Robotics, virtual worlds meet medicine

Health Industry Today,  Nov, 1992  by Sharyn Rosenbaum

Technology that combines computer models and images could revolutionize surgical training and surgical procedures. Manufacturers, especially those that supply laparoscopic instruments, are likely to begin developing a marketing strategy to capitalize on the potential medical breakthroughs.

Already being explored in surgical simulations, motion measurement and patient rehabilitation, virtual reality and telepresence offer exciting opportunities in health care.

Virtual reality is a technology that enables users to interact with three-dimensional computer worlds that resemble the real world. It combines computers and sensory mechanisms to create a controlled environment. Users can interact with the virtual environment through sight, sound or touch. The technology may not only empower patients by helping them to communicate, but also may help physicians better diagnose and treat patients in the future.

Telepresence extends sensory apparatus to a remote location through robotics. The user, from a distant site, could see, feel and touch objects in the real world.

For example, a surgeon in one location could have a similar hand control, visualization and touch as if he were actually at the operation site performing a surgical procedure. He or she could actually pick up an object from a remote location and feel its weight and texture. The technology has already found a niche in pathology diagnosis, whereby a surgical pathologist has immediate access to a microscope and a slide of a patient's biopsy at a remote site.

Telepresence is being investigated for use in laparoscopic surgery. Researchers are trying to commercialize the technology so surgeons performing minimally invasive surgery could have the same freedom of movement as in open surgery.

The real-time use virtual reality technology for direct patient care is years away, but telepresence may be used sooner, according to Dr. Eric Horvitz of Stanford University School of Medicine, Palo Alto, Calif., and president of Knowledge Industries, a Palo Alto company that produces real-world reasoning systems for physicians in a variety of medical specialty areas.

"Authentic or high-fidelity virtual reality is a long way off," he said. "Current virtual reality projects stimulate the imagination about the possibilities of using the technology in medicine decades from now.

Telepresence applications are likely to come to fruition earlier because the technology does not face the inherent computational problems associated with building rich, artificial worlds.

"For example, for generating realistic simulations of surgery, computers have great difficulty at simulating the look and feel of soggy, slippery tissue. But telepresence can make use of the world as it currently exists."

Horvitz likened the resolution and functionality of medical virtual reality systems today to that of video games. "But people are excited today about the possibilities, and today's toy-like demonstrations highlight the promise of future technology for training and patient care."

Applications abound

The technology involved in virtual reality and telepresence, Horvitz said, has applications in three areas: visualization of complex structures and anatomy; more efficient access to delicate structures; and visual integration of diverse information sources.

Through virtual reality, the user could view a high-definition image on a computer screen that mimics the actual operating field and specific anatomy. The technology could be used commercially in both surgical planning and an actual operation when its resolution is improved.

Wearing virtual reality head gear, the surgeon could have a view of the operating field that would closely represent the real thing. The surgeon could then better prepare for difficult operations and have an easier time navigating instruments during procedures.

"Virtual reality is good for the 'three Ps'--planning, performing surgery and predicting outcomes," said Dr. Joseph M. Rosen, associate professor of plastic and reconstructive surgery at Dartmouth-Hitchcock Medical Center, Lebanon, N.H. "It improves visualization and navigation and therefore improves safety and performance. It can apply to any application."

Investigators at Stanford and the National Aeronautics and Space Administration have experimented with a software prototype for operating on simulated patients.

The prototype systems enable medical students and surgeons to don virtual reality gloves and headsets and to manipulate virtual surgical tools along with structures in the abdomen. But the low resolution of the images and other factors, Horvitz said, "make the simulation a distant shadow of a real surgical experience."

The second area involves using telepresence devices to plan or perform surgeries on delicate structures. The technology would be particularly advantageous in laparoscopic surgery, where clear visualization and effective control of surgical tools are crucial.

Using a version of telepresence called micropresence, a surgeon could work on a magnified image of the real anatomy. A micropresence system electronically amplifies the size of a structure and provides a different perspective of the anatomy as the physician moves his or her head.

"Micropresence is quite feasible because the essential components of the technology exist today," Horvitz said.

The technology involves the positioning of one or more miniature television cameras and related camera-control systems in hard-to-reach or compact areas of a patient's anatomy and to identify the exact position of teleoperated microsurgery tools, Horvitz said. The technology would enable the surgeon to enter and navigate through difficult anatomical regions.

As mentioned above, telepresence also could be used to perform surgery from a remote location or having experts guide a surgeon through a procedure from another site.

The third area involves overlaying of radiologic images on a patient's anatomy. This technique holds great promise for use in such tasks as surgical planning and radiation therapy. According to Horvitz, some of these methods are already in use.

At Scripps Research Institute, La Jolla, Calif., Dr. Christopher Gallen is using this application on live patients. He records electrical activity in the brain and maps the location of motor control and sensory processing. The sensory information is overlaid on a model of the brain. The information is then used to plan delicate brain surgery.

"The future is rich with the possibilities for applying such anatomically keyed display technology," Horvitz said.

For example, computed tomography images could be overlaid onto live anatomy, allowing a surgeon to "see through" the patient and inspect a tumor, Horvitz said. "This technology could be used for highlighting the outlines of a malignancy or of delicate blood vessels during surgery," he added.

A fourth area involving the use of computational methods in medicine is robotic conducted surgery. Microcontrolled robotic scalpels are already being tested at the University of California, San Diego, for use in detailed ophthalmologic surgeries.

"The notion of a robot cutting tissue may initially frighten patients," Horvitz said. "However, for many delicate surgeries, a finely controlled robotic scalpel may be preferred to even the most steady human hand. Someday, patients may demand robotic precision for such surgeries."

Laparoscopic surgery aid

A number of companies displayed and discussed their virtual reality-like systems at the "Medicine Meets Virtual Reality" conference in June in San Diego. Products ranged from a prototype virtual reality system for utilization with MR units from Virtual Reality Inc., Pleasantville, N.Y., to a tactile feedback system from Xtensory Inc., Scotts Valley, Calif.

SRI International, Menlo Park, Calif., is demonstrating remote surgery using telepresence. Philip S. Green, director of the bioengineering research laboratory at the company, said the technology will enable surgeons to perform laparascopic surgery with the same feel of open surgery.

"The surgeon would sit at a console and look into the patient," he said. "He would reach into the patient with his instruments and perform complex surgical procedures. This is widely heralded as the coming thing in laparoscopic surgery."

Telepresence could be used in open surgery and microsurgery as well, along with surgical planning and training. The technology also would eliminate direct contact between the surgeon's hand, instruments and patient, therapy eliminating the risk of disease transmission.

"By the middle of the decade, surgeries will routinely be performed this way. I believe it to be very cost effective," Green said.

SRI's telepresence system is not yet in clinical trials.

Hands-on virtual reality

Greenleaf Medical Systems, Redwood City, Calif., is marketing a product in the virtual reality area. Its DataGlove is being used at The Advanced Technology Center of Loma Linda University Medical Center, Loma Linda, Calif., to measure tremors in patients with Parkinson's disease and for rehabilitation therapy. Greenleaf licensed the technology from VPL Research Inc., Redwood City, Calif.

The patient with Parkinson's disease puts on the DataGlove, which has fiber optic cables along its surface. When the joints in the hand bend, the fibers bend, and sensors record the movement. The recordings are then digitized and forwarded to a computer, which calculates the angle at which each joint is bent. This can be done for 10 joints simultaneously.

An image of a hand can move on the screen that represents the actual hand motion of the patient. Dr. David Warner and his colleagues analyze the data to study the effects of certain drugs on patients' motor performance.

Warner and his colleagues also have patients wear the DataGlove for rehabilitation therapy; they pick up virtual objects and practice specific motions.

Another application of the DataGlove at Loma Linda involves patient communications. Using the DataGlove and a gesture-to-speech system, patients who have suffered trauma injuries can communicate to hospital personnel by making gestures.

The system was used to treat a patient who had come out of a coma with brainstem encephalitis. She could not open her eyes or make facial expressions. The DataGlove was put on her hand, and she was told to make specific gestures--move her index finger for yes, her pinky finger for no. Then she was asked questions such as, "Are you feeling any pain?" When she gestured, the computer emitted sounds that indicated that she was responding. This kind of situation, Warner stated in an article, could take place in every rehabilitation clinic.

Loma Linda also has used Greenleaf Medical's GloveTalker software in intensive care for vocally impaired patients. The system, which speaks for the user, is an extension of Greenleaf's gesture control system. Patients wear the DataGlove and signal the computer with a set of gestures. This information is passed through to the computer's voice system that speaks for the patient.

New markets targeted

Greenleaf is aiming to expand its customer base to include medical and non-medical markets.

Along with contacting key investigators for the DataGlove and other motion enhancement products, the company also will target the disabilities market, according to John Reed, director of product development.

Greenleaf will target therapists who work with people who are motion and/or temporarily vocally impaired. A sales force of about 40 reps will initially call on physical therapists at research labs.

Another market for the products is the workplace, where upper extremity injuries frequently occur. Reed sees a huge market for the company's movement analysis system, which assesses carpel tunnel syndrome. He said that about $20 billion is spent annually on repetitive strain injuries.

The system adapts the DataGlove's fiber optics and links it with new software to quantitatively assess upper extremity function. Potential customers include orthopedic surgeons, physical therapists, occupational therapists and research clinics.

The system costs $12,000, compared to competing video systems that cost up to three times as much, Reid said. The DataGlove costs $2,000.