Doctors Test Newly Invented Gestural Interface During Brain Surgery

The new and growing field of gestural computing got a serious field test during brain surgery

Gestural interfaces are a hot new field of computing. Microsoft announced that its upcoming OS -- Windows 7 -- will rely heavily on gestures and touch. In a most basic sense, a gestural interface is controlled by movements of the hands or arms, allowing users to gesture to literally scroll around images on screen. Sometimes this is coupled with touch in devices such as the iPhone, where a pinching gesture can shrink or expand items.

Continuing the progress in the field of gestural computing, researchers at Israel's Ben-Gurion University of the Negev (BGU) developed a new gestural computer system specially designed for medical use. In the past, doctors used touch screens or mice to navigate about images during surgeries. However, by touching the screen, they risk compromising sterility and introducing infection into the surgery site.

The new system is purely gestural and requires no touch. It allows doctors to scroll around images by moving their hands in front of the screen.

The system received an impressive field test at a Washington D.C. hospital and the results are detailed in the June article “A Gesture-based Tool for Sterile Browsing of Radiology Images" in the Journal of the American Medical Informatics Association.

Juan P. Wachs, a recent Ph.D. recipient from the Department of Industrial Engineering and Management at BGU lead the research, which he describes stating, "A sterile human-machine interface is of supreme importance because it is the means by which the surgeon controls medical information, avoiding patient contamination, the operating room (OR) and the other surgeons. This could replace touch screens now used in many hospital operating rooms which must be sealed to prevent accumulation or spreading of contaminants and requires smooth surfaces that must be thoroughly cleaned after each procedure – but sometimes aren't. With infection rates at U.S. hospitals now at unacceptably high rates, our system offers a possible alternative."

The new system, known as Gestix, eliminates the need for complex and largely ineffective sterilization procedures on today's OR touch screens. When surgeons first start with the system, they train it and learn to use it by learning to move their hand in one of eight directions away from a neutral area, fast. This movement scrolls the image. They also learn to zoom in and out by rotating their hand clockwise or counterclockwise. To avoid misleading signals, when the doctor is done, they drop their hand which triggers a sleep mode.

The hand motions are captured using a Canon VC-C4 camera and they are processed by an Intel Pentium processor and a Matrox Standard II Video Capture device. The system was tested to much success at the Washington Hospital Center in Washington, D.C during two "in vivo" neurosurgical brain biopsies. This may be the first time such a system was used with an "in vivo" procedure, according to the researchers.

Wachs worked with colleagues Professors Helman Stern and Yael Edan on the project and with a variety of M.S. students, who theses pertained to the topic. Ongoing research is focusing to expand the gestural interface for use in tele-robotic and tele-operated systems. By adding voice recognition, researchers hope to create a system with many control modes (multimodal).

Other additional research on the imaging side is being developed by Prof. Helman Stern and Dr. Tal Oren of the Dept. of Industrial Engineering and Management and Dr. Amir Shapiro of the Dept. of Mechanical Engineering. They aim to help the vision impaired navigate better through use of this system and a tactile body display.

BGU's staff has not announced the current commercialization plans for the technology, but it seems likely that it will soon be finding its way into hospitals.