The work of a new startup and some pioneering doctors has recently given the term “operating system” and entirely new meaning: They are using virtual reality (VR) to experimentally plan, practice and assist in surgery.
Currently, VR technology is rapid diversifying into new and exciting applications across a widening spectrum of fields and markets. This surgical one is particularly promising because of its potential to improve medical care. Indeed, this is far beyond the VR’s more familiar domains of entertainment and media.
During the past year, a series of Subway Fold posts have covered closely related VR advancements and apps in movies, news reporting, and corporate advisory boards. (These are just three of ten posts here in the category Virtual and Augmented Reality.)
Virtual Surgical Imaging
The details of these new VR surgical systems was reported in a fascinating article entitled posted on Smithsonian.com on September 15, 2015 entitled How Is Brain Surgery Like Flying? Put On a Headset to Find Out by Michelle Z. Donahue. I highly recommend reading it in its entirety. I will summarize, annotate, and pose a few non-surgical questions of my own.
Surgeons and developers are creating virtual environments by combining and enhancing today’s standard two-dimensional medical scans. The surgeons can then use the new VR system to study a particular patient’s internal biological systems. For example, prior to brain surgery, they can explore the virtual representation of the area to be operated upon before as well as after any incision has been made.
For example, a fourth year neurosurgery resident named Osamah Choudhry at NYU’s Langone Medical Center, has had experience doing this recently in a 3D virtualization of a patient’s glioma, a form of brain tumor. His VR headset is an HTC Vive used in conjunction with a game controller that enables him to move around and view the subject from different perspectives, and see the fine details of connecting nerve and blood vessels. Furthermore, he has been able to simulate a view of some of the pending surgical procedures.
SNAP Help for Surgeons
This new system that creates these fully immersive 3D surgical virtualizations is called the Surgical Navigation Advanced Platform (SNAP). It was created by a company in Ohio called Surgical Theater ( @SurgicalTheater ). It can be used with either the Oculus Rift or HTC Vive VR headsets (neither of which has been commercially released yet). Originally, SNAP was intended for planning surgery, as it is in the US. Now it is being tested by a number of hospitals for actual usage during surgery in Europe.
Surgeons using SNAP today need to step away from their operations and change gloves. Once engaged with this system, they can explore the “surgical target” and then “return to the patient with a clear understanding of next steps and obstacles”. For instance, SNAP can assist in accurately measuring and focusing upon which parts of a brain tumor to remove as well which areas to avoid.
SNAP’s chance origin occurred when former Israeli fighter pilots Moty Avisar and Alon Geri were in Cleveland at work on a flight simulator for the US Air Force. While having a cup of coffee, some of their talk was overheard by Dr. Warren Selman who is the chair of neurosurgery at Case Western University. He inquired whether they could adapt their system for surgeons to enable them to “go inside a tumor” in order to see how best to remove a tumor while avoiding “blood vessels and nerves”. This eventually led to Avisar and Geri to form Surgical Theater. At first, their system produced a 3D model that was viewed on a 2D screen. The VR headset was integrated later on.
SNAP’s software merges a patient’s CT and MRI images to create its virtual environment. Thereafter, a neurosurgeon can, with the assistance of a handheld controller, use the VR headset to “stand next to or even inside the tumor or aneurysm“. This helps them to plan the craniotomy, the actual surgery on the skull, and additional parts of the procedures. As well, the surgeon can examine the virtual construct of the patient’s vascular system.
At NYU’s Langone Medical Center, the Chair of Neurosurgery, Dr. John Golfinos, believes that SNAP is a significant advancement in this field as doctors previously had to engage in all manner of “mental gymnastics” when using 2D medical imaging to visualize a patient’s condition. Today, with a system SNAP, simulations are much accurate in presenting patients the way that surgeons see them.
Dr. Golfinos has applied SNAP to evaluating “tricky procedures” such as whether or not to use an endoscopic tool for an operation. SNAP was helpful in deciding to proceed in this manner and the outcome was successful.
UCLA’s medical school, the David Geffen School of Medicine, is using SNAP in “research studies to plan surgeries and a procedure’s effectiveness”. The schools Neurosurgery Chair, Dr. Neil Martin, has been working with Surgical Theater to smooth over the disorientation some users experience with VR headsets.
Dr. Martin and Mr. Avisar believe that SNAP “could take collaboration on surgeries to an international level”. That is, surgeons could consult with each other on a global scale to assist with operations that place then within a shared virtual space to cooperate on an operation.
Patient Education and Future Developments
Dr. Choudhry further believes that the Oculus Rift or Vive headsets can be used to answer the questions of patients who have done their own research and devised their own questions, as well as improve the doctor/patient relationship. He has seen patients quickly “lose interest” when he uses 2D CT and MRI scans to explain these images. However, he believes that 3D VR “is intuitive” as patients recognize what they are viewing.
He also believes that future developments might lead to the integration of augmented reality systems into surgery. These present, through a transparent viewer and headset, a combination of a virtual data overlay within the user’s line of sight upon the real operating room. (Please see these eight recent Subway Fold posts about augmented reality.)
My own questions are as follows:
- Are VR surgical systems used only after a decision to operate has been made or are surgeons also using them to assess whether or not to operate?
- What other types of surgeries could benefit both doctors and patients by introducing the option of using VR surgical systems?
- Can such systems be used for other non-invasive medical applications such as physical therapy for certain medical conditions and post-operative care?
- Can such systems be adapted for non-medical applications such as training athletes? That is, can such imagery assist in further optimizing and personalizing training regimens?
May 11, 2017 Update: For a truly fascinating report on a new surgical system that has just been developed using advanced augmented realty technology, see Augmented Reality Goggles Give Surgeons X-ray Vision, by Matt Reynolds, posted on NewScientist.com on May 11, 2017.