The Microsoft headset fits over the surgeon’s head and displays transparent images that hover in the surgeon’s field of vision. The apps align images of the patient’s anatomy with the real-life view. When it’s all set up, a surgeon can walk around the patient, viewing three-dimensional holographic images of internal structures—such as arteries, veins, and internal organs—from different vantage points.
The headset could mean surgeons never have to look away to get information while they perform a procedure. They can use voice commands or hand gestures to enlarge images or move information around. Even the patient’s vital signs can be projected onto the field of vision. “It’s going to allow surgeons to be able to do things faster, more efficiently, and more accurately in an environment that is real,” says West. MediView is currently testing its HoloLens application in liver, kidney, and other abdominal tumor ablation procedures.
In a typical procedure, an interventional radiologist uses a biopsy needle to deliver microwave energy that heats and destroys abnormal issue. Seeing the patient’s anatomy in 3D will provide “a markedly increased ability to target cancerous masses deep within the body,” West says. (MediView has been using HoloLens 1 and plans to upgrade to HoloLens 2 when it becomes available.)
In May, New York startup Medivis received FDA approval for its HoloLens-based imaging product, SurgicalAR, which can be used for holographic surgical planning. It is the brainchild of two medical residents at New York University—a neurosurgeon and radiologist, two technophiles who began brainstorming about how mixed reality devices could help reveal surgical anatomy.