Holding A Heart in Your Hands - how virtual reality is shaping the future of cardiovascular science

Visiting UCL’s Institute of Cardiovascular Science is a day of firsts for me. I not only test out Virtual Reality for the first time, but I also hold a human heart. But I don’t go into a lab and touch any specimens. Instead, I put on an Oculus headset, grab two controllers, and enter a virtual operating room.

UCL PhD student Endrit Pajaziti is a co-creator of VheaRts, a VR app that creates 3D models from patients’ CT and MRI scans. VheaRts, which started out as a research tool in 2018, is revolutionising UCL’s approach to medicine, in both a clinical and educational setting. In the app, doctors and medical trainees are given specialised tools to grab, slice, highlight, and measure the internal anatomy of a heart smaller than the size of an almond. 

“Recently, we created a network version of the application,” says Pajaziti, a warm final-year doctoral student.. “Now, you can put several people in a virtual room to connect in a group session and view the same model remotely, which wouldn't be possible otherwise.” 

Pajaziti demonstrated the app at his desk at the Institute of Cardiovascular Science, at The Zayed Centre for Research into Rare Disease in Children, December 7th.

In August, UCL and The Great Ormond Street Hospital for Children (GOSH) teamed up virtually with doctors in Rio de Janeiro, using VheaRts to complete one of the most complex separation surgeries of conjoined twins ever recorded. 

Four-year-old twins Bernardo and Arthur Lima were born with fused heads, known as craniopagus twins. The international medical teams spent months using headsets to come together in the same virtual room, where they could visualise and discuss the twins' anatomy. They tested techniques for the separation procedure using interactive models created by the app. Owase Jeelani, the lead paediatric surgeon on the London team, describes it as "space-age stuff."

Owase Jeelani pictured in London connecting with Brazil’s team via video chat and VheaRts to plan the craniopagus twins’ surgery, March 2022. Courtesy of Endrit Pajaziti
 

“The surgeon found it useful because you can see the vasculature from both twins intermingling, and it's a complex network of arteries and veins. That's impossible to visualise from looking at 2D scans,” says Pajaziti, with his headset on, pointing out visible abnormalities on the heart model as I watch alongside.

VheaRts has been used in planning over 50 surgical procedures. Before the separation of the Brazilian twins, the app was used in two previous separation surgeries of twins joined at the head. 

“With VR, you can examine any angle, slice into any depth, rotate it – there is a lot more freedom,” he continues. "It's fulfilling to see people responding well to something you've worked on." 

VheaRts app, a snapshot of doctors working with a patient model, May 20th. Courtesy of Endrit Pajaziti

When most people think of Virtual Reality, they think of gaming. VheaRts was created using the game development software Unity, to use with the commercial Oculus Rift/Quest headsets made by Facebook Technologies. At a price of £400, VR headsets are becoming increasingly popular. Market analysts project that the demand for home VR devices will increase by 16 times going into 2023. This figure doesn't include the rapidly increasing demand in the medical sector. 

In 2018, George Washington University created a VR tool that enables surgeons to virtually explore patients’ brains and bodies prior to performing a procedure. A Harvard Business Review study found that VR training on the Osso VR platform improved surgical performance by 230% when compared to traditional teaching methods like 2D scans and 3D printed models. And researchers also found that mean errors in the operating room were six times less likely to occur in a VR-trained group.

When it comes to VheaRts, what makes it especially effective is the team’s ability to work closely with clinicians. 

“Virtual Reality is not the future, it is the present,” said Dr. Beatrice Bonello, a cardiologist and consultant at GOSH, who sits across the hall from Pajaziti. 

“It's important in paediatrics to have a great understanding, in 3D, of the heart. VR is perfect for this because it has an infinite library. So for education, it's a no-brainer. I wish I had it when I was a trainee. The goal is to have it integrated a hundred percent.” 

As Dr. Bonello left Pajazit’s desk, Andrew Cook, Professor of Research and Education in Fetal Cardiac Development, walked into the office behind us, sandwich in hand. Professor Cook put down his lunch and pulled up a chair next to us, excited to speak on VheaRts as an innovative tool in his classroom. 

Cook has run a 3-day ‘hands-on’ Cardiac Morphology Course for the past 15 years. The workshop teaches clinicians from a mix of professional backgrounds using UCL’s ‘gold-standard’ library of anatomical specimens. VheaRts was tested by 58 participants during this course, to assess VR’s usability, helpfulness, and level of acceptance in an educational setting. 

Professor Cook (arm raised), leading a lecture using the Oculus headsets, January 2022. Courtesy of Endrit Pajaziti

Cook and Pajaziti report that, “VheaRts was found to be very intuitive to use by 93% of the participants” and “over 89% of users were “willing” or “very willing” to implement a VR setup at their own institutions.”

They also tested the app by hosting seminars in South Africa from London. “They had some headsets on their side and we sent them our application. You have five or six students in South Africa and Cook joins remotely teaching cardiac morphology,” explains Pajaziti. 

During Covid-19, Cook sent undergraduate and postgraduate students Oculus Headsets with VheaRts software downloaded to replicate hands-on lab time.

Professor Cook, leading a lecture using the Oculus headsets during the pandemic, October 2020. Courtesy of Endrit Pajaziti

“I had a challenge because I couldn't run any anatomy labs. You can show anatomy on Zoom – we did a lot of that. But then we thought, what else could we do that would give them that three-dimensionality they would see if they were in the lab?” explained Cook, about his MSc modules.

As a result, the team sped up development and began implementing the app in more curricula. Modules held in the virtual classroom look more like the Mii Plaza on the Wii than a lecture, as students’ avatars bobble around virtual specimens. 

Professor Cook, leading a lecture in the VheaRts app during the pandemic, 2021. Courtesy of Endrit Pajaziti

Although the department has headsets ready, VR training and integration are time-consuming. So Cook and Pajaziti paused using VR as a regular tool in the classroom, but they see VR tools becoming the new normal within the next 10-15 years. 

I ask Cook what he sees as the ideal future for VheaRts: “Full integration with our teaching,” he says. “You tend to get biased because you've been using it, but I find it very difficult now to go back to lectures. The advantages are unparalleled… if you have to go back to the old method, it's the equivalent of moving from traditional three-millimetre slides to PowerPoint.”

Although I have no experience in VR or heart surgery, Pajaziti teaches me the controls with ease. Within minutes I’m using a digital laser pointer to flip through real patient models. I learn how to grab and rotate the model with one hand, and hold tools in the other. I can slice a heart in half, see through it, take it apart, and even see where the abnormalities are. This is probably the closest to a surgical room I will ever come to. 

After about 30 minutes with the headset on, I start to feel nauseous and have to take it off. As I lift it off I feel my eyes readjust back to reality, a feeling that does not fade quickly. I later discover this is known as Cybersickness. 

According to Neuroscience News, “studies show that more than half of first-time headset users experience this phenomenon within 10 minutes of being exposed to VR.” The symptoms are similar to motion sickness, caused by all the conflicting sensory information. 

Neal Patel, one of Cook and Pajaziti’s medical students, used VheaRts during his Integrated Cardiovascular module and had a similar experience. “It was a new feeling,” he says. I'd say about nearing the 20-minute mark, the headset starts to feel heavy. Your eyes become tired and your head's like, okay, I'd rather take it off. So there are limitations to how long you can use it.” 

Despite this, Neal argues VheaRts will be a game-changer for medical trainees. “Even if it is for 20 minutes, it's definitely a useful experience. To give someone the understanding that you get for 20 minutes in that room would traditionally require hours of reading, describing, and drawing.” 

VR undoubtedly comes with a learning curve, and the accuracy and efficacy of its training still need to be improved. This makes it difficult to devote the time and resources necessary to reach full integration. VR is still in its infancy and much of its development is uncharted territory.

VR also faces increasing opposition. According to a study done by the Tehran University of Medical Sciences, VR reduces face-to-face communications and lacks the regulatory laws and protocols necessary for widespread integration. 

Despite these challenges, Pajaziti has high hopes for VheaRts beyond UCL. “We want to create a business case for virtual reality as an integral service in hospitals. The perfect solution is that your clinician, doctor or surgeon, sitting at their desk, has the images from the scanner on a computer, and the images are immediately transformed into a 3D model without the aid of a person manually doing the segmentation that we do.” To achieve this, Pajaziti says the app must receive more funding and undergo a rigorous regulatory process. 

Once thought to be just science fiction, VR will soon become a part of our daily lives. The pioneering advancements offered by VheaRts prove it is not a matter of if headsets will become inevitable, but when. 

About the author:
Caroline Coyer started her multimedia journalism career in her hometown of New York City, focusing on criminal justice, LGBTQ+ lifestyle, and local community stories. She recently graduated from NYU, studying Broadcast Journalism and International Relations. After graduating, she went on to work for the New York Post as part of Page Six’s video team. She is currently on the Creative Documentary By Practice MFA at UCL, and is part of the inaugural cohort of the Student Journalism Scheme.