simulador-de-laparoscopia-lapsim

In a first approach to the world of Augmented Reality, we established the differences between technologies of mixed, augmented and virtual reality. In this second installment, we are going to review the applications that are already being made of these realities in the health sector.

Medical data visualization

Applications aiming at the agile viewing of patient data such as ultrasound, tomography images … obtaining a more accurate view of data improving diagnoses and facilitating the decision making for possible surgical interventions.

An example is the viewing of images of an ultrasound using AR, developed by the hacker Victor Skobov using an ultrasound that was to be discarded.

sistema-de-visionado-de-ultrasonidos-de-victor-skobov-1Ultrasonic viewing system by Victor Skobov

Simulation and training

RV does not need the human patient to be present, since it deals more with off-line simulation. This makes it very suitable for training.

The simulator called LapSim emulates a real surgery with the laparoscopic technique using a haptic device that allows reproducing the sensations of the realized movements.

Surgery

Both AR and VR are able to improve the surgery by allowing the preparation of them with patient data and testing various techniques to choose the most convenient. At the same time, it is able to guide and mark relevant information during the development of the surgery obtaining more efficient and less invasive surgeries.

MEVIS system allows to prepare the surgery using 3D images of x-rays and tomographies to reconstruct the locations of blood vessels. In addition, during the development of them, is able to register the planning data and display blood vessels in different colours.

Diagnosis, therapy and rehabilitation

The AR and VR have a clear application in tests for diagnosis of diseases, treatment of phobias and support and incentive in the rehabilitation generating virtual and safe situations

An example is the system of treatment of spiders phobias using AR developed by Phobos Center where they are simulated in physical world situations with spiders in order to treat the patient’s fear to these insects.

Sistema de tratamiento de fobias a las arañas de Phobos CenterSpiders Phobias Treatment System by Phobos Center

Rehabilitation system for the health platform Tratamiento 2.0 by CARTIF. The system allows managing rehabilitation exercises for the patients by the health personnel and the patients perform the exercises as a game at home with the use of a webcam. The system records the evolution of the treatment and the performance of the exercise.

Servicio de Rehabilitacion (SER) de CARTIF para la plataforma Tratamiento 2.0Rehabilitation service (SER) by CARTIF for Tratamiento 2.0 platform

Emotional evaluation system developed by CARTIF. The system generates situations and emotions through an avatar of a person to identify them by people with schizophrenia. It can be used in diagnosis, treatment and evaluation of progress.

Sistema evaluador de emociones desarrollado por CARTIFEmotional evaluation system developed by CARTIF

Education

In this aspect, the AR provides a new channel that allows improving the learning providing other points of views on the knowledge. An example is the books that through mobile applications allow seeing parts of the anatomy in 3D.

libro-de-anatomia-con-realidad-aumentadaAnatomy book with augmented reality

 The use of these technologies is based on a series of techniques that make possible its use. Any advance in these techniques greatly improves the technologies. The main techniques used are:

  • Registration of information and monitoring: It is very important to position the user to be able to correctly locate the contents in their environment even if the user or objects move or even partially cover. This is done through visual marks such as bidis that are identified by the system and allows accurate placement of the contents.
  • Display screens. It allows integration between the real and the virtual. The most striking technique is the use of head screens (smart glasses) that allows the user to see the physical world through the lens and superimpose graphic information in the user’s field of vision by reflecting it in their eyes. Other techniques are screen in hand (mobile or tablets) that capture the physical world with a camera and overlay graphic information on the video. Space projection uses digital projectors to display graphic information about physical objects.

El uso de estas tecnologías se sustenta en una serie de técnicas que hacen posible su utilización. Cualquier avance en estas técnicas mejora enormemente las tecnologías. Las principales técnicas usadas son:

  • Registration of information and monitoring. It is very important to position the user to be able to correctly locate the contents in their environment even if the user or objects move or even partially cover. This is done through visual marks such as bidis that are identified by the system and allows accurate placement of the contents.
  • Display screens. It allows integration between the real and the virtual. The most striking technique is the use of head screens (smart glasses) that allows the user to see the physical world through the lens and superimpose graphic information in the user’s field of vision by reflecting it in their eyes. Other techniques are screen in hand (mobile or tablets) that capture the physical world with a camera and overlay graphic information on the video. Space projection uses digital projectors to display graphic information about physical objects.pantalla en la cabezaHead screens

Pantallas transparentes. Sistema de guiado (STAR)Screen in hand (mobile or tablets). Guidance System (STAR)

Proyección espacialSpace projection

  • In order to interact with these systems, typical interfaces such as touch screen, mouse, keyboard or more advanced and specific interfaces such as gloves, brain interfaces or simulation tools are used as surgical material …

As we have seen, AR and VR have promising potential for use in medical applications because it provides seamless integration of data visualization with the patient’s body. This allows for improved methods of medical diagnosis and treatment.

There are technological limitations especially on the screens and registration of data that make this technology have not yet a realistic clinical application in a regular medical environment, but the progress in several R&D projects and the interest shown by the researched ones is encouraging.

Finally, it is necessary to solve a great challenge that is often overlooked and is to improve the real usability of these systems, avoiding the sensory overload and making the viewing experience more controllable, simple, agile and transparent so that the only concern of the medical staff is the patient.

Álvaro Melero Gil

Computer Engineer. He is responsible for the software engineering and development machine vision, and robotics software and integration and device control. He is member of CARTIF’s 3D laboratory and performs 3D scanning works of structure, environments and objects.
Álvaro Melero Gil

Latest posts by Álvaro Melero Gil (see all)

Facebooktwitterlinkedin

About Álvaro Melero Gil

Computer Engineer. He is responsible for the software engineering and development machine vision, and robotics software and integration and device control. He is member of CARTIF’s 3D laboratory and performs 3D scanning works of structure, environments and objects.

Leave a Reply

Your email address will not be published. Required fields are marked *