Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Virtual Reality in Medicine and Medical Engineering: VDC-Whitepaper

34 views

Published on

The Virtual Dimension Center (VDC) Fellbach has created a whitepaper that provides an up-to-date overview of applications of virtual reality technologies in medical applications.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Virtual Reality in Medicine and Medical Engineering: VDC-Whitepaper

  1. 1. Whitepaper Virtual Reality in Medicine and Medical Engineering Techniques, Applications, Results © Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC) Dr.-Ing. Dipl.-Kfm. Christoph Runde Virtual Dimension Center (VDC) Fellbach Auberlenstr. 13 70736 Fellbach www.vdc-fellbach.de Techniques, Applications, Results
  2. 2. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Environment: Necessary in training: o realistic o rare pathologies o objective evaluation of performance o easy access scenario-simulation Challenges Permanent advancement of treatment methods Coordination task team Limited visibility, indirect working Complex 3D data, difficult image interpretation Overview 2 o scenario-simulation Training still on the patient Assessment of the surgeon/ student Mastery of many knowledge domains interpretation Multimodal interaction Planning of complex operations Complex device technology Application for device development Scenario simulation for therapy
  3. 3. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Application fields VR-based diagnosis Analysis of 3D data (volume-based method) Analysis simulation data o Strength/ stress calculation o Kinematics, multibody systems Flow, fluidics Rehabilitation o Motor skills o Psychiatric therapy Digital atlases Training o Endoscopic procedures Overview 3 o Flow, fluidics Planning intervention o Procedures, routes o Irradiation Intraoperative support o Navigation support o Expanded surgical field of view Tele-Medicine Tele-presence with VR o Endoscopic procedures o Surgical activities o Procedures and processes, device technology Workflow and arrangement planning Illustration medical technology for marketing communication
  4. 4. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration VR-based diagnosis Immersive analysis of results from volume- based procedures: CT, MRI, ultrasound Spatial 6DOF interaction Using stereoscopy and motion parallax (via head tracking): improving spatial depth perception Image: Fh-IPA DigiHom VR-based Diagnosis 4 perception Image: ZGDV Darmstadt Image: VPH/ZGDV 3D-representation brain tumor with blood vessels 3D-scene extracted from contents of volume-based procedures
  5. 5. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Time lapse, slow motion: animation: for the recognition of temporal relationships Reinforcement: physical quantity is increased in order to make it visible VR-based diagnosis – Applicable techniques False color representation: a physical quantity (such as temperature) is visualized via color coding Samples: the viewer can use a measurement sensor to display the model´s Selective display: only areas with certain measurement values are displayed: problem zones Superposition: 2 simulation results or one test part and a simulation result are VR-based Diagnosis Comparative presentation: alternative processes are shown simultaneously to detect differences Subtractive representation: only the difference between two alternative processes is displayed. This makes process differences even easier to identify 5 to display the model´s measurement values. The sensor may be variable regarding the measured quantity. Sections: the 3D model is cutted into sections in order to make the viewer´s areas of interest clearly visible simulation result are graphically superimposed: optimization potential of the simulation model or fluctuations of the process Inline-support with AR: overlay measurement values/ process parameters on situs ( intraoperative support)
  6. 6. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Analysis simulation data Strength and stress calculation: Inspection of Geometry implant Location of insertion Functionality Image: Fh-IGD Analysis of statics knee implant Analysis Simulation Data 6 Functionality Strains, tensions on the digital model Imagee CADFEM Adjust implant on individual vortex geometry Tension calculation implant Tension calculation thighbone with implant; Decubitus simulation: tensions on bones Image: Visenso Image: CADFEM Image: Visenso
  7. 7. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Analysis simulation data Kinematics and multibody systems: Inspection of Movements, limits of movement, speed, accuracy of fit Forces, torques Hip joint: analysis forcesImage: Blundell Analysis Simulation Data 7 Forces, torques Tensions on the digital model Image: Fh-IGD Image: AnyBody Jaw: analysis of movement pathologies, validation of implantations Leg: digital model of muscles and bones
  8. 8. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Analysis simulation data Flow and fluidics: Inspection of Flow rates Turbulences Areas with/ without flows Image: Fh-IGD 3D modeling, simulation model flow nose interior Analysis Simulation Data 8 Areas with/ without flows on the digital model Image: Fh-IGD Image: Visenso Flow simulation nose flow Simulation model flow aneurysm
  9. 9. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Preoperative planning – Simulation of intervention Process planning Route planning Joining Application examples: Image: HS Bern Virtual osteosynthesis Planning Intervention Eingriff 9 Application examples: Osteosynthesis planning Various surgery applications Irradiation planning Image: HS Bern Image: BMBF 3D interaction with 3D data for osteosynthesis Surgery planning metastases (yellow), portal vein (orange) and hepatic veins (blue)
  10. 10. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Planning – Example of liver resection Various distribution tools to simulate an atypical resection: Planning Intervention Eingriff 10 atypical resection: a) Plane b) Sphere c) Malleable surface /ŵĂŐĞƐ͗ dh 'ƌĂnj 3D measurement tools for spatial analysis: a) Distance measurement b) Volume measurement of individual objects c) Overall volume through virtual measuring cup
  11. 11. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Excursion: Augmented Reality – AR Basic idea: superimposition of natural view with computer-generated information Context sensitivity, alpha-numeric or 3D content Simple comparison of reality vs. model: Data glasses Overview 11 Simple comparison of reality vs. model: consistency checks digital model - phys. world Digitization is not always required High comprehensibility by docking the computer graphic to a real object Usage when manuals are difficult to apply Instruction, clarification, support in the operating process Image: metaio Image: Volkswagen CAD model superimposed with physical prototype Equipment planning in factory
  12. 12. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Das Bild kann nicht angezeigt werden. Dieser Computer verfügt möglicherweise über zu wenig Arbeitsspeicher, um das Bild zu öffnen, oder das Bild ist beschädigt. Starten Sie den Computer neu, und öffnen Sie dann erneut die Datei. Wenn weiterhin das rote x angezeigt wird, müssen Sie das Bild möglicherweise löschen und dann erneut einfügen. Intraoperative support Provision of 3D data from volumetric methods Technical data, measured data Different system configurations are conceivable (tested prototypical) Image: ZGDV Possible hardware arrangement: display above patient Intraoperative Support 12 conceivable (tested prototypical) o Display between surgeon and situs (video-see-through or optical-see- through) o Projection on situs o Side by side on separated displays (next page) Image: ZGDV Image: Surgical Planning Laboratory, Harvard Gaze through display: additional information to normal view Projection of 3D data
  13. 13. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Intraoperative support 9th of October, 2012: At the ARTORG Center for Biomedical Engineering in Bern, surgical navigation technology is combined with augmented reality displays. This technology makes it possible to map the internal structure of an organ during an operation. Intraoperative Support 13 /ŵĂŐĞ͗ hŶŝǀĞƌƐŝƚćƚ ĞƌŶ structure of an organ during an operation. The new procedure was now used for the first time in a complex liver operation. The liver is shown in triplicate: A screen in the operating room shows 3D computed tomography (CT) and magnetic resonance (MRI) scans illustrating the internal anatomy of the patient's liver. On another screen endoscopic images of the miniature camera can be seen and a third screen shows a combination of both, a superimposed, reality-enhanced image. All the relevant information is summarized in a surgical cockpit on several screens – this method helps to simplify orientation and make better decisions.
  14. 14. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Das Bild kann nicht angezeigt werden. Dieser Computer verfügt möglicherweise über zu wenig Arbeitsspeicher, um das Bild zu öffnen, oder das Bild ist beschädigt. Starten Sie den Computer neu, und öffnen Sie dann erneut die Datei. Wenn weiterhin das rote x angezeigt wird, müssen Sie das Bild möglicherweise löschen und dann erneut einfügen. Tele-Medicine Tele-surgery: Telepresence applications where the surgeon and the patient are in different locations Application scenarios: On-site treatment of accident victims, soldiers, persons who are not transportable Tele- Medicine 14 soldiers, persons who are not transportable Remote diagnosis Long-distance vs. short-distance Inclusion of distant specialists: Teleconsultation Images: Internet-Blog „Geste, image, sens“ A gallbladder removal performed in 2001 is seen as a paradigm for the long-distance application. Here a patient in Strasbourg underwent a surgery by robotic arms operated by a surgeon in New York.
  15. 15. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Rehabilitation Motor skills: Support movement Deliberate restriction of unintended directions of movement Performing exercises Image: Gupta, O’Malley MAHI Exoskeleton Rehabilitation 15 Performing exercises Objective measurement of success Image: Blundell Image: O‘Malley, Ambrose Haptic work station for training after stroke 6DOF JPL arm master
  16. 16. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Rehabilitation Treatment of phobias and addictions through individual training in the simulator Treatment of Posttraumatic Stress Disorder (PTSD) Examination of patients behavior and feelings Image: Uni Würzburg VR flight scene Rehabilitation 16 feelings Cognitive-behavioral methods Accompanied by the analysis of problematic thought patterns, specific behavioral exercises and the influence of physical processes (e.g. through precise relaxation techniques) Image: VT+ Image: Uni Würzburg VR hight scene VR spider sceneImage: Uni Würzburg VR drive through tunnel
  17. 17. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Rehabilitation Rehabilitation 17 Image: VT+ VR hight scene
  18. 18. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Pain treatment In research since 1996 Central idea: distract patients 2004: patients suffering from burns were sent to virtual ice worlds Brain scans show less brain activity with VR in the areas which are responsible for pain Rehabilitation HitLab VR for the treatment of burn patients 18 in the areas which are responsible for pain perception Endorphins do not play a central role (no effects of blockers) Even simple game consoles show no effect New study (2016) of a hospital in Los Angeles: 25% reduction of pain perception through VR AppliedVR HitLab Patient with Samsung GearVR Headset Brain scan: less brain activity in areas which are responsible for pain perception with VR
  19. 19. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Motoric skillsImage: VRMMP Excursion: Virtual Reality in knowledge management Knowledge types in virtual environments: Position knowledge: What is where? Structural knowledge: How is something related? Behavioral knowledge: How does the system behave? How do I behave? Overview VR scene for the application of a stethoscope Virtual surgery with associated haptic interface 19 Image: The Medical News (2010): http://www.thaimedicalnews.com Image: VRMMP behave? How do I behave? Procedure knowledge: Which processes cause what? Learning opportunities in virtual environments: Spatial exploration Conceptual learning Learning motoric abilities Procedural learning
  20. 20. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Digital atlases Mediation of position knowledge Mediation of structural knowledge Anatomy, physiology, biomechanics Clipping (cutting away graphical areas in order to get a free view on Image: Fh-IPA DigiHom Digitale Atlases 20 areas in order to get a free view on relevant areas) Transparency Animation Image: Google Image: Bratz 3D representation of brain and heart Google Body Browser Image: BMBF
  21. 21. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Training scenarios with visual and haptic modalities Learning levels: demonstrate, accompany, examine Training Simulation environment eyesi: Education and Training examine Virtual endoscopy, laparoscopy Virtual palpation Images: Projekt HASASEM 21 environment eyesi: ophtalmo surgery Electrorheological fluid for haptic display of elasograms. Application: palpation for identification of pathological changes of tissue Image: BioSkill Image: VRmagic BioSkill simulator for endoscopy and laparoscopy
  22. 22. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Training Diagnosis: Provision of training scenarios Examples of application: Rhinoscopy Image: Fh-IGD Virtual rhinoscopy Education and Training 22 Rhinoscopy Ophtalmoscopy Arthroscopy Hysteroscopy Images: VR Magic Image: Fh-IGD Virtual ophtalmoscopy Virtual hystereoscopyVirtual arthroscopy Image: Fh-IGD
  23. 23. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Training Intervention Provision of training scenarios Examples of application: Ophthalmic surgery Image: Uni Heidelberg Ophthalmic surgery Education and Training 23 Ophthalmic surgery Vitro-retinal interventions, cataract Colonoscopy Images: Uni Heidelberg Images: Uni Heidelberg Ophthalmic surgery EndoSim, scheme colonoscopy, 3D scene virtual colonoscopy
  24. 24. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Training Procedures and processes: Provision of training scenarios for Procedures Sequences Involvement of staff Image: Texas AM University Virtual inspection eye adaption (pupil reflex) Education and Training 24 Involvement of staff Aids Points of provision Use of device technology Image: Texas AM University Image: Texas AM University Virtual palpation of spine Handling equipment technology
  25. 25. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Training Procedures and processes - emergency response simulators: Gaining overview, securing Self organisation, team organisation Image: ETC Simulation Simulation ADMS Education and Training 25 team organisation Coordination staff Image: Texas AM University Image: E-Semble Staff control Simulation Pulse Simulation XVR
  26. 26. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Workflow and arrangement planning Layout-optimization regarding reachability, minimization of routes, overview, flow principle High occupancy rate of equipment technology and low patient waiting times at the same time (resource- Image: FH Bern Virtual surgery scene (project Hovisse) Workflow and Arrangement 26 times at the same time (resource- oriented clocking) Amongst others use of material flow simulations Procedure acceleration Image: FH Bern Image: FH Bern Work group at planning desk (project Hovisse) Top view and 4 perspectives (project Hovisse)
  27. 27. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Illustration in marketing communication Realistic representation based on 3D data Create understanding Brochures, web, (stereo) films Interaction is possible 2D drafting based on configuration Image: LumoGraphics Computer graphics operating room for Berchtold Illustration 27 Mediation of product variance Customization: configuration of modular designs Consistent sales and quotation processes Reliable and correct offers even for complex products Innovative appearance Virtual operating room: extract of 3D stereo film for KLSmartin Surgical instruments: extract of 3D stereo film for KLSmartin Image: Visenso Image: Visenso
  28. 28. Intraoperative Support Digital Atlases VR-based Diagnosis Analysis Simulation Data Planning Intervention Tele- Medicine Rehabilitation Education and Training Workflow and Arrangement Overview Illustration Conclusion Very wide range of applications of VR in medicine and medical engineering Medicine is the most specialized field of application of VR Very specific device technology Challenge: o Registration Generation of 3D data in real-time Overview 28 o Generation of 3D data in real-time Complex models that reflect all facets of reality Germany, Switzerland, USA very far ahead – local specialists
  29. 29. VDC members and partners regarding this topic 29
  30. 30. Thank you very much for your interest! You are interested in this topic and you are looking for contact persons/ implementation partners? Please contact us. © Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC) Virtual Dimension Center (VDC) Fellbach Auberlenstraße 13 70736 Fellbach www.vdc-fellbach.de 30

×