Virtual Worlds And Real World


Published on

Dr Kanav Kahols presentation at AMIA Workshop on virtual environment training.

Published in: Technology, Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • In 1960, Rudolf E. Kalman published his famous paper describing a recursive solution to the discrete linear filtering problem. The Kalman filter addresses the general problem of trying to estimate the state x of a discrete-time controlled process that is governed by the linear stochastic difference equation, with a measurement z. The Kalman filter estimates a process by using a form of feedback control: the filter estimates the process state at some time and then obtains feedback in the form of (noisy) measurements. As such, the equations for the Kalman filter fall into two groups: time update equations and measurement update equations. The time update equations are responsible for projecting forward the current state and error covariance estimates to obtain the a priori estimates for the next time step. The measurement update equations are responsible for the feedback, i.e. for incorporating a new measurement into the a priori estimate to obtain an improved a posteriori estimate. If the process to be estimated or the measurement relationship to the process is non-linear, a filter that linearizes about the current mean and covariance will be needed. This filter is referred to as an extended Kalman filter.
  • Virtual Worlds And Real World

    1. 1. Virtual Worlds and Real World By the real… Of the Real… For the Real…
    2. 2. Contents <ul><li>What to capture from the real world? </li></ul><ul><li>How to simulate in the virtual world? </li></ul><ul><li>How to measure transfer in the real world? </li></ul>
    4. 4. Visual Appearance <ul><li>Need 3D Models: , , </li></ul><ul><li>Modification of Models: Need Maya, 3D Studio Max… free solutions Vorpal, MeshMan ( ) 3D Object converter. </li></ul><ul><li>Need textures… get a camera and click pics of real world. Make images small in size <100K in size. </li></ul><ul><li>Need space: , </li></ul>
    5. 5. Entities. <ul><li>Poser® allows for clicking front and side images and enables building of a model. </li></ul><ul><li>For non-human entities search the 3D models websites… </li></ul>
    6. 6. Activities and Interactions <ul><li>This is where conventional off-the-shelf approaches don’t work. </li></ul><ul><li>We need a fly on the wall… a system which much like blackbox captures interactions in real environment without much interventions. </li></ul><ul><li>We can develop models of normative behavior and simulate that and… </li></ul>
    7. 7. Interactions in a Complex Environment <ul><li>High performance, high complexity environment with high interpersonal variations and interactions that are unstructured. </li></ul><ul><li>Interactions are multimodal and not just verbal </li></ul><ul><ul><li>Speech </li></ul></ul><ul><ul><li>Movement </li></ul></ul><ul><ul><li>Proximity </li></ul></ul><ul><ul><li>Gestures </li></ul></ul>THE PROBLEM Behavioral Manifestations Cognitive Foundations
    8. 8. Capturing Interactions…. Detailed But Human Intensive and Can miss Dynamic Events Data Driven Too Coarse Not cognitively grounded Laxmisan et al. 2007 Malhotra et al. 2007 Alwan et al. 2006 Ostbye et al. 2003
    9. 9. Proposed Hybrid Method
    10. 10. Qualitative Data Ontology By Zhang et al.
    11. 11. Quantitative Data Movement/Proximity Data Speech/Voice Data
    12. 12. Capturing Movement Data <ul><li>A common approach is to focus on <x,y,z> location of entities and then developing activities as deterministic models of location. </li></ul><ul><li>The issue: sensors such as RID sensors suck when it comes to movement by themselves. </li></ul><ul><li>What can we do? </li></ul><ul><li>Well the end product is activities.. Why not develop probabilistic models of activities from noisy data. </li></ul>
    13. 13. Activity Recognition <ul><li>Activity Recognition is a burgeoning area.. </li></ul><ul><li>Routinely done through computer vision, sensor processing and data mining. </li></ul><ul><li>We use temporal modeling techniques such as Hidden markov Models for detecting activities (gestures). </li></ul>
    14. 14. Scenario
    15. 15. But I really really want location <ul><li>A multisensor approach is a valid method of finding exact locations. </li></ul><ul><li>Multisensor fusion provides multiple streams of data and enables a correction mechanism to combine multiple noisy streams to yield a single noiseless stream. </li></ul><ul><li>A possible mechanism for sensor fusion is a predictro corrector mechanism called Kalman Filter. </li></ul>
    16. 16. Kalman Filter Extended Kalman Filter Discrete Kalman Filter Assumed noise
    17. 17. Integration of Audio Data <ul><li>HIPAA regulations: security and privacy </li></ul><ul><li>Time synchronization </li></ul><ul><li>Audio Analytics </li></ul><ul><ul><li>Number of words spoken </li></ul></ul><ul><ul><li>Tone/Amplitude of the signal </li></ul></ul><ul><ul><li>Directionality of Verbal Interactions. </li></ul></ul>
    18. 18. Movement Analysis Results UT Houston Banner Health
    19. 19. Location Detection <ul><li>Kalman filter to combine accelerometer data with RID based location data </li></ul>
    20. 20. Virtual Playback and Analysis Tool Demo
    21. 21. Building Learning Environments
    22. 22. Persuasive Collaborative Framework External motivators Internal motivators Feedback on patient conditions due to decisions Encouraging group consensus, Dissent.Shared mental models
    23. 23. Tabletop exercises *courtesy ICT USC Supplemented by offline sessions on web 2.0 tools
    24. 24. Case Study Design <ul><li>Develop a decision tree for case solving stopping at different times during vignettes and asking questions. </li></ul><ul><li>Using custom software it is possible to group answers based on clinical specialtys or groups </li></ul><ul><li>Word clouds can be shown for answers by these different groups to enable visualization of differences in mental models and then promotion of shared mental models. </li></ul>
    25. 25. Validation Strategy
    26. 26. Validation Strategy <ul><li>Actual usage statistics </li></ul><ul><ul><li>Track usage statistics </li></ul></ul><ul><ul><li>Track number of times community option is used. </li></ul></ul><ul><ul><li>Track number of times alternative path of care is used. </li></ul></ul><ul><li>User preference models </li></ul><ul><ul><li>User preference for tools </li></ul></ul><ul><li>Reveal the effect of different tools on diagnostic and clinical abilities. </li></ul><ul><li>Differentiate between levels of expertise. </li></ul><ul><li>Reveal underlying trends on technology adoption with regards to attitudes to technology, demographic information etc. </li></ul><ul><li>Reveal difference in teams physically co-located and physically separated during training. </li></ul>
    27. 27. Some Interesting Studies. <ul><li>Discussion on what can be done. </li></ul>
    28. 28. Take Home Simulators <ul><li>Re-use of existing resources such as simulation gaming platform has several advantages </li></ul><ul><li>Can provide practice on psychomotor and cognitive skills </li></ul><ul><li>Engaging and fun for trainees </li></ul><ul><li>Several students can study together </li></ul><ul><li>Connectivity proficiency scores can be transmitted to database over the Internet </li></ul><ul><li>Can be deployed anywhere, remote areas as well as developing countries </li></ul>
    29. 29. Methodology For Choosing Exercises Cognitive task analysis Suturing->{setting the needle->passing suture->tying} Matching observational Parameters in the real world And virtual world Monitor progress through mechanism that work in an ambient manner Adapt gaming scores to our needs
    30. 30. Wii and fine motor skills <ul><li>Fine motor skills based games are very suitable </li></ul><ul><li>Very high correlation with basic gestures of surgery </li></ul><ul><li>Quantitatively we found that hand movement acceleration, and joint angles showed 0.78 to 0.91% correlation. </li></ul><ul><li>Cons: doesn’t have the fulcrum effect and significant weight. </li></ul>
    31. 31. Apparatus <ul><li>Gaming Extensions to Wii can be modified for surgical probe based interactions. </li></ul><ul><li>WiiMote Extension </li></ul><ul><li>Movement Constrainer </li></ul>Location of wiimote
    32. 32. Full System in Action
    33. 33. Study
    34. 34. Robotic Surgery Simulator
    35. 35. Analysis <ul><li>Exploring the error innovation continuum </li></ul><ul><li>Focus in analysis on best practices and validity of checklists in Trauma and critical care settings. </li></ul><ul><li>We performed analysis on ATLS training and execution and used the tool to analyze if activities as cited in checklists are followed. </li></ul><ul><li>Hypothesis: Experts assign criticality to steps in a procedure and tend to follow high criticality steps with precision but innovate in low criticality steps. </li></ul><ul><li>Criticality obtained by focus group sessions with physicians, nurses and residents with checklists. </li></ul>
    36. 36. Adherence to Best Practices and Accuracy Percentage
    37. 37. Future Work: Technical <ul><li>An activity segmentation algorithm: using the accelerometer, we aim to define a method to segment data into activities and then recognize them. </li></ul><ul><li>Based on Kahol (2003) , we propose to use Bayesian techniques to identify activity boundaries automatically. </li></ul><ul><li>Pilot data gathered at ASU and UT Houston. </li></ul>
    38. 38. Future Work: Analytical <ul><li>Employ the tool to further explore error-innovation continuum in critical care environment. </li></ul><ul><li>One mans error is another mans innovation. </li></ul><ul><li>This evolves extensively in critical care environments and ASU and BannerHealth will develop methodology to explore how errors and innovations evolve in critical care environments. </li></ul>
    39. 39. Future Work Interventions <ul><li>The developed tools will be employed with new year residents for training in a control group experimental group paradigm. </li></ul><ul><li>We expect that the group with exposure to workflow will be better integrated in the trauma unit workflow as compared to the group that doesn’t. </li></ul>
    40. 40. Conclusions <ul><li>Virtual from real is fun and effective… </li></ul><ul><li>Focus on the activities and interactions visual brilliance is over-rated and your main aim is education anyway… </li></ul><ul><li>Go with environments that allow programming… </li></ul><ul><li>What are you waiting for… build it! </li></ul>
    41. 41. thanks Partner LABORATORY