Successfully reported this slideshow.

Model-based Measurement of Interaction in Mobile Multimodal Environments

1

Share

1 of 17
1 of 17

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Model-based Measurement of Interaction in Mobile Multimodal Environments

  1. 1. Model-based Measurement of Interaction in Mobile Multimodal Environments Pedro Mateo – Stefan Hillmann aMMI - October 2012 Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 1 / 17
  2. 2. Agenda Model-based Measurement of Interaction in Mobile Multimodal Environments: 1 Research Questions motivation, questions and proposed solutions 2 Research and Development MMI metrics, MMI model and Android implementation 3 Conclusion advantages, disadvantages and research directions Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 2 / 17
  3. 3. I Research Questions
  4. 4. Motivation We analyze interaction in different systems... ...but independently for each system Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 4 / 17
  5. 5. Motivation And now, for each system... different quantification/evaluation metrics different representation formats different interaction analysis tools multimodal communication is not analyzed as a whole Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 5 / 17
  6. 6. Research questions and proposed solutions Question 1: MMI assessed on a common level for different modalities? ⇒ Use common and generic metrics for all modalities Question 2: Compare interactions observed in different MMI contexts? ⇒ Structure metrics in a common representation (model) Question 3: Analyze MMI dinamically, step-by-step, and using real-time data? ⇒ Use a run-time model Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 6 / 17
  7. 7. II Research and Development
  8. 8. Defining multimodal metrics Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 8 / 17
  9. 9. Modeling multimodal interaction in EMF Metacommunication User Metacommunication System Metacommunication Speech Output Speech Input GUI Output GUI Input User Turn System Turn Gesture Output Gesture Input Cooperativity Modality Turn Dialog Synchrony Modality Change Task Appropriateness Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 9 / 17
  10. 10. Modeling multimodal interaction in EMF Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 10 / 17
  11. 11. Android implementation Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 11 / 17
  12. 12. Android implementation how does it work? Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 12 / 17
  13. 13. III Conclusion
  14. 14. Good stuff... common metrics + uniform representation ⇒ unified criteria to quantify MMI in different scenarios comparison is allowed, model transformation is possible different modalities are on a level ⇒ single thread dynamic representation of MMI ⇒ run-time applications reuse of metrics and tools is improved Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 14 / 17
  15. 15. Not so good stuff... some data can’t be automatically collected criteria to record parameters in different modalities? lack of plasticity ⇒ user and context information to enrich the analysis process the HOW of interaction is not described Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 15 / 17
  16. 16. And now what? Integrate this solution in real projects Use the framework for real analysis Make the model context-aware, more “mobile” Explore new possibilities of the model E.g. behavioral biometric authentication Project info: http://www.catedrasaes.org/wiki/MIM Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 16 / 17
  17. 17. Acknowledgements Pedro Mateo – Stefan Hillmann Model-based Interaction Measurement aMMI - October 2012 17 / 17

×