SlideShare a Scribd company logo
1 of 22
Download to read offline
Distributed Multimodal Information Processing Group                                   Technische Universität München




                      MobiMed:
       Comparing Object Identification Techniques
                  on Smartphones

               Andreas Möller1, Stefan Diewald1, Luis Roalter1, Matthias Kranz2

                                   1Technische
                                      Universität München, Germany
             2Luleå University of Technology, Department of Computer Science,

                     Electrical and Space Engineering, Luleå, Sweden

                                                             October 15, 2012
                                                      NordiCHI, Copenhagen, Denmark
Distributed Multimodal Information Processing Group            Technische Universität München



Outline


                   Background and Motivation



                           Scenario and Prototype



                           User Study



                   Discussion and Conclusion



Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    2
Distributed Multimodal Information Processing Group            Technische Universität München



Background and Motivation

•     Idea of bridging the gap between the physical and the virtual world
      for easier interaction and additional functionality
       –  Connect physical objects with virtual representations by tags
          (Want et al., 1999)
       –  Physical mobile interaction (Rukzio, 2006)

•     Investigation and comparison of different interaction techniques done earlier,
      BUT:
       –  meanwhile outdated technologies (e.g. IR)
       –  older comparisons based on (nowadays) limited hardware
          (VGA cameras, small screens, slow mobile CPUs)
       –  new technologies have emerged (e.g. vision-based approaches)
       –  user knowledge and experience has changed

 Suggesting a new comparison of (state-of-the-art) interaction techniques

Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    3
Distributed Multimodal Information Processing Group            Technische Universität München



Outline


                   Background and Motivation



                           Scenario and Prototype



                           User Study



                   Discussion and Conclusion



Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    4
Distributed Multimodal Information Processing Group            Technische Universität München



Scenario for Physical Mobile Interaction

•     MobiMed:
      identifying medication packages
      with the smartphone

•     Target groups: active people
      pursuing a healthy lifestyle, elderly
      people

•     Physical mobile interaction to get
      information on drugs
       –  package insert
       –  side effects
       –  active ingredients
       –  cross-correlations


Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    5
Distributed Multimodal Information Processing Group                        Technische Universität München



Investigated Interaction Types




           Touching                                                                Scanning
           (radio tags, e.g. NFC or RFID)                      (visual tags, e.g. bar codes)




           Pointing                                                              Text Input
           (tag-less vision-based identification)                       (e.g. name, ID, …)

Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                                6
Distributed Multimodal Information Processing Group            Technische Universität München



Excursus: Pointing (Vision-based Recognition)

•     Image processing is used to detect
      visual features of an image
•     A query in feature space returns
      similar images from a reference
      database
•     Good choice of feature type allows
      very reliable results (e.g. MSER)
       –  High distinctiveness (e.g. by
          using text-related features)
       –  Scale invariance (works at
          different distances)
       –  Rotation invariance (works at
          different angles)
•     Enabled by rise in mobile CPU
      performance (multi-core...)

Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    7
Distributed Multimodal Information Processing Group            Technische Universität München



Prototype

•     Implementation as Android application
•     47,000 drugs in query database
•     100,000 reference images




Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    8
Distributed Multimodal Information Processing Group            Technische Universität München



Outline


                   Background and Motivation



                           Scenario and Prototype



                           User Study



                   Discussion and Conclusion



Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    9
Distributed Multimodal Information Processing Group            Technische Universität München



Research Questions

•     RQ1: What advantages and disadvantages of identification techniques,
             as presented in MobiMed, can be determined?
       –  ...in terms of effectiveness? large-scale, online
       –  ...in terms of efficiency? lab

•     RQ2: Which method is preferred by users?
       –  ...a priori? large-scale, online
       –  ...after practical use? lab

•     RQ3: What potential do people see for MobiMed as a whole?
       –  ...a priori? large-scale, online
       –  ...after practical use? lab




Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    10
Distributed Multimodal Information Processing Group                     Technische Universität München



Methodology

•     Online study
       –  Human Intelligence Task at Amazon mTurk
       –  149 participants
                 •  74 females, 75 males
                 •  17-79 years (average: 31, standard deviation: 11)
        –  Questionnaire survey

•     Lab study
       –  16 participants
                 •  6 females, 10 males
                 •  22-69 years (average: 31, standard deviation: 12)
        –  Experimental task + Questionnaire survey
            •  Identification of 10 packages
               with each of four methods
            •  Within-subjects design, permuted order
Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                             11
Distributed Multimodal Information Processing Group                                        Technische Universität München



Results: RQ1 (Individual Method Comparison)

Method
                                               Advantages
                      Disadvantages
Scanning
                                             Quick, precise, high             Visual code + camera
                                                      familiarity
                     required, need to find and
                                                                                       focus on code
Touching
                                             Hassle-free, fool-proof,         NFC augmentation and
                                                      quick
                           NFC-capable phone
                                                                                       required, privacy skepticism
Pointing
                                             Intuitive to use, „most          Computational demand,
                                                      human form“ of interaction,      ambiguous results possible
                                                      works from any angle, works
                                                      also with catalog/website
                                                      images, no product tagging
                                                      required
Text
                                                 Highest familiarity, accurate,   High amount of typing,
                                                      search term flexibility
          misspelling, slow, difficult



Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                                                12
Distributed Multimodal Information Processing Group            Technische Universität München



Results: RQ1 (Efficiency)




Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    13
Distributed Multimodal Information Processing Group                                      Technische Universität München



Results: RQ2 (User Preferences)




                                                               -3 = strongly disagree, +3 =strongly agree



Observations/interpretations:
•  Touching was only #3 in online survey, but rated best in lab study
•  Possible explanation: low familiarity (as soon as people used it, they liked it)


Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                                              14
Distributed Multimodal Information Processing Group                              Technische Universität München



Results: RQ3 (Utility of Tool in Scenario)

•     Information sources on drugs:                            •    Suggestions for additional
       –  Doctor/pharmacist (75%)                                   features
       –  Package insert (69%)                                       –  Price comparison
       –  Books/internet (56%)                                       –  Active ingredient analysis
                                                                     –  Self-diagnose
•     Would you be interested in                                     –  Personalized medication
      MobiMed as alternative source for                                 management
      drug information? 88%

•     Would you use a system such as
      MobiMed? 82%

•     Average amount of money subjects
      would spend: $8.40 (aged >25:
      $14.01)

Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                                      15
Distributed Multimodal Information Processing Group            Technische Universität München



Results: RQ3 (Usability of Prototype)




Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    16
Distributed Multimodal Information Processing Group            Technische Universität München



Outline


                   Background and Motivation



                           Scenario and Prototype



                           User Study



                   Discussion and Conclusion



Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    17
Distributed Multimodal Information Processing Group            Technische Universität München



Discussion and Conclusion

•     Physical Mobile Interaction is popular and efficient
       –  Was preferred over conventional (text) search
       –  Was faster than text search
•     Touching and Scanning evaluated best
       –  Fastest and most popular physical mobile interaction methods
       –  Touching faster and more popular than scanning in lab study
       –  Scanning more popular in online survey (familiarity)
•     Vision-based Search (pointing) as future alternative?
       –  Natural; works for any object (no augmentation needed)
       –  Reliability/speed improvement needed, but almost as fast as scanning
•     Best method depends on intended scenario
•     General demand for medical apps




Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    18
Distributed Multimodal Information Processing Group                   Technische Universität München




                                          Thank you for your attention!
                                                 Questions?




                                                               ?
                                                               ?
                                andreas.moeller@tum.de
                       www.vmi.ei.tum.de/team/andreas-moeller.html

Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                           19
Distributed Multimodal Information Processing Group                           Technische Universität München



References

•     Slide 3:
        –  Rukzio, E. Physical mobile interactions: Mobile devices as pervasive mediators for interactions
           with the real world. PhD thesis, 2006
        –  Want, R., Fishkin, K., Gujar, A., and Harrison, B. Bridging physical and virtual worlds with
           electronic tags. In Proceedings of the SIGCHI conference on Human factors in computing
           systems: the CHI is the limit, ACM (1999), 370–377.
•     Slide 10: https://www.mturk.com/mturk/welcome

•     All other images: Microsoft ClipArt 2012




Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                                   20
Distributed Multimodal Information Processing Group            Technische Universität München



Paper Reference

•     Please find the associated paper at:
      http://dx.doi.org/10.1145/2399016.2399022

•     Please cite this work as follows:
•     Andreas Möller, Stefan Diewald, Luis Roalter, and Matthias Kranz. 2012.
      MobiMed: comparing object identification techniques on smartphones. In
      Proceedings of the 7th Nordic Conference on Human-Computer Interaction:
      Making Sense Through Design (NordiCHI '12). ACM, New York, NY, USA,
      31-40. DOI=10.1145/2399016.2399022 http://doi.acm.org/
      10.1145/2399016.2399022




Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                    21
Distributed Multimodal Information Processing Group                                 Technische Universität München



If you use BibTex, please use the following entry
to cite this work:



 @inproceedings{Moller:2012:MCO:2399016.2399022,
  author = {M"{o}ller, Andreas and Diewald, Stefan and Roalter, Luis and Kranz, Matthias},
  title = {MobiMed: comparing object identification techniques on smartphones},
  booktitle = {Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design},
  series = {NordiCHI '12},
  year = {2012},
  isbn = {978-1-4503-1482-4},
  location = {Copenhagen, Denmark},
  pages = {31--40},
  numpages = {10},
  url = {http://doi.acm.org/10.1145/2399016.2399022},
  doi = {10.1145/2399016.2399022},
  acmid = {2399022},
  publisher = {ACM},
  address = {New York, NY, USA},
  keywords = {object identification, physical mobile interaction, pointing, scanning, touching},
 }




Oct 15, 2012     A. Möller, S. Diewald, L. Roalter, M. Kranz                                                         22

More Related Content

Similar to MobiMed: Comparing Object Identification Techniques on Smartphones

Ambient Intelligence: Definitions and Application Areas
Ambient Intelligence: Definitions and Application AreasAmbient Intelligence: Definitions and Application Areas
Ambient Intelligence: Definitions and Application AreasFulvio Corno
 
Mobility&Udi 2011
Mobility&Udi 2011Mobility&Udi 2011
Mobility&Udi 2011TingRay Chang
 
Ci2004-10.doc
Ci2004-10.docCi2004-10.doc
Ci2004-10.docbutest
 
The Art and Science of Analyzing Software Data
The Art and Science of Analyzing Software DataThe Art and Science of Analyzing Software Data
The Art and Science of Analyzing Software DataCS, NcState
 
Research Opportunities - Interactive Visual Representations, Otto J. Anshus, ...
Research Opportunities - Interactive Visual Representations, Otto J. Anshus, ...Research Opportunities - Interactive Visual Representations, Otto J. Anshus, ...
Research Opportunities - Interactive Visual Representations, Otto J. Anshus, ...The Research Council of Norway, IKTPLUSS
 
UMOBILE: Universal, mobile-centric and opportunistic communications architecture
UMOBILE: Universal, mobile-centric and opportunistic communications architectureUMOBILE: Universal, mobile-centric and opportunistic communications architecture
UMOBILE: Universal, mobile-centric and opportunistic communications architecturePaulo Milheiro Mendes
 
AudrisMockus_MSR22.pdf
AudrisMockus_MSR22.pdfAudrisMockus_MSR22.pdf
AudrisMockus_MSR22.pdfTapajitDey1
 
Methods and tools for human centered ICT: from human values to real-life inno...
Methods and tools for human centered ICT: from human values to real-life inno...Methods and tools for human centered ICT: from human values to real-life inno...
Methods and tools for human centered ICT: from human values to real-life inno...Human Centered ICT
 
Multimodal Tutor - Adaptive feedback from multimodal experience capturing
Multimodal Tutor - Adaptive feedback from multimodal experience capturingMultimodal Tutor - Adaptive feedback from multimodal experience capturing
Multimodal Tutor - Adaptive feedback from multimodal experience capturingDaniele Di Mitri
 
The Internet of Things: What's next?
The Internet of Things: What's next? The Internet of Things: What's next?
The Internet of Things: What's next? PayamBarnaghi
 

Similar to MobiMed: Comparing Object Identification Techniques on Smartphones (20)

Decision-Point Panorama-Based Indoor Navigation
Decision-Point Panorama-Based Indoor NavigationDecision-Point Panorama-Based Indoor Navigation
Decision-Point Panorama-Based Indoor Navigation
 
The Smartphone as Mobile Authorization Proxy
The Smartphone as Mobile Authorization ProxyThe Smartphone as Mobile Authorization Proxy
The Smartphone as Mobile Authorization Proxy
 
A Mobile Indoor Navigation System Interface Adapted to Vision-Based Localization
A Mobile Indoor Navigation System Interface Adapted to Vision-Based LocalizationA Mobile Indoor Navigation System Interface Adapted to Vision-Based Localization
A Mobile Indoor Navigation System Interface Adapted to Vision-Based Localization
 
Tool Support for Prototyping Interfaces
Tool Support for Prototyping InterfacesTool Support for Prototyping Interfaces
Tool Support for Prototyping Interfaces
 
Ambient Intelligence: Definitions and Application Areas
Ambient Intelligence: Definitions and Application AreasAmbient Intelligence: Definitions and Application Areas
Ambient Intelligence: Definitions and Application Areas
 
Mobility&Udi 2011
Mobility&Udi 2011Mobility&Udi 2011
Mobility&Udi 2011
 
Gamification-supported Exploration of Natural User Interfaces
Gamification-supported Exploration of Natural User InterfacesGamification-supported Exploration of Natural User Interfaces
Gamification-supported Exploration of Natural User Interfaces
 
Ci2004-10.doc
Ci2004-10.docCi2004-10.doc
Ci2004-10.doc
 
Medical image analysis and big data evaluation infrastructures
Medical image analysis and big data evaluation infrastructuresMedical image analysis and big data evaluation infrastructures
Medical image analysis and big data evaluation infrastructures
 
MobiliNet: A Social Network for Optimized Mobility
MobiliNet: A Social Network for Optimized MobilityMobiliNet: A Social Network for Optimized Mobility
MobiliNet: A Social Network for Optimized Mobility
 
Update Behavior in App Markets and Security Implications: A Case Study in Goo...
Update Behavior in App Markets and Security Implications: A Case Study in Goo...Update Behavior in App Markets and Security Implications: A Case Study in Goo...
Update Behavior in App Markets and Security Implications: A Case Study in Goo...
 
The Art and Science of Analyzing Software Data
The Art and Science of Analyzing Software DataThe Art and Science of Analyzing Software Data
The Art and Science of Analyzing Software Data
 
Nessos
NessosNessos
Nessos
 
Research Opportunities - Interactive Visual Representations, Otto J. Anshus, ...
Research Opportunities - Interactive Visual Representations, Otto J. Anshus, ...Research Opportunities - Interactive Visual Representations, Otto J. Anshus, ...
Research Opportunities - Interactive Visual Representations, Otto J. Anshus, ...
 
Smart homes
Smart homesSmart homes
Smart homes
 
UMOBILE: Universal, mobile-centric and opportunistic communications architecture
UMOBILE: Universal, mobile-centric and opportunistic communications architectureUMOBILE: Universal, mobile-centric and opportunistic communications architecture
UMOBILE: Universal, mobile-centric and opportunistic communications architecture
 
AudrisMockus_MSR22.pdf
AudrisMockus_MSR22.pdfAudrisMockus_MSR22.pdf
AudrisMockus_MSR22.pdf
 
Methods and tools for human centered ICT: from human values to real-life inno...
Methods and tools for human centered ICT: from human values to real-life inno...Methods and tools for human centered ICT: from human values to real-life inno...
Methods and tools for human centered ICT: from human values to real-life inno...
 
Multimodal Tutor - Adaptive feedback from multimodal experience capturing
Multimodal Tutor - Adaptive feedback from multimodal experience capturingMultimodal Tutor - Adaptive feedback from multimodal experience capturing
Multimodal Tutor - Adaptive feedback from multimodal experience capturing
 
The Internet of Things: What's next?
The Internet of Things: What's next? The Internet of Things: What's next?
The Internet of Things: What's next?
 

More from Distributed Multimodal Information Processing Group

More from Distributed Multimodal Information Processing Group (9)

Experimental Evaluation of User Interfaces for Visual Indoor Navigation
Experimental Evaluation of User Interfaces for Visual Indoor NavigationExperimental Evaluation of User Interfaces for Visual Indoor Navigation
Experimental Evaluation of User Interfaces for Visual Indoor Navigation
 
Visual Authentication - A Secure Single Step Authentication for User Authoriz...
Visual Authentication - A Secure Single Step Authentication for User Authoriz...Visual Authentication - A Secure Single Step Authentication for User Authoriz...
Visual Authentication - A Secure Single Step Authentication for User Authoriz...
 
Mit mobilem Lernen zur erweiterten Methodenkompetenz
Mit mobilem Lernen zur erweiterten MethodenkompetenzMit mobilem Lernen zur erweiterten Methodenkompetenz
Mit mobilem Lernen zur erweiterten Methodenkompetenz
 
Investigating Self-Reporting Behavior in Long-Term Studies
Investigating Self-Reporting Behavior in Long-Term StudiesInvestigating Self-Reporting Behavior in Long-Term Studies
Investigating Self-Reporting Behavior in Long-Term Studies
 
GymSkill - A Personal Trainer for Physical Exercises
GymSkill - A Personal Trainer for Physical ExercisesGymSkill - A Personal Trainer for Physical Exercises
GymSkill - A Personal Trainer for Physical Exercises
 
The Healthcare and Motivation Seat - A Survey with the GewoS Chair
The Healthcare and Motivation Seat - A Survey with the GewoS ChairThe Healthcare and Motivation Seat - A Survey with the GewoS Chair
The Healthcare and Motivation Seat - A Survey with the GewoS Chair
 
DriveAssist – A V2X-Based Driver Assistance System for Android
DriveAssist – A V2X-Based Driver Assistance System for Android DriveAssist – A V2X-Based Driver Assistance System for Android
DriveAssist – A V2X-Based Driver Assistance System for Android
 
Distributed Networks within ROS: Challenges and Possibilities
Distributed Networks within ROS: Challenges and PossibilitiesDistributed Networks within ROS: Challenges and Possibilities
Distributed Networks within ROS: Challenges and Possibilities
 
MobiDics: Cooperative Mobile e-Learning for Teachers
MobiDics: Cooperative Mobile e-Learning for TeachersMobiDics: Cooperative Mobile e-Learning for Teachers
MobiDics: Cooperative Mobile e-Learning for Teachers
 

Recently uploaded

Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuidePixlogix Infotech
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform EngineeringMarcus Vechiato
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization -  The evolution of API governanceAPI Governance and Monetization -  The evolution of API governance
API Governance and Monetization - The evolution of API governanceWSO2
 
Decarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceDecarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceIES VE
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsLeah Henrickson
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)Samir Dash
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAnitaRaj43
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxFIDO Alliance
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityWSO2
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityVictorSzoltysek
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistandanishmna97
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewDianaGray10
 
Modernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaModernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaWSO2
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingScyllaDB
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontologyjohnbeverley2021
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....rightmanforbloodline
 

Recently uploaded (20)

Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
JavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate GuideJavaScript Usage Statistics 2024 - The Ultimate Guide
JavaScript Usage Statistics 2024 - The Ultimate Guide
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization -  The evolution of API governanceAPI Governance and Monetization -  The evolution of API governance
API Governance and Monetization - The evolution of API governance
 
Decarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceDecarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational Performance
 
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on ThanabotsContinuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
Continuing Bonds Through AI: A Hermeneutic Reflection on Thanabots
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Intro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptxIntro to Passkeys and the State of Passwordless.pptx
Intro to Passkeys and the State of Passwordless.pptx
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
UiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overviewUiPath manufacturing technology benefits and AI overview
UiPath manufacturing technology benefits and AI overview
 
Modernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using BallerinaModernizing Legacy Systems Using Ballerina
Modernizing Legacy Systems Using Ballerina
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
 

MobiMed: Comparing Object Identification Techniques on Smartphones

  • 1. Distributed Multimodal Information Processing Group Technische Universität München MobiMed: Comparing Object Identification Techniques on Smartphones Andreas Möller1, Stefan Diewald1, Luis Roalter1, Matthias Kranz2 1Technische Universität München, Germany 2Luleå University of Technology, Department of Computer Science, Electrical and Space Engineering, Luleå, Sweden October 15, 2012 NordiCHI, Copenhagen, Denmark
  • 2. Distributed Multimodal Information Processing Group Technische Universität München Outline Background and Motivation Scenario and Prototype User Study Discussion and Conclusion Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 2
  • 3. Distributed Multimodal Information Processing Group Technische Universität München Background and Motivation •  Idea of bridging the gap between the physical and the virtual world for easier interaction and additional functionality –  Connect physical objects with virtual representations by tags (Want et al., 1999) –  Physical mobile interaction (Rukzio, 2006) •  Investigation and comparison of different interaction techniques done earlier, BUT: –  meanwhile outdated technologies (e.g. IR) –  older comparisons based on (nowadays) limited hardware (VGA cameras, small screens, slow mobile CPUs) –  new technologies have emerged (e.g. vision-based approaches) –  user knowledge and experience has changed  Suggesting a new comparison of (state-of-the-art) interaction techniques Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 3
  • 4. Distributed Multimodal Information Processing Group Technische Universität München Outline Background and Motivation Scenario and Prototype User Study Discussion and Conclusion Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 4
  • 5. Distributed Multimodal Information Processing Group Technische Universität München Scenario for Physical Mobile Interaction •  MobiMed: identifying medication packages with the smartphone •  Target groups: active people pursuing a healthy lifestyle, elderly people •  Physical mobile interaction to get information on drugs –  package insert –  side effects –  active ingredients –  cross-correlations Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 5
  • 6. Distributed Multimodal Information Processing Group Technische Universität München Investigated Interaction Types Touching Scanning (radio tags, e.g. NFC or RFID) (visual tags, e.g. bar codes) Pointing Text Input (tag-less vision-based identification) (e.g. name, ID, …) Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 6
  • 7. Distributed Multimodal Information Processing Group Technische Universität München Excursus: Pointing (Vision-based Recognition) •  Image processing is used to detect visual features of an image •  A query in feature space returns similar images from a reference database •  Good choice of feature type allows very reliable results (e.g. MSER) –  High distinctiveness (e.g. by using text-related features) –  Scale invariance (works at different distances) –  Rotation invariance (works at different angles) •  Enabled by rise in mobile CPU performance (multi-core...) Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 7
  • 8. Distributed Multimodal Information Processing Group Technische Universität München Prototype •  Implementation as Android application •  47,000 drugs in query database •  100,000 reference images Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 8
  • 9. Distributed Multimodal Information Processing Group Technische Universität München Outline Background and Motivation Scenario and Prototype User Study Discussion and Conclusion Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 9
  • 10. Distributed Multimodal Information Processing Group Technische Universität München Research Questions •  RQ1: What advantages and disadvantages of identification techniques, as presented in MobiMed, can be determined? –  ...in terms of effectiveness? large-scale, online –  ...in terms of efficiency? lab •  RQ2: Which method is preferred by users? –  ...a priori? large-scale, online –  ...after practical use? lab •  RQ3: What potential do people see for MobiMed as a whole? –  ...a priori? large-scale, online –  ...after practical use? lab Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 10
  • 11. Distributed Multimodal Information Processing Group Technische Universität München Methodology •  Online study –  Human Intelligence Task at Amazon mTurk –  149 participants •  74 females, 75 males •  17-79 years (average: 31, standard deviation: 11) –  Questionnaire survey •  Lab study –  16 participants •  6 females, 10 males •  22-69 years (average: 31, standard deviation: 12) –  Experimental task + Questionnaire survey •  Identification of 10 packages with each of four methods •  Within-subjects design, permuted order Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 11
  • 12. Distributed Multimodal Information Processing Group Technische Universität München Results: RQ1 (Individual Method Comparison) Method Advantages Disadvantages Scanning Quick, precise, high Visual code + camera familiarity required, need to find and focus on code Touching Hassle-free, fool-proof, NFC augmentation and quick NFC-capable phone required, privacy skepticism Pointing Intuitive to use, „most Computational demand, human form“ of interaction, ambiguous results possible works from any angle, works also with catalog/website images, no product tagging required Text Highest familiarity, accurate, High amount of typing, search term flexibility misspelling, slow, difficult Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 12
  • 13. Distributed Multimodal Information Processing Group Technische Universität München Results: RQ1 (Efficiency) Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 13
  • 14. Distributed Multimodal Information Processing Group Technische Universität München Results: RQ2 (User Preferences) -3 = strongly disagree, +3 =strongly agree Observations/interpretations: •  Touching was only #3 in online survey, but rated best in lab study •  Possible explanation: low familiarity (as soon as people used it, they liked it) Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 14
  • 15. Distributed Multimodal Information Processing Group Technische Universität München Results: RQ3 (Utility of Tool in Scenario) •  Information sources on drugs: •  Suggestions for additional –  Doctor/pharmacist (75%) features –  Package insert (69%) –  Price comparison –  Books/internet (56%) –  Active ingredient analysis –  Self-diagnose •  Would you be interested in –  Personalized medication MobiMed as alternative source for management drug information? 88% •  Would you use a system such as MobiMed? 82% •  Average amount of money subjects would spend: $8.40 (aged >25: $14.01) Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 15
  • 16. Distributed Multimodal Information Processing Group Technische Universität München Results: RQ3 (Usability of Prototype) Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 16
  • 17. Distributed Multimodal Information Processing Group Technische Universität München Outline Background and Motivation Scenario and Prototype User Study Discussion and Conclusion Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 17
  • 18. Distributed Multimodal Information Processing Group Technische Universität München Discussion and Conclusion •  Physical Mobile Interaction is popular and efficient –  Was preferred over conventional (text) search –  Was faster than text search •  Touching and Scanning evaluated best –  Fastest and most popular physical mobile interaction methods –  Touching faster and more popular than scanning in lab study –  Scanning more popular in online survey (familiarity) •  Vision-based Search (pointing) as future alternative? –  Natural; works for any object (no augmentation needed) –  Reliability/speed improvement needed, but almost as fast as scanning •  Best method depends on intended scenario •  General demand for medical apps Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 18
  • 19. Distributed Multimodal Information Processing Group Technische Universität München Thank you for your attention! Questions? ? ? andreas.moeller@tum.de www.vmi.ei.tum.de/team/andreas-moeller.html Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 19
  • 20. Distributed Multimodal Information Processing Group Technische Universität München References •  Slide 3: –  Rukzio, E. Physical mobile interactions: Mobile devices as pervasive mediators for interactions with the real world. PhD thesis, 2006 –  Want, R., Fishkin, K., Gujar, A., and Harrison, B. Bridging physical and virtual worlds with electronic tags. In Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, ACM (1999), 370–377. •  Slide 10: https://www.mturk.com/mturk/welcome •  All other images: Microsoft ClipArt 2012 Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 20
  • 21. Distributed Multimodal Information Processing Group Technische Universität München Paper Reference •  Please find the associated paper at: http://dx.doi.org/10.1145/2399016.2399022 •  Please cite this work as follows: •  Andreas Möller, Stefan Diewald, Luis Roalter, and Matthias Kranz. 2012. MobiMed: comparing object identification techniques on smartphones. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design (NordiCHI '12). ACM, New York, NY, USA, 31-40. DOI=10.1145/2399016.2399022 http://doi.acm.org/ 10.1145/2399016.2399022 Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 21
  • 22. Distributed Multimodal Information Processing Group Technische Universität München If you use BibTex, please use the following entry to cite this work: @inproceedings{Moller:2012:MCO:2399016.2399022, author = {M"{o}ller, Andreas and Diewald, Stefan and Roalter, Luis and Kranz, Matthias}, title = {MobiMed: comparing object identification techniques on smartphones}, booktitle = {Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design}, series = {NordiCHI '12}, year = {2012}, isbn = {978-1-4503-1482-4}, location = {Copenhagen, Denmark}, pages = {31--40}, numpages = {10}, url = {http://doi.acm.org/10.1145/2399016.2399022}, doi = {10.1145/2399016.2399022}, acmid = {2399022}, publisher = {ACM}, address = {New York, NY, USA}, keywords = {object identification, physical mobile interaction, pointing, scanning, touching}, } Oct 15, 2012 A. Möller, S. Diewald, L. Roalter, M. Kranz 22