Gestural-Vocal Coordinated Interaction on Large Displays
•Download as PPTX, PDF•
0 likes•6 views
Presentation of a Late Breaking Results paper @ EICS 2022 The paper is available at https://dl.acm.org/doi/10.1145/3531706.3536457
Recommended
Recommended
More Related Content
Similar to Gestural-Vocal Coordinated Interaction on Large Displays
Similar to Gestural-Vocal Coordinated Interaction on Large Displays (20)
Recently uploaded
Recently uploaded (20)
Gestural-Vocal Coordinated Interaction on Large Displays
- 1. Gestural-Vocal Coordinated Interaction on Large Displays Vik Parthiban, Pattie Maes, Quentin Sellier, Arthur Sluÿters, and Jean Vanderdonckt
- 2. The idea Investigate interaction techniques with multimedia contents on large screens • TVs • Projectors • AR/VR headsets Focus on gestures and voice A first version was developed: Large User Interface (LUI)
- 3. Philosophy of LUI Accessible: • Web application • Cheap off-the-shelf gesture sensor • Google Voice assistant Extensible • ReactJS framework (for modularity) Non-discrete • Real-time hand tracking • Visual feedback of the hand ultraleap.com
- 4. What is LUI?
- 5. Large User Interface Lock screen Main menu Photos Videos 3D models
- 6. Large User Interface Lock screen Main menu Photos Videos 3D models
- 7. Large User Interface Lock screen Main menu Photos Videos 3D models
- 8. Large User Interface Lock screen Main menu Photos Videos 3D models
- 9. Large User Interface Lock screen Main menu Photos Videos 3D models
- 10. Extensible with other apps: • Gesture keyboard • Games • Maps • …
- 11. Implementation
- 12. Gesture protocol
- 13. ultraleap.com Leap Motion Controller • Inexpensive (<100$) • Quite accurate (~1mm) • SDK for retrieving skeleton data developer-archive.leapmotion.com
- 14. Gestures Mix of native LMC gestures and threshold-based recognition Pinch in Air Tap Rotate Push Swipe Blossom
- 16. Voice protocol
- 17. 5% 5% 5% 5% 5% 74% 26% 11% 5% 16% 21% 11% 75% 11% 5% 5% 47% 58% 17% 68% 58% 16% 8% 11% 32% Q1. I found the system unnecessarily complex Q2. I found the system very easy to use Q3. I like to use this interface for media browsing (photos,… Q4. Did adding the voice help you navigate the interface… Q5. Most people would learn to use this system very quickly Q6. I would recommend using this kind of interaction for… Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree 3.36 0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 M
- 18. Conclusion
- 19. Conclusion and Future Work • Coordinated interaction works mostly for gestures, less for vocal apart for some actions that are more complex than manipulation • No additional device for gesture recognition • Shift the focus to voice
- 20. Thank you for your attention!
Editor's Notes
- A paper on gestural-vocal coordinated interaction on large displays. This paper was co-written with Vik Parthiban and Pattie Maes from MIT, and Quentin Sellier, Jean Vanderdonckt and myself from UCLouvain.