Gesture-Based Computing• Gestural systems offer a path for a more holistic, human interaction of people with technology• Gesture-based computing is potentially transformative technology, because it changes not only the physical and mechanical aspects of interaction with computers, but also our perception of collaborating with computers.
Gestural Computing Challenges• Need for standards (Different systems may use different conventions)• gestures are ephemeral, they do not leave behind any record of their path• Gestures also vary in meaning and use by culture
Importance of interaction design principles in creating new gestural interfaces• Visibility (also called perceived affordances or signifiers)• Feedback• Consistency (also known as standards)• Non-destructive operations (hence the importance of undo)• Discoverability: All operations can be discovered by systematic exploration of menus.• Scalability: The operation should work on all screen sizes, small and large.• Reliability: Operations should work. Norman (2010)
Recognizing the power of gestures for learningSource: Power Posing – Carney et al, 2010
Mixed Reality Using TMG to fuse video game with real world interactionSource: Marta Ferraz, Doctoral research University of Texas at Austin
Moving into a multimodal world Source: Nokia Research Center Future Technologies
A Day Made of GlassSource: A Day Made of Glass Corning
Wearable Technology ViconRevue – Life Logs Sony razor thin display that bends like paper LED display you can wear as contact lens
TMG provides new ways to support Multimodal Learning• Multiple means of representation• Multiple means of action and expression• Multiple means of engagement
Questions for Multimodal Learning Design• Scaffolding needed to prepare students to effectively use multimedia visual representations and touch, motion and gesture technologies for learning• How to minimize cognitive overload for students in interacting in a multimodal environment
Now able to capture and analyze astudent’s interactions on touch devices
Will require new ways to analyze and visualize the new massive data setsSource: Taylor Martin :Analysis of student actions with virtual manipulativeson interactive whiteboard
Education Visualization LabUniversity of Texas at Austin Learning Technology Center and Texas Advanced Computing Center
Implications of Touch and Gesture Technologies• The future trajectory of touch and gesture technologies have profound implications for: – the creation of new ways of interacting with computers and with each other – the design of new learning environments that engage more of the sensorium and are more authentic and engaging for students
The Future of Touch and Gesture Technologies for Learning• To achieve this goal, we need to: – break out of disciplinary silos – Form new interdisciplinary teams of kinesiology experts. learning scientists, neuroscientists, software designers, engineers, computer scientists and artists working together
The new TMG technologies offer excitingopportunities to move toward a new vision of learning built upon embodied cognition.