Lecture 7 in the Glass Class course. Presented on February 21st 2014 by Mark Billinghurst. This lecture discusses directions for future research using Google Glass.
17. THE GLASS CLASS
To Make the Vision Real..
Hardware/software requirements
Intelligent systems
Contact lens displays
Free space hand/body tracking
Speech/gesture recognition
Etc..
Most importantly
Usability
18. THE GLASS CLASS
Environment Sensing
Create virtual mesh over real world
Update at 10 fps – can move real objects
Use by physics engine for collision detection (virtual/real)
Use by OpenScenegraph for occlusion and shadows
19. THE GLASS CLASS
Natural Hand Interaction
Using bare hands to interact with AR content
MS Kinect depth sensing
Real time hand tracking
Physics based simulation model
22. THE GLASS CLASS
Gesture Based Interaction
3 Gear Systems
Kinect/Primesense Sensor
Two hand tracking
http://www.threegear.com
23. THE GLASS CLASS
Gesture Interaction + AR
HMD AR View
Viewpoint tracking
Two hand input
Skeleton interaction, occlusion
24. THE GLASS CLASS
Multimodal Interaction
Combined speech and Gesture Input
Free-hand gesture tracking
Semantic fusion engine (speech + gesture input history)
25. THE GLASS CLASS
User Evaluation
Change object shape, colour and position
Results
MMI signif. faster (11.8s) than gesture alone (12.4s)
70% users preferred MMI (vs. 25% speech only)
Billinghurst, M., & Lee, M. (2012). Multimodal Interfaces for Augmented Reality. In Expanding the Frontiers
of Visual Analytics and Visualization (pp. 449-465). Springer London.
26. THE GLASS CLASS
Contact Lens Display
Babak Parviz
University Washington
MEMS components
Transparent elements
Micro-sensors
Challenges
Miniaturization
Assembly
Eye-safe
31. THE GLASS CLASS
Ego-Vision Research
System
How do you capture the user's environment?
How do you provide good quality of service?
Interface
What visual and audio cues provide best experience?
How do you interact with the remote user?
Evaluation
How do you measure the quality of collaboration?
32. THE GLASS CLASS
AR + Human Computation
Human Computation
Real people solving problems
difficult for computers
Web-based, non real time
Little work on AR + HC
AR attributes
Shared point of view
Real world overlay
Location sensing
What does this say?
33. THE GLASS CLASS
Human Computation Architecture
Add AR front end to typical HC platform
34. THE GLASS CLASS
AR + HC Research Questions
System
What architecture provides best performance?
What data is needed to be shared?
Interface
What cues are needed by the human computers?
What benefits does AR provide cf. web systems?
Evaluation
How can the system be evaluated?
35. THE GLASS CLASS
Scaling Up
Seeing actions of millions of users in the world
Augmentation on city/country level
36. THE GLASS CLASS
AR + Smart Sensors + Social Networks
Track population at city scale (mobile networks)
Match population data to external sensor data
medical, environmental, etc
Mine data to improve social services
39. THE GLASS CLASS
Orange Data for Development
Orange made available 2.5 billion phone records
5 months calls from Ivory Coast
> 80 sample projects using data
eg: Monitoring human mobility for disease modeling
40. THE GLASS CLASS
Research Questions
System
How can you capture the data reliably?
How can you aggregate and correlate the information?
Interface
What data provides the most values?
How can you visualize the information?
Evaluation
How do you measure the accuracy of the model?
44. THE GLASS CLASS
More Information
Mark Billinghurst
Email: mark.billinghurst@hitlabnz.org
Twitter: @marknb00
HIT Lab NZ
http://www.hitlabnz.org/