0
Tutorial:
Building Mobile AR Applications using
the Outdoor AR Library
Gun A. Lee

Mark Billinghurst

The Human Interface ...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Welcome
 Mark Billinghurst
 Director, HIT Lab NZ
 PhD, Univ....
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Schedule








Introduction (Mark)
Outdoor AR History ...
Introduction
Mark Billinghurst
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

What is Augmented Reality?
 Definition [Azuma 97]
 Combines r...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Outdoor AR
 Using mobile/wearable
systems to overlay AR conten...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Applications

 Tourism, Gaming, Architecture, Engineering,
Etc
Outdoor AR History
Mark Billinghurst
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Evolution of Mobile AR
Camera phone

Camera phone
- Thin client...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

MIT Wearable Computing
(1996)
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Mobile AR: Touring Machine (1997)
 University of Columbia
 Fe...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

MARS View

 Virtual tags overlaid on the real world
 “Informa...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Backpack/Wearable AR
1997 Backpack AR





Feiner’s Touring...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Mobile AR - Hardware
RTK correction Antenna
GPS
Antenna

HMD
Co...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

HIT Lab NZ Wearable AR (2004)
 Highly accurate outdoor AR
trac...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Image Registration

AR Stakeout Application
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Wearable AR Video
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Location Aware Phones (2008)

Motorola Droid

Nokia Navigator
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Outdoor Information Overlay (2009)
 Mobile phone based
 Tag r...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Google Glass (2013)
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

View Through Google Glass

Always available peripheral informat...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

 Hardware
 CPU TI OMAP 4430 – 1.2 Ghz
 16 GB SanDisk Flash,1...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Competitors
 Vuzix M100
 $999, profession

 Recon Jet
 $600...
Design Guidelines
Mark Billinghurst
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

AR UI Design
 Consider your user
 Follow good HCI principles
...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Consider Your User
 Consider context of user
 Physical, socia...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Adapting Existing Guidelines
 Mobile Phone AR
 Phone HCI Guid...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS







Applying Principles to Mobile
AR
Clean
Large Video V...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

AR vs. Non AR Design
Characteristics

Non-AR Interfaces

AR Int...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Maps vs. Junaio

 Google Maps
 2D, mouse driven, text/image h...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Design to Device Constraints
 Understand the platforms and des...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

AR as Perception Problem
 Goal of AR to fool human senses – cr...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Which Object is Closest?
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Depth Cues
Pictorial: visual cues
• Occlusion, texture, relati...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Use the Following Depth Cues
 Movement parallax
 Icon/Object ...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Provide Perspective Cue

 Eg ground plane grid
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Depth Perception
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Information Presentation
• Amount of information
• Clutter, com...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Twitter 360






www.twitter-360.com
iPhone application
Se...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Wikitude – www.mobilizy.com
Blah

Blah
Blah

Blah
Blah

Blah Bl...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Information Filtering
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Information Filtering
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Outdoor AR: Limited FOV
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Possible solutions
 Overview + Detail
 spatial separation; tw...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Zooming Views

 Zooming panorama, Zooming Map
Mulloni, A., Dün...
Example Applications
Mark Billinghurst
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

HIT Lab NZ Building Viewer
 Architectural Application
 Loads ...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

2011 Christchurch Earthquake
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Christchurch Today
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

CityViewAR

 Using AR to visualize Christchurch buildings
 3D...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

User Experience

 While walking in the real world people can
s...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

List View

List of all assets
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Map View

Icons for buildings, viewpoints,
panoramas
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Building History Data
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Photographic Images
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Panorama Images

360 degree photo bubbles
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Augmented Reality View
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
Building a Mobile Outdoor AR
Application
Gun Lee
Research Directions
Mark Billinghurst
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Looking to the Future

What’s Next?
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

What’s Next
 Key Research Problems
 Wide area tracking
 Inpu...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Robust Outdoor Tracking

 Hybrid Tracking
 Computer Vision, G...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Handheld Display

Reitmayr, G., & Drummond, T. W. (2006, Octobe...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Meta Gesture Interaction

 Depth sensor + Stereo see-through
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Meta Video
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Contact Lens Display
 Babak Parviz
 University Washington

 ...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Contact Lens Prototype
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Ego-Vision Collaboration

 Google Glass
 camera + processing ...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Ego-Vision Research
 System
 How do you capture the user's en...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Massive Multiuser
 Handheld/Outdoor AR for the first time allo...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

PERSONAL VIEW
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Augmented Reality 2.0
Infrastructure
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Leveraging Web 2.0
 Content retrieval using HTTP
 XML encoded...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

AR Standards + Markup Languages







KHARMA and Argon
A...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

AR + Human Computation
 Human Computation
 Real people solvin...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Human Computation
Architecture

 Add AR front end to typical H...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Scaling Up

 Seeing actions of millions of users in the
world
...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

AR + Smart Sensors + Social Networks

 Track population at cit...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Scaling Up

 AR on a City Scale
 Using mobile phone as ubiqui...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

WikiCity Rome

http://senseable.mit.edu/wikicity/rome/
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Orange Data for Development

 Orange made available 2.5 billio...
Conclusions
Mark Billinghurst, Gun Lee
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Conclusions
 Outdoor AR hardware available
 Handhelds (GPS, c...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

Next steps
 More tutorials on our website
 http://www.hitlabn...
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

http://www.hitlabnz.org/mobileAR
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

http://www.arforglass.org/
SYMPOSIUM ON MOBILE GRAPHICS
AND INTERACTIVE APPLICATIONS

More Information
 Website
 http://www.hitlabnz.org/mobileAR
...
Upcoming SlideShare
Loading in...5
×

Building Mobile AR Applications Using the Outdoor AR Library (Part 1)

2,074

Published on

The first part of a tutorial given on November 21st at the MGIA symposium at Siggraph Asia 2013. This shows how to build Outdoor AR applications using the HIT Lab NZ's Outdoor AR library. For more information see http://www.hitlabnz.org/index.php/products/mobile-ar-framework/334

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,074
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
86
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Need to consider side bar and other optionsOverlapping will occur Ground grid is available if it’s neededCan assume average value of height that the device is being held at – around 1.3 metersReasonable assumption that the ground is flat
  • Transcript of "Building Mobile AR Applications Using the Outdoor AR Library (Part 1)"

    1. 1. Tutorial: Building Mobile AR Applications using the Outdoor AR Library Gun A. Lee Mark Billinghurst The Human Interface Technology Laboratory New Zealand University of Canterbury, New Zealand 21 Nov 2013 09:00-10:15
    2. 2. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Welcome  Mark Billinghurst  Director, HIT Lab NZ  PhD, Univ. Washington  AR, Interaction Design  Gun Lee  Post Doc, HIT Lab NZ  PhD, POSTECH  AR, Android, Google Glass
    3. 3. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Schedule        Introduction (Mark) Outdoor AR History (Mark) Design Guidelines (Mark) Example Applications (Mark) Building a Mobile Outdoor AR Application (Gun) Research Directions (Mark) Conclusions (Gun + Mark)
    4. 4. Introduction Mark Billinghurst
    5. 5. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS What is Augmented Reality?  Definition [Azuma 97]  Combines real and virtual images  Is interactive in real-time  Content registered in 3D Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355-385.
    6. 6. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Outdoor AR  Using mobile/wearable systems to overlay AR content outdoors  Technical Requirements  Tracking (GPS, compass, etc)  Display (handheld, headmounted)  Input devices  Processing, networking
    7. 7. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Applications  Tourism, Gaming, Architecture, Engineering, Etc
    8. 8. Outdoor AR History Mark Billinghurst
    9. 9. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Evolution of Mobile AR Camera phone Camera phone - Thin client AR Wearable Computers Wearable AR Handheld AR Displays Camera phone - Self contained AR PDAs -Thin client AR PDAs -Self contained AR 1995 1997 2001 2003 2004
    10. 10. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS MIT Wearable Computing (1996)
    11. 11. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Mobile AR: Touring Machine (1997)  University of Columbia  Feiner, MacIntyre, Höllerer, Webster  Combines      See through head mounted display GPS tracking Orientation sensor Backpack PC (custom) Tablet input Feiner, S., MacIntyre, B., Höllerer, T., & Webster, A. (1997). A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment. Personal Technologies, 1(4), 208-217.
    12. 12. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS MARS View  Virtual tags overlaid on the real world  “Information in place”
    13. 13. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Backpack/Wearable AR 1997 Backpack AR     Feiner’s Touring Machine AR Quake (Thomas) Tinmith (Piekarski) MCAR (Reitmayr)  Bulky, HMD based
    14. 14. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Mobile AR - Hardware RTK correction Antenna GPS Antenna HMD Controller Example self-built working solution with PCI-based 3D graphics PCI 3D Graphics Board Tracker Controller PC104 Sound Card DC to DC Converter Wearable Computer CPU PC104 PCMCIA Battery GPS RTK correction Radio Hard Drive Serial Ports Columbia Touring Machine
    15. 15. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS HIT Lab NZ Wearable AR (2004)  Highly accurate outdoor AR tracking system  GPS, Inertial, RTK system  HMD  First prototype  Laptop based  Video see-through HMD  2-3 cm tracking accuracy
    16. 16. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Image Registration AR Stakeout Application
    17. 17. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Wearable AR Video
    18. 18. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Location Aware Phones (2008) Motorola Droid Nokia Navigator
    19. 19. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Outdoor Information Overlay (2009)  Mobile phone based  Tag real world locations  GPS + Compass input  Overlay graphics data on live video  Applications  Travel guide, Advertising, etc  Wikitude, Layar, Junaio, etc..  Android based, Public API released
    20. 20. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Google Glass (2013)
    21. 21. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS
    22. 22. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS View Through Google Glass Always available peripheral information display Combining computing, communications and content capture
    23. 23. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS  Hardware  CPU TI OMAP 4430 – 1.2 Ghz  16 GB SanDisk Flash,1 GB Ram  570mAh Battery  Input  5 mp camera, 720p recording, microphone  GPS, InvenSense MPU-9150 inertial sensor  Output  Bone conducting speaker  640x360 micro-projector display
    24. 24. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Competitors  Vuzix M100  $999, profession  Recon Jet  $600, more sensors, sports  Opinvent  500 Euro, multi-view mode  Motorola Golden-i  Rugged, remote assistance
    25. 25. Design Guidelines Mark Billinghurst
    26. 26. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS AR UI Design  Consider your user  Follow good HCI principles  Adapt HCI guidelines for AR  Design to device constraints  Design for perception/info presentation  Design for evaluation
    27. 27. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Consider Your User  Consider context of user  Physical, social, emotional, cognitive, etc  Outdoor AR User       Probably Mobile One hand interaction Short application use Need to be able to multitask Use in outdoor environment Enhance interaction with real world
    28. 28. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Adapting Existing Guidelines  Mobile Phone AR  Phone HCI Guidelines  Mobile HCI Guidelines  HMD Based AR  3D User Interface Guidelines  VR Interface Guidelines
    29. 29. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS      Applying Principles to Mobile AR Clean Large Video View Large Icons Text Overlay Feedback
    30. 30. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS AR vs. Non AR Design Characteristics Non-AR Interfaces AR Interfaces Object Graphics Mainly 2D Mainly 3D Object Types Mainly virtual objects Both virtual and physical objects Object behaviors Mainly passive objects Both passive and active objects Communication Mainly simple Mainly complex HCI methods Mainly explicit Both explicit and implicit  Design Guidelines  Design for 3D graphics + Interaction  Consider elements of physical world  Support implicit interaction
    31. 31. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Maps vs. Junaio  Google Maps  2D, mouse driven, text/image heavy, exocentric  Junaio  3D, location driven, simple graphics, egocentric
    32. 32. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Design to Device Constraints  Understand the platforms and design for limitations  Hardware, software platforms  Eg Outdoor AR game with handheld       Use large screen icons Consider screen reflectivity Support one-hand interaction Consider the viewing angle Do not tire users out physically Don’t require accurate tracking
    33. 33. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS AR as Perception Problem  Goal of AR to fool human senses – create illusion that real and virtual are merged  Depth      Size Occlusion Shadows Relative motion Etc..
    34. 34. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Which Object is Closest?
    35. 35. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Depth Cues Pictorial: visual cues • Occlusion, texture, relative brightness Kinetic: motion cues • Relative motion parallax, motion perspective Physiological: motion cues • Convergence, accommodation Binocular disparity • Two different eye images
    36. 36. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Use the Following Depth Cues  Movement parallax  Icon/Object size (for close objects)  Linear perspective  To add side perspective bar.  Overlapping  Works if the objects are big enough  Shades and shadows  Depends on the available computation
    37. 37. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Provide Perspective Cue  Eg ground plane grid
    38. 38. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS
    39. 39. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Depth Perception
    40. 40. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Information Presentation • Amount of information • Clutter, complexity • Representation of information • Navigation cues, POI representation • Placement of information • Head, body, world stabilized • View combination • Multiple views
    41. 41. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Twitter 360     www.twitter-360.com iPhone application See geo-located tweets in real world Twitter.com supports geo tagging
    42. 42. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Wikitude – www.mobilizy.com Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah Blah
    43. 43. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Information Filtering
    44. 44. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Information Filtering
    45. 45. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Outdoor AR: Limited FOV
    46. 46. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Possible solutions  Overview + Detail  spatial separation; two views  Focus + Context  merges both views into one view  Zooming  temporal separation
    47. 47. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Zooming Views  Zooming panorama, Zooming Map Mulloni, A., Dünser, A., & Schmalstieg, D. (2010, September). Zooming interfaces for augmented reality browsers. In Proceedings of the 12th international conference on Human computer interaction with mobile devices and services (pp. 161-170). ACM.
    48. 48. Example Applications Mark Billinghurst
    49. 49. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS HIT Lab NZ Building Viewer  Architectural Application  Loads 3D models  a OBJ/MTL format  Positions content in space  GPS, compass  Intuitive user interface  toolkit to modify the model  Targeting museum guide/outdoor site applications
    50. 50. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS 2011 Christchurch Earthquake
    51. 51. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Christchurch Today
    52. 52. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS CityViewAR  Using AR to visualize Christchurch buildings  3D buildings, 2D images, text, panoramas  AR View, Map view, List view  Available on Android market, iOS App Store
    53. 53. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS
    54. 54. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS User Experience  While walking in the real world people can see text, 2D images and 3D content
    55. 55. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS List View List of all assets
    56. 56. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Map View Icons for buildings, viewpoints, panoramas
    57. 57. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Building History Data
    58. 58. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Photographic Images
    59. 59. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Panorama Images 360 degree photo bubbles
    60. 60. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Augmented Reality View
    61. 61. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS
    62. 62. Building a Mobile Outdoor AR Application Gun Lee
    63. 63. Research Directions Mark Billinghurst
    64. 64. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Looking to the Future What’s Next?
    65. 65. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS What’s Next  Key Research Problems  Wide area tracking  Input Methods/Displays  Collaboration  Emerging Trends     AR + Social Networking AR Standards/Platforms AR + Human Computing Scaling up
    66. 66. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Robust Outdoor Tracking  Hybrid Tracking  Computer Vision, GPS, inertial  Going Out  Reitmayer & Drummond (Univ. Cambridge)
    67. 67. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Handheld Display Reitmayr, G., & Drummond, T. W. (2006, October). Going out: robust model-based tracking for outdoor augmented reality. In Mixed and Augmented Reality, 2006. ISMAR 2006. IEEE/ACM International Symposium on (pp. 109-118). IEEE.
    68. 68. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Meta Gesture Interaction  Depth sensor + Stereo see-through
    69. 69. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Meta Video
    70. 70. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Contact Lens Display  Babak Parviz  University Washington  MEMS components  Transparent elements  Micro-sensors  Challenges  Miniaturization  Assembly  Eye-safe
    71. 71. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Contact Lens Prototype
    72. 72. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Ego-Vision Collaboration  Google Glass  camera + processing + display + connectivity
    73. 73. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Ego-Vision Research  System  How do you capture the user's environment?  How do you provide good quality of service?  Interface  What visual and audio cues provide best experience?  How do you interact with the remote user?  Evaluation  How do you measure the quality of collaboration?
    74. 74. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Massive Multiuser  Handheld/Outdoor AR for the first time allows extremely high numbers of AR users  Requires  New types of applications/games  New infrastructure (server/client/peer-to-peer)  Content distribution…
    75. 75. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS PERSONAL VIEW
    76. 76. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Augmented Reality 2.0 Infrastructure
    77. 77. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Leveraging Web 2.0  Content retrieval using HTTP  XML encoded meta information  KML placemarks + extensions  Queries  Based on location (from GPS, image recognition)  Based on situation (barcode markers)  Syndication  Community servers for end-user content  Tagging  AR client subscribes to data feeds
    78. 78. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS AR Standards + Markup Languages       KHARMA and Argon ARML AREL Patterns if Interest X3D+ KML vs. RDF + Multimedia Markup Languages
    79. 79. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS
    80. 80. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS AR + Human Computation  Human Computation  Real people solving problems difficult for computers  Web-based, non real time  Little work on AR + HC  AR attributes  Shared point of view  Real world overlay  Location sensing What does this say?
    81. 81. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Human Computation Architecture  Add AR front end to typical HC platform
    82. 82. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Scaling Up  Seeing actions of millions of users in the world  Augmentation on city/country level
    83. 83. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS AR + Smart Sensors + Social Networks  Track population at city scale (mobile networks)  Match population data to external sensor data  medical, environmental, etc
    84. 84. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Scaling Up  AR on a City Scale  Using mobile phone as ubiquitous sensor  MIT Senseable City Lab  http://senseable.mit.edu/
    85. 85. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS WikiCity Rome http://senseable.mit.edu/wikicity/rome/
    86. 86. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Orange Data for Development  Orange made available 2.5 billion phone records  5 months calls from Ivory Coast  > 80 sample projects using data  eg: Monitoring human mobility for disease
    87. 87. Conclusions Mark Billinghurst, Gun Lee
    88. 88. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Conclusions  Outdoor AR hardware available  Handhelds (GPS, compass, camera), Wearables  Many possible applications  HIT Lab NZ Outdoor AR platform  Easy for building AR applications  Multi-view support, Client/Server interface  Cross platform (handheld, Glass, etc)
    89. 89. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Next steps  More tutorials on our website  http://www.hitlabnz.org/mobileAR  Tags, LocationEvents, etc.  Projects on Google Glass  http://arforglass.org  One week workshop in Feb 2014  Stay tuned for server components  Web-based Authoring Tool closed beta (Jan 2014)
    90. 90. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS http://www.hitlabnz.org/mobileAR
    91. 91. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS http://www.arforglass.org/
    92. 92. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS More Information  Website  http://www.hitlabnz.org/mobileAR  http://arforglass.org  Gun Lee  gun.lee@hitlabnz.org  Mark Billinghurst  mark.billinghurst@hitlabnz.org
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×