A Step-by-step guide to Building a Mobile Outdoor AR Application

2,442 views

Published on

A tutorial given at MGIA, SIGGRAPH Asia 2013 on building mobile AR application using the Outdoor AR Library.

Published in: Technology, Art & Photos
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,442
On SlideShare
0
From Embeds
0
Number of Embeds
625
Actions
Shares
0
Downloads
63
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

A Step-by-step guide to Building a Mobile Outdoor AR Application

  1. 1. A Step-by-step guide to Building a Mobile Outdoor AR Application Gun Lee 21 Nov. 2013
  2. 2. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Outdoor AR Browsers Wikitude Layar Junaio
  3. 3. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS SW Dev. Tools & Frameworks  High-level content description  AR Browsers  Hard to customize app logic and UI  Low-level functional modules  Programming libraries  Lack of higher-level abstraction of AR scene and content (BYO)
  4. 4. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Outdoor AR Framework  High-level abstraction and low-level programming support in a seamless way.  Low-level functional modules - Flexible and highly customizable  Ready-to-use high level components - Focus on building the interface, behavior and content of the application rather than system integration  The high-level and low-level tools share the common scene (or POI) data structure - Both skilled and novice developers can work together under the same framework.
  5. 5. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Apps built with OAR Framework [CityViewAR 2011] [CCDU 3D 2012] [GeoBoids 3D 2012] [AntarcticAR 2012]
  6. 6. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Outdoor AR Framework  Covers full tier of ready-to-use components, yet highly customizable Mobile Device (Client) Server Mobile AR Application Application Service Mobile SW Library Data Communication Server SW Library (Optional) Scene data Media data Mobile OS Scene data Media data Server OS
  7. 7. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Server Software and Tools Mobile Client (App) Server Web Client (Browser) AR View Web-based Authoring Tool HTML Data Manager RESTful API Javascript Local cache Scene Database
  8. 8. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Server Software and Tools  Web-based Authoring Tool  Please sign up for closed beta – Jan 2014
  9. 9. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Outdoor AR Library  For mobile app (client)  Android (v2.2 or above)  Google Maps API  Scenegraph based rendering engine  OpenGL ES (1.1 and 2.0)  Free download from our website: http://www.hitlabnz.org/mobileAR
  10. 10. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Outdoor AR Library Structure Service Component Layer Views Controllers Models Map View AR View List View Map Component AR Component List Component Scene Data Manager Tracking Data Manager Functional Component Layer Scene Data Structures Tracking Sensors 3D Graphics Rendering 3D Sound UI Views Tools & Utilities
  11. 11. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Model-View-Component Controller Application Logic & Data Binding Model View Scene & Tracking Data Managers Visualization & UI Elements
  12. 12. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Mobile App Software Library  Customization by extending components  Customized AR scenes - Customize scene class data structure  Customized user interface - Customize UI layout - Customize UI behavior (e.g. onSceneSelected) - Motion and touch gesture based interaction
  13. 13. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Step-by-step Guide      Setup the development environment Create an Android app Using the Outdoor AR Library in your app Adding outdoor AR scenes with 3D models Customizing UI  Run your app on a Google Glass
  14. 14. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Setup the dev environment  Prerequisite  Android Development Environment - http://developer.android.com/sdk/index.html - Eclipse, Android SDK  Google Play Services Library (for maps)  Download the Outdoor AR Library  http://www.hitlabnz.org/mobileAR - Download, Tutorials, Forum
  15. 15. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Setup the dev environment  Import the library and sample projects  OutdoorARLibrary  SampleOAComponent  Dependency  Google (not Android) API 4.1.2 (API Level 16)  Google Play Services Library - extras/google/google_play_services/libproject  Android Support Library 4 - extras/android/compatibility/v4/ android-support-v4.jar
  16. 16. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Create an Android app  Create a new Android Application project  Blank Activity - “An activity is a single, focused thing that the user can do.” – Android SDK API Reference  Anatomy of an Android App project     src res & gen assets AndroidManifest.xml  Add a “Hello MGIA” button
  17. 17. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Using the OA Library in your app  Add the Outdoor AR Library to your project’s build path  Add permissions in the manifest  Add a new AR view (activity)  Create a custom class inheriting OAARComponentBase  Add activity to the manifest - Full screen landscape style preferred  Wire a button to start the AR view
  18. 18. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Adding scenes with 3D models  Preparing 3D model  OBJ/MTL with JPEG or PNG textures  OpenGL Coordinate frame - X-right, Y-up, Z-out from screen - North = -Z  Placing the 3D model in the real world  Latitude, Longitude - http://itouchmap.com/latlong.html  Optionally, elevation from the ground plane  SketchUp – 3D model on Google Earth
  19. 19. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Adding scenes with 3D models  Adding 3D models to the project  assets/OutdoorAR/3Dmodels(/name/model.obj)  Adding scenes to the data manager  Override onSetupScenes()  Handling the scene selection event  Override onSceneSelected()  Using a mock location  Override onCreate()  Sensor manager’s enableMockLocation()
  20. 20. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Loading and managing scenes  Add scenes by code – difficult to manage  Load from assets or local file storage  OADataManagerAssets  OADataManagerLocal  SQLite or XML file  SQLite Database Browser  Default path - (assets/)OutdoorAR/scenes.db - (assets/)OutdoorAR/scenes.xml
  21. 21. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Custom scene class  OAScene  Lat/Lon, 3D model, Name, Description, Category  Custom scene classes extend OAScene  Override OADataManager’s loadScene() to load in custom properties of your scene  Cast OAScene to your custom class in onSceneSelected() or onTouchedScene()  Different scene classes can live together - if(scene instanceof MyCustomScene)
  22. 22. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Customizing UI  Add layout resource  Override setupUILayout()  LayoutInflater - read in the layout res file  addContentView() - overlay UI layout on top of AR view  findViewById() - get reference to the UI elements  onSceneSelected() vs. onTouchedScene()
  23. 23. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Map and List views  OAMapComponentBase, OAListComponentBase  Similar class structure with OAARComponentBase - Load scenes from a data manager - setupXYZ() methods - onSceneSelected() callback  Map components needs API key  Google Maps Android API v2 - OAGoogleMapsV2ComponentBase  https://developers.google.com/maps/documentation/an droid/start
  24. 24. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Run your app on Google Glass  Startup with AR  Set AR view as a launcher activity in the app manifest file  Aim to select  Use Timer to trigger touch events at the center of the screen  Override onResume() and onPause() to start/stop the timer
  25. 25. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Other tips for Google Glass  Touch gestures  onGenericMotionEvent()  Swipe forward/backward/up  Tap, double tap, tap and hold, etc.  Head motion  Add orientation listener to sensor manager - getSensorManager().addSensorListener()  Avoid conflicts with AR scene viewing motions - Time-outs
  26. 26. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS Other tips for Google Glass  GPS sensor blocked  Stream location from a phone through Bluetooth  Mock location  Turning off camera background  Application dependent  Override setupOptions()  enableCameraBackground()
  27. 27. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS CityViewAR for Glass Demo
  28. 28. Thank you!
  29. 29. SYMPOSIUM ON MOBILE GRAPHICS AND INTERACTIVE APPLICATIONS More Information  Website  http://www.hitlabnz.org/mobileAR  http://arforglass.org  Gun Lee  gun.lee@hitlabnz.org  Mark Billinghurst  mark.billinghurst@hitlabnz.org

×