Augmented reality for consumer devices

  • 645 views
Uploaded on

In this presentation we explore three different approaches to Augmented Reality and we see examples of commercially available titles that use each. We explore the principals behind each approach and …

In this presentation we explore three different approaches to Augmented Reality and we see examples of commercially available titles that use each. We explore the principals behind each approach and the steps required by developers to build AR applications on their platform of choice. Finally, we take a look at libraries that developers can leverage to build AR applications quickly on Windows Phone.

More in: Business , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
645
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
10
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • http://msdn.microsoft.com/en-us/library/hh202956(v=VS.92).aspx
  • http://msdn.microsoft.com/en-us/library/hh202956(v=VS.92).aspx

Transcript

  • 1. Augmented Reality for Consumer Devices (and shortcuts for Windows Phone)Jared Bienzjbienz@microsoft.comjaredbienz.wordpress.com@jbienz
  • 2. What will be covered What is Augmented Reality? How AR is being used today Different approaches to AR (platform agnostic) Tools for building AR on Windows Phone Getting “crafty” Windows Phone Microsoft Corporation.
  • 3. What is Augmented Reality?“A live direct or indirect view of a physical, real-worldenvironment whose elements are augmented by computer-generated sensory input such as sound, video, graphics orGPS data.” - Wikipedia Windows Phone Microsoft Corporation.
  • 4. What is Augmented Reality?“A view of the world through a digital “lens”, which allows thescene to be changed or appear differently than the real-world environment.” - Jared Bienz Windows Phone Microsoft Corporation.
  • 5. AR Approaches Windows Phone Microsoft Corporation.
  • 6. Vision-Based (Games) “These Magic Cards Are The „Holy Shit‟ Moment of Nintendo‟s New Toy” - Gizmodo (March2011) Windows Phone Microsoft Corporation.
  • 7. Vision-Based (Apps) Kogan concept app Windows Phone Microsoft Corporation.
  • 8. Sensor-Based (Games) Live Butterflies Viewer / Game Windows Phone Microsoft Corporation.
  • 9. Sensor-Based (Apps) PhotoSynth for iPhone (yes, we know) Windows Phone Microsoft Corporation.
  • 10. Vision vs. SensorVision+ No sensors required (only access to camera feed)+ Existing libraries for tracking tags (code is fairly simple)- Tied to specially printed tags (in range, good angle)- CPU intensiveSensor+ Requires no printed tags+ Not nearly as CPU intensive- Code is more complex (due to dealing with sensors)- Most current examples are tied to a fixed location- Requires sensors Windows Phone Microsoft Corporation.
  • 11. Steps for Building Vision-Based AR1. Capture video from the camera2. Using a vision library to locate tag in scene (position) and determine its scale and skew3. Working backward, calculate the real-world camera location and direction4. Transform virtual scene (possibly using a virtual camera)5. Draw virtual scene on top of camera feed6. Repeat Windows Phone Microsoft Corporation.
  • 12. Vision-Based AR on Windows Phone 00:37 00:37 SLAR Toolkit - slartoolkit.codeplex.com Windows Phone Microsoft Corporation.
  • 13. Sensor-Based AR Virtual World + Camera and Sensors = Augmented World Windows Phone Microsoft Corporation.
  • 14. Steps for Building Sensor-Based AR1. Position items in “virtual world” (WP use XNA)2. Determine how device is rotated using Motion APIs3. Rotate virtual world to match device rotation4. Capture video from camera5. Draw virtual world on top of camera feed a) (WP) Use XNA to render in XNA objects – or – b) (WP) Convert XNA space to Silverlight transforms6. Repeat Windows Phone Microsoft Corporation.
  • 15. What Sensors are Required? Accelerometer Camera Compass Gyro (best) Windows Phone Microsoft Corporation.
  • 16. Connecting the camera to UI (WP)// Create PhotoCameracamera = new PhotoCamera(CameraType.Primary);// Create a VideoBrush and connect it to the cameracameraBrush = new VideoBrush();cameraBrush.SetSource(camera);// Fill rectangle with video from the camerasomeRectangle.Fill = cameraBrush; Windows Phone Microsoft Corporation.
  • 17. SensorsAccelerometer Gyro+ Very fast + Fast+ 3 Axis (X, Y, Z)- Not a good reference point + 3 Axis (X, Y, Z) in motion + Always a reference point, even in motionCompass - Drifts+ Simple to use- Jitters - Not always available- 1 Axis only (heading) Windows Phone Microsoft Corporation.
  • 18. Motion “Sensor” (WP) Windows Phone Microsoft Corporation.
  • 19. Using Motion “sensor” (WP)// Create Motion “sensor”if (Motion.IsSupported){ motion = newMotion(); motion.TimeBetweenUpdates = TimeSpan.FromMilliseconds(20); motion.CurrentValueChanged += motion_CurrentValueChanged; motion.Start();}// Handle movementvoid motion_CurrentValueChanged(object sender, SensorReadingEventArgs<MotionReading> e){ …} Windows Phone Microsoft Corporation.
  • 20. Hardware Availability (WP)Accelerometer & Compass are required for motion. Highestquality if Gyro is available. All 7.0 devices have Accelerometer Some 7.0 devices have Compass (Motion enabled) All 7.5 devices will have Compass (Motion enabled) Many 7.5 devices will have Gyro (Best motion) Windows Phone Microsoft Corporation.
  • 21. Debugging and Testing Only Accelerometer supported in the emulator today Test for Motion.IsSupported and fail gracefully! AR should always be tested on a physical device Windows Phone Microsoft Corporation.
  • 22. Sensor-Based AR on Windows Phone Step-by-step video - bit.ly/WPARBasic Windows Phone Microsoft Corporation.
  • 23. Geo Augmented Reality TexMex TexMex Automatically Augmented Real World Object + Your Location and + Generated = World Location Direction (sensors) Virtual World Windows Phone Microsoft Corporation.
  • 24. Steps for Building Geo-Based AR1. Get location of objects in real world (web service, etc.)2. Get location of the user (Location APIs)3. Calculate actual distance between user and objects4. Generate “virtual world” where user is at center5. Capture the direction the user is looking (Motion APIs)6. Rotate virtual world to match device rotation7. Capture video from camera8. Draw virtual items on top of camera feed9. Repeat Windows Phone Microsoft Corporation.
  • 25. Introducing: GART 00:18 Geo AR Toolkit - gart.codeplex.com Windows Phone Microsoft Corporation.
  • 26. Steps for using GART1. New Windows Phone project2. Add a reference to GART.dll3. Add ARDisplay control to page (use Expression Blend)4. Add child views (map, heading, camera, 3D)5. Create an ARItem to represent a real-world thing6. Add ARItem to ARDisplay.ARItems collection Windows Phone Microsoft Corporation.
  • 27. GART Manages SensorsGART automatically manages the camera, motion andlocation APIs for you! Call ARDisplay.StartServices in Page.OnNavigatedTo Call ARDisplay.StopServices in Page.OnNavigatedFromIf there is a problem starting or stopping any sensor it’sbubbled through the ARDisplay.ServiceError event. Windows Phone Microsoft Corporation.
  • 28. GART Demo Windows Phone Microsoft Corporation.
  • 29. Getting Crafty  Windows Phone Microsoft Corporation.
  • 30. Targus TG-42TT $14Cable Ties: 30 cm $3Cable Ties: 10 cm $3Friction Mount $16Spray Grip $3Mainstays Fabric Bin $6 Total: $45 Windows Phone Microsoft Corporation.
  • 31. Please let me help you build Windows Phone apps gart.codeplex.com jbienz@microsoft.com jaredbienz.wordpress.com @jbienz on Twitter Windows Phone Microsoft Corporation.
  • 32. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.© 2011 Microsoft Corporation.All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S.and/or other countries. Windows Phone Microsoft Corporation.