203 Is It Real or Is It Virtual? Augmented Reality on the iPhone
Upcoming SlideShare
Loading in...5
×
 

203 Is It Real or Is It Virtual? Augmented Reality on the iPhone

on

  • 3,600 views

203 Is It Real or Is It Virtual? Augmented Reality on the iPhone with Jonathan Blocksom

203 Is It Real or Is It Virtual? Augmented Reality on the iPhone with Jonathan Blocksom
Tuesday, Sept. 28, 11:15 am — 12:30 pm
San Diego

Statistics

Views

Total Views
3,600
Views on SlideShare
3,600
Embed Views
0

Actions

Likes
1
Downloads
107
Comments
2

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

203 Is It Real or Is It Virtual? Augmented Reality on the iPhone 203 Is It Real or Is It Virtual? Augmented Reality on the iPhone Presentation Transcript

  • iPhone Augmented Reality Jonathan Blocksom Jonathan Saggau blocksom@gmail.com jonathan@jonathansaggau.com @jblocksom @jonmarimba
  • About Us • Jonathan Blocksom • Jonathan Saggau (No PhD) (Also No PhD)
  • VR / AR History • 1965 Sutherland
  • VR / AR History • 1995 - UNC, Nintendo • who is that? ------>
  • Stella!
  • Existing AR Apps: Theolodite • Sort of like fancy binoculars
  • DishPointer
  • Existing AR Apps: Layar • Overlay markers on the world
  • Augmented Surreality: Numen
  • AR Drone • iPhone controlled remote flyer • Sends video stream to phone over Wifi • http://ardrone.parrot.com/parrot-ar-drone/
  • More Existing AR Apps • http://mashable.com/2009/12/05/ augmented-reality-iphone/ • http://www.youtube.com/watch? v=ps49T0iJwVg (acrossair) • Augmentia • Simple Geo • smaato • Layar • Wikitude
  • From zero to hero: starting with the “View based application” template and ending with a tagged landmark • Hello (Augmented) World • Picture Level (rotation about 1 axis) • Artificial Horizon (rotation about 2 axes) • Artificial Horizon + Compass (add cardinal direction / translation) • GPS Locator: tagging a landmark (add location / translation)
  • Baby steps to a better reality : Hello (Augmented) World • Goal: Overlay some text onto the live camera image • UIImagePicker + overlay • Step 1: collect underwear r3
  • UIImagePicker overlay -(void)viewDidLoad { [super viewDidLoad]; picker = [[UIImagePickerController alloc] init]; picker.sourceType = UIImagePickerControllerSourceTypeCamera; picker.showsCameraControls = NO; picker.navigationBarHidden = YES; picker.wantsFullScreenLayout = YES; //1:1.124.. = aspect ratio of the screen picker.cameraViewTransform = CGAffineTransformScale(picker.cameraViewTransform, 1.0f, 1.12412f); } -(void)viewDidAppear:(BOOL)animated { [super viewDidAppear:animated]; [self presentModalViewController:picker animated:NO]; UILabel *aLabel = [[UILabel alloc] initWithFrame:CGRectMake(10., 50., 300., 30.)]; [aLabel setText:@"Hello augmented world!"]; [aLabel setBackgroundColor:[UIColor clearColor]]; [aLabel setTextColor:[UIColor whiteColor]]; [aLabel setFont:[UIFont boldSystemFontOfSize:12]]; picker.cameraOverlayView = aLabel; }
  • Picture (er, shopping cart) Level • Goal: Overlay a level bar on the center of the live camera view • Requires: real time info regarding the rotation of the camera about the Z-axis • Adds UIAccelerometer r7
  • -(void)viewDidAppear:(BOOL)animated { [super viewDidAppear:animated]; [self presentModalViewController:picker animated:NO]; label = [[UILabel alloc] initWithFrame:CGRectMake(10., 50., 300., 30.)]; [label setText:@"Hello augmented world!"]; [label setBackgroundColor:[UIColor clearColor]]; [label setTextColor:[UIColor whiteColor]]; [label setFont:[UIFont boldSystemFontOfSize:12]]; CGRect viewFrame = self.view.frame; viewFrame.origin.x = viewFrame.origin.y = 0.0; UIView *overlayView = [[UIView alloc] initWithFrame:viewFrame]; [overlayView addSubview:label]; [overlayView bringSubviewToFront:label]; // Load the image to show in the overlay: UIImage *overlayGraphic = [UIImage imageNamed:@"artificialHorizon.png"]; overlayGraphicView = [[UIImageView alloc] initWithImage:overlayGraphic]; overlayGraphicView.frame = CGRectMake(30, 100, 260, 200); [overlayView addSubview:overlayGraphicView]; picker.cameraOverlayView = overlayView; [overlayView release]; [[UIAccelerometer sharedAccelerometer] setDelegate:self]; }
  • #pragma mark UIAccelerometerDelegate - (void)accelerometer:(UIAccelerometer *)accelerometer didAccelerate:(UIAcceleration *)acceleration; { double angle = -atan2(acceleration.y, acceleration.x) - M_PI/2.0; double angleInDeg = angle * 180.0f/M_PI; [label setText: [NSString stringWithFormat:@"Angle: %f", angleInDeg]]; overlayGraphicView.transform = CGAffineTransformMakeRotation(angle); } Video will go here
  • Artificial Horizon • Goal: Keep the bar on the horizon (not just level) • Compensate for rotation along x-axis • Need to know field of view of camera to find horizon position • ~53 degrees vertical • ~37.5 degrees horizontal r8
  • iPhone View Angles 53 Degrees Vertical* 37.5 Degrees Horizontal *Close to this, but not exactly on the iPhone 4
  • #pragma mark UIAccelerometerDelegate - (void)accelerometer:(UIAccelerometer *)accelerometer didAccelerate:(UIAcceleration *)acceleration; { overlayGraphicView.transform = CGAffineTransformIdentity; double vertAngle = -atan2(acceleration.y, acceleration.z) - M_PI/2.0; double vertAngleInDeg = vertAngle * 180.0f/M_PI; [label setText: [NSString stringWithFormat:@"V-Angle: %f", vertAngleInDeg]]; CGRect overlayFrame = [overlayGraphicView frame]; overlayFrame.origin.y = 240.0 - 537.8 * sin(vertAngle); [overlayGraphicView setFrame:overlayFrame]; double angle = -atan2(acceleration.y, acceleration.x) - M_PI/2.0; //double angleInDeg = angle * 180.0f/M_PI; overlayGraphicView.transform = CGAffineTransformMakeRotation(angle); }
  • Directional Horizon Marker • Goal: Artificial horizon pegged to a specific cardinal direction (North) • Adds compass direction using CLLocationManager locationManager = [[CLLocationManager alloc] init]; [locationManager setDelegate:self]; [locationManager startUpdatingHeading]; r10
  • - (void)updateUI { [label setText: [NSString stringWithFormat:@"H-Angle:%5.1f V-Angle:%5.1f Heading:%5.1f", angleInDeg, vertAngleInDeg, magCompassHeadingInDeg]]; CGPoint overlayCenter = [overlayGraphicView center]; overlayCenter.y = 240.0 - 537.8 * sin(vertAngle); overlayCenter.x = 160.0 - 497.8 * sin((magCompassHeadingInDeg) * (M_PI / 180.0)); [overlayGraphicView setCenter:overlayCenter]; overlayGraphicView.transform = CGAffineTransformMakeRotation(angle); } #pragma mark UIAccelerometerDelegate - (void)accelerometer:(UIAccelerometer *)accelerometer didAccelerate:(UIAcceleration *)acceleration; { vertAngle = -atan2(acceleration.y, acceleration.z) - M_PI/2.0; vertAngleInDeg = vertAngle * 180.0f/M_PI; angle = -atan2(acceleration.y, acceleration.x) - M_PI/2.0; angleInDeg = angle * 180.0f/M_PI; [self updateUI]; } #pragma mark CLLocationManagerDelegate - (void)locationManager:(CLLocationManager *)manager didUpdateHeading:(CLHeading *) newHeading; { magCompassHeadingInDeg = [newHeading magneticHeading]; [self updateUI]; }
  • GPS Hotel Locator • Goal: Speakers must find Hotel in any (um) state, so we’ll tag it for them. • Add GPS location with CLLocationManager • Compensate for “the hotel is over there and I’m looking over here”
  • In other words: Given the position of the hotel, the position of the iPhone, and the direction the iPhone is pointing, we want to know the cardinal direction of the hotel relative to the iPhone’s pointing direction and the elevation of the hotel relative the iPhone’s pointing direction in order to overlay a label onto the hotel in real time.
  • - (void)updateUI { CLLocation *marriot = [[LocationTools class] locationForMarriott]; float directionToMarriott = [[LocationTools class] angleFromCoordinate:[self.currentLocation coordinate] toCoordinate:[marriot coordinate]]; CLLocationDistance dist = [self.currentLocation getDistanceFrom:marriot]; double deltaAlt = [marriot altitude] - [self.currentLocation altitude]; //verticalAccuracy of -1 == invalid if ([self.currentLocation verticalAccuracy] < 0) deltaAlt = 0; double vertAngleToMarriott = atan2(deltaAlt, dist); [marriotLabel setText:[NSString stringWithFormat:@"DirHotel:%5.1f Dist: %5f VertAngle: %f", directionToMarriott * (180.0 / M_PI), dist, vertAngleToMarriott * (180.0 / M_PI)]]; [anglesLabel setText: [NSString stringWithFormat:@"Horizon H-Angle:%5.1f V-Angle:%5.1f Heading:%5.1f", angleInDeg, vertAngleInDeg, magCompassHeadingInDeg]]; double relativeDirectionToMarriott = magCompassHeadingInDeg * (M_PI / 180.) - directionToMarriott; if (relativeDirectionToMarriott < (-M_PI / 2.0)) relativeDirectionToMarriott = (-M_PI / 2.0); if (relativeDirectionToMarriott > (M_PI / 2.0)) relativeDirectionToMarriott = (M_PI / 2.0); double relativeVertAngleToMarriott = vertAngleToMarriott - vertAngle; CGPoint overlayCenter = [overlayGraphicView center]; overlayCenter.y = 240.0 - 537.8 * sin(relativeVertAngleToMarriott); overlayCenter.x = 160.0 - 497.8 * sin(relativeDirectionToMarriott); [overlayGraphicView setCenter:overlayCenter]; overlayGraphicView.transform = CGAffineTransformMakeRotation(angle); }
  • Gotchas • GPS and compass haven’t enough accuracy to place advertisements or labels on near objects. • On a “bad day” your iPhone’s version of “here” might be a 1/2 kilometer circle around a cell tower... or worse. • Location updates will drain battery... Quickly. • But the dream is alive.
  • “Best practices... ” or “stuff we noticed making a demo AR app” • Don’t try to tag a near object, inaccuracies compound as you near the target. • Check altitude / location accuracy: Phone can’t get altitude w/o good GPS fix. See documentation for accuracy (check for -1) • The broad side of the barn is easier to hit. • Bottom line: “here” isn’t here, yet.
  • Download the code • http://code.google.com/p/ar360/
  • iPhone OS 3 Stack :;;261*'&<=*$>")3$"77*$% ?@&4*"%&)9#3A% :;0<<*7*$"2*3*$%4*7*163*% !"#$%&'(")*%+,%-./% ?0<<*7*$"2*3*$A% 0#12*)3*4%5*67&38%099% B"<6C")D6)61*$% ?E',F%>"296GGA%
  • iOS 4 Ideal 0:;"#)46<")% =>&4*"%&)9#3?% E"$*%B"<")% !"#$%&'(")*%+,%-./% =0AA*7*$"2*3*$D%C8$"F?% 0#12*)3*4%5*67&38%099% @"A6<")B6)61*$% =C',D%E"296FF?%
  • AV Foundation • Allows processing of the frame before display • Very complicated to setup • Check out the WWDC samples and video
  • What’s next? • Smoothing / Filtering Sensor Data (wobble wobble) • Full-on 3D AR world -- Dancing hippos in swimming pools with Dolphins... • Image tracking to mitigate the swings •? • Profit!
  • How we might transition to a Full virtual 3D World • Need useful local coordinate system to map onto OpenGL; we’re pretty close with the demo code, actually. • Match OpenGL transform matrices with current position relative to some scene we might want to add to the real world, camera parameters; GLULookAt... • Need to not mind that compositing could be very slow...
  • ARKit : Existing AR Toolkit for iPhone • http://github.com/zac/iphonearkit/ • Marker based • Simple CoreLocation - based data model • Hasn’t been updated for a while
  • What counts as AR? • Video with... ? • Overlay? • Must be Interactive? • GPS? • Modify the virtual environment?
  • AR News Sources • Bruce Sterling’s Beyond the Beyond blog http://www.wired.com/ beyond_the_beyond/ • Tish Shute: http://www.ugotrade.com/ • Twitter streams: @anselm, @bruces, @tishshute, @brady • O’Reilly Radar: http://radar.oreilly.com