From zero to hero: starting with the
“View based application” template
and ending with a tagged landmark
• Hello (Augmented) World
• Picture Level (rotation about 1 axis)
• Artiﬁcial Horizon (rotation about 2 axes)
• Artiﬁcial Horizon + Compass (add cardinal
direction / translation)
• GPS Locator: tagging a landmark (add
location / translation)
Baby steps to a better reality :
Hello (Augmented) World
• Goal: Overlay some text onto the live
• UIImagePicker + overlay
• Step 1: collect underwear
Picture (er, shopping cart) Level
• Goal: Overlay a level bar on the center of
the live camera view
• Requires: real time info regarding the
rotation of the camera about the Z-axis
• Adds UIAccelerometer
#pragma mark UIAccelerometerDelegate
- (void)accelerometer:(UIAccelerometer *)accelerometer
double angle = -atan2(acceleration.y, acceleration.x) - M_PI/2.0;
double angleInDeg = angle * 180.0f/M_PI;
[label setText: [NSString stringWithFormat:@"Angle: %f", angleInDeg]];
overlayGraphicView.transform = CGAffineTransformMakeRotation(angle);
Video will go here
• Goal: Keep the bar on the horizon (not just
• Compensate for rotation along x-axis
• Need to know ﬁeld of view of camera to
ﬁnd horizon position
• ~53 degrees vertical
• ~37.5 degrees horizontal
iPhone View Angles
53 Degrees Vertical*
37.5 Degrees Horizontal
*Close to this, but not exactly on the iPhone 4
GPS Hotel Locator
• Goal: Speakers must ﬁnd Hotel in any (um)
state, so we’ll tag it for them.
• Add GPS location with
• Compensate for “the hotel is over there
and I’m looking over here”
In other words:
Given the position of the hotel, the position of
the iPhone, and the direction the iPhone is pointing,
we want to know the cardinal direction of the
hotel relative to the iPhone’s pointing direction
and the elevation of the hotel relative the iPhone’s
pointing direction in order to overlay a label
onto the hotel in real time.
• GPS and compass haven’t enough accuracy
to place advertisements or labels on near
• On a “bad day” your iPhone’s version of
“here” might be a 1/2 kilometer circle
around a cell tower... or worse.
• Location updates will drain battery...
• But the dream is alive.
“Best practices... ” or “stuff we noticed
making a demo AR app”
• Don’t try to tag a near object, inaccuracies
compound as you near the target.
• Check altitude / location accuracy: Phone
can’t get altitude w/o good GPS ﬁx. See
documentation for accuracy (check for -1)
• The broad side of the barn is easier to hit.
• Bottom line: “here” isn’t here, yet.
Download the code
• Allows processing of the frame before
• Very complicated to setup
• Check out the WWDC samples and video
• Smoothing / Filtering Sensor Data (wobble
• Full-on 3D AR world -- Dancing hippos in
swimming pools with Dolphins...
• Image tracking to mitigate the swings
How we might transition to
a Full virtual 3D World
• Need useful local coordinate system to
map onto OpenGL; we’re pretty close with
the demo code, actually.
• Match OpenGL transform matrices with
current position relative to some scene we
might want to add to the real world,
camera parameters; GLULookAt...
• Need to not mind that compositing could
be very slow...
Existing AR Toolkit for iPhone
• Marker based
• Simple CoreLocation - based data model
• Hasn’t been updated for a while
What counts as AR?
• Video with... ?
• Must be Interactive?
• Modify the virtual environment?
AR News Sources
• Bruce Sterling’s Beyond the Beyond blog
• Tish Shute:
• Twitter streams:
@anselm, @bruces, @tishshute, @brady
• O’Reilly Radar: