Augmented reality combines real and virtual elements. While mobile phones provide an accessible platform, developing augmented reality applications for phones poses challenges due to limitations of phone hardware and cameras. Effective augmented reality relies on sensors or computer vision techniques to accurately track the position and orientation of the device or camera in real time. Marker-based tracking provides precise tracking but requires physical markers, while markerless tracking using natural features is more difficult but more versatile. Overall, augmented reality remains limited by the accuracy and processing capabilities of mobile phones but continues to advance.

1. AUGMENTED REALITY
BEYOND THE HYPE
Dr Paul Coulton and Klen Čopič Pucihar
Banksy

2. Mixed Reality
Real Augmented Augmented Virtual
Environment Reality Virtuality Reality
MIXED REALITY CONTINUUM
Paul Milgram

3. WHAT IS
AUGMENTED
REALITY?
The term AR is being used in
all sorts of ways but the
generally accepted definition
is that it:
Combines the real and virtual
Is interactive in real time
Is registered in 3D

4. WHAT ISN'T
AUGMENTED
REALITY?
Location‐based services
Barcode detection
(QR codes)
Augmenting still images
Special effects in movies
Photo‐based object
recognition
Alessandro
Mulloni ... Yu-Gi-Oh! Zexal

5. GENERAL
CHALLENGES
OF AR
Strict real time operation
(30Hz)
High spatial precision
(1cm, 1 degree)
Robustness for operation by
human user
Alessandro
Mulloni

6. CHALLENGES
OF MOBILE AR
Same level of performance
as desktop AR
No unrealistic assumptions
about hardware
Variable operating contexts
Layar

7. Image
Denno Coil
FLAVOURS OF
MOBILE AR
WEARABLE AR
Wearable system
Head‐mounted display
AR always in view
(immersive)
Demo
Sixth Sense

8. FLAVOURS OF
MOBILE AR
HANDHELD AR
Mobile phone as platform
Phone acts as a “magic lens”
Non‐immersive view
Tom’s Hardware

9. WHY USE PHONES?
Low cost, Ubiquity, Robust, Self Contained

10. WHY NOT USE PHONES?
Low memory, Limited power, Small Screen, Limited Inputs
available, Fragmentation

11. PERCEPTUAL PROBLEMS
You see through the camera not the phone!

12. Get video frame from camera
DEVELOPING
AR
Estimate position and
orientation of the camera
APPLICATIONS Render the augmented scene
(video and virtual)
What is involved in the
process?
Render GUI
Process the user input
Update application status

13. Phones with
Handheld
Phones acceleromet
AR
with GPS ers Phones with
Displays
two
Camera cameras
Phones PDA Phone Phones
On device with 3D with
Wearable HW compass
AR AR
Phone
Thin Phones with
PDA client AR gyroscopes
Thin
client AR
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
EVOLUTION OF MOBILE AR

14. ESTIMATING
THE DEVICE
POSE
Sensor tracking
Vision‐based tracking

15. SENSOR
BASED
Used by many commercial
“AR browsers”
GPS, Compass,
Accelerometer, (Gyroscope)
Demo
Wikitude

16. GPS
Originally 24 satellites but 31
as of 2008
Bill Clinton cleared GPS for
commercial use in 1996

17. GPS
Spatial Scattering
System and Environmental
Effects

18. ACCELEROMETERS

19. ACCELEROMETERS

20. MAGNETOMETERS

21. David McCandless
Information is Beautiful DATA SOURCES
Quality, Availability, Crowd Sourcing

22. VISION BASED
MARKER
TRACKING
Standard Vision techniques
Marker provides 4 corners
from this we can get the pose

23. VISION BASED
MARKER
TRACKING
1.Convert image to
black
and white
2. Search
for
edges
3. Follow edges
4. Find rectangle corners
!*
K*
C*
:*
Daniel Wagner

24. VISION BASED
MARKER
TRACKING
5.Estimate homography using 4
corners
6. Extract pattern by sampling
7. Check Pattern
Daniel Wagner

25. MARKER TRACKING PIPELINE
Daniel Wagner

26. VISION BASED
MARKER
TRACKING
Nintendo 3DS AR

27. VISION BASED
NATURAL
FEATURE
TRACKING
Tracking features from natural
environment
More difficult than marker
Less established techniques
Slower than marker based
PTAM

28. VISION BASED
NATURAL
FEATURE
TRACKING
Edges - boundaries
Corners - local 2D structure
Blobs - regions rather than
points
Ridges - elongated structures

29. VISION BASED
NATURAL
FEATURE
TRACKING
Offline
Use still images to build data
base of features

30. Camera
Image
VISION BASED Keypoint
Detection
NATURAL
FEATURE Outlier
Removal
TRACKING Pose
Estimation
Online
Creating maps are markers RECOGNITION
online
Real-Time Pipeline
Pose

31. VISION BASED
NATURAL
FEATURE
TRACKING
Online

32. • SENSORS • VISION
• Noise Output • High Accuracy
• Low Accuracy • Local Pose
• Global Pose • Memory Intensive
• Works when • Works only when
nothing to track Objects to Track
HYBRID TECHNIQUES

33. DESIGN
MOCKUPS
AR is easy to understand, but
hard to explain in words.
Ease of creation.
Holding strong memorable
message.

34. CONCLUSIONS
• Augmented Reality is currently the hot topic from a continuum of
possible systems.
• Mobile phones provide the most obvious platform for the
widespread adoption of AR.
• Majority of current commercial offering are sensor based which
gives crude contextual sensitivity but this may be fine for your
application. These solutions generally are dependant on quality of
the data source.
• Marker based solutions give greater accuracy although we aren't
likely to cover the world with markers work well for advertising.
• Marker-less systems offer the ‘dream’ of AR but currently present
considerable technical challenges.
• If your considering AR ask what benefits it gives the user apart
from being ‘cool’.

35. OTHER INTERESTING AR
STUFF