Augmented Reality Barriers & Drivers for Widespread Adoption

Augmented Reality (AR) AIMIA Mobile Industry Group
Industry, Technology & Innovation Snapshot Sydney 13 Jul 2010

AUGMENTED REALITY
Industry, Technology & Innovation Snapshot
Widespread Adoption Barriers & Drivers
for Mobile Industry Group
Australian Interactive Media Industry Association AIMIA.com.au

Gary Hayes 13 Jul 2010

© Gary Hayes CCO MUVEDesign & Personalizemedia.com
gary@muvedesign.com personalizemedia.com muvedesign.com storylabs.com.au


Service provider, content producer, strategist, director, web 2.0 ‘generator’
•80-90s - Music Composer, Producer, Lecturer, Filmmaker then multi media editor BBC
•95- 03 - BBC Senior Dev Producer, New Media & Broadband TV - 8 yrs
•Personal TV / standards - TV-Anytime, MPEG 7/21 (metadata standards)
•2004-05 - Interactive TV Producer, LA USA -
•2005 - 2010 - Lab for Advanced Media Production, Founding Director LAMP & MultiPlatform Head at AFTRS
•2006 - MUVEDesign, Virtual World / Game Creation, Augmented & Alternate Reality Games Design & Production
•2005 - 9th Top Blog Media and Marketing Australia (139th worldwide AdAge power150)
•2010 - Founder of new development & training initiative storylabs.com.au, storylab.us and thePORTALlabs.com



My AR presentation
1.Definition, landscape & relevant areas for ‘developer’ discussion

2.Barriers & drivers for adoption

3.Ubiquitous AR issues



AR Landscapes



The AR ecosystem driven by a descending data ‘cloud’



Augmented
Reality

a lot more than
just technology
- social utility
will be it’s life force


Yes, AR is again in the hype phase



Yes, AR is again in the hype phase http://www.kzero.co.uk/blog/?p=4346



Key Types of Augmented Reality & two most accessible for widespread development

GARY’S FIVE TECHNICAL TYPES OF AUGMENTED REALITY

1.Surface or Haptic – The most understandable form of ‘reality that is augmented’ would
be screens, floors, walls etc that respond to the touch of people in them providing them with virtual
real time information or collaboration

2.Pattern, Recognition or Marker – The AR system performs simple pattern
recognition on a shape, marker (usually on a framed card in the real world scene) or face and
replaces it with a static or moving element e.g: a 3D model, info, audio, video stream or loop etc:
You view the ‘items’ in the scene with you

3.Outline – This is where your hand, eye or body outline is picked up and seamlessly ‘merged’
with the virtual elements. Simple example where you can pick up a 3D object that doesn’t exist
because the system is tracking your hand outline.

4.Location, Wayfinding, Geo-Location – Based on detailed GPS or
triangulation location & position/view of the camera/device the AR system can overlay information
precisely over buildings or people as you move through real space.

5.Hologram – Using ‘smoke & spinning mirrors’ literally in some cases, virtual or real items are
‘projected’ into the physical space you are in and can be interactive with based on cameras
tracking real world impulses e.g: hand gestures or audio signals



2. EXAMPLE Pattern, Recognition or Marker - Junaio natural tracking, using Glue



2. EAMPLE Pattern, Recognition or Marker - Ben & Jerry’s dev’d using Metaio’s Unifeye SDK



3. EXAMPLE Location, Wayfinding, Geo-Location - ‘TONY Summer Drinking Channel’ Junaio

Users can visit the TONY Summer Drinking channel by downloading the junaio 2.0 app for
iPhone and Android devices. After launching the app and selecting the TONY Summer
Drinking channel, users can point their smartphones in all directions to reveal details on
featured bars offering deals and incentives, browse venue photos, contact numbers,
directions, neighborhood maps, as well as a web link to the actual Time Out New York
summer drinking page.

http://www.metaio.com/index.php?id=1044



3. EXAMPLE Location, Wayfinding, Geo-Location - Malaysian Airlines with SITA lab

“In the case of Malaysia Airlines the app displays the cheapest deals on the airline from the user’s nearest airport to
those further afield. What is interesting about the use of augmented reality by an airline is better understood by what
Malaysia has in store further down the line. Officials say the first incarnation of the app (deals) will be developed over
time so passengers can use features related to their booking. For example, a user will be able to switch on the AR
function at an airport and find where the check-in desk is located or the direction of the departure gate, lounge,
baggage claim and other services.”
http://www.tnooz.com/2010/06/23/news/malaysia-airlines-dabbles-in-mobile-augmented-reality-with-deals-and-directions/



3. EXAMPLE Location, Wayfinding, Geo-Location growth curve begins

Layar Platform grows to 1.000 layers, covering
every country in the world
This week Layar reached three major milestones:
1.000 layers published, 3.000 layers in testing and
4.000 active developers. The platform growth has
now accelerated to over 20% per month. These
layers bring augmented content to every country
in the world.

Layar’s usage shows growth and maturity with
700K active users
In the last 6 months 1.6 million people used Layar at
least once. The active user base amounts to 716.000
people who used Layar in the past 30 days. The DONT FORGET FOURSQUARE - “Today we
Layar Reality Browser is moving from a cool closed on a $20m Series B round with Union
application to show friends to an application that is a Square Ventures, O’Reilly AlphaTech Ventures
service of value in daily life. and our newest partner, Andreessen Horowitz.”



AR Adoption



Barriers for widespread adoption of AR

1. REGULATION - particularly people & place
recognition services, not on Gov radar yet?!

2. POOR UI - cluttered screen & hard to navigate

3. PERCEPTION - AR perceived as superficial &
dubious value by consumers

4. EXPERIENCE - Mobile processing power - some
apps run at below 7 frames per sec & battery drain,
slow pings & lock-on using GPS

5. 1st GEN TOOLS - The open tools are a little clunky
& definitely buggy

6. DISCONNECT - Still an issue going from location or
marker browsing to fulfillment



What will drive widespread adoption

1. VALUE - That AR is seen as a useful life & entertainment
addition. Better education
2. BETTER UI - larger screens, wrap glasses and tablets
(see Gary’s video)
3. SOCIAL INTEGRATION - Foursquare/Gowalla meets
Layar/Junaio = TagWhat, Urbanspoon etc:
4. EXPERIENCE - Richer 3D game-like applications, faster
FPS, more natural recognition. Lock-on improved using
combinations of GPS, WiFi & 3/4G
5. UBIQUITOUS - Average prediction of users of mAR (some
form) to be greater than 50% in 2 years (2012). Already
most location browsers are on Android, iPhone & Symbian



Ubiquitous AR Issues



Improving the user experience - suggestive model & AR glasses

http://www.personalizemedia.com/augmented-worlds-video-part-1-recognition/ http://www.youtube.com/watch?v=dkYC1kBPR6o



Real business models will drive development



Regulation and emergent technologies - privacy a barrier?

No faces, baby
We don't do face recognition. Besides the technological challenges there are
also serious privacy issues.
Millions of things
We currently have over 10 million items in our database for recognition. In
comparison, the English version of Wikipedia has about 3 million articles in
total.



Regulation and emergent services, ARvertising - barriers?

If the site allows it,
will virtual AR
advertising take off?



The Google influence - Google Goggles, face recognition and now AR-like ad overlays

The Jan 2010 patent allows overlaying of new ads
over old streetview ad sites. AR implications?
"The link can be associated with a property owner, for example the property owner
which owns the physical property portrayed. The link can alternatively be associated
with an advertiser who placed the highest bid on the image recognized within the
region of interest (e.g., poster, billboard, banner, etc.). Any portion of the geographic
display image in which the region of interest is located can be selectable (e.g., hot-
linked). For example, the image of the coffee shop can be hot-linked to an
advertisement for the coffee shop."

[0038]Alternatively, or additionally, explicit user feedback may be used as a factor to
select one of the links included in a comment. When a comment is presented to a user
in connection with presentation of a particular document, the user may be given the
opportunity to provide explicit feedback on that comment. For example, the user may
indicate whether the comment is meaningful (e.g., a positive vote) or not meaningful
(e.g., a negative vote) to the user (with respect to the particular document) by selecting
an appropriate voting button. This kind of feedback may be used to select one of the
links of the comment. If users indicate, via appropriate voting, that a comment is
meaningful (or not meaningful) with regard to a particular document with which the
comment is presented, this may provide evidence that the comment contains content
relevant (or not relevant) to the content of the particular document. By using explicit
user feedback to select one of the links, server 220 may select the link leading to the
document with content that best matches the content of the comment.

http://appft.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&u=/netahtml/PTO/search-adv.html&r=2&p=1&f=G&l=50&d=PG01&S1=google.AS.&OS=an/google&RS=AN/google



Education - gov & orgs behind industry in providing clarity?



Education - gov & orgs behind industry in providing clarity?

“For those not familiar with AR, it is an environment that includes
both virtual reality and real-world elements. For instance, an AR
user might wear translucent goggles; through these, he could
see the real world, as well as computer-generated images
projected on top of that world. We define an augmented reality
system as one that

1 combines real and virtual,
2 is interactive in real-time,
3 is registered in three dimensions.
Vuzix Educational AR provides leading-edge AR technology
training solutions for the following markets:

1 Aviation & Aerospace
2 Defense & Military
3 Emergency Response
4 Medicine / medical market
5 Academia
http://www.vuzix.com/AR_Site/about/default.asp
6 AR Video Eyewear”



Proprietary standards? - Metaio’s Unifeye SDK
The Unifeye Mobile platform is the most comprehensive solution to create augmented reality applications for iPhone, Android, Symbian and WinMobile
devices. Featuring a high-level API and latest image recognition technologies it allows developers to produce high-quality applications with low efforts.

•Most important platforms supported: currently available on: iPhone, Android, Symbian OS and WinMobile. The code-base is prepared to allow easy
porting of applications.
•Optimized components for mobile hardware: Libraries and algorithms written to account for hardware limitations. As a result, the Unifeye Mobile
SDK is a fast and compact modular system.
•Comes with conﬁgurable standard-app, for simple tasks—no programming is necessary:
•By changing an XML file it is possible to load 2D and 3D-content, configure them to be shown at certain tracking patterns and trigger animations
and sounds. The configuration can be edited and tested on the PC.
•The latest image recognition (tracking) technologies: Includes a marker-based and robust 2D-texture tracking for arbitrary images. New tracking
technologies will be integrated in the future
•Supporting the .md2 model format: the rendering engine features a scene graph-system, animations (linear and splines), billboards, picking, various
texture formats (including alpha channel) based on OpenGL ES 1.x
•Different programming layers: high and low level programming is possible with individual component access (rendering, capturing, tracking)
•Content protection through state-of-the-art encryption


The social component will drive take-up - eg: TagWhat



The social component will drive take-up eg: UrbanSpoon

The big location AR driver will be location based on
the filtering of trusted recommendations

“Urbanspoon (now part of Citysearch/IAC) just
released an app update that includes AR for the
iPhone 3GS. Called “Scope” it shows a tab at the
bottom of the screen that launches the camera-
based experience. Hold the phone up and through
the camera you see the popularity of restaurants
and distance, indicated by the size of the circle
(image below). If you point the phone toward the
ground, you’ll get a traditional map instead. You can
also manually locate yourself for greater precision
(so that you can see the restaurants immediately
near you) if GPS isn’t working well.”

http://searchengineland.com/battle-of-the-augmented-reality-apps-
urbanspoon-layar-wikitude-wheremark-more-27806?
utm_source=feedburner&utm_medium=feed&utm_campaign=Feed
%3A+searchengineland+%28Search+Engine+Land%29



AR company initiatives driving developer take-up?



Locative AR, originally clunky tools but adding key features now



Creating marker & location AR is relatively easy for all - but real quality control?

Layar/Junaio location types
1 Create a list of POIs - long/lat/altitude
2 Create media assets (text, png/tga, .l3d, obj) for each POI
(use 3D model conv tools)
3 Generate a php/XML file in correct format
4 Host this on your own servers, validate
5 Request publication, go live
6 Optional with Junaio - add markers/patterns to each POI for
deeper interaction

FLAR Marker
1 FLASH SETUP
We simply create a Flash ‘container’, which links to a Flash Script file which has that runtime script and links to an attached the “ready to go” code
libraries which are:
•

Flex SDK lib – which embeds AR data inside flash
•

FLARToolkit – handless detection
•

Papervision3D code – deals with the import, position and rendering of 3D models
2 MARKER SETUP
The simple steps once you have the Flash hierarchy ready are:
a) Design a marker, start the MarkerGenerator.air application. Hold up your paper pattern to the camera, when it is ‘red’ highlighted click save
pattern. Save it as FLARPattern.pat in the FLAR folder. Check your new pattern works
3 MODEL SETUP
For the purposes of this demo we are going to only originate a single static 3D model. This ready to go AR bundle is scripted Papervision3D
models in the ‘collada’ ‘digital asset exchange’ format. You have two options to create a replacement model in your project – 1) Use the free tool
SketchUp 7 thttp://sketchup.google.com/ (installer in folder) make or get “ready made” models from http://sketchup.google.com/3dwarehouse/
(within the tool) and export as a collada model (triangulate, only selection and texture mapes)
b) Note generally use a low polygon count (less than 1500) & low rez texture files. Blender, 3dsMax and Maya and others can be used for this
c) Place new 1) 3D model, 2) textures and 3) txt files in the ‘models’ folder - to save time give the model the same name as the one already in
there ‘model.dae’ (perhaps you might like to rename the other one to ‘model0.dae’ for example). Also place associated textures in the
‘images’ folder and the textures.txt file, the level above in the ‘models’ folder.



AR Marker based tools more mature & ubiquitous
e.g: FLAR Toolkit, Flex, Air, Papervision, 3D tools also newer D’Fusion Studio etc:



Other marker focused tools - good Flash focused list on Wikipedia, ARToolKit

•FLARToolKit - an ActionScript 3 port of ARToolKit for Flash 9+.
•ARTag - alternative to ARToolKit that uses more complex image processing and digital symbol processing for reliability
and resistance to light. Only licensed for non-commercial purposes.
•ARToolKitPlus - extended version of ARToolKit, only targeted to handheld users and developers of AR-oriented
software. No longer developed.
•Studierstube Tracker - Successor of ARToolKitPlus, many new features, closed source, not available for download.
•Mixed Reality Toolkit (MRT) - University College London
•OSGART - a combination of ARToolKit and OpenSceneGraph
•SLARToolkit - a Silverlight port of NyARToolkit.
•NyARToolkit - an ARToolkit class library released for virtual machines, particularly those which host Java, C# and
Android.
•ARDesktop - ARToolKit class library that creates a three-dimensional desktop interface with controls and widgets.
•AndAR - A native port of ARToolkit to the Android platform.
•ATOMIC Authoring Tool - a Cross-platform Authoring Tool software, for Augmented Reality Applications, which is a
Front_end for the ARToolkit library. Was developed for non-programmers, to create small and simple, Augmented
Reality applications, released under the GNU GPL License.
•ATOMIC Web Authoring Tool Is a children project from: ATOMIC Authoring Tool that enables the creation of
Augmented Reality applications and export it, to any website. Developed as A front end (Graphic Interface) for the
Flartoolkit library. And it's licensed under the GNU GPL License.



Early alpha tools means opps for developer companies ...

Development
Around the world and in Sydney based
buildAR, Inscribe, The Project Factory and
MUVEDesign amongst others - making it
easier for small business & story-tellers to
enter the world of geo-location AR,
classiﬁeds as well as marker & recognition
based AR

Educational
MUVEDesign & storylabs.com.au
producing a series of short videos looking
at trends in AR helping map out a very
rapid transformation in the next few years



Mass consumer adoption - Augmented Reality Gaming a big perception changer?



Mass consumer adoption - Microsoft’s Kinect
The Dimitri Project was the codename for a video game by Lionhead
Studios that for many years had an unknown status, before surfacing
in 2009 as the technology demo "Milo" for Microsoft's Project Natal.
[1]

Work on The Dimitri Project started in 2001 after Lionhead had
finished work on Black & White, and had been reported as being
cancelled, however a 2006 interview with Peter Molyneux confirmed
that that was not the case.[2] Development on Dimitri was again
reported as stopped with reports that the game never got to a "full-
team" stage. It was reported that Dimitri had been replaced by a
second project, Project X, which may lean on aspects of Dimitri but is
distinct.

In an interview with Eurogamer at E3 in June 2009 Peter Molyneux
confirmed that The Dimitri Project had become the "Milo" technical
demo used to demonstrate Microsoft's Project Natal which was
shown at that years E3. Milo is a living AI that reacts to body
movements and facial expressions. Milo is a program that recognises
people, introduces himself to new people, and allows people to have a
conversation with him. Drawing pictures on paper and holding to
"Natal" allows it to scan the image and create a representative
onscreen that Milo can give his opinion on and interact with as a real
drawing. that for many years had an unknown status, before surfacing
in 2009 as the technology demo "Milo" for Microsoft's Project Natal.

Work on The Dimitri Project started in 2001 after Lionhead had
finished work on Black & White, and had been reported as being
canceled, however a 2006 interview with Peter Molyneux confirmed
that that was not the case. Development on Dimitri was again reported
as stopped with reports that the game never got to a "full-team"
stage. It was reported that Dimitri had been replaced by a second
project, Project X, which may lean on aspects of Dimitri but is distinct.

In an interview with Eurogamer at E3 in June 2009 Peter Molyneux confirmed that The Dimitri Project had become the "Milo" technical demo used to demonstrate Microsoft's Project Natal which was
shown at that years E3.Milo is a living AI that reacts to body movements and facial expressions. Milo is a program that recognises people, introduces himself to new people, and allows people to have a
conversation with him. Drawing pictures on paper and holding to "Natal" allows it to scan the image and create a representative onscreen that Milo can give his opinion on and interact with as a real
drawing.

Project Natal:
(pronounced "nuh-tall" is the code name for a "controller-free gaming and entertainment experience" by Microsoft for the Xbox 360 video game platform. Based on an add-on peripheral for the Xbox 360
console, Project Natal enables users to control and interact with the Xbox 360 without the need to touch a controller, such as through body gestures or spoken commands.
The project is aimed at broadening the Xbox 360's audience beyond its hardcore base of young males. Project Natal was first announced on June 1 at the 2009 Electronic Entertainment Expo. Release
pricing and timing have not been announced, but the device is not expected to be available until at least 2010.
An approximately nine-inch wide horizontal bar connected to a circular base with a ball joint pivot, the Project Natal sensor is designed to be positioned lengthwise above or below the video display. The
sensor features a "RGB camera, depth sensor, multi-array microphone, and custom processor running proprietary software," which provide full-body 3D motion capture, facial recognition, and voice
recognition capabilities.
The "Project Natal" sensor's microphone array enables the Xbox 360 to conduct multi-directional voice location tracking and ambient noise suppression, allowing for things such as headset-free party
chat over Xbox Live. The depth sensor consists of an infrared projector combined with a monochrome CMOS sensor, and allows the Project Natal sensor to see in 3D under any lighting conditions.



AR Adoption Reminder



Thanks, Questions?
These slides on slideshare
http://www.slideshare.net/hayesg31


Augmented Reality Barriers & Drivers for Widespread Adoption

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (15)

Similar to Augmented Reality Barriers & Drivers for Widespread Adoption

Similar to Augmented Reality Barriers & Drivers for Widespread Adoption (20)

More from Gary Hayes

More from Gary Hayes (10)

Recently uploaded

Recently uploaded (20)

Augmented Reality Barriers & Drivers for Widespread Adoption