1. Dr. Lyndon Nixon presented on semantics enhancing augmented reality and making reality smarter through annotation of things of interest using image recognition. 2. Annotations are saved in a data model and directly integrated with content provider databases to retrieve additional metadata and representations. 3. The SmartReality platform uses image recognition services and a JSON data model to identify objects and return relevant annotations and content to display for users in an augmented reality client application.

1. Semantics enhancing Augmented
Reality and making our reality
smarter
Presenter: Dr Lyndon Nixon, head of research, STI Research GmbH
lyndon.nixon@sti2.org
Co-authors: Jens Grubert and Gerhard Reitmeyr, Graz University of
Technology; James Scicluna, Seekda gmbh

2. Augmented Reality in 2012 –
image recognition based AR is on
the rise
•Google Googles
•Layar Vision
•Kooaba Shortcut
…
http://www.youtube.com/watch?v=im

3. SmartReality: what should be the
unique value proposition?
• For organisations with Things of Interest
– Unlock additional content about those things at low cost.
Annotate once, the content updates itself!
• For online content owners
– Provide your content in a new context. Direct access to
new consumers in an appropriate situation.
• For SmartReality reality providers
– Be the gateway to a new AR experience.
• For the user
– Discover relevant content about things around you!

4. SmartReality architecture

5. Things of Interest
• Could be anything, we start with street
posters as partner play.fm has a database of
digitized posters tied to their events & artists

6. TOI Metadata
Things of - Identifier
-
- provider
- creator
- created
Interest (TOI) Metadata
-
etc. (Dublin
Core)
• POIs often mix too strongly Annotatio
n
content and presentation
info Annotation Representati
• Require all data for the app - Identifier on
- Metadata - Dimension
to be pre-available in the - Locating - Identifier
Data - Classification
dataset - - Name
Representatio - Description
• Not easily manipulated as n - Icon
- Represents - Thumbnail
there are no „handles“ to - Segment_
other data, nor typing annotation Represen
Segment_ ts
• TOI data model looks at annotation - Identifier
- Identifier -
minimal given content and - Locating Data Classificatio
- n
most flexible retrieval of Representation - Name
further content on-the-fly - Represents - Description
- Icon
- Thumbnail

7. Annotation tool (play.fm)
• Loads poster images
from play.fm DB
• Easy conceptual
annotation saved in
TOI data model
• Direct integration
with the play.fm TOI
repository
• Supports play.fm,
DBPedia and general
LOD URIs …

8. Image recognition service
• Visual image is registered
in an object recognition service
(now using Kooaba)
• Object look-up call returns
an identifier for visible objects
„e_15149“

9. •

10. Platform – conceptual vision

11. SmartReality client
• A JSON model has been
specified which is returned
by the platform and
processed at the client.
• Content display for a
segment follows a grid
approach which has a
max_content_items
based on display size.
• Version 2 currently runs on
Android 2.3 and 4.03