This document proposes an authoring solution for augmented reality applications based on extracting facades from images. It involves: 1. Extracting facades from images to create a frame of reference for authoring augmented content. 2. Developing a facade database with 3D annotation capabilities to allow non-experts to intuitively add augmented reality content. 3. A visualization system that matches new images to the facade database to retrieve and display associated augmented reality content.

1. An Authoring Solution for
a Façade-Based AR Platform:
Infrastructure, Annotation
and Visualization
Guillaume GALES
Eric MCCLEAN
John MCDONALD
DEPARTMENT OF COMPUTER SCIENCE
NATIONAL UNIVERSITY OF IRELAND MAYNOOTH

2. Motivation
Input Image Augmented Image
2

3. Overall System
Mobile client side
Rendering
Server side
Façade Widget
extraction Retrieval
Database Authoring client
Façade
Matching
3

4. Rendering
4

5. Outline
Façade Extraction
Authoring solution
• Façade database
• 3D annotation
Visualization
Conclusion, perspectives and discussion
5

6. Façade Extraction

7. Façade Extraction
Key of the authoring solution and visualization
• Façades are the frame of references of 3D content
• Augment a view with 3D content in urban environment from a single image
Input: image of a façade
Output: homography between the façade and its image
Advantages:
• Robust matching (invariant to rotation and perspective changes)
• Façade normalization (used to build a representation of the environment)
• Camera pose estimation (used by the visualization system)
7

8. Façade Extraction
8

9. Authoring Solution

10. Façade Database
For each street, take pictures of façades
Automatic façade extraction and matching
• Geometrical constraint makes matching robust
• Invariance to rotation and perspective changes
10

11. Façade Database
11

12. Façade Database
Candidate selection
• GPS coordinates
• Bag-of-word description for selecting
candidates
Similarity constraint
Figure 2: Database infrastructure for computing planar facade mo-
¸
saics. The individual facades are stitched together into planes to
¸
12 build a frame of reference for authoring.

13. 3D Annotation
Extension of façade to 3D: frame of reference
13

14. 3D Annotation
Desktop application
• Easy and intuitive interface to use for non expert users
• Predefined list of 3D models
14

15. 3D Annotation
Mobile Application (ongoing work)
Long Tap to add widgets Manual Accurate Geolocalisation
15

16. Visualization

17. National University of National University of National University of
Ireland Maynooth
Ireland Data Retrieval
Ireland Maynooth
Ireland
Ireland Maynooth
Ireland
T
horing is an important stage in the workflow of creat-
gmented reality applications. In this paper we describe
sed database infrastructure for authoring and storing 3D
use in urban environments. It provides frames of ref-
he environment as well as a mechanism to match new
h the facades and thus retrieving associated 3D content.
¸
ucture is flexible in that we can add different 3D “lay-
tent on top of the facades and hence opens many pos-
¸
augmented reality applications in urban environments.
Façade Extraction
e the system provides a representation suitable for both
automatic content authoring.
Façade Matching
ds: Augmented Reality, Infrastructure, Authoring,
ed Database, Content Storing and Retrieving.
DUCTION Widget Retrieval
mented reality applications provide rich and useful in-
o their users about their surrounding environment. To
e augmented reality applications, an efficient infrastruc-
red. Such infrastructures involve:
ng a map of the environment ;
g content ;
ving content.
aper we propose an infrastructure that makes authoring
ve and flexible. Our goal is to create a platform for mo- 17

18. Camera Pose Estimation
intrinsics are known
extrinsics are given by the façade extraction algorithm
(homography between plane and its image decomposed
into rotation and translation)
1
⇥ ⇤
H=K R t
1
H Façade
Camera
18

19. Results
19

20. Conclusion,
Perspectives and
Discussion

21. Conclusion, Perspectives & Discussion
Easy and intuitive authoring solution for AR applications
based on façade extraction
Extension to real time computation on mobile
• Use of GPU and accelerometer
Extension to automatic dynamic authoring
• Link with OpenStreetMap through GPS coordinates
Collaborative authoring client (and database updates)
• HTML5
• Mobile version
21

22. Acknowledgment
Research presented in this paper was funded by a Strategic
Research Cluster grant (07/SRC/I1169) by Science
Foundation Ireland under the National Development Plan.
The authors gratefully acknowledge this support.
Thank you for your attention
22