2013 Lecture4: Designing AR Interfaces

COSC 426: Augmented Reality
Mark Billinghurst
mark.billinghurst@hitlabnz.org
August 2nd 2013
Lecture 4: Designing AR Interfaces

Tracking Requirements
  Augmented Reality Information Display
  World Stabilized
  Body Stabilized
  Head Stabilized
Increasing Tracking
Requirements
Head Stabilized Body Stabilized World Stabilized

Tracking Types
Magnetic
Tracker
Inertial
Tracker
Ultrasonic
Tracker
Optical
Tracker
Marker-Based
Tracking
Markerless
Tracking
Specialized
Tracking
Edge-Based
Tracking
Template-Based
Tracking
Interest Point
Tracking
Mechanical
Tracker

Marker Based Tracking: ARToolKit
http://artoolkit.sourceforge.net/

Markerless Tracking
Magnetic Tracker Inertial
Tracker
Ultrasonic
Tracker
Optical
Tracker
Marker-Based
Tracking
Markerless
Tracking
Specialized
Tracking
Edge-Based
Tracking
Template-Based
Tracking
Interest Point
Tracking
  No more Markers! Markerless Tracking

Natural Feature Tracking
  Use Natural Cues of Real Elements
  Edges
  Surface Texture
  Interest Points
  Model or Model-Free
  ++: no visual pollution
Contours
Features Points
Surfaces

Combining Sensors and Vision
  Sensors
-  Produce noisy output (= jittering augmentations)
-  Are not sufficiently accurate (= wrongly placed augmentations)
-  Gives us first information on where we are in the world,
and what we are looking at
  Vision
-  Is more accurate (= stable and correct augmentations)
-  Requires choosing the correct keypoint database to track from
-  Requires registering our local coordinate frame (online-
generated model) to the global one (world)

Outdoor AR Tracking System
You, Neumann, Azuma outdoor AR system (1999)

The Registration Problem
  Virtual and Real must stay properly aligned
  If not:
  Breaks the illusion that the two coexist
  Prevents acceptance of many serious applications

Sources of registration errors
  Static errors
  Optical distortions
  Mechanical misalignments
  Tracker errors
  Incorrect viewing parameters
  Dynamic errors
  System delays (largest source of error)
-  1 ms delay = 1/3 mm registration error

Reducing static errors
  Distortion compensation
  Manual adjustments
  View-based or direct measurements
  Camera calibration (video)

Reducing dynamic errors (1)
  Reduce system lag
  Faster components/system modules
  Reduce apparent lag
  Image deflection
  Image warping

Reducing System Lag
Tracking Calculate
Viewpoint
Simulation
Render
Scene
Draw to
Display
x,y,z
r,p,y
Application Loop
Faster Tracker Faster CPU Faster GPU Faster Display

Predictive Tracking
Time
Position
Past Future
Can predict up to 80 ms in future (Holloway)
Now

experiences
applications
tools
components
Building Compelling AR Experiences
Tracking, Display
Authoring
Interaction
Usability

3. Discovering User Needs
Mark Billinghurst

Interaction Design is User Centered

A methodology for ensuring good user
experiences with products by getting feedback
from users to inform the design.

50
Discovering Needs
• Why:
Requirements definition: the stage where
failure occurs most commonly
Getting requirements right is crucial

What, How and Why?
• What
1. Understand users, task, context
2. Produce a stable set of requirements
• How:
Data gathering activities
Data analysis activities
Expression as ‘requirements’
• Why:
Requirements definition: failure occurs most commonly
Getting requirements right is crucial
Iterate

What Are the Requirements?
  Functional
  What should the system do? What has it done?
  Data
  What type of data?
  Environment or Context of use
  Physical: dusty? noisy? light? heat? humidity? ….
  Social: sharing of files, of displays, work alone ..
  Organizational: IT support, hierarchy, training..
  Users: Who are they?
  Usability: learnability, throughput, flexibility..

Who REALLY are your Users/Stakeholders?
• Not as obvious as you think:
— those who interact directly with the product
— those who manage direct users
— those who receive output from the product
— those who make the purchasing decision
— those who use competitor’s products
• Three categories of user (Eason, 1987):
— primary: frequent hands-on
— secondary: occasional or via someone else
— tertiary: affected by its introduction, or will influence its purchase

Who are the Stakeholders?
Check-out operators
CustomersManagers and owners
• Suppliers
• Local shop  
owners

Data Gathering
Mark Billinghurst

Data Gathering Techniques (1)
  Questionnaires
  Looking for specific information
  Qualitative and quantitative results
  Good for getting data from a large, dispersed group
  Interviews
  Good for exploring issues, using props
  Structured, unstructured or semi-structured
  But are time consuming and difficult to visit everyone

Data Gathering Techniques (2)
  Workshops or focus groups
  Group interviews/activities
  Good at gaining a consensus view and/or highlighting areas
of conflict
  Observations
  Spending time with users in day to day tasks
  Good for understanding task context
  requires time and can result in a huge amount of data
  Documentation
  Procedures and rules written down in manuals

A day in the Life of.. Cultural Probes.. Role Playing..

Case Study: Equator Domestic Probes

4. Design and Prototyping
Mark Billinghurst

Elaboration and Reduction
  Elaborate - generate solutions. These are the opportunities
  Reduce - decide on the ones worth pursuing
  Repeat - elaborate and reduce again on those solutions
Source: Laseau,P. (1980) Graphic Thinking for Architects & Designers. John Wiley and Sons

The Design Funnel
  Alternate generation of ideas and convergence until resolution
Modified from Pugh, S. (1990) Total design: Integrated methods for successful products engineering.
Addison-Wesley. P. 75

Unmet Needs
  Digital Cameras for Apple
Late 80s – What would happen if we married
a computer with a digital camera?
Many studies of current photography use.
Uncovering of unmet needs in traditional experience
- not enough photos
- not knowing whether photo good or not
- ability to add sound
- ability to organize
Features determined through research and designed
Iteratively.

Brainstorming
  Best with interdisciplinary team

Other Products
  Notice all the iPod look-alikes?

Tools for Effective Design
 Personas
 Scenarios
 Storyboards
 Wireframes and Mock-ups
 Prototypes

Persona Technique
•  Personas are a design tool to help visualize who you are
designing for and imagine how person will use the product
•  A persona is an archetype that represents the behavior and
goals of a group of users
•  Based on insights and observations from customer research
•  Not real people, but synthesised from real user characteristics
•  Bring them to life with a name, characteristics, goals, background
•  Develop multiple personas

Gunther the Ad Guy
Gunther is from Germany. He
Travels extensively for work and
As he is an advertising executive
he needs to present concepts to
clients quickly and easily. He is
a person very well-versed in new
technologies and wishes he had
easier portable solutions for his
presentations…..

Scenarios
Usage Scenarios are narrative descriptions of how the product
meets the needs of a persona
Short (2 pages max)
Focus on unmet needs of persona
Concrete story
Set of stories around essential tasks, problems...
Use to test ideas

A business woman travels from SF to Paris on a business trip. On
her way to the airport she narrowly misses a traffic delay. She
avoids the jam because her Smartphone beeps and send her a
warning text message on her route from the office to the airport.
Upon arrival the location-sensitive Smartphone notifies the airline
that she‘ll check in shortly and an airline employee finds her
immediately and takes her baggage. Her display shows that her
flight is on time and provides a map to her gate. On the way she
downloads tourist information (maps, events) for Paris.

Once found her seat, she begins to review the downloaded
information. She books a ticket for an opera she wants to see.
Her Smartphone makes the booking using her credit card number
stored in memory.The security software of the Smartphone
protects her against fraud.
The Smartphone stores the opera booking along with emails
written on the plane.As soon as she steps off the plane, it makes
the calls and sends the emails.As she leaves the airport, a map
appears on the display and guides her to her hotel

Storyboarding
Sequence of sketches showing use of system in
everyday use context
Concrete example
Easier (faster) to grasp than text based stories
Means of communication with users and system
developers
Sketches, not drawings...
Use to test interaction and make sure design works

Sketching is about design
Sketching is not about drawing
It is about design.
Sketching is a tool to help you:
-  express
-  develop, and
-  communicate design ideas
Sketching is part of a process:
-  idea generation,
-  design elaboration
-  design choices,
-  engineering

Why is sketching useful?
  Early ideation
  Think through ideas
  Force you to visualize how things come together
  Communicate ideas to others to inspire new
designs
  Active brainstorming

Figure 41: A Sketch of a Dialogue with a Sketch
The “conversation” between the sketch (right bubble) and the mind (left
bubble). A sketch is created from current knowledge (top arrow). Reading,
or interpreting the resulting representation (bottom arrow), creates new
knowledge. The creation results from what Goldschmidt calls “seeing
that” reasoning, and the extraction of new knowledge results from what
she calls “seeing as.”
sketch
representation
mind
(new) knowledge
Create
(seeing that)
Read
(seeing as)
54

Sketchbook examples
Industrial designer
  ideas
  variations
  annotations
Source: sketch page from student Industrial designer Samnang Eav
Form factors for wearable computer watch

Sketchbook examples
Idea variations
Source: Nicolai Marquardt sketchbook, with permission.
Four different versions of an idea

Sketch examples – design variations
From Carloyn Snyder’s Book: Paper Prototyping (2003) Morgan Kaufmann, p350

Sketch examples – storyboard transitions
From Carloyn Snyder’s Book Paper Prototyping (2003) Morgan Kaufmann, p11

Techniques: Annotations
  Augment a sketch
  Directly on sketch
  As layer
-  Tracing
-  Photoshop layer
  Over dynamic video

Technique: Annotations
  Textual notes
  Name and/or explain thinks
  Add details
  List of items
  Questions/issues about design
  Graphical marks
  Connect notes to sketch elements
  Relates sketch elements
  Show dynamics of elements or interaction over time

Technique: Tracing
  Basic Idea
-  Copy/trace the fixed elements
of an existing interface/screen
  Technology
-  Camera, photograph, tracing
paper or..
-  Photoshop or equivalent (trace
over image on a separate layer)
  Drawing skill required
-  Almost zero

Technique Hybrid Sketches
  Mixes full and low fidelity elements
  High fidelity parts:
  Fixed design elements
  Low fidelity parts:
  Speculative components
  Contrast
  Makes certain parts of the sketch stand out

Your Turn
  Design a mobile AR user interface
  Viewing virtual buildings in Christchurch
  Views (Should be able to swap between)
  Map View
  AR View
  List View

68www.id-book.com
What is a prototype?
In interaction design it can be (among other things):
•  a series of screen sketches
•  a storyboard, i.e. a cartoon-like series of scenes
•  a Powerpoint slide show
•  a video simulating the use of a system
•  a lump of wood (e.g. PalmPilot)
•  a cardboard mock-up
•  a piece of software with limited functionality

69
www.id-book.com
Why Prototype?
•  Evaluation and feedback are central to interaction design
•  Stakeholders can see, hold, interact with a prototype
more easily than a document or a drawing
•  Team members can communicate effectively
•  You can test out ideas for yourself
•  It encourages reflection: very important aspect of design
•  Prototypes answer questions, and support designers in
choosing between alternatives

Sketch vs. Prototype
Sketch
Prototype

Invite
A)end

Suggest
Describe

Explore
Reﬁne

Ques;on
Answer

Propose
Test

Provoke
Resolve

Tenta;ve,
non
commi)al
Speciﬁc
Depic;on

The primary differences are in the intent

Types of Prototypes
Low Fidelity – quick and dirty, easy access
materials like cardboard and paper.
High Fidelity – more involved electronic
versions similar in materials to final product.

RAPID Prototyping
  Fast and inexpensive
  Identifies problems before they’re coded
  Elicits more and better feedback from users
  Helps developers think creatively
  Gets users and other stakeholders involved early
  Fosters teamwork and communication
  Avoids opinion wars
  Helps decide design directions

Paper Prototyping (Low Fidelity)
Quick and simple means of sketching interfaces
Use office materials
Easier to criticize, quick to change
Creative process (develop in team)
Can also use for usability test (focus on flow of
interaction rather than visuals)
Used a lot to test out concepts before real design begins.

Paper Prototyping with Hardware

Technique: State Transition Diagrams
  Create transition diagram
  key interaction steps
  branch points
  For each transition
  Sketch the screen
  Include the transitional diagram
  Label the transition with what triggered the
transition
(typically user input of set of system responses)

Technique: State Transition Diagrams

Half-Functional Prototypes
More hi-fi
HTML (or
Powerpoint) ...
Mostly precoded
flow, but allows
clicking and
experiencing flow

Wireframing Tools
  Pidoco (websites, mobile, desktop)
  No programming
  http://www.pidoco.com
  WireframeSketcher
  Plug-in for Eclipse
  http://wireframesketcher.com/
  Omnigraffle (Mac)
  Digramming tool
  http://www.omnigroup.com/products/omnigraffle/

Technique: Make storyboards come alive
  An interactive foam
core and paper
sketch/storyboard

Mobile Rapid Prototype Tools
  Android: App Inventor
  http://appinventor.mit.edu/

High-fidelity prototyping
•  Uses materials that you would expect to be in the
final product.
•  Prototype looks more like the final system than a
low-fidelity version.
•  For a high-fidelity software prototype common
environments include Macromedia Director,Visual
Basic, and Smalltalk.
•  Danger that users think they have a full
system…….see compromises

Interactive Role Play in Environment

Tethered Prototypes (High Fidelity)
Goal:
CommunicateVision
HIGH FIDELITY
PROTOTYE

Rapid Prototyping
  Speed development time with quick hardware mockups
  handheld device connected to PC
  LCD screen, USB phone keypad, Camera
  Can use PC development tools for rapid development
  Flash, Visual Basic, etc

‘Wizard-of-Oz’ Prototyping
• The user thinks they are interacting with a
computer, but a developer is responding to output
rather than the system.
• Usually done early in design to understand users’
expectations
>Blurb blurb
>Do this
>Why?
User

Typical Iterations for HW/SW Product:
1.  Paper prototype to evaluate conceptual model
2.  Interactive computer-based prototype with rough
screens to evaluate feature placement
3.  Tethered prototype to evaluate button + screen
interactions
4.  Real device prototyped with major features
working
5.  Real device prototyped with all features working
and graphic design implemented

ITERATIVE DESIGN PROCESS
Design

(redesign)

Prototype

Test
Design
with
Users
(Note

problems)

Evaluate

(Fix
Issues)

BUNRATTY FOLK PARK
 Irish visitor attraction run by Shannon Heritage
 19th century life is recreated
 Buildings from the mid-west have been relocated
to the 26-land surrounding Bunratty Castle
 30 buildings are set in a rural or village setting
there.

AUGMENTED REALITY
In Bunratty Folk Park:
 Allows the visitor to point a camera at an
exhibit, the device recognises it by it’s location
and layers digital information on to the display
 3-dimensional virtual objects can be positioned
with real ones on display
 Leads to dynamic combination of a live camera
view and information

NAVIGATIONAL AID
Smartphone Platform
Most people carry mobile phones and are
comfortable with them
Ideal Augmented Reality Technology
 Global tracking tools
 Wireless communication capabilities
 Location based computing
 Large display for interaction

HUMAN CENTRED DESIGN
Goal of the Navigational Aid
 Easy to use, clear and understandable
 Useful to visitors
 Creating interaction between the visitor and the aid
through the user interface
Engage the visitor
 It is necessary to understand the visitor to Bunratty
 Identify visitor motives and goals while going through
the Folk Park.

Understanding the technology and Related Works
 Literature
 Similar Technologies
 Electronic Tours in Museum Settings
 Interactions design in Outdoor Museums
Understanding the User over time
 Observations
 Interviews

Findings
 Most visitors do not use the map
 Most visitors have mobile phones
 Visitors want more information
 View the Folk Park at their own pace
 Information should be straight to the point
 Large social interaction within groups

NEXT STEPS FROM RESEARCH
 Define Visitors Goals
 Define Functionalities of the Aid
 Develop Personas – visitors who use the Aid
 Develop Scenarios – how the persona uses the
navigational aid in the Folk Park
 Draw up Storyboards on scenarios

FUNCTIONALITY
 View Options
 Camera View
 Map View
 List View
 Sub-Options
 Places
 Events
 Restaurants
 AR Features
 Text Information
 3D Objects
 3D Tour Guide
 3D Placement of Buildings

ITERATIVE DESIGN PROCESS
Prototyping and User Testing
 Low Fidelity Prototyping
 Sketches
 Paper Prototyping
 Post-It Prototyping
 PowerPoint Prototyping
 High Fidelity Prototyping
 Wikitude

INITIAL SKETCHES - Photoshop
Pros:
• Good for idea generation
• Cheap
• Concepts seem feasible
Cons:
• Not great feedback gained
• Photoshop not fast enough
for making changes

POST IT PROTOTYPING
First
DraJ

Camera
View
with
3D

Second
DraJ
Third
DraJ

• 
Selec;on
highlighted
in
blue
• 
Home
bu)on
added
for
easy

naviga;on
to
main
menu

POWERPOINT PROTOTYPING
Benefits
• Used for User Testing
• Interactive
• Functionalities work
• Quick
• Easy arrangement of slides
User Testing
• Participants found
• 15 minute sessions screen captured
• ‘Talk Allowed’ technique used
• Notes taken
• Post-Interview

WIKITUDE
 Popular AR browser
 Mapping
 Point of Interest abilities
 Multiplatform
 Shows points of interest
 Markers can be selected in and
information pop-up appears

WIKITUDE
User Testing
 Application well received
 Understandable
 Participants playful with the technology

FINAL DESIGN CONCEPT

Key Issues
  Fix issues found in previous sessions
  Design with guidelines in mind
  Appealing to the Mental Model
 Icon Design
  Aesthetic Design
 Colour/Font
 Buttons
 Look

VIDEO PROTOTYPE
 Flexible
tool
for
capturing
the
use
of

an
interface

 Elaborate
simula;on
of
how
the

naviga;onal
aid
will
work

 Does
not
need
to
be
realis;c
in

every
detail

 Gives
a
good
idea
of
how
the

ﬁnished
system
will
work

Project List
  Mobile
  Hybrid Tracking for Outdoor AR
  City Scale AR Visualization
  Outdoor AR Authoring Tool
  Outdoor AR collaborative game
  AR interaction for Google Glass
  Non-Mobile
  AR Face Painting
  AR Authoring Tool
  Tangible AR puppeteer studio
  Gesture based interaction with AR content

BuildAR Assignment
  Download and install Build AR (or HIT Lab NZ)
  http://www.buildar.org/
  Build two AR scenes
  Marker based
-  at least 3 markers, 3D models, same theme
  Image based
-  at least 2 images, 3D model, text, image/video
  Due August 16th

Project Assignment
  Design/Related work exercise
  Individual
  Each person find 2 relevant papers/videos/websites
  Write two page literature review
  As a team - prototype design
  Sketch out the user interface of application
  Design the interaction flow/Screen mockups
  3 minute Presentation in class August 16th

2013 Lecture4: Designing AR Interfaces

More Related Content

What's hot

Viewers also liked

Similar to 2013 Lecture4: Designing AR Interfaces

More from Mark Billinghurst

Recently uploaded

2013 Lecture4: Designing AR Interfaces