A look at the current state of augmented reality audio, the major players, inflection points on the horizon, and specific ways geotagged audio can apply to travel -- both today and in the future.

1. Hearing voices
AUGMENTED REALITY AUDIO CAN CHANGE
THE WAY PEOPLE EXPERIENCE THE WORLD,
BOTH AT HOME AND IN TRAVEL
Tim Haynes
Principal Storyteller
Sabre Labs

2. What is augmented reality (AR) audio?
• A layer of digital audio information that enriches the
physical world
• Usually associated with playback of location-based
(geotagged) digital audio information
• Includes the hardware and software necessary to play
layers of geotagged content in a meaningful way
• Can also include digital augmentation of aural senses
to affect a user’s perception of an environment

3. Partial graveyard of failed AR audio platforms

4. Causes for the graveyard
Discoverability — Lack of standard name or buzzword
• 2011, “dynamic spatial audio” & “ambient spatial audio”
were buzzwords for the next big thing with smartphones
• Also: location-based audio, geotagged audio, locative
audio, augmented reality audio, location-aware audio…
Failure of the “why” — Startups failed to find a broad and
compelling use case for AR audio, focusing on niches
Lack of partnership — Most AR audio apps are startups
and not integrated into, partnered with or promoted by an
existing, successful platform and user base

5. Continuing challenge: poor mobile interfaces
Content playback
• Many apps (both past and present) were never
optimized for mobile, even in-app
• If it’s hard to find or to play content, an app will fail
Content creation
• Most apps have very poor interfaces for adding
location-specific content, especially mobile creation
• The best creation interfaces have tended to be focused
on art and documentary uses in niche locations

6. Usability premise:
Audio makes the real world more interesting
This is Massachusetts Ave. in Cambridge, MA.
On the next slide, try different clips of music and see how it changes the way the scene feels.
I recommend trying Yann Tiersen, Amelie Soundtrack, “La valse d’Amelie, version original,”
followed by some classic jazz and then something hip hop or alt-country.

8. How is AR audio being used?
• Art — theater and conceptual art installations
• Documentary— sharing and cataloguing stories
• Gaming — AR and MR possibilities are endless
• Navigation — associating sounds with places
• Notifications — directions, alerts, messages, etc.
• Social — sharing experiences, both real and virtual
• Tours — cities, historical sites, museums

9. Placing sounds in space as interactive art
Art
2010 – Audio Graffiti
project, Connecticut
2010 – Scapes project*,
Decordova Sculpture
Park, Lincoln, MA
*created w/Roundware

10. Blended audio tours of art spaces
Art
Voices: FAMSF*
(2015) An “immersive
soundscape” with
multiple voices and
music centered on 21
sculptures; users can
personalize with a mix
of curators, art experts
and fellow patrons
*created w/Roundware

11. Ghosts of poets in Harvard Yard
Art
re~verse (2015)
commissioned for Harvard
Litfest, “consists of a
continuous layer of location-
sensitive music punctuated
with recorded voices of
poets who have visited
Harvard in the past and given
readings of their work.”
*created w/Roundware

12. High Street Stories
Christchurch, NZ, aftermath of 2011 earthquakes
Documentary

13. Tributaries*
Tyneside, Northern England, WWI recordings
Documentary
*created w/Roundware

14. Listening to the voices I forgot
where I was and was taken
somewhere else.”
—KERRY BOSSONS
PLAY STORE REVIEW OF TRIBUTARIES
“
Documentary

15. Location-based gaming is on the rise
Gaming
Geocaching Zombies, Run! Pokemon GO

16. Incorporating AR audio into gaming
Gaming
• Several ambitious / experimental smartphone games over the
past few years have used AR audio as their main interface:
• Inception and Dimensions produced location-based sound and music
and use the phone’s microphone to generate a unique soundscape
• Papa Sangre allowed players to navigate a virtual horror soundscape,
attempting to gather lost souls and escape enemies
• Meltdown never made it out of prototype, but was an Alien-esque
game in which you hunted aliens on foot using only your ears
• None of these AR audio games are still around (AFAIK)
• Immersion in AR games can benefit tremendously from
location-based audio

17. Using AR audio to navigate
Navigation
• Turn-by-turn directions are a common feature of
virtually all GPS-based mapping apps, but that’s the
most basic use
• Just as the sounds of a city help tell you where you
are, an AR audio layer of sound effects and music can
convey additional spatial or location information
• Art installations are examples—sound effects and
music are tied to places in a exhibition and volume and
directionality are directly affected by your location

18. MIT Media Labs, Loco-Radio project
Navigation
Music for navigation and discovery

19. Using AR audio to give people important context
Notifications
• At it’s most basic, having earbuds in can provide private
notification of messages or alerts
• More importantly, location-aware alerts can inform people of:
• Points of interest
• Transportation options
• Potential hazards
• On the commerce side—location alerts can market
promotions, sales, and/or experience availability

20. Discoverable content for the visually impaired
Notifications
Ariadne GPS (2012)
Featured as part of Apple
Keynote 2012; create
personalized maps with
alerts and access audio
information through touch
about maps in any area.
Allows users to understand
an area they are unable to
see with their eyes

21. Social elements of AR audio overlap other uses
Social
• Startups have attempted to use location-based audio
with chat, friend proximity and other social services
• Art and documentary installations connect visitors
with other participants over time
• Gaming uses have the potential to be deeply social
• Synchronized tour groups are inherently social as
shared experiences

22. People want to feel an
authentic connection with...
what makes a place different
from all the other places they
could visit in the world.”
—ANDREW MASON
DETOUR FOUNDER AND CEO
“
Tours

23. Audio tours are increasingly important to travel
Tours
• Plethora of apps for “traditional” audio tours:
Izi, Pocket Guide, Rick Steves, UniGuide, etc…
• High adoption by Millennial and Z generations
• Detour is the current AR audio darling, one-upping
traditional tour approaches and incorporating limited
use of AR audio elements

24. +

25. Detour
NPR-style tours of San Francisco +9 other cities
• Current tours are
professionally produced,
taking 3-6 months each
• Participants need to
follow tour directions
• Music and speech run
separately to account for
different movement rates
• Groups can sync tours to
share in the experience
• “Descript” turns editing
voice to word processing
Tours

26. Audio tours can be cinematic
experiences that get you
closer to the feeling of walking
in another person’s shoes than
any other medium can.”
—ANDREW MASON
DETOUR FOUNDER AND CEO
“
Tours

27. Touring doesn’t have to be proscribed
Tours
• I experimented with various platforms for AR audio content:
• Audio Walk — Assign audio files to geographic locations; all
features currently appear broken
• Echoes — Limited content (mostly European), significant login
issues and lack of support for content creation
• GeoTourist — Basic tours, decent discoverability; open-to-public,
Web-based content creation tool (no mobile creation)
• HearUsHere — Handful of tours, poor discoverability of nearby
content, and content creation done only by company
• Roundware – really excellent open-source, but it would take a
team effort to create a viable experience
• SonicMaps — Terrible interface; lots of broken links; counter-
intuitive content creation

28. What are the advantages of AR audio?
• Constant context
• Adds information without monopolizing attention
• Fully adaptive to individual pace and schedule
• No fiddling with a device – you interface by moving
• Aural illusion of directionality and distance
• The brain is really good at filtering sounds, allowing
audio AR to target different layers of your attention

29. How does AR audio apply to travel?

30. AR audio opportunities for travel: Art
• Theater performances could overlap w/games, potentially
allowing users to take on roles asynchronous w/other users
or create new kinds of improve performances
• Art installations enrich content and add a participatory
element to art cultural experiences – either standalone AR
audio or integrated w/other art – potentially driving additional
traffic to specific locations

31. AR audio opportunities for travel: Documentary
• Needs a shared social platform for documentary content for
viability
• If not a standardized creation app, at least a site or app that
aggregates, curates and maps niche documentary apps to
increase discoverability
• Documentary AR audio allows people to experience locations
from new and diverse perspectives, enriching the value of
traveling—and spending additional time—in a particular place

32. AR audio opportunities for travel: Gaming
• Huge opportunities for AR audio gaming experiences, both
standalone and with AR visuals (more common), as well as
integrated into deeper merged reality experiences
• Location-based digital souvenirs are likely to be a huge part of
future travel, largely driven by younger generations; if not the
impetus for travel, definitely part of the anticipated travel
experience
• When AR audio and visuals combine, “gamification” of
everyday life is likely to become more common; lessons
learned now by experimenting will help shape future uses

33. AR audio opportunities for travel: Navigation
• AR audio is increasingly essential to travel, both for direction
and for discoverability
• Layering a location-based soundtrack could be a powerful
wayfinding method for large buildings or city areas, tying
locations or areas to certain music or soundscapes
• Layered with gaming elements, audio navigation can
potentially drive people to visit new places
• In the near future, may start to include “superpowers” like
near-real time translation of voices and music

34. AR audio opportunities for travel: Notifications
• Being able to layer directions with other sounds makes travel
more seamless (vs. current method of most audio
apps/streams demanding exclusivity (e.g. Google Maps stops
your music to give you directions)
• Audio alerts can help maintain hands-free, eyeballs-free travel
through both indoor and outdoor spaces
• Listening to messages (and advertisements) can be layered
as part of background audio, helping with multitasking while
moving

35. AR audio opportunities for travel: Social
• To date, efforts to make AR audio social have largely failed, in part
due to the challenge of most AR audio apps doing too little to be
worth using; none have been better than existing social options
• Airbnb Tours may be the app that creates critical mass to
encourage AR audio experiences, and the leisure nature of much of
Airbnb’s business may help give AR audio an inherent social utility
of experiencing AR audio in the real world together
• A simpler route that may emerge is for a photo-first platform like
Instagram to make it easy to integrate audio clips with geotagged
images, changing how people catalogue both everyday and travel
experiences

36. AR audio opportunities for travel: Tours
• AR audio elements need to be integrated into existing city-
discoverability platforms, from mainstream services like
TripAdvisor & Rough Guides to startups like Dojo and Peek
• Tour needs to take guidance from both Detour and from
prototypes like Loco-radio, taking users and usage possibilities into
account in soundscape design
• When Detour opens its platform and Descript audio editing, it may
radically accelerate tour content development (could take off the
same way podcasts have)
• Integration of AR audio content into Google Maps (or similar)
would be a game changer, normalizing ad hoc tour design

37. Recommendations to experiment w/ AR audio

38. Considerations for implementation
• Rate of user’s movement (walking, biking, driving, etc.)
• Sensory immersion level (e.g. walking=high, driving=low)
• Scaling for distance and directionality of sound
• Foreground vs. background layers (speech vs. music or noise)
• Overlapping layers of sound
• Appropriate duration of audio elements (affected by rate of
movement and density of audio elements)
Recommendations

39. Geotourist
Top option for basic, location-based tour creation
Recommendations

40. Roundware
Top development platform for sophisticated AR audio
• Roundtable is an open-source platform being used for many of the
most creative examples of AR audio
• Ideal for creating and experimenting with “niche” experiences
• Allows real-time, geotagged collection of audio from smartphones
• Filters and plays back collective, continuous audio streams
• Creates seamless, non-linear, location-sensitive layer of audio
mixed on the fly based on user input
• Uses Python for server, Swift for iOS app, Java for Android app
• “Soon” to be expanded to include video, photos and text
Recommendations

41. Possible inflection points for AR audio taking off
• Airbnb promoting Detour, then Detour making both creation
and distribution tools open to the public
• Apple, Google or Microsoft making AR audio an integral part
of their existing map software
• Shift to wireless headphones may be a minor, but notable step
towards always-on AR audio
• Visual AR (Hololens, etc.) becoming widespread, and bringing
AR audio along with it (several years out)
Recommendations

42. Thank you for reading.
I HOPE THIS HAS BEEN HELPFUL.
IF YOU HAVE QUESTIONS OR WISH TO
DISCUSS TRAVEL IMPLICATIONS OF
AUGMENTED REALITY OR OF OTHER TECHS,
PLEASE REACH OUT.
timothy.haynes@sabre.com