From audiopaper to next generation paper

www.surrey.ac.uk/dwrc/
From audio paper to next
generation paper
Professor David Frohlich
18th October 2017

Motivation

Trend: From print to web media
e.g. domestic photography
WEB MEDIA
PRINT MEDIA

Technology: Augmented paper
Electronic paper (e-books) versus Augmented paper
DIGITAL PAPER-AND-DIGITAL
“While the electronic paper approach aims to make existing devices as
paper-like as possible, the augmented or interactive paper approach
focuses on augmenting regular paper by linking it to supplemental digital
information”. (Signer & Norrie 2010 Interactive paper: Past, present and
future. UbiComp ‘10)

Augmented paper fights back?
e.g. domestic photography
WEB MEDIA
PRINT MEDIA

Vision concept:
The Audio-photobook (https://vimeo.com/album/3430190)

Research questions for this talk
1. What is the role of augmenting paper with audio?
2. How can we combine the best of print and web media more
generally?

Plan
• Audioprints (design explorations)
• Interactive newsprint (design explorations)
• Next generation paper (vision)
• Conclusions

Audioprints

Audiophotography research
HP Labs 1995-2005, DWRC 2014
2004 2015

Audiocamera prototypes
1998
2000
2002

The multilayered audiophoto

HP audiophotography products
HP Jornada 1998 HP Photosmart 850 camera 2000
HP Memories Disc software 2002-2005HP Compact camera line 2001-2005

Current audiophoto apps 2015

The audioprint opportunity:
Playback of sound from the printed photo
Audioscanner 1996 Audioprint player 1998
Audiocard/frame 1998Audiophoto album 1998

Feedback from audiophoto trials
(Frohlich & Tallyn 1999, Frohlich 2004)
• All participants liked both screen and paper ‘playback’ .. for
different purposes. People still like tangible prints.
• The audioscanner was seen as personal, intimate and
accessible to older users
• The audioprint player was seen as a vehicle for playback of
sounds but also recording voice messages on photos
• The audioprint album was seen as too heavy to carry and more
suitable for special event collections and gifts.
• The audio cards were prefered to audio frames especially for
sending to others

The Audiophoto Desk
Frohlich, Clancy, Robinson & Costanza (2004)

The Audiophoto desk
Frohlich, Clancy, Robinson & Costanza (2004)

Feedback from an
audiophoto desk trial (Lindley & Monk 2004)
• Sound adds humour to the interaction especially when it doesn’t
match the image
• Mixing sounds and images on the desk
made the interaction more engaging
• Sound affects the conversation around
photographs
• Music was left playing in the background
for atmosphere and mood
• Ambient sounds became a talking point
in their own right
• Voiceover inhibits conversation
• Sound adds humour to the interaction especially when it doesn’t
match the image
• Mixing sounds and images on the desk
made the interaction more engaging
• Sound affects the conversation around
photographs
• Music was left playing in the background
for atmosphere and mood
• Ambient sounds became a talking point
in their own right
• Voiceover inhibits conversation

Emerging audioprint products 2015
HP Live Photos
Tap for message stickerPhoto storybook from Shutterfly
Moon Pig cards

Interactive newsprint

Interactive newsprint project
2011-2013

Interactive newsprint prototype

Audio annotations and voting
options

Newspaper formats
Print origin: print format Print origin: web format
Web origin: web format
Web origin: print format

Feedback from interactive
newsprint experiment (Frohlich et al 2017)
• Responses varied depended on users’ existing news reading
habits (paper v screen)
• There were perceived values of audio in providing extra
information, local commentary in a regional ‘voice’, and access
to original interviews and other evidence for the stories
• Users would often read and listen out of synch, to consume the
material faster ( a ‘radio-newspaper’)
• Print formats with interactive ’extras’ were seen as most
attractive. Extras should give free added value.
• Some users wanted the printed paper to be more pictorial with
textual content moving to speech

Feedback continued:
Thumb selection and pinch regions

Next generation paper

Next generation paper project
November 2017 for 30 months

Objectives: To generalise
audiopaper to ‘webpaper’
1. To connect complementary print and web media through touch
2. To support audio and visual playback of associated web media
on a variety of devices. Make paper interact with the devices
around it.
3. To develop and integrate two big classes of augmented paper in
a common architectural framework (c.f. 1G passive paper)
1. 2G optically recognised paper
2. 3G touch sensitive paper
4. To create and test compelling demonstrations for BOOKS in the
travel and tourism domain
5. To research and develop new business models www.surrey.ac.uk/dwrc/

Common architecture

Domain: travel books and records
Before During After

Conclusions

Answers to research questions
What is the role of augmenting paper with audio?
• Mixed up with augmenting text and image with audio (on e-
books)
• Audio annotation can have all kinds of values from adding
atmosphere, emotion, triggering memory, storytelling and
elaboration.
• In a news context audio can add extra information with a
regional voice, and provide access to source interviews
• Audiopaper is interactively intuitive because you read printed
material as usual, whilst listening to sounds
• It is seen as tangible, sharable, accessible, novel and
attractive, and complements other forms of screen-based
playback

Answers to research questions
How can we combine the best of print and web media more
generally?
• Generalising ‘audiopaper’ to ‘webpaper’ leads to a vision of
paper interacting with audiovisual content on a variety of
devices nearby
• Two types of technology need to be supported in a common
framework: optical (2G) and printed electronic (3G)
• Eventually links could be two-way allowing web material to be
printed with working hotlinks, or apps with paper interfaces

The webpaper opportunity
PRINT MEDIA WEB MEDIA

References
Frohlich, D.M. (2004) Audiophotography: Bringing photos to life with sounds. Kluwer Academic
Publishers.
Frohlich, D.M. (2015) Fast design, slow innovation: Audiophotography ten years on. Springer.
Frohlich, D.M. & Tallyn E. (1999) Audiophotography: Practice and prospects. CHI ’99 Extended
Abstracts: 296-297. New York, ACM Press.
Frohlich D.M., Clancy T., Robinson J. & Costanzo E. (2004) The audiophoto desk. Proceedings of
2AD: Second International Conference on Appliance Design (p139).
Frohlich, D.M. et al (2017) Designing interactive newsprint. International Journal of Human
Computer Studies 104:36-49.
Lindley, S. E., & Monk, A. F. (2005). Augmenting photographs with sound for collocated sharing.
In Home-Oriented Informatics and Telematics (pp. 155-170). Springer US.
Signer & Norrie 2010 Interactive paper: Past, present and future. UbiComp ‘10

From audiopaper to next generation paper

More Related Content

What's hot

Similar to From audiopaper to next generation paper

Recently uploaded

From audiopaper to next generation paper