This work presents WonderLens, a system of optical lenses and mirrors for enabling tangible interactions on printed paper. When users perform spatial operations on the optical components, they deform the visual content that is printed on paper, and thereby provide dynamic visual feedback on user interactions without any display devices. The magnetic unit that is embedded in each lens and mirror allows the unit to be identified and tracked using an analog Hall-sensor grid that is placed behind the paper, so the system provides additional auditory and visual feedback through different levels of embodiment, further enhancing the interactivity with the printed content on the physical paper.

1. WonderLens: Optical Lenses and Mirrors
for Tangible Interactions on Printed Paper
Rong-Hao Liang, Chao Shen, Yu-Chien Chan, Guan-Ting Chou,
Liwei Chan, De-Nian Yang, Mike Y. Chen, and Bing-Yu Chen
National Taiwan University and Academia Sinica
This is Rong-Hao from National Taiwan University,
Today we are glad to talk about WonderLens,
a system of optical lenses and mirrors for tangible interactions on printed paper.

2. Display devices become more and more portable

3. and even cheaper with lower power consumption.

4. Even so, paper still plays an important role today.
because paper is more comfortable to see and play with than electronic displays.
Also, paper affords natural interactions, because we already know how to use paper.

5. Printed content is not interactive
However, paper has its limitation, the content printed on paper is not interactive.
When we interact with the printed content,
the content does not change its state,
So, the interactions stop here.

6. The MagicBook
[Billinghurst et. al. 2001]
Handheld Displays or HUDs
Adding a Visual Display
To add interactivity on paper, many researches augmented a visual displays for users,
such as holding a display or wearing a head-mounted display.

7. The Digital Desk
[Wellner 1993]
Tabletop Pro-Cam Modules
Adding a Visual Display
mounting a projector-camera module to augment the paper.

8. MouseLight
[Song et. al. 2010]
Handheld Pro-Cam Modules
Adding a Visual Display
and making projector-camera modules graspable.

9. HideOut
[Willis et. al. 2013]
Handheld Pro-Cam Modules
Adding a Visual Display
These methods are effective to provide dynamic visual feedback.

10. HideOut
[Willis et. al. 2013]
MouseLight
[Song et. al. 2010]
The Digital Desk
[Wellner 1993]
The MagicBook
[Billinghurst et. al. 2001]
External Device and Notable Latency Reduce the Immersion
However, these external devices maybe bulky,
or may introducing additional latencies to reduce the immersion of interactions.

11. Listen Reader
[Back et. al. 2001]
Hiding the RFID sensor behind paper
Increasing Immersion
To increase immersion,
designers “hide” the sensing mechanism behind paper, and use audio feedback instead.
such as placing an RFID sensor behind paper with voice feedback.

12. JabberStamp
[Raffel et. al. 2007]
Hiding the EMR sensor behind paper
Increasing Immersion
or placing an EMR sensor behind paper with voice feedback.

13. provide only auditory feedback
Listen Reader
[Back et. al. 2001]
JabberStamp
[Raffel et. al. 2007]
However, the modality of interaction is limited to auditory.

14. Actually,
adding a visual display on paper does not necessarily reduce immersion,
For example, the magnifying glass helped us to see the printed content in details,
provides very immersive user experiences.

15. Interaction Model of Lenses & Mirrors
Optical Illusions
Printed content
on PaperUser
Spatial Operations
Tool: Spatial Operations ∼ Optical Illusions
When we move the magnifying glass,
it feeds the enlarged content back to us immediately.
So, we feel this tool useful.
Optical lenses and mirrors are likely to have this property.

16. PDMS
poly-dimethylsiloxane
Therefore, we explore the use of lenses and mirrors by molding acrylic and PDMS

17. PDMS
poly-dimethylsiloxane
By the way, PDMS is the material of making disposable contact lens.

18. ca b d e
Low N
High N
Low S
High S
tilted
cylindrical
lens
convex
lens
concave
lens
prism angled
mirror
5 basic lenses and mirrors
After analyzing the affordances and optical illusions,
we identify a set of 5 basic lenses and mirrors to be the most useful.

19. tilted cylindrical lens
Tilted Cylindrical lens.

20. tilted cylindrical lens
Rotating it on a Circular symbol results in visual illusion of rotation.

21. flexible convex lens
made by PDMS
Flexible Convex Lens

22. flexible convex lens
made by PDMS
Flexible Convex Lens not only allows for magnifying content by lifting it

23. flexible convex lens
made by PDMS
but also allows for changing the magnification by pressing it.

24. concave lens
Concave lens,

25. concave lens
Concave lens allows users to change the shrinkage by lifting it.

26. prism
Prism, the beam-splitter,

27. prism
allows users to break a line by rotating it.

28. prism
or connect the lines by moving it.

29. prism

30. π/3 angled mirror
Angled Mirror is similar to a kaleidoscope.

31. π/3 angled mirror
Moving it can proportionally scale a pattern.

32. π/2 angled mirror
generates several replicates,

33. π/2 angled mirror

34. π/2 angled mirror
or freely scales a pattern.

35. Printed content
on Paper
User
Lenses and Mirrors
Immediate
Visual & Haptic
Feedback
Close the Interaction Loop
The lenses and mirrors provide immediate visual and haptic feedback,
close the interaction loop,
and therefore allow for more interaction designs on printed paper.

36. Printed content
on Paper
User Computer
Lenses and Mirrors
Immediate
Visual & Haptic
Feedback
Dynamic Information
Close the Interaction Loop
But, to facilitate Human-Computer Interaction,
such as guiding a multi-step process.
the system should allow for communicating dynamic information.

37. Printed content
on Paper
User Computer
Lenses and Mirrors
Immediate
Visual & Haptic
Feedback
Input
Output
Close the Interaction Loop
If so,
the computer should sense users operations as input,
and display the output accordingly.

38. North South
TILTED CYLINDER CONVEX CONCAVE PRISM ANGLED MIRROR
c
a
b
Low N High NLow S High S
released pressed released pressed
ca b d e
Low
High
Low
High
On the input side, to get the lenses and mirrors sensed.

39. North South
TILTED CYLINDER CONVEX CONCAVE PRISM ANGLED MIRROR
c
a
b
Low N High NLow S High S
released pressed released pressed
Adding magnets on the lenses and mirrors
Invisible magnetic fields get tracked above the paper
We choose to add magnets on the lenses and mirrors.
Because the magnets are small,
and the Invisible magnetic fields can be tracked on and above the paper.

40. North South
TILTED CYLINDER CONVEX CONCAVE PRISM ANGLED MIRROR
c
a
b
Low N High NLow S High S
released pressed released pressed
Adding magnets on the lenses and mirrors
Invisible magnetic fields get tracked above the paper
Each magnetic unit is designed in unique pattern.

41. North South
TILTED CYLINDER CONVEX CONCAVE PRISM ANGLED MIRROR
c
a
b
Low N High NLow S High S
released pressed released pressed
Adding magnets on the lenses and mirrors
Invisible magnetic fields get tracked above the paper
So the magnetic fields can represent their ID and states.

42. GaussSense
Analog Hall-Sensor Grid
[Liang et al. 2012]
Sense the Lenses and Mirrors
To sense the magnetic lenses and mirrors,
We use GaussSense, the thin-form analog Hall-Sensor Grid,
as the sensing platform.

43. Sense the Lenses and Mirrors
The GaussSense senses multiple magnetic lenses and mirrors through the paper.

44. RFID Reader
magnets
GaussSense
Analog Hall-Sensor Grid
[Liang et al. 2012]
For detecting paper, we mount an additional RFID reader and two small magnets on the
sensing platform.

45. Paper
with RFID attached
magnets
Each piece of paper is attached with an RFID tag and two magnets.

46. When the piece of paper snaps to the platform,
the platform recognizes the RFID tag, and loads the content for interactions.
The magnets align the coordinates between paper and the GaussSense.

47. User Computer
Lenses and Mirrors
Immediate
Visual & Haptic
Feedback
Magnetic
GaussSense
RFID Reader
Printed content
on RFID Paper
[Liang et al. 2012]
Output
Environmental: ambient light, audio
Distant: remote display
Nearby: point light
Kenneth P. Fishkin. 2004. A taxonomy for and analysis of tangible interfaces.
Personal Ubiquitous Comput. 8, 5 (September 2004), 347-358.
Close the Interaction Loop
After sensed the operation,
the system can provide additional output in different levels of embodiment.
We show 3 examples to illustrate them.

48. Application #1: Storytelling
Audio Feedback
First, Output with only Audio Feedback. In storytelling,

49. Application #1: Storytelling
Audio Feedback
when a user rolls the character’s eye,
and the character vocally tells the user what he sees.

50. Application #1: Storytelling
Audio Feedback
When the two characters look at each other, they chat.

51. Application #2: CPR-Learning
Nearby Point Light
stethoscope
Second, Output with a Nearby Point Light.
In the CPR learning program,
a user uses an LED-mounted flexible convex lens as a stethoscope.

52. Application #2: CPR-Learning
Nearby Point Light
stethoscope
When placing the stethoscope on a patient’s heart,
the user sees the point light changes and hears the heartbeats.

53. Application #2: CPR-Learning
Nearby Point Light
Then, the user presses and releases the soft convex lens at a constant rate to save the
patient.
The blinking point light shows it is well done.

54. BLE+Battery
By the way,
the LED-mounted convex lens can be made wirelessly with BLE modules and battery.

55. Application #3: Hide-and-Seek
Ambient Light + Remote Display
Third, Output with Ambient Light & Remote Display.
In the hide-and-seek game,
a user sets the time of the game by placing an angled mirror.

56. Application #3: Hide-and-Seek
Ambient Light + Remote Display
then, the ambient light and remote display changes.

57. Application #3: Hide-and-Seek
Ambient Light + Remote Display
Then, use the magnifying glass to find out the character.

58. Application #3: Hide-and-Seek
Ambient Light + Remote Display
When a character is found,
the glowing lens prompts the user to check the remote display,
to see who is found and what the character is doing.

59. User Computer
Lenses and Mirrors
Immediate
Visual & Haptic
Feedback
Magnetic
GaussSense
RFID Reader
Printed content
on RFID Paper
Environmental: ambient light, audio
Distant: remote display
Nearby: point light
Kenneth P. Fishkin. 2004. A taxonomy for and analysis of tangible interfaces.
Personal Ubiquitous Comput. 8, 5 (September 2004), 347-358.
Conclusion Close the Interaction Loop
Conclusion, we introduced WonderLens,
a system of lenses and mirrors that augments tangible and embodied interactions with
printed paper.
The double interaction loop allows for communicating dynamic information.

60. Future Work
Printed Optics [Willis et al. 2012]
Magic Lens [Willis et al. 2012]
Paper Generator [Karagozler et al. 2013]
3D Optic Printing Energy Harvesting
Advanced Lens Fabrication and Visual Designs
Future work can consider using advanced methods of lens fabrication and visual designs.
or incorporating energy harvesting mechanism with paper.

61. Attachable Stylus Sensing Using Magnetic Sensor Grid
GaussSense
GaussBits
GaussBricks
GaussBits
Magnetic
Tangible Bits
[Liang et al. CHI 2013]
GaussBricks
Magnetic
Building Blocks
[Liang et al. CHI 2014]
GaussSense
Analog
Hall-Sensor Grid
[Liang et al. UIST 2012]
GaussStones
Shielded
Magnetic Tangibles
[Liang et al. UIST 2014]
GaussStones
eng†
Chao-Huai Su†
Chen‡
@cmlab.csie.ntu.edu.tw
d
c
c
ouchpad function through pinch ges-
words to the private glass display (b)
mb tip on the index fingertip. (c) The
hrough magnetic tracking by adding a
he fingernails.
splays permit personal and pri-
put methods may not offer the
voice input is commonly used
because it is expressive and ef-
put can be problematic in loud
ssues arise with its use in pub-
nput)[11]. Gesture input suffers
ns because input actions are eas-
nt research proposes subtle inter-
re based on implicit movements
cially acceptable. For example,
input through unobservable mus-
[12] detects subtle foot motions.
rics [6] have been developed to
inputs. Although these methods
allow for privacy and social ac-
uffer from limited input space.
Pad, a nail-mounted device that
o a touchpad, allowing private,
FingerPad
Wearable
Private Input
[Chan et al. UIST 2013]
GaussSense
WonderLens:
Optical Lenses and Mirrors for
Tangible Interactions on Printed Paper
Rong-Hao Liang, Chao Shen, Yu-Chien Chan, Guan-Ting Chou,
Liwei Chan, De-Nian Yang, Mike Y. Chen, and Bing-Yu Chen
National Taiwan University and Academia Sinica
WonderLens
WonderLens
TUI on
Printed Paper
[Liang et al. CHI 2015]
FingerPad
This project, WonderLens, shows another applications of GaussSense,
that is enabling tangible interactions on Printed Paper.
For makers and researchers, who want to try out the GaussSense technology,
we are happy to announce that.

62. Attachable Stylus Sensing Using Magnetic Sensor Grid
GaussSense
GaussBits
GaussBricks
GaussBits
Magnetic
Tangible Bits
[Liang et al. CHI 2013]
GaussBricks
Magnetic
Building Blocks
[Liang et al. CHI 2014]
GaussSense
Analog
Hall-Sensor Grid
[Liang et al. UIST 2012]
GaussStones
Shielded
Magnetic Tangibles
[Liang et al. UIST 2014]
GaussStones
eng†
Chao-Huai Su†
Chen‡
@cmlab.csie.ntu.edu.tw
d
c
c
ouchpad function through pinch ges-
words to the private glass display (b)
mb tip on the index fingertip. (c) The
hrough magnetic tracking by adding a
he fingernails.
splays permit personal and pri-
put methods may not offer the
voice input is commonly used
because it is expressive and ef-
put can be problematic in loud
ssues arise with its use in pub-
nput)[11]. Gesture input suffers
ns because input actions are eas-
nt research proposes subtle inter-
re based on implicit movements
cially acceptable. For example,
input through unobservable mus-
[12] detects subtle foot motions.
rics [6] have been developed to
inputs. Although these methods
allow for privacy and social ac-
uffer from limited input space.
Pad, a nail-mounted device that
o a touchpad, allowing private,
WonderLens
WonderLens
TUI on
Printed Paper
[Liang et al. CHI 2015]
FingerPad
Wearable
Private Input
[Chan et al. UIST 2013]
FingerPad
GaussSense
WonderLens:
Optical Lenses and Mirrors for
Tangible Interactions on Printed Paper
Rong-Hao Liang, Chao Shen, Yu-Chien Chan, Guan-Ting Chou,
Liwei Chan, De-Nian Yang, Mike Y. Chen, and Bing-Yu Chen
National Taiwan University and Academia Sinica
http://gausstoys.com
$19 GaussSense is coming soon!
subscribe this information on:
The 19-dollar GaussSense is coming soon.
We have worked very hard on it, and really excited to see this finally happens.
If you are interested in, please subscribe us on GaussToys.com.
So we can keep you in the loop.
Thanks for your attention, and I’m happy to take all of your questions.