Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CHI'15 - WonderLens: Optical Lenses and Mirrors for Tangible Interactions on Printed Paper

1,568 views

Published on

This work presents WonderLens, a system of optical lenses and mirrors for enabling tangible interactions on printed paper. When users perform spatial operations on the optical components, they deform the visual content that is printed on paper, and thereby provide dynamic visual feedback on user interactions without any display devices. The magnetic unit that is embedded in each lens and mirror allows the unit to be identified and tracked using an analog Hall-sensor grid that is placed behind the paper, so the system provides additional auditory and visual feedback through different levels of embodiment, further enhancing the interactivity with the printed content on the physical paper.

Published in: Technology

CHI'15 - WonderLens: Optical Lenses and Mirrors for Tangible Interactions on Printed Paper

  1. 1. WonderLens: Optical Lenses and Mirrors for Tangible Interactions on Printed Paper Rong-Hao Liang, Chao Shen, Yu-Chien Chan, Guan-Ting Chou, Liwei Chan, De-Nian Yang, Mike Y. Chen, and Bing-Yu Chen National Taiwan University and Academia Sinica This is Rong-Hao from National Taiwan University, Today we are glad to talk about WonderLens, a system of optical lenses and mirrors for tangible interactions on printed paper.
  2. 2. Display devices become more and more portable
  3. 3. and even cheaper with lower power consumption.
  4. 4. Even so, paper still plays an important role today. because paper is more comfortable to see and play with than electronic displays. Also, paper affords natural interactions, because we already know how to use paper.
  5. 5. Printed content is not interactive However, paper has its limitation, the content printed on paper is not interactive. When we interact with the printed content, the content does not change its state, So, the interactions stop here.
  6. 6. The MagicBook [Billinghurst et. al. 2001] Handheld Displays or HUDs Adding a Visual Display To add interactivity on paper, many researches augmented a visual displays for users, such as holding a display or wearing a head-mounted display.
  7. 7. The Digital Desk [Wellner 1993] Tabletop Pro-Cam Modules Adding a Visual Display mounting a projector-camera module to augment the paper.
  8. 8. MouseLight [Song et. al. 2010] Handheld Pro-Cam Modules Adding a Visual Display and making projector-camera modules graspable.
  9. 9. HideOut [Willis et. al. 2013] Handheld Pro-Cam Modules Adding a Visual Display These methods are effective to provide dynamic visual feedback.
  10. 10. HideOut [Willis et. al. 2013] MouseLight [Song et. al. 2010] The Digital Desk [Wellner 1993] The MagicBook [Billinghurst et. al. 2001] External Device and Notable Latency Reduce the Immersion However, these external devices maybe bulky, or may introducing additional latencies to reduce the immersion of interactions.
  11. 11. Listen Reader [Back et. al. 2001] Hiding the RFID sensor behind paper Increasing Immersion To increase immersion, designers “hide” the sensing mechanism behind paper, and use audio feedback instead. such as placing an RFID sensor behind paper with voice feedback.
  12. 12. JabberStamp [Raffel et. al. 2007] Hiding the EMR sensor behind paper Increasing Immersion or placing an EMR sensor behind paper with voice feedback.
  13. 13. provide only auditory feedback Listen Reader [Back et. al. 2001] JabberStamp [Raffel et. al. 2007] However, the modality of interaction is limited to auditory.
  14. 14. Actually, adding a visual display on paper does not necessarily reduce immersion, For example, the magnifying glass helped us to see the printed content in details, provides very immersive user experiences.
  15. 15. Interaction Model of Lenses & Mirrors Optical Illusions Printed content on PaperUser Spatial Operations Tool: Spatial Operations ∼ Optical Illusions When we move the magnifying glass, it feeds the enlarged content back to us immediately. So, we feel this tool useful. Optical lenses and mirrors are likely to have this property.
  16. 16. PDMS poly-dimethylsiloxane Therefore, we explore the use of lenses and mirrors by molding acrylic and PDMS
  17. 17. PDMS poly-dimethylsiloxane By the way, PDMS is the material of making disposable contact lens.
  18. 18. ca b d e Low N High N Low S High S tilted cylindrical lens convex lens concave lens prism angled mirror 5 basic lenses and mirrors After analyzing the affordances and optical illusions, we identify a set of 5 basic lenses and mirrors to be the most useful.
  19. 19. tilted cylindrical lens Tilted Cylindrical lens.
  20. 20. tilted cylindrical lens Rotating it on a Circular symbol results in visual illusion of rotation.
  21. 21. flexible convex lens made by PDMS Flexible Convex Lens
  22. 22. flexible convex lens made by PDMS Flexible Convex Lens not only allows for magnifying content by lifting it
  23. 23. flexible convex lens made by PDMS but also allows for changing the magnification by pressing it.
  24. 24. concave lens Concave lens,
  25. 25. concave lens Concave lens allows users to change the shrinkage by lifting it.
  26. 26. prism Prism, the beam-splitter,
  27. 27. prism allows users to break a line by rotating it.
  28. 28. prism or connect the lines by moving it.
  29. 29. prism
  30. 30. π/3 angled mirror Angled Mirror is similar to a kaleidoscope.
  31. 31. π/3 angled mirror Moving it can proportionally scale a pattern.
  32. 32. π/2 angled mirror generates several replicates,
  33. 33. π/2 angled mirror
  34. 34. π/2 angled mirror or freely scales a pattern.
  35. 35. Printed content on Paper User Lenses and Mirrors Immediate Visual & Haptic Feedback Close the Interaction Loop The lenses and mirrors provide immediate visual and haptic feedback, close the interaction loop, and therefore allow for more interaction designs on printed paper.
  36. 36. Printed content on Paper User Computer Lenses and Mirrors Immediate Visual & Haptic Feedback Dynamic Information Close the Interaction Loop But, to facilitate Human-Computer Interaction, such as guiding a multi-step process. the system should allow for communicating dynamic information.
  37. 37. Printed content on Paper User Computer Lenses and Mirrors Immediate Visual & Haptic Feedback Input Output Close the Interaction Loop If so, the computer should sense users operations as input, and display the output accordingly.
  38. 38. North South TILTED CYLINDER CONVEX CONCAVE PRISM ANGLED MIRROR c a b Low N High NLow S High S released pressed released pressed ca b d e Low High Low High On the input side, to get the lenses and mirrors sensed.
  39. 39. North South TILTED CYLINDER CONVEX CONCAVE PRISM ANGLED MIRROR c a b Low N High NLow S High S released pressed released pressed Adding magnets on the lenses and mirrors Invisible magnetic fields get tracked above the paper We choose to add magnets on the lenses and mirrors. Because the magnets are small, and the Invisible magnetic fields can be tracked on and above the paper.
  40. 40. North South TILTED CYLINDER CONVEX CONCAVE PRISM ANGLED MIRROR c a b Low N High NLow S High S released pressed released pressed Adding magnets on the lenses and mirrors Invisible magnetic fields get tracked above the paper Each magnetic unit is designed in unique pattern.
  41. 41. North South TILTED CYLINDER CONVEX CONCAVE PRISM ANGLED MIRROR c a b Low N High NLow S High S released pressed released pressed Adding magnets on the lenses and mirrors Invisible magnetic fields get tracked above the paper So the magnetic fields can represent their ID and states.
  42. 42. GaussSense Analog Hall-Sensor Grid [Liang et al. 2012] Sense the Lenses and Mirrors To sense the magnetic lenses and mirrors, We use GaussSense, the thin-form analog Hall-Sensor Grid, as the sensing platform.
  43. 43. Sense the Lenses and Mirrors The GaussSense senses multiple magnetic lenses and mirrors through the paper.
  44. 44. RFID Reader magnets GaussSense Analog Hall-Sensor Grid [Liang et al. 2012] For detecting paper, we mount an additional RFID reader and two small magnets on the sensing platform.
  45. 45. Paper with RFID attached magnets Each piece of paper is attached with an RFID tag and two magnets.
  46. 46. When the piece of paper snaps to the platform, the platform recognizes the RFID tag, and loads the content for interactions. The magnets align the coordinates between paper and the GaussSense.
  47. 47. User Computer Lenses and Mirrors Immediate Visual & Haptic Feedback Magnetic GaussSense RFID Reader Printed content on RFID Paper [Liang et al. 2012] Output Environmental: ambient light, audio Distant: remote display Nearby: point light Kenneth P. Fishkin. 2004. A taxonomy for and analysis of tangible interfaces. Personal Ubiquitous Comput. 8, 5 (September 2004), 347-358. Close the Interaction Loop After sensed the operation, the system can provide additional output in different levels of embodiment. We show 3 examples to illustrate them.
  48. 48. Application #1: Storytelling Audio Feedback First, Output with only Audio Feedback. In storytelling,
  49. 49. Application #1: Storytelling Audio Feedback when a user rolls the character’s eye, and the character vocally tells the user what he sees.
  50. 50. Application #1: Storytelling Audio Feedback When the two characters look at each other, they chat.
  51. 51. Application #2: CPR-Learning Nearby Point Light stethoscope Second, Output with a Nearby Point Light. In the CPR learning program, a user uses an LED-mounted flexible convex lens as a stethoscope.
  52. 52. Application #2: CPR-Learning Nearby Point Light stethoscope When placing the stethoscope on a patient’s heart, the user sees the point light changes and hears the heartbeats.
  53. 53. Application #2: CPR-Learning Nearby Point Light Then, the user presses and releases the soft convex lens at a constant rate to save the patient. The blinking point light shows it is well done.
  54. 54. BLE+Battery By the way, the LED-mounted convex lens can be made wirelessly with BLE modules and battery.
  55. 55. Application #3: Hide-and-Seek Ambient Light + Remote Display Third, Output with Ambient Light & Remote Display. In the hide-and-seek game, a user sets the time of the game by placing an angled mirror.
  56. 56. Application #3: Hide-and-Seek Ambient Light + Remote Display then, the ambient light and remote display changes.
  57. 57. Application #3: Hide-and-Seek Ambient Light + Remote Display Then, use the magnifying glass to find out the character.
  58. 58. Application #3: Hide-and-Seek Ambient Light + Remote Display When a character is found, the glowing lens prompts the user to check the remote display, to see who is found and what the character is doing.
  59. 59. User Computer Lenses and Mirrors Immediate Visual & Haptic Feedback Magnetic GaussSense RFID Reader Printed content on RFID Paper Environmental: ambient light, audio Distant: remote display Nearby: point light Kenneth P. Fishkin. 2004. A taxonomy for and analysis of tangible interfaces. Personal Ubiquitous Comput. 8, 5 (September 2004), 347-358. Conclusion Close the Interaction Loop Conclusion, we introduced WonderLens, a system of lenses and mirrors that augments tangible and embodied interactions with printed paper. The double interaction loop allows for communicating dynamic information.
  60. 60. Future Work Printed Optics [Willis et al. 2012] Magic Lens [Willis et al. 2012] Paper Generator [Karagozler et al. 2013] 3D Optic Printing Energy Harvesting Advanced Lens Fabrication and Visual Designs Future work can consider using advanced methods of lens fabrication and visual designs. or incorporating energy harvesting mechanism with paper.
  61. 61. Attachable Stylus Sensing Using Magnetic Sensor Grid GaussSense GaussBits GaussBricks GaussBits Magnetic Tangible Bits [Liang et al. CHI 2013] GaussBricks Magnetic Building Blocks [Liang et al. CHI 2014] GaussSense Analog Hall-Sensor Grid [Liang et al. UIST 2012] GaussStones Shielded Magnetic Tangibles [Liang et al. UIST 2014] GaussStones eng† Chao-Huai Su† Chen‡ @cmlab.csie.ntu.edu.tw d c c ouchpad function through pinch ges- words to the private glass display (b) mb tip on the index fingertip. (c) The hrough magnetic tracking by adding a he fingernails. splays permit personal and pri- put methods may not offer the voice input is commonly used because it is expressive and ef- put can be problematic in loud ssues arise with its use in pub- nput)[11]. Gesture input suffers ns because input actions are eas- nt research proposes subtle inter- re based on implicit movements cially acceptable. For example, input through unobservable mus- [12] detects subtle foot motions. rics [6] have been developed to inputs. Although these methods allow for privacy and social ac- uffer from limited input space. Pad, a nail-mounted device that o a touchpad, allowing private, FingerPad Wearable Private Input [Chan et al. UIST 2013] GaussSense WonderLens: Optical Lenses and Mirrors for Tangible Interactions on Printed Paper Rong-Hao Liang, Chao Shen, Yu-Chien Chan, Guan-Ting Chou, Liwei Chan, De-Nian Yang, Mike Y. Chen, and Bing-Yu Chen National Taiwan University and Academia Sinica WonderLens WonderLens TUI on Printed Paper [Liang et al. CHI 2015] FingerPad This project, WonderLens, shows another applications of GaussSense, that is enabling tangible interactions on Printed Paper. For makers and researchers, who want to try out the GaussSense technology, we are happy to announce that.
  62. 62. Attachable Stylus Sensing Using Magnetic Sensor Grid GaussSense GaussBits GaussBricks GaussBits Magnetic Tangible Bits [Liang et al. CHI 2013] GaussBricks Magnetic Building Blocks [Liang et al. CHI 2014] GaussSense Analog Hall-Sensor Grid [Liang et al. UIST 2012] GaussStones Shielded Magnetic Tangibles [Liang et al. UIST 2014] GaussStones eng† Chao-Huai Su† Chen‡ @cmlab.csie.ntu.edu.tw d c c ouchpad function through pinch ges- words to the private glass display (b) mb tip on the index fingertip. (c) The hrough magnetic tracking by adding a he fingernails. splays permit personal and pri- put methods may not offer the voice input is commonly used because it is expressive and ef- put can be problematic in loud ssues arise with its use in pub- nput)[11]. Gesture input suffers ns because input actions are eas- nt research proposes subtle inter- re based on implicit movements cially acceptable. For example, input through unobservable mus- [12] detects subtle foot motions. rics [6] have been developed to inputs. Although these methods allow for privacy and social ac- uffer from limited input space. Pad, a nail-mounted device that o a touchpad, allowing private, WonderLens WonderLens TUI on Printed Paper [Liang et al. CHI 2015] FingerPad Wearable Private Input [Chan et al. UIST 2013] FingerPad GaussSense WonderLens: Optical Lenses and Mirrors for Tangible Interactions on Printed Paper Rong-Hao Liang, Chao Shen, Yu-Chien Chan, Guan-Ting Chou, Liwei Chan, De-Nian Yang, Mike Y. Chen, and Bing-Yu Chen National Taiwan University and Academia Sinica http://gausstoys.com $19 GaussSense is coming soon! subscribe this information on: The 19-dollar GaussSense is coming soon. We have worked very hard on it, and really excited to see this finally happens. If you are interested in, please subscribe us on GaussToys.com. So we can keep you in the loop. Thanks for your attention, and I’m happy to take all of your questions.

×