Multi-Touch System

Ambient Visualisation with Devices 2008 Research Presentation Kerry Jia Yi Lin DESC 9179 Multi – Touch System Clam/Affective/Aesthetic Computing - HCI

Multi – Touch System

Research papers include: Multi – Touch System Jefferson Y. Han, “ Low-cost Multi-Touch sensing through frustrated total internal reflection ”, ACM Press New York, NY, USA, 2005 Shahram Izadi, Steve Hodges, Alex Butler, Alban Rrustemi, Bill Buxton. “ ThinSight: Integrated Optical Multi-touch Sensing through Thin Form-factor Displays ”, Symposium on User Interface Software and Technology, Proceedings of the 20th annual ACM symposium on User interface software and technology, 2007, Pages: 259 - 268 Chia Shen; Ryall, K.; Forlines, C.; Esenther, A.; Vernier, F.D.; Everitt, K.; Wu, M.; Wigdor, D.; Morris, M.R.; Hancock, M.; Tse, E. “ Informing the Design of Direct-Touch Tabletops ”, Computer Graphics and Applications, IEEE Volume: 26, 2006. Page(s) 36 – 46 Jeff Han FTIR Tabletops Design ThinSight

Jeff Han FTIR Multi – Touch System Motivation of the research projects: Multi-Touch, Multi-finger, find the medium in Frustrated Total Internal Reflection, reduce the cost Tabletops Design ThinSight Thin form factor display, Multi-Touch, Multi-finger, infrared ray, new hardware allow sensing without degradation of display capability Improve the design of tabletop user interfaces, interaction techniques and usage scenarios.

Jeff Han FTIR Multi – Touch System Contents: Video: Jeff Han on TED Talks, Monterey California, Feb 2006

Jeff Han FTIR Multi – Touch System Contents: Video Camera Projector Acrylic Sheet (1/4” or thicker)‏ 􀁹 Infra-Red LEDs 􀁹 IR-Detecting Camera 􀁹 Projector 􀁹 Diffuser (projection surface overlay)‏ 􀁹 Frame with Light Baffles 􀁹 Computer

Jeff Han FTIR Multi – Touch System Contents: Total Internal Reflection “ Total internal reflection is an optical phenomenon that occurs when a ray of light strikes a medium boundary at an angle larger than the critical angle with respect to the normal to the surface. If the refractive index is lower on the other side of the boundary no light can pass through, so effectively all of the light is reflected. The critical angle is the angle of incidence above which the total internal reflection occurs. ” ----From Wikipedia, the free encyclopedia Medium = Acrylic pane Video Camera Projector

Jeff Han FTIR Multi – Touch System Research Contributions & Significance: Medium can be seen as the most important thing in the TIR. Jeff Han found the medium which is the thin sheet of Acrylic ( cheap material) . Jeff Han's problem : Dry skin generates a weaker optional signal, User need to press harder against the sensor. Moisten Skin. Or NUI (Natural User Interface) found a way to solve this problem, which is adding a compliant surface (e.g. Silicon ) between the acrylic and the diffuser Simple – construction Inexpensive – the prototype is at $542 (not including the $2,300 projector, computer, or miscellaneous tools and “workbench” materials). Scalable Size – Hand-held to Full wall display

Multi – Touch System Contents: ThinSight (Microsoft Research Cambridge)‏ Video : http://www.youtube.com/watch?v=v_MAtjOlRqE

Multi – Touch System ThinSight Research Contributions & Significance: A new technique for optically sensing multiple objects (fingers) through thin form factor display. Supports interactions as a distance using IR pointing devices, and it enables IR-based communication through the display with other electronic devices (e.g. TV controller). Allows rich “camera-like” data ( outlines of IR-reflective objects ) to be captured by the display & is processed using computer vision techniques. Allow new types of human computer interface that exploit zero-force multi-touch & tangible interaction. Be more practical & deployable in real-world settings.

Multi – Touch System Relation, Advantage & Disadvantage Jeff Han FTIR ThinSight - Infrared ray (IR) disturbing, is unstable when putting it under the Sun or Environment with IR. Advantage Disadvantage Relation - Infrared ray (IR) used - W eaker optional signal. - Not provide “hover” information - IR can identify the shape of the objects - Work with LCD, laptop - compact, thin from-factor making it easier to deploy - “See through the screen” - high-resolution multi-touch sensing on rear-projected interactive surface - Prototype with fairly low resolution sensing & covers only a portion of the display

Multi – Touch System Tabletops Design Research the potential benefits of the Direct-Touch Tabletops from the usability challenges (Occlusion and reach, Gestural interaction, legacy application support, Group interaction techniques), usage issues, experiences and reflection (Orientation side effects,Input, precision, Nonspeech audio feedback) . Exploit the advantage and afforcement of them. Evaluate a variety of tabletop user interfaces, interaction techniques and usage scenarios. Projects from Mitsubishi Electric Research Labs : UbiTable (UbiComp 2003), LiamonSpin (CHI 2004), DT-Lens (ACM UIST 2005), IEEE Table TOP 2006, DiamondSpace (ACM UIST 2006). Contents:

Multi – Touch System Tabletops Design Research Contributions & Significance:

- Direct-Touch Tabletop is a Promising technology ‏

- Positive influence on working styles and group dynamics, compares to the Wall interactive display

- Serves external cognitive medium

- Research areas: guidelines, UI, interaction techniques, usability

- Suggest to set forth interaction techniques , user experiences, and design consideration to expand and explore the advantages of interactive tabletop system.

- Can be used to evaluate the existing Direct-Touch Tabletop

Multi – Touch System Multi – Touch technologies has a long history. They took 30 years to improve, today has got into people's life. Microsoft Surface, iPhone, Perceptivepixel, Toshiba Display... They are the innovation of the Human Computer Interaction. Conclusion Thank you!