• Save
Multi-Touch Interaction Overview
Upcoming SlideShare
Loading in...5
×
 

Multi-Touch Interaction Overview

on

  • 41,935 views

This presentation provides an overview of multi-touch hardware, products, applications and market examples as well as samples of projects of TNO. More information on http://www.tno.nl/nui

This presentation provides an overview of multi-touch hardware, products, applications and market examples as well as samples of projects of TNO. More information on http://www.tno.nl/nui

Statistics

Views

Total Views
41,935
Views on SlideShare
37,511
Embed Views
4,424

Actions

Likes
224
Downloads
0
Comments
36

69 Embeds 4,424

http://www.touchscreentables.com 1553
http://moodle.flshase.uni.lu 630
http://www.touchuserinterface.com 546
http://conociendoelmundodigital.wikispaces.com 339
http://www.slideshare.net 221
http://www.gadgetsenonderwijs.nl 193
http://abbuzz.blogspot.com 161
http://cityray.blogspot.com 142
http://cityray.blogspot.tw 103
http://www.techgig.com 82
http://narrativisions.com 62
http://ctoutcom.blogspirit.com 46
http://moita.blogs.ua.sapo.pt 36
https://tasks.crowdflower.com 35
http://abbuzz.blogspot.it 31
http://www.levidepoches.blogs.com 30
http://pjarte.fr 21
http://static.flipora.com 19
http://mindpractice.tumblr.com 15
http://translate.googleusercontent.com 11
http://user-experience-specialist.blogspot.com 11
http://dmt-digitalmedia.blogspot.com 9
http://www.levidepoches.fr 9
http://static.slidesharecdn.com 9
http://search.flipora.com 8
http://archives.moodle.flshase.uni.lux 8
http://cityray.blogspot.hk 7
http://www.linkedin.com 7
https://jujo00obo2o234ungd3t8qjfcjrs3o6k-a-sites-opensocial.googleusercontent.com 6
http://goodresults.flipora.com 6
http://archives.moodle.flshase.uni.lu 6
https://www.facebook.com 5
http://kodblog.com 4
http://narrativisions.posterous.com 4
http://user-experience-specialist.blogspot.de 3
http://www.goodresults.flipora.com 3
http://www.pearltrees.com 2
http://www.cityray.blogspot.com 2
http://abbuzz.blogspot.in 2
http://abbuzz.blogspot.com.au 2
http://edcecocity.blogspot.com 2
http://hosup.textcube.com 2
https://webcourses.niu.edu 2
http://www.slashdocs.com 2
http://10.150.200.57 2
http://www.kodblog.com 2
http://user-experience-specialist.blogspot.co.uk 1
http://user-experience-specialist.blogspot.pt 1
http://edcecocity.blogspot.in 1
http://cityray.blogspot.jp 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

110 of 36 Post a comment

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • please send me this presentation.
    archanasathya121@gmail.com
    Are you sure you want to
    Your message goes here
    Processing…
  • can u plz send me ppt on multi touch interaction immediately on my mail saranya.k_03@yahoo.in
    Are you sure you want to
    Your message goes here
    Processing…
  • can u plz send me a seminar report and ppt on multi touch technology immediately on my mail
    suryawanshik2@gmail.com
    Are you sure you want to
    Your message goes here
    Processing…
  • plz send me a seminar report and ppt on multi touch technology immediately on my mail..
    Are you sure you want to
    Your message goes here
    Processing…
  • plz send me a seminar report and ppt on multi touch technology immediately..
    Are you sure you want to
    Your message goes here
    Processing…

110 of 36

Post Comment
Edit your comment

Multi-Touch Interaction Overview Multi-Touch Interaction Overview Presentation Transcript

  • Natural User Interfaces Natural User Hardware Interfaces Markets and Markets and Hardware and software applications Software applications History and future Timeline Multi-touch interaction TNO work TNO Work Arnoud de Jong Arnout de Vries TNO Information and Communication Technologies Multi- Multi- Summary touch touch in and products the Conclusions 1 movies
  • History of Multi-touch interaction Pierre Wellner paper Foundation on multi touch and iGesture Pad and NUI Group First multi pinching motions TouchStream keyboard hand display by Fingerworks Bell Labs Palm Sony labs Microsoft Pre First MT SmartSkin surface PlayAnywhere display Toronto Mixed Reality table of Touch and object university Microsoft Research detection Hitachi Q2 2009 Smartboard 2002 1982 2005 1991 1999 2006 2001 1984 FX77 Duo 1980 2009 Jun 2007 2002 Q3 2008 Q1 2009 2003 1992 1982 1985 2006 Q1 2006 University of 2001 Movie: 3D multi-touch Toronto Tron Microsoft sensitive tablet Simon - first Multi-finger Windows 7 smartphone Multi-hand Multi-user Lucid With Mitsubishi Touch interaction iPhone touchscreen MERL R&D Lab release DiamondTouch HP-150 world’s Microsoft Surface 1983 first Jeff Han SMART touchscreen PC Movie: @TED table Minority report 2
  • Hardware technology • Multi-point detection • Multi-object detection • Setup 3
  • 4
  • 5
  • 6
  • 7
  • 8
  • 9
  • 10
  • 11
  • Setups 12
  • Products • Table tops • Tablet PC’s • Whiteboards • Large surfaces (walls, floors, tables) • Desktop PC’s • Netbooks / UMPC’s • Mobile phones • TV’s 13
  • Fraunhofer IGD multi-touch table • The table is running on Linux and uses the InstantReality framework for displaying and interacting. • The finger tracking is done by the Fraunhofer software VisionLib which is not available for pulic right now. • Table size is 150x90 cm 14
  • Mitsubishi: DiamondTouch • Front-projection on typically 32" or 42" diagonal • Multiple touches support by a single user • Distinguishes between simultaneous inputs from multiple users. • DiamondTouch SDK or DTFlash toolkit. 15
  • Gesturetek 16
  • Horizon
  • SMART table
  • Microsoft: Surface • www.surface.com • Object on table are identified by ‘domino tags’ • http://worldofitgeeks.blogspot.com/2007/10/domino-theories-for-microsofts-surface.html 19
  • HP TouchSmart 20
  • HP Touchsmart TX2 tablet PC
  • Dell Latitude XT 22
  • Hitachi Starboard FX77 Duo 23
  • Multi touch desktop LCD’s 24
  • TV 25
  • iPhone en iPod Touch • The Apple iPhone has a multitouch display for resizing, zooming and rotating purposes. • Multi touch sensative screen • Motion sensor • Types of multi touch gestures: • Zoom, rotate 26
  • G1 Google Android phone 27
  • Palm Pre • WebOS. It’s a web-centric multi- touch platform that makes us of CSS, XHTML and Javacript making it accessible and compatible with a wide range of applications. 28
  • And more mobile phone devices Meizu M8 HTC Touch Synaptics Onyx Nokia N97 29
  • Apple: MacBook Air • The Apple MacBook Air has multitouch touchpad for resizing, zooming and rotating purposes. 30
  • Eee PC 31
  • MultiTouch Cell • The MultiTouch LCDs are controlled by the MultiTouch CornerStone software. Cornerstone handles the core multitouch functionality, tracking users hands as they move on the screen. 32
  • Other vendors Accenture Panasonic Hitachi Philips Entertaible Perceptive Pixel Sony Smartskin HP Misto Sharp optical LCD Touchable
  • Market share of products with multi-touch • Multi-touch ability (or NUI) is competitor’s edge that is used in marketing • Market share is quickly rising • 10 mio iPhones sold in first 6 months • Table top sales booming since end 2008 Multi-touch is currently an “infant” technology • Different multi-touch imlementations (Apple, Microsoft, HP, etc.) • All are limited to a handful of gestures • Still a lot of traditional GUI interaction 34
  • Future product list • Input devices • Apple Mighty Mouse • Wacom tablets • Handheld game consoles • PSP2 • E-book readers • Desktops • Larger and flat tabletops • TV screens 35
  • Natural User Interfaces (NUI’s) Introduction • Touch screen interaction • Multi-touch screen interaction The vision • Evolution of computer user interfaces Reality • Natural User Interfaces (NUI) • Tangible interfaces • Multi modal interfaces • NUI design 36
  • Touch screen interaction Touch interaction • 1960: Touch • 1982: Multi touch • 2006: Lucid touch • Multi modal interfaces, controlled by • Touch • Gestures • Speech • Objects 39
  • Single Touch 40
  • Touch screen interfaces (since 1960’s) • Interaction via • Stylus, light pen, finger, hand,  Finger stress  Fingernail interaction  Fingerprints  “Gorilla arm” • No intelligent interaction • Gestures • Touch force sensing • Object sensing (finger vs hand) 41
  • Multi-touch & Natural User Interfaces 42
  • Multi-touch interaction • In the realworld we do lots of actions with two hands or more than one finger. e.g. driving a car • Operating a mouse are serial, discrete events and single point input • Multi point interaction allows more parallel interaction • Reduce task complexity of single input techniques • Increases parallesm in multi-finger interaction reduces time 43
  • Tangible interfaces 44
  • Interact with real objects 45
  • Haptics 46
  • Haptics 47
  • Haptics today 48
  • Lucid Touch 49
  • But lucid touch interaction is very complex 50
  • Gesture interaction 51
  • Gesture interaction today 52
  • Multimodal interaction 53
  • Multimodal interaction is still the future… 54
  • Paradigm shift Command Line Interfaces (CLI) 55
  • Paradigm shift Graphical User Interfaces (GUI) 56
  • Paradigm shift Natural User Interfaces (NUI) 57
  • Paradigm shift • Static • Responsive • Evocative • Disconnected • Indirect • Unmediated • High-low • Double medium • Fast few • Directed • Exploratory • Contextual • Recall • Recognition • Intuition • Abstract • Indirect • Direct • Textual • Graphical • Physical 58 Source: Dennis Wixon
  • Overview of Human Computer Interfaces Microsoft Microsoft Windows NT Windows Vista Alto PC Microsoft Microsoft Xerox Parc Windows 1.0 Windows 95 Microsoft MS-DOS Windows 98 Microsoft Microsoft Microsoft Q2 2009 Windows 3.0 Windows XP Windows 7 1993 1995 2001 1982 1985 1998 2007 1990 1973 1970 2010 Jun 2007 1987 1997 Microsoft 2001 1984 1992 2005 2009 Windows Snow Tablet PC Leopard? Apple Macintosh edition IBM OS/2 2.0 Apple Mac OS 8 Apple Mac OS X 10.5 Apple Mac OS X 10.0 Apple Mac Color Note: Linux excluded
  • Interaction styles and elements 1. Punch-card input (50’s-60’s) 2. Teletype writer connected to mainframe (60’s -80’s) EVOLUTION 3. CLI (80’s) 4. GUI (90’s-00’s) • Classic WIMP (Window, Icon, Menu, Pointer) • Desktop metaphor • Post-WIMP interfaces • Computer games • Virtual reality systems • Physical controls (e.g. iPod wheel) • Multi-touch interaction 3. NUI • STaG (Speech, Touch and Gesture) 4. XUI • Synthesis, anticipatory, constant fluid interaction 60
  • The vision of Natural User Experience (NUE) is to deliver intuitive, seamless experiences that unfold through natural human input.” • “Surfaces that respond to us” • “The content is the interface” 61
  • Surface: design for emotion • Natural & Intuitive • Unique & Magical • Social & Together • Aware & Responsive • Premium & Authentic 62
  • Surface guidelines 1. Seamless 2. Social 3. Spatial 4. Super realism 5. Contextual environments 6. Scaffolding 7. Performance aesthetics 8. Direct manipulation
  • User Interface Design for NUI’s Difference in designing Tabletop interfaces? • No fixed top-bottom orientation; • Multiple “mouse cursors”; • Multiple simultaneous users; • Multiple user contexts; • Multiple simultaneous actions, such as ‘drag & drop’; • No more tradional data input means. 64
  • Multi-touch in the movies • 1982: Tron • 2002: Minority report • 2005: The Island • 2008: Quantum of Solace (James Bond) • 2008: CSI: Miami 65
  • 1982: Tron 66
  • 2002: Minority Report 67
  • 2005: The Island 68
  • 2008: Quantum of Solace 69
  • 2008: CSI: Miami 70
  • Markets and applications • Entertainment: (serious) gaming and fun • Public safety and crisis management • Retail • Public Places • Healthcare • Education • Financial • Office • Industrial use 71
  • Healthcare • Photo tool for dementia patients • Tool for patient-doctor relationships 72
  • Entertainment • Games (serious/fun) 73
  • Public places • Information centre New York • Why? • Intuitive, interactive and focussed on discovering New York • Target group: Tourists • Device • Gesturetek illuminate tables • Application: • Maps, POI, multimedia content 74 Information centre New York
  • Museums 75
  • TV presentations CNN – presidential elections
  • Retail 78
  • Horeca • Restaurants, hotels • Viewing menu • Entertainment (games, gambling) • Tourist information Surfaceware: detect drink level 79
  • iBar: object recognizing bar • Bar detects all objects on the surface • It detects the shape, it doesn’t identify the object. • future releases will also be able to identify objects • http://www.i-bar.ch/info/ 80
  • Education 81
  • Control Rooms AT&T Telecomunication control room OODA loop 1. Static risk map 2. real-time or simulated flooding 3. Real-time information feeds 82
  • Public safety • Manages planning, tracking and report security matters surrounding an event. The Super Bowl E•SPONDER solution 83
  • Financial services Use in a bank lobby for customers to look at account features and sit down with a banker to discuss their financial needs. 84
  • i-phone multi-touch applications • Photo’s: create a collage • Graffiti • Scratch Mixmeister • Games Foto collages maken Muziek scratchen en mixen puistjes uitdrukken 85
  • Industrial use • Insight in processes • Observe, control and decide • Collaborative design • Fraunhofer: • Virtual factory on a tabletop
  • Collaborative work in the office 87
  • Office Digidesk
  • TNO Work • Multi -touch in Control Rooms • Multi –touch in Education • Multi-touch : complex gesture support • MiReCol: collaboration and co-design • Slide overview + video Maurice 89
  • Control Rooms
  • Experiment area 91
  • Visualisation of information (1.4)(2.4)(21.6)(4.2)(7.6)(2.6)(9.8)(3.0) Quick insight Large data amounts Unstructured Time critical Diverse sources
  • Data sources information exchange Virtual teams Control room teams Control room decision centre 93
  • Control Rooms AT&T Telecomunication control room OODA loop 1. Static risk map 2. real-time or simulated flooding 3. Real-time information feeds 94
  • Crisis management scenario Satellite or street map Multi-user interface Ten Boer Risk objects Real-time communication log Annotate Analyse Anticipate Change view, filter, log
  • MiReCol: Mixed Reality Collaboration
  • Mixed reality collaboration Augmented collaboration Shared Augmented virtuality understanding collaboration Measuring and facilitating collaboration
  • Collaboration • Co-creation, co-design • Decision making • CSCW support in general 100
  • Teliris telepresence
  • Education 102
  • Serious gaming & multi touch interactie in het onderwijs Research goal: Does the combination of serious gaming and multi touch interaction add value to elementary education. Serious games Target group: School children aged 8 until 12 with a social disorder (PDD-NOS). Multi touch Results: a serious game was developed to support and stimulate several collaboration forms in calculation tasks. Through multi- touch and multi user interaction these children could cooperate using one Education system. Partners: Bladergroenschool (elementary school for special education) and RENN4.
  • Supporting gesture interaction
  • Supporting gesture interaction with gesture previews Gestures • In general: non-verbal communication. • Specific: a set of measured movements over time. • Examples: letter recognition, unlocking your iPhone. Advanced gestures • Multi-touch gestures: zooming an image. • Collaborative gestures: clearing a drawing board. Problem statement • Most gestures have no intrinsic meaning. • Users have to know gestures in advance. • A steep learning curve especially for complex systems using advanced gestures.
  • 106
  • Solution: gesture previews • Overlay with visual feedback. • Shows what gestures can be performed. • Shows where others can join in. • Indicates the meaning of gestures. • Previews do not limit expert users. • Increase detection rate, decrease learning curve.
  • Proof of Concept • Bejeweled clone, uses gestures to identify users. With some twists. • Gestures are stored in XML. <?xml version="1.0" encoding="utf-8"?> <root> <segmentDeclaration name="line" uniformScaling="false" <point x="0" y="0" /> <point x="0" y="0,1" /> </segmentDeclaration>
  • Wrap up Knowledge • Started at TNO just a few years ago, experimental. Now this topic is booming, backed up by large vendors and coming to the market. Hardware/software • More vendors enabling MT for their devices • Current software / applications are dedicated solutions • Wide adoption needs more hardware and applications Interaction • Multi touch is part of the NUI paradigm shift in Human Machine Interaction Markets / applications • Wow-technology, but the real added value and useful applications still have to prove themselves TNO • Added value in collaboration environments (crisismanagement, education and many more projects) 109
  • Arnout de Vries Senior scientist Natural User Interfaces Arnout.devries@tno.nl http://www.tno.nl/nui