Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Natural User Interfaces
                                                                           Natural User
          ...
History of Multi-touch interaction



                                                        Pierre Wellner paper
       ...
Hardware technology

    • Multi-point detection
    • Multi-object detection
    • Setup




3
4
5
6
7
8
9
10
11
Setups




12
Products

     •   Table tops
     •   Tablet PC’s
     •   Whiteboards
     •   Large surfaces (walls, floors, tables)
  ...
Fraunhofer IGD multi-touch table

                     • The table is running on Linux
                       and uses the...
Mitsubishi: DiamondTouch

                            • Front-projection on
                              typically 32" or...
Gesturetek




16
Horizon
SMART table
Microsoft: Surface




                         • www.surface.com
           • Object on table are identified by ‘domino t...
HP TouchSmart




20
HP Touchsmart TX2 tablet PC
Dell Latitude XT




22
Hitachi Starboard FX77 Duo




23
Multi touch desktop LCD’s




24
TV




25
iPhone en iPod Touch

               • The Apple iPhone has a
                 multitouch display for resizing,
          ...
G1 Google Android phone




27
Palm Pre

         • WebOS. It’s a web-centric multi-
           touch platform that makes us of
           CSS, XHTML and...
And more mobile phone devices
            Meizu M8   HTC Touch     Synaptics Onyx




     Nokia N97




29
Apple: MacBook Air

                          • The Apple MacBook Air
                            has multitouch touchpad
...
Eee PC




31
MultiTouch Cell




     • The MultiTouch LCDs are controlled by the MultiTouch CornerStone software.
       Cornerstone h...
Other vendors

 Accenture             Panasonic           Hitachi




 Philips Entertaible   Perceptive Pixel     Sony Sma...
Market share of products with multi-touch

     • Multi-touch ability (or NUI) is competitor’s edge that is used in
      ...
Future product list

     • Input devices
         • Apple Mighty Mouse
         • Wacom tablets
     • Handheld game cons...
Natural User Interfaces (NUI’s)

     Introduction
     • Touch screen interaction
     • Multi-touch screen interaction

...
Touch screen interaction

     Touch interaction
     • 1960: Touch
     • 1982: Multi touch
     • 2006: Lucid touch

   ...
Single Touch


40
Touch screen interfaces (since 1960’s)

     • Interaction via
         • Stylus, light pen, finger, hand,


      Finger...
Multi-touch &
     Natural User Interfaces




42
Multi-touch interaction

     • In the realworld we do lots of actions with two hands
       or more than one finger. e.g....
Tangible interfaces




44
Interact with real objects
45
Haptics




46
Haptics




47
Haptics today




48
Lucid Touch




49
But lucid touch interaction is very complex




50
Gesture interaction




51
Gesture interaction today




52
Multimodal interaction




53
Multimodal interaction is still the future…




54
Paradigm shift




            Command Line Interfaces (CLI)
55
Paradigm shift




            Graphical User Interfaces (GUI)
56
Paradigm shift




              Natural User Interfaces (NUI)
57
Paradigm shift

     •   Static           •   Responsive      •   Evocative
     •   Disconnected     •   Indirect        ...
Overview of Human Computer Interfaces



                                                                             Micr...
Interaction styles and elements

     1. Punch-card input (50’s-60’s)

     2. Teletype writer connected to mainframe (60’...
The vision of Natural User Experience
                        (NUE) is to deliver intuitive, seamless
                    ...
Surface: design for emotion

     • Natural & Intuitive

     • Unique & Magical

     • Social & Together

     • Aware &...
Surface guidelines

1. Seamless

2. Social

3. Spatial

4. Super realism

5. Contextual environments

6. Scaffolding

7. P...
User Interface Design for NUI’s

     Difference in designing Tabletop interfaces?

     •   No fixed top-bottom orientati...
Multi-touch in the movies

     •   1982: Tron
     •   2002: Minority report
     •   2005: The Island
     •   2008: Qua...
1982: Tron




66
2002: Minority Report




67
2005: The Island




68
2008: Quantum of Solace




69
2008: CSI: Miami




70
Markets and applications

     • Entertainment: (serious) gaming and fun

     • Public safety and crisis management

    ...
Healthcare

     • Photo tool for dementia patients

     • Tool for patient-doctor relationships




72
Entertainment

     • Games (serious/fun)




73
Public places

                                      • Information centre New York

                                      ...
Museums




75
TV presentations




         CNN – presidential elections
Retail




78
Horeca

                                   • Restaurants, hotels

                                   • Viewing menu
      ...
iBar: object recognizing bar



     • Bar detects all objects on the surface
     • It detects the shape, it doesn’t iden...
Education




81
Control Rooms




     AT&T Telecomunication control room


                                                              ...
Public safety

     • Manages planning, tracking and report
       security matters surrounding an event.



             ...
Financial services



                          Use in a bank lobby for
                          customers to look at
   ...
i-phone multi-touch applications

     •   Photo’s: create a collage
     •   Graffiti
     •   Scratch Mixmeister
     • ...
Industrial use

• Insight in processes
    • Observe, control and decide
• Collaborative design




• Fraunhofer:
   • Vir...
Collaborative work in the office




87
Office




         Digidesk
TNO Work

     • Multi -touch in Control Rooms

     • Multi –touch in Education

     • Multi-touch : complex gesture sup...
Control Rooms
Experiment
area




91
Visualisation of information




(1.4)(2.4)(21.6)(4.2)(7.6)(2.6)(9.8)(3.0)


                                            Q...
Data sources


              information exchange



                                                       Virtual teams
...
Control Rooms




     AT&T Telecomunication control room


                                                              ...
Crisis management scenario   Satellite or street map


                             Multi-user interface




             ...
MiReCol: Mixed Reality Collaboration
Mixed reality collaboration
                                              Augmented collaboration




                    ...
Collaboration

      • Co-creation, co-design
      • Decision making
      • CSCW support in general




100
Teliris telepresence
Education




102
Serious gaming & multi touch interactie in het onderwijs

Research goal: Does the
combination of serious gaming and
multi ...
Supporting gesture interaction
Supporting gesture interaction with gesture previews

Gestures
• In general: non-verbal communication.
• Specific: a set o...
106
Solution: gesture previews

•   Overlay with visual feedback.
•   Shows what gestures can be performed.
•   Shows where ot...
Proof of Concept

• Bejeweled clone, uses gestures to identify users. With some twists.
• Gestures are stored in XML.
    ...
Wrap up

Knowledge
• Started at TNO just a few years ago, experimental. Now this topic is booming,
  backed up by large ve...
Arnout de Vries

Senior scientist
Natural User Interfaces

Arnout.devries@tno.nl
http://www.tno.nl/nui
Multi-Touch Interaction Overview
Multi-Touch Interaction Overview
Multi-Touch Interaction Overview
Multi-Touch Interaction Overview
Multi-Touch Interaction Overview
Upcoming SlideShare
Loading in …5
×

of

Multi-Touch Interaction Overview Slide 1 Multi-Touch Interaction Overview Slide 2 Multi-Touch Interaction Overview Slide 3 Multi-Touch Interaction Overview Slide 4 Multi-Touch Interaction Overview Slide 5 Multi-Touch Interaction Overview Slide 6 Multi-Touch Interaction Overview Slide 7 Multi-Touch Interaction Overview Slide 8 Multi-Touch Interaction Overview Slide 9 Multi-Touch Interaction Overview Slide 10 Multi-Touch Interaction Overview Slide 11 Multi-Touch Interaction Overview Slide 12 Multi-Touch Interaction Overview Slide 13 Multi-Touch Interaction Overview Slide 14 Multi-Touch Interaction Overview Slide 15 Multi-Touch Interaction Overview Slide 16 Multi-Touch Interaction Overview Slide 17 Multi-Touch Interaction Overview Slide 18 Multi-Touch Interaction Overview Slide 19 Multi-Touch Interaction Overview Slide 20 Multi-Touch Interaction Overview Slide 21 Multi-Touch Interaction Overview Slide 22 Multi-Touch Interaction Overview Slide 23 Multi-Touch Interaction Overview Slide 24 Multi-Touch Interaction Overview Slide 25 Multi-Touch Interaction Overview Slide 26 Multi-Touch Interaction Overview Slide 27 Multi-Touch Interaction Overview Slide 28 Multi-Touch Interaction Overview Slide 29 Multi-Touch Interaction Overview Slide 30 Multi-Touch Interaction Overview Slide 31 Multi-Touch Interaction Overview Slide 32 Multi-Touch Interaction Overview Slide 33 Multi-Touch Interaction Overview Slide 34 Multi-Touch Interaction Overview Slide 35 Multi-Touch Interaction Overview Slide 36 Multi-Touch Interaction Overview Slide 37 Multi-Touch Interaction Overview Slide 38 Multi-Touch Interaction Overview Slide 39 Multi-Touch Interaction Overview Slide 40 Multi-Touch Interaction Overview Slide 41 Multi-Touch Interaction Overview Slide 42 Multi-Touch Interaction Overview Slide 43 Multi-Touch Interaction Overview Slide 44 Multi-Touch Interaction Overview Slide 45 Multi-Touch Interaction Overview Slide 46 Multi-Touch Interaction Overview Slide 47 Multi-Touch Interaction Overview Slide 48 Multi-Touch Interaction Overview Slide 49 Multi-Touch Interaction Overview Slide 50 Multi-Touch Interaction Overview Slide 51 Multi-Touch Interaction Overview Slide 52 Multi-Touch Interaction Overview Slide 53 Multi-Touch Interaction Overview Slide 54 Multi-Touch Interaction Overview Slide 55 Multi-Touch Interaction Overview Slide 56 Multi-Touch Interaction Overview Slide 57 Multi-Touch Interaction Overview Slide 58 Multi-Touch Interaction Overview Slide 59 Multi-Touch Interaction Overview Slide 60 Multi-Touch Interaction Overview Slide 61 Multi-Touch Interaction Overview Slide 62 Multi-Touch Interaction Overview Slide 63 Multi-Touch Interaction Overview Slide 64 Multi-Touch Interaction Overview Slide 65 Multi-Touch Interaction Overview Slide 66 Multi-Touch Interaction Overview Slide 67 Multi-Touch Interaction Overview Slide 68 Multi-Touch Interaction Overview Slide 69 Multi-Touch Interaction Overview Slide 70 Multi-Touch Interaction Overview Slide 71 Multi-Touch Interaction Overview Slide 72 Multi-Touch Interaction Overview Slide 73 Multi-Touch Interaction Overview Slide 74 Multi-Touch Interaction Overview Slide 75 Multi-Touch Interaction Overview Slide 76 Multi-Touch Interaction Overview Slide 77 Multi-Touch Interaction Overview Slide 78 Multi-Touch Interaction Overview Slide 79 Multi-Touch Interaction Overview Slide 80 Multi-Touch Interaction Overview Slide 81 Multi-Touch Interaction Overview Slide 82 Multi-Touch Interaction Overview Slide 83 Multi-Touch Interaction Overview Slide 84 Multi-Touch Interaction Overview Slide 85 Multi-Touch Interaction Overview Slide 86 Multi-Touch Interaction Overview Slide 87 Multi-Touch Interaction Overview Slide 88 Multi-Touch Interaction Overview Slide 89 Multi-Touch Interaction Overview Slide 90 Multi-Touch Interaction Overview Slide 91 Multi-Touch Interaction Overview Slide 92 Multi-Touch Interaction Overview Slide 93 Multi-Touch Interaction Overview Slide 94 Multi-Touch Interaction Overview Slide 95 Multi-Touch Interaction Overview Slide 96 Multi-Touch Interaction Overview Slide 97 Multi-Touch Interaction Overview Slide 98 Multi-Touch Interaction Overview Slide 99 Multi-Touch Interaction Overview Slide 100 Multi-Touch Interaction Overview Slide 101 Multi-Touch Interaction Overview Slide 102 Multi-Touch Interaction Overview Slide 103 Multi-Touch Interaction Overview Slide 104 Multi-Touch Interaction Overview Slide 105 Multi-Touch Interaction Overview Slide 106 Multi-Touch Interaction Overview Slide 107 Multi-Touch Interaction Overview Slide 108 Multi-Touch Interaction Overview Slide 109 Multi-Touch Interaction Overview Slide 110
Upcoming SlideShare
Multitouch Interaction
Next

257 Likes

Share

Multi-Touch Interaction Overview

This presentation provides an overview of multi-touch hardware, products, applications and market examples as well as samples of projects of TNO. More information on http://www.tno.nl/nui

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Multi-Touch Interaction Overview

  1. 1. Natural User Interfaces Natural User Hardware Interfaces Markets and Markets and Hardware and software applications Software applications History and future Timeline Multi-touch interaction TNO work TNO Work Arnoud de Jong Arnout de Vries TNO Information and Communication Technologies Multi- Multi- Summary touch touch in and products the Conclusions 1 movies
  2. 2. History of Multi-touch interaction Pierre Wellner paper Foundation on multi touch and iGesture Pad and NUI Group First multi pinching motions TouchStream keyboard hand display by Fingerworks Bell Labs Palm Sony labs Microsoft Pre First MT SmartSkin surface PlayAnywhere display Toronto Mixed Reality table of Touch and object university Microsoft Research detection Hitachi Q2 2009 Smartboard 2002 1982 2005 1991 1999 2006 2001 1984 FX77 Duo 1980 2009 Jun 2007 2002 Q3 2008 Q1 2009 2003 1992 1982 1985 2006 Q1 2006 University of 2001 Movie: 3D multi-touch Toronto Tron Microsoft sensitive tablet Simon - first Multi-finger Windows 7 smartphone Multi-hand Multi-user Lucid With Mitsubishi Touch interaction iPhone touchscreen MERL R&D Lab release DiamondTouch HP-150 world’s Microsoft Surface 1983 first Jeff Han SMART touchscreen PC Movie: @TED table Minority report 2
  3. 3. Hardware technology • Multi-point detection • Multi-object detection • Setup 3
  4. 4. 4
  5. 5. 5
  6. 6. 6
  7. 7. 7
  8. 8. 8
  9. 9. 9
  10. 10. 10
  11. 11. 11
  12. 12. Setups 12
  13. 13. Products • Table tops • Tablet PC’s • Whiteboards • Large surfaces (walls, floors, tables) • Desktop PC’s • Netbooks / UMPC’s • Mobile phones • TV’s 13
  14. 14. Fraunhofer IGD multi-touch table • The table is running on Linux and uses the InstantReality framework for displaying and interacting. • The finger tracking is done by the Fraunhofer software VisionLib which is not available for pulic right now. • Table size is 150x90 cm 14
  15. 15. Mitsubishi: DiamondTouch • Front-projection on typically 32" or 42" diagonal • Multiple touches support by a single user • Distinguishes between simultaneous inputs from multiple users. • DiamondTouch SDK or DTFlash toolkit. 15
  16. 16. Gesturetek 16
  17. 17. Horizon
  18. 18. SMART table
  19. 19. Microsoft: Surface • www.surface.com • Object on table are identified by ‘domino tags’ • http://worldofitgeeks.blogspot.com/2007/10/domino-theories-for-microsofts-surface.html 19
  20. 20. HP TouchSmart 20
  21. 21. HP Touchsmart TX2 tablet PC
  22. 22. Dell Latitude XT 22
  23. 23. Hitachi Starboard FX77 Duo 23
  24. 24. Multi touch desktop LCD’s 24
  25. 25. TV 25
  26. 26. iPhone en iPod Touch • The Apple iPhone has a multitouch display for resizing, zooming and rotating purposes. • Multi touch sensative screen • Motion sensor • Types of multi touch gestures: • Zoom, rotate 26
  27. 27. G1 Google Android phone 27
  28. 28. Palm Pre • WebOS. It’s a web-centric multi- touch platform that makes us of CSS, XHTML and Javacript making it accessible and compatible with a wide range of applications. 28
  29. 29. And more mobile phone devices Meizu M8 HTC Touch Synaptics Onyx Nokia N97 29
  30. 30. Apple: MacBook Air • The Apple MacBook Air has multitouch touchpad for resizing, zooming and rotating purposes. 30
  31. 31. Eee PC 31
  32. 32. MultiTouch Cell • The MultiTouch LCDs are controlled by the MultiTouch CornerStone software. Cornerstone handles the core multitouch functionality, tracking users hands as they move on the screen. 32
  33. 33. Other vendors Accenture Panasonic Hitachi Philips Entertaible Perceptive Pixel Sony Smartskin HP Misto Sharp optical LCD Touchable
  34. 34. Market share of products with multi-touch • Multi-touch ability (or NUI) is competitor’s edge that is used in marketing • Market share is quickly rising • 10 mio iPhones sold in first 6 months • Table top sales booming since end 2008 Multi-touch is currently an “infant” technology • Different multi-touch imlementations (Apple, Microsoft, HP, etc.) • All are limited to a handful of gestures • Still a lot of traditional GUI interaction 34
  35. 35. Future product list • Input devices • Apple Mighty Mouse • Wacom tablets • Handheld game consoles • PSP2 • E-book readers • Desktops • Larger and flat tabletops • TV screens 35
  36. 36. Natural User Interfaces (NUI’s) Introduction • Touch screen interaction • Multi-touch screen interaction The vision • Evolution of computer user interfaces Reality • Natural User Interfaces (NUI) • Tangible interfaces • Multi modal interfaces • NUI design 36
  37. 37. Touch screen interaction Touch interaction • 1960: Touch • 1982: Multi touch • 2006: Lucid touch • Multi modal interfaces, controlled by • Touch • Gestures • Speech • Objects 39
  38. 38. Single Touch 40
  39. 39. Touch screen interfaces (since 1960’s) • Interaction via • Stylus, light pen, finger, hand,  Finger stress  Fingernail interaction  Fingerprints  “Gorilla arm” • No intelligent interaction • Gestures • Touch force sensing • Object sensing (finger vs hand) 41
  40. 40. Multi-touch & Natural User Interfaces 42
  41. 41. Multi-touch interaction • In the realworld we do lots of actions with two hands or more than one finger. e.g. driving a car • Operating a mouse are serial, discrete events and single point input • Multi point interaction allows more parallel interaction • Reduce task complexity of single input techniques • Increases parallesm in multi-finger interaction reduces time 43
  42. 42. Tangible interfaces 44
  43. 43. Interact with real objects 45
  44. 44. Haptics 46
  45. 45. Haptics 47
  46. 46. Haptics today 48
  47. 47. Lucid Touch 49
  48. 48. But lucid touch interaction is very complex 50
  49. 49. Gesture interaction 51
  50. 50. Gesture interaction today 52
  51. 51. Multimodal interaction 53
  52. 52. Multimodal interaction is still the future… 54
  53. 53. Paradigm shift Command Line Interfaces (CLI) 55
  54. 54. Paradigm shift Graphical User Interfaces (GUI) 56
  55. 55. Paradigm shift Natural User Interfaces (NUI) 57
  56. 56. Paradigm shift • Static • Responsive • Evocative • Disconnected • Indirect • Unmediated • High-low • Double medium • Fast few • Directed • Exploratory • Contextual • Recall • Recognition • Intuition • Abstract • Indirect • Direct • Textual • Graphical • Physical 58 Source: Dennis Wixon
  57. 57. Overview of Human Computer Interfaces Microsoft Microsoft Windows NT Windows Vista Alto PC Microsoft Microsoft Xerox Parc Windows 1.0 Windows 95 Microsoft MS-DOS Windows 98 Microsoft Microsoft Microsoft Q2 2009 Windows 3.0 Windows XP Windows 7 1993 1995 2001 1982 1985 1998 2007 1990 1973 1970 2010 Jun 2007 1987 1997 Microsoft 2001 1984 1992 2005 2009 Windows Snow Tablet PC Leopard? Apple Macintosh edition IBM OS/2 2.0 Apple Mac OS 8 Apple Mac OS X 10.5 Apple Mac OS X 10.0 Apple Mac Color Note: Linux excluded
  58. 58. Interaction styles and elements 1. Punch-card input (50’s-60’s) 2. Teletype writer connected to mainframe (60’s -80’s) EVOLUTION 3. CLI (80’s) 4. GUI (90’s-00’s) • Classic WIMP (Window, Icon, Menu, Pointer) • Desktop metaphor • Post-WIMP interfaces • Computer games • Virtual reality systems • Physical controls (e.g. iPod wheel) • Multi-touch interaction 3. NUI • STaG (Speech, Touch and Gesture) 4. XUI • Synthesis, anticipatory, constant fluid interaction 60
  59. 59. The vision of Natural User Experience (NUE) is to deliver intuitive, seamless experiences that unfold through natural human input.” • “Surfaces that respond to us” • “The content is the interface” 61
  60. 60. Surface: design for emotion • Natural & Intuitive • Unique & Magical • Social & Together • Aware & Responsive • Premium & Authentic 62
  61. 61. Surface guidelines 1. Seamless 2. Social 3. Spatial 4. Super realism 5. Contextual environments 6. Scaffolding 7. Performance aesthetics 8. Direct manipulation
  62. 62. User Interface Design for NUI’s Difference in designing Tabletop interfaces? • No fixed top-bottom orientation; • Multiple “mouse cursors”; • Multiple simultaneous users; • Multiple user contexts; • Multiple simultaneous actions, such as ‘drag & drop’; • No more tradional data input means. 64
  63. 63. Multi-touch in the movies • 1982: Tron • 2002: Minority report • 2005: The Island • 2008: Quantum of Solace (James Bond) • 2008: CSI: Miami 65
  64. 64. 1982: Tron 66
  65. 65. 2002: Minority Report 67
  66. 66. 2005: The Island 68
  67. 67. 2008: Quantum of Solace 69
  68. 68. 2008: CSI: Miami 70
  69. 69. Markets and applications • Entertainment: (serious) gaming and fun • Public safety and crisis management • Retail • Public Places • Healthcare • Education • Financial • Office • Industrial use 71
  70. 70. Healthcare • Photo tool for dementia patients • Tool for patient-doctor relationships 72
  71. 71. Entertainment • Games (serious/fun) 73
  72. 72. Public places • Information centre New York • Why? • Intuitive, interactive and focussed on discovering New York • Target group: Tourists • Device • Gesturetek illuminate tables • Application: • Maps, POI, multimedia content 74 Information centre New York
  73. 73. Museums 75
  74. 74. TV presentations CNN – presidential elections
  75. 75. Retail 78
  76. 76. Horeca • Restaurants, hotels • Viewing menu • Entertainment (games, gambling) • Tourist information Surfaceware: detect drink level 79
  77. 77. iBar: object recognizing bar • Bar detects all objects on the surface • It detects the shape, it doesn’t identify the object. • future releases will also be able to identify objects • http://www.i-bar.ch/info/ 80
  78. 78. Education 81
  79. 79. Control Rooms AT&T Telecomunication control room OODA loop 1. Static risk map 2. real-time or simulated flooding 3. Real-time information feeds 82
  80. 80. Public safety • Manages planning, tracking and report security matters surrounding an event. The Super Bowl E•SPONDER solution 83
  81. 81. Financial services Use in a bank lobby for customers to look at account features and sit down with a banker to discuss their financial needs. 84
  82. 82. i-phone multi-touch applications • Photo’s: create a collage • Graffiti • Scratch Mixmeister • Games Foto collages maken Muziek scratchen en mixen puistjes uitdrukken 85
  83. 83. Industrial use • Insight in processes • Observe, control and decide • Collaborative design • Fraunhofer: • Virtual factory on a tabletop
  84. 84. Collaborative work in the office 87
  85. 85. Office Digidesk
  86. 86. TNO Work • Multi -touch in Control Rooms • Multi –touch in Education • Multi-touch : complex gesture support • MiReCol: collaboration and co-design • Slide overview + video Maurice 89
  87. 87. Control Rooms
  88. 88. Experiment area 91
  89. 89. Visualisation of information (1.4)(2.4)(21.6)(4.2)(7.6)(2.6)(9.8)(3.0) Quick insight Large data amounts Unstructured Time critical Diverse sources
  90. 90. Data sources information exchange Virtual teams Control room teams Control room decision centre 93
  91. 91. Control Rooms AT&T Telecomunication control room OODA loop 1. Static risk map 2. real-time or simulated flooding 3. Real-time information feeds 94
  92. 92. Crisis management scenario Satellite or street map Multi-user interface Ten Boer Risk objects Real-time communication log Annotate Analyse Anticipate Change view, filter, log
  93. 93. MiReCol: Mixed Reality Collaboration
  94. 94. Mixed reality collaboration Augmented collaboration Shared Augmented virtuality understanding collaboration Measuring and facilitating collaboration
  95. 95. Collaboration • Co-creation, co-design • Decision making • CSCW support in general 100
  96. 96. Teliris telepresence
  97. 97. Education 102
  98. 98. Serious gaming & multi touch interactie in het onderwijs Research goal: Does the combination of serious gaming and multi touch interaction add value to elementary education. Serious games Target group: School children aged 8 until 12 with a social disorder (PDD-NOS). Multi touch Results: a serious game was developed to support and stimulate several collaboration forms in calculation tasks. Through multi- touch and multi user interaction these children could cooperate using one Education system. Partners: Bladergroenschool (elementary school for special education) and RENN4.
  99. 99. Supporting gesture interaction
  100. 100. Supporting gesture interaction with gesture previews Gestures • In general: non-verbal communication. • Specific: a set of measured movements over time. • Examples: letter recognition, unlocking your iPhone. Advanced gestures • Multi-touch gestures: zooming an image. • Collaborative gestures: clearing a drawing board. Problem statement • Most gestures have no intrinsic meaning. • Users have to know gestures in advance. • A steep learning curve especially for complex systems using advanced gestures.
  101. 101. 106
  102. 102. Solution: gesture previews • Overlay with visual feedback. • Shows what gestures can be performed. • Shows where others can join in. • Indicates the meaning of gestures. • Previews do not limit expert users. • Increase detection rate, decrease learning curve.
  103. 103. Proof of Concept • Bejeweled clone, uses gestures to identify users. With some twists. • Gestures are stored in XML. <?xml version="1.0" encoding="utf-8"?> <root> <segmentDeclaration name="line" uniformScaling="false" <point x="0" y="0" /> <point x="0" y="0,1" /> </segmentDeclaration>
  104. 104. Wrap up Knowledge • Started at TNO just a few years ago, experimental. Now this topic is booming, backed up by large vendors and coming to the market. Hardware/software • More vendors enabling MT for their devices • Current software / applications are dedicated solutions • Wide adoption needs more hardware and applications Interaction • Multi touch is part of the NUI paradigm shift in Human Machine Interaction Markets / applications • Wow-technology, but the real added value and useful applications still have to prove themselves TNO • Added value in collaboration environments (crisismanagement, education and many more projects) 109
  105. 105. Arnout de Vries Senior scientist Natural User Interfaces Arnout.devries@tno.nl http://www.tno.nl/nui
  • AshwiniKumari18

    May. 24, 2020
  • seyyedshahab

    May. 8, 2020
  • StenSwenson

    Nov. 12, 2018
  • HafeezUllah3

    May. 10, 2018
  • ssuserb1990b

    Sep. 8, 2017
  • affittatore

    Aug. 5, 2017
  • RheySingh

    Apr. 8, 2017
  • rakeshbn3

    Apr. 7, 2017
  • SibiSingh

    Mar. 26, 2017
  • PawanRohilla8

    Feb. 6, 2017
  • kamalayoub

    Jan. 20, 2017
  • hmhung

    Oct. 27, 2016
  • GamalMohamed8

    Aug. 17, 2016
  • krunalmevada

    Jul. 28, 2016
  • mixARConference

    Apr. 28, 2016
  • Harshitha56

    Apr. 27, 2016
  • MalavikaNadgir

    Mar. 11, 2016
  • PrashanthD6

    Feb. 18, 2016
  • HammadRaza6

    Feb. 14, 2016
  • BillyWahng

    Feb. 11, 2016

This presentation provides an overview of multi-touch hardware, products, applications and market examples as well as samples of projects of TNO. More information on http://www.tno.nl/nui

Views

Total views

48,408

On Slideshare

0

From embeds

0

Number of embeds

4,635

Actions

Downloads

0

Shares

0

Comments

0

Likes

257

×