1<br />The Future Of Human-Computer Input Interfaces<br />Stephanie Budiman<br />Johnny Cham<br />Karthik Nandakumar<br />...
2<br />Computing Anytime Anywhere!<br />Motivation: In today’s world, easy access to information and computing is required...
3<br />Human-Computer Interface (HCI)<br /><ul><li>HCI is the technology that connects man and machine
Robust HCIs are needed to enable ubiquitous computing</li></ul>We focus only on input interfaces in this presentation<br />
4<br />Technology Paradigms for Input Interfaces<br />Batch Interfaces<br />Graphical User Interfaces<br />Command Line In...
5<br />Value Proposition<br />
6<br />Key Components of Input Interfaces <br />
7<br />Speech Interfaces<br />Key Components<br />Microphone<br />Automated Speech Recognition (ASR) and Natural Language ...
Achieve human-level performance on ASR/NLU tasks</li></li></ul><li>8<br />Microphone Technology<br />MEMS Digital Micropho...
9<br />Microphone: Method of Improvement<br />Microphone array can mitigate background noise & interference1<br />Correspo...
10<br />Automated Speech Recognition (ASR) <br />L. Rabiner, “Challenges in Speech Recognition”, NSF Symp. on Next Gen. AS...
11<br />ASR Accuracy Improvements<br />NIST  Benchmark Test History – May 2009<br />WORD ERROR RATE (%)<br />YEAR<br />ASR...
12<br />ASR Accuracy in Text Dictation<br />*<br /><ul><li>Stand-alone speech interfaces may be useful for tasks like dict...
Speech as an important modality in multimodal user interfaces (e.g., Microsoft Kinect) may be the future</li></ul>* http:/...
13<br />Touch Interfaces<br />Key Component<br /><ul><li>Touch screen</li></ul>iPad Touch<br />Microsoft Surface<br />Key ...
14<br />Comparing Touch UI with Mouse<br /><ul><li>Fitt’s Law: Time required to move to a target area displayed on screen ...
15<br />Improving Throughput of Touch UI<br />Interaction based on large multi-touch screens can increase the throughput o...
16<br />Touch Screen Technologies<br />Resistive<br />Surface Capacitive<br />Infrared (IR)<br />Surface Acoustic Wave<br ...
17<br />Comparison of Touch Technologies<br />Source: Frost & Sullivan, Advances in Haptics & Touch Technology, 2010<br />
18<br />Future of Touch: Tangible Bits<br />Manipulating digital objects through physical objects<br />
19<br />Gesture Interfaces<br />Key Components<br />3D Camera (image sensor)<br />Tracking, Recognition & Gesture Understa...
20<br />Gesture UI: Methods of Improvement<br />Required improvements in image sensor technology<br />Accuracy<br /><ul><l...
Robustness to lighting changes (high pixel sensitivity)
More accurate depth sensing (lower depth error) </li></ul>Throughput<br /><ul><li>Higher frame rate (temporal resolution)<...
22<br />Camera Technology Improvements<br />Reducing pixel-size (green square) and improving sensitivity (Yellow circle) m...
23<br />Camera Price Improvements<br />Price per pixel for a 12 MP camera<br />Year<br />Number of pixels (resolution) has...
24<br />3D Image Sensor Technology<br /><ul><li>Time of flight (ToF) cameras require high temporal resolution* (70 picosec...
CMOS technology now has the required temporal resolution to enable production of affordable ToF 3D cameras
3D cameras will improve the accuracy of gesture UI</li></ul>Depth Error/Distance <br />Distance to image <br />Cost-effect...
25<br />Neural Interfaces<br />Key Component<br />Brain scanning device<br />Key values that need improvement are Accuracy...
26<br />Key Brain Scanning Technologies<br />Magneto Encephalo Graphy<br />Electro Encephalo Graphy<br />US$0.5M-3M<br />U...
27<br />Comparison of Technologies<br />Neuron can fire ~0.1mm (spatial) & <br />~10 ms (temporal)<br />Non-invasive<br />...
28<br />Spatial Resolution Improvement<br />While spatial resolution is important for accuracy, high temporal resolution i...
29<br />ElectroEncephaloGraphy (EEG)<br />Non-invasive interface using electrodes to pick up brain signals<br />Key limita...
30<br />MagnetoEncephaloGraphy (MEG)<br />fT<br />Baranga, A. B.-A. (2010). "Brain's Magnetic Field: a Narrow Window to Br...
31<br />MEG: Improvements in Millimeter-scale Atomic Magnetometer<br />A: 2004<br />B: 2007<br />C: 2007<br />D: 2009<br /...
Upcoming SlideShare
Loading in …5
×

Human-Computer Interfaces: When will new ones become technically and economically feasible?

1,787 views

Published on

Master's students use concepts from my (Jeff Funk) forthcoming book (Technology Change and the Rise of New Industries) to analyze the technical and economic feasibility of new human-computer interfaces (e.g., touch, gestures, voice, neural interfaces). See my other slides for details on concepts, methodology, and other new industries.

Published in: Business, Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,787
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
121
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide
  • Touch screens, in particular, are becoming increasingly popular because of their ease and versatility of operation as well as their declining price. Touch screens can include a touch panel, which can be a clear panel with a touch-sensitive surface. The touch panel can be positioned in front of or integral with a display screen so that the touch-sensitive surface covers the viewable area of the display screen. Touch screens can allow a user to make selections and move a cursor by simply touching the display screen via a finger or stylus. In general, the touch screen can recognize the touch and position of the touch on the display screen, and the computing system can interpret the touch and thereafter perform an action based on the touch event. Touch screens typically include a touch panel, a controller and a software driver. The touch panel is characteristically an optically clear panel with a touch sensitive surface that is positioned in front of a display screen so that the touch sensitive surface is coextensive with a specified portion of the display screen&apos;s viewable area (most often, the entire display area). The touch panel registers touch events and sends signals indicative of these events to the controller. The controller processes these signals and sends the resulting data to the software driver. The software driver, in turn, translates the resulting data into events recognizable by the electronic system (e.g., finger movements and selections). Single TouchSingle Touch occurs when a finger or stylus creates a touch event on the surface of a touch sensor or within a touch field so it is detected by the touch controller and the application can determine the X,Y coordinates of the touch event. These technologies have been integrated into millions of devices and typically do not have the ability to detect or resolve more than a single touch point at a time as part of their standard configuration.Single Touch with Pen InputSingle Touch with Pen input functionality can range from a simple, inactive pointer or stylus to complex, active tethered pens. Inactive pens enable the same input characteristics as a finger, but with greater pointing accuracy, while sophisticated, active pens can provide more control and uses for the touch system with drawing and palm rejection capabilities, and mouse event capabilities.Single Touch with GestureEnhancements to firmware, software and hardware by many single touch technologies have increased their touch functionality. Some touch technologies can use advanced processing capabilities to &quot;detect&quot; or recognize that a second touch event is occurring, which is called a &quot;gesture event.&quot; Since single touch systems can&apos;t resolve the exact location of the second touch event they rely on algorithms to interpret or anticipate the intended gesture event input. Common industry terms for this functionality are two-finger gestures, dual touch, dual control, and gesture touch.Two TouchTwo Touch refers to a touch system that can detect and resolve two discrete, simultaneous touch events. The best demonstration of Two Touch capability is to draw two parallel lines on the screen at the same time. Two Touch systems can also support gesturing.Multi-touchMulti-touch refers to a touch system&apos;s ability to simultaneously detect and resolve a minimum of 3+ touch points. All 3 or more touches are detected and fully resolved resulting in a dramatically improved touch experience. Multi-touch is considered by many to become a widely-used interface mainly because of the speed, efficiency and intuitiveness of the technology.
  • Touch screens, in particular, are becoming increasingly popular because of their ease and versatility of operation as well as their declining price. Touch screens can include a touch panel, which can be a clear panel with a touch-sensitive surface. The touch panel can be positioned in front of or integral with a display screen so that the touch-sensitive surface covers the viewable area of the display screen. Touch screens can allow a user to make selections and move a cursor by simply touching the display screen via a finger or stylus. In general, the touch screen can recognize the touch and position of the touch on the display screen, and the computing system can interpret the touch and thereafter perform an action based on the touch event. Touch screens typically include a touch panel, a controller and a software driver. The touch panel is characteristically an optically clear panel with a touch sensitive surface that is positioned in front of a display screen so that the touch sensitive surface is coextensive with a specified portion of the display screen&apos;s viewable area (most often, the entire display area). The touch panel registers touch events and sends signals indicative of these events to the controller. The controller processes these signals and sends the resulting data to the software driver. The software driver, in turn, translates the resulting data into events recognizable by the electronic system (e.g., finger movements and selections). Single TouchSingle Touch occurs when a finger or stylus creates a touch event on the surface of a touch sensor or within a touch field so it is detected by the touch controller and the application can determine the X,Y coordinates of the touch event. These technologies have been integrated into millions of devices and typically do not have the ability to detect or resolve more than a single touch point at a time as part of their standard configuration.Single Touch with Pen InputSingle Touch with Pen input functionality can range from a simple, inactive pointer or stylus to complex, active tethered pens. Inactive pens enable the same input characteristics as a finger, but with greater pointing accuracy, while sophisticated, active pens can provide more control and uses for the touch system with drawing and palm rejection capabilities, and mouse event capabilities.Single Touch with GestureEnhancements to firmware, software and hardware by many single touch technologies have increased their touch functionality. Some touch technologies can use advanced processing capabilities to &quot;detect&quot; or recognize that a second touch event is occurring, which is called a &quot;gesture event.&quot; Since single touch systems can&apos;t resolve the exact location of the second touch event they rely on algorithms to interpret or anticipate the intended gesture event input. Common industry terms for this functionality are two-finger gestures, dual touch, dual control, and gesture touch.Two TouchTwo Touch refers to a touch system that can detect and resolve two discrete, simultaneous touch events. The best demonstration of Two Touch capability is to draw two parallel lines on the screen at the same time. Two Touch systems can also support gesturing.Multi-touchMulti-touch refers to a touch system&apos;s ability to simultaneously detect and resolve a minimum of 3+ touch points. All 3 or more touches are detected and fully resolved resulting in a dramatically improved touch experience. Multi-touch is considered by many to become a widely-used interface mainly because of the speed, efficiency and intuitiveness of the technology.
  • Touch Technology refers to technology that can detect and process touch signals.HowitWorksTouch screen basically consists of two layers. When the user presses at a point, the two layers come in contact and a signal is created.Different Touch TechnologyCapacitiveThe operation relies on the capacitance of the human body. When a person touches the screen, a small current flows to the point of touch, causing a voltage drop which is sensed at the 4 corners. Capacitive touch screens use the body of the user as a ground for a small electric current promulgated over the screen.ResistiveWhen a person presses on the top sheet, it is deformed and its conductive side comes in contact with the conductive side of the glass, effectively closing a circuit. The voltage at the point of contact is read from a wire connected to the top sheet.The term, &quot;resistive&quot; refers to the way the system registers the touch of the user. Because a resistive touch screen responds to the pressure of the touchInfraredInfrared scanning systems register a touch when a field of infrared beams is interrupted. Surface Acoustic WaveA surface-acoustic wave touch identifies a touch by the reduction of the acoustic signal at the point of contact on the screen.
  • All a multi-touch functionalitySOURCE:Resistive offers the flexibility of activation by multiple touch devices and an attractive price, but does not offer high clarity. Overall, it is an excellent offering for retailers seeking to implement touch while minimizing cost. Capacitive offers good image clarity and excellent durability at a moderate price but is limited to activation by contact with a human finger. In traditional retail environments this combination of benefits has been widely accepted. Surface wave acoustic accepts input from several devices and offers excellent clarity and durability, but is also more sensitive to the contaminants prevalent in a retail environment. Infrared allows input from multiple devices and offers minimal calibration, dirt and debris may cause it to register unintentional touches, and it is significantly slower to react to a touch than the other technologies.
  • Anatomical Structures (Static Pictures of Living Tissues)Computer Assisted X-ray Tomography CAT (CT Scan)  US$180K- US$250KMagnetic Resonance Imaging (MRI) -&gt; $0.5M-$3MFunctional Metabolic/ Blood Flow ExplorationSingle-Photon-Emission Computed Tomography (SPECT) US$2.9MPositron-Emission Tomography (PET)  US$5M – US$7Mfunctional MRI (fMRI) –&gt; US$1- US$1.5MElectroEncephaloGraphy (EEG): US$250KMagnetoEncephaloGraphy (MEG): US$2.4MNear Infrared Spectroscopy (NIRS): &gt;US$30K without a computer
  • As we achieve finer and finer resolutions from scanning techniques that don’t require getting inside the skull, we’ll increasingly be able to better understand the details of brain processes and how things happen at various structural levels.
  • Limitation: noise issueLab Environment NoiseElevator, lamps, general equipments, magnetometer movement, Geomagnetic fluctuations, etc.Human Body NoiseEye movements and blinks, cardiac activity, electric currents in muscles.Source localization in the brain might have multiple solution.
  • The SERF magnetometers are in the range of what is required for biomagnetic measurements.Need to have gd sensitivity to detect low signal MEG: vry2 low signal  need to hv highly magnetic signal sensors, big shielding.We hv reached the target, need to reduce further
  • Human-Computer Interfaces: When will new ones become technically and economically feasible?

    1. 1. 1<br />The Future Of Human-Computer Input Interfaces<br />Stephanie Budiman<br />Johnny Cham<br />Karthik Nandakumar<br />Osbert Poniman<br />Mauhay Mary Esther Samson<br />
    2. 2. 2<br />Computing Anytime Anywhere!<br />Motivation: In today’s world, easy access to information and computing is required anytime and anywhere<br />
    3. 3. 3<br />Human-Computer Interface (HCI)<br /><ul><li>HCI is the technology that connects man and machine
    4. 4. Robust HCIs are needed to enable ubiquitous computing</li></ul>We focus only on input interfaces in this presentation<br />
    5. 5. 4<br />Technology Paradigms for Input Interfaces<br />Batch Interfaces<br />Graphical User Interfaces<br />Command Line Interfaces<br />Natural User Interfaces<br />Neural Interfaces<br />
    6. 6. 5<br />Value Proposition<br />
    7. 7. 6<br />Key Components of Input Interfaces <br />
    8. 8. 7<br />Speech Interfaces<br />Key Components<br />Microphone<br />Automated Speech Recognition (ASR) and Natural Language Understanding (NLU) Software<br />Key value that needs improvement is Accuracy<br />Possible methods of improvement are<br /><ul><li>Increase Signal to Noise Ratio (SNR) from microphone
    9. 9. Achieve human-level performance on ASR/NLU tasks</li></li></ul><li>8<br />Microphone Technology<br />MEMS Digital Microphone<br />SNR: 61 dB<br />Flatter frequency response<br />Smaller size <br />(CMOS fabrication) <br />Electret Condenser Microphone (ECM)<br />SNR: 55-58 dB<br />Noise levels in a microphone is close to the thermal noise limit <br />Analog Devices, MEMS Microphone Technology, October 2010<br />
    10. 10. 9<br />Microphone: Method of Improvement<br />Microphone array can mitigate background noise & interference1<br />Corresponding decrease in Word Error Rate<br />Improvement in SNR<br />Noise suppression algorithms can increase SNR by 18dB with just 4 microphones in an array2<br />LOUD project from MIT Computer Science and Artificial Intelligence Laboratory, 2005<br />Microphone Array project in MSR: Approach and Results, Microsoft Research, June 2004<br />
    11. 11. 10<br />Automated Speech Recognition (ASR) <br />L. Rabiner, “Challenges in Speech Recognition”, NSF Symp. on Next Gen. ASR, 2003<br />“Increase in vocabulary sizes needs exponential increase in computing power due to potential combinatorial explosions”<br />
    12. 12. 11<br />ASR Accuracy Improvements<br />NIST Benchmark Test History – May 2009<br />WORD ERROR RATE (%)<br />YEAR<br />ASR accuracy is acceptable only in some niche applications<br />
    13. 13. 12<br />ASR Accuracy in Text Dictation<br />*<br /><ul><li>Stand-alone speech interfaces may be useful for tasks like dictation
    14. 14. Speech as an important modality in multimodal user interfaces (e.g., Microsoft Kinect) may be the future</li></ul>* http://blogs.msdn.com/b/sprague/archive/2004/10/22/246506.aspx<br />
    15. 15. 13<br />Touch Interfaces<br />Key Component<br /><ul><li>Touch screen</li></ul>iPad Touch<br />Microsoft Surface<br />Key values that need improvement are <br />Throughput and Affordability<br />
    16. 16. 14<br />Comparing Touch UI with Mouse<br /><ul><li>Fitt’s Law: Time required to move to a target area displayed on screen is proportional to index of difficulty (distance to the target/width of the target area)</li></ul>Touch UI has better throughput than mouse for more difficult tasks<br />Muller, L.Y.L., “Multi-touch displays: design, applications & performance evaluation”, 2008 <br />
    17. 17. 15<br />Improving Throughput of Touch UI<br />Interaction based on large multi-touch screens can increase the throughput of touch user interfaces<br />B. Buxton, “Multi-Touch Systems”, Microsoft Research, 2007<br />
    18. 18. 16<br />Touch Screen Technologies<br />Resistive<br />Surface Capacitive<br />Infrared (IR)<br />Surface Acoustic Wave<br />Source: 3M Touch System Website<br />
    19. 19. 17<br />Comparison of Touch Technologies<br />Source: Frost & Sullivan, Advances in Haptics & Touch Technology, 2010<br />
    20. 20. 18<br />Future of Touch: Tangible Bits<br />Manipulating digital objects through physical objects<br />
    21. 21. 19<br />Gesture Interfaces<br />Key Components<br />3D Camera (image sensor)<br />Tracking, Recognition & Gesture Understanding Software<br />Key values that need improvement are <br />Accuracy, Throughput and Affordability<br />
    22. 22. 20<br />Gesture UI: Methods of Improvement<br />Required improvements in image sensor technology<br />Accuracy<br /><ul><li>Higher spatial resolution (number of pixels)
    23. 23. Robustness to lighting changes (high pixel sensitivity)
    24. 24. More accurate depth sensing (lower depth error) </li></ul>Throughput<br /><ul><li>Higher frame rate (temporal resolution)</li></ul>Affordability<br /><ul><li>Smaller pixel size reduces price per pixel</li></li></ul><li>21<br />Image Sensor Technologies<br />As pixel size decreases, resolution improves (more pixels per area), but sensitivity decreases because the “light available per pixel” will become less<br />Back illuminated CMOS technology provides better trade-off between pixel size and sensitivity than traditional charge coupled device (CCD)-based image sensors<br />CMOS-based image sensors are also expected to follow Moore’s Law in size and cost scaling<br />T. Suzuki, “Challenges of Image-Sensor Development”, ISSCC, 2010<br />
    25. 25. 22<br />Camera Technology Improvements<br />Reducing pixel-size (green square) and improving sensitivity (Yellow circle) miniaturized cameras without reducing quality<br />T. Suzuki, “Challenges of Image-Sensor Development”, ISSCC, 2010<br />
    26. 26. 23<br />Camera Price Improvements<br />Price per pixel for a 12 MP camera<br />Year<br />Number of pixels (resolution) has increased, while price per pixel has decreased <br />K. Wiley, “Digital Photography”, www.keithwiley.com<br />
    27. 27. 24<br />3D Image Sensor Technology<br /><ul><li>Time of flight (ToF) cameras require high temporal resolution* (70 picoseconds for 1cm depth resolution)
    28. 28. CMOS technology now has the required temporal resolution to enable production of affordable ToF 3D cameras
    29. 29. 3D cameras will improve the accuracy of gesture UI</li></ul>Depth Error/Distance <br />Distance to image <br />Cost-effective 3D image sensors are now becoming available (e.g., Microsoft Kinect ~ 150 USD) <br />* R. Lange, “3D Time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology”, PhD Thesis, 2000<br />
    30. 30. 25<br />Neural Interfaces<br />Key Component<br />Brain scanning device<br />Key values that need improvement are Accuracy, Throughput and Affordability<br />Required improvements in brain scanning technology<br />Accuracy – Higher spatial resolution<br />Throughput – Higher temporal resolution<br />Affordability – Size and better materials<br />
    31. 31. 26<br />Key Brain Scanning Technologies<br />Magneto Encephalo Graphy<br />Electro Encephalo Graphy<br />US$0.5M-3M<br />US$2.4M<br />US$2.9M<br />US$250K<br />SPECT<br />EEG<br />CT Scan<br />MEG<br />fMRI<br />NIRS<br />MRI<br />PET<br />1972<br />1991<br />1983<br />1950<br />1968<br />1936<br />1975<br />1973<br />US$1M-1.5M<br />US$180K- 250K<br />US$5M-7M<br />>US$30K <br />Baranga, A. B.-A. (2010). "Brain's Magnetic Field: a Narrow Window to Brain's Activity". Electromagnetic field and the human body workshop, (pp. 3-4).<br />
    32. 32. 27<br />Comparison of Technologies<br />Neuron can fire ~0.1mm (spatial) & <br />~10 ms (temporal)<br />Non-invasive<br />Invasive<br />Ideally, a non-invasive technology with high spatial resolution and high temporal resolution is required<br />Additionally, the technology must be affordable and portable in order to be useful in HCI applications<br />Gerven, M. v., et al., “The Brain-Computer Interface Cycle”, J. Neural Eng, 2009<br />
    33. 33. 28<br />Spatial Resolution Improvement<br />While spatial resolution is important for accuracy, high temporal resolution is critical for user interfaces<br />R. Kurzweil, “The Singularity is Near”, 2005<br />
    34. 34. 29<br />ElectroEncephaloGraphy (EEG)<br />Non-invasive interface using electrodes to pick up brain signals<br />Key limitation: Poor spatial resolution<br />Increasing number of EEG electrodes may provide limited improvement in spatial resolution and higher SNR<br />J. Malmivuo, “Comparison of the Properties of EEG and MEG”, Intl J of Bioelectromagnetism, 6 (1), 2004<br />
    35. 35. 30<br />MagnetoEncephaloGraphy (MEG)<br />fT<br />Baranga, A. B.-A. (2010). "Brain's Magnetic Field: a Narrow Window to Brain's Activity". Electromagnetic field and the human body workshop, (pp. 12).<br />
    36. 36. 31<br />MEG: Improvements in Millimeter-scale Atomic Magnetometer<br />A: 2004<br />B: 2007<br />C: 2007<br />D: 2009<br />Target: ~100fT and <100Hz<br />J. Kitching, et al., “Uncooled, Millimeter-Scale Atomic Magnetometers”, IEEE Sensors 2009 Conference (pp. 1844-1846)<br />
    37. 37. 32<br />Further Scope for Improvement<br />Hybrid technologies <br />Enable high levels of accuracy in diagnosis that individual modalities cannot offer.<br />E.g: MEG, EEG and MRI1 or PET/CT, PET/MRI2<br />Better shielding for noise reduction <br />Low frequency noise: use flux-entrapment shields <br />High frequency noise: use lossy magnetic shields based on induced eddy currents<br />Reduce costs<br />Open-source: OpenEEG<br />[1] Baranga, A. B.-A. (2010). "Brain's Magnetic Field: a Narrow Window to Brain's Activity". Electromagnetic field and the human body workshop, (pp.15).<br />[2] Stommen, J. (2011, Mar 25). "Superior capabilities boost hybrid imaging, says report“, Medical Device Daily<br />
    38. 38. 33<br />Opportunities in Input Interfaces<br />
    39. 39. 34<br />Market Segments for Input Interfaces<br />
    40. 40. 35<br />Conclusions<br /><ul><li>Ubiquitous computing requires new HCI paradigms
    41. 41. Natural and Neural Interfaces are the future of human-computer input interfaces
    42. 42. Touch interfaces have already diffused into the mainstream; speech and gesture interfaces are becoming more accurate and affordable
    43. 43. Neural interfaces requires development of more accurate, cheap, and portable sensors
    44. 44. Numerous entrepreneurial opportunities are available both in technology development & customization </li></li></ul><li>36<br />THANK YOU!<br />Q&A?<br />

    ×