Jason Grigsby • @grigs • cloudfour.com
Adaptive Input
http://www.flickr.com/photos/cobalt/7217055290/
Tweets: @grigs_talks
Slides: http://bit.ly/grigs-2013-07-22
http://www.flickr.com/photos/cobalt/7217055290/
http://www.flickr.com/photos/cdm/51747860/
http://www.flickr.com/photos/rheaney/4397863376
It started with TVs.
Designing for a 10-foot UI is very different.
http://www.flickr.com/photos/chrisbartow/5835428673
Larger text and fewer words.
Make up, down, left, right directions clear.
http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.jpg
How do we know what is a TV?
This is HDTV.
This is HDTV.
1980 px
1080px
Resolution does not
define the optimal
experience.
Next came responsive web apps.
https://twitter.com/freediverx/status/354698695041744896
How do I make this responsive?
How do I make this responsive?
mobiledesktop
THE ART OF WEB DEVELOPMENTTHE ART OF WEB DEVELOPMENT
Web widgets
THE ART OF WEB DEVELOPMENTTHE ART OF WEB DE...
It’s not that we’re technically incapable, but adapting a phone UI to a
tablet UI is not so dissimilar from trying to auto...
Sometimes it’s hard to envision a responsive version.
http://demos.kendoui.com/web/grid/editing.html
http://www.flickr.com/photos/jesuspresley/384080245/
We want people to be productive…
and stay in the zone.
http://www.flickr.com/photos/raccatography/8038855203
http://www.flickr.com/photos/shantellmartin/4543010568
Which seems very different from playing on an iPad.
For both the TV…
and the desktop web app…
input matters much more than screen size.
The grid is important to support d-pad interaction.
http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.j...
http://www.flickr.com/photos/royalsapien/2387707860
And keyboard and mouse are what we envision work is.
http://www.flickr.com/photos/royalsapien/2387707860
http://www.flickr.com/photos/hellogeri/6154034099/
Two years ago at BDConf, Jeremy talked about how…
http://www.flickr.com/photos/60415054@N00/14301113/
we told ourselves
that the web was…
http://www.flickr.com/photos/60415054@N00/14301113/
http://www.flickr.com/photos/60415054@N00/14301113/
640 px
480px
640 px
480px
1024 px
768px
http://www.flickr.com/photos/adactio/6153481666/
http://www.flickr.com/photos/adactio/6153481666/
Then mobile came and made us realize…
that it was a collective hallucination all along.
http://www.flickr.com/photos/garibaldi/303085857/
The web never had a fixed canvas.
http://www.flickr.com/photos/paulocarrillo/124755065/
Even our tools perpetuate the lie.
http://www.flickr.com/photos/69797234@N06/7203485148/
We’ve made tremendous progress.
But there is another collective hallucination.
http://www.flickr.com/photos/garibaldi/303085857/
=
=
Supports hover and pointer events.
Keyboard and touch.
Even the iPhone can have a keyboard.
Are these laptops or tablets?
Desktop computer with 23” touch screen
Luke nailed it.
http://static.lukew.com/unified_device_design.png
We can no longer make assumptions about input
based on screen size or form factor.
And we probably never should have.
http://www.flickr.com/photos/cblue98/7254221968
Input represents a bigger challenge than screen size.
http://www.flickr.com/photos/cblue98/7254221968
http://www.flickr.com/photos/taedc/9278192929
And it is changing more rapidly than ever before.
http://www.flickr.com/photos/taedc/9278192929
So let’s take a closer look…
Let’s start with futuristic input.
http://www.flickr.com/photos/jdhancock/3714748769/
http://uncyclopedia.wikia.com/wiki/File:Man_yelling_at_computer.JPG
VOICE
http://uncyclopedia.wikia.com/wiki/File:Man_yelling_at_computer.JPG
VOICE
http://www.98ps.com/viewnews-15222.html
Siri gets all of the hype…
http://www.98ps.com/viewnews-15222.html
but both Microsoft and Google have
compelling voice input in their products.
Explore the web on your TV with Internet Explorer for Xbox One. Use
your voice to browse your favorite sites with ease. Us...
Xbox 360 Voice Control
http://www.youtube.com/watch?v=xsfVJxakDa8
How would web pages change if we had voice control?
Google voice search
You can use speech recognition too.
http://www.google.com/intl/en/chrome/demos/speech.html
http://www.moreawesomeweb.com/d...
Web Speech API Specification
19 October 2012
Editors:
Glen Shires, Google Inc.
Hans Wennborg, Google Inc.
Please refer to ...
Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Cop...
Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Cop...
Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Cop...
Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Cop...
Hans Wennborg, Google Inc.
Please refer to the errata for this document, which may include some normative corrections.
Cop...
Special thanks to Eric Bidelman
http://moreawesomeweb.com
Speech Recognition API Support
?
Gestures?
Amazing, but too new to know
what, if anything, this technology
will mean for the web.
Let’s come back from the future and
look at something much Dumber.Dumber
Dumber
Dumber
-pad remote controlsD
-pad remote controlsD
-pad remote controlsD
http://www.flickr.com/photos/stewc/6669743035/
http://www.flickr.com/photos/stewc/6669743035/
TVs browsers that
support d-pad, send
arrow key events.
http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/
If then
http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/
is undetectable.
This is a recurring theme for input.
Sensors and camera
http://www.flickr.com/photos/retrocactus/2170677056
Sensors and camera
Camera
http://www.flickr.com/photos/retrocactus/2170677056
GPS
http://www.flickr.com/photos/3dking/149450434
GPS
GeoLocation
http://www.flickr.com/photos/3dking/149450434
Gyroscope & Accelerometer
http://www.flickr.com/photos/chrisjagers/4694134078
Back to today’s problems.
http://www.flickr.com/photos/chrisjagers/4694134078
Hover state No hover state
Hover state
Typing easier for many
No hover state
Typing often more difficult
Higher precision with mouse means
smaller targets possible
Hover state
Less precise than mouse and requires
larger touch t...
Higher precision with mouse means
smaller targets possible
Hover state
Less precise than mouse and requires
larger touch t...
http://www.flickr.com/photos/28096801@N05/5012309802
I got this. Detect touch.
http://www.flickr.com/photos/28096801@N05/5012309802
Whatever you may think, it currently isn't
possible to reliably detect whether or not the
current device has a touchscreen...
Chrome would like to turn on touch events by default.
https://code.google.com/p/chromium/issues/detail?id=159527
https://d...
Detect a mouse? Not reliably.
Surely we can detect a keyboard?
Surely we can detect a keyboard?
NOPE
Input is dynamic.
Input is dynamic.
Boris Smus’s experiments responding to input.
Boris Smus’s experiments responding to input.
http://www.flickr.com/photos/lyza/7382235106
Maybe we need to be more zen about input.
After poking at this problem for a few weeks,
my conclusion is: every desktop UI should be
designed for touch now. When an...
http://ie.microsoft.com/testdrive/ieblog/2011/Sep/20_TouchInputforIE10andMetrostyleApps_1.png
http://www.w3.org/TR/pointer...
What about those who won’t let
go of their “power” interfaces?
http://www.flickr.com/photos/ecos/4092571213/
http://www.flickr.com/photos/scarygami/5689980135/
One option: give them a choice.
Gmail display density settings
Th Dream Experience - …
Uploaded 2 years ago
More Of The Dream Exp…
Uploaded 2 years ago
The Dream Experience -…
Uploaded ...
Couch Mode+ See allCentric TV’s videos / Recently viewed / Related videos
Th Dream Experience - …
Uploaded 2 years ago
Mor...
Couch Mode+ See allCentric TV’s videos / Recently viewed / Related videos
Th Dream Experience - …
Uploaded 2 years ago
Mor...
Vimeo Couch Mode
The key benefit of this approach:
You’re designing for user need not
for a specific form factor or input.
http://www.flickr.com/photos/raver_mikey/504815463
Progressive Input?
Graph from Chapter 1 of Adaptive Web Design by Aaron Gustafson
http://easy-readers.net/books/adaptive-web-design/
Graph from Chapter 1 of Adaptive Web Design by Aaron Gustafson
http://easy-readers.net/books/adaptive-web-design/
Progress...
Who are we to judge what input is better?
http://www.flickr.com/photos/fensterbme/4783366926
We need to learn
to adapt.
http://www.flickr.com/photos/cdm/147947664/
So that’s the puzzle that has
been haunting me.
http://www.flickr.com/photos/sravishankar/3460495
http://www.flickr.com/photos/wwworks/1384952210
Thank You!Special thanks to Luke Wroblewski, Eric Bidelman and Flickr users for
generously sharing their photos under crea...
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Adaptive Input — Breaking Development Conference, San Diego
Upcoming SlideShare
Loading in...5
×

Adaptive Input — Breaking Development Conference, San Diego

1,447

Published on

Windows 8. Chromebook Pixel. Ubuntu Phone. These devices shatter another consensual hallucination that we web developers have bought into: mobile = touch and desktop = keyboard and mouse.

We have tablets with keyboards; laptops that become tablets; laptops with touch screens; phones with physical keyboards; and even phones that become desktop computers. Not to mention new forms of input like cameras, voice control, and sensors.

We've learned how to respond to screen size. Our next challenge is learning how to adapt to different forms of input.

Published in: Technology
1 Comment
3 Likes
Statistics
Notes
No Downloads
Views
Total Views
1,447
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
18
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide

Adaptive Input — Breaking Development Conference, San Diego

  1. 1. Jason Grigsby • @grigs • cloudfour.com Adaptive Input http://www.flickr.com/photos/cobalt/7217055290/
  2. 2. Tweets: @grigs_talks Slides: http://bit.ly/grigs-2013-07-22 http://www.flickr.com/photos/cobalt/7217055290/
  3. 3. http://www.flickr.com/photos/cdm/51747860/
  4. 4. http://www.flickr.com/photos/rheaney/4397863376 It started with TVs.
  5. 5. Designing for a 10-foot UI is very different. http://www.flickr.com/photos/chrisbartow/5835428673
  6. 6. Larger text and fewer words.
  7. 7. Make up, down, left, right directions clear. http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.jpg
  8. 8. How do we know what is a TV?
  9. 9. This is HDTV.
  10. 10. This is HDTV. 1980 px 1080px
  11. 11. Resolution does not define the optimal experience.
  12. 12. Next came responsive web apps. https://twitter.com/freediverx/status/354698695041744896
  13. 13. How do I make this responsive? How do I make this responsive?
  14. 14. mobiledesktop THE ART OF WEB DEVELOPMENTTHE ART OF WEB DEVELOPMENT Web widgets THE ART OF WEB DEVELOPMENTTHE ART OF WEB DEVELOPMENT Mobile widgets
  15. 15. It’s not that we’re technically incapable, but adapting a phone UI to a tablet UI is not so dissimilar from trying to automatically adapt desktop UI to a phone. They are fundamentally different platforms with different usability considerations, and something that makes sense on phones may or may not belong on tablets. —Todd Anglin, Kendo UI http://www.kendoui.com/blogs/teamblog/posts/12-09-11/universal_mobile_apps_with_html5_and_kendo_ui.aspx
  16. 16. Sometimes it’s hard to envision a responsive version. http://demos.kendoui.com/web/grid/editing.html
  17. 17. http://www.flickr.com/photos/jesuspresley/384080245/ We want people to be productive…
  18. 18. and stay in the zone. http://www.flickr.com/photos/raccatography/8038855203
  19. 19. http://www.flickr.com/photos/shantellmartin/4543010568 Which seems very different from playing on an iPad.
  20. 20. For both the TV…
  21. 21. and the desktop web app…
  22. 22. input matters much more than screen size.
  23. 23. The grid is important to support d-pad interaction. http://images.dailytech.com/nimage/29122_large_amazon_prime_screen_5.jpg
  24. 24. http://www.flickr.com/photos/royalsapien/2387707860
  25. 25. And keyboard and mouse are what we envision work is. http://www.flickr.com/photos/royalsapien/2387707860
  26. 26. http://www.flickr.com/photos/hellogeri/6154034099/ Two years ago at BDConf, Jeremy talked about how…
  27. 27. http://www.flickr.com/photos/60415054@N00/14301113/ we told ourselves that the web was…
  28. 28. http://www.flickr.com/photos/60415054@N00/14301113/
  29. 29. http://www.flickr.com/photos/60415054@N00/14301113/ 640 px 480px
  30. 30. 640 px 480px
  31. 31. 1024 px 768px
  32. 32. http://www.flickr.com/photos/adactio/6153481666/
  33. 33. http://www.flickr.com/photos/adactio/6153481666/ Then mobile came and made us realize…
  34. 34. that it was a collective hallucination all along. http://www.flickr.com/photos/garibaldi/303085857/
  35. 35. The web never had a fixed canvas. http://www.flickr.com/photos/paulocarrillo/124755065/
  36. 36. Even our tools perpetuate the lie.
  37. 37. http://www.flickr.com/photos/69797234@N06/7203485148/ We’ve made tremendous progress.
  38. 38. But there is another collective hallucination. http://www.flickr.com/photos/garibaldi/303085857/
  39. 39. = =
  40. 40. Supports hover and pointer events.
  41. 41. Keyboard and touch.
  42. 42. Even the iPhone can have a keyboard.
  43. 43. Are these laptops or tablets?
  44. 44. Desktop computer with 23” touch screen
  45. 45. Luke nailed it. http://static.lukew.com/unified_device_design.png
  46. 46. We can no longer make assumptions about input based on screen size or form factor. And we probably never should have.
  47. 47. http://www.flickr.com/photos/cblue98/7254221968
  48. 48. Input represents a bigger challenge than screen size. http://www.flickr.com/photos/cblue98/7254221968
  49. 49. http://www.flickr.com/photos/taedc/9278192929
  50. 50. And it is changing more rapidly than ever before. http://www.flickr.com/photos/taedc/9278192929
  51. 51. So let’s take a closer look…
  52. 52. Let’s start with futuristic input. http://www.flickr.com/photos/jdhancock/3714748769/
  53. 53. http://uncyclopedia.wikia.com/wiki/File:Man_yelling_at_computer.JPG VOICE
  54. 54. http://uncyclopedia.wikia.com/wiki/File:Man_yelling_at_computer.JPG VOICE
  55. 55. http://www.98ps.com/viewnews-15222.html
  56. 56. Siri gets all of the hype… http://www.98ps.com/viewnews-15222.html
  57. 57. but both Microsoft and Google have compelling voice input in their products.
  58. 58. Explore the web on your TV with Internet Explorer for Xbox One. Use your voice to browse your favorite sites with ease. Use Bing to find the best of the web. Or use Xbox SmartGlass on your phone or tablet to control your experience. http://www.xbox.com/en-US/xboxone/what-it-does
  59. 59. Xbox 360 Voice Control http://www.youtube.com/watch?v=xsfVJxakDa8
  60. 60. How would web pages change if we had voice control?
  61. 61. Google voice search
  62. 62. You can use speech recognition too. http://www.google.com/intl/en/chrome/demos/speech.html http://www.moreawesomeweb.com/demos/speech_translate.html
  63. 63. Web Speech API Specification 19 October 2012 Editors: Glen Shires, Google Inc. Hans Wennborg, Google Inc. Please refer to the errata for this document, which may include some normative corrections. Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable summary is available. Abstract This specification defines a JavaScript API to enable web developers to incorporate speech recognition and synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web pages to control activation and timing and to handle results and alternatives. Status of This Document This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA) other conditions apply. Learn more about W3C Community and Business Groups.
  64. 64. Hans Wennborg, Google Inc. Please refer to the errata for this document, which may include some normative corrections. Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable summary is available. Abstract This specification defines a JavaScript API to enable web developers to incorporate speech recognition and synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web pages to control activation and timing and to handle results and alternatives. Status of This Document This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA) other conditions apply. Learn more about W3C Community and Business Groups. All feedback is welcome. Table of Contents 1 Conformance requirements 2 Introduction 3 Use Cases 4 Security and privacy considerations 5 API Description 5.1 The SpeechRecognition Interface 5.1.1 SpeechRecognition Attributes 5.1.2 SpeechRecognition Methods 5.1.3 SpeechRecognition Events 5.1.4 SpeechRecognitionError 5.1.5 SpeechRecognitionAlternative 5.1.6 SpeechRecognitionResult
  65. 65. Hans Wennborg, Google Inc. Please refer to the errata for this document, which may include some normative corrections. Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable summary is available. Abstract This specification defines a JavaScript API to enable web developers to incorporate speech recognition and synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web pages to control activation and timing and to handle results and alternatives. Status of This Document This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA) other conditions apply. Learn more about W3C Community and Business Groups. All feedback is welcome. Table of Contents 1 Conformance requirements 2 Introduction 3 Use Cases 4 Security and privacy considerations 5 API Description 5.1 The SpeechRecognition Interface 5.1.1 SpeechRecognition Attributes 5.1.2 SpeechRecognition Methods 5.1.3 SpeechRecognition Events 5.1.4 SpeechRecognitionError 5.1.5 SpeechRecognitionAlternative 5.1.6 SpeechRecognitionResult Hans Wennborg, Google Inc. Please refer to the errata for this document, which may include some normative corrections. Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable summary is available. Abstract This specification defines a JavaScript API to enable web developers to incorporate speech recognition and synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web pages to control activation and timing and to handle results and alternatives. Status of This Document This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA) other conditions apply. Learn more about W3C Community and Business Groups. All feedback is welcome. Table of Contents 1 Conformance requirements 2 Introduction 3 Use Cases 4 Security and privacy considerations 5 API Description 5.1 The SpeechRecognition Interface 5.1.1 SpeechRecognition Attributes 5.1.2 SpeechRecognition Methods 5.1.3 SpeechRecognition Events 5.1.4 SpeechRecognitionError 5.1.5 SpeechRecognitionAlternative 5.1.6 SpeechRecognitionResult
  66. 66. Hans Wennborg, Google Inc. Please refer to the errata for this document, which may include some normative corrections. Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable summary is available. Abstract This specification defines a JavaScript API to enable web developers to incorporate speech recognition and synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web pages to control activation and timing and to handle results and alternatives. Status of This Document This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA) other conditions apply. Learn more about W3C Community and Business Groups. All feedback is welcome. Table of Contents 1 Conformance requirements 2 Introduction 3 Use Cases 4 Security and privacy considerations 5 API Description 5.1 The SpeechRecognition Interface 5.1.1 SpeechRecognition Attributes 5.1.2 SpeechRecognition Methods 5.1.3 SpeechRecognition Events 5.1.4 SpeechRecognitionError 5.1.5 SpeechRecognitionAlternative 5.1.6 SpeechRecognitionResult Hans Wennborg, Google Inc. Please refer to the errata for this document, which may include some normative corrections. Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable summary is available. Abstract This specification defines a JavaScript API to enable web developers to incorporate speech recognition and synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web pages to control activation and timing and to handle results and alternatives. Status of This Document This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA) other conditions apply. Learn more about W3C Community and Business Groups. All feedback is welcome. Table of Contents 1 Conformance requirements 2 Introduction 3 Use Cases 4 Security and privacy considerations 5 API Description 5.1 The SpeechRecognition Interface 5.1.1 SpeechRecognition Attributes 5.1.2 SpeechRecognition Methods 5.1.3 SpeechRecognition Events 5.1.4 SpeechRecognitionError 5.1.5 SpeechRecognitionAlternative 5.1.6 SpeechRecognitionResult
  67. 67. Hans Wennborg, Google Inc. Please refer to the errata for this document, which may include some normative corrections. Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable summary is available. Abstract This specification defines a JavaScript API to enable web developers to incorporate speech recognition and synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web pages to control activation and timing and to handle results and alternatives. Status of This Document This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA) other conditions apply. Learn more about W3C Community and Business Groups. All feedback is welcome. Table of Contents 1 Conformance requirements 2 Introduction 3 Use Cases 4 Security and privacy considerations 5 API Description 5.1 The SpeechRecognition Interface 5.1.1 SpeechRecognition Attributes 5.1.2 SpeechRecognition Methods 5.1.3 SpeechRecognition Events 5.1.4 SpeechRecognitionError 5.1.5 SpeechRecognitionAlternative 5.1.6 SpeechRecognitionResult Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable summary is available. Abstract This specification defines a JavaScript API to enable web developers to incorporate speech recognition and synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web pages to control activation and timing and to handle results and alternatives. Status of This Document This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA) other conditions apply. Learn more about W3C Community and Business Groups. All feedback is welcome. Table of Contents 1 Conformance requirements 2 Introduction 3 Use Cases 4 Security and privacy considerations 5 API Description 5.1 The SpeechRecognition Interface 5.1.1 SpeechRecognition Attributes 5.1.2 SpeechRecognition Methods 5.1.3 SpeechRecognition Events
  68. 68. Hans Wennborg, Google Inc. Please refer to the errata for this document, which may include some normative corrections. Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable summary is available. Abstract This specification defines a JavaScript API to enable web developers to incorporate speech recognition and synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web pages to control activation and timing and to handle results and alternatives. Status of This Document This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA) other conditions apply. Learn more about W3C Community and Business Groups. All feedback is welcome. Table of Contents 1 Conformance requirements 2 Introduction 3 Use Cases 4 Security and privacy considerations 5 API Description 5.1 The SpeechRecognition Interface 5.1.1 SpeechRecognition Attributes 5.1.2 SpeechRecognition Methods 5.1.3 SpeechRecognition Events 5.1.4 SpeechRecognitionError 5.1.5 SpeechRecognitionAlternative 5.1.6 SpeechRecognitionResult Copyright © 2012 the Contributors to the Web Speech API Specification, published by the Speech API Community Group under the W3C Community Final Specification Agreement (FSA). A human-readable summary is available. Abstract This specification defines a JavaScript API to enable web developers to incorporate speech recognition and synthesis into their web pages. It enables developers to use scripting to generate text-to-speech output and to use speech recognition as an input for forms, continuous dictation and control. The JavaScript API allows web pages to control activation and timing and to handle results and alternatives. Status of This Document This specification was published by the Speech API Community Group. It is not a W3C Standard nor is it on the W3C Standards Track. Please note that under the W3C Community Final Specification Agreement (FSA) other conditions apply. Learn more about W3C Community and Business Groups. All feedback is welcome. Table of Contents 1 Conformance requirements 2 Introduction 3 Use Cases 4 Security and privacy considerations 5 API Description 5.1 The SpeechRecognition Interface 5.1.1 SpeechRecognition Attributes 5.1.2 SpeechRecognition Methods 5.1.3 SpeechRecognition Events
  69. 69. Special thanks to Eric Bidelman http://moreawesomeweb.com Speech Recognition API Support ?
  70. 70. Gestures?
  71. 71. Amazing, but too new to know what, if anything, this technology will mean for the web.
  72. 72. Let’s come back from the future and look at something much Dumber.Dumber
  73. 73. Dumber
  74. 74. Dumber
  75. 75. -pad remote controlsD
  76. 76. -pad remote controlsD
  77. 77. -pad remote controlsD
  78. 78. http://www.flickr.com/photos/stewc/6669743035/
  79. 79. http://www.flickr.com/photos/stewc/6669743035/ TVs browsers that support d-pad, send arrow key events.
  80. 80. http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/
  81. 81. If then http://www.wasdkeyboards.com/index.php/catalog/product/gallery/id/7164/image/343/
  82. 82. is undetectable. This is a recurring theme for input.
  83. 83. Sensors and camera http://www.flickr.com/photos/retrocactus/2170677056
  84. 84. Sensors and camera Camera http://www.flickr.com/photos/retrocactus/2170677056
  85. 85. GPS http://www.flickr.com/photos/3dking/149450434
  86. 86. GPS GeoLocation http://www.flickr.com/photos/3dking/149450434
  87. 87. Gyroscope & Accelerometer
  88. 88. http://www.flickr.com/photos/chrisjagers/4694134078
  89. 89. Back to today’s problems. http://www.flickr.com/photos/chrisjagers/4694134078
  90. 90. Hover state No hover state
  91. 91. Hover state Typing easier for many No hover state Typing often more difficult
  92. 92. Higher precision with mouse means smaller targets possible Hover state Less precise than mouse and requires larger touch targets Typing easier for many No hover state Typing often more difficult
  93. 93. Higher precision with mouse means smaller targets possible Hover state Less precise than mouse and requires larger touch targets Typing easier for many No hover state Typing often more difficult Right clicking and “power” tools Single and multi-touch gestures
  94. 94. http://www.flickr.com/photos/28096801@N05/5012309802
  95. 95. I got this. Detect touch. http://www.flickr.com/photos/28096801@N05/5012309802
  96. 96. Whatever you may think, it currently isn't possible to reliably detect whether or not the current device has a touchscreen, from within the browser. —Stu Cox, You Can’t Reliably Detect a Touch Screen http://www.stucox.com/blog/you-cant-detect-a-touchscreen/
  97. 97. Chrome would like to turn on touch events by default. https://code.google.com/p/chromium/issues/detail?id=159527 https://docs.google.com/a/cloudfour.com/presentation/d/1-n1qyzewpagREbzW2zm0wOalq33UhbtbSkWf9mEdly8/edit#slide=id.gc2d80e5b_171
  98. 98. Detect a mouse? Not reliably.
  99. 99. Surely we can detect a keyboard?
  100. 100. Surely we can detect a keyboard? NOPE
  101. 101. Input is dynamic.
  102. 102. Input is dynamic.
  103. 103. Boris Smus’s experiments responding to input.
  104. 104. Boris Smus’s experiments responding to input.
  105. 105. http://www.flickr.com/photos/lyza/7382235106 Maybe we need to be more zen about input.
  106. 106. After poking at this problem for a few weeks, my conclusion is: every desktop UI should be designed for touch now. When any desktop machine could have a touch interface, we have to proceed as if they all do. —Josh Clark http://globalmoxie.com/blog/desktop-touch-design.shtml
  107. 107. http://ie.microsoft.com/testdrive/ieblog/2011/Sep/20_TouchInputforIE10andMetrostyleApps_1.png http://www.w3.org/TR/pointerevents/ http://blog.webplatform.org/2013/02/pointing-toward-the-future/ New Pointer Events spec normalizes touch and mouse Pointer Events builds on the DOM event model to offer a new way to handle input on the web, enabling developers to build touch-first experiences that work with mouse, pen, and other pointing devices as well…They are also designed from the ground up to allow modern browsers to accelerate the touch-surface performance, leading to a smoother user experience.
  108. 108. What about those who won’t let go of their “power” interfaces? http://www.flickr.com/photos/ecos/4092571213/
  109. 109. http://www.flickr.com/photos/scarygami/5689980135/ One option: give them a choice.
  110. 110. Gmail display density settings
  111. 111. Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] –VIDEOS Vimeo Couch Mode
  112. 112. Couch Mode+ See allCentric TV’s videos / Recently viewed / Related videos Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago PROMOTED War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] –VIDEOS Vimeo Couch Mode
  113. 113. Couch Mode+ See allCentric TV’s videos / Recently viewed / Related videos Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago PROMOTED War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] –VIDEOS Couch Mode+ See allCentric TV’s videos / Recently viewed / Related videos Th Dream Experience - … Uploaded 2 years ago More Of The Dream Exp… Uploaded 2 years ago The Dream Experience -… Uploaded 2 years ago The Dream Experience … Uploaded 2 years ago The Love King Breaks It… Uploaded 2 years ago PROMOTED War Paint for Trees From Lincoln Motor Company Join Log In Create Watch Upload Search s [ ] –VIDEOS Vimeo Couch Mode
  114. 114. Vimeo Couch Mode
  115. 115. The key benefit of this approach: You’re designing for user need not for a specific form factor or input.
  116. 116. http://www.flickr.com/photos/raver_mikey/504815463
  117. 117. Progressive Input?
  118. 118. Graph from Chapter 1 of Adaptive Web Design by Aaron Gustafson http://easy-readers.net/books/adaptive-web-design/
  119. 119. Graph from Chapter 1 of Adaptive Web Design by Aaron Gustafson http://easy-readers.net/books/adaptive-web-design/ Progressive enhancement contains a value judgment
  120. 120. Who are we to judge what input is better? http://www.flickr.com/photos/fensterbme/4783366926
  121. 121. We need to learn to adapt. http://www.flickr.com/photos/cdm/147947664/
  122. 122. So that’s the puzzle that has been haunting me. http://www.flickr.com/photos/sravishankar/3460495
  123. 123. http://www.flickr.com/photos/wwworks/1384952210
  124. 124. Thank You!Special thanks to Luke Wroblewski, Eric Bidelman and Flickr users for generously sharing their photos under creative commons license.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×