Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Getting touchy - an introduction to touch and pointer events / Sainté Mobile Days / Saint-Etienne 23.11.2013

2,205 views

Published on

Published in: Technology
  • Be the first to comment

Getting touchy - an introduction to touch and pointer events / Sainté Mobile Days / Saint-Etienne 23.11.2013

  1. 1. getting touchy AN INTRODUCTION TO TOUCH AND POINTER EVENTS Patrick H. Lauke / Sainté Mobile Days / Saint-Etienne / 23 November 2013
  2. 2. 1. introduction 2. touch events 3. break 4. pointer events 5. debugging/testing 6. conclusion
  3. 3. patrickhlauke.github.io/touch
  4. 4. “how can I make my website work on touch devices?”
  5. 5. you don't need touch events browsers emulate regular mouse events
  6. 6. patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html
  7. 7. patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html
  8. 8. mouseover > mousemove* > mousedown > (focus) > mouseup > click *only a single “sacrificial” event is fired
  9. 9. on first tap mouseover > mousemove > mousedown > (focus) > mouseup > click subsequent taps mousemove > mousedown > (focus) > mouseup > click tapping away mouseout > (blur) focus/blur only on focusable elements in Opera Mobile and Firefox mouseout not on iOS Safari and embedded WebView (e.g. iOS Chrome)
  10. 10. emulation works, but is limiting/problematic
  11. 11. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  12. 12. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  13. 13. patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
  14. 14. patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
  15. 15. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  16. 16. steamcommunity.com/id/redux_splintered/screenshots/?appid=8870
  17. 17. vimeo.com/ondemand
  18. 18. patrickhlauke.github.io/touch/css-dropdown/mouse-only.html
  19. 19. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  20. 20. patrickhlauke.github.io/touch/particle/2
  21. 21. patrickhlauke.github.io/touch/particle/2 (bug in iOS7: moving finger does seem to scroll and fire multiple mousemove events?!)
  22. 22. “we need to go deeper...”
  23. 23. touch events introduced in iOS 2.0, adopted in Chrome/Firefox/Opera www.w3.org/TR/touch-events
  24. 24. touchstart touchmove touchend touchcancel touchenter touchleave
  25. 25. patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
  26. 26. patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
  27. 27. touchstart > [touchmove]+ > touchend > mouseover > mousemove > mousedown > (focus) > mouseup > click
  28. 28. on first tap touchstart > [touchmove]+ > touchend > mouseover > mousemove > mousedown > (focus) > mouseup > click subsequent taps touchstart > [touchmove]+ > touchend > mousemove > mousedown > (focus) > mouseup > click tapping away mouseout > (blur) too many touchmove events abort the tap (after touchend) not all browsers consistently send the touchmove some browsers outright weird...
  29. 29. Browser/Android 4.3 (AppleWebKit/534.30) mouseover > mousemove > touchstart > touchend > mousedown > mouseup > click Browser/Blackberry PlayBook 2.0 (AppleWebKit/536.2) touchstart > mouseover > mousemove > mousedown > touchend > mouseup > click
  30. 30. touch events vs limitations/problems
  31. 31. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  32. 32. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  33. 33. patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
  34. 34. patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
  35. 35. why the delay? double-tap to zoom (mostly anyway)
  36. 36. what if browsers didn't wait?
  37. 37. Puffin/Android (popular in Korea?) puffinbrowser.com double-tap zooms and fires mouse events + click (also, doesn't support touch events at all)
  38. 38. UCWeb/Android (popular in China?) ucweb.com no delay...but only because it doesn't implement double-tap to zoom
  39. 39. when does the delay happen?
  40. 40. patrickhlauke.github.io/touch/tests/event-listener.html
  41. 41. patrickhlauke.github.io/touch/tests/event-listener.html
  42. 42. “how can we make it feel responsive like a native app?”
  43. 43. react to events fired before the 300ms delay...
  44. 44. interlude: simple feature detection for touch events
  45. 45. if ('ontouchstart' in window) { /* some clever stuff here */ }
  46. 46. /* common performance “trick” */ var clickEvent = ('ontouchstart' in window ? 'touchend' : 'click'); blah.addEventListener(clickEvent, function() { ... }, false);
  47. 47. don't make it touch-exclusive
  48. 48. hacks.mozilla.org/2013/04/detecting-touch...
  49. 49. if ('ontouchstart' in window) { /* browser supports touch events but user is not necessarily using touch (exclusively) */ }
  50. 50. hybrid devices touch + mouse + keyboard
  51. 51. patrickhlauke.github.io/touch/tests/event-listener_show-delay-naive-event-fix.html
  52. 52. bugzilla.mozilla.org/show_bug.cgi?id=888304
  53. 53. Android + mouse – behaves like touch touchstart > touchend > mouseover > mousemove > mousedown > mouseup > click
  54. 54. Blackberry PlayBook 2.0 + mouse – like desktop mouse mouseover > mousedown > mousemove > mouseup > click
  55. 55. Android + keyboard – like desktop keyboard focus > click
  56. 56. VoiceOver enables keyboard access on iOS
  57. 57. iOS + keyboard – similar to touch focus > touchstart > touchend > mouseover > mousemove > mousedown blur > mouseup > click
  58. 58. iOS + VoiceOver – similar to touch focus > touchstart > touchend > mouseover > mousemove > mousedown blur > mouseup > click
  59. 59. Android + TalkBack – keyboard/mouse hybrid focus > blur > mousedown > mouseup > click > focus(?)
  60. 60. touch or mouse or keyboard
  61. 61. touch and mouse and keyboard
  62. 62. /* doubled-up event listeners */ foo.addEventListener('touchend', someFunction, false); foo.addEventListener('click', someFunction, false);
  63. 63. /* doubled-up event listeners */ foo.addEventListener('touchend', function(e) { /* prevent delay + mouse events */ e.preventDefault(); someFunction(); /* or even e.target.click(); */ }, false); foo.addEventListener('click', someFunction, false);
  64. 64. github.com/ftlabs/fastclick
  65. 65. preventDefault kills scrolling, long-press, pinch/zoom
  66. 66. browsers working to remove delay when possible
  67. 67. patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
  68. 68. patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
  69. 69. patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html
  70. 70. patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
  71. 71. patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
  72. 72. patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html
  73. 73. patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
  74. 74. patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html
  75. 75. Bug 922896 - Optimizations to remove 300ms touch > mouse events delay [RESOLVED - FIXED]
  76. 76. Issue 169642: Remove ~300ms delay on tap for mobile sites [...]
  77. 77. [...] no more 300ms click delay, or double-tap zoom, on mobile websites
  78. 78. patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
  79. 79. iOS/Safari designed themselves into a corner with “double-tap to scroll” bugs.webkit.org/show_bug.cgi?id=122212
  80. 80. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  81. 81. steamcommunity.com/id/redux_splintered/screenshots/?appid=8870
  82. 82. vimeo.com/ondemand
  83. 83. patrickhlauke.github.io/touch/css-dropdown/mouse-only.html
  84. 84. no isolated hover (or focus) on touch devices* * iOS fakes it, Samsung Galaxy S4 + stylus, ...
  85. 85. patrickhlauke.github.io/touch/css-dropdown/mouse-only.html
  86. 86. patrickhlauke.github.io/touch/css-dropdown/mouse-only.html
  87. 87. http://developer.apple.com/library/IOS/...
  88. 88. Modernizr issue 869: Detecting a mouse user
  89. 89. www.stucox.com/blog/you-cant-detect-a-touchscreen
  90. 90. avoid hover-based interfaces? complement for keyboard / touch!
  91. 91. patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard.html
  92. 92. patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard.html
  93. 93. patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard-touch.html
  94. 94. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  95. 95. patrickhlauke.github.io/touch/particle/2
  96. 96. patrickhlauke.github.io/touch/particle/2 (bug in iOS7: moving finger does seem to scroll and fire multiple mousemove events?!)
  97. 97. touchstart > [touchmove]+ > touchend > mouseover > mousemove* > mousedown > (focus) > mouseup > click *mouse event emulation fires only a single mousemove
  98. 98. doubling up handling of mousemove and touchmove
  99. 99. var posX, posY; ... function positionHandler(e) { posX = e.clientX; posY = e.clientY; } ... canvas.addEventListener('mousemove', positionHandler, false);
  100. 100. var posX, posY; ... function positionHandler(e) { /* handle both mouse and touch? */ } ... canvas.addEventListener('mousemove', positionHandler, false); canvas.addEventListener('touchmove', positionHandler, false);
  101. 101. interface MouseEvent : UIEvent { readonly attribute long screenX; readonly attribute long screenY; readonly attribute long clientX; readonly attribute long clientY; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute unsigned short button; readonly attribute EventTarget relatedTarget; void initMouseEvent(...); }; www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent
  102. 102. interface TouchEvent : UIEvent { readonly attribute TouchList touches; readonly attribute TouchList targetTouches; readonly attribute TouchList changedTouches; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; }; www.w3.org/TR/touch-events/#touchevent-interface
  103. 103. interface Touch { readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute }; long EventTarget long long long long long long identifier; target; screenX; screenY; clientX; clientY; pageX; pageY; www.w3.org/TR/touch-events/#touch-interface
  104. 104. var posX, posY; ... function positionHandler(e) { if ((e.clientX)&&(e.clientY)) { posX = e.clientX; posY = e.clientY; } else if (e.targetTouches) { posX = e.targetTouches[0].clientX; posY = e.targetTouches[0].clientY; e.preventDefault(); } } ... canvas.addEventListener('mousemove', positionHandler, false ); canvas.addEventListener('touchmove', positionHandler, false );
  105. 105. patrickhlauke.github.io/touch/particle/3
  106. 106. patrickhlauke.github.io/touch/particle/4
  107. 107. www.splintered.co.uk/experiments/archives/paranoid_0.5
  108. 108. tracking finger movement over time ... swipe gestures
  109. 109. patrickhlauke.github.io/touch/swipe
  110. 110. patrickhlauke.github.io/touch/swipe
  111. 111. patrickhlauke.github.io/touch/picture-slider
  112. 112. don't forget mouse/keyboard !
  113. 113. bradfrostweb.com/demo/mobile-first
  114. 114. touchmove fires...a lot!
  115. 115. do absolute minimum on each touchmove (usually: store new coordinates)
  116. 116. heavy JavaScript on requestAnimationFrame setInterval
  117. 117. patrickhlauke.github.io/touch/touch-limit
  118. 118. why stop at a single point? multitouch support
  119. 119. interface TouchEvent : UIEvent { readonly attribute TouchList touches; readonly attribute TouchList targetTouches; readonly attribute TouchList changedTouches; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; }; www.w3.org/TR/touch-events/#touchevent-interface
  120. 120. for (i=0; i<e.targetTouches.length; i++) { ... posX = e.targetTouches[i].clientX; posY = e.targetTouches[i].clientY; ... }
  121. 121. patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html
  122. 122. iOS/iPad even preventDefault() can't override 4-finger gestures
  123. 123. iOS7/Safari even preventDefault() can't override back/forward swipe gestures
  124. 124. multitouch gestures
  125. 125. /* iOS/Safari has gesture events for size/rotation, not supported in Chrome/Firefox/Opera, not part of the W3C Touch Events spec. */ gesturestart, gesturechange, gestureend e.scale, e.rotation
  126. 126. patrickhlauke.github.io/touch/iosgestures
  127. 127. patrickhlauke.github.io/touch/iosgestures/image.html
  128. 128. /* with some trigonometry we can replicate these from basic principles. */ var distance = Math.sqrt(Math.pow(...)+Math.pow(...)); var angle = Math.atan2(...);
  129. 129. patrickhlauke.github.io/touch/pinch-zoom-rotate
  130. 130. shinydemos.com/picture-organizer
  131. 131. interlude: older devices/OS versions and multitouch...
  132. 132. HTC Hero – Android 2.1
  133. 133. LG Optimus 2X – Android 2.3.7
  134. 134. touch events and Internet Explorer...
  135. 135. blogs.msdn.com/...
  136. 136. www.w3.org/TR/pointerevents
  137. 137. not just some “not invented here” new approach for IE10+
  138. 138. html5labs.interoperabilitybridges.com/prototypes/...
  139. 139. code.google.com/p/chromium/issues/detail?id=162757
  140. 140. bugzilla.mozilla.org/show_bug.cgi?id=822898
  141. 141. ...what about Apple?
  142. 142. bugs.webkit.org/show_bug.cgi?id=105463 (no real activity since April 2013?)
  143. 143. patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
  144. 144. patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
  145. 145. pointerover > mouseover > pointerenter > mouseenter > pointerdown > mousedown > pointermove > mousemove > focus > pointerup > mouseup > pointerout > mouseout > pointerleave > mouseleave > click mouse events fired “inline” with pointer events (for the primary contact, e.g. first finger on screen)
  146. 146. gotpointercapture lostpointercapture
  147. 147. vendor-prefixed in IE10 MSPointerDown MSPointerMove MSPointerUp navigator.msPointerEnabled navigator.msMaxTouchPoints -ms-touch-action unprefixed in IE11
  148. 148. interface MouseEvent : UIEvent { readonly attribute long screenX; readonly attribute long screenY; readonly attribute long clientX; readonly attribute long clientY; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute unsigned short button; readonly attribute EventTarget relatedTarget; void initMouseEvent(...); }; www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent
  149. 149. /* Pointer Events extend Mouse Events vs Touch Events and their completely new objects/arrays */ interface PointerEvent : readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute }; MouseEvent { long pointerId; long width; long height; float pressure; long tiltX; long tiltY; DOMString pointerType; boolean isPrimary; www.w3.org/TR/pointerevents/#pointerevent-interface
  150. 150. simple feature detection for pointer events
  151. 151. if (navigator.pointerEnabled) { /* some clever stuff here but this covers touch, stylus, mouse, etc */ } /* still listen to click for keyboard! */
  152. 152. if (navigator.maxTouchPoints > 1) { /* multitouch-capable device */ }
  153. 153. patrickhlauke.github.io/touch/tests/pointer-multitouch-detect.html
  154. 154. pointer events vs limitations/problems of mouse event emulation
  155. 155. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  156. 156. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  157. 157. patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
  158. 158. patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
  159. 159. patrickhlauke.github.io/touch/tests/event-listener.html
  160. 160. patrickhlauke.github.io/touch/tests/event-listener.html
  161. 161. patrickhlauke.github.io/touch/tests/event-listener.html
  162. 162. “how can we make it feel responsive like a native app?”
  163. 163. preventDefault() won't work (but you can prevent most mouse compatibility events by cancelling pointerdown)
  164. 164. touch-action: auto|none|[pan-x][pan-y] www.w3.org/TR/pointerevents/#the-touch-action-css-property only prevents default touch action (e.g. double-tap to zoom) does not stop synthetic mouse events nor click
  165. 165. touch-action:none kills scrolling, long-press, pinch/zoom
  166. 166. touch-action:none
  167. 167. http://patrickhlauke.github.io/touch/tests/event-listener_touch-action-none.html
  168. 168. http://patrickhlauke.github.io/touch/tests/event-listener_touch-action-none.html
  169. 169. pointerover > mouseover > pointerenter > mouseenter > pointerdown > mousedown > pointermove > mousemove > focus > pointerup > mouseup > click > pointerout > mouseout > pointerleave > mouseleave
  170. 170. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  171. 171. msdn.microsoft.com/en-us/library/ie/dn265029(v=vs.85).aspx
  172. 172. msdn.microsoft.com/en-us/library/ie/jj152135(v=vs.85).aspx
  173. 173. patrickhlauke.github.io/touch/css-dropdown/mouse-only-aria-haspopup.html
  174. 174. channel9.msdn.com/Blogs/IE/IE11-Using-Hover-Menus-with-Touch
  175. 175. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  176. 176. patrickhlauke.github.io/touch/particle/2
  177. 177. touch-action:none
  178. 178. patrickhlauke.github.io/touch/particle/2a
  179. 179. what about multitouch?
  180. 180. patrickhlauke.github.io/touch/tracker/mouse-tracker_touch-action-none.html
  181. 181. /* PointerEvents don't have the handy touch arrays, so we have to replicate something similar... */
  182. 182. /* PointerEvents don't have the handy touch arrays, so we have to replicate something similar... */ var points = []; switch (e.type) { case 'pointerdown': /* add to the array */ break; case 'pointermove': /* update the relevant array entry's x and y */ break; case 'pointerup': /* remove the relevant array entry */ break; }
  183. 183. patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer.html
  184. 184. /* like iOS/Safari, IE10/Win has higher-level gestures, but these are not part of the W3C Pointer Events spec. Replicate these from basic principles again? */
  185. 185. www.exploretouch.ie/behind-the-scenes
  186. 186. pointer events as the future?
  187. 187. what about backwardscompatibility?
  188. 188. touchstart > [touchmove]+ > touchend > [300ms delay] > mouseover > mousemove > mousedown > mouseup > click + pointerover > mouseover > pointerdown > mousedown > pointermove > mousemove > pointerup > mouseup > pointerout > mouseout > [300ms delay] > click
  189. 189. www.w3.org/community/touchevents
  190. 190. polyfills for pointer events (code for tomorrow, today)
  191. 191. handjs.codeplex.com
  192. 192. www.catuhe.com/msdn/handjs
  193. 193. github.com/Polymer/PointerEvents
  194. 194. www.polymer-project.org
  195. 195. utility libraries (because life is too short to hand-code gesture support etc)
  196. 196. eightmedia.github.io/hammer.js
  197. 197. jQuery Mobile? Sencha Touch? … check for support of IE10+ and “this is a touch device” assumptions
  198. 198. debugging/testing
  199. 199. Issue 181204: Emulate touch events - event order different from real devices
  200. 200. Bug 920956 - DevTools touch emulation: suppress regular mouse events ...
  201. 201. developers.google.com/chrome-developer-tools/docs/console#monitoring_events
  202. 202. Bug 861876 - [...] multiple mousemoves being fired
  203. 203. further reading...
  204. 204. Matt Gaunt – Touch Feedback for Mobile Sites www.gauntface.co.uk/blog/2013/06/25/touch-feedback-for-mobile-sites Jonathan Stark – FastActive github.com/jonathanstark/FastActive Stephen Woods – HTML5 Touch Interfaces www.slideshare.net/ysaw/html5-touch-interfaces-sxsw-extended-version David Rousset – Unifying touch and mouse: how Pointer Events will make cross-browsers touch support easy blogs.msdn.com/b/davrous/archive/2013/02/20/handling-touch[...] Chris Wilson + Paul Kinlan – Touch And Mouse: Together Again For The First Time www.html5rocks.com/en/mobile/touchandmouse Patrick H. Lauke – Webseiten zum Anfassen webkrauts.de/artikel/2012/einfuehrung-touch-events Ryan Fioravanti – Creating Fast Buttons for Mobile Web Applications developers.google.com/mobile/articles/fast_buttons Boris Smus – Multi-touch Web Development www.html5rocks.com/en/mobile/touch Boris Smus – Generalized input on the cross-device web smus.com/mouse-touch-pointer
  205. 205. Rick Byers + Boris Smus (Google I/O) – Point, Click, Tap, Touch - Building Multi-Device Web Interfaces developers.google.com/events/io/sessions/361772634 Grant Goodale – Touch Events www.w3.org/conf/2011/#Touch_Events W3C – Touch Events Extensions www.w3.org/TR/2013/NOTE-touch-events-extensions-20131031 Mozilla Developer Network – Touch Events developer.mozilla.org/en-US/docs/Web/Guide/API/DOM/Events/Touch_events WebPlatform.org – Pointer Events docs.webplatform.org/wiki/concepts/Pointer_Events Rick Byers – The best way to avoid the dreaded 300ms click delay is to disable double-tap zoom plus.google.com/+RickByers/posts/ej7nsuoaaDa Tim Kadlec – Avoiding the 300ms Click Delay, Accessibly timkadlec.com/2013/11/Avoiding-the-300ms-click-delay-accessibly Microsoft – Pointer events updates (differences between IE10 and IE11) msdn.microsoft.com/en-us/library/ie/dn304886(v=vs.85).aspx
  206. 206. youtube.com/watch?v=AZKpByV5764
  207. 207. merci @patrick_h_lauke github.com/patrickhlauke/touch slideshare.net/redux paciellogroup.com splintered.co.uk

×