Your SlideShare is downloading. ×
Getting touchy - an introduction to touch and pointer events / Sainté Mobile Days / Saint-Etienne 23.11.2013
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Getting touchy - an introduction to touch and pointer events / Sainté Mobile Days / Saint-Etienne 23.11.2013

1,319
views

Published on

Published in: Technology

0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,319
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
10
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. getting touchy AN INTRODUCTION TO TOUCH AND POINTER EVENTS Patrick H. Lauke / Sainté Mobile Days / Saint-Etienne / 23 November 2013
  • 2. 1. introduction 2. touch events 3. break 4. pointer events 5. debugging/testing 6. conclusion
  • 3. patrickhlauke.github.io/touch
  • 4. “how can I make my website work on touch devices?”
  • 5. you don't need touch events browsers emulate regular mouse events
  • 6. patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html
  • 7. patrickhlauke.github.io/touch/tests/event-listener_mouse-only.html
  • 8. mouseover > mousemove* > mousedown > (focus) > mouseup > click *only a single “sacrificial” event is fired
  • 9. on first tap mouseover > mousemove > mousedown > (focus) > mouseup > click subsequent taps mousemove > mousedown > (focus) > mouseup > click tapping away mouseout > (blur) focus/blur only on focusable elements in Opera Mobile and Firefox mouseout not on iOS Safari and embedded WebView (e.g. iOS Chrome)
  • 10. emulation works, but is limiting/problematic
  • 11. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  • 12. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  • 13. patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
  • 14. patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
  • 15. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  • 16. steamcommunity.com/id/redux_splintered/screenshots/?appid=8870
  • 17. vimeo.com/ondemand
  • 18. patrickhlauke.github.io/touch/css-dropdown/mouse-only.html
  • 19. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  • 20. patrickhlauke.github.io/touch/particle/2
  • 21. patrickhlauke.github.io/touch/particle/2 (bug in iOS7: moving finger does seem to scroll and fire multiple mousemove events?!)
  • 22. “we need to go deeper...”
  • 23. touch events introduced in iOS 2.0, adopted in Chrome/Firefox/Opera www.w3.org/TR/touch-events
  • 24. touchstart touchmove touchend touchcancel touchenter touchleave
  • 25. patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
  • 26. patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
  • 27. touchstart > [touchmove]+ > touchend > mouseover > mousemove > mousedown > (focus) > mouseup > click
  • 28. on first tap touchstart > [touchmove]+ > touchend > mouseover > mousemove > mousedown > (focus) > mouseup > click subsequent taps touchstart > [touchmove]+ > touchend > mousemove > mousedown > (focus) > mouseup > click tapping away mouseout > (blur) too many touchmove events abort the tap (after touchend) not all browsers consistently send the touchmove some browsers outright weird...
  • 29. Browser/Android 4.3 (AppleWebKit/534.30) mouseover > mousemove > touchstart > touchend > mousedown > mouseup > click Browser/Blackberry PlayBook 2.0 (AppleWebKit/536.2) touchstart > mouseover > mousemove > mousedown > touchend > mouseup > click
  • 30. touch events vs limitations/problems
  • 31. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  • 32. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  • 33. patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
  • 34. patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
  • 35. why the delay? double-tap to zoom (mostly anyway)
  • 36. what if browsers didn't wait?
  • 37. Puffin/Android (popular in Korea?) puffinbrowser.com double-tap zooms and fires mouse events + click (also, doesn't support touch events at all)
  • 38. UCWeb/Android (popular in China?) ucweb.com no delay...but only because it doesn't implement double-tap to zoom
  • 39. when does the delay happen?
  • 40. patrickhlauke.github.io/touch/tests/event-listener.html
  • 41. patrickhlauke.github.io/touch/tests/event-listener.html
  • 42. “how can we make it feel responsive like a native app?”
  • 43. react to events fired before the 300ms delay...
  • 44. interlude: simple feature detection for touch events
  • 45. if ('ontouchstart' in window) { /* some clever stuff here */ }
  • 46. /* common performance “trick” */ var clickEvent = ('ontouchstart' in window ? 'touchend' : 'click'); blah.addEventListener(clickEvent, function() { ... }, false);
  • 47. don't make it touch-exclusive
  • 48. hacks.mozilla.org/2013/04/detecting-touch...
  • 49. if ('ontouchstart' in window) { /* browser supports touch events but user is not necessarily using touch (exclusively) */ }
  • 50. hybrid devices touch + mouse + keyboard
  • 51. patrickhlauke.github.io/touch/tests/event-listener_show-delay-naive-event-fix.html
  • 52. bugzilla.mozilla.org/show_bug.cgi?id=888304
  • 53. Android + mouse – behaves like touch touchstart > touchend > mouseover > mousemove > mousedown > mouseup > click
  • 54. Blackberry PlayBook 2.0 + mouse – like desktop mouse mouseover > mousedown > mousemove > mouseup > click
  • 55. Android + keyboard – like desktop keyboard focus > click
  • 56. VoiceOver enables keyboard access on iOS
  • 57. iOS + keyboard – similar to touch focus > touchstart > touchend > mouseover > mousemove > mousedown blur > mouseup > click
  • 58. iOS + VoiceOver – similar to touch focus > touchstart > touchend > mouseover > mousemove > mousedown blur > mouseup > click
  • 59. Android + TalkBack – keyboard/mouse hybrid focus > blur > mousedown > mouseup > click > focus(?)
  • 60. touch or mouse or keyboard
  • 61. touch and mouse and keyboard
  • 62. /* doubled-up event listeners */ foo.addEventListener('touchend', someFunction, false); foo.addEventListener('click', someFunction, false);
  • 63. /* doubled-up event listeners */ foo.addEventListener('touchend', function(e) { /* prevent delay + mouse events */ e.preventDefault(); someFunction(); /* or even e.target.click(); */ }, false); foo.addEventListener('click', someFunction, false);
  • 64. github.com/ftlabs/fastclick
  • 65. preventDefault kills scrolling, long-press, pinch/zoom
  • 66. browsers working to remove delay when possible
  • 67. patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
  • 68. patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
  • 69. patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html
  • 70. patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
  • 71. patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
  • 72. patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html
  • 73. patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
  • 74. patrickhlauke.github.io/touch/tests/event-listener_minimum-maximum-scale.html
  • 75. Bug 922896 - Optimizations to remove 300ms touch > mouse events delay [RESOLVED - FIXED]
  • 76. Issue 169642: Remove ~300ms delay on tap for mobile sites [...]
  • 77. [...] no more 300ms click delay, or double-tap zoom, on mobile websites
  • 78. patrickhlauke.github.io/touch/tests/event-listener_user-scalable-no.html
  • 79. iOS/Safari designed themselves into a corner with “double-tap to scroll” bugs.webkit.org/show_bug.cgi?id=122212
  • 80. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  • 81. steamcommunity.com/id/redux_splintered/screenshots/?appid=8870
  • 82. vimeo.com/ondemand
  • 83. patrickhlauke.github.io/touch/css-dropdown/mouse-only.html
  • 84. no isolated hover (or focus) on touch devices* * iOS fakes it, Samsung Galaxy S4 + stylus, ...
  • 85. patrickhlauke.github.io/touch/css-dropdown/mouse-only.html
  • 86. patrickhlauke.github.io/touch/css-dropdown/mouse-only.html
  • 87. http://developer.apple.com/library/IOS/...
  • 88. Modernizr issue 869: Detecting a mouse user
  • 89. www.stucox.com/blog/you-cant-detect-a-touchscreen
  • 90. avoid hover-based interfaces? complement for keyboard / touch!
  • 91. patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard.html
  • 92. patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard.html
  • 93. patrickhlauke.github.io/touch/css-dropdown/mouse-keyboard-touch.html
  • 94. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  • 95. patrickhlauke.github.io/touch/particle/2
  • 96. patrickhlauke.github.io/touch/particle/2 (bug in iOS7: moving finger does seem to scroll and fire multiple mousemove events?!)
  • 97. touchstart > [touchmove]+ > touchend > mouseover > mousemove* > mousedown > (focus) > mouseup > click *mouse event emulation fires only a single mousemove
  • 98. doubling up handling of mousemove and touchmove
  • 99. var posX, posY; ... function positionHandler(e) { posX = e.clientX; posY = e.clientY; } ... canvas.addEventListener('mousemove', positionHandler, false);
  • 100. var posX, posY; ... function positionHandler(e) { /* handle both mouse and touch? */ } ... canvas.addEventListener('mousemove', positionHandler, false); canvas.addEventListener('touchmove', positionHandler, false);
  • 101. interface MouseEvent : UIEvent { readonly attribute long screenX; readonly attribute long screenY; readonly attribute long clientX; readonly attribute long clientY; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute unsigned short button; readonly attribute EventTarget relatedTarget; void initMouseEvent(...); }; www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent
  • 102. interface TouchEvent : UIEvent { readonly attribute TouchList touches; readonly attribute TouchList targetTouches; readonly attribute TouchList changedTouches; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; }; www.w3.org/TR/touch-events/#touchevent-interface
  • 103. interface Touch { readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute }; long EventTarget long long long long long long identifier; target; screenX; screenY; clientX; clientY; pageX; pageY; www.w3.org/TR/touch-events/#touch-interface
  • 104. var posX, posY; ... function positionHandler(e) { if ((e.clientX)&&(e.clientY)) { posX = e.clientX; posY = e.clientY; } else if (e.targetTouches) { posX = e.targetTouches[0].clientX; posY = e.targetTouches[0].clientY; e.preventDefault(); } } ... canvas.addEventListener('mousemove', positionHandler, false ); canvas.addEventListener('touchmove', positionHandler, false );
  • 105. patrickhlauke.github.io/touch/particle/3
  • 106. patrickhlauke.github.io/touch/particle/4
  • 107. www.splintered.co.uk/experiments/archives/paranoid_0.5
  • 108. tracking finger movement over time ... swipe gestures
  • 109. patrickhlauke.github.io/touch/swipe
  • 110. patrickhlauke.github.io/touch/swipe
  • 111. patrickhlauke.github.io/touch/picture-slider
  • 112. don't forget mouse/keyboard !
  • 113. bradfrostweb.com/demo/mobile-first
  • 114. touchmove fires...a lot!
  • 115. do absolute minimum on each touchmove (usually: store new coordinates)
  • 116. heavy JavaScript on requestAnimationFrame setInterval
  • 117. patrickhlauke.github.io/touch/touch-limit
  • 118. why stop at a single point? multitouch support
  • 119. interface TouchEvent : UIEvent { readonly attribute TouchList touches; readonly attribute TouchList targetTouches; readonly attribute TouchList changedTouches; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; }; www.w3.org/TR/touch-events/#touchevent-interface
  • 120. for (i=0; i<e.targetTouches.length; i++) { ... posX = e.targetTouches[i].clientX; posY = e.targetTouches[i].clientY; ... }
  • 121. patrickhlauke.github.io/touch/tracker/multi-touch-tracker.html
  • 122. iOS/iPad even preventDefault() can't override 4-finger gestures
  • 123. iOS7/Safari even preventDefault() can't override back/forward swipe gestures
  • 124. multitouch gestures
  • 125. /* iOS/Safari has gesture events for size/rotation, not supported in Chrome/Firefox/Opera, not part of the W3C Touch Events spec. */ gesturestart, gesturechange, gestureend e.scale, e.rotation
  • 126. patrickhlauke.github.io/touch/iosgestures
  • 127. patrickhlauke.github.io/touch/iosgestures/image.html
  • 128. /* with some trigonometry we can replicate these from basic principles. */ var distance = Math.sqrt(Math.pow(...)+Math.pow(...)); var angle = Math.atan2(...);
  • 129. patrickhlauke.github.io/touch/pinch-zoom-rotate
  • 130. shinydemos.com/picture-organizer
  • 131. interlude: older devices/OS versions and multitouch...
  • 132. HTC Hero – Android 2.1
  • 133. LG Optimus 2X – Android 2.3.7
  • 134. touch events and Internet Explorer...
  • 135. blogs.msdn.com/...
  • 136. www.w3.org/TR/pointerevents
  • 137. not just some “not invented here” new approach for IE10+
  • 138. html5labs.interoperabilitybridges.com/prototypes/...
  • 139. code.google.com/p/chromium/issues/detail?id=162757
  • 140. bugzilla.mozilla.org/show_bug.cgi?id=822898
  • 141. ...what about Apple?
  • 142. bugs.webkit.org/show_bug.cgi?id=105463 (no real activity since April 2013?)
  • 143. patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
  • 144. patrickhlauke.github.io/touch/tests/event-listener_all-no-timings.html
  • 145. pointerover > mouseover > pointerenter > mouseenter > pointerdown > mousedown > pointermove > mousemove > focus > pointerup > mouseup > pointerout > mouseout > pointerleave > mouseleave > click mouse events fired “inline” with pointer events (for the primary contact, e.g. first finger on screen)
  • 146. gotpointercapture lostpointercapture
  • 147. vendor-prefixed in IE10 MSPointerDown MSPointerMove MSPointerUp navigator.msPointerEnabled navigator.msMaxTouchPoints -ms-touch-action unprefixed in IE11
  • 148. interface MouseEvent : UIEvent { readonly attribute long screenX; readonly attribute long screenY; readonly attribute long clientX; readonly attribute long clientY; readonly attribute boolean ctrlKey; readonly attribute boolean shiftKey; readonly attribute boolean altKey; readonly attribute boolean metaKey; readonly attribute unsigned short button; readonly attribute EventTarget relatedTarget; void initMouseEvent(...); }; www.w3.org/TR/DOM-Level-2-Events/events.html#Events-MouseEvent
  • 149. /* Pointer Events extend Mouse Events vs Touch Events and their completely new objects/arrays */ interface PointerEvent : readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute readonly attribute }; MouseEvent { long pointerId; long width; long height; float pressure; long tiltX; long tiltY; DOMString pointerType; boolean isPrimary; www.w3.org/TR/pointerevents/#pointerevent-interface
  • 150. simple feature detection for pointer events
  • 151. if (navigator.pointerEnabled) { /* some clever stuff here but this covers touch, stylus, mouse, etc */ } /* still listen to click for keyboard! */
  • 152. if (navigator.maxTouchPoints > 1) { /* multitouch-capable device */ }
  • 153. patrickhlauke.github.io/touch/tests/pointer-multitouch-detect.html
  • 154. pointer events vs limitations/problems of mouse event emulation
  • 155. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  • 156. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  • 157. patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
  • 158. patrickhlauke.github.io/touch/tests/event-listener_show-delay.html
  • 159. patrickhlauke.github.io/touch/tests/event-listener.html
  • 160. patrickhlauke.github.io/touch/tests/event-listener.html
  • 161. patrickhlauke.github.io/touch/tests/event-listener.html
  • 162. “how can we make it feel responsive like a native app?”
  • 163. preventDefault() won't work (but you can prevent most mouse compatibility events by cancelling pointerdown)
  • 164. touch-action: auto|none|[pan-x][pan-y] www.w3.org/TR/pointerevents/#the-touch-action-css-property only prevents default touch action (e.g. double-tap to zoom) does not stop synthetic mouse events nor click
  • 165. touch-action:none kills scrolling, long-press, pinch/zoom
  • 166. touch-action:none
  • 167. http://patrickhlauke.github.io/touch/tests/event-listener_touch-action-none.html
  • 168. http://patrickhlauke.github.io/touch/tests/event-listener_touch-action-none.html
  • 169. pointerover > mouseover > pointerenter > mouseenter > pointerdown > mousedown > pointermove > mousemove > focus > pointerup > mouseup > click > pointerout > mouseout > pointerleave > mouseleave
  • 170. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  • 171. msdn.microsoft.com/en-us/library/ie/dn265029(v=vs.85).aspx
  • 172. msdn.microsoft.com/en-us/library/ie/jj152135(v=vs.85).aspx
  • 173. patrickhlauke.github.io/touch/css-dropdown/mouse-only-aria-haspopup.html
  • 174. channel9.msdn.com/Blogs/IE/IE11-Using-Hover-Menus-with-Touch
  • 175. 1. delayed event dispatch 2. mouse-specific interfaces 3. mousemove doesn't track
  • 176. patrickhlauke.github.io/touch/particle/2
  • 177. touch-action:none
  • 178. patrickhlauke.github.io/touch/particle/2a
  • 179. what about multitouch?
  • 180. patrickhlauke.github.io/touch/tracker/mouse-tracker_touch-action-none.html
  • 181. /* PointerEvents don't have the handy touch arrays, so we have to replicate something similar... */
  • 182. /* PointerEvents don't have the handy touch arrays, so we have to replicate something similar... */ var points = []; switch (e.type) { case 'pointerdown': /* add to the array */ break; case 'pointermove': /* update the relevant array entry's x and y */ break; case 'pointerup': /* remove the relevant array entry */ break; }
  • 183. patrickhlauke.github.io/touch/tracker/multi-touch-tracker-pointer.html
  • 184. /* like iOS/Safari, IE10/Win has higher-level gestures, but these are not part of the W3C Pointer Events spec. Replicate these from basic principles again? */
  • 185. www.exploretouch.ie/behind-the-scenes
  • 186. pointer events as the future?
  • 187. what about backwardscompatibility?
  • 188. touchstart > [touchmove]+ > touchend > [300ms delay] > mouseover > mousemove > mousedown > mouseup > click + pointerover > mouseover > pointerdown > mousedown > pointermove > mousemove > pointerup > mouseup > pointerout > mouseout > [300ms delay] > click
  • 189. www.w3.org/community/touchevents
  • 190. polyfills for pointer events (code for tomorrow, today)
  • 191. handjs.codeplex.com
  • 192. www.catuhe.com/msdn/handjs
  • 193. github.com/Polymer/PointerEvents
  • 194. www.polymer-project.org
  • 195. utility libraries (because life is too short to hand-code gesture support etc)
  • 196. eightmedia.github.io/hammer.js
  • 197. jQuery Mobile? Sencha Touch? … check for support of IE10+ and “this is a touch device” assumptions
  • 198. debugging/testing
  • 199. Issue 181204: Emulate touch events - event order different from real devices
  • 200. Bug 920956 - DevTools touch emulation: suppress regular mouse events ...
  • 201. developers.google.com/chrome-developer-tools/docs/console#monitoring_events
  • 202. Bug 861876 - [...] multiple mousemoves being fired
  • 203. further reading...
  • 204. Matt Gaunt – Touch Feedback for Mobile Sites www.gauntface.co.uk/blog/2013/06/25/touch-feedback-for-mobile-sites Jonathan Stark – FastActive github.com/jonathanstark/FastActive Stephen Woods – HTML5 Touch Interfaces www.slideshare.net/ysaw/html5-touch-interfaces-sxsw-extended-version David Rousset – Unifying touch and mouse: how Pointer Events will make cross-browsers touch support easy blogs.msdn.com/b/davrous/archive/2013/02/20/handling-touch[...] Chris Wilson + Paul Kinlan – Touch And Mouse: Together Again For The First Time www.html5rocks.com/en/mobile/touchandmouse Patrick H. Lauke – Webseiten zum Anfassen webkrauts.de/artikel/2012/einfuehrung-touch-events Ryan Fioravanti – Creating Fast Buttons for Mobile Web Applications developers.google.com/mobile/articles/fast_buttons Boris Smus – Multi-touch Web Development www.html5rocks.com/en/mobile/touch Boris Smus – Generalized input on the cross-device web smus.com/mouse-touch-pointer
  • 205. Rick Byers + Boris Smus (Google I/O) – Point, Click, Tap, Touch - Building Multi-Device Web Interfaces developers.google.com/events/io/sessions/361772634 Grant Goodale – Touch Events www.w3.org/conf/2011/#Touch_Events W3C – Touch Events Extensions www.w3.org/TR/2013/NOTE-touch-events-extensions-20131031 Mozilla Developer Network – Touch Events developer.mozilla.org/en-US/docs/Web/Guide/API/DOM/Events/Touch_events WebPlatform.org – Pointer Events docs.webplatform.org/wiki/concepts/Pointer_Events Rick Byers – The best way to avoid the dreaded 300ms click delay is to disable double-tap zoom plus.google.com/+RickByers/posts/ej7nsuoaaDa Tim Kadlec – Avoiding the 300ms Click Delay, Accessibly timkadlec.com/2013/11/Avoiding-the-300ms-click-delay-accessibly Microsoft – Pointer events updates (differences between IE10 and IE11) msdn.microsoft.com/en-us/library/ie/dn304886(v=vs.85).aspx
  • 206. youtube.com/watch?v=AZKpByV5764
  • 207. merci @patrick_h_lauke github.com/patrickhlauke/touch slideshare.net/redux paciellogroup.com splintered.co.uk