Creating ResponsiveHTML5 TouchInterfacesStephen Woods
Stephen WoodsFront End Engineer       Flickr
On the desktop weworry about browsers-moz-transform:rotate(-270deg);-moz-transform-origin: bottom left;-webkit-transform: ...
On mobile we worryabout devices.
ScreenSizesMedia Queries,Break points,liquid layoutshttp://www.alistapart.com/articles/responsive-web-design/
iPhone 3GS 256mb RAMGeekbench: 271
iPhone 3GS 256mb RAMGeekbench: 271      ==
Modern mobiledevices are crappy  computers withdecent video cards.
PerceivedPerformance
On the desktop it’seasy...  Throw up a spinner.
Touch interfaces aretactile.
Touch interfaces aretactile.   Feedback must be        continuous.
When the interface  stops movingduring a gesture it feels like it died
Respect Convention
Mobile hasconventionstoo
Mobile hasconventionstoo
TouchEvent• touchstart - fires once• touchmove - fires continuously• touchend - fires once
The touches Array• You only get one on Android• You get up to 11 on iOS• Each touch gives you position  information, and s...
iOS Gesture Events• gesturestart• gesturechange• gestureend
iOS Developer      Libraryhttp://bit.ly/iOS-guide
Making Gestures Work • Prioritize user feedback • Use hardware acceleration • Manage your memory
Prioritize User-feedback• Don’t do any loading during gestures• Treat the DOM as write-only (do your  own math)• When at a...
Write-Only DOM• DOM touches are really expensive• You know where everything is• Use matrix transforms to queue up  positions
Swipe Basicsdistance = e.touches[0].pageX - startX;translate3d(+distance+px,0px,0px)
Snap back/snap      forward• Keep track of last position, use  transitions with easing to snap back• Pick a swipe distance...
A Word about scrolling• Use native if at all possible:• -webkit-overflow-scrolling:      touch;• If not, use a library to ...
Image © Brian LalorUsed with permission   Pinch to Zoom   (there will be math)
Why you can’t use native Pinch to      Zoom
First:Use Matrix TransformsMinimize DOM touches, make your transformssimpler in the long run
http://xkcd.com/184/
It’s Not That Hard!transform:          Translatematrix(1, 0, 0, 1, 10, 10);         Scale
With HardwareAcceleration... (matrix3d)            [                [1,0,0,0],                [0,1,0,0],                [0...
Transforms keep  complex statewithout DOM reads
What is happening?• Determine Center of the touch points• Determine the scale factor  (touch.scale)• Scale the element by ...
The Naive Example
The Naive Example
The Naive Example
The Right Example
The Right Example
The Right Example
Breakdown
Breakdown
Breakdown
Breakdown
translateX =scalePointX * (newWidth - oldWidth)            newWidth;
Pro Tips• Beware the virtual pixels• Moving the transform-origin won’t  really work• Remember to snap back
The Flickr Touch   Light Box
UntitledBy protohiro
UntitledBy protohiro
UntitledBy protohiro
UntitledBy protohiro
Swiping Process• Event Listener on top level for touch  events• Only visible nodes move via  translate3d• Rebuild next/pre...
Performance Tricks• Aggressive Pruning• Clean off css transforms/transitions• Write-only DOM.• Do as little as possible du...
Frustrating         Limitations• Retina screen is huge, device  memory is small• Hardware acceleration is a crash  festiva...
Stephen Woods                                  @ysawImage Credits (http://flic.kr/y/kQ5cLh)http://www.flickr.com/photos/wafe...
Creating Responsive HTML5 Touch Interfaces
Creating Responsive HTML5 Touch Interfaces
Creating Responsive HTML5 Touch Interfaces
Upcoming SlideShare
Loading in...5
×

Creating Responsive HTML5 Touch Interfaces

28,977

Published on

Published in: Technology
0 Comments
52 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
28,977
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
369
Comments
0
Likes
52
Embeds 0
No embeds

No notes for slide
  • \n
  • I built the mobile lightbox (on a great backbone from the flickr FE team)\n
  • The flickr mobile lightbox. Swiping, zooming, pretty smooth, very natural for the device.\nI’m going to talk about how we made it, but I am going to talk a lot of concepts that came out of the work first\n
  • \n
  • They all run webkit, they all have css3, transforms, etc.\nYou’ll have to do a lot of testing on screen sizes of course...but..\n
  • People have covered this topic very well elsewhere\n
  • \n
  • \n
  • \n
  • but this is going to change...\n
  • An awesome experience is possible.\n
  • \n
  • \n
  • Remember the teleporter control in stng? It had a control like in the original series, but touch\nIt needs to feel right, that means immediate feedback is continuous, rather than momentary.\n
  • The rational for snap back and momentum scroll (always give feedback)\n\n
  • We all know the conventions of desktop software\nMost people respect it when building interfaces\n
  • Slide to unlock, swipe to go forward and back, pinch to zoom, tap and hold\ni) don't break what users expect: swipes & Pinches\nexample: user reaction to m.flickr lack of pinch zoom\n
  • \n
  • Ice cream sandwich caveat noted.\nI don’t know what the eleventh touch point is for\n
  • Considered harmful, Don’t waste your time with this crap, just do the math, its way easier than forking\n
  • Best reference in the universe for this\n
  • \n
  • DOM reads aren’t free, most JS-dom stuff is blocking\nMinimize writes as well of course, but that is obvious when you just do the math your self\n
  • Don’t be insecure, math is math, no need to keep checking positions\nMore on matrices later. Remember reads aren’t free, don’t do them\n
  • use average of multiple touches if you want it to be multi-finger swipe\nYou probably shouldn’t override os gestures\nprioritize user feedback! When the user is swiping you are doing nothing else\nsnapback, snap forward. Just keep track of the place to snap back to and go there\n
  • (native uses physics, generally unecessary)\n\n
  • limitation of -webkit-overflow-scrolling: it bounces the whole page!\n
  • \n
  • You have ZERO control. Your interface elements are going to slide all over and resize\nYou could try position:fixed, but what about touch points? Font size? \nOk, you can work around all that, but that is going to be a hassle, especially when you consider\nthat pinch zoom isn’t that complex\n
  • If you have multiple transforms to apply, you can apply them to the matrix and combine the changes into one DOM write\nNot as scary as it looks\n
  • The cartoon everyone uses for this\nMatrix is a transformation applied to the vector representing the position\nYeah, you don’t really need to understand that for anything I’m talking about here, because we aren’t rotating, shearing, anything like that\n\n
  • This is the 2d\n
  • The 1s are the scale, the ts are translation. For pinch zoom (without rotation)\nTHIS IS ALL WE NEED TO KEEP TRACK OF STATE\n
  • \n
  • \n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • Scale point x is relative to object left, not page\n
  • transform origin will suffer from rounding errors caused by hiDPI virtual pixels\n
  • \n
  • \n
  • \n
  • DOM Layout is very simple. When the lightbox first opens we create three nodes (in addition to the interface, which is also simple) The DOM is also blanked to clear up memory space.\nImages are DIVs\n
  • \n
  • Remember the 256mb of memory!\nNo loading, no calculations during swipes. And no fancy business between event handler and transform,\n that is a critical performance point\n
  • You can’t use a big enough image to really benefit from pinch/zoom (subsampling of large images means they get scaled down)\nHardware accelerated stuff does not seem to manage memory very gracefully\n
  • \n
  • Creating Responsive HTML5 Touch Interfaces

    1. 1. Creating ResponsiveHTML5 TouchInterfacesStephen Woods
    2. 2. Stephen WoodsFront End Engineer Flickr
    3. 3. On the desktop weworry about browsers-moz-transform:rotate(-270deg);-moz-transform-origin: bottom left;-webkit-transform: rotate(-270deg);-webkit-transform-origin: bottom left;-o-transform: rotate(-270deg);-o-transform-origin: bottom left;filter:progid:DXImageTransform.Microsoft.BasicImage(rotation=1);
    4. 4. On mobile we worryabout devices.
    5. 5. ScreenSizesMedia Queries,Break points,liquid layoutshttp://www.alistapart.com/articles/responsive-web-design/
    6. 6. iPhone 3GS 256mb RAMGeekbench: 271
    7. 7. iPhone 3GS 256mb RAMGeekbench: 271 ==
    8. 8. Modern mobiledevices are crappy computers withdecent video cards.
    9. 9. PerceivedPerformance
    10. 10. On the desktop it’seasy... Throw up a spinner.
    11. 11. Touch interfaces aretactile.
    12. 12. Touch interfaces aretactile. Feedback must be continuous.
    13. 13. When the interface stops movingduring a gesture it feels like it died
    14. 14. Respect Convention
    15. 15. Mobile hasconventionstoo
    16. 16. Mobile hasconventionstoo
    17. 17. TouchEvent• touchstart - fires once• touchmove - fires continuously• touchend - fires once
    18. 18. The touches Array• You only get one on Android• You get up to 11 on iOS• Each touch gives you position information, and sometimes scale
    19. 19. iOS Gesture Events• gesturestart• gesturechange• gestureend
    20. 20. iOS Developer Libraryhttp://bit.ly/iOS-guide
    21. 21. Making Gestures Work • Prioritize user feedback • Use hardware acceleration • Manage your memory
    22. 22. Prioritize User-feedback• Don’t do any loading during gestures• Treat the DOM as write-only (do your own math)• When at all possible, use css transitions
    23. 23. Write-Only DOM• DOM touches are really expensive• You know where everything is• Use matrix transforms to queue up positions
    24. 24. Swipe Basicsdistance = e.touches[0].pageX - startX;translate3d(+distance+px,0px,0px)
    25. 25. Snap back/snap forward• Keep track of last position, use transitions with easing to snap back• Pick a swipe distance threshold, use that to snap forward (ontouchend)• If the user is gesturing, the element must be moving
    26. 26. A Word about scrolling• Use native if at all possible:• -webkit-overflow-scrolling: touch;• If not, use a library to simulate momentum scroll (iScroll 4, Scrollability)
    27. 27. Image © Brian LalorUsed with permission Pinch to Zoom (there will be math)
    28. 28. Why you can’t use native Pinch to Zoom
    29. 29. First:Use Matrix TransformsMinimize DOM touches, make your transformssimpler in the long run
    30. 30. http://xkcd.com/184/
    31. 31. It’s Not That Hard!transform: Translatematrix(1, 0, 0, 1, 10, 10); Scale
    32. 32. With HardwareAcceleration... (matrix3d) [ [1,0,0,0], [0,1,0,0], [0,0,1,0], [tx,ty,tz,1] ]
    33. 33. Transforms keep complex statewithout DOM reads
    34. 34. What is happening?• Determine Center of the touch points• Determine the scale factor (touch.scale)• Scale the element by the scale factor, with the center of the touch points as the scale center
    35. 35. The Naive Example
    36. 36. The Naive Example
    37. 37. The Naive Example
    38. 38. The Right Example
    39. 39. The Right Example
    40. 40. The Right Example
    41. 41. Breakdown
    42. 42. Breakdown
    43. 43. Breakdown
    44. 44. Breakdown
    45. 45. translateX =scalePointX * (newWidth - oldWidth) newWidth;
    46. 46. Pro Tips• Beware the virtual pixels• Moving the transform-origin won’t really work• Remember to snap back
    47. 47. The Flickr Touch Light Box
    48. 48. UntitledBy protohiro
    49. 49. UntitledBy protohiro
    50. 50. UntitledBy protohiro
    51. 51. UntitledBy protohiro
    52. 52. Swiping Process• Event Listener on top level for touch events• Only visible nodes move via translate3d• Rebuild next/previous happens when movement stops.
    53. 53. Performance Tricks• Aggressive Pruning• Clean off css transforms/transitions• Write-only DOM.• Do as little as possible during swipes
    54. 54. Frustrating Limitations• Retina screen is huge, device memory is small• Hardware acceleration is a crash festival• Fighting automatic optimizationhttp://bit.ly/apple-image-size-restrictions
    55. 55. Stephen Woods @ysawImage Credits (http://flic.kr/y/kQ5cLh)http://www.flickr.com/photos/wafer/5533140316/http://www.flickr.com/photos/latca/2265637876/http://www.flickr.com/photos/spine/1471217194/http://www.flickr.com/photos/williamhook/3656233025/http://www.flickr.com/photos/isriya/4656385586/http://www.flickr.com/photos/yandle/3076451873/http://www.flickr.com/photos/uberculture/6632437677/http://www.flickr.com/photos/blalor/4934146981/http://www.flickr.com/photos/torek/3280152297/http://www.flickr.com/photos/nilsrinaldi/5157809941/
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×