Successfully reported this slideshow.
Your SlideShare is downloading. ×

Creating Responsive HTML5 Touch Interfaces

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Loading in …3
×

Check these out next

1 of 58 Ad
Advertisement

More Related Content

Similar to Creating Responsive HTML5 Touch Interfaces (20)

Recently uploaded (20)

Advertisement

Creating Responsive HTML5 Touch Interfaces

  1. Creating Responsive HTML5 Touch Interfaces Stephen Woods
  2. Stephen Woods Front End Engineer Flickr
  3. On the desktop we worry about browsers -moz-transform:rotate(-270deg); -moz-transform-origin: bottom left; -webkit-transform: rotate(-270deg); -webkit-transform-origin: bottom left; -o-transform: rotate(-270deg); -o-transform-origin: bottom left; filter:progid:DXImageTransform.Microsoft .BasicImage(rotation=1);
  4. On mobile we worry about devices.
  5. Screen Sizes Media Queries, Break points, liquid layouts http://www.alistapart.com/articles/responsive-web-design/
  6. iPhone 3GS 256mb RAM Geekbench: 271
  7. iPhone 3GS 256mb RAM Geekbench: 271 ==
  8. Modern mobile devices are crappy computers with decent video cards.
  9. Perceived Performance
  10. On the desktop it’s easy... Throw up a spinner.
  11. Touch interfaces are tactile.
  12. Touch interfaces are tactile. Feedback must be continuous.
  13. When the interface stops moving during a gesture it feels like it died
  14. Respect Convention
  15. Mobile has conventions too
  16. Mobile has conventions too
  17. TouchEvent • touchstart - fires once • touchmove - fires continuously • touchend - fires once
  18. The touches Array • You only get one on Android • You get up to 11 on iOS • Each touch gives you position information, and sometimes scale
  19. iOS Gesture Events • gesturestart • gesturechange • gestureend
  20. iOS Developer Library http://bit.ly/iOS-guide
  21. Making Gestures Work • Prioritize user feedback • Use hardware acceleration • Manage your memory
  22. Prioritize User-feedback • Don’t do any loading during gestures • Treat the DOM as write-only (do your own math) • When at all possible, use css transitions
  23. Write-Only DOM • DOM touches are really expensive • You know where everything is • Use matrix transforms to queue up positions
  24. Swipe Basics distance = e.touches[0].pageX - startX; 'translate3d('+distance+'px,0px,0px)'
  25. Snap back/snap forward • Keep track of last position, use transitions with easing to snap back • Pick a swipe distance threshold, use that to snap forward (ontouchend) • If the user is gesturing, the element must be moving
  26. A Word about scrolling • Use native if at all possible: • -webkit-overflow-scrolling: touch; • If not, use a library to simulate momentum scroll (iScroll 4, Scrollability)
  27. Image © Brian Lalor Used with permission Pinch to Zoom (there will be math)
  28. Why you can’t use native Pinch to Zoom
  29. First: Use Matrix Transforms Minimize DOM touches, make your transforms simpler in the long run
  30. http://xkcd.com/184/
  31. It’s Not That Hard! transform: Translate matrix(1, 0, 0, 1, 10, 10); Scale
  32. With Hardware Acceleration... (matrix3d) [ [1,0,0,0], [0,1,0,0], [0,0,1,0], [tx,ty,tz,1] ]
  33. Transforms keep complex state without DOM reads
  34. What is happening? • Determine Center of the touch points • Determine the scale factor (touch.scale) • Scale the element by the scale factor, with the center of the touch points as the scale center
  35. The Naive Example
  36. The Naive Example
  37. The Naive Example
  38. The Right Example
  39. The Right Example
  40. The Right Example
  41. Breakdown
  42. Breakdown
  43. Breakdown
  44. Breakdown
  45. translateX = scalePointX * (newWidth - oldWidth) newWidth;
  46. Pro Tips • Beware the virtual pixels • Moving the transform-origin won’t really work • Remember to snap back
  47. The Flickr Touch Light Box
  48. Untitled By protohiro
  49. Untitled By protohiro
  50. Untitled By protohiro
  51. Untitled By protohiro
  52. Swiping Process • Event Listener on top level for touch events • Only visible nodes move via translate3d • Rebuild next/previous happens when movement stops.
  53. Performance Tricks • Aggressive Pruning • Clean off css transforms/transitions • Write-only DOM. • Do as little as possible during swipes
  54. Frustrating Limitations • Retina screen is huge, device memory is small • Hardware acceleration is a crash festival • Fighting automatic optimization http://bit.ly/apple-image-size-restrictions
  55. Stephen Woods @ysaw Image Credits (http://flic.kr/y/kQ5cLh) http://www.flickr.com/photos/wafer/5533140316/ http://www.flickr.com/photos/latca/2265637876/ http://www.flickr.com/photos/spine/1471217194/ http://www.flickr.com/photos/williamhook/3656233025/ http://www.flickr.com/photos/isriya/4656385586/ http://www.flickr.com/photos/yandle/3076451873/ http://www.flickr.com/photos/uberculture/6632437677/ http://www.flickr.com/photos/blalor/4934146981/ http://www.flickr.com/photos/torek/3280152297/ http://www.flickr.com/photos/nilsrinaldi/5157809941/

Editor's Notes

  • \n
  • I built the mobile lightbox (on a great backbone from the flickr FE team)\n
  • The flickr mobile lightbox. Swiping, zooming, pretty smooth, very natural for the device.\nI’m going to talk about how we made it, but I am going to talk a lot of concepts that came out of the work first\n
  • \n
  • They all run webkit, they all have css3, transforms, etc.\nYou’ll have to do a lot of testing on screen sizes of course...but..\n
  • People have covered this topic very well elsewhere\n
  • \n
  • \n
  • \n
  • but this is going to change...\n
  • An awesome experience is possible.\n
  • \n
  • \n
  • Remember the teleporter control in stng? It had a control like in the original series, but touch\nIt needs to feel right, that means immediate feedback is continuous, rather than momentary.\n
  • The rational for snap back and momentum scroll (always give feedback)\n\n
  • We all know the conventions of desktop software\nMost people respect it when building interfaces\n
  • Slide to unlock, swipe to go forward and back, pinch to zoom, tap and hold\ni) don't break what users expect: swipes & Pinches\nexample: user reaction to m.flickr lack of pinch zoom\n
  • \n
  • Ice cream sandwich caveat noted.\nI don’t know what the eleventh touch point is for\n
  • Considered harmful, Don’t waste your time with this crap, just do the math, its way easier than forking\n
  • Best reference in the universe for this\n
  • \n
  • DOM reads aren’t free, most JS-dom stuff is blocking\nMinimize writes as well of course, but that is obvious when you just do the math your self\n
  • Don’t be insecure, math is math, no need to keep checking positions\nMore on matrices later. Remember reads aren’t free, don’t do them\n
  • use average of multiple touches if you want it to be multi-finger swipe\nYou probably shouldn’t override os gestures\nprioritize user feedback! When the user is swiping you are doing nothing else\nsnapback, snap forward. Just keep track of the place to snap back to and go there\n
  • (native uses physics, generally unecessary)\n\n
  • limitation of -webkit-overflow-scrolling: it bounces the whole page!\n
  • \n
  • You have ZERO control. Your interface elements are going to slide all over and resize\nYou could try position:fixed, but what about touch points? Font size? \nOk, you can work around all that, but that is going to be a hassle, especially when you consider\nthat pinch zoom isn’t that complex\n
  • If you have multiple transforms to apply, you can apply them to the matrix and combine the changes into one DOM write\nNot as scary as it looks\n
  • The cartoon everyone uses for this\nMatrix is a transformation applied to the vector representing the position\nYeah, you don’t really need to understand that for anything I’m talking about here, because we aren’t rotating, shearing, anything like that\n\n
  • This is the 2d\n
  • The 1s are the scale, the ts are translation. For pinch zoom (without rotation)\nTHIS IS ALL WE NEED TO KEEP TRACK OF STATE\n
  • \n
  • \n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  • Scale point x is relative to object left, not page\n
  • transform origin will suffer from rounding errors caused by hiDPI virtual pixels\n
  • \n
  • \n
  • \n
  • DOM Layout is very simple. When the lightbox first opens we create three nodes (in addition to the interface, which is also simple) The DOM is also blanked to clear up memory space.\nImages are DIVs\n
  • \n
  • Remember the 256mb of memory!\nNo loading, no calculations during swipes. And no fancy business between event handler and transform,\n that is a critical performance point\n
  • You can’t use a big enough image to really benefit from pinch/zoom (subsampling of large images means they get scaled down)\nHardware accelerated stuff does not seem to manage memory very gracefully\n
  • \n

×