24. Making Gestures Work
• Prioritize user feedback
• Use hardware acceleration
• Manage your memory
25. Prioritize User-feedback
• Don’t do any loading during gestures
• Treat the DOM as write-only (do your
own math)
• When at all possible, use css
transitions
26. Write-Only DOM
• DOM touches are really expensive
• You know where everything is
• Use matrix transforms to queue up
positions
28. Snap back/snap
forward
• Keep track of last position, use
transitions with easing to snap back
• Pick a swipe distance threshold, use
that to snap forward (ontouchend)
• If the user is gesturing, the element
must be moving
29. A Word about scrolling
• Use native if at all possible:
• -webkit-overflow-scrolling: touch;
• If not, use a library to simulate
momentum scroll (iScroll 4,
Scrollability)
37. What is happening?
• Determine Center of the touch points
• Determine the scale factor
(touch.scale)
• Scale the element by the scale
factor, with the center of the touch
points as the scale center
55. Swiping Process
• Event Listener on top level for touch
events
• Only visible nodes move via
translate3d
• Rebuild next/previous happens when
movement stops.
56. Performance Tricks
• Aggressive Pruning
• Clean off css transforms/transitions
• Write-only DOM.
• Do as little as possible during swipes
57. Frustrating
Limitations
• Retina screen is huge, device
memory is small
• Hardware acceleration is a crash
festival
• Fighting automatic optimization
http://bit.ly/apple-image-size-restrictions
I built the mobile lightbox (on a great backbone from the flickr FE team)\n
The flickr mobile lightbox. Swiping, zooming, pretty smooth, very natural for the device.\nI’m going to talk about how we made it, but I am going to talk a lot of concepts that came out of the work first\n
\n
They all run webkit, they all have css3, transforms, etc.\nYou’ll have to do a lot of testing on screen sizes of course...but..\n
People have covered this topic very well elsewhere\n
\n
\n
\n
but this is going to change...\n
An awesome experience is possible.\n
\n
\n
Remember the teleporter control in stng? It had a control like in the original series, but touch\nIt needs to feel right, that means immediate feedback is continuous, rather than momentary.\n
The rational for snap back and momentum scroll (always give feedback)\n\n
We all know the conventions of desktop software\nMost people respect it when building interfaces\n
Slide to unlock, swipe to go forward and back, pinch to zoom, tap and hold\ni) don't break what users expect: swipes & Pinches\nexample: user reaction to m.flickr lack of pinch zoom\n
\n
Ice cream sandwich caveat noted.\nI don’t know what the eleventh touch point is for\n
Considered harmful, Don’t waste your time with this crap, just do the math, its way easier than forking\n
Best reference in the universe for this\n
\n
DOM reads aren’t free, most JS-dom stuff is blocking\nMinimize writes as well of course, but that is obvious when you just do the math your self\n
Don’t be insecure, math is math, no need to keep checking positions\nMore on matrices later. Remember reads aren’t free, don’t do them\n
use average of multiple touches if you want it to be multi-finger swipe\nYou probably shouldn’t override os gestures\nprioritize user feedback! When the user is swiping you are doing nothing else\nsnapback, snap forward. Just keep track of the place to snap back to and go there\n
(native uses physics, generally unecessary)\n\n
limitation of -webkit-overflow-scrolling: it bounces the whole page!\n
\n
You have ZERO control. Your interface elements are going to slide all over and resize\nYou could try position:fixed, but what about touch points? Font size? \nOk, you can work around all that, but that is going to be a hassle, especially when you consider\nthat pinch zoom isn’t that complex\n
If you have multiple transforms to apply, you can apply them to the matrix and combine the changes into one DOM write\nNot as scary as it looks\n
The cartoon everyone uses for this\nMatrix is a transformation applied to the vector representing the position\nYeah, you don’t really need to understand that for anything I’m talking about here, because we aren’t rotating, shearing, anything like that\n\n
This is the 2d\n
The 1s are the scale, the ts are translation. For pinch zoom (without rotation)\nTHIS IS ALL WE NEED TO KEEP TRACK OF STATE\n
\n
\n
Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
The touch center point is the point from which the object scales\n
The touch center point is the point from which the object scales\n
The touch center point is the point from which the object scales\n
The touch center point is the point from which the object scales\n
The touch center point is the point from which the object scales\n
The touch center point is the point from which the object scales\n
The touch center point is the point from which the object scales\n
The touch center point is the point from which the object scales\n
The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
Scale point x is relative to object left, not page\n
transform origin will suffer from rounding errors caused by hiDPI virtual pixels\n
\n
\n
\n
DOM Layout is very simple. When the lightbox first opens we create three nodes (in addition to the interface, which is also simple) The DOM is also blanked to clear up memory space.\nImages are DIVs\n
\n
Remember the 256mb of memory!\nNo loading, no calculations during swipes. And no fancy business between event handler and transform,\n that is a critical performance point\n
You can’t use a big enough image to really benefit from pinch/zoom (subsampling of large images means they get scaled down)\nHardware accelerated stuff does not seem to manage memory very gracefully\n
