SlideShare a Scribd company logo
1 of 58
Creating Responsive
HTML5 Touch
Interfaces
Stephen Woods
Stephen Woods
Front End Engineer
       Flickr
On the desktop we
worry about browsers
-moz-transform:rotate(-270deg);
-moz-transform-origin: bottom left;
-webkit-transform: rotate(-270deg);
-webkit-transform-origin: bottom left;
-o-transform: rotate(-270deg);
-o-transform-origin: bottom left;
filter:progid:DXImageTransform.Microsoft
.BasicImage(rotation=1);
On mobile we worry
about devices.
Screen
Sizes
Media Queries,
Break points,
liquid layouts




http://www.alistapart.com/articles/responsive-web-design/
iPhone 3GS
 256mb RAM
Geekbench: 271
iPhone 3GS
 256mb RAM
Geekbench: 271



      ==
Modern mobile
devices are crappy
  computers with
decent video cards.
Perceived
Performance
On the desktop it’s
easy...



  Throw up a spinner.
Touch interfaces are
tactile.
Touch interfaces are
tactile.


   Feedback must be
        continuous.
When the interface
  stops moving
during a gesture it
 feels like it died
Respect Convention
Mobile has
conventions
too
Mobile has
conventions
too
TouchEvent

• touchstart - fires once
• touchmove - fires continuously
• touchend - fires once
The touches Array

• You only get one on Android
• You get up to 11 on iOS
• Each touch gives you position
  information, and sometimes scale
iOS Gesture Events

• gesturestart
• gesturechange
• gestureend
iOS Developer
      Library

http://bit.ly/iOS-guide
Making Gestures Work

 • Prioritize user feedback
 • Use hardware acceleration
 • Manage your memory
Prioritize User-feedback



• Don’t do any loading during gestures
• Treat the DOM as write-only (do your
  own math)

• When at all possible, use css
  transitions
Write-Only DOM

• DOM touches are really expensive
• You know where everything is
• Use matrix transforms to queue up
  positions
Swipe Basics
distance = e.touches[0].pageX - startX;

'translate3d('+distance+'px,0px,0px)'
Snap back/snap
      forward
• Keep track of last position, use
  transitions with easing to snap back

• Pick a swipe distance threshold, use
  that to snap forward (ontouchend)

• If the user is gesturing, the element
  must be moving
A Word about scrolling

• Use native if at all possible:
• -webkit-overflow-scrolling:      touch;

• If not, use a library to simulate
  momentum scroll (iScroll 4,
  Scrollability)
Image © Brian Lalor
Used with permission




   Pinch to Zoom
   (there will be math)
Why you can’t use
 native Pinch to
      Zoom
First:
Use Matrix Transforms
Minimize DOM touches, make your transforms
simpler in the long run
http://xkcd.com/184/
It’s Not That Hard!

transform:          Translate
matrix(1, 0, 0, 1, 10, 10);

         Scale
With Hardware
Acceleration... (matrix3d)


            [
                [1,0,0,0],
                [0,1,0,0],
                [0,0,1,0],
                [tx,ty,tz,1]
            ]
Transforms keep
  complex state
without DOM reads
What is happening?

• Determine Center of the touch points
• Determine the scale factor
  (touch.scale)

• Scale the element by the scale
  factor, with the center of the touch
  points as the scale center
The Naive Example
The Naive Example
The Naive Example
The Right Example
The Right Example
The Right Example
Breakdown
Breakdown
Breakdown
Breakdown
translateX =
scalePointX * (newWidth - oldWidth)
            newWidth;
Pro Tips

• Beware the virtual pixels
• Moving the transform-origin won’t
  really work

• Remember to snap back
The Flickr Touch
   Light Box
Untitled
By protohiro
Untitled
By protohiro
Untitled
By protohiro
Untitled
By protohiro
Swiping Process

• Event Listener on top level for touch
  events

• Only visible nodes move via
  translate3d

• Rebuild next/previous happens when
  movement stops.
Performance Tricks

• Aggressive Pruning
• Clean off css transforms/transitions
• Write-only DOM.
• Do as little as possible during swipes
Frustrating
         Limitations
• Retina screen is huge, device
  memory is small

• Hardware acceleration is a crash
  festival

• Fighting automatic optimization
http://bit.ly/apple-image-size-restrictions
Stephen Woods
                                  @ysaw



Image Credits (http://flic.kr/y/kQ5cLh)
http://www.flickr.com/photos/wafer/5533140316/
http://www.flickr.com/photos/latca/2265637876/
http://www.flickr.com/photos/spine/1471217194/
http://www.flickr.com/photos/williamhook/3656233025/
http://www.flickr.com/photos/isriya/4656385586/
http://www.flickr.com/photos/yandle/3076451873/
http://www.flickr.com/photos/uberculture/6632437677/
http://www.flickr.com/photos/blalor/4934146981/
http://www.flickr.com/photos/torek/3280152297/
http://www.flickr.com/photos/nilsrinaldi/5157809941/

More Related Content

Similar to Creating Responsive HTML5 Touch Interfaces

SlickGrid Touch: Making complex JavaScript widgets work on mobile devices
SlickGrid Touch: Making complex JavaScript widgets work on mobile devicesSlickGrid Touch: Making complex JavaScript widgets work on mobile devices
SlickGrid Touch: Making complex JavaScript widgets work on mobile devices
reebalazs
 

Similar to Creating Responsive HTML5 Touch Interfaces (20)

Practical resource monitoring with munin (English editon)
Practical resource monitoring with munin  (English editon)Practical resource monitoring with munin  (English editon)
Practical resource monitoring with munin (English editon)
 
All about that reactive ui
All about that reactive uiAll about that reactive ui
All about that reactive ui
 
Espial 3d,2d transforms preso
Espial  3d,2d transforms presoEspial  3d,2d transforms preso
Espial 3d,2d transforms preso
 
LST Toolkit: Exfiltration Over Sound, Light, Touch
LST Toolkit: Exfiltration Over Sound, Light, TouchLST Toolkit: Exfiltration Over Sound, Light, Touch
LST Toolkit: Exfiltration Over Sound, Light, Touch
 
Building a Database for the End of the World
Building a Database for the End of the WorldBuilding a Database for the End of the World
Building a Database for the End of the World
 
Introduction to mobile programming with Androids.
Introduction to mobile programming with Androids. Introduction to mobile programming with Androids.
Introduction to mobile programming with Androids.
 
Design for Scale / Surge 2010
Design for Scale / Surge 2010Design for Scale / Surge 2010
Design for Scale / Surge 2010
 
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
The Dirty Little Secrets They Didn’t Teach You In Pentesting Class
 
The Good, the Bad and the Ugly things to do with android
The Good, the Bad and the Ugly things to do with androidThe Good, the Bad and the Ugly things to do with android
The Good, the Bad and the Ugly things to do with android
 
K2P workshop 3-23-13
K2P workshop 3-23-13K2P workshop 3-23-13
K2P workshop 3-23-13
 
How shit works: the CPU
How shit works: the CPUHow shit works: the CPU
How shit works: the CPU
 
SlickGrid Touch: Making complex JavaScript widgets work on mobile devices
SlickGrid Touch: Making complex JavaScript widgets work on mobile devicesSlickGrid Touch: Making complex JavaScript widgets work on mobile devices
SlickGrid Touch: Making complex JavaScript widgets work on mobile devices
 
JavaFX 101
JavaFX 101JavaFX 101
JavaFX 101
 
Building a game engine with jQuery
Building a game engine with jQueryBuilding a game engine with jQuery
Building a game engine with jQuery
 
[DBA]_HiramFleitas_SQL_PASS_Summit_2017_Summary
[DBA]_HiramFleitas_SQL_PASS_Summit_2017_Summary[DBA]_HiramFleitas_SQL_PASS_Summit_2017_Summary
[DBA]_HiramFleitas_SQL_PASS_Summit_2017_Summary
 
The Appy Hour
The Appy HourThe Appy Hour
The Appy Hour
 
Smooth Animations for Web & Hybrid
Smooth Animations for Web & HybridSmooth Animations for Web & Hybrid
Smooth Animations for Web & Hybrid
 
Adobe AIR for mobile games
Adobe AIR for mobile gamesAdobe AIR for mobile games
Adobe AIR for mobile games
 
Ottawa unity user_group_feb13_2015
Ottawa unity user_group_feb13_2015Ottawa unity user_group_feb13_2015
Ottawa unity user_group_feb13_2015
 
How to create Great App
How to create Great AppHow to create Great App
How to create Great App
 

Recently uploaded

Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Peter Udo Diehl
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
UK Journal
 

Recently uploaded (20)

Overview of Hyperledger Foundation
Overview of Hyperledger FoundationOverview of Hyperledger Foundation
Overview of Hyperledger Foundation
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101AI presentation and introduction - Retrieval Augmented Generation RAG 101
AI presentation and introduction - Retrieval Augmented Generation RAG 101
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024Enterprise Knowledge Graphs - Data Summit 2024
Enterprise Knowledge Graphs - Data Summit 2024
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
Measures in SQL (a talk at SF Distributed Systems meetup, 2024-05-22)
 
Optimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through ObservabilityOptimizing NoSQL Performance Through Observability
Optimizing NoSQL Performance Through Observability
 
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo DiehlFuture Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
Future Visions: Predictions to Guide and Time Tech Innovation, Peter Udo Diehl
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
 
Demystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John StaveleyDemystifying gRPC in .Net by John Staveley
Demystifying gRPC in .Net by John Staveley
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdfSimplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
Simplified FDO Manufacturing Flow with TPMs _ Liam at Infineon.pdf
 

Creating Responsive HTML5 Touch Interfaces

Editor's Notes

  1. \n
  2. I built the mobile lightbox (on a great backbone from the flickr FE team)\n
  3. The flickr mobile lightbox. Swiping, zooming, pretty smooth, very natural for the device.\nI’m going to talk about how we made it, but I am going to talk a lot of concepts that came out of the work first\n
  4. \n
  5. They all run webkit, they all have css3, transforms, etc.\nYou’ll have to do a lot of testing on screen sizes of course...but..\n
  6. People have covered this topic very well elsewhere\n
  7. \n
  8. \n
  9. \n
  10. but this is going to change...\n
  11. An awesome experience is possible.\n
  12. \n
  13. \n
  14. Remember the teleporter control in stng? It had a control like in the original series, but touch\nIt needs to feel right, that means immediate feedback is continuous, rather than momentary.\n
  15. The rational for snap back and momentum scroll (always give feedback)\n\n
  16. We all know the conventions of desktop software\nMost people respect it when building interfaces\n
  17. Slide to unlock, swipe to go forward and back, pinch to zoom, tap and hold\ni) don't break what users expect: swipes & Pinches\nexample: user reaction to m.flickr lack of pinch zoom\n
  18. \n
  19. Ice cream sandwich caveat noted.\nI don’t know what the eleventh touch point is for\n
  20. Considered harmful, Don’t waste your time with this crap, just do the math, its way easier than forking\n
  21. Best reference in the universe for this\n
  22. \n
  23. DOM reads aren’t free, most JS-dom stuff is blocking\nMinimize writes as well of course, but that is obvious when you just do the math your self\n
  24. Don’t be insecure, math is math, no need to keep checking positions\nMore on matrices later. Remember reads aren’t free, don’t do them\n
  25. use average of multiple touches if you want it to be multi-finger swipe\nYou probably shouldn’t override os gestures\nprioritize user feedback! When the user is swiping you are doing nothing else\nsnapback, snap forward. Just keep track of the place to snap back to and go there\n
  26. (native uses physics, generally unecessary)\n\n
  27. limitation of -webkit-overflow-scrolling: it bounces the whole page!\n
  28. \n
  29. You have ZERO control. Your interface elements are going to slide all over and resize\nYou could try position:fixed, but what about touch points? Font size? \nOk, you can work around all that, but that is going to be a hassle, especially when you consider\nthat pinch zoom isn’t that complex\n
  30. If you have multiple transforms to apply, you can apply them to the matrix and combine the changes into one DOM write\nNot as scary as it looks\n
  31. The cartoon everyone uses for this\nMatrix is a transformation applied to the vector representing the position\nYeah, you don’t really need to understand that for anything I’m talking about here, because we aren’t rotating, shearing, anything like that\n\n
  32. This is the 2d\n
  33. The 1s are the scale, the ts are translation. For pinch zoom (without rotation)\nTHIS IS ALL WE NEED TO KEEP TRACK OF STATE\n
  34. \n
  35. \n
  36. Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  37. Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  38. Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  39. Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  40. Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  41. Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  42. Just apply the scale property to the scale transform of the element\nBreaks the convention. \nIt should feel like the user is stretching the image, not turning a dial\n
  43. The touch center point is the point from which the object scales\n
  44. The touch center point is the point from which the object scales\n
  45. The touch center point is the point from which the object scales\n
  46. The touch center point is the point from which the object scales\n
  47. The touch center point is the point from which the object scales\n
  48. The touch center point is the point from which the object scales\n
  49. The touch center point is the point from which the object scales\n
  50. The touch center point is the point from which the object scales\n
  51. The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  52. The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  53. The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  54. The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  55. The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  56. The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  57. The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  58. The touch center point is the point from which the object scales\nWith the magic of matrix transforms we can apply both at the same time\n
  59. Scale point x is relative to object left, not page\n
  60. transform origin will suffer from rounding errors caused by hiDPI virtual pixels\n
  61. \n
  62. \n
  63. \n
  64. DOM Layout is very simple. When the lightbox first opens we create three nodes (in addition to the interface, which is also simple) The DOM is also blanked to clear up memory space.\nImages are DIVs\n
  65. \n
  66. Remember the 256mb of memory!\nNo loading, no calculations during swipes. And no fancy business between event handler and transform,\n that is a critical performance point\n
  67. You can’t use a big enough image to really benefit from pinch/zoom (subsampling of large images means they get scaled down)\nHardware accelerated stuff does not seem to manage memory very gracefully\n
  68. \n