Understanding Hardware Acceleration on Mobile       Browsers        Ariya Hidayat
Magical Advice                 Use translate3d for hardware                         acceleration
Challenges
Game vs Web
Game                                    Text                                           Textured objects       Animation   ...
Large but Still Bounded
Web Page                  Images           Text
Challenges    Predictable contents (assets)   ✔    Mostly text                           ✔    Mostly images               ...
Browser Rendering
From Wave to Pixels                       Network stack                                  Layout engine                    ...
WebKit Components                                   Render Engine                                       CSS               ...
Platform Abstraction           Network      Unicode      Clipboard          Graphics       Theme        Events           T...
Graphics Abstraction  GraphicsContext       iOS        Android                                    Skia                    ...
SoC = System-on-a-Chip                         CPU   GPU
GPU Workflow               Matrix  Vertices              Rendered   Textured
Optimized for Games                      “Fixed” geometry                      Transformation                      Texture...
Powerful Capabilities http://epicgames.com/technology/epic-citadel   http://www.rage.com/
http://www.trollquotes.org/619-super-spider-bat-troll-quote/
Fundamental Physical Limitations    Memory     - Can’t store too much stuff    Speed     - Quad-core CPU can do certain ...
Traffic Congestion
Primitive Drawing
Path is Everything                     Triangle   Rectangle             Path                     Ellipse         Polygon
Stroke = Outline            Solid   Dashed   Dotted   Textured
Brush = Fill   None        Solid   Gradient   Textured
Gradient = Non-uniform Pixel Values
Drawing Time: Solid vs Gradient       Only border      Solid color fill                                           14x slow...
Path Approximation          Curves                     Polygon
Smooth via Antialiasing                          Multiple levels of transparency                                        Pe...
Shadow  Involved pixel “blending”   Often obstructed
Drawing Time: Solid vs Blur Shadow                                              20x slower  Solid shadow                  ...
CSS = Box Modelhttp://www.w3.org/TR/CSS2/zindex.html   Appendix E. Elaborate description of Stacking Contexts
Shadow Abuse     12,000 pixels                     Blur shadow
Transformation     Scaling                            Perspective                 Rotation
Text Rasterization
Font Atlas             Buffer                      Bye
From Simple to Complex                                                  ello!                                             ...
Library Instrumentation                     platformInit                     savePlatformState                     transla...
Painting Performance                       16:24:04.070   D/webcoreglue(    273):   drawContent   11   ms                 ...
Backing Store
Maps       Tiles
Responsive Interface             Rendering                         Processor                                     User inte...
Pinch to Zoom  when you pinch...
Rendering vs Interaction          Rendering   Web Page                User interaction
Checkerboard Pattern                       Poor man’s indicator                         of performance
Progressive Rendering       Fast but blurry                         Crisp but slow
Perceived Responsiveness        User pinches                                            Smooth scaled                     ...
Tiling System                               ....                ....                      CPU                             ...
Tile Transformation        Panning = Translation   Zooming = Scaling
Y U NOposition:fixed
Layer & Compositing
Typical Animation              opacity, movement, scaling, rotation, ...
Principles of Fluid Animation               At the beginning, push as many resources         1               as possible t...
Immediate vs Retained                               At the beginning...                                                   ...
Mechanics of Animation                       “Hey, this is good stuff. Cache it as texture #42.”      Initialization      ...
Elements = Layer
Logical 3-D
Compositing By Force                    -webkit-transform: translateZ(0)                    Not needed for CSS animation! ...
Debugging in Safari        defaults write com.apple.Safari IncludeInternalDebugMenu 1
Compositing Indicators                         Texture (number = upload)                                Composited layer  ...
Avoid Texture Reupload            No reupload                            Upload             Opacity             Position  ...
Examples
Color Transition        @-webkit-keyframes box {            0% { -webkit-transform: background-color: blue; }          100...
Color Transition: The Trick                          Blended with
Prepare and Reuse                          Hide the layer offscreen               Viewport
Timer Latency                                                        Depending on                                         ...
Avoid Overcapacity                              Texture upload                       ....               ....              ...
Wrap-up
Hardware Acceleration       Drawing primitives   Backing stores   Layer & compositing
There is no such  Traditional graphicsprogramming has been  always full of tricks.  It will always be.
Thank You       ariya.hidayat@gmail.com    Office Hour                                 Wed 1:45 PM        ariya.ofilabs.com ...
Understanding Hardware Acceleration on Mobile Browsers
Upcoming SlideShare
Loading in...5
×

Understanding Hardware Acceleration on Mobile Browsers

8,195

Published on

GPU acceleration on mobile browsers, if it is leveraged correctly, can lead to a smooth and fluid applications, thus improving the user experience. There has been a lot of mentions and best practices of hardware acceleration these days, although so far it has been pretty general and hasn’t provided much technical direction apart from simple magical advice such as “use translate3d”. This talk sheds some more light on browser interactions with the GPU and explain what happens behind the scenes, covering the topic of acceleration of primitive drawing, the use of tiled backing store, and composited layer. Knowing the actual machinery behind hardware acceleration, you will be in the position to plan your strategy to improve the performance of your web application.

Published in: Technology, Design
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
8,195
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
78
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide

Understanding Hardware Acceleration on Mobile Browsers

  1. 1. Understanding Hardware Acceleration on Mobile Browsers Ariya Hidayat
  2. 2. Magical Advice Use translate3d for hardware acceleration
  3. 3. Challenges
  4. 4. Game vs Web
  5. 5. Game Text Textured objects Animation Transformation Physics
  6. 6. Large but Still Bounded
  7. 7. Web Page Images Text
  8. 8. Challenges Predictable contents (assets) ✔ Mostly text ✔ Mostly images ✔ Initial response immediate
  9. 9. Browser Rendering
  10. 10. From Wave to Pixels Network stack Layout engine JavaScript engine Graphics User interface
  11. 11. WebKit Components Render Engine CSS DOM SVG HTML Canvas WebCore Client Interface JavaScript Engine (JSC/V8)
  12. 12. Platform Abstraction Network Unicode Clipboard Graphics Theme Events Thread Geolocation Timer
  13. 13. Graphics Abstraction GraphicsContext iOS Android Skia CoreGraphics Graphics stack
  14. 14. SoC = System-on-a-Chip CPU GPU
  15. 15. GPU Workflow Matrix Vertices Rendered Textured
  16. 16. Optimized for Games “Fixed” geometry Transformation Textured triangles Parallelism
  17. 17. Powerful Capabilities http://epicgames.com/technology/epic-citadel http://www.rage.com/
  18. 18. http://www.trollquotes.org/619-super-spider-bat-troll-quote/
  19. 19. Fundamental Physical Limitations  Memory - Can’t store too much stuff  Speed - Quad-core CPU can do certain operations better  Bandwidth - Bottleneck can be in the data transfer
  20. 20. Traffic Congestion
  21. 21. Primitive Drawing
  22. 22. Path is Everything Triangle Rectangle Path Ellipse Polygon
  23. 23. Stroke = Outline Solid Dashed Dotted Textured
  24. 24. Brush = Fill None Solid Gradient Textured
  25. 25. Gradient = Non-uniform Pixel Values
  26. 26. Drawing Time: Solid vs Gradient Only border Solid color fill 14x slower (depending on the dimension) Linear gradient fill Radial gradient fill
  27. 27. Path Approximation Curves Polygon
  28. 28. Smooth via Antialiasing Multiple levels of transparency Perfect for parallelism
  29. 29. Shadow Involved pixel “blending” Often obstructed
  30. 30. Drawing Time: Solid vs Blur Shadow 20x slower Solid shadow (depending on the blur radius) Blur shadow
  31. 31. CSS = Box Modelhttp://www.w3.org/TR/CSS2/zindex.html Appendix E. Elaborate description of Stacking Contexts
  32. 32. Shadow Abuse 12,000 pixels Blur shadow
  33. 33. Transformation Scaling Perspective Rotation
  34. 34. Text Rasterization
  35. 35. Font Atlas Buffer Bye
  36. 36. From Simple to Complex ello! H Curves Simple shape Gradient brush Solid color Textured stroke Blur shadow Serif text Rotated Make it as an image
  37. 37. Library Instrumentation platformInit savePlatformState translate 0,0 translate 0,0 clip 1,0 0x6.95322e-310 fillRect 0,0 800x556 color ff ff ff ff restorePlatformState platformDestroy platformInit savePlatformState translate 0,0 translate 0,0 clip 1,0 0x6.95322e-310 fillRect 0,0 800x556 color ff ff ff ff restorePlatformState platformDestroy platformInit savePlatformState translate 0,0 translate 0,0 clip 1,0 0x6.95322e-310 fillRect 0,0 800x556 color ff ff ff ff fillRect 0,0 800x556 color ff ff ff ff
  38. 38. Painting Performance 16:24:04.070 D/webcoreglue(  273): drawContent 11 ms 16:24:04.110 D/webcoreglue(  273): drawContent 13 ms 16:24:04.150 D/webcoreglue(  273): drawContent 13 ms 16:24:04.190 D/webcoreglue(  273): drawContent 10 ms 16:24:04.240 D/webcoreglue(  273): drawContent 10 ms 16:24:04.280 D/webcoreglue(  273): drawContent 13 ms 16:24:04.320 D/webcoreglue(  273): drawContent 13 ms 16:24:04.360 D/webcoreglue(  273): drawContent 13 ms 16:24:06.080 D/webcoreglue(  273): drawContent 12 ms 16:24:06.140 D/webcoreglue(  273): drawContent 10 ms 16:24:06.180 D/webcoreglue(  273): drawContent 13 ms 16:24:06.230 D/webcoreglue(  273): drawContent 14 ms 16:24:06.600 D/webcoreglue(  273): drawContent 26 ms 16:24:06.640 D/webcoreglue(  273): drawContent 13 ms 16:24:06.860 D/webcoreglue(  273): drawContent 33 ms 16:24:06.890 D/webcoreglue(  273): drawContent 12 ms 16:24:06.930 D/webcoreglue(  273): drawContent 13 ms 16:24:06.960 D/webcoreglue(  273): drawContent 13 ms 16:24:07.000 D/webcoreglue(  273): drawContent 13 ms
  39. 39. Backing Store
  40. 40. Maps Tiles
  41. 41. Responsive Interface Rendering Processor User interaction
  42. 42. Pinch to Zoom when you pinch...
  43. 43. Rendering vs Interaction Rendering Web Page User interaction
  44. 44. Checkerboard Pattern Poor man’s indicator of performance
  45. 45. Progressive Rendering Fast but blurry Crisp but slow
  46. 46. Perceived Responsiveness User pinches Smooth scaled Blocky (but fast!)
  47. 47. Tiling System .... .... CPU GPU Streamed What happens if the page background changes color?
  48. 48. Tile Transformation Panning = Translation Zooming = Scaling
  49. 49. Y U NOposition:fixed
  50. 50. Layer & Compositing
  51. 51. Typical Animation opacity, movement, scaling, rotation, ...
  52. 52. Principles of Fluid Animation At the beginning, push as many resources 1 as possible to the GPU During the animation, minimize 2 CPU-GPU interaction
  53. 53. Immediate vs Retained At the beginning... draw the shape onto a buffer For every animation tick... draw the shape at (x, y) blit the buffer at (x, y) x = x + 10 x = x + 10 Off main thread
  54. 54. Mechanics of Animation “Hey, this is good stuff. Cache it as texture #42.” Initialization “Apply [operation] to texture #42.” Animation step
  55. 55. Elements = Layer
  56. 56. Logical 3-D
  57. 57. Compositing By Force -webkit-transform: translateZ(0) Not needed for CSS animation! .someid { -webkit-animation-name: somekeyframeanimation; -webkit-animation-duration: 7s; -webkit-transform: translateZ(0); } Don’t do that
  58. 58. Debugging in Safari defaults write com.apple.Safari IncludeInternalDebugMenu 1
  59. 59. Compositing Indicators Texture (number = upload) Composited layer No associated texture Container layer Overflow
  60. 60. Avoid Texture Reupload No reupload Upload Opacity Position Everything else! Size Animation “Hardware accelerated CSS”
  61. 61. Examples
  62. 62. Color Transition @-webkit-keyframes box { 0% { -webkit-transform: background-color: blue; } 100% { -webkit-transform: background-color: green; } } Need a new texture uploaded to the GPU for every in-between color
  63. 63. Color Transition: The Trick Blended with
  64. 64. Prepare and Reuse Hide the layer offscreen Viewport
  65. 65. Timer Latency Depending on user reaction •Animation end triggers the JavaScript callback •JavaScript code kicks the next animation Prepare both animation and hide the “wrong” one
  66. 66. Avoid Overcapacity Texture upload .... .... Think of the GPU memory like a cache.
  67. 67. Wrap-up
  68. 68. Hardware Acceleration Drawing primitives Backing stores Layer & compositing
  69. 69. There is no such Traditional graphicsprogramming has been always full of tricks. It will always be.
  70. 70. Thank You ariya.hidayat@gmail.com Office Hour Wed 1:45 PM ariya.ofilabs.com @AriyaHidayat
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×