Wednesday, November 2, 11
Wednesday, November 2, 11
Hardware Acceleration                on Mobile                              Ariya Hidayat & Jarred Nicholls               ...
ChallengesWednesday, November 2, 11
Game vs WebWednesday, November 2, 11
Game                                                         Text                                                         ...
Web Pages                                   Images                            TextWednesday, November 2, 11
How Browser Works                             Network stack                                         Layout engine         ...
WebKit Components                            DOM               CSS                                  WebCore              S...
Platform Abstraction                            Network     Unicode      Clipboard                            Graphics    ...
Graphics Abstraction       GraphicsContext          Mac        Chromium       Qt     Gtk                                  ...
Components of Mobile                                              Radio                            Touch interface        ...
Graphics Processor                            Divide and conquer                            Very specific purposeWednesday,...
Optimized for Games                            Fixed geometry                            Transformation                   ...
Challenges         Predictable contents (assets)   ✔         Mostly text                           ✔         Mostly images...
Primitive DrawingWednesday, November 2, 11
Path is Everything                               Triangle   Rectangle                        Path                         ...
Stroke = Outline                  Solid     Dashed   Dotted   TexturedWednesday, November 2, 11
Brush = Fill            None            Solid   Gradient   TexturedWednesday, November 2, 11
Smooth via Antialiasing                            Multiple levels of transparency                                        ...
Path Approximation                  Curves                            PolygonWednesday, November 2, 11
Gradient: Expensive                            CPU: sequential, tedious                            GPU: parallel, still a ...
Blur Shadow: Really Posh                            Involved pixel “blending”                            GPU: parallel, st...
Transformation                 Scaling                                       Perspective                            Rotati...
Text RasterizationWednesday, November 2, 11
Font Atlas                            Buffer                                     ByeWednesday, November 2, 11
Simple to Complex                                                  ello!                                                H ...
HOW FAST?Wednesday, November 2, 11
Android Drawing                                                   Log file                   WebCore                   grap...
Example: Bing    platformInit    savePlatformState    translate 0,0    translate 0,0    clip 1,0 0x6.95322e-310    fillRec...
How Fast?              #include "TimeCounter.h"              bool WebViewCore::drawContent(SkCanvas* canvas, SkColor)     ...
Example: Google News         adb logcat -v time | grep drawContent        16:24:04.070 D/webcoreglue(  273): drawContent  ...
Backing StoreWednesday, November 2, 11
Maps                            TileWednesday, November 2, 11
Pinch to Zoom         when you pinch...Wednesday, November 2, 11
Responsive UI                                           decoupled                     Rendering                           ...
Rendering vs Interaction                                                      Screen                              Backing ...
Checkerboard PatternWednesday, November 2, 11
Progressive Rendering                    Fast but blurry                                      Crisp but slowWednesday, Nov...
Tiling System                                   Texture upload                            ....               ....         ...
Tile Transformation           Panning = Translation   Zooming = ScalingWednesday, November 2, 11
Backing Store Support       Vector-based       Texture-based                            Honeycomb and laterWednesday, Nove...
Y U NO                            position:fixedWednesday, November 2, 11
DemoWednesday, November 2, 11
Layer & CompositingWednesday, November 2, 11
Elements = LayersWednesday, November 2, 11
Typical Animation                            opacity, movement, scaling, rotation, ...Wednesday, November 2, 11
Immediate Mode                            setInterval(function() {                                box.draw(x, y);         ...
Scene Graph             box.drawInto(layer);             setInterval(function() {                 layer.translate(10, 0); ...
Logical 3-DWednesday, November 2, 11
Keep the TexturesWednesday, November 2, 11
DemoWednesday, November 2, 11
Compositing Support                            Honeycomb and laterWednesday, November 2, 11
Well-known Trick                                                      forcing 3-D transform                            -we...
Debugging in Safari             defaults write com.apple.Safari IncludeInternalDebugMenuWednesday, November 2, 11
Compositing Indicators                            Texture (number = upload)                                         Compos...
Debugging in Chrome                  about:flagsWednesday, November 2, 11
Avoid Texture Reupload                            No reupload                          Upload                             ...
Avoid Overcapacity                              GPU = Limited silicon space                            Large layer ➔ broke...
Prepare & Reuse                                       Hide the layer offscreen                            ViewportWednesda...
Wrap UpWednesday, November 2, 11
Hardware Acceleration                Drawing     Backing     Layer &               primitives    stores   compositingWedne...
THANK YOU!Wednesday, November 2, 11
QUESTIONS?            ariya @ sencha.com             jarred @ sencha.com                 ariya.ofilabs.com                 ...
Upcoming SlideShare
Loading in …5
×

Hardware Acceleration on Mobile, Ariya Hidayat & Jarred Nicholls

3,765 views

Published on

GPU acceleration on mobile browsers, if it is leveraged correctly, can lead to a smooth and fluid applications, thus improving the user experience. There has been a lot of mentions and best practices of hardware acceleration these days, although so far it has been pretty general and hasn’t provided much technical direction apart from simple magical advice such as “use translate3d”. This talk sheds some more light on browser interactions with the GPU and explain what happens behind the scenes, covering the topic of acceleration of primitive drawing, the use of tiled backing store, and composited layer. Knowing the actual machinery behind hardware acceleration, you will be in the position to plan your strategy to improve the performance of your web application.

Published in: Technology, Art & Photos
  • Greate thanks, very useful!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Hardware Acceleration on Mobile, Ariya Hidayat & Jarred Nicholls

  1. 1. Wednesday, November 2, 11
  2. 2. Wednesday, November 2, 11
  3. 3. Hardware Acceleration on Mobile Ariya Hidayat & Jarred Nicholls Sencha Twitter: @ariyahidayat @jarrednichollsWednesday, November 2, 11
  4. 4. ChallengesWednesday, November 2, 11
  5. 5. Game vs WebWednesday, November 2, 11
  6. 6. Game Text Textured objects Animation Transformation PhysicsWednesday, November 2, 11
  7. 7. Web Pages Images TextWednesday, November 2, 11
  8. 8. How Browser Works Network stack Layout engine JavaScript engine Graphics User interfaceWednesday, November 2, 11
  9. 9. WebKit Components DOM CSS WebCore SVG HTML rendering JavaScriptCore WebKit LibraryWednesday, November 2, 11
  10. 10. Platform Abstraction Network Unicode Clipboard Graphics Theme Events Thread Geolocation TimerWednesday, November 2, 11
  11. 11. Graphics Abstraction GraphicsContext Mac Chromium Qt Gtk Skia CoreGraphics QPainter graphics stackWednesday, November 2, 11
  12. 12. Components of Mobile Radio Touch interface CPU Power GPUWednesday, November 2, 11
  13. 13. Graphics Processor Divide and conquer Very specific purposeWednesday, November 2, 11
  14. 14. Optimized for Games Fixed geometry Transformation Textured triangles ParallelismWednesday, November 2, 11
  15. 15. Challenges Predictable contents (assets) ✔ Mostly text ✔ Mostly images ✔ Expected run-time response immediateWednesday, November 2, 11
  16. 16. Primitive DrawingWednesday, November 2, 11
  17. 17. Path is Everything Triangle Rectangle Path Ellipse PolygonWednesday, November 2, 11
  18. 18. Stroke = Outline Solid Dashed Dotted TexturedWednesday, November 2, 11
  19. 19. Brush = Fill None Solid Gradient TexturedWednesday, November 2, 11
  20. 20. Smooth via Antialiasing Multiple levels of transparency Perfect for parallelismWednesday, November 2, 11
  21. 21. Path Approximation Curves PolygonWednesday, November 2, 11
  22. 22. Gradient: Expensive CPU: sequential, tedious GPU: parallel, still a lot of workWednesday, November 2, 11
  23. 23. Blur Shadow: Really Posh Involved pixel “blending” GPU: parallel, still tediousWednesday, November 2, 11
  24. 24. Transformation Scaling Perspective RotationWednesday, November 2, 11
  25. 25. Text RasterizationWednesday, November 2, 11
  26. 26. Font Atlas Buffer ByeWednesday, November 2, 11
  27. 27. Simple to Complex ello! H Simple shape Curves Solid color Gradient brush Textured stroke Blur shadow Make it as an Serif text image RotatedWednesday, November 2, 11
  28. 28. HOW FAST?Wednesday, November 2, 11
  29. 29. Android Drawing Log file WebCore graphics GraphicsContext SkiaWednesday, November 2, 11
  30. 30. Example: Bing platformInit savePlatformState translate 0,0 translate 0,0 clip 1,0 0x6.95322e-310 fillRect 0,0 800x556 color ff ff ff ff restorePlatformState platformDestroy platformInit savePlatformState translate 0,0 translate 0,0 clip 1,0 0x6.95322e-310 fillRect 0,0 800x556 color ff ff ff ff restorePlatformState platformDestroy platformInit savePlatformState translate 0,0 translate 0,0 clip 1,0 0x6.95322e-310 fillRect 0,0 800x556 color ff ff ff ff fillRect 0,0 800x556 color ff ff ff ffWednesday, November 2, 11
  31. 31. How Fast? #include "TimeCounter.h" bool WebViewCore::drawContent(SkCanvas* canvas, SkColor) { uint32_t timestamp = getThreadMsec(); .... painting code .... DBG_SET_LOGD("% ms", getThreadMsec() - timestamp); } external/webkit/WebKit/android/jni/WebViewCore.cppWednesday, November 2, 11
  32. 32. Example: Google News adb logcat -v time | grep drawContent 16:24:04.070 D/webcoreglue(  273): drawContent 11 ms 16:24:04.110 D/webcoreglue(  273): drawContent 13 ms 16:24:04.150 D/webcoreglue(  273): drawContent 13 ms 16:24:04.190 D/webcoreglue(  273): drawContent 10 ms 16:24:04.240 D/webcoreglue(  273): drawContent 10 ms 16:24:04.280 D/webcoreglue(  273): drawContent 13 ms 16:24:04.320 D/webcoreglue(  273): drawContent 13 ms 16:24:04.360 D/webcoreglue(  273): drawContent 13 ms 16:24:06.080 D/webcoreglue(  273): drawContent 12 ms 16:24:06.140 D/webcoreglue(  273): drawContent 10 ms 16:24:06.180 D/webcoreglue(  273): drawContent 13 ms 16:24:06.230 D/webcoreglue(  273): drawContent 14 ms 16:24:06.600 D/webcoreglue(  273): drawContent 26 ms 16:24:06.640 D/webcoreglue(  273): drawContent 13 ms 16:24:06.860 D/webcoreglue(  273): drawContent 33 ms 16:24:06.890 D/webcoreglue(  273): drawContent 12 ms 16:24:06.930 D/webcoreglue(  273): drawContent 13 ms 16:24:06.960 D/webcoreglue(  273): drawContent 13 ms 16:24:07.000 D/webcoreglue(  273): drawContent 13 msWednesday, November 2, 11
  33. 33. Backing StoreWednesday, November 2, 11
  34. 34. Maps TileWednesday, November 2, 11
  35. 35. Pinch to Zoom when you pinch...Wednesday, November 2, 11
  36. 36. Responsive UI decoupled Rendering Processor User interactionWednesday, November 2, 11
  37. 37. Rendering vs Interaction Screen Backing Store Rendering Web Page User interactionWednesday, November 2, 11
  38. 38. Checkerboard PatternWednesday, November 2, 11
  39. 39. Progressive Rendering Fast but blurry Crisp but slowWednesday, November 2, 11
  40. 40. Tiling System Texture upload .... .... CPU GPUWednesday, November 2, 11
  41. 41. Tile Transformation Panning = Translation Zooming = ScalingWednesday, November 2, 11
  42. 42. Backing Store Support Vector-based Texture-based Honeycomb and laterWednesday, November 2, 11
  43. 43. Y U NO position:fixedWednesday, November 2, 11
  44. 44. DemoWednesday, November 2, 11
  45. 45. Layer & CompositingWednesday, November 2, 11
  46. 46. Elements = LayersWednesday, November 2, 11
  47. 47. Typical Animation opacity, movement, scaling, rotation, ...Wednesday, November 2, 11
  48. 48. Immediate Mode setInterval(function() {     box.draw(x, y);     x += 10; }, 20); every animation tickWednesday, November 2, 11
  49. 49. Scene Graph box.drawInto(layer); setInterval(function() {     layer.translate(10, 0); Change transformation matrix }, 20);Wednesday, November 2, 11
  50. 50. Logical 3-DWednesday, November 2, 11
  51. 51. Keep the TexturesWednesday, November 2, 11
  52. 52. DemoWednesday, November 2, 11
  53. 53. Compositing Support Honeycomb and laterWednesday, November 2, 11
  54. 54. Well-known Trick forcing 3-D transform -webkit-transform: translateZ(0) Not needed for CSS animation!Wednesday, November 2, 11
  55. 55. Debugging in Safari defaults write com.apple.Safari IncludeInternalDebugMenuWednesday, November 2, 11
  56. 56. Compositing Indicators Texture (number = upload) Composited layer No texture Container layer OverflowWednesday, November 2, 11
  57. 57. Debugging in Chrome about:flagsWednesday, November 2, 11
  58. 58. Avoid Texture Reupload No reupload Upload Opacity Everything else! Position Size Animation “Hardware accelerated CSS”Wednesday, November 2, 11
  59. 59. Avoid Overcapacity GPU = Limited silicon space Large layer ➔ broken into “tiles”Wednesday, November 2, 11
  60. 60. Prepare & Reuse Hide the layer offscreen ViewportWednesday, November 2, 11
  61. 61. Wrap UpWednesday, November 2, 11
  62. 62. Hardware Acceleration Drawing Backing Layer & primitives stores compositingWednesday, November 2, 11
  63. 63. THANK YOU!Wednesday, November 2, 11
  64. 64. QUESTIONS? ariya @ sencha.com jarred @ sencha.com ariya.ofilabs.com ariyahidayat jarrednichollsWednesday, November 2, 11

×