The document discusses hardware acceleration in mobile graphics, detailing the challenges faced when transitioning from games to web environments. It covers topics such as rendering processes, the role of the GPU, path drawing, and the importance of responsive UI design. Additionally, it emphasizes optimization techniques for efficient rendering and the significance of compositing in modern mobile browsers.

1. Wednesday, November 2, 11

2. Wednesday, November 2, 11

3. Hardware Acceleration
on Mobile
Ariya Hidayat & Jarred Nicholls
Sencha
Twitter: @ariyahidayat @jarrednicholls
Wednesday, November 2, 11

4. Challenges
Wednesday, November 2, 11

5. Game vs Web
Wednesday, November 2, 11

6. Game
Text
Textured
objects
Animation
Transformation Physics
Wednesday, November 2, 11

7. Web Pages
Images
Text
Wednesday, November 2, 11

8. How Browser Works
Network stack
Layout engine
JavaScript engine Graphics
User interface
Wednesday, November 2, 11

9. WebKit Components
DOM CSS
WebCore SVG
HTML
rendering
JavaScriptCore
WebKit Library
Wednesday, November 2, 11

10. Platform Abstraction
Network Unicode Clipboard
Graphics Theme Events
Thread Geolocation Timer
Wednesday, November 2, 11

11. Graphics Abstraction
GraphicsContext Mac Chromium Qt Gtk
Skia
CoreGraphics
QPainter
graphics stack
Wednesday, November 2, 11

12. Components of Mobile
Radio
Touch interface
CPU
Power
GPU
Wednesday, November 2, 11

13. Graphics Processor
Divide and conquer
Very specific purpose
Wednesday, November 2, 11

14. Optimized for Games
Fixed geometry
Transformation
Textured triangles
Parallelism
Wednesday, November 2, 11

15. Challenges
Predictable contents (assets) ✔
Mostly text ✔
Mostly images ✔
Expected run-time response immediate
Wednesday, November 2, 11

16. Primitive Drawing
Wednesday, November 2, 11

17. Path is Everything
Triangle Rectangle
Path
Ellipse Polygon
Wednesday, November 2, 11

18. Stroke = Outline
Solid Dashed Dotted Textured
Wednesday, November 2, 11

19. Brush = Fill
None Solid Gradient Textured
Wednesday, November 2, 11

20. Smooth via Antialiasing
Multiple levels of transparency
Perfect for parallelism
Wednesday, November 2, 11

21. Path Approximation
Curves
Polygon
Wednesday, November 2, 11

22. Gradient: Expensive
CPU: sequential, tedious
GPU: parallel, still a lot of work
Wednesday, November 2, 11

23. Blur Shadow: Really Posh
Involved pixel “blending”
GPU: parallel, still tedious
Wednesday, November 2, 11

24. Transformation
Scaling
Perspective
Rotation
Wednesday, November 2, 11

25. Text Rasterization
Wednesday, November 2, 11

26. Font Atlas
Buffer
Bye
Wednesday, November 2, 11

27. Simple to Complex
ello!
H
Simple shape Curves
Solid color Gradient brush
Textured stroke
Blur shadow
Make it as an Serif text
image Rotated
Wednesday, November 2, 11

28. HOW FAST?
Wednesday, November 2, 11

29. Android Drawing
Log file
WebCore
graphics
GraphicsContext
Skia
Wednesday, November 2, 11

30. Example: Bing
platformInit
savePlatformState
translate 0,0
translate 0,0
clip 1,0 0x6.95322e-310
fillRect 0,0 800x556 color ff ff ff ff
restorePlatformState
platformDestroy
platformInit
savePlatformState
translate 0,0
translate 0,0
clip 1,0 0x6.95322e-310
fillRect 0,0 800x556 color ff ff ff ff
restorePlatformState
platformDestroy
platformInit
savePlatformState
translate 0,0
translate 0,0
clip 1,0 0x6.95322e-310
fillRect 0,0 800x556 color ff ff ff ff
fillRect 0,0 800x556 color ff ff ff ff
Wednesday, November 2, 11

31. How Fast?
#include "TimeCounter.h"
bool WebViewCore::drawContent(SkCanvas* canvas, SkColor)
{
uint32_t timestamp = getThreadMsec();
.... painting code ....
DBG_SET_LOGD("% ms", getThreadMsec() - timestamp);
}
external/webkit/WebKit/android/jni/WebViewCore.cpp
Wednesday, November 2, 11

32. Example: Google News
adb logcat -v time | grep drawContent
16:24:04.070 D/webcoreglue( 273): drawContent 11 ms
16:24:04.110 D/webcoreglue( 273): drawContent 13 ms
16:24:04.150 D/webcoreglue( 273): drawContent 13 ms
16:24:04.190 D/webcoreglue( 273): drawContent 10 ms
16:24:04.240 D/webcoreglue( 273): drawContent 10 ms
16:24:04.280 D/webcoreglue( 273): drawContent 13 ms
16:24:04.320 D/webcoreglue( 273): drawContent 13 ms
16:24:04.360 D/webcoreglue( 273): drawContent 13 ms
16:24:06.080 D/webcoreglue( 273): drawContent 12 ms
16:24:06.140 D/webcoreglue( 273): drawContent 10 ms
16:24:06.180 D/webcoreglue( 273): drawContent 13 ms
16:24:06.230 D/webcoreglue( 273): drawContent 14 ms
16:24:06.600 D/webcoreglue( 273): drawContent 26 ms
16:24:06.640 D/webcoreglue( 273): drawContent 13 ms
16:24:06.860 D/webcoreglue( 273): drawContent 33 ms
16:24:06.890 D/webcoreglue( 273): drawContent 12 ms
16:24:06.930 D/webcoreglue( 273): drawContent 13 ms
16:24:06.960 D/webcoreglue( 273): drawContent 13 ms
16:24:07.000 D/webcoreglue( 273): drawContent 13 ms
Wednesday, November 2, 11

33. Backing Store
Wednesday, November 2, 11

34. Maps
Tile
Wednesday, November 2, 11

35. Pinch to Zoom
when you pinch...
Wednesday, November 2, 11

36. Responsive UI
decoupled
Rendering
Processor
User interaction
Wednesday, November 2, 11

37. Rendering vs Interaction
Screen
Backing Store
Rendering
Web Page User interaction
Wednesday, November 2, 11

38. Checkerboard Pattern
Wednesday, November 2, 11

39. Progressive Rendering
Fast but blurry
Crisp but slow
Wednesday, November 2, 11

40. Tiling System
Texture upload
.... ....
CPU GPU
Wednesday, November 2, 11

41. Tile Transformation
Panning = Translation Zooming = Scaling
Wednesday, November 2, 11

42. Backing Store Support
Vector-based
Texture-based
Honeycomb and later
Wednesday, November 2, 11

43. Y U NO
position:fixed
Wednesday, November 2, 11

44. Demo
Wednesday, November 2, 11

45. Layer & Compositing
Wednesday, November 2, 11

46. Elements = Layers
Wednesday, November 2, 11

47. Typical Animation
opacity, movement, scaling, rotation, ...
Wednesday, November 2, 11

48. Immediate Mode
setInterval(function() {
box.draw(x, y);
x += 10;
}, 20);
every animation tick
Wednesday, November 2, 11

49. Scene Graph
box.drawInto(layer);
setInterval(function() {
layer.translate(10, 0); Change transformation
matrix
}, 20);
Wednesday, November 2, 11

50. Logical 3-D
Wednesday, November 2, 11

51. Keep the Textures
Wednesday, November 2, 11

52. Demo
Wednesday, November 2, 11

53. Compositing Support
Honeycomb and later
Wednesday, November 2, 11

54. Well-known Trick
forcing 3-D transform
-webkit-transform: translateZ(0)
Not needed for CSS animation!
Wednesday, November 2, 11

55. Debugging in Safari
defaults write com.apple.Safari IncludeInternalDebugMenu
Wednesday, November 2, 11

56. Compositing Indicators
Texture (number = upload)
Composited layer
No texture
Container layer
Overflow
Wednesday, November 2, 11

57. Debugging in Chrome
about:flags
Wednesday, November 2, 11

58. Avoid Texture Reupload
No reupload Upload
Opacity Everything else!
Position
Size
Animation
“Hardware accelerated CSS”
Wednesday, November 2, 11

59. Avoid Overcapacity
GPU = Limited silicon space
Large layer ➔ broken into “tiles”
Wednesday, November 2, 11

60. Prepare & Reuse
Hide the layer offscreen
Viewport
Wednesday, November 2, 11

61. Wrap Up
Wednesday, November 2, 11

62. Hardware Acceleration
Drawing Backing Layer &
primitives stores compositing
Wednesday, November 2, 11

63. THANK YOU!
Wednesday, November 2, 11

64. QUESTIONS?
ariya @ sencha.com jarred @ sencha.com
ariya.ofilabs.com
ariyahidayat jarrednicholls
Wednesday, November 2, 11