This document discusses the process of rendering views and pixels on the screen from a high level object to pixels. It covers the key steps of measure, layout, and draw and explains important concepts like double buffering, vsync, and avoiding overdraw. Key topics include how the GPU helps with rasterization, using viewtreeobserver for size changes, different root view types, and using constraint layout for a more optimized layout. It emphasizes best practices like minimizing unnecessary layout requests and using tools like hierarchy viewer and GPU profiling.

1. View To Pixel
Britt Barak
26/6/2017
Wifi: GoogleGuestPSK
pass: pUp3EkaP
3

2. Jonathan
Yarkoni
Android Developer &
Advocate
Ironsource
Android Academy Staff
Yonatan Levin
Android
Google Developer
Expert
Yossi Segev
Android Developer
Colu
Shahar
Avigezer
Android Developer
Hello Heart
Britt Barak
Android Lead
Figure8

3. Britt Barak
Britt Barak
Adventures
Android Academy
Women Techmakers

4. The Largest
Android Community
Android Academy - TLV
TLV - Android Academy
Join Us:Fundamentals
Advanced
With designers
Hackathons
Mentors

7. Has this ever happened to you?

8. Why do we love apps?

9. Smooth Experience

10. How Is Motion Perceived?

11. How Is Motion Perceived?
Smooth
Motion
60
No
Difference
60+
Flip Book
12
Movies
Frames Per Second
Fluid Motion
24
+effects

12. We Have A Winner!
Smooth
Motion
60
No
Difference
60+
Flip Book
12
Movies
Frames Per Second
Fluid Motion
24
+effects
Smooth
Motion
60
Colt McAnlis: https://youtu.be/CaMTIgxCSqU

13. 60 FPS
60 Frames / Second =
Frame / 16.666 Millisecond

14. But First Thing First
But First Thing First
Today’s Story

15. How Did My Code Become Pixels?
CPU
new
View()

16. Rasterization
From a high level object
into pixels on screen (or in texture).
https://en.wikipedia.org/wiki/Rasterisation

17. GPU - For The Rescue !

18. GPU - For The Rescue!
CPU
new
View()
GPU

19. GPU - For The Rescue!
CPU
new
View()
GPU
Polygons
Textures

20. GPU - For The Rescue!
CPU
new
View()
GPU
Polygons
Textures
DisplayList

21. GPU - For The Rescue!
CPU
new
View()
GPU
Polygons
Textures
DisplayList

22. What do we need to do in these 16ms?

23. Measure
CPU
new
View()
GPU

24. Layout
CPU GPU

25. Draw
CPU GPU
new
View()
DisplayList

26. Sync & Upload
CPU
new
View()
GPU
Polygons
Textures
DisplayList

27. Execute
CPU
new
View()
GPU
Polygons
Textures
DisplayList

28. Execute
CPU
new
View()
GPU
Polygons
Textures
DisplayList
Frame
Buffer

29. What happens if

30. What happens if
CPU GPU

31. What happens if
CPU GPU

32. What happens if
CPU GPU

34. What do we do?

35. Double buffer

36. CPU
new
View()
GPU
Polygons
Textures
DisplayList
Back
Buffer
Frame
Buffer

37. CPU
new
View()
GPU
Polygons
Textures
DisplayList
Back
Buffer
Frame
Buffer

38. CPU
new
View()
GPU
Polygons
Textures
DisplayList
Back
Buffer
Frame
Buffer

39. CPU
new
View()
GPU
Polygons
Textures
DisplayList
Back
Buffer
Frame
Buffer

40. vsync

41. So What Does That Mean?
Measure/ Layout
Draw
Sync & Upload
Execute
Swap Buffers
VSync/Misc

42. actually...

43. input

44. animation

45. Invalidation
requestLayout

46. When Things Change...
E.g. text, color, size, padding, margin...
A view notifies the system, by calling:
Invalidate - which will call the onDraw again, or
requestLayout - which will call the entire process again.

47. When Is Request Layout Called?
View1 View2

48. When Is Request Layout Called?
View2View1

49. When Is Request Layout Called?
View2View1

50. ●Adds a flag
●Recreate displaylist in next frame
invalidate()

51. requestLayout()
Bubble up to the root view.
Heavy for deep view hierarchy.
Called once a frame.

52. cheographer

53. The all-mighty LogCat
I/Choreographer(1378): Skipped 55 frames! The application may be doing too much work
on its main thread.

54. Order Is Guaranteed

55. VSync/Misc
Input
Animation
Measure/ Layout
Draw
Sync & Upload
Command Issue
Swap Buffers
So What Does That Mean?

56. Questions?

57. Reminder
Measure Layout Draw!

58. Step 1: Measure
Goal: obtain view size,
including its descendants size,
agreed by its parent.
REF: http://developer.android.com/reference/android/view/View.html#onMeasure(int, int)

59. ViewGroup.LayoutParams
How big the View wants to be
For each dimension, it can specify one of:
an exact number
MATCH_PARENT
WRAP_CONTENT
*Might be subclassed, like in RelativeLayout

60. MeasureSpec
Parent requirement for child
UNSPECIFIED: as child wants
EXACTLY: exact size
AT MOST: maximum size

61. Multiple measure() Calls
measure() may be called more than once
For example:
- Call once with UNSPECIFIED
- If size too big/small- evaluate
- Call with exact size

62. View.getWidth() == 0…
What’s up with that?!

63. Step 1: Measure - What’s View SIZE?
- width & height (aka: drawing width & drawing height) -
- Actual size on screen, at drawing time and after layout.
REF: http://developer.android.com/reference/android/view/View.html#onMeasure(int, int)

64. Step 1: Measure - What’s View SIZE?
- width & height (aka: drawing width & drawing height) -
- Actual size on screen, at drawing time and after layout.
- measured width & measured height
- how big a view wants to be within its parent
REF: http://developer.android.com/reference/android/view/View.html#onMeasure(int, int)

65. What if we want the
size during an animation?

66. ViewTreeObserver
●Provided by the view hierarchy
●Use to register listeners to view tree global changes
○ For example: layout of the whole tree, beginning of the drawing, touch mode
change....
●Use with getViewTreeObserver()
https://developer.android.com/reference/android/view/ViewTreeObserver.html

67. onPreDrawListener

68. Any Questions?

69. The Steps
Measure Layout Draw!

70. Step 2: Layout
Goal : set position for view
and all its children,
respect parent’s constraints.
REF: http://developer.android.com/reference/android/view/View.html#onLayout(boolean, int, int, int, int)

71. How many layout passes are there?

72. Root View : RelativeLayout
2 passes per change:
1)By views’ requests
a)Then relativeLayout calculates by relations & weights
2)Determine the final positions for rendering.

73. Root View : RelativeLayout
Example:
●Every view is RelativeLayout
●Per Leaf change:
2*2*2 = 8 traversals
X2
X2
X2

74. Root View : LinearLayout
●measureWithLargestChild(true)
○ Children with weights will have minimum size of largest child
●Nested weights

75. Root View : GridLayout
Allows relative positioning,
while preprocessing the child view relationships.

76. Root View : GridLayout
Generally issues 1 layout request
Unless:
●Layout_gravity: fill*
●Wrong weights

77. This is called
Double Layout Taxation

78. It Usually Works Fine… But...
●Root elements
●Deep view hierarchy
●Too many (i.e. list)
Can hurt.

79. What can we do?
●Flatten hierarchy :
Removing unneeded views
Merge on include
flatter layout
Beware double layout taxation
Minimize layout requests

80. Cool Tool
Hierarchy Viewer

81. Hierarchy Viewer
Tools→ Android → Android Device Monitor → Hierarchy Viewer
Display complete view hierarchy
Access to all the views’ properties
Get performance stats

82. Hierarchy Viewer

83. Hierarchy Viewer
- Green dot : in the faster 50%
of all the View objects
- Yellow : in the slower than 50%
- Red: the slowest
- requestLayout:

84. Any Questions?

85. Ok ok ,
but which layout should I choose??

86. Simple VS Complex Layouts
●Simple : LinearLayout, FrameLayout, TableLayout
●Complex : RelativeLayout , GridLayout

87. Simple VS Complex Layouts
●Simple : LinearLayout, FrameLayout, TableLayout
○ → Nesting → Performance Problems
●Complex : RelativeLayout , GridLayout
○ → hard to use & maintain
○ → shouldn’t be @ top hierarchy

88. Isn’t there anything
smarter we can do??

89. Constraint Layout

90. Constraint Layout
●Api 9+
●Performance oriented:
○ Reduce nesting
○ Improve measure/layout
○ Best practice : top level, without nesting layouts
User friendly:
Expressive

91. New Layout Editor
Designed for
ConstraintLayout -
but not only!

92. Concept
Similar to RelativeLayout :
Position a view relative to other layout elements.
Yet more expressive & powerful

93. Getting Started
Design Mode Blueprint Mode

95. Among Capabilities
- Bias
- Ratio
- Center
- ConstraintSet

96. Possible Attributes
layout_constraintX_toYOf
X and Y can be replaced with:
a. Top
b. Bottom
c. Left
d. Right
e. Start
f. End
g. Baseline

97. Converting To ConstraintLayout
1.Open layout on editor
2.right-click the layout → Convert <layout> to ConstraintLayout.

98. Well, that’s about it!
Not officially released, but quite stable.
Give it a try :)
https://codelabs.developers.google.com/codelabs/constraint-layout/index.html

99. Any Questions?

100. The Steps
Measure Layout Draw!

101. - View draws itself with size and position from previous steps.
- onDraw(Canvas) is called
- Canvas object generates (or Updates) a list of OpenGL-ES
commands (displayList) to send to the GPU.
Step 3: Draw (AKA Update)
REF: http://developer.android.com/reference/android/view/View.html#onDraw(android.graphics.Canvas)
Guide: http://developer.android.com/training/custom-views/custom-drawing.html

102. What Can Go Wrong?

103. Allocating objects (new XXX()) might cause a GC (blocking)
So you might drop a frame.
Allocations in onDraw()
But it goes much deeper. See Ian Ni-Lewis: https://youtu.be/HAK5acHQ53E

104. B
The number of files that a given pixel is
drawn in a frame.
Overdraw
Colt McAnlis: https://youtu.be/T52v50r-JfE

105. Detecting Overdraw

106. Detecting Overdraw
https://developer.android.com/studio/profile/dev-options-

107. Codelab: https://io2015codelabs.appspot.com/codelabs/android-

108. Fixing Overdraw
There are 2 common reasons for overdraw:
- Redundant backgrounds / Redundant transparency
- Wasteful onDraw
- Things that aren’t visible at all gets drawn (not using quickReject)
- Things that will be overdrawn gets drawn (not using clipRect)
Colt McAnlis: https://youtu.be/vkTn3Ule4Ps

109. QuickReject
A method to tell if something can be not drawn at all.
Call quickReject to see if you can skip drawing of things that will be
off screen.
REF: https://developer.android.com/reference/android/graphics/Canvas.html

110. ClipRect
ClipRect is a way to avoid OverDraw,
By keeping your GPU from drawing pixels that you know that will be
obscured by other stuff,
you refrain from overdraw.

111. Step 1:
quickReject
Step 2:
clipRect

112. ClipRect vs. QuickReject
Method return type: Detects... Helps to...
QuickReject boolean Fully invisible stuff Avoid redundant
calls to drawXXX(),
Keeping the
drawlist short.
ClipRect void Fully and Partially
invisible stuff
Avoid drawing
pixels,
but still executing
the draw-list!
boolean
But is usually used
as void

113. Any Questions?

114. Profiling tools

115. VSync/Misc
Input
Animation
Measure/ Layout
Draw
Sync & Upload
Command Issue
Swap Buffers
So What Does That Mean?

116. GPU Profiling Tool

117. GPU Profiling
● A graph per visible app
● A bar per frame
● Render time corresponds height
● 16 millis benchmark (green)
● Crossing = skipping frame
https://developer.android.com/studio/profile/dev-options-rendering.html

118. 16ms line
Dropped
Frames :(
14
accented
if >16ms
Shiney
colors!

119. GPU Monitor (Android Studio)

120. systrace

125. Vitals dashboard

127. Frame Metrics
On Android N+
- Per Frame data:
- Windos.OnFrameMetricsAvailableListener()
- Aggregated
- FrameMetricsAggregator (Support lib v26)

128. https://developer.android.com/reference/android/view/FrameMet

129. VSync/Misc
Input
Animation
Measure/ Layout
Draw
Sync & Upload
Command Issue
Swap Buffers
So What Does That Mean?

130. Vsync / Misc Time
Misc = (vsync timestamp) - (timestamp when received)
→ work on the UI thread between two consecutive frames
● Choreographer log : “Missed vsync by XX ms skipping XX frames”
High? move work off UI Thread
VSync/
Misc
Input
Animation
Measure/
Layout
Draw
Sync &
Upload
Command
Issue
Swap
Buffers

131. Input Handling
App code inside an input event callback.
High? optimize/offload processing input
VSync/
Misc
Input
Animation
Measure/
Layout
Draw
Sync &
Upload
Command
Issue
Swap
Buffers

132. Animation
Evaluate all running animators
Often : object animator, view property animator, and transitions
High?(2milis+) check custom animators / unintended work
VSync/
Misc
Input
Animation
Measure/
Layout
Draw
Sync &
Upload
Command
Issue
Swap
Buffers

133. Measure / Layout
Executing layout and measure callbacks for needed view.
More on that in a few slides :)
VSync/
Misc
Input
Animation
Measure/
Layout
Draw
Sync &
Upload
Command
Issue
Swap
Buffers

134. Draw
update (/creates) all display lists
+ cache.
High?
complex onDraw() logic
many views invalidated
VSync/
Misc
Input
Animation
Measure/
Layout
Draw
Sync &
Upload
Command
Issue
Swap
Buffers

135. Sync & Upload
Upload bitmap to the GPU.
High?
Too many images
Too big of an image
VSync/
Misc
Input
Animation
Measure/
Layout
Draw
Sync &
Upload
Command
Issue
Swap
Buffers

136. Command Issue
Execution: Android's 2D renderer issuing commands to OpenGL to draw and
redraw all display lists
High?
●Complex view (custom?)
●Many views redrawing:
○ Invalidation
○ animationVSync/
Misc
Input
Animation
Measure/
Layout
Draw
Sync &
Upload
Command
Issue
Swap
Buffers

137. Swap Buffers
CPU done rendering and wait for GPU ack.
High?
A lot of GPU work
VSync/
Misc
Input
Animation
Measure/
Layout
Draw
Sync &
Upload
Command
Issue
Swap
Buffers

138. Any Questions?

139. What Happened Here Today?

140. How Does A View Display On Screen?
ScreenGPU
Polygons
Textures
CPU
new
Button()
DisplayList

141. tools
- Hierarchy viewer
- Gpu overdraw
- Gpu profiler
- Systrace
- Vitals dashboard

142. Any Questions?

143. Thank You !

144. Community announcements