The document discusses the complexities involved in the processing and display of images on the web, including factors such as network latency, bandwidth, and memory utilization. It emphasizes the importance of efficient memory management and power consumption to enhance user satisfaction, especially on low-cost devices. Additionally, the document highlights various image formats and their impact on performance, detailing technical aspects from downloading to displaying images.

1. IMAGECON 2017
The Physics of Fast Graphics
Tobin Titus
@tobint

2. IMAGECON 2017
The Impact of Images
 100 resource requests, 18 Domains, 33 TCP Connections
54 7 23 9 7
Images CSS JavaScript HTML Other
1627 81 408 48 101
Images CSS JavaScript HTML Other
 2.467 MB Total Size, 51% Cacheable

3. IMAGECON 2017
Seemingly simplistic …
Download Decode Display

4. IMAGECON 2017
.. but surprisingly complex
Network
Disk
CPU
Memory
Memory Bus
GPU
Display
Battery
GB

5. IMAGECON 2017
Responsiveness makes it more complex!

6. IMAGECON 2017
Download
Getting data to your device

7. © 2012 mass:werk

8. IMAGECON 2017
Bandwidth vs Throughput
Bandwidth Throughput

9. IMAGECON 2017
Global Mobile Network Speeds
0.1
3.3
10.8
17.4
0
2
4
6
8
10
12
14
16
18
20
Speeds(Mbps)
2G 3G WiFI 4G LTE
Source:
State of LTE – Nov 2016

10. IMAGECON 2017
Overhead Plays Significant Role
NYC to: Latency
KM Miles Vacuum Fiber RTT 1 Packet
Seattle 3,870 2,405 12.9 19.7 39.3 157.6
London 5,576 3,465 18.6 28.3 56.6 226.4
Beijing 11,001 6,835 36.7 55.9 111.7 447.2
TCP
TLS
Data

11. IMAGECON 2017
Last Mile Latency
Source:
Measuring Broadband America - 2016

12. IMAGECON 2017
Decode
Unpacking the Downloaded Image File

13. IMAGECON 2017
Download Size vs Memory Size
Download sizes
may seem small,
but image
memory adds up
very quickly
683 x 1024
RGBA
557 KB
width * height * 4
2.67 MB Decoded

14. IMAGECON 2017
Memory Utilization
Responsible
memory
management
is crucial when
targeting low-
cost devices
with your sites
and apps
-
20
40
60
80
100
120
140
160
180
200
Megabytes
5000 sites’ image memory utilization

15. Images: Downloading
Decoding

16. Images: Continue Downloading
Decoding

17. Images : Decoding
Decoding

18. Processing
Images: Processing
Decoding

19. Images: Copy to the GPU
ProcessingDecoding

20. Images: Download to Display
ProcessingDecoding

21. Images: Download to Display
ProcessingDecoding

22. IMAGECON 2017
Display
Showing Your Images to The User

23. IMAGECON 2017
Image Formats Matter
Image
formats use
different
means to
depict colors
RGB to YCbCr Color
Conversion
Chroma
Subsampling
DCT /
Quantization
Huffman
Encoding

24. Chroma Subsampling
+
Original Luma (Y) Chroma (Cb) Chroma (Cr)
= + +
4:4:4
Original Luma (Y) Chroma (Cb) Chroma (Cr)
= +4:2:0
Original Luma (Y) Chroma (Cb) Chroma (Cr)
= + +4:2:2
62.5% less memory than RGB
(no subsampling)

25. JPEG images
were previously
decoded,
upsampled and
color-converted
on the CPU
before copying
them to the GPU
JPEG – From Download to Display
Huffman Decoding
Dequantization /
iDCT
Chroma
Upsampling
YCbCr to RGB
Color Conversion

26. IMAGECON 2017
Drain!
Powering your Device without Overheating

27. Power Consumption
Let it rest!
Power
efficiency can
drain your
users’ battery
and decrease
satisfaction
with your
application
CPU
Utilization
GPU
UtilizationVSync

28. Power Consumption
Let it rest!
Power
efficiency can
drain your
users’ battery
and decrease
satisfaction
with your
application
CPU
Utilization
GPU
UtilizationVSync

29. Power Consumption
Let it rest!
Power
efficiency can
drain your
users’ battery
and decrease
satisfaction
with your
application
CPU
Utilization
GPU
UtilizationVSync

30. IMAGECON 2017
Memory And Power Consumption
Maintaining
memory
consumes power
and generates
heat. The sites to
the left of the
graph use far less
power than the
sites on the right.
-
20
40
60
80
100
120
140
160
180
200
Megabytes
5000 sites’ image memory utilization

31. IMAGECON 2017
Navigation App or Site
Network / GPS
CPU
GPU
MemoryGB

32. IMAGECON 2017
Resources
Overview Concepts
High Performance Websites
Steve Souders, September 2007
Event Faster Websites: Best Practices
Steve Souders, June 2009
High Performance Browser Networking
Ilya Grigorik, September 2013
A Picture Costs a Thousand Words
Guy Podjamy, November 2013
High Performance Images
Colin Bendell, Tim Kadlec, Yoav Weiss,
Guy Podjarny, Kick Doyle & Mike McCall
November 2016
Microsoft Guidance
High-Performance Web Platform
Tobin Titus, Build 2014
Web Runtime Performance
Tobin Titus, Build 2013
Performance Tricks to Make Apps & Sites Faster
Jatinder Mann, Build 2012
50 Performance Tricks for Windows Store Apps
Jason Weber, Build 2011
W3C Web Performance
Web Performance Working Group Homepage
Blog Posts
1) JPEGXR updates (comparisons to WebP)
2) Debugging/Tuning Web Sites w/F12 Developer
Tools
3) Measuring Performance with ETW/XPerf
4) Measuring Performance in Lab Environments
5) Browser Subsystems Overview
6) What Common Benchmarks Measure
Tools
Windows Performance Toolkit
Fiddler Web Debugger
Contact
Tobin Titus
Twitter: @tobint
Email: tobint@microsoft.com

33. IMAGECON 2017