Presented Nov 25, 2018 Novosibrisk, Russia

1. Delivering Beautiful and Fast
Images and Video
Doug Sillars
@DougSillars
DevFest Siberia
November 24, 2018

2. Contact Me:
@DougSillars
Doug.Sillars@gmail.com
www.dougsillars.com
Doug Sillars
Freelance Developer Relations
Performance Audits: Web/Native
Workshops:
Performance/Images/Video
http://bit.ly/HighPerformanceAndroidApps

4. 0.5
0.6
0.7
0.8
0.9
Standing in Line Standing on the
edge of a virtual
cliff
Experiencing
Mobile Delays
Solving a Math
Problem
https://www.ericsson.com/res/docs/2016/mobility-report/emr-feb-2016-the-stress-of-steaming-delays.pdf
Stress

5. Large Downloads Induce Delays in Rendering
3s: 53% of Users Abandon Mobile Sites
500ms: 26% Frustration
8% Engagement
100ms: 1% Revenue Walmart & Amazon (Desktop 2001)
4% Mobile Users Throw Their Phones
https://www.doubleclickbygoogle.com/articles/mobile-speed-matters
http://bit.ly/mobileWebStress
http://www.globaldots.com/how-website-speed-affects-conversion-rates/
https://www.mobilejoomla.com/blog/172-responsive-design-vs-server-side-solutions-infographic.html

6. Images & Video make up 75% of Web Content
HTTPArchive mobile websites 15/02/18

7. Images and Video Can be Expensive

8. 7.9 MB

9. 3.5 MB

10. 4 Simple Image Optimizations

11. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

12. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

13. 4 Simple Image Optimizations
Httparchive.org Webpagetest.org

14. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

15. Image Quality
magick -quality 85 riga.jpg riga85.jpg
http://res.cloudinary.com/dougsillars/image/upload/q_85/v1520504964/IMG_20180301_114117_tzasan.jpg
Lighthouse:
Recommends 85% quality on all images
https://developers.google.com/speed/docs/insights/OptimizeImages

16. 100%
3.6 MB
http://res.cloudinary.com/dougsillars/image/upload/v1529005982/IMG_20180614_184507_ssuk1i.jpg

17. 85%
1.87 MB
q_85
http://res.cloudinary.com/dougsillars/image/upload/q_85/v1529005982/IMG_20180614_184507_ssuk1i.jpg

18. Image Quality Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

19. Image Quality Use “In The Wild”
Median Savings (50th percentile):
• 2.83 seconds faster page load
• 419KB less data

20. 50%
914 KB
q_50
http://res.cloudinary.com/dougsillars/image/upload/q_50/v1529005982/IMG_20180614_184507_ssuk1i.jpg

21. 20%
513 KB
q_20
http://res.cloudinary.com/dougsillars/image/upload/q_20/v1529005982/IMG_20180614_184507_ssuk1i.jpg

22. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

23. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

24. Automate Quality vs. File Size
• Butteraugli
• SSIM: Structural SIMilarity
https://github.com/technopagan/cjpeg-dssim
cjpeg-dssim jpegoptim riga.jpg
http://res.cloudinary.com/dougsillars/image/upload/q_auto/v1529005982/IMG_20180614_184507_ssuk1i.jpg

25. SSIM
1.46 MB
http://res.cloudinary.com/dougsillars/image/upload/q_auto/v1529005982/IMG_20180614_184507_ssuk1i.jpg

26. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

27. Results:
• Test Load on Motorola G4:

28. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

29. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

30. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

31. Scalable Vector Graphics (SVG)
Images drawn as shapes
infinitely scalable
XML - Can be added inline to HTML document
https://gist.github.com/hail2u/2884613?short_path=66a60ff

32. Scalable Vector Graphics (SVG)
Images drawn as shapes
infinitely scalable
XML - Can be added inline to HTML document
https://gist.github.com/hail2u/2884613?short_path=66a60ff

33. Scalable Vector Graphics (SVG)

34. Scalable Vector Graphics (SVG)

35. Scalable Vector Graphics (SVG)
https://github.com/google/brotli

36. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

37. File Format: WebP
https://caniuse.com/#feat=webp

39. File Format: WebP
https://caniuse.com/#feat=webp

40. File Format: WebP
https://caniuse.com/#feat=webp

41. SSIM + WebP
986 KB
http://res.cloudinary.com/dougsillars/image/upload/q_auto,f_auto/v1529005982/IMG_20180614_184507_

42. File Format: Web
<picture>
<source width = "100%" type="image/webp" srcset=”riga.webp">
<img width = "100%" src=”riga_cjpeg_dssim.jpg"
alt=”Riga, Latvia">
</picture>

43. Image Format Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

44. Image Format Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

45. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

46. Image Sizing
-

47. Image Sizing

48. Image Sizing
-

49. Image Sizing
624
832
-

50. Image Sizing
624
832
12,979,000 pixels
519,000 pixels
_________________
-
12,460,000 pixels

51. Image Sizing
624
832
12,979,000 pixels
519,000 pixels
_________________
-
12,460,000 pixels

52. Image Sizing
https://twitter.com/paulcalvano/status/928751141843808256

53. Image Sizing
-
Responsive Images:
Generate a set of images
25 KB difference in size

54. Responsive Images
624
832
625,000 pixels
-
106,000 pixels
519,000 pixels
_________________

55. Responsive Breakpoint Generation
https://github.com/cloudinary/responsive_breakpoints_generator
http://www.responsivebreakpoints.com/

56. Responsive Images

57. Responsive Images

58. Responsive Images Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

59. Responsive Images Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

60. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

61. Lazy Load
https://calendar.perfplanet.com/2017/progressive-image-loading-using-intersection-observer-and-sqip/

62. Lazy Load
https://calendar.perfplanet.com/2017/progressive-image-loading-using-intersection-observer-and-sqip/

63. Lazy Loading Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

64. Lazy Loading Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

65. Preview Images

66. Optimizing Content Delivery
Preview Images
https://github.com/technopagan/sqip

67. Lazy Loading: Experiments

68. Lazy Loading: Chrome Experiments

69. Lazy Loading: Chrome Experiments

70. Animated GIFs
Original MP4
1.4 MB

71. Animated GIFs

72. Animated GIFs
Animated GIF
3.8 MB
270% larger

73. Animated GIFs

74. Animated GIFs
MP4: 256 colors
247KB
93% smaller

75. Animated GIFs
Video Tags:
<video loop autoplay muted playsinline controls = "false” src="goats.mp4”/>
Video is not pre-loaded, will be last to download
Img tags are fast!
<picture>
<source type="video/mp4" srcset=”goats.mp4">
<source type="image/webp" srcset=”goats.webp">
<img src=”goats.gif">
</picture>
https://calendar.perfplanet.com/2017/animated-gif-without-the-gif/

76. Animated GIFs

77. Video

78. 78
Video
Buffer Rage: a state of uncontrollable
fury or violent anger induced by delayed
or interrupted streaming video content
http://www.ineoquest.com/press-releases/new-research-reveals-buffer-rage-as-techs-newest-epidemic

79. Video Quality Metrics
1.Does the Video Start?
2.Does the Video Stall?
3.Does it Look Good?

80. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018:

81. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
16.9B total Video plays
400M Fail to Start
2B Abandoned before Start
~800M hours of video
playback lost

82. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
16.9B total Video plays
400M Fail to Start
2B Abandoned before Start
~800M hours of video
playback lost

83. Video Fails To Start

84. Video Startup Failure

85. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
16.9B total Video plays
400M Fail to Start
2B Abandoned before Start

86. Video Startup Delay
After 2 seconds,
every additional
second corresponds
to 5.8% increase in
abandonment
https://www.akamai.com/kr/ko/multimedia/documents/technical-publication/video-stream-quality-impacts-viewer-behavior-inferring-causality-using-quasi-experimental-designs-technical-publication.pdf

87. Video Startup Delay
https://www.akamai.com/kr/ko/multimedia/documents/technical-publication/video-stream-quality-impacts-viewer-behavior-inferring-causality-using-quasi-experimental-designs-technical-publication.pdf

88. Preload When It Makes Sense
Ad Playback Video
Download

89. Preload When It Makes Sense
Ad Playback Video
Download

90. Preload When It Makes Sense
Ad Playback
Video
Download

91. ..be Careful with Video Preload
Preload = “auto”

92. …be Careful with Preload

93. Background Video

94. Video Background
Video (as Downloaded): 5.3 MB
Video 5 MB
Audio 250 KB 5% of file
Best Practice:
To save bandwidth, remove the audio stream from videos that
are played silently.

95. Video Background: Mobile
Best Practice:
If Viewport will not support Video…
Don’t Download it

96. Video Background

97. Video Background
33.6 MB
27s
2560 x 1226
10 MBPS

98. Video Background
Best Practice:
Resize Video to reasonable size.
33.6 MB
27s
2560 x 1226
10 MBPS

99. Video Background
Best Practice:
Resize Video to reasonable size.
PROTIP: Renaming the file is not enough…

100. Video Resizing
• 1920x1080: 8.1 MB
• 1280x720: 4.3 MB
• 1080x608: 3.3 MB
• 720x405: 1.76 MB
http://res.cloudinary.com/dougsillars/video/upload/vc_auto,w_720/v1533591785/depend_p2ryou.mp4

101. Third Parties
No initial Video Download, but expect 700KB -1MB
additional JavaScript on page load

102. Video Third Parties

103. Video Third Parties

104. Video Third Parties

105. Video Third Parties

106. Video Streaming
Manifest File:
List of Available
Streams
Player Chooses a Stream Stream Manifest:
List of Video Segments
Player Downloads
Segments into buffer
Video Plays

107. Video Plays
Optimizing Video Delivery
Video Streaming
Player can estimate
network throughput
Optimal video Bitrate
HTTPArchive
Mobile: 1.6 MBPS
Desktop 5 MBPS

108. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8
Video Tracks
Video Tracks (iFrames)
Audio Tracks &
Subtitles

109. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8
Video Tracks

110. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

111. Optimizing Video Delivery
Video Streaming
Manifest File:
List of Available
Streams
Player selects high
bitrate stream Stream Manifest:
List of Video Segments
Buffer takes a long
time to fill
Video Does Not Play
Player chooses low
bitrate
Buffer Fills Quickly
Video Plays

112. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
Average Video Start Time:
N. America: 4.91s
Asia: 3.00s
Europe: 4.30s

113. Video Streaming: Startup Time
Low -> High
High => Low
Middle (Goldilocks)

114. Video Startup
10s 11s 12s 13s 14s 15s 16s 17s
4k Low->High:
4K Middle
(Goldilocks):
4k:High->Low:

115. Video Startup
4k Low->High:
4K Middle
(Goldilocks):
4k:High->Low:
PROs: CONS:
Fast Startup Initial Quality: Low
Other Connections:
Initial Quality: Low
Very slow startup
Fast Connections:
Initial Quality: High
Initial Quality: Good
Initial Startup: Not
fast, but not slow

116. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8 Testing with WebPageTest 3G

117. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

118. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

119. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8
Testing with WebPageTest 3G

120. Video Streaming: Manifest File
600k.m3u8

121. Video Streaming: Manifest File
600k.m3u8

122. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

123. Conclusion
Optimize Image:
Quality
Format
Sizing
Lazy Load if Possible
Only download Video when displayed
Strip audio if silent
Resize Videos for Mobile
Audit 3rd Party Videos
Streaming: Start with lower bitrates to speed video playback
Streaming: Conservative bitrates *may* reduce stalls, but will lower quality
Images
Video

124. Summary
Testing:
WebPageTest https://www.webpagetest.org
HttpArchive https://httparchive.org
Images:
ImageMagick https://www.imagemagick.org
SSIM https://github.com/technopagan/cjpeg-dssim
LazySizes https://github.com/aFarkas/lazysizes
Responsive Breakpoints http://www.responsivebreakpoints.com/
Cloudinary https://www.cloudinary.com
Tooling

125. Conclusion
Images and Video
Can Be Beautiful AND Fast