Next-Generation Image Formats for the Internet
•
10 likes•14,850 views
The document discusses next-generation image formats for the internet. It focuses on JPEG XL, a new image format currently being standardized that aims to replace JPEG, PNG, and GIF. JPEG XL provides state-of-the-art compression, supports high bit depths, alpha channels, animation and is designed to be responsive by encoding images into a single file at multiple quality levels. It is also intended to be legacy-friendly by never producing larger files than the original formats and avoiding additional quality loss for existing images.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Next-Generation Image Formats for the Internet
Similar to Next-Generation Image Formats for the Internet (20)
More from Cloudinary
More from Cloudinary (15)
Recently uploaded
Recently uploaded (20)
Next-Generation Image Formats for the Internet
- 1. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE MAY 2019 NEXT-GENERATION IMAGE FORMATS FOR THE INTERNET Jon Sneyers jon@cloudinary.com
- 2. N E X T- G E N E R AT I O N I M A G E F O R M AT S F O R T H E I N T E R N E T JPEG
- 3. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G ISO/IEC JTC1/SC29/WG1 aka “Joint Photographic Experts Group” International Standardization Organization International Electrotechnical Commission International Telecommunication UnionJoint Technical Committee 1: Information Technology Subcommittee 29: Coding of audio, picture, multimedia and hypermedia information Working Group 1: Coding of still pictures “Joint”
- 4. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G JPEG is great! 4 In 1986, the Joint Photographic Experts Group started developing what became the 1992 JPEG standard. It is still the most widely used image format, and arguably the most successful image format ever. Features: • Lossy compression: • YCbCr, optional chroma subsampling (4:2:0) • 8x8 DCT (discrete cosine transform) • Sequential and progressive mode • Standard: • 8-bit and 12-bit lossy • lossless mode up to 16-bit • up to 4 components (not really specified what they are) • two choices of entropy coders: Huffman (no patents) or Arithmetic Coding (patent-encumbered) de facto standard: grayscale, RGB or CMYK
- 5. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G But… 5 With a good JPEG encoder like the (de facto) JPEG standard is still quite good! BUT… it has limitations: • Only lossy • Bad for non-photographs (sharp edges, text) • No alpha channel (transparency) • Only 8-bit (problem for wide-gamut) • No animation • Not quite state-of-the-art entropy coding (Huffman) • At lower bitrates: obvious compression artifacts (blockiness, color banding, ringing, DCT noise) }Use PNG instead ⟶ This is why GIF is still around
- 6. N E X T- G E N E R AT I O N I M A G E F O R M AT S F O R T H E I N T E R N E T ALTERNATIVES TO JPEG
- 7. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE A LT E R N AT I V E S T O J P E G Attempts to replace JPEG There have been many attempts to replace JPEG • JPEG 2000 • JPEG XR • JPEG XT • WebP • BPG • HEIF (HEIC) • AVIF So far, none of them has really succeeded (yet) (although some of them have had some success) 7 }Video codecs used as image codec
- 8. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE A LT E R N AT I V E S T O J P E G Attempts to replace JPEG There have been many attempts to replace JPEG • JPEG 2000 (1997-…) • JPEG XR • JPEG XT • WebP • BPG • HEIF (HEIC) • AVIF 8 👍 • Supports lossless • Alpha • High bit depth • Less blockiness 👎 • Slow • Blur • Not that much better • Initially no open source software
- 9. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE A LT E R N AT I V E S T O J P E G Attempts to replace JPEG There have been many attempts to replace JPEG • JPEG 2000 • JPEG XR (2007-2012) • JPEG XT • WebP • BPG • HEIF (HEIC) • AVIF 9 👍 • Supports lossless • Alpha • High bit depth • Less blockiness • Faster than J2K 👎 • Worse than J2K • Single vendor pushing it (Microsoft)
- 10. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE A LT E R N AT I V E S T O J P E G Attempts to replace JPEG There have been many attempts to replace JPEG • JPEG 2000 • JPEG XR • JPEG XT (2013-2016) • WebP • BPG • HEIF (HEIC) • AVIF 10 👍 • Backwards compatible • Supports lossless • Alpha • High bit depth 👎 • Backwards compatible • Little incentive to upgrade decoders • No compression improvement
- 11. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE A LT E R N AT I V E S T O J P E G Attempts to replace JPEG There have been many attempts to replace JPEG • JPEG 2000 • JPEG XR • JPEG XT • WebP (2010-2019) • BPG • HEIF (HEIC) • AVIF 11 👍 • Supports lossless • Alpha • Animation • Open source • Royalty-free 👎 • Only 8-bit • No progressive mode • Obligatory 4:2:0 • Max dimensions are 16383x16383 • No formal standard
- 12. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE A LT E R N AT I V E S T O J P E G Attempts to replace JPEG There have been many attempts to replace JPEG • JPEG 2000 • JPEG XR • JPEG XT • WebP • BPG (2014-2018) • HEIF (HEIC) (2013-2017) • AVIF 12 👍 • Supports lossless • Alpha, Depth • High bit depth • Animation • Very good compression 👎 • No progressive mode • Patent mess
- 13. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE A LT E R N AT I V E S T O J P E G Attempts to replace JPEG There have been many attempts to replace JPEG • JPEG 2000 • JPEG XR • JPEG XT • WebP • BPG • HEIF (HEIC) • AVIF (2018-…) 13 👍 • Supports lossless • Alpha • Up to 12-bit • Animation • Very good compression • Royalty-free 👎 • No progressive mode • Slow • Complicated
- 14. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE A LT E R N AT I V E S T O J P E G Attempts to replace JPEG 14 Patent mess Limitations (8 bit, 4:2:0) Complexity No progressive, only sequential Compression improvement: “meh”
- 15. 15 A LT E R N AT I V E S T O J P E G Attempts to replace JPEG 1. JPEG 2. GIF 3. PNG 4. TIFF 5. BMP 6. PPM 7. JPEG 2000 8. WebP 9. JPEG XR 10. JPEG XT 11. BPG 12. FLIF 13. HEIC 14. AVIF 1. JPEG 2. GIF 3. PNG 4. TIFF 5. BMP 6. PPM 7. JPEG 2000 8. WebP 9. JPEG XR 10. JPEG XT 11. BPG 12. FLIF 13. HEIC 14. AVIF 15. JPEG XL xkcd.com/927
- 16. N E X T- G E N E R AT I O N I M A G E F O R M AT S F O R T H E I N T E R N E T JPEG XL
- 17. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE 17 Pik Image Compression ♠ Alexander Rhatushnyak, Evgenii Kliuchnikov, Jan Wassenberg, Jeffrey Lim, Jyrki Alakuijala, Krzysztof Potempa, Lode Vandevenne, Renata Khasanova, Robert Obryk, Sami Boukortt, Thomas Fischbacher, Zoltan Szabadka = + proposal 2 proposal 3 proposal 4 proposal 5 proposal 6 proposal 1 proposal 7 > ISO/IEC JTC1/SC29/WG1 J P E G X L History
- 18. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G How it works 18
- 19. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L How it works 19
- 20. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Features / Goals 20 • Royalty-free, Free & Open Source software available from the start • All the features expected for a modern image codec: • Alpha (and other extra channels like Depth) • High bit depth (up to 16-bit) • Supports lossless • State-of-the-art compression • Animation • Progressive mode / “Responsive by design” • Legacy-friendly • High quality • Can replace JPEG, PNG and GIF
- 21. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Responsive by design 21
- 22. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Responsive by design 22 In the 1990s… Internet was slow for everyone Screen sizes and resolutions didn’t vary much 14.4 - 33.6 kbit/s 14 to 21 inch 800x600 or 1024x768
- 23. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Responsive by design 23 Today Internet speed and screen sizes and resolutions vary a lot! Smartwatch 2 inch, 360x480 8K Smart TV 80 inch, 7680x4320
- 24. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Responsive by design 24 ❌ ✅
- 25. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Responsive by design 25 w4x.jpg, 1.9 MBw2x.jpg, 490 KBw1x.jpg, 110 KBw0.5x.jpg, 30 KB LQIP.gif 271 bytes Current approach: many files
- 26. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Responsive by design 26 w.jxl, 1.7 MBw.jxl, first 806 KBw.jxl, first 206 KBw.jxl, first 50 KB w.jxl first 200 bytes JPEG XL approach: one file!
- 27. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L How it works: Squeeze 27
- 28. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L How it works: Squeeze 28
- 29. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L How it works: Squeeze 29
- 30. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L How it works: Squeeze 30
- 31. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L How it works: Squeeze 31
- 32. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Legacy-friendly 32
- 33. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Legacy-friendly 33 JPEG PNG GIF New format Pixels Legacy decoder New encoder Additional loss and/or more bytes than original! Existing image files
- 34. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Legacy-friendly 34 sunset.jpg 6173 bytes sunset.webp 11110 bytes
- 35. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Legacy-friendly 35 JPEG PNG GIF New format Pixels Legacy decoder New encoder Additional loss and/or more bytes than original! Existing image files
- 36. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Legacy-friendly 36 JPEG PNG8 GIF JPEG XL No additional loss, always smaller than original! Existing image files PNG24 Pixels Palette pixelsDCT coefficients
- 37. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Legacy-friendly 37 sunset.jpg 6173 bytes sunset.jxl 3320 bytes sunset.jpg 6173 bytes reversible
- 38. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Legacy-friendly 38 sunset.jpg 6173 bytes sunset.jxl 3320 bytes sunset.jpg 6173 bytes reversible Improved dequantization (reduces blockiness / color banding)
- 39. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L High quality 39
- 40. 40TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L High quality • Video codecs focus on low bitrates • They have a lot of pixels to encode! • You only see a single frame for 16-42 ms • A lot of smoothing and “distilling” • Great for video • Not so good for high-quality still images Original HEIC
- 41. 41TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L High quality • Visually lossless = lossy, but human eyes cannot see the difference • XYB color space models human vision • Adaptive quantization (guided by a perceptual metric) Distribution of cone cells in the fovea
- 42. 42TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L How it works: adaptive quantization • JPEG: Same quality in every region • JPEG XL: Quality can be different in different regions; it is adjusted automatically based on perceptual metrics
- 43. TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Universal 43
- 44. 44
- 45. 45TRANSFORMING THE DIGITAL MEDIA EXPERIENCE J P E G X L Universal JPEG XL is designed to be a universal and future-proof image codec • Any kind of image content: photographs, illustrations, screenshots, rendered images, document scans, medical imaging, game graphics, UI elements, … • Support for extra channels like alpha, depth, spot colors, … • No limits on the image dimensions • High bit depth / wide gamut / HDR • Covers the whole quality spectrum from very low bitrates all the way to lossless • Various trade-offs between encode/decode speed and compression density
- 46. 46 J P E G X L Universal Non-photographic 8-bit photo 12-bit photo TRANSFORMING THE DIGITAL MEDIA EXPERIENCE
- 47. Conclusion 47 Responsive by design Legacy-friendlyUniversal TRANSFORMING THE DIGITAL MEDIA EXPERIENCE High quality