Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to PR-340: DVC: An End-to-end Deep Video Compression Framework
Similar to PR-340: DVC: An End-to-end Deep Video Compression Framework (20)
More from Hyeongmin Lee
More from Hyeongmin Lee (20)
Recently uploaded
Recently uploaded (20)
PR-340: DVC: An End-to-end Deep Video Compression Framework
- 1. DVC: An End-to-end Deep Video Compression Framework Hyeongmin Lee Yonsei University / Adobe Research PR-340 2021.8.15
- 3. Entropy Coding Image Compression 0011010100111...
- 4. Entropy Coding Entropy Coding A,B,C,D: 4 Letters (need at least 2 bits per letter) Sample Bits A 00 B 10 C 01 D 11 100100001111110101000100010001 (30 bits) https://www.programiz.com/dsa/huffman-coding
- 5. Entropy Coding Entropy Coding A,B,C,D: 4 Letters (need at least 2 bits per letter) Sample Bits A 11 B 100 C 0 D 101 1000111110110110100110110110 (28 bits) Lower Bound?? Huffman Coding
- 6. Entropy Coding Entropy 𝑯𝑯 = − � 𝒑𝒑𝒊𝒊𝒍𝒍𝒍𝒍𝒈𝒈𝟐𝟐𝒑𝒑𝒊𝒊 Sample 𝒑𝒑𝒊𝒊 A 5/15 B 1/15 C 6/15 D 3/15
- 7. Entropy Coding Entropy 2040x1356 / 6657 KB 2040x1368 / 2296 KB
- 9. Image Compression Image Compression 0011010100111... Lossless Coding: bmp Coding the Integers Itself Lossy Coding: JPEG Reducing the ‘Entropy’ of samples
- 10. Image Compression JPEG 8x8 cutting Discrete Cosine Transform Quantization Entropy Coding... https://www.whydomath.org/node/wavlets/basicjpg.html
- 11. Image Compression End-to-end Optimized Image Compression [PR328] Ballé, Johannes, Valero Laparra, and Eero P. Simoncelli. "End-to-end optimized image compression." 5th International Conference on Learning Representations, ICLR 2017. 2017. 𝝀𝝀𝝀𝝀 + 𝑹𝑹
- 12. Image Compression JPEG High Quality High Entropy High Bitrate Low Quality Low Entropy Low Bitrate
- 13. Image Compression Bitrate-Distortion Tradeoff
- 15. Video Compression Decoding Time Stamp (DTS) Intra Coding: I-Frame compression Inter Coding: P-Frame compression
- 16. Video Compression Inter Compression 𝑰𝑰𝟏𝟏, 𝑰𝑰𝟐𝟐, 𝑰𝑰𝟑𝟑, 𝑰𝑰𝟒𝟒, … , 𝑰𝑰𝑵𝑵 𝑰𝑰𝟏𝟏, 𝑹𝑹𝟐𝟐, 𝑹𝑹𝟑𝟑, 𝑹𝑹𝟒𝟒, … , 𝑹𝑹𝑵𝑵 𝑭𝑭𝟏𝟏𝟏𝟏, 𝑭𝑭𝟐𝟐𝟐𝟐, 𝑭𝑭𝟑𝟑𝟑𝟑, … , 𝑭𝑭𝑵𝑵−𝟏𝟏,𝑵𝑵
- 17. Video Compression Traditional Video Compression • H.264 • H.265 (HEVC) 1 𝒙𝒙𝒕𝒕 � 𝒙𝒙𝒕𝒕−𝟏𝟏 Motion Estimation 𝒗𝒗𝒕𝒕 Entropy Coding 11000110101
- 18. Video Compression Traditional Video Compression • H.264 • H.265 (HEVC) 2 � 𝒙𝒙𝒕𝒕−𝟏𝟏 Warping � 𝒙𝒙𝒕𝒕 𝒗𝒗𝒕𝒕
- 19. Video Compression Traditional Video Compression • H.264 • H.265 (HEVC) 3 𝒓𝒓𝒕𝒕 = 𝒙𝒙𝒕𝒕 − � 𝒙𝒙𝒕𝒕
- 20. Video Compression Traditional Video Compression • H.264 • H.265 (HEVC) 4 𝒓𝒓𝒕𝒕 Transform 𝒚𝒚𝒕𝒕 Quantization � 𝒚𝒚𝒕𝒕 Inverse Transform � 𝒓𝒓𝒕𝒕 Entropy Coding 11000110101 Buffer � 𝒙𝒙𝒕𝒕 = � 𝒙𝒙𝒕𝒕 + � 𝒓𝒓𝒕𝒕
- 21. DVC: An End-to-end Deep Video Compression Framework
- 22. DVC DVC 𝒙𝒙𝒕𝒕 � 𝒙𝒙𝒕𝒕−𝟏𝟏 Dense Optical Flow Estimation Net 𝒗𝒗𝒕𝒕 Entropy Coding 11000110101 𝒎𝒎𝒕𝒕 Motion Vector Encoder Net Quantization � 𝒎𝒎𝒕𝒕 Motion Vector Decoder Net � 𝒗𝒗𝒕𝒕 1
- 24. DVC DVC 3 𝒓𝒓𝒕𝒕 = 𝒙𝒙𝒕𝒕 − � 𝒙𝒙𝒕𝒕
- 25. DVC DVC 4 𝒓𝒓𝒕𝒕 Encoder Network 𝒚𝒚𝒕𝒕 Quantization � 𝒚𝒚𝒕𝒕 Decoder Network � 𝒓𝒓𝒕𝒕 Entropy Coding 11000110101 Buffer � 𝒙𝒙𝒕𝒕 = � 𝒙𝒙𝒕𝒕 + � 𝒓𝒓𝒕𝒕
- 26. DVC Motion Estimation Network Ranjan, Anurag, and Michael J. Black. "Optical flow estimation using a spatial pyramid network." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
- 27. DVC Motion Vector Encoder/Decoder Network Ballé, Johannes, Valero Laparra, and Eero P. Simoncelli. "End-to-end optimized image compression." arXiv preprint arXiv:1611.01704 (2016).
- 28. DVC Motion Compensation Network
- 29. DVC Loss Function Ballé, Johannes, et al. "Variational image compression with a scale hyperprior." International Conference on Learning Representations. 2018.
- 30. Experiments
- 33. Experiments Motion Estimation Network: Fix vs Train
- 34. Experiments Bit Rate Analysis
- 35. Thank You!