Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

A short history of video coding

6,670 views

Published on

Video coding is an essential component of video streaming, digital TV, video chat and many other technologies. This presentation, an invited lecture to the US Patent and Trade Mark Office, describes some of the key developments in the history of video coding.

Many of the components of present-day video codecs were originally developed before 1990. From 1990 onwards, developments in video coding were closely associated with industry standards such as MPEG-2, H.264 and H.265/HEVC.

The presentation covers:
- Basic concepts of video coding
- Fundamental inventions prior to 1990
- Industry standards from 1990 to 2014
- Video coding patents and patent pools.

Published in: Technology
  • -- DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT -- ......................................................................................................................... ......................................................................................................................... Download FULL PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... (Unlimited)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • .
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

A short history of video coding

  1. 1. A History of Video Coding Invited Talk, United States Patent and Trademark Office Iain Richardson December 2014 Slide 1 vcodex.com
  2. 2. What this talk is about Slide 2
  3. 3. Slide 3 From To
  4. 4. Video coding : the basics Slide 4
  5. 5. A video codec (1) Slide 5 Video Source Decompress (Decode) Compress (Encode) Video Display Coded video enCOder / DECoder
  6. 6. A video codec (2) Slide 6 video source partition transform entropy encode predict (add) inverse transform entropy decode compressed video video output VIDEO ENCODER VIDEO DECODER predict (subtract) reconstruct
  7. 7. 1. Partitioning Slide 7 frame or picture slice or tile macroblock (MB) or coding tree unit (CTU) sequence
  8. 8. 2. Prediction Slide 8 Original Frame Prediction Frame Residual
  9. 9. 3. Transform + Quantize Block of samples After transform After quantization Slide 9
  10. 10. 4. Entropy coding Slide 10 1 1 0 1 1 1 1 1 0 0 0 1 1 coded components (syntax elements) bitstream
  11. 11. source frames ........ ........ coded bitstream decoded frames ENCODE DECODE Slide 11
  12. 12. Video Encoder Form prediction Transform + quantize Inverse transform + quantize Entropy encoder Current MB Prediction MB Residual MB Decoded Residual MB Intra Inter .... Current frame or field Previously coded frames or fields + Coded bitstream - + + Slide 12
  13. 13. Video Decoder Coded bitstream .... Entropy decoder Inverse transform + quantize Decoded Residual MB Current decoded frame or field Inter Previously decoded frames or fields Prediction MB Form prediction Intra Decoded MB + + Slide 13
  14. 14. Ancient history Slide 14
  15. 15. Slide 15 1851
  16. 16. 1910 Slide 16
  17. 17. Slide 17 1964
  18. 18. The foundations Slide 18
  19. 19. Slide 19 1952 Variable length coding 1972 Frame differencing 1974 Discrete Cosine Transform 1981 Motion compensated video codec 1990 Bidirectional prediction Variable block size motion compensation
  20. 20. Huffman variable length coding, 1952 Slide 20 D A Huffman, Minimum redundancy codes
  21. 21. Frame differencing, 1972 Slide 21 US 3,679,821, Schroeder, Bell Labs
  22. 22. Discrete Cosine Transform, 1974 Slide 22 The Discrete Cosine Transform Ahmed, Natarajan and Rao
  23. 23. Video codec with motion compensation, 1981 Slide 23 Displacement measurement and its application in interframe video coding, Jain and Jain
  24. 24. Bidirectional + variable block size motion prediction, circa 1990 Slide 24 MPEG-1, LeGall, 1991 Chan et al, 1990
  25. 25. By 1991, the basic codec was complete Slide 25 MPEG-1 encoder, 1991
  26. 26. Video Encoder Form prediction Transform + quantize Inverse transform + quantize Entropy encoder Current MB Prediction MB Residual MB Decoded Residual MB Intra Inter .... Current frame or field Previously coded frames or fields + Coded bitstream - + + Slide 26
  27. 27. The video coding standards Slide 27
  28. 28. Slide 28 Moving Picture Experts Group - MPEG standards - Broadcast / entertainment Joint Video Team / Joint Collaborative Team Video Coding Experts Group - H.26x standards - Communications
  29. 29. Slide 29
  30. 30. Slide 30 1990 1993 1995 1996 1999 2003 2013 ITU-T JOINT ISO MPEG MPEG-1 H.263 H.262/MPEG-2 H.264/AVC H.265/HEVC MPEG-4 H.261 ….
  31. 31. ITU-T H.261 (1990) Slide 31 2-way video telephony ISDN lines (p x 64kbps)
  32. 32. ITU-T H.261 (1990) Slide 32 16x16 blocks 8x8 DCT I P P P Forward prediction …011011000100… Variable length coding
  33. 33. MPEG-1 Video (1991-93) Slide 33 Video CD playback
  34. 34. MPEG-1 Video (1991-93) Slide 34 I B B P B B P B B P B B I 0 1 2 3 4 5 6 7 8 9 10 11 12 Intra, forward and bidirectional prediction Half-pixel motion vectors
  35. 35. MPEG-2 Video (1995) Slide 35 Digital TV broadcasting, DVD
  36. 36. MPEG-2 Video (1995) Slide 36 16x16 and Interlaced video 16x8 block sizes
  37. 37. H.263 (1996), MPEG-4 Part 2 (1999) Slide 37
  38. 38. H.263 (1996), MPEG-4 Part 2 (1999) Slide 38 16x16, 8x8 blocks Arithmetic coding Error resilience Advanced prediction Deblocking filter Object coding Mesh coding Face animation Many optional coding tools
  39. 39. H.264 / Advanced Video Coding (2003) Slide 39
  40. 40. H.264 / Advanced Video Coding (2003) Slide 40 16x16 down to 4x4 blocks Intra prediction 0 1 2 3 4 5 6 Multiple reference frames + context adaptive coding (CABAC / CAVLC), ¼ pixel motion vectors, loop filter….
  41. 41. H.265 / High Efficiency Video Coding (2013) Slide 41
  42. 42. H.265 / High Efficiency Video Coding (2013) Courtesy of Parabola Research Slide 42 64x64 down to 4x4 blocks Parallel processing support + Lots of “H.264 like” optimizations….
  43. 43. HEVC analyzer screenshot Courtesy of Parabola Research Slide 43
  44. 44. Performance Slide 44
  45. 45. Compression performance Slide 45 MPEG-2 ~2x better H.264/AVC ~2x better H.265/HEVC
  46. 46. Performance examples Slide 46
  47. 47. Slide 47 MPEG-1 MPEG-2 H.264 HEVC
  48. 48. Slide 48 Difference image (MPEG-1) Difference image (HEVC)
  49. 49. Patents and patent pools Slide 49
  50. 50. Example Slide 50 September 2001 Patent application is filed September 2001 Proposal document is submitted to Joint Video Team May 2003 H.264 is published, including the concept of the application & proposal 2005 Patent is granted
  51. 51. Slide 51 Published Standard Patents Implementors Patent pool
  52. 52. Patents and patent pools: the early standards Slide 52 H.261, MPEG-1, H.263 : - no formal patent pool MPEG-2 : - first patent pool, managed by MPEG-LA MPEG-4 Part 2 : - patent pool, managed by MPEG-LA
  53. 53. Patents and patent pools: H.264 / AVC Slide 53 2001-2003 Royalty Free Baseline intention announced by 32 patent holders May 2003 First edition of H.264 published Late 2003 MPEG-LA and Via Licensing announce patent pools. Royalty Free Baseline abandoned. 2004 onwards Via Licensing pool “folds”. MPEG-LA pool dominates.
  54. 54. Patents and patent pools: recent developments Slide 54 Open source VP8 format: MPEG-LA attempts to create patent pool, Google and MPEG-LA reach agreement and pool is abandoned. HEVC: Patent pool created by MPEG-LA (2014).
  55. 55. Conclusion Slide 55 Many of the key components of modern video codecs originated before 1990. In the last 25 years, developments in video coding have been strongly influenced by standardisation. Performance continues to improve, driven by optimization of compression tools.
  56. 56. Useful information sources Slide 56 Low bit rate / Video Coding Experts Group archives, 1996-present: http://wftp3.itu.int/av-arch/video-site/ Joint Video Team archives (H.264 development): http://wftp3.itu.int/av-arch/jvt-site/ JCT-VC archives (HEVC development): http://wftp3.itu.int/av-arch/jctvc-site/ Video coding resources: http://vcodex.com/

×