An introduction to
Video Compression

  Iain Richardson
     20 June 2012




                      Slide 1
Agenda

• 1. What is Video Compression?
• 2. How does a video codec work?
   –   - prediction
   –   - transform + quantization
   –   - coding
   –   - examples
• 3. Video compression standards
   – - H.264
   – - HEVC
• 4. Video compression patents
• 5. Going further

                                    Slide 2
Agenda

• 1. What is Video Compression?
• 2. How does a video codec work?
   –   - prediction
   –   - transform + quantization
   –   - coding
   –   - examples
• 3. Video compression standards
   – - H.264
   – - HEVC
• 4. Video compression patents
• 5. Going further

                                    Slide 3
Video compression



             MPEG-2       H.264    HEVC        ……




Broadcast       DVD /      IPTV        Web      Mobile
Digital TV      Blu-Ray                Video    Video

                           ……


                                                    Slide 4
The need for video compression

 Original 2 hour movie: 1,800,000,000,000 bits




                                  =53 DVD disks

                       COMPRESS




                                  =1 DVD disk


                                                  Slide 5
Video Source                                     Video Display

               Compress   Transmi   Decompress
               (Encode)     t or     (Decode)
                           Store




                                                      Slide 6
Slide 7
Slide 8
Slide 9
Agenda

• 1. What is Video Compression?
• 2. How does a video codec work?
   –   - prediction
   –   - transform + quantization
   –   - coding
   –   - examples
• 3. Video compression standards
   – - H.264
   – - HEVC
• 4. Video compression patents
• 5. Going deeper

                                    Slide 10
Digital video




                            Luma (Y)




   Displayed frame


                     Chroma (Cr)       Chroma (Cb)


                                             Slide 11
Macroblock




             Slide 12
Video Encoder




                Slide 13
Video decoder




                Slide 14
Prediction




             Slide 15
Prediction




             Slide 16
Intra prediction




                   Slide 17
Slide 18
Inter prediction




                   Slide 19
Slide 20
Types of predicted picture




I : Intra prediction only
P : Inter prediction (one reference frame) or Intra prediction
B : Biprediction (two reference frames), Inter (one reference) or Intra prediction




                                                                                     Slide 21
Transform + Quantization




                           Slide 22
Video Encoder




                Slide 23
Forward Transform




                    Slide 24
Inverse Transform




                    Slide 25
Forward Quantize




                   Slide 26
Forward Transform + Quantize


58   64   51   58   961   -41   15    -48    48   -1   0    -1
52   64   56   66   -34   72    -30   -104   -1   1    -1   -2
62   63   61   64   -15   3     15    24     0    0    0    0
59   51   63   69   13    81    -5     8     0    1    0    0


Block of samples    After transform          Quantized, QP=18




                                                            Slide 27
Inverse Quantize




                   Slide 28
Inverse Transform




                    Slide 29
58   64   51   58   48   -1   0    -1   55   66   54   58
52   64   56   66   -1   1    -1   -2   54   62   58   63
62   63   61   64   0    0    0    0    61   59   61   62
59   51   63   69   0    1    0    0    60   55   65   67


Block of samples    Quantized, QP=18    Rescaled +
                                        inverse
                                        transformed




                                                       Slide 30
Entropy coding




                 Slide 31
Video Encoder




                Slide 32
Entropy coding

                         Fixed length
                         codes


                           Variable length
        Headers            codes

Prediction parameters
                             Context Adaptive
Quantized coefficients       Variable Length
                             Codes

                           Context Adaptive
                           Binary Arithmetic
                           Codes

                                                Slide 33
Slide 34
Slide 35
Slide 36
Examples




           Slide 37
Slide 38
Slide 39
Slide 40
Slide 41
……etc



        Slide 42
Agenda

• 1. What is Video Compression?
• 2. How does a video codec work?
   –   - prediction
   –   - transform + quantization
   –   - coding
   –   - examples
• 3. Video compression standards
   – - H.264
   – - HEVC
• 4. Video compression patents
• 5. Going deeper

                                    Slide 43
H.264 Advanced Video Compression




                               Slide 44
H.264/AVC

•   First released in 2003
•   Published by ISO MPEG and ITU-T VCEG
•   Roughly 2x compression efficiency of MPEG-2
•   Used for:
    –   Blu-Ray Disks
    –   High Definition TV broadcasting
    –   Internet video
    –   ……




                                                  Slide 45
Slide 46
High Efficiency Video Compression
              (HEVC)




                                    Slide 47
HEVC

• Under development by ISO MPEG and ITU-T VCEG
• Roughly 2x compression efficiency of H.264
• Due for publication in 2013




                                                 Slide 48
Slide 49
Agenda

• 1. What is Video Compression?
• 2. How does a video codec work?
   –   - prediction
   –   - transform + quantization
   –   - coding
   –   - examples
• 3. Video compression standards
   – - H.264
   – - HEVC
• 4. Video compression patents
• 5. Going deeper

                                    Slide 50
Source: USPTO database




                         Slide 51
Types of video compression patent

• Encoder

• Decoder

• Transport




                                    Slide 52
Patent Pools
• E.g. MPEG-LA, H.264/AVC patent pool:
   –   Licenses patents on behalf of patent holders
   –   Unrelated to ISO/IEC MPEG
   –   Accepts patents based on determination of “essentiality”
   –   250+ US patents
   –   900+ licensees




                                                                  Slide 53
Agenda

• 1. What is Video Compression?
• 2. How does a video codec work?
   –   - prediction
   –   - transform + quantization
   –   - coding
   –   - examples
• 3. Video compression standards
   – - H.264
   – - HEVC
• 4. Video compression patents
• 5. Going further

                                    Slide 54
To find out more

• http://www.itu.int - Download the H.264 standard
• http://www.vcodex.com - Free tutorials

• Follow me on Twitter: @onecodec




                                                     Slide 55

Iain Richardson: An Introduction to Video Compression

  • 1.
    An introduction to VideoCompression Iain Richardson 20 June 2012 Slide 1
  • 2.
    Agenda • 1. Whatis Video Compression? • 2. How does a video codec work? – - prediction – - transform + quantization – - coding – - examples • 3. Video compression standards – - H.264 – - HEVC • 4. Video compression patents • 5. Going further Slide 2
  • 3.
    Agenda • 1. Whatis Video Compression? • 2. How does a video codec work? – - prediction – - transform + quantization – - coding – - examples • 3. Video compression standards – - H.264 – - HEVC • 4. Video compression patents • 5. Going further Slide 3
  • 4.
    Video compression MPEG-2 H.264 HEVC …… Broadcast DVD / IPTV Web Mobile Digital TV Blu-Ray Video Video …… Slide 4
  • 5.
    The need forvideo compression Original 2 hour movie: 1,800,000,000,000 bits =53 DVD disks COMPRESS =1 DVD disk Slide 5
  • 6.
    Video Source Video Display Compress Transmi Decompress (Encode) t or (Decode) Store Slide 6
  • 7.
  • 8.
  • 9.
  • 10.
    Agenda • 1. Whatis Video Compression? • 2. How does a video codec work? – - prediction – - transform + quantization – - coding – - examples • 3. Video compression standards – - H.264 – - HEVC • 4. Video compression patents • 5. Going deeper Slide 10
  • 11.
    Digital video Luma (Y) Displayed frame Chroma (Cr) Chroma (Cb) Slide 11
  • 12.
    Macroblock Slide 12
  • 13.
  • 14.
  • 15.
    Prediction Slide 15
  • 16.
    Prediction Slide 16
  • 17.
  • 18.
  • 19.
  • 20.
  • 21.
    Types of predictedpicture I : Intra prediction only P : Inter prediction (one reference frame) or Intra prediction B : Biprediction (two reference frames), Inter (one reference) or Intra prediction Slide 21
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
    Forward Transform +Quantize 58 64 51 58 961 -41 15 -48 48 -1 0 -1 52 64 56 66 -34 72 -30 -104 -1 1 -1 -2 62 63 61 64 -15 3 15 24 0 0 0 0 59 51 63 69 13 81 -5 8 0 1 0 0 Block of samples After transform Quantized, QP=18 Slide 27
  • 28.
  • 29.
  • 30.
    58 64 51 58 48 -1 0 -1 55 66 54 58 52 64 56 66 -1 1 -1 -2 54 62 58 63 62 63 61 64 0 0 0 0 61 59 61 62 59 51 63 69 0 1 0 0 60 55 65 67 Block of samples Quantized, QP=18 Rescaled + inverse transformed Slide 30
  • 31.
  • 32.
  • 33.
    Entropy coding Fixed length codes Variable length Headers codes Prediction parameters Context Adaptive Quantized coefficients Variable Length Codes Context Adaptive Binary Arithmetic Codes Slide 33
  • 34.
  • 35.
  • 36.
  • 37.
    Examples Slide 37
  • 38.
  • 39.
  • 40.
  • 41.
  • 42.
    ……etc Slide 42
  • 43.
    Agenda • 1. Whatis Video Compression? • 2. How does a video codec work? – - prediction – - transform + quantization – - coding – - examples • 3. Video compression standards – - H.264 – - HEVC • 4. Video compression patents • 5. Going deeper Slide 43
  • 44.
    H.264 Advanced VideoCompression Slide 44
  • 45.
    H.264/AVC • First released in 2003 • Published by ISO MPEG and ITU-T VCEG • Roughly 2x compression efficiency of MPEG-2 • Used for: – Blu-Ray Disks – High Definition TV broadcasting – Internet video – …… Slide 45
  • 46.
  • 47.
    High Efficiency VideoCompression (HEVC) Slide 47
  • 48.
    HEVC • Under developmentby ISO MPEG and ITU-T VCEG • Roughly 2x compression efficiency of H.264 • Due for publication in 2013 Slide 48
  • 49.
  • 50.
    Agenda • 1. Whatis Video Compression? • 2. How does a video codec work? – - prediction – - transform + quantization – - coding – - examples • 3. Video compression standards – - H.264 – - HEVC • 4. Video compression patents • 5. Going deeper Slide 50
  • 51.
  • 52.
    Types of videocompression patent • Encoder • Decoder • Transport Slide 52
  • 53.
    Patent Pools • E.g.MPEG-LA, H.264/AVC patent pool: – Licenses patents on behalf of patent holders – Unrelated to ISO/IEC MPEG – Accepts patents based on determination of “essentiality” – 250+ US patents – 900+ licensees Slide 53
  • 54.
    Agenda • 1. Whatis Video Compression? • 2. How does a video codec work? – - prediction – - transform + quantization – - coding – - examples • 3. Video compression standards – - H.264 – - HEVC • 4. Video compression patents • 5. Going further Slide 54
  • 55.
    To find outmore • http://www.itu.int - Download the H.264 standard • http://www.vcodex.com - Free tutorials • Follow me on Twitter: @onecodec Slide 55