Compression Synopsis H264-H265

19 October 2016 ALL DATA IS COMPANY CONFIDENTIAL Sheet 1
Compression Synopsis
h.264 vs. h.265
By
Paul Hightower
CEO, Instrumentation Technology Systems
Presented to
Optical Systems Group
Fall 2016
At
NASA Armstrong Flight Research Center

30 Years of Compression
Each major system has added new compression tools
JPEG
MPEG 1
MPEG 2/H.262
MPEG 4H.261
H.264
H.265
1986 1994 2000 2013
H.263
• Targeted @ video conferencing
• Introduced I &P frames
• Targeted @ HD distribution
• variable block size
• Deblocking filters
• 8 modes motion vectors
• Context-adaptive binary
arithmetic coding
• Targeted @ SD/ED distribution
• Introduced motion vectors
• Bidirectional estimation
• ½ pixel motion vectors
• Intraframe prediction
• More frame formats
• Targeted @ UHD distribution
• 4K and 8K at up to 120 FPS
• Larger more flexible block sizes
using Coding Tree Units
• Larger transform size
• 33 modes motion vectors
• Merge/Skip vector modes
• Targeted at single images
• Wavelet transform

Why Compression?
Content delivery
 Television & Cable have narrow bands available to deliver content
• Compression is an enabler
• Allocations are in the 10s of Mb/sec
 Internet Streaming
• Premium speed limit is 500 Mb/s
 Wi-Fi
• 802.11ac max is 1,300 Mb/sec
Content Storage
 After Theater Movie Sales
• DVD holds 4.7 GB
• Blu-ray holds 50 GB
• Hard Drives 1000-4000 GB now
• Cinema industries stores 10s of PB/day!
Format Raw Data Rate
SD/30 270 Mb/s
720/60 1,485 Mb/s
1080/60 2,970 Mb/s
2160/60 (4K) 11,880 Mb/s
Format Single Frame Size 2 Hours
SD/30 1.1 MB 250 GB
720/60 3.1 MB 1,400 GB
1080/60 6.2 MB 2,700 GB
2160/60 (4K) 24.8 MB 10,700 GB

Challenges of Compression
A 2 hour movie on a Blu-ray disk
Transport over GigE Ethernet;
 ONLY 1 Channel, not shared with other traffic
Transport via 100Mb/s Wi-Fi link
 Not shared with other traffic
Format Minimum Compression Ratio
SD >6:1
720/60 >30:1
1080/60 >50:1
2160/60 >200:1
SD none
720/60 2:1
1080/60 4:1
2160/60 >14:1
SD 3:1
720/60 >15:1
1080/60 >30:1
2160/60 >120:1

Objectives
 Reduce Frame Size (bytes) as much as possible
• Take advantage of redundancies in the image data of a frame (spatial)
• Take advantage of image areas that are same frame to frame (temporal)
• Take advantage of information that can be derived across a group of frames (prediction)
• Create standard algorithms that analyze and transform image data into compact nuggets
 Enable decoder to render visually good imagery
• Largely a subjective measure
• Take advantage of eye sensitivities to changes in brightness, color and bandwidth
• Different criteria than still imagery
 Manage Latency
• The greater the compression:
 The more time it takes to crunch it
 The more time it takes to re-render the images
 The more buffering may be needed at the decoder end

Latency
 Introduction of I & P frames (MPEG) introduces interframe dependencies
 Latency is dynamic and is continuously variable
• Latency varies with the number of dependent frames, scene complexity, buffer sizes,
transport bandwidth, hardware and software from source to destination
Image Quality
 Unlike data compression, image compression takes advantage of human perception
• Temporal and spatial image characteristics replace image areas (macroblocks)
• Encoded Pixels are replaced with transform coefficients.
• Decode Filters, motion prediction, pixel interpolation reduce the perceived errors
 Bit-depth may be compromised
• Many encoders change sampling from 4:2:2 to 4:2:0 effectively reducing “shades”
resolution of pixels and shift color
 Can vary with the encoder, transport activity, rendering horsepower

Image Quality
Image quality is largely based on human perception
 Noise perceived
 Sharpness perceived
 Blur
 Color Space
 Bandwidth sensitivities and limits
No one metric stands alone
 “beauty is in the eye of the beholder”
 Encode/Decode results may differ from encoder vendor to encoder vendor
 There are many settings that vary with the camera, transport, view needs, lighting
and content
MPEG & H.xxx specifications define the bit streams and data structures (Decoding)
 Encoding is left to the developer
 End-to-End quality is not controlled, only output given the same input

Image Quality
Common Metrics of Image Quality
 Peak Signal to Noise Ratio (PSNR)
• Basic, but flawed
• PSNR values do not correlate well with perceived picture quality due to the complex,
highly non-linear behavior of the human visual system.
( source: https://sonnati.wordpress.com/2014/06/20/h265-part-i-technical-overview/)
Source: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/VELDHUIZEN/node18.html
All images have the same PSNR

Image Quality
 Structural Similarity (SSIM) : Math
• Compares groups of original pixels with the
post-decoded pixels in the same image area.
• This objective analysis compares favorably with
MOS
 Mean Opinion Score (MOS): Observation
• Subjective measure
• Uses standard clips
• Uses standard viewing environments
• Human subjects score what they see compared
to raw clips.

H.265
Standard Managed by
 ITU-T Video Coding Experts Group (VCEG)
 ISO/IEC Moving Picture Experts Group (MPEG)
First joint HEVC standard released Jan 2013
Design Goals
 50% reduction of bit rate compared to H.264 at the same perceptual image quality
 Support transport of 4K video and beyond (more pixels)
 Support high dynamic range color (more shades representable)
 Support higher frame rates (120 and up)
 Increase data loss resilience
Enabling Technologies
 Parallel processing
• Needed for encoding (image analysis) and decoding (re-rendering)
 New modulation techniques increasing the bits/Hz ratio
• QAM, quadrature modulation increases the bits/Hz in the transport channel

What is common H.264-H.265
Split frames into Coding Tree Units (macro blocks)
 This is essentially the starting point for an I frame
Transform (DCT), scaling & quantization
 Spatial motion prediction (Intrapicture) & differences compared
• Prediction of block data derived from within the I (reference) frame alone
 The two elements are transformed (DCT), quantized and scaled
 Entropy Encoding; Context adaptive binary arithmetic coding (CABAC)
• Lossless data compression (the final step)
 The result is put in to the bit stream

What is common H.264-H.265
Temporal Encoding
 Motion prediction of elements across a group of frames (GOP) from the I frame
• Results in differential frames
• B and P frames rely on data from the I frame
 P frames (predictive) and can be used for prediction blocks in B frames
 B frames process predictions based on data before and after the current frame.
• The resultant data for B and P frames is very small
• Most of the average compression comes from P and B frames
• P & B frames may be lost if the reference I frame is corrupted

What is different H.264-H.265
Larger macroblocks
More macroblock types
Spatial prediction; 9 modes > 35 modes
Temporal prediction; H.265 adds rectangular blocks
Transform sizes; H.264 max 16x16; H.265 adds 32x32, 64x64 plus non-square forms
 Larger block sizes are more coding efficient

Is H.265 Better?
H.265 is not MORE compression; it is smarter
 Increased flexibility
 More fine control of prediction vectors
 Better pixel interpolation
 Finer deblocking

Is H.265 Better?

Is H.265 Better?
A range of scenes with similar PSNR and MOS compared to H.264 scenarios
Bit rates are better, but vary with scene complexity
 Note a ±25% bit rate variance with the same encoder/decoder and hardware

Is H.265 Better?
A 2 hour movie on a Blu-ray disk
Transport over GigE Ethernet
 UDP with exclusive use of the network
Transport via 100Mb/s Wi-Fi link
 Exclusive use at full bandwidth
H.264 H.265
SD >6:1 3:1
720/60 >30:1 14:1
1080/60 >50:1 28:1
2160/60 >200:1 111:1
H.264 H.265
SD none None
720/60 2:1 None
1080/60 4:1 2:1
2160/60 >14:1 8:1
H.264 H.265
SD 3:1 2:1
720/60 >15:1 8:1
1080/60 >30:1 16:1
2160/60 >120:1 62:1

Conclusions
H.265 expands on legacy of tools
 Macro blocks
 Spatial and Temporal prediction
 Motion vectors
 DCT and Entropy
H.265 is not MORE compression; Smarter Processing
 More tools, more analytical choices
More Computing Power Required
 Parallel processing a must
 Real-time (30 to 60 fps) does not change with more work to do encoding/decoding
 Encoding can be longer even with more capable hardware
Compression goal met at 48-50% for same image quality
Latency?
Positive Side Effect:
 At current transport bandwidths available 720p/60 and 1080p/60 should survive
with better image fidelity than with H.264

Concerns
Greater compression may mean more sensitivity to bit errors in transport
 Encoders will vary from source to source
 Image quality needs to be scored
H.265 Encoder/Decoder sets must come on line with metadata support
 Two H.264 Encoder/Decoders have been tested for Metadata by ITS
 Delta Digital 4480E and 9600 (decoder)
 Haivision Makito X

Concerns
Decode side is specified
 Encode side is up to the supplier
 Encoders must be evaluated for image quality
 Image quality for engineering test is different than moviegoers
• More details are important
• Critical frames are often most different frames
Latency
 For the same channel width is latency greater or lesser?
 Does latency vary more?

Bibliography of Resources
Overview of the High Efficiency Video Coding (HEVC) Standard
 Gary J. Sullivan,Fellow, IEEE, Jens-RainerOhm,Member, IEEE, Woo-Jin Han, Member, IEEE, and Thomas Wiegand, Fellow, IEEE;
 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,VOL. 22, NO. 12, DECEMBER 2012
Design and Implementation of Next Generation Video Coding Systems (H.265/HEVC Tutorial)
 Vivienne Sze (sze@mit.edu);Madhukar Budagavi (m.budagavi@samsung.com)
 ISCAS Tutorial 2014
Video Coding Basics
 Yao Wang, Polytechnic University,Brooklyn, NY11201, yao@vision.poly.edu,2003
“Intra Prediction in HEVC,”High Efficiency Video Coding (HEVC): Algorithms and Architectures
 Springer, 2014. J. Lainema, W.---J. Han,

Compression Synopsis H264-H265

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (8)

Similar to Compression Synopsis H264-H265

Similar to Compression Synopsis H264-H265 (20)

Compression Synopsis H264-H265