Introduction and Description of Error Resiliency and Concealment in ISO/IEC Video Codec H.264 MPEG4 Part10

1. Error Resiliency & Concealment in H.264 Code from jm9.8/x264 http:// getonebyone .com/ H264.html

2. What is Error Resiliency? “ To enable the compressed bit-stream to better resist channel errors” OR “ Error resilience techniques enable the compressed bit-stream to resist channel errors so that the impact on the reconstructed image quality is minimal”

6. H.264 Eco System Heterogeneous Environments Transport

8. Entropy Coding Data Partition NAL Packets (nal-units or slices) NAL Entropy Decoding Decode Prediction Info, Quantized Transform Coeffs…. VLC or AC VLC CodedData Coded MBs Coded Slice/Partition Packets (nal-units or slices) Encode Abstract Level Flow Diagram internet

9. Resilience & Concealment in Heterogeneous Environment

16. FMO cont.. FMOInit FmoGenerateMapUnitToSliceGroupMap FmoGenerateMBAmap FmoGenerateType0MapUnitMap FmoGenerateType1MapUnitMap FmoGenerateType2MapUnitMap FmoGenerateType3MapUnitMap FmoGenerateType4MapUnitMap FmoGenerateType5MapUnitMap FmoGenerateType6MapUnitMap Main Encode_one_frame() Frame_picture() Field_picture () FMOInit

21. Concealment in Data Partitioning Error concealment mechanism implemented by the decoder *A = header info *B = Intra CBPs and I-block trans-coeffs *C = Inter CBPs and P-block trans-coeffs

23. Data Partition (DP) Code_a_picturre Encode_one_Slice() Start_slice() Generates Slice (Partiton) Header Main Encode_one_frame() Frame_picture() Field_picture () SliceGroupId, Picture *, TotalCodedMBs Picture * Picture * Picture * Picture * #define MAXSLICEPERPICTURE 100 typedef struct { int no_slices; Slice *slices[MAXSLICEPERPICTURE]; …… } Picture;

29. Rate Distortion Optimization The error propagation and drift due to predictive coding can be eliminated by periodically inserting intra-coded MBs in the bitstream. Adaptive Intra Refresh (AIR): H.264 uses intelligent intra-block refreshing by R-D (rate-distortion) control, such that an appropriate block coding option o* will be selected that minimizes the Lagrangian cost function , unlike the periodic or random intra refresh in earlier standards (e.g., MPEG-4, H.263)

30. Intra Block Refreshing by R-D Control Here, o = block coding mode (inter/intra mode and block size) D(o) = distortion introduced by encoding with mode o and is computed by the SAD (sum of absolute difference) in low complexity mode and by SSD (sum of squared difference) in high complexity mode R(o) = corresponding coding rate, whereas for inter-block mode it represents the block residual and corresponding motion vector(s). λ = Lagrange parameter for appropriate weighting of rate and distortion.

31. double RDCost_for_8x8blocks ( int* cnt_nonz, // --> number of nonzero coefficients int64* cbp_blk, // --> cbp blk double lambda, // <-- lagrange multiplier int block, // <-- 8x8 block number int mode, // <-- partitioning mode short pdir, // <-- prediction direction short ref, // <-- reference frame short bwd_ref // <-- abp type ) Error Concealment API Desc: Calculates Cost function to find the appropriate block coding option. Return: Get Rate-Distortion Cost Intra Block Refreshing by R-D Control

32. Random Intra MB Refresh In order to enhance robustness to data loss, the H.264/AVC design contains a new ability to allow an encoder to send redundant representations of regions of pictures. void RandomIntraInit ( int xsize, int ysize, int refresh ) Desc: Initializes Random Intra module. xsize, ysize = size of picture in MBs refresh = refresh rate in MBs per picture void RandomIntraNewPicture ( ) Desc: Selects new set of MBs for forced Intra based on the RefreshPattern initialized in RandomIntraInit()

35. Encoder Side Error Resilience Error Concealment ( ) Encode ALL Frames( ) (main) Encode_One_Frame ( ) Field/Frame Picture ( ) (Picture *)

36. Error Concealment Decoder Error Concealment Decode ALL Frames (main) Docode_One_Frame ( Flag, DecParams, DecOutputs ) Read_New_Slice ( DecParams, DecInputs, DecOutputs ) Flag is the state variable for error detector and concealor ( DecParams, DecInputs, DecOutputs )

37. The End http:// getonebyone .com/ H264.html