H.264 Library


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

H.264 Library

  1. 1. H.264 Library Detailed Reference for Verification and Design Exploration H.264 Overview I Previous video coding standards use an 8x8 in raster scan order. Two slice types are sup- Broadcast television and home entertainment real discrete cosine transform (DCT) to exploit ported in H.264 baseline profile. In an I-slice, have been revolutionized by the advent of the spatial redundancy in the 8x8 block of all macro blocks are encoded in intra mode. digital TV and DVD-video. These applications image data. In H.264/AVC, a smaller 4x4 In a P-slice, some macro blocks are predicted and many more were made possible by the integer DCT is used which significantly using a motion compensated prediction with standardization of video compression technol- reduces ringing artifacts associated with one reference frame among the set of reference ogy. The recent standard in the MPEG (Moving the transform. frames and some macro blocks are encoded in Pictures Expert Group) series, MPEG-4, is I Various block sizes from 16x16 to 4x4 are intra mode. H.264 decoder processes the data enabling a new generation of internet-based allowed to perform motion compensation on a macro block by macro block basis. For video applications whilst the ITU-T H.263 prediction. every macro block depending on its charac- standard for video compression is now widely teristics, it will be constructed by the predicted I Previous video coding standards used a used in video conferencing systems. part of the macro block and the residual maximum of half-pixel accuracy for motion (error) part, which is coded using CAVLC. H.264/AVC (aka MPEG-4 Part 10), the result estimation. of the collaboration between the ISO/IEC I Inter prediction mode allows multiple Figure 1 shows the overall block diagram of MPEG group and the ITU-T Video Coding reference frames for block-based motion an H.264 baseline profile video decoder. Experts Group (VCEG), is the latest video compensation prediction. H.264 bit-stream passes through the “slice coding standard. The goals of this standard- I Context-adaptive variable length coding header parsing” block. This block extracts ization effort were: enhanced compression (CAVLC) and context-adaptive binary the information about each slice. In H.264 efficiency; network friendly video representa- arithmetic coding (CABAC) are used for video coding, each macro block is categorized tion for interactive applications (video tele- entropy encoding/decoding which improves as either coded or skipped. If the macro block phony) and non-interactive applications compression by 10% compared to previous is skipped, then the macro block is completely (broadcast applications, storage media appli- schemes. reconstructed using the inter prediction cations, etc). H.264/AVC provides gains in module. In this case, the residual information compression efficiency up to 50% over a wide H.264 Baseline Profile Video Decoder is zero. If the macro block is coded, then based range of bit rates and video resolutions com- Compressed H.264 bit-stream data is avail- on the prediction mode it passes through the pared to previous standards. Compared to able on slice-by-slice basis whereas a slice is “Intra 4x4 prediction” block or “Intra 16x16 previous standards, the decoder complexity usually a group of macro blocks processed prediction” block or “Inter prediction” block. is about four times that of MPEG-2 and two times that of MPEG-4 visual simple profile. Skip MB Relative to prior video coding standards, H.264 H.264/AVC introduces the following changes Elementary Bit Stream Inter Slice Macro Sub MB Prediction Header Block Parsing I In order to reduce the blocking artifacts, an Parsing Parsing Prediction Path Deblocking adaptive loop filter is used in the prediction Intra 4x4 Prediction Filter loop to reduce blocking artifacts. I A prediction scheme called intra prediction Intra 16x16 Prediction is used that exploits spatial redundancy. In this scheme, data from previously processed CAVLC Scale & macroblocks is used to predict the data for Decoding Inverse Transform the current macroblock in the current Residual Path encoding frame. Figure 1: Block diagram of H.264 baseline profile video decoder.
  2. 2. H.264 Library 2 The output macro block is reconstructed using the prediction output from the predic- tion module and the residual output from the “scale and transform” module. Once all the macro blocks in a frame are reconstructed, de-blocking filter will be applied for the entire frame. The “macro block parsing module” parses the information related to the macro block, such as prediction type, number of blocks coded in a macro block, partition type, motion vectors, etc. The “sub macro block” parsing module parses the information if the macro block is split into sub macro blocks of one of the sizes 8x8, 8x4, 4x8, and 4x4 when the macro block is coded as inter macro block. If the macro block is not split into sub macro blocks, all the three prediction types Figure 2.: CoWare Signal Processing Designer View of H.264 baseline decoder operation (Intra16x16, Intra4x4, or Inter) can be used. In inter prediction module, the motion compensated predicted blocks are predicted H.264/AVC baseline profile video decoder is is a 2x2 inverse Hadamard transform, which from the previous frames, which are already using CAVLC entropy coding method to is used to form the DC coefficients of the decoded. decode the encoded quantized residual chrominance blocks. transform coefficients. In CAVLC module, Intra prediction means that the samples of a the number of non-zero quantized transform The 4x4 block transform and motion com- macro block are predicted by using the already coefficients, the actual size and the position pensation prediction can be the source of transmitted macro blocks of the same image. of each coefficient are decoded separately. blocking artifacts in the decoded image. In H.264/AVC, two different types of intra The tables used for decoding these parameters Filtering the block edges will improve the prediction modes are available for coding are adaptively changed depending on the final visual quality of decoded image. H.264 luminance component of the macro block. previously decoded syntax elements. After standard is using in-loop deblocking filter to The first type is called INTRA_4x4 mode and decoding, the coefficients are inverse zigzag remove the blocking artifacts. the second type is called INTRA_16x16 scanned and form a 4x4 blocks which are mode. In INTRA_4x4 prediction mode, each given to “scale and inverse transform module”. CoWare Signal Processing Designer Library of macro block of size 16x16 is divided into H.264 Baseline Profile Video Decoder small blocks of size 4x4 and prediction is In “scale and inverse transform module”, The Signal Processing Designer library of carried out individually for each sub-block inverse quantization and inverse transforma- H.264 baseline profile video decoder system, using one of the nine prediction modes avail- tion are done on the decoded coefficients developed by iDeaWorks, is shown in Figure 2. able. In INTRA_16x16 prediction mode, the and form a residual data suitable for inverse prediction. Three different types of transforms The decoder takes H.264 baseline profile prediction is carried out at macro block level are used in H.264 standard. The first type is elementary bit stream from a source as NAL using one of the four prediction modes 4x4 inverse integer discrete cosine transform units and decodes the compressed video available. Intra prediction for chrominance (DCT), which is used to form the residual streams and displays and dumps the decoded components of a macro blocks is similar blocks of both luminance and chrominance frames. Frame width, frame height, and NAL to the INTRA_16x16 prediction of the lumi- blocks. Second type is a 4x4 inverse Hadamard unit size can be set using global parameters. nance component. transform, which is used to form the DC There are nine major blocks in the top level coefficients of the 16 luminance blocks of the hierarchy of H.264 video decoder, which are: INTRA_16x16 macro blocks. Third transform
  3. 3. 3 Source Block Frame Construct Block The algorithm model meets these industry This block converts H.264 baseline profile This is a multirate block, in which the input standards. elementary bit stream into fixed sized frame is coming in as MBs and the output is going I ITU-T H.264 standard OR NAL units. Each frame NAL unit contains all out as frame. This block takes vectors of size I ISO/IEC MPEG-4 Part 10 standard NAL units of a single frame. The block takes 532 bytes and forms a single frame. There H.264 bit stream file name and frame NAL are three modules in this block, which are Included with the library are reference test unit size as parameters. It gives frame NAL benches consisting of standard media video I Polymorphic vector-to-scalars unit as a vector of size specified by the input feeds capable of exercising the algorithm and conversion block parameter and end of sequence (EOS) flag any implementation of the reference library. I Polymorphic scalars-to-vector The decoder library is qualified with the JVT as outputs. conversion block standards compliance video streams. Parser Block I Pur filter data block This module is used to parse the following The following packages are available: Display Reorder Block set of information This block is used to reorder the decoded I CoWare Signal Processing Designer H.264 I Control data required for doing intra and YUV display data and is also used to crop the Reference Library inter prediction output YUV data according to cropping –H.264 models as partitioned verification I Motion compensation data required for parameters. This block takes decoded frame reference quarter pixel interpolation data, display re-order control data, and end of sequence flag + cropping parameters as –Reference media streams I Quantized residual transform coefficients inputs and gives display frame data and a –Value: Validated, partitioned executable I Control parameters for performing scale flag indicating whether the display frame reference inside CoWare flow for either and transform data is valid or not. customer H.264 algorithm optimization I Control data required for applying the I CoWare Signal Processing Designer H.264 deblocking filter on the decoded image Sink Block Design Library This block is used to dump the decoded I Control data required for display re-ordering –Same as reference library, but down to very YUV data into an output file. This block The parser works at frame level. takes display frame data, flag that indicates low level of granularity the validity of frame data, control data for –Reference module test benches Frame to Macroblock Conversion Block cropping, and end of sequence flag as inputs. –Value: Verification and detailed reference This is a multirate block, which converts The output file name can be set using para- for the design and optimization of sub frame level parameters to macroblock level meters of sink block. functions parameters. The parser block drives this block. The output rate is ((Frame Width x –Documentation: Detailed Algorithm docu- Frame Height)/256) times the input rate. H.264 Library Use Models mentation representing, Control, dataflow The H.264 Baseline Video Decoder was devel- and Memory requirements. Macroblock Process Block oped as a optimized, architecture neutral ref- –Test Vectors: the scaling transform, predic- The macroblock process block consists of erence standard for users incorporating tion, and de-blocking filter modules have a four major modules, which are advanced video decoding in embedded or full ten frames of test vectors for QCIF, CIF, I Scale and transform module stand-alone products. It consists of a fully and HD 720p resolutions. decomposed software C-based algorithm I Prediction module I CoWare Signal Processing Designer H.264 encapsulated within CoWare Signal I Deblocking module Source Library Processing Designer. This environment I Intra feedback data module insures users can use this product as a refer- –Requires H.264 Design Library This block operates at macroblock level. ence tool for FPGA, ASIC, SoC, or embedded –Source code license agreement IC implementations. The algorithms have –Specific jump-start service package included been decomposed down to the ‘leaf ‘or lowest –All leaf level blocks as source code common functions. Parallelism and concur- rency is fully exposed in the CoWare Signal –Value: working C-code for embedded imple- Processing Designer H.264 Design Library mentation or starting point for detailed and CoWare Signal Processing Designer RTL design H.264 Source Library.
  4. 4. H.264 Library 4 Customer Focus CoWare provides a complete range of training, support, design methodology consulting, and integration services. Technical support requests are handled directly by experienced design engineers who are fully familiar with the application of CoWare tools and method- ologies to real-world designs. Training courses are available at CoWare offices or at the customer site and can be tailored to meet the specific needs of the design team. Sales Offices CoWare has sales offices in the U.S., Europe, Asia Pacific and Japan. For a complete listing with contact information visit www.CoWare.com. For technical or sales information call 1-888-CoWare8 or email info@CoWare.com. The ESL Design Leader CoWare, Inc. Corporate Headquarters 1732 N. First Street San Jose, CA 95112 Main: 408-436-4720 Fax: 408-436-4740 www.CoWare.com CoWare and the CoWare logo are registered trademarks of CoWare, Inc. All other marks are the property of their respective owners. ©2006 CoWare, Inc.