Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Video Transcoding with Intel IPP Eric Shufro April 27, 2004 COT6930
  2. 2. Introduction to Transcoding <ul><li>What is transcoding? </li></ul><ul><li>Why transcode? </li></ul><ul><li>What is involved? </li></ul><ul><li>Performance </li></ul><ul><li>Quality </li></ul><ul><li>Intel IPP </li></ul><ul><li>Applications </li></ul>
  3. 3. Transcoding Overview <ul><li>Reduced bitrate </li></ul><ul><li>Reasonable Quality </li></ul><ul><li>Performance Gain </li></ul>MPEG-2 bit stream MPEG-4 bit stream MPEG-2 bit stream MPEG-4 bit stream Decode Encode Partial Decode and Encode 1 2
  4. 4. Applications <ul><li>Streaming video for both broadband and narrow band networks. </li></ul><ul><li>Decreased video bitrate for playback on mobile or other small embedded devices. </li></ul><ul><li>Conversion and modification of pre-encoded bit streams. </li></ul><ul><li>Perhaps steganography? (jpg, mp3…) </li></ul>
  5. 5. Integrated Performance Primitives <ul><li>Provides source code and libraries for media types such as MP3, MPEG-2, MPEG-4, H.263, JPEG, JPEG2000, GSM-AMR, G.723, and computer vision. [Intel] </li></ul><ul><li>Well documented. </li></ul><ul><li>Easy to use. </li></ul>
  6. 6. MPEG-2 Decoder <ul><li>Runs in two separate threads. </li></ul><ul><li>Responsible for splitting the input stream into two separate parts, audio and video and then decoding the video stream into YUV components. </li></ul><ul><li>YUV buffers are available through the transcoder class to the input of the encoder. </li></ul>
  7. 7. MPEG-4 Encoder <ul><li>Uses input data from the decoder YUV frame buffers on a frame by frame basis. </li></ul><ul><li>Creates an MPEG-4 bit stream file, out.cms, between 800KB and 1MB in size, video only. </li></ul><ul><li>Motion Estimation can be disabled for testing purposes. </li></ul><ul><li>Requires input parameters via a parameter file, though some elements can be ignored. </li></ul>
  8. 8. Transcoder Architecture <ul><li>MPEG-2 decoder and MPEG-4 encoder based on the IPP. </li></ul><ul><li>Transcoder class encapsulates both the encoder and decoder. </li></ul><ul><li>Memory is accessible between the encoder and the decoder. </li></ul><ul><li>Transcoder runs in three separate threads. </li></ul>
  9. 9. Transcoder Initialization <ul><li>Read application parameters, source file, encoder parameter file and output file name. </li></ul><ul><li>Read the encoder parameters into memory. </li></ul><ul><li>Create decoder and splitter thread. </li></ul><ul><ul><li>DecoderInit() </li></ul></ul><ul><ul><li>EncoderInit(), overwrite parameters. </li></ul></ul><ul><ul><li>Splitter – Open bit stream </li></ul></ul><ul><li>Encode MPEG-4 header. </li></ul><ul><li>Begin transcoding. </li></ul>
  10. 10. Input Stream <ul><li>MPEG-2. </li></ul><ul><li>Contains both audio and video. </li></ul><ul><li>147 frames @ 720x480. </li></ul><ul><li>Source file is 3.57 MB. </li></ul><ul><li>Intra and Inter coded frames (I, P, B). </li></ul>
  11. 11. Encoder Modifications <ul><li>ExpandFrame() bypassed. </li></ul><ul><li>StepLuma and StepChroma artificially set after decoder init. </li></ul><ul><li>Parameters overwritten after encoder init. </li></ul><ul><li>mp4_MacroBlock changed to a public member of ippVideoEncoderMPEG4. </li></ul><ul><li>Motion Estimation can be disabled. (avoids Sum of Absolute Difference) </li></ul>
  12. 12. Modified Parameters <ul><li>Input filename (ignored). </li></ul><ul><li>Resolution (ignored). </li></ul><ul><li>Frame count (ignored). </li></ul><ul><li>Frame rate = 30. </li></ul><ul><li>ME algorithm and accuracy. </li></ul><ul><li>Number of motion vectors, 1 or 4. </li></ul>
  13. 13. Decoder / Encoder Interfacing <ul><li>memcpy(mp4enc.mCurrPtr Y , frame->Y_comp_data, mp4par.Width * mp4par.Height); </li></ul><ul><li>memcpy(mp4enc.mCurrPtr U , frame->U_comp_data, mp4par.Width * mp4par.Height /4 ); </li></ul><ul><li>memcpy(mp4enc.mCurrPtr V , frame->V_comp_data, mp4par.Width * mp4par.Height /4 ); </li></ul>Transcoder is hard coded to work with 4:2:0 only
  14. 14. Motion Vectors <ul><li>No ME for I-VOP’s </li></ul><ul><li>Better quality with 4MV </li></ul><ul><li>With ME disabled, all MV copied, but correct number coded. </li></ul>mp4enc.MBinfo->mv[0].dx = this->context->macroblock.vector_luma[0]; //x mp4enc.MBinfo->mv[0].dy = this->context->macroblock.vector_luma[1]; //y mp4enc.MBinfo->mv[1].dx = this->context->macroblock.vector_luma[2]; //x mp4enc.MBinfo->mv[1].dy = this->context->macroblock.vector_luma[3]; //y mp4enc.MBinfo->mv[2].dx = this->context->macroblock.vector_luma[4]; //x mp4enc.MBinfo->mv[2].dy = this->context->macroblock.vector_luma[5]; //y mp4enc.MBinfo->mv[3].dx = this->context->macroblock.vector_luma[6]; //x mp4enc.MBinfo->mv[3].dy = this->context->macroblock.vector_luma[7]; //y 16x16, 1MV 8x8, 4MV
  15. 15. Output Stream <ul><li>MPEG-4. </li></ul><ul><li>Contains only video. </li></ul><ul><li>147 frames @ 720x480. </li></ul><ul><li>Output file is 824 KB. </li></ul><ul><li>Intra and Inter coded VOP’s (I, P). </li></ul>
  16. 16. Image Comparison MPEG-2 to MPEG-4 with 1 MV and ME Enabled Mpeg-2 Mpeg-4
  17. 17. Peak Signal to Noise Ratio <ul><li>F(i,j), The Average Decoded Luminance Pixel Shade (0-255) </li></ul><ul><li>N^2 = The Number of pixels present </li></ul>Error Image
  18. 18. PSNR of Sample Clip 1 MV 4 MV Graphs created by CalcPSNR, a freely distributed product of VideoSoft Inc. Motion Estimation Enabled
  19. 19. PSNR of Sample Clip 1 MV 4 MV Graphs created by CalcPSNR, a freely distributed product of VideoSoft Inc. Motion Estimation Disabled
  20. 20. Conclusion <ul><li>Transcoding is practical for many applications and quality can be maintained. </li></ul><ul><li>PSNR is reasonable measure of quality, but does reveal everything. </li></ul><ul><li>Partial decoding and encoding along with motion vector reuse can save execution time (168%)! </li></ul><ul><li>Dramatic difference in execution time between AMD and Intel processors of near equivalent speed due to the use of the Intel IPP. </li></ul>
  21. 21. Limitations <ul><li>Resolution (Input = Output) </li></ul><ul><li>Format (4:2:0) </li></ul><ul><li>Audio (None) </li></ul>
  22. 22. References <ul><li>Intel - </li></ul><ul><li>VideoSoft - (for PSNR) </li></ul><ul><li>HK – H. Kalva, A. Vetro, and H. Sun, &quot;Performance Optimization of the MPEG-2 to MPEG-4 Video Transcoder“ , May 2003. </li></ul><ul><li>GIT - Seong Hwan Jang, Nikil Jayant (Georgia Institute of Technology) </li></ul>