Video Transcoding in H.264 Prof. Maurizio Bonuccelli Francesca Martelli Francesca Lonetti PISATEL
Outline <ul><li>H.264 coding and transcoding </li></ul><ul><li>Motion Vector Composition algorithms </li></ul><ul><li>Over...
H.264 coding <ul><li>H.264 mean features: </li></ul><ul><li>Variable and small block sizes (4×4) </li></ul>PISATEL Average...
H.264 temporal transcoding  <ul><li>Skipping frames to reduce the output bit-rate </li></ul><ul><li>Three main issues: </l...
Motion Vector Composition <ul><li>The motion vectors are computed by  </li></ul><ul><ul><li>Motion Vector Composition Algo...
Telescopic Vector Composition F(n-1) ( skipped ) MB n MB 4 n-1 MB 1 n-1 MB 2 n-1 MV n MB 3 n-1 MV 1 n-1 MV 2 n-1 MV 4 n-1 ...
Bilinear Interpolation MV BI =(1-α)(1-β)MV 1 n-1 +(α)(1-β)MV 2 n-1 +(1-α)(β)MV 3 n-1 +(α)(β)MV 4 n-1 F(n-2) F(n-1) ( skipp...
H.264 features  <ul><li>New  in H.264: variable macroblock partition (16 motion vectors for each macroblock) </li></ul><ul...
MVC in H.264 MB1 MB2 MB3 MB4 LIV.0 Mv_MVC= f(mv_MB1,mv_B2, mv_B3, mv_B4) MB1 B1 B2 MB3 B1 B2 B3 B4 MB4 B1 B2 LIV.1 MB2 B1 ...
A new MVC algorithm <ul><li>Basic Idea : looking for a reference area, the most similar to the current macroblock </li></u...
New MVC algorithm: example V f = argmin  i Є S   MSE(A, A i ) =  argmin  i Є S   (1/NxM |A-A i | 2 ) F(n) A MB N M F(n-2) ...
MVC performance (time) PISATEL TVC BI New ME
MVC performance (quality) PSNR(dB) Frames Akiyo Motion Estimation New MVC Algorithm
MVC performance (quality) Akiyo PSNR(dB) Frames New MVC Algorithm BI Algorithm TVC Algorithm
MVC performance (quality) PSNR(dB) Frames Coastguard Motion Estimation New MVC Algorithm
MVC performance (quality) PSNR(dB) Frames Coastguard New MVC Algorithm BI Algorithm TVC Algorithm
Our activities in video transcoding <ul><li>We started in studying video features in MPEG4 </li></ul><ul><li>With this cod...
MPEG4 transcoder architectures <ul><li>After a skipped frame: </li></ul><ul><ul><li>In the FSC architecture, motion vector...
Motion Vector Composition <ul><li>In literature we found 4 MVC algorithms: </li></ul><ul><ul><li>Bilinear interpolation </...
Architectures and MVC algorithms <ul><li>M. A. Bonuccelli, F. Lonetti, F. Martelli.  Video Transcoding Architectures for M...
Skipping policies <ul><li>We developed 4 skipping policies, all of them based on the transcoder buffer occupancy: </li></u...
H.263 Video Transcoder <ul><li>Then, we studied the H.263 video codec, and we developed a temporal transcoder, based on DF...
H.263 Video Transcoder <ul><li>In our transcoder architecture, every frame is first transcoded, and then transmitted or dr...
Finally, H.264 <ul><li>This codec is the product of the union between VCEG (H.263) and MPEG, forming the Joint VideoTeam (...
Optimizing H.264 encoder <ul><li>We operated some modifications to the reference software in order to obtain acceptable en...
H.264 transcoder <ul><li>We first implemented the cascade pixel domain transcoder by simply concatenating a decoder with a...
H.264 rate control <ul><li>Finally, we are implementing the TMN8 rate control algorithm to be used in the front encoder </...
Master Theses <ul><li>Luigi D’Amaro.  Algoritmi per la transcodifica video . </li></ul><ul><li>Gianni Rosa.  Transcodifica...
Conclusions <ul><li>We studied the video transcoding problem in real-time communications </li></ul><ul><li>We developed te...
Acknowledgements <ul><li>We thank all ERI people who introduced us in this research area, for the helpful discussions and ...
Upcoming SlideShare
Loading in...5
×

martelli.ppt

474

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
474
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

martelli.ppt

  1. 1. Video Transcoding in H.264 Prof. Maurizio Bonuccelli Francesca Martelli Francesca Lonetti PISATEL
  2. 2. Outline <ul><li>H.264 coding and transcoding </li></ul><ul><li>Motion Vector Composition algorithms </li></ul><ul><li>Overview of our temporal transcoding results (MPEG4, H.263) </li></ul>PISATEL
  3. 3. H.264 coding <ul><li>H.264 mean features: </li></ul><ul><li>Variable and small block sizes (4×4) </li></ul>PISATEL Average bit rate reduction! High complexity! <ul><li>Quarter-pixel resolution </li></ul><ul><li>NAL units </li></ul><ul><li>New entropic coding algorithms (CABAC) </li></ul><ul><li>Deblocking filter </li></ul><ul><li>Moto compensation with multiple reference pictures </li></ul>
  4. 4. H.264 temporal transcoding <ul><li>Skipping frames to reduce the output bit-rate </li></ul><ul><li>Three main issues: </li></ul><ul><ul><li>Motion Vector Computation(MVC) </li></ul></ul><ul><ul><li>Prediction Error Computation </li></ul></ul><ul><ul><li>Frame Skipping Policy </li></ul></ul>PISATEL Transcoder Input bit-rate 64 Kb Output bit-rate 32 Kb 1 2 3 1 3 1 2 3
  5. 5. Motion Vector Composition <ul><li>The motion vectors are computed by </li></ul><ul><ul><li>Motion Vector Composition Algorithms (BI, TVC, FDVS, ADVS) </li></ul></ul><ul><ul><li>Restricted Motion Estimation (RME) </li></ul></ul>MVC skipped F(n-1) F(n) F(n-2) mv n-1 mv n
  6. 6. Telescopic Vector Composition F(n-1) ( skipped ) MB n MB 4 n-1 MB 1 n-1 MB 2 n-1 MV n MB 3 n-1 MV 1 n-1 MV 2 n-1 MV 4 n-1 MV 3 n-1 F(n) MV n =MV n +MV TVC New reference area F(n-2) MB n MV TVC = MV 1 n-1
  7. 7. Bilinear Interpolation MV BI =(1-α)(1-β)MV 1 n-1 +(α)(1-β)MV 2 n-1 +(1-α)(β)MV 3 n-1 +(α)(β)MV 4 n-1 F(n-2) F(n-1) ( skipped ) MB n MB 4 n-1 MB 1 n-1 MB 2 n-1 MV n MB 3 n-1 MB n MV 1 n-1 MV 2 n-1 MV 4 n-1 MV 3 n-1 F(n) MV n =MV n +MV BI New reference area MV BI α β
  8. 8. H.264 features <ul><li>New in H.264: variable macroblock partition (16 motion vectors for each macroblock) </li></ul><ul><li>Transcoder keeps the same partitions of the remote encoder (most efficient solution) </li></ul><ul><li>How to apply MVC in H.264? </li></ul><ul><ul><li>BI and TVC adaptation </li></ul></ul><ul><ul><li>New MVC algorithm </li></ul></ul>? H.264 ? Previous standards
  9. 9. MVC in H.264 MB1 MB2 MB3 MB4 LIV.0 Mv_MVC= f(mv_MB1,mv_B2, mv_B3, mv_B4) MB1 B1 B2 MB3 B1 B2 B3 B4 MB4 B1 B2 LIV.1 MB2 B1 B3 B4 B2 Mv_B4 LIV.3 Mv_B2 Mv_B2 Mv_B2 Mv_B1 Mv_B1= f(sb1,sb2) Mv_B2 Mv_MB4= f(mv_B1,mv_B2) Mv_MB3= f(mv_B2,mv_B4) Mv_MB2= f(mv_B1,mv_B2) Mv_MB1= mv_B2 B1 sb1 B2 B2 B2 B4 B2 LIV.2 B1 sb2
  10. 10. A new MVC algorithm <ul><li>Basic Idea : looking for a reference area, the most similar to the current macroblock </li></ul><ul><li>Two steps : </li></ul><ul><ul><li>For each motion vector of a macroblock (or block) of the reference area in the skipped frame, compute the difference between the macroblock and an area pointed by the motion vector with the same size of the current macroblock </li></ul></ul><ul><ul><li>Choose the motion vector of the reference area that minimizes this difference </li></ul></ul>
  11. 11. New MVC algorithm: example V f = argmin i Є S MSE(A, A i ) = argmin i Є S (1/NxM |A-A i | 2 ) F(n) A MB N M F(n-2) mv F(n-1) MB4 MB3 MB1 MB2 b1 b2 b3 b4 b2 b2 b1 b1 (skipped) mv2 A2 A4 mv4 A3 mv3 A6 mv6 mv8 A8 mv7 A7 mv5 A5 mv1 A1 new_mv
  12. 12. MVC performance (time) PISATEL TVC BI New ME
  13. 13. MVC performance (quality) PSNR(dB) Frames Akiyo Motion Estimation New MVC Algorithm
  14. 14. MVC performance (quality) Akiyo PSNR(dB) Frames New MVC Algorithm BI Algorithm TVC Algorithm
  15. 15. MVC performance (quality) PSNR(dB) Frames Coastguard Motion Estimation New MVC Algorithm
  16. 16. MVC performance (quality) PSNR(dB) Frames Coastguard New MVC Algorithm BI Algorithm TVC Algorithm
  17. 17. Our activities in video transcoding <ul><li>We started in studying video features in MPEG4 </li></ul><ul><li>With this codec, we developed two temporal transcoders: </li></ul><ul><ul><li>One based on an architecture known as FSC (Frame Skipping Control) </li></ul></ul><ul><ul><li>One based on an architecture known as DFS (Dynamic Frame Skipping) </li></ul></ul><ul><li>By simulation, we realized that the second architecture results better in terms of video quality, while the first one results better in terms of processing times </li></ul>
  18. 18. MPEG4 transcoder architectures <ul><li>After a skipped frame: </li></ul><ul><ul><li>In the FSC architecture, motion vectors and prediction errors are computed without executing motion estimation  worse quality, re-encoding errors amassment </li></ul></ul><ul><ul><li>In the DFS architecture, motion vectors and prediction errors are computed with a restricted motion estimation  better quality, but worse time performance </li></ul></ul>
  19. 19. Motion Vector Composition <ul><li>In literature we found 4 MVC algorithms: </li></ul><ul><ul><li>Bilinear interpolation </li></ul></ul><ul><ul><li>Telescopic Vector Composition </li></ul></ul><ul><ul><li>Forward Dominant Vector Selection </li></ul></ul><ul><ul><li>Activity Dominant Vector Selection </li></ul></ul><ul><li>By using the DFS architecture (that performs motion estimation) we realized by simulation that these algorithms are equivalent </li></ul>
  20. 20. Architectures and MVC algorithms <ul><li>M. A. Bonuccelli, F. Lonetti, F. Martelli. Video Transcoding Architectures for Multimedia Real Time Services , ERCIM News No. 62, pp. 39-40, July 2005. </li></ul>
  21. 21. Skipping policies <ul><li>We developed 4 skipping policies, all of them based on the transcoder buffer occupancy: </li></ul><ul><ul><li>Buffer based </li></ul></ul><ul><ul><li>Motion activity </li></ul></ul><ul><ul><li>Consecutive skipping </li></ul></ul><ul><ul><li>Random </li></ul></ul><ul><li>M. A. Bonuccelli, F. Lonetti, F. Martelli. Temporal Transcoding for Mobile Video Communication . In Proceedings of 2 nd Annual International Conference on Mobile and Ubiquitous Systems: Networking and Services (Mobiquitous 2005), pp.502-506, July 17-21, 2005, San Diego, CA. </li></ul>
  22. 22. H.263 Video Transcoder <ul><li>Then, we studied the H.263 video codec, and we developed a temporal transcoder, based on DFS architecture. </li></ul><ul><li>By observing the test results, we realize that video quality is influenced by the rate control of the front encoder </li></ul><ul><li>Then, we tested and implemented several rate control algorithms for the front encoder: </li></ul><ul><ul><li>TMN5 </li></ul></ul><ul><ul><li>TMN8 </li></ul></ul><ul><ul><li> domain </li></ul></ul><ul><ul><li>Perceptual rate control </li></ul></ul><ul><ul><li>Multiple zone (Activity) </li></ul></ul>
  23. 23. H.263 Video Transcoder <ul><li>In our transcoder architecture, every frame is first transcoded, and then transmitted or dropped. </li></ul><ul><li>To avoid the computation of frames that will be discarded, we developed a new frame skipping policy that predicts the frames to be transcoded </li></ul><ul><li>M. A. Bonuccelli, F. Lonetti, F. Martelli. A Fast Skipping Policy for H.263 Video Transcoder . In Proceedings of 12 th International Workshop on Systems, Signals and Image Processing (IWSSIP'05). September 22-24, 2005, Chalkida, Greece. </li></ul>
  24. 24. Finally, H.264 <ul><li>This codec is the product of the union between VCEG (H.263) and MPEG, forming the Joint VideoTeam (JVT) </li></ul><ul><li>It includes all benefits of previous standards in order to achieve good quality performance even at low bit-rate </li></ul><ul><li>We worked in two directions: </li></ul><ul><ul><li>Optimizing the codec, to speed up it </li></ul></ul><ul><ul><li>Implementing a transcoder </li></ul></ul>
  25. 25. Optimizing H.264 encoder <ul><li>We operated some modifications to the reference software in order to obtain acceptable encoding times: </li></ul><ul><ul><li>instead of computing all half and quarter pixels in two rounds, we compute them in only one round </li></ul></ul><ul><ul><li>fast way for choosing the optimal partitioning: instead of using the SAD (Sum of Absolute Differences) measure as decision parameter, we use other metrics: </li></ul></ul><ul><ul><ul><li>the number of differences in terms of pixels </li></ul></ul></ul><ul><ul><ul><li>the maximum difference value </li></ul></ul></ul><ul><ul><ul><li>the average difference value </li></ul></ul></ul><ul><ul><ul><li>the most popular difference value </li></ul></ul></ul><ul><ul><li>compared with proper self-adjusting thresholds. </li></ul></ul>
  26. 26. H.264 transcoder <ul><li>We first implemented the cascade pixel domain transcoder by simply concatenating a decoder with an encoder, in order to develop the motion vector composition algorithms </li></ul><ul><li>Then we are implementing the frame skipping to obtain a transcoder that decides the frames to be dropped </li></ul>
  27. 27. H.264 rate control <ul><li>Finally, we are implementing the TMN8 rate control algorithm to be used in the front encoder </li></ul><ul><li>We think that, with a rate control algorithm able to skip frame in encoding phase, the transcoding process may be improved both in terms of quality and computation time </li></ul>
  28. 28. Master Theses <ul><li>Luigi D’Amaro. Algoritmi per la transcodifica video . </li></ul><ul><li>Gianni Rosa. Transcodifica video per comunicazione mobile: studio di rate control . </li></ul><ul><li>Luca Leonardi. Transcodifica video temporale: politiche di frame skipping . </li></ul><ul><li>Marina Paletta. Realizzazione di un transcodificatore video temporale H.264 per video comunicazione mobile . </li></ul><ul><li>Riccardo Vagli. Implementazione di un transcoder video basato sullo standard H.264/AVC . </li></ul><ul><li>Alsona Dema. Rate Control in H.264 . </li></ul>
  29. 29. Conclusions <ul><li>We studied the video transcoding problem in real-time communications </li></ul><ul><li>We developed temporal transcoders with MPEG4, H.263 and H.264 codecs </li></ul><ul><li>We developed some skipping policies to be used in each transcoder </li></ul><ul><li>We developed three MVC algorithms to be used in the H.264 transcoder </li></ul>
  30. 30. Acknowledgements <ul><li>We thank all ERI people who introduced us in this research area, for the helpful discussions and advices </li></ul><ul><li>We thank PisaTel Lab people </li></ul><ul><li>We thank all students who worked with us in this project </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×