11/11/2004 MPEG4 Codec for Access Grids National Center for High Performance Computing Speaker:  Barz Hsu Email:   [email_...
Outline <ul><li>Definition of HDTV </li></ul><ul><li>Brief Introduction to MPEG-4 Standard </li></ul><ul><li>History and P...
What’s High Definition Video? <ul><li>Frame Resolution: </li></ul><ul><ul><li>>= 720x480 or 640x480 </li></ul></ul><ul><li...
Another Key Factors for Access Grids <ul><li>Real-Time </li></ul><ul><ul><li>Minimize latency & jitter </li></ul></ul><ul>...
Status of MPEG4  <ul><li>Part 2:  Advanced Simple Profile (ASP) </li></ul><ul><li>Part 10: Advanced Video Coding (AVC or H...
Performance Comparison
Interoperability Issues <ul><li>Communicate with Polycom, iChat AV (Tiger) </li></ul><ul><li>Transport Stream </li></ul><u...
VIC:  Vi deo Conferencing  T ool <ul><li>Developed by Network Research Group  in U.C. Berkeley </li></ul><ul><li>Mbone Too...
The Problems of VIC <ul><li>Hard to maintain  </li></ul><ul><li>Lack of high-quality video codec </li></ul><ul><ul><li>onl...
Our Goal <ul><li>Broadcast in Television Quality </li></ul><ul><ul><li>Leverage with Existing Open Source Resources </li><...
MPEG4 Codec <ul><li>Related Open Source Project </li></ul><ul><ul><li>FFmpeg, xvid, and sklmp4 </li></ul></ul><ul><li>FFmp...
Benefits from MPEG4 codec <ul><li>Support arbitrary sizes of a video stream   </li></ul><ul><ul><li>in contrast to H.261 <...
MPEG4 Performance <ul><li>Typical bit-rate is   1Mbps </li></ul><ul><ul><li>720x480 (NTSC) with 25 frames/sec and PSNR val...
Deinterlace <ul><li>Convert interlaced video into progressive one </li></ul>
Efficient Color Conversion <ul><li>VIC didn’t give correct color representation. </li></ul><ul><li>Efficient Color Convers...
Scaling Viewing Windows <ul><li>Original VIC </li></ul><ul><ul><li>only support limited sizes of viewing windows </li></ul...
Full Screen Snapshot
Real-Time Video Streaming <ul><li>Real-time </li></ul><ul><ul><li>the compressed data are transferred at a speed that matc...
Error Resilience <ul><li>Error Progratation </li></ul>
Error Resilience (cont’d) <ul><li>Resynchronization marker </li></ul><ul><ul><li>the data between the error location and t...
Minimize Latency <ul><li>Multithread Encoding </li></ul><ul><ul><li>Take advantage of symmetric multiple processor archite...
Future Work <ul><li>Support H.264 codec </li></ul><ul><li>Support MPEG-4 High Efficient AAC (HE-AAC) </li></ul>
Conclusion <ul><li>Project Website: </li></ul><ul><li>http://vic.nchc.org.tw </li></ul>
Thanks for  your attendance
Upcoming SlideShare
Loading in …5
×

MPEG4 codec for Access Grid

777 views
743 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
777
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Good morning, everybody. I am Barz Hsu, come from Taiwan. Currently, work in National Center for High Performance Computing, abbreviated as NCHC. This talk is entitled to “ MPEG4 Codec for Access Grids.”
  • This is the outline. First, we want to give a definition about HDTV Then, give a brief introduction to MPEG4 standard . Then, we would like to demonstrate our contributions to the AG community. Especially, we have incorporated MPEG4-codec to provide high definition and high quality video streaming. However, it is not straightforward to deploy MPEG4 codec into access grid, because it is required much more computational resource and also becomes error-sensitive to pack loss. So, we would pay attention over performance issues and error-resilience capability of MPEG4 codec.
  • Mul
  • Let review the history of VIC VIC is a video conferencing application developed by the Network Research Group in U.C. Berkeley. This group creates a set of Mbone tools. They are fully used by Access Grid , inSORS, and VRVS community. VIC is based on Real Time Protocol and Real Time Control Protocol to provide real-time video streaming. It supports multicasting transmission to relieve the exponential growth of bandwidth requirement, and also support unicasting. VIC is written by hybrid of Tcl/Tk and C++ languages, and currently supports codec including H.261, H.263, H263++, Motion JPEG, nv, and etc.
  • VIC is a critical component for AG, however, there are still many problems. First, it is somewhat hard to maintain owing to it is ten years old and uses mixed languages of Tcl/Tk and C++. The hybrid nature makes VIC hard to trace and debug. LACK OF HIGH-QUALITY VIDEO CODEC Although VIC supports many video codecs, unfortunately, only H.261 is relatively usable, other codecs have some drawbacks such as easy crash, burst bit-rate, and obvious block artifacts. THERE ARE SOME LIMITATIONS ON FRAME RESOLUTION The supported video frame resolutions are only CIF and QCIF or 4CIF However, even using H.261, fast movement can result in degradation and parts of the video feed not being updated for several seconds, depending upon network condition. Also, the viewing windows can only be resized in very limited sizes. Video quality is subjectively good when displayed in medium or small windows but can look pixellated when displayed in a large window. The other serious problem is that VIC lacks of exhaustive optimizations on critical components, for examples, codec, render, grabber, or even the memcpy function. I believe the performance is incredible critical if we want Access Grid to be scalable to accommodate large number of sites.
  • Our goal is that we want Access Grid can be broadcasting in television quality, or even in cinema quality. The goal is simple to be understood, however difficult to be realized. Fortunately, we don’t need to reinvent the wheel to do everything. Instead, we leverage with existing open source resources, such as VLC, MPlayer, and FFmpeg, To provide “sense of presence”, first, the video conference should be as real-time as possible. For example If not, you might not realize that your partner was angry until 5 seconds later. Next, we are going to supporting high-definition video source and studio-quality video conferencing by incorporate with up-to-date codecs such as MPEG2, MPEG4, and H.264. Due to advanced codecs designs, in same bit-rate these codecs can provide higher quality and less compression artifacts than H.261’s. Furthermore, error resilience and concealment are essential to maintain quality even when the situation of pack loss and error are heavy. We will detail these in the following.
  • MPEG4 codec for Access Grid

    1. 1. 11/11/2004 MPEG4 Codec for Access Grids National Center for High Performance Computing Speaker: Barz Hsu Email: [email_address]
    2. 2. Outline <ul><li>Definition of HDTV </li></ul><ul><li>Brief Introduction to MPEG-4 Standard </li></ul><ul><li>History and Problems of VIC </li></ul><ul><li>Contributions </li></ul>
    3. 3. What’s High Definition Video? <ul><li>Frame Resolution: </li></ul><ul><ul><li>>= 720x480 or 640x480 </li></ul></ul><ul><li>Frame Rate </li></ul><ul><ul><li>>= 20 fps </li></ul></ul><ul><li>Peak Signal to Noise Ratio (PSNR) </li></ul><ul><ul><li>>= 30 dB </li></ul></ul>
    4. 4. Another Key Factors for Access Grids <ul><li>Real-Time </li></ul><ul><ul><li>Minimize latency & jitter </li></ul></ul><ul><li>Error Resilience </li></ul><ul><ul><li>RTP over UDP is unreliable </li></ul></ul><ul><li>Large Scale Video Conferencing </li></ul><ul><ul><li>Selection of suitable coding tools </li></ul></ul><ul><ul><li>Optimization </li></ul></ul><ul><li>Adaptive </li></ul><ul><ul><li>network bandwidth </li></ul></ul><ul><ul><li>computing resources </li></ul></ul>
    5. 5. Status of MPEG4 <ul><li>Part 2: Advanced Simple Profile (ASP) </li></ul><ul><li>Part 10: Advanced Video Coding (AVC or H.264) </li></ul><ul><li>Match MPEG-2 quality at up to half data rate </li></ul><ul><li>Deliver excellent video quality across entire bandwidth spectrum – from 3G to HD </li></ul><ul><ul><li>(from 40Kbps to 10MBps) </li></ul></ul>
    6. 6. Performance Comparison
    7. 7. Interoperability Issues <ul><li>Communicate with Polycom, iChat AV (Tiger) </li></ul><ul><li>Transport Stream </li></ul><ul><ul><li>RFC 3016: RTP Payload Format for MPEG-4 Audio/Visual Streams. </li></ul></ul><ul><ul><li>RFC 3640: RTP Payload Format for Transport of MPEG-4 Elementary Streams </li></ul></ul><ul><ul><li>MPEG2TS </li></ul></ul>
    8. 8. VIC: Vi deo Conferencing T ool <ul><li>Developed by Network Research Group in U.C. Berkeley </li></ul><ul><li>Mbone Tools used by Access Grid, inSORS, and VRVS </li></ul><ul><ul><li>vic for video and rat for audio </li></ul></ul><ul><li>Based on RTP/RTCP to provide real-time video </li></ul><ul><li>Support Multicast/Unicast Protocol </li></ul><ul><li>Hybrid of Tcl/Tk and C++ </li></ul><ul><li>Supported codecs </li></ul><ul><ul><li>H.261, H.263, H.263++, MJPEG, nv , and etc. </li></ul></ul>
    9. 9. The Problems of VIC <ul><li>Hard to maintain </li></ul><ul><li>Lack of high-quality video codec </li></ul><ul><ul><li>only H.261 is relative useable </li></ul></ul><ul><li>Limitation on frame resolution </li></ul><ul><ul><li>Only QCIF (176x144), CIF (352x288) </li></ul></ul><ul><li>Limitation on viewing resolution </li></ul><ul><li>Lack of exhaustive optimization on codec, render, grabber , and etc </li></ul>
    10. 10. Our Goal <ul><li>Broadcast in Television Quality </li></ul><ul><ul><li>Leverage with Existing Open Source Resources </li></ul></ul><ul><ul><ul><li>VLC, MPlayer, and FFmpeg </li></ul></ul></ul><ul><ul><li>Providing “ Sense of Presence ” </li></ul></ul><ul><ul><li>As Real-Time As Possible </li></ul></ul><ul><ul><li>High-Definition, High Quality </li></ul></ul><ul><ul><li>Incorporate with up-to-date codecs such as MPEG2, MPEG4 (ASP), H.264 </li></ul></ul><ul><ul><li>Less Compression Artifact </li></ul></ul><ul><ul><li>Error Resilience/Concealment </li></ul></ul>
    11. 11. MPEG4 Codec <ul><li>Related Open Source Project </li></ul><ul><ul><li>FFmpeg, xvid, and sklmp4 </li></ul></ul><ul><li>FFmpeg/libavcodec </li></ul><ul><ul><li>High performance of CPU usage </li></ul></ul><ul><ul><li>Support various codec </li></ul></ul><ul><ul><ul><li>MPEG4, MPEG2, MPEG1 </li></ul></ul></ul><ul><ul><ul><li>H.263++, H.263, H.261 </li></ul></ul></ul><ul><ul><li>Provide error resilience tools </li></ul></ul><ul><ul><ul><li>slices, data partition </li></ul></ul></ul><ul><ul><li>Cross Platform </li></ul></ul><ul><ul><ul><li>Linux, Windows, and Mac OS </li></ul></ul></ul>
    12. 12. Benefits from MPEG4 codec <ul><li>Support arbitrary sizes of a video stream </li></ul><ul><ul><li>in contrast to H.261 </li></ul></ul><ul><ul><li>the largest size depends on your video grabber card and camera </li></ul></ul><ul><li>Less obvious block artifacts </li></ul>
    13. 13. MPEG4 Performance <ul><li>Typical bit-rate is 1Mbps </li></ul><ul><ul><li>720x480 (NTSC) with 25 frames/sec and PSNR value is about 40 dB </li></ul></ul><ul><li>Consistent bit-rate </li></ul><ul><ul><li>adjusting bit-rate only affects to the quality; not influence the frame-rate </li></ul></ul><ul><li>Up to 40 CIF, or 10 NTSC video streams. </li></ul>
    14. 14. Deinterlace <ul><li>Convert interlaced video into progressive one </li></ul>
    15. 15. Efficient Color Conversion <ul><li>VIC didn’t give correct color representation. </li></ul><ul><li>Efficient Color Conversion </li></ul><ul><ul><li>Grabber (YUYV) to Encoder (YUV420) </li></ul></ul><ul><ul><li>Decoder (YUV420) to Display (RGB15/RGB16/RGB24) </li></ul></ul>
    16. 16. Scaling Viewing Windows <ul><li>Original VIC </li></ul><ul><ul><li>only support limited sizes of viewing windows </li></ul></ul><ul><ul><li>based on unrealistic pixel replication method </li></ul></ul><ul><li>NCHC-VIC </li></ul><ul><ul><li>scaling viewing windows arbitrarily without too much overhead </li></ul></ul><ul><ul><li>interpolation by 9 tap FIR filter </li></ul></ul><ul><ul><li>hardware assisted by Xvideo extension in XFree86 </li></ul></ul><ul><ul><li>full screen mode </li></ul></ul>
    17. 17. Full Screen Snapshot
    18. 18. Real-Time Video Streaming <ul><li>Real-time </li></ul><ul><ul><li>the compressed data are transferred at a speed that matches the coded video source rate. </li></ul></ul><ul><li>Latency, Jitter </li></ul><ul><ul><li>average end-to-end delay </li></ul></ul><ul><ul><li>delay variation </li></ul></ul>
    19. 19. Error Resilience <ul><li>Error Progratation </li></ul>
    20. 20. Error Resilience (cont’d) <ul><li>Resynchronization marker </li></ul><ul><ul><li>the data between the error location and the first following resynchronization marker would have to be discarded. </li></ul></ul><ul><li>Data Partition </li></ul><ul><ul><li>motion vectors and related syntactic elements are transmitted first, followed by syntactic elements like MB headers, motion vectors, and DCT coefficients. </li></ul></ul>
    21. 21. Minimize Latency <ul><li>Multithread Encoding </li></ul><ul><ul><li>Take advantage of symmetric multiple processor architecture </li></ul></ul><ul><li>Pipeline the latency of transmission and encoding </li></ul><ul><li>SIMD Instruction Ses </li></ul><ul><ul><li>Optimized by SSE2/SSE, MMX </li></ul></ul>
    22. 22. Future Work <ul><li>Support H.264 codec </li></ul><ul><li>Support MPEG-4 High Efficient AAC (HE-AAC) </li></ul>
    23. 23. Conclusion <ul><li>Project Website: </li></ul><ul><li>http://vic.nchc.org.tw </li></ul>
    24. 24. Thanks for your attendance

    ×