Pretty as a Picture: Assessing Quality in Image/Video IP


Published on

Any good IP core will conform to the specification it claims to support. But for video and image IP, matching a standard is not the same as producing the degree of image quality required for demanding applications. High quality output requires clever, high-quality development in all the crucial steps of a core's operation. In this talk from SoCIP China 2011, we look at key quality factors using JPEG 2000 compression and video deinterlacing as examples. Visit or to learn more.

Published in: Technology, Art & Photos
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Pretty as a Picture: Assessing Quality in Image/Video IP

  1. 1. Pretty as a Picture:Assessing Quality inImage/Video IP<br />Nikos ZervasVice President, CAST, Inc.<br />
  2. 2. SoCIP 2011<br />2<br />Video and Image Quality <br />Quality can be crucial, especially in applications with high-resolution displays or analytics.<br />Customers automatically judge by the quality, they see, regardless of the end application.<br />
  3. 3. Compression and Quality <br />Myths<br />Supporting an industry support defines a core’s quality.<br />A particular set of features automatically defines quality.<br />I can judge quality with vendor-provided tests.<br />Realities<br />Standards typically describe a decoder, but quality is mainly determined by the encoder.<br />Within conformance to a standard, developers have great freedom in choosing algorithms and implementation details.<br />Image or video tests can be tuned to do well with particular encoders/decoders, and may not reflect your media content.<br />SoCIP 2011<br />3<br />
  4. 4. Compression and Quality <br />Standards do not mandate quality.<br />There are potential pitfalls with every compression technology.<br />MPEG2 JPEG JPEG 2000H.264 LJPEG JPEG-LS …<br />Let’s look at one: JPEG 2000.<br />SoCIP 2011<br />4<br />
  5. 5. JPEG 2000 — Pitfalls<br />JPEG 2000 offers the best lossless and great lossy compression efficiency. <br />True only when all encoding switches are turned off.<br />JPEG 2000 offers the best lossless and great lossy compression. <br />True for Variable Bit Rate (VBR).<br />But under Constant Bit Rate (CBR), quality depends on Rate Control efficiency. <br />JPEG 2000 offers Region of Interest (ROI) for selective quality.<br />True, but the implementation of ROI may make ROI unusable.<br />JPEG 2000 minimizes blocking artifacts. <br />True, but tiling artifacts can be present when you get more than one tile per image.<br />SoCIP 2011<br />5<br />
  6. 6. JPEG 2000 — Encoding Switches<br />Parallel Mode and Bypass <br />Simplify hardware design.<br />Destroy JPEG 2000 coding Efficiency.<br />6<br />SoCIP 2011<br />-3dB<br />-1dB<br />
  7. 7. JPEG 2000 — Region of Interest<br />slide 7<br />SoCIP 2011<br />Requires higher quality for the region of interest (ROI).<br />But you still care about the quality of the background.<br />Example: Aerial photography with ROI compressed at 50:1<br />Some encoders only do this:<br />But others can do this:<br />
  8. 8. JPEG 2000 — Rate Control<br />Not defined by the standard, but needed for many applications.<br />Determines the quality you will get at a specific rate (compression ratio).<br />Its accuracy defines the buffering/storage requirements and latency on your system.<br />SoCIP 2011<br />slide 8<br />Your SIP provider should be able to provide comparative data with respect to the de facto reference software (kakadu)<br />
  9. 9. Image Processing and Quality<br />There are no standards for most image processing algorithms.<br />There are potential pitfalls with every image processing algorithm.<br />Image Scaling White BalanceBad Pixel CorrectionDe-interlacing …<br />Let’s look at one: De-interlacing.<br />SoCIP 2011<br />9<br />
  10. 10. De-interlacing – Pitfalls<br />Quality is preserved by minimizing motion artifacts<br />True, but detail needs to be preserved too.<br />Motion detection is the best de-interlacing method<br />Not really. Motion detection algorithms can be easily fooled by small motion, image noise etc<br />SoCIP 2011<br />10<br />DSP techniques provide great quaiity<br />There is no continuous time signal in video, so DSP techniques just fail<br />Treats all pixels equally<br />
  11. 11. De-interlacing- Quality<br />Detail without Motion Artifacts is preserved when filtering adopts not only to motion (temporal) but also to spatial variations <br />e.g. edges should be treated differently than flat areas<br />Content Adaptive algorithms deliver the highest quality results<br />SoCIP 2011<br />11<br />DSP Content Adaptive<br />Motion Detection<br />
  12. 12. Considerations for SIP Selection<br />How easy is it to integrate my image/video Semiconductor IP?<br />Streaming versus SoC bus interfaces.<br />External memory bandwidth and tolerance to memory latencies.<br />Run-time programmability.<br />System-level buffering requirements.<br />How much external processing does it require?<br />For example: some H.264 SIPs work as accelerators rather than complete stand-alone solutions.<br />SoCIP 2011<br />12<br />
  13. 13. Example: Camera SoC<br />13<br />SoCIP 2011<br />Good<br />Pixel data streamed from sensor to output; Host controls the SoC.<br />Easy SoC bus and memory arbitration, low software complexity.<br />Bad<br />Pixel data transferred over SoC bus; Host controls the SoC.<br />Challenging SoC bus and memory arbitration, moderate complexity.<br />Ugly<br />Pixel data transferred over SoC bus; Host performs some processing and controls the SoC.<br />SoC bus and memory arbitration becomes difficult, High complexity.<br />3 – 4 weeks to integrate, Low risk<br />2 – ??? Months to integrate, High risk<br />
  14. 14. Takeaways<br />Evaluate your video and image IP for quality before making a purchase decision.<br />Ask for software modelor reference design, or ask your vendors to encode your own clips using your settings.<br />Compare results using your perception, reference implementations, and stream analysis tools. <br />Consider integration complexity.<br />Cores that are difficult to integrate can end up costing you several times more than the core license itself.<br />Failures in system-level integration can delay your product development, cripple your product and make it non-competitive.<br />SoCIP 2011<br />14<br />