Choosing the Best IP for Compressing Images & Video


Published on

Introduces some of the key factors to consider when choosing IP cores -- and a supplier -- for image and video compression applications. Covers compression algorithms, hardware implications, and evaluation methods for JPEG, JPEG 2000, H.264, and more.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • This slide is how to evaluate a vendor’s implementation of an algorithm. To evaluate the actual algorithm, download Codecs from the web. There are also a number of compression algorithm comparison technical papers on the web. Synthesis results should be provided for your target technology by the IP provider Engineering expertise can be determined during the pre-sales period. Also need system expertise.
  • Choosing the Best IP for Compressing Images & Video

    1. 1. Choosing the Best IP for Compressing Images and Video CAST, Inc. +1 201.391.8300 [email_address]
    2. 2. Selecting the Best IP <ul><li>Algorithms </li></ul><ul><ul><li>Image Quality </li></ul></ul><ul><ul><li>Formats </li></ul></ul><ul><li>Hardware </li></ul><ul><ul><li>Area </li></ul></ul><ul><ul><li>Speed </li></ul></ul><ul><ul><li>Memory </li></ul></ul><ul><ul><li>System Integration </li></ul></ul><ul><ul><li>Application Requirements </li></ul></ul><ul><li>Evaluation </li></ul><ul><li>Expertise </li></ul>JPEG JPEG-LS JPEG 2000 Lossless JPEG H.264 Motion JPEG
    3. 3. Compression Algorithms <ul><li>Lossy discards image data but looks good enough </li></ul><ul><li>Lossless discards no image data </li></ul><ul><li>Near-Lossless visually identical, or numerically acceptable data loss </li></ul>
    4. 4. <ul><li>JPEG </li></ul><ul><ul><li>Good quality < 20:1 </li></ul></ul><ul><ul><li>Moderate quality 20:1 – 50:1 </li></ul></ul><ul><ul><li>Poor quality > 50:1 </li></ul></ul><ul><li>JPEG 2000 </li></ul><ul><ul><li>Good up to 80:1 </li></ul></ul><ul><li>H.264 </li></ul><ul><ul><li>Best for > 50:1 </li></ul></ul>Image Quality: Lossy
    5. 5. <ul><li>LJPEG </li></ul><ul><ul><li>Compresses 2:1 </li></ul></ul><ul><ul><li>Not good for text, computer graphics </li></ul></ul><ul><li>JPEG-LS </li></ul><ul><ul><li>2.3:1 </li></ul></ul><ul><ul><li>Usually best </li></ul></ul><ul><li>JPEG 2000 </li></ul><ul><ul><li>2.1:1 </li></ul></ul><ul><ul><li>Flexible (lossy, too) </li></ul></ul>Image Quality: Lossless
    6. 6. Input Format: Image Types <ul><li>Still images vs. motion video </li></ul><ul><ul><li>Still images — JPEG, LJPEG, JPEG-LS, JPEG 2000 </li></ul></ul><ul><ul><li>Motion video — Motion JPEG, H.264 </li></ul></ul><ul><li>H.264 Motion Estimation </li></ul><ul><ul><li>Predicts where motion will occur </li></ul></ul><ul><ul><li>Represents motion, not entire frame: fewer bits </li></ul></ul>
    7. 7. Input Format: Data Factors <ul><li>Number of bits/pixel </li></ul><ul><ul><li>JPEG — 8 </li></ul></ul><ul><ul><li>LJPEG — up to 16 </li></ul></ul><ul><ul><li>JPEG-LS — up to 16 </li></ul></ul><ul><ul><li>JPEG 2000 — up to 16 </li></ul></ul><ul><li>Number of pixels: frame size </li></ul><ul><li>Representation method </li></ul>CGA 320 x 200 64,000 pixels QSXGA 2560 x 2048 5,242,880 pixels
    8. 8. Input Format: Subsampling <ul><li>Eye more sensitive to changes in brightness ( luma ) than color ( chroma ) </li></ul><ul><li>Can use just some of the chroma info (a subsample ) to compress an image </li></ul><ul><li>Encoders use different subsample formats </li></ul><ul><ul><li>JPEG — 4:4:4, 4:2:2, 4:1:1, 4:2:0, grayscale </li></ul></ul><ul><ul><li>LJPEG — 4:4:4, grayscale </li></ul></ul><ul><ul><li>JPEG-LS — 4:4:4, 4:2:2, 4:1:1, 4:2:0, grayscale </li></ul></ul><ul><ul><li>JPEG 2000 — 4:4:4, 4:2:2, 4:1:1, 4:2:0, grayscale </li></ul></ul><ul><ul><li>H.264 — 4:2:0 </li></ul></ul>Color Space Notation Y:C b :C r Y = luma value C b :C r together = chroma value Can include fewer chroma values in various ways. Another popular color space is similar: Y:U:V
    9. 9. Hardware & Algorithm Trade-Offs <ul><li>Area </li></ul><ul><li>Speed </li></ul><ul><li>Memory </li></ul><ul><li>System Integration </li></ul><ul><li>Application Requirements </li></ul>
    10. 10. Area <ul><li>JPEG 2000 Parameters </li></ul><ul><li>No. of Entropy Encoders </li></ul><ul><li>Filter Types & Levels </li></ul><ul><li>Quantization Tables </li></ul><ul><li>Block Size </li></ul><ul><li>Whole Frame or Tiles </li></ul>
    11. 11. Speed <ul><li>JPEG fastest on the market, 1 clock/sample </li></ul><ul><li>JPEG 2000 1.4 clocks/sample, depending on configuration </li></ul><ul><li>H.264 2.5 clocks/sample </li></ul><ul><li>LJPEG and JPEG-LS very fast, 1 clock/sample </li></ul>Max clock rate of the technology limits the frame size and fps
    12. 12. Memory <ul><li>Dependent on frame size </li></ul><ul><li>H.264 can use SDRAM, DDR or DDR2 (more flexible than MPEG4) </li></ul><ul><li>JPEG 2000 large demands, but can reduce by tiling (at expense of speed) </li></ul>WVGA 480 x 854 1024 x 768 1080p 1920 x 1080
    13. 13. Integration <ul><li>Latency </li></ul><ul><ul><li>Time from first input sample to first compressed output </li></ul></ul><ul><li>CPU required? </li></ul><ul><ul><li>Core may have a CPU for configuration, or read from input stream </li></ul></ul><ul><li>Memory Interface </li></ul><ul><ul><li>Best not embedded in compression core to allow flexibility </li></ul></ul>
    14. 14. Application Requirements <ul><li>Rate Control </li></ul><ul><li>Error Resilience </li></ul><ul><li>Special Features </li></ul><ul><ul><li>Multiple Resolutions </li></ul></ul><ul><ul><li>Quality Layers </li></ul></ul><ul><ul><li>Rectangular Regions of Interest </li></ul></ul>
    15. 15. Rate Control <ul><li>Trades off transmission bit rate and video quality </li></ul><ul><li>Essential for applications with constrained channel bandwidth or storage capacity </li></ul><ul><li>Must: </li></ul><ul><ul><li>Guarantee channel will never be overflowed </li></ul></ul><ul><ul><li>Maximize quality for even very low bit rates </li></ul></ul><ul><ul><li>Be adaptable to any type of scene or motion </li></ul></ul><ul><ul><li>Provide a fast response to changes in video characteristics </li></ul></ul><ul><ul><li>Allow simple system integration </li></ul></ul><ul><li>Hardware Bit Rate Control is best </li></ul>
    16. 16. Multiple Resolution Levels <ul><li>JPEG 2000 can store Images at multiple resolutions </li></ul><ul><li>Thumbnails quickly pulled from storage and analyzed </li></ul><ul><li>Higher resolution images then requested from storage for in-depth analysis </li></ul><ul><li>Excellent for reconnaissance & surveillance </li></ul>From:
    17. 17. Error Resilience <ul><li>JPEG Restart markers, but works on blocks </li></ul><ul><li>JPEG-LS None </li></ul><ul><li>JPEG 2000 Packet headers in stream, other features </li></ul><ul><li>H.264 Reference frame errors extend to dependent frames; CAST adds multiple slices to increase error resilience </li></ul>
    18. 18. Quality Layers with JPEG 2000 <ul><li>Military applications: thermal IR & satellite imagery </li></ul><ul><li>Multiple layers with different compression ratios </li></ul>Original Lossless 710 Kbytes 20 Quality Layers 8:1 Compression 88.76Kbytes 10 Quality Layers 16:1 Compression 44.28 Kbytes 5 Quality Layers 256:1 Compression 2774 bytes 1 Quality Layer
    19. 19. Rectangular Regions of Interest <ul><li>Specify an area is more important than the rest of the image </li></ul><ul><li>Vary quality of the region and the background per: </li></ul><ul><ul><li>Desired compression ratio </li></ul></ul><ul><ul><li>Relative importance of the region </li></ul></ul>50:1 Compression Region: no special importance 50:1 Compression Region: slightly important 50:1 Compression Region: moderately important 50:1 Compression Region: extremely important
    20. 20. Image & Video IP Evaluation <ul><li>Get a Bit Accurate Model ( BAM ) to run your images </li></ul><ul><li>Try a Prototype Board with the IP in hardware </li></ul><ul><li>Get references in your market sector (military is tough) </li></ul><ul><li>Study representative synthesis results for area and max operating frequency </li></ul><ul><li>Make sure you will have access to compression engineers </li></ul>
    21. 21. CAST Compression Reference Design System <ul><li>Complete FPGA System Board: </li></ul><ul><ul><li>Learn compression </li></ul></ul><ul><ul><li>Evaluate our cores with your media </li></ul></ul><ul><ul><li>Get a system head start </li></ul></ul><ul><li>Combines multiple cores and essential software </li></ul>
    22. 22. Vendor Expertise <ul><li>CAST : </li></ul><ul><li>Delivering IP for 16 Years </li></ul><ul><ul><li>Privately-held, independent </li></ul></ul><ul><ul><li>1,000 sales to 600 customers </li></ul></ul><ul><li>Expert Development Partners </li></ul><ul><li>Extreme Customer Focus </li></ul><ul><ul><li>24/7, fast-response culture </li></ul></ul><ul><ul><li>Global team of 100 people </li></ul></ul>Airborne Reconnaissance Audio/Video Platform Broadcast Equipment Camera Module Cell Phone Camera Image Processing Chip Digital Still Camera Digital Video Recorder Earth Observation Satellite High-Definition Monitor High-Resolution Airborne Reconnaissance Sensor High-Speed Video Camera Imaging Chips Military - Classified Mobile Camera SoC Mobile Digital Video Recorder Platform Mobile Phone SIM Card Multiple Kill Vehicle Office Class Copier/printer Prototype 3D Camera Satellite - Classified Satellite Camera and Measurement Device Security and Surveillance Camera Security LCD Monitor SoC-on-panel Display Surveillance Camera Technology Proof-of-Concept Camera TV Surveillance Camera Display USB Module Video Compression Video Conferencing System Video Image Acquisition System for the Military Video Surveillance for Road Tolling Weather Satellites Wireless Rear-view Camera X-ray Dental Project Some Customer Multimedia Applications
    23. 23. CAST Broad IP Cores Line Multimedia JPEG JPEG Scalado Speedtags JPEG Ext. 12-bit JPEG Lossless JPEG-LS JPEG 2000 H.264 Video Audio: I2S, SPDIF Image Conversion System IP Subsystems 8051/USB, 8051/MAC SoC Platforms ARM, eASIC AMBA Infrastructure Replacements Processors, Bit Slices Controllers, Peripherals, Timers, UARTs, TTL Parts Encryption AES DES, Triple DES MD5 SHA-1, SHA-256 Communications Consumer IR Controllers Synchronous UARTS SDLC, HDLC Interfaces PCI & PCI Express Ethernet MACs USB & OTG CAN, LIN, I2C, SPI ECP Memory & Peripherals Memory Controllers DDR1/2 SDRAM SDIO Host SDR mobile SDRAM NAND flash Serial flash Storage Controllers ATA/IDE Interface Device Controllers Smart Card Reader, TV Display, Hi-Res Processors 8-bit: 8051 fast, tiny, entry Z80, C6502 16-bit: 68000, 80186xl, 80186ec, 387L 32-bit 68000 AHB DSP: 32025, 32025TX See more at
    24. 24. CAST Image & Video IP Cores Multimedia JPEG JPEG Scalado Speedtags JPEG Ext. 12-bit JPEG Lossless JPEG-LS JPEG 2000 H.264 Video Audio: I2S, SPDIF Image Conversion Cameras, Printers, Copiers, etc. Pro Video, Medical Imaging Best Lossless Surveillance, Medical Imaging Replacing MPEG4 Fast Cameras High Quality Imaging Block to Raster Raster to Block Downscaling Color Space Conversion