This document presents a lightweight H.264-based hardware-accelerated image compression library. The library aims to provide better compression rates and reduced compression times compared to software-only approaches by utilizing NVIDIA's hardware-accelerated video encoder. It was integrated into the ParaView visualization system and compared to existing compressors, demonstrating over 25x better bandwidth savings and modest improvements to processing times compared to the next best compressor. While using a lossy compression method, the induced artifacts were found to be minor with high image fidelity maintained.

1. A LIGHTWEIGHT H.264-BASED
HARDWARE-ACCELERATED IMAGE COMPRESSION LIBRARY
LDAV October 23, 2016 Baltimore, Maryland, USA
MOTIVATION
• Most remote visualization systems are
based on transmitting high-resolution
images over the network.
• NVIDIA’s hardware-accelerated video
encoder can provide both better
compression rates and reduced
compression times. Our goals were:
i. Easy integration to existing systems
ii. Reduced bandwidth requirements
iii. Software fallback for
less-capable hardware
• The library wraps around FFmpeg to
provide both a hardware-accelerated
encoder, as well as a software fallback.
• We abstract away complex settings
and use parameters appropriate for
low-latency, real-time visualization.
• Benchmark experiments used an Ubuntu
16.04 machine with a GeForce®
GTX 1080
(NVIDIA Pascal™
) graphics card.
• Our library uses the lossy variant of
H.264. The introduced artifacts are minor
and image fidelity remains satisfactory.
• Encoding/decoding time is linear to the
total number of pixels.
DESIGN AND
IMPLEMENTATION
BACKGROUND
Image Coherency and Video Encoding
• H.264 is a block-oriented, motion
compensation-based video compression
standard. H.264 exploits inter-frame
coherency but comes at significant
computational expense.
The compression ratio for an 8-bit
RGB image is:
Where fm
is the projected inter-frame
coherency on a scale from 1 to 4.
USE CASE
ParaView Integration
• We integrated our library into
ParaView and compared our hardware-
accelerated approach (H.264_hw) and
software fallback (H.264_sw) to built-in
compressors LZ4, SQUIRT, and zlib.
• The experiments used 300 iterations of
1080p images generated from a rotating
volume dataset. We used rotations of
1.2º per frame for ‘high-coherency’
mode and 45º per frame for ‘low-
coherency’ mode. Both improve
performance in both compression ratio
and processing time.
• Our hardware-accelerated approach
achieves the best frame rate while
the software approach is slower
than existing ParaView compressors.
Plus, our library achieves 25X better
compression than the closest competitor,
zlib.
CONCLUSION
• We save greater than 25X bandwidth
and achieve modest improvements to
compression and decompression time.
• While we currently use lossy H.264,
the induced artifacts are minor.
• The compatible software approach
expands accessibility, but introduces a
tradeoff between compression ratio and
compression time.
EXPERIMENT
Image Fidelity and Benchmark
Figure 1: Integrating the compression library into remote visualization system.
Overall latency is
Tend-to-end
=Trender
+Tcompress
+Tcommunication
+Tdecompress
+Tblit
I-frame P-frame B-frame I-frame
Figure 2 : I-frames are self-referential and do not require other frames to
decode; P-frames encode changes to previous frames; B-frames use both
previous and forward frames for data reference, acquiring best compression
at the cost of frame latency. We do not use B-frames to minimize latency.
Figure 4 : FFmpeg provides NVIDIA Video Codec as a hardware-
accelerated codec and libx264 as the software fallback.
Figure 6 : Image fidelity with high (fm
=4, 30 FPS) bitrate.
Compression of the original (left) creates only minor errors (right)
with a structural similarity of 99.3%
Table 1 : Bandwidth requirement and processing time for typical resolutions.
Aggregated processing latency for a 1080p image is less than 10ms.
Figure 3 : High-coherency image sequences require less bandwidth
(achieved using a lower fm
) to preserve image fidelity.
Figure 5 : Compression pipeline in detail. An FFmpeg interface that accepted
GPU-resident data would be a boon to our implementation.
Figure 7 : Frame rate of compression and decompression in ParaView
integration experiment. Hardware-based H.264 achieves slightly better
performance than competitors, while the software fallback is only better
than zlib.
LOW COHERENCY HIGH COHERENCY
Resolution Bitrate(mbps) Compress(ms) Decompress(ms)
1024x768 6.606 2.8057 2.1170
1920x1080 17.418 5.4358 4.2876
4096x2160 74.318 21.2504 15.0976
LZ4 SQUIRT zlib H.264_hw H.2644_sw
rc
1.29:1 1.43:1 3.43:1 85:1 88:1
PerformanceFPS
175
150
125
100
75
50
25
0
LZ4
SQUIRT
zlib
H.2647_hw
H.2647_sw
Consecutive Frames
Decompression
Compression
I-frame P-frame B-frame I-frame
© 2016 NVIDIA Corporation. All rights reserved. NVIDIA, the NVIDIA logo, GeForce, and NVIDIA Pascal are trademarks and/or registered trademarks of NVIDIA Corporation in the U.S. and other countries. Other company and product names may be trademarks of the respective companies with which they are associated.
Jie Jiang [University of Illinois] [NVIDIA], Thomas Fogal [NVIDIA], Cliff Woolley [NVIDIA], Peter Messmer [NVIDIA]
246174 LDAV 2016_Poster_FNL.indd 1 10/20/16 12:02 PM