Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Published on

  • Be the first to comment

  • Be the first to like this


  1. 1. Motivation <ul><li>Video Communication over </li></ul><ul><li>Heterogeneous Networks </li></ul><ul><ul><li>Diverse client devices </li></ul></ul><ul><ul><li>Various network connection </li></ul></ul><ul><ul><li>bandwidths </li></ul></ul><ul><li>Limitations of Scalable Video </li></ul><ul><li>Coding Schemes </li></ul><ul><ul><li>Limited layers supported </li></ul></ul><ul><ul><li>No video format changes </li></ul></ul><ul><li>Video Transcoding Provides </li></ul><ul><li>Dynamic Solutions </li></ul><ul><ul><li>Channel bandwidth adaptation </li></ul></ul><ul><ul><li>Video coding format adaptation </li></ul></ul>
  2. 2. Challenges in Video Transcoding <ul><li>Improve Efficiency of Video Transcoding </li></ul><ul><ul><li>Large data volume </li></ul></ul><ul><ul><li>High computational complexity </li></ul></ul><ul><li>Optimize Visual Quality for a Given Bit Rate </li></ul><ul><ul><li>Human vision system (HVS) based video transcoding is desirable </li></ul></ul>
  3. 3. Proposed Solutions <ul><li>Exploit Foveation Property of the HVS in Video Transcoding </li></ul><ul><li>Develop Fast Algorithms for Video Transcoding </li></ul><ul><ul><li>DCT-domain foveation filtering technique </li></ul></ul><ul><ul><li>Fast algorithms for DCT-domain inverse motion compensation </li></ul></ul><ul><ul><ul><li>Local bandwidth constrained DCT-domain inverse motion compensation </li></ul></ul></ul><ul><ul><ul><li>Look-up-table based DCT-domain inverse motion compensation </li></ul></ul></ul>
  4. 4. Foveation <ul><li>The Human Eye Samples Visual Field Non-uniformly </li></ul><ul><ul><li>The highest sampling resolution is at Fovea </li></ul></ul><ul><ul><li>The sampling resolution decreases rapidly as away from Fovea </li></ul></ul><ul><li>Retinal Images are Inherently Non-uniform in Spatial Resolution </li></ul>Eccentricity (deg) Cells per degree Eccentricity (left eye)
  5. 5. Foveation Modelling <ul><li>Foveated Contrast Threshold [Geisler & Perry 98] </li></ul><ul><li>Foveated Cut-off Frequency f c </li></ul><ul><li>Spatial Frequencies Beyond </li></ul><ul><li>the Cut-off Frequency is </li></ul><ul><li>Invisible ( Foveated Image ) </li></ul><ul><li>f : Spatial frequency (cyc/degree) </li></ul><ul><li>e : Retinal eccentricity(degree) </li></ul><ul><li>a : Spatial frequency decay constant </li></ul><ul><li>e 2 : Half-resolution eccentricity </li></ul><ul><li>CT 0 : Minimum contrast threshold </li></ul><ul><li>CT : Contrast threshold </li></ul>Local cut-off frequency (cyc/deg) Pixel position relative to foveation point (unit: pixel) Image size: 512 x 512 Unit of v: image height
  6. 6. Foveated Images Foveation point is marked by ‘X’ JPEG-coded Uniform Image (168KB) JPEG-coded Foveated Image (136KB)
  7. 7. Foveated Contrast Sensitivity Function (FCSF) <ul><li>Foveated Contrast Sensitivity Function (FCSF) </li></ul><ul><li>Shape the Compression Distortion According to FCSF </li></ul>Image size: 512 x 512 Viewing distance: 3 times the image height Normalized contrast sensitivity of human eye Distance from foveation point (unit: pixel)
  8. 8. Video Transcoding Architecture <ul><li>Open-Loop Video Transcoding </li></ul><ul><ul><li>Simple and fast </li></ul></ul><ul><ul><li>Error drift </li></ul></ul>Transcoding Error Propagation
  9. 9. Drift Free Video Transcoders <ul><li>Cascaded Pixel Domain Video Transcoding </li></ul><ul><ul><li>Low efficiency </li></ul></ul><ul><ul><li>Long delay </li></ul></ul><ul><li>Fast Pixel Domain Video Transcoding </li></ul><ul><ul><li>Save motion estimation, one frame memory and one IDCT operation </li></ul></ul><ul><li>Fast DCT-Domain Video Transcoding </li></ul><ul><ul><li>No IDCT-DCT operations; Lower data volume </li></ul></ul><ul><ul><li>DCT-domain inverse motion compensation is complex (Research topic) </li></ul></ul>Fast Pixel Domain Video Transcoder Fast DCT Domain Video Transcoder
  10. 10. Foveation Embedded DCT Domain Video Transcoding
  11. 11. Foveation Filtering <ul><li>Pixel Domain Foveation Filtering Technique [Lee, 99] </li></ul><ul><ul><li>High computational complexity </li></ul></ul>
  12. 12. DCT-Domain Foveation Filtering <ul><li>DCT-Domain Block Mirror Filtering [Rao, 90] </li></ul><ul><li>Pros </li></ul><ul><ul><li>Significantly simplified </li></ul></ul><ul><ul><li>Combine with inverse quantization </li></ul></ul><ul><ul><li>Easy to parallelize </li></ul></ul><ul><li>Cons </li></ul><ul><ul><li>Blocking artifacts </li></ul></ul>Filter Kernel DCT of f h f H. R. Sheikh, S. Liu, B. L. Evans and A. C. Bovik, “ Real-Time Foveation Techniques for H.263 Video Encoding in Software ”, ICASSP 2001.
  13. 13. Multipoint Video Conferencing H. R. Sheikh, S. Liu, Z. Wang and A. C. Bovik,“ Foveated Multipoint Videoconferencing at Low Bit Rates ”, ICASSP 2002, accepted.
  14. 14. Simulation Results Uniform resolution video at 256 kb/s Foveated video at 256 kb/s Foveation point is at the center of the upper-left quadrant
  15. 15. Foveation Point Selection <ul><li>Interactive Methods </li></ul><ul><ul><li>Mouse, eye tracker </li></ul></ul><ul><ul><li>Reverse channel is assumed </li></ul></ul><ul><ul><li>End to end delay is assumed short enough </li></ul></ul><ul><li>Automatic Methods </li></ul><ul><ul><li>Fixation points analysis (Very challenging) </li></ul></ul><ul><ul><li>Application oriented methods </li></ul></ul><ul><li>DCT-Domain Human Face Detection [Wang & Chang, 97] </li></ul><ul><ul><li>Skin color region segmentation </li></ul></ul><ul><ul><li>Face template constraint </li></ul></ul><ul><ul><li>Spatial Verification </li></ul></ul>