Motivation <ul><li>Video Communication over  </li></ul><ul><li>Heterogeneous Networks </li></ul><ul><ul><li>Diverse client...
Challenges in Video Transcoding <ul><li>Improve Efficiency of Video Transcoding </li></ul><ul><ul><li>Large data volume </...
Proposed Solutions <ul><li>Exploit Foveation Property of the HVS in Video Transcoding </li></ul><ul><li>Develop Fast Algor...
Foveation <ul><li>The Human Eye Samples Visual Field Non-uniformly </li></ul><ul><ul><li>The   highest sampling resolution...
Foveation Modelling <ul><li>Foveated Contrast Threshold  [Geisler & Perry 98] </li></ul><ul><li>Foveated Cut-off Frequency...
Foveated Images Foveation point is marked by ‘X’ JPEG-coded Uniform Image (168KB) JPEG-coded Foveated Image (136KB)
Foveated Contrast Sensitivity Function (FCSF) <ul><li>Foveated Contrast Sensitivity Function (FCSF) </li></ul><ul><li>Shap...
Video Transcoding Architecture <ul><li>Open-Loop Video Transcoding </li></ul><ul><ul><li>Simple and fast </li></ul></ul><u...
Drift Free Video Transcoders <ul><li>Cascaded Pixel Domain Video Transcoding </li></ul><ul><ul><li>Low efficiency  </li></...
Foveation Embedded DCT Domain Video Transcoding
Foveation Filtering <ul><li>Pixel Domain Foveation Filtering Technique  [Lee, 99] </li></ul><ul><ul><li>High computational...
DCT-Domain Foveation Filtering <ul><li>DCT-Domain Block Mirror Filtering  [Rao, 90] </li></ul><ul><li>Pros </li></ul><ul><...
Multipoint Video Conferencing H. R. Sheikh, S. Liu, Z. Wang and A. C. Bovik,“ Foveated Multipoint Videoconferencing at Low...
Simulation Results Uniform resolution video at 256 kb/s  Foveated video at 256 kb/s  Foveation point is at the center of t...
Foveation Point Selection <ul><li>Interactive Methods </li></ul><ul><ul><li>Mouse, eye tracker </li></ul></ul><ul><ul><li>...
Upcoming SlideShare
Loading in …5
×

Slides

345 views
323 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
345
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • The frequency domain equalizer is just a complex division per subchannel Channel shortening equalizer is an 20-30 tap FIR filter My focus is on channel shortening
  • Slides

    1. 1. Motivation <ul><li>Video Communication over </li></ul><ul><li>Heterogeneous Networks </li></ul><ul><ul><li>Diverse client devices </li></ul></ul><ul><ul><li>Various network connection </li></ul></ul><ul><ul><li>bandwidths </li></ul></ul><ul><li>Limitations of Scalable Video </li></ul><ul><li>Coding Schemes </li></ul><ul><ul><li>Limited layers supported </li></ul></ul><ul><ul><li>No video format changes </li></ul></ul><ul><li>Video Transcoding Provides </li></ul><ul><li>Dynamic Solutions </li></ul><ul><ul><li>Channel bandwidth adaptation </li></ul></ul><ul><ul><li>Video coding format adaptation </li></ul></ul>
    2. 2. Challenges in Video Transcoding <ul><li>Improve Efficiency of Video Transcoding </li></ul><ul><ul><li>Large data volume </li></ul></ul><ul><ul><li>High computational complexity </li></ul></ul><ul><li>Optimize Visual Quality for a Given Bit Rate </li></ul><ul><ul><li>Human vision system (HVS) based video transcoding is desirable </li></ul></ul>
    3. 3. Proposed Solutions <ul><li>Exploit Foveation Property of the HVS in Video Transcoding </li></ul><ul><li>Develop Fast Algorithms for Video Transcoding </li></ul><ul><ul><li>DCT-domain foveation filtering technique </li></ul></ul><ul><ul><li>Fast algorithms for DCT-domain inverse motion compensation </li></ul></ul><ul><ul><ul><li>Local bandwidth constrained DCT-domain inverse motion compensation </li></ul></ul></ul><ul><ul><ul><li>Look-up-table based DCT-domain inverse motion compensation </li></ul></ul></ul>
    4. 4. Foveation <ul><li>The Human Eye Samples Visual Field Non-uniformly </li></ul><ul><ul><li>The highest sampling resolution is at Fovea </li></ul></ul><ul><ul><li>The sampling resolution decreases rapidly as away from Fovea </li></ul></ul><ul><li>Retinal Images are Inherently Non-uniform in Spatial Resolution </li></ul>Eccentricity (deg) Cells per degree Eccentricity (left eye)
    5. 5. Foveation Modelling <ul><li>Foveated Contrast Threshold [Geisler & Perry 98] </li></ul><ul><li>Foveated Cut-off Frequency f c </li></ul><ul><li>Spatial Frequencies Beyond </li></ul><ul><li>the Cut-off Frequency is </li></ul><ul><li>Invisible ( Foveated Image ) </li></ul><ul><li>f : Spatial frequency (cyc/degree) </li></ul><ul><li>e : Retinal eccentricity(degree) </li></ul><ul><li>a : Spatial frequency decay constant </li></ul><ul><li>e 2 : Half-resolution eccentricity </li></ul><ul><li>CT 0 : Minimum contrast threshold </li></ul><ul><li>CT : Contrast threshold </li></ul>Local cut-off frequency (cyc/deg) Pixel position relative to foveation point (unit: pixel) Image size: 512 x 512 Unit of v: image height
    6. 6. Foveated Images Foveation point is marked by ‘X’ JPEG-coded Uniform Image (168KB) JPEG-coded Foveated Image (136KB)
    7. 7. Foveated Contrast Sensitivity Function (FCSF) <ul><li>Foveated Contrast Sensitivity Function (FCSF) </li></ul><ul><li>Shape the Compression Distortion According to FCSF </li></ul>Image size: 512 x 512 Viewing distance: 3 times the image height Normalized contrast sensitivity of human eye Distance from foveation point (unit: pixel)
    8. 8. Video Transcoding Architecture <ul><li>Open-Loop Video Transcoding </li></ul><ul><ul><li>Simple and fast </li></ul></ul><ul><ul><li>Error drift </li></ul></ul>Transcoding Error Propagation
    9. 9. Drift Free Video Transcoders <ul><li>Cascaded Pixel Domain Video Transcoding </li></ul><ul><ul><li>Low efficiency </li></ul></ul><ul><ul><li>Long delay </li></ul></ul><ul><li>Fast Pixel Domain Video Transcoding </li></ul><ul><ul><li>Save motion estimation, one frame memory and one IDCT operation </li></ul></ul><ul><li>Fast DCT-Domain Video Transcoding </li></ul><ul><ul><li>No IDCT-DCT operations; Lower data volume </li></ul></ul><ul><ul><li>DCT-domain inverse motion compensation is complex (Research topic) </li></ul></ul>Fast Pixel Domain Video Transcoder Fast DCT Domain Video Transcoder
    10. 10. Foveation Embedded DCT Domain Video Transcoding
    11. 11. Foveation Filtering <ul><li>Pixel Domain Foveation Filtering Technique [Lee, 99] </li></ul><ul><ul><li>High computational complexity </li></ul></ul>
    12. 12. DCT-Domain Foveation Filtering <ul><li>DCT-Domain Block Mirror Filtering [Rao, 90] </li></ul><ul><li>Pros </li></ul><ul><ul><li>Significantly simplified </li></ul></ul><ul><ul><li>Combine with inverse quantization </li></ul></ul><ul><ul><li>Easy to parallelize </li></ul></ul><ul><li>Cons </li></ul><ul><ul><li>Blocking artifacts </li></ul></ul>Filter Kernel DCT of f h f H. R. Sheikh, S. Liu, B. L. Evans and A. C. Bovik, “ Real-Time Foveation Techniques for H.263 Video Encoding in Software ”, ICASSP 2001.
    13. 13. Multipoint Video Conferencing H. R. Sheikh, S. Liu, Z. Wang and A. C. Bovik,“ Foveated Multipoint Videoconferencing at Low Bit Rates ”, ICASSP 2002, accepted.
    14. 14. Simulation Results Uniform resolution video at 256 kb/s Foveated video at 256 kb/s Foveation point is at the center of the upper-left quadrant
    15. 15. Foveation Point Selection <ul><li>Interactive Methods </li></ul><ul><ul><li>Mouse, eye tracker </li></ul></ul><ul><ul><li>Reverse channel is assumed </li></ul></ul><ul><ul><li>End to end delay is assumed short enough </li></ul></ul><ul><li>Automatic Methods </li></ul><ul><ul><li>Fixation points analysis (Very challenging) </li></ul></ul><ul><ul><li>Application oriented methods </li></ul></ul><ul><li>DCT-Domain Human Face Detection [Wang & Chang, 97] </li></ul><ul><ul><li>Skin color region segmentation </li></ul></ul><ul><ul><li>Face template constraint </li></ul></ul><ul><ul><li>Spatial Verification </li></ul></ul>

    ×