• Save
2D to 3D conversion at CRC: A visual perception approach.
Upcoming SlideShare
Loading in...5
×
 

2D to 3D conversion at CRC: A visual perception approach.

on

  • 602 views

Invited panel presentation at the First Toronto International Stereoscopic 3D Conference

Invited panel presentation at the First Toronto International Stereoscopic 3D Conference

Statistics

Views

Total Views
602
Views on SlideShare
597
Embed Views
5

Actions

Likes
0
Downloads
0
Comments
0

2 Embeds 5

http://www.linkedin.com 4
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

2D to 3D conversion at CRC: A visual perception approach. 2D to 3D conversion at CRC: A visual perception approach. Presentation Transcript

  • 2D to 3D conversion at CRCA visual perception approach Carlos Vazquez and Wa James Tam 3D-TV Research Advanced Video Systems
  • 2D to S3D video conversion Why? Where/When? How? •Many S3D cinema screens available •Post-production: For cinema and •Fully manual •Introduction of 3D-enabled TV sets TV production •Semi-automatic •Large amount of 2D content •Broadcasting: For live content and •Automatic off-line available legacy material •Automatic in real-time •Limited content in original •TV set, STB: Legacy content from stereoscopic format viewer library or 2D program •Still difficult to capture stereoscopic contentJune 4, 2012 Advanced Video Systems -- 3D-TV Research 2
  • Real-time vs. off-line conversion On-line 2D-to-3D conversion  Fully automatic  Artefacts are not removed  Speed of conversion is critical  Non-veridical depth information frequently used Off-line 2D-to-3D conversion  Automatic, semi-automatic or manual  Automatically generated content can be corrected  Normally human-assisted (post production)  High quality and high cost June 4, 2012 Advanced Video Systems -- 3D-TV Research 3
  • 2D to S3D conversion: General framework Strategy: Define depth structure and render new virtual view Input: 2D video sequence Output: Stereoscopic/multi-view video sequence June 4, 2012 Advanced Video Systems -- 3D-TV Research 4
  • How to get the depth Manually assignment of • Human interpretation of the scene depth by operator analysis of translates into depth information the scene Extracting depth from • Depth from static images monocular cues in images • Depth from motion related cues Assigning of depth from • Apply a know depth model automatic analysis of the • Use surrogates for the depth scene informationJune 4, 2012 Advanced Video Systems -- 3D-TV Research 5
  • Automatic generation of depth information Two approaches are used:  Extract depth information from pictorial depth cues in images  Assign depth based on general assumptions about scenes Pictorial depth cues:  Size, texture density, occlusion, atmospheric haze, shadows  Perspective  Focus/defocus General assumptions  Scene depth models Single image with several pictorial  Surrogate depth based on colour and depth cues shadings June 4, 2012 Advanced Video Systems -- 3D-TV Research 6
  • Visual perception: An active process Content dependency:  Aoccdrnig to rscheearch at an Elingsh uinervtisy, Past experience help in interpretation of visual content it deosnt mttaer in waht oredr the ltteers in a Auto-correction: wrod are, olny taht the frist and lsat ltteres are  Fill-in missing information or minimize conflicting information at the rghit pcleas. The rset can be a toatl mses and you can sitll raed it wouthit a porbelm. Tihs is bcuseae we do not raed ervey lteter by ilstef, but the wrod as a wlohe. http://kybele.psych.cornell.edu/~edelman/ June 4, 2012 Advanced Video Systems -- 3D-TV Research 8
  • Visual perception: Depth filling June 4, 2012 Advanced Video Systems -- 3D-TV Research 9
  • Surrogate depth mapsMotivation Shadows offer relative depth inside objects Small image features (rain drops, snow flakes, dust) have depth Good separation between objects in the scene Green and blue go back (Trees and sky) Red goes to the front (Skin tones) June 4, 2012 Advanced Video Systems -- 3D-TV Research 10
  • CRC-i3D: Real-time 2D to 3D video conversion Surrogate Depth Maps:  Single Cue – using the Chroma component of the images.  Minimal adjustments to original depth map Rendering:  Real-Time and no buffering required.  Avoids disocclusion by smoothing depth map.  Manual control of depth to ensure comfort according to viewing conditions. Advantages:  Very low processing requirements.  No storage requirements.  This all translates into less processing, heat and power requirements. June 4, 2012 Advanced Video Systems -- 3D-TV Research 11
  • Off-line conversion: depth-based approach Depth generation Depth map editing New image generation Generate Select Edit depth Render Fill-in2D depth regions values image holes S3D Generate initial depth: Automatic operation Edit depth map: Manual or semi-automatic  Select objects: Semi-automatic operation  Change depth of objects: Semi-automatic Render new view: Automatic operation Fill-in holes: Automatic operation with human supervision. June 4, 2012 Advanced Video Systems -- 3D-TV Research 12
  • Depth-based approach: Main characteristics Intermediate depth master: Can be used to render the stereoscopic content for different targets Depth relations are easy to evaluate Depth is continuous: Slow variations in depth are easily represented Depth information is represented as an image: Easy to store and transmit June 4, 2012 Advanced Video Systems -- 3D-TV Research 13
  • CRC-DMEG: Depth editing toolJune 4, 2012 Advanced Video Systems -- 3D-TV Research 14
  • CRC-DMEG: Depth editing tool Simple histogram equalization of green channelJune 4, 2012 Advanced Video Systems -- 3D-TV Research 15
  • June 4, 2012 Advanced Video Systems -- 3D-TV Research 16