Your SlideShare is downloading. ×
0
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
3DTV - Past, Present and Future
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

3DTV - Past, Present and Future

9,561

Published on

Slides of my seminar at Digitale Telenett 2011, Trondheim, Norway, January 4th, 2011.

Slides of my seminar at Digitale Telenett 2011, Trondheim, Norway, January 4th, 2011.

Published in: Education
2 Comments
18 Likes
Statistics
Notes
No Downloads
Views
Total Views
9,561
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
2
Likes
18
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • http://www.seereal.com/en/holography/papers/2010%20SPIE%202010%20SPIE%20Defense%20Security%20Sensing%20-%20DSS10-7690A-10%20-%20for%20web.pdf\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • [Mobile3DTV D5.1]\n
  • [\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • BBC News - Quality warnings issued over 3DTV\n""'We don't think it's right to confuse consumers this early on with second-rate conversion technology,' Fabrice Estornel, product manager at Panasonic TV, told website Home Cinema Choice (HCC)."\nhttp://www.bbc.co.uk/news/10446419\n\nMichael Bay And James Cameron Skeptical Of 3D Conversions: “The Jury Is Out” – Deadline.com\n"In any event, the 3D experience doesn’t at all feel natural, much less “immersive.”"\nhttp://www.deadline.com/2010/03/michael-bay-james-cameron-skeptical-of-3d-conversions-the-jury-is-out/\n\nMichael Bay And James Cameron Skeptical Of 3D Conversions: “The Jury Is Out” – Deadline.com\n"I was able to see 2D footage that was converted to 3D by one of these companies mentioned in the article. Long story short, it looked awful and cheesy. While yes, the images did seem to pop off of the screen, it still looked flat and lacked the depth that AVATAR has."\nhttp://www.deadline.com/2010/03/michael-bay-james-cameron-skeptical-of-3d-conversions-the-jury-is-out/\n\n3-D movies like Monsters vs. Aliens hurt your eyes. They always have, and they always will. - By Daniel Engber - Slate Magazine\n"(A Katzenbergian mantra: "Making your customers sick is not a recipe for success.")"\nhttp://www.slate.com/id/2215265/\n\n3-D movies like Monsters vs. Aliens hurt your eyes. They always have, and they always will. - By Daniel Engber - Slate Magazine\n"As much as it pains me to say this—I love 3-D, I really do—these films are unpleasant to watch."\nhttp://www.slate.com/id/2215265/\n\n3-D movies like Monsters vs. Aliens hurt your eyes. They always have, and they always will. - By Daniel Engber - Slate Magazine\n"Did the RealD projection give me a headache or was it the movie's lamebrained script?"\nhttp://www.slate.com/id/2215265/pagenum/2\n\n3D Sickness: 3D movies make me sick...literally!\n"I don't get motion sickness from a car or sea yet 3D movies tent to give me headache."\nhttp://www.squidoo.com/3dsickness\n\nAvatar 3D glasses of sub-standard quality? « Fenil and Bollywood\n"“The 3D glasses offered to me ruined my movie-watching experience to say the least. They were uncomfortable to wear as they only had a one-size-fits-all."\nhttp://fenilandbollywood.wordpress.com/2009/12/24/avatar-3d-glasses-of-bad-sub-standard-quality/\n
  • Why is the 3-D so bad in Clash of the Titans? - By Daniel Engber - Slate Magazine\n"The film "redefines 3-D but in the wrong way," reports the St. Petersburg Times"\nhttp://www.slate.com/id/2249527\n\nWhy is the 3-D so bad in Clash of the Titans? - By Daniel Engber - Slate Magazine\n"The film "redefines 3-D but in the wrong way," reports the St. Petersburg Times"\nhttp://www.slate.com/id/2249527\n\nToday3D Forum\n"Earlier this week James Cameron warned against such 2D to 3D conversions, saying specifically of the Clash of the Titans conversion that “If you want to make a movie in 3-D, make the movie in 3-D.”"\nhttp://www.today3d.com/2010/03/clash-of-titans-2d-to-3d-conversion-is.html\n\nMichael Bay And James Cameron Skeptical Of 3D Conversions: “The Jury Is Out” – Deadline.com\n"The 3D trend is annoying…what’s so bad about a really beautiful 2D composition? Even the best 3D still darkens the picture and muddies the color ever so slightly"\nhttp://www.deadline.com/2010/03/michael-bay-james-cameron-skeptical-of-3d-conversions-the-jury-is-out/\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript

    • 1. 1 3DTVPast, Present and Future Touradj Ebrahimi Professor II at NTNU/Q2S Professor at EPFL Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 2. 2 Outline• Introduction – Motivation – A bit of History• 3D perception – Human visual system – Depth perception• 3D processing – Creation – Representation – Coding – Visualization• 3D quality• Future of 3DTV• Conclusion Indicates that the anaglyph glasses should be used to view the content of the slides Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 3. 3INTRODUCTION Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 4. 4 Motivation• Evolution of visual content towards greater realism – Black and white – Color – High resolution – 3DTV• 3 major factors for success – Technology: Ability to capture, process and show 3D – Content: Availability of interesting 3D content – Quality: Attractive to consumers Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 5. 5 History of 3D• 1840: Invention of stereoscopy and stereoscope by C. Wheatstone• 1890: First patent for 3D motion pictures using stereoscope• 1915: First 3D footage in cinema using anaglyph glasses• 1922: Invention of „Teleview“ a shutter based technique• 1936: First demonstration of polarization based projection• 1952: Golden era of 3D movies due to invention of television• 1961: Single film solution „Space-Vision 3D“ using polarization• 1980: IMAX 70mm projectors for non- fiction short films• 2003: First full length 3D feature film for IMAX screens by J. Cameron• 2004: Animation „Polar Express“ makes 14 times more revenue in 3D than 2D Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 6. 6 Example of most recent 3D successes• Movies – Beowulf (2007) – Avatar (2009)• Music – U2 3D (2008) – In Concert 3D (2009)• Documentary – Biodiversity (2009)• Sports – NBA All Star Game (2009) – Six Nations Cup (2010) – FIFA World Cup (2010)• Games – 19 PS3 titles in 2010• 3D Blu-ray Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 7. 73D Perception Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 8. 8 3D perception• Human visual system uses several depth cues to build a mental model of a perceived 3D scene – Oculomotor cues – Monocular cues – Binocular cues• Understanding the human visual perception of Text 3D is an important aspect to develop high quality 3D systems and services Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 9. 9 Visual perception• Eye – Optical part of the visual perception – Amount of light is controlled by iris – Image is brought into focus by lens – Retinal cells capture information• Visual pathways – Carry visual information from the retina to the brain• Visual cortex – Primary visual cortex responds to low level visual information such as frequencies, color and direction – Dorsal and vental streams are dealing with motions and objects Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 10. 10 Eye movements• Achieved by muscles attached to the eye itself and the lens• Accommodation – Change to optical parameter of the lens to bring an object into focus• Vergence – Movement of the eyes to opposite directions to gaze at an object• Version – Movement of the eyes to same direction to pursue movements• Saccadic – Rapid movement of the eyes to scan the scene Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 11. 11 Stereopsis• Human eyes are separated horizontally by approx. 6.3 cm• Existence of different retinal images leads to binocular disparity• Binocular disparity provides cues about the relative depth of objects and their environment• Very effective for large disparities at close distances Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 12. 12 Depth cuesDigitale telenett 2011January 4th, 2010, Trondheim, Norway
    • 13. 13 Oculomotor depth cues• Accommodation – Change of the focal length of the lens – Quite a weak depth cue and only effective for distances < 2 m• Convergence – Rotation of the eyes towards each other for closer objects – Quite a weak depth cue and only effective for distances < 10 m• Myosis – Size of the pupil determines both amount of light and depth of field (DOF) – Very weak depth cue for short distances Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 14. 14 Monocular depth cues• Mainly experiential and learned over time• Shadow• Illumination and shadow• Relative sizes differences• Motion parallax• Aerial perspective• Linear perspective• Interposition• Texture gradient• Intensity gradient• ... Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 15. 15 Monocular depth cuesDigitale telenett 2011January 4th, 2010, Trondheim, Norway
    • 16. 16 Binocular depth cues• Two eyes observe scene from two slightly different angles• Most important depth cue for medium viewing distances• Basic idea behind any stereoscopic display technology• Around 5% of the population have difficulties with binocular depth Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 17. 17 Motion parallax• Motion cues are created when the viewer moves his eyes or head• Relative object motion around a fixation point serves as depth cue• Very important depth cue for a large range of scene depths Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 18. 18 Depth perception layersDigitale telenett 2011January 4th, 2010, Trondheim, Norway
    • 19. 193D Processing Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 20. 20 3D processing chainDigitale telenett 2011January 4th, 2010, Trondheim, Norway
    • 21. 21 3D creation• Computer generated imagery (CGI)• Stereoscopic cameras• Video + depth sensor• 2D to 3D conversion Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 22. 22 Stereoscopic camera• Matching cameras• Matching photography – White balance – Sensitivity – Shutter speed – Aperture• Matching optics – Focal length• Matching geometry – Distance – Angle• Synchronization Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 23. 23 Camera distance• Single most important parameter in stereoscopy• Regulates strength of 3D effect and object size• Optimal camera distance depends on many factors – Near and far distance – Focal length – Maximum disparity Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 24. 24 Camera distanceDigitale telenett 2011January 4th, 2010, Trondheim, Norway
    • 25. 25 Camera convergence• Second most important parameter in stereoscopy• Parallel camera setup – All objects are in front of screen – Depth position can be set later• Converged camera setup – Objects partially behind the screen – Depth position fixed to objects where optical axes converge – May lead to keystone effect Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 26. 26 Camera convergenceDigitale telenett 2011January 4th, 2010, Trondheim, Norway
    • 27. 27 2D to 3D conversion• Basic idea is to convert available 2D content to 3D• Exploit monocular depth cues to generate 3D content• Automatic conversion is difficult and may cause artifacts• Semiautomatic conversion based on computer vision technologies for best tradeoff between efficiency and quality – Structure from Motion – Monocular Depth Estimation Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 28. 28 3D post-processing• Geometric alignment – Horizontal and vertical alignment of cameras may not be perfect – Align images to avoid vertical disparities and adjust depth position• Color adjustment – White balance and exposure of individual cameras may differ – Leads to color and brightness variations between stereo images – Correction using histogram matching techniques Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 29. 29 3D scene representation• Stereoscopic – Left-Right – Above-Below – Interlaced• Image + Depth – Synchronized 2D and depth image or video• Multiview – Several synchronized 2D video streams – Also possible to include depth for each view Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 30. 30 3D compression standards• Simulcast – Possible with any video coding standard• Stereoscopic – MPEG-4/AVC: Stereo SEI message and different L/R packings• Image+depth – MPEG-C Part A: Auxiliary data representation for depth maps• Multiview video coding (MVC) – H.264/MVC: Extension of H. 264/AVC to multiview video• Multiview+depth – 3D Video Coding (3DVC) Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 31. 31 Asymmetric coding• Individual views are coded independently• Each view is coded with a different quality• According to binocular suppression theory the perceived quality is dominated by the higher quality view• Any video coding standard can be used Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 32. 32 3D content displays (devices)• Head Mounted Displays (HMD)• Monitors – Anaglyph – Polarization – Shutter – Autostereoscopic• Projectors – Anaglyph – Polarization – Shutter Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 33. 33 3D content display (technologies)• Stereograms• Passive anaglyph• Active shutter• Passive polarization – Linear – Circular• Autostereoscopic – Single view – Multi view Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 34. 34 Autostereoscopic displays• Requires no glasses• Sends the left and right views directly to the eyes• Lenticular lens – Tiny cylindrical plastic lenses attached to the front of the screen – Does not support 2D mode• Parallax barrier – Fine grating of liquid crystals placed in front of the screen – Supports 2D/3D mode by switching parallax barrier off/on Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 35. 35 Sweet spot• Correct 3D perception depends largely on the viewers position and the used display technology• Wrong distance to the screen leads to crosstalk between the left and the right images• Wrong horizontal position leads to inverted images and causes headache Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 36. 36 Single vs. multi view displays• Single view displays provide the same view for each position• Multi view displays provide different views depending on position Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 37. 373D Quality Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 38. 38 Quality of Experience is the Key• "As much as it pains me to say this - I love 3-D, I really do - these films are unpleasant to watch.“• "In any event, the 3D experience doesn’t at all feel natural, much less immersive.”• "I dont get motion sickness from a car or sea yet 3D movies tent to give me headache.”• "The 3D trend is annoying…what’s so bad about a really beautiful 2D composition? Even the best 3D still darkens the picture and muddies the color ever so slightly“• "The 3D glasses offered to me ruined my movie-watching experience to say the least. They were uncomfortable to wear as they only had a one-size-fits-all.” Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 39. 39 Good vs. Bad 3D Quality• Avatar (2009) • Clash of the Titans (2010) – Combination of 3D shooting, – Shot completely in 2D and motion capture and and CGI converted to 3D in 8 weeks – Budget of 237 million USD – Budget of 125 million USD – "It was an absolute marvel and I – "The film redefines 3-D but in the am left in awe after seeing it.“ wrong way.” Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 40. 40 3D Quality• Improved quality of experience (QoE) is an important aspect for the success of any new technology such as 3DTV• 2D image and video quality assessment is relatively mature• Emerging 3D video quality assessment needs to consider additional aspects such as depth perception, immersion and presence Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 41. 41 3D QualityNintendo WarnsParents Not To LetChildren Under 6Play 3DSDecember 29th, 2010 Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 42. 42 3D processing chain• Capture – Special care needs to be taken when positioning cameras or selecting rendering parameters – Unnatural correspondences are source of many 3D artifacts• Conversion – Various representations for 3D video exist (e.g. multiview and image+depth) – Converting between them may introduce different artifacts Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 43. 43 3D processing chain• Coding – Coding schemes exploit spatial, temporal and inter view redundancies – May lead to various 2D and 3D artifacts• Transmission – Digital wireless transmission are subject to packet losses – Resilience and error concealment algorithms may introduce artifacts Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 44. 44 3D processing chain• Visualization – Various approaches exist for 3D scene visualization with different degree of scene approximation – Each 3D display technology may cause specific artifacts and the amount of artifacts is largely scene dependent Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 45. 45 2D artifacts• Not specific to 3D video but also present in 2D video• Mainly degrade the 2D quality but may also affect depth perception• Examples include blocking, ringing, blurring, color bleeding, jerkiness Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 46. 46 3D artifacts• Distortions that lead to a perceptual difference between the real 3D structure of a scene and its representation• May have different effects on the depth perception from a feeling of unnaturalness to visual discomfort and headache• Includes keystone distortion, puppet theater effect, crosstalk, cardboard effect, shear distortion, disocclusions Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 47. 47 A key challenge in 3D• Subjective and objective quality evaluation protocols and metrics that measure: – Artifacts of 3D processing chain – Naturalness – Sense of presence – Fatigue – Eye strain – Headache – … Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 48. 48 Holographic 3DTV• 3DTV with and without glasses are milestones towards holographic TV Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 49. 49 Challenges in Holographic 3DTV• Efficient and practical capture and display• Compression, transmission and storage• Conversion of legacy 2D and even multi- view content• Applications and viable business models Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 50. 50CONCLUSION Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 51. 51 Conclusion• 3DTV is among one of the emerging multimedia trends• Success will largely depend on improved quality of experience• Interesting content and good 3D quality provides novel experience to users (e.g. Avatar)• Bad 3D quality may limit the acceptance of 3D technologies (e.g. Clash of the Titans)• Subjective and objective quality assessment of 2D image and video quite mature• Methodologies and metrics need to be adopted for 3D considering the special characteristics of 3D perception• Huge opportunities in research, technology, art, and business Digitale telenett 2011 January 4th, 2010, Trondheim, Norway
    • 52. 52 Thanks for your attentionDigitale telenett 2011January 4th, 2010, Trondheim, Norway
    • 53. 53 Acknowledgements• Many of the slides presented in this seminar were collected/produced diligently by Dr. Lutz Goldmann.• Great efforts by Martin Rerabek for preparation of appropriate 3D anaglyph glasses are appreciated and acknowledged. Digitale telenett 2011 January 4th, 2010, Trondheim, Norway

    ×