• Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
2,372
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Why is the 3-D so bad in Clash of the Titans? - By Daniel Engber - Slate Magazine\n"The film "redefines 3-D but in the wrong way," reports the St. Petersburg Times"\nhttp://www.slate.com/id/2249527\n\nWhy is the 3-D so bad in Clash of the Titans? - By Daniel Engber - Slate Magazine\n"The film "redefines 3-D but in the wrong way," reports the St. Petersburg Times"\nhttp://www.slate.com/id/2249527\n\nToday3D Forum\n"Earlier this week James Cameron warned against such 2D to 3D conversions, saying specifically of the Clash of the Titans conversion that “If you want to make a movie in 3-D, make the movie in 3-D.”"\nhttp://www.today3d.com/2010/03/clash-of-titans-2d-to-3d-conversion-is.html\n\nMichael Bay And James Cameron Skeptical Of 3D Conversions: “The Jury Is Out” – Deadline.com\n"The 3D trend is annoying…what’s so bad about a really beautiful 2D composition? Even the best 3D still darkens the picture and muddies the color ever so slightly"\nhttp://www.deadline.com/2010/03/michael-bay-james-cameron-skeptical-of-3d-conversions-the-jury-is-out/\n
  • \n
  • \n
  • http://www.seereal.com/en/holography/papers/2010%20SPIE%202010%20SPIE%20Defense%20Security%20Sensing%20-%20DSS10-7690A-10%20-%20for%20web.pdf\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • [Mobile3DTV D5.1]\n
  • [\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Several methods have been proposed for the quality evaluation of 2D [9] and 3D [10] images and videos including single stimulus (SS), double stimulus (DS) and stimulus comparison (SC). Since judging the quality of different 2D and 3D restitution techniques individually may be quite difficult, the SC method seems to be the most suitable method for the subjective test.\n\nSince the IPD (Inter-Pupillary Distance) of all participating subjects has not been measured, the two camera baselines, used to generate narrow and wide subset, are compared each other.\nFifteen subjects (8 male and 7 female) participated in the subjective test experiments. They reported normal or corrected to normal vision according to [10].\n
  • Preference and tie probabilities of the individual pairs considering the overall quality.\n
  • Preference and tie probabilities of the individual pairs considering the perceived depth.\n
  • Preference probabilities of condition A vs. B.\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Transcript

  • 1. 1 Vers une mesurede qualité visuelle 3D pour le multimédia de demain Touradj Ebrahimi Touradj.Ebrahimi@epfl.ch CORESA 24-25 mai 2012, Lille, France
  • 2. 1 Vers une mesureTowards 3Dvisuelle 3D de qualité visual quality assessment for future pour le multimédia de demain multimedia Touradj Ebrahimi Touradj.Ebrahimi@epfl.ch CORESA 24-25 mai 2012, Lille, France
  • 3. 2 Outline• Motivations• 3D perception• 3D processing chain and sources of distortions• 3D quality – Subjective quality assessment – Objective quality assessment• Some last words Anaglyph glasses recommended CORESA 24-25 mai 2012, Lille, France
  • 4. 3INTRODUCTION CORESA 24-25 mai 2012, Lille, France
  • 5. 4 Motivation• Evolution of visual content towards greater realism– Black and white– Color– High resolution– 3DTV• 3 major factors for success– Technology: Ability to capture, process and show 3D– Content: Availability of interesting 3D content– Quality: Attractive to consumers CORESA 24-25 mai 2012, Lille, France
  • 6. 5 History of 3D• 1840: Invention of stereoscopy and stereoscope by C. Wheatstone• 1890: First patent for 3D motion pictures using stereoscope• 1915: First 3D footage in cinema using anaglyph glasses• 1922: Invention of „Teleview“ a shutter based technique• 1936: First demonstration of polarization based projection• 1952: Golden era of 3D movies due to invention of television• 1961: Single film solution „Space-Vision 3D“ using polarization• 1980: IMAX 70mm projectors for non- fiction short films• 2003: First full length 3D feature film for IMAX screens by J. Cameron• 2004: Animation „Polar Express“ makes 14 times more revenue in 3D than 2D CORESA 24-25 mai 2012, Lille, France
  • 7. 6 Example of some recent 3D successes• Movies – Beowulf (2007) – Avatar (2009)• Music – U2 3D (2008) – In Concert 3D (2009)• Documentary – Biodiversity (2009)• Sports – NBA All Star Game (2009) – Six Nations Cup (2010) – FIFA World Cup (2010)• Games – 19 PS3 titles in 2010• 3D Blu-ray CORESA 24-25 mai 2012, Lille, France
  • 8. 7 Good vs. Bad 3D Quality• Avatar (2009) • Clash of the Titans (2010) – Combination of 3D shooting, – Shot completely in 2D and motion capture and and CGI converted to 3D in 8 weeks – Budget of 237 million USD – Budget of 125 million USD – "It was an absolute marvel and I – "The film redefines 3-D but in the am left in awe after seeing it.“ wrong way.” CORESA 24-25 mai 2012, Lille, France
  • 9. 83D Perception CORESA 24-25 mai 2012, Lille, France
  • 10. 9 3D perception• Human visual system uses several depth cues to build a mental model of a perceived 3D scene• Understanding the human visual perception of 3D is an important aspect to develop 3D quality metrics and to come up with efficient 3D systems and services CORESA 24-25 mai 2012, Lille, France
  • 11. 10 Depth cuesCORESA24-25 mai 2012, Lille, France
  • 12. 11 Oculomotor depth cues• Accommodation – Change of the focal length of the lens – Quite a weak depth cue and only effective for distances < 2 m• Convergence – Rotation of the eyes towards each other for closer objects – Quite a weak depth cue and only effective for distances < 10 m• Myosis – Size of the pupil determines both amount of light and depth of field (DOF) – Very weak depth cue for short distances CORESA 24-25 mai 2012, Lille, France
  • 13. 12 Monocular depth cues• Mainly experiential and learned over time• Shadow• Illuminations• Relative sizes differences• Aerial perspective• Linear perspective• Interposition• Texture gradient• Intensity gradient• ... CORESA 24-25 mai 2012, Lille, France
  • 14. 13 Stereopsis• Human eyes are separated horizontally by approx. 6.3 cm• Existence of different retinal images leads to binocular disparity CORESA 24-25 mai 2012, Lille, France
  • 15. 14 Binocular depth cues• Two eyes observe scene from two slightly different angles• Most important depth cue for medium viewing distances• Basic idea behind any stereoscopic display technology• 5%-10% of population have difficulties with binocular depth CORESA 24-25 mai 2012, Lille, France
  • 16. 15 Motion parallax• Motion cues are created when the viewer moves his eyes or head• Relative object motion around a fixation point serves as depth cue• Very important depth cue for a large range of scene depths CORESA 24-25 mai 2012, Lille, France
  • 17. 16 Depth perception layersCORESA24-25 mai 2012, Lille, France
  • 18. 173D Processing chain and sourcesof distortions CORESA 24-25 mai 2012, Lille, France
  • 19. 18 3D processing chainCORESA24-25 mai 2012, Lille, France
  • 20. 19 3D creation• Computer generated imagery (CGI)• Stereoscopic cameras• Video + depth sensor• 2D to 3D conversion CORESA 24-25 mai 2012, Lille, France
  • 21. 20 Stereoscopic camera• Matching cameras• Matching photography – White balance – Sensitivity – Shutter speed – Aperture• Matching optics – Focal length• Matching geometry – Distance – Angle• Synchronization CORESA 24-25 mai 2012, Lille, France
  • 22. 21 Camera distance• Single most important parameter in stereoscopy• Regulates strength of 3D effect and object size• Optimal camera distance depends on many factors – Near and far distance – Focal length – Maximum disparity CORESA 24-25 mai 2012, Lille, France
  • 23. 22 Camera distanceCORESA24-25 mai 2012, Lille, France
  • 24. 23 Camera convergence• Second most important parameter in stereoscopy• Parallel camera setup – All objects are in front of screen – Depth position can be set later• Converged camera setup – Objects partially behind the screen – Depth position fixed to objects where optical axes converge – May lead to keystone effect CORESA 24-25 mai 2012, Lille, France
  • 25. 24 Camera convergenceCORESA24-25 mai 2012, Lille, France
  • 26. 25 3D post-processing• Geometric alignment – Horizontal and vertical alignment of cameras may not be perfect – Align images to avoid vertical disparities and adjust depth position• Color adjustment – White balance and exposure of individual cameras may differ – Leads to color and brightness variations between stereo images – Correction using histogram matching techniques CORESA 24-25 mai 2012, Lille, France
  • 27. 26 3D scene representation• Stereoscopic – Left-Right – Above-Below – Interlaced• Image + Depth – Synchronized 2D and depth image or video• Multiview – Several synchronized 2D video streams – Also possible to include depth for each view CORESA 24-25 mai 2012, Lille, France
  • 28. 27 3D compression standards• Simulcast – Possible with any video coding standard• Stereoscopic – MPEG-4/AVC: Stereo SEI message and different L/R packings• Image+depth – MPEG-C Part A: Auxiliary data representation for depth maps• Multiview video coding (MVC) – H.264/MVC: Extension of H. 264/AVC to multiview video• Multiview+depth – 3D Video Coding (3DVC) CORESA 24-25 mai 2012, Lille, France
  • 29. 28 Asymmetric coding• Individual views are coded independently• Each view is coded with a different quality• According to binocular suppression theory the perceived quality is dominated by the higher quality view• Any video coding standard can be used CORESA 24-25 mai 2012, Lille, France
  • 30. 29 3D content displays (devices)• Head Mounted Displays (HMD)• Monitors – Anaglyph – Polarization – Shutter – Autostereoscopic• Projectors – Anaglyph – Polarization – Shutter CORESA 24-25 mai 2012, Lille, France
  • 31. 30 3D content display (technologies)• Stereograms• Passive anaglyph• Active shutter• Passive polarization – Linear – Circular• Autostereoscopic – Single view – Multi view CORESA 24-25 mai 2012, Lille, France
  • 32. 31 Autostereoscopic displays• Requires no glasses• Sends the left and right views directly to the eyes• Lenticular lens – Tiny cylindrical plastic lenses attached to the front of the screen – Does not support 2D mode• Parallax barrier – Fine grating of liquid crystals placed in front of the screen – Supports 2D/3D mode by switching parallax barrier off/on CORESA 24-25 mai 2012, Lille, France
  • 33. 32 Sweet spot and crosstalk• Correct 3D perception depends largely on the viewers position and the used display technology• Wrong distance to the screen leads to crosstalk between the left and the right images• Wrong horizontal position leads to inverted images and causes headache CORESA 24-25 mai 2012, Lille, France
  • 34. 333D Quality CORESA 24-25 mai 2012, Lille, France
  • 35. 34 3D Quality• Improved quality of experience (QoE) is an important aspect for the success of any new technology such as 3DTV• Emerging 3D video quality assessment needs to consider additional aspects such as depth perception, immersion, naturalness, and presence CORESA 24-25 mai 2012, Lille, France
  • 36. 35 3D QualityNintendo WarnsParents Not To LetChildren Under 6Play 3DSDecember 29th, 2010 CORESA 24-25 mai 2012, Lille, France
  • 37. 36 2D artifacts• Not specific to 3D video but also present in 2D video• Mainly degrade the 2D quality but may also affect depth perception• Examples include blocking, ringing, blurring, color bleeding, jerkiness CORESA 24-25 mai 2012, Lille, France
  • 38. 37 3D artifacts• Distortions that lead to a perceptual difference between the real 3D structure of a scene and its representation• May have different effects on the depth perception from a feeling of unnaturalness to visual discomfort and headache• Includes keystone distortion, puppet theater effect, crosstalk, cardboard effect, shear distortion, disocclusions CORESA 24-25 mai 2012, Lille, France
  • 39. 38 A key challenge in 3D• Subjective and objective quality evaluation protocols and metrics that measure: – Artifacts of 3D processing chain – Naturalness – Sense of presence – Fatigue – Eye strain – Headache – … CORESA 24-25 mai 2012, Lille, France
  • 40. 393D subjective evaluations CORESA 24-25 mai 2012, Lille, France
  • 41. 40 Evaluation methodology in MPEG 3DVC• ISO subjective test campaign for the evaluation 1) of proponent technologies Registration and for the 3D video coding Screening standard (3DV) during the month of October 2011 2) Training• Coordinated by QUALINET COST Action 3) Test• Network of 12 labs all around the world• Circa 120 subjects in each test laboratory taking part, during 1 week CORESA 24-25 mai 2012, Lille, France
  • 42. 41 Evaluation methodology in MPEG 3DVC• Double Stimulus Impairment Scale (DSIS) evaluation “Rate the quality of each stimulus B, keeping in mind that of stimulus A”• 11-grade numerical categorical scale – 10: highest quality (i.e. test sequence indistinguishable from the reference) – 0: lowest quality CORESA 24-25 mai 2012, Lille, France
  • 43. Scatter plots (EPFL – UBC) 42 Autostereo dataCORESA24-25 mai 2012, Lille, France
  • 44. Scatter plots (NTNU – Acreo) 43 Stereo dataCORESA24-25 mai 2012, Lille, France
  • 45. 44 Paired comparison subjective evaluation• Evaluation of motion parallax versus binocular based mobile 3D display CORESA 24-25 mai 2012, Lille, France
  • 46. 45 Wiggle stereoscopy• Rapid change between two views provides motion parallax cues• Automatic alternation leads to annoyingly jerky image• Quality of wiggle stereoscopy can be improved in two ways – Use multiple views to achieve smoother transition – Interactive alternation between images depending on the viewing position CORESA 24-25 mai 2012, Lille, France
  • 47. Video 46CORESA24-25 mai 2012, Lille, France
  • 48. Video 46CORESA24-25 mai 2012, Lille, France
  • 49. Subjective quality evaluation 47• Comparison of methods in terms of overall and depth quality• Paired comparison with sequential presentation and 3 rating possibilities (first, equal, second)• Multiview content from the MPEG 3DV dataset (Ballons, Kendo, Mobile)• 7 different test conditions – 2D as reference (2D) – Anaglyph with narrow (ANN) and wide (ANW) camera baseline – Motion Parallax with narrow (MVN) and wide (MVW) camera baseline – Autostereoscopic with narrow (Sn) and wide (Sw) camera baseline• 15 participants (11 male, 4 female) CORESA 24-25 mai 2012, Lille, France
  • 50. Overall quality 48 (a) Ballons (b) Kendoions from multi- ultiscopic image (c) Mobile (d) Overalleline resulting in image of 21 im- Fig. 5. CORESA and tie probabilities of the individual pairs Preference 24-25 mai 2012, Lille, France in larger motion considering the overall quality.
  • 51. Depth quality 49 (a) Ballons (b) Kendo (a) Overall quali Fig. 7. Preferenc (c) Mobile (d) OverallFig. 6.CORESA and tie probabilities of the individual pairs Preference 24-25 mai 2012, Lille, Franceconsidering the perceived depth. (a) Overall quali
  • 52. 50 Preference probabilities (a) Overall quality (b) Perceived depthFig. 7. Preference probabilities of condition A vs. B. CORESA 24-25 mai 2012, Lille, France
  • 53. (a) Overall quality (b) Perceived depth 51 Probability of preference & Confidence Interval Fig. 7. Preference probabilities of condition A vs. B. (a) Overall quality (b) Perceived depthFig. 8. Preference probability of choosing the given restitu-tion technique among the others. CORESA 24-25 mai 2012, Lille, France
  • 54. 52 Normalized MOS and CI (a) Overall quality (b) Perceived depthFig. 9. MOS/CI of the individual test conditions. CORESA 24-25 mai 2012, Lille, France
  • 55. 53 Subjective 3D quality assessment• Psycho-perceptual approaches – Examine relationship between physical stimuli and sensorial experience – Usually conducted in highly controlled laboratory environment – Judgement of overall quality or predefined quality attributes (e.g. depth)• User-centered approaches – Relate quality evaluation to the use of the system or service – Takes into account typical users (persona) and required system characteristics – Goal-related evaluation in potential usage scenarios CORESA 24-25 mai 2012, Lille, France
  • 56. 54 Psycho-perceptual evaluation• Long history of subjective quality evaluation according to standardized protocols• Methods for 2D video quality are standardized in ITU-R BT. 500-11• Methods differ mainly in – Number of stimuli (single, double) – Type of scale (quality, impairment, comparison) – Granularity of scale (discrete, continuous)• According to ITU-R BT.1438 also recommended for 3D video quality• Assessment predefined quality attributes (overall quality, depth quality, naturalness)• No support for experienced quality factors and ergonomic aspects CORESA 24-25 mai 2012, Lille, France
  • 57. 55Methods overview [MOBILE3DTV D4.2] CORESA 24-25 mai 2012, Lille, France
  • 58. 56 User centered evaluation• Increasing importance to evaluate quality from a user perspective rather than from a system perspective• Constrains the tests to potential users and stimuli and evaluates the acceptability with respect to the task and the context• Identify underlying quality factors beside the overall quality CORESA 24-25 mai 2012, Lille, France
  • 59. Personas and scenarios [Strohmeier2009] 57• Personas (user profiles) – Archetypical user representing the needs, behaviors and goals of a particular group of users – Not real people but fictional characters derived from user requirements• Scenarios (context) – Realistic usage environment CORESA 24-25 mai 2012, Lille, France
  • 60. 58 Discussion• Current status – Most 3D studies are based on psycho-perceptual evaluation methodologies – Perceived 3D quality is difficult to describe using fixed quality attributes – Only a few studies consider ergonomical aspects of 3D video – Common standards for subjective evaluation of 3D quality are missing• Future directions – Assess suitability of established 2D methodologies for 3D quality evaluation considering the fundamental differences between 2D and 3D perception and sources of distortions – User centered evaluation to identify the various quality factors for 3D video – Need for long-term studies which consider visual fatique and motion sickness – Definition of standard protocols for subjective evaluation of 3D quality including display and viewing conditions – Evaluation of systems in realistic usage scenarios and with relevant content CORESA 24-25 mai 2012, Lille, France
  • 61. 59 Objective quality assessment• Subjective tests are time consuming and expensive and can not be user for online quality evaluation• Objective quality metrics are mathematical models that approximate subjective quality evaluation• High correlation between subjective and objective scores is desired• Can be grouped based on the availability of a reference – Full reference (FR) – Reduced reference (RR) – No reference (NR)• Can be categorized according to the approach – Model based (top down) – Feature based (bottom up) CORESA 24-25 mai 2012, Lille, France
  • 62. Objective 3D quality metrics 60• Fairly new research topic with only a few proposed metrics so far• Straightforward way is to apply 2D image quality metrics to individual channels – Works for impairment that affect both channels equally – Does not consider inter channel masking effects• Several factors affect the 3D image quality – Binocular suppression – Inter channel relations – Image content• Most of the proposed metrics are full reference metrics – Not suitable for online scenarios or when the reference is not available CORESA 24-25 mai 2012, Lille, France
  • 63. Left/right image [Campisi2007] 61• Applies common 2D image quality metrics to left and right image• Combines scores using average, main eye or visual acuity approach CORESA 24-25 mai 2012, Lille, France
  • 64. 62 3D Quality ǂ Σ2D• Encode left/right images with JPEG and different QPs (0-100)• Show images with decreasing quality to the subjects• Determine limit of transparency for left, right and stereo image• Compute PSNR of left and right images and average for stereo• Find PSNR which corresponds to the QP limit for each image• Average PSNRs for each image across the individual subjects CORESA 24-25 mai 2012, Lille, France
  • 65. 3D Quality ǂ Σ2D 63CORESA24-25 mai 2012, Lille, France
  • 66. 64 MVC assessment using PSNR as metricSample comparison of simulcast vs inter-view prediction 8 views (640 480), consider rate for all views ~25% bit rate savings over all views Ballroom Race1 40 42 39 41 40 38 39 37 38 PSNR (db)PSNR (db) 36 37 35 36 34 35 33 Simulcast 34 Simulcast 32 MVC 33 MVC 31 32 0 200 400 600 800 1000 1200 1400 1600 1800 0 200 400 600 800 1000 1200 1400 1600 Bitrate (Kb/s) Bitrate (Kb/s) CORESA 24-25 mai 2012, Lille, MPEG 3D Video Coding Activity Vetro: Update on France
  • 67. MVC:assessment by subjective evaluation MVC Subjective Performance 65 • Main finding: MVC achieves comparable quality to simulcast with as little as 25% rate for dependent view 4.50Mean Opinion Score 4.00 3.50 3.00 2.50 2.00 1.50 1.00 al ) t ct ct ct ct ct ct VC c in 5P P P P P P P rig 50 35 25 20 15 10 +A L_ O L_ L_ L_ L_ L_ L_ VC 12 12 12 12 12 12 12 (A t cas ul m Si Base view fixed at 12Mbps Dependent view at varying percentage of base view rate CORESA Vetro: Update on MPEG 3D Video Coding Activity 24-25 mai 2012, Lille, France
  • 68. 66 Performance of PSNR in 3D video quality evaluation Left View Depth 3D Video ViewIntermediate Synthesized Synthesis Estimation Codec Synthesis Right Decoded CORESA 24-25 mai 2012, Lille, France
  • 69. and is irrelevant for the goal of the measure performance anal- ever, they are more clustered for either the decoded view orysis. At the same time, it avoids the risk of data over fit- the average of the decoded and synthesized views, especially 67 Performance of PSNR inting, which may occur when considering non-linear regres- 3D sequence S01. Therefore,evaluation for video quality the PSNR of the decoded viewsion. The linear regression is of the form: and the average PSNR of the decoded and synthesized views MOSp (VQR) = a · VQR + b should have a better correlation with the perceived quality. For some sequences (S03, S05 and S06), a few synthe-Then, the Pearson linear correlation coefficient (PCC) and sized (intermediate) views have a low PSNR value but the cor-the root-mean-square error (RMSE) are computed between responding stereo pairs have a high MOS. These data points a) PSNR of the decoded view, calculated between the de- coded view and the original view b) PSNR of the intermediate view, calculated between the synthesized view at the decoder side and the intermedi- ate view from the original data (if available)MOSp and MOS to estimateview, calculated between theVQR. To es- at the decoder side and the distant from thethe encoder side scatter plot of c) PSNR of the synthesized the accuracy of the synthesized view are significantly synthe- sized view at trend in thetimate monotonicity and consistency, d) Average PSNR of the decoded view andthe intermedi- ate view, computed as the synthesized (intermediate) view. It is known that one pro- the Spearman rank or- the mean value of a) and b) e) Average PSNR of the decoded view and the synthesized view, computed as the mean value of a) and c)der correlation coefficient (SCC) and the outlier ratio (OR), ponent used a different view synthesis algorithm. Our hy-are computed between MOSp and MOS, respectively [13]. pothesis is that those results are from this specific proponent a) b) c) d) e) a) b) c) d) e) Table 2. Accuracy, monotonicity, and consistency indexes of the objective video quality models under consideration. Decoded and Decoded and Decoded and Decoded and Decoded Intermediate Synthesized Decoded Intermediate Synthesized intermediate synthesized intermediate synthesized Pearson linear correlation coefficient Spearman rank order correlation coefficient S01 0.9216 - 0.4489 - 0.8502 0.9452 - 0.4211 - 0.8509 S02 0.9469 - 0.8158 - 0.9406 0.9534 - 0.7958 - 0.9373 S03 0.9366 0.8130 0.8370 0.9199 0.9240 0.9362 0.8504 0.8674 0.9219 0.9266 S04 0.8889 - 0.8476 - 0.8843 0.8372 - 0.8055 - 0.8412 S05 0.9073 0.4592 0.4877 0.8498 0.8309 0.9078 0.5910 0.5831 0.8182 0.8019 S06 0.9558 0.6477 0.6145 0.9235 0.9039 0.9773 0.6370 0.5934 0.9288 0.9049 S07 0.9219 0.1119 0.3788 0.9186 0.8284 0.9158 0.1935 0.4704 0.9278 0.8723 S08 0.8813 0.8972 0.9043 0.9024 0.9044 0.8185 0.8449 0.9006 0.8492 0.8747 Mean 0.9200 0.5858 0.6668 0.9028 0.8834 0.9114 0.6234 0.6797 0.8892 0.8762 Root-mean-square error Outliers ratio S01 0.9476 - 2.1812 - 1.2850 0.0220 - 0.2088 - 0.0659 S02 0.7882 - CORESA - 1.4169 0.8316 0.0000 - 0.0978 - 0.0217 S03 0.7861 1.3062 1.2277 mai 2012, 24-25 0.8795 Lille, France 0.0000 0.8575 0.0312 0.0208 0.0104 0.0104 S04 1.1044 - 1.2790 - 1.1255 0.0217 - 0.0543 - 0.0217
  • 70. 68 Disparity map and cyclopean image [Boev2006]• Compare cyclopean images using perceptual quality metric (SSIM)• Compare disparity maps using absolute difference CORESA 24-25 mai 2012, Lille, France
  • 71. 69 Discussion• Current status – Current 3D quality metrics are rather simple extensions to 2D quality metrics – They do not take into account the interaction between 2D and 3D perception – Mainly full reference metrics not suitable for assessment of 3D creation – Do not consider 3D display characteristics and influence on 3D perception• Future directions – Important to quantify the influence of 3D distortions originating from every step within the whole processing chain – More accurate models for 2D and 3D human visual perception are needed – Consider interaction between monocular and binocular depth cues – Incorporate information about 3D content creation and display – Consider focus of attention and accommodation/convergence rivalry CORESA 24-25 mai 2012, Lille, France
  • 72. 70A FEW LAST WORDS CORESA 24-25 mai 2012, Lille, France
  • 73. 71 Take home message• 3DTV is among one of the emerging multimedia trends• Success will largely depend on improved quality of experience• Interesting content and good 3D quality provides novel experience to users (e.g. Avatar)• Bad 3D quality may limit the acceptance of 3D technologies (e.g. Clash of the Titans)• Subjective and objective quality assessment of 3D image and video not mature yet• Methodologies and metrics need to be adopted for 3D considering the special characteristics of 3D perception• Huge opportunities in research, technology, art, and business CORESA 24-25 mai 2012, Lille, France
  • 74. 72 Thanks for your attentionCORESA24-25 mai 2012, Lille, France