Adapting ISO 20462 Softcopy Quality Ruler Method for on-line Image Quality Studies

  • 344 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • It's really a great idea to engage on-line 'crowd' to participate to this kind of image quality evaluation study: a very valuable piece of work, indeed!
    Are you sure you want to
    Your message goes here
    Be the first to like this
No Downloads

Views

Total Views
344
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
8
Comments
1
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • 02/13/13
  • 02/13/13 Image texture is the term given to the information-bearing fluctuations such as those for skin, grass and fabrics. Since image processing aimed at reducing unwanted fluctuations (noise are other artifacts) can also remove important texture, good product design requires a balance. To aid in the image quality evaluation of digital and mobile-telephone cameras a method is being developed as part of an international standards effort. The method addresses the retention of image texture
  • 02/13/13 Image texture is the term given to the information-bearing fluctuations such as those for skin, grass and fabrics. Since image processing aimed at reducing unwanted fluctuations (noise are other artifacts) can also remove important texture, good product design requires a balance. To aid in the image quality evaluation of digital and mobile-telephone cameras a method is being developed as part of an international standards effort. The method addresses the retention of image texture
  • 02/13/13
  • 02/13/13
  • 02/13/13

Transcript

  • 1. Adapting ISO 20462 Softcopy Quality Ruler Method for On-line Image Quality Studies   Peter D. Burns, Jonathan B. Phillips1 and Don Williams2 Burns Digital Imaging, 1NVIDIA Corp. and 2 Image Science AssociatesReference:P. D. Burns, J. B. Phillips and D Williams, Adapting ISO 20462 Softcopy Quality RulerMethod for On-line Image Quality Studies, Proc. SPIE Vol. 8653, 86530E-1, 2013 IS&T and SPIE Electronic Imaging Symposium, Feb. 2013
  • 2. Outline1. ISO 20462 Image quality ruler method2. Extended to softcopy image quality3. Adaptation for Internet-based image quality evaluations • Viewing method • Image selection and preparation • Design for judgments4. Verification study • Presentation of softcopy ruler (images) • Previous texture-loss experiment5. Conclusions and recommendations • Comparison with ISO softcopy ruler results IS&T and SPIE Electronic Imaging 2013 2
  • 3. IntroductionISO 20462 Image quality ruler method • Set of calibrated reference hardcopy images • Primary Standard Quality Scale, SQS1, in overall JND (just noticeable difference) units • Ruler is a set of prints based on a single scene, varying from ideal to lower image qualityExtended to softcopy image quality • Digital Reference stimuli, updated ISO 20462 • Calibrated display and viewing conditions, verified with four labs. • Secondary Standard Quality Scale, SQS2Internet-based image quality evaluations • Our objective: Develop and test method that uses commercial online presentation • efficient methods for gathering evaluation data (e.g. crowdsourcing) that can be related to the ISO method • when commonly used (uncontrolled) viewing conditions, hardware, software and lighting are needed IS&T and SPIE Electronic Imaging 2013 3
  • 4. ISO softcopy ruler methodRuler image is adjusted until it matches test image on right• GUI software used slider Updated by slider movement IS&T and SPIE Electronic Imaging 2013 4
  • 5. Previous work: ISO softcopy ruler application for texture blurMeasurement of image quality degradation due to loss of texture,caused by noise-cleaning image processing • Ruler varied in image sharpness • test scenes varied in texture loss Memorial Art Gallery, Flowers, Girl and Mountain. Scenes used in their original form (1088 x 816, or 1253 x 834 pixels),
  • 6. Previous work: ISO softcopy ruler application for texture blurNoise-cleaned test images compared to ruler images varying insharpness to obtain SQS2 JNDs for each test image Ruler, Standard Quality Scale 30 Ruler, Standard Quality Scale 15 noise-cleaned sample treatment texture loss level 6 IS&T and SPIE Electronic Imaging 2013 6
  • 7. Adaptation of ISO Method for On-line Evaluations Presentation and Image Selection, Preparation1. Presentation • Internet-browser based • Commercial online presentation, Survey Monkey – Likely maximum display size and resolution • Presentation of images without cropping or scaling – Image file size limitations • Static presentation of ruler, rather than dynamic slider-controlled – Button selection with anchors (two sets)2. Image selection and preparation • Preserve image microstructure – Crop selected ruler and test images • Use previous texture-loss study and images – Ruler based on range of image sharpness – Texture loss due to noise-cleaning image processing IS&T and SPIE Electronic Imaging 2013 7
  • 8. Adaptation of ISO Method for On-line Evaluations1. Presentation • Button selection with two anchor reference images • Each test image was rated twice, compared with two sets of references [0 (low), 15] and [15, 30 (high)] SQS2 values IS&T and SPIE Electronic Imaging 2013 8
  • 9. Adaptation of ISO Method for On-line Evaluations2. Image selection and preparation • Selection of scenes based on JND sensitivity found in previous work • Texture loss due to noise-cleaning image processing (1088 x 816, or 1253 x 834 pixels), IS&T and SPIE Electronic Imaging 2013 9
  • 10. Adaptation of ISO Method for On-line Evaluations2. Image selection and preparation• Commercial service: Survey Monkey• URL vs. upload• Display field aimed at display of 1600 x 900 pixel Original scenes in previous work (1088 x 816, or 1253 x 834 pixels) Cropped scenes in current experiment (525 x 330 pixels per scene) IS&T and SPIE Electronic Imaging 2013 10
  • 11. Study: Image Quality Due to Noise-Cleaning 1. Same scenes and treatments that were used in previous verification study 2. Initial subjective results with 21 overall quality buttons • Observers with and without involvement in the imaging industry. (N=21) 3. Second study with 9 overall quality buttons for the observers to select • Similar observer population, though not the same observers (N=11) • Experienced observers in image quality evaluation (N=20) Rate the quality of the test image based on the position between the left and right reference images IS&T and SPIE Electronic Imaging 2013 11
  • 12. ISO Softcopy Results for the Four Scenes• From ISO method study rsults• SQS2 JNDs for the 4 scenes and noise-cleaning treatments levels for calibrated reference points from previous study.• Error bars are standard errors, N=17 IS&T and SPIE Electronic Imaging 2013 12
  • 13. Results 1: Adapted Online MethodInferred SQS2 JNDs for the 4 scenes and treatments levels for 21buttons of overall quality • 21 observers IS&T and SPIE Electronic Imaging 2013 13
  • 14. Results 2: Adapted Online MethodInferred SQS2 JNDs for the 4 scenes and treatments levels for 9buttons of overall quality • 11 observers IS&T and SPIE Electronic Imaging 2013 14
  • 15. Results 3: Adapted Online MethodComparison of our inferred SQS2 JND results with those of thereference experiment (previous work) with calibrated SQS2 JNDs.Averaged over observers and scenes. IS&T and SPIE Electronic Imaging 2013 15
  • 16. Summary1. Softcopy ruler method has been adapted for unconstrained on-line image quality studies • Static ruler-anchor display vs. slider-ruler selection • Button selection of image quality rating • Cropped scenes2. Results show similar scene-differences in IQ sensitivity3. 9- and 21-button results were similar, implying a quantizing of ruler scale response, as intended4. Pooled results data for all four scenes were consistent with results from the original softcopy ruler study5. Uncontrolled, on-line results showed more sensitivity to the overall image quality degradation (rather than less, as anticipated) • Possibly due to cropping most salient image areas, or uncontrolled viewing distance6. Conclusion: Internet-based image quality evaluations can be related to ISO 20462 standard if common reference images are used. IS&T and SPIE Electronic Imaging 2013 16
  • 17. SUBJECTIVE IMAGEQUALITY EVALUATION(Not presented at EI 2013)Image quality ruler
  • 18. ISO 20462 Part 3 – The Quality RulerMethod“Photography—Psychophysical experimental methods to estimate image quality—Part 3: Quality ruler method”Standardization of anchored pair method based on JND unitsSubstitute length units with an image set having a series of sharpness levelsQuality ruler approach allows for repeatable and definable results High quality (sharp) Low quality (blurry)Compare test image to images on ruler and pick best match on ruler
  • 19. ISO 20462 Part 3 – The Quality Ruler Method Standard Quality Scale (SQS) 32 24 16 8 0 Excellent Very Good Fair Poor Not Worth Good Keeping Standard Reference Stimuli (SRS) or Digital Reference Stimuli (DRS)JND = 75:25 split in paired comparison;JNDs = advertisable differences;JNDs = interval of image quality categories
  • 20. Softcopy Quality RulerSoftcopy quality ruler method based on ISO 20462 Part 3 • 30" Apple Cinema Display monitor • Controlled viewing distance with chin/head rest* – 25", 34", or 43" • Simultaneous viewing of ruler and test images • Quality ruler reference stimuli span 30 just noticeable difference values in sharpness * e.g., University of Houston College of Optometry (UHCO) HeadSpot
  • 21. Softcopy Ruler Method Development Raw Demosaic Linear sRGB Camera MTF image WB + CCM image Response Monitor displays PSF image captured Monitor MTF Single bright pixel and analyzed Response System MTF = Camera MTF * Monitor MTF * Downsampling MTF ( 1MP)
  • 22. 21 Quality Rulers • 21 scene choices • 30 ruler images per scene • SQS from 2 to 31 in increments of approximately 1 JND • Matlab code for running quality ruler Soon to be officially located at:http://www.imaing.org/ist/resources/standards/standards.cfm Secondary location: http://www.aptina.com/ImArch/
  • 23. Softcopy Quality Ruler Screen Screen Capture In situ
  • 24. Softcopy Quality Ruler Method Workflow One-time setup Running the tests Obtain ruler Set up viewing Obtain Obtain test Perform Analyze data images environment software tools images psychovisual and report studies results Fixed sets of Specifications Software Specifications Specifications Tools for digital anchor on viewing applications to on obtaining on subject analyzing the images, i.e. distance and display images test images at selection, vision results and - 21 scenes ambient and collect any time of the screening, report- 30 images per lighting. viewer year and number of template scene Create ICC responses. around the subjects, etc. profile for the globe in a monitor. timely manner