• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
High dynamic images between devices and vision limits
 

High dynamic images between devices and vision limits

on

  • 1,041 views

Alessandro Rizzi, University of Milan, lecture at Media Integration and Communication Center 10/06/2011

Alessandro Rizzi, University of Milan, lecture at Media Integration and Communication Center 10/06/2011

Statistics

Views

Total Views
1,041
Views on SlideShare
1,014
Embed Views
27

Actions

Likes
0
Downloads
7
Comments
0

1 Embed 27

http://www.micc.unifi.it 27

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    High dynamic images between devices and vision limits High dynamic images between devices and vision limits Presentation Transcript

    • Le immagini ad alta dinamica tra i limiti dei dispositivi e quelli della visione Alessandro Rizzi Dipartimento di Informatica e Comunicazione Università degli Studi di MilanoFriday, June 10, 2011
    • Outline HDR imaging HDR in practice: measuring the limits Using HDRFriday, June 10, 2011
    • The dynamic rangeFriday, June 10, 2011
    • Friday, June 10, 2011
    • Define HDR ? do we need a threshold number ?Friday, June 10, 2011
    • Define HDR ? do we need a threshold number ? NOFriday, June 10, 2011
    • Define HDR A rendition of a scene with greater dynamic range than the reproduction mediaFriday, June 10, 2011
    • That is ?Friday, June 10, 2011
    • Friday, June 10, 2011
    • Annibale  Carracci      (1560-­‐1609)    PaesaggioFriday, June 10, 2011
    • Photo: C. OleariFriday, June 10, 2011
    • Photo: C. OleariFriday, June 10, 2011
    • Annibale  Carracci      (1560-­‐1609)    PaesaggioFriday, June 10, 2011
    • Source/lamp Average Luminance cd/ Light Xenon  short  arc m2 200  000  ÷  5  000  000  000 levels Sun Metal  halide 1  600  000  000 10  000  000  ÷  60  000  000 Incandescent 20  000  000  ÷  26  000  000 compact  Fluorescent   20  000  ÷  70  000 Fluorescent 5  000  ÷  30  000 Sunlit  clouds 10  000 Candle 7  500 blue  sky 5  000 Preferred  values  for   50  ÷  500 indoor  lighIng White  paper  at  sun 10  000 White  paper  at  500  lx 100 White  paper  at  5  lx 1 Courtesy: C. OleariFriday, June 10, 2011
    • Dynamic rangesFriday, June 10, 2011
    • Dynamic ranges ?Friday, June 10, 2011
    • Range limits and quantization: the ‘salame’ metaphorFriday, June 10, 2011
    • Friday, June 10, 2011
    • Range compression from incorrect pixel perspectiveFriday, June 10, 2011
    • Range compression from incorrect pixel perspectiveFriday, June 10, 2011
    • Range compression from incorrect pixel perspective Very wide range obtained with isolated stimuli impossible to obtain in an imageFriday, June 10, 2011
    • The “salame” metaphor Dynamic range QuantizationFriday, June 10, 2011
    • The “salame” metaphor Dynamic range Quantization More bits do not mean wider range Less bits do not mean shorter rangeFriday, June 10, 2011
    • 28=256 8 bit 2-3 log unit Scene Sensor DR DR 216=65536 16 bit 4-5 log unitFriday, June 10, 2011
    • 28=256 8 bit 2-3 log unit Scene DR Sensor DR NO 216=65536 16 bit 4-5 log unitFriday, June 10, 2011
    • 8 bit 16 bit 2-3 log unit Scene Sensor Scene Sensor DR DR DR DR 8 bit 16 bit 4-5 log unitFriday, June 10, 2011
    • Scene Sensor Scene Sensor DR DR DR DR 8 bit 16 bitFriday, June 10, 2011
    • Scene Sensor Scene Sensor DR DR DR DR 8 bit 16 bitFriday, June 10, 2011
    • The HDR idea http://www.adolfo.trinca.name/public/2010/11/ ahdrdiagram.jpgFriday, June 10, 2011
    • The HDR idea How ? general solution ? rendering intent ? http://www.adolfo.trinca.name/public/2010/11/ ahdrdiagram.jpgFriday, June 10, 2011
    • http://www.digitalcameratracker.com/how-to-create-high- definition-range-hdr-photos/Friday, June 10, 2011
    • Two sides of the coin • Objective data: recording/displaying physical light colorimetric distribution • Subjective data: reproducing appearance (or different rendering intent)Friday, June 10, 2011
    • Mapping the world: the characteristic curveFriday, June 10, 2011
    • H&D curveFriday, June 10, 2011
    • H&D curveFriday, June 10, 2011
    • H&D curveFriday, June 10, 2011
    • H&D curveFriday, June 10, 2011
    • Olympus E-3 http://www.dpreview.com/reviews/olympuse3/page21.aspFriday, June 10, 2011
    • Exposure problemFriday, June 10, 2011
    • Friday, June 10, 2011
    • Friday, June 10, 2011
    • History of HDR imagingFriday, June 10, 2011
    • HDR 1858 H.P. Robinson “Fading AwayFriday, June 10, 2011
    • “The Fundamentals of Photography” Mees (1920) 2 negative printFriday, June 10, 2011
    • Ansel AdamsFriday, June 10, 2011
    • Ansel Adams - Zone System ISCC 11/05-McCannFriday, June 10, 2011
    • Jones and Condit, 1941 Measurements of dynamic range of real scenes REFLECTANCE RANGE OF PRINTS SCENE RANGE OF WORLD Minimum Average of 126 outdoor scenes Maximum 0.0 1.5 3.0 log rangeFriday, June 10, 2011
    • L.A.Jones & H.R.Condit, JOSA,1941Friday, June 10, 2011
    • Retinex starting idea digit ~ luminance 119 119 Green record 55 146 88 230 ratio = ratio = 0.62 0.62 Ratios are constant in sun and shadeFriday, June 10, 2011
    • 1980Friday, June 10, 2011
    • Retinex cameraFriday, June 10, 2011
    • Capturing and reproducing the sceneFriday, June 10, 2011
    • Friday, June 10, 2011
    • Sensors dynamic range Limited !Friday, June 10, 2011
    • Is HDR a technological problem ?Friday, June 10, 2011
    • Expanding sensors dynamic range • Sensors that compress their response to light due to their logarithmic transfer function; • Multimode sensors that have a linear and a logarithmic response at dark and bright illumination levels, (switches between linear and logarithmic modes of operation); • Sensors with a capacity well adjustment method; • Frequency-based sensors, sensor output is converted into pulse frequency; • Time-to-saturation [(TTS); time-to-first spike] sensors, signal is the time the to saturated pixel; • Sensors with global control over the integration time; • Sensors with autonomous control over the integration time, where each pixel has control over its own exposure. Spivak A, Belenky A, Fish A & Yadid-Pecht O (2009) Wide dynamic-range CMOS image sensors: A comparative performance analysis, IEEE Trans. on Electron Devices, 56, 2446-2461.Friday, June 10, 2011
    • Friday, June 10, 2011
    • Friday, June 10, 2011
    • The HDR idea http://www.adolfo.trinca.name/public/2010/11/ ahdrdiagram.jpgFriday, June 10, 2011
    • The HDR idea How ? http://www.adolfo.trinca.name/public/2010/11/ ahdrdiagram.jpgFriday, June 10, 2011
    • Multiple image acquisitionFriday, June 10, 2011
    • CameraDigit = (radiance * time) • Multiple Exposures • Use Multiple Times • Recover scene radiances at all pixels from camera digits New goal: Accurately measure radiancesFriday, June 10, 2011
    • Multiple Exposures Flux = Luminance * time Scene Luminance = Flux / time Scene Luminance = Camera Digit / timeFriday, June 10, 2011
    • Multiple Exposures One Spot (ScaleD) 250 200 1/8 sec 1/4 sec 1/2 sec Camera Digit 150 Camera 1 sec 2 sec 4 sec Digit 100 8 sec 16 sec 32 sec 64 sec 50 FIT 0 0.0001 0.0010 0.0100 0.1000 1.0000 10.0000 100.0000 1000.0000 Exposure Flux [(cd/m2) * sec] Flux = Luminance * timeFriday, June 10, 2011
    • HDR file formats Source: Reinhard et al., High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting (The Morgan Kaufmann Series in Computer Graphics)Friday, June 10, 2011
    • HDR file formats Source: Reinhard et al., High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting (The Morgan Kaufmann Series in Computer Graphics)Friday, June 10, 2011
    • Acquisition limitsFriday, June 10, 2011
    • Friday, June 10, 2011
    • The glare problemFriday, June 10, 2011
    • The glare problemFriday, June 10, 2011
    • Friday, June 10, 2011
    • Effect of illumination 1.0 refl * 1.0 illum = 1.0 cd/m2 0.2 refl *0.01 illum = 0.002 cd/m2 Assumes 0.0 glareFriday, June 10, 2011
    • Glare is image dependent 1.0 refl * 1.0 illum = 1.0 cd/m2 0.002 cd/m2 *0.001 = 0.000002 0.001 1.0 cd/m2 *0.001 = 0.001 0.001 0.2 refl *0.01 illum = 0.002 cd/m2 Assumes 0.001 glareFriday, June 10, 2011
    • Ratio Signal/Glare 1.0 cd/m2)/(0.000002) = 5*10^5 ( 0.002 cd/m2)) / (0.001) = 2 Assumes 0.001 glareFriday, June 10, 2011
    • Sowerby, “Dictionary of Photography”, 1956Friday, June 10, 2011
    • Parasitic ImagesFriday, June 10, 2011
    • Camera limits • Glare • Unwanted scattered light in camera • air - glass reflections • lens (number of elements) • aperture • angle off optical axis • camera wall reflections • sensor surface reflections • We must measure actual veiling glare limitFriday, June 10, 2011
    • Measuring overall camera glareFriday, June 10, 2011
    • Friday, June 10, 2011
    • HDR Test SetupFriday, June 10, 2011
    • digit 255 = 2094.2 cd/m2 digit 0 = 0.11 cd/m2 Synthetic HDR (High-Dynamic Range) Images Text 18,619:1 Goal Image 2094.2 cd/m2 = 18,619 0.11 cd/m2Friday, June 10, 2011
    • 20:1 18,619:1 TargetsFriday, June 10, 2011
    • 16 sec exposure - Target 1scaleBlackFriday, June 10, 2011
    • 16 sec exposure - Target 4scaleBlackFriday, June 10, 2011
    • 16 sec exposure - Target 4scaleBlackFriday, June 10, 2011
    • Target 1B Text Target 4B Text Target 4W 16 sec exposureFriday, June 10, 2011
    • Constant Luminance - Variable SurroundFriday, June 10, 2011
    • Minimum GlareFriday, June 10, 2011
    • Mild GlareFriday, June 10, 2011
    • Maximum GlareFriday, June 10, 2011
    • Friday, June 10, 2011
    • Friday, June 10, 2011
    • 4.3 log10 scene ----> 3.0 log10 image Scene In-camera Maximum Scene Dynamic Accurate Error Range Range (% radiance) 1scaleB 20:1 20:1 0 4scaleB 18,619:1 3,000:1 1 300% Min 4scaleW 18,619:1 100:1 10,000% Max Measure In-camera AccuracyFriday, June 10, 2011
    • Side Dupe FilmFriday, June 10, 2011
    • Slide Dupe FilmFriday, June 10, 2011
    • One Negative Capture 4scale Black - Single Negative 2.50 3.5 Log10 units 2.30 2.10 Log digit 1.90 1.70 1.50 -1.00 -0.50 0.00 0.50 1.00 1.50 2.00 2.50 3.00 3.50 Log Cd/m2Friday, June 10, 2011
    • Dynamic Range (OD)Friday, June 10, 2011
    • HDR from cameras • Range of usable captured information • Range of accurate luminance information (much smaller) • Scene dependentFriday, June 10, 2011
    • Courtesy: M. FairchildFriday, June 10, 2011
    • Glare insertion Gregory Ward Larson, Holly Rushmeier, and Christine Piatko, “A Visibility Matching Tone Reproduction Operator for High Dynamic Range Scenes”, IEEE Trans on VISUALIZATION AND COMPUTER GRAPHICS, VOL. 3, NO. 4, oct-dec 1997Friday, June 10, 2011
    • Display: measuring the human limitsFriday, June 10, 2011
    • Friday, June 10, 2011
    • Magnitude estimates (100-1)Friday, June 10, 2011
    • • Luminance does not correlate uniquely with appearance • No global tone scale can render the appearanceFriday, June 10, 2011
    • Magnitude Estimation of Appearance Change Surrounds 100 90 Magnitude Estimation 80 70 60 50 40 30 20 10 0 0.10 1.00 10.00 100.00 1000.00 10000.00 Log Luminance (cd/m2) Min [0 cd/m2] Max [2094 cd/m2]Friday, June 10, 2011
    • •White surround •adds glare •changes surround (simultaneous contrast) We need a new range target •Vary dynamic range with •constant glare •contrast surroundFriday, June 10, 2011
    • Center/Surround Basic Unit Gray test areas 12% (small differences) Fixed contrast surround 88%Friday, June 10, 2011
    • 90o rotationFriday, June 10, 2011
    • Friday, June 10, 2011
    • Testing different glares % of white surround 100% 50% 0%Friday, June 10, 2011
    • Single & Double Density Transparencies Single = 2.7 log10 range Double = (superimposed) 5.4 log10 rangeFriday, June 10, 2011
    • 5.4 & 2.7 log10 Ranges Constant Glare & SurroundFriday, June 10, 2011
    • White[100] = 0.0 rOD - Black [1] = 2.89 rOD 100.0 90.0 80.0 70.0 magnitude estimation 60.0 50.0 40.0 30.0 50% white surround 20.0 10.0 0.0 6 5 4 3 2 1 0 relative optical density 50% Single DensityFriday, June 10, 2011
    • 100 90 80 70 2.3 log10 units magnitude estimation 60 50 40 50% white 30 surround 20 10 0 6 5 4 3 2 1 0 relative optical density 50% Double Density 50% Single DensityFriday, June 10, 2011
    • 100 90 2.0 log10 units 80 70 magnitude estimation 60 50 40 100% white surround 30 20 10 0 6 5 4 3 2 1 0 relative optical density White Double Density White Single DensityFriday, June 10, 2011
    • 100 90 5.0 log10 units 80 70 magnitude estimation 60 50 40 0% white 30 surround 20 10 0 6 5 4 3 2 1 0 relative optical density Black Double Density Black Single DensityFriday, June 10, 2011
    • 100 90 5.0 log10 units 80 Over 20 70 not big improvement magnitude estimation 60 50 40 0% white 30 surround 20 10 0 6 5 4 3 2 1 0 relative optical density Black Double Density Black Single DensityFriday, June 10, 2011
    • Measurements of apparent range (depends on area of white) •100% = 2.0 log units 10 • 50% = 2.3 log units 10 • 8% = 2.9 log units 10Friday, June 10, 2011
    • DD DD DD DDFriday, June 10, 2011
    • Test summary • Double transmission contrast • Double dynamic range • very small change in appearance range • Visual limit ~ area of white surround • area of white controls glareFriday, June 10, 2011
    • What is on the retina: calculated retinal luminanceFriday, June 10, 2011
    • Friday, June 10, 2011
    • What comes to the retina is different from the image High glare Low glareFriday, June 10, 2011
    • Veiling glare increases gray luminance Contrast offsets glare Contrast decreases gray appearance Glare vs. ContrastFriday, June 10, 2011
    • Discussion • Glare lowers the physical contrast • Spatial comparisons increase the contrast of appearance. • The two act in opposition. • Change with distance are different and the cancellation is far from exact.Friday, June 10, 2011
    • Glare Spread Function 1Vos, J.J. and van den Berg, T.J.T.P, CIE Research note 135/1, “Disability Glare”, ISBN 3900734976 (1999). PIGMENT Blue eyed Caucasian 1.21 Blue green Caucasian 1.02 Mean over all Caucasian 1.00 Brown eyed Caucasian 0.50 Non Caucasian with pigmented skin and dark brown eyes 0.00Friday, June 10, 2011
    • Glare Spread Function Plotted in log scaleFriday, June 10, 2011
    • Dynamic Range = 5.4 OD or 251,189:1 False-color LookUpTable (LUT)Friday, June 10, 2011
    • Same LUT applied to SD & DD Visualize HDR targetsFriday, June 10, 2011
    • Retinal imageFriday, June 10, 2011
    • Same LUT applied to SD & DD Visualize Retinal ImagesFriday, June 10, 2011
    • Same LUT applied to SD & DD Change LUT for Retinal ImagesFriday, June 10, 2011
    • Change LUT for Retinal ImagesFriday, June 10, 2011
    • Scene Retina Appearance 1,000,000:1 100:1 1,000:1 Spatial Spatial Glare Contrast Two scene-dependent spatial mechanisms: glare and contrast Glare masks the strength of spatial contrastFriday, June 10, 2011
    • RangesFriday, June 10, 2011
    • Tone-rendering problem and spatial comparisonsFriday, June 10, 2011
    • Friday, June 10, 2011
    • Choosing a rendering intentFriday, June 10, 2011
    • 124Friday, June 10, 2011
    • 124Friday, June 10, 2011
    • Friday, June 10, 2011
    • Friday, June 10, 2011
    • Friday, June 10, 2011
    • Land experimentFriday, June 10, 2011
    • Land experimentFriday, June 10, 2011
    • Land experiment ProjectorFriday, June 10, 2011
    • Land experiment ProjectorFriday, June 10, 2011
    • Land experiment ES=100 EM=100 EL=100 ProjectorFriday, June 10, 2011
    • Land experiment ES=100 EM=100 EL=100 Projector ColorimeterFriday, June 10, 2011
    • Land experiment ES=100 EM=100 EL=100 LS=255 LM=115 LL=255 Projector ColorimeterFriday, June 10, 2011
    • Land experiment Observer ES=100 EM=100 EL=100 LS=255 LM=115 LL=255 Projector ColorimeterFriday, June 10, 2011
    • Land experiment PINK Observer ES=100 EM=100 EL=100 LS=255 LM=115 LL=255 Projector ColorimeterFriday, June 10, 2011
    • Land experiment Observer Projector ColorimeterFriday, June 10, 2011
    • Land experiment Observer ES=50 EM=111 EL=50 Projector ColorimeterFriday, June 10, 2011
    • Land experiment Observer ES=50 EM=111 EL=50 LS=128 LM=128 LL=128 Projector ColorimeterFriday, June 10, 2011
    • Land experiment PINK Observer ES=50 EM=111 EL=50 LS=128 LM=128 LL=128 Projector ColorimeterFriday, June 10, 2011
    • Land experiment GRAY PINK Observer ES=50 EM=111 EL=50 LS=128 LM=128 LL=128 Projector ColorimeterFriday, June 10, 2011
    • visual sensationFriday, June 10, 2011
    • HVS: local compression of rangeFriday, June 10, 2011
    • HVS: local compression of rangeFriday, June 10, 2011
    • Tone mapping vs Tone rendering No tone mapping operator (global) can mimic vision We need an image dependent tone renderer operator (local)Friday, June 10, 2011
    • Black and White MondrianFriday, June 10, 2011
    • HP 945 Images without “Frames of Reference”Friday, June 10, 2011
    • Some examplesFriday, June 10, 2011
    • Friday, June 10, 2011
    • Bob Sobol, HPR. Sobol, “ Improving the Retinex algorithm for rendering wide dynamic range photographs”, in Human Vision and Electronic Imaging VII, B. E. Rogowitz and T. N.Pappas, ed., Proc. SPIE 4662-41, 341-348, 2002.Friday, June 10, 2011
    • Friday, June 10, 2011
    • ACE Original ACE Original ACEFriday, June 10, 2011
    • STRESS Tone RenderingFriday, June 10, 2011
    • Judging the resultsFriday, June 10, 2011
    • Beauty contest C. Gatta, A. Rizzi, D. Marini, “Perceptually inspired HDR images tone mapping with color correction”, Journal of Imaging Systems and Technology, Volume 17 Issue 5, pp. 285-294 (2007).Friday, June 10, 2011
    • HDR is in the middle Glare Post-LUT Sensor Spatial graphics Pre-LUT Algorithm card Image Spatial Scene in CPU Image Display memory in CPUFriday, June 10, 2011
    • SummaryFriday, June 10, 2011
    • • To understand HDR we need a new perspective! 1.Veiling glare limits the range on the retina 2. Neural processing (spatial) determines appearance 3. Neural is stronger than it appears [neural cancels glare] 4. General Solution requires spatial process [mimic vision] 5. Tone-Scale is limited, we need Tone-rendering [scene dependent]Friday, June 10, 2011
    • Take home points • HDR limits are not (only) technological • Glare limits both acquisition and vision • Glare is scene dependent • Human vision use spatial comparison to overcome this limit • Tone renderer operator can use the same approachFriday, June 10, 2011
    • Take home points HDR works very well • because preserves image information • not because are more accurate (not possible)Friday, June 10, 2011
    • References • J. J. McCann, A. Rizzi, “Camera and visual veiling glare in HDR images” Journal of the Society for Information Display 15/9, 721–730 (2007). • J. J. McCann, “Art, Science and Appearance in HDR” Journal of the Society for Information Display 15/9, 709–719 (2007). • A. Rizzi, J. J. McCann, “Glare-limited Appearances in HDR Images”, Journal of the Society for Information Display, 17/1, pp. 3-12, (2009). • J. J. McCann, A. Rizzi, “Retinal HDR Images: Intraocular Glare and Object Size” Journal of the Society for Information Display, 17/11, pp. 913-920, (2009).Friday, June 10, 2011
    • The art and science of HDR imaging J.J. McCann, A. Rizzi (expected publication date autumn 2011)Friday, June 10, 2011
    • Thank you alessandro.rizzi@unimi.itFriday, June 10, 2011