Image-Based Illumination for Electronic Display of Artistic Paintings Da Young Ju, Jin-Ho Yoo, Gregory Sharp and Sang Wook Lee July 25, 2002 Sogang University University of Michigan Image-based illumination for

Goal How can we create artificial illumination environments comparable to good art museums?

Process for Electronic Display & Printing Output Poster and Electronic display - Photographed images under fixed lighting Photographing Scanning Jpeg,Tiff.. Poster Monitor Web

Gallery vs. Electronic Display Gallery Display - Color - Size - Brush stroke texture

Effect of Illumination on Appearance

Solution 1: Moving the Viewer Requires difficult registration of views

Solution 1: Moving the Viewer Requires difficult registration of views

No problem aligning views But how to deal with the large data size? Solution 2: Moving the Illumination

Solution 3: Hybrid Modeling Residual image database can be smaller than full image database Residual Image Database Model Parameters Captured images Rendered images Residual images

Hybrid Modeling f (x,y,  ) + r(x,y,  ) Image Model Residual I(x,y,  ) = Diffuse (Lambertian) Specular (Phong) I d (x,y,  ) + r(x,y,  ) I s (x,y,  ) + I(x,y,  ) =

Parametric Modeling (with a backup plan) Captured images Rendered images Fit Parametric Lighting Model

Parametric Modeling (with a backup plan) Captured images Rendered images Residual images = + We will use a simple parametric model: Lambertian diffuse + Phong specular Residual images will be compressed using PCA

Lambertian Diffuse Term I d = C d ( N • L ) = C d cos  L : light direction N : surface normal N L  N L 

Phong Specular Term I s = C s ( V • R ) n V : viewing direction L : light direction N : surface normal R : lighting reflection unit vector (mirror of L about N ) R N L  V Assumption: V N = C s [ 2( L • N )( V • N )- V • L ] n = C s cos n  C s [ N • L ] n

C s [ N • L ] n

Acquisition of Image Irradiance from a Painting N V L  Measure intensity I [  ] for many  C

Acquisition of Image Irradiance from a Painting V I [  ] = C d cos  + C s cos n  + r [  ] L   N Measure intensity I [  ] for many  C

Acquisition of Image Irradiance from a Painting V L    N Measure intensity I [  ] for many  I [  ] = C d cos [  -  ] + C s cos n [  -  ] + r [  ] C

Optimization I [  ] = C d cos [  -  ] + C s cos n [  -  ] + r [  ] Four unknowns for each pixel C d Diffuse coefficient C s Specular coefficient  Structure n Shininess

Optimization For each pixel, minimize MSE residual over all lighting angles φ

Compression of Residuals Compression of residual terms using PCA

Reconstruction = Model + Residuals Model  =30 Original Model + 1 component Model + 10 components

Results

Acquiring Images

Original Diffuse Structure Residual Specular Shininess

Experimental Results / Oil painting 2 Original Diffuse Structure Residual Specular Shininess

Experimental Results / Water color painting Original Diffuse Structure Residual Specular Shininess

Live demonstration with browser Portrait

Conclusion The first work on artistic painting Creating artificial illumination environments Important 3D effects in paintings with low data dimension – brush stroke and canvas texture

The first work on artistic painting

Creating artificial illumination environments

Important 3D effects in paintings with low data

dimension

– brush stroke and canvas texture

Future work