Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Realism of image composits

247 views

Published on

Published in: Technology
  • Be the first to comment

Realism of image composits

  1. 1. Understanding  and  Improving  the  Realism  of  Image  Composites   Aliya  Ibragimova     University  of  Fribourg  
  2. 2. Agenda    •  Realis7c  Image  composi7ng  •  Iden7fying  key  sta7s7cs    •  Human  subject  experiments  •  Algorithm    •  Results  
  3. 3. Image  composi7ng  
  4. 4. Composi7ng  procedure  1.     Foreground   Alpha  maAe  2.   New  background   Composite  
  5. 5. Color  Transfer  Technique  (CTT)     Reinhard  et  al.  2001    ‘Match  color’  feature  of  Photoshop    
  6. 6. Color  transfer  technique:  limita7ons   Cut-­‐and-­‐paste   Match  Color  Conflates  the  effects  of  reflectance  and  illumina7on  
  7. 7. Improvements  of  CTT  (ColorComp)     Lalonde  and  Effros  2007  •  Analyze  huge  dataset  of  natural  images  :   difference  in  distribu7on  of  realis7c  and   unrealis7c  images  •  Recolor  regions  for  realis7c  composi7ng    Limita7ons:  requires  a  large  dataset,  depends  on  the  presence  of  images  that  are  similar  to  the  target    
  8. 8. #  of  pixels   Professional  compositors   Shadows   Midtones   Highlights   Brightness   •  Isolate  highlights  and  match  their  colors  and   brightness   •  Balance  mid-­‐tones  with  gamma  correc7on   •  Match  the  shadow  regions  
  9. 9. Color  Harmony    Cohen-­‐Or  et  al.  2006  Harmonic  colors  are  sets  of  colors  that  are  aesthe7cally  pleasing  in  terms  of  human  visual  percep7on.  
  10. 10. Color  Harmony    Cohen-­‐Or  et  al.  2006  Limita7ons:  obtained  images  are  not  necessary  realis7c,  ignores  luminance  and  contrast,  the  approach  has  not  been  quan7ta7vely  evaluated  
  11. 11. Alterna7ve  to  alpha  maAe:  seamlessly  blending  •  Feathering  •  Laplacian  pyramids  [Odgen  et  al.  1985]  •  Gradient-­‐domain  composi7ng  [Perez  et  al.   2003]  
  12. 12. Alterna7ve  to  alpha  maAe:  seamlessly  blending   foreground   background   Cut-­‐and-­‐paste   Gradient-­‐domain   Limita7ons:  2  source  images  should  have  similar   colors  and  textures  
  13. 13. Problem  statement  •  Which  sta7s7cs  control  realism?  •  How  do  these  sta7s7cs  affect  human   judgment  of  realism?  •  Automa7c  algorithm  to  improve  realism?    
  14. 14. Good  sta7s7cs  •  Highly  correlated  between  foreground  and   background  •  Easy  to  adjust  •  Independent  from  each  other  
  15. 15. Categories  of  sta7s7cal  measures  •  Luminance  •  Color  temperature  (CCT)  •  Satura7on  •  Local  contrast  •  Hue  (circular  sta7s7cs)    
  16. 16. Sta7s7cal  measures   Standard  devia7on  #  of  pixels   mean   Brightness   •  Mean   •  Standard  devia7on   •  High   •  Low   •  Kurtosis   •  Entropy  
  17. 17. Find  correla7on  •  Pearson  correla7on  coefficient  •  Standard  devia7on  of  offset  δi  =  Mif  –  Mib       M  –  measure   f  –  foreground   b  –  background   i  –  sta7s7cs    
  18. 18. Sta7s7cal  experiment  •  Use  large  (4126  images)  labeled  dataset  •  Select  the  most  correlated  sta7s7cs  
  19. 19. Luminance  
  20. 20. Color  Temperature  
  21. 21. Satura7on  
  22. 22. Sta7s7cal  experiment  
  23. 23. Results:  sta7s7cal  experiment    •   luminance,  color  temperature,  satura7on,   local  contrast  are  most  correlated  •   mean  of  zones  correlate  more  than  other   sta7s7cal  measures  •   mean  of  high  and  low  zones  correlate  more   than  mean  of  en7re  histogram  
  24. 24. Human  subjects  experiment  Experiment  with  human  subjects  on  Amazon  Mechanical  Turk  (MTurk)    •  20  natural  images  •  3  key  sta7s7cs  (luminance,  color  temperature,  satura7on)  
  25. 25. Human  subject  experiments  Foreground  luminance   Background  luminance  
  26. 26. Human  subjects  experiment  
  27. 27. Human  subjects  experiment  
  28. 28. Automa7c  composite  adjustment  •  Zone  selec7on  using  machine  learning   (random  forest  classifier)        T  =  s  *  σg,  s  =  0.1    Three  binary  classifiers:  •  Pick  smallest  changes  if  mul7ple  zones  •  Features  for  foreground  and  background  and   per  sta7s7c:  std,  skew,  kurt,  entropy,  p1,  p2  … p20.      
  29. 29. Pipeline  Input:  foreground  and  background  image  1.  Match  H-­‐zone  of  contrast  using  S-­‐shape  2.  Select  zone  and  adjust  mean  of  luminance  3.  Select  zone  and  adjust  mean  of  CCT  4.  Select  zone  and  adjust  satura7on    Adjust  algorithm  is  greedy,  could  iterate  several  7mes  if  needed    
  30. 30. Results  and  evalua7on  Cut-­‐and-­‐paste   Manual   MatchColor   ColorComp   Ours  
  31. 31. Results  and  evalua7on  Cut-­‐and-­‐paste   Manual   MatchColor   ColorComp   Ours  
  32. 32. Results  and  evalua7on  
  33. 33. Results  and  evalua7on  Thurstone’s  Law  of  Compara7ve  Judgement  
  34. 34. Results  and  evalua7on  •  BeAer  than  previous  methods  •  Close  to  manual  edi7ng  Limita7ons:  •  Religh7ng  •  Theory  for  zone  selec7on    
  35. 35. Ques7ons?  

×