Your SlideShare is downloading. ×
Realism of image composits
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Realism of image composits

122
views

Published on

Published in: Technology

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
122
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Understanding  and  Improving  the  Realism  of  Image  Composites   Aliya  Ibragimova     University  of  Fribourg  
  • 2. Agenda    •  Realis7c  Image  composi7ng  •  Iden7fying  key  sta7s7cs    •  Human  subject  experiments  •  Algorithm    •  Results  
  • 3. Image  composi7ng  
  • 4. Composi7ng  procedure  1.     Foreground   Alpha  maAe  2.   New  background   Composite  
  • 5. Color  Transfer  Technique  (CTT)     Reinhard  et  al.  2001    ‘Match  color’  feature  of  Photoshop    
  • 6. Color  transfer  technique:  limita7ons   Cut-­‐and-­‐paste   Match  Color  Conflates  the  effects  of  reflectance  and  illumina7on  
  • 7. Improvements  of  CTT  (ColorComp)     Lalonde  and  Effros  2007  •  Analyze  huge  dataset  of  natural  images  :   difference  in  distribu7on  of  realis7c  and   unrealis7c  images  •  Recolor  regions  for  realis7c  composi7ng    Limita7ons:  requires  a  large  dataset,  depends  on  the  presence  of  images  that  are  similar  to  the  target    
  • 8. #  of  pixels   Professional  compositors   Shadows   Midtones   Highlights   Brightness   •  Isolate  highlights  and  match  their  colors  and   brightness   •  Balance  mid-­‐tones  with  gamma  correc7on   •  Match  the  shadow  regions  
  • 9. Color  Harmony    Cohen-­‐Or  et  al.  2006  Harmonic  colors  are  sets  of  colors  that  are  aesthe7cally  pleasing  in  terms  of  human  visual  percep7on.  
  • 10. Color  Harmony    Cohen-­‐Or  et  al.  2006  Limita7ons:  obtained  images  are  not  necessary  realis7c,  ignores  luminance  and  contrast,  the  approach  has  not  been  quan7ta7vely  evaluated  
  • 11. Alterna7ve  to  alpha  maAe:  seamlessly  blending  •  Feathering  •  Laplacian  pyramids  [Odgen  et  al.  1985]  •  Gradient-­‐domain  composi7ng  [Perez  et  al.   2003]  
  • 12. Alterna7ve  to  alpha  maAe:  seamlessly  blending   foreground   background   Cut-­‐and-­‐paste   Gradient-­‐domain   Limita7ons:  2  source  images  should  have  similar   colors  and  textures  
  • 13. Problem  statement  •  Which  sta7s7cs  control  realism?  •  How  do  these  sta7s7cs  affect  human   judgment  of  realism?  •  Automa7c  algorithm  to  improve  realism?    
  • 14. Good  sta7s7cs  •  Highly  correlated  between  foreground  and   background  •  Easy  to  adjust  •  Independent  from  each  other  
  • 15. Categories  of  sta7s7cal  measures  •  Luminance  •  Color  temperature  (CCT)  •  Satura7on  •  Local  contrast  •  Hue  (circular  sta7s7cs)    
  • 16. Sta7s7cal  measures   Standard  devia7on  #  of  pixels   mean   Brightness   •  Mean   •  Standard  devia7on   •  High   •  Low   •  Kurtosis   •  Entropy  
  • 17. Find  correla7on  •  Pearson  correla7on  coefficient  •  Standard  devia7on  of  offset  δi  =  Mif  –  Mib       M  –  measure   f  –  foreground   b  –  background   i  –  sta7s7cs    
  • 18. Sta7s7cal  experiment  •  Use  large  (4126  images)  labeled  dataset  •  Select  the  most  correlated  sta7s7cs  
  • 19. Luminance  
  • 20. Color  Temperature  
  • 21. Satura7on  
  • 22. Sta7s7cal  experiment  
  • 23. Results:  sta7s7cal  experiment    •   luminance,  color  temperature,  satura7on,   local  contrast  are  most  correlated  •   mean  of  zones  correlate  more  than  other   sta7s7cal  measures  •   mean  of  high  and  low  zones  correlate  more   than  mean  of  en7re  histogram  
  • 24. Human  subjects  experiment  Experiment  with  human  subjects  on  Amazon  Mechanical  Turk  (MTurk)    •  20  natural  images  •  3  key  sta7s7cs  (luminance,  color  temperature,  satura7on)  
  • 25. Human  subject  experiments  Foreground  luminance   Background  luminance  
  • 26. Human  subjects  experiment  
  • 27. Human  subjects  experiment  
  • 28. Automa7c  composite  adjustment  •  Zone  selec7on  using  machine  learning   (random  forest  classifier)        T  =  s  *  σg,  s  =  0.1    Three  binary  classifiers:  •  Pick  smallest  changes  if  mul7ple  zones  •  Features  for  foreground  and  background  and   per  sta7s7c:  std,  skew,  kurt,  entropy,  p1,  p2  … p20.      
  • 29. Pipeline  Input:  foreground  and  background  image  1.  Match  H-­‐zone  of  contrast  using  S-­‐shape  2.  Select  zone  and  adjust  mean  of  luminance  3.  Select  zone  and  adjust  mean  of  CCT  4.  Select  zone  and  adjust  satura7on    Adjust  algorithm  is  greedy,  could  iterate  several  7mes  if  needed    
  • 30. Results  and  evalua7on  Cut-­‐and-­‐paste   Manual   MatchColor   ColorComp   Ours  
  • 31. Results  and  evalua7on  Cut-­‐and-­‐paste   Manual   MatchColor   ColorComp   Ours  
  • 32. Results  and  evalua7on  
  • 33. Results  and  evalua7on  Thurstone’s  Law  of  Compara7ve  Judgement  
  • 34. Results  and  evalua7on  •  BeAer  than  previous  methods  •  Close  to  manual  edi7ng  Limita7ons:  •  Religh7ng  •  Theory  for  zone  selec7on    
  • 35. Ques7ons?