Automatic Photo Selection for  Media and Entertainment  Applications Ekaterina Potapova,  Marta Egorova,  Ilia Safonov Nat...
Applications Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 2
Applications Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 2
Applications – photo book Images are taken from printbook.ru, ehow.com, snapfish.com.au, smilebooks.co.uk GraphiCon 2009 3...
Applications – slide show Photos from ITaS’2008 GraphiCon 2009 4 Automatic Photo Selection for Media and Entertainment App...
General workflow GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications
GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications General workflow Detection of low-qual...
General workflow GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications Detection of low-qual...
General workflow GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications Selection of appealin...
Detection of low-quality photos GraphiCon 2009 6 Automatic Photo Selection for Media and Entertainment Applications
Estimation of JPEG quality A.Foi et al.,2007 Images are taken from en.wikipedia.org Quantization Table GraphiCon 2009 7 Au...
Detection of backlit, low-contrast & blurred photos Two Ada Boost classifiers committee:  -for detection of low-contrast a...
Detection of backlit and low-contrast photos  - 1   S1/S2 -  ratio of tones in shadows to midtones GraphiCon 2009 9 Automa...
S11/S12  - ratio of tones in first to second part of shadows Detection of backlit and low-contrast photos  - 1   GraphiCon...
M1/M2  -  ratio of the histogram maximum in shadows to the maximum in midtones Detection of backlit and low-contrast photo...
P1   - location of the histogram maximum in shadows P1 Detection of backlit and low-contrast photos  - 1   GraphiCon 2009 ...
C  –  global contrast H 0 C 0 C 1 H 1 Detection of backlit and low-contrast photos  - 1   GraphiCon 2009 9 Automatic Photo...
Training set:  480 photos Error rate on cross-validation test :  ~0.055 Testing set:  1830 with 2% affected by backlit and...
Image Intensity image Z 1 =[-1 1] Z 2 =[-1  0 1] Z 3 =[-1  0 0 1] Z 10 =[-1  0 0 0 0 0 0 0 0 0 1] I.Safonov et al.,2008 … ...
Crete et al., 2007 F.Crete et al.,2007 ? Image Blurred image Edge image Edge image Comparison of the  images HPF=[1 -1] LP...
Training set:  416 photos Error rate on cross-validation test :  ~0.07 Testing set:  1830 with 171 blurred photos The numb...
Time and camera-based quantization i  is an index of source   L  is time between the least and the most time for the large...
GraphiCon 2009 12 Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection The most appe...
Conspicuity  maps Gaussian pyramids Image Intensity image r-channel g-channel b-channel R-channel G-channel B-channel Y-ch...
original image saliency map intensity map color map orientation map ROI Automatic Photo Selection for Media and Entertainm...
Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection GraphiCon 2009 15 124 88 11 100...
Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection GraphiCon 2009 15 83 11 124 Sal...
<ul><li>Main Disadvantages: </li></ul><ul><li>average number of FP increases a lot with picture size </li></ul>We consider...
Photos ranking Heuristic formula, experiments have shown that value w=25 gives the best result Automatic Photo Selection f...
Photos ranking Heuristic formula, experiments have shown that value w=25 gives the best result Automatic Photo Selection f...
Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18
Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18 Autocollage ch...
Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18
Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18 Autocollage ch...
Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18
Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18 Autocollage ch...
Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 19 Proposed AutoC...
? Automatic Photo Selection for Media and Entertainment Applications Questions & Answers GraphiCon 2009 8
Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 9 Thank you for your attention =)
Upcoming SlideShare
Loading in...5
×

Automatic Photo Selection For Media And Entertainment Applications

759

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
759
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
9
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Automatic Photo Selection For Media And Entertainment Applications"

  1. 1. Automatic Photo Selection for Media and Entertainment Applications Ekaterina Potapova, Marta Egorova, Ilia Safonov National Nuclear Research University MEPhI Moscow, Russia GraphiCon 2009 5-9 October
  2. 2. Applications Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 2
  3. 3. Applications Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 2
  4. 4. Applications – photo book Images are taken from printbook.ru, ehow.com, snapfish.com.au, smilebooks.co.uk GraphiCon 2009 3 Automatic Photo Selection for Media and Entertainment Applications
  5. 5. Applications – slide show Photos from ITaS’2008 GraphiCon 2009 4 Automatic Photo Selection for Media and Entertainment Applications
  6. 6. General workflow GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications
  7. 7. GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications General workflow Detection of low-quality photos
  8. 8. General workflow GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications Detection of low-quality photos Adaptive quantization on time-camera plane
  9. 9. General workflow GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications Selection of appealing photos Detection of low-quality photos Adaptive quantization on time-camera plane
  10. 10. Detection of low-quality photos GraphiCon 2009 6 Automatic Photo Selection for Media and Entertainment Applications
  11. 11. Estimation of JPEG quality A.Foi et al.,2007 Images are taken from en.wikipedia.org Quantization Table GraphiCon 2009 7 Automatic Photo Selection for Media and Entertainment Applications
  12. 12. Detection of backlit, low-contrast & blurred photos Two Ada Boost classifiers committee: -for detection of low-contrast and backlit photos -for detection of blurred photos GraphiCon 2009 8 Automatic Photo Selection for Media and Entertainment Applications + Good photo Bad photo True False … …
  13. 13. Detection of backlit and low-contrast photos - 1 S1/S2 - ratio of tones in shadows to midtones GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
  14. 14. S11/S12 - ratio of tones in first to second part of shadows Detection of backlit and low-contrast photos - 1 GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
  15. 15. M1/M2 - ratio of the histogram maximum in shadows to the maximum in midtones Detection of backlit and low-contrast photos - 1 GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
  16. 16. P1 - location of the histogram maximum in shadows P1 Detection of backlit and low-contrast photos - 1 GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
  17. 17. C – global contrast H 0 C 0 C 1 H 1 Detection of backlit and low-contrast photos - 1 GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
  18. 18. Training set: 480 photos Error rate on cross-validation test : ~0.055 Testing set: 1830 with 2% affected by backlit and low-contrast photos The number of False Positives (FP) is 10 The number of False Negatives (FN) is 3 Low-contrast photo Backlit photo Detection of backlit and low-contrast photos - 2 GraphiCon 2009 10 Automatic Photo Selection for Media and Entertainment Applications
  19. 19. Image Intensity image Z 1 =[-1 1] Z 2 =[-1 0 1] Z 3 =[-1 0 0 1] Z 10 =[-1 0 0 0 0 0 0 0 0 0 1] I.Safonov et al.,2008 … Edge image Histogram Normalized entropy Entropy to [0, 1] ? ? ? ? An An GraphiCon 2009 11 Detection of blurred photos Automatic Photo Selection for Media and Entertainment Applications
  20. 20. Crete et al., 2007 F.Crete et al.,2007 ? Image Blurred image Edge image Edge image Comparison of the images HPF=[1 -1] LPF=[1 1 1 1 1 1 1 1 1]/9 Detection of blurred photos GraphiCon 2009 11 Automatic Photo Selection for Media and Entertainment Applications
  21. 21. Training set: 416 photos Error rate on cross-validation test : ~0.07 Testing set: 1830 with 171 blurred photos The number of False Positives (FP) is 34 The number of False Negatives (FN) is 10 Detection of blurred photos GraphiCon 2009 11 Automatic Photo Selection for Media and Entertainment Applications
  22. 22. Time and camera-based quantization i is an index of source L is time between the least and the most time for the largest source Nps is a number of sources H = L/M M is count of images GraphiCon 2009 11 Automatic Photo Selection for Media and Entertainment Applications N region < N group N region < M Calculation of bounding boxes Partition into 2 app. equal subregions Seeking for the biggest region 1200 3600 2400 7200 0 36000 T, s 21600
  23. 23. GraphiCon 2009 12 Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection The most appealing photo is the most salient photo L.Itti, C.Koch et al. Images are taken from the Internet
  24. 24. Conspicuity maps Gaussian pyramids Image Intensity image r-channel g-channel b-channel R-channel G-channel B-channel Y-channel Orientation map Intensity map Color map Saliency map Feature maps Gabor pyramids GraphiCon 2009 13 Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection
  25. 25. original image saliency map intensity map color map orientation map ROI Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection GraphiCon 2009 14 Image is taken from the Internet
  26. 26. Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection GraphiCon 2009 15 124 88 11 100 81 92 62 83 105 70 Saliency Index
  27. 27. Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection GraphiCon 2009 15 83 11 124 Saliency Index 81 88 62 92 105 70 100
  28. 28. <ul><li>Main Disadvantages: </li></ul><ul><li>average number of FP increases a lot with picture size </li></ul>We consider, that images of people attracts more attention <ul><li>processing time also increases a lot with picture size </li></ul>Six places were detected erroneously <ul><li>Modifications: </li></ul><ul><li>image down sampling is applied at preprocessing step </li></ul><ul><li>optimization of search using color information – skin tone detection </li></ul>P.Viola, M.Jones, 2001 Automatic Photo Selection for Media and Entertainment Applications Face Detection GraphiCon 2009 16 Viola-Jones, Intel OpenCV Before modifications After modifications
  29. 29. Photos ranking Heuristic formula, experiments have shown that value w=25 gives the best result Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 17 124 88 11 116 92 118 148 95 62 100
  30. 30. Photos ranking Heuristic formula, experiments have shown that value w=25 gives the best result Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 17 118 62 124 88 11 100 116 92 148 95
  31. 31. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18
  32. 32. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18 Autocollage choice Our choice
  33. 33. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18
  34. 34. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18 Autocollage choice Our choice
  35. 35. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18
  36. 36. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18 Autocollage choice Our choice
  37. 37. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 19 Proposed AutoCollage Random 14 1 4 3 3 3 Unacceptable 21 5 2 4 5 5 Acceptable 15 4 4 3 2 2 Agree with experts 9 1 4 1 1 2 Unacceptable 24 4 0 7 7 6 Acceptable 17 5 6 2 2 2 Agree with experts 4 1 1 0 1 1 Unacceptable 17 2 4 4 4 3 Acceptable 29 7 5 6 5 6 Agree with experts Sum Set 5 Set 4 Set 3 Set 2 Set 1
  38. 38. ? Automatic Photo Selection for Media and Entertainment Applications Questions & Answers GraphiCon 2009 8
  39. 39. Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 9 Thank you for your attention =)
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×