Automatic Photo Selection For Media And Entertainment Applications
1. Automatic Photo Selection for Media and Entertainment Applications Ekaterina Potapova, Marta Egorova, Ilia Safonov National Nuclear Research University MEPhI Moscow, Russia GraphiCon 2009 5-9 October
4. Applications – photo book Images are taken from printbook.ru, ehow.com, snapfish.com.au, smilebooks.co.uk GraphiCon 2009 3 Automatic Photo Selection for Media and Entertainment Applications
5. Applications – slide show Photos from ITaS’2008 GraphiCon 2009 4 Automatic Photo Selection for Media and Entertainment Applications
7. GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications General workflow Detection of low-quality photos
8. General workflow GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications Detection of low-quality photos Adaptive quantization on time-camera plane
9. General workflow GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications Selection of appealing photos Detection of low-quality photos Adaptive quantization on time-camera plane
10. Detection of low-quality photos GraphiCon 2009 6 Automatic Photo Selection for Media and Entertainment Applications
11. Estimation of JPEG quality A.Foi et al.,2007 Images are taken from en.wikipedia.org Quantization Table GraphiCon 2009 7 Automatic Photo Selection for Media and Entertainment Applications
12. Detection of backlit, low-contrast & blurred photos Two Ada Boost classifiers committee: -for detection of low-contrast and backlit photos -for detection of blurred photos GraphiCon 2009 8 Automatic Photo Selection for Media and Entertainment Applications + Good photo Bad photo True False … …
13. Detection of backlit and low-contrast photos - 1 S1/S2 - ratio of tones in shadows to midtones GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
14. S11/S12 - ratio of tones in first to second part of shadows Detection of backlit and low-contrast photos - 1 GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
15. M1/M2 - ratio of the histogram maximum in shadows to the maximum in midtones Detection of backlit and low-contrast photos - 1 GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
16. P1 - location of the histogram maximum in shadows P1 Detection of backlit and low-contrast photos - 1 GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
17. C – global contrast H 0 C 0 C 1 H 1 Detection of backlit and low-contrast photos - 1 GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
18. Training set: 480 photos Error rate on cross-validation test : ~0.055 Testing set: 1830 with 2% affected by backlit and low-contrast photos The number of False Positives (FP) is 10 The number of False Negatives (FN) is 3 Low-contrast photo Backlit photo Detection of backlit and low-contrast photos - 2 GraphiCon 2009 10 Automatic Photo Selection for Media and Entertainment Applications
19. Image Intensity image Z 1 =[-1 1] Z 2 =[-1 0 1] Z 3 =[-1 0 0 1] Z 10 =[-1 0 0 0 0 0 0 0 0 0 1] I.Safonov et al.,2008 … Edge image Histogram Normalized entropy Entropy to [0, 1] ? ? ? ? An An GraphiCon 2009 11 Detection of blurred photos Automatic Photo Selection for Media and Entertainment Applications
20. Crete et al., 2007 F.Crete et al.,2007 ? Image Blurred image Edge image Edge image Comparison of the images HPF=[1 -1] LPF=[1 1 1 1 1 1 1 1 1]/9 Detection of blurred photos GraphiCon 2009 11 Automatic Photo Selection for Media and Entertainment Applications
21. Training set: 416 photos Error rate on cross-validation test : ~0.07 Testing set: 1830 with 171 blurred photos The number of False Positives (FP) is 34 The number of False Negatives (FN) is 10 Detection of blurred photos GraphiCon 2009 11 Automatic Photo Selection for Media and Entertainment Applications
22. Time and camera-based quantization i is an index of source L is time between the least and the most time for the largest source Nps is a number of sources H = L/M M is count of images GraphiCon 2009 11 Automatic Photo Selection for Media and Entertainment Applications N region < N group N region < M Calculation of bounding boxes Partition into 2 app. equal subregions Seeking for the biggest region 1200 3600 2400 7200 0 36000 T, s 21600
23. GraphiCon 2009 12 Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection The most appealing photo is the most salient photo L.Itti, C.Koch et al. Images are taken from the Internet
24. Conspicuity maps Gaussian pyramids Image Intensity image r-channel g-channel b-channel R-channel G-channel B-channel Y-channel Orientation map Intensity map Color map Saliency map Feature maps Gabor pyramids GraphiCon 2009 13 Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection
25. original image saliency map intensity map color map orientation map ROI Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection GraphiCon 2009 14 Image is taken from the Internet
26. Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection GraphiCon 2009 15 124 88 11 100 81 92 62 83 105 70 Saliency Index
27. Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection GraphiCon 2009 15 83 11 124 Saliency Index 81 88 62 92 105 70 100
28.
29. Photos ranking Heuristic formula, experiments have shown that value w=25 gives the best result Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 17 124 88 11 116 92 118 148 95 62 100
30. Photos ranking Heuristic formula, experiments have shown that value w=25 gives the best result Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 17 118 62 124 88 11 100 116 92 148 95
31. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18
32. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18 Autocollage choice Our choice
33. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18
34. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18 Autocollage choice Our choice
35. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18
36. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18 Autocollage choice Our choice
37. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 19 Proposed AutoCollage Random 14 1 4 3 3 3 Unacceptable 21 5 2 4 5 5 Acceptable 15 4 4 3 2 2 Agree with experts 9 1 4 1 1 2 Unacceptable 24 4 0 7 7 6 Acceptable 17 5 6 2 2 2 Agree with experts 4 1 1 0 1 1 Unacceptable 17 2 4 4 4 3 Acceptable 29 7 5 6 5 6 Agree with experts Sum Set 5 Set 4 Set 3 Set 2 Set 1
38. ? Automatic Photo Selection for Media and Entertainment Applications Questions & Answers GraphiCon 2009 8
39. Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 9 Thank you for your attention =)
