Automatic Photo Selection For Media And Entertainment Applications

  • 698 views
Uploaded on

 

More in: Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
698
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
8
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Automatic Photo Selection for Media and Entertainment Applications Ekaterina Potapova, Marta Egorova, Ilia Safonov National Nuclear Research University MEPhI Moscow, Russia GraphiCon 2009 5-9 October
  • 2. Applications Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 2
  • 3. Applications Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 2
  • 4. Applications – photo book Images are taken from printbook.ru, ehow.com, snapfish.com.au, smilebooks.co.uk GraphiCon 2009 3 Automatic Photo Selection for Media and Entertainment Applications
  • 5. Applications – slide show Photos from ITaS’2008 GraphiCon 2009 4 Automatic Photo Selection for Media and Entertainment Applications
  • 6. General workflow GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications
  • 7. GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications General workflow Detection of low-quality photos
  • 8. General workflow GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications Detection of low-quality photos Adaptive quantization on time-camera plane
  • 9. General workflow GraphiCon 2009 5 Automatic Photo Selection for Media and Entertainment Applications Selection of appealing photos Detection of low-quality photos Adaptive quantization on time-camera plane
  • 10. Detection of low-quality photos GraphiCon 2009 6 Automatic Photo Selection for Media and Entertainment Applications
  • 11. Estimation of JPEG quality A.Foi et al.,2007 Images are taken from en.wikipedia.org Quantization Table GraphiCon 2009 7 Automatic Photo Selection for Media and Entertainment Applications
  • 12. Detection of backlit, low-contrast & blurred photos Two Ada Boost classifiers committee: -for detection of low-contrast and backlit photos -for detection of blurred photos GraphiCon 2009 8 Automatic Photo Selection for Media and Entertainment Applications + Good photo Bad photo True False … …
  • 13. Detection of backlit and low-contrast photos - 1 S1/S2 - ratio of tones in shadows to midtones GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
  • 14. S11/S12 - ratio of tones in first to second part of shadows Detection of backlit and low-contrast photos - 1 GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
  • 15. M1/M2 - ratio of the histogram maximum in shadows to the maximum in midtones Detection of backlit and low-contrast photos - 1 GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
  • 16. P1 - location of the histogram maximum in shadows P1 Detection of backlit and low-contrast photos - 1 GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
  • 17. C – global contrast H 0 C 0 C 1 H 1 Detection of backlit and low-contrast photos - 1 GraphiCon 2009 9 Automatic Photo Selection for Media and Entertainment Applications
  • 18. Training set: 480 photos Error rate on cross-validation test : ~0.055 Testing set: 1830 with 2% affected by backlit and low-contrast photos The number of False Positives (FP) is 10 The number of False Negatives (FN) is 3 Low-contrast photo Backlit photo Detection of backlit and low-contrast photos - 2 GraphiCon 2009 10 Automatic Photo Selection for Media and Entertainment Applications
  • 19. Image Intensity image Z 1 =[-1 1] Z 2 =[-1 0 1] Z 3 =[-1 0 0 1] Z 10 =[-1 0 0 0 0 0 0 0 0 0 1] I.Safonov et al.,2008 … Edge image Histogram Normalized entropy Entropy to [0, 1] ? ? ? ? An An GraphiCon 2009 11 Detection of blurred photos Automatic Photo Selection for Media and Entertainment Applications
  • 20. Crete et al., 2007 F.Crete et al.,2007 ? Image Blurred image Edge image Edge image Comparison of the images HPF=[1 -1] LPF=[1 1 1 1 1 1 1 1 1]/9 Detection of blurred photos GraphiCon 2009 11 Automatic Photo Selection for Media and Entertainment Applications
  • 21. Training set: 416 photos Error rate on cross-validation test : ~0.07 Testing set: 1830 with 171 blurred photos The number of False Positives (FP) is 34 The number of False Negatives (FN) is 10 Detection of blurred photos GraphiCon 2009 11 Automatic Photo Selection for Media and Entertainment Applications
  • 22. Time and camera-based quantization i is an index of source L is time between the least and the most time for the largest source Nps is a number of sources H = L/M M is count of images GraphiCon 2009 11 Automatic Photo Selection for Media and Entertainment Applications N region < N group N region < M Calculation of bounding boxes Partition into 2 app. equal subregions Seeking for the biggest region 1200 3600 2400 7200 0 36000 T, s 21600
  • 23. GraphiCon 2009 12 Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection The most appealing photo is the most salient photo L.Itti, C.Koch et al. Images are taken from the Internet
  • 24. Conspicuity maps Gaussian pyramids Image Intensity image r-channel g-channel b-channel R-channel G-channel B-channel Y-channel Orientation map Intensity map Color map Saliency map Feature maps Gabor pyramids GraphiCon 2009 13 Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection
  • 25. original image saliency map intensity map color map orientation map ROI Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection GraphiCon 2009 14 Image is taken from the Internet
  • 26. Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection GraphiCon 2009 15 124 88 11 100 81 92 62 83 105 70 Saliency Index
  • 27. Automatic Photo Selection for Media and Entertainment Applications Salient Photo Selection GraphiCon 2009 15 83 11 124 Saliency Index 81 88 62 92 105 70 100
  • 28.
    • Main Disadvantages:
    • average number of FP increases a lot with picture size
    We consider, that images of people attracts more attention
    • processing time also increases a lot with picture size
    Six places were detected erroneously
    • Modifications:
    • image down sampling is applied at preprocessing step
    • optimization of search using color information – skin tone detection
    P.Viola, M.Jones, 2001 Automatic Photo Selection for Media and Entertainment Applications Face Detection GraphiCon 2009 16 Viola-Jones, Intel OpenCV Before modifications After modifications
  • 29. Photos ranking Heuristic formula, experiments have shown that value w=25 gives the best result Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 17 124 88 11 116 92 118 148 95 62 100
  • 30. Photos ranking Heuristic formula, experiments have shown that value w=25 gives the best result Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 17 118 62 124 88 11 100 116 92 148 95
  • 31. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18
  • 32. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18 Autocollage choice Our choice
  • 33. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18
  • 34. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18 Autocollage choice Our choice
  • 35. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18
  • 36. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 18 Autocollage choice Our choice
  • 37. Automatic Photo Selection for Media and Entertainment Applications Results and discussion GraphiCon 2009 19 Proposed AutoCollage Random 14 1 4 3 3 3 Unacceptable 21 5 2 4 5 5 Acceptable 15 4 4 3 2 2 Agree with experts 9 1 4 1 1 2 Unacceptable 24 4 0 7 7 6 Acceptable 17 5 6 2 2 2 Agree with experts 4 1 1 0 1 1 Unacceptable 17 2 4 4 4 3 Acceptable 29 7 5 6 5 6 Agree with experts Sum Set 5 Set 4 Set 3 Set 2 Set 1
  • 38. ? Automatic Photo Selection for Media and Entertainment Applications Questions & Answers GraphiCon 2009 8
  • 39. Automatic Photo Selection for Media and Entertainment Applications GraphiCon 2009 9 Thank you for your attention =)