1/26




Red Eye Removal
     y


                         Tony Meccio


       Red eye removal   17/04/2008
Overview                                                2/26



   • I t d ti
     Introduction t the Red Eye problem
    ...
The Red Eye problem                                      3/26



 The Red E
 Th R d Eye phenomenon is a well-
            ...
Why the Red Eye?                                              4/26



                        Red Eye Cone
        Eye

  ...
Red Eye prevention                                           5/26


        Eye
                           Red Eye Cone

 ...
Red Eye detection                                             6/26



    Most f th ti
    M t of the times, red eyes must...
Semi-automatic methods                            7/26


             The eyes are manually selected
             using a ...
Automatic methods                                          8/26



   Automatic methods attempt to find red eyes on
   A t...
Face-based methods                                               9/26



   •F
    Faces are looked for using a multiple f...
Eye-based methods                                             10/26



   Similar t face detection, but more complex
   Si...
Skin-based methods                                                11/26



   • Ski i d t t d first by pixel colors;
     ...
Flash-noFlash methods                                       12/26


 • Two different pictures,
 one with flash and one
   ...
An algorithm in detail                                                13/26


                                            ...
Example                                                                                  14/26




  Input Image          ...
Input/Output comparison               15/26




               Red eye removal   17/04/2008
Algorithm drawbacks                                        16/26



   • U bl t perform single red eye detection;
     Una...
Red eye correction                                        17/26



    Once red eyes have been detected, they must be
    ...
Desaturation                                             18/26




 Desaturation means lowering/zeroing the chrominance
 c...
Washed-out irises                                             19/26




     Washed-out irises                Wrong correc...
Inpainting techniques                                          20/26




 Some tools completely reconstruct the irises and...
False positives                                                21/26

  One of the biggest issue in red eye removal are fa...
Unnatural corrections                                          22/26


 • Unnatural corrections are another
   important i...
Red Eye removal examples                  23/26




             StopRedeyes®      AutoRemover®




                 Red e...
Red Eye removal examples                  24/26




            StopRedeyes®        AutoRemover®




                  Red...
Red Eye removal examples                    25/26




     StopRedeyes®            AutoRemover®



                    Red...
References                                                                                     26/26

 • J.Y. Hardeberg, “...
Upcoming SlideShare
Loading in …5
×

Red Eye Removal

2,659 views
2,471 views

Published on

A presentation by Tony Mecciio on red eye removal algorithms and a comparison of the state-of-the-art products

Published in: Education, Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,659
On SlideShare
0
From Embeds
0
Number of Embeds
75
Actions
Shares
0
Downloads
62
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Red Eye Removal

  1. 1. 1/26 Red Eye Removal y Tony Meccio Red eye removal 17/04/2008
  2. 2. Overview 2/26 • I t d ti Introduction t the Red Eye problem to th R d E bl • Red Eye prevention • Red Eye detection • Semi-automatic methods • Automatic methods •R dE Red Eye correction ti • Desaturation • Inpainting techniques • False positives and unnatural corrections a po a du au a o o • Red Eye removal examples Red eye removal 17/04/2008
  3. 3. The Red Eye problem 3/26 The Red E Th R d Eye phenomenon is a well- h i ll known problem which happens when taking flash-lighted p g g pictures of people. The pupils in the picture appear red instead of black. This h Thi happens more often when ft h using compact, consumer-oriented cameras. a a Red eye removal 17/04/2008
  4. 4. Why the Red Eye? 4/26 Red Eye Cone Eye α β • The red eye cone shines from the flashed eye back at the flash with an angle α; g ; • Its red color is caused by the reflection of the flash off the blood vessels o the retina; a o b ood of a; • The camera will record this red hue if the angle β be ee between the flash and the camera is not greater e as a d e ca e a s o g ea e than α. Red eye removal 17/04/2008
  5. 5. Red Eye prevention 5/26 Eye Red Eye Cone α β • If the equipment allows it, the flash can be spaced further away from the sensor in order to increase y β (not possible on compact devices); • One o more additional flashes before picture O or o add o a a b o p u acquisition make the iris tighten and decrease α; • This methods reduce the probability of the Red s e ods educe e p obab y o e ed Eye phenomenon but don’t remove it entirely. Red eye removal 17/04/2008
  6. 6. Red Eye detection 6/26 Most f th ti M t of the times, red eyes must be removed d tb d during post-processing. For d F red eyes to be successfully removed, they must t b f ll d th t be first detected then corrected. Methods are classified according to the detection phase: h • Semi-automatic methods ask the user to manually l ll localize th red eyes; li the d • Automatic methods detect the red eyes themselves. th l Red eye removal 17/04/2008
  7. 7. Semi-automatic methods 7/26 The eyes are manually selected using a visual interface ( g (Adobe Photoshop ®, Corel Paint Shop Pro ®, ACD See ®, etc.) Pros: • Eyes are easy to localize for y y men. Cons: • It may be difficult to have such an interface on a mobile device; • Automatic methods are easier uo a od a a to use and more appealing. Red eye removal 17/04/2008
  8. 8. Automatic methods 8/26 Automatic methods attempt to find red eyes on A t ti th d tt t t fi d d their own. The task is harder than it may seem: No “ N “perfect” R d E f t” Red Eye d t ti detection method has th d h been developed yet. Automatic methods extract features from images in A t ti th d t tf t f i i order to identify red eyes. Different methods work on different features: • Face detection • Skin detection • Eye detection • Flash-noFlash comparison Red eye removal 17/04/2008
  9. 9. Face-based methods 9/26 •F Faces are looked for using a multiple feature l k df i lti l f t object based approach; •O Once th f the face features have been found then the f t h b f d th th research is restricted to red pixels. Face Extraction Extracted Multiple Masks Area Red-Eye Red Eye Research Red eye removal 17/04/2008
  10. 10. Eye-based methods 10/26 Similar t face detection, but more complex Si il to f d t ti b t l because the features are less evident: •EEyes are seeked, matching fi d t k d t hi fixed templates at l t t different resolutions with regular eyes present into the images, or looking for red pixels using g , g p g computed color LUTs. Initial Single Eye g y Pairingg Candidate Verification Verification Detection Red eye removal 17/04/2008
  11. 11. Skin-based methods 11/26 • Ski i d t t d first by pixel colors; Skin is detected fi t b i l l • Red circular patches near the skin are then looked for. l k df This approach is simpler and does not take into account th presence of more complex features. t the f l f t Skin Detection Red Circular Pairingg Patches Verification Research Optional Red eye removal 17/04/2008
  12. 12. Flash-noFlash methods 12/26 • Two different pictures, one with flash and one ith fl h d without, are acquired one after the other;; • Red eyes are detected as patches whose color is p red in the “flash” image and black in the “nonflash” i “ fl h” image. This approach has several drawbacks: • The dimension of the buffer must double; • The two images may be mis-aligned; e o ages ay s a g ed; • The subject(s) may move between acquisitions. Red eye removal 17/04/2008
  13. 13. An algorithm in detail 13/26 Red Color Input Skin Morphological Detection in Detection Operators Skin Area Image Morphological Operators Corrected Red-Eye Pairing Single Eye Verification Verification Image g Correction The Algorithm is Skin Feature Extraction based. • First the skin is extracted and morphologically modified to blob the enhanced areas; • Then a successive red color detection is performed to find red eyes in the skin areas; • The red regions are then dilated, eroded and analyzed to identify the Red-Eyes pairs. Red eye removal 17/04/2008
  14. 14. Example 14/26 Input Image Skin Detection Morphological Red Color Detection Operators in Skin Area Morphological Pairing Corrected Operators Verification Image Red eye removal 17/04/2008
  15. 15. Input/Output comparison 15/26 Red eye removal 17/04/2008
  16. 16. Algorithm drawbacks 16/26 • U bl t perform single red eye detection; Unable to f i l d d t ti • The skin detection is performed over the whole image (slow); • Big ( g (slow) morphological operators p ) p g p permit to g get good results only on small images (less than 1 Mpixels): it would require even bigger (and slower) operators t operate on larger images. t to t l i Red eye removal 17/04/2008
  17. 17. Red eye correction 17/26 Once red eyes have been detected, they must be O d h b d t t d th tb corrected. Red eye correction is quite simpler than detection, but there are more diffi lt cases than others. b t th difficult th th Correction may vary from simple desaturation to complete reconstruction of iris and pupil. Red eye removal 17/04/2008
  18. 18. Desaturation 18/26 Desaturation means lowering/zeroing the chrominance components while mantaining the luminance component component. It is the best way to correct “easy” red eyes. Red eye removal 17/04/2008
  19. 19. Washed-out irises 19/26 Washed-out irises Wrong correction Sometimes irises are totally washed out by reflected light. In these cases a simple desaturation or color correction is not enough. It is necessary to use a more complex method to reconstruct a realistic image of the eye. Red eye removal 17/04/2008
  20. 20. Inpainting techniques 20/26 Some tools completely reconstruct the irises and the pupils to t replace the red eye (Jasc Paint Shop Pro ®). l th d (J P i t Sh P ®) The results, however, are often unrealistic and look like glass eyes. Red eye removal 17/04/2008
  21. 21. False positives 21/26 One of the biggest issue in red eye removal are false p positives in the detection phase. p Unwanted corrections are much less desirable than missing ones ones. Red eye removal 17/04/2008
  22. 22. Unnatural corrections 22/26 • Unnatural corrections are another important issue. • The most common ones are: • Partial correction: only a Partial correction portion of the red pupil has f been corrected; • Noisy correction: the presence of heavy noise or jpeg compression can introduce false red pixels around the pupil and Noisy correction thus a strange correction is made over the iris; • Wrong luminance correction: in this case the disk has been correctly found but the correction is unnatural due to Wrong luminance wrong luminance distribution. correction Red eye removal 17/04/2008
  23. 23. Red Eye removal examples 23/26 StopRedeyes® AutoRemover® Red eye removal 17/04/2008
  24. 24. Red Eye removal examples 24/26 StopRedeyes® AutoRemover® Red eye removal 17/04/2008
  25. 25. Red Eye removal examples 25/26 StopRedeyes® AutoRemover® Red eye removal 17/04/2008
  26. 26. References 26/26 • J.Y. Hardeberg, “Red Eye Removal using Digital Color Image Processing”, Conexant Systems, Inc. , Redmond, Washington, USA • M. Gaubatz, R. Ulichney, “Automatic Red-Eye Detection and Correction”, HP Lab, Cornell University, University ICIP 2002 • S. Ioffe, “Red Eye Detection With Machine Learning”, Fujifilm Software, California, ICIP 2003. • F Gasparini, R. Schettini “Automatic Redeye Removal for Smart Enhancement of Photos of F. Gasparini R Schettini, Automatic Unknown Origin”, DISCO University of Milano-Bicocca, VIS 2005. • H. Luo, J. Yen, D. Tretter, “An Efficient Redeye Detection and Correction Algorithm”, HP labs, Palo Alto, California, ICPR 2004. • A. Patti, K. Konstantinides, D. Tretter, Q. Lin, “Automatic Digital Redeye Reduction”, HP Labs, Palo Alto California, ICIP 1998. • L. Zhang, Y. Sun, M. Li, H. Zhang, “Automated Red-Eye Detection and Correction in Digital Photographs”, Photographs” Microsoft Research Asia China ICIP 2004 Asia, China, • B. Smolka, K. Czubin, J.Y. Hardeberg, K.N. Palataniotis, M. Szczepanski, K. Wojciechowski, “Towars Automatic Redeye Effect Removal”, Pattern Recognition Letter 2003. • J S Schildkraut R T Gray “A Fully Automatic Redeye Detection and Correction Algorithm”, J.S. Schildkraut, R. T. Gray, A Algorithm Kodak Company, NY, ICIP 2002. • R. Schettini, F. Gasparini, F. Chazli, ”A Modular Procedure for Automatic Redeye Correction in Digital Photos”, DISCO University of Milano-Bicocca, SPIE 2004 • PETSCHNIGG, G., AGRAWALA, M., HOPPE, H., SZELISKI, R., COHEN, M., and TOYAMA, K. 2004. Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics 23, 3 (Aug.), 664–672. Red eye removal 17/04/2008

×