• Like
Red Eye Removal
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Red Eye Removal


A presentation by Tony Mecciio on red eye removal algorithms and a comparison of the state-of-the-art products

A presentation by Tony Mecciio on red eye removal algorithms and a comparison of the state-of-the-art products

Published in Education , Health & Medicine
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On SlideShare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. 1/26 Red Eye Removal y Tony Meccio Red eye removal 17/04/2008
  • 2. Overview 2/26 • I t d ti Introduction t the Red Eye problem to th R d E bl • Red Eye prevention • Red Eye detection • Semi-automatic methods • Automatic methods •R dE Red Eye correction ti • Desaturation • Inpainting techniques • False positives and unnatural corrections a po a du au a o o • Red Eye removal examples Red eye removal 17/04/2008
  • 3. The Red Eye problem 3/26 The Red E Th R d Eye phenomenon is a well- h i ll known problem which happens when taking flash-lighted p g g pictures of people. The pupils in the picture appear red instead of black. This h Thi happens more often when ft h using compact, consumer-oriented cameras. a a Red eye removal 17/04/2008
  • 4. Why the Red Eye? 4/26 Red Eye Cone Eye α β • The red eye cone shines from the flashed eye back at the flash with an angle α; g ; • Its red color is caused by the reflection of the flash off the blood vessels o the retina; a o b ood of a; • The camera will record this red hue if the angle β be ee between the flash and the camera is not greater e as a d e ca e a s o g ea e than α. Red eye removal 17/04/2008
  • 5. Red Eye prevention 5/26 Eye Red Eye Cone α β • If the equipment allows it, the flash can be spaced further away from the sensor in order to increase y β (not possible on compact devices); • One o more additional flashes before picture O or o add o a a b o p u acquisition make the iris tighten and decrease α; • This methods reduce the probability of the Red s e ods educe e p obab y o e ed Eye phenomenon but don’t remove it entirely. Red eye removal 17/04/2008
  • 6. Red Eye detection 6/26 Most f th ti M t of the times, red eyes must be removed d tb d during post-processing. For d F red eyes to be successfully removed, they must t b f ll d th t be first detected then corrected. Methods are classified according to the detection phase: h • Semi-automatic methods ask the user to manually l ll localize th red eyes; li the d • Automatic methods detect the red eyes themselves. th l Red eye removal 17/04/2008
  • 7. Semi-automatic methods 7/26 The eyes are manually selected using a visual interface ( g (Adobe Photoshop ®, Corel Paint Shop Pro ®, ACD See ®, etc.) Pros: • Eyes are easy to localize for y y men. Cons: • It may be difficult to have such an interface on a mobile device; • Automatic methods are easier uo a od a a to use and more appealing. Red eye removal 17/04/2008
  • 8. Automatic methods 8/26 Automatic methods attempt to find red eyes on A t ti th d tt t t fi d d their own. The task is harder than it may seem: No “ N “perfect” R d E f t” Red Eye d t ti detection method has th d h been developed yet. Automatic methods extract features from images in A t ti th d t tf t f i i order to identify red eyes. Different methods work on different features: • Face detection • Skin detection • Eye detection • Flash-noFlash comparison Red eye removal 17/04/2008
  • 9. Face-based methods 9/26 •F Faces are looked for using a multiple feature l k df i lti l f t object based approach; •O Once th f the face features have been found then the f t h b f d th th research is restricted to red pixels. Face Extraction Extracted Multiple Masks Area Red-Eye Red Eye Research Red eye removal 17/04/2008
  • 10. Eye-based methods 10/26 Similar t face detection, but more complex Si il to f d t ti b t l because the features are less evident: •EEyes are seeked, matching fi d t k d t hi fixed templates at l t t different resolutions with regular eyes present into the images, or looking for red pixels using g , g p g computed color LUTs. Initial Single Eye g y Pairingg Candidate Verification Verification Detection Red eye removal 17/04/2008
  • 11. Skin-based methods 11/26 • Ski i d t t d first by pixel colors; Skin is detected fi t b i l l • Red circular patches near the skin are then looked for. l k df This approach is simpler and does not take into account th presence of more complex features. t the f l f t Skin Detection Red Circular Pairingg Patches Verification Research Optional Red eye removal 17/04/2008
  • 12. Flash-noFlash methods 12/26 • Two different pictures, one with flash and one ith fl h d without, are acquired one after the other;; • Red eyes are detected as patches whose color is p red in the “flash” image and black in the “nonflash” i “ fl h” image. This approach has several drawbacks: • The dimension of the buffer must double; • The two images may be mis-aligned; e o ages ay s a g ed; • The subject(s) may move between acquisitions. Red eye removal 17/04/2008
  • 13. An algorithm in detail 13/26 Red Color Input Skin Morphological Detection in Detection Operators Skin Area Image Morphological Operators Corrected Red-Eye Pairing Single Eye Verification Verification Image g Correction The Algorithm is Skin Feature Extraction based. • First the skin is extracted and morphologically modified to blob the enhanced areas; • Then a successive red color detection is performed to find red eyes in the skin areas; • The red regions are then dilated, eroded and analyzed to identify the Red-Eyes pairs. Red eye removal 17/04/2008
  • 14. Example 14/26 Input Image Skin Detection Morphological Red Color Detection Operators in Skin Area Morphological Pairing Corrected Operators Verification Image Red eye removal 17/04/2008
  • 15. Input/Output comparison 15/26 Red eye removal 17/04/2008
  • 16. Algorithm drawbacks 16/26 • U bl t perform single red eye detection; Unable to f i l d d t ti • The skin detection is performed over the whole image (slow); • Big ( g (slow) morphological operators p ) p g p permit to g get good results only on small images (less than 1 Mpixels): it would require even bigger (and slower) operators t operate on larger images. t to t l i Red eye removal 17/04/2008
  • 17. Red eye correction 17/26 Once red eyes have been detected, they must be O d h b d t t d th tb corrected. Red eye correction is quite simpler than detection, but there are more diffi lt cases than others. b t th difficult th th Correction may vary from simple desaturation to complete reconstruction of iris and pupil. Red eye removal 17/04/2008
  • 18. Desaturation 18/26 Desaturation means lowering/zeroing the chrominance components while mantaining the luminance component component. It is the best way to correct “easy” red eyes. Red eye removal 17/04/2008
  • 19. Washed-out irises 19/26 Washed-out irises Wrong correction Sometimes irises are totally washed out by reflected light. In these cases a simple desaturation or color correction is not enough. It is necessary to use a more complex method to reconstruct a realistic image of the eye. Red eye removal 17/04/2008
  • 20. Inpainting techniques 20/26 Some tools completely reconstruct the irises and the pupils to t replace the red eye (Jasc Paint Shop Pro ®). l th d (J P i t Sh P ®) The results, however, are often unrealistic and look like glass eyes. Red eye removal 17/04/2008
  • 21. False positives 21/26 One of the biggest issue in red eye removal are false p positives in the detection phase. p Unwanted corrections are much less desirable than missing ones ones. Red eye removal 17/04/2008
  • 22. Unnatural corrections 22/26 • Unnatural corrections are another important issue. • The most common ones are: • Partial correction: only a Partial correction portion of the red pupil has f been corrected; • Noisy correction: the presence of heavy noise or jpeg compression can introduce false red pixels around the pupil and Noisy correction thus a strange correction is made over the iris; • Wrong luminance correction: in this case the disk has been correctly found but the correction is unnatural due to Wrong luminance wrong luminance distribution. correction Red eye removal 17/04/2008
  • 23. Red Eye removal examples 23/26 StopRedeyes® AutoRemover® Red eye removal 17/04/2008
  • 24. Red Eye removal examples 24/26 StopRedeyes® AutoRemover® Red eye removal 17/04/2008
  • 25. Red Eye removal examples 25/26 StopRedeyes® AutoRemover® Red eye removal 17/04/2008
  • 26. References 26/26 • J.Y. Hardeberg, “Red Eye Removal using Digital Color Image Processing”, Conexant Systems, Inc. , Redmond, Washington, USA • M. Gaubatz, R. Ulichney, “Automatic Red-Eye Detection and Correction”, HP Lab, Cornell University, University ICIP 2002 • S. Ioffe, “Red Eye Detection With Machine Learning”, Fujifilm Software, California, ICIP 2003. • F Gasparini, R. Schettini “Automatic Redeye Removal for Smart Enhancement of Photos of F. Gasparini R Schettini, Automatic Unknown Origin”, DISCO University of Milano-Bicocca, VIS 2005. • H. Luo, J. Yen, D. Tretter, “An Efficient Redeye Detection and Correction Algorithm”, HP labs, Palo Alto, California, ICPR 2004. • A. Patti, K. Konstantinides, D. Tretter, Q. Lin, “Automatic Digital Redeye Reduction”, HP Labs, Palo Alto California, ICIP 1998. • L. Zhang, Y. Sun, M. Li, H. Zhang, “Automated Red-Eye Detection and Correction in Digital Photographs”, Photographs” Microsoft Research Asia China ICIP 2004 Asia, China, • B. Smolka, K. Czubin, J.Y. Hardeberg, K.N. Palataniotis, M. Szczepanski, K. Wojciechowski, “Towars Automatic Redeye Effect Removal”, Pattern Recognition Letter 2003. • J S Schildkraut R T Gray “A Fully Automatic Redeye Detection and Correction Algorithm”, J.S. Schildkraut, R. T. Gray, A Algorithm Kodak Company, NY, ICIP 2002. • R. Schettini, F. Gasparini, F. Chazli, ”A Modular Procedure for Automatic Redeye Correction in Digital Photos”, DISCO University of Milano-Bicocca, SPIE 2004 • PETSCHNIGG, G., AGRAWALA, M., HOPPE, H., SZELISKI, R., COHEN, M., and TOYAMA, K. 2004. Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics 23, 3 (Aug.), 664–672. Red eye removal 17/04/2008