The document discusses the complexities of face recognition technology including its process, benchmarks, and challenges like environmental conditions and demographic factors. It highlights various applications in law enforcement and identity verification, along with a comparison to other biometric modalities like fingerprints and iris scans. Additionally, it raises concerns about the accuracy and ethical implications of face recognition systems, emphasizing the need for ongoing research and better standards.

1. Face Recognition Benchmarks, Caveats, Comparisons and Directions Gurumurthi V. Ramanan AU-KBC RESEARCH CENTRE MIT, ANNA UNIVERSITY CHENNAI

2. Face Recognition-The Problem Given a digitised image containing the face of a person, extract the face region in the image and identify or verify the person in the image, from a database of face images.

3. Face Recognition- From Face to Biometric Template The main steps involved in converting a face image into a biometric template (vector): Feature extraction, which includes face localisation, facial feature detection and extraction A pattern recognition methodology for classifying and comparing the data that has been extracted Each of these steps has its own success rate

4. Face Recognition- From Face to Biometric Template Feature extraction involves: determination of the position of a single face (face localisation). Typical algorithm : Viola-Jones algorithm that locates the face using the statistics of skin texture. A literature survey in 1998 mentions more than 150 face detection algorithms. determination of the presence and location of features such as eyes, nose, lips (feature detection). All other features are usually located after first locating these. a pattern recognition methodology that processes the data from the above two steps. This methodology will be used to classify and compare the faces.

5. Face Recognition- Enrolment Enrolment/ Training: The system enrols an individual using digitised and good quality images containing the frontal face view. This image is converted to a biometric template using pattern recognition techniques and stored in the database. By a pattern recognition technique, we mean a mathematical model, which represents biometric templates as vectors in some high dimensional space with an inbuilt notion of similarity / distance.

6. Face Recognition- Verification, Identification & Watch list Verification (1-1): Are you who you say you are? Identification (1-many): I know you are in my database. Can I find you ? Watch list: Can I find you in my database? If I can, who are you? Ranking: The system can also be made to report the top matches from the database. These matches can be ordered by rank . The system’s most likely candidate is assigned the rank 1. A rank-5 success rate denotes the rate at which the correct identity is within the top five matches.

7. Face Recognition- The Process

8. Face Recognition-Rates The verification (identification) rate is the rate at which legitimate users are verified (identified). The false accept rate (FAR) is the rate at which impostors are verified or identified by the system. The false reject rate (FRR) is the rate at which legitimate users are wrongly verified (identified) as impostors by the system. The performance of a facial recognition system is measured by the above rates. An ideal system would have a verification rate of 100% and a false accept rate of 0%. Such systems do not exist. Good systems balance this trade-off between the verification rates and the false accept rates depending on the application context and security needs.

9. Face Recognition- Databases One of the largest publicly available face database is the FERET database. It consists of 14,051 eight-bit grayscale images of human heads of 1072 subjects, with views ranging from frontal to left and right profiles The HCInt dataset used in the FRVT 2002 test consisted of 121,589 facial images of 37,437 persons . This is not a publicly available database. This is supposed to be a part of a much larger database of 6.8 million images collected from Visa applicants at US Consular offices in a controlled environment. India does not have any such database.

10. Face Recognition-Benchmarks Verification Rates Fig. 1. Verification performance is reported for all participants on the HCInt visa dataset. Verification performance is reported at a false accept rate of 1%. (Source: FRVT 2002 )

11. Face Recognition-Benchmarks Degradation in Verification Rates due to lighting conditions Fig. 2. Verification performance is reported for five categories of frontal facial images. Performance is reported for the best system and average of the top three systems in each category. The verification rate is reported at a false accept rate 1%. (Source: FRVT 2002 )

12. Face Recognition-Benchmarks Identification Rates Fig. 3. Identification performance for the three best systems on the HCInt visa dataset. The database consisted of 37,437 persons. Identification rates are reported for ranks 1, 10, and 50. (Source: FRVT 2002 )

13. Face Recognition-Benchmarks Identification Rates due to age Fig. 4. Identification performance is reported broken out by age of a person. Each bin is labelled by the age range it contains (five year intervals). Identification rate is the average for the top three systems. Performance is on a database of 37,437. (Source: FRVT 2002 )

14. Face Recognition-Benchmarks Degradation in Identification Rates due to time Fig. 5. Identification performance is reported broken out by elapsed time between database and new image. Performance is reported in 60-day intervals. The average rank one identification rate for the top three systems is reported on a database of 37,437 persons. (Source: FRVT 2002 )

15. Misperceptions about Commercial Products: 100% match to any image at any angle Instantly recognises any person Database capabilities running to millions Reality (Research challenges): Affected by lighting, angle,size of face, quality of captured and known image Technology demonstrations done under lab / studio conditions Face Recognition-Some Caveats

16. Research done across Face Recognition (FR) algorithms indicate that they are susceptible to factors such as Race, age, gender and time. These factors are not well quantified even in the research literature. A recent study on the FERET database indicated Wearing glasses makes people more recognisable contradicting previous study Relative to the majority white faces, the Asian faces and African – American faces are significantly easier to recognise. (Total – 1072 subjects: White - 720, Asian - 143, African-American -121, other races- 88). Different FR algorithms exhibit different behavior when confronted with factors such as gender. Face Recognition- Factors

17. In the US, State Departments of Motor Vehicles (DMV) are using software developed by Visionics and Polaroid, to prevent criminals from obtaining multiple licenses under different names. Computerised identity verification is in use by 37 of the 50 American States. The Californian DMV database contains millions of images. The violence surrounding the Euro 2000 football games in the Netherlands was analyzed through CCTV cameras. FR systems recognized many individuals entering and leaving the country resulting in arrests by the Dutch authorities. http://news6.thdo.bbc.co.uk/1/hi/euro2000/teams/england/796242.stm. Face Recognition- Pilots

18. Royal Canadian Mounted Police (RCMP) is using a face-scanning camera in the cell area of Pearson Airport to match people who are known criminals or terrorists. Anser also uses Visionics’ FaceIt system as part of their project with the National Center for Missing and Exploited Children to locate missing children on the Internet. In the July 2000 presidential elections in Mexico, Visionics’ FaceIt facial recognition tools were used to build a database of 8 million voters in efforts to eliminate voter fraud. Testing and internal market research is also big business within the biometric industry. The results of many of these pilots are confidential and some are sold as IP. Face Recognition- Pilots

19. In Super Bowl in Tampa Bay, Florida, FR system was used during the game and the week prior to it. The police claimed that they uncovered 19 people with criminal records in the crowd of over 100,000 at the Super Bowl. But American Civil Liberties Union (ACLU) claimed that ‘ a study of the police logs showed that the system never correctly matched a face in its criminal database or resulted in any arrests ’. Civil liberties groups fear a ‘ function creep’ could occur i.e., biometrics could be introduced for one reason and then turned around and used for another different one. Face Recognition- Apprehensions

20. In 1893, the Home Ministry Office, UK, accepted that no two individuals have the same fingerprints leading to the development of Automatic Fingerprint Identification Systems (AFIS) in the 1960s. The current AFIS system at FBI consists of a large database of approximately 46 million ‘ten prints’ and conducts, on an average, approximately 50,000 searches per day. Lesson: It takes time for wide acceptance of a new biometric. First modern systematic use of fingerprints seems to be in India in order to prevent the rich from paying the poor to serve in the prison in their place. Face Recognition vs. Fingerprints

21. Some aspersions on fingerprints : stigma of criminality fingerprint quality dependent on subject population and collection environment. a 2004 fingerprint algorithms contest (NIST) revealed that fingerprint matching algorithms have false non-match error rate of 2%. This means a 100,000 transactions/day (typical in a high throughput environment) would result in 2,000 false rejects/day. lack of standards hampers inter-operability between proprietary but inexpensive systems. Face Recognition vs. Fingerprints

22. Verification rates of the top three FR systems vs. single fingerprint matcher (Source: FRVT 2002 Evaluation report) Face Recognition vs. Fingerprints

23. Comparative performance: At false accept rates around 0.0l, verification performance is comparable. At false accept rates below 0.01, fingerprint performance is better. At rates above 0.01, the best face recognition systems perform better. For False accepts around 0.01, that face recognition performance is now comparable to large-scale fingerprint systems available in 1998. In 2005 FRVT, FR algorithms are expected to catch up. Face Recognition vs. Fingerprints

24. Iris has low error rates but these are not verifiable due to lack of publicly available databases. This hampers research as well as verifiability. Fragility in recent pilot studies - relatively high failure to enroll rates An Iris pilot at a school reported 22% of unsuccessful transactions in a total of 9412 transactions over three months of which 16% was due to camera capture errors and 6% due to access attempts by unknown users. More research needs to be done before any meaningful comparisons can be made. Face Recognition vs. Iris

25. Face Recognition under outdoor lighting conditions Face Recognition for Surveillance with a watch-list size of at least 100 faces 3D FR algorithms FR algorithms for victim identification FR algorithms for applications such as missing children FR algorithms that measure resemblance A deeper understanding of the manifold of faces Fusion of biometrics Face Recognition- Future Directions

26. For national id and using the body as a passport, these questions are the most challenging: How to acquire repeatable and distinctive patterns from a broad population? How to accurately and efficiently represent and recognise biometric patterns? (Anil Jain) Biometrics - The Grand Challenge

27. Thank you