This presentation outlined preliminary work in testing and evaluating face recognition at Purdue University airport.

1. Facial Recognition at Purdue
University’s Airport
2003-2008
Jeremy M. Morton, C. Michael Portell, Stephen J.
Elliott, Ph.D. & Eric P. Kukula
Presented By: Eric Kukula
Biometric Standards, Performance, and Assurance Laboratory
www.biotown.purdue.edu
Department of Industrial Technology, School of Technology,
Purdue University, West Lafayette, IN 47906
© Purdue University 2006
1

2. Identifying Individuals
• There are three common ways of distinguishing
someone’s identity
– Through something that an individual knows
• A password
– Something an individual has
• Identification card
– Something they own
• Biometric
• In many airport applications, individuals gain access to
specific areas by providing a card and personal
identification number (PIN).
– A combination of these identifiers provides a more robust
security option.
• Biometric identification is defined as:
– The “automatic identification or identity verification of
(living) individuals based on behavioral and physiological
characteristics”
© Purdue University 2006
2

3. Introduction
• Physiological biometrics include:
– Facial recognition
– Finger
– Face
– Eye
– Hand
• Behavioral biometrics include:
– Speaker (voice)
– Keystroke
– Dynamic signature verification.
• Furthermore, a biometric must be:
– Measurable
– Robust
– Distinctive
• Within the research community, there is interest in the performance of a
biometric over an extended period of time.
• According to Wayman template aging is defined as:
– “the increase in error rates caused by time related changes in the biometric pattern”,
and that “longer time intervals generally make for more difficulty in matching samples
to templates”
– Furthermore, a study of template aging will also require a cohort of participants who are
available over an extended period of time.
– In this study, the time period will last up to five years.
© Purdue University 2006
3

4. Motivation
• Cross Disciplinary Research Opportunity with
Aviation Technology
• The existence of an airport facility on campus,
with a potential cohort of individuals to
participate over a five year program
– Flight students
– Faculty
– Staff
• Interdisciplinary relationships with other
departments
• Need to understand how images change over
a period of time
© Purdue University 2006
4

5. Experimental Setup
• Evaluation takes place in Hanger 6, the
Student Flight Operations Center at the airport
• Evaluation of the area completed in
conjunction with a graduate class in Biometric
Technology
• Constraints:
– Had to fit in with the existing operation of the airport
– Had to be as unobtrusive as possible
– Had to be able to collect data ongoing for a 5 year
period, therefore setup of camera was critical
– In line with the labs philosophy
• Therefore commercially available software had to
be used, with all images stored
© Purdue University 2006
5

6. Experimental Setup
© Purdue University 2006
6

7. Experimental Setup
• The face recognition system was setup in the Hanger 6
area with no additional environmental controls.
• A Logitech QuickCam Pro 4000 was used. This camera
has a video resolution of 640x480 VGA CCD, with up to
30 frames per second.
• The camera also has a still image capture of up to 1280
x 960 pixels, 1.3 mega pixels.
• The video camera was connected to Dell Omniplex
GX260 2.0 GHZ, 512 MB RAM computers through a 35ft
cable with USB boosters.
• The camera was 6 inches off the table. The angle of the
camera also accommodated all of the participants,
regardless of their height. As lighting was constant, it
was not measured continually.
© Purdue University 2006
7

8. Volunteer Crew
• Recruitment of the students occurs at the
flight operations safety briefings
– Held once per month.
• It is anticipated that over 300 students will be
enrolled over the duration of the test.
• Currently in Cohorts 1 and 2
© Purdue University 2006
8

9. Cohorts
• Cohorts are used to describe the groupings of
students
– Currently 2 cohorts
• So far there are 2 cohort groups
– Group 1
• 71 individuals
– 60 males
– 11 females
– Enrolled in a 4 day period in May 2003
– Group 2
• 56 individuals
– 54 male
– 2 female
© Purdue University 2006
9

10. Enrollment
• Each enrollment required an individual to stand in front
of the camera, and move their head to the left, right, up
and down.
• 100 images were taken for each enrollment, an
enrollment taking approximately 1.5 minutes
• The experimenters enrolled people in batches, so later
enrollees were more habituated to the enrollment
process than those at the beginning of the enrollment
period.
• It was noticed that sometimes, in a batch enrollment,
people would speed up their movement, causing the
FRS system to loose track of the individual
– This was the only significant issue with enrollment
• There was 0% Failure to Enroll (FTE)
• There was also a 0% Failure to Acquire (defined as
successfully acquiring 100 images).
© Purdue University 2006
10

11. Initial Results from the Study
• As the data is collected in a 1:M classification
mode, ie: there is no claim to identity, each
image collected by the system has to be
manually verified against its original template
• The system returns two different variables
– Classify
– Classify Failure
© Purdue University 2006
11

12. Initial Results
• Classify
– The classification rate in a 1:M scenario is currently
between 70-73%
– We are now seeing a decline in the classification
success rate and an increase in the classification
failure rate due to the following reasons:
• It is a 1:M system, so the user is not presenting
their token or password to be verified as a 1:1
match.
– Therefore, when an individual walks by the camera,
the system takes an image, and compares it to all
known individuals in the database
• However, there are lessons to be learned
regarding the testing and evaluation of biometric
equipment in operational tests
© Purdue University 2006
12

13. Issues with 1:M operational testing
• The result of a classification failure in a 1:M
situation may be an understatement of the
true performance of the FRS system due to:
– A False Match
– A Failure to Acquire
– An unknown person
© Purdue University 2006
13

14. Issues with Operational Testing
• The false non-match rate used in this
evaluation is defined as the error rate of the
matching algorithm from a single attempt-
template comparison in a genuine attempt.
• This rate will be established by the research
team using additional software currently being
developed
© Purdue University 2006
14

15. Issues with Operational Testing
• Failure to Acquire
– The failure to acquire rate is the proportion of
attempts for which the system is unable to capture
or locate the user’s face
– In this situation, as the camera is in a high traffic
area, the camera cannot find a face, therefore times
out
– The failure to acquire rate is counted as a
classification failure
© Purdue University 2006
15

16. Issues with Operational Testing
• Unknowns
– An unknown is also a classification failure. An
unknown is defined for this study as someone who
has not yet been entered into the system
– The FRS captures an image, tries to match that
image with all those enrolled in the database, yet
fails to find one
– It then returns classification failure
© Purdue University 2006
16

17. Issues and Conclusions with Operational Testing and the
Reporting of Results
• The issue with operational testing is that the
commercially available software (COTS) used in this
study overstates the classification failure rate due to the
issues discussed:
– False Match
– Failure to Acquire
– Unknowns
• Research is currently underway at the Biometrics
Standards, Performance, and Assurance Laboratory to
resolve these issues, and provide a comprehensive best
practice testing and reporting guide for 1:M applications
© Purdue University 2006
17