SlideShare a Scribd company logo

Face recognition vendor test 2006 face recognition vendor test 2006 experiment 4 covariate study experiment 4 covariate study

•
•
S
Sungkwan Park

Face recognition vendor test 2006 face recognition vendor test 2006 experiment 4 covariate study experiment 4 covariate study

Technology
1 of 41
Download to read offline
Face Recognition Vendor Test 2006 Face Recognition Vendor Test 2006 Experiment 4 Covariate Study Experiment 4 Covariate Study Dr. J. Ross Beveridge Dr. J. Ross Beveridge Dr. Dr. Geof Geof H. Givens H. Givens Dr. Bruce Draper Dr. Bruce Draper Mr. Mr. Yui Yui Man Man Lui Lui Colorado State University Colorado State University Dr. P. Dr. P. Jonathon Phillips Jonathon Phillips National Institute of Standards and Technology National Institute of Standards and Technology The work was funded in part by the Technical Support Working Group (TSWG) under Task T-1840C.
Motivation Motivation Factors that influence face recognition Factors that influence face recognition
4/21/2008 3 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation - - Attributes of People Attributes of People What makes recognition harder/easier? What makes recognition harder/easier? Young Young … …
4/21/2008 4 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation - - Attributes of People Attributes of People Gender? Gender? Age Age
4/21/2008 5 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation - - Attributes of People Attributes of People Race? Race? Age Age Gender Gender
4/21/2008 6 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation Smile? - - Smile? Expression? Expression? Age Age Gender Gender Race Race
4/21/2008 7 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation - - Environment Environment Control: Mugshot vs. posed indoor or outdoor Control: Mugshot vs. posed indoor or outdoor Age Age Gender Gender Race Race Expression Expression
4/21/2008 8 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation - - Glasses Glasses Age Age Gender Gender Race Race Expression Expression Glasses in uncontrolled imagery. Glasses in uncontrolled imagery. Uncontrolled Uncontrolled
4/21/2008 9 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation - Recap - Recap Age Age Gender Race Expression Glasses Uncontrolled … Gender Race Expression Glasses Uncontrolled …
4/21/2008 10 FRVT 2006 Experiment 4 Covariate Analysis But Wait, There But Wait, There’ ’s More, Quality s More, Quality   You cannot do much about You cannot do much about   Gender, Age, Race, Gender, Age, Race, … …   Some control over Some control over   Setting, Glasses, Expression, Setting, Glasses, Expression, … …   What about measurable image properties? What about measurable image properties?   Resolution, Focus, Resolution, Focus, … … ISO SC 37 "Biometrics” - Factors Affecting Face Image Quality Imaging ACQUISITION PROCESS AND CAPTURE DEVICE PROPERTIES … 2. physical properties (e.g. resolution and contrast) ISO SC 37 "Biometrics ISO SC 37 "Biometrics” ” - - Factors Affecting Face Image Quality Imaging Factors Affecting Face Image Quality Imaging ACQUISITION PROCESS AND CAPTURE DEVICE PROPERTIES ACQUISITION PROCESS AND CAPTURE DEVICE PROPERTIES … … 2. physical properties (e.g. resolution and contrast) 2. physical properties (e.g. resolution and contrast)
Covariate Analysis Covariate Analysis 4/21/2008 11 FRVT 2006 Experiment 4 Covariate Analysis
4/21/2008 12 FRVT 2006 Experiment 4 Covariate Analysis For Analysis We Need For Analysis We Need … …   Lots of Performance Data Lots of Performance Data   FRVT 2006 FRVT 2006   Methodology Methodology   Generalized Linear Mixed Effect Model Generalized Linear Mixed Effect Model   Specific Problem Specific Problem   Uncontrolled frontal still against mugshot gallery Uncontrolled frontal still against mugshot gallery
4/21/2008 13 FRVT 2006 Experiment 4 Covariate Analysis Introduction Introduction - - More on FRVT More on FRVT 22 participants, 10 countries 22 participants, 22 participants, 10 countries 10 countries China China Denmark Denmark Germany Germany Israel Israel Japan Japan Romania Romania Spain Spain South South Korea Korea United United Kingdom Kingdom United United States States 10 US, 7 foreign, 5 with offices in and outside the US 10 US, 7 foreign, 5 with offices in and outside the US Executing Agency
4/21/2008 14 FRVT 2006 Experiment 4 Covariate Analysis Introduction Introduction - - Progress Progress For controlled frontal still images For controlled frontal still images 2006 - Falsely turn away 1/100 people, when only admitting 1/1000 imposters. 2006 2006 - - Falsely turn away 1/100 people, Falsely turn away 1/100 people, when only admitting 1/1000 imposters. when only admitting 1/1000 imposters.
4/21/2008 15 FRVT 2006 Experiment 4 Covariate Analysis Our Focus Our Focus - - Uncontrolled Stills Uncontrolled Stills
Turn Away 20/100 (at 1/1,000 FAR) FRVT 2002 Performance (Controlled vs Controlled) 4/21/2008 16 FRVT 2006 Experiment 4 Covariate Analysis Uncontrolled to Controlled Still Uncontrolled to Controlled Still 2006 2006 - - 2006 - Falsely turn away 10/100 to 40/100 people, when only admitting 1/1000 impostors. Falsely turn away 10/100 to 40/100 people, Falsely turn away 10/100 to 40/100 people, when only admitting 1/1000 impostors. when only admitting 1/1000 impostors.
4/21/2008 17 FRVT 2006 Experiment 4 Covariate Analysis FRVT Covariate Analysis FRVT Covariate Analysis   Algorithm Algorithm - - score fusion of 3 top performers. score fusion of 3 top performers.   Uncontrolled match to Controlled. Imagery Imagery - - Uncontrolled match to Controlled.   Subset of FRVT 2006 Experiment 4 Subset of FRVT 2006 Experiment 4   345 subjects and 345 subjects and 110,514 110,514 match scores. match scores.
4/21/2008 18 FRVT 2006 Experiment 4 Covariate Analysis Performance Variable Performance Variable   Verification Outcome: Success / Failure Verification Outcome: Success / Failure Verified Verified FAR FAR Gender Gender Race Race … … Yes Yes 1/100 1/100 Female Female Asian Asian … … Verified Verified FAR FAR Gender Gender Race Race … … No No 1/1,000 1/1,000 Male Male Asian Asian … … Verified Verified FAR FAR Gender Gender Race Race … … Yes Yes 1/10,000 1/10,000 Male Male White White … … * Outcomes for illustration purposes only. Levels 1/100 1/1,000 & 1/10,000 Levels 1/100 1/1,000 & 1/10,000 FAR is a factor FAR is a factor Distributed across pairs. Distributed across pairs. There are 110,514 pairs like this! There are 110,514 pairs like this! There are 110,514 pairs like this!
4/21/2008 19 FRVT 2006 Experiment 4 Covariate Analysis Face Region In Focus Measure Face Region In Focus Measure FRIFM: Sum of FRIFM: Sum of Sobel Sobel edge magnitude inside an edge magnitude inside an ellipse bounding the face. ellipse bounding the face. “Active Computer Vision by Cooperative Focus and Stereo” by Eric Krotkov.
4/21/2008 20 FRVT 2006 Experiment 4 Covariate Analysis Face Region In Focus Measure Face Region In Focus Measure Low FRIFM examples High FRIFM examples
Fitting a Statistical Model Fitting a Statistical Model 4/21/2008 21 FRVT 2006 Experiment 4 Covariate Analysis Generalized Linear Mixed Model Generalized Linear Generalized Linear Mixed Model Mixed Model … … Verification Outcomes Verification Outcomes Age Age Gender Gender Race Race Glasses Glasses Focus Focus Head Tilt Head Tilt Resolution Resolution FAR FAR
Using the Statistical Model Using the Statistical Model 4/21/2008 22 FRVT 2006 Experiment 4 Covariate Analysis Fitted Generalized Linear Fitted Generalized Linear Mixed Model Fitted Generalized Linear Mixed Model Mixed Model Age Age Gender Gender Race Race Glasses Glasses Focus Focus Head Tilt Head Tilt Resolution Resolution FAR FAR P(success | covariates)
4/21/2008 23 FRVT 2006 Experiment 4 Covariate Analysis   Let Let A A and and B B be 2 covariates that might influence be 2 covariates that might influence algorithm performance. For example, A=gender algorithm performance. For example, A=gender (categorical) and B=Query (categorical) and B=Query- -Eye Eye- -Distance (continuous). Distance (continuous).   Let a index levels of A. Let a index levels of A.   Let j index the FAR setting, Let j index the FAR setting, α αj j   Y Ypabj pabj is is   1 if Person 1 if Person p p is verified correctly, 0 otherwise. is verified correctly, 0 otherwise.   Y Ypabj pabj depends on: depends on:   person person p p, covariates , covariates A A and and B B, and , and   false alarm rate false alarm rate α αj j. . Analysis is: Mixed Effects Logistic Regression with Repeated Measures on People. Generalized Linear Mixed Model Generalized Linear Mixed Model
4/21/2008 24 FRVT 2006 Experiment 4 Covariate Analysis GLMM Model Continued GLMM Model Continued … …
4/21/2008 25 FRVT 2006 Experiment 4 Covariate Analysis The outcomes, i. e. verification success/failure, are uncorrelated when testing different people but correlated when testing the same person under different configurations. The outcomes, i. e. verification success/failure, are The outcomes, i. e. verification success/failure, are uncorrelated when testing different people but uncorrelated when testing different people but correlated when testing the same person under correlated when testing the same person under different configurations. different configurations. This means: This means: The Mixed in Generalized Linear The Mixed in Generalized Linear Mixed Mixed effect Model. effect Model. Subject Variation Subject Variation
Vendor Test Covariate Analysis Findings Vendor Test Covariate Analysis Findings From the highly expected From the highly expected … … … … to the unexpected. to the unexpected.
4/21/2008 27 FRVT 2006 Experiment 4 Covariate Analysis Finding 1: False Accept Rate Finding 1: False Accept Rate Solid = Indoors Dashed= Outdoors
4/21/2008 28 FRVT 2006 Experiment 4 Covariate Analysis Finding 2: Gender Finding 2: Gender Solid = Indoors Dashed= Outdoors
4/21/2008 29 FRVT 2006 Experiment 4 Covariate Analysis Finding 3: Race Finding 3: Race Solid = Indoors Dashed= Outdoors
4/21/2008 30 FRVT 2006 Experiment 4 Covariate Analysis Finding 4: Glasses Finding 4: Glasses Solid = Indoors Dashed= Outdoors
4/21/2008 31 FRVT 2006 Experiment 4 Covariate Analysis Finding 5: Finding 5: Small Medium Large
Small Medium Large Size of query image (distance between eyes) 4/21/2008 32 FRVT 2006 Experiment 4 Covariate Analysis Finding 5: Finding 5:
Query environment 4/21/2008 33 FRVT 2006 Experiment 4 Covariate Analysis Finding 5: Finding 5: Small Medium Large
Boundary of observed data 4/21/2008 34 FRVT 2006 Experiment 4 Covariate Analysis Finding 5: Finding 5: Small Medium Large
Large PV range ~0.90 − ~0.10 4/21/2008 35 FRVT 2006 Experiment 4 Covariate Analysis Finding 5: Finding 5: Small Medium Large
Low FRIFM good; even for one image 4/21/2008 36 FRVT 2006 Experiment 4 Covariate Analysis Finding 5: Finding 5: Small Medium Large
4/21/2008 37 FRVT 2006 Experiment 4 Covariate Analysis FRIFM Conclusion FRIFM Conclusion   Large performance variation. Large performance variation.   Indoors [>0.95, ~.0.70] Indoors [>0.95, ~.0.70]   Outdoors [~0.90, ~0.10]. Outdoors [~0.90, ~0.10].   Interaction between covariates Interaction between covariates   Environments (indoors, outdoors) Environments (indoors, outdoors)   Query image size Query image size   Target and query FRIFM Target and query FRIFM   Low FRIFM good Low FRIFM good   Effect if control for only one image Effect if control for only one image   Outdoors: query size very important Outdoors: query size very important
4/21/2008 38 FRVT 2006 Experiment 4 Covariate Analysis FRIFM Conclusion FRIFM Conclusion   According to this analysis According to this analysis   Out of focus is higher quality Out of focus is higher quality   Remember, edge density surrogate for focus Remember, edge density surrogate for focus   Is this really quality, Is this really quality, … …   Or other environmental factors, Or other environmental factors, … …   Or algorithm aberration? Or algorithm aberration?
4/21/2008 39 FRVT 2006 Experiment 4 Covariate Analysis GLMM and Quality Standards GLMM and Quality Standards Factors Affecting Face Image Quality Character RICHNESS OF IDENTIFYING CHARACTERISTIC Š BIOLOGICAL CHARACTERS Behavior SPOOFING Imaging ACQUISITION PROCESS AND CAPTURE DEVICE PROPERTIES Environment AMBIENT CONDITION FACE 1. anatomical characteristic (e.g. head dimensions, eye position) 2. injuries and scars 3. ethnic group 4. impairment 5. Heavy facial wears, such as thick or dark glasses 1. closed eyes 2. (exaggerated) expression 3. hair across the eye 4. head pose 5. makeup 6. subject posing (frontal / non- frontal to camera) 1. image enhancement and data reduction process 2. physical properties (e.g. resolution and contrast) 3. optical distortions 4. static properties of the background (e.g. wallpaper) 5. camera characteristics • sensor resolution 6. scene characteristics • geometric distortion 1. dynamic characteristics of the background like moving objects 2. variation in lighting and relate potential defects as • deviation from the symmetric lighting • uneven lighting on the face area • extreme strong or weak illumination 3. subject posing, e.g.: • too far (face too small), or too near (face too big) • out of focus (low sharpness) • partial occlusion of the face From Covariates to Quality Measures …. From Covariates to Quality Measures From Covariates to Quality Measures … …. .
4/21/2008 40 FRVT 2006 Experiment 4 Covariate Analysis Conclusion Conclusion   Quality is NOT in the eyes of the beholder Quality is NOT in the eyes of the beholder   It is in the performance numbers It is in the performance numbers   Model quantifies performance change. Model quantifies performance change.   Turn the knob. Turn the knob.   Read off the change in performance. Read off the change in performance.   Interaction between covariates. Interaction between covariates.   Tells us where to put our efforts Tells us where to put our efforts   Indoors it is FRIFM. Indoors it is FRIFM.   Outdoors it is Query Image Size. Outdoors it is Query Image Size.   These models are used in other fields. These models are used in other fields.   e.g., Biomedical. e.g., Biomedical.   Studies of Biometrics should use them. Studies of Biometrics should use them.
Thank You Thank You

Recommended

Policy as a Concept, Procedures and Case Study
Policy as a Concept, Procedures and Case StudyPolicy as a Concept, Procedures and Case Study
Policy as a Concept, Procedures and Case StudyDeddy Supriady Bratakusumah
 
Policy as a Concept, Procedures and Case Study
Policy as a Concept, Procedures and Case StudyPolicy as a Concept, Procedures and Case Study
Policy as a Concept, Procedures and Case StudyDeddy Supriady Bratakusumah
 
Abdm4064 week 09 10 sampling
Abdm4064 week 09 10 samplingAbdm4064 week 09 10 sampling
Abdm4064 week 09 10 samplingStephen Ong
 
Genetic purity testing
Genetic purity testingGenetic purity testing
Genetic purity testingsunil kumar
 
Quantitative research methods in medicine dr. baxi
Quantitative research methods in medicine dr. baxiQuantitative research methods in medicine dr. baxi
Quantitative research methods in medicine dr. baxiIndian Health Journal
 
Risk Model Methodologies
Risk Model MethodologiesRisk Model Methodologies
Risk Model Methodologiesyamanote
 
Use of Biostatics in Dentistry /certified fixed orthodontic courses by Indian...
Use of Biostatics in Dentistry /certified fixed orthodontic courses by Indian...Use of Biostatics in Dentistry /certified fixed orthodontic courses by Indian...
Use of Biostatics in Dentistry /certified fixed orthodontic courses by Indian...Indian dental academy
 
Blue ocean strategy
Blue ocean strategyBlue ocean strategy
Blue ocean strategyEngSabreen Doghmosh
 

Recommended

Policy as a Concept, Procedures and Case Study
Policy as a Concept, Procedures and Case StudyPolicy as a Concept, Procedures and Case Study
Policy as a Concept, Procedures and Case StudyDeddy Supriady Bratakusumah
 
Policy as a Concept, Procedures and Case Study
Policy as a Concept, Procedures and Case StudyPolicy as a Concept, Procedures and Case Study
Policy as a Concept, Procedures and Case StudyDeddy Supriady Bratakusumah
 
Abdm4064 week 09 10 sampling
Abdm4064 week 09 10 samplingAbdm4064 week 09 10 sampling
Abdm4064 week 09 10 samplingStephen Ong
 
Genetic purity testing
Genetic purity testingGenetic purity testing
Genetic purity testingsunil kumar
 
Quantitative research methods in medicine dr. baxi
Quantitative research methods in medicine dr. baxiQuantitative research methods in medicine dr. baxi
Quantitative research methods in medicine dr. baxiIndian Health Journal
 
Risk Model Methodologies
Risk Model MethodologiesRisk Model Methodologies
Risk Model Methodologiesyamanote
 
Use of Biostatics in Dentistry /certified fixed orthodontic courses by Indian...
Use of Biostatics in Dentistry /certified fixed orthodontic courses by Indian...Use of Biostatics in Dentistry /certified fixed orthodontic courses by Indian...
Use of Biostatics in Dentistry /certified fixed orthodontic courses by Indian...Indian dental academy
 
Blue ocean strategy
Blue ocean strategyBlue ocean strategy
Blue ocean strategyEngSabreen Doghmosh
 
Blue ocean strategy
Blue ocean strategyBlue ocean strategy
Blue ocean strategyEngSabreen Doghmosh
 
Identification1
Identification1Identification1
Identification1veesingh
 
What can Community Challenges do for You?
What can Community Challenges do for You?What can Community Challenges do for You?
What can Community Challenges do for You?Iddo
 
bgfact_stat.ppt
bgfact_stat.pptbgfact_stat.ppt
bgfact_stat.pptdeepika563208
 
Data Safety Monitoring Boards in Pediatric Clinical Trials
Data Safety Monitoring Boards in Pediatric Clinical TrialsData Safety Monitoring Boards in Pediatric Clinical Trials
Data Safety Monitoring Boards in Pediatric Clinical TrialsCarlo Carandang
 
Mba2216 week 09 10 sampling
Mba2216 week 09 10 samplingMba2216 week 09 10 sampling
Mba2216 week 09 10 samplingStephen Ong
 
NG BB 33 Hypothesis Testing Basics
NG BB 33 Hypothesis Testing BasicsNG BB 33 Hypothesis Testing Basics
NG BB 33 Hypothesis Testing BasicsLeanleaders.org
 
NG BB 33 Hypothesis Testing Basics
NG BB 33 Hypothesis Testing BasicsNG BB 33 Hypothesis Testing Basics
NG BB 33 Hypothesis Testing BasicsLeanleaders.org
 
Experimental Research
Experimental Research Experimental Research
Experimental Research ShadzDhan
 
Chap017
Chap017Chap017
Chap017Dhamo daran
 
Improvement 7 tools
Improvement 7 toolsImprovement 7 tools
Improvement 7 toolsAhmed Nady
 
Behavior of Indian Investor: A Market Research
Behavior of Indian Investor: A Market ResearchBehavior of Indian Investor: A Market Research
Behavior of Indian Investor: A Market ResearchAkash Jauhari
 
Causal comparative study
Causal comparative studyCausal comparative study
Causal comparative studywawaaa789
 
Economics of screening for atherosclerosis final
Economics of screening for atherosclerosis finalEconomics of screening for atherosclerosis final
Economics of screening for atherosclerosis finalSociety for Heart Attack Prevention and Eradication
 
ASSIGNMENT INFORMATION1. Due DateMonday, July 17, 20171159 PM.docx
ASSIGNMENT INFORMATION1. Due DateMonday, July 17, 20171159 PM.docxASSIGNMENT INFORMATION1. Due DateMonday, July 17, 20171159 PM.docx
ASSIGNMENT INFORMATION1. Due DateMonday, July 17, 20171159 PM.docxrock73
 
The ASA president Task Force Statement on Statistical Significance and Replic...
The ASA president Task Force Statement on Statistical Significance and Replic...The ASA president Task Force Statement on Statistical Significance and Replic...
The ASA president Task Force Statement on Statistical Significance and Replic...jemille6
 
Sergio Destefanis - The OECD Diagnostic Tool in Italy: Relevance, Determining...
Sergio Destefanis - The OECD Diagnostic Tool in Italy: Relevance, Determining...Sergio Destefanis - The OECD Diagnostic Tool in Italy: Relevance, Determining...
Sergio Destefanis - The OECD Diagnostic Tool in Italy: Relevance, Determining...OECD CFE
 
Alexis Diamond - quasi experiments
Alexis Diamond - quasi experimentsAlexis Diamond - quasi experiments
Alexis Diamond - quasi experimentsMicrofinance Gateway
 
1. F A Using S P S S1 (Saq.Sav) Q Ti A
1. F A Using S P S S1 (Saq.Sav) Q Ti A1. F A Using S P S S1 (Saq.Sav) Q Ti A
1. F A Using S P S S1 (Saq.Sav) Q Ti AZoha Qureshi
 
Face recognition vendor test 2002 technical appendices
Face recognition vendor test 2002 technical appendicesFace recognition vendor test 2002 technical appendices
Face recognition vendor test 2002 technical appendicesSungkwan Park
 
Face recognition vendor test 2002 evaluation report
Face recognition vendor test 2002 evaluation reportFace recognition vendor test 2002 evaluation report
Face recognition vendor test 2002 evaluation reportSungkwan Park
 
Face recognition vendor test 2002 evaluation report
Face recognition vendor test 2002 evaluation reportFace recognition vendor test 2002 evaluation report
Face recognition vendor test 2002 evaluation reportSungkwan Park
 

More Related Content

Similar to Face recognition vendor test 2006 face recognition vendor test 2006 experiment 4 covariate study experiment 4 covariate study

Identification1
Identification1Identification1
Identification1veesingh
 
What can Community Challenges do for You?
What can Community Challenges do for You?What can Community Challenges do for You?
What can Community Challenges do for You?Iddo
 
Data Safety Monitoring Boards in Pediatric Clinical Trials
Data Safety Monitoring Boards in Pediatric Clinical TrialsData Safety Monitoring Boards in Pediatric Clinical Trials
Data Safety Monitoring Boards in Pediatric Clinical TrialsCarlo Carandang
 
Mba2216 week 09 10 sampling
Mba2216 week 09 10 samplingMba2216 week 09 10 sampling
Mba2216 week 09 10 samplingStephen Ong
 
NG BB 33 Hypothesis Testing Basics
NG BB 33 Hypothesis Testing BasicsNG BB 33 Hypothesis Testing Basics
NG BB 33 Hypothesis Testing BasicsLeanleaders.org
 
NG BB 33 Hypothesis Testing Basics
NG BB 33 Hypothesis Testing BasicsNG BB 33 Hypothesis Testing Basics
NG BB 33 Hypothesis Testing BasicsLeanleaders.org
 
Experimental Research
Experimental Research Experimental Research
Experimental Research ShadzDhan
 
Improvement 7 tools
Improvement 7 toolsImprovement 7 tools
Improvement 7 toolsAhmed Nady
 
Behavior of Indian Investor: A Market Research
Behavior of Indian Investor: A Market ResearchBehavior of Indian Investor: A Market Research
Behavior of Indian Investor: A Market ResearchAkash Jauhari
 
Causal comparative study
Causal comparative studyCausal comparative study
Causal comparative studywawaaa789
 
ASSIGNMENT INFORMATION1. Due DateMonday, July 17, 20171159 PM.docx
ASSIGNMENT INFORMATION1. Due DateMonday, July 17, 20171159 PM.docxASSIGNMENT INFORMATION1. Due DateMonday, July 17, 20171159 PM.docx
ASSIGNMENT INFORMATION1. Due DateMonday, July 17, 20171159 PM.docxrock73
 
The ASA president Task Force Statement on Statistical Significance and Replic...
The ASA president Task Force Statement on Statistical Significance and Replic...The ASA president Task Force Statement on Statistical Significance and Replic...
The ASA president Task Force Statement on Statistical Significance and Replic...jemille6
 
Sergio Destefanis - The OECD Diagnostic Tool in Italy: Relevance, Determining...
Sergio Destefanis - The OECD Diagnostic Tool in Italy: Relevance, Determining...Sergio Destefanis - The OECD Diagnostic Tool in Italy: Relevance, Determining...
Sergio Destefanis - The OECD Diagnostic Tool in Italy: Relevance, Determining...OECD CFE
 
Alexis Diamond - quasi experiments
Alexis Diamond - quasi experimentsAlexis Diamond - quasi experiments
Alexis Diamond - quasi experimentsMicrofinance Gateway
 
1. F A Using S P S S1 (Saq.Sav) Q Ti A
1. F A Using S P S S1 (Saq.Sav) Q Ti A1. F A Using S P S S1 (Saq.Sav) Q Ti A
1. F A Using S P S S1 (Saq.Sav) Q Ti AZoha Qureshi
 

Similar to Face recognition vendor test 2006 face recognition vendor test 2006 experiment 4 covariate study experiment 4 covariate study (19)

Blue ocean strategy
Blue ocean strategyBlue ocean strategy
Blue ocean strategy
 
Identification1
Identification1Identification1
Identification1
 
What can Community Challenges do for You?
What can Community Challenges do for You?What can Community Challenges do for You?
What can Community Challenges do for You?
 
bgfact_stat.ppt
bgfact_stat.pptbgfact_stat.ppt
bgfact_stat.ppt
 
Data Safety Monitoring Boards in Pediatric Clinical Trials
Data Safety Monitoring Boards in Pediatric Clinical TrialsData Safety Monitoring Boards in Pediatric Clinical Trials
Data Safety Monitoring Boards in Pediatric Clinical Trials
 
Mba2216 week 09 10 sampling
Mba2216 week 09 10 samplingMba2216 week 09 10 sampling
Mba2216 week 09 10 sampling
 
NG BB 33 Hypothesis Testing Basics
NG BB 33 Hypothesis Testing BasicsNG BB 33 Hypothesis Testing Basics
NG BB 33 Hypothesis Testing Basics
 
NG BB 33 Hypothesis Testing Basics
NG BB 33 Hypothesis Testing BasicsNG BB 33 Hypothesis Testing Basics
NG BB 33 Hypothesis Testing Basics
 
Experimental Research
Experimental Research Experimental Research
Experimental Research
 
Chap017
Chap017Chap017
Chap017
 
Improvement 7 tools
Improvement 7 toolsImprovement 7 tools
Improvement 7 tools
 
Behavior of Indian Investor: A Market Research
Behavior of Indian Investor: A Market ResearchBehavior of Indian Investor: A Market Research
Behavior of Indian Investor: A Market Research
 
Causal comparative study
Causal comparative studyCausal comparative study
Causal comparative study
 
Economics of screening for atherosclerosis final
Economics of screening for atherosclerosis finalEconomics of screening for atherosclerosis final
Economics of screening for atherosclerosis final
 
ASSIGNMENT INFORMATION1. Due DateMonday, July 17, 20171159 PM.docx
ASSIGNMENT INFORMATION1. Due DateMonday, July 17, 20171159 PM.docxASSIGNMENT INFORMATION1. Due DateMonday, July 17, 20171159 PM.docx
ASSIGNMENT INFORMATION1. Due DateMonday, July 17, 20171159 PM.docx
 
The ASA president Task Force Statement on Statistical Significance and Replic...
The ASA president Task Force Statement on Statistical Significance and Replic...The ASA president Task Force Statement on Statistical Significance and Replic...
The ASA president Task Force Statement on Statistical Significance and Replic...
 
Sergio Destefanis - The OECD Diagnostic Tool in Italy: Relevance, Determining...
Sergio Destefanis - The OECD Diagnostic Tool in Italy: Relevance, Determining...Sergio Destefanis - The OECD Diagnostic Tool in Italy: Relevance, Determining...
Sergio Destefanis - The OECD Diagnostic Tool in Italy: Relevance, Determining...
 
Alexis Diamond - quasi experiments
Alexis Diamond - quasi experimentsAlexis Diamond - quasi experiments
Alexis Diamond - quasi experiments
 
1. F A Using S P S S1 (Saq.Sav) Q Ti A
1. F A Using S P S S1 (Saq.Sav) Q Ti A1. F A Using S P S S1 (Saq.Sav) Q Ti A
1. F A Using S P S S1 (Saq.Sav) Q Ti A
 

More from Sungkwan Park

Face recognition vendor test 2002 technical appendices
Face recognition vendor test 2002 technical appendicesFace recognition vendor test 2002 technical appendices
Face recognition vendor test 2002 technical appendicesSungkwan Park
 
Face recognition vendor test 2002 evaluation report
Face recognition vendor test 2002 evaluation reportFace recognition vendor test 2002 evaluation report
Face recognition vendor test 2002 evaluation reportSungkwan Park
 
Face recognition vendor test 2002 evaluation report
Face recognition vendor test 2002 evaluation reportFace recognition vendor test 2002 evaluation report
Face recognition vendor test 2002 evaluation reportSungkwan Park
 
Face in video evaluation (five)
Face in video evaluation (five)Face in video evaluation (five)
Face in video evaluation (five)Sungkwan Park
 
Face recognition vendor test 2002 supplemental report
Face recognition vendor test 2002 supplemental reportFace recognition vendor test 2002 supplemental report
Face recognition vendor test 2002 supplemental reportSungkwan Park
 
생체신호 인증 알고리즘 성능 지침
생체신호 인증 알고리즘 성능 지침 생체신호 인증 알고리즘 성능 지침
생체신호 인증 알고리즘 성능 지침Sungkwan Park
 
A rendering architecture
A rendering architectureA rendering architecture
A rendering architectureSungkwan Park
 

More from Sungkwan Park (7)

Face recognition vendor test 2002 technical appendices
Face recognition vendor test 2002 technical appendicesFace recognition vendor test 2002 technical appendices
Face recognition vendor test 2002 technical appendices
 
Face recognition vendor test 2002 evaluation report
Face recognition vendor test 2002 evaluation reportFace recognition vendor test 2002 evaluation report
Face recognition vendor test 2002 evaluation report
 
Face recognition vendor test 2002 evaluation report
Face recognition vendor test 2002 evaluation reportFace recognition vendor test 2002 evaluation report
Face recognition vendor test 2002 evaluation report
 
Face in video evaluation (five)
Face in video evaluation (five)Face in video evaluation (five)
Face in video evaluation (five)
 
Face recognition vendor test 2002 supplemental report
Face recognition vendor test 2002 supplemental reportFace recognition vendor test 2002 supplemental report
Face recognition vendor test 2002 supplemental report
 
생체신호 인증 알고리즘 성능 지침
생체신호 인증 알고리즘 성능 지침 생체신호 인증 알고리즘 성능 지침
생체신호 인증 알고리즘 성능 지침
 
A rendering architecture
A rendering architectureA rendering architecture
A rendering architecture
 

Recently uploaded

How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonetsnaman860154
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024The Digital Insurer
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUK Journal
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdfhans926745
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking MenDelhi Call girls
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Miguel Araújo
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Servicegiselly40
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsMaria Levchenko
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?Antenna Manufacturer Coco
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Scriptwesley chun
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 

Recently uploaded (20)

How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdfUnderstanding Discord NSFW Servers A Guide for Responsible Users.pdf
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf[2024]Digital Global Overview Report 2024 Meltwater.pdf
[2024]Digital Global Overview Report 2024 Meltwater.pdf
 
08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men08448380779 Call Girls In Friends Colony Women Seeking Men
08448380779 Call Girls In Friends Colony Women Seeking Men
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law DevelopmentsTrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
TrustArc Webinar - Stay Ahead of US State Data Privacy Law Developments
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Handwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed textsHandwritten Text Recognition for manuscripts and early printed texts
Handwritten Text Recognition for manuscripts and early printed texts
 
What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?What Are The Drone Anti-jamming Systems Technology?
What Are The Drone Anti-jamming Systems Technology?
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘🐬 The future of MySQL is Postgres 🐘
🐬 The future of MySQL is Postgres 🐘
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
Automating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps ScriptAutomating Google Workspace (GWS) & more with Apps Script
Automating Google Workspace (GWS) & more with Apps Script
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 

Face recognition vendor test 2006 face recognition vendor test 2006 experiment 4 covariate study experiment 4 covariate study

  • 1. Face Recognition Vendor Test 2006 Face Recognition Vendor Test 2006 Experiment 4 Covariate Study Experiment 4 Covariate Study Dr. J. Ross Beveridge Dr. J. Ross Beveridge Dr. Dr. Geof Geof H. Givens H. Givens Dr. Bruce Draper Dr. Bruce Draper Mr. Mr. Yui Yui Man Man Lui Lui Colorado State University Colorado State University Dr. P. Dr. P. Jonathon Phillips Jonathon Phillips National Institute of Standards and Technology National Institute of Standards and Technology The work was funded in part by the Technical Support Working Group (TSWG) under Task T-1840C.
  • 2. Motivation Motivation Factors that influence face recognition Factors that influence face recognition
  • 3. 4/21/2008 3 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation - - Attributes of People Attributes of People What makes recognition harder/easier? What makes recognition harder/easier? Young Young … …
  • 4. 4/21/2008 4 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation - - Attributes of People Attributes of People Gender? Gender? Age Age
  • 5. 4/21/2008 5 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation - - Attributes of People Attributes of People Race? Race? Age Age Gender Gender
  • 6. 4/21/2008 6 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation Smile? - - Smile? Expression? Expression? Age Age Gender Gender Race Race
  • 7. 4/21/2008 7 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation - - Environment Environment Control: Mugshot vs. posed indoor or outdoor Control: Mugshot vs. posed indoor or outdoor Age Age Gender Gender Race Race Expression Expression
  • 8. 4/21/2008 8 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation - - Glasses Glasses Age Age Gender Gender Race Race Expression Expression Glasses in uncontrolled imagery. Glasses in uncontrolled imagery. Uncontrolled Uncontrolled
  • 9. 4/21/2008 9 FRVT 2006 Experiment 4 Covariate Analysis Motivation Motivation - Recap - Recap Age Age Gender Race Expression Glasses Uncontrolled … Gender Race Expression Glasses Uncontrolled …
  • 10. 4/21/2008 10 FRVT 2006 Experiment 4 Covariate Analysis But Wait, There But Wait, There’ ’s More, Quality s More, Quality   You cannot do much about You cannot do much about   Gender, Age, Race, Gender, Age, Race, … …   Some control over Some control over   Setting, Glasses, Expression, Setting, Glasses, Expression, … …   What about measurable image properties? What about measurable image properties?   Resolution, Focus, Resolution, Focus, … … ISO SC 37 "Biometrics” - Factors Affecting Face Image Quality Imaging ACQUISITION PROCESS AND CAPTURE DEVICE PROPERTIES … 2. physical properties (e.g. resolution and contrast) ISO SC 37 "Biometrics ISO SC 37 "Biometrics” ” - - Factors Affecting Face Image Quality Imaging Factors Affecting Face Image Quality Imaging ACQUISITION PROCESS AND CAPTURE DEVICE PROPERTIES ACQUISITION PROCESS AND CAPTURE DEVICE PROPERTIES … … 2. physical properties (e.g. resolution and contrast) 2. physical properties (e.g. resolution and contrast)
  • 11. Covariate Analysis Covariate Analysis 4/21/2008 11 FRVT 2006 Experiment 4 Covariate Analysis
  • 12. 4/21/2008 12 FRVT 2006 Experiment 4 Covariate Analysis For Analysis We Need For Analysis We Need … …   Lots of Performance Data Lots of Performance Data   FRVT 2006 FRVT 2006   Methodology Methodology   Generalized Linear Mixed Effect Model Generalized Linear Mixed Effect Model   Specific Problem Specific Problem   Uncontrolled frontal still against mugshot gallery Uncontrolled frontal still against mugshot gallery
  • 13. 4/21/2008 13 FRVT 2006 Experiment 4 Covariate Analysis Introduction Introduction - - More on FRVT More on FRVT 22 participants, 10 countries 22 participants, 22 participants, 10 countries 10 countries China China Denmark Denmark Germany Germany Israel Israel Japan Japan Romania Romania Spain Spain South South Korea Korea United United Kingdom Kingdom United United States States 10 US, 7 foreign, 5 with offices in and outside the US 10 US, 7 foreign, 5 with offices in and outside the US Executing Agency
  • 14. 4/21/2008 14 FRVT 2006 Experiment 4 Covariate Analysis Introduction Introduction - - Progress Progress For controlled frontal still images For controlled frontal still images 2006 - Falsely turn away 1/100 people, when only admitting 1/1000 imposters. 2006 2006 - - Falsely turn away 1/100 people, Falsely turn away 1/100 people, when only admitting 1/1000 imposters. when only admitting 1/1000 imposters.
  • 15. 4/21/2008 15 FRVT 2006 Experiment 4 Covariate Analysis Our Focus Our Focus - - Uncontrolled Stills Uncontrolled Stills
  • 16. Turn Away 20/100 (at 1/1,000 FAR) FRVT 2002 Performance (Controlled vs Controlled) 4/21/2008 16 FRVT 2006 Experiment 4 Covariate Analysis Uncontrolled to Controlled Still Uncontrolled to Controlled Still 2006 2006 - - 2006 - Falsely turn away 10/100 to 40/100 people, when only admitting 1/1000 impostors. Falsely turn away 10/100 to 40/100 people, Falsely turn away 10/100 to 40/100 people, when only admitting 1/1000 impostors. when only admitting 1/1000 impostors.
  • 17. 4/21/2008 17 FRVT 2006 Experiment 4 Covariate Analysis FRVT Covariate Analysis FRVT Covariate Analysis   Algorithm Algorithm - - score fusion of 3 top performers. score fusion of 3 top performers.   Uncontrolled match to Controlled. Imagery Imagery - - Uncontrolled match to Controlled.   Subset of FRVT 2006 Experiment 4 Subset of FRVT 2006 Experiment 4   345 subjects and 345 subjects and 110,514 110,514 match scores. match scores.
  • 18. 4/21/2008 18 FRVT 2006 Experiment 4 Covariate Analysis Performance Variable Performance Variable   Verification Outcome: Success / Failure Verification Outcome: Success / Failure Verified Verified FAR FAR Gender Gender Race Race … … Yes Yes 1/100 1/100 Female Female Asian Asian … … Verified Verified FAR FAR Gender Gender Race Race … … No No 1/1,000 1/1,000 Male Male Asian Asian … … Verified Verified FAR FAR Gender Gender Race Race … … Yes Yes 1/10,000 1/10,000 Male Male White White … … * Outcomes for illustration purposes only. Levels 1/100 1/1,000 & 1/10,000 Levels 1/100 1/1,000 & 1/10,000 FAR is a factor FAR is a factor Distributed across pairs. Distributed across pairs. There are 110,514 pairs like this! There are 110,514 pairs like this! There are 110,514 pairs like this!
  • 19. 4/21/2008 19 FRVT 2006 Experiment 4 Covariate Analysis Face Region In Focus Measure Face Region In Focus Measure FRIFM: Sum of FRIFM: Sum of Sobel Sobel edge magnitude inside an edge magnitude inside an ellipse bounding the face. ellipse bounding the face. “Active Computer Vision by Cooperative Focus and Stereo” by Eric Krotkov.
  • 20. 4/21/2008 20 FRVT 2006 Experiment 4 Covariate Analysis Face Region In Focus Measure Face Region In Focus Measure Low FRIFM examples High FRIFM examples
  • 21. Fitting a Statistical Model Fitting a Statistical Model 4/21/2008 21 FRVT 2006 Experiment 4 Covariate Analysis Generalized Linear Mixed Model Generalized Linear Generalized Linear Mixed Model Mixed Model … … Verification Outcomes Verification Outcomes Age Age Gender Gender Race Race Glasses Glasses Focus Focus Head Tilt Head Tilt Resolution Resolution FAR FAR
  • 22. Using the Statistical Model Using the Statistical Model 4/21/2008 22 FRVT 2006 Experiment 4 Covariate Analysis Fitted Generalized Linear Fitted Generalized Linear Mixed Model Fitted Generalized Linear Mixed Model Mixed Model Age Age Gender Gender Race Race Glasses Glasses Focus Focus Head Tilt Head Tilt Resolution Resolution FAR FAR P(success | covariates)
  • 23. 4/21/2008 23 FRVT 2006 Experiment 4 Covariate Analysis   Let Let A A and and B B be 2 covariates that might influence be 2 covariates that might influence algorithm performance. For example, A=gender algorithm performance. For example, A=gender (categorical) and B=Query (categorical) and B=Query- -Eye Eye- -Distance (continuous). Distance (continuous).   Let a index levels of A. Let a index levels of A.   Let j index the FAR setting, Let j index the FAR setting, α αj j   Y Ypabj pabj is is   1 if Person 1 if Person p p is verified correctly, 0 otherwise. is verified correctly, 0 otherwise.   Y Ypabj pabj depends on: depends on:   person person p p, covariates , covariates A A and and B B, and , and   false alarm rate false alarm rate α αj j. . Analysis is: Mixed Effects Logistic Regression with Repeated Measures on People. Generalized Linear Mixed Model Generalized Linear Mixed Model
  • 24. 4/21/2008 24 FRVT 2006 Experiment 4 Covariate Analysis GLMM Model Continued GLMM Model Continued … …
  • 25. 4/21/2008 25 FRVT 2006 Experiment 4 Covariate Analysis The outcomes, i. e. verification success/failure, are uncorrelated when testing different people but correlated when testing the same person under different configurations. The outcomes, i. e. verification success/failure, are The outcomes, i. e. verification success/failure, are uncorrelated when testing different people but uncorrelated when testing different people but correlated when testing the same person under correlated when testing the same person under different configurations. different configurations. This means: This means: The Mixed in Generalized Linear The Mixed in Generalized Linear Mixed Mixed effect Model. effect Model. Subject Variation Subject Variation
  • 26. Vendor Test Covariate Analysis Findings Vendor Test Covariate Analysis Findings From the highly expected From the highly expected … … … … to the unexpected. to the unexpected.
  • 27. 4/21/2008 27 FRVT 2006 Experiment 4 Covariate Analysis Finding 1: False Accept Rate Finding 1: False Accept Rate Solid = Indoors Dashed= Outdoors
  • 28. 4/21/2008 28 FRVT 2006 Experiment 4 Covariate Analysis Finding 2: Gender Finding 2: Gender Solid = Indoors Dashed= Outdoors
  • 29. 4/21/2008 29 FRVT 2006 Experiment 4 Covariate Analysis Finding 3: Race Finding 3: Race Solid = Indoors Dashed= Outdoors
  • 30. 4/21/2008 30 FRVT 2006 Experiment 4 Covariate Analysis Finding 4: Glasses Finding 4: Glasses Solid = Indoors Dashed= Outdoors
  • 31. 4/21/2008 31 FRVT 2006 Experiment 4 Covariate Analysis Finding 5: Finding 5: Small Medium Large
  • 32. Small Medium Large Size of query image (distance between eyes) 4/21/2008 32 FRVT 2006 Experiment 4 Covariate Analysis Finding 5: Finding 5:
  • 33. Query environment 4/21/2008 33 FRVT 2006 Experiment 4 Covariate Analysis Finding 5: Finding 5: Small Medium Large
  • 34. Boundary of observed data 4/21/2008 34 FRVT 2006 Experiment 4 Covariate Analysis Finding 5: Finding 5: Small Medium Large
  • 35. Large PV range ~0.90 − ~0.10 4/21/2008 35 FRVT 2006 Experiment 4 Covariate Analysis Finding 5: Finding 5: Small Medium Large
  • 36. Low FRIFM good; even for one image 4/21/2008 36 FRVT 2006 Experiment 4 Covariate Analysis Finding 5: Finding 5: Small Medium Large
  • 37. 4/21/2008 37 FRVT 2006 Experiment 4 Covariate Analysis FRIFM Conclusion FRIFM Conclusion   Large performance variation. Large performance variation.   Indoors [>0.95, ~.0.70] Indoors [>0.95, ~.0.70]   Outdoors [~0.90, ~0.10]. Outdoors [~0.90, ~0.10].   Interaction between covariates Interaction between covariates   Environments (indoors, outdoors) Environments (indoors, outdoors)   Query image size Query image size   Target and query FRIFM Target and query FRIFM   Low FRIFM good Low FRIFM good   Effect if control for only one image Effect if control for only one image   Outdoors: query size very important Outdoors: query size very important
  • 38. 4/21/2008 38 FRVT 2006 Experiment 4 Covariate Analysis FRIFM Conclusion FRIFM Conclusion   According to this analysis According to this analysis   Out of focus is higher quality Out of focus is higher quality   Remember, edge density surrogate for focus Remember, edge density surrogate for focus   Is this really quality, Is this really quality, … …   Or other environmental factors, Or other environmental factors, … …   Or algorithm aberration? Or algorithm aberration?
  • 39. 4/21/2008 39 FRVT 2006 Experiment 4 Covariate Analysis GLMM and Quality Standards GLMM and Quality Standards Factors Affecting Face Image Quality Character RICHNESS OF IDENTIFYING CHARACTERISTIC Š BIOLOGICAL CHARACTERS Behavior SPOOFING Imaging ACQUISITION PROCESS AND CAPTURE DEVICE PROPERTIES Environment AMBIENT CONDITION FACE 1. anatomical characteristic (e.g. head dimensions, eye position) 2. injuries and scars 3. ethnic group 4. impairment 5. Heavy facial wears, such as thick or dark glasses 1. closed eyes 2. (exaggerated) expression 3. hair across the eye 4. head pose 5. makeup 6. subject posing (frontal / non- frontal to camera) 1. image enhancement and data reduction process 2. physical properties (e.g. resolution and contrast) 3. optical distortions 4. static properties of the background (e.g. wallpaper) 5. camera characteristics • sensor resolution 6. scene characteristics • geometric distortion 1. dynamic characteristics of the background like moving objects 2. variation in lighting and relate potential defects as • deviation from the symmetric lighting • uneven lighting on the face area • extreme strong or weak illumination 3. subject posing, e.g.: • too far (face too small), or too near (face too big) • out of focus (low sharpness) • partial occlusion of the face From Covariates to Quality Measures …. From Covariates to Quality Measures From Covariates to Quality Measures … …. .
  • 40. 4/21/2008 40 FRVT 2006 Experiment 4 Covariate Analysis Conclusion Conclusion   Quality is NOT in the eyes of the beholder Quality is NOT in the eyes of the beholder   It is in the performance numbers It is in the performance numbers   Model quantifies performance change. Model quantifies performance change.   Turn the knob. Turn the knob.   Read off the change in performance. Read off the change in performance.   Interaction between covariates. Interaction between covariates.   Tells us where to put our efforts Tells us where to put our efforts   Indoors it is FRIFM. Indoors it is FRIFM.   Outdoors it is Query Image Size. Outdoors it is Query Image Size.   These models are used in other fields. These models are used in other fields.   e.g., Biomedical. e.g., Biomedical.   Studies of Biometrics should use them. Studies of Biometrics should use them.