Stark Individual Differences: Face Recognition Ability Influences the Relationship Between Confidence and Accuracy in a Recognition Test of Game of Thrones Actors - Jesse Grabman, Chad Dodson
•Download as POTX, PDF•
0 likes•22 views
Presented at TARMAC 2020. Access recording via link: https://www.youtube.com/watch?v=Z1EQpdNeblo&list=PLfjltJSHA9mqEI3gjUYXylY-t6hBm6fE2&index=4
More Related Content
Similar to Stark Individual Differences: Face Recognition Ability Influences the Relationship Between Confidence and Accuracy in a Recognition Test of Game of Thrones Actors - Jesse Grabman, Chad Dodson
Similar to Stark Individual Differences: Face Recognition Ability Influences the Relationship Between Confidence and Accuracy in a Recognition Test of Game of Thrones Actors - Jesse Grabman, Chad Dodson (10)
More from Heather Flowe
More from Heather Flowe (20)
Recently uploaded
Recently uploaded (20)
Stark Individual Differences: Face Recognition Ability Influences the Relationship Between Confidence and Accuracy in a Recognition Test of Game of Thrones Actors - Jesse Grabman, Chad Dodson
- 1. Stark Individual Differences: Face Recognition Ability Influences the Relationship Between Confidence and Accuracy in a Recognition Test of Game of Thrones Actors Jesse H. Grabman & Chad S. Dodson TARMAC June 11th, 2020
- 2. Eyewitness misidentifications are the leading cause (>70%) of false convictions in the United States 2
- 3. High eyewitness confidence usually signals accuracy… 3
- 4. High eyewitness confidence usually signals accuracy… 4
- 5. High eyewitness confidence usually signals accuracy… 5
- 6. High eyewitness confidence usually signals accuracy… 6
- 7. High eyewitness confidence usually signals accuracy…
- 8. High eyewitness confidence usually signals accuracy…
- 9. Are there factors that systematically predict high confidence errors?
- 10. Background Individual Differences Game of Thrones Discussion
- 11. Heterogeneity in Face Recognition • The ability to identify unfamiliar faces varies considerably from person to person. Prosopagnosia Super-Recognizers Most People 11
- 12. Measuring Face Recognition Ability • Cambridge Face Memory Test (CFMT) Measures poor to above-average performance on face memory. 72 trials of varying difficulty • Score = simple sum of correct responses. Duchaine & Nakayama, 2006; Cho et al., 2015 12
- 13. Face Memory Test
- 14. Face Memory Test
- 15. Face Memory Test
- 16. Does Face Recognition Ability Influence the Confidence-Accuracy Relationship? 16
- 17. Standard Eyewitness paradigm Participants encode 12 faces 17 … … …
- 18. Standard Eyewitness paradigm Then perform a lineup task after a delay. 18
- 19. Provide confidence in decision 19
- 20. Poor recognizers Good recognizers
- 21. Poor recognizers Good recognizers Error bands = 95% CI 21 Face Recognition ability influences the confidence-accuracy relationship (N = 569).
- 22. Face Recognition ability influences the confidence-accuracy relationship (N = 569). Poor recognizers Good recognizers Error bands = 95% CI 22
- 23. Poor recognizers Good recognizers Error bands = 95% CI 23 Face Recognition ability influences the confidence-accuracy relationship (N = 569).
- 24. Poor recognizers Good recognizers Error bands = 95% CI CFMT = 68 ~86% Accurate CFMT = 48 ~61% Accurate 24 Face Recognition ability influences the confidence-accuracy relationship (N = 569).
- 25. How Well Does this Apply to Real- World Contexts?
- 26. How Well Does this Apply to Real- World Contexts?
- 27. How Well Does this Apply to Real- World Contexts?
- 28. How Well Does this Apply to Real- World Contexts?
- 29. Background Individual Differences Game of Thrones Discussion
- 31. Advantages • Faces are encountered under naturalistic conditions. • Dynamic lighting, contexts, views, age (etc.)
- 32. Advantages • Incidental encoding • All 32 participants watched 6 seasons (in order) as the show aired.
- 33. Retention-Interval • Last appeared on the show during: • Season 6 (18 actors; short) • Season 4-5 (32 actors; medium) • Season 1-3 (34 actors; long)
- 34. Exposure Duration • Time on screen: • Leading actors (27 actors; 20-90 min) • Supporting actors (27 actors; 9-19 min) • ‘Bit parts’ (30 actors; short; < 9 min)
- 36. Old/New Recognition 84 Actors 84 Foils CFMT+Watch Show
- 37. Old/New Recognition 84 Actors 84 Foils 84 Actors 84 Foils CFMT+Watch Show
- 38. Task
- 39. Task ‘K’ = Yes ‘L’ = No Have you seen this face before?
- 40. Task ‘K’ = Yes ‘L’ = No Have you seen this face before? Correct!
- 41. Task How confident are you that you have seen this face before? 1 2 3 4 5 Not at all confident Totally confident
- 42. Is Confidence Indicative of Accuracy? Error bars represent 95% HDIs
- 43. Is Confidence Indicative of Accuracy? Error bars represent 95% HDIs
- 44. Retention Interval Error bars represent 95% HDIs
- 45. Exposure Duration Error bars represent 95% HDIs
- 46. Similarity Error bars represent 95% HDIs
- 47. Good recognizersPoor recognizers Does CFMT Influence the Confidence-Accuracy Relationship?
- 48. Good recognizersPoor recognizers Does CFMT Influence the Confidence-Accuracy Relationship?
- 49. Retention Interval Error bars represent 95% CIs
- 50. Exposure Duration Error bars represent 95% CIs
- 52. Take-home Messages 1. Confidence ratings were generally predictive of accuracy in a difficult, naturalistic test of face recognition.
- 53. Take-home Messages 1. Confidence ratings were generally predictive of accuracy in a difficult, naturalistic test of face recognition. 2. Replicated that face recognition ability influence the reliability of high confidence ratings.
- 54. Background Individual Differences Game of Thrones Discussion
- 55. Questions for Discussion 1. How can we better map insights from the lab onto real-world face recognition?
- 56. Questions for Discussion 1. How can we better map insights from the lab onto real-world face recognition? 2. Which other individual differences apply to face recognition and meta-cognitive tasks?
- 57. 57
- 58. Error bars represent 95% HDIs Overall Accuracy
- 59. Retention Interval Error bars represent 95% HDIs
- 60. Exposure Duration Error bars represent 95% HDIs
- 61. Similarity Error bars represent 95% HDIs
- 62. Are there *Stark* Individual Differences? Error bars represent 95% HDIs
- 63. Retention Interval Error bars represent 95% HDIs
- 64. Exposure Duration Error bars represent 95% HDIs
- 65. Similarity Error bars represent 95% HDIs
- 66. ROC
- 67. ROC
- 68. ROC
- 69. ROC
Editor's Notes
- Good afternoon everyone! Thank you for the opportunity to present our work!
- To start off, eyewitness misidentifications are the leading cause of false convictions in the United States, accounting for 70% of cases.
- Research so far has supported that eyewitness confidence is a robust predictor of accuracy. What I’m showing here is an influential 2017 review paper by Wixted and Wells, that has already been cited over 200 times. Their analyses showed that eyewitness confidence was protective against a number of factors that we know influence eyewitness memory.
- For example, we know that the race of the police suspect can influence accuracy. People generally are worse at remembering faces of a different race from than themselves. For instance, when a black victim attempts to identify a white perpetrator from a lineup.
- Based on Wixted and Wells’ analysis, we see evidence for a cross-race effect at lower levels of confidence. When confidence was low, participants in this study were more accurate at identifying a same-race culprit, than if the crime was committed by a person of a different race.
- Yet, at higher levels of confidence, there is seemingly no difference. The review paper finds a similar confidence-accuracy relationship for many other variables.
- For example, the weapon focus effect…
- And retention interval
- Despite these findings, we still see high confidence errors in the lab and in the real-world. For this reason, our research examines factors that systematically predict high confidence errors.
- To start off, I’d like to talk about some of my previous work that has investigated this issue. One thing that is common in the eyewitness memory field is to treat all witnesses the same. However, intuitively, we know that this is not the case.
- In fact, the ability to recognize unfamiliar faces varies widely across the population. At the low end are those with developmental prosopagnosia, or face-blindness, who may have difficulty recognizing even family members and close friends. While at the high end there are super-recognizers, who exhibit exceptional face recognition ability. But, most individuals fall somewhere in the middle.
- So, how do we test face recognition ability in the lab? We use a well validated measure -- the Cambridge Face Memory Test. This test measures poor to above average face recognition ability over 72 trials of varying difficulty. An individual’s score is simply the number of trials they get correct, where higher scores indicate greater face recognition ability.
- Here is an example of what the test looks like. First, individuals memorize a target face in three separate orientations.
- Then, they must identify this face among two additional fillers.
- And, the test grows more challenging over subsequent trials. For example, by incorporating visual noise.
- Recently, I investigated how face recognition ability influences the rate of high confidence errors in eyewitness memory.
- In this study, participants first encoded 12 still images of faces, each exhibiting a casual expression, such as a smile.
- After a delay, participants performed a standard eyewitness lineup procedure. In this procedure, the participant attempts to identify whether a previously seen face is present in a 2x3 array of mugshots.
- After each decision, participants provided a confidence rating, indicating how certain they were about the accuracy of their response.
- Next they performed the Cambridge Face Test. Here is the distribution of the scores from the CFMT for our 569 participants. The median score is 61 and the blue shading indicates 1 Median Absolute Deviation above and below this average score. Again, higher scores reflect stronger face recognition ability
- Let’s get to the results. To orient you to the graph, the x-axis represents the participants face recognition ability, with higher scores indicating greater ability level. This dashed line represents the average face recognition score across participants. The y-axis gives the percentage of identifications that resulted in a correct decision. And, the colored lines will represent different levels of confidence.
- First, let’s look at the results from participants who were above average face recognizers. As we can see, the results are concordant with previous studies. As represented by the red line, when good face recognizers use a 100% confidence rating they tend to be pretty accurate.
- However, if we look at below-average face recognizers we start to see a different story. High confidence identifications become less indicative of accuracy as face recognition ability decreases.
- In fact, if we compare the accuracy of high confidence identifications from participants who were one standard deviation above the average score to those one deviation below, we see a gap of about 25 percentage points. So, results from this study indicated that the value of a high confidence rating differs across face recognition ability.
- However, one question I had was whether the results of the lineup study would extend to situations that more closely mirror our daily experiences recognizing people.
- For example, you may need to recognize people from strange angles…
- Or at different ages…
- Or when they make the regrettable decision to bleach their hair in Secondary school.
- To address this question, in a recent paper, I re-analyzed a dataset from Devue and colleagues
- In Devue et al’s study, 32 participants were tested on their ability to recognize actors from the popular television series, Game of Thrones.
- This approach comes with several advantages. For example, actors are encountered under more naturalistic conditions, with differences in lighting, viewing angle, and age.
- Most importantly, there was incidental encoding in a much less controlled environment than my previous study. All participants watched 6 seasons in order as the show aired, without knowing that they would later need to recognize the actors.
- And, if you’re a fan of Game of Thrones, you know that many characters are killed throughout the series. This creates some natural retention intervals based on the last time the actor appeared on the show. Participants needed to recognize faces from as far back as Season 1, or 6 years.
- And actors had varying amounts of screen time. Leading actors have the most screen-time, whereas bit parts have the least.
- Upon entering the study, participants took the CFMT Plus, which is an expanded version of the CFMT. They then completed an old/new recognition test of 168 faces, half old actors and half new foils.
- Half of the participants completed the task with actors who looked similar to their most recent appearance in the show.
- Whereas the other half saw a version where the actors were dissimilar as possible.
- And, just to acquaint you to the task, this is an actor that the participants viewed in the show.
- On each trial, participants needed to choose whether or not they had seen this person before.
- And in this case, a yes response would be a correct identification.
- After making their decision, participants provided a confidence rating ranging from 1 to 5. One is the least confident, whereas five is the most confident.
- So, is confidence predictive of identification accuracy in this highly uncontrolled context?
- The answer appears to be yes!
- This pattern holds for large retention-intervals, in this case spanning 3 or more years.
- Moreover, it largely holds for changes in exposure duration. However, you’ll notice that performance for bit parts is poor across levels of confidence. We think this is explained by chance levels of performance in this condition.
- Finally, high confidence seems to be indicative of good identification accuracy for both similar and dissimilar faces.
- However, our main focus was whether this confidence-accuracy relationship changes across the span of face recognition ability.
- We find that it does. Poor face recognizers are once again more likely to make high confidence misidentifications than better face recognizers.
- This pattern holds across retention intervals.
- Across exposure durations, again except for bit parts.
- And finally, across face similarity.
- So, the take-home messages from this study are that confidence ratings were generally predictive of accuracy in a difficult, naturalistic test of face recognition
- And, we replicated that face recognition ability influences the reliability of high confidence ratings, despite using other researchers’ data and a more ecologically valid context.
- Concluding, I would like to bring up a couple points for discussion
- First, how can we better map insights from the lab onto real-world face recognition. TV shows are a good start, but there are still some clear differences from everyday life. For example, what is the plot-line for the person you met at the dinner party the other day?
- Second, we need to better understand the influence of individual differences on experimental outcomes. I believe face recognition ability likely explains some, but not all of the differences we see the predictive value of a confidence statement.
- Finally, I want to thank all of my collaborators, both at the University of Virginia and elsewhere. Especially, my advisor Chad Dodson for helping me extensively with these projects.