The problem with HR Analytics: Bias in the Machine

W E A R E
V I D E O - L E D T E C H N O L O G Y A N D
H R S A A S - B A S E D D ATA A N A LY T I C S

1 www.launchpadrecruits.com
WE MAKE PEOPLE DECISIONS DAILY
Hiring
Promotions
Role Changes
Appraisals Retention
Probation
© 2016 LaunchPad. All rights reserved

EVERY OPPORTUNITY IS AN OPPORTUNITY FOR BIAS
Like Me Diversity Gender

CASE STUDY
Entrepreneur
Silicon Valley
CEO
MD Venture Capital
Board Member
Competent. Would
be enjoyable to work for.
Likeable.
Competent. Power-
hungry. Self-promoting.
Selfish. Not the person
you would want to hire
or work for.
”
“
”
“
HOWARD ROIZEN
HEIDI ROIZEN
GENDER BIAS IN ACTION (COLUMBIA UNIVERSITY)

A CLEAR PATTERN RELATED
TO THE TIME OF DAY
OTHER FORMS OF BIAS
Average Score, across all reviewers, each hour Negative reviewer vs Positive reviewer – Score distribution
Average Score per review criteria
SOME REVIEWERS ARE CONSISTENTLY
NEGATIVE, OTHERS ARE POSITIVE
SOME CRITERIA DO NOT
DIFFERENTIATE CANDIDATES
Negative correlation – Average answer vs count of videos viewed
REVIEWERS BECOME MORE NEGATIVE
AS THEY REVIEW MORE CANDIDATES

The central contradiction of hidden bias training is that you can’t train
something you can’t control. Understanding implicit bias does not actually
provide you the tools to do something about it
Tony Greenwald, Psychology Professor, University of Washington”
THE PROBLEM IS, IT’S UNCONSCIOUS
“

Identical Experience Reviewable DecisionsStructured
ADDRESSING BIAS

InterviewCV Review Telephone Screen
SUBJECTIVE, UNSTRUCTURED, UNIQUE
EXAMPLE: HIRING DECISIONS

Automated Screening Recorded Video Assessment Validation
SYSTEMATIC BIAS REMOVAL
OBJECTIVE, STRUCTURED, CONSISTENT EXPERIENCE

MI InsightsSignals
Machine Learning AlgorithmBehaviour
monitoring
VALIDATION IN ACTION
Real Time
feedback
REVIEWER
A
REVIEWER
B
REVIEWER
C
Monitor reviewer behaviours
and identify potential bias
Maximise candidate score
confidence with optimised
candidate allocation
Test reviewers against known
‘correct’ scores
Provide real-time feedback
for reviewers
Assessment Design
Feedback to help recruiters
optimise the assessment
design
Reviewer Ratings
Rate reviewer performance,
provide actionable feedback
and potential training
requirements
Candidate Score Confidence
Quantify confidence in scores
given to individual candidates
© 2015 LaunchPad. All rights reserved | Nov 2015 | Confidential – Not for redistribution

ASSESSMENT DESIGN FEEDBACK
© 2015 Launchpad. All rights reserved | Sep 2015

11 www.launchpadrecruits.com© 2015 Launchpad. All rights reserved | Sep 2015
RECRUITMENT CAMPAIGN
USE OF SCORES
REVIEWER 1 REVIEWER 2

REVIEW CRITERIA DESIGN
RANGE OF SCORES BY CRITERIA
Criteria DESCRIPTION TRAFFIC LIGHT
1 CRITERIA ONE
2 CRITERIA TWO
3 CRITERIA THREE
4 CRITERIA FOUR
5 CRITERIA FIVE

REVIEW CRITERIA INTER-RATER AGREEMENT
Criteria DESCRIPTION TRAFFIC LIGHT
1 CRITERIA ONE
2 CRITERIA TWO
3 CRITERIA THREE
4 CRITERIA FOUR
5 CRITERIA FIVE

REVIEW CRITERIA INTER-RATER AGREEMENT BY GENDER
Criteria DESCRIPTION FEMALE MALE
1 CRITERIA ONE
2 CRITERIA TWO
3 CRITERIA THREE
4 CRITERIA FOUR
5 CRITERIA FIVE

USE OF SCORES BY CRITERIA & GENDER FOR OVERLAP CANDIDATES
Criteria DESCRIPTION FEMALE AVG
SCORE
MALE AVG
SCORE
DIFFERENCE
1 CRITERIA ONE 3.6 3.1 0.5
2 CRITERIA TWO 3.2 2.8 0.4
3 CRITERIA THREE 3.3 2.9 0.4
4 CRITERIA FOUR 3.4 2.9 0.5
5 CRITERIA FIVE 3.6 2.9 0.7

STRICTNESS OF REVIEWERS
ARE REVIEWERS BEING TOO HARSH OR GENEROUS?
Reviewer AVERAGE SCORE GROUP AVERAGE TRAFFIC LIGHT
1 3.32 3.34
2 3.35 3.34

RANGE OF SCORES OF REVIEWERS
ARE REVIEWERS USING A HEALTHY RANGE OF SCORES?
Reviewer TRAFFIC LIGHT
1
2

INTER-RATER AGREEMENT - OVERLAP CANDIDATES
DO THE REVIEWERS AGREE ON SCORES?
Reviewer Agreement  Reviewers overall only have a fair agreement

*Income Growth = Revenue Growth minus Cost of Revenue
THE VALUE OF SUPERIOR PERFORMANCE
Cost Savings
Number of Employees TBC
Reduction in Attrition £X Million
Process Efficiency Improved
Time to Hire Reduced
Value Creation
Employee Engagement Improved
Diversity & Inclusion Improved
Candidate Experience Improved
Quality of Talent Improved

www.launchpadrecruits.com
Questions?

The problem with HR Analytics: Bias in the Machine

Recommended

Recommended

More Related Content

Similar to The problem with HR Analytics: Bias in the Machine

Similar to The problem with HR Analytics: Bias in the Machine (20)

More from TALiNT Partners

More from TALiNT Partners (20)

The problem with HR Analytics: Bias in the Machine