1. The Tragedy of Bias in
Technical Hiring
in Five Acts
Kelsey Foley
Oct 10, 2014
2014
#GHC14
2014

2. Why are there so few women in tech?
1. “The Pipeline” – not enough trained women
2014

3. Why are there so few women in tech?
1. “The Pipeline” – not enough trained women
1. Industry doesn’t know how to recruit and
2014
hire women.
1. Industry doesn’t know how to retain women.
(Hint: Industry must hire women before retaining them!)

4. Synopsis
 The Birthplace of Bias – and how to
combat it
 How bias manifests in:
− Job descriptions
− The Interview Process
− The Hire or No-Hire Decision
2014

5. Act 1: The Players
“All the world’s a stage, and all the men
and women merely players.”
- William Shakespeare, As You Like It
2014

6. Meet Julie Ette:
2014
• BS in CS from StateU
• 5 years work experience with two
mobile software companies
• Looking for a new job

7. Meet Monty and Ben:
2014
Monty Gue,
Engineering manager at
hot mobile startup Roam.io
Ben Volio,
Technical recruiter at
Roam.io

8. Will Julie find a match with Monty’s team?
2014
Let’s find out…

9. Act 2: The Birthplace of Bias
“Wisely and slow. They stumble that run fast.”
- William Shakespeare, Romeo and Juliet
2014

10. The Two-Systems Model of Judgment and Choice
System 1 Thinking:
 Fast
 Effortless
 Automatic, involuntary
 Takes mental short cuts
 Driven by impressions,
patterns, intuitions,
memories, and feelings
 Prone to error and bias
2014
unless checked by
System 2
System 2 Thinking:
 Slow
 Effortful, limited energy
budget
 Conscious engagement
required
 Can be lazy
 Applies methodical,
reasoned, and coherent
thinking to the System 1
raw data
(Kahneman, Daniel. Thinking, Fast and Slow. Farrar, Straus and Giroux. 2011.)

11. System 1 Source Data
Comes from the cultural soup we experience
every day since infancy:
• Role models - parents, teachers, siblings, and
caregivers
• TV, books, music, and cultural memes
• Peers and their own source data!
System 1 creates a meaningful story from
our senses and experiences!
(Efforts to fix The Pipeline change the next generation’s patterns.)
2014

12. The Birthplace of Bias
Cognitive bias happens when System 1
decides without System 2 helping to
catch errors, assumptions, biases, and
mental short cuts!
(We all do this! Don’t feel bad. It’s part of being human!)
2014

13. Tech Company Culture Exacerbates Bias
“People who are cognitively busy are also
more likely to make selfish choices, use
sexist language, and make superficial
judgments in social situations…. but of
course cognitive load is not the only cause
of weakened self-control. A few drinks have
the same effect, as does a sleepless night.”
- Dr. Daniel Kahneman, Thinking, Fast and Slow, pp.41
2014

14. Common Biases in Hiring
2014
 Casuistry
− using specious reasoning to rationalize behavior
or decisions
 The Halo Effect
− First impressions influence later experience
 Affect Heuristic
− People answer an easy question with System 1
instead of a harder one with System 2

15. Common Biases in Hiring
2014
 Confirmation Bias
− Seeking data that confirms our ideas
 Fundamental Attribution Error, or the
Negativity Effect
− Over-emphasizing traits in others while
under-emphasizing situations (luck) in
ourselves

16. Common Biases in Hiring
 Predicting by Representativeness
− Making decisions using association with a
2014
stereotype
 Projection Bias
− Unconsciously assuming that others share
our own perspectives, thoughts, and values

17. So… How do we overcome our biases?
“What can be done about biases? How can we
improve judgments and decisions, both our own and
those of the institutions that we serve and that serve
us?...
The way to block errors that originate in System 1 is
simple in principle: recognize the signs that you are
in a cognitive minefield, slow down, and ask for
reinforcement from System 2. Unfortunately, this
sensible procedure is least likely to be applied when
it is needed most.”
- Dr. Daniel Kahneman, Thinking, Fast and Slow, pp.417
2014

18. System 1 in Interviews
“The optimal time to make a decision about the candidate is
about three minutes after the end of the interview…. I ask
interviewers to write immediate feedback after the interview,
either a “hire” or “no hire”, followed by a one or two paragraph
justification. It’s due 15 minutes after the interview ends.”
“Never say “Maybe, I can’t tell.” If you can’t tell, that means No
Hire. It’s really easier than you’d think. Can’t tell? Just say no! If
you are on the fence, that means No Hire… Mechanically
translate all the waffling to “no” and you’ll be all right.”
- Joel Spolsky, The Guerrilla Guide to Interviewing v3.0, Oct 25, 2006
http://www.joelonsoftware.com/articles/GuerrillaInterviewing3.html
2014

19. Act 3: Attracting Diverse Candidates
2014

20. Subtle Cues in Job Descriptions
The purpose of a job description?
1. Internal: communicate hiring requirements
2. External: promote the job and company
How can the job description project bias?
(See also: Gaucher, D., Friesen, J., & Kay, A. C. (2011, March 7). Evidence That
Gendered Wording in Job Advertisements Exists and Sustains Gender Inequality. Journal
of Personality and Social Psychology. )
2014

21. Bad (and real!) examples
“Do you have a passion for quality gaming and auto
racing? XXXX Game Studios is hiring!
You are a Senior Software Development Engineer
with broad game development experience and world-class
software engineering skills. You’re the kind of
person who drives projects to completion, sometimes
across multiple functions and groups.”
(See any Projection Bias? Casuistry? Representativeness?)
2014

22. Bad (and real!) examples
“The Application Programmer Analyst plays a vital role on
the ZZZZ Medical Group Support team, demonstrating our
values of patient-centered care and service; respect, caring
and compassion; teamwork and partnership; continuous
learning and improvement; and leadership.
In this position you will:
Enhance existing computer programs to add value
throughout the organization…”
(Representative stereotypes exist for female-dominated roles too)
2014

23. Bad (and real!) examples
“QQQ Software runs a developer paradise: the latest technologies
and platforms and an elite team of great developers. No résumé
needed! Great work speaks for itself. We'd love links to your GitHub or
StackExchange profile!
Your Profile:
You live, eat and dream about code and test! Your drive to know
more and do better makes you evolve...”
(Projection Bias and Representativeness)
2014

24. Some recent (real!) examples
Education Required:
BS (Technical ), Masters preferred
Experience Required:
Prior experience at the Director level or equivalent
Physical Requirements:
Must be able to execute a two-handed reverse dunk on a ten-foot
rim without the aid of a trampoline.
2014

25. Who wants these as
coworkers?
(And why are they all holding weapons?)
2014

26. To Attract More Diverse Candidates:
 Be aware of the impact of language.
 Write job descriptions that don’t create role biases!
− Look carefully for values, traits, behaviors, and
2014
motivations
− Find gender-neutral ways to “sell” the job
 Circulate several versions to attract candidates
with diverse motivations and career objectives.
 Get people with different perspectives to edit job
descriptions – how would someone in marketing
write an engineering job description?

27. Monty’s JD Ben’s JD
Roam.io is hiring versatile
software engineers with a passion
for making products that impact
our customer’s lives. Our
developers support and
challenges each other to
continuously learn and improve.
We believe in working at a
sustainable pace following Agile
philosophies. Roam.io offers
flexible schedules and a
respectful and fun work
environment. Join us and make a
difference!
Do you live, breathe, eat, and
dream about coding and mobile?
Do you crave fanfare and
adulation from users? Roam.io is
hiring! We’re looking for software
geeks who can thrive in our high-energy
2014
open office environment.
We offer great benefits, free food
and beer, foosball, and tons of
fun. Come join our elite team and
push the barriers in mobile
technology!
(Which one would Julie apply for? What about Julio?)

28. Act 4: The Interview
2014

29. The Classic Software Interview
 Short call with a recruiter
 A technical phone screen, some coding
 On-site interview with 4-6 sessions, all
with heavy coding
 Many tech companies do no training on
how to interview
− Some focus on legal areas of questioning
− A few give training but do not monitor how
these techniques are used in interviews
2014

30. How effective are tech interviews?
“For the record, we don’t think that the way
interviewing is done today is necessarily the
way it should be done. The current paradigm
puts too much emphasis on the ability to solve
puzzles and familiarity with a relatively limited
body of knowledge, and it generally fails to
measure a lot of the skills that are critical to
success in industry.”
- Mongan, John, Eric Giguere, and Noah Kindler. Programming
Interviews Exposed: Secrets to Landing Your Next Job, 2013, pp. xxvi
2014

31. Schmidt, F.L. & Hunter, J.E. (1998) The validity and utility of selection methods in personnel
psychology: Practical and theoretical implications of 85 years of research findings,” Psychological
Bulletin, 124, 262–274.
Personnel Measures Predictive Validity (r)
Work sample tests 0.54
General cognitive ability
0.51
tests
2014
Employment interviews,
structured
0.51
Job knowledge tests 0.48
Behavioral questions 0.45
Employment interviews,
0.38
unstructured
Reference checks 0.26
Job experience 0.18
Years of education 0.10
Age -0.01

32. What can this look like in practice?
 Train interviewers about cognitive biases
 Ask some coding questions
 Also ask behavioral, work habits, and job
knowledge questions to assess all the
other success factors
 A great resource for technical managers to
use with their teams is this book:
Rothman, Johanna. Hiring Geeks That Fit. Rothman
Consulting Group, Inc. 2013.
2014

33. Julie’s interview:
2014
Welcome with recruiter Ben Volio
Software lifecycle, Agile, communication style,
personal work habits with Merlin Cutio
Petra Escalus: CS fundamentals - complexity, networks,
threads, databases, OS
Ty Balt and Amy Bram: lunch at Café Verona
and behavioral and culture fit
Cindy Paris: mad programming skillz - languages, algorithms,
data structures, coding
Phil Laurence: Debugging and testing in mobile & embedded
Finish with Hiring Manager Monty Gue

34. Interviewing is bi-directional!
Julie is also evaluating:
− The manager
− Potential coworkers
− The company
− The workplace environment
− The technology stack
The interview experience will impact Julie’s
final decision!
2014

35. Act 5: The Decision
2014

36. The Post-Interview Debrief
 Review job requirements first
 Avoid the Affect Heuristic!
− Ask: “What evidence did you see that Julie has the skills
for job requirement #1?”
− Don’t ask: “What did you think of Julie?”
 Watch for cognitive landmines:
− personality traits, intuitions, impressions, or stereotypes,
− Slow down, engage System 2
− Dig into possible bias with questions
 Give a numerical score for each job requirement
− Numerical scores engage System 2
− Don’t use “hire” or “no hire” which uses System 1.
2014

37. Epilogue
“If this were play'd upon a stage now, I
could condemn it as an improbable fiction.”
- William Shakespeare, Twelfth Night
2014

38. Outcomes
Changing technical interviews won’t convert
every hire from a tragedy to a romance…
But, it sets the stage for candidates of all
kinds to audition on equal terms.
2014

39. And what of Julie and fair Roam.io?
2014

40. Thank you!
Want to use #System2Hiring in
your workplace?
Or:
Want to experience
#System2Hiring yourself?
My team is hiring!
2014
Kelsey Foley
kelsey@moz.com
@EnigmaticKelsey

41. Got Feedback?
Rate and Review the session using the
GHC Mobile App
To download visit www.gracehopper.org
2014