Slides from Mary Ann Brennan's ML4ALL Machine Learning conference talk on Machine Bias. Machine Learning algorithms often exhibit systematic, unfair bias against certain groups of people, all while the supposed impartiality of machines dissolves accountability. I touch on definitions, criminal justice, and solutions and finish with a success story from NLP and a slide listing resources. Video here: https://www.youtube.com/watch?v=VksAAI60GEc
2. What is machine bias?
• Machine Bias, or Algorithmic Bias, occurs
when machine intelligence exhibits bias against
groups of people, usually systematic and unfair
• Algorithmic Fairness describes efforts to
ensure people are treated equally by algorithms
Not related to bias/
variance or model bias
3. Human bias
Racial Bias
Motherhood Penalty
Gender Bias Ableism
Religious Bias
Unconscious Bias
Xenophobia
Ageism
Positive Bias
Fatherhood Bonus
“People who wear kilts are the worst”
Bernie Bros
“Android users are fools”
4. Machines are seen as impartial
Machine learning is like
money laundering for bias.
It's a clean, mathematical
apparatus that gives the status
quo the aura of logical inevitability.
—Maciej Cegłowski
http://idlewords.com/talks/sase_panel.htm
5. PredPol - Predictive policing
Higher observed
crime rates in an area
Additional policing
of the area
Additional observation
of crimes
To predict and serve?, Volume: 13, Issue: 5, Pages: 14-19, First published: 07 October
2016, DOI: (10.1111/j.1740-9713.2016.00960.x)
Drug arrests vs. estimated use
To predict and serve?, Volume: 13, Issue: 5, Pages: 14-19, First published: 07 October 2016,
DOI: (10.1111/j.1740-9713.2016.00960.x)
Simulated effects of additional observed crime
To predict and serve? Lum, Isacc 2016
6. COMPAS - Risk assessment
White Black
Higher risk,
didn’t reoffend 23.5% 44.9%
Low risk, did
reoffend 47.7% 28.0%
Machine Bias - ProPublica
White Black
Higher risk, re-
offended 505 1369
Total higher
risk 854 2174
Positive
predictive
value (PPV)
59% 63%
Demo at http://bit.ly/BIAS4ALL
7. COMPAS - Risk assessment
A Popular Algorithm Is No Better at Predicting
Crimes Than Random People
The Atlantic - January 17, 2018
The accuracy, fairness, and limits of predicting recidivism (Dressel and Farid 2018)
COMPAS:
65.2%
Average human:
62.8%
Pooled humans:
67%
8. Protected attributes
Gender Age College Major
Favorite
Movie Genre
Female 23 English
Romantic
Comedy
Male 78 Physics
Science
Fiction
Neutral 48 Economics
Action/
Adventure
10. More diverse teams
• A more diverse team will likely test with a wider
variety of data
• Members of diverse groups are more likely to
find biases in code
• This is a good idea anyway
11. Other ways to uncover bias
• Community Policing: allow users to report biased
results
• Algorithmic auditing: analyze performance on
different inputs
• Ask for help! Algorithmic Justice League offers
to help detect bias, ORCAA consulting firm
17. Google Translate is sexist
Turkish:
• o bir doktor
• o bir hemşire
• o evli
• o bekar
English:
• he is a doctor
• she is a nurse
• she is married
• he is single
19. What can we do?
What happens when an algorithm cuts your health care
The Verge - Mar 21, 2018
PEOPLE cut
• Support tools, standards, and regulation! GDPR,
algorithmic impact assessment, new NYC law
• Talk about these issues with your communities!
20. Resources
Organizations and conferences
• Orgs: Algorithmic Justice League, AI
Now, Algorithm Watch, AFOG - Berkeley,
fairness.haverford.edu
• Confs: FAT/ML, FAT Conference
• Consulting: ORCAA
• Data-focused: Data & Society, Human
Rights Data Analysis Group
Technical resources
• Debiaswe - word embeddings and
source from Bolukbasi
• COMPAS Analysis on GitHub
• Deep Learning on Coursera
Human bias/justice
• Project Implicit - Implicit bias tests
• @pdxlawgrrrl, @shaunking,
@Data4BlackLives, so many more
• hiremorewomenintech.com (and
all underrepresented people)
Books
• Weapons of Math Destruction by
Cathy O’Neil
• Automating Inequality by Virginia
Eubanks
