SOCIAM all-hands meeting, September, University of Oxford

1. UNDERSTANDING
ALGORITHMIC DECISIONS
Updates on work in progress from the SOCIAM team
at Oxford CS…
Dr. Reuben Binns, Dr Jun Zhao, Dr Max Van Kleek, Prof. Sir. Nigel Shadbolt
reuben.binns@cs.ox.ac.uk
Dept. Computer Science, University of Oxford

2. QUESTION: WHAT DO THEY DO WITH THE DATA?
▸ Transparency over data collection is important, but then
what happens to it?
▸ How will they use it? Will they treat me differently?
UNDERSTANDING ALGORITHMIC DECISIONS

3. UNDERSTANDING ALGORITHMIC DECISIONS
…BUILD MODELS!
▸ ML systems: build a model
which can predict or classify
things
▸ Examples:
▸ What products will this
person buy?
▸ will they pay back their loan?
▸ Is this email spam?

4. MACHINE LEARNING AND SOCIAL MACHINES
▸ People label data (‘spam’ / ‘not spam’, ‘good credit risk’ /
‘bad credit risk’), machines build models from it
▸ Models used to decide things:
▸ what adverts are seen
▸ who gets a loan
▸ what goes in the spam box
UNDERSTANDING ALGORITHMIC DECISIONS

5. ACCOUNTABILITY, TRANSPARENCY, FAIRNESS
▸ How do the biases of humans in training data find their way
into machine models?
▸ How should machines explain the outputs of their models
to humans? Can explanations help people assess the
fairness of those outputs?
UNDERSTANDING ALGORITHMIC DECISIONS

6. AUTOMATED CONTENT MODERATION
▸ Manual, community-driven
flagging
▸ Paid moderators
▸ Blacklisted words

7. AUTOMATED CONTENT MODERATION

8. ‘TOXICITY’ SCORES

9. ‘TOXICITY’ SCORES

10. ALGORITHMIC MODERATION AND BIAS
▸ 100k Wikipedia talk page comments, each annotated by 10 different
people for `toxicity’.
▸ Do different demographic sub-groups have different norms of offence?
▸ Yes: men and women often disagreed.
▸ Women had more diverse norms of offence.
y
n♀ ♀
y
y
♂ ♂

11. CREATING BIASED TRAINING DATA
▸ Created 30 training data sets, sampling men / women /
mixed genders from original Detox dataset
▸ Trained new offensive text classifiers based on these
biased samples
♀ ♀ ♀
♀ ♀ ♀
♀♀ ♀
♀ ♂ ♂
♂ ♂ ♂
♂♂ ♂
♂ ⚥ ⚥
♂ ⚥ ⚥
⚥⚥ ⚥
⚥
⚥
⚥
⚥
♂
♀
test upon

12. TESTING BIASED OFFENCE DETECTORS
▸ Test on unseen examples, labelled by each group (male /
female / balanced)
▸ All classifiers performed worse on female-labelled test
data
▸ Different coefficients between m / f.
Female Male Balanced
0.96 0.97 0.98 0.96 0.97 0.98 0.96 0.97 0.98
0.44
0.48
0.52
Specificity (true negative rate)
Sensitivity(true
positiverate)
Training set
Female
Balanced
Male
Test

13. EXPLAINING ALGORITHMIC DECISIONS
▸ ML systems used to decide:
▸ Who gets a loan
▸ Who to invite to an interview
▸ Insurance premiums
▸ How should these decisions be explained?

15. WHY DOES COMPUTER SAY NO?
▸ Data protection laws require organisations to provide
`meaningful information about the logic’ behind
automated decisions
▸ US laws require credit scoring companies to provide
`statements of reasons’

16. DECISION TREES?

17. LOCAL, INTERPRETABLE, MODEL-AGNOSTIC EXPLANATIONS
▸ E.g. Ribeiro, Marco Tulio, Sameer Singh, and
Carlos Guestrin. "Why should i trust you?:
Explaining the predictions of any classifier."
Proceedings of the 22nd ACM SIGKDD
International Conference on Knowledge
Discovery and Data Mining. ACM, 2016.

18. SENSITIVITY
▸ What would I have to
change in order to
get a different result?

19. CASE BASED
▸ Marian is like Vivian,
and Vivian paid back
her loan, so Marian
will pay back her loan
Nugent, Conor, and Pádraig
Cunningham. "A case-based explanation
system for black-box systems." Artificial
Intelligence Review 24.2 (2005):
163-178.

20. DEMOGRAPHIC
▸ What are the
characteristics of
people who received
this outcome?
▸ What outcomes did
other people in my
demographic
categories get?
Ardissono, Liliana, et al. "Intrigue: personalized recommendation of tourist attractions for desktop and hand
held devices." Applied Artificial Intelligence 17.8-9 (2003): 687-714.

21. DO EXPLANATIONS AFFECT PERCEPTIONS OF JUSTICE?
▸ Tested people’s perceptions of justice in response to
various hypothetical cases using different explanation
styles…

22. DO EXPLANATIONS AFFECT PERCEPTIONS OF JUSTICE?
“She’s been a victim of
this computer system
that has to generalise
based on, like,
somebody else”
“If we were in a court of
law, I would argue we don’t
know his circumstances,
but given this computer
model and the way it works
it’s deserved”
“This is just simply
reducing a human being
to a percentage”