The document discusses the implications of algorithmic bias in machine learning, emphasizing the need for transparency and accountability in these systems to avoid reinforcing societal inequalities. It highlights historical insights on query-by-browsing as a potential solution for creating interpretable algorithms, and the importance of creating scrutable internal representations. The author calls for a transformative approach beyond mere publication, advocating for active engagement in policy and public discourse regarding algorithmic ethics.

1. Sufficient Reason
Alan Dix
Computational Foundry
Swansea
http://alandix.com/academic/talks/sufficient-reason-2018

2. Tiree
Tiree Tech Wave
25-30 Oct 2018
Computational Foundry
Swansea University

3. the foundry
building
mission
community

4. computational foundry
opportunities
ECR programme
• emerging research leaders in the UK
• now in third year
escalator funds
• Swansea academic (not nec. computing)
• non-Swansea academic
• non-academic partner (industry, community, gov.)

5. today I am not talking about …
• physicality and product design
• the long tail of small data
• IT for small communities
• walking round Wales
• REF
• digital light
• digital humanities
• creativity and Bad Ideas
• virtual crackers and slow time
• modeling dreams, regret and the emergence of self

7. 25 years back …

9. warns of the danger of gender and ethnic bias in
black-box machine learning systems
gives example: database queries using ID3
offers (partial) solution: Query-by-Browsing
and even some broader heuristics
inter alia …

10. yes, 25 years ago!

12. Query-by-Browsing
creating scructable
internal representations

13. Query by Browsing
user chooses records of interest
 tick for those wanted
 cross for those not wanted
system infers query
web version uses rule induction
variant of Quinlan’s ID3
www.meandeviation.com/qbb

14. Query by Browsing
what it looks like
user asks
system to
make a query
system infers
SQL query
query results
highlighted

15. Query by Browsing
dual representation
query (intensional)
for precision
listing (extensional)
for understanding

16. Query by Browsing – how it works
examples
machine
learning
SQL query
cond
cond
decision
tree

18. it is not just about
being accurate
not just right
but also upright

19. algorithms reflect society

20. mimicking human behaviour and choices

21. pandering to human bias
(effective outcomes?)
• dating sites using ethnicity (CHI 2018!)
• young pretty waitresses sell more drinks
• Trump (reportedly) hiding black employees at
casino when certain rich customers arrived
• BBC (& others) paying male presenters more
because they are more popular

22. ‘good’ business
but is it good?

23. reinforcing societal/cultural norms
at school
boys more likely to study STEM subjects
girls more likely to study humanities
so, on average, with no other information
gender is an (albeit poor) predictor
of communication skills
and engineering knowledge

24. as a society we choose
to use other (and better)
predictors

25. innate (but largely irrelevant) differences
men are (on average) larger and stronger
so gender is a Bayesian predictor of strength
this may explain gender differences in some jobs
but …
it does NOT justify employment discrimination

26. bias is not about
algorithmic correctness
it is about social choice

27. Note:
human reasoning is
poor at ignoring low quality cues
even when we have better ones

28. algorithms may be better?

29. however …
not sufficient to remove explicit indicators:
gender/ethnicity/disability/religion
potential correlating factors e.g. clothing
algorithms need to actively avoid discrimination

30. and how do we know our
algorithms are OK?

31. Not just bias
safety – e.g. autonomous cars
democracy – e.g. social media, fake news
health and well being – e.g. soft-drink adverts
social issues – e.g. credit ratings

32. we need to ask
Why?
algorithmic transparency
c.f. court judgment

34. an AIX Kitbag
AI explainability
how to make sense of
black-box machine-learning algorithms

35. crucial insight …
human–human explanations
rarely utterly precise or reproducible
but are
sufficient to inspire confidence and trust

36. white-box black-box
grey-box
creating scructable
internal representations
analysing and
understanding
from the outside
peeking within
understanding
internal representations

38. but … this was all evident
25 years ago
why didn’t I do more?
if it is important
not sufficient to publish
you need to transform into
publicity and policy

40. white-box methods
creating scructable
internal representations

41. WB0. choose a white box classifier!
training set
scrutable
rules
white-box
algorithm
unseen data white-box classifier outputs

42. WB1. black-box generation of white box
classifier
training set
scrutable
rules
black-box
algorithm
unseen data white-box classifier outputs

43. WB2. Adversarial examples for white-box
learning
case-base of
behaviour scrutable
rules
black-box
adversarial learning
white-box
learning

44. WB3. Simplification of rule set
scrutable
rules
black-box
learning
training
set
inscrutable
rules tweak

45. black-box methods
analysing and understanding
from the outside

46. BB1. exploration analysis for human
visualisation
black-box
learning
training
set
inscrutable
rules
lots of
examples
black-box
classifier
visualise
input-output

47. BB2. perturbation/exploration analysis for
key feature detection
black-box
learning
inscrutable
rules
randomly vary
feature values
black-box
classifier
hotspot
visualisation

48. BB3. perturbation analysis for central and
boundary cases
lots of
examples
black-box
classifier
central and
boundary
cases
user
visualisation
white-box
learning

49. BB3. close up
central cases
perturbations
do not change class
boundary cases
small perturbations
change class
penumbra
larger perturbations
change class

50. BB4. black-box oracle – white-box learning
input
examples
black-box
classifier
scrutable
rules
white-box
learning
input–output
pairs as
training set
output
classes

51. grey-box methods
peeking within

52. GB0a. sensitivity analysis – weights
perturb parameters in
the inscrutable rules
lots of
examples
black-box
classifier
hotspot analysis
on parameters

53. GB0b. sensitivity analysis – activation
input
example
black-box classifier
(low level)
extract
intermediate
activation
black-box classifier
(high level)
perturb
activations
hotspot analysis
of nodes

54. GB0c. sensitivity analysis – algorithmic
apply black-box
algorithm
inverse
algorithm

55. GB1. high level model generation
input
examples
black-box
classifier
extract
intermediate
activation
scrutable
rules
white-box
learning
activations with
output class
as training set
output
classes

56. GB2. Clustering and comprehension of
low level
input
examples
black-box
classifier
extract
intermediate
activation
clusters
various
algorithms
activations
as input
MDS
SOM

57. GB3. triad distinctions
input
examples
black-box classifier
(low level)
A
B
C
hotspot analysis
of nodes
compare

58. GB4. apply generatively
output to input
activation to input
output to activation
between layers