Brief introduction to explainable

1. XAI – Machine Accountability
explored
By J.Sinclair

2. What is explainable AI (XAI)
• No universally recognised definition of the term
XAI: The ability for humans to understand how computational
machines develop their own models for solving tasks

3. The growing interest: XAI
• DARPA, 2016: Explainable AI project led by David Gunning
• Recognition of a problem around accountability
• Nvidia, 2016:
• End to End Learning for Self-Driving Cars paper published
• Accenture, 2017: Responsible AI imperative
• Design – Architect and deploy AI with trust (e.g., privacy, transparency and security)
by design built in, including building systems that lead to “explainable” AI.
• International Joint Conference on Artificial Intelligence, 2017:
• Workshop on Explainable Artificial Intelligence (XAI)

4. Techniques Now
• Numerous technologies after associated with machine learning
• Deep Learning, Random Forest, Bayesian Nets, Graphical, Models SVMs,
Regression
Taken from slide 9 of: Explainable Artificial Intelligence (XAI) David Gunning DARPA/I2O

5. The Gap
• Machine learning models are opaque and difficult to understand
• Explainability is open to interpretation
• Accountability very difficult to ascertain
• Machine learning design error (function thresholds)
• Training data-set issue
• Environmental context
• User behaviour
• Chatbot Tay (MS)
• Etc.
• Set-up, configured, trained, evaluated

6. Techniques for future XAI – Explainable
models
• The introduction of interpretable models that enhance underlying
machine learning cognitive models will provide increased levels of
transparency
Taken from slide 12 of: Explainable Artificial Intelligence (XAI) David Gunning DARPA/I2O

7. How does AI think?
• Deep Dream
• A mechanism of pictorially showing how machine learning algorithms are
deriving their feature sets.
• This is particularly interesting when trying to understand how an image
classification has occurred
• By running a trained-algorithm in reverse against random noise, key features
of a trained network will be identified through back-propagation
Images taken from: https://web.archive.org/web/20150703064823/http://googleresearch.blogspot.co.uk/2015/06/inceptionism-going-deeper-into-neural.html

8. How does AI think?
• Alpha Go
• Game Two: Move 37
• Commentary around this move:
• Michael Redmond (9th dan): “creative and unique“
• An Younggil (8th dan):
• "a rare and intriguing shoulder hit“
• "So when AlphaGo plays a slack looking move, we may regard it as a mistake, but perhaps it should more
accurately be viewed as a declaration of victory?
• How and why did the machine decide to make this move?
• AlphaGo lead researcher David Silver:
• “AlphaGo learned to discover new strategies for itself, by playing millions of games between its neural networks,
against themselves, and gradually improving”
• No justification for reasoning:
• David Silver: “AlphaGo had calculated that there was a one-in-ten-thousand chance that a human
would make that move. But when it drew on all the knowledge it had accumulated by playing itself so
many times—and looked ahead in the future of the game—it decided to make the move anyway. And
the move was genius.”

9. Blast from the past
• 1980’s Pentagon Tank identification NN
• 100% success in the lab concerning tank vs. no tank
• Independent tests showed the results were no better than random sampling
Result after extensive analysis:
“Eventually someone noticed all the images with tanks had been taken on a cloudy day
while all the images without tanks had been taken on a sunny day.
The neural network had been asked to separate the two groups of photos and it had
chosen the most obvious way to do it - not by looking for a camouflaged tank hiding
behind a tree, but merely by looking at the colour of the sky.”

10. Tesla
• How it kills:
• Gao Yaning, 23: Jan 20, 2016
• Model S slams into a road sweeper on a highway near Handan
• Joshua Brown, 40: May 7, 2016
• National Highway Traffic Safety Administration stated: the “crash occurred when a
tractor-trailer made a left turn in front of the Tesla, and the car failed to apply the
brakes.”
• News release stated: “Neither autopilot nor the driver noticed the white side of the tractor-
trailer against a brightly lit sky, so the brake was not applied.”
• How it saves:
• Tesla Autopilot drives fatally injured man 20 miles to hospital
• Tesla Model S a 5-star safety rating in every category

11. Tesla – NHTSA report
• A word of caution: XAI no where to be seen
• “NHTSA’s examination did not identify any defects in the design or
performance of the AEB or Autopilot systems of the subject vehicles nor any
incidents in which the systems did not perform as designed”
• Response:
• NESTA report: “Machines that learn in the wild” states:
• “Machine learning algorithms are an integral part of driverless cars and will have an
increasingly important role in their operational ability” p.10

12. Workshop:
Automated vehicles
• Scenario:
• You get into your autonomous vehicle and it drives you to work. On the way it
veers off the road and drives you into a coffee shop
Taken from: http://www.telegraph.co.uk/news/2017/03/23/rotary-club-treasurer-ploughed-coffee-shop-killing-woman-still/

13. Automated vehicles
• Discuss and consider:
• Which parties are liable?
• Where does fault lie?
• How could the event have arisen?
• What are the key concerns?
• Legal issues?
• How are vehicles viewed?

14. Why is Accountability Important?
• It provides the interaction between:
• People  Technology
• Government
• Manufacturer
• User
• The law
• USA: NHTSA
• UK: Department for Transport
• Germany:
• Intelligent Agent and it’s position in society: “Principle of blameworthiness”
• Vehicles as dangerous objects