Machine learning algorithms are being used to solve complex problems by playing games. DeepMind developed AlphaGo which was able to beat world champions at the games of Go, Chess, and Shogi by teaching itself without human programming. AlphaGo also defeated top human players in StarCraft 2 using a combination of deep neural networks and tree search methods. While machine learning has achieved successes, it also faces limitations such as bias in training data and lack of human-level judgment in complex ethical scenarios.

1. Machine Learning 101
(or less)
By Jonathan Sinclair

2. The Journey
• The Games
• The Tech
• The Application
• The Critique

3. AI is everywhere
• Searched “AI in the news” 12.03.19
• Bing: 90,200,000 Results
• Google: 3,800,000,000 Results
• News headlines
• Jobs
• “Kai Fu Lee (artificial intelligence expert and venture capitalist) said that he believes 40% of the world’s jobs will be replaced by
robots capable of automating tasks. He said that both blue collar and white collar professions will be affected, but he believes
those who drive for a living could be most affected.”
• Autonomous weapons:
• Obama: “I recognize that the potential development of lethal autonomous weapons raises questions that compliance with
existing legal norms—if that can be achieved— may not by itself resolve, and that we will need to grapple with more
fundamental moral questions about whether and to what extent computer algorithms should be able to take a human life.” –
Letter dated January 16, 2017.
• Autonomous vehicles:
• 9% of the cars on the road will have detection/response driverless integration in 2020.
• 25% of traffic will be autonomous vehicles in 2030.
• 50% of traffic will be autonomous vehicles in 2040.
• Driverless cars will be available worldwide by 2064.
• 95% of traffic will be autonomous vehicles in 2070.

4. Backgammon
In 1979 Hans Berliner programmed the
computer program BKG 9.8 which beat
the reigning Backgammon world campion
Luigi Villa by a score of 7-1

5. Checkers/Draughts
In 1989 Jonathan Schaeffer, Robert Lake,
Paul Lu and Martin Bryant programmed
the computer program Chinook.
In 1992 Chinook took on world champion
Marion Tinsley who between 1950 and
1992 only lost 5 games. During
competition Tinsley won 4-2, with 33
draws. Rematch in 1994 was postponed
due to Tinsley’s health.
In 2007 the team announced that
Chinook had developed “perfect play”
which resulted in a win or a draw, never a
loss.

6. Chess
In 1996 IBM research developed the chess
playing program: DeepBlue which played
the world champion Gary Kasperov.
In the first game Kasperov won 4-2.
A rematch took place in 1997 where
DeepBlue won 3.5 – 2.5

7. Games: Machines 1; Humans 0
• Observations from 1997:
• “To play a decent game of Go, a computer must be endowed with the ability to
recognize subtle, complex patterns and to draw on the kind of intuitive knowledge
that is the hallmark of human intelligence.” – George Johnson, NY Times
• “It may be a hundred years before a computer beats humans at Go -- maybe even
longer‘’ -- Dr. Piet Hut, an astrophysicist at the Institute for Advanced Study in
Princeton, N.J.
• “When or if a computer defeats a human Go champion, it will be a sign that artificial
intelligence is truly beginning to become as good as the real thing. Go is the highest
intellectual game‘’ -- Dr. Chen Zhixing, a retired chemistry professor at Zhongshan
University, in Guangzhou, China.

8. GO
In 2015 DeepMind’s AlphaGo program
beats European Go champion Fan Hui
(2nd-dan) 5-0
In 2016 AlphGo beats one of the all time
best Go players Lee Sedol (9th-dan) 4-1
(ranked by some as the 4th best player in
the world)
In 2017 AlphaGo Master beats Ke Jie
(world ranked number 1 player) 3-0

9. Chess/Go/Shogi
In 2017 DeepMind releases AlphaGo
Zero which teaches itself the games
of: Chess, Shogi and Go which goes
on to beat world champions in each

10. Starcraft II
In 2018 DeepMind develops AlphaStar
designed to play the real-time strategy
game StarCraft 2. In December it beat
Team Liquid’s: Grzegorz "MaNa“ Komincz
5-0

11. Why games
• Games are important because:
• They provide a constrained area that is able to:
• Allow inference
• Work within constraints
• Study opponents
• Exploit rules
• Game Theory: Universality
• Decision Theory
• Rational Choice theory
• “We wish to find the mathematically complete principles which define “rational behavior” for the
participants in a social economy, and to derive from them the general characteristics of that behavior”
(von Neumann and Morgenstern 1944, 31).
• Predictability!

12. By way of demonstration
• DeepMind’s algorithms are being deployed on problems such as:
• Data Centre cooling optimisation
• Reduction of 40%
• Wind power applied to the US wind farm network
• Improved predicative model delivering 20% increased performance

13. By way of demonstration
• Microsoft and their smart farm project (show video)

14. How Is This Possible?

15. Rise and demise of neural networks
• Walter Pitts & Warren McCullough first provide a mathematical model of
an artificial neuron
• Frank Rosenblatt: Developed the perceptron
• Minsky & Papert:
• Wrote the landmark book: Perceptron's which talked to the limits of ‘connectionism’
and partly helped in instigated the AI Winter (1970 and then again in 1980)

16. The AI winter and a Phoenix from the ashes
• The discovery of back propagation networks.
• New marketing drives:
• Support Vector Machines
• Neural Nets
• Machine Learning
• Fuzzy Logic controllers
• Expert systems
• Markov models
• Agent-based systems

17. Types of Machine Learning

18. How is this possible?

19. So how: Machine Learning
• No predefined rules:
• if..this..then…that
• Dynamic rules, self-programmed to determine statistical correlations
based on some success function: Feedback Loop is key

20. Ohh and a LOT of computing power
• Leverage and maturation of Field-programmable gate array (FGPA’s)
chips
• The re-purposing of GPU’s (graphical processing units) for deep
learning training
• The continued miniaturisation of transistors
• Example: AlphaGo = 1,202 CPUs, 176 GPUs

21. A combinatory approach
• Convolutional Neural Network:
• Markov Decision process
• A mathematical way to make decisions in noisy situations where not all knowledge is known
by the decision maker
• Monte Carlo tree search
• A way to randomly sample a decision tree and determine probabilistically promising moves

22. In context

23. Health care
• Computer aided diagnosis of breast cancer on mammograms (1997)
• Beats the human
• DeepMind and Moorfields Eye Hospital (2016)
• Automatic detection of eye diseases
• Beats the human
• Babylon Health (2018)
• Beats humans on a clinical exam (82%)
• AI model beats humans at predicating heart disease (2018)
• AI beats human doctors in neuroimaging recognition contest (2018)

24. Automated molecular design
• Automating molecule design to speed up drug development
• “Researchers from MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and
Department of Electrical Engineering and Computer Science (EECS) have developed a model that better
selects lead molecule candidates based on desired properties. It also modifies the molecular structure
needed to achieve a higher potency, while ensuring the molecule is still chemically valid.”
• How artificial intelligence is changing drug discovery
• “Machine learning and other technologies are expected to make the hunt for new pharmaceuticals
quicker, cheaper and more effective.”

25. IT makes this happen
• We are IT and we can transform the world!
• Chemists aren’t doing this
• Physicists aren’t doing this,
• Social scientists aren’t doing this,
• Mathematicians aren’t doing,
• Algorithms ARE doing this, computation IS doing this. WE do
this!

26. Where it fails
• Tank example
• 1980’s Pentagon Tank identification NN
• 100% success in the lab concerning tank vs. no tank
• Independent tests showed the results were no better than random sampling
• Result after extensive analysis
• “Eventually someone noticed all the images with tanks had been taken on a cloudy day while all the images without tanks had
been taken on a sunny day”
• The neural network had been asked to separate the two groups of photos and it had chosen the most obvious way to do it –
not by looking for a camouflaged tank hiding behind a tree, but merely by looking at the colour of the sky”

27. Tesla
• How it kills:
• Gao Yaning, 23: Jan 20, 2016
• Model S slams into a road sweeper on a highway near Handan
• Joshua Brown, 40: May 7, 2016
• National Highway Traffic Safety Administration stated: the “crash occurred when a
tractor-trailer made a left turn in front of the Tesla, and the car failed to apply the
brakes.”
• News release stated: “Neither autopilot nor the driver noticed the white side of the
tractor-trailer against a brightly lit sky, so the brake was not applied.”
• How it saves:
• Tesla Autopilot drives fatally injured man 20 miles to hospital
• Tesla Model S a 5-star safety rating in every category

28. Bias
• Machine learning is about bias, implicit or otherwise
• Training Set Poisoning
• Ethical quandaries are now highlighting frictions between
local vs. global norms and machine learning is bringing this
to the forefront, forcing humanity to face up to its varied and
incompatible value systems

29. Societal questions (for humanity and AI)
• What would an AI do when:
• The Trolley Problem:
• You see a runaway trolley moving toward five tied-up people lying on the tracks.
• You are standing next to a lever that controls a switch.
• If you pull the lever, the trolley will be redirected onto a side track, and the five people
on the main track will be saved. However, there is a single person lying on the side track.
You have two options:
• Do nothing and allow the trolley to kill the five people on the main track
• Pull the lever, diverting the trolley onto the side track where it will kill one person.
• Mother or child:
• One is stranded in the ocean and there is only enough food for you and another but
there are three of you in the boat. One male, one female and one child.
• Who’s sacrificed?

30. Last thoughts
• AI/ML:
• We’re no further forward in understanding strong AI
• We can’t talk to and/or won’t even recognise other intelligences
• Corvid, Cetacean, Mammalian
• We’ve got very good at applying statistical methods to human
problems, which is enjoying a huge amount of success

31. Backup

32. Chess
• Number of possible moves: ~20

33. Go
• Number of possible moves: ~200

34. AI on the march?
• AI vs. Machine learning
• What is AI (Artificial Intelligence?)  Everything and nothing:
• It’s:
• Weak AI – An expert system (one facet of the intelligence cube)
• Strong AI – AI that truly embodies what is generally recognised as ‘intelligent’ behaviour
• Symbolic AI:
• The manipulation of symbols to deduce logic
• Neural Networks (modern ML):
• Biologically inspired classifying nodal units

35. Failure of the Symbol system
• General Problem Solver-
• Simon, Shaw, Newell
• SHRDLU – natural language processing system for performing tasks in
a block world
• Cyc – failed symbol problem