This document provides an incomplete history of machine learning from 1946 to 2016. It describes some of the major developments in the field including the first general purpose computer (ENIAC), Arthur Samuel creating the first machine learning program to play checkers in 1955, the development of the perceptron in 1958, Marvin Minsky's influential work establishing limits of perceptrons, the AI winter from 1970-1980, the rediscovery of backpropagation in the 1980s reigniting neural networks research, support vector machines gaining popularity in the 1990s, IBM's Deep Blue beating Garry Kasparov at chess in 1997, advances in image recognition with challenges like ImageNet, AlphaGo defeating top Go players in 2016, and Geoffrey Hinton's vision

1. Incomplete History of
Machine Learning
1946 - 2016
Bob Colner

2. General AI is the quest to build
intelligent machines.
Machine learning is a branch of AI
focusing on systems that learn
from their environment (i.e. data).
The goal is to generalize this
training to act intelligent in new
environments.
Machine Learning
&
Artificial Intelligence

3. The Quest for Artificial
Intelligence begins with the
first computer.

4. 1946
ENICA
Electronic Numerical
Integrator & Computer
First general purpose digital
computer. Powered by vacuum
tubes
Built by US Army, used by John von
Neumann to develop the H-Bomb.
Heralded as a Giant Brain by the
press.
Prompted Alan Turing to devise a
test to detect artificial intelligence.
The Turing-Test has yet to be
definitively passed.

5. 1955
Arthur Samuel is recognized as the
first learning machine which leaned
to play (and win) checkers.
His algorithms used a heuristic
search memory to learn from
experience.
By the mid 1970’s his program was
beating capable human players.
Give machines the ability to
learn without explicitly
programming them
- Arthur Samuel, 1955
“

6. 1958
The Perceptron was the first
artificial neural network.
Developed by Frank Rosenblatt at
the US office of Naval Research for
visual recognition tasks.
The New York Times reported the
perceptron to be: "the embryo of an
electronic computer that will be able
to walk, talk, see, write, reproduce
itself and be conscious of its
existence."Perceptron
Mark I

7. 1969
“ Cognitive scientist in the field of AI.
Co-founder of the MIT AI lab.
Published influential book
describing fundamental limits of
the perceptron (and all 2 layer
neural networks).
This was a bummer.
Will robots inherit the
Earth? Yes, but they will
be our children.
- Marvin Minsky

8. AI Winter
1970-1980

9. In the mid 1980’s multiple people
independently (re)discovered the
Backpropagation algorithm.
Allowed more powerful neural
networks with hidden layers to be
trained.
Reinvigorated research; many
people excited about Neural Nets as
model for the mind/brain
(connectionism) & commercial
applications.
1985
Neural Network
Breakthrough

10. 1995
By the 1990’s neural networks had
fallen out of favor (again).
Lack of good theory, tendency to
overfit, biologically implausible.
New methods like Support Vector
Machines are in-vogue.
SVM are amenable to rigorously
mathematical analysis and achieve
state-of-the art performance.
Statistical Approaches to
Machine Learning

11. IBM’s Deep Blue beats Chess Grandmaster
Garry Kasparov. (Brute Force != ML).
1997

12. The ImageNet
Challenge
Classify 1M Images
into 1K Categories
Microsoft’s deep rectified model
exceeds human accuracy in 2015

13. What are
DeepNets
Doing?

14. Google’s AlphaGo Model defeats
Euro Go Campaign
2016

15. Boomtrain Editor
2016

16. Looking to the (Near) Future
2015 Royal Society Keynote:
Geoffrey Hinton