Brian gives a light introduction to the master class on Deep Learning he will present at Predict.

1. 13/08/15
1
A
Short(er)
Introduc0on
To
Deep
Learning
Dr.
Brian
Mac
Namee
University
College
Dublin

2. 13/08/15
2
Deep Learning
Google Trends: http://www.google.com/trends/
2005 2007 2009 2011 2013 2015
Kaggle Digit Recogniser Contest
https://www.kaggle.com/c/digit-recognizer
MNIST Dataset from Yan LeCun
http://yann.lecun.com/exdb/mnist/index.html

3. 13/08/15
3
The standard approach to using machine learning
to build a system to recognise these different digits
is to first engineer a high-level respresentation
Percent filled: 0.37
Number of loops: 2
Direction 0: 0.2
Direction 1: 0.6
Direction 2: 0.1
Direction 3: 0.4
Percent filled: 0.11
Number of loops: 0
Direction 0: 0.1
Direction 1: 0.4
Direction 2: 0.0
Direction 3: 0.5
0
1
2
3
Percent filled: 0.29
Number of loops: 1
Direction 0: 0.2
Direction 1: 0.5
Direction 2: 0.4
Direction 3: 0.2
The standard approach to using machine learning
to build a system to recognise these different digits
is to first engineer a high-level respresentation
Percent filled: 0.37
Number of loops: 2
Direction 0: 0.2
Direction 1: 0.6
Direction 2: 0.1
Direction 3: 0.4
Percent filled: 0.11
Number of loops: 0
Direction 0: 0.1
Direction 1: 0.4
Direction 2: 0.0
Direction 3: 0.5
0
1
2
3
Percent filled: 0.29
Number of loops: 1
Direction 0: 0.2
Direction 1: 0.5
Direction 2: 0.4
Direction 3: 0.2
Using this reperesentation (6 features) we could
train a decision tree that would manage to correctly
recognise about 8 out of every 10 digits

4. 13/08/15
4
Engineering
representa0ons,
is
one
of
the
most
important
and
>me
consuming
jobs
in
most
predic>ve
analy>cs
projects,
and
needs
a
blend
of
technical
exper>se
and
domain
exper>se
Representa0on
learning
is
a
set
of
methods
that
allows
a
machine
to
be
fed
with
raw
data
and
to
automa>cally
discover
the
representa>ons
needed
for
detec>on
or
classifica>on
[LeCun
etal,
2014]
Deep Learning
Yann LeCun, Yoshua Bengio & Geoffrey Hinton
http://www.nature.com/nature/journal/v521/n7553/full/nature14539.html
Rosenbla='s
perceptron
from
1957
was
the
earliest
example
of
representa0on
learning,
and
the
first
neural
network

5. 13/08/15
5
Each image is composed of
28 x 28 = 784 pixels each
containing a grayscale
value between 0 and 255

6. 13/08/15
6
This low-level representation uses a
vector of 784 features, each with values
between 0 and 255

7. 13/08/15
7
0
1
2
3
4
5
6
7
8
9
A simple
representation learning
approach to the digit
recognition problem
could use a multi-
layer perceptron to
make predictions using
the low-level
representation
0
1
2
3
4
5
6
7
8
9

8. 13/08/15
8
0
1
2
3
4
5
6
7
8
9
This neural network
could manage to
correctly recognise
about 9 out of
every 10 digits
Deep-­‐learning
methods
are
representa0on-­‐
learning
methods
with
mul>ple
levels
of
representa>on,
obtained
by
composing
simple
but
non-­‐linear
modules
that
each
transform
the
representa>on
at
one
level
(star>ng
with
the
raw
input)
into
a
representa>on
at
a
higher,
slightly
more
abstract
level.
[LeCun
etal,
2014]
Deep Learning
Yann LeCun, Yoshua Bengio & Geoffrey Hinton
http://www.nature.com/nature/journal/v521/n7553/full/nature14539.html

9. 13/08/15
9
0
1
2
3
4
5
6
7
8
9
0
1
2
3
4
5
6
7
8
9
A deep learning
appraoch could
manage to correctly
recognise about 10
out of every 10
digits

10. 13/08/15
10
Deep
neural
networks
seem
to
brilliantly
address
the
selec0vity-­‐invariance
dilemma
that
is
fundamental
to
all
efforts
to
learn
to
classify
objects:
they
produce
representa>ons
that
are
selec>ve
to
the
aspects
of
the
image
that
are
important
for
discrimina>on,
but
that
are
invariant
to
irrelevant
aspects
Deep
networks
hold
records
for
problems
in
image
recogni0on,
speech
recogni0on,
and
text
classifica0on
amongst
other
areas
Hardware
Data
Algorithms
Applica>ons

11. 13/08/15
11
Thank You
Questions?
Fundamentals of Machine
Learning for Predictive Data
Analytics: Algorithms, Worked
Examples, and Case Studies
John D. Kelleher, Brian Mac Namee and
Aoife D'Arcy
www.machinelearningbook.com