1. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
Andrew
Jeavons
Mass
Cogni+on
18
August
2016
An
Introduc+on
to
Neural
Networks
2. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
The
Beginning
• “A
Logical
Calculus
of
the
ideas
immanent
in
Nervous
Ac2vity”
– McCulloch
and
PiDs,
1943
• Organiza2on
of
neurons
and
logic.
– Mimicking
organiza2on
of
neurons
in
brain
• Beginning
of
“neural
networks”
approach
– But
not
a
simple
path.
3. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
Neuron:
All
or
Nothing
What
comes
out
is
not
necessarily
what
goes
in.
“Transfer
Func2on”
–
controls
what
the
output
is.
4. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
Ar+ficial
Neuron:
All
or
Nothing
5. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
Ar+ficial
Neuron
SoTware
“copy”
of
the
structure
of
a
physical
neuron.
Much
simplified,
many
aspects
of
neuron
func2on
were/are
hard
to
simulate.
Arranged
into
more
complex
mul2
layer
structures.
6. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
ANN:
Backpropoga+on
Network
7. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
Types
of
Training:
Supervised
Supervised:
Weights
adjusted
itera2vely
to
bring
down
the
difference
between
the
output
generated
by
the
ANN
by
the
training
data
set
and
the
test
value.
For
supervised
learning
divide
training
data
set
into
train
and
test
segments.
ANN
learns
on
the
training
data
set
and
tests
this
training
on
the
test
dataset.
8. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
Types
of
Training:
Unsupervised
Usually
different
network
structure.
For
example
a
Kohonen
Self
Organizing
Map:
All
inputs
connect
to
ALL
outputs.
Can
have
hidden
layers.
Acts
as
a
classifier,
produces
a
map
(similar
to
segmenta2on)
of
the
input
data.
9. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
Types
of
Training:
Unsupervised
Example
of
Kohonen
Map:
10. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
Types
of
Training:
Unsupervised
Select
an
input
at
random
Find
which
ar2ficial
neuron
is
most
similar
based
on
weight
value.
Adjust
values
of
“winner”
neuron
to
be
close
to
Input.
Keep
doing
this.
Then
you
can
map
a
new
input
“vector”
and
get
classifica2on.
11. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
• Lots
of
arithme2c.
– Video
games
GPU’s
have
helped
alleviate
this
problem.
• Complex
– As
bigger
memory
capacity
became
available
bigger
networks
feasible.
• Coding
of
input.
• Architecture
selec2on.
– Thousands
of
architectures
available.
ANN:
Challenges
12. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
ANN:
New
and
Interes+ng
• “Genera2ve”
Networks.
– Produce
new
output
rather
than
classifica2on
and
predic2on.
• Large
scale
networks
for
text
analysis.
– Google
word2vec
and
doc2vec.
13. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
ANN:
LSTM
Recurrent
ANN
• Long
Term
Short
Term
Memory
ANN.
– Used
to
generate
a
short
movie.
– Trained
on
scripts
of
Scifi
movies/TV
shows
on
interweb.
– (hDp://arstechnica.com/the-‐mul2verse/2016/06/an-‐ai-‐
wrote-‐this-‐movie-‐and-‐its-‐strangely-‐moving/)
– AI
came
up
with
its
own
name,
Benjamin,
in
an
interview.
– Biologically
Plausible
ANN,
uses
cogni2ve
psychology
constructs.
14. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
ANN:
Doc2vec
• Word2vec/doc2vec
– Take
a
word
or
document
– Create
a
“vector”
(list
of
numbers)
300+
long.
– All
vectors
are
fixed
length
– Vectors
can
be
compared
• word2vec
can
show
“odd
man
out”
– “cat,
dog,
dolphin,
chair”
– Uses
“distance”
between
vectors
to
decide
odd
man
out
15. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
ANN:
Doc2vec
• Doc2vec
does
the
same
thing
for
documents.
– Converts
documents
to
fixed
length
vectors.
– Can
calculate
the
similarity
of
documents.
– Can
be
interrogated
as
in
“which
color
is
best”
• Prints
out
most
similar
documents
to
input
.
• Input
phrase
does
not
have
to
be
in
text
corpus
trained
with.
• Encodes
seman2cs.
• Poten2al
for
mapping
and
exploring
text
• Computa2onally
intensive
!
16. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
Thank
You!
www.masscogni2on.com
apj@masscogni2on.com
17. An
Introduc+on
to
Neural
Networks
–
Agility,
Automa+on
&
AI
Andrew
Jeavons,
Mass
Cogni2on,
2016
Q
&
A
Ray
Poynter
The
Future
Place
Andrew
Jeavons
Mass
Cogni2on
