This document introduces artificial neural networks (ANNs) and convolutional neural networks (ConvNets). ANNs are modeled after biological neurons and learn through training algorithms like supervised learning. ConvNets are a type of ANN used for image analysis. They apply convolutional kernels to detect features like edges. ConvNets have multiple convolutional and pooling layers that allow for translation and rotation invariant feature detection. Examples of ConvNet applications include image classification, speech recognition, and data analysis.

1. Introduction to Artificial Neural Networks
and Convolutional Neural Networks
Aiko Klostermann
October 4, 2016
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 1 / 19

2. Agenda
1 Artificial Neural Networks
Origin
Behaviour
Learning
2 Convolutional Neural Networks
Structure
Usage
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 2 / 19

3. What are Neurons?
Neuron:
cell that processes signals
The human brain has
about 100.000.000.000
(100 billion)
linked with other neurons
via synapses / axons Figure: Visualisation of a human
neuron
typically multiple inputs and one output
If the input reaches a certain threshold the neuron fires
serves as a model for artificial neurons
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 3 / 19

4. Artificial Neurons
Figure: Artificial Neuron
The output activation is calculated by multiplying each input with its
weight and summing all of them up.
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 4 / 19

5. Artificial Neural Network(ANN)
Figure: structure of an
ANN
1 Input Layer
1 Output Layer
n Hidden Layer
(Deep Learning: many)
Neuron linked to every neuron of
the previous and following layer
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 5 / 19

6. ANN Blackbox Behaviour
Figure: Representation of the black box behavior of an ANN with example input a
of black / white picture showing the (handwritten) number 6
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 6 / 19

7. Example
Figure: Caffe Demo Screenshot
Links:
1 Image Search ”bernese mountain dog”
2 Caffe Demo
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 7 / 19

8. But how does the network know this?
first of all every network has to be trained
In the course of training: the weights are adjusted (gradient descent)
In general: neural networks approximate a function
Most common method:
Supervised learning
Training with input and expected oputput
train the network until it generates the expected output for each
given input
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 8 / 19

9. Alternative methods of training
Reinforced learning
No expected output
feeback based on the output
Unsupervised learning
no expected output or feedback given
recognizes pattern
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 9 / 19

10. Convolutional Neural Networks (ConvNets)
Convolution refers to a
technique that is often
used in image processing
where a convolution
kernel is applied to
several pixels in the
original image to
calculate a pixel in the
target image.
Figure: Function of a Convolutional
Kernel
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 10 / 19

11. Edge detection
Figure: Convolutional kernel
left: Convolutional kernel for
edge detection
bottom: edge detection
applied to a grayscale image of
a house
Figure: Detecting edges with convolutional kernel
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 11 / 19

12. Structure of ConvNets
Figure: Structure of a ConvNet
Figure: Feature maps of a ConvNet
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 12 / 19

13. ConvNet: Feature detection in consecutive layers
Figure: Feature detection in consecutive layers
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 13 / 19

14. Advantages over usual neural networks
Why Convolutional?
Optimized for the classification of 2-dimensional input
robust character recognition
insensitive to:
Rotation
Translation
Scaling
Line thickness
robust face detection
and much more
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 14 / 19

15. Areas of application
Image classification
People, Faces, Traffic signs, Passers-by (autonomous vehicles), Species,
OCR-Systems, Signature checking
Audioclassification
Speech to Text, Identify feelings (Service robots), QA for ceramics, Engine
diagnostics
Data analysis and prediction
Weather forecast, stock analysis, creditworthiness analysis and insolvency
test
Optimization
”Knapsack Problem”, ”Traveling Salesman Problem”
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 15 / 19

16. Live Example
Figure: TensorFlow Demo App classifying a Laptop
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 16 / 19

17. Googles so called ’Deep Dream’
Figure: Photo of Toast(left) and ”seen” by Google’s DeepDream(right)
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 17 / 19

18. Conclusion / Future of ConvNets
The training data must be screened and classified (for supervised
learning) [Big Data]
high resource requirements
Training parameters, number and size of the layers must be
determined experimentally
Only approximation, no certainty!
Easy evaluation of personal data. e.g. already customer classification
and tracking in the supermarket.[Big Data]
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 18 / 19

19. Any Questions?
Aiko Klostermann Intro to ANNs and ConvNets October 4, 2016 19 / 19