The document describes the process of a basic neural network. It takes in input values which are multiplied by weights and summed to produce output values. It then describes different types of activation functions that can be applied such as threshold, sigmoid, rectifier and hyperbolic tangent. It also lists additional reading on deep sparse rectifier neural networks.

1. © SuperDataScienceDeep Learning A-Z

2. © SuperDataScienceDeep Learning A-Z
Input value 1
Input value 2
Input value m
y
2nd step:
X1
X2
Xm
w1
w2
wm
Output value
3rd step

3. © SuperDataScienceDeep Learning A-Z
y
0
1

4. © SuperDataScienceDeep Learning A-Z
0
y
Threshold Function
1

5. © SuperDataScienceDeep Learning A-Z
y
1
0
Sigmoid

6. © SuperDataScienceDeep Learning A-Z
0
y
Rectifier
1

7. © SuperDataScienceDeep Learning A-Z
0
y
1
-1
Hyperbolic Tangent (tanh)

8. © SuperDataScienceDeep Learning A-Z
Deep sparse rectifier
neural networks
By Xavier Glorot et al. (2011)
Link:
http://jmlr.org/proceedings/papers/v15/glorot11a/glorot11a.pdf
Additional Reading:

9. © SuperDataScienceDeep Learning A-Z
Input value 1
Input value 2
Input value m
y
2nd step:
X1
X2
Xm
w1
w2
wm
Output value
3rd step
If threshold activation function:
If sigmoid activation function:
Assuming the DV is binary (y = 0 or 1)

10. © SuperDataScienceDeep Learning A-Z
y
4
5
6
7
Input value 1
Input value 2
Input value m
Output value
Input
Layer
Hidden
Layer
Output
Layer
X2
X1
Xm