Featuring pointers for: Single-layer neural networks and multi-layer neural networks, gradient descent, backpropagation. Slides are for introduction, for deep explanation on deep learning, please consult other slides.

1. Artificial Neural Networks
Fariz Darari
(fariz@ui.ac.id)
Feel free to contact me for any partnership opportunities!

2. Organic Neural Network
• The human brain has about 1011 neurons
• Switching time 0.001s (computer ≈ 10-10s)
• Connections per neuron: 104 - 105
• 0.1s for face recognition!
• Strengths: Parallelism and distributedness

3. Biological Neurons
• Dendrit menerima input informasi dalam bentuk sinyal elektrik,
yang diakumulasikan di badan sel saraf (= soma).
• Ketika akumulasi informasi mencapai threshold tertentu,
sel saraf akan menembakkan output informasi
yang dikirim melalui axon
• Axon tersambung dengan dendrit yang ada
di badan sel saraf lain melalui synapses
• Pemelajaran/learning dilakukan dengan
adaptasi synaptical weight

4. McCulloch-Pitts Processing Unit (1943)
Contoh:

5. ANN: Basic Idea
• Artificial Neuron
• Each input is multiplied by a weighting factor
• Output is: 1 if sum of weighted inputs exceeds threshold;
0 otherwise
• Network is programmed by adjusting weights using feedback from
examples

6. Back to McCulloch-Pitts example...
Given that:

7. Back to McCulloch-Pitts example...
Given that:

8. Back to McCulloch-Pitts example...
is >= 0 (if the activation function called step function is used)

9. Generalization of McCulloch-Pitts Processing Unit

10. Activation Functions
g(x)

11. Activation Functions

12. When to leverage ANN?

13. Implementing logical functions

14. Single-layer NN vs. Multi-layer NN

15. ANN Learning

16. ANN Learning Using Gradient Descent

17. ANN Learning Using Gradient Descent
For an excellent step-by-step tutorial on Gradient Descent:
https://mccormickml.com/2014/03/04/gradient-descent-derivation/

18. Multi-layer NN

19. Multi-layer NN

20. NN with Two Layers

21. NN with Two Layers
3
a4 =

22. Backpropagation
1. Computes the error term for the output units using the observed error.
2. From the output layer, repeat:
• propagating the error term back to the previous layer, and
• updating the weights between the two layers
until the earliest hidden layer is reached.

23. Combining NN layers can create difficult shapes

24. Intuition: Multilayered neural networks

25. Intuition: Multilayered neural networks

26. Intuition: Multilayered neural networks

27. Intuition: Multilayered neural networks

28. Intuition: Multilayered neural networks

29. Intuition: Multilayered neural networks

30. Intuition: Multilayered neural networks

31. Intuition: Multilayered neural networks

32. Intuition: Multilayered neural networks

33. Intuition: Multilayered neural networks

34. Intuition: Multilayered neural networks

35. Intuition: Multilayered neural networks

36. Neural network in action: Iris classification

37. Neural network in action: Iris classification

38. Neural network in action: Iris classification

39. Neural network in action: Facial recognition

40. Neural network in action: Facial recognition

41. Neural network in action: Facial recognition

42. Neural network in action: Facial recognition

43. Neural network in action: AlphaGo
More info about CNN:
https://towardsdatascience.com/the-most-intuitive-and-easiest-guide-for-convolutional-neural-network-3607be47480

44. Neural network in action: Self-driving car
More info about CNN:
https://towardsdatascience.com/the-most-intuitive-and-easiest-guide-for-convolutional-neural-network-3607be47480