The document discusses the fundamentals of artificial neural networks (ANNs) by comparing them to biological neural networks, highlighting key aspects such as neuron structure, learning processes, and the significance of activation functions. It explains the McCulloch-Pitts processing unit and introduces techniques like backpropagation and gradient descent for training multi-layered networks. Various applications of neural networks, including face recognition and self-driving cars, are also mentioned.

1. Artificial Neural Networks
Fariz Darari
(fariz@ui.ac.id)
Feel free to contact me for any partnership opportunities!

2. Organic Neural Network
• The human brain has about 1011 neurons
• Switching time 0.001s (computer ≈ 10-10s)
• Connections per neuron: 104 - 105
• 0.1s for face recognition!
• Strengths: Parallelism and distributedness

3. Biological Neurons
• Dendrit menerima input informasi dalam bentuk sinyal elektrik,
yang diakumulasikan di badan sel saraf (= soma).
• Ketika akumulasi informasi mencapai threshold tertentu,
sel saraf akan menembakkan output informasi
yang dikirim melalui axon
• Axon tersambung dengan dendrit yang ada
di badan sel saraf lain melalui synapses
• Pemelajaran/learning dilakukan dengan
adaptasi synaptical weight

4. McCulloch-Pitts Processing Unit (1943)
Contoh:

5. ANN: Basic Idea
• Artificial Neuron
• Each input is multiplied by a weighting factor
• Output is: 1 if sum of weighted inputs exceeds threshold;
0 otherwise
• Network is programmed by adjusting weights using feedback from
examples

6. Back to McCulloch-Pitts example...
Given that:

7. Back to McCulloch-Pitts example...
Given that:

8. Back to McCulloch-Pitts example...
is >= 0 (if the activation function called step function is used)

9. Generalization of McCulloch-Pitts Processing Unit

10. Activation Functions
g(x)

11. Activation Functions

12. When to leverage ANN?

13. Implementing logical functions

14. Single-layer NN vs. Multi-layer NN

15. ANN Learning

16. ANN Learning Using Gradient Descent

17. ANN Learning Using Gradient Descent
For an excellent step-by-step tutorial on Gradient Descent:
https://mccormickml.com/2014/03/04/gradient-descent-derivation/

18. Multi-layer NN

19. Multi-layer NN

20. NN with Two Layers

21. NN with Two Layers
3
a4 =

22. Backpropagation
1. Computes the error term for the output units using the observed error.
2. From the output layer, repeat:
• propagating the error term back to the previous layer, and
• updating the weights between the two layers
until the earliest hidden layer is reached.

23. Combining NN layers can create difficult shapes

24. Intuition: Multilayered neural networks

25. Intuition: Multilayered neural networks

26. Intuition: Multilayered neural networks

27. Intuition: Multilayered neural networks

28. Intuition: Multilayered neural networks

29. Intuition: Multilayered neural networks

30. Intuition: Multilayered neural networks

31. Intuition: Multilayered neural networks

32. Intuition: Multilayered neural networks

33. Intuition: Multilayered neural networks

34. Intuition: Multilayered neural networks

35. Intuition: Multilayered neural networks

36. Neural network in action: Iris classification

37. Neural network in action: Iris classification

38. Neural network in action: Iris classification

39. Neural network in action: Facial recognition

40. Neural network in action: Facial recognition

41. Neural network in action: Facial recognition

42. Neural network in action: Facial recognition

43. Neural network in action: AlphaGo
More info about CNN:
https://towardsdatascience.com/the-most-intuitive-and-easiest-guide-for-convolutional-neural-network-3607be47480

44. Neural network in action: Self-driving car
More info about CNN:
https://towardsdatascience.com/the-most-intuitive-and-easiest-guide-for-convolutional-neural-network-3607be47480