hacker's guide to neural networks or Data Driven Code -101 is a bold attempt to teach basics of computation and mathematics required for a newbie to get started with playing around with a neural network

1. Hacker’s Guide to Neural Networks
a. k. a.
Data Driven Code – 101
Anoop Thomas Mathew
@atmb4u
PyCon Canada 2016

2. WHAT WE WILL NOT COVER
• Recent developments in
• United States of America! ;)
• Deep Learning, CNN, RNN, DCGAN etc.
• Frameworks like TensorFlow, Theano, Keras etc.
• Advanced concepts of neural networks

3. WHAT WE WILL COVER
• Basic Concepts of
• Parameter Optimization
• Entropy (Sparse Coding)
• Little bit of Mathematics
• Linear Algebra – Matrix Multiplication
• Differential Calculus – Sigmoid Function (2 equations only)
• (Try to) Build a 2 layer neural network
• Way Forward

4. PARAMETER OPTIMIZATION
• Infinite Monkey Theorem
• Any problem is fundamentally a parameter optimization problem
𝑎𝑥3 + 𝑏𝑥2 + 𝑐𝑥 + 𝑑 = 0

5. 0/1
noise bit
data bits
28 = 256
Imagine:
• divided into 4 spaces
• each bit – a feature
• generalize data bits
• attenuates noise bit
ENTROPY
0/1
0/1
• Sparse Coding

6. LITTLE LINEAR ALGEBRA
𝑤1 𝑤2
𝑤3 𝑤4
×
𝑥1
𝑥2
=
𝑤1 𝑥1 + 𝑤2 𝑥2
𝑤3 𝑥1 + 𝑤4 𝑥2

7. LITTLE LINEAR ALGEBRA (matrix multiplication)
𝑤1 𝑤2
𝑤3 𝑤4
×
𝑥1
𝑥2
=
𝑤1 𝑥1 + 𝑤2 𝑥2
𝑤3 𝑥1 + 𝑤4 𝑥2

8. ACTIVATION FUNCTION (differential calculus part)
σ(x) =
/
1+e−x
(Sigmoid Function)
0 ∂σ(x)
02
= σ(x) × (1−σ(x))
(Derivative of Sigmoid Function)
Stochastic Gradient Decent

9. QUICK RECAP
• Activation Function(combining inputs to a representation)
• Layers (Weight Matrices)
• Network (collection of weight matrices)
• Dataset (inputs and expect outputs)
• Features (patterns in dataset)
• Sparse Encoding (roomfor error)
• Backpropagation (learn from past mistakes)
• Parameter Optimization (correct values into weight matrix)

10. LET’S MAKE ONE NOW !
CODE + DATA = MAGIC
(Teach machine XOR truth table)

11. XOR GATE

12. SHOW ME THE
CODE

14. DEEP LEARNING

15. DEEP LEARNING

16. WAY FORWARD
Play with Neural networks in the browser
http://playground.tensorflow.org/
Comprehensive list of resources available online on Deep Learning
https://github.com/ChristosChristofidis/awesome-deep-learning
Very active online Machine Learning Community
https://www.kaggle.com/

17. REFERENCE LINKS / IMAGE COURTESY
• https://medium.com/technology-invention-and-more/how-to-build-a-simple-
neural-network-in-9-lines-of-python-code-cc8f23647ca1
• https://medium.com/technology-invention-and-more/how-to-build-a-multi-
layered-neural-network-in-python-53ec3d1d326a#.dis92by0e
• http://iamtrask.github.io/2015/07/12/basic-python-network/
• http://iamtrask.github.io/2015/07/27/python-network-part2/
• http://sebastianraschka.com/faq/docs/logisticregr-neuralnet.html

18. THANK YOU
// atm@infiniteloop.in
• Code - https://github.com/atmb4u/data-driven-code
• Slides – https://slideshare.net/atmb4u/data-driven-code
• Medium - https://medium.com/@atmb4u/data-driven-code-101-17003f7b88ec