The document outlines the fundamentals of deep learning, discussing various neural network types such as feed forward, recurrent, and convolutional networks. It highlights practical applications in diverse industries, including self-driving cars and chatbots, and emphasizes best practices for building AI systems. Additionally, the document includes hands-on sessions for practical learning using datasets like iris and mnist.

1. Deep Learning
Fundamentals
Satnam Singh, PhD
Chief Data Scientist
May 29, 2018

2. Outline
• What is deep learning?
• Feed Forward Neural Network (FFNN)
• Recurrent Neural Network (RNN)& LSTM
• Convolutional Neural Network (CNN)
• How to build a deep learning network?

3. Applications of Deep Learning
• Self-Driving Cars
• Alexa
• Chatbots
• Almost every industry has use cases
• Pls, share use cases from your industry….

4. ML vs. DL

5. ML vs. DL

6. What is Deep Learning?

9. Single Neuron

10. Single Neuron - “AND” node

11. Single Neuron - “OR” node
How to
compute the
weights?

12. Forward Propagation - 1

13. Forward Propagation - 2

14. Back Propagation
Source: Python Machine Learning Book by Sebastian Raschka

19. How to
select
the
model?

20. 1. Hands-on Session:
Iris Dataset

21. 2. Hands-on Session:
Mnist dataset

22. Feed Forward Neural Network
Inputs and outputs are independent

23. Recurrent Neural
Networks (RNN)

24. Time Series/ Sequence Prediction
Stock Market Prediction
Text Sequence Prediction The colour of the sky _____
Speech Translation

25. Recurrent Neural Network
Output is dependent on Previous output

26. Problems in RNN
The colour of the sky _____
RNN can predict: BLUE
I grew up in India… I speak fluent ____
RNN CANNOT predict HINDI

27. RNN - Memory
Source: Colah’s blog

28. RNN - Missing Long Term Memory
RNN has Vanishing Gradient Problem

29. Long Short Term Memory
(LSTM) Networks

30. Information Flow in RNN
Source: Deep learning with Keras - Chollet

31. LSTM - Carry Bits

32. LSTM

33. 3. Hands-on Session:
CnC Detection

34. Command and Control Detection
C&C domain examples:
• DGA based: gvludcvhcrjwmgq.in, uqvwxfrhhwreddf.yt
• non DGA based: thisisyourchangeqq.com, homejobsinstitute.biz
Ransomware
Malware
Enterprise Network
Main DB
Webserver
C&C server
Data
Command
Attacker

35. Update Models
DNS
data
Ranking of
Malicious
C&C
domains
C&C
domains
Classify benign vs
C&C domains using
LSTM
C&C Detection: Pipeline

36. Convolutional
Neural Networks
(CNN)

37. CNN: Data Flow

38. CNN

39. Convolution

40. Convolution

41. Understanding Image Features

42. Understanding Image Features

43. Dropout

44. Softmax

45. Back Propagation

46. Hands-on Session:
Cat vs. Dogs

47. Best Practices for Building AI Systems

48. 1. Start Small and Simple -
First cut of AI system need not
be the best

49. 2. Capturing the domain
knowledge, data pre-
processing is important.

50. 3. Need to follow Software
Engg practices and tools -
logging, data flow diagrams,
abstraction, design patterns

51. 4. Identify and define
performance metrics

52. 5. Integration with UI and Data Infra layers needs
to be ironed out, use data interfaces, AI layer
should be isolated

53. @satnam74s
Where to go from here:
• “Awesome Deep Learning - Github”
https://github.com/ChristosChristofidis/awesome-
deep-learning
https://www.linkedin.com/in/satnamsinghdatascientist/