Presentation for my session at IISc, Bangalore

1. Improving efficiency for Deep Networks
August 23, 2019
Indian Institute of Science
Akshay Bahadur

2. Acknowledgement

3. Akshay Bahadur
● SDE – Symantec
● Intel Software Innovator
● Google Developer Expert
(Machine Learning)

4. Agenda
● Normalization
● Optimizers
● Activation
● Advanced Activation
● Focal Loss instead categorical loss
● Learning Rate Decay with Adam
● Network Architecture optimization
● Autopilot research paper by NVIDIA
● Handling out of memory issue
● Scaling Images
● Adding filters and Splitting channels
● Fit Generator
● DeepSign

5. Tania’s Story

6. Normalization
Normalization is a technique often applied as part of data preparation for machine learning. The goal of normalization is to
change the values of numeric columns in the dataset to a common scale, without distorting differences in the ranges of
values.

14. Optimizers

15. ● Stochastic Gradient Descent
● RMSprop

16. ● ADAM
● AdaDelta

17. Activation

18. ● Sigmoid
● Rectified Linear Unit

19. Advanced Activation

20. ● Thresholded ReLU
● Leaky ReLU

21. Focal Loss instead of categorical loss

22. Focal Loss

23. Learning Rate Decay with Adam

25. EMNIST

26. DigiEncoder

27. Network Architecture Optimization
Cifar 10

28. FC Architecture

30. CNN Architecture

32. Autopilot research paper by NVIDIA

34. Total pixels per image = 66*200*3
= 39,600

36. Handling out of memory issue
● Scaling

37. Total pixels per image = 50*50*3
= 7500

39. ● Filtering

41. Total pixels per image = 50*50
= 2500

43. ● Using fit_generator

44. Total pixels per image = 200*200*3
= 120,000

46. Autopilot V2

47. Emojinator
● Image generator for Data Augmentation

51. Emojinator

52. Malaria Detection

53. ● Filtering

54. ● Filtering

55. DeepSign