This document provides an overview of deep learning including: - A definition of deep learning as a powerful class of machine learning using multi-layered neural networks. - Examples of where deep learning is used including speech recognition, image recognition, and natural language processing. - Descriptions of common deep learning architectures like convolutional neural networks and recurrent neural networks. - An overview of popular deep learning libraries including Caffe, Torch, Theano, and TensorFlow. - A demonstration of building and training a simple deep learning model in Keras.

1. 21.09.2017
Burcu Kaptıkaçtı

2. Index
 What is DeepLearning?
 History
 What changed?
 Where we use?
 DeepLeaarning Architectures
 DeepLearning Libraries
 Demo

3. What is DeepLearning?
 A powerful class of machine learning model
 Multi-layered Neural Network

4. What is DeepLearning?

5. History

6. <<Hype Cycle>>

7. What Changed?

8. What Changed?
 Amount of data

9. What Changed?
 Amount of data
 GPU

10. What Changed?
 Amount of Data
 GPU
 Multi-layered Structure

11. Where we use?
 Speech Recognition
 LSTM
 Cortana,Xbox,Skype Translator, Google Now, Siri, Baidu
 Image Recognition
 CNN
 ImageNet,MNIST
 Natural Language Processing
 Word Embedding

12. Where we use?
 https://www.youtube.com/watch?v=MVBe6_o4cMI
 https://www.youtube.com/watch?v=Voz4dosVGSM
 https://www.youtube.com/watch?v=V1eYniJ0Rnk

13. Neural Networks

14. Neural Networks

15. Neural Networks

16. DeepLearning Architectures
 Convolutional Neural Networks (CNN)
 Recurrent Neural Networks (RNN)
 Long Short Term Memort (LSTM)

17. Convolutional Neural Networks (CNN)

18. Convolutional Neural Networks (CNN)

19. Convolutional Neural Networks (CNN)

20. Convolutional Neural Networks (CNN)

21. Convolutional Neural Networks (CNN)
 Kernel(K) or Filter

22. Convolutional Neural Networks (CNN)

23. Convolutional Neural Networks (CNN)

24. Convolutional Neural Networks (CNN)

25. Convolutional Neural Networks (CNN)

27. DeepLearning Libraries
 Caffe

28. DeepLearning Libraries
 Caffe
 Torch

29. DeepLearning Libraries
 Caffe
 Torch
 Theano

30. Deep Learning Libraries
 Caffe
 Torch
 Theano
 TensorFlow

31. DeepLearning Libraries
 Caffe
 Torch
 Theano
 TensorFlow
 NDIVA DIGIT

32. DeepLearning Libraries
 Caffe
 Torch
 Theano
 TensorFlow
 NDIVA DIGIT
 Keras
 http://deeplearning.net/software_links/

33. DeepLearning Libraries
 Caffe2 --Facebook
 Torch --Facebook
 TensorFlow --Google
 CNTK--Microsoft
 MxNet– Amazon/Apache

34. Deep Learning Libraries

35. Knet
 Koç University deep learning framework
 https://github.com/denizyuret/Knet.jl

36. Keras
 Python Deep Learning Library
 Documentation
 https://keras.io/

37. Keras
 Sequential Model
 Functional Model

38. Keras-Sequential Model

39. Keras-Sequential Model
 Input
 Output

40. Keras-Sequential Model
 Compile
model.compile(optimizer=... , loss=..., metrics=[...])

41. Keras- Optimization Functions
 'sgd': SGD
 'rmsprop': RMSprop
 'adagrad': Adagrad
 'adadelta': Adadelta
 'adam': Adam
 'adamax': Adamax
 'nadam': Nadam

42. Loss Functions
 MSE
 Mean Absolute
 hinge
 squared_hinge
….
 https://keras.io/losses/

43. Keras –Loss Functions
 mean_squared_error / mse
 mean_absolute_error / mae
 mean_absolute_percentage_error / mape
 mean_squared_logarithmic_error / msle
 squared_hinge
 Hinge
model.compile(loss='categorical_crossentropy',...)

44. Activation Functions
 Sigmoid
 Softmax
 Tanh
 Relu
 https://keras.io/activations/

45. Keras - Save
model.fit(giris,cikis,epochs=1,validation_split=0.1)
model.save("kayıt")
model= keras.models.load_model("kayıt")

46. Overfitting
 Dropout

47. Keras
 Python Deep Learning Library
 Documentation
 https://keras.io/
 https://repo.continuum.io/archive/index.html
 conda install -c conda-forge keras

48. Datasets
 https://archive.ics.uci.edu/ml/datasets/
 https://datamarket.com/data/
 https://www.cs.toronto.edu/~kriz/cifar.html

50. Demo -1
 Dataset
 https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Diagnostic)