•

0 likes•183 views

Report

Share

Deep learning is the subset of Machine learning. In this neural network is designed with more than one hidden layer between input and output layer.

- 1. DEEP LEARNING By A. Yamini Saraswathi Y18MTSP401 Under the guidance of Mr. T. Krishna chaithanya (M.Tech)
- 2. Contents • Artificial intelligence, machine learning • What is deep learning • computer vision features • Fundamentals of deep learning • Back propogation algorithm • Convolutional neural network • Recurrent neural network • Deep learning framework • Applications
- 3. Artificial intelligence Artficial intelligence is the simulation of human intelligence processes by machines especially computer systems. In this process include learning, reasoning to reach to an approximate or accurate output.
- 5. Continution.... Fig: Artificial neural network Fig: Biological neuron
- 6. Continution.... Applications of artficial intelligence • Autonomous vehicles(as drones and self driven cars) • Medical diagnosis • Playing games (such as chess) • Online assistance (such as siri) • Image recognition • Speech recognition • Natural language processing • Etc......
- 7. Continution... Learning of a neural netwwork can be of two types • Supervised learning:- It is used to perform classification when we want to map input data to output labels. In this data is labeled with classes. • Unsupervised learning:- This learning is used to perform clustering and in this data is not labeled. It is mostly used in exploratory analysis.
- 8. Machine learning • Which is a subclass of artificial intelligence that automatially learn and improve from the experience without being epxlicitly programmed. • Intelligent systems also built on machine learning algorithms which have the capability to learn from the past experience.
- 10. Limitations of machine learning • High dimensionality of the data • Unable to solve crucial AI problems • One of the biggest challenge is feature extraciton • Requires only significant amount of data
- 11. What is deep learning Which is a subclass of machine learning. In this neural network is designed with more than one hidden layer between input and output layer. It reduces the complexity of high dimensions of the data. Fig: Deep neural network
- 12. Computer vision features • Camera obscura • www.youtube.com/watch?v=10Hay06LJ4 • Black world • SIFT feature • Object recognition • Etc......
- 13. Fundamentals of deep learning Activation function:- These are really important for artificial neural network to learn and make sense of something really complicated and make non-linear complex functional mapping between inputs and response variable. It should be always non-linear and there are so many types Sigmoid Tanh Softmax Relu Leaky Relu
- 14. Continution.... Sigmoid It looks like an S- shaped curve. It's mathematical form is f(x) = 1 / 1 + exp(-x) and its range is from 0 to 1 easy to understand and apply but it has some major issues Vanishing gradient It's output is not zero centered Sigmoids saturate and kill gradients Sigmoids have slow convergence
- 15. Continution.... Tanh It also mostly looked like S-shaped curve and its range is from -1 to 1 and it's mathematical form f(x) = 1-exp(-2x) / 1 + exp(-2x) in this optimizations is easier. it's major issue is Vanishing gradient problem
- 16. Continution.... Relu It has become more popular in last couple of years. It was recently proved that it had 6 times improvement in convergence compared to Tanh function it also solves vanishing gradient problem there are two major issues • It can be used in hidden layers only • Some gradients may die
- 17. Continution.... Leaky Relu To fix the problem of dying neurons in the Relu we will use Leaky Relu which is the modification of Relu which introdues a small slope to keep the updates alive
- 18. Loss function • It is having an important part in ANN which is used to measure the inconsistency between predicted value and actual value. These are very helpful for training the neural network. Y=Predicted output-actual output • Many types of loss functions are present to deal with different types of tasks but most widely used loss function is Mean Square Error(MSE)
- 19. Optimization Weights and bias are modifiedusing a function calledoptimization function in such a way error is minimized. For updating weights we use an equation Optimazation algorithms are pf 2 types • First order optmization algorithm • Second order optimization algorithm
- 20. Optimzation algorithms Gradient descent is the most popularly used optmization algorithm. Which is of two types • Stochastich gradient descent θ=θ−η∇J(θ) where ∇J(θ) is the Gradient of Loss function J(θ) w.r.t parameters 'θ’ • Mini batch gradient descent
- 21. Weight initialization The method that we follow for weight initilalization is 'Xavier intialization'. This method tries to intialize the weights with a smarter value
- 22. Normalization It is most important part in ANN because because most of the data obtained from the real world are very distant each other. So to keep the data at a comparable range we need to normalize the input data of input layer. Common normalization approach includes. • Statistical normalization • Sigmoid normalization
- 23. Batch normalizaton • Training deep neural network with more number of layers is more challenging one to avoid this problem 'Batch normalization' is introduced. • It trains deep neural network that standardizes the inputs to a layernfor each mini-batch and Benifits of this batch normalization are • Network train • Makes weights easier to initialize • Simplifies the creation of deeper network • Makes more activation functions viable
- 24. Continution...Normalization Batch normalization Fig: Deep neural network
- 25. Regularization When we build a neural network it performs well for trained data but not on the test data. This is the sign of over fitting to reduce this problem there are two ways • Use more data • Regularization Getting more data is sometimes impossible and expoensive so for this purpose we use 'Regularization' So to solve this we should make our model generalize over the trained data which is done using various regularization techniques.
- 26. Continuation... Regularization techniques • L1 & L2 regularization Which updates general cost function by adding another term known as regularization term There is a small difference between L1 and L2 in the equation. Mostly L1 is preferred
- 27. Continuation.... • Dropout regularization This is mostly used one in the deep learning. At every iteration it randomly selects some nodes and removes remaining with all of their incoming signals and outgoing connections So at each iteration has a different set of nodes which result a different set of output At last we ensemble models which usually perform better than a normal neural network Fig: Drop out Neural network
- 28. Back propogation algorithm Neural network is trained by 'Back propogation algorithm' after forwarding inputs to neural network then inputs and weights are summed and multiplied then apply to activation function at last we predict the error so to reduce this error we need to update weights using gradient descent and bias which propogate backwwards to network
- 29. Convolutional neural network In todays, convnets are used in every where it has been very effective in such as image classification and image recognition it has been very successful in identifying faces, objects and traffic signs LeNet was one of the first convolutional neural network which helped the path to deep learning Convent is mainly build based on 3 layers • Convolution layer & Non linearity • Pooling layer • Fully connected layer
- 30. Continution... Fig: Convolutional neural network
- 31. Continution.... Convolution operation Fig:Input image Fig: Kernal filter Fig: Feature map K=1
- 32. Continution.... Padding & pooling operation Fig: Pooling operation Fig: Padding operation
- 34. Continuation... Fig: Practical example of Convnet
- 35. Recurrent Neural Network • RNN are also a feed forward neural network, however with ecurrent meomory loops which takes input from the previous and /or same layers of states • This gives them a unique capability to model along the time dimension and arbitary sequence of even inputs. • RNN's are used for sequenced data analysis such as time series, sentimental nalysis, NLP, language translation, speech recognition, image captioning, script recognition and other othings. • These are aslo calles networks with memory as the previous inputs or states may persist in the model to do a sequential analysis. These memories become input as well
- 36. Google Net • It is almost different compared to other architectures like Alex Net • It is the winner of Image Net 2014 and can train network faster which is developed by Google • In this global average pooling is used at the end of architecture it leads to reduce number of weights are reduced and accuracy increases to 0.6%
- 37. Continuation…
- 38. Deep Learning Framework • Deep learning frameworks were built in academia in first generation Caffe from Berkley university Torch from NYU and also collaboration with Facebook • In second generation all deep learning frameworks were developed from industry's Caffe2 from Facebook Pytarch from Facebook Tensor flow is from Facebook MXNet from Amazon CNTK form Microsoft
- 39. Importance of Deep learning frame work • Easily can build big computational graphs • Easily compute gradients in computational graphs • Run it all efficiently n GPU
- 40. Continuation... • This framework is an open source which aims to simplify the implementation of compels and large scale deep learning models using these amazing frameworks we can build complex models like CNN. Some types of frameworks listed below. Tensor flow Keras Pytorch Caffe Deeplearning4j • One of the most popular frame work is 'Tensorflow'
- 41. Applications • Automatic speech recognition • Image recognition • Visual art processing • Natural language processing • Drug discovery and toxicology • Bioinformatics • Medical Image Analysis • Mobile advertising • Image restoration • Financial fraud detection • Military
- 42. Google Arts & Culture app This app uses automatic image analysis to compare lot of pictures in the museums and in institutions throughout the world with your picture. For this process mainly deep learning is mainly involved and in the next slide I have taken the pictures practically.