Cypher 2017 - Lalit Sachan - Workshop

Copyright 2016 RESTRICTED CIRCULATION
Making Data Science Great Again – Applications of
Deep learning Using Tensorflow

Copyright 2016 RESTRICTED CIRCULATION ‹#›
➢ In last 4+ years , we have helped thousands of independent
individuals and 100+ corporate clients towards building their
capabilities in data science
➢Here are few of our clients :
About Edvancer

About Me
Lalit Sachan
eMail : lalit.sachan@edvancer.in
Linkedin : https://in.linkedin.com/in/lalitsachan

Preview

➢one line summary for the talk : What can we do
with deep learning and how to get started .
➢Flow :
➢Part 1 : From simple neural networks to deep
neural nets
➢Part 2 : Introduction to tensor flow
➢Part 3 : Tensorflow and Tensorboard in Action
➢Part 4 : Popular Models
➢Part 5 : Summary and Few other things
Contd..

Part 1

➢Cost Functions and Gradient Descent
➢Back-propagation
➢Deep Neural Networks
From simple neural networks to deep neural networks

Cost Functions and Gradient Descent
• GradientDescent
• Adam
• Adadelta
• Adagrad
• RMSprop

Back-Propogation
Reference : http://colah.github.io/posts/2015-08-Backprop/

Deep Neural Networks

Part 2

➢What is Tensorflow?
➢Why Tensorflow?
➢Compuation graph for Linear Regression
➢Computation graph for Neural Network with no
hidden layer
➢Computation graph for neural Network with
hidden layers
Tensorflow

➢Developed by Google Brain Team
➢Open sourced November, 2015
➢Numerical computation using data flow graphs
➢Allows various models to be built from scratch
What is Tensorflow?

➢Python API
➢Tensorboard
➢Auto Differentiation
➢Large and Active community
Why Tensorflow

➢Y_predicted = X*w + b
➢loss = (Y-Y_predicted)*(Y-Y_predicted)
Linear Regression
Reference : http://web.stanford.edu/class/cs20si/lectures/notes_03.pdf

Computation Graph : Linear Regression
Reference : http://web.stanford.edu/class/cs20si/lectures/notes_03.pdf

Neural Network with no hidden layers ( Classification)

Contd..

Computation Graph

Neural Network with hidden layers ( Classification)

Computation Graph expand

Part 3

➢Create an account on google cloud platform ($300 credit which can
easily last couple of months for your learning purposes)
➢Create a project , go to compute machine , create a VM instance with
following properties (what i use for the demo)
➢Customise : 4 CPUs , 1 GPU (K Tesla)( for the first time you’ll get an
error saying exceeding quota , with an option for request increase
[becasue of GPU])
➢dont forget to select zone which has GPUs ( different zones will
differ in pricing and latency depending on where you are, chose
accordingly )
➢OS : ubuntu 16.04
➢Increase size of disk to say 50GBs
➢Allow http and https connection
➢In networking , add a firewall rule allowing connections from all
ips(0.0.0.0/0), add ports tcp and udp 8888,6006 ( for accessing
jupyter and tensor board remotely)
Setting up tensor flow: Creating VM Instance

Preparing VM instance for tensor flow
• Install cuda ( Lets you use GPUS)
• go to VM and launch using SSH on browser
sudo apt-get install python3-pip
sudo pip3 install jupyter
sudo jupyter notebook --ip 0.0.0.0 --port 8888 --allow-root
• Go to your_external_ip_for_vm:8888 (on chrome
preferably )
• create an Nvidia Developer account , download cudnn
version 6 for cuda 8 for linux
• upload the file to your VM using upload feature of jupyter

Contd..
tar -xzvf cudnn-8.0-linux-x64-v6.0.tgz
sudo cp cuda/include/cudnn.h /usr/local/cuda/include
sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/
libcudnn*
nano ~/.bashrc
export LD_LIBRARY_PATH=/usr/local/cuda/lib64
export CUDA_HOME=/usr/local/cuda
source ~/.bashrc
sudo pip3 install tensorflow-gpu

Last thing for the unfortunates ( Me)
jupyter notebook --generate-config
jupyter --config-dir
• Go to the file with nano file_with_path
• Add things mentioned below
• Reboot your VM
import os
c = get_config()
os.environ['LD_LIBRARY_PATH'] = '/usr/local/cuda/lib64:usr/local/cuda/
lib64/libcudnn.so.6'
c.Spawner.env.update('LD_LIBRARY_PATH')
c.Spawner.env_keep.append('LD_LIBRARY_PATH')

Problem : Identifying handwritten digits

Demo 1: Neural Networks with no hidden layer

Demo 2 : Neural Network with a single hidden layer

➢A convolution network
➢Regularization : dropout
➢Batch Normalisation and learning rate decay
➢A fairly quick implementation on your own gives
accuracy of 99.3% , state of the art is 99.7%
What more we could have done

Part4

➢Convolutional Networks
➢RNN,LSTM,GRU
➢Hopfield Networks
➢Autoencoders
➢Generative Adversarial Networks
Various Ideas and Architectures

Convolution Neural Networks
• Make use of information in pixel positions
• Convolution and pooling pick different features of shapes
present in the image

LSTM
• Gates in each unit change/preserve states(memory) from
previous occurrences in the sequence
• Used in sequence modelling : next word , market forecast
• Can also be used with other kind of sequential data such as
music, speech etc

Hopfield Networks
• Learns how a data distribution or input looks like
• Can be used in regenerating data from partial inputs or de-
noising images

Autoencoders
• Input and output are same
• Used in extracting lower dimension features . An important
unsupervised algorithms

Generatve Adversarial Networks
• Made up of a generator and discriminator
• Among many interesting use cases , text to image
generation

Summary and few other things

➢Deeplearning outperforms other algorithms and has
many variations to connect it with ai tasks
➢With tensorflow and easy access to otherwise
difficult/expensive hardware its approachable
➢We can move beyond simple predictions to more
general problems
Key Takeaways

➢What do we compete against? Whats the limit to
how well our model can do?
➢What do we do when we surpass that?
➢How do i decide between having more data, making
bigger models Vs Regularisation?
➢What if we don’t have data?
➢What if we have data but not exactly the same kind i
need for my application?
➢How do i pick a certain architecture for my project to
start with?
And other things…

Thank you!
Questions?

Cypher 2017 - Lalit Sachan - Workshop

Recommended

Recommended

More Related Content

What's hot

What's hot (9)

Similar to Cypher 2017 - Lalit Sachan - Workshop

Similar to Cypher 2017 - Lalit Sachan - Workshop (20)

More from Analytics India Magazine

More from Analytics India Magazine (20)

Recently uploaded

Recently uploaded (20)

Cypher 2017 - Lalit Sachan - Workshop