Demystifying Containerization Principles for Data Scientists - An introductory tutorial on how Dockers can be used as a development environment for data science projects

1. Demystifying Containerization
Principles for Data Scientists
A quick preview of Dockers
Dr Ganesh Neelakanta Iyer
Amrita Vishwa Vidyapeetham, Coimbatore
Associate Professor, Dept of Computer Science and Engg

2. About Me • Associate Professor, Amrita Vishwa Vidyapeetham
• Masters & PhD from National University of Singapore (NUS)
• Several years in Industry/Academia
• Sasken Communications, NXP Semiconductors, Progress
Software, IIIT-HYD, NUS (Singapore)
• Architect, Manager, Technology Evangelist, Visiting Faculty
• Talks/workshops in USA, Europe, Australia, Asia
• Cloud/Edge Computing, IoT, Game Theory, Software QA
• Kathakali Artist, Composer, Speaker, Traveler, Photographer
GANESHNIYER http://ganeshniyer.com

3. Prelude

4. Data scientist – Basic software needs

5. Tough part is
• Setting them up and running
• Python latest version is 3.7 (as of 24th Oct 2018)
– TensorFlow support only up to Python 3.6
• TensorFlow 1.10 is incompatible with numPy > 1.14.5
• Primary problem is that when we do “pip instal yyy” it installs
latest version always – We may not know all these
dependencies and end up redoing all over again and again
Dr Ganesh Neelakanta Iyer 5

6. Flashback
Lets go back to pre-1960’s

8. Multiplicityof
Goods
Multiplicityof
methodsfor
transporting/storing
DoIworryabout
howgoodsinteract
(e.g.coffeebeans
nexttospices)
CanItransport
quicklyandsmoothly
(e.g.fromboatto
traintotruck)
Cargo Transport Pre-1960

9. ? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
Also an M x N Matrix

10. Multiplicityof
Goods
Multiplicityof
methodsfor
transporting/storing
DoIworryabout
howgoodsinteract
(e.g.coffeebeans
nexttospices)
CanItransport
quicklyand
smoothly
(e.g.fromboatto
traintotruck)
Solution: Intermodal Shipping Container
…in between, can be loaded and
unloaded, stacked, transported
efficiently over long distances,
and transferred from one mode
of transport to another
A standard container that is
loaded with virtually any
goods, and stays sealed until
it reaches final delivery.

11. This eliminated the M x N problem…

12. and spawned an Intermodal Shipping Container Ecosystem
• 90% of all cargo now shipped in a standard container
• Order of magnitude reduction in cost and time to load and unload ships
• Massive reduction in losses due to theft or damage
• Huge reduction in freight cost as percent of final goods (from >25% to <3%) massive globalizations
• 5000 ships deliver 200M containers per year

13. Static website
Web frontend
User DB
Queue Analytics DB
Background workers
API endpoint
nginx 1.5 + modsecurity + openssl + bootstrap 2
postgresql + pgv8 + v8
hadoop + hive + thrift + OpenJDK
Ruby + Rails + sass + Unicorn
Redis + redis-sentinel
Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs +
phantomjs
Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client
Development VM
QA server
Public Cloud
Disaster recovery
Contributor’s laptop
Production Servers
The Challenge
Multiplicityof
Stacks
Multiplicityof
hardware
environments
Production Cluster
Customer Data Center
Doservicesand
appsinteract
appropriately?
CanImigrate
smoothlyand
quickly?

14. Results in M x N compatibility nightmare
Static website
Web frontend
Background workers
User DB
Analytics DB
Queue
Development
VM
QA Server
Single Prod
Server
Onsite
Cluster
Public Cloud
Contributor’s
laptop
Customer
Servers
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?
? ? ? ? ? ? ?

15. Static website Web frontendUser DB Queue Analytics DB
Development
VM
QA server Public Cloud Contributor’s
laptop
Docker is a shipping container system for
code
Multiplicityof
Stacks
Multiplicityof
hardware
environments
Production
Cluster
Customer Data
Center
Doservicesand
appsinteract
appropriately?
CanImigrate
smoothlyand
quickly
…that can be manipulated using
standard operations and run
consistently on virtually any
hardware platform
An engine that enables any
payload to be encapsulated
as a lightweight, portable,
self-sufficient container…

16. Static website Web frontendUser DB Queue Analytics DB
Development
VM
QA server Public Cloud Contributor’s
laptop
Or…put more simply
Multiplicityof
Stacks
Multiplicityof
hardware
environments
Production
Cluster
Customer Data
Center
Doservicesand
appsinteract
appropriately?
CanImigrate
smoothlyand
quickly
Operator: Configure Once, Run
Anything
Developer: Build Once, Run
Anywhere (Finally)

17. Static website
Web frontend
Background workers
User DB
Analytics DB
Queue
Development
VM
QA Server
Single Prod
Server
Onsite
Cluster
Public Cloud
Contributor’s
laptop
Customer
Servers
Docker solves the M x N problem

18. Docker containers
• Wrap up a piece of software in a
complete file system that contains
everything it needs to run:
– Code, runtime, system tools, system
libraries
– Anything you can install on a server
• This guarantees that it will always
run the same, regardless of the
environment it is running in

19. Why containers matter
Physical Containers Docker
Content Agnostic The same container can hold almost
any type of cargo
Can encapsulate any payload and its
dependencies
Hardware Agnostic Standard shape and interface allow
same container to move from ship to
train to semi-truck to warehouse to
crane without being modified or
opened
Using operating system primitives (e.g.
LXC) can run consistently on virtually
any hardware—VMs, bare metal,
openstack, public IAAS, etc.—without
modification
Content Isolation and
Interaction
No worry about anvils crushing
bananas. Containers can be stacked
and shipped together
Resource, network, and content
isolation. Avoids dependency hell
Automation Standard interfaces make it easy to
automate loading, unloading, moving,
etc.
Standard operations to run, start, stop,
commit, search, etc. Perfect for devops:
CI, CD, autoscaling, hybrid clouds
Highly efficient No opening or modification, quick to
move between waypoints
Lightweight, virtually no perf or start-up
penalty, quick to move and manipulate
Separation of duties Shipper worries about inside of box,
carrier worries about outside of box
Developer worries about code. Ops
worries about infrastructure.

20. Docker containers
Lightweight
• Containers running on
one machine all share
the same OS kernel
• They start instantly
and make more
efficient use of RAM
• Images are
constructed from
layered file systems
• They can share
common files, making
disk usage and image
downloads much
more efficient
Open
• Based on open
standards
• Allowing containers to
run on all major Linux
distributions and
Microsoft OS with
support for every
infrastructure
Secure
• Containers isolate
applications from
each other and the
underlying
infrastructure while
providing an added
layer of protection for
the application

21. Docker / Containers vs. Virtual Machine
https://www.docker.com/whatisdocker/
Containers have similar resource
isolation and allocation benefits as
VMs but a different architectural
approach allows them to be much
more portable and efficient

22. Virtual Machines
Virtual machines run guest operating systems—note the OS
layer in each box. This is resource intensive, and the
resulting disk image and application state is an entanglement
of OS settings, system-installed dependencies, OS security
patches, and other easy-to-lose, hard-to-replicate ephemera
Containers vs Virtual Machines
Containers
Containers can share a single kernel, and the only
information that needs to be in a container image is the
executable and its package dependencies, which never need
to be installed on the host system. These processes run like
native processes, and you can manage them individually

23. Why are Docker containers lightweight?
Bins/
Libs
App
A
Original App
(No OS to take
up space, resources,
or require restart)
AppΔ
Bins
/
App
A
Bins/
Libs
App
A’
Gues
t
OS
Bins/
Libs
Modified App
Union file system allows
us to only save the diffs
Between container A
and container
A’
VMs
Every app, every copy of an
app, and every slight modification
of the app requires a new virtual server
App
A
Guest
OS
Bins/
Libs
Copy of
App
No OS. Can
Share bins/libs
App
A
Guest
OS
Guest
OS
VMs Containers

24. What are the basics of the Docker system?
Source
Code
Repository
Dockerfile
For
A
Docker Engine
Docker
Container
Image
Registry
Build
Docker Engine
Host 2 OS 2 (Windows / Linux)
Container
A
Container
B
Container
C
ContainerA
Push
Search
Pull
Run
Host 1 OS (Linux)

25. Changes and Updates
Docker Engine
Docker
Container
Image
Registry
Docker Engine
Push
Update
Bins/
Libs
App
A
AppΔ
Bins
/
Base
Container
Image
Host is now running A’’
Container
Mod A’’
AppΔ
Bins
/
Bins/
Libs
App
A
Bins
/
Bins/
Libs
App
A’’
Host running A wants to upgrade to A’’.
Requests update. Gets only diffs
Container
Mod A’

26. Easily Share and Collaborate on Applications
• Distribute and share content
– Store, distribute and manage your Docker images in your Docker
Hub with your team
– Image updates, changes and history are automatically shared
across your organization.
• Simply share your application with others
– Ship your containers to others without worrying about different
environment dependencies creating issues with your application.
– Other teams can easily link to or test against your app without
having to learn or worry about how it works.
Docker creates a common framework for developers and sysadmins to work together on distributed
applications

27. Get Started with Docker
• Install Docker
• Run a software image in a container
• Browse for an image on Docker Hub
• Create your own image and run it in a
container
• Create a Docker Hub account and an
image repository
• Create an image of your own
• Push your image to Docker Hub for
others to use
https://www.docker.com/products/docker
https://www.docker.com/products/docker-toolbox

28. Docker Container as a Service (CaaS)
Deliver an IT secured and managed application environment for developers to build and deploy
applications in a self service manner

29. Typical Use cases

30. App Modernization

31. Continuous Integration and Deployment (CI / CD)
• The modern development pipeline is fast, continuous and automated
with the goal of more reliable software
• CI/CD allows teams to integrate new code as often as every time
code is checked in by developers and passes testing
• A cornerstone of devops methodology, CI/CD creates a real time
feedback loop with a constant stream of small iterative changes that
accelerates change and improves quality
• CI environments are often fully automated to trigger a test at git push
and to automatically build a new image if the test is successful and
push to a Docker Registry
• Further automation and scripting can deploy a container from the
new image to staging for further testing.

32. Microservices
• App architecture is changing from monolithic code bases with waterfall development
methodologies to loosely coupled services that are developed and deployed
independently
• Tens to thousands of these services can be connected to form an app
• Docker allows developers are able to choose the best tool or stack for each service
and isolates them to eliminate any potential conflicts and avoids the “matrix from
hell.”
• These containers can be easily shared, deployed, updated and scaled instantly and
independently of the other services that make up the app
• Docker’s end to end security features allow teams to build and operate a least
privilege microservices model where services only get access to the resources
(other apps, secrets, compute) they need to run at just the right time to create.

33. IT Infrastructure optimization
• Docker and containers help optimize the utilization and cost
of your IT infrastructure
• Optimization not just cost reduction, it is ensuring the right
amount of resources are available at the right time and
used efficiently
• Because containers are lightweight ways of packaging and
isolating app workloads, Docker allows multiple workloads
to run on the same physical or virtual server without conflict
• Businesses can consolidate datacenters, integrate IT from
mergers and acquisitions and enable portability to cloud
while reducing the footprint of operating systems and
servers to maintain

34. Hybrid Cloud
• Docker guarantees apps are cloud enabled - ready
to move across private and public clouds with a
higher level of control and guarantee apps will
operate as designed
• The Docker platform is infrastructure independent
and ensures everything the app needs to run is
packaged and transported together from one site to
another
• Docker uniquely provides flexibility and choice for
businesses to adopt a single, multi or hybrid cloud
environment without conflict

35. How does this help you build better software?
• Stop wasting hours trying to setup developer environments
• Spin up new instances and make copies of production code to run locally
• With Docker, you can easily take copies of your live environment and run on any new
endpoint running Docker.
Accelerate Developer Onboarding
• The isolation capabilities of Docker containers free developers from the worries of using
“approved” language stacks and tooling
• Developers can use the best language and tools for their application service without
worrying about causing conflict issues
Empower Developer Creativity
• By packaging up the application with its configs and dependencies together and shipping
as a container, the application will always work as designed locally, on another machine,
in test or production
• No more worries about having to install the same configs into a different environment
Eliminate Environment Inconsistencies

37. First Hand Experience

38. Setting up
• Before we get started, make sure your system has the latest version of
Docker installed.
• Docker is available in two editions: Community Edition
(CE) and Enterprise Edition (EE).
• Docker Community Edition (CE) is ideal for developers and small teams
looking to get started with Docker and experimenting with container-based
apps. Docker CE has two update channels, stable and edge:
– Stable gives you reliable updates every quarter
– Edge gives you new features every month
• Docker Enterprise Edition (EE) is designed for enterprise development
and IT teams who build, ship, and run business critical applications in
production at scale.

39. Supported Platforms
https://docs.docker.com/install/

40. © 2017 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.© 2017 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.
In this session, I use Docker for Windows Desktop

41. Docker for
Windows

42. If your windows is not in latest version…
https://docs.docker.com/docker-for-windows/release-notes/#docker-community-edition-17062-ce-win27-2017-09-06-stable

43. Docker for Windows
When the whale in the status
bar stays steady, Docker is up-
and-running, and accessible
from any terminal window.

44. Hello-world
• Open command prompt / windows power shell and run
docker run hello-world
 Now would also be a good time to make sure you are using
version 1.13 or higher. Run docker --version to check it out.

45. Building an app the Docker way
• In the past, if you were to start writing a Python app, your first order
of business was to install a Python runtime onto your machine
• But, that creates a situation where the environment on your machine
has to be just so in order for your app to run as expected; ditto for
the server that runs your app
• With Docker, you can just grab a portable Python runtime as an
image, no installation necessary
• Then, your build can include the base Python image right alongside
your app code, ensuring that your app, its dependencies, and the
runtime, all travel together
• These portable images are defined by something called a Dockerfile

46. Define a container with a Dockerfile
• Dockerfile will define what goes on in the environment
inside your container
• Access to resources like networking interfaces and disk
drives is virtualized inside this environment, which is
isolated from the rest of your system, so you have to map
ports to the outside world, and be specific about what files
you want to “copy in” to that environment
• However, after doing that, you can expect that the build of
your app defined in this Dockerfile will behave exactly
the same wherever it runs

47. Dockerfile
• Create an empty directory
• Change directories (cd) into the new directory, create a
file called Dockerfile

48. Dockerfile
• In windows, open notepad, copy the content below, click on Save as, type “Dockerfile”
This Dockerfile refers to a couple of files we
haven’t created yet, namely app.py and
requirements.txt. Let’s create those next.

49. The app itself
• Create two more files,
requirements.txt and app.py, and
put them in the same folder with the
Dockerfile
• This completes our app, which as you
can see is quite simple
• When the above Dockerfile is built
into an image, app.py and
requirements.txt will be present
because of that Dockerfile’s ADD
command, and the output from app.py
will be accessible over HTTP thanks to
the EXPOSE command.

50. The App itself
Requirements.txt
app.py
That’s it! You don’t need Python
or anything in
requirements.txt on your
system, nor will building or
running this image install them
on your system. It doesn’t seem
like you’ve really set up an
environment with Python and
Flask, but you have.

51. Building the app
• We are ready to build the app. Make sure you are still at the
top level of your new directory. Here’s what ls should show
• Now run the build command. This creates a Docker image,
which we’re going to tag using -t so it has a friendly name.

52. Building the app
• docker build -t friendlyhello .

53. Where is your built images?
• docker images

54. Run the app
• Run the app, mapping your machine’s port 4000 to the container’s
published port 80 using –p
• docker run -p 4000:80 friendlyhello
• You should see a notice that Python is serving your app at
http://0.0.0.0:80. But that message is coming from inside the
container, which doesn’t know you mapped port 80 of that container to
4000, making the correct URL http://localhost:4000
• Go to that URL in a web browser to see the display content served up on
a web page, including “Hello World” text, the container ID, and the Redis
error message

56. End the process
• Hit CTRL+C in your terminal to quit
• Now use docker stop to end the process, using the
CONTAINER ID, like so

57. • Now let’s run the app in the background, in detached mode:
• docker run -d -p 4000:80 friendlyhello
• You get the long container ID for your app and then are kicked back
to your terminal. Your container is running in the background. You
can also see the abbreviated container ID with docker container ls
(and both work interchangeably when running commands):
• docker container ls

58. Share image
• To demonstrate the portability of what we just created, let’s
upload our built image and run it somewhere else
• After all, you’ll need to learn how to push to registries when you
want to deploy containers to production
• A registry is a collection of repositories, and a repository is a
collection of images—sort of like a GitHub repository, except the
code is already built. An account on a registry can create many
repositories. The docker CLI uses Docker’s public registry by
default
• If you don’t have a Docker account, sign up for one at
cloud.docker.com. Make note of your username.

61. Login with your docker id
• Log in to the Docker public registry on your local machine.
• docker login

62. Tag the image
• The notation for associating a local image with a repository on a
registry is username/repository:tag. The tag is optional, but
recommended, since it is the mechanism that registries use to give
Docker images a version. Give the repository and tag meaningful
names for the context, such as get-started:part1. This will put
the image in the get-started repository and tag it as part1.
• Now, put it all together to tag the image. Run docker tag image
with your username, repository, and tag names so that the image will
upload to your desired destination. The syntax of the command is:

63. Tag the image

64. Publish the image
• Upload your tagged image to the repository
• docker push username/repository:tag
• Once complete, the results of this upload are publicly available. If
you log in to Docker Hub, you will see the new image there, with its
pull command

65. Publish the image
• Upload your tagged image to the repository
• docker push username/repository:tag
• Once complete, the results of this upload are publicly available. If
you log in to Docker Hub, you will see the new image there, with its
pull command

67. Pull and run the image from the remote
repository
• From now on, you can use docker run and run your app on any
machine with this command:
• docker run -p 4000:80 username/repository:tag
• If the image isn’t available locally on the machine, Docker will pull it
from the repository.
• If you don’t specify the :tag portion of these commands, the tag of
:latest will be assumed, both when you build and when you run
images. Docker will use the last version of the image that ran without
a tag specified (not necessarily the most recent image).
No matter where executes, it pulls your image, along with Python and all the dependencies
from , and runs your code. It all travels together in a neat little package, and the host machine
doesn’t have to install anything but Docker to run it.

68. What have you seen so far?
• Basics of Docker
• How to create your first app in the Docker way
• Building the app
• Run the app
• Sharing and Publishing images
• Pull and run images

69. © 2017 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.© 2017 Progress Software Corporation and/or its subsidiaries or affiliates. All rights reserved.
Docker for data scientists
Reference: Blog post is authored by Shaheen Gauher,
Data Scientist at Microsoft
https://blogs.technet.microsoft.com/machinelearning/2018/03/15/demystifying
-docker-for-data-scientists-a-docker-tutorial-for-your-deep-learning-projects/

70. Docker for Data scientists
If you have tried to install and set up a deep learning framework (e.g. CNTK,
Tensorflow etc.) on your machine you will agree that it is challenging
The proverbial stars need to align to make sure the dependencies and requirements
are satisfied for all the different frameworks that you want to explore and experiment
with
Getting the right anaconda distribution, the correct version of Python, setting up the
paths, the correct versions of different packages, ensuring the installation does not
interfere with other Python-based installations on your system is not a trivial exercise

71. Docker for Data scientists
Using a Docker image saves us this trouble as it provides a pre-
configured environment ready to start work in
Even if you manage to get the framework installed and running in
your machine, every time there’s a new release, something could
inadvertently break
Making Docker your development environment shields your project
from these version changes until you are ready to upgrade your code
to make it compatible with the newer version
Dr Ganesh Neelakanta Iyer 71

72. Docker for Data scientists
Using Docker also makes sharing projects with others a painless process
You don’t have to worry about environments not being compatible, missing dependencies or
even platform conflicts
When sharing a project via a container you are not only sharing your code but your
development environment as well ensuring that your script can be reliably executed, and your
work faithfully reproduced
Furthermore, since you work is already containerized, you can easily deploy it using services
such as Kubernetes, Swarm etc.
Dr Ganesh Neelakanta Iyer 72

73. The right image
• Go to Docker hub
https://hub.docker.com/r/
microsoft/cntk/
• Download your preferred
version of the CNTK
image
Dr Ganesh Neelakanta Iyer 73

74. The right image
• The command will download the CNTK 2.2 CPU runtime
configuration set up for Python 3.5
• After pulling the image if we execute the docker images command,
the image that was just pulled should be listed in the output
Dr Ganesh Neelakanta Iyer 74

75. Image to containers
• Use this image to start a container
• To create a new container, we must specify an image
name from which to derive the container from and an
optional command to run (/bin/bash here to access the
bash shell).
docker run [OPTIONS] microsoft/cntk:2.2-cpu-
python3.5 /bin/bash
Dr Ganesh Neelakanta Iyer 75

76. Inside the container
Transfer these to our working directory in the container
The training and test data along with our script are on the local machine
pip install lightgbm inside the container as the CNTK image does not come with lightgbm
Create a working directory called mylightgbmex as we want to train a lightgbm model
To start the deep learning project, lets jump inside the container in a bash shell and use it as our development environment
Dr Ganesh Neelakanta Iyer 76

77. Inside the container
• To run an interactive shell in the image, the docker run command can be executed
using the following options
• docker run -i -t –name mycntkdemo microsoft/cntk:2.2-cpu-
Python3.5 /bin/bash
• -t, –tty=false Allocate a pseudo-TTY
• -i, –interactive Keep STDIN open even if not attached
• The above command starts the container in an interactive mode and puts us in
a bash shell as though we were working directly in our terminal. Once inside the
shell, we can use any editor (the CNTK image comes with vi editor) to write our
code. We can start the Python interpreter by typing Python on the command line.
Dr Ganesh Neelakanta Iyer 77

78. Dr Ganesh Neelakanta Iyer 78

79. Inside the container
• Next, copy the training and
test data along with Python
script from local machine to
the working folder in the
container mycntkdemo using
the docker cp command
• With the files available
inside the container I will
jump back inside and
execute my script
• Once we have the output
from running our script, we
could transfer it back to our
local machine using the
docker cp command again
Dr Ganesh Neelakanta Iyer 79

80. • Alternatively we could map the
folder C:dockertut on the host
machine to the directory
mylightgbmex in the Docker
container when starting the
container by using the -v flag with
docker run command
• docker run -it –name
mycntkdemo -v
C:dockertut:/root/myligh
tgbmex
microsoft/cntk:2.2-cpu-
python3.5 /bin/bash
• When inside the container, we will
see a directory mylightgbmex
with the contents of the folder
C:dockertut in it.
Dr Ganesh Neelakanta Iyer 80

81. Custom Image
• In the exercise above we installed lightgbm in our container and by doing
so we added another layer to the image we started with
• If we want to save these changes, we need to commit the container’s file
changes or settings into a new image
• docker commit mycntkdemo mycntkwlgbm:version1
• The above command will create a new image called mycntkwlgbm and
should be listed in the output of Docker images command
• This new image will contain everything that the CNTK image came with
plus lightgbm, all the files we transferred from our machine and the output
from executing our script
• We can continue using this new image by starting a container with it.
Dr Ganesh Neelakanta Iyer 81

82. Dockerfile
Dr Ganesh Neelakanta Iyer 82

83. Jupyter Notebook
• Jupyter notebook is a favorite tool of data scientists
• Both CNTK and Tensorflow images come with Jupyter
installed
• In Docker, the containers themselves can have
applications running on ports
• To access these applications, we need to expose the
containers internal port and bind the exposed port to a
specified port on the host
Dr Ganesh Neelakanta Iyer 83

84. Jupyter Notebook
• In the example below, we will access the Jupyter notebook
application running inside my container
• Starting a container with -p flag will explicitly map the port of
the Docker host to the port number on our localhost to access
the application running on that port in the container (port 8888
is default for Jupyter notebook application)
docker run -it -p 8888:8888 –name mycntkdemo2
microsoft/cntk:2.2-cpu-python3.5 /bin/bash
Dr Ganesh Neelakanta Iyer 84

85. Jupyter Notebook
• Once in the container shell, the Jupyter notebook application can be
started using the command
jupyter-notebook –no-browser –ip=0.0.0.0 –notebook-
dir=/cntk/Tutorials –allow-root
Dr Ganesh Neelakanta Iyer 85

86. Type the url with the
token above
http://localhost:8888/?tok
en=************* in your
favorite browser to see
the notebook dashboard
Dr Ganesh Neelakanta Iyer 86

87. Repeat
• In the examples above we used the CNTK framework
• To work with other frameworks, we can simply repeat the
above exercises with the appropriate image
• For example, to work on a Tensorflow project, we can access
the Jupyter notebook application running in the container as:
docker run -it -p 8888:8888 tensorflow/tensorflow
• The command above will get the latest image for CPU only
container and start the Jupyter notebook application
Dr Ganesh Neelakanta Iyer 87

88. Repeat
Dr Ganesh Neelakanta Iyer 88

89. Dr Ganesh Neelakanta Iyer 89
Type the url with the token above http://localhost:8888/?token=*************
in your favorite browser to see the notebook dashboard

90. Takeaways
With Docker containers as the development environment for your
deep learning projects, you can hit the ground running
You are spared the overhead of installing and setting up the
environment for the various frameworks and can start working on your
deep learning projects right away
Scripts are guaranteed to run everywhere and will run the same every
time
Dr Ganesh Neelakanta Iyer 90

91. Additional References
• CNTK
– https://hub.docker.com/r/microsoft/cntk/
– https://www.born2data.com/2017/deeplearning_install-part4.html
• TensorFlow
– https://www.tensorflow.org/install/docker
– https://www.tensorflow.org/serving/docker
• Jupyter
– https://blog.sicara.com/tensorflow-gpu-opencv-jupyter-docker-10705b6cd1d
– https://www.dataquest.io/blog/docker-data-science/
– https://jupyter-docker-stacks.readthedocs.io/en/latest/using/selecting.html
– https://towardsdatascience.com/jupyter-data-science-stack-docker-in-under-15-
minutes-19d8f822bd45
Dr Ganesh Neelakanta Iyer 91

92. Dr Ganesh Neelakanta Iyer
ni_amrita@cb.amrita.edu
ganesh.vigneswara@gmail.com
GANESHNIYER