Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Docker @ Data Science Meetup

Docker is a very useful tool in every data scientists toolbox. In this talk I present motivations to use Docker and made some live demos of typical tools used in data science, such as RStudio, Jupyter Notebook, or Elasticsearch.

  • Login to see the comments

Docker @ Data Science Meetup

  1. 1. for Data Science Daniel Nüst Institute for Geoinformatics University of Münster @nordholmen | http://nö Science-Meetup-Muenster
  2. 2.
  3. 3. Docker for Data Science nce-in-containers/ (Hackday container - nice!) part-2-dockerized-data-science/ -talk/ ence-and-Machine-Learning “Isolation! Portability! Repeatability!”
  4. 4. Agenda What is Docker? Why? What can it be used for? Live Demo Crossed fingers created by Michael A. Salter from Noun Project (
  5. 5. Why containerization? Why Docker?
  6. 6. Motivation Pets vs. Cattle
  7. 7. Motivations for Docker in mainstream IT
  8. 8. Science Reproducibility is at the of Motivation for Reproducible Research
  9. 9. Executable Research Compendium Docker logo courtesy of Docker Inc.; Trafic lights Bluemix via Wikimedia Commons; crowbar by Delapouite via; zipper by RRZEIcons, cursor by Subhashish Panigrahi, via Wikimedia Commons;
  10. 10. via Daniel Whitenack, and-engineering-part-2-dockerized-data-science/ Motivation for Data Scientists - Proof by quote
  11. 11. Slide by Docker inventor & Docker, Inc. CTO Solomon Hykes, DockerCon 2014
  12. 12. Application packaging using kernel features namespaces libcontainer, LXC cgroups resources Houses vs. Appartments | “binary” vs. OS “Containerization”
  13. 13. Docker basics Dockerfile ENV RUN CMD Docker Image pause stop/kill start logs cp exec rm stats build Docker CLI run Docker Container Docker Engine Docker Registry run use n Docker Container Docker Container Docker Container up down docker-compose configuration one: … two: … …
  14. 14. DockerHub
  15. 15. docker run --rm -it -p 8787:8787 rocker/rstudio http://localhost:8787/ (rstudio/rstudio) Great example: docker run --rm -it -p 8787:8787 benmarwick/mjb1989excavationpaper http://localhost:8787/ (rstudio/rstudio)
  16. 16. Jupyter Notebook wget re_data_scientists_and_tools.ipynb docker run -it -p 8888:8888 -v $(pwd):/home/jovyan/work jupyter/datascience-notebook http://localhost:8888/ See also
  17. 17. ELK stack git clone cd docker-elk # add filter to logstash/config/logstash.conf: # filter { # grok { match => { "message" => " %{COMBINEDAPACHELOG}"} # } #} docker-compose up http://localhost:5601/app/kibana http://localhost:9200/ Example data: nc localhost 5000 > access_log_Aug95 docker-compose down (-v)
  18. 18. docker run --rm -it -p 3000:3000 -p 8080:8080 -p 8181:8181 --name carto sverhoeven/cartodb sudo sh -c 'echo cartodb.localhost >> /etc/hosts' docker run --rm -it -p 80:80 --link carto:cartodb.localhost spawnthink/cartodb-nginx http://cartodb.localhost dev/pass1234
  19. 19. Deep convolutional network in Amazon Cloud git clone docker run -d -p 8888:8888 -v /home/ubuntu/pug_classifier:/home/jovyan/work mdagost/pug_classifier_notebook
  20. 20. Interested in “geo”? Go to OSGeo wiki +
  21. 21. Core arguments for Data Scientists (all the Docker advantages… write once, biz ops, cloud, etc.) Reproducibility Project separation + don’t clutter dev machine Environment (re)creation, documentation Adopt good practices on the way (dev cred) Easy collaboration Easy transition from testing to production
  22. 22. More from the Dockerverse Docker Machine (provision remote host or clusters) Docker Cloud (hosting of Dockerized apps) Docker Toolbox (for older Mac and Windows OS) Docker Universal Control Pane (custer management and monitoring UI) Docker Swarm mode (container orchestration) Docker Trusted Registry (own enterprise image storage) Kubernetes (container orchestration)
  23. 23. Thanks for your attention! What are your questions? http://nö Want more Docker? Watch Dockercon Keynote!