MIT Licensed - Reuse freely, but attribute "Hamilton Turner"
An introduction to the Docker container engine. Focuses on how to use Docker and implications of Docker for Cloud-based services. Shows multiple examples of rapidly starting complex environments using Docker. Very minor discussion on how Docker works technically.
Presentation source is available at https://github.com/hamiltont/intro-to-docker
Shipping Applications to Production in Containers with DockerJérôme Petazzoni
Docker is an Open Source engine to build, run, and manage Linux Containers. Containers use less resources than virtual machines, they boot faster, but they have similar guarantees of portability and repeatability for Linux applications. Those features made Docker and Linux Containers extremely popular for development and testing environments. But what does it take to use Docker and Containers for production workloads?
Docker has created enormous buzz in the last few years. Docker is a open-source software containerization platform. It provides an ability to package software into standardised units on Docker for software development. In this hands-on introductory session, I introduce the concept of containers, provide an overview of Docker, and take the participants through the steps for installing Docker. The main session involves using Docker CLI (Command Line Interface) - all the concepts such as images, managing containers, and getting useful work done is illustrated step-by-step by running commands.
Introduction to Docker and all things containers, Docker Meetup at RelateIQdotCloud
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
Shipping Applications to Production in Containers with DockerJérôme Petazzoni
Docker is an Open Source engine to build, run, and manage Linux Containers. Containers use less resources than virtual machines, they boot faster, but they have similar guarantees of portability and repeatability for Linux applications. Those features made Docker and Linux Containers extremely popular for development and testing environments. But what does it take to use Docker and Containers for production workloads?
Docker has created enormous buzz in the last few years. Docker is a open-source software containerization platform. It provides an ability to package software into standardised units on Docker for software development. In this hands-on introductory session, I introduce the concept of containers, provide an overview of Docker, and take the participants through the steps for installing Docker. The main session involves using Docker CLI (Command Line Interface) - all the concepts such as images, managing containers, and getting useful work done is illustrated step-by-step by running commands.
Introduction to Docker and all things containers, Docker Meetup at RelateIQdotCloud
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
Docker and Containers for Development and Deployment — SCALE12XJérôme Petazzoni
Docker is an Open Source engine to build, run, and manage containers. We'll explain what are Linux Containers, what powers them (under the hood), and what extra value Docker brings to the table. Then we'll see what the typical Docker workflow looks like from a developer point of view. We'll also give an Ops perspective, including deployment options. If you already saw a "Docker 101", consider this presentation as the February 2014 update! :-)
Introduction to Docker - Docker workshop @TwitterdotCloud
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
From development environments to production deployments with Docker, Compose,...Jérôme Petazzoni
In this session, we will learn how to define and run multi-container applications with Docker Compose. Then, we will show how to deploy and scale them seamlessly to a cluster with Docker Swarm; and how Amazon EC2 Container Service (ECS) eliminates the need to install,operate, and scale your own cluster management infrastructure. We will also walk through some best practice patterns used by customers for running their microservices platforms or batch jobs. Sample code and Compose templates will be provided on GitHub afterwards.
Docker containers are other piece of the new Connections architecture that makes it a highly extensible and flexible collaboration platform. Flashing back to IBM Connect 17 in San Francisco, I knew Docker was going to be a topic of high interest as the Docker session was standing room only. Predicated on this I decided to conduct an introduction to Docker session at Social Connections 11.
An introduction to Docker and docker-compose. Starting from single docker run commands we discover docker file basics, docker-compose basics and finally we play around with scaling containers in docker-compose.
Running Cassandra in a docker environment to give you a flexible development environment that uses only a very small set of resources, both locally and with your favorite cloud provider. Lessons learned running Cassandra with a very small set of resources are applicable to both your local development environment and larger, less constrained production deployments.
From http://www.meetup.com/Docker-Santa-Clara/events/232789407/
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...Codemotion
In less than two years Docker went from first line of code to major Open Source project with contributions from all the big names in IT. Everyone is excited, but what's in for me - as a Dev or Ops? In short, Docker makes creating Development, Test and even Production environments an order of magnitude simpler, faster and completely portable across both local and cloud infrastructure. We will start from Docker main concepts: how to create a Linux Container from base images, run your application in it, and version your runtimes as you would with source code, and finish with a concrete example.
Docker and Containers for Development and Deployment — SCALE12XJérôme Petazzoni
Docker is an Open Source engine to build, run, and manage containers. We'll explain what are Linux Containers, what powers them (under the hood), and what extra value Docker brings to the table. Then we'll see what the typical Docker workflow looks like from a developer point of view. We'll also give an Ops perspective, including deployment options. If you already saw a "Docker 101", consider this presentation as the February 2014 update! :-)
Introduction to Docker - Docker workshop @TwitterdotCloud
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
From development environments to production deployments with Docker, Compose,...Jérôme Petazzoni
In this session, we will learn how to define and run multi-container applications with Docker Compose. Then, we will show how to deploy and scale them seamlessly to a cluster with Docker Swarm; and how Amazon EC2 Container Service (ECS) eliminates the need to install,operate, and scale your own cluster management infrastructure. We will also walk through some best practice patterns used by customers for running their microservices platforms or batch jobs. Sample code and Compose templates will be provided on GitHub afterwards.
Docker containers are other piece of the new Connections architecture that makes it a highly extensible and flexible collaboration platform. Flashing back to IBM Connect 17 in San Francisco, I knew Docker was going to be a topic of high interest as the Docker session was standing room only. Predicated on this I decided to conduct an introduction to Docker session at Social Connections 11.
An introduction to Docker and docker-compose. Starting from single docker run commands we discover docker file basics, docker-compose basics and finally we play around with scaling containers in docker-compose.
Running Cassandra in a docker environment to give you a flexible development environment that uses only a very small set of resources, both locally and with your favorite cloud provider. Lessons learned running Cassandra with a very small set of resources are applicable to both your local development environment and larger, less constrained production deployments.
From http://www.meetup.com/Docker-Santa-Clara/events/232789407/
Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...Codemotion
In less than two years Docker went from first line of code to major Open Source project with contributions from all the big names in IT. Everyone is excited, but what's in for me - as a Dev or Ops? In short, Docker makes creating Development, Test and even Production environments an order of magnitude simpler, faster and completely portable across both local and cloud infrastructure. We will start from Docker main concepts: how to create a Linux Container from base images, run your application in it, and version your runtimes as you would with source code, and finish with a concrete example.
Running the Oracle SOA Suite Environment in a Docker ContainerGuido Schmutz
Docker is all about making it easier to create, deploy, and run applications by using containers. Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package. Docker helps creating, moving and duplicating environments.
This presentation will give an introduction to Docker, the ideas behind containerization and explain why there is so much hype around Docker and why you should be taking notice. I will show how Docker containers can be used to setup different environments, such as SOA Suite, Service Bus, Business Activity Monitoring and Event Processing and Stream Explorer. The talk will also include various short live demos.
Introduction to Docker - Learning containerization XP conference 2016XP Conference India
Docker is an open-source platform which provides a great way to package and deploy applications. With its lightweight resource consumption pattern, it helps in making CI/CD environments faster and predictable. Learn how to setup Docker and deploy a basic web application.
An introduction to Docker (Docker.io). Just a brief overview that describes a little of what Docker has to offer. The talk was hosted at kWantera HQ in Pittsburgh PA.
Docker is popular open-source software containerization platform. It provides an ability to package software into standardised units on Docker for software development. In this hands-on introductory session, I introduce the concept of containers and provide an overview of Docker. Participants can learn important concepts in Docker step-by-step and learn by example by running commands with me. The main session involves using Docker CLI (Command Line Interface) covering all the key concepts such as creating images and managing containers. What is more, this workshop ends with a complete example of getting some amazing work done with ease using Docker. Presented in OSI Days '16: http://opensourceindia.in/osidays/workshops-osi-2016/
ExpoQA 2017 Using docker to build and test in your laptop and JenkinsElasTest Project
In this workshop the basics about container use in the development environment are presented. Then we go further by describing how to leverage containers in the CI server, using Jenkins and Pipelines.
Presentato al sesto WebMeetup del Machine Learning / Data Science Meetup Roma: https://www.meetup.com/it-IT/Machine-Learning-Data-Science-Meetup/events/273089965/
Docker is an amazing tool, but unless you work with it every day, you're probably left with a ton of questions. What's a container? What's an image? What's the difference between Docker, Machine, Compose, and Swarm? Why the heck should I care? Well, Docker makes it easier than ever to deploy and scale your applications and services. In addition, it lets you simulate your production environment on your local machine without heavy virtual machines. In this talk, we'll explore the basics of Docker, create a custom image for a web application, create a group of containers, and look at how you can put your apps into production on various cloud providers. At the end of the talk, you'll have the knowledge you need to put this to use with your own applications.
Docker 1 0 1 0 1: a Docker introduction, actualized for the stable release of...Jérôme Petazzoni
If you're not familiar yet with Docker, here is your chance to catch up. This presentation includes a quick overview of the Open Source Docker Engine, and its associated services delivered through the Docker Hub. Recent features are listed, as well as a glimpse at what's next in the Docker world.
This presentation was given during OSCON, at a meet-up hosted by New Relic, with co-presentations from CoreOS and Rackspace OnMetal.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Transcript: Selling digital books in 2024: Insights from industry leaders - T...BookNet Canada
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Slack (or Teams) Automation for Bonterra Impact Management (fka Social Soluti...Jeffrey Haguewood
Sidekick Solutions uses Bonterra Impact Management (fka Social Solutions Apricot) and automation solutions to integrate data for business workflows.
We believe integration and automation are essential to user experience and the promise of efficient work through technology. Automation is the critical ingredient to realizing that full vision. We develop integration products and services for Bonterra Case Management software to support the deployment of automations for a variety of use cases.
This video focuses on the notifications, alerts, and approval requests using Slack for Bonterra Impact Management. The solutions covered in this webinar can also be deployed for Microsoft Teams.
Interested in deploying notification automations for Bonterra Impact Management? Contact us at sales@sidekicksolutionsllc.com to discuss next steps.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
How world-class product teams are winning in the AI era by CEO and Founder, P...
Introduction To Docker
1. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Next-Gen Cloud Computing and DevOps with
Docker Containers
Hamilton Turner
hamiltont@gmail.com
March 12, 2014
March 2014 1 / 43
2. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Who am I?
Who am I?
• Ph.D. Candidate at Virginia Tech
• Defending in Fall 2014
• Member of Magnum Research Group
(http://www.magnum.io/)
• We focus on mobile and cloud applications
How do I use Docker?
• Stress test 100+ web apps, without installing natively
• Manage computer cluster for research group
• Personal playground for learning new technologies
March 2014 2 / 43
3. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
DevOps Challenges: The Matrix From Hell
django web frontend ? ? ? ? ? ?
node.js async API ? ? ? ? ? ?
background worker tasks ? ? ? ? ? ?
SQL Database ? ? ? ? ? ?
distributed DB, big data tools ? ? ? ? ? ?
message queue ? ? ? ? ? ?
my laptop your laptop QA system staging production on Cloud VM production on bare metal
• Each environment looks different
• Different developer needs
• Different operations environments
The ‘Big Question’
• How do we make all these environments identical?!
Credit: J´erˆome Petazzoni at http://www.slideshare.net/jpetazzo/
introduction-docker-linux-containers-lxc
March 2014 3 / 43
4. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Review of Virtual Machines
Virtual Machines are one current solution
• An ‘entire computer’ is
executed as a program
• No developer’s local
system has to be modified
• The entire VM disk image
can be shared
• No more differences
between dev and
production environments!
• This is great! ....Right?
March 2014 4 / 43
5. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Comparison of Docker Containers And VMs
Credit: quay.io - A Secure Hosting Solution For Private Docker Repositories
Docker Container Virtual Machine
Avg Host Resources Consumed Low High
Clean Startup Time seconds minutes/hours
Environment (FS) Sharing Small (Union filesystem) Large (Entire Snapshot)
Environment Reproducibility High Moderate (AWS image)
Software Modifications Needed? Perhaps (one process) Unlikely
Attack Surface Untested Small
System Monitoring Use Linux Tools Custom systems
March 2014 5 / 43
6. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
The How of Docker
Docker shares the kernel with the host, uses Linux
namespaces+cgroups+union filesystems to isolate
• process trees (PIDs)
• mounts (mnt)
• network (net)
• inter-process
communication (ipc)
• user accounts (user)
• hostnames (utc)
• memory
• CPU
• Disk access (blkio)
• Device access (devices)
Summary: Docker combines and standardizes a number of
existing Linux components (kernel 3.8+)
March 2014 6 / 43
7. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
The How of Docker, Union Filesystem Version
• Each layer of the FS is mounted
on top of prior layers
• The first layer is the base image
• Current base images include
debian, ubuntu, busybox, fedora,
cent os, etc
• Each read-only layer is called an
image (A layer is just a
collection of files and folders!)
• The top layer is the only
modifiable layer - it’s termed the
container
March 2014 7 / 43
8. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Docker 101: Run Interactive Container
$ sudo docker run −i −t ubuntu / bin / bash
• sudo : Docker has to be run as root!
• run : we are running a container
• -i -t : we want a terminal (stdin and stdout), and we
want to be connected to those so we can interact with
the continer
• ubuntu : The base image for this container
• /bin/bash : Let’s run bash
$ sudo docker run −i −t ubuntu / bin / bash
root@03711559d57d :/ # cat /etc/*release*
DISTRIB ID=Ubuntu
DISTRIB RELEASE=12.04
DISTRIB CODENAME=p r e c i s e
DISTRIB DESCRIPTION=”Ubuntu 12.04 LTS”
root@03711559d57d :/ # exit
March 2014 8 / 43
9. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Docker 101: Run Non-Interactive Container
Flags -i and -t are good for interacting with a container, but
for scripting or long-running tasks, you’ll want to use detached
(-d) mode
$ sudo docker run −d ubuntu / bin / bash −c ” echo h i ”
94490365 f464bab1f009ec0971e1691213b4562dbaeb04b2e33ad
$
Odd things:
• There was no ‘hi’
• You were given this long string
• You are back at your original shell, even though you ran
bash
In detached mode, docker immediately returns a container ID.
This ID can be used to fetch container stdout, check container
status, stop the container, etc
March 2014 9 / 43
10. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Docker 101: Run Non-Interactive Container, Part
Two
Ok, let’s see what’s happening using our container ID
$ sudo docker run −d ubuntu / bin / bash −c ” echo h i ”
d2026870efedf09e29dbea146d399e60493e9dd0ebbf6124347d6
$ sudo docker l o g s d2026870efedf09e29dbea146d399e6049
h i
Container ID’s can be referenced by unique prefix too
$ sudo docker l o g s d202
h i
docker ps shows you what containers are running
$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS
d2026870ef ubuntu : 1 2 . 0 4 / bin / bash −c w h i l e t 1 minute ago Up 1 min
March 2014 10 / 43
11. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
More on Container IDs
Typically, you will want to store the ID
$ MY ECHO=$( sudo docker run −d ubuntu / bin / bash
−c ” echo h i ” )
$ sudo docker l o g s $MY ECHO
h i
• Detached Mode (e.g. docker run -d)
• Docker run response is the container ID
• To capure the output, we use $(...)
• This output is stored into variable MY ECHO,
and later retrieved with $MY ECHO
• Interactive Mode (e.g. docker run -i -t)
• Run container, modify, then exit. Container is now stopped
• Use docker ps -a to show all containers, incl. stopped
ones
• Or use docker ps -l -q to show the last container ID
$ sudo docker ps −a
CONTAINER ID IMAGE COMMAND CREATED STATUS
d2026870ef ubuntu : 1 2 . 0 4 / bin / bash −c w h i l e t 1 minute ago E x i t 0
$ sudo docker ps −q −l
d2026870ef
Note: Docker now supports container names
March 2014 11 / 43
12. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Storing A Container For Reuse
(a.k.a. Building an Image)
• Recall: the container filesystem is the
final union with a stack of images
• docker commit converts this container
filesystem into an image
• The image can then be used to run
other containers
$ APP=$ ( sudo docker run −d ubuntu / bin / bash −c
‘ ‘ echo h i > c o n f i g . out ’ ’ )
$ sudo docker commit $APP hamiltont /myapp
$ sudo docker run −i −t hamiltont /myapp / bin / bash
root@3a1f0c28b822 :/ # cat config.out
h i
If you could share this image...then others could build new
containers based on this image!
March 2014 12 / 43
13. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Sharing An Image For Reuse
• Images can be shared using a registry
• docker push and docker pull
• There is a public registry available, or your company can
host it’s own private registry Check out quay.io
• If docker run does not find the image locally, it will
automatically search known registries
$ sudo docker push hamiltont /myapp
$ sudo docker p u l l hamiltont / myotherapp
• The images subcommand can be used to list local images
$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
hamiltont /myapp l a t e s t d100b411c51e 2 minutes ago 204.4 MB
hamiltont / myotherapp l a t e s t 7 cb2d9010d39 11 days ago 410.6 MB
ubuntu 12.04 9 cd978db300e 5 weeks ago 204.4 MB
ubuntu l a t e s t 9 cd978db300e 5 weeks ago 204.4 MB
March 2014 13 / 43
14. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Benefits of Using Union Filesystems For Images
• Hypothetical:
• I run a ubuntu container, make changes, and exit
• I commit my changes to an image and run docker push
• My colleage wants to docker pull my image
• What do they need to download?
• Answer:
• Just your changes!
• They have probably already downloaded the ubuntu base
image
• No inherent need for multi-GB images
• Download only files, not arbitrary filesystem junk
• While YMMV,
80% of images are ≤ 50MB, 95% are ≤ 500MB
March 2014 14 / 43
15. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Linking Host/Container Filesystems
A nice seperation of concerns would place application inside the
container, logs outside the container.
The -v flag can mount a host volume to the container
$ sudo mkdir / app logs
$ sudo docker run −i −t −v / app logs :/ l o g s ubuntu
/ bin / bash
root@842fa9699353 :/ # cd /logs/
root@842fa9699353 :/ l o g s # echo "My application log"
> log . out
root@842fa9699353 :/ l o g s # exit
$ cat / app logs / log . out
My a p p l i c a t i o n log
-v can also be used to access configuration on the host
$ sudo mkdir / app conf
$ sudo docker run −i −t −v / app conf :/ etc /app ubuntu
/ bin / bash
root@842fa9699353 :/ # my_app --conf /etc/app
March 2014 15 / 43
16. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Exposing Container Network Ports
Docker container ports are not published unless requested.
The -p flag can be used to publish a port
$ SERVER=$ ( docker run −d −p 8000 ubuntu / bin / bash
−c ‘ w hi le t r u e ; do s l e e p 5; done ’ )
$ sudo docker port $SERVER 8000
0 . 0 . 0 . 0 : 4 9 1 5 8
Breakdown:
• Run a bash process inside the container, looping
indefinitely
• -p 8000 caused Docker to find an unused host port and
link it with the container-internal port 8000
• We used the port subcommand to find this public port
• There is nothing listening on port 8000 in the container,
so this is kind of boring
March 2014 16 / 43
17. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Exposing Container Network Ports, Part Two
So let’s run an actual webserver!
Method 1: Build my own webserver image
Method 2: Reuse someone else’s pre-built image
$ WEB SERVER=$ ( sudo docker run −t −d
−p 8000 hamiltont /python−s i m p l e h t t p s e r v e r )
$ sudo docker l o g s $WEB SERVER
Serving HTTP on 0 . 0 . 0 . 0 port 8000 . . .
$ sudo docker port $WEB SERVER 8000
0 . 0 . 0 . 0 : 4 9 1 8 6
Note:
• I chose to reuse hamiltont/python-simplehttpserver
• Navigating to http://localhost:49186 will now
connect me to the webserver
• The container knew what command to run! More on this
next...
March 2014 17 / 43
18. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Building Images with Dockerfiles
• We know how to run a container, modify it, and commit it
as an image
• A Dockerfile lists the steps needed to build an images
• Similar to a Makefile
• docker build is used to run a Dockerfile
• Can define default command for docker run, ports to
expose, etc
March 2014 18 / 43
19. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Locating Community Images
• There are hundreds of community-contributed and/or
official images online at http://index.docker.io
• This is the official registry, you can also host your own
• You can also use the docker search subcommand to
interact with index.docker.io
$ sudo docker s earch wordpress
NAME DESCRIPTION STARS TRUSTED
c t l c / wordpress 4 [OK]
j b f i n k / docker−wor Same as j b f i n k / wordpress , j u s t a t r u s t e d b 2 [OK]
skxskx / wordpress Wordpress & SSH i n a c o n t a i n e r . 2
eugeneware / docker 1 [OK]
tutum/ wordpress Wordpress Docker image − l i s t e n s i n port 80 1 [OK]
j b f i n k / wordpress Wordpress 3.8 1
March 2014 19 / 43
20. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Docker 101 Review: Topics Covered
• Running Docker Containers In Interactive or Detached
Mode
• Container IDs (and names)
• Viewing output of detached container (docker logs)
• Committing containers to create images
• Sharing images via push/pull
• Mounting Filesystems From Host inside container
• Exposing container network ports
• Basic concept of Dockerfiles
March 2014 20 / 43
21. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Docker 101: Things We Didn’t Cover
• Detach from running container, then re-attaching
• Diff’ing filesystems
• Storing container/image as tar file
• Using docker kill to stop a running container
• Deleting unused containers/images
• Examining processes inside container from the host
• Linking containers together (via filesystem or network)
• Trusted builds
• Limiting container memory and CPU consumption
Any questions at this point?
March 2014 21 / 43
22. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
A (small) selection of Dockerized Projects
Data Storage Redis, MySQL, MongoDB,
memcached
Server Stacks Nginx, Apache+PHP, SSH
Development
Environments
Python, Go, Ruby, Java,
Chef/Puppet, node.js,
X11+SSH Desktop
Blogging / CMS Wordpress, Ghost
Single-use tool usage Redis CLI, Latex, subuser
Continuous Integration Jenkins, Drone
Proxy Software Hipache, Nginx/Apache/noje.js
PaaS Cocaine (from Yandex), Deis,
Flynn, Bowery, Dokku
March 2014 22 / 43
23. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Example: Wordpress
# WP=$(docker run -d -p 80 tutum/wordpress)
# docker port $WP 80
0 . 0 . 0 . 0 : 4 9 1 5 9
March 2014 23 / 43
24. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Example: Redis Commander Web GUI
# RD=$(docker run -d -p 8081 elsdoerfer/redis -commander)
# docker port $RD 8081
0 . 0 . 0 . 0 : 4 9 1 5 9
March 2014 24 / 43
25. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Example: Sandboxed Dev Environment
# docker run -t -i -p 22 magglass1/docker -browser -over -ssh
IP a dd r es s : 1 7 2 . 1 7 . 0 . 4
Password : N24DjBM86gPubuEE
F i r e f o x : ssh −X webuser@172 . 1 7 . 0 . 4 f i r e f o x
Google Chrome : ssh −X webuser@172 . 1 7 . 0 . 4 google−chrome −−no−sandbox
March 2014 25 / 43
26. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Example: SSH Server
# SSH=$(docker run -d -p 22 dhrp/sshd)
# docker port $SSH 22
0 . 0 . 0 . 0 : 4 9 1 6 0
Ok, SSH is running. Now to connect!
$ ssh −p 49161 root@10 . 0 . 0 . 2
root@10 . 0 . 0 . 2 ’ s password :
Welcome to Ubuntu 12.04 LTS (GNU/ Linux 3.11.0−18− g e n e r i c x86 64 )
∗ Documentation : h t t p s :// help . ubuntu . com/
The programs i n c l u d e d with the Ubuntu system are f r e e s o f t w a r e ;
the exact d i s t r i b u t i o n terms f o r each program are d e s c r i b e d i n the
i n d i v i d u a l f i l e s i n / usr / share /doc /∗/ c o p y r i g h t .
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the e x t e n t permitted by
a p p l i c a b l e law .
root@7d45b427eca1 :˜ #
March 2014 26 / 43
27. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Example: Continuous Integration
# JEN=$($docker run -d -p 8080 --cpu -shares =20 lzhang/jenkins)
# docker port $JEN 8080
0 . 0 . 0 . 0 : 4 9 1 6 0
Note: I’ve limited the CPU shares allowed
March 2014 27 / 43
28. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Example: Quick Web Server
# WEB=$(docker run -d -v /Programming :/ www -p 8000 hamiltont/python - simplehtt
# docker port $WEB 8000
0 . 0 . 0 . 0 : 4 9 1 6 0
Note: I’ve mounted my /Programming folder to /www
March 2014 28 / 43
29. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Example: Using Redis CLI Without Installing It
# docker run -i -t crosbymichael/redis -cli -h
r e d i s . myhost . com ‘KEYS ∗ ’
1) ” f o u r ”
2) ”one”
3) ”two”
• Redis is not installed on the host
• I can use complex (and stateful!) commands without
modifying the host machine
• I could even pull my dotfiles onto a production machine
• No noticable run delay
March 2014 29 / 43
30. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Example: Preexisting Automation Tools
• Docker doesn’t exclude existing tools like Chef, Puppet,
etc
• Current base systems are limited to (mostly) Debian
• Most current automation systems can be used seamlessly
• But...you no longer have to worry as much about OS
updates as the Docker image is static ;-)
CentOS+Chef:
# docker run -i -t raybeam/chef /bin/bash
bash −4.1# knife
ERROR: You need to pass a sub−command
( e . g . , k n i f e SUB−COMMAND)
. . . <snip >
March 2014 30 / 43
31. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Docker Is Not a Panacea: Considerations
• No Shared Libraries
• The price of process isolation
• ADD-only filesystem
• Scenario: Run a huge build and then rm /tmp/*
• Result: All of /tmp will be downloaded, and
then masked with an empty directory
• Root access required for simple operations!
• Progress is being made on this front
• Orchestration of Containers is limited...more on this next
March 2014 31 / 43
32. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Docker Service Orchestration and Service Discovery
If my application has 10 containers, how do I organize them all?
No clear winner here, but many solutions in progress:
• Raw Docker
• docker run -v can be used to call out interesting volumes
within contianers
• volumes from can be used to share these volumnes with
other containers
• Containers can be linked to share their network
• Fig: Uses simple config to start/link containers
• Serf: General service discovery solution
• Shipyard: Web-based system for managing docker-driven
applications
• CoreOS: OS designed for running cloud apps such as
Docker
• SkyDock: DNS-based Docker service discovery
March 2014 32 / 43
33. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Rough Test of Docker Overhead
• Launch and sleep tiny containers (busybox, 125Kb)
• Docker overhead hopefully trumps this
• Using 4GB, 4-core system
# while true
> do
> docker run −d busybox / bin / ash −c
” wh il e t r u e ; do echo a l i v e ; s l e e p 60000; done”
> docker ps | wc −l
> done
Fast Fail Results:
• 250 containers launched before “Too many open files”
• <2GB memory used, load of 3 (all sleeping)
March 2014 33 / 43
34. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Recap: Docker and Virtual Machines
Q: Are Docker Container just ‘Better Virtual Machines’?
• Docker monitors one process, VMs have system daemons
running (cron, syslog, upstart, etc)
• You could run/monitor a process manager (e.g.
supervisord)
• Let’s consider separation of concerns
• Given: Containers are light, VMs are heavy
• It’s unlikely you allocate two VMs and communicate - you
shove your entire ecosystem into a single VM
• This jumbles concerns together and reduces
maintainability, predictability, and security
• Containers emulate processes, VM’s emulate machines
• Often ‘Dev’ means you work inside the container, and
‘Ops’ means you work outside the container
March 2014 34 / 43
35. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Docker and Hadoop
What do I need?
• Development Environment
• With full source code / docs
• Good base image for configuring IDE
• Production Environment
• Minimal Hadoop footprint
• All dependencies
• Native 64-bit libraries
• Commonalities?
• Yes, Java
• Ok, so three images: Java, Dev,
Production
Ubuntu:12.04
Java
Hadoop
Development
Hadoop
Production
March 2014 35 / 43
36. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Developing Oracle Java7 Image
• Need to base our image on a stable parent
• Need to carefully record our steps
• Want to share this with all developers
• Use Dockerfile!
March 2014 36 / 43
37. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
March 2014 37 / 43
38. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Developing Hadoop-Dev Image
• Build on top of Java7 image
• Install build tools
• Install hadoop build dependencies
• Run build process
• Of Note: This image is huge! > 1GB due to all the build
dependencies downloaded by maven
March 2014 38 / 43
39. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Developing Hadoop Docker
March 2014 39 / 43
40. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Developing Hadoop Image
• Use docker cp to extract the tar from hadoop-dev
• Add that to hadoop, extract tar file
• Update environment PATH
• Install SSH server
• Pull in configuration files
• Automatically start SSH, hadoop processes
Final product is online at
github.com/hamiltont/hadoop-docker
March 2014 40 / 43
41. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Developing Hadoop Docker
March 2014 41 / 43
42. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Conclusions: Who and What is Docker
Officially, Docker is...
• Open Source, 200+ contributors
• Corporate parent named Docker as well ;-)
• Very active IRC + mailing lists
• A project to combine and standardize existing features of
Linux
Unofficially, Docker is...
• The forefront in Linux Containers
• A huge step beyond current VM’s w.r.t. machine
utilization and DevOps workflow
• A pragmatic improvement that is here to stay
March 2014 42 / 43
43. Who Am I?
The DevOps
Challenge
Beyond VMs
The How of
Docker
Docker 101
Docker
Examples
Docker Limits
Hadoop Demo
Conclusions
Thank You For Your Time
Questions?
Please feel free to reuse/modify presentation if you wish, just
remember to leave my name in there somewhere. It’s online at
https://github.com/hamiltont/intro-to-docker
March 2014 43 / 43