SlideShare a Scribd company logo
1 of 44
Download to read offline
Docker at DevTable
What is DevTable?
DevTable is a browser-based, hosted,
collaborative IDE
Develop in the cloud with the same power as
your desktop applications
Code
Collaborate
Debug and Test
Deploy
● Google App Engine
● REST
● SCP
● Git (Heroku and other providers)
Sealed evil in a can
There are a lot of neat things that we run for
our users, but they are all potentially very
dangerous:
● App Engine Development Server
● Debuggers and Emulators
● REPLs (Python, etc)
● Terminal support (which means all of the
above as well)
Why this is a problem
● Without a containment system of some kind,
any of these awesome features would allow
users to cause mayhem:
○ A REPL use could open any file
○ A DevServer can execute arbitrary code
○ A terminal could allow anything to happen
Why not simply use permissions?
● Permissions solve the file access problem
● Permissions do not prevent users from
causing other system issues: instability,
exhaustion, escalation, etc
Solution: containers!
To contain the insecurity of running live code,
we run all non-custom code in a container,
with only the user’s project mounted and
available
Evil (not to scale)
Project data
Container
Ideal container properties
● Lightweight
● Secure
● Easy to manage
● FAST
Originally we used LXC...
● Lightweight (sort of…)
● Secure
● Easy to manage (sort of...)
● FAST
In the beginning, there was LXC...
… and it was slow.
● Typical startup times for our containers were
on the order of minutes
● Starting a debugger or shell is not fun at
those speeds
● Getting the security and management just
right was quite painful
Then the community said “let there
be Docker”...
Yo!
… and it made things amazing.
Our average startup time for a container has
dropped from over a minute to just under
four seconds.
LXC
Docker
Go make a cup of coffee and play swords on office chairs
Go!
Before Docker
But, but Docker is just... LXC...
Almost, Docker does some things that make
starting up single processes lightning quick:
● Incremental by default
● Replace distro init process with lightweight
version
● No DHCP, upstart, dnsmasq, etc.
● Aufs seems to be faster than OverlayFS
● Build process is MUCH better (Dockerfiles)
Docker at DevTable
The fun technical details!
DevTable overview
Clients
Web
browsers
Clients -
Web
browsers
Frontends
Python
Clients -
Web
browsers
Backends
C#
WebSocket Socket
DFS
Clients -
Web
browsers
Container
Servers
Python
Thrift
SSH
HTTP
?
Images
Things we’ll discuss today
Clients
Web
browsers
Clients -
Web
browsers
Frontends
Python
Clients -
Web
browsers
Backends
C#
WebSocket Socket
DFS
Clients -
Web
browsers
Container
Servers
Python
Thrift
HTTP
SSH
?
Images
How we use docker now
● Python Docker API bindings
● Run a single instance per project
● Mount only the files relevant to the project in
the container
● Run an SSH “command and control” process
● Execute user processes through SSH
● Dynamic version of Docker port forwarding
Backend <-> Container server
Backends
C#
Container
Servers
Python
Thrift
Container server
The container server is the server in charge of
managing all aspects related to the Docker
containers
● Written in Python
● Conforms to a Thrift interface
● Called by the Backends to start containers,
stop containers, run commands, mount file
systems in containers, etc
Container server
startContainer
Starts a new container for a project.
runCommand
Runs a command inside a container
stopCommand
Stops a command inside a container
notifyFilesModifed
Notifies a container that a file has been modified by the backend
stopContainer
Stops a container
Handling file changes
● Changes made by the container or the
backend to the DFS are propagated
automatically
● However, both sides have code that
depends on notification of changes
● Each server notifies the other about
changes that occur via a notification service
DFS change notifications
Backend
C#
Container
Server
Python
Hey, a user added file “test.txt” in container 1234
Backend
C#
Container
Server
Python
Hey, the user changed file “foo.py” in container 1
How we handle file changes in
Docker
● The container server watches changes
inside the container using inotify, and
reports changes to the backend
● The backend reports changes to the
container server which will touch files that
have been added or changed
Container server <-> Docker
Clients -
Web
browsers
Container
Servers
Python
SSH
Container server <-> Docker
We use the Python Docker bindings to create a
new image and load it with a temporary ssh
key
New container requests bring up the container
with the known session SSH key and issue
commands to the container via SSH
Much better than LXC issuing commands via
subprocess
Docker <-> Outside world
For many services we run (such as the App
Engine Development Server), we need to
expose the server running inside Docker to
the outside world
Docker <-> Outside world
HTTP
Clients
Web
browsers
HTTP
Container Server
HAProxy
Docker <-> Outside world
Services inside of Docker as exposed via
dynamic port mapping to a HAProxy
running on the container server
The HAProxy exposes the port by remapping it
to the external port and a custom
subdomain
Docker <-> Outside world
Container Server
93nx83ndsc34mn.c4.devtable.io:80Clients
Web
browsers
Port 38563
HAProxy
Example: running a dev server
1. Backend requests a container from the
server
Backend
C#
Container
Server
Python
I need a container for project “testapplication”
Container “container1234” started for project
Example: running a dev server
2. Backend registers for file notification events
Backend
C#
Container
Server
Python
Let me know if any files change
Duly noted
Example: running a dev server
3. Backend asks for the dev server to be
started and port 80 to be forwarded
Backend
C#
Container
Server
Python
Please start the dev server and forward port 80
Dev server started and port is forwarded at
subdomain foobarbaz
Example: running a dev server
1. Container server tells Docker to start a
container
Container
Server
Python
create_container, mount_filesystem,
forward_port, start_ssh
Done. Port exposed: 84639
Example: running a dev server
2. Container server tells HAProxy to forward
the port returned by docker
Container
Server
Python
Forward port 84639 as subdomain
foobarbaz
HAProxy
Example: running a dev server
3. Container server tells Docker to run the dev
server
Container
Server
Python
ssh command_for_devserver
Summary
Docker has allowed DevTable to run amazing
tools securely and fast, without a large
management overhead
Future opportunities
Docker presents some amazing new
opportunities for DevTable and the
community:
● Ability to quickly load (and save) complete
development environments, securely
● Ability to quickly write custom plugins and
run them in our IDE (want to analyze and
build Go? just give us a URL or a
Dockerfile!)
But wait…
There’s something that has been
bugging us…
How should we distribute our
private images in production?
Quay Demo
At this point in the live talk we unveiled and
gave a demo of our hosted private docker
registry called Quay.io.
Questions? Comments? Witty
anecdotes?
devtable.com
Jacob Moshenko - jake@devtable.com
Joseph Schorr - jschorr@devtable.com

More Related Content

What's hot

Docker Introduction
Docker IntroductionDocker Introduction
Docker IntroductionRobert Reiz
 
Introduction to Containers - SQL Server and Docker
Introduction to Containers - SQL Server and DockerIntroduction to Containers - SQL Server and Docker
Introduction to Containers - SQL Server and DockerChris Taylor
 
Microservices using relocatable Docker containers
Microservices using relocatable Docker containersMicroservices using relocatable Docker containers
Microservices using relocatable Docker containersMauricio Garavaglia
 
Tokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker SecurityTokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker SecurityPhil Estes
 
Docker and the Container Ecosystem
Docker and the Container EcosystemDocker and the Container Ecosystem
Docker and the Container Ecosystempsconnolly
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker IntroductionSparkbit
 
Deep Dive in Docker Overlay Networks - Laurent Bernaille - Architect, D2SI
Deep Dive in Docker Overlay Networks - Laurent Bernaille - Architect, D2SIDeep Dive in Docker Overlay Networks - Laurent Bernaille - Architect, D2SI
Deep Dive in Docker Overlay Networks - Laurent Bernaille - Architect, D2SIDocker, Inc.
 
Troubleshooting Tips from a Docker Support Engineer
Troubleshooting Tips from a Docker Support EngineerTroubleshooting Tips from a Docker Support Engineer
Troubleshooting Tips from a Docker Support EngineerJeff Anderson
 
Continuous Integration using Docker & Jenkins
Continuous Integration using Docker & JenkinsContinuous Integration using Docker & Jenkins
Continuous Integration using Docker & JenkinsB1 Systems GmbH
 
Docker for Devs - John Zaccone, IBM
Docker for Devs - John Zaccone, IBMDocker for Devs - John Zaccone, IBM
Docker for Devs - John Zaccone, IBMDocker, Inc.
 
Docker Swarm Meetup (15min lightning)
Docker Swarm Meetup (15min lightning)Docker Swarm Meetup (15min lightning)
Docker Swarm Meetup (15min lightning)Mike Goelzer
 
Docker, the Future of DevOps
Docker, the Future of DevOpsDocker, the Future of DevOps
Docker, the Future of DevOpsandersjanmyr
 
Docker & JVM: A Perfect Match
Docker & JVM: A Perfect MatchDocker & JVM: A Perfect Match
Docker & JVM: A Perfect MatchMatthias Grüter
 
Introduction to Docker, December 2014 "Tour de France" Edition
Introduction to Docker, December 2014 "Tour de France" EditionIntroduction to Docker, December 2014 "Tour de France" Edition
Introduction to Docker, December 2014 "Tour de France" EditionJérôme Petazzoni
 
Techtalks: taking docker to production
Techtalks: taking docker to productionTechtalks: taking docker to production
Techtalks: taking docker to productionmuayyad alsadi
 
Building Reusable Development Environments with Docker
Building Reusable Development Environments with DockerBuilding Reusable Development Environments with Docker
Building Reusable Development Environments with DockerRevelation Technologies
 
ExpoQA 2017 Using docker to build and test in your laptop and Jenkins
ExpoQA 2017 Using docker to build and test in your laptop and JenkinsExpoQA 2017 Using docker to build and test in your laptop and Jenkins
ExpoQA 2017 Using docker to build and test in your laptop and JenkinsElasTest Project
 
Introduction of Docker and Docker Compose
Introduction of Docker and Docker ComposeIntroduction of Docker and Docker Compose
Introduction of Docker and Docker ComposeDr. Ketan Parmar
 

What's hot (20)

Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
 
Introduction to Containers - SQL Server and Docker
Introduction to Containers - SQL Server and DockerIntroduction to Containers - SQL Server and Docker
Introduction to Containers - SQL Server and Docker
 
Microservices using relocatable Docker containers
Microservices using relocatable Docker containersMicroservices using relocatable Docker containers
Microservices using relocatable Docker containers
 
Tokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker SecurityTokyo OpenStack Summit 2015: Unraveling Docker Security
Tokyo OpenStack Summit 2015: Unraveling Docker Security
 
Docker and the Container Ecosystem
Docker and the Container EcosystemDocker and the Container Ecosystem
Docker and the Container Ecosystem
 
Docker Introduction
Docker IntroductionDocker Introduction
Docker Introduction
 
Deep Dive in Docker Overlay Networks - Laurent Bernaille - Architect, D2SI
Deep Dive in Docker Overlay Networks - Laurent Bernaille - Architect, D2SIDeep Dive in Docker Overlay Networks - Laurent Bernaille - Architect, D2SI
Deep Dive in Docker Overlay Networks - Laurent Bernaille - Architect, D2SI
 
Troubleshooting Tips from a Docker Support Engineer
Troubleshooting Tips from a Docker Support EngineerTroubleshooting Tips from a Docker Support Engineer
Troubleshooting Tips from a Docker Support Engineer
 
Continuous Integration using Docker & Jenkins
Continuous Integration using Docker & JenkinsContinuous Integration using Docker & Jenkins
Continuous Integration using Docker & Jenkins
 
Docker for Devs - John Zaccone, IBM
Docker for Devs - John Zaccone, IBMDocker for Devs - John Zaccone, IBM
Docker for Devs - John Zaccone, IBM
 
What is Docker
What is DockerWhat is Docker
What is Docker
 
Docker Swarm Meetup (15min lightning)
Docker Swarm Meetup (15min lightning)Docker Swarm Meetup (15min lightning)
Docker Swarm Meetup (15min lightning)
 
Docker, the Future of DevOps
Docker, the Future of DevOpsDocker, the Future of DevOps
Docker, the Future of DevOps
 
Docker & ci
Docker & ciDocker & ci
Docker & ci
 
Docker & JVM: A Perfect Match
Docker & JVM: A Perfect MatchDocker & JVM: A Perfect Match
Docker & JVM: A Perfect Match
 
Introduction to Docker, December 2014 "Tour de France" Edition
Introduction to Docker, December 2014 "Tour de France" EditionIntroduction to Docker, December 2014 "Tour de France" Edition
Introduction to Docker, December 2014 "Tour de France" Edition
 
Techtalks: taking docker to production
Techtalks: taking docker to productionTechtalks: taking docker to production
Techtalks: taking docker to production
 
Building Reusable Development Environments with Docker
Building Reusable Development Environments with DockerBuilding Reusable Development Environments with Docker
Building Reusable Development Environments with Docker
 
ExpoQA 2017 Using docker to build and test in your laptop and Jenkins
ExpoQA 2017 Using docker to build and test in your laptop and JenkinsExpoQA 2017 Using docker to build and test in your laptop and Jenkins
ExpoQA 2017 Using docker to build and test in your laptop and Jenkins
 
Introduction of Docker and Docker Compose
Introduction of Docker and Docker ComposeIntroduction of Docker and Docker Compose
Introduction of Docker and Docker Compose
 

Similar to Docker at DevTable

PuppetConf 2017: What’s in the Box?!- Leveraging Puppet Enterprise & Docker- ...
PuppetConf 2017: What’s in the Box?!- Leveraging Puppet Enterprise & Docker- ...PuppetConf 2017: What’s in the Box?!- Leveraging Puppet Enterprise & Docker- ...
PuppetConf 2017: What’s in the Box?!- Leveraging Puppet Enterprise & Docker- ...Puppet
 
Powercoders · Docker · Fall 2021.pptx
Powercoders · Docker · Fall 2021.pptxPowercoders · Docker · Fall 2021.pptx
Powercoders · Docker · Fall 2021.pptxIgnacioTamayo2
 
Dockerizing Aurea - Docker Con EU 2017
Dockerizing Aurea - Docker Con EU 2017Dockerizing Aurea - Docker Con EU 2017
Dockerizing Aurea - Docker Con EU 2017Matias Lespiau
 
Introduction to Docker at the Azure Meet-up in New York
Introduction to Docker at the Azure Meet-up in New YorkIntroduction to Docker at the Azure Meet-up in New York
Introduction to Docker at the Azure Meet-up in New YorkJérôme Petazzoni
 
ContainerDayVietnam2016: Dockerize a small business
ContainerDayVietnam2016: Dockerize a small businessContainerDayVietnam2016: Dockerize a small business
ContainerDayVietnam2016: Dockerize a small businessDocker-Hanoi
 
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...Ambassador Labs
 
Introduction to Docker and Linux Containers @ Cloud Computing Rhein Main
Introduction to Docker and Linux Containers @ Cloud Computing Rhein MainIntroduction to Docker and Linux Containers @ Cloud Computing Rhein Main
Introduction to Docker and Linux Containers @ Cloud Computing Rhein MainPuja Abbassi
 
Docker for developers
Docker for developersDocker for developers
Docker for developersAnvay Patil
 
Detailed Introduction To Docker
Detailed Introduction To DockerDetailed Introduction To Docker
Detailed Introduction To Dockernklmish
 
Domino on docker version 2
Domino on docker version 2Domino on docker version 2
Domino on docker version 2Slobodan Lohja
 
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...Ambassador Labs
 
Docker puebla bday #4 celebration
Docker puebla bday #4 celebrationDocker puebla bday #4 celebration
Docker puebla bday #4 celebrationRamon Morales
 
DCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development PipelineDCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development PipelineDocker, Inc.
 
Docker up and Running For Web Developers
Docker up and Running For Web DevelopersDocker up and Running For Web Developers
Docker up and Running For Web DevelopersBADR
 
Docker Up and Running for Web Developers
Docker Up and Running for Web DevelopersDocker Up and Running for Web Developers
Docker Up and Running for Web DevelopersAmr Fawzy
 
ContainerDays NYC 2015: "Easing Your Way Into Docker: Lessons From a Journey ...
ContainerDays NYC 2015: "Easing Your Way Into Docker: Lessons From a Journey ...ContainerDays NYC 2015: "Easing Your Way Into Docker: Lessons From a Journey ...
ContainerDays NYC 2015: "Easing Your Way Into Docker: Lessons From a Journey ...DynamicInfraDays
 

Similar to Docker at DevTable (20)

PuppetConf 2017: What’s in the Box?!- Leveraging Puppet Enterprise & Docker- ...
PuppetConf 2017: What’s in the Box?!- Leveraging Puppet Enterprise & Docker- ...PuppetConf 2017: What’s in the Box?!- Leveraging Puppet Enterprise & Docker- ...
PuppetConf 2017: What’s in the Box?!- Leveraging Puppet Enterprise & Docker- ...
 
Powercoders · Docker · Fall 2021.pptx
Powercoders · Docker · Fall 2021.pptxPowercoders · Docker · Fall 2021.pptx
Powercoders · Docker · Fall 2021.pptx
 
Dockerizing Aurea - Docker Con EU 2017
Dockerizing Aurea - Docker Con EU 2017Dockerizing Aurea - Docker Con EU 2017
Dockerizing Aurea - Docker Con EU 2017
 
Introduction to Docker at the Azure Meet-up in New York
Introduction to Docker at the Azure Meet-up in New YorkIntroduction to Docker at the Azure Meet-up in New York
Introduction to Docker at the Azure Meet-up in New York
 
ContainerDayVietnam2016: Dockerize a small business
ContainerDayVietnam2016: Dockerize a small businessContainerDayVietnam2016: Dockerize a small business
ContainerDayVietnam2016: Dockerize a small business
 
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
 
Introduction to Docker and Linux Containers @ Cloud Computing Rhein Main
Introduction to Docker and Linux Containers @ Cloud Computing Rhein MainIntroduction to Docker and Linux Containers @ Cloud Computing Rhein Main
Introduction to Docker and Linux Containers @ Cloud Computing Rhein Main
 
Containerization using docker and its applications
Containerization using docker and its applicationsContainerization using docker and its applications
Containerization using docker and its applications
 
Containerization using docker and its applications
Containerization using docker and its applicationsContainerization using docker and its applications
Containerization using docker and its applications
 
Docker for developers
Docker for developersDocker for developers
Docker for developers
 
Detailed Introduction To Docker
Detailed Introduction To DockerDetailed Introduction To Docker
Detailed Introduction To Docker
 
Docker Container Introduction
Docker Container IntroductionDocker Container Introduction
Docker Container Introduction
 
Domino on docker version 2
Domino on docker version 2Domino on docker version 2
Domino on docker version 2
 
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
 
Docker puebla bday #4 celebration
Docker puebla bday #4 celebrationDocker puebla bday #4 celebration
Docker puebla bday #4 celebration
 
DCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development PipelineDCEU 18: Building Your Development Pipeline
DCEU 18: Building Your Development Pipeline
 
Docker up and Running For Web Developers
Docker up and Running For Web DevelopersDocker up and Running For Web Developers
Docker up and Running For Web Developers
 
Docker Up and Running for Web Developers
Docker Up and Running for Web DevelopersDocker Up and Running for Web Developers
Docker Up and Running for Web Developers
 
ContainerDays NYC 2015: "Easing Your Way Into Docker: Lessons From a Journey ...
ContainerDays NYC 2015: "Easing Your Way Into Docker: Lessons From a Journey ...ContainerDays NYC 2015: "Easing Your Way Into Docker: Lessons From a Journey ...
ContainerDays NYC 2015: "Easing Your Way Into Docker: Lessons From a Journey ...
 
Container Days
Container DaysContainer Days
Container Days
 

More from Docker, Inc.

Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience Docker, Inc.
 
How to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker BuildHow to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker BuildDocker, Inc.
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSDocker, Inc.
 
Securing Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINXSecuring Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINXDocker, Inc.
 
How To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and ComposeHow To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and ComposeDocker, Inc.
 
Distributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at SalesforceDistributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at SalesforceDocker, Inc.
 
The First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker HubThe First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker HubDocker, Inc.
 
Monitoring in a Microservices World
Monitoring in a Microservices WorldMonitoring in a Microservices World
Monitoring in a Microservices WorldDocker, Inc.
 
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...Docker, Inc.
 
Predicting Space Weather with Docker
Predicting Space Weather with DockerPredicting Space Weather with Docker
Predicting Space Weather with DockerDocker, Inc.
 
Become a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio CodeBecome a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio CodeDocker, Inc.
 
How to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container RegistryHow to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container RegistryDocker, Inc.
 
Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!Docker, Inc.
 
Kubernetes at Datadog Scale
Kubernetes at Datadog ScaleKubernetes at Datadog Scale
Kubernetes at Datadog ScaleDocker, Inc.
 
Labels, Labels, Labels
Labels, Labels, Labels Labels, Labels, Labels
Labels, Labels, Labels Docker, Inc.
 
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment ModelUsing Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment ModelDocker, Inc.
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSDocker, Inc.
 
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...Docker, Inc.
 
Developing with Docker for the Arm Architecture
Developing with Docker for the Arm ArchitectureDeveloping with Docker for the Arm Architecture
Developing with Docker for the Arm ArchitectureDocker, Inc.
 

More from Docker, Inc. (20)

Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience
 
How to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker BuildHow to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker Build
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
 
Securing Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINXSecuring Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINX
 
How To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and ComposeHow To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and Compose
 
Hands-on Helm
Hands-on Helm Hands-on Helm
Hands-on Helm
 
Distributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at SalesforceDistributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at Salesforce
 
The First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker HubThe First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker Hub
 
Monitoring in a Microservices World
Monitoring in a Microservices WorldMonitoring in a Microservices World
Monitoring in a Microservices World
 
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
 
Predicting Space Weather with Docker
Predicting Space Weather with DockerPredicting Space Weather with Docker
Predicting Space Weather with Docker
 
Become a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio CodeBecome a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio Code
 
How to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container RegistryHow to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container Registry
 
Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!
 
Kubernetes at Datadog Scale
Kubernetes at Datadog ScaleKubernetes at Datadog Scale
Kubernetes at Datadog Scale
 
Labels, Labels, Labels
Labels, Labels, Labels Labels, Labels, Labels
Labels, Labels, Labels
 
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment ModelUsing Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
 
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWSBuild & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
 
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
 
Developing with Docker for the Arm Architecture
Developing with Docker for the Arm ArchitectureDeveloping with Docker for the Arm Architecture
Developing with Docker for the Arm Architecture
 

Docker at DevTable

  • 2. What is DevTable? DevTable is a browser-based, hosted, collaborative IDE Develop in the cloud with the same power as your desktop applications
  • 6. Deploy ● Google App Engine ● REST ● SCP ● Git (Heroku and other providers)
  • 7. Sealed evil in a can There are a lot of neat things that we run for our users, but they are all potentially very dangerous: ● App Engine Development Server ● Debuggers and Emulators ● REPLs (Python, etc) ● Terminal support (which means all of the above as well)
  • 8. Why this is a problem ● Without a containment system of some kind, any of these awesome features would allow users to cause mayhem: ○ A REPL use could open any file ○ A DevServer can execute arbitrary code ○ A terminal could allow anything to happen
  • 9. Why not simply use permissions? ● Permissions solve the file access problem ● Permissions do not prevent users from causing other system issues: instability, exhaustion, escalation, etc
  • 10. Solution: containers! To contain the insecurity of running live code, we run all non-custom code in a container, with only the user’s project mounted and available Evil (not to scale) Project data Container
  • 11. Ideal container properties ● Lightweight ● Secure ● Easy to manage ● FAST
  • 12. Originally we used LXC... ● Lightweight (sort of…) ● Secure ● Easy to manage (sort of...) ● FAST
  • 13. In the beginning, there was LXC... … and it was slow. ● Typical startup times for our containers were on the order of minutes ● Starting a debugger or shell is not fun at those speeds ● Getting the security and management just right was quite painful
  • 14. Then the community said “let there be Docker”... Yo!
  • 15. … and it made things amazing. Our average startup time for a container has dropped from over a minute to just under four seconds. LXC Docker Go make a cup of coffee and play swords on office chairs Go!
  • 17. But, but Docker is just... LXC... Almost, Docker does some things that make starting up single processes lightning quick: ● Incremental by default ● Replace distro init process with lightweight version ● No DHCP, upstart, dnsmasq, etc. ● Aufs seems to be faster than OverlayFS ● Build process is MUCH better (Dockerfiles)
  • 18. Docker at DevTable The fun technical details!
  • 19. DevTable overview Clients Web browsers Clients - Web browsers Frontends Python Clients - Web browsers Backends C# WebSocket Socket DFS Clients - Web browsers Container Servers Python Thrift SSH HTTP ? Images
  • 20. Things we’ll discuss today Clients Web browsers Clients - Web browsers Frontends Python Clients - Web browsers Backends C# WebSocket Socket DFS Clients - Web browsers Container Servers Python Thrift HTTP SSH ? Images
  • 21. How we use docker now ● Python Docker API bindings ● Run a single instance per project ● Mount only the files relevant to the project in the container ● Run an SSH “command and control” process ● Execute user processes through SSH ● Dynamic version of Docker port forwarding
  • 22. Backend <-> Container server Backends C# Container Servers Python Thrift
  • 23. Container server The container server is the server in charge of managing all aspects related to the Docker containers ● Written in Python ● Conforms to a Thrift interface ● Called by the Backends to start containers, stop containers, run commands, mount file systems in containers, etc
  • 24. Container server startContainer Starts a new container for a project. runCommand Runs a command inside a container stopCommand Stops a command inside a container notifyFilesModifed Notifies a container that a file has been modified by the backend stopContainer Stops a container
  • 25. Handling file changes ● Changes made by the container or the backend to the DFS are propagated automatically ● However, both sides have code that depends on notification of changes ● Each server notifies the other about changes that occur via a notification service
  • 26. DFS change notifications Backend C# Container Server Python Hey, a user added file “test.txt” in container 1234 Backend C# Container Server Python Hey, the user changed file “foo.py” in container 1
  • 27. How we handle file changes in Docker ● The container server watches changes inside the container using inotify, and reports changes to the backend ● The backend reports changes to the container server which will touch files that have been added or changed
  • 28. Container server <-> Docker Clients - Web browsers Container Servers Python SSH
  • 29. Container server <-> Docker We use the Python Docker bindings to create a new image and load it with a temporary ssh key New container requests bring up the container with the known session SSH key and issue commands to the container via SSH Much better than LXC issuing commands via subprocess
  • 30. Docker <-> Outside world For many services we run (such as the App Engine Development Server), we need to expose the server running inside Docker to the outside world
  • 31. Docker <-> Outside world HTTP Clients Web browsers HTTP Container Server HAProxy
  • 32. Docker <-> Outside world Services inside of Docker as exposed via dynamic port mapping to a HAProxy running on the container server The HAProxy exposes the port by remapping it to the external port and a custom subdomain
  • 33. Docker <-> Outside world Container Server 93nx83ndsc34mn.c4.devtable.io:80Clients Web browsers Port 38563 HAProxy
  • 34. Example: running a dev server 1. Backend requests a container from the server Backend C# Container Server Python I need a container for project “testapplication” Container “container1234” started for project
  • 35. Example: running a dev server 2. Backend registers for file notification events Backend C# Container Server Python Let me know if any files change Duly noted
  • 36. Example: running a dev server 3. Backend asks for the dev server to be started and port 80 to be forwarded Backend C# Container Server Python Please start the dev server and forward port 80 Dev server started and port is forwarded at subdomain foobarbaz
  • 37. Example: running a dev server 1. Container server tells Docker to start a container Container Server Python create_container, mount_filesystem, forward_port, start_ssh Done. Port exposed: 84639
  • 38. Example: running a dev server 2. Container server tells HAProxy to forward the port returned by docker Container Server Python Forward port 84639 as subdomain foobarbaz HAProxy
  • 39. Example: running a dev server 3. Container server tells Docker to run the dev server Container Server Python ssh command_for_devserver
  • 40. Summary Docker has allowed DevTable to run amazing tools securely and fast, without a large management overhead
  • 41. Future opportunities Docker presents some amazing new opportunities for DevTable and the community: ● Ability to quickly load (and save) complete development environments, securely ● Ability to quickly write custom plugins and run them in our IDE (want to analyze and build Go? just give us a URL or a Dockerfile!)
  • 42. But wait… There’s something that has been bugging us… How should we distribute our private images in production?
  • 43. Quay Demo At this point in the live talk we unveiled and gave a demo of our hosted private docker registry called Quay.io.
  • 44. Questions? Comments? Witty anecdotes? devtable.com Jacob Moshenko - jake@devtable.com Joseph Schorr - jschorr@devtable.com