This document introduces Docker and provides an overview of its key features and benefits. It explains that Docker allows developers to package applications into lightweight containers that can run on any Linux server. Containers deploy instantly and consistently across environments due to their isolation via namespaces and cgroups. The document also summarizes Docker's architecture including storage drivers, images, and the Dockerfile for building images.
Introduction to Docker (and a bit more) at LSPE meetup SunnyvaleJérôme Petazzoni
What's Docker, why does it matter, how does it use Linux Containers, why should you use it, and how? You'll find answers to those questions (and a bit more) in this presentation, given February 20th 2014 at the Large Scale Production Engineering Meet-Up at Yahoo, in Sunnyvale.
Docker and Containers for Development and Deployment — SCALE12XJérôme Petazzoni
Docker is an Open Source engine to build, run, and manage containers. We'll explain what are Linux Containers, what powers them (under the hood), and what extra value Docker brings to the table. Then we'll see what the typical Docker workflow looks like from a developer point of view. We'll also give an Ops perspective, including deployment options. If you already saw a "Docker 101", consider this presentation as the February 2014 update! :-)
While probably the most prominent, Docker is not the only tool for building and managing containers. Originally meant to be a "chroot on steroids" to help debug systemd, systemd-nspawn provides a fairly uncomplicated approach to work with containers. Being part of systemd, it is available on most recent distributions out-of-the-box and requires no additional dependencies.
This deck will introduce a few concepts involved in containers and will guide you through the steps of building a container from scratch. The payload will be a simple service, which will be automatically activated by systemd when the first request arrives.
If you're not familiar with Docker yet, here is your chance to catch up: a quick overview of the Open Source Docker Engine, and its associated services delivered through the Docker Hub. It also includes Jérôme will also discuss the new features of Docker 1.0, and briefly explain how you can run and maintain Docker on Azure. In addition, an Azure team member will demonstrate how deploy docker to Azure. The presentation will be followed by a Q&A session!
Hands on Virtualization with Ganeti (part 1) - LinuxCon 2012Lance Albertson
Ganeti is a robust cluster virtualization management software tool. It’s built on top of existing virtualization technologies such as Xen and KVM and other Open Source software. Its integration with various technologies such as DRBD and LVM results in a cheaper High Availability infrastructure and linear scaling.
This hands-on tutorial will cover a basic overview of Ganeti, the step-by-step install & setup of a single-node and multi-node Ganeti cluster, operating the cluster, and some best practices of Ganeti.
Most Linux distributions now feature systemd at their core. This presentation shows how to leverage it for your own services -- all the way from the most basic, two-line service configuration to advanced resource and security options.
Containerization is more than the new Virtualization: enabling separation of ...Jérôme Petazzoni
Docker offers a new, lightweight approach to application
portability. Applications are shipped using a common container format,
and managed with a high-level API. Their processes run within isolated
namespaces which abstract the operating environment, independently of
the distribution, versions, network setup, and other details of this
environment.
This "containerization" has often been nicknamed "the new
virtualization". But containers are more than lightweight virtual
machines. Beyond their smaller footprint, shorter boot times, and
higher consolidation factors, they also bring a lot of new features
and use cases which were not possible with classical virtual machines.
We will focus on one of those features: separation of operational
concerns. Specifically, we will demonstrate how some fundamental tasks
like logging, remote access, backups, and troubleshooting can be
entirely decoupled from the deployment of applications and
services. This decoupling results in independent, smaller, simpler
moving parts; just like microservice architectures break down large
monolithic apps in more manageable components.
Introduction to Docker (and a bit more) at LSPE meetup SunnyvaleJérôme Petazzoni
What's Docker, why does it matter, how does it use Linux Containers, why should you use it, and how? You'll find answers to those questions (and a bit more) in this presentation, given February 20th 2014 at the Large Scale Production Engineering Meet-Up at Yahoo, in Sunnyvale.
Docker and Containers for Development and Deployment — SCALE12XJérôme Petazzoni
Docker is an Open Source engine to build, run, and manage containers. We'll explain what are Linux Containers, what powers them (under the hood), and what extra value Docker brings to the table. Then we'll see what the typical Docker workflow looks like from a developer point of view. We'll also give an Ops perspective, including deployment options. If you already saw a "Docker 101", consider this presentation as the February 2014 update! :-)
While probably the most prominent, Docker is not the only tool for building and managing containers. Originally meant to be a "chroot on steroids" to help debug systemd, systemd-nspawn provides a fairly uncomplicated approach to work with containers. Being part of systemd, it is available on most recent distributions out-of-the-box and requires no additional dependencies.
This deck will introduce a few concepts involved in containers and will guide you through the steps of building a container from scratch. The payload will be a simple service, which will be automatically activated by systemd when the first request arrives.
If you're not familiar with Docker yet, here is your chance to catch up: a quick overview of the Open Source Docker Engine, and its associated services delivered through the Docker Hub. It also includes Jérôme will also discuss the new features of Docker 1.0, and briefly explain how you can run and maintain Docker on Azure. In addition, an Azure team member will demonstrate how deploy docker to Azure. The presentation will be followed by a Q&A session!
Hands on Virtualization with Ganeti (part 1) - LinuxCon 2012Lance Albertson
Ganeti is a robust cluster virtualization management software tool. It’s built on top of existing virtualization technologies such as Xen and KVM and other Open Source software. Its integration with various technologies such as DRBD and LVM results in a cheaper High Availability infrastructure and linear scaling.
This hands-on tutorial will cover a basic overview of Ganeti, the step-by-step install & setup of a single-node and multi-node Ganeti cluster, operating the cluster, and some best practices of Ganeti.
Most Linux distributions now feature systemd at their core. This presentation shows how to leverage it for your own services -- all the way from the most basic, two-line service configuration to advanced resource and security options.
Containerization is more than the new Virtualization: enabling separation of ...Jérôme Petazzoni
Docker offers a new, lightweight approach to application
portability. Applications are shipped using a common container format,
and managed with a high-level API. Their processes run within isolated
namespaces which abstract the operating environment, independently of
the distribution, versions, network setup, and other details of this
environment.
This "containerization" has often been nicknamed "the new
virtualization". But containers are more than lightweight virtual
machines. Beyond their smaller footprint, shorter boot times, and
higher consolidation factors, they also bring a lot of new features
and use cases which were not possible with classical virtual machines.
We will focus on one of those features: separation of operational
concerns. Specifically, we will demonstrate how some fundamental tasks
like logging, remote access, backups, and troubleshooting can be
entirely decoupled from the deployment of applications and
services. This decoupling results in independent, smaller, simpler
moving parts; just like microservice architectures break down large
monolithic apps in more manageable components.
Introduction to Docker at Glidewell Laboratories in Orange CountyJérôme Petazzoni
In this presentation we will introduce Docker, and how you can use it to build, ship, and run any application, anywhere. The presentation included short demos, links to further material, and of course Q&As. If you are already a seasoned Docker user, this presentation will probably be redundant; but if you started to use Docker and are still struggling with some of his facets, you'll learn some!
This presentation gives overview or our Ganeti deployment
There is related YouTube playlist (in Croatian) with presentations https://www.youtube.com/playlist?list=PLDMnMa3XBHD_K6Rl2FBe2CC-MS6mdTOrJ
Docker storage drivers by Jérôme PetazzoniDocker, Inc.
The first release of Docker only supported AUFS, and AUFS was available (out of the box) only on Debian and Ubuntu kernel. Then Red Hat wanted Docker to run on its distros, and contributed the Device Mapper driver, and later the BTRFS driver, and recently the overlayfs driver.
Jérôme presents how those drivers compare from a high-level perspective, explaining their pros and cons.
Then he showed each driver in action, and look at low-level implementation details. We won't dive into the golang implementation code itself, but we will explain the concepts of each driver. This will help to better understand how they work, and give some hints when it comes to troubleshoot their behaviour.
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxConJérôme Petazzoni
Containers are everywhere. But what exactly is a container? What are they made from? What's the difference between LXC, butts-nspawn, Docker, and the other container systems out there? And why should we bother about specific filesystems?
In this talk, Jérôme will show the individual roles and behaviors of the components making up a container: namespaces, control groups, and copy-on-write systems. Then, he will use them to assemble a container from scratch, and highlight the differences (and likelinesses) with existing container systems.
Ganeti is a robust cluster virtualization management software tool. It’s built on top of existing virtualization technologies such as Xen and KVM and other Open Source software. Its integration with various technologies such as DRBD and LVM results in a cheaper High Availability infrastructure and linear scaling.
This hands-on tutorial will cover a basic overview of Ganeti, the step-by-step install & setup of a single-node and multi-node Ganeti cluster, operating the cluster, and some best practices of Ganeti.
If attendees want to participate in the optional hands-on portions of the tutorial, there will be virtual machine images available online and at the tutorial itself. We’ll be using VirtualBox and Vagrant to deploy Ganeti on two to three Ubuntu nodes.
This document explains how to use docker container in Ubuntu 14.04 VM for trying out cgroups without actually using the host/guest operating system. It talks about using 'cpu' subcomponent and demostrates the effect of process isolation by 'htop' utility.
The docker containers are very effective way of trying out things by launching a container using standard/custom docker image from docker hub or your own image repository.
Kernel Recipes 2015 - Porting Linux to a new processor architectureAnne Nicolas
Getting the Linux kernel running on a new processor architecture is a difficult process. Worse still, there is not much documentation available describing the porting process.
After spending countless hours becoming almost fluent in many of the supported architectures, I discovered that a well-defined skeleton shared by the majority of ports exists. Such a skeleton can logically be split into two parts that intersect a great deal.
The first part is the boot code, meaning the architecture-specific code that is executed from the moment the kernel takes over from the bootloader until init is finally executed. The second part concerns the architecture-specific code that is regularly executed once the booting phase has been completed and the kernel is running normally. This second part includes starting new threads, dealing with hardware interrupts or software exceptions, copying data from/to user applications, serving system calls, and so on.
In this talk I will provide an overview of the procedure, or at least one possible procedure, that can be followed when porting the Linux kernel to a new processor architecture.
Joël Porquet – Joël was a post-doc at Pierre and Marie Curie University (UPMC) where he ported Linux to TSAR, an academic processor. He is now looking for new adventures.
Introduction to Docker at Glidewell Laboratories in Orange CountyJérôme Petazzoni
In this presentation we will introduce Docker, and how you can use it to build, ship, and run any application, anywhere. The presentation included short demos, links to further material, and of course Q&As. If you are already a seasoned Docker user, this presentation will probably be redundant; but if you started to use Docker and are still struggling with some of his facets, you'll learn some!
This presentation gives overview or our Ganeti deployment
There is related YouTube playlist (in Croatian) with presentations https://www.youtube.com/playlist?list=PLDMnMa3XBHD_K6Rl2FBe2CC-MS6mdTOrJ
Docker storage drivers by Jérôme PetazzoniDocker, Inc.
The first release of Docker only supported AUFS, and AUFS was available (out of the box) only on Debian and Ubuntu kernel. Then Red Hat wanted Docker to run on its distros, and contributed the Device Mapper driver, and later the BTRFS driver, and recently the overlayfs driver.
Jérôme presents how those drivers compare from a high-level perspective, explaining their pros and cons.
Then he showed each driver in action, and look at low-level implementation details. We won't dive into the golang implementation code itself, but we will explain the concepts of each driver. This will help to better understand how they work, and give some hints when it comes to troubleshoot their behaviour.
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxConJérôme Petazzoni
Containers are everywhere. But what exactly is a container? What are they made from? What's the difference between LXC, butts-nspawn, Docker, and the other container systems out there? And why should we bother about specific filesystems?
In this talk, Jérôme will show the individual roles and behaviors of the components making up a container: namespaces, control groups, and copy-on-write systems. Then, he will use them to assemble a container from scratch, and highlight the differences (and likelinesses) with existing container systems.
Ganeti is a robust cluster virtualization management software tool. It’s built on top of existing virtualization technologies such as Xen and KVM and other Open Source software. Its integration with various technologies such as DRBD and LVM results in a cheaper High Availability infrastructure and linear scaling.
This hands-on tutorial will cover a basic overview of Ganeti, the step-by-step install & setup of a single-node and multi-node Ganeti cluster, operating the cluster, and some best practices of Ganeti.
If attendees want to participate in the optional hands-on portions of the tutorial, there will be virtual machine images available online and at the tutorial itself. We’ll be using VirtualBox and Vagrant to deploy Ganeti on two to three Ubuntu nodes.
This document explains how to use docker container in Ubuntu 14.04 VM for trying out cgroups without actually using the host/guest operating system. It talks about using 'cpu' subcomponent and demostrates the effect of process isolation by 'htop' utility.
The docker containers are very effective way of trying out things by launching a container using standard/custom docker image from docker hub or your own image repository.
Kernel Recipes 2015 - Porting Linux to a new processor architectureAnne Nicolas
Getting the Linux kernel running on a new processor architecture is a difficult process. Worse still, there is not much documentation available describing the porting process.
After spending countless hours becoming almost fluent in many of the supported architectures, I discovered that a well-defined skeleton shared by the majority of ports exists. Such a skeleton can logically be split into two parts that intersect a great deal.
The first part is the boot code, meaning the architecture-specific code that is executed from the moment the kernel takes over from the bootloader until init is finally executed. The second part concerns the architecture-specific code that is regularly executed once the booting phase has been completed and the kernel is running normally. This second part includes starting new threads, dealing with hardware interrupts or software exceptions, copying data from/to user applications, serving system calls, and so on.
In this talk I will provide an overview of the procedure, or at least one possible procedure, that can be followed when porting the Linux kernel to a new processor architecture.
Joël Porquet – Joël was a post-doc at Pierre and Marie Curie University (UPMC) where he ported Linux to TSAR, an academic processor. He is now looking for new adventures.
Starr Long: Video Game History & MethodsStarr Long
This is a lecture I gave at the University of Texas about my history in the video game business. Includes a deepdive into my current project Shroud of the Avatar, currently the second highest crowdfunded video game of all time.
medical malpractice insurance, medical malpractice insurance new york, malpractice insurance ny, new york state medical malpractice, medical malpractice in new york, medical malpractice new york city, medical malpractice nyc. For More Information, Visit: http://www.malpracticenys.com
Πως μπορούν να αντιμετωπιστούν τα προβλήματα που προκύπτουν από τη Ψηφιακή Μετάβαση για τους Επαγγελματίες, μεγάλων δικτύων, Ξενοδοχείων & Νοσοκομείων;
IPTV και πρόσθετες υπηρεσίες φιλοξενίας ξενοδοχείων.
Τι είναι και πως χρησιμεύουν?
Moving Beyond Mobile: Delivering a Seamless Digital Experience from Online to...Mozu
Learn how to deliver a seamless digital experience with this presentation from Jason Wallis, CTO of Mozu, featuring Adam Silverman, Principal Analyst at Forrester Research.
Watch the webinar replay here: http://info.mozu.com/ecommerce-responsive-design-considerations-webinar.html
Earlier this week we saw economic data that pointed towards a recovery, and that coupled with a strong dollar kept gold under pressure. To take a look for weekly updates in market is necessary, theequicom provides genuine information and important updates through this newsletter.
Introduction to Docker (as presented at December 2013 Global Hackathon)Jérôme Petazzoni
Not on board of the Docker ship yet? This presentation will get you up to speed, and explain everything you want to know about Linux Containers and Docker, including the new features of the latest 0.7 version (which brings support for all Linux distros and kernels).
Introduction to Docker at SF Peninsula Software Development Meetup @GuidewiredotCloud
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
Introduction to Docker, December 2014 "Tour de France" Bordeaux Special EditionJérôme Petazzoni
Docker, the Open Source container Engine, lets you build, ship and run, any app, anywhere.
This is the presentation which was shown in December 2014 for the last stop of the "Tour de France" in Bordeaux. It is slightly different from the presentation which was shown in the other cities (http://www.slideshare.net/jpetazzo/introduction-to-docker-december-2014-tour-de-france-edition), and includes a detailed history of dotCloud and Docker and a few other differences.
Special thanks to https://twitter.com/LilliJane and https://twitter.com/zirkome, who gave me the necessary motivation to put together this slightly different presentation, since they had already seen the other presentation in Paris :-)
Introduction to Docker, December 2014 "Tour de France" EditionJérôme Petazzoni
Docker, the Open Source container Engine, lets you build, ship and run, any app, anywhere.
This is the presentation which was shown in December 2014 for the "Tour de France" in Paris, Lille, Lyon, Nice...
Docker is the Open Source container engine. It lets you author, run, and manage software containers. Escape from dependency hell, and make deployment a breeze! This presentation includes the standard Docker intro (actualized for Docker 0.11) as well as some insights about how to perform orchestration and multi-host container linking.
Internal presentation of Docker, Lightweight Virtualization, and linux Containers; at Spotify NYC offices, featuring engineers from Yandex, LinkedIn, Criteo, and NASA!
"Lightweight Virtualization with Linux Containers and Docker". Jerome Petazzo...Yandex
Lightweight virtualization", also called "OS-level virtualization", is not new. On Linux it evolved from VServer to OpenVZ, and, more recently, to Linux Containers (LXC). It is not Linux-specific; on FreeBSD it's called "Jails", while on Solaris it’s "Zones". Some of those have been available for a decade and are widely used to provide VPS (Virtual Private Servers), cheaper alternatives to virtual machines or physical servers. But containers have other purposes and are increasingly popular as the core components of public and private Platform-as-a-Service (PAAS), among others.
Just like a virtual machine, a Linux Container can run (almost) anywhere. But containers have many advantages over VMs: they are lightweight and easier to manage. After operating a large-scale PAAS for a few years, dotCloud realized that with those advantages, containers could become the perfect format for software delivery, since that is how dotCloud delivers from their build system to their hosts. To make it happen everywhere, dotCloud open-sourced Docker, the next generation of the containers engine powering its PAAS. Docker has been extremely successful so far, being adopted by many projects in various fields: PAAS, of course, but also continuous integration, testing, and more.
A Gentle Introduction To Docker And All Things ContainersJérôme Petazzoni
Docker is a runtime for Linux Containers. It enables "separation of concern" between devs and ops, and solves the "matrix from hell" of software deployment. This presentation explains it all! It also explains the role of the storage backend and compares the various backends available. It gives multiple recipes to build Docker images, including integration with configuration management software like Chef, Puppet, Salt, Ansible. If you already watched other Docker presentations, this is an actualized version (as of mid-November 2013) of the thing!
Introduction to Docker and all things containers, Docker Meetup at RelateIQdotCloud
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
Docker 1 0 1 0 1: a Docker introduction, actualized for the stable release of...Jérôme Petazzoni
If you're not familiar yet with Docker, here is your chance to catch up. This presentation includes a quick overview of the Open Source Docker Engine, and its associated services delivered through the Docker Hub. Recent features are listed, as well as a glimpse at what's next in the Docker world.
This presentation was given during OSCON, at a meet-up hosted by New Relic, with co-presentations from CoreOS and Rackspace OnMetal.
Docker Intro at the Google Developer Group and Google Cloud Platform Meet UpJérôme Petazzoni
Docker is the Open Source engine to author, run, and manage Linux Containers. This is a short introduction to Docker, what it is, what is for; it was given in the context of the Google Developer Group and Google Cloud Platform Meet-Up in San Francisco, end of March 2014.
Introduction to Docker at the Azure Meet-up in New YorkJérôme Petazzoni
This is the presentation given at the Azure New York Meet-Up group, September 3rd.
It includes a quick overview of the Open Source Docker Engine and its associated services delivered through the Docker Hub. It also covers the new features of Docker 1.0, and briefly explains how to get started with Docker on Azure.
Use the Source or Join the Dark Side: differences between Docker Community an...Jérôme Petazzoni
The Docker Project delivers a complete open source platform to “build, ship, and run” any application, anywhere, using containers. The Docker Engine and the other main components (Compose, Machine,
and the SwarmKit orchestration system) are free; but Docker Inc. (the company who started the Docker Project) also has a complete commercial offering named “Docker EE” (for Enterprise Edition) that adds an extra set of features geared at larger organizations, as well as an extended support and release cycle.
In this talk, I will explain (and show with demos) what you can do using exclusively Docker CE (community, free edition) and which features are added by Docker EE. This talk is for you if you are in the process of selecting a container platform; or if you’re just curious, and want to know exactly what you can do (and cannot do) with Docker CE and EE.
Orchestration, resource scheduling…What does that mean? Is this only relevant for data centers with thousands of nodes? Should I care about Mesos, Kubernetes, Swarm, when all I have is a handful of virtual machines? The motto of public cloud IAAS is "pay for what you use," so in theory, if I deploy my apps there, I'm already getting the best "resource utilization" aka "bang for my buck," right? In this talk, we will answer those questions, and a few more. We will define orchestration, scheduling, and others, and show what it's like to use a scheduler to run containerized applications there.
Cgroups, namespaces, and beyond: what are containers made from? (DockerCon Eu...Jérôme Petazzoni
Linux containers are different from Solaris Zones or BSD Jails: they use discrete kernel features like cgroups, namespaces, SELinux, and more. We will describe those mechanisms in depth, as well as demo how to put them together to produce a container. We will also highlight how different container runtimes compare to each other.
This talk was delivered at DockerCon Europe 2015 in Barcelona.
Docker : quels enjeux pour le stockage et réseau ? Paris Open Source Summit ...Jérôme Petazzoni
Présentation donnée le 18 novembre 2015 au Paris Open Source Summit par Hervé Leclerc (Alterway) et Jérôme Petazzoni (Docker), présentant entre autres les nouvelles fonctionalités de Docker pour le stockage et le réseau arrivées dans la version 1.9 du Docker Engine.
Making DevOps Secure with Docker on Solaris (Oracle Open World, with Jesse Bu...Jérôme Petazzoni
Docker, the container Engine and Platform, is coming to Oracle Solaris! This is the talk that Jérôme Petazzoni (Docker) and Jesse Butler (Oracle) gave at Oracle Open World in November 2015.
Containers, docker, and security: state of the union (Bay Area Infracoders Me...Jérôme Petazzoni
Docker is two years old. While security has always been at the core of the questions revolving around Docker, the nature of those questions has changed. Last year, the main concern was "can I safely colocate containers on the same machine?" and it elicited various responses. Dan Walsh, SELinux expert, notoriously said: "containers do not contain!", and at last year's LinuxCon, Jérôme delivered a presentation detailing how to harden Docker and containers to isolate them better. Today, people have new concerns. They include image transport, vulnerability mitigation, and more.
After a recap about the current state of container security, Jérôme will explain why those new questions showed up, and most importantly, how to address them and safely deploy containers in general, and Docker in particular.
From development environments to production deployments with Docker, Compose,...Jérôme Petazzoni
In this session, we will learn how to define and run multi-container applications with Docker Compose. Then, we will show how to deploy and scale them seamlessly to a cluster with Docker Swarm; and how Amazon EC2 Container Service (ECS) eliminates the need to install,operate, and scale your own cluster management infrastructure. We will also walk through some best practice patterns used by customers for running their microservices platforms or batch jobs. Sample code and Compose templates will be provided on GitHub afterwards.
How to contribute to large open source projects like Docker (LinuxCon 2015)Jérôme Petazzoni
Contributing to a large open source project can seem daunting at first; but fear not! You too can join thousands of successful contributors. First, you don't have to be an expert in Golang, Python, or C, to contribute to Docker, OpenStack, or the Linux Kernel. Many projects also need help with documentation, translation, testing, triaging issues, and more. Very often, just going through bug reports to reproduce them and confirm "this also happens on my setup, with version XYZ" is extremely helpful.
If you decide to take the leap and propose a change (be it code or documentation), each open source project has different contribution guidelines and workflows.
In this talk, Arnaud and Jérôme will explain some of those workflows, how maintainers review your patches, and highlight the details that make your changes more likely to be merged into the project.
Containers, Docker, and Security: State Of The Union (LinuxCon and ContainerC...Jérôme Petazzoni
Docker is two years old. While security has always been at the core of the questions revolving around Docker, the nature of those questions has changed. Last year, the main concern was "can I safely colocate containers on the same machine?" and it elicited various responses. Dan Walsh, SELinux expert, notoriously said: "containers do not contain!", and at last year's LinuxCon, Jérôme delivered a presentation detailing how to harden Docker and containers to isolate them better.
Today, people have new concerns. They include image transport, vulnerability mitigation, and more.
After a recap about the current state of container security, Jérôme will explain why those new questions showed up, and most importantly, how to address them and safely deploy containers in general, and Docker in particular.
Deploy microservices in containers with Docker and friends - KCDC2015Jérôme Petazzoni
Docker lets us build, ship, and run any Linux application, on any platform. It found many early adopters in the CI/CD industry, long before it reached the symbolic 1.0 milestone and was considered "production-ready." Since then, its stability and features attracted enterprise users in many different fields, including very demanding ones like finance, banking, or intelligence agencies.
We will see how Docker is particularly suited to the deployment of distributed applications, and why it is an ideal platform for microservice architectures. In particular, we will look into three Docker related projects that have been announced at DockerCon Europe last December: Machine, Swarm, and Compose, and we will explain how they improve the way we build, deploy, and scale distributed applications.
Containers: from development to production at DevNation 2015Jérôme Petazzoni
In Docker, applications are shipped using a lightweight format, managed with a high-level API, and run within software containers which abstract the host environment. Operating details like distributions, versions, and network setup no longer matter to the application developer.
Thanks to this abstraction level, we can use the same container across all steps of the life cycle of an application, from development to production. This eliminates problems stemming from discrepancies between those environments.
Even so, these environments will always have different requirements. If our quality assurance (QA) and production systems use different logging systems, how can we still ship the same container to both? How can we satisfy the backup and security requirements of our production stack without bloating our development stack?
In this sess, you will learn about the unique features in containers that allow you to cleanly decouple system administrator tasks from the core of your application. We’ll show you how this decoupling results in smaller, simpler containers, and gives you more flexibility when building, managing, and evolving your application stacks.
Immutable infrastructure with Docker and containers (GlueCon 2015)Jérôme Petazzoni
"Never upgrade a server again. Never update your code. Instead, create new servers, and throw away the old ones!"
That's the idea of immutable servers, or immutable infrastructure. This makes many things easier: rollbacks (you can always bring back the old servers), A/B testing (put old and new servers side by side), security (use the latest and safest base system at each deploy), and more.
However, throwing in a bunch of new servers at each one-line CSS change is going to be complicated, not to mention costly.
Containers to the rescue! Creating container "golden images" is easy, fast, dare I say painless. Replacing your old containers with new ones is also easy to do; much easier than virtual machines, let alone physical ones.
In this talk, we'll quickly recap the pros (and cons) of immutable servers; then explain how to implement that pattern with containers. We will use Docker as an example, but the technique can easily be adapted to Rocket or even plain LXC containers.
The Docker ecosystem and the future of application deploymentJérôme Petazzoni
Ten years ago, virtualization ignited a revolution which gave birth to the Cloud and the DevOps initiative. Today, with containers, we are at the dawn of a similar breakthrough.
How can we capture the value of containers? How can we use their features to implement microservices and immutable infrastructures, while retaining as much as possible of our existing practices? The answer is in the rich ecosystem that developed around Docker, an open-source platform to build, ship, and run applications in containers.
In this keynote we’ll explore what the applications of tomorrow will look like, how they’ll be deployed and distributed – and how to leverage those tools today.
Docker landed almost two years ago, making it possible to build, ship, and run
any Linux application, on any platform, it was quickly adopted by developers
and ops, like no other tool before. The CI/CD industry even took it to
production long before it was stamped "production-ready."
Why does everyone (or almost!) love Docker? Because it puts powerful
automation abilities within the hands of normal developers. Automation
almost always involves building distribution packages, virtual machine
images, or writing configuration management manifests. With Docker,
those tasks are radically transformed: sometimes they're far easier than before,
other times they're no longer needed at all. Either way, the intervention
of a seasoned sysadmin guru is no longer required.
Containers, Docker, and Microservices: the Terrific TrioJérôme Petazzoni
One of the upsides of Microservices is the ability to deploy often,at arbitrary schedules, and independently of other services, instead of requiring synchronized deployments happening on a fixed time.
But to really leverage this advantage, we need fast, efficient, and reliable deployment processes. That's one of the value propositions of Containers in general, and Docker in particular.
Docker offers a new, lightweight approach to application portability.It can build applications using easy-to-write, repeatable, efficient recipes; then it can ship them across environments using a common container format; and it can run them within isolated namespaces which abstract the operating environment, independently of the distribution,versions, network setup, and other details of this environment.
But Docker can do way more than deploy your apps. Docker also enables you to generalize Microservices principles and apply them on operational tasks like logging, remote access, backups, and troubleshooting.This decoupling results in independent, smaller, simpler moving parts.
Pipework: Software-Defined Network for Containers and DockerJérôme Petazzoni
Pipework lets you connect together containers in arbitrarily complex scenarios. Pipework uses cgroups and namespaces and works with "plain" LXC containers (created with lxc-start), and with the awesome Docker.
It's nothing less than Software-Defined Networking for Linux Containers!
This is a short presentation about Pipework, given at the Docker Networking meet-up November 6th in Mountain View.
More information:
- https://github.com/jpetazzo/pipework
- http://www.meetup.com/Docker-Networking/
Docker Tips And Tricks at the Docker Beijing MeetupJérôme Petazzoni
This talk was presented in October at the Docker Beijing Meetup, in the VMware offices.
It presents some of the latest features of Docker, discusses orchestration possibilities with Docker, then gives a briefing about the performance of containers; and finally shows how to use volumes to decouple components in your applications.
Dans cette présentation, nous donnons plusieurs réponses aux grandes questions classiques associées au déploiement avec Docker. Entre autres :
- comment orchestrer des déploiements non triviaux (plusieurs containers sur plusieurs machines) ?
- comment avoir des métriques sur les ressources utilisées par les containers ?
- comment optimiser les performances de Docker, en particulier pour des applications où ces performances sont critiques ?
- comment intégrer Docker avec des outils de "configuration management" comme Puppet, Chef, Salt, Ansible ?
- comment implémenter la "service discovery", ou, de manière générale, connecter entre eux plusieurs containers ?
La présentation a été donnée Lundi 8 Septembre à Paris dans les locaux de Zenika, cabinet spécialisé dans l'architecture informatique et les méthodes Agiles possédant une triple compétence de conseil, réalisation et formation.
3. @jpetazzo
●
Wrote dotCloud PAAS deployment tools
– EC2, LXC, Puppet, Python, Shell, ØMQ...
●
Docker contributor
– Docker-in-Docker, VPN-in-Docker,
router-in-Docker... CONTAINERIZE ALL THE THINGS!
●
Runs Docker in production
– You shouldn't do it, but here's how anyway!
10. Deploy reliably & consistently
● If it works locally, it will work on the server
● With exactly the same behavior
● Regardless of versions
● Regardless of distros
● Regardless of dependencies
11. Deploy efficiently
● Containers are lightweight
– Typical laptop runs 10-100 containers easily
– Typical server can run 100-1000 containers
● Containers can run at native speeds
– Lies, damn lies, and other benchmarks:
http://qiita.com/syoyo/items/bea48de8d7c6d8c73435
13. Is there really
no overhead at all?
● Processes are isolated,
but run straight on the host
● CPU performance
= native performance
● Memory performance
= a few % shaved off for (optional) accounting
● Network performance
= small overhead; can be reduced to zero
15. High level approach:
it's a lightweight VM
● Own process space
● Own network interface
● Can run stuff as root
● Can have its own /sbin/init
(different from the host)
« Machine Container »
16. Low level approach:
it's chroot on steroids
● Can also not have its own /sbin/init
● Container = isolated process(es)
● Share kernel with host
● No device emulation (neither HVM nor PV)
« Application Container »
17. How does it work?
Isolation with namespaces
● pid
● mnt
● net
● uts
● ipc
● user
18. pid namespace
jpetazzo@tarrasque:~$ ps aux | wc -l
212
jpetazzo@tarrasque:~$ sudo docker run -t -i ubuntu bash
root@ea319b8ac416:/# ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 18044 1956 ? S 02:54 0:00 bash
root 16 0.0 0.0 15276 1136 ? R+ 02:55 0:00 ps aux
(That's 2 processes)
23. user namespace
● No demo, but see LXC 1.0 (just released)
● UID 0→1999 in container C1 is mapped to
UID 10000→11999 in host;
UID 0→1999 in container C2 is mapped to
UID 12000→13999 in host; etc.
● what will happen with copy-on-write?
– double translation at VFS?
– single root UID on read-only FS?
24. How does it work?
Isolation with cgroups
● memory
● cpu
● blkio
● devices
25. memory cgroup
● Keeps track pages used by each group:
– file (read/write/mmap from block devices; swap)
– anonymous (stack, heap, anonymous mmap)
– active (recently accessed)
– inactive (candidate for eviction)
● Each page is « charged » to a group
● Pages can be shared (e.g. if you use any COW FS)
● Individual (per-cgroup) limits and out-of-memory killer
26. cpu and cpuset cgroups
● Keep track of user/system CPU time
● Set relative weight per group
● Pin groups to specific CPU(s)
– Can be used to « reserve » CPUs for some apps
– This is also relevant for big NUMA systems
27. blkio cgroups
● Keep track IOs for each block device
– read vs write; sync vs async
● Set relative weights
● Set throttle (limits) for each block device
– read vs write; bytes/sec vs operations/sec
Note: earlier versions (<3.8) didn't account async correctly.
3.8 is better, but use 3.10 for best results.
28. devices cgroups
● Controls read/write/mknod permissions
● Typically:
– allow: /dev/{tty,zero,random,null}...
– deny: everything else
– maybe: /dev/net/tun, /dev/fuse, /dev/kvm, /dev/dri...
● Fine-grained control for GPU, virtualization, etc.
29. How does it work?
Copy-on-write storage
● Create a new machine instantly
(Instead of copying its whole filesystem)
● Storage keeps track of what has changed
● Since 0.7, Docker has a storage plugin system
30. Storage:
many options!
Union
Filesystems
Snapshotting
Filesystems
Copy-on-write
block devices
Provisioning Superfast
Supercheap
Fast
Cheap
Fast
Cheap
Changing
small files
Superfast
Supercheap
Fast
Cheap
Fast
Costly
Changing
large files
Slow (first time)
Inefficient (copy-up!)
Fast
Cheap
Fast
Cheap
Diffing Superfast Superfast Slow
Memory usage Efficient Efficient Inefficient
(at high densities)
Drawbacks Random quirks
AUFS not mainline
!AUFS more quirks
ZFS not mainline
BTRFS not as nice
Higher disk usage
Great performance
(except diffing)
Bottom line Ideal for PAAS and
high density things
This is the Future
(probably)
Dodge Ram 3500
45. Docker-what?
The Big Picture
● Open Source engine to commoditize LXC
● Using copy-on-write for quick provisioning
● Allowing to create and share images
● Standard format for containers
(stack of layers; 1 layer = tarball+metadata)
● Standard, reproducible way to easily build
trusted images (Dockerfile, Stackbrew...)
46. Docker-what?
History
● Rewrite of dotCloud internal container engine
– original version: Python, tied to dotCloud PaaS
– released version: Go, legacy-free
47. Docker-what?
Under the hood
● The Docker daemon runs in the background
– manages containers, images, and builds
– HTTP API (over UNIX or TCP socket)
– embedded CLI talking to the API
48. Docker-what?
Take me to your dealer
● Open Source
– GitHub public repository + issue tracking
https://github.com/dotcloud/docker
● Nothing up the sleeve
– public mailing lists (docker-user, docker-dev)
– IRC channels (Freenode: #docker #docker-dev)
– public decision process
49. Docker-what?
The ecosystem
● Docker Inc. (formerly dotCloud Inc.)
– ~30 employees, VC-backed
– SAAS and support offering around Docker
● Docker, the community
– more than 360 contributors, 1600 forks on GitHub
– dozens of projects around/on top of Docker
– x100k trained developers
50. One-time setup
● On your servers (Linux)
– Packages (Ubuntu, Debian, Fedora, Gentoo, Arch...)
– Single binary install (Golang FTW!)
– Easy provisioning on Rackspace, Digital Ocean, EC2, GCE...
● On your dev env (Linux, OS X, Windows)
– Vagrantfile
– boot2docker (25 MB VM image)
– Natively (if you run Linux)
51. The Docker workflow 1/2
● Work in dev environment
(local machine or container)
● Other services (databases etc.) in containers
(and behave just like the real thing!)
● Whenever you want to test « for real »:
– Build in seconds
– Run instantly
52. The Docker workflow 2/2
Satisfied with your local build?
● Push it to a registry (public or private)
● Run it (automatically!) in CI/CD
● Run it in production
● Happiness!
Something goes wrong? Rollback painlessly!
54. 1) docker run ubuntu bash
2) apt-get install this and that
3) docker commit <containerid> <imagename>
4) docker run <imagename> bash
5) git clone git://.../mycode
6) pip install -r requirements.txt
7) docker commit <containerid> <imagename>
8) repeat steps 4-7 as necessary
9) docker tag <imagename> <user/image>
10) docker push <user/image>
55.
56. Authoring images
with run/commit
● Pros
– Convenient, nothing to learn
– Can roll back/forward if needed
● Cons
– Manual process
– Iterative changes stack up
– Full rebuilds are boring, error-prone
58. FROM ubuntu
RUN apt-get -y update
RUN apt-get install -y g++
RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe ...
RUN apt-get install -y libmozjs185-dev libicu-dev libtool ...
RUN apt-get install -y make wget
RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf-
RUN cd /tmp/apache-couchdb-* && ./configure && make install
RUN printf "[httpd]nport = 8101nbind_address = 0.0.0.0" >
/usr/local/etc/couchdb/local.d/docker.ini
EXPOSE 8101
CMD ["/usr/local/bin/couchdb"]
docker build -t jpetazzo/couchdb .
59.
60. Authoring images
with a Dockerfile
● Minimal learning curve
● Rebuilds are easy
● Caching system makes rebuilds faster
● Single file to define the whole environment!
66. Stability and performance
● Many, many, many bugfixes
● Performance improvements
– When Docker starts
– When creating/destroying containers
(Especially en masse)
– Better memory footprint
67. ADD caching
● ADD no longer breaks caching
● You can now use the following pattern:
ADD requirements.txt /src/requirements.txt
RUN pip install r requirements.txt
ADD . /src
68. New ONBUILD instruction
● Register « triggers » to be executed later,
when building an image on top of this one
● Triggers are executed in downstream context
Example:
RUN aptget install buildessential
ONBUILD ADD . /src
ONBUILD RUN cd /src; ./configure; make install
70. Storage:
many options!
Union
Filesystems
Snapshotting
Filesystems
Copy-on-write
block devices
Provisioning Superfast
Supercheap
Fast
Cheap
Fast
Cheap
Changing
small files
Superfast
Supercheap
Fast
Cheap
Fast
Costly
Changing
large files
Slow (first time)
Inefficient (copy-up!)
Fast
Cheap
Fast
Cheap
Diffing Superfast Superfast Slow
Memory usage Efficient Efficient Inefficient
(at high densities)
Drawbacks Random quirks
AUFS not mainline
!AUFS more quirks
ZFS not mainline
BTRFS not as nice
Higher disk usage
Great performance
(except diffing)
Bottom line Ideal for PAAS and
high density things
This is the Future
(probably)
Dodge Ram 3500
71. BTRFS storage driver
● Available on « normal » kernels
● Supposedly faster than DM (but YMMV)
● Doesn't use BTRFS delta stream yet
(i.e. commit/diff is not optimized yet)
● Use it, abuse it, break it!
72. Socket activation
● Plays nice with systemd
● Avoids race condition in boot scripts
Docker doesn't handle API requests when it's
starting, so without socket activation, you need
e.g. reconnection logic.
73. OS X support
● The CLI now runs on OS X
● boot2docker is awesome
79. Coming Soon
● Network acceleration
● Container-specific metrics
● Plugins (e.g. for logging)
● Orchestration hooks
Those things are already possible,
but will soon be part of the core.