Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The DIY Punk Rock
DevOps Playbook
CTO
bryan@joyent.com
Bryan Cantrill
@bcantrill
1990s/2000s: Architectural big album rock
• The late 1990s saw the rise of three-tier architectures consisting
of presenta...
Post-AJAX punks
• Why should HTTP be restricted to the web?
• Why should REST be restricted to web apps?
• Why should Java...
Proto-punk: The Unix philosophy
• The Unix philosophy, as articulated by Doug McIlroy:
• Write programs that do one thing ...
Punk architecture: Microservices
• Microservices do one thing, and strive to do it well
• Replace a small number of monoli...
Punks in prod: Deploying microservices
• Microservices are tautologically small
• One physical machine per service is clea...
Hardware-level virtualization?
• The historical answer to virtualization — since the 1960s — has
been to virtualize the ha...
Platform-level virtualization?
• Virtualizing at the application platform layer addresses the
tenancy challenges of hardwa...
OS-level virtualization!
• Virtualizing at the operating system hits a sweet spot:
• A single operating system (i.e. a sin...
OS-level virtualization at Joyent
• Joyent runs OS containers in the cloud via SmartOS — and we
have run containers in mul...
Containers as PaaS foundation?
• Some saw the power of OS containers to facilitate up-stack
platform-as-a-service abstract...
...and Docker was born
Docker revolution
• Docker has used the rapid provisioning + shared underlying
filesystem of containers to allow developers...
Broader container revolution
• The Docker model has pointed to the future of containers
• Docker’s challenges today are la...
The future of containers in production?
• Can we achieve an ideal world that combines the development
model of Docker with...
Upcoming SlideShare
Loading in …5
×

The DIY Punk Rock DevOps Playbook

3,220 views

Published on

My (very brief!) presentation at Interzone.io on March 11, 2015. A more in depth exploration of these ideas can be found at http://www.slideshare.net/bcantrill/docker-and-the-future-of-containers-in-production video: https://www.joyent.com/developers/videos/docker-and-the-future-of-containers-in-production

Published in: Software
  • Be the first to comment

The DIY Punk Rock DevOps Playbook

  1. 1. The DIY Punk Rock DevOps Playbook CTO bryan@joyent.com Bryan Cantrill @bcantrill
  2. 2. 1990s/2000s: Architectural big album rock • The late 1990s saw the rise of three-tier architectures consisting of presentation, application logic and data tiers • Many names for roughly the same notion: “Service-oriented architecture”, “Model/View/Controller”, etc. • The AJAX+REST revolution of the mid-2000s gave rise to true web applications in which application logic could live on the edge • Led to a punk uprising...
  3. 3. Post-AJAX punks • Why should HTTP be restricted to the web? • Why should REST be restricted to web apps? • Why should JavaScript be restricted to the client? • Instead of having one monolithic architecture, why not have a series of (smaller) services that merely did one thing well? • In case this sounds vaguely familiar...
  4. 4. Proto-punk: The Unix philosophy • The Unix philosophy, as articulated by Doug McIlroy: • Write programs that do one thing and do it well • Write programs to work together • Write programs that handle text streams, because that is a universal interface • The single most important revolution in software systems thinking! • Applying it to HTTP-based services...
  5. 5. Punk architecture: Microservices • Microservices do one thing, and strive to do it well • Replace a small number of monoliths with many services that have well-documented, small HTTP-based APIs • Larger systems can be composed of these smaller services • And there is an organizational analogue: microservices can break large organizations into smaller, more effective ones
  6. 6. Punks in prod: Deploying microservices • Microservices are tautologically small • One physical machine per service is clearly uneconomical… • ...but deploying many orthogonal services on a single machine is a well-known operational nightmare (e.g. conflicting dependencies, shared fault domain) • The key is to virtualize — but at what layer of the stack? • Virtualization has ramifications with respect to performance and density — which is to say, economics
  7. 7. Hardware-level virtualization? • The historical answer to virtualization — since the 1960s — has been to virtualize the hardware: • A virtual machine is presented upon which each tenant runs an operating system that they choose (and must manage) • There are as many operating systems on a machine as tenants! • Can run entire legacy stacks unmodified... • ...but operating systems are heavy and don’t play well with others with respect to resources like DRAM, CPU, I/O devices, etc. • With microservices, overhead dominates!
  8. 8. Platform-level virtualization? • Virtualizing at the application platform layer addresses the tenancy challenges of hardware virtualization, and presents a much more nimble (& developer friendly!) abstraction... • ...but at the cost of dictating abstraction to the developer • This is the “Google App Engine” problem: developers are in a straightjacket where toy programs are easy — but sophisticated applications are impossible • Virtualizing at the application platform layer poses many other challenges with respect to security, containment, etc.
  9. 9. OS-level virtualization! • Virtualizing at the operating system hits a sweet spot: • A single operating system (i.e. a single kernel) allows for efficient use of hardware resources, maximizing tenancy and performance • Disjoint instances are securely compartmentalized by the operating system • Gives tenants what appears to be a virtual machine (albeit a very fast one) on which to run higher-level software: PaaS ease with IaaS generality • Also: boots like a bandit! • Model was pioneered by FreeBSD jails and taken to their logical extreme by Solaris zones — and then aped by Linux containers
  10. 10. OS-level virtualization at Joyent • Joyent runs OS containers in the cloud via SmartOS — and we have run containers in multi-tenant production since ~2006 • Adding support for hardware-based virtualization circa 2011 strengthened our resolve with respect to OS-based virtualization • This is especially true for microservices: as services get small, overhead and latency become increasingly important — and OS containers become a bigger and bigger win • We emphasized their operational characteristics — performance, elasticity, tenancy — and for many years, we were a lone voice...
  11. 11. Containers as PaaS foundation? • Some saw the power of OS containers to facilitate up-stack platform-as-a-service abstractions • For example, dotCloud — a platform-as-a-service provider — built their PaaS on OS containers • Struggling as a PaaS, dotCloud pivoted — and open sourced their container-based orchestration layer...
  12. 12. ...and Docker was born
  13. 13. Docker revolution • Docker has used the rapid provisioning + shared underlying filesystem of containers to allow developers to think operationally • Developers can encode deployment procedures via an image • Images can be reliably and reproducibly deployed as a container • Images can be quickly deployed — and re-deployed • Docker complements the DIY ethos of microservices • Docker will do to apt what apt did to tar
  14. 14. Broader container revolution • The Docker model has pointed to the future of containers • Docker’s challenges today are largely operational: network virtualization, persistence, security, etc. • Security concerns are not due to Docker per se, but rather to the architectural limitations of the Linux “container” substrate • For multi-tenancy, state-of-the-art for Docker containers is to run in hardware virtual machines (!!) • Deploying OS containers in hardware virtual machines negates their economic advantage!
  15. 15. The future of containers in production? • Can we achieve an ideal world that combines the development model of Docker with the deployment model of SmartOS? • To facilitate this ideal, we at Joyent have: • Added the ability to run Linux binaries in a SmartOS container • Added a Docker API end point to SmartDataCenter, the container-native stack that runs our public cloud • Open sourced everything • Here’s to the punk rock revolution of microservices + containers!

×