Are VM Passé?


Published on

Are VM passé? Ben Golub's awesome presentation about Docker, the project roadmap and its ecosystem at Interop Las Vegas

Published in: Technology

Are VM Passé?

  1. 1. Are VM’s Passé? Ben Golub CEO Docker, Inc.
  2. 2. They told me that I needed a provocative title for this talk. So…
  3. 3. Q: Are VMs Passé?
  4. 4. A: No
  5. 5. Thank you for attending
  6. 6. A different question… • Q: Is there a better alternative for many use cases & environments? –application management & creation? –application deployment across clusters & clouds? –CI & CD? –scale out? –high performance? –collaborative development?
  7. 7. A: Yes
  8. 8. Agenda • Server Proliferation and the rise of the VM • The Matrix from Hell and the need for lightweight, interoperable containers • Step 1: Making lightweight containers: • Containers vs. VMs: how they work • Step 2: Making containers really interoperable: • Step 3: Making containers really lightweight • Step 4: Creating a container-based system for app mgt & deployment • Step 5: Creating an ecosystem around containers • Where to Use VMs vs. Containers/Docker • Learn more
  9. 9. Some ancient history: Where did VM’s come from? • PROBLEM IN 2000: Server Proliferation/Consolidation - I’ve got a single purpose, physical Microsoft Exchange Server, Mac print server, and Custom Unix inventory server - Machines are getting more powerful - I want to consolidate all those single purpose, physical servers onto a single server - BTW--It takes too damn long to provision a physical machine • ANSWER: Create a Virtual Machine
  10. 10. Results • Single purpose physical application servers become single purpose virtual servers • Provisioning a “server” goes from days/weeks to minutes • Huge cost savings • An awesome solution to the server consolidation problem • An awesome solution for creating flexible infrastructures • Mature ecosystem and tool set for isolation, security, management
  11. 11. What has changed since the VM was developed? 2000 2014 Apps are long lived Development is iterative and constant Apps are monolithic and developed on a single stack Apps are created from loosely coupled components, themselves created from a multitude of “stacks” Deployment is to a single server Deployment is to a variety of servers: VM, physical, cluster, open stack, public cloud, +++ • Result: An application isn’t easily represented or managed as a single purpose server (whether physical or virtual)
  12. 12. Static website Web frontend User DB Queue Analytics DB Background workers API endpoint nginx 1.5 + modsecurity + openssl + bootstrap 2 postgresql + pgv8 + v8 hadoop + hive + thrift + OpenJDK Ruby + Rails + sass + Unicorn Redis + redis-sentinel Python 3.0 + celery + pyredis + libcurl + ffmpeg + libopencv + nodejs + phantomjs Python 2.7 + Flask + pyredis + celery + psycopg + postgresql-client Development VM QA server Public Cloud Disaster recovery Contributor’s laptop Production Servers The Problem in 2014MultiplicityofStacks Multiplicityof hardware environments Production Cluster Customer Data Center Doservicesandapps interact appropriately? CanImigrate smoothlyand quickly?
  13. 13. The Matrix From Hell Static website Web frontend Background workers User DB Analytics DB Queue Development VM QA Server Single Prod Server Onsite Cluster Public Cloud Contributor’s laptop Customer Servers ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?
  14. 14. MultiplicityofGoods Multipilicityof methodsfor transporting/storing DoIworryabout howgoodsinteract (e.g.coffeebeans nexttospices) CanItransportquickly andsmoothly (e.g.fromboattotrain totruck) An Inspiration…and some really ancient history: Cargo Transport Pre-1960
  15. 15. MultiplicityofGoods Multiplicityof methodsfor transporting/storing DoIworryabout howgoodsinteract (e.g.coffeebeans nexttospices) CanItransport quicklyandsmoothly (e.g.fromboatto traintotruck) Solution: Intermodal Shipping Container …in between, can be loaded and unloaded, stacked, transported efficiently over long distances, and transferred from one mode of transport to another A standard container that is loaded with virtually any goods, and stays sealed until it reaches final delivery.
  16. 16. This spawned an Intermodal Shipping Container Ecosystem • 90% of all cargo now shipped in a standard container • Order of magnitude reduction in cost and time to load and unload ships • Massive reduction in losses due to theft or damage • Huge reduction in freight cost as percent of final goods (from >25% to <3%)  massive globalization • 5000 ships deliver 200M containers per year
  17. 17. Static website Web frontendUser DB Queue Analytics DB Development VM QA server Public Cloud Contributor’s laptop Let’s create a shipping container system for codeMultiplicityofStacks Multiplicityof hardware environments Production Cluster Customer Data Center Doservicesandapps interact appropriately? CanImigrate smoothlyandquickly …that can be manipulated using standard operations and run consistently on virtually any hardware platform An engine that enables any payload to be encapsulated as a lightweight, portable, self-sufficient container…
  18. 18. Static website Web frontend Background workers User DB Analytics DB Queue Development VM QA Server Single Prod Server Onsite Cluster Public Cloud Contributor’s laptop Customer Servers Eliminate the matrix from Hell
  19. 19. App A Step One: Create a lightweight container (vs. VMs) Hypervisor (Type 2) Host OS Server Guest OS Bins/ Libs App A’ Guest OS Bins/ Libs App B Guest OS Bins/ Libs AppA’ Host OS Server Bins/Libs AppA Bins/Libs AppB AppB’ AppB’ AppB’ VM Container Containers are isolated, but share OS kernel and, where appropriate, bins/libraries Guest OS Guest OS …result is significantly faster deployment, much less overhead, easier migration, faster restart
  20. 20. A great slide stolen from IBM: Why Containers?  Provision in seconds / milliseconds  Near bare metal runtime performance  10 x greater density  VM-like agility – it’s still “virtualization”  Flexibility – Containerize a “system” – Containerize “application(s)”  Lightweight – Just enough Operating System (JeOS) – Minimal per container penalty  Open source – free – lower TCO  Supported with OOTB modern Linux kernel  Growing in popularity 3/28/2014 20 “Containers as poised as the next VM in our modern Cloud era…” Manual VM LXC Provision Time Days Minutes Seconds / ms linpack performance @ 45000 0 50 100 150 200 250 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 BM vcpus GFlops Google trends - LXC Google trends - docker
  21. 21. Step 2: Make the containers easy to use, standardized, interoperable, automatable • Shipping containers are a standard size, and have hooks and holes in all the same places • With Docker, Containers get the following: - Ease of use, tooling - Re-usable components - Ability to run on any Linux server today: physical, virtual, VM, cloud, OpenStack, +++ - (Stay tuned for other O/S’s) - Ability to move between any of the above in a matter of seconds-no modification or delay - Ability to share containerized components - Interoperability with all existing devops tools - Self contained environment—no dependency hell - Tools for how containers work together: linking, nesting, discovery, orchestration, ++ • You get ability to separate app management from infrastructure management
  22. 22. Technical & cultural revolution: separation of concerns • Dan the Developer • Worries about what’s “inside” the container • His code • His Libraries • His Package Manager • His Apps • His Data • All Linux servers look the same • Oscar the Ops Guy • Worries about what’s “outside” the container • Logging • Remote access • Monitoring • Network config • All containers start, stop, copy, attach, migrate, etc. the same way
  23. 23. Step 3: Make containers super lightweight Bins/ Libs App A Original App (No OS to take up space, resources, or require restart) AppΔ Bins/ App A Bins/ Libs App A’ Guest OS Bins/ Libs Modified App Copy on write allows us to only save the diffs Between container A and container A’ VMs Every app, every copy of an app, and every slight modification of the app requires a new virtual server App A Guest OS Bins/ Libs Copy of App No OS. Can Share bins/libs App A Guest OS Guest OS VMs Containers
  24. 24. Prod Hosts 21000+: OS (Linux)Prod Hosts 21000+: OS (Linux) Step 4: Build a System for creating, managing, deploying code Source Code Repository Dockerfile For A Docker Registry Build Docker Prod Hosts: OS (Linux) ContainerA ContainerB ContainerC ContainerA Push Search Pull Run Dev Host : OS (Linux) Docker
  25. 25. Including a System for Changes and Updates Docker Registry Push Update Bins/ Libs App A AppΔ Bins/ Base Container Image Hosts are now running A’’ Container Mod A’’ AppΔ Bins/ Bins/ Libs App A Bins/ Bins/ Libs App A’’ Host(s) running A want to upgrade to A’’. Requests update. Gets only diffs Container Mod A’ Prod Hosts 21000+: OS (Linux)Prod Hosts 21000+: OS (Linux)Prod Hosts: OS (Linux) Prod Hosts 21000+: OS (Linux)Prod Hosts 21000+: OS (Linux)Prod Hosts: OS (Linux)
  26. 26. Including a System for the Full Lifecycle Develop Test/QA Build Run Scale Hyper scale APP CREATION APP DEPLOYMENT APP MANAGEMENT
  27. 27. Docker engine Docker engine Docker engine Docker engine Containers can be linked and assembled into complex service- oriented stacks. v0.7 Stacks can span multiple machines, using encrypted and authenticated tunnels. v0.9 v0.really soon Docker deployments can span multiple datacenters and cloud providers by using the service. acts as a hub and federates authentication, service discovery and orchestration across all docker engines across an organization Including a System for Complex Apps
  28. 28. Step 5: Create an Ecosystem 45,000
  29. 29. Open Ecosystem Momentum • Truly open: Apache license, open design, open tooling, non-Docker maintainers • Downloads: Over 1.2 m container downloads • Users: Over 45,000 trained developers • Content: Over 8000 repositories now publishing containers to Docker Index • Contributors: 380 contributors, 95% of whom don’t work for Docker, In.c • Meetups: Over 80 cities in 30 countries have Docker meetups • Integrations: OpenStack, RHEL, Ubuntu, Salt, Chef, Puppet, Salt +++ • Github • Over 10,000 stars • Over 1.7 K forks • Over 350 derivative projects
  30. 30. Who is using Docker? … and hundreds of other small and big companies
  31. 31. Four major use cases • Continuous Integration/Continuous Delivery: • Go from developer’s laptop, through automated test, to production, and through scaling without modification • Alternative form of virtualization for multi-tenant services • Scale-out: • Rapidly scale same application across hundreds or thousands of servers…and scale down as rapidly • Cross Cloud Deployment • Move the same application across multiple clouds (public, private, or hybrid) without modification or noticeable delay
  32. 32. Where should I use VMs? • VMs are definitely the way to go to solve many problems • Heterogeneous O/S families: Run Windows app on a Mac Server • Using O/S or kernel that doesn’t support containers • Your real problem is infrastructure management • You want the maturity of the VM toolset • VM requires unique kernel setup which is not applicable to other VMs on the host (i.e. per VM kernel config) • Need to freeze state and live migrate • But… you can pursue a hybrid strategy: containers on VMs • Stay tuned for better Docker/Container answers for many of the above
  33. 33. Conclusion • Multiple forces are driving a reconsideration of how applications should be created, built, deployed, scaled, and managed • We believe that the right approach is to decouple application management from infrastructure management • Container based approach (vs. VM approach) provides right level of abstraction • Enables infrastructure to be managed consistently and stably • Enables applications to be built flexibly and deployed flexibly • Provides greater degree of visibility, control, and management of what runs where and what components are allowed • Massive cost, speed, efficiency savings • Docker is becoming the standard for containerization
  34. 34. Learn More • LXC Technical discussion: • Docker project: • Follow Docker on Twitter: • Take the Docker interactive tutorial: • Join Docker on IRC: • Go to the Docker repository on GitHub: • Go to a meetup: • See what others are doing: • Come to DockerCon, Jun 9-10, San Francisco:
  35. 35. Are VM’s Passé? Ben Golub CEO Docker, Inc.