Advertisement

More Related Content

Slideshows for you(20)

Advertisement
Advertisement

Alibaba Cloud Conference 2016 - Docker Enterprise

  1. What’s New in Docker John Willis Director of Ecosystem Development
  2. About Me • One of the founding members of the “Devops” movement. • Author of the “Devops Handbook”. • Author of the “Introduction to Devops” on Linux Foundation edX. • Podcaster at devopscafe.org • Devops Enterprise Summit - Cofounder • Found of Socketplane (Acquired by Docker) • Formally Director of Devops at Dell • Formally Director at Chef • 10 Startups over 25 years 2 Github: botchagalupe/my-presentations Twitter: botchagalupe Wechat: botchagalupe
  3. Devops Practices and Patterns • Continuous Delivery • Everything in version control • Small batch principle • Trunk based deployments • Manage flow (WIP) • Automate everything
 • Culture • Everyone is responsible • Done means released • Stop the line when it breaks • Remove silos 3 itrevolution.com/devops-handbook http://www.bookdepository.com/
  4. The world needs tools of mass innovation
  5. A programmable Internet would be the ultimate tool of mass innovation
  6. Let’s eliminate friction in the development cycle
  7. A commercial product, built on a development platform, built on infrastructure, built on standards. Docker is building a stack to program the Internet
  8. 8 Docker Project Sponsor •Primary authors, contributor maintainer •6B+ Downloads, 3000+ Contributors, 500,000+ Applications •100’s of ecosystem partners •Millions of developers use Docker. Millions of servers run Docker Commercial Docker Solutions •Integrated solutions to build, ship, run Docker at scale • Orchestration, registry, security, workflow, control plane • CaaS (containers as a service) •Official providers of commercial technical support •10K’s cloud customers, 300+ F500 customers About Docker, Inc. Gerber, Anna. “The State of Containers and the Docker Ecosystem: 2015” O’Reilly, September 2015 Docker users already running in production 60%
  9. China is part of Worldwide Docker Community Meetups in Beijing, Changsha, Chengdu, Chongqing, Dalian, Fuzhou, Guangzhou, Hangzhou, Hong Kong, Nanjing, Qingdao, Shanghai, Shenzen, Suzhou, Tianjin, Wuhan, Xi’an, Xiamen, and Zhuhai
  10. Cloud and Docker Status Update in China China cloud relative to the West – DB ! China’s cloud spending to reach 24.5% of the West by 2018 from 5.7% in 2015 ! More than 80% respondents are already using or plan to use containers, but only 10% used in production environment. Survey result from 350+ feedbacks by Alibaba Cloud, June 2016 Docker Adoption in China
  11. Other Government Sample of Docker Customers Financial Services Healthcare Consumer Education Tech Services 11
  12. Dockerizing applications 12 Legacy App: One container per app. Microservices: one service per container. App comprised of many containers linked together
  13. Enterprises Are Containerizing both Legacy and Microservices Applications 13 80% 46% plan to build new microservices in 2016 #4 container workload is traditional databases 1 in 3 have already containerized legacy apps Will leverage Docker to enable hybrid cloud initiatives Docker platform standardizes environment AND enables workload portability
  14. 3 Paths to Containers Adoption 1 Containerize Monoliths Build-Test for CI; Migrate to the Cloud; Get Better CapEx/OpEx than VM 2 3 Containerize Monolith; Transform to Microservices Look for Shared Services to Transform Enable New Microservices and Apps Greenfield CaaS
  15. Migrate any workload anywhere Infrastructure agnostic platform • Standardize: Docker abstracts away the infrastructure and virtualization away from the standardized app containers • Portability: Containers move without friction from one environment to another – no recoding needed • Lift and Shift: Containerize legacy and microservices to gain portability 15 Bins/Libs App OS Bins/Libs App OS Bins/Libs App OS Before: VM formats are proprietary Bins/Libs App Bins/Libs App Bins/Libs App OS After: Docker abstracts above VMs for portability Docker Engine OS Docker Engine
  16. Infrastructure optimization with Docker Swisscom • 20:1 VM consolidation ratio • Running the same 400 MongoDB instances in 400 containers in 20 VMs • Reduce CapEx and OpEx costs Leading Energy Company • Containerize legacy apps for portability • Entire cloud to datacenter site migration in 5 months • Dramatically accelerated release process 16
  17. 17 Enable Modern App Initiatives with CaaS Cloud Migration Hybrid Cloud Multi Cloud Modernizing Apps DevOps CI/CD DevOps Cloud DevOpsApps
  18. 18 Enterprises Can Decide How To Adopt Enable CI and DevOps Better Resource Usage Enable Cloud Initiatives Transform iteratively Ship faster, with better reliability Enable developers to self service Enable business transformations Containerize Build New AppsContainerize & Transform
  19. The Docker ecosystem Dev Tools Official Repositories Operating Systems Big Data Service Discovery Build / Continuous Integration Configuration Management Consulting &Training Management Storage Clustering & Scheduling Networking Infrastructure & Service Providers Storage Security Monitoring & Logging 19
  20. Docker and Alibaba Announce 
 Commercial Agreement • Localized Docker image store and distribution for Docker Hub on Alibaba Cloud • Alibaba will resell Commercially Supported (CS) Docker Engine and Docker Datacenter, enabling enterprises to manage their production workloads across the entire application lifecycle. • Alibaba will provide enterprise support options for CS Docker Engine and Docker Datacenter, backed by Docker, Inc. 20
  21. Docker Platform
  22. COMPARING CONTAINERS AND VIRTUAL MACHINES
  23. Isolation using Linux kernel features namespaces ● pid ● mnt ● net ● uts ● ipc ● user cgroups ● memory ● cpu ● blkio ● devices
  24. Image layers
  25. Docker Engine extensibility and plugins • Built in orchestration expands the opportunity for the plugin to manage swarm wide vs a single Engine • Updated architecture standardizes plugin process for ecosystem partners – Benefits users and vendors – Standardized process of granting plugin permissions – Containerized plugins on roadmap Docker Engine Networking Swarm Mode Volumes Plugins
  26. Developer experience
  27. 1. Get out of the way The best tools… 2. Adapt to you 3. Make the powerful simple
  28. Docker for Mac Docker for Windows
  29. Orchestration
  30. Introducing the best way to orchestrate Docker: Docker. Docker 1.12: now with orchestration built-in.
  31. Swarm mode Service API Node identity Built-in routing mesh Docker 1.12: now with orchestration built-in.
  32. Combine your engines in swarms of any scale Self-organizing, self-healing No external data store required
 No single points of failure Infrastructure-agnostic topology Swarm mode
  33. Desired state reconciliation Scaling
 Rolling updates Advanced scheduling Application-specified health checks Rescheduling on node failure Docker Service API
  34. How service deployment works Declare ScheduleReconcile $ docker service create declares the service name, network, port, image:tag and scale Managers break down service into tasks, schedules them and workers execute tasks Engines check to see what is running and compared to what was declared to “true up” the environment
  35. API Allocator Orchestrator Scheduler Dispatcher R A F T Manager Node Worker Executor Worker Node Accepts command from client and creates service object Reconciliation loop for service objects and creates tasks Allocates IP addresses to tasks Assigns tasks to nodes Checks in on workers docker service create Connects to dispatcher to check on assigned tasks How service deployment works Executes the tasks assigned to worker node
  36. Manager ManagerManager Worker WorkerWorkerWorker Worker Example service on a swarm The declarative command describes a new service: • Named Frontend • Made of 5 containers based on the latest my_frontend_image • Connected on an overlay network called “my overlay” • Assigned to port 80 $ docker service create --replicas 5 --name frontend --network myoverlay --publish 80/tcp mikegoelzer/ my_frontend_image:latest
  37. Manager ManagerManager Worker WorkerWorkerWorker Worker Example service on a swarm This state is what swarm mode and the service deployment API will maintain. Check to ensure 5 containers are always running for the frontend service$ docker service create --replicas 5 --name frontend --network myoverlay --publish 80/tcp mikegoelzer/ my_frontend_image:latest
  38. Manager ManagerManager Worker WorkerWorkerWorker Worker Example service on a swarm This command creates a new service to join an existing overlay network to communicate with frontend $ docker service create --name redis --network myoverlay redis:latest
  39. Built-in Routing Mesh Swarm-wide overlay networking Container-native load-balancing DNS-based service discovery No separate cluster to setup Works with your existing load-balancers Rock-solid kernel-only data path with IPVS
  40. Routing mesh for application services • Container-aware dynamic load balancing • Assign ports to service that do not change • Built in load balancing into the Engine • Automatic service discovery Worker Load Balancing Worker Load Balancing Worker Load Balancing Port 1000Port 1000 Port 1000 Manager ManagerManager Docker user assigns a global port for a service End user accessing www.website.com
  41. Docker 1.12 is Simple > docker swarm init > docker service create
  42. How to get Docker 1.12 Mac OS X Developer Workstations Cloud Providers Windows PC AWS Azure Commercially Supported Docker Datacenter CS Docker Engine Docker Cloud Open Source Docker Engine installers for the following Linux distros Optimized for and integrated directly into the underlying platform with custom plugins and drivers
  43. Ops experience
  44. Deep integration with native load-balancers, templates, SSH keys, ACLs, scaling groups, firewall rules… beta.docker.com
  45. Distributed Application Bundle experimental www.docker.com/dab A portable format for multi-container applications
  46. 5 Goldilocks and the 3 XaaS Just rightToo highToo low IaaS PaaS CaaS
  47. 5 Goldilocks and the 3 XaaS Platform As A Service Infrastructure As A Service Software As A Service Too high Too low Just right Container As A Service
  48. Docker Datacenter core values 48 Management at scale Integrated Content Trust Secure Access (RBAC) Integrates with existing systems Full support of Docker API Seamless dev to prod workflow Infrastructure, network and storage portability Easy to setup and use Native Docker solution Extend existing Docker developer experience + +Agility Portability Control
  49. Agility, portability and control for developers and IT Developers IT Operations • Freedom to create and deploy apps fast • Define and package application needs • Quickly and flexibly respond to changing needs • Standardize, secure, and manage Frictionless portability across teams, environments, infrastructure 49
  50. Docker Datacenter platform Management Universal Control Plane Security Content Trust, RBAC, LDAP/AD Orchestration Swarm Container Runtime Engine Registry Service Trusted Registry
  51. BUILD Development Environments SHIP Registry: Secure Content & Collaboration RUN Control Plane: Deploy, Orchestrate, Manage, Scale Networking Volumes MonitoringLoggingConfig MgtCI/CD IT Operations Developers IT Operations Docker CaaS Workflow
  52. Docker Containers as a Service platform 52 BUILD Developer Workflows SHIP Registry Services RUN Management Docker for Mac and Windows Docker Trusted Registry Docker Universal Control Plane Docker Cloud Docker Container Engine Ecosystem Plugins and Integrations
  53. UCP Permission Model
  54. Docker UCP 1.1 - DTR 2.0 • HA • Unified Auth • Compose deployment • UI to add nodes
  55. Security scanning in Docker Cloud
  56. Adoption
  57. 2013-05 2013-06 2013-07 2013-08 2013-09 2013-10 2013-11 2013-12 2014-01 1,000,000 0 2014-02 2014-03 2014-04 2014-05 2014-06 2014-07 2014-08 2014-09 2014-10 2014-11 2014-12 2015-01 2015-02 2015-03 2015-04 2015-05 2015-06 2015-07 2015-08 2015-09 2015-10 2015-11 2015-12 2016-01 1,000,000,000 ~ 10,000,000 9,000,000 8,000,000 7,000,000 6,000,000 5,000,000 4,000,000 3,000,000 2,000,000 6,000,000,000 5,750,000,000 5,500,000,000 5,250,000,000 5,000,000,000 4,750,000,000 4.500,000,000 4,250,000,000 4,000,000,000 3,750,000,000 3,500,000,000 3,250,000,000 3,000,000,000 2,750,000,000 2,500,000,000 2,250,000,000 2,000,000,000 1,750,000,000 1,500,000,000 1,250,000,000
  58. 2013-05 2013-06 2013-07 2013-08 2013-09 2013-10 2013-11 2013-12 2014-01 1,000,000 0 2014-02 2014-03 2014-04 2014-05 2014-06 2014-07 2014-08 2014-09 2014-10 2014-11 2014-12 2015-01 2015-02 2015-03 2015-04 2015-05 2015-06 2015-07 2015-08 2015-09 2015-10 2015-11 2015-12 2016-01 ~ 2016-09 1,000,000,000 ~ 10,000,000 9,000,000 8,000,000 7,000,000 6,000,000 5,000,000 4,000,000 3,000,000 2,000,000 6,000,000,000 5,750,000,000 5,500,000,000 5,250,000,000 5,000,000,000 4,750,000,000 4.500,000,000 4,250,000,000 4,000,000,000 3,750,000,000 3,500,000,000 3,250,000,000 3,000,000,000 2,750,000,000 2,500,000,000 2,250,000,000 2,000,000,000 1,750,000,000 1,500,000,000 1,250,000,000 Notary runC • containerd • HyperKit , VPNKit, DataKit • SwarmKit • libcontainer • libnetwork • • Docker 1.8 : Docker Content Trust • Docker for Mac Docker for Windows • Docker 1.12 with built-in orchestration • Docker 0.9 : Pluggable execution • Docker 1.7 : Multi-Host Networking • Docker 1.11: OCI support
  59. Docker on Windows
  60. Extra
  61. InfraKit A toolkit for building declarative, self- healing infrastructure.
  62. Declarative • JSON configuration for desired infrastructure state: • Specification of instances — vm image, instance type, etc. • Group properties — size, logical identifiers, etc.
 • Design patterns encourage • encapsulation • composition
 • Config is input to all operations — system figures out what to do
 66
  63. Self-healing • Composed of a set of active components / processes that • monitor infrastructure state • detect state divergence • take actions
 • Continuous monitoring and reconciliation — always on
 • No downtime — rolling update 67
  64. Toolkit • Primitives for managing collections of resources • create, scale, destroy • rolling update • Abstractions & Developer SPI • Group - manages collection of resources • Instance - describes the physical resource • Flavor - extra semantics for handling instances • A collection of executable, active components — plugins • Initially, Go daemons in the toolkit • Soon, easy management via Docker Plugins (runc)
  65. Architecture
  66. Instance Plugin • Spec: specification / model of an instance (e.g. vagrant, EC2): • Logical ID, Init, Tags, and attachment • Platform-specific properties • Methods: • /Instance.Validate • /Instance.Provision • /Instance.Destroy • /Instance.DescribeInstances • Examples: instance plugins for EC2, Azure VM, Vagrant, … 70
  67. Flavor Plugin • Gives more context about the group members: • Size, or list of Logical ID’s (e.g. IP addresses for ‘pets’) • Application-specific notions of ‘health’ 
 Is the node not only present but also joined a swarm? • Methods: • /Flavor.Validate • /Flavor.Prepare • /Flavor.Healthy • Examples: flavor for Zookeeper members, Docker swarm nodes 71
  68. Group Plugin • Main entry point for user interaction: • Create, describe update, update, destroy • Config JSON is always the input • Composed of Instance and Flavor — mix and match to manage cattle (fungible) or pets (special) • Methods: • /Group.Watch • /Group.Unwatch • /Group.Inspect 72 • /Group.DescribeUpdate • /Group.Update • /Group.StopUpdate • /Group.Destroy
  69. Configuration Example config file (zk.conf): Group configuration = Instance + Flavor { "Properties": { /* raw configuration */ } } { "groups" : { "my_zookeeper_nodes" : { "Properties" : { "Instance" : { "Plugin": "instance-vagrant", "Properties": { "Box": "bento/ubuntu-16.04" } }, "Flavor" : { "Plugin": "flavor-zookeeper", "Properties": { "type": "member", "IPs": ["192.168.1.200", "192.168.1.201", "192.168.1.202"] } } } } }
  70. Operations • Make sure the plugins are running: • infrakit/group &; infrakit/zookeeper &; infrakit/vagrant &;
 • “Watch” the group starts management: • infrakit/cli group watch zk.conf
 • Update the config, e.g. change size or add IP address • Describe changes before committing — 
 infrakit/cli group describe zk.conf • Begin update —
 infrakit/cli group update zk.conf 74
  71. Today 75 • InfraKit is just getting started… only primitives for working with groups like clusters of hosts
 • But we have big plans • Improve group management strategies • More resource types — networking, load balancers, storage…
 • A cohesive framework for active management of infrastructure — physical, virtual, or containers
  72. Get Involved • Help define and implement new and interesting plugins • Instance plugins for different infrastructure providers • Flavor plugins for systems like etcd or mysql clusters • Group controller plugins — metrics-driven auto scaling and more
 • Help define interfaces and implement new infrastructure resource types — load balancers, networks and storage volume provisioners 76
  73. More Info • Github: 
 https://github.com/docker/infrakit • A quick tutorial: 
 https://github.com/docker/infrakit/blob/master/docs/ tutorial.md 77
Advertisement