Download as PDF, PPTX
Uploaded byAndrew Kennedy
Using Clocker with Project Calico - Running Production Workloads in the Cloud
The document discusses Clocker, an open source application that provides Docker container management and orchestration capabilities. It summarizes Clocker's key features as managing Docker clusters across clouds, serving containers on demand, and managing composite application deployments. It also discusses Clocker's use of Apache Brooklyn for application modeling and Calico for software-defined networking between containers.
More Related Content
![[DevDay 2017] OpenShift Enterprise - Speaker: Linh Do - DevOps Engineer at Ax...](https://cdn.slidesharecdn.com/ss_thumbnails/linhdoopenshiftenterprises-170420095822-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Using Clocker with Project Calico - Running Production Workloads in the Cloud
Using Clocker with Project Calico - Running Production Workloads in the Cloud
- 1. Using Clocker withProject Calico Running Production Workloads in the Cloud Andrew Kennedy, SoftwareCircus, 11 September 2015
- 2. @grkvlt Agenda • Introduction • ApplicationManagement • Networking with Calico • Demonstration • Roadmap • Questions Copyright 2015 by Cloudsoft Corporation Limited
- 3.
- 5. @grkvlt Clocker Statistics • OpenSource • 1 Year Old • 725 Commits • 153 Pull Requests • 11 Contributors • 2 External • 15 KLOC • 26 Releases http://www.redotheweb.com/CodeFlower/ Copyright 2015 by Cloudsoft Corporation Limited
- 6. @grkvlt • Application ManagementPlatform • Deploy, Manage and Monitor Blueprints • Provisioning, Installation and Customization • Management • AutoScaling,Resilience, Performance, Security Apache Brooklyn
- 7. @grkvlt Apache jclouds • JavaCloud Library • API Agnostic • SoftLayer, OpenStack, AWS EC2, GCE… • Create Virtual Machines • Return SSH Endpoint • Create Containers • Docker REST API Copyright 2015 by Cloudsoft Corporation Limited
- 8. @grkvlt Docker • Containers • Isolation •Performance • Composable • Huge Ecosystem • Compute Abstraction • Process Wrapper
- 9. @grkvlt • Software-‐Defined Networking •Calico • Weave • Storage and Volume Management • Flocker • More Being Developed… • Native Plugins Docker Extras
- 10. @grkvlt What does itdo? 1. Spins up and Manages Docker Clusters in the Clouds 2. Serves up Containers on Demand 3. Manages Composite Application Deployments Copyright 2015 by Cloudsoft Corporation Limited
- 11. @grkvlt What does itprovide? • Infrastructure Management • Docker Hosts • Swarm Controller • Multi Host and Multi Container Applications • Seamless Networking • CommunicationBetween Services • Orchestration and Clustering • Control of Containers • Container Management Copyright 2015 by Cloudsoft Corporation Limited
- 12. @grkvlt Clocker and BrooklynSummary • What is it? • Brooklyn Application • Brooklyn Location • What does it provide? • First Class Docker Support in Brooklyn • Optimized Brooklyn Blueprints for Docker • Container Orchestration Copyright 2015 by Cloudsoft Corporation Limited
- 13.
- 14. @grkvlt Brooklyn Blueprints • DescribeApplications • OASIS CAMP Standard • TOSCA and Compose in Development • List of Services • NoSQL Database Clusters • Web Servers and Load Balancers • Shell or Python Scripts • Targeting Multiple Destinations • VM, Container,Bare Metal • Sensors, Effectorsand Policies Copyright 2015 by Cloudsoft Corporation Limited
- 15. @grkvlt Application Management • Sensors •Data from Services • Effectors • Brooklyn Policies • Attached to Entities in Application • Nothing Docker Specific • ElasticScaling and Cluster Resizing • Service Resilience and Replacement Copyright 2015 by Cloudsoft Corporation Limited
- 16. @grkvlt Blueprint Example 1 Copyright2015 by Cloudsoft Corporation Limited id: redis-‐service name: "Docker Hub Redis Service" origin: "https://registry.hub.docker.com/_/redis/" locations: -‐ my-‐docker-‐cloud services: -‐ type: docker:redis:3 openPorts: -‐ 6379 directPorts: -‐ 6379
- 17. @grkvlt Blueprint Example 2 Copyright2015 by Cloudsoft Corporation Limited id: redis-‐cluster name: "Redis Cluster" locations: -‐ jclouds:aws-‐ec2: region: eu-‐central-‐1 services: -‐ type: org.apache.brooklyn.entity.nosql.redis.RedisCluster initialSize: 3
- 18. @grkvlt Blueprint Example 3 id: appserver-‐with-‐policy name: "Tomcat Scaling Webapp Server" location: jclouds:aws-‐ec2:eu-‐west-‐1 services: -‐ type: org.apache.brooklyn.entity.webapp.ControlledDynamicWebAppCluster initialSize: 3 memberSpec: $brooklyn:entitySpec: type: org.apache.brooklyn.entity.webapp.tomcat.Tomcat8Server brooklyn.config: wars.root: https:// s3-‐eu-‐west-‐1.amazonaws.com/brooklyn-‐clocker/brooklyn-‐example-‐hello-‐world-‐sql-‐webapp-‐0.6.0.war http.port: 8080+ java.sysprops: brooklyn.example.db.url: $brooklyn:formatString("jdbc:%s%s?user=%s&password=%s", component("db").attributeWhenReady("datastore.url"), "visitors", "brooklyn", "br00k11n") brooklyn.policies: -‐ policyType: org.apache.brooklyn.policy.autoscaling.AutoScalerPolicy brooklyn.config: metric: $brooklyn:sensor("org.apache.brooklyn.entity.webapp.DynamicWebAppCluster", "webapp.reqs.perSec.windowed.perNode") metricLowerBound: 10 metricUpperBound: 100 minPoolSize: 1 maxPoolSize: 5 -‐ type: org.apache.brooklyn.entity.database.mysql.MySqlNode id: db name: DB HelloWorld Visitors brooklyn.config: datastore.creation.script.url: https:// s3-‐eu-‐west-‐1.amazonaws.com/brooklyn-‐clocker/visitors-‐creation-‐script.sql Copyright 2015 by Cloudsoft Corporation Limited
- 19. @grkvlt Clocker Blueprints Copyright 2015by Cloudsoft Corporation Limited
- 20. @grkvlt Docker Cloud • BrooklynBlueprint for Docker Cluster • Docker Engine on Cloud VM or Bare Metal • Configuration for Host • TLS Certificates • Setup Volumes • Logging • Install SDN Agents • Manage Capacity or Headroom Copyright 2015 by Cloudsoft Corporation Limited
- 21. @grkvlt Headroom • Ensure resourcesavailable • Based on MaxContainersstrategy limit • Or Percentage Utilisation • Or CPU and RAM allocation • Scale Docker Host Cluster Automatically • Add new Docker hosts • Remove emptyDocker hosts Copyright 2015 by Cloudsoft Corporation Limited
- 22. @grkvlt Clocker 1.x Architecture Copyright2015 by Cloudsoft Corporation Limited
- 23. @grkvlt Docker Cloud 1. On-demand 2.Multi-Tenant 3. Hardware Independent 4. Application Level Copyright 2015 by Cloudsoft Corporation Limited
- 24. @grkvlt Clocker Features • DockerExtensions to Brooklyn –Docker Image as First-Class Service Type –Placement Strategiesfor Containers –Create Docker Images and Networks • Manages Docker Engine and Swarm –Deployment and Management –Installation and Configuration –Software-Defined Networking Copyright 2015 by Cloudsoft Corporation Limited
- 25. @grkvlt Container Management • Sources •Docker Image Definition • Docker Hub or Registry • Dockerfile • Brooklyn EntityDefinition • Create Image Automatically • Commit or Push for Reuse Copyright 2015 by Cloudsoft Corporation Limited
- 26. @grkvlt Clocker Orchestration Copyright 2015by Cloudsoft Corporation Limited Docker Engine Virtual Machine Container Clocker Network Segment SDN Provider Cloud Provider Brooklyn
- 27.
- 28. @grkvlt Software-Defined Networking • Neededfor Seamless Provisioning • Host to Host Communication • Same LAN Segment • No Port Forwarding • NaturalApplication Configuration • Initial Driver was EPMD Applications • Useful for any opinionated applications Copyright 2015 by Cloudsoft Corporation Limited
- 29. @grkvlt Networking Providers • ImplementationAgnostic • L2 overlay,L3 routing etc. • Similarto Hypervisorin Clouds • Generic Interfaces • Host Component • Service Component (or Endpoint) • Same idea as Docker Network Plugins Copyright 2015 by Cloudsoft Corporation Limited
- 30. @grkvlt Networking Capabilities • AttachContainers to Networks • Create Networksas Required • Also Attachto VMs and Metal • Provide Multiple Networks • Per-Applicationor Shared • Segmented PrivateAddress Space • Docker Port Forwarding for Ingress Copyright 2015 by Cloudsoft Corporation Limited
- 31. @grkvlt Network Provisioning • Minimal(Zero!)Configuration • Use Sensible Defaults • Allows SDN or Cloud SpecificConfiguration • Allocate Address Space on Demand • IP Pool Controlled byClocker -‐ type: brooklyn.networking.VirtualNetwork networkId: database-‐net cidr: 192.168.34.0/24 gateway: 192.168.34.1 dnsServers: -‐ $brooklyn:entity("bind-‐server").attributeWhenReady("host.address") addIptablesRules: true -‐ type: brooklyn.networking.OpenStackVirtualNetwork networkId: couchbase-‐net Copyright 2015 by Cloudsoft Corporation Limited
- 32. @grkvlt Clocker Networking ContainerHost SDN Bridge Container Internet SDN Gateway Copyright 2015by Cloudsoft Corporation Limited
- 33. @grkvlt Metaswitch Project Calico •SDN for Bare Metal, VMs and Containers • Layer 3 • Uses OS IP routing and forwarding • Configuration in an etcd Cluster • Version 0.4.9 in Clocker • 0.6.0 with libnetwork when Docker stable • Spans VMs and Containers • OpenStack Neutron network driver Copyright 2015 by Cloudsoft Corporation Limited
- 34. @grkvltCopyright 2015 byCloudsoft Corporation Limited Cross-Target Deployment
- 35. @grkvlt Networking Capabilities • WideArea and Multi Region SDN • VPN or IPIP and NAT configuration • Cross Platform SDN • Both VMs and Containers on one VLAN • Name Resolution and Service Discovery • Contributing to Weave DNS for orchestration • Use traditional external BIND service entity • Brooklyn can inject correctendpointaddress Copyright 2015 by Cloudsoft Corporation Limited
- 36.
- 38. @grkvlt Application Blueprint 1 Copyright2015 by Cloudsoft Corporation Limited id: my-‐application location: my-‐docker-‐cloud services: -‐ type: docker:redis:3 id: redis-‐service openPorts: 6379 -‐ type: docker:dnmonster:latest id: dnmonster-‐service openPorts: 8080 -‐ type: docker:grkvlt/myapp:latest id: app-‐service mappedPorts: 80: 8080 links: redis: $brooklyn:component("redis-‐service") dnmonster: $brooklyn:component("dnmonster-‐service")
- 39. @grkvlt Application Blueprint 2 Copyright2015 by Cloudsoft Corporation Limited id: my-‐application location: my-‐docker-‐cloud services: -‐ type: org.apache.brooklyn.entity.nosql.RedisStore id: redis-‐service install.version: 3.0.0 -‐ type: docker:dnmonster:latest id: dnmonster-‐service openPorts: 8080 -‐ type: docker:grkvlt/myapp:latest id: app-‐service mappedPorts: 80: 8080 links: redis: $brooklyn:component("redis-‐service") dnmonster: $brooklyn:component("dnmonster-‐service")
- 40. @grkvlt Application Blueprint 3 Copyright2015 by Cloudsoft Corporation Limited id: my-‐application location: my-‐docker-‐cloud services: -‐ type: org.apache.brooklyn.entity.nosql.RedisStore location: jclouds:aws-‐ec2:eu-‐west-‐1 id: redis-‐service install.version: 3.0.0 -‐ type: docker:dnmonster:latest id: dnmonster-‐service openPorts: 8080 -‐ type: docker:grkvlt/myapp:latest id: app-‐service mappedPorts: 80: 8080 links: redis: $brooklyn:component("redis-‐service") dnmonster: $brooklyn:component("dnmonster-‐service")
- 41. @grkvlt Application Blueprint 4 Copyright2015 by Cloudsoft Corporation Limited id: my-‐application location: jclouds:aws-‐ec2:eu-‐west-‐1 services: -‐ type: org.apache.brooklyn.entity.nosql.RedisStore id: redis-‐service install.version: 3.0.0 -‐ type: docker:dnmonster:latest id: dnmonster-‐service openPorts: 8080 -‐ type: docker:grkvlt/myapp:latest id: app-‐service mappedPorts: 80: 8080 links: redis: $brooklyn:component("redis-‐service") dnmonster: $brooklyn:component("dnmonster-‐service")
- 42. @grkvlt • Orchestrated Dockerdeployment and configuration, with Project CalicoSDN • Brooklyn applicationblueprints deployed with network topology linked to OpenStack using Project Calico • Automated attachment of containers to multiple dynamic networks • Zero Config Multi-Target Deployment Copyright 2015 by Cloudsoft Corporation Limited Features
- 43.
- 44. Apache Mesos ... •Distributed Systems Kernel • Cluster Management • Resource Sharing and Placement • Calico SDN Support • Frameworks • Aurora and Marathon • Riak, Spark, Hadoop, Storm et al • Brooklyn Scheduler
- 45. @grkvlt Roadmap • Mesos Integration •Deploy the Mesos infrastructure • Brooklyn asa Mesos framework • Provide Mesos asanother Brooklyn endpoint • Container Mobility • StatelessServices • Brooklyn Core Integration Copyright 2015 by Cloudsoft Corporation Limited
- 46. @grkvlt Roadmap Copyright 2015 byCloudsoft Corporation Limited
- 47. Clocker Solves: – DockerCloud Networking – Container Placement and Provisioning – Composite Application Management Summary
- 48.
- 49.