Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Concourse, Spinnaker, Cloud Foundry, Oh My! Creating Sophisticated Deployment Workflows - Cameron Stewart

378 views

Published on

Concourse, Spinnaker, Cloud Foundry, Oh My! Creating Sophisticated Deployment Workflows with Cameron Stewart at SpringOne Tour 2019

Published in: Software
  • Be the first to comment

Concourse, Spinnaker, Cloud Foundry, Oh My! Creating Sophisticated Deployment Workflows - Cameron Stewart

  1. 1. © Copyright 2019 Pivotal Software, Inc. All rights Reserved. Cameron Stewart, Partner Platform Architect SpringOne Tour Burlington July 26th Concourse, Spinnaker, CF, Oh My! Automate all the things.
  2. 2. Who knows what Toil is?
  3. 3. Who likes Toil?
  4. 4. Cover w/ Image Toil Toil Defined “So what is toil? Toil is the kind of work tied to running a production service that tends to be manual, repetitive, automatable, tactical, devoid of enduring value, and that scales linearly as a service grows.” Chapter 5 - Eliminating Toil
  5. 5. “... delivering useful software shapes our world… ...[software] is only useful when we deliver it.” - you’ll see
  6. 6. Getting code into Production is the most important thing we can do
  7. 7. Everything as Code ➔ You need automation; and you need to be able to manage your infrastructure programmatically - at every layer ➔ As delivery velocity increases, we need to adopt the practices to deploy applications to other areas ➔ Infrastructure as Code - writing code that provisions and manages our infrastructure in a predictable way ➔ Delivery pipelines as first class citizens
  8. 8. “We recognize that delivering useful software shapes our world. We recognize that code is the best way to specify precise action. We recognize that code is only useful when we deliver it. Delivery is not a detail, it is our job. Now is the time to apply our core skills to our own work. Now is the time to engineer our delivery. We divide our work between ourselves and computers: humans for decisions, and automation for tasks.”
  9. 9. Continuous Delivery “Continuous Delivery is about shipping small batches of software to production constantly, through automation. Continuous delivery makes the act of releasing dull and reliable, so organizations can deliver frequently, at less risk, and get feedback faster from end users.”
  10. 10. Continuous Delivery Pipeline
  11. 11. Getting code into is the most important thing we can do
  12. 12. Continuous Delivery Pipeline Automated builds Unit tests Compliance checks Service tickets Performance tests Security validation Blue/Green deploys Canary analysis A/B testing Monitoring Security scans Chaos engineering Test-driven dev Iterative coding/fixing Frequent integration Commit code change Store binaries & build artifacts
  13. 13. Continuous Delivery Pipeline Automated builds Unit tests Blue/Green deploys Canary analysis A/B testing Monitoring Security scans Chaos engineering Test-driven dev Iterative coding/fixing Frequent integration Commit code change Store binaries & build artifacts Continuous Integration Continuous Delivery Compliance checks Service tickets Performance tests Security validation
  14. 14. Continuous Delivery Pipeline Automated builds Unit tests Blue/Green deploys Canary analysis A/B testing Monitoring Security scans Chaos engineering Test-driven dev Iterative coding/fixing Frequent integration Commit code change Store binaries & build artifacts Continuous Integration Continuous Delivery Compliance checks Service tickets Performance tests Security validation
  15. 15. Hello, Concourse
  16. 16. The Concourse Pipeline Flexible integration of resources Simple modeling of components Pipeline status is immediately visible Build components are expressed as code
  17. 17. Concourse Concepts: Simple Primitives Resources Detecting, fetching, creation of externally versioned “things” Jobs Compose resources and tasks together to do something (run tests, ship, etc). Tasks Run a script in a container with its dependent inputs # pipeline.yml resources: - name: source-code type: git source: uri: https://github.com/... branch: master - name: source-code type: git source: uri: https://github.com/... branch: master # pipeline.yml jobs: - name: unit plan: - get: source-code trigger: true - task: unit-tests file: source-code/ci/unit.yml # unit.yml platform: linux image_resource: type: docker-image source: repository: java tag: '8' inputs: - name: source-code run: path: source-code/mvnw args: [ clean, test ]
  18. 18. Concourse Pipeline Visualization
  19. 19. Platform Automation Automate the installation and updates of PCF to stay stable and secure always ● Concourse is an essential part of your upgrade path, enabling you to automate small, defined actions for interacting with OpsManager. ● The resulting perpetual upgrade machine helps you to maintain secure, stable PCF instantiations all the time. ● It also ensures developers always have access to the latest features and capabilities. Application CI/CD Automate the CI/CD pipeline to deliver apps/services to PCF fast and safely ● Concourse presents a general approach to automation that makes it ideal for CI/CD workflows. ● Think Concourse for automating continuous integration (CI) and production deployment readiness (e.g., automate change tickets, compliance, security). ● Integrate with CD solutions like Spinnaker for continuous delivery/deployment ApplicationPlatformConcourse for PCF Use Cases Automate Modern Cloud Delivery
  20. 20. Embedded OS (Windows & Linux) NSX-T CPI (15 methods) v1 v2 v3 ... CVEs Product Updates Java | .NET | NodeJS Pivotal Application Service (PAS) Application Code & Frameworks Buildpacks | Spring Boot | Spring Cloud | Steeltoe Elastic | Packaged Software | Spark Pivotal Container Service (PKS) YOU build the containerWE build the container vSphere Azure & Azure StackGoogle CloudAWSOpenstack Pivotal Network “3Rs” Github Concourse Concourse Pivotal Services Marketplace Pivotal and Partner Products Continuous delivery Public Cloud Services Customer Managed Services OpenServiceBrokerAPI Repair — CVEs Repave Rotate — Credhub
  21. 21. What’s good for your apps, is good for your platform. “ ”
  22. 22. Continuously Delivering Your Platform
  23. 23. Cloud Pipelines
  24. 24. Cover w/ Image (Spring) Cloud Pipelines ➔ Creation of a common deployment pipelines in Concourse or Jenkins ➔ Propagation of good testing and deployment practices ➔ Reducing the time required to create pipelines ➔ Project crawler ➔ Implements tests for API contracts using Spring Cloud Contract ➔ Integrates with flyway Database schema migrations
  25. 25. Continuous Delivery Pipeline Automated builds Unit tests Blue/Green deploys Canary analysis A/B testing Monitoring Security scans Chaos engineering Test-driven dev Iterative coding/fixing Frequent integration Commit code change Store binaries & build artifacts Continuous Integration Continuous Delivery Compliance checks Service tickets Performance tests Security validation
  26. 26. G’day, Spinnaker
  27. 27. Spinnaker Is an OSS Multi-Cloud Delivery Platform Spinnaker Community “...the passionate open source community dedicated to making deployment pain go away.” https://www.spinnaker.io/publications/ebook/
  28. 28. Sophisticated Deployment Strategies https://www.spinnaker.io/concepts/
  29. 29. Application Centric Control Plane =
  30. 30. Multi-foundation View
  31. 31. Spinnaker microservices https://www.spinnaker.io/reference/architecture/
  32. 32. Spinnaker Ecosystem Cloud Providers App Engine Amazon Web Services Azure Cloud Foundry DC/OS Google Compute Engine Kubernetes Openstack Oracle CI Systems Jenkins Travis CI Wercker Concourse Artifact Support Docker Google Cloud Storage GitHub HTTP S3 Artifactory Monitoring Datadog Prometheus Stackdriver Atlas SignalFx (10+ more on roadmap) Notifications Email HipChat Slack SMS via Twilio
  33. 33. Hello. G’day.
  34. 34. Stateless: Rather than sharing state, every task runs in its own container, controlling its own dependencies. Ephemeral: Clean and consistent builds. Minimizes test infrastructure to maintain and gives you flexibility to scale with the cloud. Continuous integration: Track versions of external artifacts used for CI (e.g., Git, S3, docker image). Build components expressed as code with simple modeling of modular components. Flexible workflow automation: Extensible for compliance or other customized tasks in a pipeline. Stateful: Maintains awareness of the topography (multiple clouds, PCF Foundations, etc.) and keeps record of all pipeline activities for compliance and audit. Application inventory: Maintains inventory of deployed applications, supporting activities like rollback, canary analysis and identifying security vulnerabilities. Sophisticated deployment scenarios: Canary deploys, red/black or blue/green deploys, progressive deployments (e.g., by time zone), multi-cloud/PCF Foundation deployments. How Does Spinnaker Relate to Concourse?
  35. 35. Interactive Continuous Delivery Pipeline CI Production Arbitrary Jobs
  36. 36. Spinnaker + Concourse Together Bidirectional Integration ● Using Concourse as a trigger ○ Trigger a Spinnaker pipeline stage ○ Pass Concourse trigger info on to subsequent stages ● Including Concourse as part of a Spinnaker pipeline ○ Create a stage that watches Concourse jobs ○ Concourse tells Spinnaker what jobs to watch ○ When Concourse jobs complete, Spinnaker stage completes
  37. 37. Concourse Trigger
  38. 38. Trigger Configuration
  39. 39. Using Concourse trigger info in subsequent stages
  40. 40. Using Concourse trigger info in subsequent stages Ex: use the version to identify the application binary in a Deploy stage
  41. 41. 1. Concourse job runs Any Concourse out resource metadata will be available to a running Spinnaker pipeline.
  42. 42. 2. Spinnaker pipeline is triggered
  43. 43. Concourse Stage
  44. 44. Stage Configuration The Concourse resource uses these to identify whether a running stage is a match against the job it is attached to.
  45. 45. 1. Stage starts
  46. 46. 2. Concourse check resource identifies the stage.
  47. 47. 3. Concourse tells Spinnaker which job to watch.
  48. 48. 4. Concourse job completes.
  49. 49. 5. Spinnaker sees the Concourse job is finished, and completes the stage.
  50. 50. Virtuous cycle of automated and iterative progression, feedback, and improvement of software Continuous Delivery to the Cloud Monitoring Securing Resiliency Develop Testing Experimenting Compliance Pivotal Cloud Foundry Deploy Operate Planning Test-Driven Dev CI Development Users
  51. 51. Who likes Toil?
  52. 52. Who wants to eliminate Toil?
  53. 53. Transforming how the world builds software © Copyright 2019 Pivotal Software, Inc. All rights Reserved.

×