Download as PDF, PPTX
![Pre-Fabricated Pipelines
pipeline {
agent any
environment {
APP_NAME = 'hello'
}
stages {
stage('dependencies') {
when {
expression { opts.stageDeps ==
'true' }
}
steps {
sh "go mod download"
}
}
stage('tests-parallel') {
when {
expression { opts.stageTest ==
'true' }
}
failFast true
parallel {
stage('vet') {
steps {
script {
sh "go vet ${env.APP_NAME}"
}
}
}
stage('unit') {
steps {
script {
sh "go test -v $
{env.APP_NAME}"
...
Pipeline as Code
ci ([
stageDeps: 'true',
stageTest: 'true',
stageBuild: ‘true’,
stageDeploy: ‘true’
])
Pre-fabricated Pipeline
@gregsypolt](https://image.slidesharecdn.com/keynote-agile-devops-2019-190611130410/75/Agile-DevOps-Keynote-The-Evolution-of-a-Continuous-Integration-Pipeline-20-2048.jpg)
Uploaded byGreg Sypolt
Agile + DevOps Keynote - The Evolution of a Continuous Integration Pipeline
The document discusses the evolution of the continuous integration (CI) pipeline at the USA Today Network, highlighting its transformation from manual self-service to a hosted CI platform. It emphasizes the importance of shifting left in DevOps, continuous testing, and collaboration to improve software quality and reduce costs. Additionally, it includes metrics showcasing the improvements achieved, such as significant cost reductions and increased build activity.
More Related Content
Similar to Agile + DevOps Keynote - The Evolution of a Continuous Integration Pipeline
Agile + DevOps Keynote - The Evolution of a Continuous Integration Pipeline
- 1. The Evolution ofa Continuous Integration Pipeline Greg Sypolt unsplash.com
- 2. The Scale OfUSA TODAY NETWORK Gannett is a leading local-to- national media & marketing solutions company, and the largest local media company in America, with 100+ newsrooms spanning 34 states. Who we are. 2,000+ 100+ 124M+ 1.1B 500+ JOURNALISTS NEWSROOMS UNIQUE MONTHLY USERS TOTAL PAGES PER MONTH DIGITAL PRODUCTS @gregsypolt
- 3. Happy CustomerReduce Cost AddressingThe Problem With Innovation Speed @gregsypolt
- 4. Evolution of ContinuousIntegration Hosted CI Platform Manual On-Premise Self-Service @gregsypolt
- 5. The Vision Shifting LEFTwith DevOps and Automated Testing Code Build Test Deploy MonitorPlanCollaborate building quality into every stage Repeat It takes a COMMUNITY Dev | Staging | Prod Discovery Execute Release Review How We Test Things Shift RIGHT @gregsypolt
- 6.
- 7. ConfidenceShareAdapt Expectations for ContinuousIntegration Goals @gregsypolt
- 8.
- 9. CI Ownership asa Community Product EngineeringPlatform Engineering Quality Engineering </> </> Build Unit Integration Pull Request Code √ √ √ Tests √ √ √ √ √ √ Others Stage Acceptance Others √ √ √ √ √ √ Tests Production Others √ √ √ Synthetics √ √ √ Tests Build better software by improving speed, efficiency, and collaboration Teamwork Vision Goals StrategyInnovation Success MeasureElevate Ideas Culture of Quality Best Practices Collect Requirements Coach and Mentor Create Pipeline Code Maintain Infrastructure Containerization Monitoring Results Identifying Quality Gaps @gregsypolt
- 10. Defining A SinglePath To Production @gregsypoltunsplash.com
- 11. Pre-Commit Commit AcceptancePerformance Exploratory Production Pipeline Stages Infrastructure Any plan that defers testing is BROKEN … Defining a Single Path to Production @gregsypolt
- 12. Defining a ContinuousTesting Strategy @gregsypoltunsplash.com
- 13. Define a ContinuousTesting Strategy Commit Creation Pull Request Merged Pull Request Deployment Trigger Points Development Staging Production Environments Infrastructure Unit Module/Component API Integration User Acceptance Visual Accessibility Targeted Performance Stress Security Scalability Quality Playbook Test Continuously Instant Feedback on Quality Reduce Risk Build Confidence Faster Release Cycles Shift Left Disposable Infrastructure Automate Everything Application Monitoring Blue/Green Deployments Canary DevOps @gregsypolt
- 14. Designing a CI/CDPlatform @gregsypoltunsplash.com
- 15. The CI/CD Platform PlatformEng APIs& Tools Plumbing Helm Source Control Artifact Storage Artifact Storage Dev Stage ProdD CI/CD Server CI/CD Workers Sauce Labs Applitools SumoLogic Selenoid Grid @gregsypolt
- 16.
- 17. CI/CD Standards Multiple Teams JenkinsfileNotifications Manage SecretsWebhooksMonitoring More … Kubernetes Containers Shared Libraries Dashboards Artifact Storage Google Storage @gregsypolt
- 18. CI/CD Plumbing Change Logs Consul HTTP Kubectl Artifactory DNS Deployment LightsOut Release Notes Tagging Vault Credentials Test Aggregator GitHub Repo Inform Semantic Versioning Build Status Color JIRA Tickets @gregsypolt 16
- 19. Pipeline as Code Agent Stages Stage Parallel BasicPipeline Terms Restricting Builds Retrieving Repositories Docker Image Build (Compiling Code) Running Tests Deploy to Environment Reporting Test Details Push Build Artifacts Notification Scripting Pipeline Code In general, most Pipeline scripts look more or less like this… Jenkinsfile Commit Acceptance Performance Exploratory Production @gregsypolt
- 20. Pre-Fabricated Pipelines pipeline { agentany environment { APP_NAME = 'hello' } stages { stage('dependencies') { when { expression { opts.stageDeps == 'true' } } steps { sh "go mod download" } } stage('tests-parallel') { when { expression { opts.stageTest == 'true' } } failFast true parallel { stage('vet') { steps { script { sh "go vet ${env.APP_NAME}" } } } stage('unit') { steps { script { sh "go test -v $ {env.APP_NAME}" ... Pipeline as Code ci ([ stageDeps: 'true', stageTest: 'true', stageBuild: ‘true’, stageDeploy: ‘true’ ]) Pre-fabricated Pipeline @gregsypolt
- 21.
- 22. Shifting Ownership Responsibilities 27 LEGACYCI/CD SELF-SERVICE 13 HOSTED CI/CD PLATFORM @gregsypolt
- 23. By The Numbers @gregsypoltunsplash.com
- 24. Availability LEGACY CI/CD SELF-SERVICEHOSTED CI/CD PLATFORM Q4 2018 Q1 2019 Q2 2019 99.97% 99.77% 99.89% @gregsypolt
- 25. Legacy CI/CD Solution $14,000/yr AVERAGELEGACY CI/CD COST SCALR FARM LEGACY CI/CD SELF-SERVICE @gregsypolt
- 26. Hosted | CI/CDWithout Autoscaling 80%COST REDUCTION $3,000/yr AVERAGE CI/CD COST W/O AUTOSCALING HOSTED CI/CD PLATFORM @gregsypolt
- 27. 2 4 6 8 10 12 10:30 10:35 10:4010:45 10:50 10:55 11:00 11:05 Active Build Count (running + queued) Worker Pod Count BULK BUILD STARTS BUILDS COMPLETE COOLDOWN BEGINS COOLDOWN ENDS Auto-Scaling Workers @gregsypolt
- 28. Hosted | CI/CDPlatform With Autoscaling 93%COST REDUCTION $960/yr AVERAGE CI/CD COST WITH AUTOSCALING HOSTED CI/CD PLATFORM @gregsypolt
- 29. 4,000+AVERAGE BUILDS PERWEEK 600%BUILDS INCREASED HOSTED CI/CD PLATFORM Builds | Hosted CI/CD Platform @gregsypolt
- 30. 1.08M May 97.9% 1.36MApril 97.5% 1.17M March 98.5% Tests | Hosted CI/CD Platform HOSTED CI/CD PLATFORM Total Tests @gregsypolt
- 31. The CI/CD EvolutionPath Innovation Adoption Maintain You don’t know your limits until you push yourself @gregsypoltunsplash.com
- 32. Thank You! Remember whyyou walked in here today It’s entirely up to you push your evolution forward Nothing feels better than new knowledge @gregsypoltunsplash.com