SlideShare a Scribd company logo
Continuous Delivery
5 years later
About me
Giovanni Toraldo
Open Source Enthusiast, Java
coder, writer of the OpenNebula
book, lead developer &
co-founder at Cloudesire.com,
shooting to 2 euro coin at 36
meters with medieval crossbow
2
Monetization & Brokering
Platform for immediate
SaaSification and automated
distribution of business
applications and services
3
4
What is Continuous Delivery?
“Continuous delivery (CD) is a software engineering approach
in which teams produce software in short cycles, ensuring
that the software can be reliably released at any time. It
aims at building, testing, and releasing software faster and
more frequently. The approach helps reduce the cost, time,
and risk of delivering changes by allowing for more
incremental updates to applications in production.”
https://en.wikipedia.org/wiki/Continuous_delivery
5
6
Continuous Delivery top-level checklist
● Code quality should be kept high
○ Write tests until nausea to avoid regressions
○ Don’t Repeat Yourself
○ Keep It Simple Stupid
○ Don’t reinvent the wheel
● Automate (almost) everything
○ Time spent writing code is never lost
○ Project building must be automated
○ Test phase must be automated
○ Deploy/Release should be automated
● Team awareness (no working in silos)
○ Everyone should know what is going on
○ Monitoring
○ Metrics
7
Our development pipeline
1. Pick an open issue
2. Write code and tests in a new branch
3. Open a pull request
4. Wait for the first green build
5. Ask for a code review
6. Handle received feedback
7. Merge code into master branch
8. Have green builds on dependant projects
9. Eventually test on staging environment (especially the UX)
10. Release to end-users
8
Issue management (Github)
● One catch-all repository for high-level issues (epic, stories),
easily understandable by non-developers
● We use issue labels to mark:
○ Bug/enhancement/task
○ Affected components
○ Effort estimate
■ Light
■ Medium
■ Heavy
■ Epic
○ Project (customer) assignment
○ Workflow steps
■ Triaged
■ Ready (to be release)
■ Closed (released)
9
10
Issue management
● We plan weekly sprints, where developers pick-up new issues
and treat them as priorities (github milestones)
● Technical details are discussed on project repository
● Workflow labels:
○ Triaged: someone has looked to the issue and defined impact
○ Awaiting-feedback: more upstream information is required to proceed
○ Backend-ready: backend code is merged into master
○ Frontend-ready: frontend code is merged into master
○ Ready: issue will be into the next release
11
12
13
Pull request and code review
Code review is a revolution for code quality and team awareness
● Different eyes catch different bugs
● Push code laziness away
● Knowledge sharing
● Code changes awareness
● Asynchronous interaction between developers
14
15
Continuous Integration - Jenkins
Our previous experience with Jenkins:
● Build parallelization bound to number of nodes
○ More nodes to increase parallelization of a single project
● Isolation across projects fragile
○ Hack to use different ports for each project
● Failed builds may interferes with next builds
○ Hack to ensure everything get cleaned properly
● Toolchains needs to coexist together and maintained
○ Beefy node configuration recipes
○ Rbenv, nvm, pyenv and similar is a must
● Github PR plugin integration broke multiple times
○ Leaving us in panic
16
Continuous Integration - CircleCI
The day after all the jenkins github builds broke, we decided to
migrate CI to a paid service and keep headaches only for the code
we write
Enhancements of CircleCI over Jenkins:
● Build isolation guaranteed by the platform
● Parallelization bound to the bought plan
● No infrastructure to manage
● Project build configuration versioned into repository
● Finally a decent UI
17
CircleCI versions
CircleCI 1.0 (EOL August 31st, 2018)
● Simple builds
● Node image based on ubuntu 14.04 with all toolchains
preinstalled
CircleCI 2.0
● Multi-step builds (workflows)
● Docker based building
○ CircleCI maintained images
○ Docker hub images
● Cron-based builds (nightly)
18
19
20
21
22
23
Static code analysis - SonarQube
Why static code analysis matters:
● Catch bugs automatically
● Learn new things
● Enforce standards
With SonarQube is possible to:
● Run on every build for every project
● New warnings added on PR as comments
● Generate reports about current projects status
24
25
26
27
28
FUN: CTO “suggesting” to write good code
29
Application packaging - Docker
Started experimenting with docker since v1.0.0 (2014)
Nowadays, every project:
● Have a docker-compose.yml launched via autoenv
● Build a docker image and runs integration tests against it
● Push image to the registry as final build step
○ Build number is the image tag
● Get released via chef recipes
30
Configuration Management - Chef
Opscode Chef configuration used since day 0.
● Chef-zero (serverless)
● Provision via fabric script
○ Sourcing node ssh details from ssh-config
○ Copy packaged cookbooks via rsync
○ Run chef-zero
● Platform modules deployed as docker containers
○ Consul + registrator to access containers running on different hosts
○ Consul-template to autoregister upstream on nginx
● All cookbooks tested via test-kitchen + serverspec (migrating to
inspec)
31
Infrastructure as code - Terraform
Design, implement, and deploy infrastructure with known software
best practices:
● Code versioning
● Code reuse (modularization/abstraction)
● Code sharing
In order to achieve:
● Repeatability
● Speed
● Reliability
32
33
Automatic release of new versions
Release a new version means restart containers with updated
images: image versions are stored inside chef environment json,
modified with a hand-made script that:
● Iterate over circleci projects looking for last successful build
● Ensure that latest master dependencies are green (no regressions)
● Retrieve commit list via Github API between current version and next
release
○ Print which new commits are going into this release
○ Reference support ticket for automatic closing
● Commit version change and run deploy job (parametrized for different
envs)
34
35
Error reporting - Sentry
Error tracking platform:
● Automatic event aggregation
● Full-text search on error message, function names, stacktraces
● Link GitHub issue
● Remember if an error was already marked as fixed (regression)
36
37
38
API monitoring - updown.io
Dead-simple HTTP health check service
● Multiple locations
● Slack/Email alerts
● Beautiful report pages
● Public status pages
39
40
41
42
Team communication - Slack
Slack as a team communication tool and notification system
● #dev: developers communication, notification from github and
circleci
● #exceptions: high priority notification from sentry and updown.io
● #ops: notification from chef runs, rss from external system
status
● #marketplace: notifications generated by cloudesire platform for
a specific environment
● #bugs: issue creation/closing and deployment to production
environments notifications
43
44
45
46
47
48
Metrics - Prometheus + Grafana
Prometheus is a monitoring system / time-series database
● Server scrape and store time series data
● Metrics exporter for nodes and applications (nginx, mysql, etc)
● Client libraries to instrument code
● Alert manager to send notifications
● Node autodiscovery via consul
49
50
Centralized Logging - Graylog
Cat | grep anyone?
Impossibile to avoid when having multiple backends
Graylog is an open source log management platform
● ElasticSearch backend for logs storage/indexing
● MongoDB backend for webapp persistence
● Rsyslog TCP+SSL input
● GELF input for application logs
51
52
Future?
Things currently on our radar:
● Kubernetes/Mesos to replace chef orchestration
○ Avoid complexity in chef recipes
○ Scalable/fault-tolerant infrastructure
● Kotlin to replace Java for backend modules
○ Concise code
○ Backward-compatibility
● Full-remote developers
○ Currently working 2 days a week from home
53
Questions?
54

More Related Content

What's hot

Flash Camp Chennai - Build automation of Flex and AIR applications
Flash Camp Chennai - Build automation of Flex and AIR applicationsFlash Camp Chennai - Build automation of Flex and AIR applications
Flash Camp Chennai - Build automation of Flex and AIR applications
RIA RUI Society
 
Software Delivery in 2016 - A Continuous Delivery Approach
Software Delivery in 2016 - A Continuous Delivery ApproachSoftware Delivery in 2016 - A Continuous Delivery Approach
Software Delivery in 2016 - A Continuous Delivery Approach
Giovanni Toraldo
 
Preventing Supply Chain Attacks on Open Source Software
Preventing Supply Chain Attacks on Open Source SoftwarePreventing Supply Chain Attacks on Open Source Software
Preventing Supply Chain Attacks on Open Source Software
All Things Open
 
GITS Class #16: CI/CD (Continuous Integration & Continuous Deployment) with G...
GITS Class #16: CI/CD (Continuous Integration & Continuous Deployment) with G...GITS Class #16: CI/CD (Continuous Integration & Continuous Deployment) with G...
GITS Class #16: CI/CD (Continuous Integration & Continuous Deployment) with G...
GITS Indonesia
 
Docs or it didn’t happen
Docs or it didn’t happenDocs or it didn’t happen
Docs or it didn’t happen
All Things Open
 
ETICS supporting compliance and interoperability, Gabriele Giammatteo, Engine...
ETICS supporting compliance and interoperability, Gabriele Giammatteo, Engine...ETICS supporting compliance and interoperability, Gabriele Giammatteo, Engine...
ETICS supporting compliance and interoperability, Gabriele Giammatteo, Engine...
OW2
 
Lightweight continuous delivery for small schools
Lightweight continuous delivery for small schoolsLightweight continuous delivery for small schools
Lightweight continuous delivery for small schools
Charles Fulton
 
Expedia 3x3 presentation
Expedia 3x3 presentationExpedia 3x3 presentation
Expedia 3x3 presentation
Drew Hannay
 
CodiLime Tech Talk - Dawid Trzebiatowski i Wojciech Urbański: Opening the Flo...
CodiLime Tech Talk - Dawid Trzebiatowski i Wojciech Urbański: Opening the Flo...CodiLime Tech Talk - Dawid Trzebiatowski i Wojciech Urbański: Opening the Flo...
CodiLime Tech Talk - Dawid Trzebiatowski i Wojciech Urbański: Opening the Flo...
CodiLime
 
DevOps in realtime
DevOps in realtimeDevOps in realtime
DevOps in realtime
Andriy Samilyak
 
A brief history of automation in Software Engineering
A brief history of automation in Software EngineeringA brief history of automation in Software Engineering
A brief history of automation in Software Engineering
Georg Buske
 
OSDC 2016 - rkt and Kubernentes what's new with Container Runtimes and Orches...
OSDC 2016 - rkt and Kubernentes what's new with Container Runtimes and Orches...OSDC 2016 - rkt and Kubernentes what's new with Container Runtimes and Orches...
OSDC 2016 - rkt and Kubernentes what's new with Container Runtimes and Orches...
NETWAYS
 
Dealing with large code bases. cd ams meetup
Dealing with large code bases. cd ams meetupDealing with large code bases. cd ams meetup
Dealing with large code bases. cd ams meetup
Viktor Sadovnikov
 
Using Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and JenkinsUsing Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and Jenkins
Micael Gallego
 
Development with Qt for Windows CE
Development with Qt for Windows CEDevelopment with Qt for Windows CE
Development with Qt for Windows CE
account inactive
 
Survival of the Continuist
Survival of the ContinuistSurvival of the Continuist
Survival of the Continuist
Paul Blundell
 
FTC6 Jean-Emmanuel Rodriguez welcome Frogans Technology 6 2016/02/16
FTC6 Jean-Emmanuel Rodriguez welcome Frogans Technology 6 2016/02/16FTC6 Jean-Emmanuel Rodriguez welcome Frogans Technology 6 2016/02/16
FTC6 Jean-Emmanuel Rodriguez welcome Frogans Technology 6 2016/02/16
Organization for the Promotion, Protection and Progress of Frogans Technology
 
Gitlab CI/CD
Gitlab CI/CDGitlab CI/CD
Gitlab CI/CD
JEMLI Fathi
 
BKK16-407 AOSP Toolchain Evolution and experimental languages on AOSP
BKK16-407 AOSP Toolchain Evolution and experimental languages on AOSPBKK16-407 AOSP Toolchain Evolution and experimental languages on AOSP
BKK16-407 AOSP Toolchain Evolution and experimental languages on AOSP
Linaro
 
BKK16-310 The HiKey AOSP collaborative experience
BKK16-310 The HiKey AOSP collaborative experience BKK16-310 The HiKey AOSP collaborative experience
BKK16-310 The HiKey AOSP collaborative experience
Linaro
 

What's hot (20)

Flash Camp Chennai - Build automation of Flex and AIR applications
Flash Camp Chennai - Build automation of Flex and AIR applicationsFlash Camp Chennai - Build automation of Flex and AIR applications
Flash Camp Chennai - Build automation of Flex and AIR applications
 
Software Delivery in 2016 - A Continuous Delivery Approach
Software Delivery in 2016 - A Continuous Delivery ApproachSoftware Delivery in 2016 - A Continuous Delivery Approach
Software Delivery in 2016 - A Continuous Delivery Approach
 
Preventing Supply Chain Attacks on Open Source Software
Preventing Supply Chain Attacks on Open Source SoftwarePreventing Supply Chain Attacks on Open Source Software
Preventing Supply Chain Attacks on Open Source Software
 
GITS Class #16: CI/CD (Continuous Integration & Continuous Deployment) with G...
GITS Class #16: CI/CD (Continuous Integration & Continuous Deployment) with G...GITS Class #16: CI/CD (Continuous Integration & Continuous Deployment) with G...
GITS Class #16: CI/CD (Continuous Integration & Continuous Deployment) with G...
 
Docs or it didn’t happen
Docs or it didn’t happenDocs or it didn’t happen
Docs or it didn’t happen
 
ETICS supporting compliance and interoperability, Gabriele Giammatteo, Engine...
ETICS supporting compliance and interoperability, Gabriele Giammatteo, Engine...ETICS supporting compliance and interoperability, Gabriele Giammatteo, Engine...
ETICS supporting compliance and interoperability, Gabriele Giammatteo, Engine...
 
Lightweight continuous delivery for small schools
Lightweight continuous delivery for small schoolsLightweight continuous delivery for small schools
Lightweight continuous delivery for small schools
 
Expedia 3x3 presentation
Expedia 3x3 presentationExpedia 3x3 presentation
Expedia 3x3 presentation
 
CodiLime Tech Talk - Dawid Trzebiatowski i Wojciech Urbański: Opening the Flo...
CodiLime Tech Talk - Dawid Trzebiatowski i Wojciech Urbański: Opening the Flo...CodiLime Tech Talk - Dawid Trzebiatowski i Wojciech Urbański: Opening the Flo...
CodiLime Tech Talk - Dawid Trzebiatowski i Wojciech Urbański: Opening the Flo...
 
DevOps in realtime
DevOps in realtimeDevOps in realtime
DevOps in realtime
 
A brief history of automation in Software Engineering
A brief history of automation in Software EngineeringA brief history of automation in Software Engineering
A brief history of automation in Software Engineering
 
OSDC 2016 - rkt and Kubernentes what's new with Container Runtimes and Orches...
OSDC 2016 - rkt and Kubernentes what's new with Container Runtimes and Orches...OSDC 2016 - rkt and Kubernentes what's new with Container Runtimes and Orches...
OSDC 2016 - rkt and Kubernentes what's new with Container Runtimes and Orches...
 
Dealing with large code bases. cd ams meetup
Dealing with large code bases. cd ams meetupDealing with large code bases. cd ams meetup
Dealing with large code bases. cd ams meetup
 
Using Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and JenkinsUsing Docker to build and test in your laptop and Jenkins
Using Docker to build and test in your laptop and Jenkins
 
Development with Qt for Windows CE
Development with Qt for Windows CEDevelopment with Qt for Windows CE
Development with Qt for Windows CE
 
Survival of the Continuist
Survival of the ContinuistSurvival of the Continuist
Survival of the Continuist
 
FTC6 Jean-Emmanuel Rodriguez welcome Frogans Technology 6 2016/02/16
FTC6 Jean-Emmanuel Rodriguez welcome Frogans Technology 6 2016/02/16FTC6 Jean-Emmanuel Rodriguez welcome Frogans Technology 6 2016/02/16
FTC6 Jean-Emmanuel Rodriguez welcome Frogans Technology 6 2016/02/16
 
Gitlab CI/CD
Gitlab CI/CDGitlab CI/CD
Gitlab CI/CD
 
BKK16-407 AOSP Toolchain Evolution and experimental languages on AOSP
BKK16-407 AOSP Toolchain Evolution and experimental languages on AOSPBKK16-407 AOSP Toolchain Evolution and experimental languages on AOSP
BKK16-407 AOSP Toolchain Evolution and experimental languages on AOSP
 
BKK16-310 The HiKey AOSP collaborative experience
BKK16-310 The HiKey AOSP collaborative experience BKK16-310 The HiKey AOSP collaborative experience
BKK16-310 The HiKey AOSP collaborative experience
 

Similar to Continuous Delivery: 5 years later (Incontro DevOps 2018)

Free GitOps Workshop
Free GitOps WorkshopFree GitOps Workshop
Free GitOps Workshop
Weaveworks
 
Paris.py
Paris.pyParis.py
Programming for non tech entrepreneurs
Programming for non tech entrepreneursProgramming for non tech entrepreneurs
Programming for non tech entrepreneurs
Rodrigo Gil
 
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
Ambassador Labs
 
Netflix Open Source: Building a Distributed and Automated Open Source Program
Netflix Open Source:  Building a Distributed and Automated Open Source ProgramNetflix Open Source:  Building a Distributed and Automated Open Source Program
Netflix Open Source: Building a Distributed and Automated Open Source Program
aspyker
 
Building a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at NetflixBuilding a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at Netflix
All Things Open
 
3x3: Speeding Up Mobile Releases
3x3: Speeding Up Mobile Releases3x3: Speeding Up Mobile Releases
3x3: Speeding Up Mobile Releases
Drew Hannay
 
3x3 Speeding Up Mobile Releases
3x3 Speeding Up Mobile Releases3x3 Speeding Up Mobile Releases
3x3 Speeding Up Mobile Releases
Mobile Delivery Days
 
Delivery at Scale
Delivery at ScaleDelivery at Scale
Delivery at Scale
Agilar
 
Delivery at Scale
Delivery at ScaleDelivery at Scale
Delivery at Scale
Adrian Perreau de Pinninck
 
Nagpur Mulesoft Meetup on CICD using Jenkins
Nagpur Mulesoft Meetup on CICD using JenkinsNagpur Mulesoft Meetup on CICD using Jenkins
Nagpur Mulesoft Meetup on CICD using Jenkins
pqrs1234
 
Modern Web-site Development Pipeline
Modern Web-site Development PipelineModern Web-site Development Pipeline
Modern Web-site Development Pipeline
GlobalLogic Ukraine
 
MuleSoft_Noida_Meetup_CICD_Azure_07_May_2022.pptx
MuleSoft_Noida_Meetup_CICD_Azure_07_May_2022.pptxMuleSoft_Noida_Meetup_CICD_Azure_07_May_2022.pptx
MuleSoft_Noida_Meetup_CICD_Azure_07_May_2022.pptx
Shiva Sahu
 
Microservices Development Process at Predix.io
Microservices Development Process at Predix.ioMicroservices Development Process at Predix.io
Microservices Development Process at Predix.io
Constantine Grigel
 
The 10 Commandments of Release Engineering
The 10 Commandments of Release EngineeringThe 10 Commandments of Release Engineering
The 10 Commandments of Release Engineering
Solano Labs
 
Cloud Native Applications on Kubernetes: a DevOps Approach
Cloud Native Applications on Kubernetes: a DevOps ApproachCloud Native Applications on Kubernetes: a DevOps Approach
Cloud Native Applications on Kubernetes: a DevOps Approach
Nicola Ferraro
 
OpenStack Cinder On-Boarding Room - Vancouver Summit 2018
OpenStack Cinder On-Boarding Room - Vancouver Summit 2018OpenStack Cinder On-Boarding Room - Vancouver Summit 2018
OpenStack Cinder On-Boarding Room - Vancouver Summit 2018
Jay Bryant
 
Devoxx : being productive with JHipster
Devoxx : being productive with JHipsterDevoxx : being productive with JHipster
Devoxx : being productive with JHipster
Julien Dubois
 
Free GitOps Workshop (with Intro to Kubernetes & GitOps)
Free GitOps Workshop (with Intro to Kubernetes & GitOps)Free GitOps Workshop (with Intro to Kubernetes & GitOps)
Free GitOps Workshop (with Intro to Kubernetes & GitOps)
Weaveworks
 
OpenDataPlane Testing in Travis
OpenDataPlane Testing in TravisOpenDataPlane Testing in Travis
OpenDataPlane Testing in Travis
Dmitry Baryshkov
 

Similar to Continuous Delivery: 5 years later (Incontro DevOps 2018) (20)

Free GitOps Workshop
Free GitOps WorkshopFree GitOps Workshop
Free GitOps Workshop
 
Paris.py
Paris.pyParis.py
Paris.py
 
Programming for non tech entrepreneurs
Programming for non tech entrepreneursProgramming for non tech entrepreneurs
Programming for non tech entrepreneurs
 
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
Velocity NYC 2017: Building Resilient Microservices with Kubernetes, Docker, ...
 
Netflix Open Source: Building a Distributed and Automated Open Source Program
Netflix Open Source:  Building a Distributed and Automated Open Source ProgramNetflix Open Source:  Building a Distributed and Automated Open Source Program
Netflix Open Source: Building a Distributed and Automated Open Source Program
 
Building a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at NetflixBuilding a Distributed & Automated Open Source Program at Netflix
Building a Distributed & Automated Open Source Program at Netflix
 
3x3: Speeding Up Mobile Releases
3x3: Speeding Up Mobile Releases3x3: Speeding Up Mobile Releases
3x3: Speeding Up Mobile Releases
 
3x3 Speeding Up Mobile Releases
3x3 Speeding Up Mobile Releases3x3 Speeding Up Mobile Releases
3x3 Speeding Up Mobile Releases
 
Delivery at Scale
Delivery at ScaleDelivery at Scale
Delivery at Scale
 
Delivery at Scale
Delivery at ScaleDelivery at Scale
Delivery at Scale
 
Nagpur Mulesoft Meetup on CICD using Jenkins
Nagpur Mulesoft Meetup on CICD using JenkinsNagpur Mulesoft Meetup on CICD using Jenkins
Nagpur Mulesoft Meetup on CICD using Jenkins
 
Modern Web-site Development Pipeline
Modern Web-site Development PipelineModern Web-site Development Pipeline
Modern Web-site Development Pipeline
 
MuleSoft_Noida_Meetup_CICD_Azure_07_May_2022.pptx
MuleSoft_Noida_Meetup_CICD_Azure_07_May_2022.pptxMuleSoft_Noida_Meetup_CICD_Azure_07_May_2022.pptx
MuleSoft_Noida_Meetup_CICD_Azure_07_May_2022.pptx
 
Microservices Development Process at Predix.io
Microservices Development Process at Predix.ioMicroservices Development Process at Predix.io
Microservices Development Process at Predix.io
 
The 10 Commandments of Release Engineering
The 10 Commandments of Release EngineeringThe 10 Commandments of Release Engineering
The 10 Commandments of Release Engineering
 
Cloud Native Applications on Kubernetes: a DevOps Approach
Cloud Native Applications on Kubernetes: a DevOps ApproachCloud Native Applications on Kubernetes: a DevOps Approach
Cloud Native Applications on Kubernetes: a DevOps Approach
 
OpenStack Cinder On-Boarding Room - Vancouver Summit 2018
OpenStack Cinder On-Boarding Room - Vancouver Summit 2018OpenStack Cinder On-Boarding Room - Vancouver Summit 2018
OpenStack Cinder On-Boarding Room - Vancouver Summit 2018
 
Devoxx : being productive with JHipster
Devoxx : being productive with JHipsterDevoxx : being productive with JHipster
Devoxx : being productive with JHipster
 
Free GitOps Workshop (with Intro to Kubernetes & GitOps)
Free GitOps Workshop (with Intro to Kubernetes & GitOps)Free GitOps Workshop (with Intro to Kubernetes & GitOps)
Free GitOps Workshop (with Intro to Kubernetes & GitOps)
 
OpenDataPlane Testing in Travis
OpenDataPlane Testing in TravisOpenDataPlane Testing in Travis
OpenDataPlane Testing in Travis
 

More from Giovanni Toraldo

About code review and BUGS
About code review and BUGSAbout code review and BUGS
About code review and BUGS
Giovanni Toraldo
 
Introduction to Traefik
Introduction to TraefikIntroduction to Traefik
Introduction to Traefik
Giovanni Toraldo
 
Managing GCP Projects with Terraform (devfest Pisa 2018)
Managing GCP Projects with Terraform (devfest Pisa 2018)Managing GCP Projects with Terraform (devfest Pisa 2018)
Managing GCP Projects with Terraform (devfest Pisa 2018)
Giovanni Toraldo
 
From zero to Docker
From zero to DockerFrom zero to Docker
From zero to Docker
Giovanni Toraldo
 
Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy
Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) ItalyClustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy
Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy
Giovanni Toraldo
 
Docker in production service discovery with consul - road to opscon 2015
Docker in production  service discovery with consul - road to opscon 2015Docker in production  service discovery with consul - road to opscon 2015
Docker in production service discovery with consul - road to opscon 2015
Giovanni Toraldo
 
When Docker ends, Chef begins ~ #idi2015 Incontro DevOps Italia
When Docker ends, Chef begins ~ #idi2015 Incontro DevOps ItaliaWhen Docker ends, Chef begins ~ #idi2015 Incontro DevOps Italia
When Docker ends, Chef begins ~ #idi2015 Incontro DevOps Italia
Giovanni Toraldo
 
Easy applications deployment on OpenStack clouds
Easy applications deployment on OpenStack cloudsEasy applications deployment on OpenStack clouds
Easy applications deployment on OpenStack clouds
Giovanni Toraldo
 
Introduction to Continuous Delivery
Introduction to Continuous DeliveryIntroduction to Continuous Delivery
Introduction to Continuous Delivery
Giovanni Toraldo
 
ClouDesire @ Italian DevOps Initiative 2013 #idi2013
ClouDesire @ Italian DevOps Initiative 2013 #idi2013ClouDesire @ Italian DevOps Initiative 2013 #idi2013
ClouDesire @ Italian DevOps Initiative 2013 #idi2013
Giovanni Toraldo
 
Cloud Computing in practice with OpenNebula ~ Develer workshop 2012
Cloud Computing in practice with OpenNebula ~ Develer workshop 2012Cloud Computing in practice with OpenNebula ~ Develer workshop 2012
Cloud Computing in practice with OpenNebula ~ Develer workshop 2012
Giovanni Toraldo
 
An introduction to cloud computing
An introduction to cloud computingAn introduction to cloud computing
An introduction to cloud computing
Giovanni Toraldo
 
EVA Florence 2012 ~ Open low-cost HA cluster cloud
EVA Florence 2012 ~ Open low-cost HA cluster cloudEVA Florence 2012 ~ Open low-cost HA cluster cloud
EVA Florence 2012 ~ Open low-cost HA cluster cloud
Giovanni Toraldo
 
Open@BNCF
Open@BNCFOpen@BNCF
Open@BNCF
Giovanni Toraldo
 

More from Giovanni Toraldo (14)

About code review and BUGS
About code review and BUGSAbout code review and BUGS
About code review and BUGS
 
Introduction to Traefik
Introduction to TraefikIntroduction to Traefik
Introduction to Traefik
 
Managing GCP Projects with Terraform (devfest Pisa 2018)
Managing GCP Projects with Terraform (devfest Pisa 2018)Managing GCP Projects with Terraform (devfest Pisa 2018)
Managing GCP Projects with Terraform (devfest Pisa 2018)
 
From zero to Docker
From zero to DockerFrom zero to Docker
From zero to Docker
 
Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy
Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) ItalyClustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy
Clustering with Docker Swarm - Dockerops 2016 @ Cento (FE) Italy
 
Docker in production service discovery with consul - road to opscon 2015
Docker in production  service discovery with consul - road to opscon 2015Docker in production  service discovery with consul - road to opscon 2015
Docker in production service discovery with consul - road to opscon 2015
 
When Docker ends, Chef begins ~ #idi2015 Incontro DevOps Italia
When Docker ends, Chef begins ~ #idi2015 Incontro DevOps ItaliaWhen Docker ends, Chef begins ~ #idi2015 Incontro DevOps Italia
When Docker ends, Chef begins ~ #idi2015 Incontro DevOps Italia
 
Easy applications deployment on OpenStack clouds
Easy applications deployment on OpenStack cloudsEasy applications deployment on OpenStack clouds
Easy applications deployment on OpenStack clouds
 
Introduction to Continuous Delivery
Introduction to Continuous DeliveryIntroduction to Continuous Delivery
Introduction to Continuous Delivery
 
ClouDesire @ Italian DevOps Initiative 2013 #idi2013
ClouDesire @ Italian DevOps Initiative 2013 #idi2013ClouDesire @ Italian DevOps Initiative 2013 #idi2013
ClouDesire @ Italian DevOps Initiative 2013 #idi2013
 
Cloud Computing in practice with OpenNebula ~ Develer workshop 2012
Cloud Computing in practice with OpenNebula ~ Develer workshop 2012Cloud Computing in practice with OpenNebula ~ Develer workshop 2012
Cloud Computing in practice with OpenNebula ~ Develer workshop 2012
 
An introduction to cloud computing
An introduction to cloud computingAn introduction to cloud computing
An introduction to cloud computing
 
EVA Florence 2012 ~ Open low-cost HA cluster cloud
EVA Florence 2012 ~ Open low-cost HA cluster cloudEVA Florence 2012 ~ Open low-cost HA cluster cloud
EVA Florence 2012 ~ Open low-cost HA cluster cloud
 
Open@BNCF
Open@BNCFOpen@BNCF
Open@BNCF
 

Recently uploaded

Il Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazioneIl Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazione
confluent
 
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
confluent
 
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
revolutionary575
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
neshakor5152
 
Amadeus Travel API, Amadeus Booking API, Amadeus GDS
Amadeus Travel API, Amadeus Booking API, Amadeus GDSAmadeus Travel API, Amadeus Booking API, Amadeus GDS
Amadeus Travel API, Amadeus Booking API, Amadeus GDS
aadhiyaeliza
 
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
singhlata50dh
 
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdfAI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
Daniel Zivkovic
 
Hotel Management Software Development Company
Hotel Management Software Development CompanyHotel Management Software Development Company
Hotel Management Software Development Company
XongoLab Technologies LLP
 
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Deliverybangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
Srinivas Dukka
 
Predicting Test Results without Execution (FSE 2024)
Predicting Test Results without Execution (FSE 2024)Predicting Test Results without Execution (FSE 2024)
Predicting Test Results without Execution (FSE 2024)
andrehoraa
 
當測試開始左移
當測試開始左移當測試開始左移
當測試開始左移
Jersey (CHE-PING) Su
 
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
3610stuck
 
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
45unexpected
 
Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...
Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...
Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...
simran hot girls
 
Crafting highly scalable and performant Modern Data Platforms
Crafting highly scalable and performant Modern Data PlatformsCrafting highly scalable and performant Modern Data Platforms
Crafting highly scalable and performant Modern Data Platforms
Sameer Paradkar
 
Applitools Autonomous 2.0 Sneak Peek.pdf
Applitools Autonomous 2.0 Sneak Peek.pdfApplitools Autonomous 2.0 Sneak Peek.pdf
Applitools Autonomous 2.0 Sneak Peek.pdf
Applitools
 
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
norina2645
 
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
dream girl
 
Tour and travel website management in odoo,
Tour and travel website management in odoo,Tour and travel website management in odoo,
Tour and travel website management in odoo,
Axis Technolabs
 

Recently uploaded (20)

Il Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazioneIl Data Streaming per un’AI real-time di nuova generazione
Il Data Streaming per un’AI real-time di nuova generazione
 
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
Unleashing the Future: Building a Scalable and Up-to-Date GenAI Chatbot with ...
 
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
Busty Girls Call Mumbai 9930245274 Unlimited Short Providing Girls Service Av...
 
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in CityGirls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
Girls Call Mysore 000XX00000 Provide Best And Top Girl Service And No1 in City
 
Amadeus Travel API, Amadeus Booking API, Amadeus GDS
Amadeus Travel API, Amadeus Booking API, Amadeus GDSAmadeus Travel API, Amadeus Booking API, Amadeus GDS
Amadeus Travel API, Amadeus Booking API, Amadeus GDS
 
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
High Girls Call Chennai 000XX00000 Provide Best And Top Girl Service And No1 ...
 
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdfAI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
AI - Your Startup Sidekick (Leveraging AI to Bootstrap a Lean Startup).pdf
 
Hotel Management Software Development Company
Hotel Management Software Development CompanyHotel Management Software Development Company
Hotel Management Software Development Company
 
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Deliverybangalore Girls call  👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
bangalore Girls call 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
AWS DevOps-Tutorial CHANAKYA SRIYAN DUKKA.
 
Predicting Test Results without Execution (FSE 2024)
Predicting Test Results without Execution (FSE 2024)Predicting Test Results without Execution (FSE 2024)
Predicting Test Results without Execution (FSE 2024)
 
當測試開始左移
當測試開始左移當測試開始左移
當測試開始左移
 
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
Mumbai Girls Call Mumbai 🎈🔥9930687706 🔥💋🎈 Provide Best And Top Girl Service A...
 
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
Celebrity Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Servic...
 
Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...
Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...
Girls Call Jogeshwari 9967584737 Provide Best And Top Girl Service And No1 in...
 
Crafting highly scalable and performant Modern Data Platforms
Crafting highly scalable and performant Modern Data PlatformsCrafting highly scalable and performant Modern Data Platforms
Crafting highly scalable and performant Modern Data Platforms
 
Applitools Autonomous 2.0 Sneak Peek.pdf
Applitools Autonomous 2.0 Sneak Peek.pdfApplitools Autonomous 2.0 Sneak Peek.pdf
Applitools Autonomous 2.0 Sneak Peek.pdf
 
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
Celebrity Girls Call Mumbai 🛵🚡9910780858 💃 Choose Best And Top Girl Service A...
 
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
Russian Girls Call Mumbai 🛵🚡9833363713 💃 Choose Best And Top Girl Service And...
 
Tour and travel website management in odoo,
Tour and travel website management in odoo,Tour and travel website management in odoo,
Tour and travel website management in odoo,
 

Continuous Delivery: 5 years later (Incontro DevOps 2018)

  • 2. About me Giovanni Toraldo Open Source Enthusiast, Java coder, writer of the OpenNebula book, lead developer & co-founder at Cloudesire.com, shooting to 2 euro coin at 36 meters with medieval crossbow 2
  • 3. Monetization & Brokering Platform for immediate SaaSification and automated distribution of business applications and services 3
  • 4. 4
  • 5. What is Continuous Delivery? “Continuous delivery (CD) is a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time. It aims at building, testing, and releasing software faster and more frequently. The approach helps reduce the cost, time, and risk of delivering changes by allowing for more incremental updates to applications in production.” https://en.wikipedia.org/wiki/Continuous_delivery 5
  • 6. 6
  • 7. Continuous Delivery top-level checklist ● Code quality should be kept high ○ Write tests until nausea to avoid regressions ○ Don’t Repeat Yourself ○ Keep It Simple Stupid ○ Don’t reinvent the wheel ● Automate (almost) everything ○ Time spent writing code is never lost ○ Project building must be automated ○ Test phase must be automated ○ Deploy/Release should be automated ● Team awareness (no working in silos) ○ Everyone should know what is going on ○ Monitoring ○ Metrics 7
  • 8. Our development pipeline 1. Pick an open issue 2. Write code and tests in a new branch 3. Open a pull request 4. Wait for the first green build 5. Ask for a code review 6. Handle received feedback 7. Merge code into master branch 8. Have green builds on dependant projects 9. Eventually test on staging environment (especially the UX) 10. Release to end-users 8
  • 9. Issue management (Github) ● One catch-all repository for high-level issues (epic, stories), easily understandable by non-developers ● We use issue labels to mark: ○ Bug/enhancement/task ○ Affected components ○ Effort estimate ■ Light ■ Medium ■ Heavy ■ Epic ○ Project (customer) assignment ○ Workflow steps ■ Triaged ■ Ready (to be release) ■ Closed (released) 9
  • 10. 10
  • 11. Issue management ● We plan weekly sprints, where developers pick-up new issues and treat them as priorities (github milestones) ● Technical details are discussed on project repository ● Workflow labels: ○ Triaged: someone has looked to the issue and defined impact ○ Awaiting-feedback: more upstream information is required to proceed ○ Backend-ready: backend code is merged into master ○ Frontend-ready: frontend code is merged into master ○ Ready: issue will be into the next release 11
  • 12. 12
  • 13. 13
  • 14. Pull request and code review Code review is a revolution for code quality and team awareness ● Different eyes catch different bugs ● Push code laziness away ● Knowledge sharing ● Code changes awareness ● Asynchronous interaction between developers 14
  • 15. 15
  • 16. Continuous Integration - Jenkins Our previous experience with Jenkins: ● Build parallelization bound to number of nodes ○ More nodes to increase parallelization of a single project ● Isolation across projects fragile ○ Hack to use different ports for each project ● Failed builds may interferes with next builds ○ Hack to ensure everything get cleaned properly ● Toolchains needs to coexist together and maintained ○ Beefy node configuration recipes ○ Rbenv, nvm, pyenv and similar is a must ● Github PR plugin integration broke multiple times ○ Leaving us in panic 16
  • 17. Continuous Integration - CircleCI The day after all the jenkins github builds broke, we decided to migrate CI to a paid service and keep headaches only for the code we write Enhancements of CircleCI over Jenkins: ● Build isolation guaranteed by the platform ● Parallelization bound to the bought plan ● No infrastructure to manage ● Project build configuration versioned into repository ● Finally a decent UI 17
  • 18. CircleCI versions CircleCI 1.0 (EOL August 31st, 2018) ● Simple builds ● Node image based on ubuntu 14.04 with all toolchains preinstalled CircleCI 2.0 ● Multi-step builds (workflows) ● Docker based building ○ CircleCI maintained images ○ Docker hub images ● Cron-based builds (nightly) 18
  • 19. 19
  • 20. 20
  • 21. 21
  • 22. 22
  • 23. 23
  • 24. Static code analysis - SonarQube Why static code analysis matters: ● Catch bugs automatically ● Learn new things ● Enforce standards With SonarQube is possible to: ● Run on every build for every project ● New warnings added on PR as comments ● Generate reports about current projects status 24
  • 25. 25
  • 26. 26
  • 27. 27
  • 28. 28
  • 29. FUN: CTO “suggesting” to write good code 29
  • 30. Application packaging - Docker Started experimenting with docker since v1.0.0 (2014) Nowadays, every project: ● Have a docker-compose.yml launched via autoenv ● Build a docker image and runs integration tests against it ● Push image to the registry as final build step ○ Build number is the image tag ● Get released via chef recipes 30
  • 31. Configuration Management - Chef Opscode Chef configuration used since day 0. ● Chef-zero (serverless) ● Provision via fabric script ○ Sourcing node ssh details from ssh-config ○ Copy packaged cookbooks via rsync ○ Run chef-zero ● Platform modules deployed as docker containers ○ Consul + registrator to access containers running on different hosts ○ Consul-template to autoregister upstream on nginx ● All cookbooks tested via test-kitchen + serverspec (migrating to inspec) 31
  • 32. Infrastructure as code - Terraform Design, implement, and deploy infrastructure with known software best practices: ● Code versioning ● Code reuse (modularization/abstraction) ● Code sharing In order to achieve: ● Repeatability ● Speed ● Reliability 32
  • 33. 33
  • 34. Automatic release of new versions Release a new version means restart containers with updated images: image versions are stored inside chef environment json, modified with a hand-made script that: ● Iterate over circleci projects looking for last successful build ● Ensure that latest master dependencies are green (no regressions) ● Retrieve commit list via Github API between current version and next release ○ Print which new commits are going into this release ○ Reference support ticket for automatic closing ● Commit version change and run deploy job (parametrized for different envs) 34
  • 35. 35
  • 36. Error reporting - Sentry Error tracking platform: ● Automatic event aggregation ● Full-text search on error message, function names, stacktraces ● Link GitHub issue ● Remember if an error was already marked as fixed (regression) 36
  • 37. 37
  • 38. 38
  • 39. API monitoring - updown.io Dead-simple HTTP health check service ● Multiple locations ● Slack/Email alerts ● Beautiful report pages ● Public status pages 39
  • 40. 40
  • 41. 41
  • 42. 42
  • 43. Team communication - Slack Slack as a team communication tool and notification system ● #dev: developers communication, notification from github and circleci ● #exceptions: high priority notification from sentry and updown.io ● #ops: notification from chef runs, rss from external system status ● #marketplace: notifications generated by cloudesire platform for a specific environment ● #bugs: issue creation/closing and deployment to production environments notifications 43
  • 44. 44
  • 45. 45
  • 46. 46
  • 47. 47
  • 48. 48
  • 49. Metrics - Prometheus + Grafana Prometheus is a monitoring system / time-series database ● Server scrape and store time series data ● Metrics exporter for nodes and applications (nginx, mysql, etc) ● Client libraries to instrument code ● Alert manager to send notifications ● Node autodiscovery via consul 49
  • 50. 50
  • 51. Centralized Logging - Graylog Cat | grep anyone? Impossibile to avoid when having multiple backends Graylog is an open source log management platform ● ElasticSearch backend for logs storage/indexing ● MongoDB backend for webapp persistence ● Rsyslog TCP+SSL input ● GELF input for application logs 51
  • 52. 52
  • 53. Future? Things currently on our radar: ● Kubernetes/Mesos to replace chef orchestration ○ Avoid complexity in chef recipes ○ Scalable/fault-tolerant infrastructure ● Kotlin to replace Java for backend modules ○ Concise code ○ Backward-compatibility ● Full-remote developers ○ Currently working 2 days a week from home 53