ODP, PDF77 views

Decentralizing CI/CD Pipelines (In Go)

This document discusses decentralizing CI/CD pipelines using Go. It provides examples of using the Holocron tool to add a new service and environment. The presentation introduces Holocron for decentralized CI/CD and demonstrates commands for adding a service and environment to manage builds and deployments across multiple regions.

Software◦

Decentralizing CI/CD Pipelines (In Go)

1.
Gophercon 2018 NeilS Primmer @neilprimmer Decentralizing CI/CD Pipelines Using Go
2.
Gophercon 2018 NeilS Primmer @neilprimmer Logo Provided by the Jenkins Project (https://jenkins.io)
3.
Gophercon 2018 NeilS Primmer @neilprimmer
4.
Gophercon 2018 NeilS Primmer @neilprimmer $ curl -X PUT holocron.local:8080/v1/services -H 'Content-type: application/json' -d '{"name":"gopherone"}' {"name":"gopherone","branches": ["master"]}
5.
Gophercon 2018 NeilS Primmer @neilprimmer $ curl -X POST holocron.local:8080/v1/environments -H 'Content-type: application/json' -d '{ "name": "gophercon", "account": "dev", "gate": false, "region": "us-east-1", "deploy_class": "dev", "protected": false}' {"name": "gophercon", "account_id": 1, "region_id": 1, "gate": false, "services": [], "branches": []}
6.
Gophercon 2018 NeilS Primmer @neilprimmer Golang Gopher by Renee French
7.
Gophercon 2018 NeilS Primmer @neilprimmer $holocron --local addService --servicegophertwo {"nam e":"gophertwo","branches":["m aster"]}
8.
Gophercon 2018 NeilS Primmer @neilprimmer $holocron--local addEnvironm ent --environm entgophercon-sandiego --accountdev --region us-east-1 --deployClassdev {"nam e":"gophercon-sandiego","account_id":1,"region_id":1,"gate": false,"services":[],"branches":[]}
9.
Gophercon 2018 NeilS Primmer @neilprimmer What’s Next?
10.
Gophercon 2018 NeilS Primmer @neilprimmer What’s Next?

Editor's Notes

#2 Hi Everybody! My name is Neil Primmer, and I’m a Senior Software Developer, on the Developer Experience team, at PIX System. Over the past three years, I’ve worked on a number of projects, but the largest and most instrumental changes I’ve been a part of have been the evolution of our CI/CD Pipelines! So, I’m going to talk a little bit today about how this evolution has gone for us, starting from the first generation of our build systems, through what we’re doing now for our future.
#3 Show of hands, who knows this gentleman? Well, at PIX, and from what I’ve been able to gather, many other places, this guy has been a force for both good and ill. He provides solutions to problems, but also ample opportunity for overengineering and frustration.
#4 So, we’ve been able to iterate on our CI/CD pipeline. Originally, it was all click-button running jobs, where we built a single artifact and manually deployed it. Then, we used webhook integrations that triggered the job to run on a push, so that we could deploy from ‘master’ and test on other branches. Now, we’ve been able to integrate numerous microservices to help add more power to this pipeline, such as automating branch-to-environment deployments. One of the biggest factors that have allowed us to make this work is our Microservice ‘Holocron’, which is responsible for storing associations between our different internal and external software products, the representations of those products (generally through git branches or git tags) and the different environments that our development teams use. Holocron has allowed us to simplify dedicated development environments for new features and products easily.
#5 Now, this is the point at which over-engineering comes into play. We have powerful resources and the tooling to use them. But what about the developers’ ability to develop and manage their services? Well, in the beginning, there was curl.
#6 Not so bad for a simple call, just registering a new service. Well, let’s look at registering an environment, with much more necessary metadata.
#7 To help address this, we provided tooling that replaces parts of the development process with utilities that aren’t strictly tied to running as part of Jenkins, and not only make these tools available to developers, but make them what Jenkins uses for the same purpose. We chose Go for this for three reasons: We can deliver single-binary artifacts to developers and Jenkins alike, removing the headache of dependency resolution just for a command line tool. We can easily cross-compile for developers who use Linux, MacOS, or Windows, removing the need to manage different build chains for individual deliverable artifacts.
#8 So now, adding a new service is slightly simpler.
#9 But now, we have a tool that is able to utilize safe conditional defaults (such as Development environments not needing approval for deployment) and toss managing the JSON syntax out the window as well.
#10 With these advances in place, we are intending to deliver a self-service experience for every developer at PIX, allowing them to iterate in their development environments significantly quicker. Once we’ve been able to replicate the core functionality that Jenkins provides, we expect that a developer should be able to publish their code to an environment independently, even automatically while working, using additional tooling such as Git commit hooks or IDE integrations. By this time next year, we intend for not only that PIX developers should be able to build and deploy as completely disconnected steps in their development workflow, but also that they should be able to do both without utilizing Jenkins directly at all.

Decentralizing CI/CD Pipelines (In Go)

More Related Content

What's hot

Similar to Decentralizing CI/CD Pipelines (In Go)

Recently uploaded

Decentralizing CI/CD Pipelines (In Go)

Editor's Notes