Stop wasting time
Automate your deployments
Jerry Jalava
Cloud GDE Helsinki
@W_I
“Companies waste $300 Billion of
developer productivity every year”
“Developers spend 42% of their time on
code maintenance”
- Stripe (The Developer Coefficient study)
Using CI/CD
Saves you time and money, brings reliability and trust
Essentially it means this
Picture from "Setting up GitLab CI/CD for Android projects"
Multiple options
Landscape of CI/CD tools
The number of different CI/CD tools and solutions have grown rapidly in
the past years
CNCF Product listing today
● Source code location(s)
Github, Gitlab, Cloud Source
● Target systems
Kubernetes, Cloud Run, VMs
● Target Environments
Development, Testing,
Staging, QA, Production
Your solution needs may vary
Not all projects are created equal
● Code structure
Multiple repos, Monorepo
● Deployment tooling
Helm charts, Chef, etc.
● And so on...
● Keep your Pipelines fast
● Run the Fastest Tests Early
● Run tests locally before committing
● Think about your branches
● Secure and Isolate your CI/CD environment
● Allow Production deployments only through CI/CD
● If the process is painful, you are doing it wrong!
Some key things to consider
Your solution needs may vary
Demo time
Let’s start with Cloud Build
Tools we will be using
Google Cloud Build
Cloud Build lets you build software quickly across all languages. Get
complete control over defining custom workflows for building, testing,
and deploying across multiple environments such as VMs, serverless,
Kubernetes, or Firebase.
Pricing: 120 min/day free (10 concurrent builds), $0.003 per minute
https://cloud.google.com/cloud-build
https://github.com/GoogleCloudPlatform/cloud-builders-community
Performance tweaks
Best practices for speeding up builds
Build leaner containers
https://cloud.google.com/cloud-build/docs/building-leaner-containers
Use Kaniko cache (in demo)
Use a cached Docker image (in demo)
Use custom virtual machine sizes (Defaults to 2 high-CPU VMs)
Avoid uploading of unnecessary files (.gcloudignore)
Architecture
# Create Build project and Enable Billing
$ gcloud projects create $BUILD_PROJECT_ID
$ gcloud --quiet beta billing projects link $BUILD_PROJECT_ID --billing-account $BILLING_ID
# Enable APIs in the build project
$ ENABLE_APIS=(
"cloudresourcemanager.googleapis.com" 
"servicemanagement.googleapis.com" 
"sourcerepo.googleapis.com" 
"cloudbuild.googleapis.com" 
"run.googleapis.com" 
"containerregistry.googleapis.com"
)
$ gcloud services enable --project=$BUILD_PROJECT_ID ${ENABLE_APIS[@]}
Preparations
DEMO
https://github.com/jerryjj/gdg-cicd-presentation
demos/cloudbuild-std/README.md & demos/cb-trigger-demo/README.md
Deploy to environment
Google Cloud Repository, Google Cloud Run
- Prepare projects
- Setup repository
- Setup Cloud Trigger
- Check the Magic
Architecture
# Create Deploy project and Enable Billing
$ gcloud projects create $DEPLOY_PROJECT_ID
$ gcloud --quiet beta billing projects link $DEPLOY_PROJECT_ID --billing-account $BILLING_ID
# Enable APIs in the build project
$ ENABLE_APIS=(
"cloudresourcemanager.googleapis.com" 
"compute.googleapis.com" 
"container.googleapis.com" 
"run.googleapis.com" 
"storage-api.googleapis.com" 
"servicenetworking.googleapis.com"
)
$ gcloud services enable --project=$DEPLOY_PROJECT_ID ${ENABLE_APIS[@]}
Preparations
# Create Cloud Repository
$ gcloud source repos create cb-deploy-run 
--project $BUILD_PROJECT_ID
# Allow Cloud Run to fetch containers from Build Project
$ DEPLOY_PROJECT_NUMBER=$(gcloud projects describe $DEPLOY_PROJECT_ID --format='value(projectNumber)')
$ RUN_SA_EMAIL="service-$DEPLOY_PROJECT_NUMBER@serverless-robot-prod.iam.gserviceaccount.com"
$ BUCKET_PATH="gs://eu.artifacts.$BUILD_PROJECT_ID.appspot.com"
$ gsutil iam ch serviceAccount:$RUN_SA_EMAIL:objectViewer $BUCKET_PATH
# Allow Cloud Build to deploy to Cloud Run on Deploy Project
$ BUILD_PROJECT_NUMBER=$(gcloud projects describe $BUILD_PROJECT_ID --format='value(projectNumber)')
$ CB_SA_EMAIL="$BUILD_PROJECT_NUMBER@cloudbuild.gserviceaccount.com"
$ COMPUTE_SA_EMAIL="$DEPLOY_PROJECT_NUMBER-compute@developer.gserviceaccount.com"
$ gcloud projects add-iam-policy-binding $DEPLOY_PROJECT_ID 
--member=serviceAccount:$CB_SA_EMAIL 
--role roles/run.admin
$ gcloud iam service-accounts add-iam-policy-binding 
$COMPUTE_SA_EMAIL 
--project $DEPLOY_PROJECT_ID 
--member="serviceAccount:$CB_SA_EMAIL" --role=roles/iam.serviceAccountUser
Preparations
DEMO
https://github.com/jerryjj/gdg-cicd-presentation
demos/cb-deploy-run/README.md
Deploy to environment
Google Cloud Repository, Google Kubernetes Engine
- Prepare projects
- Setup repository
- Setup Cloud Trigger
- Check the Magic
Architecture
# Create GKE Cluster
$ gcloud beta container clusters create "demo-cluster" 
--project $DEPLOY_PROJECT_ID 
--zone $COMPUTE_ZONE...
# Set yourself as the cluster admin
$ export CONTEXT=`kubectl config view | awk '{print $2}' | grep "demo-cluster" | tail -n 1`
$ ACCOUNT=$(gcloud info --format='value(config.account)')
$ NAME=$(echo "${ACCOUNT%@*}")
$ kubectl --context $CONTEXT create clusterrolebinding $NAME-cluster-admin-binding 
--clusterrole=cluster-admin --user=$ACCOUNT
Preparations
# Create Cloud Repository
$ gcloud source repos create cb-deploy-k8s 
--project $BUILD_PROJECT_ID
# Allow GKE to fetch containers from Build Project
$ DEPLOY_PROJECT_NUMBER=$(gcloud projects describe $DEPLOY_PROJECT_ID --format='value(projectNumber)')
$ GKE_SA_EMAIL="$DEPLOY_PROJECT_NUMBER-compute@developer.gserviceaccount.com"
$ BUCKET_PATH="gs://eu.artifacts.$BUILD_PROJECT_ID.appspot.com"
$ gsutil iam ch serviceAccount:$GKE_SA_EMAIL:objectViewer $BUCKET_PATH
# Allow Cloud Build to deploy to GKE on Deploy Project
$ BUILD_PROJECT_NUMBER=$(gcloud projects describe $BUILD_PROJECT_ID --format='value(projectNumber)')
$ CB_SA_EMAIL="$BUILD_PROJECT_NUMBER@cloudbuild.gserviceaccount.com"
$ gcloud projects add-iam-policy-binding $DEPLOY_PROJECT_ID 
--member=serviceAccount:$CB_SA_EMAIL 
--role roles/container.developer
Preparations
DEMO
https://github.com/jerryjj/gdg-cicd-presentation
demos/cb-deploy-k8s/README.md
There is a lot more you could do
Eg. GitOps-style continuous delivery with Cloud Build
https://cloud.google.com/kubernetes-engine/docs/tutorials/gitops-cloud-build
Github Triggers
Using the new App Triggers
DEMO
https://github.com/jerryjj/gdg-cicd-presentation
demos/cb-deploy-k8s/GITHUP_APP.md
This is only the
beginning
We have just scratched the surface
Some of the solutions
I suggest looking into
- Tekton https://tekton.dev/
- Jenkins X https://jenkins-x.io/
- Gitlab Kubernetes Executor
- Github Actions https://github.com/features/actions
- Flux https://docs.fluxcd.io/en/latest/
- Flagger https://flagger.app/
Thank You!
Jerry Jalava
Cloud GDE Helsinki
@W_I

Ci/CD - Stop wasting time, Automate your deployments

  • 1.
    Stop wasting time Automateyour deployments Jerry Jalava Cloud GDE Helsinki @W_I
  • 2.
    “Companies waste $300Billion of developer productivity every year” “Developers spend 42% of their time on code maintenance” - Stripe (The Developer Coefficient study)
  • 3.
    Using CI/CD Saves youtime and money, brings reliability and trust
  • 4.
    Essentially it meansthis Picture from "Setting up GitLab CI/CD for Android projects"
  • 5.
    Multiple options Landscape ofCI/CD tools The number of different CI/CD tools and solutions have grown rapidly in the past years
  • 6.
  • 7.
    ● Source codelocation(s) Github, Gitlab, Cloud Source ● Target systems Kubernetes, Cloud Run, VMs ● Target Environments Development, Testing, Staging, QA, Production Your solution needs may vary Not all projects are created equal ● Code structure Multiple repos, Monorepo ● Deployment tooling Helm charts, Chef, etc. ● And so on...
  • 8.
    ● Keep yourPipelines fast ● Run the Fastest Tests Early ● Run tests locally before committing ● Think about your branches ● Secure and Isolate your CI/CD environment ● Allow Production deployments only through CI/CD ● If the process is painful, you are doing it wrong! Some key things to consider Your solution needs may vary
  • 9.
    Demo time Let’s startwith Cloud Build
  • 10.
    Tools we willbe using Google Cloud Build Cloud Build lets you build software quickly across all languages. Get complete control over defining custom workflows for building, testing, and deploying across multiple environments such as VMs, serverless, Kubernetes, or Firebase. Pricing: 120 min/day free (10 concurrent builds), $0.003 per minute https://cloud.google.com/cloud-build https://github.com/GoogleCloudPlatform/cloud-builders-community
  • 11.
    Performance tweaks Best practicesfor speeding up builds Build leaner containers https://cloud.google.com/cloud-build/docs/building-leaner-containers Use Kaniko cache (in demo) Use a cached Docker image (in demo) Use custom virtual machine sizes (Defaults to 2 high-CPU VMs) Avoid uploading of unnecessary files (.gcloudignore)
  • 12.
  • 13.
    # Create Buildproject and Enable Billing $ gcloud projects create $BUILD_PROJECT_ID $ gcloud --quiet beta billing projects link $BUILD_PROJECT_ID --billing-account $BILLING_ID # Enable APIs in the build project $ ENABLE_APIS=( "cloudresourcemanager.googleapis.com" "servicemanagement.googleapis.com" "sourcerepo.googleapis.com" "cloudbuild.googleapis.com" "run.googleapis.com" "containerregistry.googleapis.com" ) $ gcloud services enable --project=$BUILD_PROJECT_ID ${ENABLE_APIS[@]} Preparations
  • 14.
  • 15.
    Deploy to environment GoogleCloud Repository, Google Cloud Run - Prepare projects - Setup repository - Setup Cloud Trigger - Check the Magic
  • 16.
  • 17.
    # Create Deployproject and Enable Billing $ gcloud projects create $DEPLOY_PROJECT_ID $ gcloud --quiet beta billing projects link $DEPLOY_PROJECT_ID --billing-account $BILLING_ID # Enable APIs in the build project $ ENABLE_APIS=( "cloudresourcemanager.googleapis.com" "compute.googleapis.com" "container.googleapis.com" "run.googleapis.com" "storage-api.googleapis.com" "servicenetworking.googleapis.com" ) $ gcloud services enable --project=$DEPLOY_PROJECT_ID ${ENABLE_APIS[@]} Preparations
  • 18.
    # Create CloudRepository $ gcloud source repos create cb-deploy-run --project $BUILD_PROJECT_ID # Allow Cloud Run to fetch containers from Build Project $ DEPLOY_PROJECT_NUMBER=$(gcloud projects describe $DEPLOY_PROJECT_ID --format='value(projectNumber)') $ RUN_SA_EMAIL="service-$DEPLOY_PROJECT_NUMBER@serverless-robot-prod.iam.gserviceaccount.com" $ BUCKET_PATH="gs://eu.artifacts.$BUILD_PROJECT_ID.appspot.com" $ gsutil iam ch serviceAccount:$RUN_SA_EMAIL:objectViewer $BUCKET_PATH # Allow Cloud Build to deploy to Cloud Run on Deploy Project $ BUILD_PROJECT_NUMBER=$(gcloud projects describe $BUILD_PROJECT_ID --format='value(projectNumber)') $ CB_SA_EMAIL="$BUILD_PROJECT_NUMBER@cloudbuild.gserviceaccount.com" $ COMPUTE_SA_EMAIL="$DEPLOY_PROJECT_NUMBER-compute@developer.gserviceaccount.com" $ gcloud projects add-iam-policy-binding $DEPLOY_PROJECT_ID --member=serviceAccount:$CB_SA_EMAIL --role roles/run.admin $ gcloud iam service-accounts add-iam-policy-binding $COMPUTE_SA_EMAIL --project $DEPLOY_PROJECT_ID --member="serviceAccount:$CB_SA_EMAIL" --role=roles/iam.serviceAccountUser Preparations
  • 19.
  • 20.
    Deploy to environment GoogleCloud Repository, Google Kubernetes Engine - Prepare projects - Setup repository - Setup Cloud Trigger - Check the Magic
  • 21.
  • 22.
    # Create GKECluster $ gcloud beta container clusters create "demo-cluster" --project $DEPLOY_PROJECT_ID --zone $COMPUTE_ZONE... # Set yourself as the cluster admin $ export CONTEXT=`kubectl config view | awk '{print $2}' | grep "demo-cluster" | tail -n 1` $ ACCOUNT=$(gcloud info --format='value(config.account)') $ NAME=$(echo "${ACCOUNT%@*}") $ kubectl --context $CONTEXT create clusterrolebinding $NAME-cluster-admin-binding --clusterrole=cluster-admin --user=$ACCOUNT Preparations
  • 23.
    # Create CloudRepository $ gcloud source repos create cb-deploy-k8s --project $BUILD_PROJECT_ID # Allow GKE to fetch containers from Build Project $ DEPLOY_PROJECT_NUMBER=$(gcloud projects describe $DEPLOY_PROJECT_ID --format='value(projectNumber)') $ GKE_SA_EMAIL="$DEPLOY_PROJECT_NUMBER-compute@developer.gserviceaccount.com" $ BUCKET_PATH="gs://eu.artifacts.$BUILD_PROJECT_ID.appspot.com" $ gsutil iam ch serviceAccount:$GKE_SA_EMAIL:objectViewer $BUCKET_PATH # Allow Cloud Build to deploy to GKE on Deploy Project $ BUILD_PROJECT_NUMBER=$(gcloud projects describe $BUILD_PROJECT_ID --format='value(projectNumber)') $ CB_SA_EMAIL="$BUILD_PROJECT_NUMBER@cloudbuild.gserviceaccount.com" $ gcloud projects add-iam-policy-binding $DEPLOY_PROJECT_ID --member=serviceAccount:$CB_SA_EMAIL --role roles/container.developer Preparations
  • 24.
  • 25.
    There is alot more you could do Eg. GitOps-style continuous delivery with Cloud Build https://cloud.google.com/kubernetes-engine/docs/tutorials/gitops-cloud-build
  • 26.
    Github Triggers Using thenew App Triggers
  • 28.
  • 29.
    This is onlythe beginning We have just scratched the surface
  • 30.
    Some of thesolutions I suggest looking into - Tekton https://tekton.dev/ - Jenkins X https://jenkins-x.io/ - Gitlab Kubernetes Executor - Github Actions https://github.com/features/actions - Flux https://docs.fluxcd.io/en/latest/ - Flagger https://flagger.app/
  • 31.