Downloaded 15 times
Uploaded bySteve Mactaggart
Jenkins as a Service - Code all the way down
The document outlines a comprehensive framework for implementing Jenkins as a service within an AWS ECS infrastructure, focusing on infrastructure and application configurations as code. It emphasizes using Docker containers, Groovy scripts, and Jenkins DSL for automating and simplifying CI/CD workflows while enhancing traceability and consistency. Additional resources and examples, such as Dockerfiles and Groovy configurations, are provided to support the Jenkins-as-Code approach and facilitate discussions on migration and best practices.
More Related Content
Similar to Jenkins as a Service - Code all the way down
Jenkins as a Service - Code all the way down
- 1. Jenkins as aService Code all the way down Steve Mactaggart Application Delivery Evangineer
- 2. The moving parts Infrastructure(AWS / ECS) Application (Jenkins) Configuration (Jenkins Jobs)
- 3. The moving parts- AS CODE Infrastructure (AWS / ECS) Application (Jenkins) Configuration (Jenkins Jobs) CloudFormation Docker containers Groovy / Jenkins DSL
- 4. Everything as code •Consistently repeatable process • Enables a strong self service workflow • Highly visible traceability of change • Roll forward / rollback
- 5. The moving parts- Infrastructure Infrastructure (AWS / ECS) Application (Jenkins) Configuration (Jenkins Jobs) CloudFormation Docker containers Groovy / Jenkins DSL
- 6. Infrastructure • Infrastructure capturedas Code • Designed for easy re-use per team • Deployable into existing VPCs • Common location to share learned lessons Infrastructure / AWS
- 7. EC2 Infrastructure - Simplified ap-southeast-2bap-southeast-2a ECSagent auto-scaling group ECS master auto-scaling group EC2 EC2 Infrastructure / AWS / CloudFormation ECS configuration Master ECR Agent ECR AWS ECS CloudWatch logs
- 8. EC2 Infrastructure ap-southeast-2bap-southeast-2a ECS agent auto-scalinggroup ECS master auto-scaling groupECS configuration Master ECR Agent ECR Legend ECS ScaleDown function ECS Memory Reservation Alarm Termination Event AWS ECS EC2 EC2 Query for tasks running on host Continue the ASG lifecycle hook Publish intent to terminate Watch for ScaleUp / ScaleDown alarm Consume message Containers download configuration ECSLaunches AgentTask Launch / Terminate EC2 Instance Jenkins Schedules new Task via ECS JenkinsAgent phoneshometo Mastercontainer Direct TCP call S3 download AWS API Call Retrieve Master container from ECR Retrieve Agent container from ECR EC2instance registerswith ECS Agentcontainer registerswithmaster ALB routes web traffic Infrastructure / AWS / CloudFormation CloudWatch logs EC2 / ECS logs shipped to CloudWatch
- 9. Infrastructure • Example CloudFormation •https://bitbucket.org/stevemac/dock erfiles/src/282408c04239/cloudforma tion/?at=master • 2 stacks • Infrastructure (2x ECS clusters) • Cluster - Jenkins Master Service
- 10. The moving parts- Application Infrastructure (AWS / ECS) Application (Jenkins) Configuration (Jenkins Jobs) CloudFormation / Terraform Docker containers Groovy / Jenkins DSL
- 11. EC2 Jenkins in Docker ap-southeast-2bap-southeast-2a ECSagent auto-scaling group ECS master auto-scaling group EC2 EC2 ECS configuration Master ECR Agent ECR AWS ECS CloudWatch logs Jenkins Application / Docker containers Jenkins Master Jenkins Agents
- 12. The Jenkins Masterand its Agents • Jenkins is an Application • We understand the Docker application workflow • Immutable Applications increase confidence of changes • Based of official JenkinsCI base containers • From jenkinsci/Jenkins • From jenkinsci/slave • Extended to include specific build and deployment tools • Python / AWSCli / etc Jenkins Application / Docker containers
- 13. Jenkins master -Dockerfile Jenkins Application / Docker containers
- 14. Jenkins agent -Dockerfile Jenkins Application / Docker containers
- 15. Jenkins in Docker •Many plugins in-use: • build-pipeline-plugin • amazon-ecs • delivery-pipeline-plugin • build-monitor-plugin • gitlab-merge-request-Jenkins • Etc… • State persisted through Docker volume mount • Example Master and Agent Docker containers • https://bitbucket.org/stevemac/dockerfiles/src/
- 16. The moving parts- Configuration Infrastructure (AWS / ECS) Application (Jenkins) Configuration (Jenkins Jobs) CloudFormation / Terraform Docker containers Groovy / Jenkins DSL
- 17. Configuration as Code- setup • Start of the configuration delivered in Dockerfiles • Configuration and ssh keys downloaded from S3 bucket • Supplied to container as Environment variable Configuration / Groovy
- 18. Configuration as Code- setup • Runtime configuration automated through use of custom Groovy initialisation scripts within the container • Groovy is used to configure: • Jenkins credentials • ECS Cluster configuration • Jenkins Bootstrap job • Executed as part of Dockerfile entrypoint Configuration / Groovy
- 19. Pipeline as Code- runtime • Using the Jenkins DSL pipeline • https://bitbucket.org/jenkinspipeline/jenkins-dsl-examples • Using the power of Groovy code to create reusable elements • Version controlled through Git • Repeatable / reliable / extensible • See https://github.com/jenkinsci/job-dsl-plugin/wiki/Tutorial---Using-the-Jenkins-Job-DSL • Automatic documentation available within each Jenkins instance Configuration / Jenkins DSL Lets talk about a Jenkinsfile approach over a beer
- 20. Pipeline as Code– groovy constructs Configuration / Groovy / Jenkins DSL
- 21. Pipeline as Code- example https://bitbucket.org/jenkinspipeline/jenkins-dsl-examples 1. build_pipeline.groovy Creation of example build jobs - CI 2. delivery_pipeline.groovy Creation of example jobs for a candidate delivery pipeline - CD 3. views.groovy Creation of some standard pipeline views Configuration / Groovy / Jenkins DSL
- 22. Bringing it together Letour codebases combine
- 23. Executed within ECS Jenkinsservice Result of infrastructure deploy Performed by operator Delivery workflow Build and Publish Docker Containers Launch Stack Publish keys to S3 bucket Create ECS Service Retrieve latest Docker image from ECR Launch ECS Task Download keys from S3 Clone groovy config repo Launch Jenkins Master Process Groovy config executed Bootstrap job generates pipelines
- 24. Followup discussions 1. Migratingto a Jenkins-as-code approach 2. Jenkins DSL vs Jenkinsfiles 3. All about Jenkins State 4. Security in a stateless world 5. Advanced Jenkins DSL patterns
- 25. Thanks ! Steve Mactaggart •Application Delivery Evangineer at Cevo • @stevemac Example working code available at: https://bitbucket.org/stevemac/dockerfiles