Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Jenkins vs. AWS CodePipeline
Rafael Schleetz Benvenuti & Steffen Gebert | 09.09.2019
Community Day 2019 Sponsors
@rafaelbenvenuti
@StGebert
Abstract
Who has not used Jenkins? Who does not have a love-hate relationship with it?
At EMnif...
@rafaelbenvenuti
@StGebert
Agenda
3
Current
State
Costs Feature
Comparison
Combining
Both
@rafaelbenvenuti
@StGebert
Your Guides
4
Dr. Steffen Gebert
@StGebert
Head of SRE
Würzburg
Rafael Schleetz Benvenuti
@rafa...
@rafaelbenvenuti
@StGebert
EMnify GmbH
• Cloud-based virtual mobile network operator (MVNO) focussed on IoT
• Provide glob...
@rafaelbenvenuti
@StGebert
Jenkins and AWS CodePipeline
Current State
@rafaelbenvenuti
@StGebert
Running Jenkins
• Jenkins Master
• EC2 instance, ECS+EFS, or EKS, Fargate (requires RWO file sy...
@rafaelbenvenuti
@StGebert
Job Definition
Job DSL Organization Folder
Everything as Code
8
Jenkins Pipeline
Build Agents
C...
@rafaelbenvenuti
@StGebert
Out of Scope: Jenkins X
9
@rafaelbenvenuti
@StGebert
AWS CodePipeline
10
@rafaelbenvenuti
@StGebert
Pipeline
Implementation
@rafaelbenvenuti
@StGebert
Jenkins Pipeline
12
Domain-specific language
Built-in control structures
Steps
•sh, bat
•readFi...
@rafaelbenvenuti
@StGebert
Jenkins Pipeline Steps
• Plugins contribute additional steps
• Online reference: https://jenkin...
@rafaelbenvenuti
@StGebert
CodePipeline Pipeline
14
@rafaelbenvenuti
@StGebert
CodePipeline Pipeline
15
@rafaelbenvenuti
@StGebert
CodePipeline Pipeline
16
@rafaelbenvenuti
@StGebert
CodePipeline Pipeline
17
@rafaelbenvenuti
@StGebert
CodePipeline Pipeline
18
@rafaelbenvenuti
@StGebert
CodePipeline Actions
19
Source Github Amazon S3 Amazon ECR
Amazon
CodeCommit
Build Amazon
CodeB...
@rafaelbenvenuti
@StGebert
Pricing
@rafaelbenvenuti
@StGebert
Jenkins
Pricing
21
Setup Costs
•Initial Setup Costs
Usage Costs
•Computing resources used
•EC2 ...
@rafaelbenvenuti
@StGebert
AWS Pricing
22
CodePipeline
•Each pipeline costs $1.00 per month
CodeCommit
•Free for first 5 u...
@rafaelbenvenuti
@StGebert
Price Comparison for Build Capacity
23
Jenkins Master Instance Jenkins Agent Instance
c5.large ...
@rafaelbenvenuti
@StGebert
Price Comparison for Pipelines
24
Jenkins Master Instance
c5.large RESERVED
2 vCPUS | 4 GB Ram
...
@rafaelbenvenuti
@StGebert
Visualization
@rafaelbenvenuti
@StGebert
CodePipeline Basic Pipeline
26
@rafaelbenvenuti
@StGebert
Visualization in Jenkins
27
@rafaelbenvenuti
@StGebert
28
@rafaelbenvenuti
@StGebert
29
@rafaelbenvenuti
@StGebert
30
@rafaelbenvenuti
@StGebert
31
@rafaelbenvenuti
@StGebert
CodeBuild Log Output
32
@rafaelbenvenuti
@StGebert
CodePipeline Execution Summary
33
@rafaelbenvenuti
@StGebert
Execution History
34
@rafaelbenvenuti
@StGebert
CodeBuild Metrics
35
@rafaelbenvenuti
@StGebert
Let’s Start the
Pipeline
@rafaelbenvenuti
@StGebert
Access Control
@rafaelbenvenuti
@StGebert
Access Control
• Jenkins
• Users and groups, built-in or via
SSO/OAuth
• Per-project/folder per...
@rafaelbenvenuti
@StGebert
CodePipeline Service Permissions
39
@rafaelbenvenuti
@StGebert
Building
@rafaelbenvenuti
@StGebert
Jenkins Build Environments
• Any agent (running Java) connecting to
the master (Linux, Windows,...
@rafaelbenvenuti
@StGebert
CodeBuild Build Environments
• A CodeBuild build environment represents a combination of:
• ope...
@rafaelbenvenuti
@StGebert
CodeBuild BuildSpec
43
@rafaelbenvenuti
@StGebert
Secrets Handling
@rafaelbenvenuti
@StGebert
Jenkins Credentials Management
45
@rafaelbenvenuti
@StGebert
CodeBuild Credentials Management
46
@rafaelbenvenuti
@StGebert
Collecting
Test Results
@rafaelbenvenuti
@StGebert
Jenkins: Collecting Test Results
48
@rafaelbenvenuti
@StGebert
CodeBuild: Collecting Test Results
49
@rafaelbenvenuti
@StGebert
Working With
Multiple Branches
@rafaelbenvenuti
@StGebert
Different Behavior per Branch
51
@rafaelbenvenuti
@StGebert
Implementing Gitflow in CodePipeline
52 Source: https://aws.amazon.com/blogs/devops/implementin...
@rafaelbenvenuti
@StGebert
Getting Approvals
And Input
@rafaelbenvenuti
@StGebert
Approval
54
@rafaelbenvenuti
@StGebert
Getting Input
• Input step
• Simple approval
• Values/text can be dynamically computed
• During...
@rafaelbenvenuti
@StGebert
Pipeline
Super Powers
@rafaelbenvenuti
@StGebert
Jenkins Scripted Pipelines
• More power compared to Declarative Pipelines
• Full Groovy power
•...
@rafaelbenvenuti
@StGebert
CodePipeline Custom Actions
58
@rafaelbenvenuti
@StGebert
Deployment
@rafaelbenvenuti
@StGebert
Deployments
60
@rafaelbenvenuti
@StGebert
Deploying to EC2 Instances
61
AWS CodeDeploy
Jenkins Master
Jenkins Agent
Security
Group
EKS
@rafaelbenvenuti
@StGebert
CodeDeploy AppSpec Example for EC2
62
@rafaelbenvenuti
@StGebert
CodeDeploy Rollbacks
63
@rafaelbenvenuti
@StGebert
Jenkins and Code*
Working Together
@rafaelbenvenuti
@StGebert
From CodePipeline to Jenkins
• AWS CodePipeline Plugin (polls for build jobs, publishes results...
@rafaelbenvenuti
@StGebert
From Jenkins to AWS
66
@rafaelbenvenuti
@StGebert
Summary &
Conclusion
@rafaelbenvenuti
@StGebert
Summary & Conclusion
• Right tool depends on your situation and requirements - still in 2019
• ...
Upcoming SlideShare
Loading in …5
×

Jenkins vs. AWS CodePipeline

124 views

Published on

Talk at AWS Community Summit Germany
Hamburg, 09.09.2019

Published in: Technology
  • Be the first to comment

Jenkins vs. AWS CodePipeline

  1. 1. Jenkins vs. AWS CodePipeline Rafael Schleetz Benvenuti & Steffen Gebert | 09.09.2019 Community Day 2019 Sponsors
  2. 2. @rafaelbenvenuti @StGebert Abstract Who has not used Jenkins? Who does not have a love-hate relationship with it? At EMnify, we are heavy Jenkins users, but we re also always considering alternatives where hosted services could make our life easier. Therefore, we recently - once again - looked at AWS CodePipeline and its friends CodeCommit, CodeBuild, and CodeDeploy. In this talk, we will compare the current state of the two ecosystems regarding their simplicity and flexibility for implementing both trivial as well as complex pipelines. Further, we cover topics like: deployment, maintenance, security, costs, and usability. 2
  3. 3. @rafaelbenvenuti @StGebert Agenda 3 Current State Costs Feature Comparison Combining Both
  4. 4. @rafaelbenvenuti @StGebert Your Guides 4 Dr. Steffen Gebert @StGebert Head of SRE Würzburg Rafael Schleetz Benvenuti @rafaelbenvenuti Site Reliability Engineer Berlin
  5. 5. @rafaelbenvenuti @StGebert EMnify GmbH • Cloud-based virtual mobile network operator (MVNO) focussed on IoT • Provide global connectivity for IoT devices • Running virtualized, self-developed mobile core on AWS • Located in Würzburg & Berlin • Technology stack: AWS, Terraform, Java, Akka, Perl, C++, Go 5
  6. 6. @rafaelbenvenuti @StGebert Jenkins and AWS CodePipeline Current State
  7. 7. @rafaelbenvenuti @StGebert Running Jenkins • Jenkins Master • EC2 instance, ECS+EFS, or EKS, Fargate (requires RWO file system) • Recommendation: run it in Docker and include your plugins • Build Agents • Optional J (building on master is usually a bad idea) • EC2 spot (using ec2 plugin) or EKS (using kubernetes plugin) 7
  8. 8. @rafaelbenvenuti @StGebert Job Definition Job DSL Organization Folder Everything as Code 8 Jenkins Pipeline Build Agents Configuration as Code Jenkins Master AWS CloudFormation
  9. 9. @rafaelbenvenuti @StGebert Out of Scope: Jenkins X 9
  10. 10. @rafaelbenvenuti @StGebert AWS CodePipeline 10
  11. 11. @rafaelbenvenuti @StGebert Pipeline Implementation
  12. 12. @rafaelbenvenuti @StGebert Jenkins Pipeline 12 Domain-specific language Built-in control structures Steps •sh, bat •readFile, readMavenPom •stash, archiveArtifacts •build pipeline { stages { stage("Build") { steps { echo "Starting engines.." sh "make" } } stage("Unit") { steps { sh "mvn test" } } } agent { docker { label "maven:latest" } } }
  13. 13. @rafaelbenvenuti @StGebert Jenkins Pipeline Steps • Plugins contribute additional steps • Online reference: https://jenkins.io/doc/pipeline/steps/ 13
  14. 14. @rafaelbenvenuti @StGebert CodePipeline Pipeline 14
  15. 15. @rafaelbenvenuti @StGebert CodePipeline Pipeline 15
  16. 16. @rafaelbenvenuti @StGebert CodePipeline Pipeline 16
  17. 17. @rafaelbenvenuti @StGebert CodePipeline Pipeline 17
  18. 18. @rafaelbenvenuti @StGebert CodePipeline Pipeline 18
  19. 19. @rafaelbenvenuti @StGebert CodePipeline Actions 19 Source Github Amazon S3 Amazon ECR Amazon CodeCommit Build Amazon CodeBuild Cloudbees Jenkins Teamcity Test Amazon CodeBuild Amazon Device Farm Jenkins Deploy Amazon CodeDeploy Amazon CloudFormation Amazon Elastic Beanstalk Amazon Service Catalog Amazon ECS Amazon S3 Invoke AWS Lambda
  20. 20. @rafaelbenvenuti @StGebert Pricing
  21. 21. @rafaelbenvenuti @StGebert Jenkins Pricing 21 Setup Costs •Initial Setup Costs Usage Costs •Computing resources used •EC2 instance costs •On-Demand •Reserved •Spot •A bit of storage, traffic etc. Maintenance Costs •Operational Costs
  22. 22. @rafaelbenvenuti @StGebert AWS Pricing 22 CodePipeline •Each pipeline costs $1.00 per month CodeCommit •Free for first 5 users •Each additional user costs $1.00 per month CodeBuild •Small instance (2 vCPUs, 3 GB RAM) – $0.005/min •Medium instance (4 vCPUs, 7 GB RAM) - $0.01/min •Large instance (8 vCPUs, 15 GB RAM) - $0.02/min CodeDeploy •Free
  23. 23. @rafaelbenvenuti @StGebert Price Comparison for Build Capacity 23 Jenkins Master Instance Jenkins Agent Instance c5.large RESERVED c5.large SPOT 2 vCPUS | 4 GB Ram 2 vCPUS | 4 GB Ram $39.42/month $0.000325/min CodeBuild Instance general1.small 2 vCPUS | 3 GB Ram $0.005 / min
  24. 24. @rafaelbenvenuti @StGebert Price Comparison for Pipelines 24 Jenkins Master Instance c5.large RESERVED 2 vCPUS | 4 GB Ram CodePipelines No Instances Required No Resources Required
  25. 25. @rafaelbenvenuti @StGebert Visualization
  26. 26. @rafaelbenvenuti @StGebert CodePipeline Basic Pipeline 26
  27. 27. @rafaelbenvenuti @StGebert Visualization in Jenkins 27
  28. 28. @rafaelbenvenuti @StGebert 28
  29. 29. @rafaelbenvenuti @StGebert 29
  30. 30. @rafaelbenvenuti @StGebert 30
  31. 31. @rafaelbenvenuti @StGebert 31
  32. 32. @rafaelbenvenuti @StGebert CodeBuild Log Output 32
  33. 33. @rafaelbenvenuti @StGebert CodePipeline Execution Summary 33
  34. 34. @rafaelbenvenuti @StGebert Execution History 34
  35. 35. @rafaelbenvenuti @StGebert CodeBuild Metrics 35
  36. 36. @rafaelbenvenuti @StGebert Let’s Start the Pipeline
  37. 37. @rafaelbenvenuti @StGebert Access Control
  38. 38. @rafaelbenvenuti @StGebert Access Control • Jenkins • Users and groups, built-in or via SSO/OAuth • Per-project/folder permissions 38 • CodePipeline • IAM users and roles • Configure and run permissions AWS Identity and Access Management
  39. 39. @rafaelbenvenuti @StGebert CodePipeline Service Permissions 39
  40. 40. @rafaelbenvenuti @StGebert Building
  41. 41. @rafaelbenvenuti @StGebert Jenkins Build Environments • Any agent (running Java) connecting to the master (Linux, Windows, MacOS, ..) • Docker images • Kubernetes Pod with multiple containers
  42. 42. @rafaelbenvenuti @StGebert CodeBuild Build Environments • A CodeBuild build environment represents a combination of: • operating system / programming language runtime / set of tools • Essentially, a build environment definition is a Docker Image • You can use Docker images stored in: • The Official Codebuild Image Repository • AWS ECR Repositories • DockerHub
  43. 43. @rafaelbenvenuti @StGebert CodeBuild BuildSpec 43
  44. 44. @rafaelbenvenuti @StGebert Secrets Handling
  45. 45. @rafaelbenvenuti @StGebert Jenkins Credentials Management 45
  46. 46. @rafaelbenvenuti @StGebert CodeBuild Credentials Management 46
  47. 47. @rafaelbenvenuti @StGebert Collecting Test Results
  48. 48. @rafaelbenvenuti @StGebert Jenkins: Collecting Test Results 48
  49. 49. @rafaelbenvenuti @StGebert CodeBuild: Collecting Test Results 49
  50. 50. @rafaelbenvenuti @StGebert Working With Multiple Branches
  51. 51. @rafaelbenvenuti @StGebert Different Behavior per Branch 51
  52. 52. @rafaelbenvenuti @StGebert Implementing Gitflow in CodePipeline 52 Source: https://aws.amazon.com/blogs/devops/implementing-gitflow-using-aws-codepipeline-aws-codecommit-aws-codebuild-and-aws-codedeploy/
  53. 53. @rafaelbenvenuti @StGebert Getting Approvals And Input
  54. 54. @rafaelbenvenuti @StGebert Approval 54
  55. 55. @rafaelbenvenuti @StGebert Getting Input • Input step • Simple approval • Values/text can be dynamically computed • During pipeline execution • Jenkins Parametrized build • Only supplied with job execution • Can come from other job / via POST 55
  56. 56. @rafaelbenvenuti @StGebert Pipeline Super Powers
  57. 57. @rafaelbenvenuti @StGebert Jenkins Scripted Pipelines • More power compared to Declarative Pipelines • Full Groovy power • Control/data structures • Classes • Variable assignment • Groovy errors only triggered during execution • Build error handling up to user (try/catch) • Greater flexibility in build agent allocation • Declarative: use script{} for super powers • Shared libraries to maintain custom steps 57
  58. 58. @rafaelbenvenuti @StGebert CodePipeline Custom Actions 58
  59. 59. @rafaelbenvenuti @StGebert Deployment
  60. 60. @rafaelbenvenuti @StGebert Deployments 60
  61. 61. @rafaelbenvenuti @StGebert Deploying to EC2 Instances 61 AWS CodeDeploy Jenkins Master Jenkins Agent Security Group EKS
  62. 62. @rafaelbenvenuti @StGebert CodeDeploy AppSpec Example for EC2 62
  63. 63. @rafaelbenvenuti @StGebert CodeDeploy Rollbacks 63
  64. 64. @rafaelbenvenuti @StGebert Jenkins and Code* Working Together
  65. 65. @rafaelbenvenuti @StGebert From CodePipeline to Jenkins • AWS CodePipeline Plugin (polls for build jobs, publishes results) 65
  66. 66. @rafaelbenvenuti @StGebert From Jenkins to AWS 66
  67. 67. @rafaelbenvenuti @StGebert Summary & Conclusion
  68. 68. @rafaelbenvenuti @StGebert Summary & Conclusion • Right tool depends on your situation and requirements - still in 2019 • Accepting that not everything is covered by one tool makes change easier • Integrating both allows • Getting the best of both • Cheaper solution • Smoother migration • Tip: Try CodeDeploy, it looks pretty interesting 68

×