Talk by Bhuvaneswari Subramani and Vijaya Nirmala G on the topic "CI/CD using AWS developer tools" at AWS Community Day, Bangalore 2018

1. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Bengaluru

2. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
CI CD using AWS Developer Tools
Bhuvaneswari Subramani & Vijaya Nirmala

3. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Bhuvaneswari Subramani (Bhuvana)
Director, Engineering Operations
– GT Nexus, an Infor Company
Currently working on Cloud Computing,
DevOps &, Performance QA
https://installjournal.blogspot.com
Vijaya Nirmala G (Nirmala)
DevOps Solutions Lead
– Sonata Software
Currently working on DevOps Solutions
implementation
https://cloudgoddess.blogspot.com/

4. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
• AWS CI CD Services Overview
• Use cases leveraging AWS CI CD Services
1. Web Application Deployments
2. Infrastructure Rollouts
3. Serverless Application Deployments
• buildspec, appspec & SAM Template
• Debugging Build & Deployment
• Rollback Deployments
• Generic Pipeline for Multiple Environments
• Metrics to gauge DevOps
Agenda

5. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AWS CI CD Services - Overview

6. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Software Release Phases vs AWS CI CD Services
Source Build Test Deploy
AWS CodeCommit AWS CodeBuild Third Party
Tooling
AWS CodeDeploy
AWS
CodePipeline

7. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AWS CodeCommit
• Secure, Scalable, and managed Git Source control
• Use Standard git tools
• Scalability, availability and durability of Amazon S3
• No repo size limit
• Post commit hooks to call out to SNS / Lambda

8. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Visualizer Compare Pull Request

9. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AWS CodeBuild • Fully managed build service that compiles source code,
run tests, and produces software packages
• Scales continuously and processes multiple builds
• You can provide custom build environments suited to
your needs via Docker images
• Only pay by the minute for the compute resources you
use
• Launched with Services like CodePipeline or Jenkins

10. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AWS CodeDeploy
• Automates code deployments to EC2 / On-prem
• Handles the complexity of updating your applications
• Minimal deployment downtime
• Rollback automatically, if failure detected
• Supports varied languages and operating systems
• Integrates with third-party tools like Jenkins

11. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AWS
CodeDeploy Revision 1
Application
Revision 2
Revision 3
Revision n
Instance 1
Deployment Group
Instance 2
Instance 3
Instance n
Deployments
Deploy
Config
appspec.
yml
In-place
deployment

12. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AWS
CodeDeploy Revision 1
Application
Revision 2
Revision 3
Revision n
Instance 1
Deployment Group
Instance 2
Instance 3
Instance n
Deployments
Deploy
Config
appspec.
yml
In-place
deployment
What to
deploy?
Where to
deploy?
How to
deploy?

13. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AWS CodePipeline
• Continuous delivery service for fast and reliable
application updates
• Model and visualize your software release process
• Builds, tests and deploys your code on every
commit
• Integrates with third-party tools

14. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AWS CodePipeline Action Action
Stage
Action Action
Stage
Action
Transition

15. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.

16. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AWS CI CD
Workflows
Web Application Deployments
Infrastructure Rollouts
Serverless Application Deployments

17. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Web Application Deployments
Infrastructure Rollouts
Serverless Application Deployments
AWS CI CD
Workflows

18. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Web Application Deployments

19. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Web Application
Deployments
AW S
CodePipeline

20. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Web Application Deployments
Infrastructure Rollouts
Serverless Application Deployments
AWS CI CD
Workflows

21. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Infrastructure Rollouts

22. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Infrastructure
Rollouts
AW S
CodePipeline

23. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Web App vs
Infrastructure
Rollouts
AW S
CodePipeline
VS

24. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AWS CI CD
Workflows
Web Application Deployments
Infrastructure Rollouts
Serverless Applications
DeploymentsServerless Application Deployments

25. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Serverless Application Deployments

26. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Serverless
Application
Deployments
AW S
CodePipeline

27. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Serverless
Application
Deployments
AW S
CodePipeline

28. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AW S
CodeBuild
buildspec.yml from Application Deployments

29. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AW S
CodeBuild
Create & store an artifact in S3
buildspec.yml fromApplication Deployments

30. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AW S
CodeDeploy
appspec.yml fromApplication Deployments

31. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AW S
CodeDeploy
Files to be copied to the instance
during deployment’s install
appspec.yml fromApplication Deployments

32. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
AW S
CodeDeploy
template.yml fromServerless Application Deployments

33. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Debugging Builds
• Enable Logging in CloudWatch / S3

34. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Debugging Builds
• Enable Logging in CloudWatch / S3
• Local Build Support for AWS CodeBuild
• Get the AWS CodeBuild docker image file from AWS Github
• Build the CodeBuild image locally
• Setup CodeBuild local Agent
• Use the local agent to build your project

35. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Debugging Deployments
• CodeDeploy agent generates agent, updater & deployment logs
• Send them to CloudWatch for debugging
• CodeDeploy agent logs [EC2/On-prem only]
• /var/log/aws/codedeploy-agent/codedeploy-agent.log

36. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Debugging Deployments
• CodeDeploy agent generates agent, updater & deployment logs
• Send them to CloudWatch for debugging
• CodeDeploy agent logs [EC2/On-prem only]
• /var/log/aws/codedeploy-agent/codedeploy-agent.log
• CodeDeploy deployment logs
• /opt/codedeploy-agent/deployment-root/deployment-logs/
• CodeDeploy updater logs
• /tmp/codedeploy-agent.update.log

37. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Rollback Deployments
• Rollback in AWS is actually ====>
• Configure re-deploy by enabling
• Code deploy -> Rollbacks ->
• Code deploy -> ‘Automatic Rollbacks’
• Deployment Configuration to enable rollback based on healthy instances

38. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Generic Pipeline Setup
• Provision resources via Cloudformation with
Environment as Parameter
• Create Environment specific configuration.json
• Map the relevant configuration.json in CodePipeline
Stage

39. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
CloudFormation Template
Parameters
Mappings
Resources
Output template
Parameter
Values
create /
update
Env -> QA /
PreProd / Prod
stack

40. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Metrics to gauge DevOps
• Rapid Delivery with Increased Efficiency
• Increased Feature releases with reduced TTM
• Quicker feedback (system/user) to teams & stake holders
• Greater % of defects detected in testing (Unit & SIT Cycles)
• Ticket / Incidents volume reduction in PreProd & Prod
• High Availability
• Elastic & Scalable Infrastructure

41. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
CI CD Workflow MindMap
- The devil in the detail

42. ©2018, AmazonWebServices, Inc. or its Affiliates. All rights reserved.
Thank you!
You may refer https://installjournal.blogspot.com for AWS CI CD LAB Series