End-to-End CI/CD at scale with Infrastructure-as-Code on AWS

© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
End-to-End CI/CD at scale with
Infrastructure-as-Code on AWS
Bhuvaneswari Subramani (she/hers)
D1DEV102
Director, Engineering Operations, Infor
AWS DevTools Hero

Quick Intro
• Bhuvaneswari Subramani (Bhuvana)
• Director, Engineering Operations, Infor
• AWS Hero since 2019
• Organizer, AWS User Group Bengaluru
• Leading cloud computing, DevOps,
and Performance QA
https://bhuvana.pro
/bhuvanas
@installjournal

Agenda
• Why Infrastructure as Code (IaC) for CI CD ?
• CI CD multi-account Architecture
• Build & Deployment Infrastructure Setup
• Best Practices

Why Infrastructure as Code
for CI CD ?

Key Benefits
Speed Consistency Traceability

CI CD with multi-account
architecture

VPC Gateway
EndPoint
Continuous Integration
VPC
Developer Service Account
Private subnet
Build & Deploy
Server
Worker Nodes
Lin 1
2
3
Mirror Git
Repo
CI
1 - Initiate Build process
2 - Get Source code
3 - Compile & generate artifacts
4 - Publish artifacts
5 - Run code validation suite
AWS Cloud

VPC Gateway
EndPoint
VPC
Private subnet
Build & Deploy
Server
Worker Nodes
Lin 1
2
3
Mirror Git
Repo
Publish build
artifacts
4
S3 Buckets
CI
2 - Get Source code
AWS Cloud

VPC Gateway
EndPoint
VPC
Private subnet
Build & Deploy
Server
Worker Nodes
Lin 1
2
3
Mirror Git
Repo
Publish build
artifacts
4
S3 Buckets
CI
2 - Get Source code
5
AWS Cloud

VPC Gateway
EndPoint
VPC
Build & Deploy
Server
Worker Nodes
Lin
Private subnet
1
2
3
Mirror Git
Repo
Publish build
artifacts
4
S3 Buckets
5
App artifacts
Config artifacts
Data system artifacts
Test artifacts
AWS Cloud

Mirror Git
Repo
Lin
VPC
Continuous Deployment – Nano Env
AWS Cloud AWS Cloud
Developer Service Account Nano Environment(s) Account
https://<env-id>.nano.infornexus.com
Public Subnet
Internet gateway
Windows
Peer
Linux
Peer
Users
Availability Zone
VPC
Private subnet
Build & Deploy
Server
S3 Buckets
Deploy

Nano to Full Scale Environments
Beta
…

Continuous Integration – Alpha / Beta / Prod Env
VPC
Private subnet
Build Server
Worker Nodes
Lin 1
2
3
Mirror Git
Repo
CI
2 - Get Source code
AWS Cloud

VPC
Private subnet
Build Server
Worker Nodes
Lin 1
2
3
Mirror Git
Repo
Publish build
artifacts
4
S3 Buckets
CI
2 - Get Source code
AWS Cloud
VPC Gateway
EndPoint

VPC
Private subnet
Build Server
Worker Nodes
Lin 1
2
3
Mirror Git
Repo
Publish build
artifacts
4
S3 Buckets
CI
2 - Get Source code
AWS Cloud
VPC Gateway
EndPoint
ECR
Publish container
images
VPC Interface
EndPoint
4

Continuous Deployment – Alpha / Beta / Prod Env
VPC
Private subnet
Build Server
Worker Nodes
Lin 1
2
3
Mirror Git
Repo
Publish build
artifacts
4
S3 Buckets
CI
2 - Get Source code
5
AWS Cloud
VPC Gateway
EndPoint
AWS Cloud
Prod Account
Deploy Server
ECR
Publish container
images
VPC Interface
EndPoint
4

Mirror Git
Repo
Lin
AWS Cloud
VPC
Private subnet
Build Server
VPC
AWS Cloud
Alpha / Beta / Prod Environment Account
Private subnet
App Container Services
• Web Tier
• Integration Tier
Intermediate Tier
• Caching
• Lock co-ordination
• Service Discovery
Data Tier
• Queueing
• AWS Hosted Data Services
• AWS Managed Data Services
Logging
&
Monitoring
Supply
Chain
Intelligence
Deploy Server
Deploy Nodes
A
Publish build
artifacts
ECR
B Publish
container
images

Mirror Git
Repo
Lin
AWS Cloud
VPC
Private subnet
Build Server
VPC
AWS Cloud
Private subnet
• Web Tier
Intermediate Tier
• Caching
Data Tier
• Queueing
Logging
&
Monitoring
Supply
Chain
Intelligence
Deploy Server
Deploy Nodes
A
Publish build
artifacts
C
ECR
B Publish
container
images

Mirror Git
Repo
Lin
AWS Cloud
VPC
Private subnet
Build Server
VPC
AWS Cloud
Private subnet
• Web Tier
Intermediate Tier
• Caching
Data Tier
• Queueing
Logging
&
Monitoring
Supply
Chain
Intelligence
Deploy Server
Deploy Nodes
A
Publish build
artifacts
C
ECR
B Publish
container
images
D
D
VPC Gateway
EndPoint
VPC Interface
EndPoint

Build & Deployment
Infrastructure Setup

Build Infrastructure Components
AWS
CloudFormation
Mirror Git Repo
Jenkins Master
S3 Bucket
IAM Roles
Secrets
Jenkins Win Agent Jenkins Linux
Agent
VPC EndPoints
ECR

Setup Mirror Git Repo using IaC
Create Git repo tar.gz &
Upload to S3
Create EC2 Instance
Pull Git repo tar.gz from S3
and place in EC2
Configure Cron job to sync
Git repo
AWS
CloudFormation
Mirror Git Repo

Setup Jenkins Master using IaC
Pipeline
Create
EC2
Salt
minion
Jenkins
Server
Gitlab
IaC Developers
Source - Bootstrap
Bootstrap
Jenkins
Linux Agent
Windows Agent
Jenkins Job
Configuration
Source - IaC

Dev Environment for Jenkins Bootstrapping
Run Jenkins as Container
Install Docker
Setup local repo
Bind mount the directories into
Docker Containers
Install Jenkins as a Container
Configure Jenkins

Best Practices

Best Practices
• Operational excellence
• Security
• Reliability
• Performance efficiency
• Cost optimization
• Sustainability
AWS Well-Architected Framework

Operational excellence
• Continually improving quality process
• Design for failure
• Opportunities for improvement
• Clearly defined Promotion model for IaC

Operational Excellence – Promotion Model
Stable
Staging
Feature
• Holds IaC for services that are live on an Env
• Protected Branch ; Changes needs approval
• Holds IaC for services ready to go live
• Protected Branch ; Changes needs approval
• To apply changes to production or alpha
branch

[STAGING]
[FEATURE-1]
{Create} {rebase} {promote}
[FEATURE-N] {Create}
{Promote to Stable for go
live}
[ … ]
Conventions
o commit
+ branch create
* rebase
^ promote
/ code flow
[ ] branch

[STAGING]
[FEATURE-1]
{Create} {rebase} {promote}
[FEATURE-N] {Create}
[STABLE]
Conventions
o commit
+ branch create
* rebase
^ promote
/ code flow
[ ] branch
{Promote to Stable for go
live}
[ … ]
[FEATURE-1] {Create} {rebase} {promote}
[FEATURE-N]
{Create}
[ … ]
{Code flow to Staging}

Security
• Infrastructure protection
• Centralized identities with SAML 2.0
• End-to-end Traceability
• Data Protection

Reliability
• Monitor workload Resources
• Adapt to changes in Demand
• Implement resiliency for reliable workload

Performance Efficiency
• Capability to deploy the workload in multiple AWS
Regions
• Use serverless architectures
• Monitor your resources

Cost Optimization
• Implement cloud financial management
• Monitor Cost and Usage
• Stop spending money on undifferentiated heavy
lifting

Sustainability
• Remove or refactor workload components with low or
no use
• Increase utilization of build environments
• Optimize areas of code that consume the most time or
resources

Additional resources
Link 1
About Link 1
AWS speakers,
more resources
are on the wiki
Link 1
About Link 1
Link 1
About Link 1
Link 1
About Link 1

Deepen your skills with digital
learning on demand
Access 500+ free digital courses
and Learning Plans
Earn an industry-recognized credential
AWS Skill Builder AWS Certifications
Explore resources with a variety of
skill levels and 16+ languages to
meet your learning needs
Join the AWS Certified community and
get exclusive benefits
Receive Foundational,
Associate, Professional, and
Specialty certifications
Train now
Access new
exam guides
Learn in-demand AWS Cloud skills

Thank you!
Bhuvaneswari Subramani
https://bhuvana.pro
/bhuvanas
@installjournal

!
Please complete
the session survey

End-to-End CI/CD at scale with Infrastructure-as-Code on AWS

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to End-to-End CI/CD at scale with Infrastructure-as-Code on AWS

Similar to End-to-End CI/CD at scale with Infrastructure-as-Code on AWS (20)

More from Bhuvaneswari Subramani

More from Bhuvaneswari Subramani (20)

Recently uploaded

Recently uploaded (20)

End-to-End CI/CD at scale with Infrastructure-as-Code on AWS