Cloud solutions have evolved tremendously in recent years, but many customers still shy away from moving their mission-critical workloads to the cloud, due to security and compliance concerns. Join our session to get insights on how Continuous Integration/Continuous Delivery/Security Pipelines and Infrastructure as Code can address these concerns and enable successful Hybrid Cloud Solutions.Key Takeaways:- Understand benefits of Hybrid Cloud- Understand cloud concepts, Dev/Sec/Ops and technologies for CI/CD and Infrastructure as Code- Identify practical scenarios for using Infrastructure as Code for multiple Industries considering Hybrid cloud deployments and solutions.
1. Infrastructure as Code Helping You
Conquer the Cloud
Ana Biazetti,
Distinguished Engineer,
IBM Cloud for Financial Services
Master Inventor
Shikha Srivastava,
Distinguished Engineer - IBM SaaS
Master Inventor
2. Agenda
• Landscape of Modernization and Hybrid Cloud
• Challenges and Opportunities
• Continuous Integration and Continuous Delivery
• Infrastructure as Code
• Scenarios and Examples
• Summary and Call to Action
3. Rise of Cloud and Modernization
Mainframe
Client Server
Web
Cloud
Pain Points with Monoliths
• Large Applications are
complex
• Full application upgrade
required for minor or
major updates
• Difficult to scale
• Bug in one module brings
down the entire
application
• Painful to adopt new
technologies
4. Cloud has changed expectations
Availability
24X7
Hyperscale Agility
Start small
and
grow to enterprise scale
Uptime with
defined SLAs
Deliver in-time
and
deliver at regular interval
5. An approach to modernization
Rationalize and decommission
20%
Hybrid IT
Modernize and Migrate to Cloud
…on any cloud
platform
Private Cloud
on premises
Dedicated/Hosted
off-premises
Public Cloud
20%
Traditional
on premises
Re-Host (lift & shift)
Re-Platform
Re-Architect (Microservices / APIs)
10%
10%
15%
Public SaaS
Retain on Traditional
+ for selected
apps:
- Automate
- Containerise
Replace with “as a service” solution
10%
50%
Refactor
15%
6. Hybrid Cloud – Best of the both worlds
Public Cloud
Fully Customizable
Secure by Design
Full control and
robust management
Elastic
Pay-as-you go mode
Low entry cost
Maximize returns
Match workload to best fit
infrastructure
Meet Capacity
Highly scalable
Innovate at the speed of your choice
Private Cloud
7. What does it take to modernize
Collaboration and
Communication
Continuous Integration Continuous Delivery Observability
Change and incident
management
Security and
Compliance
Test and Quality Scale
Plan Build Test Secure Release Operate
Modernization
Ideas
Modernization
Ideas
Iterate
8. Dev and SRE collaboration
Modernization requires Dev and SRE to work together towards DevSecOps
approach
Developers
Create
SRE Manage
Shift Left SRE ( DevSecOps)
Communication
Collaboration
Integration
12. /
Type
DevOps
GitOps
GitOps
Structure
Each part
of the
pipeline is
1 YAML
Each part
of the
pipeline is
1 YAML
Only 1
Jenkins file
Pipelines
Manager
X
X
--
Language
Go
Go
Java
with
Plugins
Tekton
Tekton
Server/VM
K8s-native
Part of K8
project
(Cloud
Native)
Cloud
Native
No
(Server/V
M)
Execution
Multiple
PODs
Paralelis
m
Multiple
PODs
Only 1
POD,
Sequential
Execution
V3.0
Pipeline Technologies
14. Infrastructure as Code (IaC) helping with
Modernization to Hybrid Cloud
Codify infrastructure in a way that makes
provisioning automated, faster, and repeatable.
Infrastructure as Code Technology:
○ Terraform
○ Ansible
○ Chef
○ Puppet
Benefits:
● Improve speed
● Improve reliability
● Prevent configuration drift
● Support experimentation, testing, and
optimization
15. Terraform
● Terraform is an open source
“Infrastructure as Code” tool, created by
HashiCorp.
● A declarative coding tool, describes the
desired “end-state” infrastructure
● simple syntax
● provision infrastructure across multiple
cloud and on-premises data centers
https://www.terraform.io/
19. Summary – Modernization is a journey
Identify what
to modernize
Nail down the
hybrid design
and
dependencies
Reliable
repeatable
automation –
“Everything-
as-Code”
Drive
standardizati
on and
reuse
Iterate
Mitigate
security risks
as early as
possible –
“Shift-left”
20. References
CI/CD Pipelines
● Continuous Integration
● Continuous Delivery
● Tekton
● Tekton versus Jenkins
● Speed and resiliency: two sides of the same coin
● DevSecOps Intro
● Pipelines enabling Audit and Compliance
● CI template
● CD template
● IBM Cloud for Financial Services pattern
infrastructure as Code
• Terraform
• Infrastructure as Code
• IBM Cloud Terraform Provider
• IBM Cloud Terraform Modules
• Terraform versus other infrastructure
as code options
Manage Infrastructure
Manage configurations
Observability
Manage Alerts
Manage Changes in production
Manage security incidents
Manage networking
-----------------
Take requirements
Design feature and function
Write code
Write tests ( may be)
Test and automate test (may be)
Refactor
Push to production
-----------------
Take requirements
Design feature and function
Write code
Write tests ( may be)
Test and automate test (may be)
Refactor
Push to production
-----------------
Challenges organizations face today:
Lack of deep security expertise in app dev teams
Manual effort for audit and compliance readiness
Ever changing security landscape
Custom DevOps investments across teams
Dev, Ops and Security teams working in silos
Continuous integration is a software development process where developers integrate the new code they've written more frequently throughout the development cycle, adding it to the code base at least once a day. Automated testing is done against each iteration of the build to identify integration issues earlier, when they are easier to fix , which also helps avoid problems at the final merge for the release. Overall, continuous integration helps streamline the build process, resulting in higher-quality software and more predictable delivery schedules.
Continuous delivery is the automated movement of code through the development lifecycle (sometimes called the delivery lifecycle); continuous deployment is the automated movement of that code into production, once it passes the required automated tests.
Others, like Travis, GitLab CI. etc
How to get started:
Instrument legacy CI/CD orchestration with reusable assets for evidence collection
Orchestrate legacy tools using DevSecOps reference implementation
Fully leverage DevSecOps reference implementation, eliminating legacy tools
https://developer.ibm.com/videos/tekton-versus-Jenkins
DevOps premise: Velocity and quality can coexist!
DevOps practices have become mainstream
Supported by vibrant ecosystem of open source and commercial tools
Automate everything
Manual/human processes and handoffs remain a bottleneck, in addition to introducing error and inconsistency
We’ve seen progress in quality –
More consumable static analysis, lighter-weight test automation frameworks like Junit, cloud-based test labs, architectures that limited blast-radius, canary testing etc. reduce or eliminate traditional test cycles
We’ve seen progress in deployments –
Microservice architectures, containers, and technique like canary testing and progressive rollout speed up and de-risk deployments. No blackout windows, release weekend events, etc.
PLUG OUR TEKTON WORK HERE---
Everything as code
Consistent management, governance, auditability, branching/exploration model, sharing/contribution model, etc. break-down silos of specialized tools/expertise and enable the “full-stack developer”.
While maintaining Separation of concerns
E.g. Terraform, Ansible, Schematics, Tekton, …
What are we seeing now? PLUG GITSECURE HERE----
“DevSecOps” – Emphasis on Security and Compliance
Standardizing automation, e.g. through common pipeline tasks, of release-readiness checks, e.g. vulnerability scanning, test automation, code coverage, code review, etc.
Maintaining audit-readiness by capturing results/outputs of above checks
Shift-left – Risk is reduced and velocity increased the sooner feedback reaches the developer
How many of the release readiness checks can be done on every pull request before it’s merged? E.g. scanning for source, package, OS vulnerabilities
“GitOps” – extend Kubernetes-style declarative/state-based approach to describing production systems
E.g. Razee, ArgoCD, …
Consolidation/standardization through open source communities
E.g. Continuous Delivery Foundation, under LF, has mission to drive tools AND approaches/practices
E.g. Tekton
IaC uses code to define, deploy and update a given infrastructure. This means that networks, routes, load balancer, compute instances, storage buckets, etc are managed in code
IaC allows developers to codify infrastructure in a way that makes provisioning automated, faster, and repeatable. It’s a key component of Agile and DevOps practices such as version control, continuous integration, and continuous deployment.
Infrastructure as code can help with the following:
Improve speed: Automation is faster than manually navigating an interface when you need to deploy and/or connect resources.
Improve reliability: If your infrastructure is large, it becomes easy to misconfigure a resource or provision services in the wrong order. With IaC, the resources are always provisioned and configured exactly as declared.
Prevent configuration drift: Configuration drift occurs when the configuration that provisioned your environment no longer matches the actual environment. (See ‘Immutable infrastructure’ below.)
Support experimentation, testing, and optimization: Because Infrastructure as Code makes provisioning new infrastructure so much faster and easier, you can make and test experimental changes without investing lots of time and resources; and if you like the results, you can quickly scale up the new infrastructure for production.
IaC tools include:
- Provisioning tools
- configuration management tools (eg, Chef , Puppet)– focus on installation an management of SW on existent servers)Also declarative versus procedural:
Terraform and Puppet: declararive
Ansible and Chef: procedural
Terraform is purely a declarative tool (see above), Ansible combines both declarative and procedural configuration. In procedural configuration, you specify the steps, or the precise manner, in which you want to provision infrastructure to the desired state. Procedural configuration is more work but it provides more control.
While Terraform is cloud agnostic, CloudFormation is a managed service offered by AWS that comes with a common language for modeling and provisioning third-party app resources. It is done for your cloud platform with a single point of reference for infrastructure mapping & development.
A declarative coding tool, Terraform enables developers to describe the desired “end-state” cloud or on-premises infrastructure for running an application. It then generates a plan for reaching that end-state and executes the plan to provision the infrastructure.
Because Terraform uses a simple syntax, can provision infrastructure across multiple cloud and on-premises data centers, and can safely and efficiently re-provision infrastructure in response to configuration changes, it is currently one of the most popular infrastructure automation tools available, specially for hybrid cloud or multicloud environments
The deployment of a solution consists of three stages: the init-stage, the plan-stage and the apply-stage.
In the init-stage, the modules that the solution are dependent on will be downloaded to the hidden directory .terraform .
In the plan-stage, Terraform will determine what differences there are between the configuration file(s) and the environment where the solution is deployed to. Plan will also return what objects will be added, changed or deleted.
The deployment itself is done in a seperate step, which is called terraform apply.
The deployment can be removed using terraform destroy.
A terraform template is a collection of files that, together, define the state of your infrastructure to be achieved. They include different configuration files such as variables, resources, and modules.
Terraform modules are small, reusable Terraform configurations for multiple infrastructure resources that are used together. Terraform modules are useful because they allow complex resources to be automated with re-usable, configurable constructs. Writing even a very simple Terraform file results in a module. A module can call other modules—called child modules—which can make assembling configuration faster and more concise.
Terraform providers are plugins that implement resource types. Providers contain all the code needed to authenticate and connect to a service—typically from a public cloud provider—on behalf of the user. You can find providers for the cloud platforms and services you use, add them to your configuration, and then use their resources to provision infrastructure. Providers are available for nearly every major cloud provider,
Reliable repeatable automation – “Everything-as-Code”
Application-as-Code
Infrastructure-as-Code
Reliability-as-Code
Continuous integration, delivery and deployment as-Code
Mitigate security risks as early as possible – “Shift-left”
Automate scanning/testing throughout development (i.e., every change)
Gate progression of changes with automated policies
Maintain history with evidence for audit readiness integrated with Security and Compliance Center
Drive standardization and reuse
Toolchain templates embody DevSecOps practices
Reference architectures include toolchain automation
Integrating mix of tools from third parties, open source, in-house