Confidential │ ©2020 VMware, Inc.
Data Driven
Decisions in DevOps
Add More To Your Pipelines
Leon Stigter
Product Manager @VMware
July 2nd, 2020
2Confidential │ ©2020 VMware, Inc.
Technology Is
Changing Rapidly
@retgits | @MyDevSecOps
3Confidential │ ©2020 VMware, Inc.
Changing Technology
Means Changing
Infrastructure
@retgits | @MyDevSecOps
4Confidential │ ©2020 VMware, Inc.
Changing
Infrastructure
Changing Means
Rethink How You
Deploy…
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 5
2020 stack
Gone are the “simple” days of centralized stacks
Application Architectures Are Changing
2000’s stack
App
Database
Webserver
Environment
App
Environment
Cache
Database
Logs
@retgits | @MyDevSecOps
6Confidential │ ©2020 VMware, Inc.
“Amazon.com does
136,000 fully
automated
deployments per
day”
Ken Exner, Director AWS Dev Resources
Source: https://techbeacon.com/devops/going-big-devops-how-scale-continuous-delivery-success
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 7
Applications Are The New Unit of Measure For Ops
AWS does 136,000 fully automated
deployments every day, but…
Ø Less than 50% of organizations has CI/CD
pipelines
Ø Less than 6% has fully automated CI/CD
pipelines
Deployments
@retgits | @MyDevSecOps
8Confidential │ ©2020 VMware, Inc.
“I really want to spend
my day sitting in front
of screens watching
DevOps pipelines…”
- No one ever…
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 9
Who am I?
Ø Product Manager, turned
Developer Advocate,
turned Product Manager
again…
Ø Passionate about
Serverless, Containers,
and all things Cloud
Ø I love dadjokes,
cheesecake, and Go
@retgits
Leon Stigter, Product Manager App Toolkit
go.retgits.com/mydevsecops
@retgits | @MyDevSecOps
10Confidential │ ©2020 VMware, Inc.
“Y’all only do VMs at
VMware, right?”
- Pretty much everyone, ever
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 11
We deeply care about
building apps, for
example with the
Kubernetes ecosystem…
@retgits | @MyDevSecOps
12Confidential │ ©2020 VMware, Inc.
But I’m Not Here To
Sell You Things…
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 13
Let’s get back to DevOps
@retgits | @MyDevSecOps
14Confidential │ ©2020 VMware, Inc.
“software
deployment” should
be high quality and
repeat-able as it
“leaves the factory”
@retgits | @MyDevSecOps
15Confidential │ ©2020 VMware, Inc.
“Efficiency comes
through the (re)use
of common
components”
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 16
CI/CD Pipelines are the vehicle for deployment
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 17
Most operations checks are done post deployment
Re-deploy
@retgits | @MyDevSecOps
Cost
Security
Utilization
Performance
Confidential │ ©2020 VMware, Inc. 18
Continuous Verification
“A process of querying external
system(s) and using information from
the response to make decision(s) to
improve the development and
deployment process.”
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 19
Increase Efficiency And Reduce Operational Overhead
Traditional Pipeline
Extending Pipeline(s)
External
Actor
External
Actor
Continuous
Verification
@retgits | @MyDevSecOps
20Confidential │ ©2020 VMware, Inc.
“Automating your entire
pipeline, including cost
and governance, gives you
more time to spend on the
beach”
#BeachOps
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 21
“A process of querying
external system(s) and using
information from the
response to make
decision(s) to improve the
development and
deployment process.”
ØContinuous Security (environment
validations, deployment security, etc)
ØCost and Resource Management in
CI/CD
ØImage Verification
ØAuthorization (normalized IAM/RBAC
between AWS, Azure, GCP, and on
prem)
ØApplication performance and SLA
Continuous Verification
What is it?
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 22
Continuous Verification Scenario
Do we have capacity &
budget in the
project/account/etc? Do we follow best
practices and
security guidelines
for accounts and
permissions?
Look at performance
stats in test/dev
environment
Are my dependencies
secure?
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 23
Answering questions, like
A)Will this deployment exceed the budget now?
B) Is there enough capacity for resources in EC2, RDS, etc. below the utilization budget?
C) What are the optimal resources (i.e. rightsized EC2 instance, RI, etc) to use for this deployment?
D) Is the deployment running within budget with the right resources and are there recommendations?
Because money is important…
Cost
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 24
Clair Vulnerability scans
EC2 security group - port 22 is
publicly accessible
S3 Bucket or Blob Storage is
open to the world
Database instances (RDS / S3)
are not encrypted
User has active keys but has
not rotated them
AWS VPC Flow logs / Azure
Network Security Group Flow
Logs - “What’s happening in
my cloud network?”
AWS Cloud Trail / Azure
Activity Log - “Who, when and
what CRUD operations were
performed on which services
in the account?”
AWS Guard Duty / Azure
Advanced Threat Protection -
“That looks sketchy !!”
Avoids data exposure to
anonymous users/applications
Prevents privileged escalation
and unauthorized access to
resources
Ensures compliance with
regulations like PCI, HIPPA
Security
Safety matters to everyone
Pre Deployment Post Deployment Why In A Pipeline?
Confidential │ ©2020 VMware, Inc. 25
Open Source Security
Source: State of Open Source Security Report 2020, Snyk
Confidential │ ©2020 VMware, Inc. 26
Answers questions, like:
Does my K8s cluster have enough capacity to deploy this new app
Where are the bottlenecks in my architecture
Are the spikes in CPU usage normal
How can I visualize which path traffic takes through my architecture
Is there latency issues on my app using high volume traffic generation
Can the database scale and support the needed capacity
The User experience matters too!
Performance Checks
Confidential │ ©2020 VMware, Inc. 27
Let’s Go See That in
Action…
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 28
One Small Disclaimer…
I’m about to use tools that I’m comfortable
with, you should use tools that you choose…
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc. 29
Recap
Do we have capacity &
budget in the
project/account/etc?
Do we follow best
practices and
security guidelines
for accounts and
permissions?
Look at performance
stats in test/dev
environmentAre my dependencies
secure?
@retgits | @MyDevSecOps
Confidential │ ©2020 VMware, Inc.
Thank You
@retgits
The awesome icons in this deck are made by Freepik and Flat Icons from www.flaticon.com
retgits.com
go.retgits.com/ mydevsecops

Data Driven Decisions in DevOps

  • 1.
    Confidential │ ©2020VMware, Inc. Data Driven Decisions in DevOps Add More To Your Pipelines Leon Stigter Product Manager @VMware July 2nd, 2020
  • 2.
    2Confidential │ ©2020VMware, Inc. Technology Is Changing Rapidly @retgits | @MyDevSecOps
  • 3.
    3Confidential │ ©2020VMware, Inc. Changing Technology Means Changing Infrastructure @retgits | @MyDevSecOps
  • 4.
    4Confidential │ ©2020VMware, Inc. Changing Infrastructure Changing Means Rethink How You Deploy… @retgits | @MyDevSecOps
  • 5.
    Confidential │ ©2020VMware, Inc. 5 2020 stack Gone are the “simple” days of centralized stacks Application Architectures Are Changing 2000’s stack App Database Webserver Environment App Environment Cache Database Logs @retgits | @MyDevSecOps
  • 6.
    6Confidential │ ©2020VMware, Inc. “Amazon.com does 136,000 fully automated deployments per day” Ken Exner, Director AWS Dev Resources Source: https://techbeacon.com/devops/going-big-devops-how-scale-continuous-delivery-success @retgits | @MyDevSecOps
  • 7.
    Confidential │ ©2020VMware, Inc. 7 Applications Are The New Unit of Measure For Ops AWS does 136,000 fully automated deployments every day, but… Ø Less than 50% of organizations has CI/CD pipelines Ø Less than 6% has fully automated CI/CD pipelines Deployments @retgits | @MyDevSecOps
  • 8.
    8Confidential │ ©2020VMware, Inc. “I really want to spend my day sitting in front of screens watching DevOps pipelines…” - No one ever… @retgits | @MyDevSecOps
  • 9.
    Confidential │ ©2020VMware, Inc. 9 Who am I? Ø Product Manager, turned Developer Advocate, turned Product Manager again… Ø Passionate about Serverless, Containers, and all things Cloud Ø I love dadjokes, cheesecake, and Go @retgits Leon Stigter, Product Manager App Toolkit go.retgits.com/mydevsecops @retgits | @MyDevSecOps
  • 10.
    10Confidential │ ©2020VMware, Inc. “Y’all only do VMs at VMware, right?” - Pretty much everyone, ever @retgits | @MyDevSecOps
  • 11.
    Confidential │ ©2020VMware, Inc. 11 We deeply care about building apps, for example with the Kubernetes ecosystem… @retgits | @MyDevSecOps
  • 12.
    12Confidential │ ©2020VMware, Inc. But I’m Not Here To Sell You Things… @retgits | @MyDevSecOps
  • 13.
    Confidential │ ©2020VMware, Inc. 13 Let’s get back to DevOps @retgits | @MyDevSecOps
  • 14.
    14Confidential │ ©2020VMware, Inc. “software deployment” should be high quality and repeat-able as it “leaves the factory” @retgits | @MyDevSecOps
  • 15.
    15Confidential │ ©2020VMware, Inc. “Efficiency comes through the (re)use of common components” @retgits | @MyDevSecOps
  • 16.
    Confidential │ ©2020VMware, Inc. 16 CI/CD Pipelines are the vehicle for deployment @retgits | @MyDevSecOps
  • 17.
    Confidential │ ©2020VMware, Inc. 17 Most operations checks are done post deployment Re-deploy @retgits | @MyDevSecOps Cost Security Utilization Performance
  • 18.
    Confidential │ ©2020VMware, Inc. 18 Continuous Verification “A process of querying external system(s) and using information from the response to make decision(s) to improve the development and deployment process.” @retgits | @MyDevSecOps
  • 19.
    Confidential │ ©2020VMware, Inc. 19 Increase Efficiency And Reduce Operational Overhead Traditional Pipeline Extending Pipeline(s) External Actor External Actor Continuous Verification @retgits | @MyDevSecOps
  • 20.
    20Confidential │ ©2020VMware, Inc. “Automating your entire pipeline, including cost and governance, gives you more time to spend on the beach” #BeachOps @retgits | @MyDevSecOps
  • 21.
    Confidential │ ©2020VMware, Inc. 21 “A process of querying external system(s) and using information from the response to make decision(s) to improve the development and deployment process.” ØContinuous Security (environment validations, deployment security, etc) ØCost and Resource Management in CI/CD ØImage Verification ØAuthorization (normalized IAM/RBAC between AWS, Azure, GCP, and on prem) ØApplication performance and SLA Continuous Verification What is it? @retgits | @MyDevSecOps
  • 22.
    Confidential │ ©2020VMware, Inc. 22 Continuous Verification Scenario Do we have capacity & budget in the project/account/etc? Do we follow best practices and security guidelines for accounts and permissions? Look at performance stats in test/dev environment Are my dependencies secure? @retgits | @MyDevSecOps
  • 23.
    Confidential │ ©2020VMware, Inc. 23 Answering questions, like A)Will this deployment exceed the budget now? B) Is there enough capacity for resources in EC2, RDS, etc. below the utilization budget? C) What are the optimal resources (i.e. rightsized EC2 instance, RI, etc) to use for this deployment? D) Is the deployment running within budget with the right resources and are there recommendations? Because money is important… Cost @retgits | @MyDevSecOps
  • 24.
    Confidential │ ©2020VMware, Inc. 24 Clair Vulnerability scans EC2 security group - port 22 is publicly accessible S3 Bucket or Blob Storage is open to the world Database instances (RDS / S3) are not encrypted User has active keys but has not rotated them AWS VPC Flow logs / Azure Network Security Group Flow Logs - “What’s happening in my cloud network?” AWS Cloud Trail / Azure Activity Log - “Who, when and what CRUD operations were performed on which services in the account?” AWS Guard Duty / Azure Advanced Threat Protection - “That looks sketchy !!” Avoids data exposure to anonymous users/applications Prevents privileged escalation and unauthorized access to resources Ensures compliance with regulations like PCI, HIPPA Security Safety matters to everyone Pre Deployment Post Deployment Why In A Pipeline?
  • 25.
    Confidential │ ©2020VMware, Inc. 25 Open Source Security Source: State of Open Source Security Report 2020, Snyk
  • 26.
    Confidential │ ©2020VMware, Inc. 26 Answers questions, like: Does my K8s cluster have enough capacity to deploy this new app Where are the bottlenecks in my architecture Are the spikes in CPU usage normal How can I visualize which path traffic takes through my architecture Is there latency issues on my app using high volume traffic generation Can the database scale and support the needed capacity The User experience matters too! Performance Checks
  • 27.
    Confidential │ ©2020VMware, Inc. 27 Let’s Go See That in Action… @retgits | @MyDevSecOps
  • 28.
    Confidential │ ©2020VMware, Inc. 28 One Small Disclaimer… I’m about to use tools that I’m comfortable with, you should use tools that you choose… @retgits | @MyDevSecOps
  • 29.
    Confidential │ ©2020VMware, Inc. 29 Recap Do we have capacity & budget in the project/account/etc? Do we follow best practices and security guidelines for accounts and permissions? Look at performance stats in test/dev environmentAre my dependencies secure? @retgits | @MyDevSecOps
  • 30.
    Confidential │ ©2020VMware, Inc. Thank You @retgits The awesome icons in this deck are made by Freepik and Flat Icons from www.flaticon.com retgits.com go.retgits.com/ mydevsecops