MEASUREAwaytothinkaboutDevSecOps
@wickett
JamesWickett
Head of Research @ Verica.io
Author, DevOps on LinkedIn Learning
Organizer, DevOps Days Austin, DevSecOps Days Austin
@wickett
@wickett
Getthe slides ontwitter
@wickett
The Storyofa
StartupinaLarge
Enterprise
@wickett
William Crush
@wickett
Disruptingthe
Entertainment
Industry
@wickett
MVP: Google
Ads +
KickStarter
Campaign
@wickett
September 15
1896@wickett
Crush,Texas
Population 40,000
@wickett
@wickett
@wickett
SafetyMargin
@wickett
@wickett
4pm, September 15th,
1896
@wickett
“The rumble ofthetwo
trains, faintand far offat
firstbut growing nearer
and more distinctwith
each fleeting second,was
likethe gathering force ofa
cyclone”
@wickett
@wickett
@wickett
UnmitigatedVulnerability
inthe System
@wickett
Steam, Iron,Wood
filledthe sky
@wickett
@wickett
@wickett
Aftermath:
» 4 people died
» Crush fired
» Widespread injuries
» Lawyers brought in for settlements
@wickett
@wickett
@wickett
Worked for
MKT44 more
years
@wickett
CopycatEvents
@wickett
Learnings
» Safety Margin exists in all Systems
» Configuration errors and bullwhip effect
» Experimentation can find vulnerabilities
» Root cause is a myth
@wickett
DEVSECOPS
@wickett
DevOps isan epistemological
breakthroughthatjoins disparate
peoplearoundacommon problem ina
distrubuted computearchitecture
@wickett
10:1
DEV:OPS
@wickett
Securityfinds itselfinthe
same positionthat
operations did inthe
movementofDevOps
@wickett
100:10:1
DEV:OPS:SEC
@wickett
Security's
Siloization
@wickett
“Companiesare
spendingagreatdeal
on security, butwe
read ofmassive
computer-related
attacks. Clearly
something iswrong.
The rootofthe problem
istwofold:we’re
protectingthewrong
things,andwe’re
hurting productivity
inthe process.”
“While engineeringteams
are busy deploying
leading-edgetechnologies,
securityteamsare still
focused on fighting
yesterday’s battles.”
SANS 2018 DevSecOps Survey
@wickett
"manysecurity
teamsworkwitha
worldviewwhere
their goalisto
inhibit change
as muchas possible"
A Highly Desireable New Breed:
The DevSecOp
@wickett
...notatool
...notaCI/CD pipeline
...can’tbe bought
@wickett
The DevSecOpan inclusive person participating in
the movementofsecurityinto devops.
@wickett
DevSecOps Framework:
MEASURE
@wickett
Maker Driven
@wickett
Maker Driven
Experimentation
@wickett
Maker Driven
Experimentation
Automation
@wickett
Maker Driven
Experimentation
Automation
Safety
@wickett
Maker Driven
Experimentation
Automation
Safety
Unrestrained Sharing
@wickett
Maker Driven
Experimentation
Automation
Safety
Unrestrained Sharing
Rugged
@wickett
Maker Driven
Experimentation
Automation
Safety
Unrestrained Sharing
Rugged
Empathy
@wickett
Maker Driven
@wickett
Securityisaspecific
discipline ofsoftware
engineering
@wickett
Securitysolves problems by
writing code
@wickett
Whyisthis considered
ahottake in our industry?
@wickett
Securityalreadyuses DSLs
@wickett
@wickett
ASecurityTeamthatParticipates
in Software Delivery
» Empathy building
» Familiarity with tools and teams
» Able to shift left in the pipeline
» Policy as Code approach
@wickett
Abug isabug isabug
@wickett
Defectdensity
is never zero
@wickett
My500 LOC can
easilybe
400,000 LOC IRL
Securityand Developer Collab
» TDD/BDD/ATDD
» Team Standards, reviews/config/comments/commits
» Code Smells, Patterns, Refactoring
» Instrumentation, Observability
@wickett
“The goalshould beto
come upwithasetof
automatedtests that
probeand check
security
configurations and
runtime system
behavior for
securityfeatures
thatwillexecute
everytimethe system
is builtand every
time itis deployed.”
Maker Driven means
» See security as part of engineering
» Use code, not vendors to solve problems
» View quality as a way to bring security in
@wickett
MEASURE
@wickett
Experimentation
@wickett
@wickett
“Securityincidents
are not effective
measures ofdetection
becauseatthatpoint
it'salreadytoo late”
Aaron Rinehart
@wickett
Create Objective
Feedback Loops
aboutSecurity
Effectiveness
@wickett
“The securitydiscipline of
experimentation is done in
orderto build confidence
inthe system’sabilityto
defendagainstmalicious
conditions.”
@wickett
SecurityChaos Engineering (SCE)
» Experiments that span eng and security
» Manual opt-out
» Valuable Learning
» Controlled experiment blast radius
@wickett
“[Chaos Engineering is]
empiricalratherthan formal.
We don’tuse modelsto
understandwhatthe system
should do.We run experiments
to learnwhat itdoes.”
Michael Nygard, Release It 2nd Ed.
@wickett
SecurityProblems in Complex Systems
» Configuration drift over time
» Regressions in code
» Role and privilege drift
» Additive code or microservices
» Security controls in wrong locations
» Bullwhip effect
@wickett
SCE does not
» validate a config, it exercises it
» check auth privileges, it attempts to thwart them
» trust network settings, it sends real traffic
» check app policy, it interacts with the
application
@wickett
4 Steps ofSecurityChaos
Engineering
» Define expected behavior of a security defense
» Hypothesize that when security turbulence is introduced
it will be either prevented, remediated, or detected.
» Introduce a variable that introduces security turbulence.
» Try to disprove the hypothesis by looking for a
difference in expected behavior and actual behavior
@wickett
Benefitsto Experimentation
» Measured, Repeatable
» Results based on your needs
» Actionable Outcomes
» A proven method to uncover truths in complex
systems
@wickett
Resources
» principlesofchaos.org
» Release It! 2nd ed., Nygard
» DevOps Ent Summit Talk youtu.be/yuOuVC8xljw
» Chaos Engineering, Rosenthal and Jones verica.io/
book
@wickett
MEASURE
@wickett
Automation
@wickett
“[Deploys] can be
treatedas
standard or
routine changes
thathave been
pre-approved by
management,and
thatdon’trequire
a heavyweight
change review
meeting.”
“Continuous
Deliveryis how
littleyou can
deployatonetime”
Jez Humble & David Farley
@wickett
Securityinthe Pipeline
» Software composition analysis
» Lang linters, git-hound, ...
» Scanners, gauntlt
» Monitoring and telemetry
@wickett
linkedin.com/learning/devsecops-building-a-secure-
continuous-delivery-pipeline
@wickett
linkedin.com/learning/devsecops-automated-security-
testing
@wickett
MEASURE
@wickett
Safety
@wickett
RootCause isaMyth in Complex
Systems
» Lacks full picture
» Complex systems are not linear
» Result of blame culture
» Forgets organizational decisions
» Puts the focus on the event over situation
@wickett
Simple Systems:
» Linear in nature
» Easy to Predict
» Able to comprehend
@wickett
Complex Systems:
» Non-linear (bullwhip effect)
» Unpredictable in nature
» No mental model available
@wickett
“Failure isan
inevitable by-
productofa
complex
system's
normal
functioning”
@wickett
“Failures are a
systems problem
because there is not
enough safety
margin. ”
@adrianco
@wickett
Where SecurityFits
» Know your safety margin
» Stop root cause analysis, go blameless retros
» Telemetry and instrumentation
» ...more to explore in this area
@wickett
Resources
» Drift into Failure by Dekker
» Understanding Human Error Video Series youtu.be/
Fw3SwEXc3PU
» Richard Cook paper bit.ly/2ydDQS2
@wickett
MEASURE
@wickett
Unrestrained
Sharing
@wickett
“Asecurityteamwho
embraces openness
aboutwhatitdoes
and why, spreads
understanding.”
Rich Smith
@wickett
20% ofdevelopers
don'tknowwhat
securityexpects of
them
@wickett
Againstsecurity's
standard operating
procedure
@wickett
Four Keysto DevSecOps Culture
» Mutual Understanding
» Shared Language
» Shared Views
» Collaborative Tooling
@wickett
Resources
» Phoenix Project
» Agile Application Security
» dearauditor.org
@wickett
MEASURE
@wickett
Rugged
@wickett
@wickett
Software BillofMaterials
@wickett
Favor ShortLived Systems
Cattle notPets
@wickett
Distributed
Immutable
Ephemeralsource: @sounilyu
@wickett
Rugged in 20211.Advanced Deception
2. ContinuousVerification
@wickett
Deception
» Honeypots, Tarpits, Mantraps
» Simple to get started (http headers)
» HoneyPy, DeceptionLogic
@wickett
Resources
» Aaron Rinehart's talk at RSA youtu.be/wLlME4Ve1go
» Phillip Maddux's talk: youtu.be/k81xKjCEeqE
» Herb Todd's talk: youtu.be/Cf_XXmRLnRQ
@wickett
MEASURE
@wickett
Empathy
@wickett
Developers don't
have enoughtimeto
spend on security
“those stupid
developers”
Security
@wickett
“youwantamachine
powered offand
unplugged”
Developer
@wickett
“Culture isthe most
importantaspectto
devops succeeding
inthe enterprise”
Patrick DeBois
@wickett
Maker Driven
Experimentation
Automation
Safety
Unrestrained Sharing
Rugged
Empathy
@wickett
Complimentary copy of the
Chaos Engineering book
verica.io/book
@wickett
Getthe slides ontwitter
@wickett

A Way to Think about DevSecOps: MEASURE