This document summarizes James Wickett's presentation at DevOps Days Kansas City. It discusses Wickett's journey in IT and his involvement with DevOps. It outlines some of his initial questions about DevOps culture and whether it has been distorted from its original goals. The presentation then contrasts the traditional "old path" of IT operations with a proposed "new path" that more fully incorporates DevOps and security best practices like feedback loops, non-blocking processes, and testing approaches like chaos engineering.
Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...
DevOps Days KC - The Path of DevOps Enlightenment for InfoSec
1. DevOps Days Kansas City @WICKETT
THE PATH OF
DEVOPS ENLIGHTENMENT
FOR INFOSEC
JAMES WICKETT
SIGNAL SCIENCES
2. DevOps Days Kansas City @WICKETT
Want the slides?
james@signalsciences.com
3. DevOps Days Kansas City @WICKETT
‣ HEAD OF RESEARCH AT SIGNAL SCIENCES
‣ ORGANIZER OF DEVOPS DAYS AUSTIN
‣ LYNDA.COM AUTHOR ON DEVOPS
‣ BLOG AT THEAGILEADMIN.COM
@WICKETT
4. DevOps Days Kansas City @WICKETT
‣ WHY DO WE HAVE DEVOPS?
‣ DID WE BUILD DEVOPS PROPERLY?
‣ IS THE DEVOPS CULTURE LOST?
‣ CAN WE GET IT BACK?
‣ CAN WE PROTECT DEVOPS FROM FURTHER
DISTORTION?
QUESTIONS ON MY MIND
6. DevOps Days Kansas City @WICKETT
‣ WEB AND ECOMM FOR $1B COMPANY
‣ BRUTAL ONCALL ROTATIONS
‣ +24HR DEPLOYMENTS
‣ WATERFALL, WATERFALL, WATERFALL
‣ FRIENDS ARE BORN FROM ADVERSITY
FIRST BIGCO JOB
7. DevOps Days Kansas City @WICKETT
‣ IN 2007 WENT STARTUP AND AWS CLOUD
‣ LEARNED A BIT ABOUT FAILURE AND
HAPPINESS
‣ REJOINED OLD TEAM IN 2010 FOR NEW CLOUD
VENTURE BACK IN BIGCO
CLOUDING FOR PROFIT
8. DevOps Days Kansas City @WICKETT
‣ DEVOPS AND INFRA AS CODE
‣ NOT CD, BUT DEPLOYS DAILY
‣ AT BIGCO DELIVERED 4 SAAS PRODUCTS IN 2
YEARS WITH DEVOPS AND CLOUD
ENTER DEVOPS
9. DevOps Days Kansas City @WICKETT
‣ FOUND RUGGED SOFTWARE
‣ MET GENE KIM IN 2012 IN A BAR IN AUSTIN
‣ CREATED GAUNTLT
‣ LATER, JOINED SIGNAL SCIENCES
DEVOPS AND SECURITY
20. DevOps Days Kansas City @WICKETT
‣ WHY DO WE HAVE DEVOPS?
‣ DID WE BUILD DEVOPS PROPERLY?
‣ IS THE DEVOPS CULTURE LOST?
‣ CAN WE GET IT BACK?
‣ CAN WE PROTECT DEVOPS FROM FURTHER
DISTORTION?
QUESTIONING DEVOPS
23. DevOps Days Kansas City @WICKETT
‣ TEACH THREE DEVOPS CLASSES IN THE DEVOPS
FOUNDATIONS SERIES AT LYNDA / LINKEDIN
LEARNING
‣ WRITE DEVOPS AND SECURITY ARTICLES AS
PART OF MY ROLE AT SIGNAL SCIENCES
31. DevOps Days Kansas City @WICKETT
Companies are spending a great deal on
security, but we read of massive computer-
related attacks. Clearly something is wrong.
The root of the problem is twofold:
we’re protecting the wrong things,
and we’re hurting productivity in the process.
THINKING SECURITY, STEVEN M. BELLOVIN 2015
33. DevOps Days Kansas City @WICKETT
[Security by risk assessment]
introduces a dangerous fallacy: that
structured inadequacy is almost as
good as adequacy and that
underfunded security efforts plus risk
management are about as good as
properly funded security work
34. DevOps Days Kansas City @WICKETT
Security is often the
cultural outlier in an
organization
35. DevOps Days Kansas City @WICKETT
many security teams work
with a worldview where their
goal is to inhibit change as
much as possible
36. DevOps Days Kansas City @WICKETT
“SECURITY PREFERS A SYSTEM POWERED
OFF AND UNPLUGGED”
- DEVELOPER
37. DevOps Days Kansas City @WICKETT
“…THOSE STUPID DEVELOPERS”
- SECURITY PERSON
38. DevOps Days Kansas City @WICKETT
It is 30 times cheaper to
fix security defects in dev
vs. Prod
NIST, 2002, The Economic Impacts of Inadequate Infra for Software Testing
39. DevOps Days Kansas City @WICKETT
It is 30 times cheaper to
fix security defects in dev
vs. Prod
NIST, 2002, The Economic Impacts of Inadequate Infra for Software Testing
41. DevOps Days Kansas City @WICKETT
“every aspect of managing WAFs is an ongoing
process. This is the antithesis of set it and forget it
technology. That is the real point of this research.
To maximize value from your WAF you need to go
in with everyone’s eyes open to the effort required
to get and keep the WAF running productively.”
- WHITEPAPER FROM AN UNDISCLOSED WAF VENDOR
44. DevOps Days Kansas City @WICKETT
THE AVERAGE TIME TO DELIVER CORPORATE IT PROJECTS
HAS INCREASED FROM ~8.5 MONTHS TO OVER 10
MONTHS IN THE LAST 5 YEARS
Revving up your Corporate RPMs, Fortune Magazine, Feb 1, 2016
THE GROWTH OF [SECURITY] FUNCTIONS WHICH IS TOO
OFTEN POORLY COORDINATED… [RESULTING IN] A
PROLIFERATION OF NEW TASKS IN THE AREAS OF
COMPLIANCE, PRIVACY AND DATA PROTECTION.
45. DevOps Days Kansas City @WICKETT
Many security professionals
have a hard time adapting their
existing practices to a world
where requirements can change
every few weeks, or where they
are never written down at all.
48. DevOps Days Kansas City @WICKETT
DevOps
A New Traveling Companion
for Security
(…and probably the only way to survive)
49. DevOps Days Kansas City @WICKETT
High performers spend 50 percent less
time remediating security issues than
low performers.
By better integrating information security
objectives into daily work, teams achieve
higher levels of IT performance and build
more secure systems.
2016 State of DevOps Report
50. DevOps Days Kansas City @WICKETT
High performing orgs achieve
quality by incorporating
security (and security teams)
into the delivery process
2016 State of DevOps Report
51. DevOps Days Kansas City @WICKETT
http://www.youtube.com/watch?v=jQblKuMuS0Y
53. DevOps Days Kansas City @WICKETT
OLD PATH VS. NEW PATH
Embrace Secrecy Create Feedback Loops
Just Pass Audit! Compliance adds Value
Enforce Stability Create Chaos
Build a Wall Zero Trust Networks
Slow Validation Fast and Non-blocking
Certainty Testing Adversity Testing
Test when Done Shift Left
Process Driven The Paved Road
54. DevOps Days Kansas City @WICKETT
OLD PATH VS. NEW PATH
Embrace Secrecy Create Feedback Loops
Just Pass Audit! Compliance adds Value
Enforce Stability Create Chaos
Build a Wall Zero Trust Networks
Slow Validation Fast and Non-blocking
Certainty Testing Adversity Testing
Test when Done Shift Left
Process Driven The Paved Road
55. DevOps Days Kansas City @WICKETT
A security team who embraces
openness about what it does and
why, spreads understanding.
- Rich Smith
56. DevOps Days Kansas City @WICKETT
Runtime is arguably the
most important place to
create feedback loops
57. DevOps Days Kansas City @WICKETT
‣ ACCOUNT TAKEOVER ATTEMPTS
‣ AREAS OF THE SITE UNDER ATTACK
‣ MOST LIKELY VECTORS OF ATTACK
‣ BUSINESS LOGIC FLOWS
DETECT WHAT MATTERS
61. DevOps Days Kansas City @WICKETT
Options: RASP, NGWAF or
Web Protection Platform
62. DevOps Days Kansas City @WICKETT
OLD PATH VS. NEW PATH
Embrace Secrecy Create Feedback Loops
Just Pass Audit! Compliance adds Value
Enforce Stability Create Chaos
Build a Wall Zero Trust Networks
Slow Validation Fast and Non-blocking
Certainty Testing Adversity Testing
Test when Done Shift Left
Process Driven The Paved Road
63. DevOps Days Kansas City @WICKETT
‣ POLICIES AND PROCEDURES IN PLACE
‣ EFFECTIVE EXECUTION OF THOSE POLICIES TO
ALLOW YOU TO KEEP FUNCTIONING
‣ MOST OF PCI AND OTHER FRAMEWORKS
PROVIDE REASONABLY GOOD PRACTICES *IF*
YOU REMOVE ALL THE WATERFALL BITS
UNDERSTAND AUDITORS
64. DevOps Days Kansas City @WICKETT
[Deploys] can be treated as
standard or routine changes
that have been pre-approved
by management, and that
don’t require a heavyweight
change review meeting.
66. DevOps Days Kansas City @WICKETT
Developers with Access to
Production, Oh My!!!
https://www.schellmanco.com/blog/2012/12/auditing-devops-
developers-with-access-to-production/
67. DevOps Days Kansas City @WICKETT
Check out DevOps Audit
Defense Toolkit
https://cdn2.hubspot.net/hubfs/228391/Corporate/
DevOps_Audit_Defense_Toolkit_v1.0.pdf
68. DevOps Days Kansas City @WICKETT
OLD PATH VS. NEW PATH
Embrace Secrecy Create Feedback Loops
Just Pass Audit! Compliance adds Value
Enforce Stability Create Chaos
Build a Wall Zero Trust Networks
Slow Validation Fast and Non-blocking
Certainty Testing Adversity Testing
Test when Done Shift Left
Process Driven The Paved Road
69. DevOps Days Kansas City @WICKETT
‣ ADD IN CHAOS TO YOUR SYSTEM AND
APPLICATION
‣ CHAOS MONKEY
‣ ANTI-FRAGILE
‣ RELEASE IT! BOOK
CHAOS ENGINEERING
71. DevOps Days Kansas City @WICKETT
‣ ADDS MISCONFIG TO THE STACK AND CHECKS
TO SEE IF IT GETS DETECTED
‣ NEW OPEN SOURCE TOOL!
‣ RUNS AS A LAMBDA
CHAOS SLINGR
72. DevOps Days Kansas City @WICKETT
‣ I AM BEING PEN TESTED ANYWAY, WHY NOT
FIND OUT WHAT THEY ARE FINDING?
‣ 24/7 PEN TESTING
‣ BUILDS DEVELOPER CONFIDENCE
‣ FINDS MIX OF LOW HANGING FRUIT AND
SOMETIMES MUCH MORE!
BUG BOUNTIES
73. DevOps Days Kansas City @WICKETT
OLD PATH VS. NEW PATH
Embrace Secrecy Create Feedback Loops
Just Pass Audit! Compliance adds Value
Enforce Stability Create Chaos
Build a Wall Zero Trust Networks
Slow Validation Fast and Non-blocking
Certainty Testing Adversity Testing
Test when Done Shift Left
Process Driven The Paved Road
75. DevOps Days Kansas City @WICKETT
‣ NO PERIMETER SECURITY
‣ ASSUME COMPROMISE
‣ INSTRUMENT ALL LAYERS
‣ EXTENDS FROM LAPTOPS TO WEB
APPS TO CUSTOMER ACCOUNTS
ZERO TRUST NETWORKS
76. DevOps Days Kansas City @WICKETT
OLD PATH VS. NEW PATH
Embrace Secrecy Create Feedback Loops
Just Pass Audit! Compliance adds Value
Enforce Stability Create Chaos
Build a Wall Zero Trust Networks
Slow Validation Fast and Non-blocking
Certainty Testing Adversity Testing
Test when Done Shift Left
Process Driven The Paved Road
77. DevOps Days Kansas City @WICKETT
‣ DON’T SLOW DELIVERY
‣ CONTINUOUS TESTING AND VALIDATION
‣ TESTING ON THE SIDE OF THE PIPELINE
‣ PENETRATION TESTING OUTSIDE OF DELIVERY
FAST AND NON-BLOCKING
78. DevOps Days Kansas City @WICKETT
Currently, at Signal
Sciences we do about 15
deploys per day
79. DevOps Days Kansas City @WICKETT
Roughly 10,000 deploys in
the last 2.5 yrs
84. DevOps Days Kansas City @WICKETT
Signal Sciences is a
software as a service
company and a security
company
85. DevOps Days Kansas City @WICKETT
Security is part of CI/CD
and the overall delivery
pipeline
86. DevOps Days Kansas City @WICKETT
‣DESIGN
‣INHERIT
‣BUILD
‣DEPLOY
‣OPERATE
PIPELINE PHASES
87. DevOps Days Kansas City @WICKETT
‣INHERIT
‣BUILD
‣OPERATE
SECURITY
CONSIDERATIONS
What have I bundled into my
app that leaves me
vulnerable?
Do my build acceptance
tests and integration tests
catch security issues before
release?
Am I being attacked right
now? Is it working?
88. DevOps Days Kansas City @WICKETT
OLD PATH VS. NEW PATH
Embrace Secrecy Create Feedback Loops
Just Pass Audit! Compliance adds Value
Enforce Stability Create Chaos
Build a Wall Zero Trust Networks
Slow Validation Fast and Non-blocking
Certainty Testing Adversity Testing
Test when Done Shift Left
Process Driven The Paved Road
90. DevOps Days Kansas City @WICKETT
The goal should be to come up with a
set of automated tests that probe and
check security configurations and
runtime system behavior for security
features that will execute every time
the system is built and every time it is
deployed.
91. DevOps Days Kansas City @WICKETT
Security tools are
intractably noisy and
difficult to use
92. DevOps Days Kansas City @WICKETT
A method of collaboration
was needed for devs, ops
and security eng.
93. DevOps Days Kansas City @WICKETT
There needed to be a new
language to span the
parties
96. DevOps Days Kansas City @WICKETT
Open source, MIT License
Gauntlt comes with pre-canned steps that
hook security testing tools
Gauntlt does not install tools
Gauntlt wants to be part of the CI/CD pipeline
Be a good citizen of exit status and stdout/
stderr
101. DevOps Days Kansas City @WICKETT
$ gem install gauntlt
# download example attacks from github
# customize the example attacks
# now you can run gauntlt
$ gauntlt
102. DevOps Days Kansas City @WICKETT
@slow @final
Feature: Look for cross site scripting (xss) using arachni
against a URL
Scenario: Using arachni, look for cross site scripting and verify
no issues are found
Given "arachni" is installed
And the following profile:
| name | value |
| url | http://localhost:8008 |
When I launch an "arachni" attack with:
"""
arachni —check=xss* <url>
"""
Then the output should contain "0 issues were detected."
Given
When
Then
What?
103. DevOps Days Kansas City @WICKETT
“We have saved millions of
dollars using Gauntlt for the
largest healthcare industry
project.”
- Aaron Rinehart, UnitedHealthCare
105. DevOps Days Kansas City @WICKETT
‣ 8 LABS FOR GAUNTLT
‣ HOW TO USE GAUNTLT FOR NETWORK CHECKS
‣ GAUNTLT FOR XSS, SQLI, OTHER APSES
‣ HANDLING REPORTING
‣ USING ENV VARS
‣ CI SYSTEM SETUP
WORKSHOP INCLUDES:
111. DevOps Days Kansas City @WICKETT
Most teams use Gauntlt
in Docker containers
112. DevOps Days Kansas City @WICKETT
https://github.com/
gauntlt/gauntlt-docker
113. DevOps Days Kansas City @WICKETT
OLD PATH VS. NEW PATH
Embrace Secrecy Create Feedback Loops
Just Pass Audit! Compliance adds Value
Enforce Stability Create Chaos
Build a Wall Zero Trust Networks
Slow Validation Fast and Non-blocking
Certainty Testing Adversity Testing
Test when Done Shift Left
Process Driven The Paved Road
117. DevOps Days Kansas City @WICKETT
OVER 30% OF OFFICIAL IMAGES IN
DOCKER HUB CONTAIN HIGH PRIORITY
SECURITY VULNERABILITIES
https://banyanops.com/blog/analyzing-docker-hub/
118. DevOps Days Kansas City @WICKETT
OLD PATH VS. NEW PATH
Embrace Secrecy Create Feedback Loops
Just Pass Audit! Compliance adds Value
Enforce Stability Create Chaos
Build a Wall Zero Trust Networks
Slow Validation Fast and Non-blocking
Certainty Testing Adversity Testing
Test when Done Shift Left
Process Driven The Paved Road
119. DevOps Days Kansas City @WICKETT
‣ MAKE IT EASY FOR PEOPLE TO DO THE RIGHT
THING
‣ JASON CHAN, NETFLIX
‣ GOLD IMAGES
‣ BLESSED BUILDS AND DEPENDENCIES
THE PAVED ROAD
120. DevOps Days Kansas City @WICKETT
Don’t be a blocker, be an
enabler of the business
121. DevOps Days Kansas City @WICKETT
Contact me
james@signalsciences.com
@wickett