1.
DevOps for the

2.
James Wickett
james@wickett.me
Austin, TX
Work at Signal Sciences
Rugged Dev Podcast
Gauntlt Core Team
LASCON Founder and Organizer
DevOps Days Austin and Global Organizer

3.
We’re making AppSec effective and practical
signalsciences.com

4.
in honor of SkyMall

5.
If you find
yourself in
Austin, stop by!
Austin OWASP (last
Tuesday of the month)
LASCON Oct 22-23

6.
Conclusions
It is easy to get discouraged in our industry, but
there is hope!
Agile, DevOps and Continuous Delivery practices
have an impact for AppSec / InfoSec
InfoSec is behind but has a unique opportunity to
add value

7.
Conclusions continued
Integrating into the build pipeline and operational
tooling wins
Unit and Integration tests are not enough, we need
testing that focuses on attack tooling

8.
you ==
“person you
once knew at
a previous
company”

9.
Do you feel unable to
cause positive change in
your organization?

10.
Is security ever left out of
important engineering
decisions?

11.
Have you ever reported a serious
internal vulnerability only to see
it open 6 months or more later?

12.
Why we are here

13.
Sony, Sony, Sony, Sony, Sony, Sony, Sony, Sony,
Sony, Sony, Sony, Sony, Sony, Sony, Sony, Sony,
Sony, Sony, Sony, Sony, Sony, Sony, Sony, Sony,
Sony, Sony, Sony, Sony, Sony, Sony, Sony, Sony,
Sony, Sony, Sony, Sony, Sony, Sony, Sony, Sony,
Sony, Sony, Sony, Sony, Sony, Sony, Sony, Sony,
Sony,Sony, Sony, Sony, Sony, Sony, Sony, Sony,
Sony, Sony, Sony, Sony, Sony, Sony, Sony, Sony,
Sony, Sony, Sony, Sony, Sony, Sony, Sony, Sony,
Sony, Sony, Sony, Sony, Sony, Sony, Sony, Sony,
Sony, Sony, Sony, Sony, Sony, Sony, Sony, Sony

14.
Humans optimize for
the probable

15.
We optimize for the
probable

16.
Unit Testing

17.
Integration Testing

18.
Happy Path
Engineering

19.
We optimize for the
possible

20.
Over Engineering

21.
Stress and load testing

22.
We optimize for the
perceived probable

23.
How do we perceive
what is probable?

24.
Epistemological Problem
of Software Development

25.
We attempt to solve it
by gathering data or
rhetoric

26.
3 approaches to solve the
Epistemological Problem
of Software Development

27.
Arc 1:
Agile

28.
Agile side-steps the
problem

29.
Agile says we don’t know
what we are building

30.
Solution: release features
to customers rapidly

31.
Just Ship It!

32.
Behavior Driven Dev

33.
Behavior Driven Development is a second-
generation, outside–in, pull-based,
multiple-stakeholder, multiple-scale, high-
automation, agile methodology. It
describes a cycle of interactions with well-
defined outputs, resulting in the delivery
of working, tested software that matters.
Dan North , 2009

34.
Amplify
the
feedback
loop

35.
TLDR;
Rapid Iterations Win

36.
Agile is our
guiding
light

37.
We don't sell software like we
used to

38.
Software as a Service

39.
The last 15 years have
brought a complete
change in our delivery
cadence, distribution, and
revenue models

40.
DevOps is the
application of Agile
methodology to system
administration
- The Practice of Cloud System
Administration Book

41.
Arc 2: DevOps

42.
Agile Infrastructure
@littleidea @patrickdebois at Velocity 2009
http://itrevolution.com/the-history-of-devops/

43.
http://www.slideshare.net/jallspaw/10-deploys-per-day-dev-and-ops-cooperation-at-flickr

44.
First DevOps Days,
Ghent 2009
@patrickdebois

45.
The opposite of DevOps is despair
- Gene Kim

46.
DevOps is the
operationalism of the world
- Theo Schlossnagle

47.
DevOps is a
community movement

48.
http://dev2ops.org/blog/2010/2/22/what-is-devops.html

49.
DevOps realized that
Ops doesn't know what
Devs know and vice
versa

50.
DevOps is an
epistemological
breakthrough joining
disparate people around
a common problem

51.
DevOps is an inclusive
movement that
codifies a culture
- Adam Jacobs

52.
Dev : Ops
10 : 1
Traditional Dev to
Ops Ratio

53.
“That the word #devops gets
reduced to technology is a
manifestation of how badly
we need a cultural shift”
- @patrickdebois
http://www.slideshare.net/cm6051/london-devops-31-5-years-of-devops

54.
Culture is the most
important aspect to DevOps
succeeding in the enterprise

55.
What we
value
determines
our culture

56.
Mutual Understanding
Shared Language
Openness
Visualization
Tooling

57.
DevOps is the inevitable
result of needing to do
efficient operations in a
[distributed computing and
cloud] environment.
- Tom Limoncelli

58.
DevOps is not a
technological problem.
DevOps is a business
problem.
- Damon Edwards

59.
http://puppetlabs.com/sites/default/files/2014-state-of-devops-report.pdf

60.
the first scientific study of the
relationship between organizational
performance, IT performance and
DevOps practices

61.
DevOps practices
improve IT performance

62.
Culture
Automation
Measurement
Sharing
@botchagalupe
@damonedwards

63.
Culture Influencers
Decrease time from development to release
Blameless post-mortems
Reward failure and have a high emphasis on
testing
Unite different disciplines (like dev + ops) to solve
problems
http://www.slideshare.net/wickett/the-devops-way-of-delivering-results-in-the-enterprise

64.
Antipattern:
Rebrand
your ops
team to
devops team

65.
Automation

66.
Seek automation to
increase repeatability

67.
Chef, Puppet, Ansible, CfEngine
Rundeck, Mcollective
Jenkins, Travis, Kitchen
Cucumber, Gauntlt, ServerSpec
Vagrant, Docker
A Sample of the
Automation toolspace

68.
Antipattern:
Manual
config of
production
environment

69.
Beware of the
DevOps
Software
Solution

70.
Measurement

71.
Business Metrics
Event Correlation
Usage based monitoring

72.
Sharing

73.
Dashboards for all
Deploy Bot

74.
Arc 3:
Continuous
Delivery

75.
Continuous Delivery is
not merely how often
you deliver but how little
you can deliver at a time

76.
Batch size of 1

77.
Old Way
Changes break stuff, so
limit them and batch
them all together

78.
New Way
Delivery of one change at
a time reduces outages,
increases performance,
and limits technical debt

79.
You must deploy your stuff

80.
Never Pass Defects to
the Next Step
The Practice of Cloud System Administration

81.
Increase the Flow of
Work
The Practice of Cloud System Administration

82.
Let the bots troll the
users for the lolz.

83.
Allocate time to enhance
the build, test and deploy
system
The Practice of Cloud System Administration

84.
Reduce code latency and
increase code velocity

85.
The Next Arc: Security
Rugged

86.
“… those stupid
developers”
- Security person

87.
“Security prefers a system
powered off and unplugged”
- Developer

88.
Cultural unrest with
security in an organization

89.
Compliance Driven
Culture: PCI, SOX, …

90.
“[risk assessment] introduces a
dangerous fallacy: that
structured inadequacy is almost
as good as adequacy and that
underfunded security efforts
plus risk management are about
as good as properly funded
security work”

91.
Ratio Problem
Devs : Ops : Security
100 : 10 : 1

92.
Security Tools
are run out-of-band

93.
Security tools are
confusing

94.
and when they are done they
give you this lovely gem

95.
The tide is changing

96.
Resiliency
Engineering

97.
Netflix
famously
released
chaos
monkey

98.
Rugged

99.
The Rugged Manifesto
(excerpts)

100.
I am rugged and, more importantly,
my code is rugged.
I recognize that software has become
a foundation of our modern world.
I recognize the awesome
responsibility that comes with this
foundational role.

101.
I am rugged because my code
can face these challenges and
persist in spite of them.

102.
http://www.slideshare.net/wickett/putting-rugged-into-your-devops-toolchain

103.
ruggeddev.org

104.
Rugged Journey
Quality
Transparency
Value Creation
Culture infusion

105.
http://videos.2012.appsecusa.org/video/54250716

106.
http://www.youtube.com/watch?v=jQblKuMuS0Y

107.
https://speakerdeck.com/garethr/security-monitoring-penetration-testing-meets-monitoring

108.
https://speakerdeck.com/mkonda/appsecusa-2013-insecure-expectations
http://vimeo.com/75930344

109.
Security Tooling to
Delivery Pipeline

110.
…to influence Culture,
Automation, Measurement
and Sharing

111.
Security Testing
Static Code Analysis
Dynamic Testing
Virus Scanning
Code Signing Checks
Business logic/flow testing

112.
Wouldn’t it be great if we
could automate our
security tests

113.
http://static.hothdwallpaper.net/51b8e4ee5a5ae19808.jpg

114.
You could say things
like…

115.
The login page on my
application should not be
vulnerable to SQL injection

116.
The search page should
not be vulnerable to XSS

117.
Our app should not
have a backdoor in it

118.
Enter Gauntlt

119.
gauntlt.org

120.
Integration tests meet
Dynamic App Security
Testing

121.
Gauntlt Philosophy
Gauntlt comes with pre-canned steps that hook
security testing and attack tooling
Gauntlt functions as part of the CI/CD pipeline
Gauntlt is a good citizen of exit status and stdout/
stderr
Gauntlt does not install tools
MIT Open Source License

122.
Security + Cucumber =
Gauntlt

123.
Attack Logic
GIVEN
WHEN
THEN

124.
Who uses Gauntlt?
CabForward

125.
arachni
nmap
sqlmap
sslyze
dirb
garmr
generic

126.
sqli
xss
fuzzing
forceful browsing
info leaks
heartbleed
…

127.
TLDR;
Gauntlt automates
attack tooling

128.
TLDR;
Gauntlt facilitates
collaboration

129.
more on gauntlt
• Google Group > https://groups.google.com/d/
forum/gauntlt
• Wiki > https://github.com/gauntlt/gauntlt/wiki
• Twitter > @gauntlt
• IRC > #gauntlt on freenode
• Issue tracking > http://github.com/gauntlt/gauntlt

130.
Try
this at
home

131.
Fully functioning
attacking pipeline

132.
Fork this repo

133.
https://github.com/secure-pipeline/rails-travis-example

134.
labs in ./velocity

135.
./velocity/lab_2/.travis.yml

136.
./Gemfile

137.
./velocity/lab_2/.travis.yml

138.
./Rakefile

139.
./test/attacks/xss.attack

140.
./test/attacks/xss.attack

141.
./test/attacks/backdoors.attack

142.
./test/attacks/sql_injection.attack

143.
Gauntlt Next steps
Gauntlt currently doesn't install attack tooling, we
are working on a gauntlt docker container to
change that
Integrate into Kali distro

144.
but wait,
there is
more…

145.
github.com/secure-pipeline/
jenkins-example

146.
+----------------------------------+--------+-----------------------------------------+
| Alert | Risk | URL |
+----------------------------------+--------+-----------------------------------------+
| Cross Site Scripting (Reflected) | High | http://localhost:3000/forgot_password |
+----------------------------------+--------+-----------------------------------------+

147.
Conclusions
It is easy to get discouraged in our industry, but
there is hope!
Agile, DevOps and Continuous Delivery practices
have an impact for AppSec / InfoSec
InfoSec is behind but has a unique opportunity to
add value

148.
Conclusions continued
Integrating into the build pipeline and operational
tooling wins
Unit and Integration tests are not enough, we need
testing that focuses on attack tooling