DevOps for Defenders from SecCon 2014. Geared towards security engineers in the enterprise.

1. DevOps for
Defenders
Ruggedizing the Pipeline

2. James Wickett
james@wickett.me
Austin, TX
Rugged Dev Podcast
Gauntlt Core Team
DevOps Days Austin Organizer
DevOps Days Global Organizer

3. My Journey
Clouding since 2008 and DevOpsing since 2010!
Led National Instruments R&D Cloud Ops team
IoT and Cloud products at Mentor Graphics
Working at Signal Sciences Corp

4. We’re making AppSec effective and practical
signalsciences.com

5. Conclusions
We optimize for the perceived probable
Agile, DevOps and Continuous Delivery practices
have approached this problem in different ways
InfoSec is behind but has a chance to add value
Integrating into the build pipeline wins

6. Humans optimize for
the probable

7. We optimize for the
probable

8. Happy Path
Engineering

9. We optimize for the
possible

10. Over Engineering

11. We optimize for the
perceived probable

12. How do we perceive
what is probable?

13. How do we know
anything?

14. Epistemological Problem
of Software Development

15. We attempt to solve it
by gathering data or
rhetoric

16. Approaches to solve
perceived probable
problem

17. Arc 1: Agile

18. Agile discovered we don’t
know what we are building

19. Solution: release features
to customers rapidly

20. Just Ship It!

21. Behavior Driven Development is a second-generation,
outside–in, pull-based,
multiple-stakeholder, multiple-scale, high-automation,
agile methodology. It
describes a cycle of interactions with well-defined
outputs, resulting in the delivery
of working, tested software that matters.
Dan North , 2009

22. Agile Summary
Rapid Iterations Win

24. Agile eventually births
DevOps

26. Arc 2: DevOps

27. Agile Infrastructure
@littleidea @patrickdebois at Velocity 2009
http://itrevolution.com/the-history-of-devops/

28. http://www.slideshare.net/jallspaw/10-deploys-per-day-dev-and-ops-cooperation-at-flickr

29. First DevOps Days,
Ghent 2009
@patrickdebois

30. DevOps is a
community movement

31. http://dev2ops.org/blog/2010/2/22/what-is-devops.html

32. DevOps realized that
Ops doesn't know what
Devs know and vice
versa

33. DevOps is an
epistemological
breakthrough joining
disparate people around
a common problem

34. Culture

36. Traditional Dev to
Ops Ratio
Dev : Ops
10 : 1

37. “That the word #devops gets
reduced to technology is a
manifestation of how badly
we need a cultural shift”
- @patrickdebois
http://www.slideshare.net/cm6051/london-devops-31-5-years-of-devops

38. Culture is the most
important aspect to DevOps
succeeding in the enterprise

39. What we value
determines our culture

41. Culture affects

42. Mutual Understanding
Shared Language
Openness
Visualization
Tooling

44. http://puppetlabs.com/sites/default/files/2014-state-of-devops-report.pdf

45. the first scientific study of the
relationship between organizational
performance, IT performance and
DevOps practices

46. Firms with
high-performing IT
organizations were twice
as likely to exceed their
profitability, market share
and productivity goals.

47. DevOps practices
improve IT performance

48. Organizational culture is one of the
strongest predictors of both IT
performance and overall
performance of
the organization.

49. Job satisfaction is the No. 1
predictor of organizational
performance.

50. Culture
Automation
Measurement
Sharing
@botchagalupe
@damonedwards

51. Antipattern:
Rebrand
your ops
team to
devops team

52. Culture Influencers
Decrease time from development to release
Blameless post-mortems
Reward failure and have a high emphasis on
testing
Unite different disciplines (like dev + ops) to solve
problems
http://www.slideshare.net/wickett/the-devops-way-of-delivering-results-in-the-enterprise

53. Automation

54. Antipattern:
Manual
config of
production
environment

55. Beware of the
DevOps
Software
Solution

56. Seek automation to
increase repeatability

57. A Sample of the
Automation toolspace
Chef, Puppet, Ansible, CfEngine
Rundeck, Mcollective
Jenkins, Travis, Kitchen
Cucumber, Gauntlt, ServerSpec
Vagrant, Docker

58. Decrease barriers to
Deploy

59. Measurement

60. Old Way:
CPU, Mem, Avg Load

61. New Way
Metrics mapped to stuff
you actually care about

62. Business Metrics
Event Correlation
Usage based monitoring

63. Sharing

64. Dashboards for all to see
Cultural adjustment
Deploy Bot

66. Arc 3: Continuous
Delivery

68. DevOps Deming Style!

69. Manufacturing Wisdom
of the 50’s and 60’s

70. Goldratt
Deming
Toyota

71. Black Belts
Six Sigma
Kanban
Lean

72. Batch size of 1

73. Old Way
Changes break stuff, so
limit them and batch
them all together

74. Change Control Windows
Roll Backs

75. New Way
Delivery of one change at
a time reduces outages,
increases performance,
and limits technical debt

78. Anyone can deploy…
You must deploy your stuff

79. Continuous Delivery is
not merely how often
you deliver but how little
you can deliver at a time

81. The Next Arc: Security

82. The Next Arc: Security
Rugged

83. “… those stupid
developers”
- Security person

84. “Security prefers a system
powered off and unplugged”
- Developer

85. Cultural unrest with
security in an organization

86. Differing priorities

87. Compliance Driven
Culture: PCI, SOX, …

88. “[risk assessment] introduces a
dangerous fallacy: that
structured inadequacy is almost
as good as adequacy and that
underfunded security efforts
plus risk management are about
as good as properly funded
security work”

89. Ratio Problem
Devs : Ops : Security
100 : 10 : 1

91. Security Tools
are run out-of-band

92. Security tools are
confusing

94. and when they are done they
give you this lovely gem

95. The tide is changing

96. Resiliency Engineering

97. Netflix
famously
released
chaos
monkey

98. Rugged

100. The Rugged
Manifesto

101. I am rugged and, more importantly,
my code is rugged.
I recognize that software has become
a foundation of our modern world.
I recognize the awesome
responsibility that comes with this
foundational role.

102. I recognize that my code will be used in
ways I cannot anticipate, in ways it
was not designed, and for longer than
it was ever intended.
I recognize that my code will be
attacked by talented and persistent
adversaries who threaten our physical,
economic and national security.

103. I recognize these things – and I
choose to be rugged.
I am rugged because I refuse to be a
source of vulnerability or weakness.
I am rugged because I assure my
code will support its mission.

104. I am rugged because my code can
face these challenges and persist in
spite of them.
I am rugged, not because it is easy,
but because it is necessary and I
am up for the challenge.

105. http://www.slideshare.net/wickett/putting-rugged-into-your-devops-toolchain

106. Dev / Ops / Sec
Join forces

107. The Society of Rugged
Developers
ruggeddev.org

109. Rugged Journey
Quality
Transparency
Value Creation
Culture infusion

110. #RuggedDevOps

111. http://www.slideshare.net/wickett/putting-rugged-into-your-devops-toolchain

112. http://videos.2012.appsecusa.org/video/54250716

113. http://www.youtube.com/watch?v=jQblKuMuS0Y

114. https://speakerdeck.com/garethr/security-monitoring-penetration-testing-meets-monitoring

115. http://vimeo.com/75930344
https://speakerdeck.com/mkonda/appsecusa-2013-insecure-expectations

116. Try this at home

117. Add Security Tooling
to Delivery Pipeline

118. …to influence Culture,
Automation, Measurement
and Sharing

119. Security Testing
Static Code Analysis
Dynamic Testing
Virus Scanning
Code Signing Checks
Business logic/flow testing

120. Wouldn’t it be great if we
could automate our
security tests…

121. http://static.hothdwallpaper.net/51b8e4ee5a5ae19808.jpg

122. Enter Gauntlt

123. gauntlt.org

124. Gauntlt Philosophy
Gauntlt comes with pre-canned steps that hook
security testing tools
Gauntlt does not install tools
Gauntlt can be part of the CI/CD pipeline
Be a good citizen of exit status and stdout/stderr
MIT Open Source License

125. Security + Cucumber =
Gauntlt

126. Attack Logic
GIVEN
WHEN
THEN

128. Who uses Gauntlt?

129. arachni
nmap
sqlmap
sslyze
dirb
garmr
generic

130. sqli
xss
fuzzing
forceful browsing
info leaks
heartbleed
…

131. TLDR;
Gauntlt automates
security tools

132. TLDR;
Gauntlt facilitates
collaboration

133. more on gauntlt
• Google Group > https://groups.google.com/d/
forum/gauntlt
• Wiki > https://github.com/gauntlt/gauntlt/wiki
• Twitter > @gauntlt
• IRC > #gauntlt on freenode
• Issue tracking > http://github.com/gauntlt/gauntlt

134. Free Gauntlt Book
request a copy
book@gauntlt.org
Caveat Emptor:
Under
development!
Valid until Dec 3rd

135. Try this at home

136. Fully functioning
attacking pipeline

137. Fork this repo

138. https://github.com/secure-pipeline/rails-travis-example

139. Go through the labs
in ./velocity

141. Conclusions
We optimize for the perceived probable
Agile, DevOps and Continuous Delivery practices
have approached this problem in different ways
InfoSec is behind but has a chance to add value
Integrating into the build pipeline wins