This document summarizes a talk about lessons learned embracing DevOps and security. It discusses how security practices need to change in a DevOps world where changes happen much faster. Specifically, it argues that security must bake into the development process and provide visibility and feedback. It provides examples of using bug bounty programs and penetration testing to obtain better and more real-time feedback on security issues. The goal is to surface security data for everyone and help catch attacks more quickly through modern feedback loops that combine different testing approaches.

1. Lessons Learned Embracing
DevOps + Security
zane@signalsciences.com
@zanelackey

2. Who you’ll be heckling today
• Started out in offense
– iSEC Partners / NCC Group
• Moved to defense
– First head of security at Etsy, built and lead the four
security groups
• Now scaling defense for many orgs
– Co-founder / CSO at Signal Sciences, delivering a
product that defends web applications in the
DevOps/Cloud world

3. So what is this talk about anyway?
Lessons learned being at the forefront of the shift
to DevOps/Cloud

4. Spoiler: Security shifts from being a gatekeeper
to enabling teams to be secure by default

5. What has changed?

6. The new realities in a DevSecOps world:
1. Changes happen multiple orders of magnitude
faster than previously
2. Security only becomes successful if it can bake in
to the Development/DevOps process
3. For many apps, cost of attack is so low you will be
attacked even if you’re not a brand name

7. The new realities in a DevSecOps world:
1. Changes happen multiple orders of magnitude
faster than previously
2. Security only becomes successful if it can bake in
to the Development/DevOps process
3. For many apps, cost of attack is so low you will be
attacked even if you’re not a brand name

8. The new realities in a DevSecOps world:
1. Changes happen multiple orders of magnitude
faster than previously
2. Security only becomes successful if it can bake in
to the Development/DevOps process
3. For many apps, cost of attack is so low you will be
attacked even if you’re not a brand name

9. Let’s change our approach

10. What new concepts should security focus on?

11. What new concepts should security focus on?
Visibility + Feedback

12. Except… These aren’t new concepts!

13. Performance monitoring, data analytics,
A/B testing are all about visibility + feedback

14. The same hard lessons are slowly shifting to
security

15. First, a story from the old days…

18. How can we improve?

19. Ex: Which of these is a quicker way to spot an
attack?

22. Surface security visibility for everyone, not just
the security team
(if the security team even exists)

24. Obtaining better feedback

25. Three keys to modern feedback loops:
1. Combination of bug bounty + pentests
2. Bounty is not a replacement for pentest, it
augments pentest
3. Bounty gives general but more real time
feedback, pentest shifts to giving more directed
but less frequent feedback

26. Three keys to modern feedback loops:
1. Combination of bug bounty + pentests
2. Bounty is not a replacement for pentest, it
augments pentest
3. Bounty gives general but more real time
feedback, pentest shifts to giving more directed
but less frequent feedback

27. Three keys to modern feedback loops:
1. Combination of bug bounty + pentests
2. Bounty is not a replacement for pentest, it
augments pentest
3. Bounty gives general but more real time
feedback, pentest shifts to giving more directed
but less frequent feedback

28. Visibility + Feedback success story:
“I discovered the vulnerability late Friday afternoon and
wasn't quite ready to email it to them … [Etsy] had
detected my requests and pushed a patch Saturday
morning before I could email them. This was by far
the fastest response time by any company I've reported
to.”
- Source:
https://www.reddit.com/r/netsec/comments/vbrzg/etsy_has_been_one_of_the_
best_companies_ive

29. Embrace DevOps, Cloud, and other means of
increasing velocity. But do safely by obtaining:
Visibility + Feedback

30. Thanks!
zane@signalsciences.com @zanelackey