DevOpsSec: Appling DevOps Principles to Security, DevOpsDays Austin 2012
DevOpsSec Applying DevOps Principles to SecurityNick Galbreath email@example.com @ngalbreath DevOpsDays, Austin Texas, April 3, 2012http://client9.com/20120403 firstname.lastname@example.org
Slides! Video!• Originally presented on April 3, 2012• Latest Slides! Streaming Video! http://client9.com/20120403• Related interview: http://youtu.be/Afd0u5DGxr8• Original video stream: http://www.ustream.tv/recorded/21568549
whoami• Development background• Lots o’ startups, book, patents,blahblahblah• Director of Engineering at Etsy covering • Security, Fraud, Biz Analytics, Email Infra, Internal Systems, and everything else not www.etsy.com “Enterprise” • Second time working with Allspaw!• “Oh you mean there is a name for this?”
ContextMy biases for this talk is (Web) ApplicationSecurity, not classic Network Security or ITSecurity.
Double-click to edit• Double-click to edit uhhhhhh....
...with theacknowledgement that• We are working in a complex system• And in complex systems failure happens• And failure can happen when everyone does nothing wrong• And given this, how can one increase reward and reduce risk for the business
What does this mean for....People?Processes (workﬂow)?Machines?
An Only Slightly Contrived Example• I trust MCR to run our network• I can verify this by looking at our dataporn• He trusts me that when things go wrong, the graphs won’t be used to burn him.• He can verify this by... seeing our Post Mortems in action (they are open at Etsy)
How Fast Can You Deploy or Rebuild• Your Firewall,VPN, Load Balancer• Your Operating System, Critical Servers• Your Database, server, schema, data• Your Application, patches• Any other conﬁguration ﬁle in a consistent, sane manner
Being able to deploy quickly is my #1 security featureThis implies a standardized, automatedsystem and conﬁguration management.
I Call BullshitDoesn’t the rapid rate ofchange in a continuousintegration environmentmean things are less secure? Well compare this to....
We’ll rush that security ﬁx.It will go out in next releasein about 6 weeks. former vendor at Etsy
It’s ok if we have a few extraﬁremen waiting around incase there is a ﬁre I’m more concerned we won’t know there is a ﬁre until the house is burnt down Conversation between Chad Dickerson and Nick Galbreath, Etsy 2011
Segmentation Faults• Why is your server falling over?• From the same IP address.• Over and over Maybe time to patch? Also check out your server 500 errors.
Database Syntax Errors Almost game over here. Whose is responsible for these anyways?Demand zero-tolerance for database syntax errors.
SQLi AttacksThe shittest check that works.Will undercount by at least 10xProTip #1: using regexp for sqli is failProTip #2: write a unit test for this.
And Graph ItSecurity is no longer a binary event.
Got that?Security is not a binary event. You are beingattacked constantly.THIS IS AWESOME
Attack Driven Testing Security testing in dev is hard. Use actual attacks/probes to guide testing.• Server 500 errors• Core Dumps Can you automate• CSRF Failures veriﬁcation?• XSS Attempts• Login Failures• Password Resets
TESTINGIf infrastructure is code, then doesn’t it need testing too?
assertyour production environment Who knew that writing solid C code is similar to running a complex system. Writing Solid Code Steve Maguire 1993
assert this• This page is always SSL• This page requires sign-in• This page never is publicly available• Google never crawls this page• This page is not being routed by the CDN• This port is never open
Reuse your unit test framework to test production conﬁg
Post Mortems• All security issues are “P1” or “P2” (ﬁx now, or ﬁx by end of week)• Even for internal applications.• All get a post-mortem• Great educational experience and knowledge transfer
Hiring• Strict no asshole policy.• Security is a services business.• “Product Security” is in-house consultancy.• Can you take people who are interested and train them? TBD. http://www.sans.org/ https://www.owasp.org/
Extend the Perimeter• Working on a training program for both in- house employees, contractors, and external vendors.• “Device Tune-Up Day” -- employees bring their home computers in for a tune-up.
Nick Galbreath email@example.com @ngalbreath DevOpsDays Austin Tx 2012http://client9.com/20120403 firstname.lastname@example.org