This document discusses integrating security testing into continuous delivery pipelines. It argues that security testing should be performed continuously and automated like other tests, rather than as a separate process. The document recommends that development, operations, and security teams work together in a "SecDevOps" model where security testing is integrated into regular testing workflows and everyone shares responsibility. It presents the BDD-Security framework as an example of how behavior-driven development can be used to automate continuous security testing that runs with each code change.

1. LONDON 2015Join the conversation #devseccon
Continuous Security
Testing
Stephen de Vries

2. About Me
Founder and CTO Continuum Security
70% Developer / 20% Security Analyst
Involved in OWASP since 2004
Created BDD-Security framework
@stephendv

3. Security Testing
• Performed after build
• Outsourced to external experts
• Process is opaque to dev/ops
Unit/Integration/Acceptance Testing
• Performed during build
• Owned by dev/test
• Tests visible to the team

4. Design Build
Unit
Tests
Integration
Tests
Acceptance
Tests
Deploy
Development Pre-prod Production
Agile
• Short iterative cycles
• Extensive automated testing
• Low/zero cost to test
• Tests can replace documentation

5. Design Build
Unit
Tests
Integration
Tests
Acceptance
Tests
Deploy
Development Pre-prod Production
Continuous Delivery
Automated acceptance
tests

6. Design Build
Unit
Tests
Integration
Tests
Acceptance
Tests
Deploy
Development Pre-prod Production
Continuous Delivery into Production
• Etsy: 50+ deploys per day
• Gov.uk: 10+ deploys per day
• Amazon: 300+ per hour
Security Tests?

7. • Everyone is responsible for
• Move testing closer to the code
• Continuous automated testing
quality
quality
security
security
security
^

8. BDD: Behaviour Driven Development

9. BDD: Behaviour Driven Development

10. https://github.com/continuumsecurity/bdd-security
JBehave +
OWASP ZAP +
Nessus +
Internal security tools +
Pre-written baseline security specifications
Selenium/WebDriver +

12. HTTP/S Proxy
Manual Application Security Testing with OWASP ZAP

13. HTTP/S Proxy
Manual Application Security Testing with OWASP ZAP
Automated
^
BDD-Security

15. Demo
https://vimeo.com/89848072

16. Who owns the security tests?
Option 1: Security Team
• Low cost test runs
• Slower feedback to dev
• Poor collaboration
• Lack of ownership by DevOps

17. Design Build Integration Tests
Unit
Tests
Acceptance
Tests
Deploy
Development Pre-prod Production
Semi-SecDevOps: Parallel tests
Manual Security Tests
Auto. Security Tests

18. Who owns the security tests?
Option 2: DevOps team with oversight by Security
• Better collaboration
• Sense of ownership of security
• Good stepping stone to…
SecDev
Ops
Option 3: Sec+Dev+Ops in a cross-
functional team
• Security testing is our problem
• We have the tools and skills to manage
it

19. Design
Auto. Security Tests
Build
Integration TestsUnit
Tests
Acceptance
Tests
Deploy
Development Pre-prod Production
SecDevOps: Inline blocking tests
Manual Security Tests

20. Related Tools
• Mittn (Python + Burp Intruder) https://github.com/F-Secure/
mittn
• ZAP-JUnit (Java) https://github.com/continuumsecurity/zap-
webdriver
• Guantlet (Ruby) http://gauntlt.org/
• OWASP ZAP Jenkins plugin https://wiki.jenkins-ci.org/display/
JENKINS/Zapper+Plugin

21. LONDON 2015Join the conversation #devseccon
Thank you!
www.continuumsecurity.net
@continuumsecure
@stephendv