Continuous Security Testing in a Devops World #OWASPHelsinki
Upcoming SlideShare
Loading in...5

Continuous Security Testing in a Devops World #OWASPHelsinki



Performing continuous security testing in a DevOps environment with short release cycles and a continuous delivery pipeline.

Performing continuous security testing in a DevOps environment with short release cycles and a continuous delivery pipeline.



Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • The most notable things are that after 9 years dedicated to security, I now consider myself a developer 50% of the time.But before we get there, let’s put this brave new devops world into perspective.
  • The first place to start talking about a devops world is Agile, becaue it was with agile that..Because of manual processes in getting that shippable product tested in a staging environment and finally deployed into live.Bottleneck is there because of the processes in place during testing and deployment.
  • Operations want a stable environment and a stable product and because of their reliance on manual configuration, migration and testing.
  • Take the same agile culture in dev and apply it to operations.Break down the wall between dev and ops, and have our devs participate in the testing and deployment process; and we can have operations adopting automation and dev tools.
  • As you ramp up from just Agile to Continuous Deployment, the amount of DevOps you’re doing needs to increase.Continuous delivery goes one step further than continuous integration in that you have a deliverable product. At the end of a Continuous delivery cycle you have a shippable product that can be delivered to staging environment or to user acceptance test environment. Devops starts becoming important when you move from CI to CD, because you need to delivery a working product in a realistic environment.
  • We both understand what we need to delivery, and how to deliver it. In fact we understand it so well, that we’ve automated the delivery processes.
  • The answer of a traditional security team, would be more along the lines of a continuous annoyment model.Answering no to everything:You can’t have security testing at the same tempo as you have deploysAnd you definitely can’t deploy to prod without security testing
  • For me the most important realisation when moving from security to development is that security is not special.It is an emergent property of software, just like scalability and performance.When you want to build a scalable and performant application you have to think about those two requirements from the start. And security is no different.The same way you don’t add performance or scalability to an application, shouldn’t be adding security.
  • And we don’t have to start from zero.Security can definitely learn a lot about how QA and acceptance tests are automated.Write both functional and non-functional security requirements as stories.Collaboration means exposing your internal processes
  • We have recorded 2 distinct artifacts here: Our intention, AND the steps to verify that our intention has been implemented.Problems with this approach: Navigation logic is tightly coupled to the security test. Selenium does not allow you to inspect HTTP layer. Excludes non-developers.
  • These limitations were what inspired the BDD-Security framework
  • Because the framework is based on other standard technologies, it integrates quite easily with jenkins
  • Sanity checking, show session management tests.

Continuous Security Testing in a Devops World #OWASPHelsinki Continuous Security Testing in a Devops World #OWASPHelsinki Presentation Transcript

  • Continuous Security Testing in a DevOps World
  • About Me • Stephen de Vries – CTO Continuum Security – 70% Security Consultant – 30% Developer – Author of BDD-Security Project – (Ex) Co-founder of OWASP Java Project – @stephendv
  • Agile: • Small incremental changes • Fast feedback from tests • Fast feedback from users • Easily adapts to change • Lower risk of project failure Bottleneck between “Shippable” and “Deployed”
  • Change Developers • Value faster incremental changes • Automated testing Operations • Value stability • Manual processes • Manual testing
  • Dev Ops Business value: Faster Culture Tools • Systems view • Accelerate feedback loops • Trust & Accountability • Communication • Version control • Automated deployment • Automated Configuration • Automated testing
  • Continuous Delivery Agile Plan/Code/Build/Test Continuous Integration Int. Test Delivery Continuous Deployment Deploy DevOps Pre-prod ProdDev • Requirements as stories • Unit testing • Automated Functional testing • Auto. Config + deploy • Auto. Acceptance testing • Monitoring • Easy rollback
  • The DevOps challenge to security: • As DevOps we understand the process of built, test and deploy • We’ve largely automated this process in a delivery pipeline • We deploy to production multiple times per day How can we do this securely?
  • • Dead documents • Reliance on manual processes • Tools don’t fit the deployment pipeline Traditional Security Approach ? ?
  • How can we provide security at DevOps speed?
  • Security is not special Don’t add security… …make it disappear
  • What can security learn from Agile/CD/DevOps? • Security goals must be driven by the business and must be clearly stated • Collaboration and communication means exposing your processes • Do it well enough and there is no “them” “Never send a human to do a machine’s job” – Agent Smith • Record manual security tests for automation • Automate scanning process • Automated tests are the security requirements
  • First attempt:
  • @Test public void change_session_ID_after_login() { driver.get("http://localhost:9110/ropeytasks/user/login"); Cookie preLoginSessionId = getSessionId("JESSSIONID"); login("bob", "password"); Cookie afterLoginSessionId = getSessionId("JESSSIONID"); assertThat(afterLoginSessionId.getValue(), not(preLoginSessionId.getValue())); } public void login(String u, String p) { driver.findElement("username")).clear(); driver.findElement("username")).sendKeys(u); driver.findElement("password")).clear(); driver.findElement("password")).sendKeys(p); driver.findElement("_action_login")).click(); } • Navigation logic is embedded in the test • Selenium does not expose HTTP • Excludes non-developers
  • Introducing BDD-Security
  • • Tests must be understandable by all stakeholders • Behaviour Driven Development (BDD): JBehave • Must be able to automate manual security testing • Selenium + OWASP ZAP API + Nessus + … • Must fit into dev workflow and CI/CD pipelines • Runs in IDE, cmd line • Runs in Jenkins • Test results in JUnit wrapper + HTML • The logic of the security tests should be independent from navigation code • Provide a baseline of ready-to-use security tests
  • Getting Started with BDD-Security
  • Integration with Jenkins
  • Real world challenges • Not Anti-CSRF aware • No difference between test error and test failure • Test Maintenance • Do sanity checks along the way • Try to find generic solution • E.g.: ISomeBehaviour • CAPTCHA • ICaptcha +
  • Old way:
  • New way:
  • Questions?
  • Resources: • • OWASP ZAP Pure Java client API • Resty-Burp RESTful API into Burp Suite • Nessus Java Client • SSLTest Java SSL analyser • Related projects: • Gauntlt BDD wrapper for sec tools:
  • Thank you @stephendv