Successfully reported this slideshow.

Addressing Security Regression Through Unit Testing

0

Share

Upcoming SlideShare
Api fundamentals
Api fundamentals
Loading in …3
×
1 of 35
1 of 35

Addressing Security Regression Through Unit Testing

0

Share

Download to read offline

These slides accompanied a talk given by Christopher Grayson at QCon NYC 2017 by the same name. The talk discusses how unit testing can be used to help address security regression in codebases.

A blog post detailing the contents of this talk can be found here:

https://l.avala.mp/?p=169

These slides accompanied a talk given by Christopher Grayson at QCon NYC 2017 by the same name. The talk discusses how unit testing can be used to help address security regression in codebases.

A blog post detailing the contents of this talk can be found here:

https://l.avala.mp/?p=169

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

Addressing Security Regression Through Unit Testing

  1. 1. Security Regression Addressing Security Regression by Unit Testing Christopher Grayson @_lavalamp
  2. 2. Introduction
  3. 3. WHOAMI 3 • ATL • Web development • Academic researcher • Haxin’ all the things • (but I rlllly like networks) • Founder • Red team @_lavalamp
  4. 4. • Security regression is a huge problem • Lots of infrastructure built around regression testing already • Let’s leverage all of that existing infrastructure to improve application security posture at a minimal cost to development teams WHY’S DIS 4
  5. 5. 1. Background 2. Dynamic Security Test Generation 3. Non-dynamic Security Test Generation 4. Conclusion Agenda 5
  6. 6. Background
  7. 7. • I’ve always loved breaking into things, have been doing this professionally since 2012 • Go in, break app, help client with remediation, check that remediation worked – great! • Come back 3-6 months later and test again, same vulns are back (commonly in the same places) • Offensive testing is good at diagnosing - not solving A Bit More on Motivation… 7
  8. 8. • Standard tool in any development team’s toolbox • Unit tests to ensure code does not regress to a prior state of instability • Lots of great tools (especially in the CI/CD chain) for ensuring tests are passing before deployment Regression Testing 8
  9. 9. Why not take the problem of security regression and use all of the tools already built for regression testing to improve the security posture of tested applications? Putting it All Together 9
  10. 10. • Street Art Around the World! • Written in Django (standard framework, no API, full post-back) • Same techniques work for any programming language and framework that support introspection • These examples require a framework that has explicit URL mapping The Demo Application 10 https://github.com/lavalamp-/security-unit-testing
  11. 11. Dynamic Generation
  12. 12. • Django requires users to write views and then explicitly map these views to URL routes where they are served from • Views come from a set of pre- defined base classes that support default functionality (UpdateView, DeleteView, DetailView, FormView, etc) Django Registered Routes 12
  13. 13. • We can use introspection to enumerate all of the views registered within an application • Now that we know the views, how can we support testing functionality that issues requests to all of the view functionality? • Enter the Requestor class Testing Registered Routes 13
  14. 14. • Requestors mapped to views they are meant to send requests to via Python decorators • Singleton registry contains mapping of views to requestors • Importing all of the views automatically establishes all of the mappings Requestor Registry Architecture 14
  15. 15. • We now can enumerate all of the views and access classes that are designed to submit requests to the views • With this capability we can dynamically generate test cases for all of the views in an application • Test cases take view classes and HTTP verbs as arguments to constructors Dynamic Test Generation 15
  16. 16. If we are relying on requestor classes being defined for all views, then let’s test for it! Testing for Requestors 16
  17. 17. We’ve got the ability to test every known HTTP verb of every registered view, so let’s test for successful HTTP responses. Testing for Denial of Service 17
  18. 18. Test to ensure that the methods supported by requestors match the methods returned by OPTIONS request. Testing for Unknown Methods 18
  19. 19. • Tell the requestors whether or not the tested view requires authentication • Can improve upon this demo by checking for inheritance of the LoginRequiredMixin • Check that unauthenticated request is denied Testing for Auth Enforcement 19
  20. 20. Response Header Inclusion 20
  21. 21. We already built out requestors based on the OPTIONS response, so now let’s make sure that the OPTIONS response included the correct HTTP verbs. Testing for OPTIONS Accuracy 21
  22. 22. Test to ensure that CSRF tokens are required for function invocation on non-idempotent view functionality. Testing for CSRF Enforcement 22
  23. 23. • We now have guarantees that • Our app contains no hidden functionality • All of our views are working as intended given expected input • Authentication is being properly enforced • Security headers are present • CSRF is properly protected against What Have We Gained? 23
  24. 24. • Those guarantees are great and all, but can’t we just write individual unit tests to test for them? • In a development team we have multiple people contributing code all the time • Through dynamic generation, these tests will automatically be applied to all new views, providing the same guarantees to code that hasn’t even been written yet Why Dynamic Generation? 24
  25. 25. • Other things that we could write dynamic tests for • Rate-limiting • Fuzzing of all input values to POST/PUT/PATCH/DELETE (introspection into forms used to power the views) • Proper updating, creation, and deletion of new models based on input data Where Can We Go? 25
  26. 26. Testing Other Vulns
  27. 27. Test for proper encoding of output data! Testing for Cross-site Scripting 27
  28. 28. Submit two requests to the server, one making the SQL query match none and another making the SQL query match all, test to see if the results match the none and all expected responses Testing for SQL Injection 28
  29. 29. Submit malicious input and see if HTTP redirect response redirects to full URL Testing for Open Redirects 29
  30. 30. Conclusion
  31. 31. • Initial overhead is greater than writing individual unit tests, but new views added to the application also benefit from the tests • Provide us with strong guarantees about known application functionality and basic HTTP-based security controls Benefits of Dynamic Generation 31
  32. 32. • Security guarantees now enforced by CI/CD integration • Test Driven Development? Great – have your security testers write failing unit tests that you then incorporate into your test suite • A new interface for how security and development teams can work together in harmony Benefits of Sec. Unit Testing 32
  33. 33. • Security regression is a big problem • We can use the development paradigm of regression testing to address security regression • Dynamic test generation can take us a long way • Individual tests for individual cases further augment dynamic test generation capabilities Recap 33
  34. 34. • Security Unit Testing Project https://github.com/lavalamp-/security-unit-testing • Lavalamp’s Personal Blog https://l.avala.mp/ • Django Web Framework https://www.djangoproject.com/ Resources 34
  35. 35. THANK YOU! @_lavalamp chris [AT] websight [DOT] io github.com/lavalamp-

Editor's Notes

  • Show an example of a View (CreatePostView)
    Show the base requestor class (BaseRequestor)
    Show an implementation of a requestor class (CreatePostViewRequestor)
  • Checkout HEAD of master branch (git checkout master)
    Show the registry class (TestRequestorRegistry)
    Show the decorator (requested_by in tests/registry.py)
    Show the add_mapping method in TestRequestorRegistry
    Show the implementation of requested_by (CreatePostView)
    Run the following code to show the mapping: python manage.py shell -c "from sectesting import urls; from streetart.tests import TestRequestorRegistry; registry = TestRequestorRegistry.instance(); registry.print_mappings()"
  • Checkout HEAD of master branch (git checkout master)
    Show contents of build_suite method in StreetArtTestRunner
    Show contents of __get_requestor_class_tests in StreetArtTestRunner
    Show contents of url_patterns property in StreetArtTestRunner

    Point out the anonymous subclassing thing
  • Ensure that TEST_FOR_REQUESTOR_CLASSES is the only True value in settings
    Check out tag v0.1 (git checkout tags/v0.1)
    Show contents of ViewHasRequestorTestCase
    Show contents of __get_requestor_class_tests in StreetArtTestRunner
    Comment out a requested_by on one of the views
    Run tests and show failing (python manage.py test)
    Uncomment out requested_by
    Run tests and show all passing
  • Ensure that TEST_FOR_DENIAL_OF_SERVICE is the only True value in settings
    Check out tag v0.1 (git checkout tags/v0.1)
    Show contents of RegularViewRequestIsSuccessfulTestCase
    Show contents of ____get_dos_class_tests in StreetArtTestRunner
    Add raise PermissionDenied to EditPostView GET
    Run tests and show failing (python manage.py test)
    Remove the PermissionDenied raise
    Run tests and show passing (python manage.py test)
  • Ensure that TEST_FOR_UNKNOWN_METHODS is the only True value in settings
    Check out tag v0.1 (git checkout tags/v0.1)
    Show contents of RegularUnknownMethodsTestCase
    Run tests and show failing (python manage.py test)
    Checkout the next tag (git checkout tags/v0.2)
    Show contents of DeletePostViewRequestor with new verbs
    Run tests and show passing (python manage.py test)

    Unknown functionality is a constant problem with web apps (especially in frameworks that provide significant amounts of functionality out of the box)
  • Ensure that TEST_FOR_AUTHENTICATION_ENFORCEMENT is the only True value in settings
    Check out tag v0.3 (git checkout tags/v0.3)
    Show contents of AuthenticationEnforcementTestCase
    Run tests and show failing (python manage.py test)
    Checkout the next tag (git checkout tags/v0.4)
    Show contents of MyPostsListView with new auth check
    Run tests and show passing (python manage.py test)
  • Ensure that TEST_FOR_RESPONSE_HEADERS is the only True value in settings
    Check out tag v0.5 (git checkout tags/v0.5)
    Ensure that streetart.middleware.SecurityHeadersMiddleware is commented out in settings.py
    Show contents of HeaderKeyExistsTestCase and HeaderValueAccurateTestCase
    Run tests and show failing (python manage.py test)
    Checkout the next tag (git checkout tags/v0.6)
    Uncomment streetart.middleware.SecurityHeadersMiddleware in settings.py
    Run tests and show passing (python manage.py test)

  • Ensure that TEST_FOR_OPTIONS_ACCURACYis the only True value in settings
    Check out tag v0.9 (git checkout tags/v0.9)
    Show contents of RegularVerbNotSupportedTestCase and AdminVerbNotSupportedTestCase
    Show contents of MyPostsListView trace method
    Run tests and show failing (python manage.py test)
    Checkout the next tag (git checkout tags/v0.10)
    Show contents of MyPostsListView
    Run tests and show passing (python manage.py test)
  • Ensure that TEST_FOR_CSRF_ENFORCEMENT is the only True value in settings
    Check out tag v0.11 (git checkout tags/v0.11)
    Show contents of CsrfEnforcementTestCase
    Show contents of CreatePostView with reference to csrf_exempt
    Run tests and show failing (python manage.py test)
    Checkout the next tag (git checkout tags/v0.12)
    Show contents of CreatePostView with no reference to csrf_exempt
    Run tests and show passing (python manage.py test)

    Just because CSRF values are present doesn’t mean they are properly enforced

  • Ensure that all test cases are enabled
    Check out tag v0.15 (git checkout tags/v0.15)
    Show contents of NewCreatePostView
    Run tests and show failing (python manage.py test)
    Talk about how author neglected to add requestor
    Checkout the next tag (git checkout tags/v0.16)
    Show that requestor has been added to NewCreatePostView
    Run tests and show failing (python manage.py test)
  • Check out tag v0.13 (git checkout tags/v0.13)
    Show contents of ErrorDetailsXSSTestCase
    Show contents of error_details.html and point out |safe pipe
    Run tests and show failing (python manage.py test)
    Checkout the next tag (git checkout tags/v0.14)
    Show contents of error_details.html and point out lack of pipe
    Run tests and show failing (python manage.py test)

  • Check out tag v0.19 (git checkout tags/v0.19)
    Show contents of SQLInjectionGetPostsByViewTestCase
    Show contents of GetPostsByTitleView
    Run tests and show failing (python manage.py test)
    Checkout the next tag (git checkout tags/v0.20)
    Show contents of GetPostsByTitleView and point out parameter binding
    Run tests and show failing (python manage.py test)
  • Check out tag v0.21 (git checkout tags/v0.21)
    Show contents of RedirectViewTestCase
    Show contents of RedirectView
    Run tests and show failing (python manage.py test)
    Checkout the next tag (git checkout tags/v0.21)
    Show contents of RedirectView and point out parameter sanitization
    Run tests and show failing (python manage.py test)
  • ×