Presentation about my experience of enabling security automation for a big enterprise

1. Rollercoaster of
Security Automation

2. Artem Rozumenko
S o l u t i o n A r c h i t e c t
10+ years in Software testing
7+ years in non-functional testing
5+ years in Solution Architecture
Preferable CI: Jenkins
Preferable language: Python
Preferable load tool: Gatling

3. 5
Billions US
Dollars
MARKET
2x
Security Breaches
in 2018 vs 2017
>2BPersonal records
LEAKED
WHAT ABOUT SECURITY ?
4
Billions US
Dollars
FINES
CARRIER | Continuous Test Execution Platform

4. Continuous Security Test
Execution
Integrate security tests/scans
as a quality gate of CI/CD
process
Security Testing Service
Perform manual security tests of
application
Application Security Program
Enable S-SDLC for the whole
company
SCOPE OF SECURITY TESTING
CARRIER | Continuous Test Execution Platform

5. Run security scanners in
your CI, what a big deal ?
CARRIER | Continuous Test Execution Platform

6. RUN
STATIC
SCANS
RUN
DYNAMIC
SCANS
VALIDATE
SCAN
RESULTS
VALIDATE
FIXES
25%68%6%1%
85% of findings are false-positives
CARRIER | Continuous Test Execution Platform
LET’S SEE WHERE THE TIMEOF AN ENGINEER IS

7. CARRIER | Continuous Test Execution Platform
LET’S SEE WHERE THE TIMEOF AN ENGINEER IS

8. CARRIER | Continuous Test Execution Platform
WHAT WE HAVE NOW
• Many products release with significant security
issues that cause data leaks or service failures
• Out of the box solutions mostly build for security
engineers and barely suitable for CI
• Scanners generate enormous amount of noise that
results in complete ignorance from development

9. CARRIER | Continuous Test Execution Platform
SOUNDS LIKE WE GOT A TARGET
• Run Static Scan in CI as a quality gate
• Make it run for less then 10 minutes
• Create results that won’t be ignored by developers
• Make Dynamic scans useful

10. EXPERIMENT
CARRIER | Continuous Test Execution Platform
• Take a public repo
• Run Standard approach
• Run Carrier approach
• Compare results

11. CARRIER | Continuous Test Execution Platform
SONAR + SpotBugs
19minutes
24Kfindings

12. CARRIER | Continuous Test Execution Platform
CARRIER + SpotBugs
4.5minutes
4.5Kfindings

13. How to make results actionable with less efforts?
CARRIER | Continuous Test Execution Platform

14. CARRIER | Continuous Test Execution Platform
RE-PROCESSING OUTCOME
• Group same type of vulnerabilities in same file
• Translate findings to actions
• Automatically filter false-positives
• Do not convert all the valid findings into issues
https://github.com/carrier-io/sast

15. This is how it works in production
CARRIER | Continuous Test Execution Platform

16. CARRIER | Continuous Test Execution Platform
CANONICAL DATA MODEL

17. CARRIER | Continuous Test Execution Platform
DEDUPLICATION ON THE GO
DEDUPLICATION
GROUPING

18. CARRIER | Continuous Test Execution Platform
FILTERING FALSE-POSITIVES
https://github.com/reportportal/service-analyzer-equals

19. WHAT ABOUT DYNAMIC SCANS
CARRIER | Continuous Test Execution Platform

20. CARRIER | Continuous Test Execution Platform
THERE ARE DIFFERENT SCANS
External Intruder Internal Intruder

21. EXPERIMENT
CARRIER | Continuous Test Execution Platform
• Take some well known site
• Take single well known tool
• Limit scope of vulnerabilities
• Perform authenticated scan
• Performance unauthenticated scan
• Compare results

22. CARRIER | Continuous Test Execution Platform
RESULTS
External Intruder Internal Intruder
13M req
2h scan
17 findings
100% FPs
4M req
20h scan
200 findings
99% FPs

23. CARRIER | Continuous Test Execution Platform
WE HAVE ANOTHER ONE
Automated Tests ran
through Security Tool

24. CARRIER | Continuous Test Execution Platform
RESULTS
Internal Intruder
True Automated
Intruder
200K req
25m scan
350 findings
95% FPs
4M req
20h scan
200 findings
99% FPs
https://hub.docker.com/r/getcarrier/dast

25. CARRIER | Continuous Test Execution Platform

26. CARRIER | Continuous Test Execution Platform
AS A SUMMARY
• You should test security as a part of your delivery
pipeline
• Post-processing of results helps to save time on analysis
• Automated tests helps to find great vulnerabilities
• Complex vulnerabilities should be added to functional
testing framework

27. CARRIER | Continuous Test Execution Platform
THANKS
https://github.com/carrier-io/sast
https://github.com/carrier-io/dast
https://hub.docker.com/r/getcarrier/sast
https://hub.docker.com/r/getcarrier/dast