You’re tasked with ‘doing DevSecOps’ for your company and you’ve got more apps and issues than you know how to deal with. How do you make sense of the different tool outputs for all your different apps let alone shrink the pile of work already on your plate? In this talk, we’ll discuss the key decision points and requirements to set up a program that moves as fast as it needs to without your team burning out. Learn how to keep moving forward while keeping your sanity. After learning to be nimble from dealing with teams that are doing 75 production deployments per week, the surviving ideas have been distilled into a collection of tenants. We’ll cover: How to handle CI/CD tests versus traditional security assessments? How to best manage SLAs? How to keep data for auditors and regulatory requirements while also doing continuous testing? Understanding health checks versus continuous testing versus manual testing. How to deal with false positives, risk acceptances and the lifecycle of a security issue? By using these tenants, security assessments at one company grew from 44 to 414 in 2 years or 9.4 times all while losing some headcount. Time to turn chaos into calm and distress into success.

1. Tenants for Going
at DevSecOps Speed
matt@defectdojo.com
matt.tesauro@owasp.org
https://www.linkedin.com/in/matttesauro/
Matt Tesauro

2. Who is this guy?
● Reformed programmer & AppSec Engineer
● CTO & Founder of DefectDojo Inc
● 15+ years in the OWASP community
○ OWASP DefectDojo (core maintainer)
○ OWASP Podcast (host)
○ OWASP Global Board of Directors
○ OWASP AppSec Pipeline (co-leader)
○ OWASP WTE (leader)
● 22+ years using FLOSS and Linux
● Go language fanboy
● Ee Dan in Tang Soo Do (2nd degree black belt)

3. 01
2
Fundamental
Truths
The 4
What’s
Overview
02 03 04
Conclusion

4. Overview
01 Let’s start at the beginning

5. —Me (Matt Tesauro)
“Time to turn
chaos into calm
and
distress into success.”

6. Background
This talk was created from the perspective that:
● You’ve been dropped into a AppSec / DevSecOps /
Product Security team as it’s lead
● You need to “do AppSec”
● You have a limited team and budget
● You need:
“A simple system that adapts to complex situations”

7. The 6
things of
DevSecOps
2 Fundamental Truths
The 4 What’s

8. Simple Machines
Definition:
“Any of several devices which few or no moving parts
that are used to modify motion and magnitude of a
force in order to perform work.”
● The inclined plane
● The lever
● The wedge
● The wheel and axle
● The pulley
● The screw
Combine these to make complex machines

9. Simple Machines
+ =
Archimedes’ screw

10. Let’s Define the
‘Simple Machines’
of DevSecOps
You can then mix and match them to
fit the context in which you work.

11. The 6
things of
simple machines
DevSecOps
2 Fundamental Truths
The 4 What’s

12. 2 Fundamental Truths
02 The two things the rest depend on

13. The Ground Truth
Know where you’re starting from
The initial assessment

14. The Ground Truth
Know where you’re starting from
● Initial broad assessment
● Why?
○ Demonstrate progress
● Measure when you start and regularly re-measure
○ Can’t measure change till the 2nd measurement
○ Get an early measurement to show progress early

15. What the assessment will find
● Ranges from greenfield to scorched earth
● Nothing exists to a complete, validated list of all apps
within the company
● SaaS company with a single product
○ Warning “Product” is a charged term
○ Product != repo
○ Microservices, Web and Mobile versions, …

16. What is a Product?
● You need to figure out what this means in context
● Stick to the definition you established
● Maybe make a poster out of it
● Avoids incredible amounts of future confusion
and misunderstanding
● Product X was assessed
versus
A repo that is one of the 6 repos that
makes up Product X was assessed

17. Understand the scope of your work
● Hopefully covered clearly before you were hired
● There’s no standard
○ Write it down / define it
○ Another poster?
● Apps only, App and Infra, Containers, Cloud, …
● How much of the vulnerability lifecycle do you own?
○ Reporting, Mitigation, Remediation, Retesting?
● Who are your upstream & downstream stakeholders?

18. Where to get started?
There’s some good frameworks you can use to do an initial
assessment, but first:
● Decide how much time to spend
● Accuracy vs speed
● Iterative is the thing
● Solo vs collaborative
Things to use as a guide:
● OWASP SAMM (Software Assurance Maturity Model)
● OWASP ASVS (Application Security Verification Standard)
● OWASP DSOMM (DevSecOps Maturity Model)

19. Single Source of Truth
As you move forward, you need a
canonical representation of reality

20. Canonical Representation of Reality
● It’s very likely you won’t have a full picture
● You need something that adapts and grows over time
● Tool venders will have you log into 27 web consoles
● How to pick a single source of truth?
○ Ability to hold all the data you need flexibly
○ Ability to filter, sort, modify, combine the data
○ Differing views based on stakeholders
○ Dedup, False Positives, Grouping/Merging

21. I know what you’re thinking..
Current tool for 90% of Enterprise Vulnerability Management Programs

22. But WHY?

23. Might I suggest…

24. Comes in 2 flavors:
OWASP DefectDojo

25. Comes in 2 flavors:
DefectDojo Pro (SaaS)

26. Now in beta:
New UI

27. DefectDojo - a single source of truth
DevSecOps tool created by DevSecOps professionals
for DevSecOps professionals.
● Manages the DevSecOps security program
● Application inventory with robust metadata
● Sort, filter, munge and export the data in multiple ways
● Engagement / assessment tracking
● Supports manual and automated security work
● Deduplication + false positive tracking of findings
● Custom report creation
● Tagging on multiple levels
● Calendar of security activities
● Historical knowledge of past assessments

28. Why was DefectDojo created?
DefectDojo was born from the frustrations of the product
security team at Rackspace with:
● Tracking manual testing efforts
● Combining output from multiple tools
● Tools having a snowflake way to represent issues
● De-duping and merging/grouping findings
● False-positive management
● Reporting on infra & app issues
● Logging into N vendor motherships
● The need to automate (REST API)

29. Active project with large community
Monthly release (2.x.0)
Bugfix release every week in between (2.x.y)

30. Commercial Support is available
https://www.defectdojo.com/

31. Where to get more info?
Search Youtube for “purple DefectDojo”

32. The 4 What’s
03 What needs to happen for DevSecOps

33. Starting
with the
Big Picture

37. The 4 What’s
Intake
What work do
you own or
comes into
scope?
Triage
What resources
should be used
to fulfill the
goal?
Test
What is the
current security
state?
Deliver
What addresses
the needs of the
downstream
stakeholders?

38. Intake
What work do you own or comes into
your scope?

39. What #1 - Intake
What work do you own or comes into scope?
● Remember that initial assessment, you nailed down scope,
right?
● Be DRY when gathering data (Don’t Repeat Yourself)
● Several types of work can come in:
○ Event vs Calendar
○ Ad-hoc vs Planned
● Ideally, try to limit intake - tricky politically
○ Automated intake can help with this

40. Triage
What resources should you use to fulfill
the goal?

41. What #2 - Triage
What resources should be used to fulfill the goal?
● The intake request has a goal, right?
● Bucket level of effort based on:
○ Risk / Criticality of the target
○ Automation in place
○ Accuracy of the tool
● Reduce testing scope by ‘pre-calculating’ with automation
● Keep the bucketing as simple as possible
○ No leaky abstractions with items
falling between buckets

42. What #2 - Triage
What resources should be used to fulfill the goal?
● Wrong choice can be better than no choice
○ You have (and should) iterate over time
● Don’t start with 100% accuracy
○ It’s great if you can pull it off but very unlikely
○ Consistent ‘stick’ to measure distance
○ Accuracy can arrive over time (if needed)
○ Avoid early optimizations

43. Test
What is the current security state?

44. What #3 - Test
What is the current security state?
● Triage showed you how much, now go test!
● Drivers of testing
○ Compliance / Regulation / Audit (internal or external)
○ Proactive testing
■ Prep for compliance, check for drift
■ Continual compliance automation
○ Updated code / new release
○ Published vulnerability

45. What #3 - More on Testing
What is the current security state?
Cadence considerations:
● Compliance / regulation
○ Calendar-based driven by policy or reporting requirements
● Proactive Testing
○ Calendar-based prior to reporting requirements
○ Continual is usually calendar-based too (just small)
● Update / new release
○ Event-based
○ Highly dependant on release frequency
○ Likely differs between teams
● Published Vulnerabilities
○ Event-based & Unplanned
○ Priority dictated by compensating controls

46. What #3 - More on Testing
What is the current security state?
Type of testing:
● Static vs Dynamic
● Source or running App
● Isolation or integration
Replicating environments in complex systems can be painful
● Microservices
● k8s native apps

47. What #3 - More on Testing
How far left can you go?
● Not everything to the left is “testable”
● Pick your battles, fast feedback loops are crucial
● If you “left test” one service of you 253 microservices-based
product
○ Have you really tested that product?
○ What about service to service interaction?

48. What #3 - More on Testing
CICD imperatives:
● Zero false positives
● Health checks vs scanning
● Specific issue tests
○ Ninja re-testing
● Long running tests vs quick tests
○ NOOP during test runs
○ Better eventually then not at all

49. Deliver
What addresses the needs of the
downstream stakeholders?

50. What #4 - Deliver
What addresses the needs of the downstream stakeholders?
● Where does your mandate stop in the finding life cycle?
○ Reporting
○ Remediation
○ Retesting
● Speak the native tongue
○ Issue trackers for devs
○ Summary data, charts & graphs for VPs and above
○ Dashboards for broader visibility
○ Engage pride as a lever
“The X team figured it out…”

51. Conclusion
04 Let’s wrap all this up

52. Putting it all together
● Understand your environment and scope
● Create your workflow / AppSec Pipeline
● Rely on constant iteration and feedback loops
● Let the system tell you rather then you tell the system
○ Continual learning via experimentation
● People (and their time) are the critical resource
○ Automate all the things
that don’t need a human brain

53. The 6
things of
simple machines
DevSecOps
2 Fundamental Truths
The 4 What’s

54. Simple Machines of DevSecOps
2 Fundamental Truths
(1) The Ground Truth
Know where your starting
(2) Single Source of Truth
A canonical representation of reality
The 4 What’s Fundamental Truths
1. Intake - What work do you own or comes into scope?
2. Triage - What resources should be used to fulfill the goal?
3. Test - What is the current security state?
4. Deliver - What addresses the needs of the downstream
stakeholders?

55. —Me (Matt Tesauro)
“Time to turn
chaos into calm
and
distress into success.”

56. CREDITS: This presentation template was
created by Slidesgo, including icons by Flaticon,
and infographics & images by Freepik.
THANKS!
Questions?
matt@defectdojo.com
matt.tesauro@owasp.org
defectdojo.com
https://www.linkedin.com/in/matttesauro/