"Threat Model Every Story": Practical Continuous Threat Modeling Work for Your Team
•Download as PPTX, PDF•
6 likes•1,319 views
How to do threat modeling in the age of Agile and DevOps. A practical methodology for teams focusing on developers. Also, an introduction to PyTM as a tool for threat-modeling-with-code.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to "Threat Model Every Story": Practical Continuous Threat Modeling Work for Your Team
Similar to "Threat Model Every Story": Practical Continuous Threat Modeling Work for Your Team (20)
Recently uploaded
Recently uploaded (20)
"Threat Model Every Story": Practical Continuous Threat Modeling Work for Your Team
- 1. Threat Model Every Story: Practical Continuous Threat Modeling Work for Your Team or ”What Do You Mean Threat Model EVERY Story Who Has That Kind of Time Go Away and Take Your Threat Model With you!?!”
- 2. The Bureau of Statistics informs: No surveys were harmed in the making of this talk. All data is purely anecdotal and open to subjective interpretation based on the reader’s experience.
- 3. • Lead Security Architect at Autodesk Inc., focusing on application security • SAFECode collaborator, IEEE Center For Security Design collaborator • Curious argumentative all-around kvetcher Who am I?
- 4. • Show of hands – You threat model every day – You want to add threat modeling to your practice – You do research work on Threat Modeling – You are in the wrong room and too shy to leave after three slides into the presentation Who are you?
- 5. • Level setting – threat modeling, what and why? • My personal threat modeling journey • Challenges encountered in this unfinished journey • How I am trying to solve that – Continuous Threat Modeling – How can you use it? – Tools – References What are we doing here today?
- 6. • A conceptual exercise that aims to identify security- related flaws in the design of a system and identify modifications or activities that will mitigate those flaws. • Formally, it can be “A technique to identify the attacks a system must resist and the defenses that will bring the system to a desired state” (Brook Schoenfield) • Four Fundamental Questions (Adam Shostack) – What are we working on? – What can go wrong? – What are we going to do about it? – Did we do a good job? Threat Modeling – what & why
- 7. A Threat Modeling (ongoing) personal journey Role TM Methodology developer Discuss the latest published exploits. Wonder aloud if they apply to my code. developer STRIDE developer/ architect STRIDE/element security consultant Threat Library security consultant A frank conversation about the system and what it wants to be (“SME-led”) security consultant TM “spikes”
- 8. But what was I looking for? • Accessible: can a product team do it independently after a brief period of instruction? Can they keep doing it? • Scalable: can the same thing be done over many teams and products? In a short time? • Educational: can it teach instead of correct? • Useful: are the results found useful for both product and security? • Agile: is it repeatable, does it negatively impact the product team’s velocity? • Representative: how does the system being modeled compare with the model? • Unconstrained: once the “known suspects” are evaluated, is the team led to explore further?
- 9. Did those methods reach the goals?
- 10. The Case For Continuous TM
- 11. Threat Model Every Story • build a baseline - involving everyone. Use whatever technique works for your team. At Autodesk we are currently focusing on a “subject based” list of points of interest • designate one or more “threat model curators” who will be responsible for maintaining the canonical threat model document and the findings queue • instruct your developers to evaluate each one of their stories with focus on security: – if the story has no “security value”, continue as usual – if the story generates a security “notable event”, either fix it (and document as a mitigated finding) or pop it up as a “threat model candidate finding” for the curator to take notice of (at Autodesk we are doing this using labels on JIRA tickets) • make sure your curators are on top of the finding and candidate finding queues
- 12. But…how do my developers know what has “security value”? Subject areas Question and then continue questioning during “official design time” or when building a baseline Checklist Verify that the principles have been followed at implementation time
- 13. Handbook and Subject areas
- 15. Threat Model Every Story • build a baseline - involving everyone. Use whatever technique works for your team. At Autodesk we are currently focusing on a “subject based” list of points of interest • designate one or more “threat model curators” who will be responsible for maintaining the canonical threat model document and the findings queue • instruct your developers to evaluate each one of their stories with focus on security: – if the story has no “security value”, continue as usual – if the story generates a security “notable event”, either fix it (and document as a mitigated finding) or pop it up as a “threat model candidate finding” for the curator to take notice of (at Autodesk we are doing this using labels on JIRA tickets) • make sure your curators are on top of the finding and candidate finding queues
- 17. • “Uh...what?” • “This is still too heavy” • “But how do I know I did everything?” • “I never saw a room of architects excited about threat modeling before” Reactions from product teams
- 18. Caveat Emptor: This Is Not Perfect • Difficult to convince teams that the Subject List is not a threat library and developers that the Checklist is not a requirements list – not exhaustive, just a starting point • The resulting TM won’t be perfect – evolutionary • A SME or security group is still necessary for education • GIGO – garbage-in, garbage-out
- 19. So…about that automation thing. • What are the parts of Threat Modeling we can most easily automate? – Diagraming - cross-platform, over the network, simple and quick yet representative – Reporting - having a standard and keeping to it; information passing – Threat ranking - CVSS or some other agreed ranking system – Low-hanging fruit - threats that can be immediately derived from a formal description of the system should emerge • Tooling should help discuss the system, keep the model as close as possible to the reality of the system, disseminate information, and not hinder collaboration:
- 20. What is available today? • There are many threat modeling tools; some are platform- dependent, like the MS Tool, others are web-based • Some start the process with a questionnaire along the lines of “what do you want to build” and generate a list of requirements that the developers must follow • Others get a description of the system and generate threats based on characteristics of the design • But … developers write code; why not have them feed the threat model with something that looks like code? • “TM-as-code” is in the same place “DevOps” was a couple of years ago. There is talk of, people want to do it, but the definition of what it actually means is murky
- 21. Three current practical approaches ThreatSpec Fraser Scott @zeroXten Threat modeling IN code ThreatPlaybook Abhay Bhargav @abhaybargav Threat modeling FROM code PyTM Threat modeling WITH code
- 22. PyTM – A Pythonic way of TM’ing Matt Coles, @coles_matthewj Nick Ozmore, @nozmore Rohit Shambhuni, @rshambho Izar Tarandach, @izar_t
- 23. PyTM – Creating a Threat Model
- 24. PyTM – Elements and Attributes
- 25. PyTM – Generating a DFD
- 26. PyTM – Sequence Diagrams
- 27. PyTM – Listing threats
- 28. PyTM – Report template
- 29. PyTM – generating reports
- 30. PyTM – how is it being used? • during team meetings to create the initial diagram • in discussions with the product team - “it is missing this attribute”, “why is this a threat”, “what if?” • keep threat models in revision control, together with the code they describe and generate automated, standard threat model reports
- 31. PyTM – invitation to collaborate • More threats • More elements • Documentation • A better rule engine • Integration with other tools • Suggestions, bug fixes, requirements, sample use cases • Discussion!
- 32. Thank you! • Questions ? • Don’t forget to leave feedback!
- 33. References • OWASP Threat Modeling Slack Channel – https://owasp.slack.com #threat-modeling • OWASP Application Threat Modeling - https://www.owasp.org/index.php/Application_Threat_Modeling • SAFECode “Tactical Threat Modeling” - https://safecode.org/wp-content/uploads/2017/05/SAFECode_TM_Whitepaper.pdf • ThreatSpec - https://threatspec.org/ • ThreatPlaybook -https://we45.gitbook.io/threatplaybook/ • PyTM - https://github.com/izar/pytm
Editor's Notes
- Running for 8 or so months. Results are promising but not enough empirical data yet. not saying one thing is better than another just offering the justification and the initial findings of the experiment
- 8 years at EMC IBM start-up game consulting work
- calibrate comments - need to know who you are
- just a brief agenda
- Ideally, traditionally performed at the design stage. The objective is to generate findings, but may serve as a framework for other stages of the secure software development cycle. Widely agreed to be an activity with high returns on investment: flaw correction, learning and documentation are immediate products, with training, testing and of course development also impacted. The four questions start with understanding, then wondering, then inventing, then measuring.
- As a developer I was all about the results. I just wanted to know which holes I had to plug. As an architect, I focused on coverage. Every single aspect of the system had to be considered to exhaustion, lest something fall between the lines. As a security consultant, it all became about scalability and being able to help the product team as much as possible, hopefully at some point growing security champions and architects that would take over the consulting. It was also about adapting to the rhythm of the team.
- The issue with STRIDE and STRIDE/element is that they need someone who knows how things can be broken in order to figure out what is potentially possible as a flaw, or they need the use of attack trees that add complexity and weight to the process. As a learning tool they have their value since they lead to discussion of those findings, but they still depend on having someone, a SME, to do that teaching. They are not constrained, but at the same time a lack of structure may leave areas unresolved or unexamined. They don’t scale well due to the time needed, the knowledge needed, the need for a SME. The threat library is an improvement on needing a SME, as the team looks at issues that have historically been a problem. On the other side, that tends to constrain the team to only those issues, as they look to discharge themselves of the requirement of a threat model, and the exercise becomes a compliance checklist rather than an examination. The issues are very descriptive and lead to over-focus. It scales better than STRIDE, but still needs the SME to explore beyond the library. Having a tailored conversation on each product amounts to a SME-led threat model. It will surely generate the most correct findings, but at the same time it will not scale at all, it will not teach much to the product team, but in small shops it may be the easiest and best way to go The idea of continuously improving a threat model is not new - it has been pushed, for example, by Microsoft for a long time - but it was in a vague sense, that “a threat model should be updated in every sprint” without giving a practical way to do so that didn’t fall foul of the other issues. It assumes that all the resources needed for a “threat modeling spike” are in place and willing to do what’s needed, and that the process is easily reduced to the scope of a sprint.
- The saying that “threat modeling happens at the design stage” used to be fine when we were waterfalling, since you could threat model until you had an acceptable design that would then be implemented. Nowadays with agile methods and emergent designs we don’t have that luxury anymore. Seymour Cray - The problem with programmers is that you can never tell what a programmer is doing until it is too late.
- to the meat of the thing.
- Richard Feynman: Do your own homework. To truly use first principles, don’t rely on experts or previous work. Approach new problems with the mindset of a novice. Truly understand the fundamental data, assumptions and reasoning yourself. Be curious. This is where the education angle comes in. We are training our developers wrong. We are providing huge amounts of information that do not correspond to huge amounts of useability. We are doing the busywork of teaching people how the RSA algorithm works without focusing on the aspects of choosing the right key length, algorithm and secret protection scheme that their system needs. In order to create sensitivity to what “security notable events” are, we at Autodesk are experimenting with providing developers with a checklist that they use as part of the definition of done of their stories.
- The subject areas are more important than the sample questions.
- This checklist follows a “if this then that” model - the developer only needs to relate to those items that are relevant to the work at hand The language on the “if” side is developer language. There is no need to decipher what the security team intends in order to figure out if something is relevant or not The checklist is limited in length - one double sided printed page should be the limit so ideally it can be printed and kept at hand The “then that” side is not prescriptive. It pushes the developer to search for the information that relates to whatever environment or framework they are using. This is for three reasons - to keep the list short, to make it somewhat open-ended, and to tickle the curiosity innate to most developers. Pointers are given but not “absolute solutions” The checklist is backed by documentation and live support by the security team It is made clear to the developer - once you don’t need the list anymore, throw it away. The list focuses on teaching fundamentals, not formulas.
- If we look at the threat modeling spike in detail what we see is that at the end of the sprint, the same process used to generate the baseline threat model should be again used to update it. The mitigator is that this time only those things that changed will need to be revisited. That is well and fine, but it still doesn’t answer the basic questions: who is responsible for doing the update? the whole team? the owner of the tm who will provide guidance? is a SME available? when will the findings be fixed? is a finding enough to hold back a story? are they automatically addressed in the next sprint?
- Tm-IN-code – threat modeling happens as code is written and mixes with the code, encapsulates the problem with the solution Tm-from-code - deriving previously identified threats from other tools, validating or discovering threats present in code and providing a proper language to talk about these threats Tm-with-code - we use code to express the system to be modeled and derive information about it