Successfully reported this slideshow.
Your SlideShare is downloading. ×

How to get the best AppSec test of your life

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

How to get the best AppSec test of your life

  1. 1. Josh Grossman OWASP AppSec Israel 2017 How to get the best AppSec test of your life
  2. 2. Josh Grossman OWASP AppSec Israel 2017 How to get the best AppSec test of your life
  3. 3. The data Sky Shady side of pyramid Sunny side of pyramid
  4. 4. I can doez pen testing
  5. 5. I can doez pen testing
  6. 6. Who am I? • 10 years of IT Security, IT Risk and development experience • Last several years focused on Application Security and Cloud • Team Lead in the AppSec Department at Comsec Global. • Married + 2, living in Modi’in
  7. 7. Disclaimer • Thoughts and opinions are my own and do not represent my employer
  8. 8. Today’s Goals • Something for everyone • Defenders/Builders – Ideas, ideas, ideas • Breakers – Are you ready? • Questions at the end please 
  9. 9. https://www.flickr.com/photos/purpleslog/2907496392/in/photostream/
  10. 10. Principle 1: “Anything worth doing, is worth doing right” - Hunter S. Thompson
  11. 11. Principle 2 –You get what you put in
  12. 12. Principle 3:
  13. 13. Three opportunities 1.Scoping 2.Preparation 3.Reporting
  14. 14. Security to the left • The earlier, the easier • Architecture review • Key design decisions S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  15. 15. • Value, not realism • More coverage, more quality, less time • Improvement process, not an exam The whitest white box S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  16. 16. An expert comes from outside Smart suit Know it-all From outside Realtor Intruder Your boss Expert • Shiny report is more persuasive • Should push your concerns • Both sides benefit S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  17. 17. Old hand or a Fresh Start? • Yes! • Old hand for a few cycles • Then a fresh pair of eyes later S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  18. 18. A full project? S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  19. 19. A full project Scoping Detailed overview (Mini design review) Testing supported by source code Deliver report Explanatory meeting to present findings Assist developers with finding solutions Perform retesting and prepare for next project Zero day card (credit Haroon Meer, 44con 2011) S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  20. 20. Scoping 1.Security to the left 2.The whitest white box 3.An expert comes from outside 4.Old hand or a Fresh Start 5.A full project cycle S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  21. 21. http://rickvandenbosch.net/blog/security-workshop-hack-yourself-first/ Plus credit to Jeremiah Grossman,Troy Hunt and AverageSecurityGuy • Catch low hanging fruit • It’s fun and tools are cheap • But…requires time Hack yourself first S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  22. 22. S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5 • Vulnerabilities already in the backlog • Also areas you are concerned about • Not a competition, we want efficiency! Disclose known vulnerabilities https://www.youtube.com/watch?v=7t0EtKlQxyo
  23. 23. Security by non-testability • Great security technologies exist • We want to test the app, not them • If necessary, check findings afterwards S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  24. 24. https://imgur.com/gallery/MKk5l The testing setup • No 100% safe test • Ideally, dedicated environment • Let us use our testing builds, please! S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  25. 25. Be ready • Agree a start date • Get a written request list • Test it all S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  26. 26. Preparation 1.Hack yourself first 2.Disclose known vulnerabilities 3.Security by non-testability 4.The testing setup 5.Be ready S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  27. 27. Progress reports • Status vs Findings • Problems and Critical findings – ASAP • Everything else – later • Ongoing comms is important S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  28. 28. Executive Summary 1.3 Graph of Findings 1.4 Detailed Findings List ACME Corp – Security Test Report Nation-state levelTLS stuff XSS More XSS No last login time False positives 2017T10RC1A7 Insufficient Attack Protection FINDINGS BYTYPE • Summarise report • Should look good and be client friendly! • Business impact crucial S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  29. 29. Clearly explained with specific actions https://my2ndheartbeat.files.wordpress.com/2017/08/blah-blah-blah.jpg • Easy to understand the finding • Have full repro information • How many instances? • Specific and relevant recommendations S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  30. 30. Prioritised action plan • Include testers and R&D • Balance urgency and difficulty • Maybe include short and long term fixes S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  31. 31. Assistance with fixes http://www.picturesinboxes.com/2014/09/04/superman-jerk/ • Discuss the fix with the tester • Avoid misunderstandings • Fix should address the risk • Worst case is fixing wrong S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  32. 32. Reporting 1.Progress reports 2.Executive Summary 3.Clearly explained with specific actions 4.Prioritised action plan 5.Assistance with fixes S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5
  33. 33. 1. Security to the left 2. The whitest white box 3. An expert comes from outside 4. Old hand or a Fresh Start 5. A full project cycle 1. Hack yourself first 2. Disclose known vulnerabilities 3. Security by non-testability 4. The testing setup 5. Be ready 1. Progress reports 2. Executive Summary 3. Clearly explained with specific actions 4. Prioritised action plan 5. Assistance with fixes 1. Scoping 2. Preparation 3. Reporting
  34. 34. Key takeaways: 1. Every little helps, not all or nothing 2. Efficiency, efficiency, efficiency 3. Build a dialogue joshg@comsecglobal.com joshcgrossman@gmail.com @JoshCGrossman #TrevorForget S1 S2 S3 S4 S5 P1 P2 P3 P4 P5 R1 R2 R3 R4 R5

Editor's Notes

  • Line
    Line
    Line
    Line
    Line
    Line
  • 50% build, 30% def, 20% break,
    Story of Arch Review + PT
    Initially aimed at attackers
  • If you are a breaker, there are tons of tips
    These are cool, great talks
  • About a year or so ago, I started thinking
    What about the defenders and builders
    See less of this sort of thing
  • These aren’t Comsec’s views
    I’m not a Comsec spokesman
    This isn’t a Comsec corporate advert
    This is about spreading ideas, not selling services
    Appreciate, enjoy, proud to work there
    Although through Comsec I get the opportunity
  • Something for everyone, big and small
    Defenders/builders – appsec tests deliver real value and make an application more secure.
    Breakers – are you ready for this?
    Questions via email
  • Why?
    Regulation
    Customer’s requirements
    But you’d do it anyway, right? …Right?
  • You are already paying for it
    Story about the cheap supplier and the poor report
  • Pay a little more and get some more value
    BUT NOT JUST cost, Testing based on hours estimation – want to maximize
  • Not every project is the same
    Ideas will be relevant to different people
  • Scoping and deciding on the project
    Getting ready for the project
    The project output and beyond
  • Not new but still important
    War story - why didn’t you tell us before
    design/architecture review
    review key design decisions
    The earlier, the easier
    Discovered late, lots of simple findings or worse complex findings
  • Wordpress blackbox story
    If you want realism, simulation – If you want that, insult North Korea
    NO, want VALUE!
    Most coverage, Highest quality, Shortest time
    Give me the github
    This isn’t an exam, this is improvement
  • Very culture dependent
    your boss might listen to you
    more likely to listen to consultant report
    Make sure pushing your points
    cannot influence misreport but still benefit
    make sure issues receive prominence
  • Yes, both
    old hand for a few times and then if you start seeing less important findings, try switching
  • Basic outline
    Not a full project
    Not to aim for
    Opportunities are being missed
  • Proper kickoff
    Design review
    A session to explain the findings
    A session to discuss proposed fixes or ongoing discussion
    A retest of the findings discovered
    Also consider infra element, what if Equifax
  • Don’t do a full summary
    This is what’s possible at the scoping stage
    Now we move on to prep
  • Zap is free, burp is cheap
    Catch the low hanging fruit first
    Also, it's fun :)
    But I get not everyone is time rich
  • may already have known issues
    areas you are concerned about
    implemented in a non-standard way.
    Disclose them up front
    isn't competition, about efficiency
  • great technologies to screw up attackers
    WAF - blocking or throttling.
    Am I testing your WAF? Disable it
    Maybe afterwards but is it really worth it…
    WAF War story
  • No 100% safe - either lying or feather bashing
    Use a testing/dedicated instance
    Backup the database
    Verify findings on production
    Client with the Javascript DoS
    Don’t make us use your PCs/laptop
  • request list - in writing
    Agree ready date
    Test it all yourself
    lose half a day on stuff not working
  • Don’t do a full summary
    This is what’s possible at the prep stage
    Now we move on to reporting
  • findings vs status
    Problems and Critical Findings- ASAP
    Everything else, wait until the end
    - Consider findings together in context and full impact
    No duplication of effort
    Ongoing comms – important
  • A little obvious
    Should summarise the report
    Must look good
    Agree a format that you can show clients
    Business impact crucial, you need to justify fixing to someone, ammunition must be in the report
  • Also a little obvious
    Make sure every finding is easy to understand + repro
    This the only instance? Others?
    If lots, how can you find this across the codebase?
    Different XSS fixes.
  • Step that gets missed
    When you are going to fix
    testers to show urgency and devs to understand difficulty
    Short term/long term actions for findings
  • Discuss the proposed fix with tester
    No misunderstandings
    Make sure the fix properly addresses the risk
    don’t want to fix wrong, fix incompletely or fix wrong thing
    CSRF story
  • Progress reports
    Executive Summary
    Clearly explained with specific actions
    Prioritised action plan
    Assistance with fixes
  • There is a lot here to remember, I will write it up somewhere
    This isn't all or nothing, anything you can do will help
    Build an ongoing dialogue with your testers/consultants

×