Successfully reported this slideshow.

StarWest 2009 - Detective Work For Testers: Finding Workflow Based Defects


Published on

Do you know why your software testing strategy isn't finding many of the "really big" bugs hidden in the web-based software your company churns out? Find out now...

Published in: Technology
  • Be the first to comment

  • Be the first to like this

StarWest 2009 - Detective Work For Testers: Finding Workflow Based Defects

  1. 1. 1 30 January 2015 Detective Work for Testers finding workflow-based defects Rafal M. Los HP ASC Sr. Security Solutions Expert
  2. 2. Security Quality is a Tough Business 2
  3. 3. 3 Workflow-based security defects in Web applications are especially difficult to identify because they evade traditional, point-and-scan vulnerability detection techniques. Understanding these potential defects and why black-box scanners typically miss them, are key to creating a testing strategy for successful detection and mitigation. Rafal Los describes the critical role that testers play in assessing application work flows and how business process-based testing techniques can uncover these flaws. Rafal demystifies the two main types of workflow-based application vulnerabilities—business process logic vulnerabilities and parameter-based vulnerabilities—and provides you with a sound basis to improve your testing strategies. Become a security testing sleuth and learn to find the workflow-based security defects before your system is compromised. Abstract
  4. 4. Background What we know for sure… − Security vulnerabilities exist in virtually all web sites − “Security in a vacuum” is your largest risk − A cooperative SDL mingling QA and IT Security is key QA’s advantage − Understanding site flow − Understanding business logic  The advantage of knowing the application! 4
  5. 5. Challenges Traditional issues − “Security is IT Security’s problem, right?” − Domain separation (developers vs. QA vs. security) − Lack of tools… processes… education Web 2.0 challenges − Extremely complex technologies − “The Hydra” problem − “The Maze” problem 5
  6. 6. Workflow-Based Security Defects Defined: Any security defects which can only be identified by following a specific flow, or logic within the site or application. To be clear… − Not a new type of attack vector − Identifying old attacks in their hiding places − Workflows often hide untested defects (like land-mines) 6
  7. 7. Workflow-Based Security Defects Common examples  Registration Form  Legitimate Transactions (rewards points) 7 Form 1: Enter User Details Form 2: Enter Account Info (userID, pwd) Form 3: Enter email, finalize reg. Form 4: Confirm All Details Page 1: Select a retailer Page 2: Enter purchase details Page 3: Complete sale Page 4: Receive rewards points
  8. 8. An(almost)Real-Life Example! Just another online banking app Multiple steps = multiple hiding spots 8
  9. 9. An(almost)Real-Life Example! Just another online banking app Transferring money… Form inputs  2 drop-downs, 2 free-form 9
  10. 10. An(almost)Real-Life Example! Just another online banking app Can we spot logic flaws? confirmation page  get free money? 10
  11. 11. An(almost)Real-Life Example! What do we learn? Defects  “?”  forks  workflows  use-cases − Most use-cases are non-linear* − *Non-linear = users have options/choices − Automation cannot intelligently trace non-linear use- cases on its own… human intellect is required 11
  12. 12. Workflow Complexity Workflow complexity is a critical factor Low-Order Workflow − One to several steps − No forks − Pre-defined selection fields (no free-form input) High-Order Workflows − Potentially dozens of steps − Multiple forks − Free-form input fields (with back-end logic) 12
  13. 13. Why Automation Fails • Automation relies on spidering technology − essentially link processing • Spiders submit data & click buttons − Most spiders submit garbage data  fail − Good spiders allow user to pre-define a data set  possibly fail − Great spiders submit a number of combinations per form 13
  14. 14. Spiders, Clicks and Data-Sets What about- highly complex forms? anti-automation technologies? 14 30 January 2015 Link: … Link: … Link: … Form__ Param1= Param2= Param3= [button] Link: … Link: … Link: … Link: … Link: … Link: … Link: … Link: … Link: …
  15. 15. Complexity vs. Automation The problem with highly complex structures… 15
  16. 16. High Complexity Case Study Online trading application − 100 forms+ − ~10,000 combinations each − >1,000,000 possible execution paths across the application… Can be solved… − Built an EFD… What’s an EFD you ask? Hold that thought! 16
  17. 17. Break the problem down • Over 1,000,000 possible permutations? • Where do you begin!? … it’ll make your head spin! Here’s the secret… 1. Build an EFD [Execution-Flow Diagram] 2. Identify pivotal inputs 3. Identify 2 additional non-pivotals 4. Test unique execution paths +2 Tackling Complexity 17
  18. 18. The EFD Tracking execution flow Think of an EFD as the ultimate cheat through a maze… • 2 possible exits to the maze • 1,000,000 possible ways through, only 2 get you out! • EFD is a cheat-sheet (or guide) for following the 2 paths out EFD… Business logic + Form Variations = Complete test 18
  19. 19. Building an EFD Step 1: Business logic Sample bill-pay flow 19 User Access Bill Pay Tab Pick payee Enter payment details Add Payee Details Verify Out-of-Band Submit payment Receive confirmation Return to bill pay tab New payee
  20. 20. Building an EFD Step 2: Possible form inputs Pick a step and expand… 20 User Access Bill Pay Tab Pick payee Enter payment details Add Payee Details Verify Out-of-Band Submit payment Receive confirmation Return to bill pay tab New payee
  21. 21. Building an EFD Step 2: Possible form inputs Pick a step and expand… Enter payment details 4 Form Fields Payee 1 Payee 2 New One-Time Recurring Notes Field Checking Savings Line-of- Credit 2 Pivotals Here
  22. 22. Focus on the Pivotals & Forks Maximize coverage Minimize effort • IT Security uses a hammer– don’t see flows − QA succeeds here, this is your core function • QA builds fork/flow –driven tests − Minimizes test cycle time (no time wasted repeating) − Maximizes test-case, workflow completeness • Much more complete approach to testing 22
  23. 23. EFD to Test Plan Execution Execute the test plan 1. Fire up your security testing “tool of choice” 2. Map the application  EFD 3. Choose depth or breadth –first technique 4. Execute the test cases • Use your use-case “insider” knowledge • Emphasize forks & pivotals 5. Verify & validate results 23
  24. 24. Start to Finish Follow the yellow brick road… 24 Map out business logic Identify pivotals/forks Execute test cases Validate & Verify results
  25. 25. Bring It Home Identifying risks in applications involves • Security  “breakers” or hackers • Quality  “testers” … use-case knowledge • Developers  “builders” … engineers … one cannot mitigate risks without the others. 25 30 January 2015
  26. 26. Rafal Los Security Specialist HP Application Security Center 26 Blog Following the White Rabbit Email Direct (404) 606-6056 Twitter