1 30 January 2015
Detective Work for Testers
finding workflow-based defects
Rafal M. Los
HP ASC Sr. Security Solutions Exp...
Security Quality is a Tough
Business
2
3
Workflow-based security defects in Web applications are
especially difficult to identify because they evade traditional,...
Background
What we know for sure…
− Security vulnerabilities exist in virtually all web sites
− “Security in a vacuum” is ...
Challenges
Traditional issues
− “Security is IT Security’s problem, right?”
− Domain separation (developers vs. QA vs. sec...
Workflow-Based Security Defects
Defined: Any security defects which can only be
identified by following a specific flow, o...
Workflow-Based Security Defects
Common examples
 Registration Form
 Legitimate Transactions (rewards points)
7
Form 1:
E...
An(almost)Real-Life Example!
Just another online banking app
Multiple steps = multiple hiding spots
8
An(almost)Real-Life Example!
Just another online banking app
Transferring money…
Form inputs  2 drop-downs, 2 free-form
9
An(almost)Real-Life Example!
Just another online banking app
Can we spot logic flaws?
confirmation page  get free money?
...
An(almost)Real-Life Example!
What do we learn?
Defects  “?”  forks  workflows  use-cases
− Most use-cases are non-line...
Workflow Complexity
Workflow complexity is a critical factor
Low-Order Workflow
− One to several steps
− No forks
− Pre-de...
Why Automation Fails
• Automation relies on spidering technology
− essentially link processing
• Spiders submit data & cli...
Spiders, Clicks and Data-Sets
What about-
highly complex forms?
anti-automation technologies?
14 30 January 2015
Link: …...
Complexity vs. Automation
The problem with highly complex structures…
15
High Complexity Case Study
Online trading application
− 100 forms+
− ~10,000 combinations each
− >1,000,000 possible execu...
Break the problem down
• Over 1,000,000 possible permutations?
• Where do you begin!? … it’ll make your head spin!
Here’s ...
The EFD
Tracking execution flow
Think of an EFD as the ultimate cheat through a
maze…
• 2 possible exits to the maze
• 1,0...
Building an EFD
Step 1: Business logic
Sample bill-pay flow
19
User Access
Bill Pay Tab
Pick
payee
Enter payment
details
A...
Building an EFD
Step 2: Possible form inputs
Pick a step and expand…
20
User Access
Bill Pay Tab
Pick
payee
Enter payment
...
Building an EFD
Step 2: Possible form inputs
Pick a step and expand…
Enter payment
details
4 Form Fields
Payee
1
Payee
2
N...
Focus on the Pivotals & Forks
Maximize coverage
Minimize effort
• IT Security uses a hammer– don’t see flows
− QA succeeds...
EFD to Test Plan
Execution
Execute the test plan
1. Fire up your security testing “tool of choice”
2. Map the application ...
Start to Finish
Follow the yellow
brick road…
24
Map out business logic
Identify pivotals/forks
Execute test cases
Validat...
Bring It Home
Identifying risks in applications involves
• Security  “breakers” or hackers
• Quality  “testers” … use-ca...
Rafal Los
Security Specialist
HP Application Security Center
26
Blog Following the White Rabbit
http://www.communities.hp....
Upcoming SlideShare
Loading in...5
×

StarWest 2009 - Detective Work For Testers: Finding Workflow Based Defects

801

Published on

Do you know why your software testing strategy isn't finding many of the "really big" bugs hidden in the web-based software your company churns out? Find out now...

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
801
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • … last point… “if only there was a name for that”…  Requirements Management! 
  • Traditional Issues
    Security is slowly starting to permeate into the other aspects of the SDL… but still largely misunderstood as “security’s problem”
    IT Security cannot possibly handle the challenge of securing complex web technologies alone
    Code complexity, servers, systems, integrations are too much for IT Security to try and comprehend
    Domain separation continues to exist – development, qa/testing, security… none of these cooperatively work to reduce risks
    Domain separation causes the ‘downward spiral of infighting’… and the blame game
    Distinct lack of tools… that’s right – LACK OF TOOLS! So many vendors but… where is security?
    How can security folks bring security defect-finding tools to QA folks and make it comprehensible? Usable?
    Web 2.0 Challenges
    Extremely complex technologies are making testing miserable… creating test cases is an ever-increasing exercise in madness
    Flash and “rich media” applications
    AJAX , mash-ups and multi-point applications
    Complexity is the ancient enemy of good security… like a battlefield scenario
    The “Hydra Problem”
    There are too many “heads” in the development organization
    Consistency of thought (wrt security) is nearly impossible in some enterprises
    Many poor practices, ways to get the solution wrong…
    No matter how many heads you cut off, it seems like there is always another one popping up!
    The “Maze Problem”
    Complex web sites/applications are like a “maze”
    Many, many data ingress/egress points, interactive systems, partner data feeds, exports, etc…
    Complex execution paths make it almost a certainty that IT Security alone will miss critical issues
    Only QA has the map to navigate the maze (or at least the best chance to do so)
  • Workflow-based defects are not a new type of attack, only a new way to find these types of attacks within an application (such as an XSS buried within a 3-part form, on page 3)… etc
  • Multiple steps means that a “scanner” or automation must know how to proceed from one step to the next
    Scanners aren’t intelligent but make best-guesses… there is no guarantee they will succeed in stepping-through
    Automation cannot think therefore has the chance to fail
  • Form-based fields can almost always be manipulated in applications where security isn’t integrated in
    These forms typically either echo-back (XSS) or access a database (SQLi)
  • A logic flaw may allow us (in an extreme case) to make a “free transfer”…
    Transfer money, without losing it from the origin account!
    Many opportunities for logic flaws within these types of application flows
  • Automation relies on “spidering” a site…
    Spiders fail at work-flows because spiders do not enter data (typically)
    Even GOOD spiders (which have the ability to enter data) fail because data must be specific
    One-time actions complicate workflows and fail even the best spiders
  • Workflows can be tricky because they often require specific combinations of inputs (or worse… free-form fields) to continue
    Some workflows are low order (only a few steps, can be navigated using a spider)
    Some workflows are high order (spidering cannot navigate the flow)
  • Automation alone will almost always fail…
    Simply following links and blindly submitting forms (like most spiders do) isn’t going to walk through a flow
    How does a spider know it’s actually moving forward and hasn’t forked or gone off the flow?
    Submitting garbage (pre-defined static data)  FAIL
    Submitting user-defined static data  still some FAIL
    Submitting one-time forms, fields  the KEY!
    How do you account for a CAPTCHA?
    What about a one-time password?
    What about other form complexities? What does the audience have to work with??
  • Looking at this page it’s easy to see why many black-box testing tools fail…
    Too many options… consume too many resources
    CPU + Memory
    Tables in memory have to handle each path, results, and diff… that’s not very likely in “high complexity” situations
  • When you’re faced with a form or set of forms that allows for 100,000+ permutations, how do you solve the problem?
    Refer to the Data-Flow Diagram (DFD)
    Building an execution-flow-diagram (EFD) is absolutely critical!
    Identify “pivotal inputs” as – input which causes your execution flow to fork
    Execute 2 additional non-pivotal data paths to ensure understanding of execution path
    Make sure you’re only testing unique + 2
    2 extras for certainty
  • You can explain an EFD best by using the “maze” example… it’s like having a turn-by-turn map!
    You know which paths lead to nowhere (or the same place)
    Business logic diagrams (Data-flow diagrams or DFDs) are combined with form variations to complete the picture!
  • Break out each form by the field and options for that field
    Highlight each of the options which are “pivotal” (which ones create a new fork?)
  • IT Security will miss defects (false-negatives) by not fully exercising the application and all workflows
    QA does a better job natively by focusing on the execution, application understanding and coverage
    QA’s gap is the security aspect of testing – but automation can compensate for some, manual security analysis for the rest!
    QA’s role is crucial because…
    MINIMIZE TIME
    MAXIMIZE COMPLETENESS

    Save time, find more bugs… why isn’t everyone doing this already?
  • StarWest 2009 - Detective Work For Testers: Finding Workflow Based Defects

    1. 1. 1 30 January 2015 Detective Work for Testers finding workflow-based defects Rafal M. Los HP ASC Sr. Security Solutions Expert
    2. 2. Security Quality is a Tough Business 2
    3. 3. 3 Workflow-based security defects in Web applications are especially difficult to identify because they evade traditional, point-and-scan vulnerability detection techniques. Understanding these potential defects and why black-box scanners typically miss them, are key to creating a testing strategy for successful detection and mitigation. Rafal Los describes the critical role that testers play in assessing application work flows and how business process-based testing techniques can uncover these flaws. Rafal demystifies the two main types of workflow-based application vulnerabilities—business process logic vulnerabilities and parameter-based vulnerabilities—and provides you with a sound basis to improve your testing strategies. Become a security testing sleuth and learn to find the workflow-based security defects before your system is compromised. Abstract
    4. 4. Background What we know for sure… − Security vulnerabilities exist in virtually all web sites − “Security in a vacuum” is your largest risk − A cooperative SDL mingling QA and IT Security is key QA’s advantage − Understanding site flow − Understanding business logic  The advantage of knowing the application! 4
    5. 5. Challenges Traditional issues − “Security is IT Security’s problem, right?” − Domain separation (developers vs. QA vs. security) − Lack of tools… processes… education Web 2.0 challenges − Extremely complex technologies − “The Hydra” problem − “The Maze” problem 5
    6. 6. Workflow-Based Security Defects Defined: Any security defects which can only be identified by following a specific flow, or logic within the site or application. To be clear… − Not a new type of attack vector − Identifying old attacks in their hiding places − Workflows often hide untested defects (like land-mines) 6
    7. 7. Workflow-Based Security Defects Common examples  Registration Form  Legitimate Transactions (rewards points) 7 Form 1: Enter User Details Form 2: Enter Account Info (userID, pwd) Form 3: Enter email, finalize reg. Form 4: Confirm All Details Page 1: Select a retailer Page 2: Enter purchase details Page 3: Complete sale Page 4: Receive rewards points
    8. 8. An(almost)Real-Life Example! Just another online banking app Multiple steps = multiple hiding spots 8
    9. 9. An(almost)Real-Life Example! Just another online banking app Transferring money… Form inputs  2 drop-downs, 2 free-form 9
    10. 10. An(almost)Real-Life Example! Just another online banking app Can we spot logic flaws? confirmation page  get free money? 10
    11. 11. An(almost)Real-Life Example! What do we learn? Defects  “?”  forks  workflows  use-cases − Most use-cases are non-linear* − *Non-linear = users have options/choices − Automation cannot intelligently trace non-linear use- cases on its own… human intellect is required 11
    12. 12. Workflow Complexity Workflow complexity is a critical factor Low-Order Workflow − One to several steps − No forks − Pre-defined selection fields (no free-form input) High-Order Workflows − Potentially dozens of steps − Multiple forks − Free-form input fields (with back-end logic) 12
    13. 13. Why Automation Fails • Automation relies on spidering technology − essentially link processing • Spiders submit data & click buttons − Most spiders submit garbage data  fail − Good spiders allow user to pre-define a data set  possibly fail − Great spiders submit a number of combinations per form 13
    14. 14. Spiders, Clicks and Data-Sets What about- highly complex forms? anti-automation technologies? 14 30 January 2015 Link: … Link: … Link: … Form__ Param1= Param2= Param3= [button] Link: … Link: … Link: … Link: … Link: … Link: … Link: … Link: … Link: …
    15. 15. Complexity vs. Automation The problem with highly complex structures… 15
    16. 16. High Complexity Case Study Online trading application − 100 forms+ − ~10,000 combinations each − >1,000,000 possible execution paths across the application… Can be solved… − Built an EFD… What’s an EFD you ask? Hold that thought! 16
    17. 17. Break the problem down • Over 1,000,000 possible permutations? • Where do you begin!? … it’ll make your head spin! Here’s the secret… 1. Build an EFD [Execution-Flow Diagram] 2. Identify pivotal inputs 3. Identify 2 additional non-pivotals 4. Test unique execution paths +2 Tackling Complexity 17
    18. 18. The EFD Tracking execution flow Think of an EFD as the ultimate cheat through a maze… • 2 possible exits to the maze • 1,000,000 possible ways through, only 2 get you out! • EFD is a cheat-sheet (or guide) for following the 2 paths out EFD… Business logic + Form Variations = Complete test 18
    19. 19. Building an EFD Step 1: Business logic Sample bill-pay flow 19 User Access Bill Pay Tab Pick payee Enter payment details Add Payee Details Verify Out-of-Band Submit payment Receive confirmation Return to bill pay tab New payee
    20. 20. Building an EFD Step 2: Possible form inputs Pick a step and expand… 20 User Access Bill Pay Tab Pick payee Enter payment details Add Payee Details Verify Out-of-Band Submit payment Receive confirmation Return to bill pay tab New payee
    21. 21. Building an EFD Step 2: Possible form inputs Pick a step and expand… Enter payment details 4 Form Fields Payee 1 Payee 2 New One-Time Recurring Notes Field Checking Savings Line-of- Credit 2 Pivotals Here
    22. 22. Focus on the Pivotals & Forks Maximize coverage Minimize effort • IT Security uses a hammer– don’t see flows − QA succeeds here, this is your core function • QA builds fork/flow –driven tests − Minimizes test cycle time (no time wasted repeating) − Maximizes test-case, workflow completeness • Much more complete approach to testing 22
    23. 23. EFD to Test Plan Execution Execute the test plan 1. Fire up your security testing “tool of choice” 2. Map the application  EFD 3. Choose depth or breadth –first technique 4. Execute the test cases • Use your use-case “insider” knowledge • Emphasize forks & pivotals 5. Verify & validate results 23
    24. 24. Start to Finish Follow the yellow brick road… 24 Map out business logic Identify pivotals/forks Execute test cases Validate & Verify results
    25. 25. Bring It Home Identifying risks in applications involves • Security  “breakers” or hackers • Quality  “testers” … use-case knowledge • Developers  “builders” … engineers … one cannot mitigate risks without the others. 25 30 January 2015
    26. 26. Rafal Los Security Specialist HP Application Security Center 26 Blog Following the White Rabbit http://www.communities.hp.com/securitysoftware/blogs/rafal/default.aspx Email Rafal.Los@hp.com Direct (404) 606-6056 Twitter http://twitter.com/RafalLos

    ×