Your SlideShare is downloading. ×
Web Bypass Testing
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Web Bypass Testing

1,060
views

Published on


0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,060
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. An Industrial Case Study of Bypass Testing on Web Applications Joint research Dr. Ye Wu , Xiaochen Du , Hong Huang , Vasileios Papadimitriou , Qingxiang Wang and Joann J. Ordille of Avaya Labs Research Based on papers in ISSRE 2004 and ICST 2008 Jeff Offutt Software Engineering George Mason University Fairfax, VA USA www.cs.gmu.edu/~offutt/ [email_address] Expanded version of a talk given at the first International Conference on Software Testing, Verification and Validation
  • 2. Outline of Talk
    • Motivation
    • Applying bypass testing – early results
    • Automating bypass testing
    • Real-world examples
    • Industrial case study
    • Conclusions and future work
    ICST 2008 © Jeff Offutt
  • 3. Web Application Input Validation Sensitive Data Client Server
    • Bad Data
    • Corrupts data base
    • Crashes server
    • Security violations
    Check data Check data Malicious Data Can “bypass” data checking
  • 4. Deploying Software
    • Bundled : Pre-installed on computer
    • Shrink-wrap : Bought and installed by end-users
    • Contract : Purchaser pays developer to develop and install, usually for a fixed price
    • Embedded : Installed on a hardware device, usually with no direct communication with user
    © Jeff Offutt, 2004
    • Web : Executed across the Internet through HTTP
  • 5. Problem Parameters
    • HTTP is a stateless protocol
      • Each request is independent of previous request
    • Servers have little information about where a request comes from
    • Web site software is extremely loosely coupled
      • Coupled through the Internet – separated by space
      • Coupled to diverse hardware devices
      • Written in diverse software languages
    © Jeff Offutt ICST 2008
  • 6. Bypass Testing
    • “ bypass ” client-side constraint enforcement
    • Bypass testing constructs tests to intentionally violate constraints :
      • Eases test automation
      • Validates input validation
      • Checks robustness
      • Evaluates security
    © Jeff Offutt ICST 2008
  • 7. © Jeff Offutt User Name: Small $150 Version to purchase: Age: Large $500 Medium $250 Simple Example Web Page ICST 2008
  • 8. © Jeff Offutt User Name: Small $150 Version to purchase: Age: Large $500 Medium $250 Proper Behavior ICST 2008 Username should be plain text only. Age should be between 18 and 150. Invalid data, please correct … Alan < Turing 500
  • 9. Abbreviated HTML © Jeff Offutt <FORM > <INPUT Type=“text” Name=“username” Size=20> <INPUT Type=“text” Name=“age” Size=3 Maxlength=3> <P> Version to purchase: … <INPUT Type=“radio” Name=“version” Value=“150” Checked> <INPUT Type=“radio” Name=“version” Value=“250”> <INPUT Type=“radio” Name=“version” Value=“500”> <INPUT Type=&quot;submit&quot; onClick=&quot;return checkInfo(this.form)&quot;> <INPUT Type=“hidden” isLoggedIn=“no”> </FORM> ICST 2008
  • 10. Bypass Behavior
    • Extremely loose coupling …
    • combined with the stateless protocol …
    • allows users to easily bypass client-side checking :
    © Jeff Offutt Users can save and modify the HTML ICST 2008
  • 11. Saved & Modified HTML © Jeff Offutt <FORM > <INPUT Type=“text” Name=“username” Size=20> <INPUT Type=“text” Name=“age” Size=3 Maxlength=3> <P> Version to purchase: … <INPUT Type=“radio” Name=“version” Value=“150”> <INPUT Type=“radio” Name=“version” Value=“250”> <INPUT Type=“radio” Name=“version” Value=“500” Checked> <INPUT Type=&quot;submit&quot; onClick=&quot;return checkInfo(this.form)&quot;> <INPUT Type=“hidden” isLoggedIn= “no” > </FORM> Allows an input with arbitrary age, no checking, cost=$25 … ‘ <‘ can crash an XML parser Text fields can have SQL statements 25 yes ICST 2008
  • 12. SQL Injection © Jeff Offutt, 2004 User Name: Password: turing enigma Original SQL: SELECT username FROM adminuser WHERE username='turing' AND password ='enigma' “ injected” SQL: SELECT username FROM adminuser WHERE username='turing‘ OR ‘1’ = ‘1’ AND password ='enigma‘ OR ‘1’ = ‘1’ ’ OR ‘1’=‘1 ’ OR ‘1’=‘1
  • 13. Applying Bypass Testing
    • Analyze HTML to extract each form element
    • Model constraints imposed by HTML and JavaScript
    • Rules for data generation :
      • From client-side constraints
      • Typical security violations
      • Common input mistakes
    © Jeff Offutt ICST 2008 Validating input data on the client is like asking your opponent to hold your shield in a sword fight
  • 14. Example Client-Side Constraint Rules
    • Violate size restrictions on strings
    • Introduce values not included in static choices
      • Radio boxes
      • Select ( drop-down ) lists
    • Violate hard-coded values
    • Use values that JavaScripts flag as errors
    • Change “ transfer mode ” (get, post, …)
    • Change destination URL s
    © Jeff Offutt, 2004
  • 15. Example Server-Side Constraint Rules
    • Data type conversion
    • Data format validation
    • Inter-field constraint validation
    • Inter-request data fields (cookies, hidden)
    © Jeff Offutt, 2004
  • 16. Example Security Violation Rules © Jeff Offutt, 2004 Potential Illegal Character Symbol Empty String Commas , Single and double quotes ’ or ” Tag symbols Tag symbols < and > Directory paths .. ../ Strings starting with forward slash / Strings starting with a period . Ampersands & Control character NIL, newline Characters with high bit set 254 and 255 Script symbols < javascript > or < vbscript >
  • 17. First Example CyberChair
    • CyberChair : Web-based conference management
      • www.cyberchair.org
      • ICSE , ICST, ISSRE, ICSM, …
    • Bypass testing found 5 types of faults
      • Submission without authentication
      • Unsafe use of hidden form field
      • Disclosing information (program crashes)
      • Lack of validation of file type
      • Allows papers of negative length
    © Jeff Offutt, 2004
  • 18. Automating Bypass Testing
    • Autobypass : A web application that accepts a URL and generates input data for the HTML form fields
      • Also accepts any needed login data
      • MS thesis by Vasileios Papadimitriou
    • Built on top of HttpUnit
      • Parses HMTL pages
      • Identifies forms and their fields
      • Creates bypass test cases
      • Submits test cases to the application’s server
    ICST 2008 © Jeff Offutt
  • 19. Types of Client Input Validation
    • Client side input validation is performed by HTML form controls, their attributes, and client side scripts that access DOM
    • Validation types are categorized as HTML and scripting
      • HTML supports syntactic validation
      • Client scripting can perform both syntactic and semantic validation
    ICST 2008 © Jeff Offutt HTML Constraints Scripting Constraints
    • Length (max input characters)
    • Value (preset values)
    • Transfer Mode (GET or POST)
    • Field Element (preset fields)
    • Target URL (links with values)
    • Data Type (e.g. integer check)
    • Data Format (e.g. ZIP code format)
    • Data Value (e.g. age value range)
    • Inter-Value (e.g. credit # + exp. date)
    • Invalid Characters (e.g. <,../,&)
  • 20. Example Interface: yahoo registration form ICST 2008 © Jeff Offutt Limited Length (HTML) Preset Values (HTML) Preset Transfer Mode in form definition (HTML) Preset No of Fields (HTML) URL with preset Values (HTML) Data Value, Type, & Format validation (script) Inter Value validation (script)
  • 21. Test Value Selection
    • Challenge :
      • How to automatically provide effective test values?
    • Semantic Domain Problem (SDP)
      • Values within the application domain are needed
      • Enumeration of all possible test values is inefficient
    • Possible Solutions
      • Random Values (ineffective – lots of junk )
      • Automatically generated values (very hard )
      • Taking values from session log files (feasible but incomplete )
      • Tester input ( feasible )
    • AutoBypass uses an input domain created by parsing the interface and tester input
    ICST 2008 © Jeff Offutt
  • 22. AutoBypass
    • AutoBypass steps (the big picture)
    ICST 2008 © Jeff Offutt Parse Interface Set Default Values Generate Test Cases & Run Tests Review Results
    • All HTML violation rules are used to generate test cases
    • First version of AutoBypass does NOT automatically violate scripting validation , but :
      • AutoBypass behaves as a browser with scripts disabled
      • Tester can provide test inputs that will bypass scripting validation.
  • 23. Real-World Examples ICST 2008 © Jeff Offutt atutor.ca Atalker demo.joomla.or Poll, Users phpMyAdmin Main page, Set Theme, SQL Query, DB Stats brainbench.com Submit Request Info, New user myspace.com Events & Music Search bankofamerica.com ATM locator, Site search comcast.com Service availability ecost.com Detail submit, Shopping cart control google.com Froogle, Language tools pageflakes.com Registration wellsfargolife.com Quote search nytimes.com Us-markets mutex.gmu.edu Login form yahoo.com Notepad, Composer, Search reminder, Weather Search barnesandnoble.com Cart manager, Book search/results amazon.com Item dispatch, Handle buy Pure black-box testing means no source (or permission) needed !
  • 24. Classifying Output Responses
    • ( V ) Valid Responses : invalid inputs are adequately processed by the server
    • ( F ) Faults & Failures : invalid inputs that cause abnormal server behavior (typically caught by web server when application fails to handle the error)
    • ( E ) Exposure : invalid input is not recognized by the server and abnormal software behavior is exposed to the users
    ICST 2008 © Jeff Offutt ( V1 ) Server acknowledges the invalid request and provides an explicit message regarding the violation ( V2 ) Server produces a generic error message ( V3 ) Server apparently ignores the invalid request and produces an appropriate response ( V4 ) Server apparently ignores the request completely
  • 25. Results ICST 2008 © Jeff Offutt v
  • 26. Research to Practice ICST 2008 © Jeff Offutt “ Knowing is not enough, we must apply. Willing is not enough, we must do.” Goethe They’re teaching a new way of plowing over at the Grange tonight - you going? Naw - I already don’t plow as good as I know how...
  • 27. Industrial Case Study
    • Inventions from scientists are slow to move into industrial practice
    • We wanted to investigate whether the obstacles are :
      • Technical difficulties of applying to industrial use
      • Social barriers
      • Business constraints
    • Tried to technology transition bypass testing to the research arm of a software company
    ICST 2008 © Jeff Offutt
  • 28. Avaya’s NPP Technology
    • Avaya Labs Research creates research prototypes of software systems, then turns successful prototypes over to product groups
    • NPP : Notification Preference Portal
      • Users specify how and when they should be contacted
      • Types include phone, email and SMS
      • Contacts an be made in parallel or sequentially
    • Used to notify users of events
    • This study was part of system testing
      • NPP is now in production
    ICST 2008 © Jeff Offutt
  • 29. NPP Design and Implementation
    • NPP is a highly user interactive web application
    • Uses many screens
    • Javascript is used on the client to :
      • Validate inputs
      • Dynamically modify screen by manipulating DOM
      • Encode input data into XML before sending to the server
    • The second two uses necessitated changes to how bypass testing was applied
    ICST 2008 © Jeff Offutt
  • 30. NPP Bypass Tests
    • Bypass testing analyzes HTML ( statically ) and generates inputs that violate input constraints
    • The extensive modification of the HTML DOM meant HTML could not be analyzed statically
    • Instead, the input requirements of the server software were identified, and tests encoded in XML
      • A special-purpose tool was written to convert XML tests into HtmlUnit tests
    ICST 2008 © Jeff Offutt
  • 31. NPP Testing Results
    • Six NPP screens were tested
    • Tests are invalid inputs – exceptions are expected
    • Effects on back-end were not checked
      • Failure analysis just based on response screens
    ICST 2008 © Jeff Offutt Web Screen Tests Failing Tests Unique Failures Points of Contact 42 23 12 Time Profile 53 23 23 Notification Profile 34 12 6 Notification Filter 26 16 7 Change PIN 5 1 1 Create Account 24 17 14 TOTAL 184 92 63
  • 32. Types of Faults
    • Invalid data saved into the database
      • These lead to more visible failures later
      • Example : An invalid password was accepted, but the account could not subsequently be used
      • Example : Subsequent messages could not be sent to invalid contacts
    • No response at all
      • Probably a software component failed
      • Database or web server sometimes crashed
    • Exposure errors
      • Internal exception message sent in the response screen
    ICST 2008 © Jeff Offutt
  • 33. Conclusions
    • Bypass testing worked very well in an industrial context
      • There is no technical obstacle to adoption
      • Source is not needed
    • Even hand generation of tests was quite cheap in comparison with other methods
      • There is no valid business barrier
    • Most problems are unlikely with non-malicious users and a correct implementation
      • But client-side validation is notoriously error-prone
    ICST 2008 © Jeff Offutt We conclude the primary obstacle is social
  • 34. Future Work
    • A major observability problem with web application testing is detecting invalid database values
      • A comprehensive valid data model could allow database auditors to be developed
    • Javascript needs to be fully parsed and analyzed
      • Implement scripting violation rules
    • Widen the scope of testing from a form/ to a site
      • Test sequence of events
      • Application level Input Domain
    • Explore possibilities for automated response evaluation
    • Ajax allows client-server messages to be sent asynchronously through message passing
      • This introduces more controllability and observability problems
    ICST 2008 © Jeff Offutt

×