Testingfor Sw Security


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Media player plays flawlessly but manages to do so by writing the files out to unencrypted temporary storage. oops.
  • se.fit.edu Software Engineering Florida Institute of Technology?
  • Environmental stress refers to situations outside the program that cause poor performance/responses by the program. For example, if there is not enough memory, even though your program has no memory leak that is causing that, your app may crash if it isn’t programmed to handle that as a possible “return code” on a request for more memory, etc.
  • The authors scoured bug databases incident reports advisories etc. Identified 2 broad categories of attacks that can be used to expose vulnerabilities
  • Apps rely heavily on their environment to work properly: OS to provide memory, disk space, other resources Filesystem to read and write data Registry to store and retrieve information etc.
  • Need to integrate failures into tests to evaluate their impact on the security of the product itself and its data.
  • “ why” is on next slide -- namely, You redirect a particular system call to your own impostor function
  • Most extreme vulnerabilities is when password data or other sensitive data is stored unprotected in the registry. 1 for purchased 0 for trial
  • Subsequent slides explain these
  • Reveal unintended information -- example Reporting invalid username separately from invalid password for that username ... duh!
  • The “good guys” can get mad/ laid off/ etc. This was the hack used in the court case I testified in last year :$$ DATA
  • Testingfor Sw Security

    1. 1. Testing for Software Security ECEN5053 Software Engineering of Distributed Systems University of Colorado, Boulder Testing for Software Security , Hebert Thompson, James Whittaker, Dr. Dobb’s Journal, November, 2002, pp. 24-34
    2. 2. When is a security bug not like a bug? <ul><li>Traditional non-security bugs -- often defined as a violation of a specification. </li></ul><ul><li>Security bugs -- additional behavior, not originally intended </li></ul><ul><ul><li>Meanwhile, it is doing what it is supposed to do </li></ul></ul><ul><ul><li>Traditional techniques not good at finding </li></ul></ul><ul><ul><li>Even in inspections, tend to look for </li></ul></ul><ul><ul><ul><li>missing behavior </li></ul></ul></ul><ul><ul><ul><li>incorrect behavior </li></ul></ul></ul><ul><ul><li>Neglect to look for ... undesirable side-effects </li></ul></ul>
    3. 3. Intended vs. Implemented Behavior Traditional faults Intended Functionality Actual Software Functionality Unintended, undocumented, unknown functionality
    4. 4. Traditional faults <ul><li>Incorrect </li></ul><ul><ul><li>Supposed to do A but did B instead </li></ul></ul><ul><li>Missing </li></ul><ul><ul><li>Supposed to do A and B but did only A. </li></ul></ul>
    5. 5. Security Bugs <ul><li>Side effects </li></ul><ul><ul><li>Supposed to do A, and it did. </li></ul></ul><ul><ul><li>In the course of doing A, it also did B </li></ul></ul><ul><li>Monitoring for side effects and their impact on security can be challenging </li></ul><ul><ul><li>Side effects can be subtle and hidden </li></ul></ul><ul><ul><li>Examples: file writes, registry entries, extra network packets with unencrypted data </li></ul></ul>
    6. 6. Tools <ul><li>Commercially available tools </li></ul><ul><ul><li>Mutek’s AppSight </li></ul></ul><ul><li>http://www.identify.com/products/appsightsuite.html </li></ul><ul><ul><li>Holodeck Lite </li></ul></ul><ul><li>http://se.fit.edu/holodeck/ </li></ul><ul><li>(freeware developed by the authors using techniques similar to those in the handout Listing One to help easily monitor and obstruct common system calls) </li></ul><ul><ul><li>Write your own customized monitoring solution </li></ul></ul>
    7. 7. Ways in / sources of interaction with the environment <ul><li>Human interface -- UI </li></ul><ul><ul><li>Set of API’s get input from kbd, mouse, etc. </li></ul></ul><ul><ul><li>Concerns: unauthorized access, privilege escalation and sabotage </li></ul></ul><ul><li>File system </li></ul><ul><ul><li>Provides data stored in binary or text format </li></ul></ul><ul><ul><li>Trusted to store sensitive data </li></ul></ul><ul><ul><li>Test how stored, retrieved, encrypted, managed </li></ul></ul><ul><li>API -- input in form of return values of API calls </li></ul><ul><li>Operating system kernel -- memory, file pointers, time and date functions, etc. </li></ul>
    8. 8. Memory is vulnerable <ul><li>Any information an app uses passes through memory eventually </li></ul><ul><ul><li>Encrypted -- usually ok </li></ul></ul><ul><ul><li>Decrypted and stored -- at risk of being read </li></ul></ul><ul><ul><li>Encryption keys, CD keys, passwords and other sensitive information are eventually used in an unencrypted form </li></ul></ul><ul><ul><ul><li>Must protect their exposure in memory </li></ul></ul></ul>
    9. 9. Stress <ul><li>Stress testing for low memory and other faulty operating conditions that may cause an application to crash </li></ul><ul><li>App’s tolerance to environmental stress can prevent </li></ul><ul><ul><li>denial of service </li></ul></ul><ul><ul><li>situations where app may crash before completing an important task like encrypting passwords </li></ul></ul><ul><ul><ul><li>Once it crashes, the state of stored data is ... ? </li></ul></ul></ul>
    10. 10. What won’t work <ul><li>Look at each method of input delivery </li></ul><ul><li>Bombard that interface with input </li></ul><ul><li>Why that won’t work well enough </li></ul><ul><li>Most revealing attacks require you to apply inputs through multiple interfaces </li></ul><ul><li>Search said these categories can be used to expose vulnerabilities </li></ul><ul><ul><li>Dependency attacks </li></ul></ul><ul><ul><li>Design-and-implementation attacks </li></ul></ul>
    11. 11. Attacking Dependencies <ul><li>Apps rely heavily on their environment to work properly </li></ul><ul><li>Not as overt as human input but there are lots of sources of input </li></ul><ul><ul><li>Like any input, if software receives a value outside of its expected range, it can fail. </li></ul></ul><ul><li>Environment failures lead to calls on error-handling code (if it exists) </li></ul><ul><ul><li>Error handlers are the security weak point of an application </li></ul></ul>
    12. 12. Testing error handlers <ul><li>Failures in the sw’s environment that exercise these code paths are difficult to produce in a test lab situation </li></ul><ul><ul><li>Tests that involve disk errors, memory failures, and network problems are only superficially explored </li></ul></ul><ul><ul><li>Illusion of security but ... </li></ul></ul><ul><ul><ul><li>Servers do run out of disk space </li></ul></ul></ul><ul><ul><ul><li>Network connectivity can be intermittent </li></ul></ul></ul><ul><ul><ul><li>File permissions can be set improperly </li></ul></ul></ul><ul><li>Need to integrate failures into tests to evaluate </li></ul>
    13. 13. Just do the impossible ... <ul><li>Create environmental failures </li></ul><ul><ul><li>Tamper with the application code </li></ul></ul><ul><ul><li>Simulate specific failing responses from the o.s. </li></ul></ul><ul><ul><li>Yeah, right </li></ul></ul><ul><ul><ul><li>Takes huge amount of time, effort, and expertise needed to simulate such a failure </li></ul></ul></ul><ul><ul><ul><li>Determine where in the code the app uses these resources </li></ul></ul></ul><ul><ul><ul><li>How to make appropriate changes to simulate a real failure </li></ul></ul></ul>
    14. 14. Alternative approach <ul><li>Run-time fault injection </li></ul><ul><ul><li>Simulating errors to the app in a black-box fashion at run time </li></ul></ul><ul><li>Advantage </li></ul><ul><ul><li>Nonintrusive </li></ul></ul><ul><ul><li>Lets you test production binaries, not contrived versions of the app that have phony return values hard-coded </li></ul></ul>
    15. 15. HOW <ul><li>Overwrite the first few bytes of the actual function to be called in the process space and insert a JMP statement to fault injection code in its place. </li></ul><ul><li>Modify import address tables (For doing this in Windows environment, authors recommend Jeffrey Richter, Programming Applications for Microsoft Windows, 4th Edition, Microsoft Press, 1999) </li></ul>
    16. 16. WHY? <ul><li>You redirect a particular system call to your own impostor function </li></ul><ul><li>Can log events </li></ul><ul><ul><li>Watch the application for file, memory, and registry activity </li></ul></ul><ul><li>In control, you can forward a system request to the actual OS function or deny the request by returning any error message you choose. </li></ul>
    17. 17. What about registry bugs? <ul><li>Security problem with the “registry” is trust </li></ul><ul><li>When developers read from the registry, assume </li></ul><ul><ul><li>values are accurate </li></ul></ul><ul><ul><li>haven’t been tampered with maliciously </li></ul></ul><ul><ul><li>especially, if their code wrote those values in the first place </li></ul></ul><ul><li>“ Try and buy” -- users have limited functionality or a time limit in which to try the software. </li></ul><ul><ul><li>App can be unlocked if purchased/registered </li></ul></ul><ul><ul><li>App may check a registry key at startup </li></ul></ul><ul><ul><ul><li>protected with weak encryption or it’s a 1/0 </li></ul></ul></ul>
    18. 18. Test app’s use of corrupted files & names <ul><li>App reads/writes hundreds of files </li></ul><ul><ul><li>Handle bad data gracefully without </li></ul></ul><ul><ul><ul><li>exposing sensitive information </li></ul></ul></ul><ul><ul><ul><li>allowing unsafe behavior </li></ul></ul></ul><ul><li>Test with the approach that can happen but is often not tested for </li></ul><ul><ul><li>Change a file in a way not covered in requirements </li></ul></ul><ul><ul><ul><li>Numerical data --> add text characters </li></ul></ul></ul><ul><ul><li>“ Successful” test results in denial of service </li></ul></ul><ul><ul><ul><li>Crashes the application or the whole system </li></ul></ul></ul><ul><ul><ul><li>May expose data during the crash </li></ul></ul></ul>
    19. 19. Test low memory/disk/network availability <ul><li>Deprive the app of resources to see robustness under stress </li></ul><ul><li>Rule of thumb -- block a resource when an app seems most in need of it </li></ul><ul><ul><li>memory when in an intense computation </li></ul></ul><ul><ul><li>disk errors -- introduce faults when doing writes/reads (modify code in Listing One to intercept appropriate system functions) </li></ul></ul>
    20. 20. Attack (Test) Design and Implementation <ul><li>There are subtle security implications made during the design phase </li></ul><ul><li>Even if design is secure, choices made during implementation can impact security </li></ul><ul><li>Tests that expose these: </li></ul><ul><ul><li>Force all error messages </li></ul></ul><ul><ul><li>Seek unprotected test APIs </li></ul></ul><ul><ul><li>Overflow input buffers </li></ul></ul><ul><ul><li>Connect to all ports </li></ul></ul>
    21. 21. Force all error messages <ul><li>Purposes </li></ul><ul><ul><li>Try values that should result in error msgs to see how many are handled properly </li></ul></ul><ul><ul><li>Make sure error msgs do not reveal unintended information </li></ul></ul>
    22. 22. Seek unprotected APIs <ul><li>Difficult to test complex apps by relying on the APIs extended for normal users alone. </li></ul><ul><ul><li>Many designers therefore include additional hooks (extended but not published API) </li></ul></ul><ul><ul><ul><li>These often bypass normal security because they are intended for use by the good guys </li></ul></ul></ul><ul><ul><ul><li>Intended for custom test harnesses </li></ul></ul></ul><ul><ul><ul><li>They mean to remove them ... but then ... they’ll be handy for the next release </li></ul></ul></ul><ul><ul><ul><li>So integrated into the code and testing process, removal may destabilize the code </li></ul></ul></ul><ul><li>If purposely left in, verify they are not trouble if found </li></ul>
    23. 23. Overflow input buffers <ul><li>Test ability to handle long strings in input fields </li></ul><ul><ul><li>Especially where long strings are entered into fields that have an assumed, but not enforced, length, e.g. ZIP codes, state names </li></ul></ul><ul><li>API calls notorious for unconstrained inputs </li></ul><ul><ul><li>GUI can filter inputs as they are entered </li></ul></ul><ul><ul><li>API parameters dealt with internally </li></ul></ul><ul><ul><ul><li>checks to ensure values are appropriate before they are used </li></ul></ul></ul><ul><ul><li>Most vulnerable -- seldom used or supporting legacy functionality </li></ul></ul>
    24. 24. Connect to all ports <ul><li>Sometimes apps open custom ports to connect with remote servers </li></ul><ul><ul><li>Create maintenance channels to automatic updates </li></ul></ul><ul><ul><li>Relic from test automation </li></ul></ul><ul><ul><li>http://www.ntbugtraq.com/ for many cases where these are left open and unsecured </li></ul></ul><ul><li>If this is done in the app </li></ul><ul><ul><li>Test the port in the release version </li></ul></ul><ul><ul><li>Check what kind of data flows through it </li></ul></ul>
    25. 25. Conclusion <ul><li>These attacks by testers can expose vulnerabilities before release </li></ul><ul><li>Only part of complete security-testing methods </li></ul><ul><li>This area of research is in the infant stages </li></ul>