Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Software Security Testing:

1,058 views

Published on

  • Be the first to comment

  • Be the first to like this

Software Security Testing:

  1. 1. Software Security Testing: Seeking security in an insecure world Gary McGraw, Ph.D. CTO, Cigital http://www.cigital.com © 2009 Cigital
  2. 2. Software security is getting harder The Trinity of Trouble   Connectivity   The Internet is everywhere The network is and most software is on it the computer.
   Complexity   Networked, distributed, mobile code is hard   Extensibility   Systems evolve in unexpected ways and are changed on the fly …is this complex program .NET
 This simple interface… © 2009 Cigital
  3. 3. Old school security is reactive   Defend the “perimeter” with a firewall   To keep stuff out   Promulgate “penetrate and patch”   “Review” products when they’re complete   Throw it over the wall testing   Too much weight on penetration testing The “network guy with keys” does not really understand   Over-rely on security functions software testing. Builders are   “We use SSL” only recently getting involved in security. © 2009 Cigital
  4. 4. Making software behave is hard   Can you test in quality?   How do you find (adaptive) defects in code?   What about bad guys doing evil on purpose?   What’s the difference between security testing and functional testing?   How can you analyze security design?   How can you codify non-functional, emergent requirements like security?   Can you measure security? © 2009 Cigital
  5. 5. Software vulnerability growth Software Vulnerabilities 10000 9000 8064 8000 7236 7000 5690 6000 5000 4129 3784 3780 4000 3000 2437 2000 1090 1000 0 2000 2001 2002 2003 2004 2005 2006 2007 © 2009 Cigital
  6. 6. The classic security tradeoff Windows Complexity 45 40 35 Millions of Lines 30 Software Vulnerabilities 25 20 10000 15 9000 8064 10 5 8000 0 7000 5690 Win Win Win 95 NT 4.0 Win 98 NT 5.0 Win XP 6000 3.1 NT (1997) (1998) (1999) (2000) 2K (2002) 5000 4129 3784 3780 (1990) (1995) (2001) 4000 3000 2437 2000 1090 1000 0 2000 2001 2002 2003 2004 2005 2006 © 2009 Cigital
  7. 7. Security problems are complicated IMPLEMENTATION BUGS ARCHITECTURAL FLAWS   Buffer overflow   Misuse of cryptography 50% 50%   String format   Compartmentalization   One-stage attacks problems in design   Race conditions   Privileged block protection   TOCTOU (time of check to failure (DoPrivilege()) time of use)   Catastrophic security failure (fragility)   Unsafe environment variables   Unsafe system calls   Type safety confusion error   System()   Insecure auditing   Untrusted input problems   Broken or illogical access control (RBAC over tiers)   Method over-riding problems (subclass issues)   Signing too much code © 2009 Cigital
  8. 8. Security = breaking stuff + building stuff   Security requires two hats   Offense and defense   Building and breaking   Security design based on software engineering   Security analysis based on attack   Testing has two flavors   Functional security testing (constructive)   Risk-based security testing (destructive) © 2009 Cigital
  9. 9. A Software Security Framework   Four domains   Twelve practices   See informIT article   http://www.informit.com/articles/article.aspx?p=1271382 © 2009 Cigital
  10. 10. Security as Knowledge Intensive © 2009 Cigital
  11. 11. Knowledge: 48 Attack Patterns   Make the Client Invisible   User-Controlled Filename   Target Programs That Write to Privileged OS Resources   Passing Local Filenames to Functions That Expect a   Use a User-Supplied Configuration File to Run URL Commands That Elevate Privilege   Meta-characters in E-mail Header   Make Use of Configuration File Search Paths   File System Function Injection, Content Based   Direct Access to Executable Files   Client-side Injection, Buffer Overflow   Embedding Scripts within Scripts   Cause Web Server Misclassification   Leverage Executable Code in Nonexecutable Files   Alternate Encoding the Leading Ghost Characters   Argument Injection   Using Slashes in Alternate Encoding   Command Delimiters   Using Escaped Slashes in Alternate Encoding   Multiple Parsers and Double Escapes   Unicode Encoding   User-Supplied Variable Passed to File System Calls   UTF-8 Encoding   Postfix NULL Terminator   URL Encoding   Postfix, Null Terminate, and Backslash   Alternative IP Addresses   Relative Path Traversal   Slashes and URL Encoding Combined   Client-Controlled Environment Variables   Web Logs   User-Supplied Global Variables (DEBUG=1, PHP   Overflow Binary Resource File Globals, and So Forth)   Overflow Variables and Tags   Session ID, Resource ID, and Blind Trust   Overflow Symbolic Links   Analog In-Band Switching Signals (aka “Blue Boxing”)   MIME Conversion   Attack Pattern Fragment: Manipulating Terminal Devices   HTTP Cookies   Simple Script Injection   Filter Failure through Buffer Overflow   Embedding Script in Nonscript Elements   Buffer Overflow with Environment Variables   XSS in HTTP Headers   Buffer Overflow in an API Call   HTTP Query Strings   Buffer Overflow in Local Command-Line Utilities   Parameter Expansion   String Format Overflow in syslog() © 2009 Cigital
  12. 12. Attack pattern 1: Make the client invisible   Remove the client from the communications loop and talk directly to the server   Leverage incorrect trust model (never trust the client)   Example: hacking browsers that lie (opera cookie foo) © 2009 Cigital
  13. 13. Attacker’s toolkit: buffer overflow foo   Find targets with static analysis   Trampolining past a canary   Change program control flow   Heap attacks Function arguments   Stack smashing Return Address   Trampolining Canary Value   Arc injection   Particular examples Frame Pointer   Overflow binary resource Local Variable: Buffer A files (used against Netscape) Local Variable: Pointer A   Overflow variables and tags (Yamaha MidiPlug) Local Variable: Buffer B   MIME conversion fun (Sendmail)   HTTP cookies (apache) © 2009 Cigital
  14. 14. Warning! Knowledge can be easily misused Software security “Application security”   Requires input into design   Works for COTS software and implementation   Low expertise   High expertise   Protect installed software   Design software to be from harm secure   Protection against malicious   Build secure code code   Security analysis   Policy issues   Security testing   Outside  In   Inside  Out badness-ometer © 2009 Cigital
  15. 15. Top 11 reasons why top 10 lists don’t work 1.  Executives don’t care about 6.  Bug lists change with the technical bugs prevailing technology winds 2.  Too much focus on bugs 7.  Top ten lists mix levels 3.  Vulnerability lists help 8.  Automated tools can find auditors more than bugs---let them developers 9.  Metrics built on top ten lists are 4.  One person’s bug is another misleading person’s yawner 10.  When it comes to testing, 5.  Using bug parade lists for security requirements are more training leads to awareness important than vulnerability but does not educate. lists. 11.  Ten is not enough. http://www.informit.com/articles/article.aspx?p=1322398 © 2009 Cigital
  16. 16. BSIMM-Ten surprising things 1.  Bad metrics hurt 6.  ARA is hard 2.  Secure-by default 7.  Practitioners don’t frameworks talk attacks 3.  Nobody uses 8.  Training is WAFs advanced 4.  QA can’t do 9.  Pen testing is software security diminishing 5.  Evangelize over 10.  Fuzz testing audit   http://www.informit.com/articles/article.aspx?p=1315431 © 2009 Cigital
  17. 17. Attackers are Software People © 2009 Cigital
  18. 18. Attackers do not distinguish bugs and flaws   Both bugs and flaws lead to vulnerabilities that can be exploited   Attackers write code to break code   Defenders are network operations people   Code?! What code? © 2009 Cigital
  19. 19. The attacker’s toolkit   The standard attacker’s toolkit has lots of (software analysis) stuff   Disassemblers and decompilers   Binary scanners   Control flow, data flow, and coverage tools   APISPY32   Breakpoint setters and monitors   Buffer overflow kits   Shell code, payloads (multi-platform)   Rootkits (kernel, hardware) © 2009 Cigital
  20. 20. Attacker’s toolkit: other miscellaneous tools   Debuggers (user-mode)   Kernel debuggers   SoftIce   Fault injection tools   FUZZ   Failure simulation tool   Hailstorm   Holodeck   Boron tagging   The “depends” tool   Grammar rewriters © 2009 Cigital
  21. 21. Is FUZZ good?   Wisconsin academics invent   FUZZ can be useful the notion of sending   SPIKE random noise to UNIX   Peachfuzz utilities   Mangle (HTML)   Fuzz I: 1990   FileFuzz   Fuzz II: ten years later   beStrorm   Fifteen years later, security   Codenomicon people hit on the same idea   White box testing is better © 2009 Cigital
  22. 22. Breaking stuff is important   Learning how to think like an attacker is essential (especially for good testing)   Think hard about the “can’ts” and “won’ts”   Do not shy away from teaching attacks   Engineers learn from stories of failure   Testers must deeply understand how things break © 2009 Cigital
  23. 23. Resources on security testing   Building Secure Software (Viega/McGraw)   Writing Secure Code (Howard/ LeBlanc)   How to Break Software Security (Whittaker/Thompson)   Web Security Testing Cookbook (Hope/Walther) © 2009 Cigital
  24. 24. Web Security Testing Cookbook © 2009 Cigital
  25. 25. What it ain’t © 2009 Cigital
  26. 26. What it IS   Reference and job aid for web testers   Exploratory testing   Regression testing   Automated testing   Unit testing   Starts really basic   Ends rather complicated © 2009 Cigital
  27. 27. Table of Contents 1.  Introduction 2.  Installing Free Tools 3.  Basic Observation 4.  Web-Oriented Data Encoding 5.  Tampering with Input 6.  Automated Bulk Scanning 7.  Automating Tasks with cURL 8.  Automating Tasks with LibWWWPerl 9.  Seeking Design Flaws 10.  Attacking AJAX 11.  Manipulating Sessions 12.  Multifaceted Tests © 2009 Cigital
  28. 28. Example: Login to eBay ${CURL} -s -L -A "${UA}" -c "${JAR}"   Series of -b "${JAR}" -e ";auto” -d MfcISAPICommand=SignInWelcome commands -d siteid=0 -d co_partnerId=2 -d UsingSSL=1 -d ru= -d pp= -d pa1= -d pa2= -d pa3=   Build up state/ -d i1=-1 -d pageType=-1 -d rtmData= -d userid="${USER}" -d pass="${PASS}" cookies -o "step-${step}.html" "https://signin.ebay.com/ws/..."   Check for if [ $? = 0 ]; then username in output step=$step+1 echo -n "OK] [${step} " else echo "FAIL]" exit 1 fi © 2009 Cigital
  29. 29. Other topics   LDAP injection   All of these are   Zip of death scripted   Billion laughs   All are repeatable   Pathological XML   All can be part of a   Malicious cookies routine QA process © 2009 Cigital
  30. 30. Stuff that Works for Cigital © 2009 Cigital
  31. 31. Three pillars of software security   Risk management framework   Touchpoints   Knowledge © 2009 Cigital
  32. 32. Software security touchpoints © 2009 Cigital
  33. 33. Touchpoint: Abuse cases   Use cases formalize normative behavior (and assume correct usage)   Describing non-normative behavior is a good idea   Prepare for abnormal behavior (attack)   Misuse or abuse cases do this   Uncover exceptional cases   Leverage the fact that designers know more about their system than potential attackers do   Document explicitly what the software will do in the face of illegitimate use   Abuse cases are great for test planning © 2009 Cigital
  34. 34. Touchpoint: Security testing   Test security functionality   Cover non-functional requirements   Security software probing   Risk-based testing   Use architectural risk analysis results to drive scenario- based testing   Concentrate on what “you can’t do”   Think like an attacker   Informed red teaming © 2009 Cigital
  35. 35. Touchpoint: Risk-based testing   Identify areas of potential risk in the system   Requirements   Design   Architecture   Use abuse cases to drive testing according to risk   Build attack and exploit scenarios based on identified risks   Test risk conditions explicitly   Example: Overly complex object-sharing system in Java Card © 2009 Cigital
  36. 36. Touchpoint: Penetration testing   A very good idea since software is bound in an environment   How does the complete system work in practice?   Interaction with network security mechanisms   Firewalls   Applied cryptography   Penetration testing should be driven by risks uncovered throughout the lifecycle   Not a silver bullet! © 2009 Cigital
  37. 37. Always: External review   Having outside eyes look at   Red teaming is a weak form your system is essential of external review   Designers and   Penetration testing is too developers naturally get often driven by blinders on outsidein perspective   External just means outside of the project   External review must   This is knowledge include architecture intensive analysis   Outside eyes make it easier   Security expertise and to “assume nothing” experience really helps   Find assumptions, make them go away © 2009 Cigital
  38. 38. Where to Learn More © 2009 Cigital
  39. 39. informIT & Justice League   www.cigital.com/justiceleague   In-depth thought leadership blog from the Cigital Principals   Scott Matsumoto   www.informIT.com   Gary McGraw   No-nonsense monthly security   Sammy Migues column by Gary McGraw   Craig Miller   John Steven © 2009 Cigital
  40. 40. IEEE Security & Privacy Magazine + 2 Podcasts   Building Security In   Software Security Best Practices column edited by John Steven   www.computer.org/security/bsisub/   www.cigital.com/silverbullet   www.cigital.com/realitycheck © 2009 Cigital
  41. 41. Software Security: the book   How to DO software security   Best practices   Tools   Knowledge   Cornerstone of the Addison- Wesley Software Security Series   www.swsec.com © 2009 Cigital
  42. 42. For more   Cigital’s Software Security Group invents and delivers Software Quality Management   WE NEED GREAT PEOPLE   See the Addison-Wesley Software Security series   Send e-mail: gem@cigital.com “So now, when we face a choice between adding features and resolving security issues, we need to choose security.” -Bill Gates © 2009 Cigital

×