Software Security Testing


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Software Security Testing

  1. 1. Building Security In Editor: Gary McGraw, Software Security Testing S ecurity testing has recently moved beyond the be easier to exploit. Because attacks are now becoming more sophisti- realm of network port scanning to include probing cated, the notion of which vulnera- bilities actually matter is changing. software behavior as a critical aspect of system be- Although timing attacks, including the well-known race condition, havior (see the sidebar). Unfortunately, testing were considered exotic just a few years ago, they’re common now. software security is a commonly misunderstood task. Security Similarly, two-stage buffer overflow attacks using trampolines were once BRUCE POTTER testing done properly goes deeper Security is always relative to the the domain of software scientists, but Booz Allen than simple black-box probing on the information and services being pro- now appear in 0day exploits.4 Hamilton presentation layer (the sort performed tected, the skills and resources of ad- Design-level vulnerabilities are by so-called application security versaries, and the costs of potential the hardest defect category to han- GARY tools)—and even beyond the func- assurance remedies; security is an ex- dle, but they’re also the most preva- MCG RAW tional testing of security apparatus. ercise in risk management. Risk lent and critical. Unfortunately, as- Cigital Testers must use a risk-based ap- analysis, especially at the design level, certaining whether a program has proach, grounded in both the sys- can help us identify potential secu- design-level vulnerabilities requires tem’s architectural reality and the at- rity problems and their impact.1 great expertise, which makes finding tacker’s mindset, to gauge software Once identified and ranked, soft- such flaws not only difficult, but par- security adequately. By identifying ware risks can then help guide soft- ticularly hard to automate. risks in the system and creating tests ware security testing. Examples of design-level prob- driven by those risks, a software se- A vulnerability is an error that an lems include error handling in ob- curity tester can properly focus on attacker can exploit. Many types of ject-oriented systems, object sharing areas of code in which an attack is vulnerabilities exist, and computer and trust issues, unprotected data likely to succeed. This approach pro- security researchers have created tax- channels (both internal and exter- vides a higher level of software secu- onomies of them.2 Security vulnera- nal), incorrect or missing access con- rity assurance than possible with bilities in software systems range trol mechanisms, lack of auditing/ classical black-box testing. from local implementation errors logging or incorrect logging, and or- (such as use of the gets() function dering and timing errors (especially What’s so different call in C/C++), through interpro- in multithreaded systems). These about security? cedural interface errors (such as a sorts of flaws almost always lead to Software security is about making race condition between an access security risk. software behave in the presence of a control check and a file operation), malicious attack, even though in the to much higher design-level mis- Risk management real world, software failures usually takes (such as error handling and re- and security testing happen spontaneously—that is, covery systems that fail in an insecure Software security practitioners per- without intentional mischief. Not fashion or object-sharing systems form many different tasks to manage surprisingly, standard software test- that mistakenly include transitive software security risks, including ing literature is only concerned with trust issues). Vulnerabilities typically what happens when software fails, fall into two categories—bugs at the • creating security abuse/misuse regardless of intent. The difference implementation level and flaws at cases; between software safety and software the design level.3 • listing normative security re- security is therefore the presence of Attackers generally don’t care quirements; an intelligent adversary bent on whether a vulnerability is due to a • performing architectural risk breaking the system. flaw or a bug, although bugs tend to analysis; 32 PUBLISHED BY THE IEEE COMPUTER SOCIETY ■ 1540-7993/04/$20.00 © 2004 IEEE ■ IEEE SECURITY & PRIVACY
  2. 2. Building Security In • building risk-based security test Security External Static Penetration plans; requirements review analysis testing • wielding static analysis tools; (tools) Abuse Risk Risk-based Risk • performing security tests; cases analysis security tests analysis Security • performing penetration testing in breaks the final environment; and • cleaning up after security breaches. Three of these are particularly Requirements Design Test Code Test Field and use cases plans results feedback closely linked—architectural risk analysis, risk-based security test plan- Figure 1. The software development life cycle. Throughout this series, we’ll focus on ning, and security testing—because a specific parts of the cycle; here, we’re examining risk-based security testing. critical aspect of security testing relies on probing security risks. Last issue’s installment1 explained how to ap- problem. Software testers charged rity and Exploiting Software help edu- proach a software security risk analy- with security testing often fall prey to cate testing professionals on how to sis, the end product being a set of se- the same thinking. think like an attacker.4,5 Nevertheless, curity-related risks ranked by business software exploits are surprisingly so- or mission impact. (Figure 1 shows How to approach phisticated these days, and the level of where we are in our series of articles security testing discourse found in books and articles about software security’s place in the Like any other form of testing, secu- is only now coming into alignment. software development life cycle.) rity testing involves determining White- and black-box testing and The pithy aphorism, “software se- who should do it and what activities analysis methods both attempt to un- curity is not security software” pro- they should undertake. derstand software, but they use differ- vides an important motivator for secu- ent approaches depending on rity testing. Although security features Who whether the analyst or tester has ac- such as cryptography, strong authenti- Because security testing involves two cess to source code. White-box cation, and access control play a criti- approaches, the question of who analysis involves analyzing and under- cal role in software security, security it- should do it has two answers. Stan- standing source code and the design. self is an emergent property of the dard testing organizations using a It’s typically very effective in finding entire system, not just the security traditional approach can perform programming errors (bugs when au- mechanisms and features. A buffer functional security testing. For ex- tomatically scanning code and flaws overflow is a security problem regard- ample, ensuring that access control when doing risk analysis); in some less of whether it exists in a security mechanisms work as advertised is a cases, this approach amounts to pat- feature or in the noncritical GUI. classic functional testing exercise. tern matching and can even be auto- Thus, security testing must necessarily On the other hand, traditional mated with a static analyzer (the sub- involve two diverse approaches: QA staff will have more difficulty ject of a future installment of this performing risk-based security test- department). One drawback to this 1. testing security mechanisms to ing. The problem is one of expertise. kind of testing is that it might report a ensure that their functionality is First, security tests (especially those potential vulnerability where none properly implemented, and resulting in complete exploit) are actually exists (a false positive). Nev- 2. performing risk-based security difficult to craft because the designer ertheless, using static analysis methods testing motivated by under- must think like an attacker. Second, on source code is a good technique standing and simulating the at- security tests don’t often cause direct for analyzing certain kinds of soft- tacker’s approach. security exploit and thus present an ware. Similarly, risk analysis is a white- observability problem. A security box approach based on a deep under- Many developers erroneously test could result in an unanticipated standing of software architecture. believe that security involves only outcome that requires the tester to Black-box analysis refers to ana- the addition and use of various secu- perform further sophisticated analy- lyzing a running program by probing rity features, which leads to the in- sis. Bottom line: risk-based security it with various inputs. This kind of correct belief that “adding SSL” is testing relies more on expertise and testing requires only a running pro- tantamount to securing an applica- experience than we would like. gram and doesn’t use source-code tion. Software security practitioners analysis of any kind. In the security bemoan the over-reliance on “magic How paradigm, malicious input can be crypto fairy dust” as a reaction to this Books like How to Break Software Secu- supplied to the program in an effort ■ IEEE SECURITY & PRIVACY 33
  3. 3. Building Security In cards with new functionality—such From outside→in to inside→out as the ability to provide secure card- holder identification or remember personal preferences—many credit- T raditional approaches to computer and network security testing focus on network infra- structure, firewalls, and port scanning. The notion is to protect vulnerable systems (and software) from attack by identifying and defending a perimeter. In this paradigm, testing card companies are turning to multi- application smart cards. These cards focuses on an outside→in approach. One classic example is the use of port scanning with tools use resident software applications to such as nmap <> to probe network ports and see process and store thousands of times what service is listening. Figure A shows a classic outside→in paradigm focusing on firewall more information than traditional placement. magnetic-stripe cards. By contrast, we advocate an inside→out approach to security, whereby software inside the Security and fraud issues are criti- LAN (and exposed on LAN boundaries) is itself subjected to rigorous risk management and cal concerns for the financial institu- security testing. tions and merchants spearheading smart-card adoption. By developing and deploying smart-card technol- Internet ogy, credit-card companies provide Public important new tools in the effort to network lower fraud and abuse. For instance, smart cards typically use a sophisti- Hardware, firewall, cated crypto system to authenticate usually part of a TCP/IP router transactions and verify the identities of the cardholder and issuing bank. However, protecting against fraud and maintaining security and privacy are both very complex problems be- cause of the rapidly evolving nature Secure private network of smart-card technology. Portable local area network Public network The security community has been involved in security risk analy- Figure A: The outside→in approach. A firewall protects a LAN by blocking various sis and mitigation for Open Platform network traffic on its way in; outside→in security testing involves probing the LAN (now known as Global Platform, or with a port scanner to see which ports are “open” and what services are listening GP) and Java Card since early 1997. on those ports. A major security risk associated with this approach is that the Because product security is an essen- services traditionally still available through the firewall are implemented with tial aspect of credit-card companies’ insecure software. brand protection regimen, these companies spend plenty of time and effort on security testing. One cen- tral finding emphasizes the impor- to break it: if the program breaks spend very little time understanding tance of testing particular vendor during a particular test, then we or probing nonfunctional security implementations according to our might have discovered a security risks. Exacerbating the problem, the two testing categories: adherence to problem. Black-box testing is possi- QA process is often broken in many functional security design and ble even without access to binary commercial software houses due to proper behavior under particular at- code—that is, a program can be time and budget constraints and the tacks motivated by security risks. tested remotely over a network. If belief that QA is not an essential part The latter category, risk-based the tester can supply the proper input of software development. security testing (linked directly to (and observe the test’s effect), then risk analysis findings) ensures that black-box testing is possible. An example: Java cards can perform securely in the Any testing method can reveal Card security testing field even when under attack. Risk possible software risks and potential Doing effective security testing re- analysis results can be used to guide exploits. One problem with almost quires experience and knowledge. manual security testing. As an exam- all kinds of security testing (regard- Examples and case studies like the ple, consider the risk that, as de- less of whether it’s black or white one we present now are thus useful signed, the object-sharing mecha- box) is the lack of it—most QA or- tools for understanding the approach. nism in Java Card is complex and ganizations focus on features and In an effort to enhance payment thus is likely to suffer from security- 34 IEEE SECURITY & PRIVACY ■ SEPTEMBER/OCTOBER 2004
  4. 4. Building Security In critical implementation errors on functional requirements is an ab- uncovered under normal functional any given card. Testing for this sort of solute necessity for customers to ac- security testing. Fielding cards with risk involves creating and manipulat- cept the cards. After all, they must these vulnerabilities would let an at- ing stored objects where sharing is perform properly worldwide. tacker execute successful attacks on involved. Given a technical descrip- However, every card submitted to live cards issued to the public. Be- tion of this risk, building specific the risk-based testing paradigm ex- cause of proper risk-based security probing tests is possible. hibited some manner of failure when testing, the vendors were notified of tested with the hostile applet suite. the problems and corrected the code Automating Some failures pointed directly to crit- responsible before release. security testing ical security vulnerabilities on the Over the years, we (the authors) have card; others were less specific and re- been involved in several projects that quired further exploration to deter- have identified architectural risks in the GP/Java Card platform, sug- mine the card’s true security posture. As an example, consider that risk T here is no silver bullet for soft- ware security; even a reasonable security testing regimen is just a start. gested several design improvements, analysis of Java Card’s design docu- Unfortunately security continues to and designed and built automated ments indicates that proper imple- be sold as a product, and most defen- security tests for final products (each mentation of atomic transaction sive mechanisms on the market do of which had multiple vendors). processing is critical for maintaining little to address the heart of the prob- Several years ago, Cigital began a secure card. Java Card has the capa- lem, which is bad software. Instead, developing an automated security bility of defining transaction bound- they operate in a reactive mode: test framework for GP cards built on aries to ensure that if a transaction don’t allow packets to this or that Java Card 2.1.1 and based on exten- fails, data roll back to a pre-transac- port, watch out for files that include sive risk analysis results. The end re- tion state. In the event that transac- this pattern in them, throw partial sult is a sophisticated test framework tion processing fails, transactions can packets and oversized packets away that runs with minimal human inter- go into any number of possible states, without looking at them. Network vention and results in a qualitative se- depending on what the applet was traffic is not the best way to approach curity testing analysis of a sample attempting. In the case of a stored- this predicament, because the soft- smart card. value card, bad transaction process- ware that processes the packets is the The first test set, the functional ing could allow an attacker to “print problem. By using a risk-based ap- security test suite, directly probes money” by forcing the card to roll proach to software security testing, low-level card security functionality. back value counters while actually testing professionals can help solve It includes automated testing of class purchasing goods or services. security problems while software is codes, available commands, and When creating risk-based tests to still in production. crypto functionality. This test suite probe transaction processing, we di- also actively probes for inappropriate rectly exercised transaction-process- References card behavior of the sort that can lead ing error handling by simulating an 1. D. Verndon and G. McGraw, “Risk to security compromise. attacker attempting to violate a Analysis in Software Design,” IEEE The second test set, the hostile transaction—specifically, transac- Security & Privacy, vol. 2, no. 4, applet test suite, is a sophisticated tions were aborted or never commit- 2004, pp. 79–84. set of intentionally hostile Java ted, transaction buffers were com- 2. C.E. Landwehr et al., A Taxonomy Card applets designed to probe pletely filled, and transactions were of Computer Program Security Flaws, high-risk aspects of the GP on a nested (a no-no according to the Java with Examples, tech. report NRL/ Java Card implementation. Card specification). These tests were FR/5542—93/9591, Naval not based strictly on the card’s func- Research Laboratory, Nov. 1993. Results: tionality—instead, security test en- 3. G. McGraw, “Software Security,” Nonfunctional security gineers intentionally created them, IEEE Security & Privacy, vol. 2, no. testing is essential thinking like an attacker given the 2, 2004, pp. 80–83. Most cards tested with the automated results of a risk analysis. 4. G. Hoglund and G. McGraw, test framework pass all functional se- Several real-world cards failed Exploiting Software, Addison-Wes- curity tests, which we expect because subsets of the transaction tests. The ley, 2004. smart-card vendors are diligent with vulnerabilities discovered as a result 5. J. Whittaker and H. Thompson, functional testing (including security of these tests would let an attacker How to Break Software Security, functionality). Because smart cards terminate a transaction in a poten- Addison-Wesley, 2003. are complex embedded devices, ven- tially advantageous manner, a critical dors realize that exactly meeting test failure that wouldn’t have been Bruce Potter is a senior associate with ■ IEEE SECURITY & PRIVACY 35
  5. 5. Building Security In Booz Allen Hamilton. He is also the founder of the Shmoo Group of security professionals. His areas of expertise include wireless security, large-scale net- work architectures, smartcards, and pro- motion of secure software engineering practices. Potter coauthored the books 802.11 Security (O’Reilly and Associates, 2003) and Mac OS X Security (New Rid- ers, 2003); he’s currently coauthoring Master FreeBSD and OpenBSD Security (O’Reilly and Associates, summer 2004). He was trained in computer science at the University of Alaska, Fairbanks. Gary McGraw is chief technology officer of Cigital. His real-world experience is grounded in years of consulting with major corporations and software pro- ducers. He serves on the technical advi- sory boards of Authentica, Counterpane, Fortify, and Indigo. He also is coauthor of Exploiting Software (Addison-Wesley, 2004), Building Secure Software (Addi- son-Wesley, 2001), Java Security (John Wiley & Sons, 1996), and four other books. He has a BA in philosophy from the University of Virginia and a dual PhD in computer science and cognitive science from Indiana University. Contact him at 36 IEEE SECURITY & PRIVACY ■ SEPTEMBER/OCTOBER 2004