nullcon 2010 - Penetration Testing versus Source Code Review

Penetration Testing versus Source Code Review 1. Penetration Testing vs. Source Code Review Within the domain of application security assessments, there are multiple approaches that can be considered. The most common ones are black-box penetration testing, application design review and source code reviews. Of course, more arcane ones such as reverse engineering or binary analysis can also be considered in specific situations. But for the moment, let’s focus on the two most common approaches to application security assessments – penetration testing and source code review. Currently, there is a debate raging within the information security community as to which approach is better, with some participants going so far as to say that penetration testing is dead! (ref). With the experience of having performed hundreds of penetration tests and dozens of source code reviews, I have come to the conclusion that each technique has its own place. It is also extremely worthwhile for the developer and security communities to be aware of the advantages and disadvantages of both the techniques. In fact, in my judgment a well-timed combination of both approaches can be extremely beneficial in identifying and fixing security issues at the earliest. Let’s look at some of the issues that we have discovered in penetration tests, which would be near to impossible to discover in a source code review. Penetration Testing Risk-based testing Nowadays, even penetration tests are becoming risk-based. This means, that we spend time learning about the business of the client and the risks to the business through the Internet based infrastructure – primarily web applications – in order to focus our penetration testing work in line with these risks. A source code review can never simulate the penetration testing approach, which requires building business-specific test cases, and then testing for those. For example, during the penetration test of a local search engine website, the business risk is not so much from a cross-site scripting, as it would be from someone running repeated search requests, downloading the HTML results, and then parsing the results to recreate the database – essentially an online data mining attack. Similarly, while pen-testing an ERP system, the emphasis would not be so much on unpatched vulnerabilities, as it would be on executing attacks that would simulate a financial fraud. For instance, can we violate the existing access controls to not only accept vendor quotations, but also to approve a selected vendor, under the guise of Confidential Page 1

Penetration Testing versus Source Code Review another user? So privilege escalation attacks would be of far greater consequence than regular SQL injection and cross-site scripting attacks. Search engine indexing faulty During a recent penetration test, we found a search functionality on the web application, and upon entering interesting words such as “password” and “administrator”, some very interesting documents were revealed. Upon further searching, we discovered that the terms “.jpg”, and “.bmp” revealed scanned copies of customers’ personal information such as driving licenses and passport copies. Again, it is difficult to imagine a source code review discovering these sorts of issues. Weak Authentication Mechanisms While static source code analysis tools would be able to discover weak authentication schemes (pages, which allow access without authentication), specific issues such as weak CAPTCHA implementations, or if the web application allows short and weak passwords to be selected, could be obtained from a penetration test more easily than from an automated source code review. A manual source code review would possibly be able to discover these flaws. Similarly, the ability to enumerate users due to different error messages (one when the username is incorrect, and another one when the password is incorrect), is also easily discovered in a penetration test. In another instance, we discovered that the credit card numbers were being shown up in plaintext. An automated source code review failed to show up these issues. {Other points yet to work on: Configuration issues with the web server and the operating system Manipulation of business rules, such as for web-based ERPs in terms of transaction processing controls being violated Source code not being available, of course means penetration testing is the only option Where it is a regulatory requirement – such as for Internet Banking} Source Code Reviews Code review is systematic examination of computer source code intended to find and fix mistakes and security vulnerabilities overlooked in the initial development phase, improving both the overall quality of software and the developers' skills. Largely source code developers encounter two typical errors/mistakes/issues/hurdles: Confidential Page 2

Penetration Testing versus Source Code Review Implementation errors: These quality-style defects in code are fairly atomic, and typically stand alone when identified, and remediation is applied. They are caused by bad or "loose" programming practices. Examples include buffer overflows, which result from mismanagement of memory, and race conditions, which result from call timing mismatches. Design errors: These include the failure to utilize or adequately implement security related functions. This includes authentication, encryption, and the use of insecure external code types, and validation of data input and application output. Source code reviews can be done in couple of ways – manual or automated. Even manual reviews cannot look through each line of code, but are supposed to identify key functions and then review those. An automated source code review looks through the call-tree using proprietary algorithms, and offers the following distinct advantages over a penetration test: Unlike PT, it’s more of a white box approach to testing the application. This gives the in-sight of the application security issues. Identify weak encryption algorithms or weak implementation of strong encryption algorithms, such as by choosing a shorter key length or insecure storage of the key Almost the complete set of cross-site scripting and SQL injection possible vulnerabilities. A penetration test in contrast ends up finding fewer of these, as some of them are missed out by both manual as well as automated pen-testing methods Getting down to the root-function that is vulnerable. A pen-test will typically identify the web page or form that is vulnerable, but a source code review will drill down to the common function or module, which when fixed, would end up resolving multiple security issues, thus providing for a tremendous savings in time and cost With source code reviews being embedded into IDEs, developers can identify and fix security issues almost immediately. This tremendously reduces the cost of fixing application security issues. SCR can also check whether secure coding practices and other development practices have been followed or not It could happen that in black box PT, the pen tester might miss out certain security vulnerabilities in the application. This possibility is very less in SCR, since the whole source is in-front. {Other points yet to be added here in support of source code reviews} Confidential Page 3

nullcon 2010 - Penetration Testing versus Source Code Review

by n|u - The Open Security Community on Apr 02, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 64

Statistics

nullcon 2010 - Penetration Testing versus Source Code Review — Document Transcript