BSidesLondon 20th April 2011 - Rory Mccune (@raesene) ----------- "Penetration testing" has become a staple of a the security programmes of a lot of companies around the world and particularly in the UK. Unfortunately in most cases it's poorly understood, the value for customers is minimal and it bears absolutely no resemblence to what a modern attacker would do. So it's time for it to die. ------ for more info about Rory Mccune go to www.7elements.co.uk

1. PENETRATION TESTING MUST DIE
Rory McCune

2. INTRODUCTION
• IT/Information Security person for the last 15 years
• Currently a director of 7 Elements ( http://www.7elements.co.uk )
• IT Security consultancy based in Scotland.
• Scottish chapter lead for the OWASP project.

3. AGENDA
• Why must it die
• Overloaded Terminology.
• Clients aren’t ready.
• Mission Impossible.
• How do you fix it?
• Better Terms.
• Better Scoping.

4. WHAT’S IN A NAME?
• First thing is to define what we mean by penetration testing
• One definition from Wikipedia “A penetration test, occasionally pentest, is a
method of evaluating the security of a computer system or network by
simulating an attack from a malicious source, known as a Black Hat Hacker, or
Cracker.”

5. CHARACTERISTICS OF A PENETRATION TEST
• Black-Box
• When we’re assuming the role of an attacker (unless it’s an insider) the
testing should be black box
• Goal based
• Trying to compromise the target system/network/company
• Trade off against coverage of every possible avenue
• Realistic
• Mimicking the “real thing”
• Although…. Which Real thing?

6. OVERLOADED TERMINOLOGY
• Like many things in security the term “penetration test” is overloaded
• Vulnerability Scans as Penetration Tests
• Web Application Security Assessments as Penetration Tests
• Code Reviews as Penetration Tests
• …

7. CLIENTS AREN’T READY
• What’s the purpose of a penetration test?
• From above it’s to mimic an attack
• Trade off realism against coverage
• Test controls that should be in place
• Implies that clients are ready for that
• Know what controls should be in place
• Think that they’re operating effectively
• Some (most?) clients want coverage not proof

8. MISSION IMPOSSIBLE
• Accurately mimicking high-end attackers is increasingly difficult
• Where’s the data?
• Are all their 3 rd parties in-scope?
• Is their cloud providers infrastructure in-scope?
• Out-of bounds methods
• Spear Phishing? Home and work e-mail?
• Renting botnets with zombies already onsite?
• Purchase 0-days for discovered software?
• Time
• Time to develop 0-days for discovered software?

9. FIXING THE PROBLEM - TERMS
• Use different terms for differing job types
• Vulnerability Scan
• Vulnerability Assessment
• Security Assessment
• Penetration Test

10. FIXING THE PROBLEM - SCOPING
• Threat modelling needs to be done first.
• Right type of test for the customer
• Assessment style testing for establishing controls in place (less developed
customers).
• Penetration style testing for mature companies to prove what they think
should be in place.
• The Underwriters Lab Approach
• Testing specifies the type of attacker being emulated.
• Specifies what’s not in-scope.
• Resistant for a specified duration.