Rory McCune argues that penetration testing is becoming obsolete due to overloaded terminology, clients' lack of readiness, and the complexity of accurately simulating high-end attackers. He suggests redefining terms and improving scoping through threat modeling to better align tests with client maturity and needs. The document emphasizes the importance of differentiating between various types of security assessments and tailoring approaches accordingly.

1. PENETRATION TESTING MUST DIE
Rory McCune

2. INTRODUCTION
• IT/Information Security person for the last 15 years
• Currently a director of 7 Elements ( http://www.7elements.co.uk )
• IT Security consultancy based in Scotland.
• Scottish chapter lead for the OWASP project.

3. AGENDA
• Why must it die
• Overloaded Terminology.
• Clients aren’t ready.
• Mission Impossible.
• How do you fix it?
• Better Terms.
• Better Scoping.

4. WHAT’S IN A NAME?
• First thing is to define what we mean by penetration testing
• One definition from Wikipedia “A penetration test, occasionally pentest, is a
method of evaluating the security of a computer system or network by
simulating an attack from a malicious source, known as a Black Hat Hacker, or
Cracker.”

5. CHARACTERISTICS OF A PENETRATION TEST
• Black-Box
• When we’re assuming the role of an attacker (unless it’s an insider) the
testing should be black box
• Goal based
• Trying to compromise the target system/network/company
• Trade off against coverage of every possible avenue
• Realistic
• Mimicking the “real thing”
• Although…. Which Real thing?

6. OVERLOADED TERMINOLOGY
• Like many things in security the term “penetration test” is overloaded
• Vulnerability Scans as Penetration Tests
• Web Application Security Assessments as Penetration Tests
• Code Reviews as Penetration Tests
• …

7. CLIENTS AREN’T READY
• What’s the purpose of a penetration test?
• From above it’s to mimic an attack
• Trade off realism against coverage
• Test controls that should be in place
• Implies that clients are ready for that
• Know what controls should be in place
• Think that they’re operating effectively
• Some (most?) clients want coverage not proof

8. MISSION IMPOSSIBLE
• Accurately mimicking high-end attackers is increasingly difficult
• Where’s the data?
• Are all their 3 rd parties in-scope?
• Is their cloud providers infrastructure in-scope?
• Out-of bounds methods
• Spear Phishing? Home and work e-mail?
• Renting botnets with zombies already onsite?
• Purchase 0-days for discovered software?
• Time
• Time to develop 0-days for discovered software?

9. FIXING THE PROBLEM - TERMS
• Use different terms for differing job types
• Vulnerability Scan
• Vulnerability Assessment
• Security Assessment
• Penetration Test

10. FIXING THE PROBLEM - SCOPING
• Threat modelling needs to be done first.
• Right type of test for the customer
• Assessment style testing for establishing controls in place (less developed
customers).
• Penetration style testing for mature companies to prove what they think
should be in place.
• The Underwriters Lab Approach
• Testing specifies the type of attacker being emulated.
• Specifies what’s not in-scope.
• Resistant for a specified duration.