Master Thesis

USE AUTHORIZATION
In presenting this thesis in partial fulfillment of the requirements for an advanced
degree at Idaho State University, I agree that the Library shall make it freely
available for inspection. I further state that permission to download and/or print
my thesis for scholarly purposes may be granted by the Dean of the Graduate
School, Dean of my academic division, or by the University Librarian. It is
understood that any copying or publication of this thesis for financial gain shall not
be allowed without my written permission.
Signature _____________________________
Date________________________________

Penetration Testing: Closing the gap between perfection and reality
by
Kenon M. Fenton
Thesis
Submitted in partial fulfillment
of the requirements for the degree of
Master of Business Administration in the College of Business
Idaho State University
Spring 2016

COMMITTEE APPROVAL:
To the Graduate Faculty:
The members of the committee appointed to examine the thesis of KENON M.
FENTON find it satisfactory and recommend that it be accepted.
Dr. Corey Schou
Major Advisor
Dr. Alexander Bolinger
Committee Member
Dr. David Beard
Graduate Faculty Representative

Dedication
I would first like to thank Dr. Schou and Dr. Frost, and not just because they are
the ones who decide if my life can now move forward. Dr. Schou has given me an avenue
to become what I want to become, and to do what I have dreamed of doing. He took a
chance on me and has convinced everyone that he has believed in me ever since. For that,
I will be forever grateful. Despite what I may think, Dr. Frost has “provided strong evidence
supporting the theory” that I am not perfect. He has challenged me in ways I could not have
been challenged otherwise, and has pushed me far beyond my comfort zone.
I’d also like to thank Ryan Lind. To be a good cowboy, you need a good clown. I
do not know if I have been the clown or the cowboy, but I do know that without Ryan, I
could not have dodged some of the horns that I did. Without Ryan, I would never have
made it out of the fairgrounds in one piece. I believe every NIATEC student owes a great
deal of their successes to Ryan Lind, and I am certainly no exception.
Ashley and Laura deserve their fair share of credit for my successes as well. I could
not have asked for better team players and leaders to work with. My lessons learned are
without number and will continue going forward. I will never let the relationships
developed with them die. I have no doubt that they will be sources of learning and support
for many years to come.
Of all the people who have endured me the most, I cannot continue and not thank
my wife. When I first arrived at NIATEC, Dr. Schou told me to “kiss your spouse goodbye,
she can have you back when you graduate.” Even though she is very excited that I will
soon have a full-time job at only 40 hours a week, she has always supported the program
and understood the importance of the commitments made therein. I would not be who,
what, or where I am without her.

vi
TABLE OF CONTENTS
USE AUTHORIZATION ................................................................... I
TABLE OF CONTENTS..........................................................................VI
List of Figures...................................................................................................... viii
ABSTRACT ...........................................................................................IX
CHAPTER 1: INTRODUCTION ............................................................. 10
CHAPTER 2: LITERATURE REVIEW ................................................... 12
Social Engineering.................................................................................................14
Social Engineering: The Art of Human Hacking..........................................14
Social engineering: a serious underestimated problem.................................15
Social engineering in information assurance curricula.................................17
Social engineering: the “Dark Art”...............................................................17
Two methodologies for physical penetration/SE..........................................19
Understanding Data Breaches................................................................................20
Verizon Data Breach Investigations Report .................................................21
Investigating Sophisticated Security Breaches .............................................26
Recent Data Breaches: Case by Case.....................................................................29
Target ............................................................................................................29
Home Depot..................................................................................................32
eBay Inc. .......................................................................................................34
Sony Pictures Entertainment.........................................................................36
Penetration Testing ................................................................................................40
Penetration Testing: A Hands-On Introduction to Hacking .........................40
The Basics of Hacking and Penetration Testing...........................................41
NASA Denies Hackers Hijacked Its Drone ..................................................42

vii
CHAPTER 3: PEN TESTING IN THEORY.............................................. 43
The ‘What’.............................................................................................................43
MSR Model...................................................................................................44
Technology ..........................................................................................47
Policy & Procedures ............................................................................47
People...................................................................................................47
The ‘Who’..............................................................................................................48
PCI Compliant Entities .................................................................................48
FFIEC Regulated Financial Institutions........................................................49
Government Agencies...................................................................................50
The ‘Why’..............................................................................................................51
Breaches........................................................................................................51
Associated Costs ...........................................................................................52
The ‘When’ ............................................................................................................53
The ‘How’..............................................................................................................54
CHAPTER 4: PENETRATION TESTING IN REALITY ............................ 55
The Target, Home Depot, and eBay Breaches.......................................................58
The Sony Breach....................................................................................................59
The Gaps: Threat Space Trends.............................................................................62
CHAPTER 5: CONCLUSIONS ............................................................... 67
GLOSSARY.......................................................................................... 69
REFERENCES ...................................................................................... 71

viii
List of Figures
Figure 1 – Frequency of Incident Classification Patterns by Security Incident ....25
Figure 2 – The Defender-Detection Deficit ...........................................................28
Figure 3 – MSR Model, Information Assurance ...................................................46
Figure 4 – Cost Per Record by Records Lost.........................................................52
Figure 5 – Sony Pictures Entertainment Security Breach Image...........................61

ix
ABSTRACT
Penetration Testing: Closing the gap between perfection and reality
Kenon M. Fenton, M.B.A.
Idaho State University, 2016
Supervisor: Dr. Corey Schou
Penetration testing has recently fallen into a comfortable lull. Not entirely the fault of the
penetration testers themselves, clients and business leaders have also receded into a
lackluster attitude about this critical component to cyber security. The rising number of
data breaches, their frequency, and the severity of such incidents provide evidence that
penetration testing is not currently meeting its objectives as originally intended. This
research describes penetration testing in theoretical perfection, and then identifies the
major flaws in the general approach to penetration testing in the status quo. However, the
shortcomings in penetration testing identified today may not be relevant in 5 years. The
author’s main objective is not to correct the current pen test itself, but to redirect the
approach to penetration testing altogether; it is a reminder of the original intents and
goals of penetration testing in its purest form and a plea to readdress those intentions
going forward. Critical thinking is key in avoiding the lethargic approach to penetration
testing that currently plagues the cybersecurity culture of the United States.

10
CHAPTER 1: INTRODUCTION
The cyber security culture in the United States is comprised of many different
organizations, security firms, educators, and businessmen. Cyber security is no longer a
concept that is confined to a specific field or work force. It is something that must be
considered by everyone.
It is believed that Sun Tzu, an ancient Chinese military strategist, when describing
the art of war once said, “If you know your enemy, and know yourself, you need not fear
the results of a hundred battles.” It is from this strategy that cyber security has adopted
the practice of what is now called penetration testing.
One leading objective of this document is to describe in detail what a theoretically
perfect penetration test might look like and is done so in chapter 3. In short, penetration
testing is the act of attacking your own organization in order to identify any
vulnerabilities that an enemy might be able to take advantage of. If done correctly,
penetration testing accomplishes the goal of knowing oneself and the enemy. It has
proven itself extremely successful in the past, and is now a normal practice amongst large
organizations, sometimes even required by policy and standards. But as time has gone on,
penetration testing has fallen into a comfortable slump. Security breaches are on the rise
and hackers are becoming more profitable. The research found herein is an attempt to
identify the currently lackluster areas of penetration testing and to provide the basis for a
new mindset going forward, one that will prevent future attitudes of a lax sense of
security.

11
This will be done by comparing penetration testing in theoretical perfection with
the data and details of real life security breaches and trends in the current threat space.
The behaviors and successes of the threats that are infiltrating the country’s networks are
indicators of flaws in the current penetration testing culture in the United States.
It is also important to note that the benefits of effective penetration testing will be
felt throughout an entire organization and add value to any entity that practices it
correctly. While penetration testing, and security in general, has been seen as a cost in the
past, it is easy to see the value that is added as the details of major data breaches are
examined. Some breaches that are analyzed in this research cost some organizations up to
$10 billion, while others have all but completely destroyed other companies. The
objective of effective cyber security should always be to help meet business objectives
and organizational goals. Penetration testing is a crucial component of cybersecurity and
accomplishes just that.

12
CHAPTER 2: LITERATURE REVIEW
This Literature Review was written as a brief overview of the general information
that was researched and understood during the development of this thesis. The idea is that
summarizing and describing the content found within the articles listed hereafter should
prove that the topic is, in at least some regard, understood as seen by industry and
professional community.
The literature that is presented here comes from a variety of sources, anywhere from
published journals, to news clippings, to industry standard reports. The fact that some
sources referenced here may be biased and less viable than others is recognized; however,
as the abstract written above takes a theoretical approach to describing an issue, it is crucial
to look at this topic from all sides of the coin.
Additionally, some of the breaches and security incidents are simply not found in
any works that have been published under rigorous peer review. The breaches are as recent
and pertinent as possible, which also means that there has not been ample time for them to
find their way into lengthy, published works. The world of cybersecurity changes so fast
that the focus and methods of penetration testing, or any security practice for that matter,
must change as rapidly as the enemy does. It would be pointless to compare today’s threats
with yesterday’s security procedures, and seeing as this paper is written to analyze the
theoretical distance between today’s threat surface and today’s penetration testing1 culture,
only the newest and current information can be applicable. There are concepts and theories
1 Penetration Testing – See Glossary, See also Penetration Testing section of Chapter 2

13
that run deep; these may never change and are ever relevant to this paper as well. You will
see these make their due appearances in building a security mindset for the reader to
understand. However, when analyzing specific incidents and company procedures, it is
recognized that some sources of information will be undoubtedly biased and perceived as
weak sources. This will be offset by the analysis of such incidents from multiple
perspectives, and collectively, these opinions should shape a more reliable picture as a
whole.
The Literature Review has been organized into four different sections. Each section
is also subdivided by individual articles related to the topic, indicated by the article name
being displayed as a bolded heading. These sections will relate directly to topics discussed
throughout the rest of the paper. This has been done with the purpose of guiding the reader
to become familiar with the term Social Engineering. This will be important to understand
before looking at the statistics and data about breaches and the impacts that breaches might
have on victims in general. Finally, a large portion of the Literature Review will focus on
specific recent data breaches in the United States, as well as the current penetration testing
efforts in the United States that failed to prevent such attacks.

14
Social Engineering
The Social Engineering section of the Literature Review is written to help the reader
understand the meaning of the term. Social Engineering has become a major part of today’s
security world and will be referred to throughout the document. As it is generally
understood by industry professionals and used liberally with little to no explanation when
used casually, it would seem prudent to gain an understanding of the term and its uses prior
to analyzing statements made about the thesis topic by professionals and scholars. This
section will be a simple education on the meaning of Social Engineering.
SOCIAL ENGINEERING: THE ART OF HUMAN HACKING
After showing various definitions of the words ‘social’ and ‘engineering’ from
dictionaries and such, Hadnagy defines social engineering as:
“the act of manipulating a person to take action that may or may not be
in their best interest, by their own free will.”
This may include manipulating them to give away information, help you gain access to
something, or have them perform any action by their own free will. By this definition,
social engineering can be used for both good and bad.
Hadnagy gives many examples of this in our everyday lives, excluding any
example of technical nature. He mentions doctors, psychologists, and therapists using
social engineering to ‘manipulate’ their patients to take actions that have good outcomes.
A con man would use social engineering to convince a victim to take an action that would
result in a loss. Doctors and psychologists can use questions to lead a patient into making

15
a good decision, whereas a con man will use questions to move a target into a vulnerable
position.
This concept is easily put into the context of cybersecurity. Anyone who attempts
to manipulate you into revealing information or otherwise is engaged in social
engineering. Some of the most famous forms of social engineering are described here as
well, demonstrating the most basic concepts of social engineering. The 419 scam is a
classic example (also known as the letter from the Nigerian Prince.) In this type of scam,
someone lies about who they are and convinces their victim to send a little help money
with an empty promise for a return. By understanding the definition of social
engineering, we see that these victims have been socially engineered, or in other words,
they were manipulated into taking an action that was desired by the social engineer.
Social engineering is now the most popular form of exploitation in cybersecurity.
The idea of social engineering in cyberattacks is to take advantage of the natural trust
inherently found in humans in order to gain access to computer systems. Convincing
someone to tell you their password, or tricking employees into breaking security policies
are both general examples of social engineering in cyber security. (Hadnagy, 2011)
SOCIAL ENGINEERING: A SERIOUS UNDERESTIMATED PROBLEM
This article outlines a quick and easy experimentation done on an undisclosed firm
using Social Engineering. Two experiments were done in this case, one that targeted
random employees of the company, and another that specifically targeted administrators in
the IT department of the company. Both experiments were conducted by making phone

16
calls with restricted numbers to the targets with the intent of manipulating them into giving
away important information like usernames and passwords.
In the first experiment, those who were conducting the experiment (the attackers)
claimed to be the IT department. They used multiple scenarios to produce a reason for
needing the targets passwords, such as wanting to install a new patch, or to verify if the
user had any installed games on the system. The attackers were able to convince 3 of the 6
targets to provide their usernames and passwords over the phone, and only 1 of the 6 was
concerned enough to mention the call to their colleagues.
The second experiment produced similar results. The attackers contacted two
different administrators and made longer, more delicate approaches to obtain login
credentials for the administrators. This time, the attackers did some extra research by using
social media and the company website in order to find useful information that was used in
the phone call. The attackers identified themselves by name as people who the
administrators might know by name in the company but not personally. They were able to
mention other colleagues by name and position and even bring up the “coincidence” that
the attackers and the targets attended the same university. Once the attackers gained some
fake trust from the targets, they asked for the credentials as they did in the first experiment.
The first target was willing to provide his password, but not over the phone. The second
target however, agreed to provide his credentials right then and there.
The results in both experiments had about a 50% success rate if obtaining
credentials over the phone is the only metric. One could argue the success rate even higher
if you consider other objectives, but the point is that these results are disturbing. This was

17
only one experiment done in one company, but it is not a unique example. Hundreds of
these examples exist in many different situations and circumstances, many more successful
than even this one. (Rößling & Müller, 2009)
SOCIAL ENGINEERING IN INFORMATION ASSURANCE CURRICULA
Twitchell suggests that advances in security technology have made it more difficult
for hackers to simply find a vulnerability in a system and then exploit it. For this reason,
social engineering has become a more popular means of attack in recent years. The
unfortunate reality is that most information assurance curricula do not directly consider
detailed countermeasures for social engineering, nor do they treat social engineering as the
prominent threat that it is.
There are currently 3 commonly suggested ways to counter social engineering
attacks, namely education, training and awareness (ETA), policy, and auditing. However,
these countermeasures are only taught in generalities across the country, and should be
taken more seriously in information assurance curricula as the main defense against the
majority of cyberattacks involving social engineering. (Twitchell, 2006)
SOCIAL ENGINEERING: THE “DARK ART”
Thornburgh argues that the key to maintaining confidentiality, integrity, and
availability of an organizations intellectual property is to control who can access what
information. This includes being able to verify the true identity of requestors. In the world
of information technology, social engineering has widely become known as:

18
“a social/psychological process by which an individual can gain
information from an individual about a targeted organization.”
Social engineers are not hackers by definition, but are enablers to the same cause. In
many cases, they are the same person taking on 2 different but connected roles.
Psychologically, people can be persuaded to act in two different ways. The first
being through sound analytical processing of the facts. The second is through emotions.
These are called the central and peripheral routes to persuasion, respectively. Since social
engineering is generally a misrepresentation or unethical coercion, the central route is not
generally an option for social engineers. Sound logic would say, “the password policy is
in place to protect me. I want to be protected. I must follow the password policy and not
give out my password.” Instead, a social engineer must make their target feel something
strong enough to make them willingly forego established procedures that normally would
be more logical to follow. Generally, the target knows better, but can justify their actions
to make them feel better about it. This is referred to as a “mental shortcut.”
Thornburgh argues that since every bit of information gathered by an attacker can
be useful in one way or another, any elicitation of information from an individual can be
considered a success, even if it does not alone give the attacker what they need to gain
full access to a system. In fact, many social engineering attacks are done in order to gain
information that will be useful in bigger, better social engineering attacks. (Thornburgh,
2004)

19
TWO METHODOLOGIES FOR PHYSICAL PENETRATION/SE
A normal penetration test that aims to find vulnerabilities in an organization’s
technology can also be coupled with a physical penetration test and social engineering in
order to find vulnerabilities in an organization’s people, policies, and physical security.
Many companies will avoid these kinds of penetration tests since the testers will be directly
interacting with employees of the company. These kinds of interactions, based on
deception and trust exploitation, can be damaging to a company’s relationship with their
employees. Employees can become upset after these kinds of tests; they may feel that it
violates their privacy and might lead to lawsuits and loss of productivity. Companies will
often opt out of this option for that reason. (Dimkov, Cleeff, Pieters, & Hartel, 2010)

20
Understanding Data Breaches
This section is meant to help us better understand data breaches in a general sense.
Who is attacking? How are they attacking? Are they successful? What are they targeting?
Who is getting attacked? All of these questions can be answered on a very broad level using
the statistics that have been gathered by tech leaders all over the country.
While other targeted articles are discussed here, the main source for this
information will be the Verizon Data Breach Investigations Report (Verizon DBIR). The
authority and authenticity of this report cannot be stressed enough, as it is a major building
block that is used in determining this information here. This report is compiled using
information and statistics from 70 authoritative and recognized organizations, including
the following:
 Department of Homeland Security
 US-CERT
 SANS
 United States Secret Service
 Tripwire
 United States Defense Security
Service
 McAfee Security
 MITRE Labs
 Splunk
 National Cyber Security Centre
 Center for Internet Security
 Kaspersky Lab
 FireEye
 Palo Alto Networks
 Council on Cyber Security
 Akamai
 Risk I/O
 ThreatSim
 Wombat Security

21
The report focuses on gathering data, analyzing the data in context, and comparing
changes in the data against data collected in years previous in order to discover changes in
security posture, attack vectors, and black market trends. The 2015 report begins by
pointing out that The New York Times devoted more than 700 articles related to data
breaches in 2014, versus the previous year in which we saw less than 125. There is no
doubt that the world of cyber security is not developing quick enough to keep up with the
competition. (Verizon Enterprise, 2015)
VERIZON DATA BREACH INVESTIGATIONS REPORT
The Verizon Data Breach Investigations Report starts by looking at the threat actors
that are carrying out cyberattacks and the victims of said attacks. This is called threat
intelligence2, or the gathering and analyzing of information for the purpose of
understanding threat actors3 and their motives. Threat actors are defined by the report as:
“individuals or groups involved in or that have the potential to be
responsible for a data breach4 or security incident5.”
In a general sense, threat actors will have motive against a particular victim, just as in any
crime. Some breaches are result of a mass attack on anyone that might be vulnerable, where
some breaches are more targeted and specifically aimed at one or more companies in
particular.
2 Threat Intelligence – See Glossary
3 Threat Actor – See Glossary
4 Data Breach – See Glossary
5 Security Incident – See Glossary

22
The newest metric introduced into the analysis of this year’s data was the idea of a
second motive. An attack against a specific company for the purpose of exploiting that
specific company would simply indicate a single motive… to compromise that specific
company. In an alternate scenario, an attack may be launched against a victim for the
purpose of gaining access to a separate victim, who may be the actual target. This would
be called a secondary motive. This year’s data has provided new insight into the way that
the bad guys are thinking in this regard. For over 70% of the attacks that have a confirmed
motive, there is also a secondary victim. Actors have aggressively been attacking
companies and individuals for the purpose of infecting someone else. Understanding this
mindset will be important in later topics, when entry methods and security holes are
discussed in better detail.
Not only did the analysis find that attacks are used to spread to other victims, they
found that the spread is happening incredibly fast! “Based on attacks observed by
RiskAnalytics during 2014, 75% of attacks spread from Victim 0 to Victim 1 within one
day (24 hours). Over 40% hit the second organization in less than an hour. That puts quite
a bit of pressure on us as a community to collect, vet, and distribute indicator based
intelligence very quickly in order to maximize our collective preparedness.” (Verizon
Enterprise, 2015)
The study also found some alarming statistics about phishing attacks and their
continued successfulness. In the 2013 Verizon Data Breach Investigations Report, phishing

23
was associated with over 95% of incidents attributed to state-sponsored actors6. Not only
is it a huge issue with state-sponsored actors, but phishing attacks also comprise over two
thirds of all “cyber-espionage7” related attacks, a trend that was also noticed in the previous
year’s report.
Phishing attacks in the current day and age are especially dangerous, as society is
helplessly attached at the hip to their email. The median time to first click across all the
campaigns that were analyzed is a mere 1 minute and 22 seconds. This is an impossible
amount of time for security teams and IT departments to detect and react to such
campaigns. Additionally, 50% of all users open emails and click phishing links within an
hour. “The reality is that you don’t have time on your side when it comes to detecting and
reacting to phishing events.” (Verizon Enterprise, 2015)
The main purpose of phishing is to harvest user credentials. It is the quickest, easiest
way to manipulate users into supplying their credentials to unauthorized parties. Chris
Kirsch of Rapid7 explains some parts of the 2014 version of the Verizon report in a
breakdown video that was posted by Rapid7, a leading company in exploitation
technology. His notes point out the fact that Verizon discovered an increase in value of
credentials over the past couple of years. In fact, credentials on the black market now sell
with a higher dollar value than credit card information. With no surprise, this seems to be
heavily correlated with the recent rise in phishing and social engineering attacks targeting
credentials of victims. (Kirsch, 2014)
6 State-Sponsored Actors – See Glossary
7 Cyber Espionage – See Glossary, see also Incident Classification Patterns

24
Apart from social engineering and phishing campaigns, there is a huge need for
cracking down on known vulnerabilities. According to Risk I/O, who has been aggregating
vulnerability exploit data since their inception, 99.9% of the exploited vulnerabilities in
this report were compromised more than a year after the CVE8 was published.
Possibly the most important analysis from the report pertaining to the topic being
discussed here is the categorization and organizing of incidents into nine incident
classification patterns, namely miscellaneous errors, crime ware, insider misuse, physical
theft/loss, web app attacks, denial of service, cyber-espionage, point-of-sale intrusions, and
payment card skimmers. The following is a graphical representation of the 96% of all
incidents collected in 2014 falling into one of the nine classification patterns.
8 Common Vulnerabilities and Exposures – See Glossary

25
Figure 1 – Frequency of Incident Classification Patterns by Security Incident
(Verizon Enterprise, 2015)
One of the most interesting tidbits of information that we gain from seeing this
information is the commonality amongst the highest recorded patterns. To be more
specific, the common denominator across the top 4 patterns (which accounts for nearly
90% of all incidents) is people9.
Even looking into the 5th
category, 95% of these incidents involve harvesting
credentials stolen from customer devices and using them to log into web applications with
9 Please refer to the MSR Model discussion found in the “MSR Model” section of Chapter 3.

26
them. For these attacks that are launched specifically in the financial sector the report
explains that “a look through the details of these incidents shows a common sequence of
“phish customer ≥ get credentials ≥ abuse web application ≥ empty bank/bitcoin account.”
The point here is, the world’s security incidents are being dominated by attacks that involve
human beings. If you assumed that the majority of attacks come from simply exploiting
flaws in technology, according to this data, you would be quite wrong. (Verizon Enterprise,
2015)
INVESTIGATING SOPHISTICATED SECURITY BREACHES
From a forensics perspective, locating and preserving evidence from security
breaches, specifically in large networks, is a logistical nightmare. The nature of a large
network presents a large number of challenges for forensic investigators. Networks may
produce large amounts of traffic and rapidly changing environments that make a unique
challenge for forensic experts. In physical forensic investigations, crime scenes can be
blocked off to public access and somewhat preserved for analysis. This is not the case for
cyber forensic professionals. The crime scene of a cyber breach can be in motion for
months, or even years before the live production systems can be taken offline for
inspection, and even then, systems that are simply turned on make millions of changes to
a computer environment every minute.
In order for a productive forensic investigation to take place, it must take place
directly after the incident has occurred. This rarely tends to be the case. Unless the attackers
inform the target organizations on purpose, it often takes some time to pass before a

27
company or government might realize that they have been breached. Those who can detect
a breach as it is happening will also generally have the ability to stop an attack from
completing.
Not only is it extremely difficult to perform these kinds of investigations on such
sensitive systems, the forensics experts have another force working against them. Security
postures in modern organizations are often very lax. All security comes at the cost of
convenience. Logging and backups are not generally so strict that a breach could be
identified by simply reading the logs. Even in organizations that have very strict logging
policies, many breaches are concluded with the attackers erasing logs and covering their
tracks. This is almost always the case with these kinds of sophisticated intrusions from
foreign actors.
The foreign actors that are causing the major breaches in the United States could
be state funded, government operated, part of an organized crime unit, or any combination
thereof. Attacks from these actors are generally more sophisticated and focused, rather than
loosely executed blanket attacks on “whichever systems might be vulnerable.”
(Casey, 2006)
As described above, the detection time for an incident is extremely important, not
only for forensic purposes, but for minimizing the impact of the breach itself. The longer
that an attacker goes undetected in a network, the more damage they can potentially do.

28
Figure 2 – The Defender-Detection Deficit
The above figure shows the deficit between the amount of time it takes to breach a
company and the time it takes companies to detect a breach. It is important to note that the
distance between the averages here have been steadily diverging over the past decade.
Good news however, is that 2014 shows the smallest deficit between the two; however,
this is not expected to be a trend but a simple outlier in the pattern. Either way, Verizon
has discovered that in 60% of the 2014 cases, attackers were able to compromise an
organization within minutes. (Verizon Enterprise, 2015)

29
Recent Data Breaches: Case by Case
This section of the literature review will show data and statistics from some of the
most recent high-profile data breaches in the United States. Naturally, information on these
incidents will not find their way into books or other published works for quite some time.
In order to maintain the integrity of the argument made later based on these breaches,
pertinent information is documented here from multiple sources for each incident,
including official statements, news articles, and posts from prominent and respected
industry professionals. The speed at which cybersecurity changes as well as the threats that
intend to undermine it is simply astonishing. The argument being made throughout this
paper cannot effectively be made if it must wait for properly published information on the
issue; it will already be too late. For this purpose, many readily available, yet less desirable
resources have been gathered to reconstruct a reliable assessment of these incidents.
TARGET
Official Statement: Target Data Breach FAQ
Target releases a FAQ and statements from the CEO about the major data breach
from November 27 to December 15 of 2013, right in the middle of Christmas shopping
season. In the FAQ and statements that are made, Target admits that information including
names, mailing addresses, email addresses, phone numbers, and debit/credit card
information for customers had been accessed by criminals. They also officially state that
up to 70 million individuals may have been affected; 40 million of those include
credit/debit card information leakage, according to their investigation. CVV information
was also stolen for the applicable customers.
Target also tries to brush off the consequences of the attack stating that:

30
“Because this is generally publicly available information, the primary
risk is increased exposure to consumer scams, such as phishing, web
scams and social engineering.”
The statements also explain that only in store purchasers in the United States were
affected; customers in Canada and online purchases were not affected. (Target. Corp,
2015)
Inside Target Corp., Days After 2013 Breach
The Target breach is a prime example of current common security incidents, as this
breach spurred similar breaches of many companies in the following months. In December
of 2013, 40 million Target customer debit and credit card accounts were exposed after a
massive breach.
Target Corp. hired a group of cybersecurity professionals from Verizon to do an
analysis of Targets networks and report any weaknesses found in the implementation
therein. The results are astonishing. Verizon’s reports revealed the following:
“Once inside Target’s network, there was nothing to stop attackers
from gaining direct and complete access to every single cash register
in every Target store... (Professionals) found no controls limiting their
access to any system, including devices within stores such as point of
sale (POS) registers and servers.”
An important note in this investigation also revealed that the security consultants
were even able to communicate directly with cash registers after compromising a deli meat
scale located in a different store.
Official investigators eventually released information about the source of the
original breach. Fazio Mechanical, a Target contracted heating and air company in

31
Pennsylvania, had malware delivered to their systems as an email attachment. Hackers
were able to steal some credentials once they entered the network from the malware in the
email and used the credentials to access Target’s core network. Again, access to Target’s
core network is also free access to every device that is owned by the company. The
attackers pushed their own malicious software out to every cash register in over 1,800
Target stores nationwide.
The article outlines some obvious findings from the reported suggestions made to
retailers and larger companies; segment your network, limit access to certain portions of
your network, and restrict employee network access based on job function.
(Krebs, 2015)
Target to Settle Claims Over Data Breach
After a massive data breach in 2013, Target Corp. agrees to pay $67 million for
costs to thousands of financial institutions. The agreement was made with Visa Inc.,
representing banks and other firms who issued credit and debit cards to Target customers
affected by the breach. MasterCard Inc. is working on a similar agreement with Target
Corp. as well.
Heartland Payment Systems Inc. made a nearly identical deal with Visa Inc. and
MasterCard Inc. in 2010 over a large data breach that they suffered back in 2008.
Similar breaches that followed Target’s loss of data include Home Depot Inc.,
Neiman Marcus Group Ltd. and P.F. Chang’s China Bistro Inc. (Sidel, Target to Settle
Claims Over Data Breach, 2015)

32
HOME DEPOT
Official Statement: The Home Depot Data Breach Investigation
The Home Depot released an official statement stating that criminals used third-
party vendor usernames and passwords to enter the perimeter of the Home Depot’s
network. Once they were inside the perimeter of the network, they were able to elevate
their privileges, and install custom-built malware on all the self-checkout systems in the
United States and Canada based stores.
Approximately 53 million email addresses were also taken from files on the
network. For this reason, Home Depot is offering free identity protection and credit
monitoring services to any customer who used a payment card at Home Depot in 2014, any
time after April. This official statement also outlines the additional security enhancements
that they are implementing in light of the breach. (The Home Depot, 2014)
Home Depot’s 56 Million Card Breach Bigger Than Target’s
Home Depot Inc. says the five-month long attack on their payment terminals has
compromised the information of over 56 million debit and credit cards nationwide, a bigger
attack than that of the massive Target Corp. breach.
Big card-issuing bank is like J.P. Morgan Chase & Co. began replacing customers’
debit and credits cards much earlier than in earlier attacks that year. They recognized the
costs with waiting for official investigations and such to complete before acting (most due
to previous experience.) Other companies like Capital One Financial Corp followed suit.

33
At the time this article was written, Home Depot had not officially disclosed the
point of entry for the attacks, but assured customers that it had been closed and the malware
eliminated. However, the scariest part about this breach was that Home Depot had to be
informed by officials that they had been breached. For five months, Home Depot went
unaware and never actually realized they had been compromised until someone else
informed them. (Sidel, Home Depot's 56 Million Card Breach Bigger Than Target's, 2014)
Home Depot: Will The Impact Of The Data Breach Be Significant?
The latest security breach in a chain of cyberattacks against retailers, 56 million
credit and debit card numbers were exposed. The breach came with a significant cost to the
Home Depot in the form of legal help, credit card fraud, and card re-issuance costs. The
purpose of this article is to quantify the amount of costs associated with this particular
incident for the Home Depot.
Ponemon Research estimates that compromised records can cost a company up to
$194 a piece. Typical costs are related to investigations, remediation efforts, notification
to customers affected, identity theft and repair, credit monitoring, regulatory fines, and
disruptions to normal business operations to name a few. Home Depot reported a $43
million pre-tax expense for investigation and remediation related to this incident alone. At
49 cents per stamp, $27.44 million was spent in notifications to customers. Another $560
million was spent on credit monitoring and similar services. The biggest expense is always
going to be in lawsuits. The 44 current lawsuits against Home Depot, which could go up at
any time, will likely incur about $3 billion in fraud expenses. After calculating business

34
disruptions and other miscellaneous costs, this breach will cost Home Depot approximately
$10 billion, or about $176 per compromised record in total. (View Interactive Institutional
Research, 2015)
EBAY INC.
Official Statement: eBay Inc. To Ask eBay Users To Change Passwords
eBay asks its users to change their passwords due to a “cyberattack that
compromised a database containing encrypted passwords and other non-financial data.”
eBay stated that a small number of credentials were stolen from employees and
used to access the corporate network. They were also working with law enforcement at the
time as well as leading security experts and aggressively investigating and applying the
best forensics tools to protect its customers. (eBay Inc. Staff, 2014)
eBay Database Hacked With Stolen Employee Credentials
According to a statement that eBay posted online, the online retail giant says that:
“Attackers compromised a small number of employee log-in
credentials, allowing unauthorized access to eBay’s corporate
network.”
After acquiring these few credentials, the attackers were able to gain access to a large and
steal a “plethora of information,” including customer names, encrypted passwords, email
passwords, physical addresses, phone numbers, and birthdays. Since financial data is stored
separately for PayPal users on PayPal systems, no financial data was stolen for those using
PayPal.

35
Trey Ford, a Global Security Strategist for Rapid7, says to
“Expect an uptick in phishing, do not click links in email, or discuss
anything over the phone. Call customer service or go directly to
websites as you normally would.”
He explains that attackers who have now accessed all this data about the victims can sound
much more legitimate in their attempts to social engineer the affected customers. The
attackers have a lot of information that they can use in order to convince people to give
them even more information by mistake. (Prince, 2014)
eBay Hacked, Bleeds Data and Why You Need To Act
James Lyne of Forbes points out that there are a few details that eBay seems to
avoid addressing on purpose. They mention that a small number of employees had their
credentials stolen, but they do not say how. Mr. Lyne suggests that phishing must have
been involved, just as it had been involved in ‘attacks of late,’ referring to the various
breaches that covered major news headlines at the time.
This article also describes the possible dangers of having your hashed passwords
exposed to criminals. With enough computing power, your hashed password can be
‘cracked’ with enough time, depending on how strong your password is. He references the
LinkedIn hack where over 5 million hashed passwords were stolen, with 60% of those
hashes being cracked within 2 days of the breach.
This article supports the suggestion made by eBay in their official announcement
to customers to change their passwords to reduce the impact of the breach. The intent was

36
partially to explain and bring to light the ease of cracked the hashed passwords that eBay
simply brushed off as ‘encrypted and protected.’
(Lyne, 2014)
SONY PICTURES ENTERTAINMENT
Official Statement: Sony – Statement on the Hacking
On Monday, November 24th
, 2014, Sony Pictures Entertainment (SPE) experienced
a significant system disruption. The company now has identified the incident as an attack.
Security consultants and law enforcement have been informed and are now involved in the
investigation.
The breach contained personally identifiable information about current and former
employees, who were at the time, extremely difficult to reach or communicate with. The
scope of the attack is still officially being investigated and details cannot be released at this
time. Information lost likely includes name, address, social security number, driver’s
license number, passport number, government identification numbers, bank account
information, credit card information for corporate travel and expense, usernames and
passwords, compensation, HIPAA10 protected information including claims appeals, date
of birth, home address, member ID number, and health/medical information including
specific details about medical procedures and prescriptions.
The statement notifies employees of the identity theft protection services that will
be offered by SPE and warns against further social engineering attacks. As we later learn,
10 HIPAA – See Glossary

37
SPE was having difficulty finding phones numbers or names of employees and could not
effectively communicate with them. The end of the statement includes phone numbers and
email address that employees can reach the company with for more information and status
updates from the company. (Sony Pictures Entertainment, 2014)
Official Statement: FBI - Update on Sony Investigation
The FBI would like to commend Sony Pictures Entertainment (SPE) for the quick
report submitted to the FBI following the discovery of the security breach. SPE reported
the incident to the FBI within hours of discovery, as should all companies who suffer
successful cyberattacks.
In light of the excessive media coverage on the incident, the FBI would also like to
clarify any facts that have been uncovered, so as to subside any rumors or conspiracies that
may be circulating. The following is stated with extreme clarity:
“As a result of our investigation, and in close collaboration with other
U.S. government departments and agencies, the FBI now has enough
information to conclude that the North Korean government is
responsible for these actions.”
This statement comes in conjunction with a list of reasons that would lead them to
believe that the North Korean government is responsible, including technical analysis of
the data deletion malware being connected to other confirmed attacks originating from
North Korea, IP addresses from the countries IP space, and other similarities found in
attacks by the government used against its own North Korean banks and media outlets.

38
Perhaps the main reason for releasing this official statement by the FBI can be
found in the paragraph following the identification of the source, stating:
“We are deeply concerned about the destructive nature of this attack
on a private sector entity and the ordinary citizens who worked there.
Further, North Korea’s attack on SPE reaffirms that cyber threats pose
one of the gravest national security dangers to the United States.
Though the FBI has seen a wide variety and increasing number of
cyber intrusions, the destructive nature of this attack, coupled with its
coercive nature, sets it apart. North Korea’s actions were intended to
inflict significant harm on a U.S. business and suppress the right of
American citizens to express themselves. Such acts of intimidation fall
outside the bounds of acceptable state behavior. The FBI takes
seriously any attempt—whether through cyber-enabled means, threats
of violence, or otherwise—to undermine the economic and social
prosperity of our citizens.”
It is made very clear in this statement that the attack is being taken seriously by the
United States government, and that this attack is unique in that a nation state is attacking
the private sector of another country. (FBI National Press Office, 2014)
Sony Hackers Used Phishing Emails to Breach Company Networks
The CEO of a computer security firm, Stuart McClure, found that hackers in the
Sony Pictures Entertainment (SPE) security breach used phishing emails to gain access to
Sony’s networks. There is clear evidence that shows a massive phishing campaign
launched against the company, using Apple as the disguise for malicious activity. Emails
were sent to employees and executives, including SPE CEO Michael Lynton, that asked
for an Apple ID verification. A link sent victims to a website called “ioscareteam.net” and
prompted the users to enter their Apple ID and password. This website, obviously
fraudulent, stored the credentials for the attackers to use as they please.

39
Evidence also points to the attackers using information from employees’ LinkedIn
profiles to identify probable login information for each employee, hoping that employees
would reuse passwords for both work and personal accounts. As the story goes, they seem
to have been correct. (Bisson, 2015)

40
Penetration Testing
This section will describe the basic definition and purposes of penetration testing
in a cybersecurity setting. There will also be information about what penetration testers in
the United States are finding when performing their assessments.
PENETRATION TESTING: A HANDS-ON INTRODUCTION TO HACKING
Weidman defines penetration testing as simulating real attacks to assess the risk
associated with potential security breaches. Penetration testing should not be confused with
vulnerability testing, or vulnerability assessments. A pentest11 (penetration test)
distinguishes itself from a common vulnerability test in the fact that a pentest requires the
testers to not only discover the vulnerabilities, but to actually exploit them in order to assess
what real attackers might gain access to after a successful exploitation.
Penetration tests generally go through similar phases, each slightly different as it is
defined by a given company or entity. But in some form or another, penetration tests must
go through an initial ‘pre-engagement’ phase, where the scope of the test is defined and
details of the test are agreed upon. This phase will also deal with payment information,
testing windows, and anything else that might be involved in the signing of the contract.
This is a very important phase and always involves lawyers and legal departments to ensure
the safety of both parties involved.
The second general phase is the actual test itself. During the test, teams will use
some form of attack methodology that involves information gathering, scanning and
enumeration, gaining access, maintaining access, and covering their tracks. This model will
11 A penetration test is commonly referred to by professionals as a ‘pentest.’

41
always be slightly different due to agreements and changes made to the process during the
pre-engagement phase.
Lastly, there is always a reporting phase of some sort. The team performing the
penetration test will report its findings to the target company. After all, the purpose of the
test is for the company to discover where it can improve, and without an efficient and
accurate reporting phase, the information would never be passed onto the company itself.
Many people view this as the most important phase for the theoretical purposes of a pentest.
As the main objective of a pentest, a company must be able to understand and know how
to improve security from the reporting phase; otherwise, the pentest should be considered
a failure altogether.
(Weidman, 2014)
THE BASICS OF HACKING AND PENETRATION TESTING
Engebretson has a definition for penetration testing that differs slightly from that
of Weidman, but maintains a similar meaning. In the very first paragraph of his book, he
calls penetration testing a legal and authorized attempt to locate and successfully exploit
computer systems for the purpose of making those systems more secure. He later adds that
a penetration test does not have to successfully exploit a vulnerability in order for the test
to be successful. Additionally, a proper pentest will always result with specific
recommendations for addressing of fixing issues found during the test. A straight forward
‘reporting phase’ as discussed in the previous article.
Other names for penetration testing are also attributed by this author. Pen testing is
commonly referred to as ethical hacking, and white hat hacking. Ethical and white hat are
added to differentiate between having permission to attack a system, versus regular hacking

42
or black hat hacking, attacking a system without permission or with unethical, malicious
intent.
(Engebretson, 2011)
NASA DENIES HACKERS HIJACKED ITS DRONE
This article is primarily about a NASA drone incident, but the underlying theme is
the important part for this discussion. As taught by hundreds of other penetration testers
and security professionals, companies and other entities are focusing their cybersecurity on
perimeter defense. In this incident, a similar issue is discussed and an administrator of
AnonSec explains:
"Once you get past the main lines of defense, it’s pretty much smooth
sailing propagating through a network as long as you can maintain
access. Too many corporations and governments focus 99% on
preventing intruders instead of having viable solutions once there is a
security breach, which is guaranteed to happen."
In short, once an attacker gets past the perimeter defense, there is nothing stopping him in
most cases to navigate the rest of the network as he pleases. (Claburn, 2016)

43
CHAPTER 3: PEN TESTING IN THEORY
The purpose of this chapter is to look at penetration testing from a theoretical
perspective. Imagine starting every sentence with, “theoretically.” The topics covered here
answer questions about penetration testing, theoretically speaking. In theory, what is the
perfect pen test? In theory, who is performing penetration tests? In a perfect world, when
should penetration testing be happening? Answering these questions will produce a
baseline that can be compared to what is actually happening in the real world.
The ‘What’
Cyber security should be nothing more than a complicated game of chess. In a game
of chess, each player has a goal. In order to achieve that goal, you cannot allow your
opponent to achieve his/her goal first, or else the game is over. Each player tries to
anticipate their opponents next move, trying to understand what they are after next, and
how they plan to achieve it. And so it goes, back and forth, attacking and defending,
anticipating one another.
In this game, those who work in cyber security related fields are constantly battling
an array of opponents. Most people think that these opponents are so-called hackers,
college kids in hoods, hacking away at companies from the comfort of their basements full
of pizza and caffeine. The reality is that these opponents may be organized crime units or
governments of other countries. They could be other companies competing for the
competitive advantage, or they could be your own disgruntled employees. As the faces and

44
goals of our enemies change, so should the strategies and techniques change that are
employed against them.
So what is a penetration test and how does it fit into this greater game of chess that
has just been described? There are various definitions of penetration testing floating around
the cybersecurity world, but none stray too far from the others. Definitions from authors
like Weidman and Engebretson are quoted in the literature review. No matter who you ask,
the answer will be similar. Penetration testing is in one form or another, taking the
perspective of the bad guy and looking at an organization through their eyes. It is taking
action against yourself, as if you were your own opponent. It is seeing the organization
from outside its own walls. It the best way to anticipate your opponents next move.
Penetration testing is how you win the game.
MSR12 MODEL
Each organization will have its own set of goals and objectives. Many companies
are out to make money. Some are trying to change the world. Others are created as a hobby
and the main objective is to enjoy the industry. No matter what the organization, be it
private or government related, no entity has ever been created for the sole purpose of being
100% secure. It’s neither fun nor lucrative…
However, all these objectives can be heavily impeded or totally lost when an
opponent begins winning the chess match. Data loss and security breaches are the cause of
billions of dollars in losses every year. Denial of service attacks can shut a company down
12 Maconachy, Schou, and Ragsdale Model – See Glossary

45
very quickly, cutting them off from their customers entirely. It is difficult for companies to
make money without customers. Many companies find competitive advantage in
information; secrets that make them more profitable than their competition. Opponents in
the game might steal a secret or two, removing a company’s competitive advantage. Cyber
security is hardly relevant by itself, but becomes an important necessity for most
organizations if they are to meet their own objectives without falling to their opponents.
Before understanding how penetration testing relates to the Maconachy, Schou, and
Ragsdale (MSR) model, we have to understand a few things about the MSR model itself.
The MSR model defines security services, security countermeasures, information states,
and time as the dimensions in which information assurance operates. Penetration testing
relates very directly to 2 of those, and indirectly to all 4.
In relation to the MSR model, an organization must consider the 5 security
services13 when assessing its cybersecurity strategies for defending its information.
Confidentiality – Do I need to keep this information confidential?
Integrity – How important is it to keep this information accurate?
Availability – If this information suddenly becomes unavailable, what effects will
it have on my objectives?
Authentication – How can I confirm that the people accessing my information
really are who they say they are?
13 MSR Model – See Glossary;

46
Non-Repudiation – If I ever need to prove that a certain someone accessed my
information, can I actually prove it?
Figure 3 – MSR Model, Information Assurance
(Schou & Hernandez, 2014)
If a company can identify which of these security services are important in meeting the
company’s goals, then it can provide these services by addressing the 3 security
countermeasures.

47
Technology
This is what most people think of when they hear the words “cyber security.” Using
technology means using firewalls, intrusion prevention systems, and passwords to
provide the security services.
Policy & Procedures
Policies and procedures can also be used to provide security services. Only allowing
certain people to access certain information, cycling job duties between employees,
and requiring witnesses for certain activities are all examples of policies and
procedures that provide the security services.
People
Perhaps the most important of the three, people are also very important to consider
when providing the security services. People are the things that run the technology,
they develop and carry out the procedures, they make decisions and perform all
kinds of activities within a company.
That was a long introduction to a very simple idea, but this is where penetration
testing fits in. Since the opponent’s objective is to deny an organization its security
services, then they must do so by exploiting one of the security countermeasures. Proper
penetration testing should be able to exploit each of the countermeasures as if they were
the opponent and provide insight to a company about what an attackers next move might

48
be. If a penetration test results in a successful exploitation of a firewall, then the company
knows that it needs to patch a hole in the firewall. Once its patched, the real opponent no
longer has access to that hole in the firewall. The risk has been mitigated.
In theory, if an organization is performing penetration testing and always knows
what the opponents next move will be, then the company is going to win in a game of
chess.
The ‘Who’
Though not flatly required by law across the board, a vast majority of all large
companies and government agencies are mandated or required in some form or another to
perform regular penetration testing. In theory, each company or agency who is required to
perform penetration testing should be doing so.
PCI COMPLIANT ENTITIES
To start off, one of these regulations that probably touches the most entities in the
country is the Payment Card Industry Data Security Standard (PCI DSS). PCI standards
are developed by the PCI Security Standards Council. The Council is a joint entity created
by American Express, Visa Inc., Discover, MasterCard, and JCB International and was
founded in 2006. Together, they produce the PCI DSS standards that are required of any
entity that stores, transmits, or processes cardholder data for the major credit card
companies listed above. Non-compliance can be punished with hefty fines and/or loss of
service for certain credit cards due to breached contract. Merchants are not penalized

49
directly by the Council. Instead, the Council will penalize the merchants’ banks and
financial institutions, which leaves the banks to enforce the rule themselves through their
own contracts with customer merchants. Loss of card use by a bank would affect a lot more
people than the loss of card use by a single merchant, which reasons that enforcement by
banks is generally stricter, especially for large companies like Home Depot and Target.
Since March of 2008, PCI DSS14 standards have included requirement 11.3:
Penetration Testing. The requirement begins by explaining how it specifically differs from
the former requirement 11.2: Vulnerability Assessments.
“A vulnerability assessment simply identifies and reports noted
vulnerabilities, whereas a penetration test attempts to exploit the
vulnerabilities to determine whether unauthorized access or other
malicious activity is possible.” (PCI Standards Council, 2008)
The rule states that such tests must be performed and reported annually and additionally
whenever significant upgrades or changes are made to applications or hardware. The
standard also requires that both internal as well as external tests are to be performed in
order to be PCI compliant. (PCI Standards Council, 2008)
FFIEC REGULATED FINANCIAL INSTITUTIONS
So PCI compliant companies are performing penetration testing; that covers a lot
of organizations, but there is more. The Federal Financial Institutions Examination
Council (FFIEC) consists of many government and national accreditation organizations
14 PCI DSS 3.1, the latest PCI DSS version, was released in April of 2015 and further maintains the
penetration testing requirements.

50
including the Board of Governors of the Federal Reserve System (FRB), the Federal
Deposit Insurance Corporation (FDIC), the National Credit Union Administration
(NCUA), the Office of the Comptroller of the Currency (OCC), and the Consumer
Financial Protection Bureau (CFPB). The FFIEC sets its own government mandated
regulations for financial institutions. One of which is the requirement to perform regular
penetration testing. Without repeating too many similar details as listed above, penalties
for non-compliance can be various and harsh. Cases ranging from a slap on the wrist to
devastating implications have both been recorded regarding non-compliant financial
institutions.
GOVERNMENT AGENCIES
Government agencies may actually be the most regulated of the most common
groups. Depending on what branch of government an agency belongs to, or what purpose
it serves, each agency falls under a whole slew of laws and regulations that may change
from month to month. The Federal Information Security Management Act (FISMA) is a
public law that is mandated across any government entity with information systems that
contain information related to national security. FISMA is made up of general guidelines
that reference more detailed Special Publications (SP series) that are published by the
National Institute of Standards and Technology (NIST). Various publications that are
referenced in major sections of FISMA including SP 800-53A15 and SP 800-11516 address
the requirement for penetration testing as a proper annual security testing requirement.
15 Special Publication 800-53A: Guide for Assessing the Security Controls in Federal Information Systems
and Organizations
16 Special Publication 800-115: Technical Guide to Information Security Testing and Assessment

51
(National Institute of Stardards and Technology, 2014) (National Institute of Stardards and
Technology, 2008)
On May 21st
of 2015, Secretary of Homeland Security Jeh Johnson signed and
issued the first Binding Operation Directive (BOD) that requires non-Department of
Defense federal agencies to undergo scanning and penetration testing by the Department
of Homeland Security (DHS) and to mitigate any findings within 30 days. As authority
was given by Congress and with FISMA as the momentum, nearly every civilian side
federal agency is now undergoing penetration testing and scanning from the National
Cybersecurity Assessments and Technical Services (NCATS) team from DHS. (Secretary
of Homeland Security, Jeh Johnson, 2015)
These are just a few examples of penetration testing being regulated by law or being
required by industry standard. Basically, a lot of people are required to do penetration
testing.
The ‘Why’
‘The Why’ is possibly the only question that most business executives would be
concerned about. No matter who you are, the why is always ‘to add value to your
organization.’
BREACHES
As described in various chapters of this paper, security breaches and data loss
account for billions of dollars in losses every year across the country. In theory, penetration
testing will prevent everyone one of these breaches. A breach can only occur if an opponent
exploits a vulnerability that the company does not know about, or the opponent exploits a

52
vulnerability that the company knows about that they simply have not addressed. A perfect
penetration test strategy would suggest that there is no vulnerability that a company does
not know about.
ASSOCIATED COSTS
Figure 4 – Cost Per Record by Records Lost
“How much does it cost?” will likely be another question that executives ask, and
rightfully so. Depending on your networks, the size of your company, the scope of the test,
and the countermeasures that you want to have tested, a penetration test can cost anywhere
between $4,000 and anything else. Some pen tests can end up costing over $50,000.

53
Security has always been seen as a cost, until recently. People are finally realizing that
security is more of a positive. It may not make you much money, but if you can spend
$10,000 on a pen test in order to prevent a $10 million security breach, you have made
your M.B.A. professors proud.
Security breaches can also result in other costs besides extra red on the balance
sheet. As mentioned before, information is generally at the heart of a company, and stolen
secrets can really diminish an organizations competitive advantage. Data loss can also
effect a company’s reputation and damage the trust and relationships between a company
and its customers. Theoretically, the extent of the damage may have no end.
The ‘When’
When should an organization conduct penetration testing? In a perfect world,
penetration testing would be free and a company could be testing itself around the clock,
24/7, but even in theory, this isn’t expected. However vague it may be, the best answer
really is, “before you experience a breach.” While any time is a good time for a pent test,
it is important to look at what your protecting and figure out how much you are willing to
spend on protecting it. This section gets a lot more interesting when you’re not talking
about the question “theoretically.” The real answer is “it depends,” so long as it happens
before you have lost the chess game.

54
The ‘How’
“The How” question brings the whole issue back to the MSR Model. A good pen
tester will look at the security services and security countermeasures as well. The only
difference is a pen tester should be looking at them through the eyes of the opponent. What
does a hacker want to achieve? Which service would be most beneficial to deny? What
countermeasure would be easiest to exploit? Do the hacker care if he is caught? To truly
beat the hacker, you must think like the hacker.
Not only must a pen tester ask these questions from a hacker’s perspective, but they
must also be asked from the perspective of all an organization’s enemies. What does a
disgruntled employee want to achieve? Which service would a competing company want
to deny? What counter measure would be easiest for a foreign government to exploit? The
issue has to be tackled from all angles.
Without getting extremely detailed, the penetration tester will focus most of their
attention on exploiting the countermeasures – technology, policies, and people. Finding
and exploiting the vulnerabilities in technology will allow the company to see what an
opponent can achieve when targeting their technology. Finding and exploiting
vulnerabilities in a company’s policy will give the company an opportunity to find
solutions and alter their policies accordingly. Finding and exploiting vulnerabilities in the
people involved with the company should shed light on what an opponent can achieve by
doing so, and consequently influence the company to find solutions to these vulnerabilities
as well. The penetration testing process must systematically approach each of these
dimensions in the MSR Model. Many methodologies have come and gone in attempting
this, but few, if any, have been completely successful.

55
CHAPTER 4: PENETRATION TESTING IN REALITY
After understanding the premise of chapter 4, which is that penetration testers
should be emulated the threats that organizations are defending themselves against, it is
necessary to no look at the threats themselves. Who are the threats? What are they looking
for? Why are they attacking a company? By answering these and other questions, it should
be very clear who, what, and how a pentest should be executed if it were to simulate the
enemy.
It is important to look at both individual breaches as well as data about security
breaches in general. This makes it easier to identify specific details about breaches and
different possibilities while also being able to identify general trends in the industry as a
whole. The following are details about 4 individual high-profile breaches that for one
reason or another, have attracted more attention than others. Following the descriptions of
each breach, there will be some information about the general trends that have been
identified from Verizon’s Data Breach Investigations Report.
The Target Breach
In the 2013 Target Inc. security breach, the names, mailing addresses, email
addresses, phone numbers, and debit/credit card information for over 70 million customers
were compromised, 40 million of those even included CVV information.
While information about the attackers remains unknown (or simply undisclosed),
forensic reports and security teams were able to identify the point of entry and map out
how the attack was actually launched. The initial point of entry was not within Target at
all, but inside Fazio Mechanical, the heating and air company contracted by Target in

56
Pennsylvania. Malware was delivered to the Fazio Mechanical company via an email
attachment, which stole usernames and passwords for the Fazio Mechanical systems. The
attackers used the stolen credentials to access Target’s core networks, posing as Fazio
Mechanical employees. The investigators said,
“Once inside Target’s network, there was nothing to stop attackers
from gaining direct and complete access to every single cash register
in every Target store... We found no controls limiting their access to
any system, including devices within stores such as point of sale (POS)
registers and servers.”
Security consultants testing the claim were even able to communicate directly with cash
registers in one store after compromising a deli meat scale located in a different store. From
there, the attackers loaded software onto the registers and point of sale devices, essentially
reading every card that was swiped during the Christmas season of that year.
The settlements with financial institutions alone cost Target Inc. over $67 million.
Target has since made sizable additions to its security team, and mandated rigorous
employee training to help better protect its customer’s information.
The Home Depot Breach
In April of 2014, unknown attackers used credentials that it had stolen from an
undisclosed third party vendor to access Home Depot’s networks. The attackers then had
the ability to push their own custom built malware out to every credit card terminal that
belonged to Home Depot across the United States and Canada based stores. Over the course
of five months, this malware transmitted all credit card information out of the Home Depot
network and back out to the attackers. After five months, the FBI noticed a pattern in the

57
large amounts of new data being sold on the dark net17 markets and notified Home Depot
that they believed the information was coming from Home Depot customers. Home Depot
had to be informed by someone else that their networks had been compromised.
Over 56 million customers were affected by this breach. It is estimated that this
breach has cost Home Depot around $10 billion. The postage costs of mailing notifications
to every customer alone cost just over $27 million dollars.
The eBay Breach
May is the month that marks the next high-profile attack of 2014. Hackers stole the
credentials of three corporate employees and used them to access many different areas of
the eBay networks. The exact method used by the attackers has not been disclosed,
however, leaks here and there and some serious speculation have all suggested a targeted
phishing attack, but has not been officially confirmed.
After visiting many different areas of the network, the attackers finally found the
database that stores the user information. The personal information including usernames
and passwords of 145 million users was stolen from this database in a single attack. This
information was found being sold on the dark net as well, which leads investigators to
believe that the attackers were cyber criminals looking to simply sell the information for
money. The company was breached again 5 months later in a similar manner, with
corporate employee credentials. The second breach resulted in the website sending users
to fake websites posing as eBay.
17 ‘Dark Net’ or ‘Deep Web’ markets are the ‘underground’ black markets on the internet where hackers
can sell the information that they have stolen.

58
The Target, Home Depot, and eBay Breaches
In theory, if these companies were performing “perfect penetration tests,” than
many of the costs of each data breach could have been reduced, or even eliminated. It is
time to look at these cases and identify the benefits that could have come from proper
penetration testing. If a penetration test emulates a threat, then a good pen test would have
tipped each of these companies off to the issues that were revealed after the actual breach.
The Target, Home Depot, and eBay breaches share some disturbing similarities that
display major gaps between the security cultures of the companies and the perspective that
has been outlined earlier in chapter 3.
Were the companies pen testing before the breach? Each company, at the time of
their corresponding incident, were deemed to be PCI compliant. However, each company
is also facing PCI compliance fines after being further investigated, due to the breaches.
Penetration tests were happening but they were obviously not good enough. In each case,
elaborate additions were made to security teams and additional precautions have been
taken. If the companies could have been breached in a similar manor by a controlled
penetration test that did not cost billions of dollars, a need for these changes could have
been realized and implemented prior to a real breach.
The attackers in all of these cases were targeting customer information in order to
sell the information on the black market. The attackers made money off of the customer
information. Notice that no damage was done to any of the companies’ infrastructures.
Malware was not installed to effect performance, steal secrets, or hinder the day to day
services provided by the companies.
Perhaps the most important similarity between the attacks is the how. In every case,
including the Sony breach, credentials were stolen and used to access core systems. Not

59
only were they stolen and used, but the credentials that were stolen somehow allowed
attackers to get anywhere they wanted. In both the Target and Home Depot incidents, the
credentials of third party vendors were the targets and used as gateways into the real
target’s network. What does it really mean when credentials are used to breach a company
in relation to the perspective outlined in chapter 3? When an attacker steals credentials,
they can legitimately authenticate to the system and totally bypass any perimeter defense
like a firewall. A penetration test that emulates this kind of attack should really look a
network from the perspective of any employees that can authenticate to a system. If an
attacker logs in as a secretary, what kind of damage can they do as oppose to logging in
with the credentials of the CEO? A proper penetration test will answer this question for
security experts to use in hardening a system.
The Sony Breach
The data breach at Sony Pictures Entertainment is certainly a unique case, and
should be classified as much more than a security breach. The breaches at Target, Home
Depot, and eBay were part of a long string of recent attacks that map out a very similar
attack methodology being used in other recent cases as well, like P.F. Changs, Snapchat,
Kickstarter, and Apple’s iCloud. These security incidents resulted primarily of data being
sold for money. There was a monetary motivation behind the attacks. The ‘incident’ at
Sony, however, was really a full-fledged attack aimed to bring the entire Sony company to
its knees.

60
The FBI has concluded that the attack came for the North Korean government,
seemingly a retaliation against Sony for releasing a movie that parodies a fictional attempt
to assassinate the North Korean dictator Kim Jong-Un.
The attackers sent phishing emails to various corporate employees, pretending to
be Apple. These emails asked employees to verify their Apple ID credentials by clicking
on a link and submitting credentials into a fake Apple website. The attackers then used
these credentials in conjunction with job titles and names of employees found on LinkedIn
to guess various valid credentials used at Sony by the employees. Using the credentials,
North Korea spent more than a year inside the networks of Sony Pictures Entertainment,
gathering information and mapping out the network. In November of 2014, they finally
revealed their intentions by displaying disturbing pictures on workstations and locking
employees out of all systems.

61
Figure 5 – Sony Pictures Entertainment Security Breach Image
(Peters, 2014)
The extent of the attack is quite disturbing. Servers and machines were literally
damaged so badly that they had to be replaced. Investigators believe that 100% of all the
data stored by Sony was compromised. Financial information, social security numbers of
all employees and clients, including many celebrities, health records, all stolen and used
for vengeful purposes. The motive is unique in that money was not made by the
compromising of the data. Instead, the attackers used the info to blackmail employees into
leaving the company, posted sensitive health information and counseling sessions on social
media, and damaged credit scores and identities by identity theft and abuse. The costs
associated with this attack are much harder to account for, given the variety of effect the
attack had on the company. From a financial standpoint, it is probably not as costly as the

62
Home Depot hack, but from the perspective of the company, this attack was much more
devastating to the company itself than any of the other incidents discussed here.
It may go without saying, but the source of the attack is much different here than in
more common security breaches. A nation state attack, coming from someone like North
Korea, should be much harder to avoid than an attack from a small group hoping to make
a quick buck. Interestingly enough, the method of entry here was still the same: phishing
emails and stealing credentials.
The Gaps: Threat Space Trends
Finally, a look at the gaps between the perfect pentest and reality. Perfection in this
light can never be achieved, but there are certainly leaps and bounds that can be made in
the right direction from where the country’s security culture is now. The current trends in
the threat space are not matching up to how the country is penetration testing. As discussed
in the Literature Review, this information comes from the Verizon Data Breach
Investigations Report (DBIR)18. (Verizon Enterprise, 2015) This report is compiled by over
70 trusted and known institutions that have gathered data about breaches from the previous
year, including many authoritative government and industry experts. It is the most reliable
and respected source of such data in the industry. By combining the data from the report
with the analysis done herein on specific instances, an accurate ‘big picture’ of the threat
space can be constructed with a high level of confidence.
18 See ‘Understanding Data Breaches’ in Chapter 2: Literature Review

63
The first question posed in chapter 4 was, “What is a penetration test?” The answer
is that a penetration test should emulate an attack from a real threat. So what does an attack
from a real threat look like? In the past, attacks have focused on the flaws in technology
and exploited the vulnerabilities. That hasn’t exactly gone away, but advances in
technology and a general focus on security has helped improve that area to a degree. The
examples examined earlier provide evidence of the trend moving toward targeting poor
policies and people.
That being said, there are still organizations that are breached purely by flaws in
technology. The DBIR found that 99.9% of all the technical vulnerabilities exploited in
2014 were exploited over a year after the CVE and mitigation recommendations were
published. What’s worse is that 10 vulnerabilities make up 97% of all those exploits. This
indicates that when an attacker specifically exploits a vulnerability in technology, they are
being successful by using the same ten vulnerabilities. This does not indicate that the
technology isn’t good enough to beat hackers, it indicates that either the people or the
policies in an organization are not fixing the problems when the problems are found.
That could be an easy fix, if that were the only way the bad guys are getting in, but
it’s not. Recent advances in technology and in cybersecurity have eliminated many of these
vulnerabilities in technology that are being exploited. So long as an organization decides
to eliminate them, this forces hackers to look at the other two countermeasures besides
technology. A real attack from a real threat does not follow rules or focus on a single
countermeasure but will use any combination of vulnerabilities it can find in people, the
policies and procedures of an organization, or its technology.

64
The current trends in the threat space provide evidence supporting this argument.
The new thing to do, according to the report, is to attack the vulnerabilities in trust that is
naturally inherent to humans. Phishing and other social engineering attacks are not only on
the rise, but now also the most common form of attack. Phishing alone is used in over two
thirds of all cyber espionage attacks. All of the point of sale systems, yes, 100% of the POS
systems were breached by compromised or stolen credentials. People are tricked into
giving their passwords away. They are misplacing them, misusing them, or just bad at
managing them. Why would an attacker go through the work of hacking into a system when
they can get a password that authenticates? They don’t. They get the password.
The secondary victim/secondary motive is a new and prominent metric for
understanding data breaches that has been introduced in this year’s report. For 70% of the
attacks where there is a known motive, there is also a secondary motive or victim. As shown
in the Target and Home Depot breaches, the initial target was not the actual victim of the
attack. Many companies must allow some kind of access to third party vendors in order to
accomplish business objectives and that is has become a very popular means of entry. In
fact, 75% of the breaches with a secondary victim spread from victim 0 to victim 1 within
24 hours of compromise. What does this mean in relation to the MSR model?
This is really an attack using the policies and procedures countermeasure. Although
there are countless others, this example will be easiest to understand, given the analysis
done on the Target and Home Depot breaches already. Should a secretary be able to access
the financial database? Should the janitor’s login give him access to a system domain
controller? These may seem like technical issues but the policies and procedures of a

65
company are what drive the implementation of these kinds of rules. Should the credentials
from a third party HVAC company allow you to access the POS terminals of a company
in every store across the entire country? This kind of flaw in policy and procedure could
have been identified before the attackers were ever able to take advantage of it.
Ironically, asking a ‘why’ question over and over again is a good way to answer the
“what,” “when,” “who,” and “how”. The results of current penetration testing continue to
lead technical solutions. The perspective is that every breach is the result of a technical
exploit and thus, continue pen testing for vulnerabilities in the technology alone. Solutions
are made to address the symptoms of real problems, while the real problems are rarely
identified. For example:
Payment card data was captured from an e-commerce web application.
1. Why? — Because the threat actor made changes in the payment application code
to capture and send data when processed.
2. Why? — They bypassed authentication to upload a backdoor to the server via
Remote File Inclusion (RFI).
3. Why? — Because the JBoss version was outdated and vulnerable to a widely
known attack.
4. Why? — Because the server software hadn’t been updated in years.
5. Why? — This is where it gets tricky. Because… they thought their third-party
vendor would do it? Because… they didn’t know they had to? Because… they

66
thought they had, but failed to check implementation? Because… they had
insufficient processes in place to manage their risk? (Verizon Enterprise, 2015)
Penetration testing should be able to do what the bad guys are doing. In this
example, it would be exploiting an e-commerce web application. If this were the case, a
company could perform this exact analysis, asking some “why’s” about their processes,
discovering the issues in their technology, their policies, or their people before the actual
bad guys are able to exploit them at a much higher cost.

67
CHAPTER 5: CONCLUSIONS
The penetration testing that is being performed in the country is far from perfection,
and will certainly never reach perfection as it has been described previously in this
document. However, penetration testing in the status quo could be improved by leaps and
bounds in the direction towards theoretical perfection. When an organization is surprised
to find out that a vulnerability was exploited by a real threat actor, it is evidence that points
to a faulty pen test. This is what a penetration test should prevent. In order to be truly
effective, the security culture and penetration testing mind set of security professionals
must gravitate aggressively toward the mindset of the adversary and emulate their every
attack.
The current trends of the threat space behaviors have been analyzed and provided
herein, but the end goal is not to have penetration testing match these specific trends. The
whole argument made in this document is really that penetration testing should do what
was done here. These trends won’t be relevant in three to five years, they will change again,
and if every pen tester read this paper and decided to change their methods to match the
trends specified here going forward, then the author has not met his goal. The security
culture must be constantly analyzing the threat space and understand it as it changes.
Furthermore, these changes must be adapted to and integrated into all layers of the security
posture of an organization. The country cannot afford to half-heartedly point fingers at the
symptoms of problems any longer; the real problems must be identified and addressed at
their core.

68
The hope and the plea of this author is that the security culture in the United States
will change, that security professionals stop doing only what they know, to stop doing what
is comfortable to them, but to begin looking at their organizations through the eyes of their
enemies. Organizations cannot survive the long game of chess against their enemies
without expanding their perspective and considering all the countermeasures in the MSR
model as platforms for their ever-lurking attackers.

69
GLOSSARY
Common Vulnerabilities and Exposures – Referred to more commonly as CVEs, the
Common Vulnerabilities and Exposures are a list of the known information security
vulnerabilities and flaws with associated information about severity and mitigation
recommendations for each vulnerability. These are contributed to by many different
organizations and are peer reviewed and confirmed.
Cyber Espionage – Cyber espionage is an attack on an entity for the purpose of stealing
confidential secrets and intellectual property. This is not an attack where the end result is
to steal and sell credit card and/or other privacy information.
Data Breach – An incident that resulted in confirmed disclosure to an unauthorized
party. The term is used interchangeably with the term “data compromise” in this report.
HIPAA – Health Insurance Portability and Accountability Act. This act protects health
information and requires due diligence in protecting such information.
Incident Classification Patterns – The 9 incident classification patterns are as follows:
miscellaneous errors, crime ware, insider misuse, physical theft/loss, web app attacks,
denial of service, cyber-espionage, point-of-sale intrusions, and payment card skimmers.
Incident Classification Patterns – These are the different types of patterns that have
been identified during the analysis of breach information. These are specific to industry
and may change from year to year as the reports are released.
MSR Model – Maconachy, Schou, and Ragsdale Model. This model adds Non-
Repudiation and Authentication to the original three security services described in the
McCumber Cube: Confidentiality, Integrity, and Availability. These three original
services are commonly known as the CIA Triad.
Penetration Testing – The act of attacking one’s own organization in order to identify
vulnerabilities and the potential impact of a security incident. This must be done with
proper authorization by all parties involved. As opposed to a vulnerability assessment,
vulnerabilities are actually exploited in a penetration test in order to assess the potential
impact such vulnerabilities being exploited by real enemies. Generally speaking,
penetration testing refers to testing components of an organization that relate specifically
to cyber security.
Security Incident – Any unintentional or unauthorized event that compromises the
confidentiality, integrity, or availability of an information asset. (Verizon Enterprise,
2015)

70
State-Sponsored Actors – A State-Sponsored Actor is a threat actor that may be
receiving support, protection, or funding from a country’s government. These are
amongst the most dangerous threat actors as the resources available in performing an
attack may be substantial.
Threat Actor – A threat actor can be any person, group, state, country, or organization
that poses a threat to the 5 security services that cyber security provides an entity:
confidentiality, integrity, availability, non-repudiation, and authentication. These may be
hackers, nation states, disgruntled employees, con artists, competing/rival companies, etc.
Threat Intelligence – The gathering and analyzing information about an organizations
enemies and threat actors. The purpose of threat intelligence is to truly understand who
the threat actors are, what are their goals, why are they your enemies, how they can
achieve their goals, etc.

71
REFERENCES
Bisson, D. (2015, April 22). Sony Hackers Used Phishing Emails to Breach Company
Networks. Retrieved from The State of Security: http://www.tripwire.com/state-
of-security/latest-security-news/sony-hackers-used-phishing-emails-to-breach-
company-networks/
Casey, E. (2006, February). Investigating sophisticated security breaches.
Communications of the ACM - Next-generation cyber forensics, 49(2), 48-55.
Claburn, T. (2016, February 2). NASA Denies Hackers Hijacked Its Drone. Retrieved
from Information Week:
http://www.informationweek.com/government/cybersecurity/nasa-denies-hackers-
hijacked-its-drone-/d/d-id/1324154
Dimkov, T., Cleeff, A. v., Pieters, W., & Hartel, P. (2010). Two methodologies for
physical penetration testing using social engineering. ACSAC '10 Proceedings of
the 26th Annual Computer Security Applications Conference (pp. 399-408). New
York: ACM.
eBay Inc. Staff. (2014, May 21). Official Statement. Retrieved from eBay Inc.:
https://www.ebayinc.com/stories/news/ebay-inc-ask-ebay-users-change-
passwords/?utm_source=301Redirect&utm_medium=301Redirect&utm_campaig
n=301Redirect
Engebretson, P. (2011). The Basics of Hacking and Penetration Testing. Waltham, MA:
Elsevier Inc.
FBI National Press Office. (2014, December 19). Update on Sony Investigation.
Retrieved from FBI: https://www.fbi.gov/news/pressrel/press-releases/update-on-
sony-investigation
Hadnagy, C. (2011). Social Engineering: The Art of Human Hacking. Indianapolis:
Wiley Publishing, Inc.
Kirsch, C. (2014, May 21). eBay Hacked: Need-to-Know Details for Protection. Rapid7.
Retrieved from http://www.rapid7.com/resources/videos/ebay-hack-details.jsp
Krebs, B. (2015, September 21). Inside Target Corp., Days After 2013 Breach. Retrieved
from KrebsOnSecurity: http://krebsonsecurity.com/2015/09/inside-target-corp-
days-after-2013-breach/
Lyne, J. (2014, May 21). eBay Hacked, Bleeds Data And Why You Need To Act.
Retrieved from Forbes: http://www.forbes.com/sites/jameslyne/2014/05/21/ebay-
hacked-bleeds-data-why-you-need-to-act/#5b0ed1eb468e
National Institute of Stardards and Technology. (2008). Special Publication 800-115.
National Institute of Stardards and Technology. (2014). Special Publication 800-53A
Revision 4.
PCI Standards Council. (2008). DSS Information Supplement 11.3: Penetration Testing.
Peters, S. (2014). Sony Hackers Knew Details Of Sony's Entire IT Infrastructure.
DarkReading.
Prince, B. (2014, May 21). eBay Database Hacked With Stolen Employee Credentials.
Retrieved from Dark Reading: http://www.darkreading.com/attacks-

Master Thesis

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (8)

Similar to Master Thesis

Similar to Master Thesis (20)

Master Thesis