This document summarizes Daniel Miessler's blog post describing different types of security assessments. It discusses vulnerability assessments, penetration tests, red team assessments, audits, white/grey/black box assessments, risk assessments, threat assessments, threat modeling, and bug bounties. For each type, it provides a definition, what they are commonly confused with, and what they are best used for. The key messages are that vulnerability assessments aim to find all issues while penetration tests validate security, and organizations should perform assessments before tests to identify and fix issues.

1. Reprint of IOActive’s Daniel Miessler’s blog post describing the major types of
security assessment, along with what differentiates them.
UNDERSTANDING INFORMATION
SECURITY ASSESSMENT TYPES

2. “There are many different types of security assessments…
...and they’re not always easy to keep separately in our minds (especially for sales types).”
Daniel Miessler is a well-known
information security professional based
in San Francisco. For more than 20 years,
he’s been writing about his infosec
projects and other interests, as he puts it,
“as a means of organizing everything
I have learned and want to learn.”
With organization and education in mind,
Daniel wrote a helpful post describing
the major types of security assessments
and how they’re unique and we asked if
we could re-share it. If you’re one of the
“sales types” Daniel mentions above, or
just looking to educate yourself on
infosec topics, then click ahead.

3. VULNERABILITY ASSESSMENT
What is it?
A vulnerability assessment is a technical assessment
designed to yield as many vulnerabilities as possible
in an environment, along with severity and
remediation priority information.
What’s it commonly confused with?
The vulnerability assessment is most often confused
(and/or conflated) with the Penetration Test. This is
primarily because sales people think the latter
sounds cooler, bless their hearts.

4. VULNERABILITY
ASSESSMENT
What’s it best at?
The vulnerability assessment is best used when
security maturity is low to medium, when you
need a prioritized list of everything that’s wrong,
where the goal is to fix as many things as
possible as efficiently as possible.

5. What is it?
A Penetration Test is a technical assessment
designed to achieve a specific goal, e.g., to steal
customer data, to gain domain administrator, or to
modify sensitive salary information.
What’s it commonly confused with?
The Penetration Test is most often confused (and/or
conflated) with the vulnerability assessment. See
‘Sales People’ for more information. Another way to
think about this is to imagine vulnerability
assessments as looking for security problems when
you know/assume they exist, and penetration testing
as validating a configuration when you believe it to
be secure.
PENETRATION TEST

6. PENETRATION TEST What’s it best at?
Because a Penetration Test is designed to achieve
one or more specific goals, they should not be
commissioned by low or medium security
organizations in most cases. Performing a
Penetration Test against a low or medium security
shop will simply yield recommendation
all-time-greats like, “Implement patching across the
organization.”, “Disable inactive users.”, and—one of
my favorites—”Understand where your sensitive
data is.” Do not waste money on a Penetration Test
unless you’ve already undergone one or seventeen
vulnerability assessments and then remediated
everything that was found. Penetration Tests are for
testing security that is assumed to be strong, not
documenting the contents of a soup sandwich.

7. PENETRATION TESTING: HACKERONE CAN HELP
TRY A HACKERONE CHALLENGE TODAY AND SEE FOR YOURSELF
Crowdsourced penetration testing has been proven to find more critical
vulnerabilities, at a lower cost than traditional penetration tests.
Now for a quick break from Daniel’s awesome content to show you this HackerOne commercial...
Or read the guide on hacker-powered pen tests for more information

8. RED TEAM ASSESSMENT
What is it?
A Red Team “assessment” is something of a
misnomer in the corporate context since corporate
Red Team services should ideally be continuous
rather than point-in-time. So it should ideally be
more of a service than an assessment. But
regardless of that distinction, the central purpose of
a corporate Red Team is to improve the quality of
the corporate information security defenses, which,
if one exists, would be the company’s Blue Team.
In fact, that’s what a lowercase “red team” is: an
independent group that challenges an organization
to improve its effectiveness. In the case of
corporate Red Teams, the org they’re improving is
the Blue Team.
Red Team services should, in my opinion, always
have the following five elements: Organizational
Independence, Defensive Coordination, Continuous
Operation, Adversary Emulation, and Efficacy
Measurement.

9. RED TEAM ASSESSMENT
What’s it commonly confused with?
Red Team services are most commonly confused
with Penetration Testing. Sales and marketing
groups are using the terms nearly interchangeably,
as are many internal security groups. People
confusing the two are basically seeing “Red Teaming”
as a sexier, more elite type of Penetration Test. They
are not the same. A Penetration Test is a defined,
scoped, and point-in-time assessment that has
specific goals for success or failure. A corporate
Red Team (whether internal or external) is a
continuous service that emulates real-world
attackers for the purpose of improving the Blue
Team. They may share TTPs at times, but they have
very different purposes.

10. RED TEAM ASSESSMENT What it’s best at?
Red Team services are best used when an
organization has covered the basics of strong
vulnerability management and has at least some
capability to detect and respond to malicious or
suspicious behavior in the environment. If an
organization is still struggling with basic asset
management, patching, egress traffic control, and
other fundamentals, it’s usually best that they get
those solved before hiring or building a “Red Team”.
Red Teams are for testing mature security postures
in a real-world way, not for enumerating issues in
low-maturity environments. If you don’t have a Blue
Team, you probably don’t need a Red Team.

11. AUDIT
What is it?
An audit can be technical and/or
documentation-based, and focuses on how an
existing configuration compares to a desired
standard. This is an important point. It doesn’t prove
or validate security; it validates conformance with a
given perspective on what security means. These
two things should not be confused.
What’s it commonly confused with?
Audits are often confused with pretty much any
other type of security assessment where the goal is
to find vulnerabilities and fix them. That could be
part of an audit, if there’s an item in the standard
that says you shouldn’t have vulnerabilities, but the
key attribute is mapping current state against an
arbitrary standard.

12. AUDIT
What’s it best at?
Organizations use audits to demonstrate compliance.
Importantly, compliance should not be used to
demonstrate security. Secure organizations are
significantly more likely to be compliant (if checked), but
compliant organizations should lay no claims to being
secure just because they are in accordance with
standard X or Y.

13. BEYOND THE AUDIT: HACKERONE CAN HELP
SEE HOW HACKERONE RESPONSE, THE ISO 29147 COMPLIANT SOLUTION, CAN HELP
Now for a quick break from Daniel’s awesome content to show you this HackerOne commercial...
Compliance does not equal security but it is a
necessary box to check.
Or read our guide on the 5 Critical Components of a VDP for more information

14. WHITE/GREY/BLACK ASSESSMENT
What is it?
The white/grey/black assessment parlance is used to
indicate how much internal information a tester will
get to know or use during a given technical
assessment. The levels map light to internal
transparency, so a white-box assessment is where
the tester has full access to all internal information
available, such as network diagrams, source code,
etc. A grey-box assessment is the next level of
opacity down from white, meaning that the tester
has some information but not all. The amount varies.
A black-box assessment—as you’re hopefully
guessing—is an assessment where the tester has
zero internal knowledge about the environment, i.e.
it’s performed from the attacker perspective.

15. WHITE/GREY/BLACK ASSESSMENT
What’s it commonly confused with?
The largest source of confusion around
white/grey/black-box nomenclature is not realizing
that they aren’t really an assessment type but rather
an aspect of one. They’re most commonly paired
with vulnerability assessments where you’re trying to
find the most issues possible, and that provides
significant incentive to open the curtains a bit.
Remember that the goal of a vulnerability
assessment is to find as many issues as possible, so
hiding internal information from a tester that keeps
them from finding issues doesn’t hurt them—it hurts
you. Don’t confuse wanting to know what attackers
can see/do with wanting to know what problems you
have. These are two separate things and need to be
approached separately. If you want to know what an
attacker can do, fix all your issues until you’re
confident you’re as secure as possible, and then get
a Penetration Test.

16. WHITE/GREY/BLACK ASSESSMENT
What’s it best at?
White-box assessments are best used with
vulnerability assessments because you want to find
as many issues as possible, regardless of how the
tester came to discover them. Grey-box
assessments are often used when people are
confused about the difference between a
Penetration Test and a vulnerability assessment.
They want to give some information, but not all.
Let’s be clear: if you’re trying to find all of your issues,
you shouldn’t withhold information from the tester.
If you’re doing a Penetration Test,
however, you shouldn’t give the tester anything,
which is a black-box assessment. Keep these clear in
your mind and you’ll be ok.

17. RISK ASSESSMENT What is it?
Risk Assessments, like threat models, are extremely
broad in both how they’re understood and how
they’re carried out. At the highest level, a risk
assessment should involve determining what the
current level of acceptable risk is, measuring the
current risk level, and then determining what can be
done to bring these two in line where there are
mismatches. Risk Assessments commonly involve
the rating of risks in two dimensions: probability, and
impact, and both quantitative and qualitative models
are used. In many ways, risk assessments and threat
modeling are similar exercises, as the goal of each is
to determine a course of action that will bring risk to
an acceptable level.

18. RISK ASSESSMENT
What’s it commonly confused with?
Risk Assessments are commonly confused with
threat assessments, as both are pursuing similar
goals. The primary differentiator is in where
assessments start and where they place their focus.
Threat Models focus on attack scenarios and then
move into the agents, the vulns, the controls, and
the potential impacts. Risk Assessments often start
from the asset side, rating the value of the asset and
the map onto it the potential threats, probabilities of
loss, the impact of loss, etc.
What’s it best at?
Risk Assessments should arguably be considered an
umbrella term for determining what you have of
value, how it can be attacked, what you would lose if
those attacks were successful, and what should be
done to address the issues. It’s important that when
someone says they’re going to do a risk assessment
that you delve deeper into exactly what is meant by
that, i.e. what approach or methodology will be used,
what the artifacts will be, etc.

19. THREAT ASSESSMENT
What is it?
A threat assessment is a type of security review that’s
somewhat different than the others mentioned. In
general it pertains more to physical attacks than
technology, but the lines are blurring. The primary
focus of a threat assessment is to determine whether
a threat (think bomb threat or violence threat) that
was made, or that was detected some other way, is
credible. The driver for the assessment is to
determine how many resources—if any—should be
spent on addressing the issue in question.

20. THREAT ASSESSMENT
What’s it commonly confused with?
The term “threat” is used numerous ways within security,
which leads to significant confusion. In this case the term is
used as in, “a threat was made”, or “determining whether
the threat was real”, as opposed to the “threat-agent”
usage. The origin comes from the Secret Service
investigating school violence, and the challenge was
determining which of the thousands of threats they
received they should respond to with extremely limited
resources. This is in stark contrast to what many think of
when they hear threat assessment, which is investigating
potential threat-agents, such as hackers, governments, etc.
What’s it best at?
A threat assessment is best used in situations
where someone has made a claim around
performing an attack in the future, or such a
potential is uncovered somehow. The goal in
that case would be to learn whether the
situation is worth spending resources on
addressing.

21. THREAT MODELING What is it?
Threat Modeling is not a well-understood type of
security assessment to most organizations, and part
of the problem is that it means many different things
to many different people. At the most basic level,
threat modeling is the process of capturing,
documenting, and (often) visualizing how
threat-agents, vulnerabilities, attacks,
countermeasures, and impacts to the business are
related for a given environment. As the name
suggests, the focus often starts with the threat agent
and a given attack scenario, but the subsequent
workflow then captures what vulnerabilities may be
taken advantage of, what exploits may be used, what
countermeasures may exist to stop/diminish such
an attack, and what business impact may result.

22. THREAT MODELING
What’s it commonly confused with?
Threat Modeling is confusing in general. Much of the
confusion comes from debates around definitions
and semantics, as threat modeling often includes
discussions around threats, threat-agents,
vulnerabilities, exploits, controls, risks, and impacts.
Each of these is loaded on its own, and when you
start trying to have a conversation with all of them at
the same time religious wars often result. The other
issue is that people lose track of the goal because
there are so many elements in play. Are we trying to
identify vulnerabilities? Are we trying to profile
threat-agents? Are we documenting potential
business impacts? Etc. The best way to summarize is
to say that Threat Modeling brings a dose of
potential reality to a security posture. It shows you,
through attack scenarios, where gaps exist that
could lead to real-world consequences.

23. THREAT MODELING What’s it best at?
Organizations should be using threat modeling early
and often, and they should definitely be part of the
development process. They are a way of ensuring
that known potential attack scenarios can actually be
handled by a given security posture. They can also
be extraordinarily illuminating from a pure
documentation and visibility standpoint. Seeing your
potential threat-actors, how they’re likely to attack
your app or system, using what vulns and what
exploits, and what it’ll likely do to your organization is
often a sobering experience. They’re especially
useful for showing non-security-people how
compliance and security products do not a security
program make.

24. BUG BOUNTY What is it?
A Bug Bounty is a type of technical security
assessment that leverages crowdsourcing to find
vulnerabilities in a system. The central concept is
simple: security testers, regardless of quality, have
their own set of strengths, weaknesses, experiences,
biases, and preferences, and these combine to yield
different findings for the same system when tested
by different people. In other words, you can give 100
experienced security testers the exact same testing
methodology and they’re likely to find widely
different vulnerabilities. The bug bounty concept is
to embrace this difference instead of fighting it by
harnessing multiple testers on a single assessment.

25. BUG BOUNTY What’s it commonly confused with?
Bug bounties are a relatively new approach to doing
technical security testing, and there is some
confusion around whether they should be done
instead of another security test or in addition. The
best answer, I’d argue, is that a bug bounty should
be considered a vulnerability assessment in its goal
of finding as many issues to remediate as possible,
but be considered a Penetration Test in that you
should do classical vulnerability assessments first.
The reason for this is that bug bounties, because
they use many people, excel in finding uncommon
and eccentric issues, and the exercise is somewhat
wasted on identifying the common problems that
can be uncovered using automation and
single-tester assessments.

26. BUG BOUNTY
What’s it best at?
Bug bounties are best used when you have already
performed one or more standard vulnerability
assessments (which should have included both
automated and manual testing) and then you’ve
remediated everything that was found. Consider
them an optional step between classical vulnerability
assessments and a Penetration Test, which, as noted
above, does not seek to find all issues but rather to
confirm that the security posture is where it needs to
be by pursuing specific goals.

27. Want to find critical vulnerabilities? You need
continuous testing by the best hackers
BUG BOUNTY PROGRAMS: HACKERONE CAN HELP
HACKERONE CAN HELP YOU START A BUG BOUNTY PROGRAM TODAY
Now for a quick break from Daniel’s awesome content to show you this HackerOne commercial...
Or read our Bug Bounty Field Manual guide on how to plan, launch, and operate a bug bounty program for more information

28. Most Frequently Confused
ASSESS THEN TEST!
Miessler says: If you aren’t confident in your
security posture and know already that it’s
not solid, you should be doing Vulnerability
Assessments—not Penetration Testing.
Penetration testing is for testing your posture
once you have it where you want it.
BOUNTIES SURFACE BUGS NOT FOUND WITH OTHER METHODS!
Miessler says: The best way to think about Bug Bounties is an
enhancement to the discovery phase of a Vulnerability Assessment.
Vulnerability Assessments have two pieces: Discovery (finding as many
issues as possible), and Prioritization (ranking what should be fixed
first). Bug Bounties are great at the first part, and not good at the
second. As such, they are best used when you have done multiple
Vulnerability Assessments already and have already found the easy
stuff. Bug Bounties excel at finding issues not found using other
methods.

29. Most Frequently Confused
RED TEAMS ARE CONTINUOUS!
Miessler says: Because marketing and sales drive
the infosec industry, people are constantly
conflating Red Teaming and Penetration Testing.
Because Red Teams are meant to emulate the
adversary they generally only work if they are
both continuous and run over long
periods—ideally permanently. So if you have
some company offering to do a 2-week “Red
Team” engagement, this is probably better
described as a Penetration Test. So the key
distinctions are the emulation of real-world
attackers, including their tenacity, the permanent
duration of the attack, the TTP sophistication, etc.
Assessments that lack those elements are
Penetration Tests, not Red Team engagements.

30. Expand & Reinforce Your Security Efforts
Receive vulnerability reports on a secure platform, perform discreet
tests using ethical hackers, and run bug bounties at any scale.
HACKERONE CAN HELP!
LEARN HOW

31. Did You Like This Presentation? You Can Get More
InfoSec Insights by following Daniel Miessler
Daniel Miessler is an information security
professional and writer based in the San
Francisco Bay Area. Learn more about
Daniel and read his 2,500 essays, posts,
and other content on his website,
danielmiessler.com. And, subscribe to his
weekly podcast and newsletter,
Unsupervised Learning, to get Daniel’s
curated look at the most interesting stories
covering security, technology, and humans.

32. MAKE THE
INTERNET
SAFER.
WWW.HACKERONE.COM / SALES@HACKERONE.COM / +1 (415) 891-0777