The document discusses the challenges in assessing and mitigating cyber risk, highlighting the need for a data-driven approach to enhance response to emerging threats. It emphasizes the importance of partnerships to collect cyber risk data from multiple sources and developing tools for threat analysis and visualization. Key questions are raised about relevant data, sources, and the value of cyber risk tools in providing actionable information for better understanding and managing these risks.

1. Data-­‐Driven
Assessment
of
Cyber
Risk:
Challenges
in
Assessing
and
Mi;ga;ng
Cyber
Risk
Mustaque
Ahamad,
Saby
Mitra
and
Paul
Royal
Georgia
Tech
Informa;on
Security
Center
Georgia
Tech
Research
Ins;tute
(In
collabora;on
with
the
World
Economic
Forum)
1

2. WEF
2015
Global
Risks
Report
2

3. Talking
About
Cyber
Risk
• Risk
=
Prob.[adverse
event]*Impact[adverse
event]
• AQacks
occur
when
threat
sources
exploit
vulnerabili;es
• Mean-­‐;me-­‐to-­‐compromise?
• Mean-­‐;me-­‐to-­‐recover?
(assuming
detec;on)
• Tradi;onal
assump;ons
and
solu;ons
do
not
apply.
3

4. Why
Even
Try
It?
• Current
cyber
risk
is
anecdotal
and
percep3on
based
and
we
lack
the
ability
to
objec;vely
assess
the
risk
posed
by
ever
evolving
cyber
threats.
• Current
cyber
security
threat
data
is
fragmented
and
collected
by
disparate
en;;es
such
as
security
vendors,
vendors
serving
different
sectors
and
academic
research
centers.
• Publicly
available
cyber
security
data
is
o:en
delayed
and
does
not
provide
the
ability
to
quickly
respond
to
new
threats
that
require
coordinated
effort
within
a
short
;me.
• A
trusted
data
sharing
and
analysis
pla<orm
that
brings
data
from
mul;ple
sources
and
provides
novel
analysis
will
increase
our
ability
to
respond
to
emerging
threats
quickly
and
effec;vely.
4

5. Approach
Develop
partnerships
to
collect
cyber
risk
relevant
data
from
mul3ple
sources
and
analyze
it
to
create
metrics
that
summarize
current
cyber
security
threats
• Combine
public
and
proprietary
data
sources
on
cyber
threats
such
as
soYware
vulnerabili;es,
drive-­‐by
downloads
and
malware
from
a
variety
of
cyber
security
organiza;ons.
• Provide
threat
analy0cs
and
visualiza0on
tools
suitable
for
novice
and
advanced
users,
and
that
can
be
customized
based
on
industry,
technology
pla[orm,
or
geographic
region
5

6. Key
Ques;ons
• What
data
is
relevant?
– Vulnerabili;es,
alerts
from
IDS
system,
compromised
or
malicious
services?
• Where
does
the
data
come
from?
– Public,
proprietary
from
security
vendors
or
government
or
private
en;;es?
• What
can
we
do
with
such
data
for
beQer
understanding
of
cyber
risk?
– Analysis,
visualiza;on,
predic;on?
• What
value
does
a
cyber
risk
tool
offer?
– Ac;onable
informa;on?

7. Current
Data
Sources
• Public
data
– Vulnerabili;es
reported
to
NVD
• Summarized
proprietary
data
– Drive-­‐by-­‐download
risk
data
from
a
major
security
vendor
• Poten;ally
malicious
network
traffic
targe;ng
an
enterprise
– IDS/IPS
alert
data
captured
from
Georgia
Tech
networks

8. Overall
System
Architecture
Vulnerabili3es
and
Threat
Intelligence
Errors
in
commonly
used
soYware
that
can
be
used
to
compromise
personal
or
corporate
systems
Malware
SoYware
used
to
disrupt
opera;ons,
gather
sensi;ve
informa;on,
or
gain
access
to
private
computer
systems.
Public
Na;onal
vulnerabili;es
database
(NVD),
Secunia,
Security
Focus,
and
others
Proprietary
Threat
intelligence
from
security
organiza;ons
IDS
data
from
security
service
providers
New
vulnerability
data
from
soYware
vendors
Data
Extractors
SoYware
to
interpret
data
sources
and
extract
data
to
populate
a
common
database
Database
A
structured
and
consolidated
view
of
the
public
and
proprietary
cyber
security
data
Visualiza3on
and
Predic3ve
Analy3cs
A
tool
to
display
cyber
security
metrics
and
analysis
that
is
customized
to
a
specific
technology
profile,
industry
or
region
Cyber
Risk
Relevant
Data
Possible
Data
Sources
Data
Warehouse
Dashboard
&
Decision
Support
Research
Centers
(e.g.,
Georgia
Tech
Informa3on
Security
Center)
GTISC
uses
proprietary
systems
to
iden;fy
drive-­‐by
downloads
(malware)
in
popular
domains.
GTISC
collects
5
million
malware
samples
every
month
and
iden;fies
command
and
control
domains
setup
by
criminals
to
issue
direc;ves
.
8

9. The
Why
and
What
Vulnerabili3es
Malware
Public
Vulnerability
Data
Na;onal
vulnerabili;es
database
(NVD),
Secunia,
Security
Focus,
and
others
Threat
Intelligence
Emerging
threat
intelligence
from
security
organiza;ons
Alert
Data
Intrusion
Detec8on
System
Data
from
security
service
providers
like
IBM
and
Dell
New
Vulnerabili3es
New
Vulnerability
Data
from
soYware
vendors
GT
Informa3on
Security
Center
GTISC
collec;on
of
5
million
malware
samples
every
month,
as
well
as
command
and
control
(C&C)
domains.
What
we
have
What
we
need
Predic3ve
Analysis
Expected
volume/severity
of
aQacks
on
a
day
Expected
number
of
0
day
vulnerabili;es
on
a
day
Coordinated
Response
Sharing
of
countermeasures
/
response
to
threats
Why
we
need
Malware
samples
and
C&C
Domains
Addi;onal
malware
samples
and
C&C
domains
from
security
service
providers
and
security
vendors
to
be
shared
within
a
trusted
group
More
Comprehensive
Response
More
malware
samples
and
more
C&C
domains
will
provide
for
a
more
protected
environment
for
everyone
9

10. Challenge
I
–
Access
to
Real-­‐world
Threat
Data
10
Data
Sources:
Partnerships
with
various
organiza;ons
to
obtain
cyber
risk
relevant
data
is
cri;cal
for
the
success
of
the
project
Security
Vendors
and
Service
Providers
Consumers
of
Security
Solu;ons
SoYware
Vendors
Client
Companies
&
Govt.
Agencies
Dell
Secureworks
IBM
ISS
Symantec
CERTs
Banks
MicrosoY
Oracle
SAP
IDS
data
Malware
samples
C&C
domain
list
Vulnerabili;es
Malware
samples
C&C
domain
list
Vulnerabili;es
Countermeasures
Typical
profiles
Security
Needs
IDS
Data
Cri;cal
partnerships
Suppor;ng
partnerships

11. Challenge
II
–
Analy;cs
11
Analy0cs:
While
combining
data
sets
provides
new
opportuni;es,
developing
customized
tools
will
depend
on
the
data
feeds
available
Drive-­‐by
Download
Risk
Compromised
websites
infect
user
machines
just
because
they
visit
Serious
threats
for
everyday
users
Georgia
Tech
can
detect
likelihood
of
such
infec;ons
Behavior
Fingerprints
of
Malware
Rapidly
changing
malware
means
we
must
focus
on
execu;on
behavior
Georgia
Tech
processes
about
250,000
samples
each
day
Malware
families
and
spread
What
is
My
Cyber
Risk
Today?
IT
profile
and
security
posture
Value
associated
with
target
Observed
malicious
ac;vity
Mi;ga;on
op;ons
and
ability
Predic3ve
Analy3cs
Epidemiological
analysis
How
far
can
an
aQack
spread?
How
rapidly
can
it
spread?
Are
certain
sectors
under
higher
risk?
“What
if”
scenarios
How
would
these
change
with
a
specific
mi;ga;on
plan?

12. Challenge
III
–
Threat
Visualiza;on
for
Ac;onable
Informa;on
12
Visualiza0on:
Aggrega;ng
all
the
data
feeds
in
a
meaningful
way
to
provide
a
cyber
threat
barometer
is
difficult.
Using
Visualiza3on
for
Naviga3ng
Large
Amounts
of
Threat
Data
Data
overload
is
a
serious
problem
“Flower
field”
metaphor
for
presen;ng
big
picture
Threatened
assets
can
be
easily
iden;fied
for
addi;onal
analysis
From
Big
Picture
to
Deeper
Insights
An
abnormal
asset
visualiza;on
points
to
increased
risk
Click
on
it
can
provide
details
of
vulnerabili;es,
exploits
and
aQack
informa;on
BeQer
situa;on
awareness
and
response
strategy

13. Example
of
System
Provided
Intelligence:
Malware
Source
13

14. Vulnerability
Disclosure
Calendar
14

15. Vulnerability
Data
Visualiza;on
Demo

16. Poten;al
Benefits
• Data-­‐driven
cyber
risk
assessment
can
enhance
cyber
resilience
– Modeling
aQacks:
Will
we
ever
have
be
MTTA
and
MTTR
for
cyber
aQacks?
– Predic;ve
value:
early
aQack
warning
&
proac;ve
response
–
BeQer
intelligence
about
emerging
threats
and
vulnerabili;es
– More
effec;ve
human-­‐in-­‐the-­‐loop
decision
making
with
analy;cs
and
visualiza;on
• “CERT
2.0”
– Real-­‐;me
access
to
threat
informa;on
16

17. Cyber
Threat
Weather
Reports
• Public
vulnerability
data
collec;on
and
analysis
– Calendar
style
visualiza;on
shows
high
level
trends
and
allows
drill
down
for
deeper
insights
– Customiza;on
for
given
informa;on
technology
profile
(sector
or
organiza;on
specific)
• Malware
Threat
Intelligence
– Drive-­‐by-­‐download
risk
by
daily
analysis
of
popular
websites
• “AQempted
aQack”
data
visualiza;on
and
and
;me-­‐
based
trends
• Others….
17

18. Conclusions
• Is
data-­‐driven
cyber
insurance
even
feasible?
• Are
there
objec;ves
indicators
that
can
help
beQer
inform
us?
• Why
will
anyone
provide
data?
– Incen;ves?
• Who
should
do
it?
– Cyber
CDC
– CERT
2.0
18