safe browser

1.
ANALYSIS
BRIEF
–
September
2012
IS
YOUR
BROWSER
PUTTING
YOU
AT
RISK?
PART
1
–
GENERAL
MALWARE
BLOCKING
Authors
-­‐
Bob
Walder,
Francisco
Artes,
Stefan
Frei,
Ken
Baylor,
Jayendra
Pathak,
Vikram
Phatak
Overview
The
ineffectiveness
of
Web
browser
security
is
one
of
the
most
common
reasons
for
malware
infection.
Browsers
offer
a
direct
and
unique
route
for
infection,
bypassing
corporate
protection
layers
and
bringing
malware
deep
into
the
corporate
environment,
often
protecting
it
from
detection
using
SSL.
Browsers
must
provide
a
strong
layer
of
defense
from
malware,
rather
than
defer
to
operating
system
antimalware
solutions.
This
series
examines
the
effectiveness
of
leading
browsers
to
block
malware.
The
four
leading
browsers
were
tested
against
three
million
samples
of
real
world
malicious
software.
Major
discrepancies
were
noted
in
their
ability
to
block
malware.
Data
represented
in
this
report
was
captured
over
one
hundred
and
seventy-­‐five
(175)
days
through
NSS
Labs’
unique
live
testing
harness,
and
provides
in-­‐depth
insight
into
the
built-­‐in
protection
capabilities
of
modern
browsers,
including
Chrome,
Firefox,
Internet
Explorer,
and
Safari.
This
series
of
papers
will
examine
the
ability
of
the
four
leading
browsers
to
block
each
of
the
five
main
purposes
of
malware
and
malware
monetization.
Monetization
of
malware
is
achieved
by
multiple
means,
including
click
fraud,
fake
antivirus,
account
/
password
theft,
bank/financial
fraud,
and
gaming
fraud.
Collectively
they
account
for
billions
of
dollars
worth
of
corporate
and
consumer
theft
per
year,
yet
browsers
vary
widely
in
their
ability
to
block
malware,
despite
adverse
effects
on
business
and
individual
users
alike.
Tested
Products
• Apple
Safari
5
• Google
Chrome
15
-­‐
19
• Microsoft
Internet
Explorer
9
• Mozilla
Firefox
7
–
13
Over
3,000,000
test
cases
were
used
in
the
data
sampling
captured
via
NSS
Labs’
unique
live
testing
harness.
An
initial
sample
set
of
227,841
unique
and
suspicious
URLs
entered
the
system;
84,396
were
found
active
and
malicious
and
met
the
criteria
for
entry
into
the
test.
In
total
3,038,324
test
runs
were
performed
by
the
four
browsers
against
these
unique
84,396
URLs
–
resulting
in
over
750,000
tests
cases
per
browser.

2. NSS
Labs
Analysis
Brief
–
Is
Your
Browser
Putting
You
At
Risk?
Part
1
Testing
was
repeated
every
six
(6)
hours
until
the
target
URL
was
no
longer
active.
Samples
that
did
not
pass
the
validation
criteria
were
removed,
including
false
positives
and
adware.
Ultimately,
1,407,233
URL
test
cases
passed
the
post-­‐validation
process
and
are
included
in
the
results.
Each
sample
payload
was
validated
internally.
MD5
hashes
of
samples
were
submitted
to
VirusTotal
and
the
resulting
scanner
reports
were
then
used
to
classify
malware
types.
Additionally,
the
test
samples
were
verified
by
multiple
independent
external
sources
to
confirm
distribution
accuracy
and
malware
classification.
100%
80%
60%
Firefox
40% Chrome
Internet Explorer
20% Safari
0%
Figure 1 – Malware Block Rate Over Time with 10-Day Moving Average (higher % is better)
During
the
testing
period,
Internet
Explorer
maintained
a
malware
block
rate
of
95%
while
Firefox
and
Safari’s
block
rate
remained
just
under
6%.
Over
the
same
time
period,
Chrome’s
block
rate
varied
from
13%
to
just
over
74%.
This
could
be
attributed
to
changing
protection
tactics
over
time
that
is
indicative
of
the
ongoing
battle
between
antimalware
developers
and
malicious
actors.
NSS
Lab
Findings:
• Browsers
offer
the
largest
attack
surface
in
most
enterprise
networks
and
are
the
most
common
vector
for
malware
installations
• The
use
of
SSL
by
browsers
presents
additional
problems
to
enterprises
since
it
offers
the
opportunity
to
bypass
many
layers
of
corporate
security
protection
• The
leading
browsers
show
a
significant
variance
in
their
ability
to
block
malware.
• Given
the
increasing
mobility
of
users
and
devices,
blocking
malware
is
not
only
extremely
important,
but
potentially
the
only
means
of
reducing
risk
when
outside
of
the
corporate
perimeter
of
protection.
• Web
browsing
is
the
primary
attack
vector
of
criminals
attempting
to
monetize
malware,
using
a
variety
of
means,
including
click
fraud,
fake
antivirus,
account
/
password
theft,
bank/financial
fraud,
and
gaming
fraud.
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
2

3. NSS
Labs
Analysis
Brief
–
Is
Your
Browser
Putting
You
At
Risk?
Part
1
• The
tolerance
of
browsers
with
low
malware
block
rates
may
present
undue
risk
to
an
organization.
NSS
Labs
Recommendations:
• Users
should
evaluate
browser
security
as
part
of
their
layered
security
strategy.
• Enterprises
should
perform
a
risk
analysis
of
the
browsers
in
the
organization
and
remove
those
with
unjustified
high
risk
where
possible.
• Enterprise
and
individual
users
should
use
the
findings
in
this
report
to
assist
in
the
selection
of
the
browser
most
appropriate
to
their
protection
needs.
However,
malware
infection
rather
than
exploits
were
the
subject
of
this
test,
and
readers
should
not
draw
conclusions
based
upon
this
analysis
brief
alone.
Analysis
As
the
most
widely
used
and
ubiquitous
means
of
accessing
the
Internet,
web
browsers
are
uniquely
positioned
to
filter
and
stop
malware
at
an
early
stage.
This
capability
becomes
even
more
important
given
the
increasing
mobility
of
devices,
which
means
corporate
perimeter
and
network
protection
services
cannot
always
be
relied
upon.
To
complement
traditional
defenses
and
to
address
the
highly
dynamic
nature
of
current
attacks
and
attack
distribution
methods,
modern
web
browsers
employ
technologies
to
block
access
to
malicious
URLs
before
loading
the
content.
Blocking
access
to
malicious
URLs
is
a
formidable
first
line
of
defense,
since
it
provides
complete
protection
against
malware
entering
the
system.
However,
little
is
known
or
published
on
the
effectiveness
of
web
browser’s
internal
blocking
technology
and
performance.
This
analysis
examines
the
ability
of
four
different
web
browsers
to
protect
users
from
malware
downloads,
also
known
as
socially-­‐engineered
malware.1
Modern
web
browsers
offer
an
added
layer
of
protection
against
these
threats
by
leveraging
in-­‐the-­‐cloud,
reputation-­‐based
mechanisms
to
warn
users
of
potential
infection.
However,
not
all
vendors
have
taken
the
same
approach.
Browser
protection
contains
two
main
functional
components.
The
foundation
is
an
“in-­‐the-­‐cloud”
reputation-­‐
based
system
which
scours
the
Internet
for
malicious
web
sites
and
categorizes
content
accordingly,
either
by
adding
it
to
a
black
or
white
list,
or
assigning
a
score
(depending
on
the
vendor’s
approach.)
This
categorization
may
be
performed
manually,
automatically,
or
using
both
methods.
Some
vendors
will
utilize
feedback
from
user
agents
on
their
customers’
endpoints
to
report
back
to
the
reputation
system
automatically,
providing
information
relevant
to
the
trustworthiness,
or
otherwise,
of
applications
and
files
downloaded
from
the
Internet.
The
second
functional
component
resides
within
the
web
browser
itself,
and
requests
reputation
information
from
the
in-­‐the-­‐
cloud
systems
about
specific
URLs
and
then
enforces
warning
and
blocking
functions.
1
Exploits that install malware without the user being aware (also referred to as “drive-by downloads”) are not included in this particular study.
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
3

4. NSS
Labs
Analysis
Brief
–
Is
Your
Browser
Putting
You
At
Risk?
Part
1
When
results
are
returned
that
a
site
is
“bad,”
the
web
browser
redirects
the
user
to
a
warning
message
or
page
informing
that
the
URL
is
malicious.
In
the
event
that
the
URL
links
to
a
download,
the
web
browser
instructs
the
user
that
the
content
is
likely
malicious
and
that
the
download
should
be
cancelled.
Conversely,
when
a
website
is
determined
to
be
“good,”
the
web
browser
takes
no
action
and
the
user
is
unaware
that
a
security
check
was
performed.
Internet
Explorer
Warning
Chrome
Warning
Firefox
Warning
Safari
Warning
Figure
1
–
Browser
Warnings
Functionality
unique
to
Chrome
NSS
Labs
determined
that
Safe
Browsing
API
v2
includes
additional
functionality
that
has
been
integrated
into
Chrome,
but
not
Firefox
or
Safari.
This
functionality
provides
reputation
services
for
executable
files,
or
as
Google
describes
them
“malicious
downloads”.
Figure
2
-­‐
Chrome
Safe
Browsing
Warning
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
4

5. NSS
Labs
Analysis
Brief
–
Is
Your
Browser
Putting
You
At
Risk?
Part
1
Malware
Block
Performance
Each
browser’s
individual
block
performance
was
tracked
over
time
and
mapped
by
malware
purpose.
When
aggregated
an
overall
block
rate
of
all
collected
malware
by
browser
was
developed.
A
browser’s
overall
block
rate
is
defined
as
the
percentage
of
successful
blocks
divided
by
the
total
number
of
test
cases.
With
tests
conducted
every
6
hours,
a
URL
that
was
online
for
48
hours
will
be
tested
8
times.
A
browser
blocking
it
on
6
(out
of
a
maximum
8)
test
runs
will
achieve
a
block
rate
of
75%.
Figure
3
shows
the
overall
block
performance
of
the
four
browsers
tested.
As
expected,
since
Firefox
and
Safari
using
the
same
technology
they
achieve
similar
block
rates.
However,
the
large
difference
of
the
average
block
rate
between
browsers
is
noteworthy,
with
results
ranging
from
4.7%
up
to
94%.
Chrome 27.6%
Firefox 5.0%
Internet Explorer 94.0%
Safari 4.7%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Figure
3
–
Overall
Malware
Block
Rate
by
Browser
(higher
%
is
better)
To
assess
the
effectiveness
of
different
blocking
technologies,
the
NSS
test
harness
also
records
the
mechanism
that
blocked
access
to
a
URL.
Of
the
three
browsers
using
Google’s
Safe
Browsing
API,
Chrome
is
the
only
one
to
also
utilize
Google’s
malicious
download
technology.
Figure
4
shows
the
block
performance
of
the
URL
blocking
component
and
the
additional
download
block
component
used
only
by
Google’s
Chrome.
The
URL
blocking
performance
of
these
three
Safe
Browsing
browsers
was
consistent
at
around
5%.
Google’s
malicious
download
protection
proved
to
be
almost
five
times
more
effective
than
URL
blocking
alone.
As
seen
in
Figure
,
it
increases
overall
blocking
performance
by
28%
compared
to
URL
blocking
alone,
and
accounts
for
the
majority
of
the
blocking
performance
of
Google
Chrome.
The
core
protection
technology
in
Internet
Explorer
is
SmartScreen,
which
provides
URL-­‐based
protection
from
attacks
via
an
integrated
cloud-­‐based
URL-­‐reputation
service.
SmartScreen
also
works
with
Download
Manager
to
prevent
malicious
downloads.
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
5

6. NSS
Labs
Analysis
Brief
–
Is
Your
Browser
Putting
You
At
Risk?
Part
1
Chrome! 4.6% 23.0%
Firefox! 5.0%
Internet Explorer! 94.0%
Safari! 4.7%
0%! 10%! 20%! 30%! 40%! 50%! 60%! 70%! 80%! 90%! 100%!
Safari! Internet Explorer! Firefox! Chrome!
SmartScreen! 94.0%!
SafeBrowsing! 4.7%! 5.0%! 4.6%!
Malicious Download! 0.0%! 0.0%! 23.0%!
Figure
4
–
Blocking
technologies
used
by
browsers
(higher
%
is
better)
Time
to
block
Malicious
Sites
Every
time
a
new
campaign
is
launched
by
malicious
actors,
it
is
vital
that
it
is
detected
as
quickly
as
possible
by
security
solutions
deployed
in
the
enterprise.
The
following
response
time
graph
shows
how
long
it
took
each
of
the
browsers
to
block
a
threat
once
it
was
introduced
into
the
test
cycle.
Cumulative
protection
rates
are
calculated
each
day
until
blocked.
100%
90%
80%
70%
60%
Block Rate
Internet Explorer
50%
Chrome
40%
Firefox
30% Safari
20%
10%
0%
0 5 10 15 20 25 30
Days
Figure
5
-­‐
Time
to
Block
Malicious
Sites
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
6

7. NSS
Labs
Analysis
Brief
–
Is
Your
Browser
Putting
You
At
Risk?
Part
1
Days
Firefox
Chrome
Internet
Explorer
Safari
1
4%
20%
91%
4%
2
5%
22%
92%
4%
3
5%
23%
92%
4%
4
5%
24%
92%
4%
5
5%
25%
93%
4%
6
5%
25%
93%
5%
7
5%
26%
93%
5%
10
5%
27%
93%
5%
15
5%
28%
94%
5%
20
5%
28%
94%
5%
25
5%
28%
94%
5%
30
5%
28%
94%
5%
Table
1-­‐
Time
to
Block
Malicious
Sites
Ultimately,
the
results
reveal
significant
variations
in
the
abilities
of
the
browsers
to
protect
against
malware.
Chrome
provides
more
protection
than
Safari
or
Firefox
using
the
Safe
Browsing
feed,
apparently
due
to
its
malicious
download
protection.
Trends
show
minor
differences
between
Firefox
and
Safari.
Results
from
these
tests
indicate
that
the
four
browsers
vary
both
in
their
approach
and
effectiveness
in
blocking
different
malware
categories.
It
was
decided
to
further
categorize
the
malware
behind
the
suspicious
URLs
to
measure
the
browser’s
block
performance
for
each
class
of
malware.
The
ability
of
the
four
leading
browsers
to
block
each
of
the
five
main
purposes
of
malware:
click
fraud,
banking/financial
fraud,
fake
antivirus,
password/account
theft
and
game
fraud
was
examined
and
will
be
detailed
in
subsequent
papers
in
this
series.
Reading
List
Analysis
Brief:
Did
Google
Pull
a
Fast
One
on
Firefox
and
Safari
Users?
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
7

8. NSS
Labs
Analysis
Brief
–
Is
Your
Browser
Putting
You
At
Risk?
Part
1
Appendix
A
–
Methodology
Client
Host
Description
All
tested
browser
software
was
installed
on
identical
virtual
machines
with
the
following
specifications:
Microsoft
Windows
7
2GB
RAM
40GB
hard
drive
Browser
machines
were
tested
prior
to,
and
during,
the
test
to
ensure
proper
functionality.
Browsers
were
given
full
access
to
the
Internet
to
enable
them
to
visit
live
sites.
Tested
Browsers
The
browsers,
or
products
under
test,
were
obtained
independently
by
NSS
Labs.
Generally,
available
software
releases
were
used
in
all
cases.
Each
product
was
updated
to
the
most
current
version
available
at
the
time
testing
began.
The
following
is
a
current
list
of
the
web
browsers
that
were
tested:
TM
• Google
Chrome
v15-­‐19
® ®
• Microsoft
Internet
Explorer
9
® ®
• Mozilla
Firefox
v7-­‐13
®
• Safari
v5.
Once
testing
began,
the
product
version
was
monitored
and
new
updates
were
applied
in
a
realistic
patching
methodology.
As
a
new
version
of
a
browser
was
made
publicly
available
during
the
testing
window,
NSS
would
begin
updating
the
test
harness
machines
and
run
both
versions
in
parallel
over
the
course
of
a
two-­‐week
phase-­‐
out
of
the
prior
version
of
the
browser.
This
maintained
the
integrity
of
the
virtual
instances
that
were
under
test
while
allowing
for
fresh
instances
to
start
with
the
new
browser
version.
This
test
relied
upon
Internet
access
for
the
reputation
systems
and
access
to
live
content.
Generally,
there
is
a
configurable
separation
between
software
updates
and
database
or
signature
updates,
to
draw
analogies
from
anti-­‐virus,
intrusion
prevention,
and
general
software
practices.
Network
Description
The
browsers
were
tested
for
their
ability
to
protect
the
client
in
“connected”
use
cases.
Thus,
the
tests
consider
and
analyze
the
effectiveness
of
browser
protection
in
NSS
Labs’
real-­‐world,
live
Internet
testing
harness.
The
host
system
had
one
network
interface
card
(NIC)
and
was
connected
to
the
network
via
a
1Gb
switch
port.
For
the
purposes
of
this
test,
NSS
Labs
utilized
384
desktop
systems
each
running
a
web
browser.
Results
were
recorded
into
a
MySQL
database.
Test
Duration
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
8

9. NSS
Labs
Analysis
Brief
–
Is
Your
Browser
Putting
You
At
Risk?
Part
1
NSS
Labs’
browser
test
was
performed
continuously
(24
x
7)
for
175
days.
Throughout
the
duration
of
the
test,
new
URLs
were
added
as
they
were
discovered.
Test
Frequency
Over
the
course
of
the
test,
each
URL
was
run
through
the
test
harness
every
six
hours.
Regardless
of
success
or
failure,
NSS
Labs
continued
to
attempt
to
download
a
malware
sample
with
the
web
browser
for
the
duration
of
the
test.
Collect New
Suspicious Malicous
Sites from Sources
Pre-Filter, Validate,
Results Collected & Prune & Archive
Archived
Sites
Test Clients Visit Site Distribute to Test
& Record Block/Allow Clients
Sample
Sets
for
Malware
URLs
Freshness
of
malware
sites
is
a
key
attribute
of
this
type
of
test.
In
order
to
utilize
the
freshest,
most
representative
URLs,
NSS
Labs
received
a
broad
range
of
samples
from
a
number
of
different
sources.
Sources
NSS
Labs
operates
its
own
network
of
spam
traps
and
honeypots.
These
e-­‐mail
accounts
with
high-­‐volume
traffic
yield
thousands
of
unique
e-­‐mails
and
URLs
per
day.
In
addition,
NSS
Labs
maintains
relationships
with
other
independent
security
researchers,
networks,
and
security
companies
that
provide
access
to
URLs
and
malicious
content.
Sample
sets
contain
malicious
URLs
distributed
via:
e-­‐mail,
instant
messaging,
social
networks,
and
malicious
websites.
No
content
is
used
from
the
tested
parties.
Malicious
URLs
targeting
users
throughout
the
globe
are
identified
and
selected
for
inclusion
in
this
test.
Users
are
defined
as
individuals
residing
within
the
North
America,
South
American,
European,
and
Asia-­‐Pacific
regions,
including:
Argentina,
Australia,
Austria,
Brazil,
Canada,
China,
France,
Germany,
India,
Italy,
Japan,
Indonesia,
Mexico,
New
Zealand,
Singapore,
Spain,
South
Korea,
Sweden,
Thailand,
the
United
Kingdom,
the
United
States
of
America,
and
Vietnam.
This
report
is
comprised
only
of
data
from
the
United
States
of
America
samples;
future
papers
will
include
the
additional
data.
The
ultimate
determinant
of
whether
or
not
a
malicious
URL
is
included
in
this
test
is
its
participation
in
a
malware
campaign
targeting
users.
Lastly,
just
because
a
malicious
URL
is
included
in
a
campaign
targeting
an
Asia-­‐Pacific
or
a
North
American
user
does
not
mean
that
the
URL
is
not
used
in
other
campaigns
targeting
users
from
other
regions.
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
9

10. NSS
Labs
Analysis
Brief
–
Is
Your
Browser
Putting
You
At
Risk?
Part
1
Exploits
containing
malware
payloads
(exploits
plus
malware),
also
known
as
“clickjacking”
or
“drive-­‐by
downloads”
are
excluded
from
the
test.
Every
effort
is
made
to
consider
submissions
that
reflect
a
real-­‐world
distribution
of
malware—categorically,
geographically,
and
by
platform.
In
addition,
NSS
Labs
maintains
a
collection
of
“clean
URLs”
which
includes
sites
from
Yahoo,
Amazon,
Microsoft,
Google,
NSS
Labs,
major
banks,
and
others.
Periodically,
clean
URLs
are
run
through
the
system
to
verify
that
the
browsers
are
not
over-­‐blocking.
Catalog
URLs
New
sites
are
added
to
the
URL
consideration
set
as
soon
as
possible.
The
date
and
time
each
sample
is
introduced
is
noted.
Most
sources
are
automatically
and
immediately
inserted,
while
some
methods
require
manual
handling
and
can
be
processed
in
under
30
minutes.
All
items
in
the
consideration
set
are
cataloged
with
a
unique
NSS
Labs
ID,
regardless
of
their
validity.
This
enables
correct
tracking
of
effectiveness
of
sample
sources.
Confirm
Sample
Presence
of
URLs
Time
is
of
the
essence
since
the
objective
is
to
test
the
effectiveness
against
the
freshest
possible
malware
sites.
Given
the
nature
of
the
feeds,
and
the
velocity
of
change,
it
is
not
possible
to
validate
each
site
in
depth
before
the
test,
since
the
sites
could
quickly
disappear.
Thus,
each
of
the
test
items
is
given
a
cursory
review
to
verify
it
is
present
and
accessible
on
the
live
Internet.
In
order
to
be
included
in
the
execution
set,
URLs
must
be
live
during
the
test
iteration.
At
the
beginning
of
each
test
cycle,
the
availability
of
the
URL
is
confirmed
by
ensuring
that
the
site
can
be
reached
and
is
active,
such
that
a
non-­‐404
web
page
is
returned.
This
validation
occurs
within
minutes
of
receiving
the
samples
from
NSS
sources.
Note:
These
classifications
are
further
validated
after
the
test,
and
URLs
are
reclassified
and/or
removed
accordingly.
Archive
active
URL
content
The
active
URL
content
is
downloaded
and
saved
to
an
archive
server
with
a
unique
NSS
ID
number.
This
enables
NSS
Labs
to
preserve
the
URL
content
for
control
and
validation
purposes.
Dynamically
Execute
Each
URL
A
client
automation
utility
requests
each
of
the
URLs
deemed
“present”
(based
upon
results
of
the
test
described
in
Section
5.4)
via
each
of
the
web
browsers
in
the
test.
NSS
Labs
records
whether
or
not
the
malware
is
downloaded
and
if
the
download
attempt
triggers
a
warning
from
the
browser’s
malware
protection.
Scoring
and
Recording
the
results
The
resulting
response
is
recorded
as
either
“Allowed”
or
“Blocked
and
Warned.”
Success:
NSS
Labs
defines
success
based
upon
a
web
browser
successfully
preventing
malware
from
being
downloaded
and
correctly
issuing
a
warning.
Failure:
NSS
Labs
defines
a
failure
based
upon
a
web
browser
failing
to
prevent
the
malware
from
being
downloaded
and/or
failing
to
issue
a
warning.
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
10

11. NSS
Labs
Analysis
Brief
–
Is
Your
Browser
Putting
You
At
Risk?
Part
1
Pruning
Throughout
the
test,
lab
engineers
review
and
remove
non-­‐conforming
URLs
and
content
from
the
test
execution
set.
For
example,
a
URL
that
was
initially
classified
as
malware,
but
that
has
since
been
replaced
with
a
generic
splash
page,
will
be
removed
from
the
test.
If
a
URL
sample
becomes
unavailable
for
download
during
the
course
of
the
test,
the
sample
is
removed
from
the
test
collection
for
that
iteration.
NSS
Labs
continually
verifies
each
sample’s
presence
(availability
for
download)
and
adds/removes
each
sample
from
the
test
set
accordingly.
Should
a
malware
sample
be
unavailable
for
a
test
iteration
and
then
become
available
again
for
a
subsequent
iteration,
it
will
be
added
back
into
the
test
collection.
Unavailable
samples
are
not
included
in
calculations
of
success
or
failure
by
a
web
browser.
Post-­‐Test
Validation
Post-­‐test
validation
enables
NSS
Labs
to
reclassify
and
even
remove
samples
that
were
either
not
malicious
or
not
available
before
the
test
started.
NSS
Labs
uses
two
different
commercial
sandboxes
to
prune
and
validate
the
®
malware
(Sunbelt’s
CWSandbox
and
Norman
Analyzer).
Further
validation
is
performed
using
proprietary
tools,
system
instrumentation,
and
code
analysis
as
needed.
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
11

12. NSS
Labs
Analysis
Brief
–
Is
Your
Browser
Putting
You
At
Risk?
Part
1
NSS
Labs
Test
Environment
and
Methodology
NSS
Labs
has
created
a
complex
“live”
test
environment
and
methodology
to
assess
the
protective
capabilities
of
Internet
browsers
under
the
most
real-­‐world
conditions
possible,
while
also
maintaining
control
and
verification
of
the
procedures.
The
purpose
of
the
study
was
to
determine
how
well
current
web
browsers
protect
users
from
the
most
prevalent
malware
threats
on
the
Internet
today.
A
key
aspect
in
any
test
of
this
nature
is
the
timing.
Given
the
rapid
rate
and
aggression
with
which
criminals
propagate
and
manipulate
malicious
websites,
a
key
objective
is
to
ensure
that
the
“freshest”
sites
possible
are
included
in
the
test.
NSS
Labs
has
developed
a
unique
proprietary
“Live
Testing”
harness
and
methodology.
As
part
of
this
methodology,
NSS
Labs
continually
collects
web-­‐based
threats
from
multiple
sources,
including
partners
and
NSS’
own
servers
and
high-­‐interaction
honeynets.
Potential
threats
are
vetted
algorithmically
before
being
inserted
into
the
test
queue;
threats
are
being
inserted
and
vetted
continually.
Unique
in
this
procedure
is
that
NSS
Labs
validates
the
samples
before
and
after
the
test.
Actual
testing
of
the
threats
is
repeated
every
six
hours
and
starts
with
validation
of
the
site’s
existence
and
conformance
to
the
test
definition.
All
tests
are
executed
in
a
highly
controlled
manner,
and
results
are
meticulously
recorded
and
archived
at
each
interval.
Figure
2
-­‐
NSS
Test
Framework
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
12

13. NSS
Labs
Analysis
Brief
–
Is
Your
Browser
Putting
You
At
Risk?
Part
1
Contact
Information
NSS
Labs,
Inc.
6207
Bee
Caves
Road,
Suite
350
Austin,
TX
78746
USA
+1
(512)
961-­‐5300
info@nsslabs.com
www.nsslabs.com
This
analysis
brief
was
produced
as
part
of
NSS
Labs’
independent
testing
information
services.
Leading
products
were
tested
at
no
cost
to
the
vendor,
and
NSS
Labs
received
no
vendor
funding
to
produce
this
analysis
brief.
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
No
part
of
this
publication
may
be
reproduced,
photocopied,
stored
on
a
retrieval
system,
or
transmitted
without
the
express
written
consent
of
the
authors.
Please
note
that
access
to
or
use
of
this
report
is
conditioned
on
the
following:
1.
The
information
in
this
report
is
subject
to
change
by
NSS
Labs
without
notice.
2.
The
information
in
this
report
is
believed
by
NSS
Labs
to
be
accurate
and
reliable
at
the
time
of
publication,
but
is
not
guaranteed.
All
use
of
and
reliance
on
this
report
are
at
the
reader’s
sole
risk.
NSS
Labs
is
not
liable
or
responsible
for
any
damages,
losses,
or
expenses
arising
from
any
error
or
omission
in
this
report.
3.
NO
WARRANTIES,
EXPRESS
OR
IMPLIED
ARE
GIVEN
BY
NSS
LABS.
ALL
IMPLIED
WARRANTIES,
INCLUDING
IMPLIED
WARRANTIES
OF
MERCHANTABILITY,
FITNESS
FOR
A
PARTICULAR
PURPOSE,
AND
NON-­‐INFRINGEMENT
ARE
DISCLAIMED
AND
EXCLUDED
BY
NSS
LABS.
IN
NO
EVENT
SHALL
NSS
LABS
BE
LIABLE
FOR
ANY
CONSEQUENTIAL,
INCIDENTAL
OR
INDIRECT
DAMAGES,
OR
FOR
ANY
LOSS
OF
PROFIT,
REVENUE,
D ATA,
COMPUTER
PROGRAMS,
OR
OTHER
ASSETS,
EVEN
IF
ADVISED
OF
THE
POSSIBILITY
THEREOF.
4.
This
report
does
not
constitute
an
endorsement,
recommendation,
or
guarantee
of
any
of
the
products
(hardware
or
software)
tested
or
the
hardware
and
software
used
in
testing
the
products.
The
testing
does
not
guarantee
that
there
are
no
errors
or
defects
in
the
products
or
that
the
products
will
meet
the
reader’s
expectations,
requirements,
needs,
or
specifications,
or
that
they
will
operate
without
interruption.
5.
This
report
does
not
imply
any
endorsement,
sponsorship,
affiliation,
or
verification
by
or
with
any
organizations
mentioned
in
this
report.
6.
All
trademarks,
service
marks,
and
trade
names
used
in
this
report
are
the
trademarks,
service
marks,
and
trade
names
of
their
respective
owners.
©
2012
NSS
Labs,
Inc.
All
rights
reserved.
13