This document summarizes the results of testing the abilities of web browsers like Firefox, Chrome, and Edge to block socially engineered malware (SEM) and phishing attempts. The testing found that while all browsers improved over time, Microsoft Edge with its AppRep reputation system had the highest block rates for both SEM and phishing, blocking over 99% of SEM and 91.4% of phishing attempts on average. It also blocked threats the fastest, within an average of 0.16 hours for SEM and 0.4 hours for phishing. The document concludes reputation-based systems in browsers can help protect users from web-based threats.

1. Web Browser Security
Socially Engineered Malware and Phishing
@nsslabs
Thomas Skybakmoen | Dis;nguished Research Director, NSS Labs
Jayendra Pathak | Chief Architect, NSS Labs, Inc.

2. 2
Who is NSS Labs?
Research & Advisory
• Solu;on trends
• Best prac;ce solu;on
architecture guidance
• Analyst inquiries
• Security advisory days
• Webinars/educa;on
Objec3ve Purchase
Insight
• Product modeling
• RFP templates
• TCO modeling kits
Security Vendor Tes3ng
• Security efficacy
• Solu;on performance
• Cost of ownership
Cyber Advanced
Warning System™
• Con;nuous exploit visibility
• Con;nuous target asset
iden;fica;on
• Con;nuous security
measurement
• Product compara;ves
• SaaS or API

3. 3
NSS Labs Testing: Timeline and Process
• Coverage and tests are growing – 10+ years of security
tes;ng
• 2016 – 6+ tests, 40+ vendors, 40+ devices
• Workflow for test development:
1. Market assessment
2. Primary research
3. Enterprise planning
4. Methodology
5. Test harness development
6. Group test, aggregate, review
7. Publish results

4. 4
Socially Engineered Malware (SEM)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Q1 2009 Q2 2009 Q1 2010 Q3 2010 Q3 2011 Q3 2012 Q1 2013 Q1 2014 Q4 2016
Microsoft Mozilla Firefox Google Chrome
• What is SEM?
• Historical trends

5. 5
Phishing
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
2009 2012 2013 2016
Microsoft Mozilla Firefox Google Chrome
• What is phishing?
• Historical trends

6. 6
What is CAWS?
The CAWS (Cyber Advanced Warning System) pladorm enables
con3nuous valida3on of the collec;ve effec;veness of layered
network security defenses, revealing the security posture in real 2me.
ADAPT
Con2nuously validate
the effec;veness of your
defenses in real ;me.
PRIORITIZE
Focus your efforts
on threats that mafer to your
specific environment.
RESPOND
Act with precision
using validated, contextual
threat details and metadata.
IDENTIFY
Pinpoint your exposure
to exploits that are ac;ve in
the wild right now.

7. 7
2 | Exploit Harves3ng
Vic;m machines are
commanded to visit malicious
sites and then exploited.
Exploit interac;on is recorded
in detail.
4 | Exploit Replay
Exploits are replayed against
customer profile to test efficacy
of security products.
Customer gets real-;me,
validated results of risk posture.
5 | Real-3me Security Posture
1) How are my defenses performing?
2) Where am I exposed so I can focus
my efforts?
3) What are the cri;cal threat details
that will help me avoid a breach?
Cyber Advanced Warning System – How it Works
3 | Customer Profile
Customer selects the
applica;ons and versions
present in its environment.
Customer selects the defenses it
has in place.
NSS BaitNET™
Mimicked Customer Environment
NSS Virtual Infrastructure
1 | Exploit Source Capture
Malicious URLs and IP
addresses are collected,
analyzed, and de-duped
NSS Labs
NSS Unique Intelligence
How CAWS Works

8. 8
Why is Testing Important?
• Evaluate the efficacy of a browser reputa;on system.
o Browsers are the first line of defense against web-borne threats.
o Browsers reputa;on systems protect users from themselves. (Don’t
download free apps that are actually malware.)
• Can a browser reputa;on system replace an an;virus (AV)
product to protect against web-borne threats?

9. 9
SEM: Average Block Rate
78.3%
85.8%
99.0%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mozilla Firefox
Google Chrome (w/Download Protection)
Micosoft Edge w/AppRep

10. 10
SEM: Zero-Hour Protection
0-hr 1d 2d 3d 4d 5d 6d 7d Total
Firefox 78.3% 81.6% 81.9% 81.9% 81.9% 81.9% 81.9% 81.9% 81.9%
Microsoft Edge 98.7% 99.0% 99.3% 99.3% 99.3% 99.3% 99.3% 99.3% 99.3%
Chrome (w/Download Protection) 92.8% 94.4% 95.1% 95.4% 95.4% 95.7% 95.7% 95.7% 95.7%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Coverage

11. 11
SEM: Average Time to Block
3.76
2.66
0.16
0 1 2 3 4
Firefox
Google Chrome (w/Download Protection)
Microsoft Edge w/AppRep
Hours

12. 12
SEM: Consistency of Protection
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Google Chrome (w/Download Protection) Mozilla Firefox Microsoft Edge w/AppRep Test Average

13. 13
Phishing: Average Block Rate
81.4%
82.4%
91.4%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Mozilla Firefox
Google Chrome
Microsoft Edge

14. 14
Phishing: Response Time
0-hr 1d 2d 3d 4d 5d 6d 7d Total
Google Chrome 82.7% 85.6% 85.6% 85.6% 85.6% 85.6% 85.6% 85.6% 85.6%
Microsoft Edge 92.1% 92.9% 92.9% 92.9% 92.9% 92.9% 92.9% 92.9% 92.9%
Mozilla Firefox 84.0% 84.9% 84.9% 84.9% 84.9% 84.9% 84.9% 84.9% 84.9%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Coverage

15. 15
Phishing: Average Time to Block
1.41
1.02
0.40
0.0 0.5 1.0 1.5
Google Chrome
Mozilla Firefox
Microsoft Edge
Hours

16. 16
Phishing: Protection over Time
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Google Chrome Microsoft Edge Mozilla Firefox

17. Thank you
Ques3ons? info@nsslabs.com