2012 ab is-your-browser-putting-you-at-risk

1,096 views
1,012 views

Published on

safe browser

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,096
On SlideShare
0
From Embeds
0
Number of Embeds
683
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

2012 ab is-your-browser-putting-you-at-risk

  1. 1.  ANALYSIS  BRIEF  –  September  2012  IS  YOUR  BROWSER  PUTTING  YOU  AT  RISK?    PART  1  –  GENERAL  MALWARE  BLOCKING    Authors  -­‐  Bob  Walder,  Francisco  Artes,  Stefan  Frei,  Ken  Baylor,  Jayendra  Pathak,  Vikram  Phatak    Overview  The  ineffectiveness  of  Web  browser  security  is  one  of  the  most  common  reasons  for  malware  infection.  Browsers  offer  a  direct  and  unique  route  for  infection,  bypassing  corporate  protection  layers  and  bringing  malware  deep  into  the  corporate  environment,  often  protecting  it  from  detection  using  SSL.  Browsers  must  provide  a  strong  layer  of  defense  from  malware,  rather  than  defer  to  operating  system  antimalware  solutions.  This  series  examines  the  effectiveness  of  leading  browsers  to  block  malware.  The  four  leading  browsers  were  tested  against  three  million  samples  of  real  world  malicious  software.  Major  discrepancies  were  noted  in  their  ability  to  block  malware.  Data  represented  in  this  report  was  captured  over  one  hundred  and  seventy-­‐five  (175)  days  through  NSS  Labs’  unique  live  testing  harness,  and  provides  in-­‐depth  insight  into  the  built-­‐in  protection  capabilities  of  modern  browsers,  including  Chrome,  Firefox,  Internet  Explorer,  and  Safari.  This  series  of  papers  will  examine  the  ability  of  the  four  leading  browsers  to  block  each  of  the  five  main  purposes  of  malware  and  malware  monetization.  Monetization  of  malware  is  achieved  by  multiple  means,  including  click  fraud,  fake  antivirus,  account  /  password  theft,  bank/financial  fraud,  and  gaming  fraud.    Collectively  they  account  for  billions  of  dollars  worth  of  corporate  and  consumer  theft  per  year,  yet  browsers  vary  widely  in  their  ability  to  block  malware,  despite  adverse  effects  on  business  and  individual  users  alike.  Tested  Products   • Apple  Safari  5   • Google  Chrome  15  -­‐  19     • Microsoft  Internet  Explorer  9   • Mozilla  Firefox  7  –  13  Over  3,000,000  test  cases  were  used  in  the  data  sampling  captured  via  NSS  Labs’  unique  live  testing  harness.    An  initial  sample  set  of  227,841  unique  and  suspicious  URLs  entered  the  system;  84,396  were  found  active  and  malicious  and  met  the  criteria  for  entry  into  the  test.  In  total  3,038,324  test  runs  were  performed  by  the  four  browsers  against  these  unique  84,396  URLs  –  resulting  in  over  750,000  tests  cases  per  browser.  
  2. 2. NSS  Labs   Analysis  Brief  –  Is  Your  Browser  Putting  You  At  Risk?  Part  1    Testing  was  repeated  every  six  (6)  hours  until  the  target  URL  was  no  longer  active.  Samples  that  did  not  pass  the  validation  criteria  were  removed,  including  false  positives  and  adware.  Ultimately,  1,407,233  URL  test  cases  passed  the  post-­‐validation  process  and  are  included  in  the  results.    Each  sample  payload  was  validated  internally.    MD5  hashes  of  samples  were  submitted  to  VirusTotal  and  the  resulting  scanner  reports  were  then  used  to  classify  malware  types.  Additionally,  the  test  samples  were  verified  by  multiple  independent  external  sources  to  confirm  distribution  accuracy  and  malware  classification.     100% 80% 60% Firefox 40% Chrome Internet Explorer 20% Safari 0%   Figure 1 – Malware Block Rate Over Time with 10-Day Moving Average (higher % is better)During  the  testing  period,  Internet  Explorer  maintained  a  malware  block  rate  of  95%  while  Firefox  and  Safari’s  block  rate  remained  just  under  6%.  Over  the  same  time  period,  Chrome’s  block  rate  varied  from  13%  to  just  over  74%.  This  could  be  attributed  to  changing  protection  tactics  over  time  that  is  indicative  of  the  ongoing  battle  between  antimalware  developers  and  malicious  actors.    NSS  Lab  Findings:   • Browsers  offer  the  largest  attack  surface  in  most  enterprise  networks  and  are  the  most  common  vector   for  malware  installations     • The  use  of  SSL  by  browsers  presents  additional  problems  to  enterprises  since  it  offers  the  opportunity  to   bypass  many  layers  of  corporate  security  protection   • The  leading  browsers  show  a  significant  variance  in  their  ability  to  block  malware.   • Given  the  increasing  mobility  of  users  and  devices,  blocking  malware  is  not  only  extremely  important,  but   potentially  the  only  means  of  reducing  risk  when  outside  of  the  corporate  perimeter  of  protection.     • Web  browsing  is  the  primary  attack  vector  of  criminals  attempting  to  monetize  malware,  using  a  variety   of  means,  including  click  fraud,  fake  antivirus,  account  /  password  theft,  bank/financial  fraud,  and  gaming   fraud.      ©  2012  NSS  Labs,  Inc.  All  rights  reserved.     2      
  3. 3. NSS  Labs   Analysis  Brief  –  Is  Your  Browser  Putting  You  At  Risk?  Part  1     • The  tolerance  of  browsers  with  low  malware  block  rates  may  present  undue  risk  to  an  organization.    NSS  Labs  Recommendations:   • Users  should  evaluate  browser  security  as  part  of  their  layered  security  strategy.     • Enterprises  should  perform  a  risk  analysis  of  the  browsers  in  the  organization  and  remove  those  with   unjustified  high  risk  where  possible.   • Enterprise  and  individual  users  should  use  the  findings  in  this  report  to  assist  in  the  selection  of  the   browser  most  appropriate  to  their  protection  needs.  However,  malware  infection  rather  than  exploits   were  the  subject  of  this  test,  and  readers  should  not  draw  conclusions  based  upon  this  analysis  brief   alone.    Analysis  As  the  most  widely  used  and  ubiquitous  means  of  accessing  the  Internet,  web  browsers  are  uniquely  positioned  to  filter  and  stop  malware  at  an  early  stage.  This  capability  becomes  even  more  important  given  the  increasing  mobility  of  devices,  which  means  corporate  perimeter  and  network  protection  services  cannot  always  be  relied  upon.  To  complement  traditional  defenses  and  to  address  the  highly  dynamic  nature  of  current  attacks  and  attack  distribution  methods,  modern  web  browsers  employ  technologies  to  block  access  to  malicious  URLs  before  loading  the  content.  Blocking  access  to  malicious  URLs  is  a  formidable  first  line  of  defense,  since  it  provides  complete  protection  against  malware  entering  the  system.  However,  little  is  known  or  published  on  the  effectiveness  of  web  browser’s  internal  blocking  technology  and  performance.  This  analysis  examines  the  ability  of  four  different  web  browsers  to  protect  users  from  malware  downloads,  also  known  as  socially-­‐engineered  malware.1  Modern  web  browsers  offer  an  added  layer  of  protection  against  these  threats  by  leveraging  in-­‐the-­‐cloud,  reputation-­‐based  mechanisms  to  warn  users  of  potential  infection.  However,  not  all  vendors  have  taken  the  same  approach.      Browser  protection  contains  two  main  functional  components.  The  foundation  is  an  “in-­‐the-­‐cloud”  reputation-­‐based  system  which  scours  the  Internet  for  malicious  web  sites  and  categorizes  content  accordingly,  either  by  adding  it  to  a  black  or  white  list,  or  assigning  a  score  (depending  on  the  vendor’s  approach.)  This  categorization  may  be  performed  manually,  automatically,  or  using  both  methods.  Some  vendors  will  utilize  feedback  from  user  agents  on  their  customers’  endpoints  to  report  back  to  the  reputation  system  automatically,  providing  information  relevant  to  the  trustworthiness,  or  otherwise,  of  applications  and  files  downloaded  from  the  Internet.  The  second  functional  component  resides  within  the  web  browser  itself,  and  requests  reputation  information  from  the  in-­‐the-­‐cloud  systems  about  specific  URLs  and  then  enforces  warning  and  blocking  functions.                                                                                                                                      1 Exploits that install malware without the user being aware (also referred to as “drive-by downloads”) are not included in this particular study.©  2012  NSS  Labs,  Inc.  All  rights  reserved.     3      
  4. 4. NSS  Labs   Analysis  Brief  –  Is  Your  Browser  Putting  You  At  Risk?  Part  1    When  results  are  returned  that  a  site  is  “bad,”  the  web  browser  redirects  the  user  to  a  warning  message  or  page  informing  that  the  URL  is  malicious.  In  the  event  that  the  URL  links  to  a  download,  the  web  browser  instructs  the  user  that  the  content  is  likely  malicious  and  that  the  download  should  be  cancelled.  Conversely,  when  a  website  is  determined  to  be  “good,”  the  web  browser  takes  no  action  and  the  user  is  unaware  that  a  security  check  was  performed.                      Internet  Explorer  Warning                                                                                                                Chrome  Warning                       Firefox  Warning                               Safari  Warning   Figure  1  –  Browser  WarningsFunctionality  unique  to  Chrome  NSS  Labs  determined  that  Safe  Browsing  API  v2  includes  additional  functionality  that  has  been  integrated  into  Chrome,  but  not  Firefox  or  Safari.    This  functionality  provides  reputation  services  for  executable  files,  or  as  Google  describes  them  “malicious  downloads”.     Figure  2  -­‐  Chrome  Safe  Browsing  Warning  ©  2012  NSS  Labs,  Inc.  All  rights  reserved.     4      
  5. 5. NSS  Labs   Analysis  Brief  –  Is  Your  Browser  Putting  You  At  Risk?  Part  1    Malware  Block  Performance  Each  browser’s  individual  block  performance  was  tracked  over  time  and  mapped  by  malware  purpose.  When  aggregated  an  overall  block  rate  of  all  collected  malware  by  browser  was  developed.    A  browser’s  overall  block  rate  is  defined  as  the  percentage  of  successful  blocks  divided  by  the  total  number  of  test  cases.  With  tests  conducted  every  6  hours,  a  URL  that  was  online  for  48  hours  will  be  tested  8  times.  A  browser  blocking  it  on  6  (out  of  a  maximum  8)  test  runs  will  achieve  a  block  rate  of  75%.  Figure  3  shows  the  overall  block  performance  of  the  four  browsers  tested.  As  expected,  since  Firefox  and  Safari  using  the  same  technology  they  achieve  similar  block  rates.  However,  the  large  difference  of  the  average  block  rate  between  browsers  is  noteworthy,  with  results  ranging  from  4.7%  up  to  94%.   Chrome 27.6% Firefox 5.0% Internet Explorer 94.0% Safari 4.7% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%   Figure  3  –  Overall  Malware  Block  Rate  by  Browser  (higher  %  is  better)  To  assess  the  effectiveness  of  different  blocking  technologies,  the  NSS  test  harness  also  records  the  mechanism  that  blocked  access  to  a  URL.  Of  the  three  browsers  using  Google’s  Safe  Browsing  API,  Chrome  is  the  only  one  to  also  utilize  Google’s  malicious  download  technology.    Figure  4  shows  the  block  performance  of  the  URL  blocking  component  and  the  additional  download  block  component  used  only  by  Google’s  Chrome.  The  URL  blocking  performance  of  these  three  Safe  Browsing  browsers  was  consistent  at  around  5%.  Google’s  malicious  download  protection  proved  to  be  almost  five  times  more  effective  than  URL  blocking  alone.    As  seen  in  Figure  ,  it  increases  overall  blocking  performance  by  28%  compared  to  URL  blocking  alone,  and  accounts  for  the  majority  of  the  blocking  performance  of  Google  Chrome.    The  core  protection  technology  in  Internet  Explorer  is  SmartScreen,  which  provides  URL-­‐based  protection  from  attacks  via  an  integrated  cloud-­‐based  URL-­‐reputation  service.  SmartScreen  also  works  with  Download  Manager  to  prevent  malicious  downloads.    ©  2012  NSS  Labs,  Inc.  All  rights  reserved.     5      
  6. 6. NSS  Labs   Analysis  Brief  –  Is  Your  Browser  Putting  You  At  Risk?  Part  1     Chrome! 4.6% 23.0% Firefox! 5.0% Internet Explorer! 94.0% Safari! 4.7% 0%! 10%! 20%! 30%! 40%! 50%! 60%! 70%! 80%! 90%! 100%! Safari! Internet Explorer! Firefox! Chrome! SmartScreen! 94.0%! SafeBrowsing! 4.7%! 5.0%! 4.6%! Malicious Download! 0.0%! 0.0%! 23.0%!   Figure  4  –  Blocking  technologies  used  by  browsers  (higher  %  is  better)  Time  to  block  Malicious  Sites    Every  time  a  new  campaign  is  launched  by  malicious  actors,  it  is  vital  that  it  is  detected  as  quickly  as  possible  by  security  solutions  deployed  in  the  enterprise.  The  following  response  time  graph  shows  how  long  it  took  each  of  the  browsers  to  block  a  threat  once  it  was  introduced  into  the  test  cycle.  Cumulative  protection  rates  are  calculated  each  day  until  blocked.             100% 90% 80% 70% 60% Block Rate Internet Explorer 50% Chrome 40% Firefox 30% Safari 20% 10% 0% 0 5 10 15 20 25 30 Days   Figure  5  -­‐  Time  to  Block  Malicious  Sites  ©  2012  NSS  Labs,  Inc.  All  rights  reserved.     6      
  7. 7. NSS  Labs   Analysis  Brief  –  Is  Your  Browser  Putting  You  At  Risk?  Part  1     Days   Firefox   Chrome   Internet  Explorer   Safari   1   4%   20%   91%   4%   2   5%   22%   92%   4%   3   5%   23%   92%   4%   4   5%   24%   92%   4%   5   5%   25%   93%   4%   6   5%   25%   93%   5%   7   5%   26%   93%   5%   10   5%   27%   93%   5%   15   5%   28%   94%   5%   20   5%   28%   94%   5%   25   5%   28%   94%   5%   30   5%   28%   94%   5%   Table  1-­‐  Time  to  Block  Malicious  Sites    Ultimately,  the  results  reveal  significant  variations  in  the  abilities  of  the  browsers  to  protect  against  malware.  Chrome  provides  more  protection  than  Safari  or  Firefox  using  the  Safe  Browsing  feed,  apparently  due  to  its  malicious  download  protection.    Trends  show  minor  differences  between  Firefox  and  Safari.      Results  from  these  tests  indicate  that  the  four  browsers  vary  both  in  their  approach  and  effectiveness  in  blocking  different  malware  categories.  It  was  decided  to  further  categorize  the  malware  behind  the  suspicious  URLs  to  measure  the  browser’s  block  performance  for  each  class  of  malware.  The  ability  of  the  four  leading  browsers  to  block  each  of  the  five  main  purposes  of  malware:  click  fraud,  banking/financial  fraud,  fake  antivirus,  password/account  theft  and  game  fraud  was  examined  and  will  be  detailed  in  subsequent  papers  in  this  series.    Reading  List  Analysis  Brief:  Did  Google  Pull  a  Fast  One  on  Firefox  and  Safari  Users?    ©  2012  NSS  Labs,  Inc.  All  rights  reserved.     7      
  8. 8. NSS  Labs   Analysis  Brief  –  Is  Your  Browser  Putting  You  At  Risk?  Part  1    Appendix  A  –  Methodology  Client  Host  Description  All  tested  browser  software  was  installed  on  identical  virtual  machines  with  the  following  specifications:     Microsoft  Windows  7   2GB  RAM   40GB  hard  drive  Browser  machines  were  tested  prior  to,  and  during,  the  test  to  ensure  proper  functionality.  Browsers  were  given  full  access  to  the  Internet  to  enable  them  to  visit  live  sites.    Tested  Browsers  The  browsers,  or  products  under  test,  were  obtained  independently  by  NSS  Labs.  Generally,  available  software  releases  were  used  in  all  cases.  Each  product  was  updated  to  the  most  current  version  available  at  the  time  testing  began.  The  following  is  a  current  list  of  the  web  browsers  that  were  tested:     TM • Google  Chrome  v15-­‐19   ® ® • Microsoft  Internet  Explorer  9     ® ® • Mozilla  Firefox  v7-­‐13   ® • Safari  v5.  Once  testing  began,  the  product  version  was  monitored  and  new  updates  were  applied  in  a  realistic  patching  methodology.  As  a  new  version  of  a  browser  was  made  publicly  available  during  the  testing  window,  NSS  would  begin  updating  the  test  harness  machines  and  run  both  versions  in  parallel  over  the  course  of  a  two-­‐week  phase-­‐out  of  the  prior  version  of  the  browser.    This  maintained  the  integrity  of  the  virtual  instances  that  were  under  test  while  allowing  for  fresh  instances  to  start  with  the  new  browser  version.  This  test  relied  upon  Internet  access  for  the  reputation  systems  and  access  to  live  content.  Generally,  there  is  a  configurable  separation  between  software  updates  and  database  or  signature  updates,  to  draw  analogies  from  anti-­‐virus,  intrusion  prevention,  and  general  software  practices.    Network  Description  The  browsers  were  tested  for  their  ability  to  protect  the  client  in  “connected”  use  cases.  Thus,  the  tests  consider  and  analyze  the  effectiveness  of  browser  protection  in  NSS  Labs’  real-­‐world,  live  Internet  testing  harness.  The  host  system  had  one  network  interface  card  (NIC)  and  was  connected  to  the  network  via  a  1Gb  switch  port.  For  the  purposes  of  this  test,  NSS  Labs  utilized  384  desktop  systems  each  running  a  web  browser.  Results  were  recorded  into  a  MySQL  database.  Test  Duration  ©  2012  NSS  Labs,  Inc.  All  rights  reserved.     8      
  9. 9. NSS  Labs   Analysis  Brief  –  Is  Your  Browser  Putting  You  At  Risk?  Part  1    NSS  Labs’  browser  test  was  performed  continuously  (24  x  7)  for  175  days.  Throughout  the  duration  of  the  test,  new  URLs  were  added  as  they  were  discovered.  Test  Frequency    Over  the  course  of  the  test,  each  URL  was  run  through  the  test  harness  every  six  hours.  Regardless  of  success  or  failure,  NSS  Labs  continued  to  attempt  to  download  a  malware  sample  with  the  web  browser  for  the  duration  of  the  test.   Collect New Suspicious Malicous Sites from Sources Pre-Filter, Validate, Results Collected & Prune & Archive Archived Sites Test Clients Visit Site Distribute to Test & Record Block/Allow Clients  Sample  Sets  for  Malware  URLs  Freshness  of  malware  sites  is  a  key  attribute  of  this  type  of  test.  In  order  to  utilize  the  freshest,  most  representative  URLs,  NSS  Labs  received  a  broad  range  of  samples  from  a  number  of  different  sources.    Sources  NSS  Labs  operates  its  own  network  of  spam  traps  and  honeypots.  These  e-­‐mail  accounts  with  high-­‐volume  traffic  yield  thousands  of  unique  e-­‐mails  and  URLs  per  day.  In  addition,  NSS  Labs  maintains  relationships  with  other  independent  security  researchers,  networks,  and  security  companies  that  provide  access  to  URLs  and  malicious  content.  Sample  sets  contain  malicious  URLs  distributed  via:  e-­‐mail,  instant  messaging,  social  networks,  and  malicious  websites.  No  content  is  used  from  the  tested  parties.      Malicious  URLs  targeting  users  throughout  the  globe  are  identified  and  selected  for  inclusion  in  this  test.    Users  are  defined  as  individuals  residing  within  the  North  America,  South  American,  European,  and  Asia-­‐Pacific  regions,  including:  Argentina,  Australia,  Austria,  Brazil,  Canada,  China,  France,  Germany,  India,  Italy,  Japan,  Indonesia,  Mexico,  New  Zealand,  Singapore,  Spain,  South  Korea,  Sweden,  Thailand,  the  United  Kingdom,  the  United  States  of  America,  and  Vietnam.    This  report  is  comprised  only  of  data  from  the  United  States  of  America  samples;  future  papers  will  include  the  additional  data.  The  ultimate  determinant  of  whether  or  not  a  malicious  URL  is  included  in  this  test  is  its  participation  in  a  malware  campaign  targeting  users.    Lastly,  just  because  a  malicious  URL  is  included  in  a  campaign  targeting  an  Asia-­‐Pacific  or  a  North  American  user  does  not  mean  that  the  URL  is  not  used  in  other  campaigns  targeting  users  from  other  regions.  ©  2012  NSS  Labs,  Inc.  All  rights  reserved.     9      
  10. 10. NSS  Labs   Analysis  Brief  –  Is  Your  Browser  Putting  You  At  Risk?  Part  1    Exploits  containing  malware  payloads  (exploits  plus  malware),  also  known  as  “clickjacking”  or  “drive-­‐by  downloads”  are  excluded  from  the  test.  Every  effort  is  made  to  consider  submissions  that  reflect  a  real-­‐world  distribution  of  malware—categorically,  geographically,  and  by  platform.      In  addition,  NSS  Labs  maintains  a  collection  of  “clean  URLs”  which  includes  sites  from  Yahoo,  Amazon,  Microsoft,  Google,  NSS  Labs,  major  banks,  and  others.  Periodically,  clean  URLs  are  run  through  the  system  to  verify  that  the  browsers  are  not  over-­‐blocking.  Catalog  URLs  New  sites  are  added  to  the  URL  consideration  set  as  soon  as  possible.  The  date  and  time  each  sample  is  introduced  is  noted.  Most  sources  are  automatically  and  immediately  inserted,  while  some  methods  require  manual  handling  and  can  be  processed  in  under  30  minutes.  All  items  in  the  consideration  set  are  cataloged  with  a  unique  NSS  Labs  ID,  regardless  of  their  validity.  This  enables  correct  tracking  of  effectiveness  of  sample  sources.  Confirm  Sample  Presence  of  URLs  Time  is  of  the  essence  since  the  objective  is  to  test  the  effectiveness  against  the  freshest  possible  malware  sites.  Given  the  nature  of  the  feeds,  and  the  velocity  of  change,  it  is  not  possible  to  validate  each  site  in  depth  before  the  test,  since  the  sites  could  quickly  disappear.  Thus,  each  of  the  test  items  is  given  a  cursory  review  to  verify  it  is  present  and  accessible  on  the  live  Internet.    In  order  to  be  included  in  the  execution  set,  URLs  must  be  live  during  the  test  iteration.  At  the  beginning  of  each  test  cycle,  the  availability  of  the  URL  is  confirmed  by  ensuring  that  the  site  can  be  reached  and  is  active,  such  that  a  non-­‐404  web  page  is  returned.  This  validation  occurs  within  minutes  of  receiving  the  samples  from  NSS  sources.  Note:  These  classifications  are  further  validated  after  the  test,  and  URLs  are  reclassified  and/or  removed  accordingly.  Archive  active  URL  content  The  active  URL  content  is  downloaded  and  saved  to  an  archive  server  with  a  unique  NSS  ID  number.  This  enables  NSS  Labs  to  preserve  the  URL  content  for  control  and  validation  purposes.    Dynamically  Execute  Each  URL  A  client  automation  utility  requests  each  of  the  URLs  deemed  “present”  (based  upon  results  of  the  test  described  in  Section  5.4)  via  each  of  the  web  browsers  in  the  test.  NSS  Labs  records  whether  or  not  the  malware  is  downloaded  and  if  the  download  attempt  triggers  a  warning  from  the  browser’s  malware  protection.  Scoring  and  Recording  the  results  The  resulting  response  is  recorded  as  either  “Allowed”  or  “Blocked  and  Warned.”    Success:  NSS  Labs  defines  success  based  upon  a  web  browser  successfully  preventing  malware  from  being  downloaded  and  correctly  issuing  a  warning.  Failure:  NSS  Labs  defines  a  failure  based  upon  a  web  browser  failing  to  prevent  the  malware  from  being  downloaded  and/or  failing  to  issue  a  warning.  ©  2012  NSS  Labs,  Inc.  All  rights  reserved.     10      
  11. 11. NSS  Labs   Analysis  Brief  –  Is  Your  Browser  Putting  You  At  Risk?  Part  1    Pruning  Throughout  the  test,  lab  engineers  review  and  remove  non-­‐conforming  URLs  and  content  from  the  test  execution  set.  For  example,  a  URL  that  was  initially  classified  as  malware,  but  that  has  since  been  replaced  with  a  generic  splash  page,  will  be  removed  from  the  test.  If  a  URL  sample  becomes  unavailable  for  download  during  the  course  of  the  test,  the  sample  is  removed  from  the  test  collection  for  that  iteration.  NSS  Labs  continually  verifies  each  sample’s  presence  (availability  for  download)  and  adds/removes  each  sample  from  the  test  set  accordingly.  Should  a  malware  sample  be  unavailable  for  a  test  iteration  and  then  become  available  again  for  a  subsequent  iteration,  it  will  be  added  back  into  the  test  collection.  Unavailable  samples  are  not  included  in  calculations  of  success  or  failure  by  a  web  browser.  Post-­‐Test  Validation  Post-­‐test  validation  enables  NSS  Labs  to  reclassify  and  even  remove  samples  that  were  either  not  malicious  or  not  available  before  the  test  started.  NSS  Labs  uses  two  different  commercial  sandboxes  to  prune  and  validate  the   ®malware  (Sunbelt’s  CWSandbox  and  Norman  Analyzer).  Further  validation  is  performed  using  proprietary  tools,  system  instrumentation,  and  code  analysis  as  needed.      ©  2012  NSS  Labs,  Inc.  All  rights  reserved.     11      
  12. 12. NSS  Labs   Analysis  Brief  –  Is  Your  Browser  Putting  You  At  Risk?  Part  1    NSS  Labs  Test  Environment  and  Methodology  NSS  Labs  has  created  a  complex  “live”  test  environment  and  methodology  to  assess  the  protective  capabilities  of  Internet  browsers  under  the  most  real-­‐world  conditions  possible,  while  also  maintaining  control  and  verification  of  the  procedures.  The  purpose  of  the  study  was  to  determine  how  well  current  web  browsers  protect  users  from  the  most  prevalent  malware  threats  on  the  Internet  today.  A  key  aspect  in  any  test  of  this  nature  is  the  timing.  Given  the  rapid  rate  and  aggression  with  which  criminals  propagate  and  manipulate  malicious  websites,  a  key  objective  is  to  ensure  that  the  “freshest”  sites  possible  are  included  in  the  test.  NSS  Labs  has  developed  a  unique  proprietary  “Live  Testing”  harness  and  methodology.  As  part  of  this  methodology,  NSS  Labs  continually  collects  web-­‐based  threats  from  multiple  sources,  including  partners  and  NSS’  own  servers  and  high-­‐interaction  honeynets.  Potential  threats  are  vetted  algorithmically  before  being  inserted  into  the  test  queue;  threats  are  being  inserted  and  vetted  continually.  Unique  in  this  procedure  is  that  NSS  Labs  validates  the  samples  before  and  after  the  test.  Actual  testing  of  the  threats  is  repeated  every  six  hours  and  starts  with  validation  of  the  site’s  existence  and  conformance  to  the  test  definition.    All  tests  are  executed  in  a  highly  controlled  manner,  and  results  are  meticulously  recorded  and  archived  at  each  interval.   Figure  2  -­‐  NSS  Test  Framework  ©  2012  NSS  Labs,  Inc.  All  rights  reserved.     12      
  13. 13. NSS  Labs   Analysis  Brief  –  Is  Your  Browser  Putting  You  At  Risk?  Part  1    Contact  Information  NSS  Labs,  Inc.  6207  Bee  Caves  Road,  Suite  350  Austin,  TX  78746  USA  +1  (512)  961-­‐5300  info@nsslabs.com  www.nsslabs.com      This  analysis  brief  was  produced  as  part  of  NSS  Labs’  independent  testing  information  services.  Leading  products  were  tested  at  no  cost  to  the  vendor,  and  NSS  Labs  received  no  vendor  funding  to  produce  this  analysis  brief.  ©  2012  NSS  Labs,  Inc.  All  rights  reserved.  No  part  of  this  publication  may  be  reproduced,  photocopied,  stored  on  a  retrieval    system,  or  transmitted  without  the  express  written  consent  of  the  authors.      Please  note  that  access  to  or  use  of  this  report  is  conditioned  on  the  following:    1.    The  information  in  this  report  is  subject  to  change  by  NSS  Labs  without  notice.    2.    The  information  in  this  report  is  believed  by  NSS  Labs  to  be  accurate  and  reliable  at  the  time  of  publication,  but  is  not  guaranteed.  All  use  of  and  reliance  on  this  report  are  at  the  reader’s  sole  risk.  NSS  Labs  is  not  liable  or  responsible  for  any    damages,  losses,  or  expenses  arising  from  any  error  or  omission  in  this  report.  3.    NO  WARRANTIES,  EXPRESS  OR  IMPLIED  ARE  GIVEN  BY  NSS  LABS.  ALL  IMPLIED  WARRANTIES,  INCLUDING  IMPLIED  WARRANTIES  OF  MERCHANTABILITY,  FITNESS  FOR  A  PARTICULAR  PURPOSE,  AND  NON-­‐INFRINGEMENT  ARE  DISCLAIMED  AND  EXCLUDED  BY  NSS  LABS.  IN  NO  EVENT  SHALL  NSS  LABS  BE  LIABLE  FOR  ANY  CONSEQUENTIAL,  INCIDENTAL  OR  INDIRECT  DAMAGES,  OR  FOR  ANY  LOSS  OF  PROFIT,  REVENUE,  D ATA,  COMPUTER  PROGRAMS,  OR  OTHER  ASSETS,  EVEN  IF  ADVISED  OF  THE  POSSIBILITY  THEREOF.  4.    This  report  does  not  constitute  an  endorsement,  recommendation,  or  guarantee  of  any  of  the  products  (hardware  or  software)  tested  or  the  hardware  and  software  used  in  testing  the  products.  The  testing  does  not  guarantee  that  there  are  no  errors  or  defects  in  the  products  or  that  the  products  will  meet  the  reader’s  expectations,  requirements,  needs,  or  specifications,  or  that  they  will  operate  without  interruption.    5.    This  report  does  not  imply  any  endorsement,  sponsorship,  affiliation,  or  verification  by  or  with  any  organizations  mentioned  in  this  report.    6.    All  trademarks,  service  marks,  and  trade  names  used  in  this  report  are  the  trademarks,  service  marks,  and  trade  names  of  their  respective  owners.    ©  2012  NSS  Labs,  Inc.  All  rights  reserved.     13      

×