The Difference between Track and Testing Performance

776 views

Published on

Presented at the International Antivirus Testing Workshop 2007 by Roel Schouwenberg, Senior Antivirus Researcher, Kaspersky Lab Benelux.

Published in: Travel, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
776
On SlideShare
0
From Embeds
0
Number of Embeds
49
Actions
Shares
0
Downloads
499
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The Difference between Track and Testing Performance

  1. 1. <ul><li>The difference between track and testing performance </li></ul><ul><li>Roel Schouwenberg, Senior Anti-Virus Researcher </li></ul><ul><li>Kaspersky Lab Benelux </li></ul><ul><li>[email_address] </li></ul>
  2. 2. About:Roel <ul><li>Malware analysis </li></ul><ul><li>AV research </li></ul><ul><li>Incident response </li></ul>
  3. 3. Overview <ul><li>Testing AV engine </li></ul><ul><li>Testing AVendor’s response time </li></ul><ul><li>Product technologies </li></ul><ul><li>Conclusions </li></ul>
  4. 4. Current testing <ul><li>On-demand </li></ul><ul><ul><li>WildList (won’t go there) </li></ul></ul><ul><ul><li>Large (zoo) test bed </li></ul></ul><ul><li>Retrospective </li></ul><ul><ul><li>using x month old product </li></ul></ul><ul><li>On-access (not so common or detailed) </li></ul>
  5. 5. On-demand: obvious flaws <ul><li>Trash files </li></ul><ul><li>Age of samples </li></ul><ul><li>Lack of transparency </li></ul><ul><li>Response time is not a factor </li></ul><ul><li>Lack of resources to perfect testing </li></ul><ul><li>Etc. </li></ul>
  6. 6. Infectors / Trojanizers <ul><li>Trojanizers (PE, script) </li></ul><ul><li>Real infectors </li></ul><ul><li>Check response time for detection and disinfection </li></ul><ul><ul><li>Creating trojanizer test bed can take a long time </li></ul></ul>
  7. 7. Online scan services <ul><li>JottiScan, VirusTotal (and others) </li></ul><ul><li>Much trash and ‘trash’ </li></ul><ul><li>False positive issues </li></ul><ul><li>Additional checks needed </li></ul><ul><ul><li>SFX archives and so on </li></ul></ul>
  8. 8. Testing vs track performance <ul><li>Detection on/of packer/crypter </li></ul><ul><li>Compare results with and without packer detection </li></ul><ul><li>Differentiate between packers </li></ul><ul><ul><li>Regular vs custom packer/crypter </li></ul></ul><ul><ul><li>Generic vs detecting specific family </li></ul></ul><ul><li>Age of samples </li></ul><ul><ul><li>1/2/3/6/12 months old </li></ul></ul>
  9. 9. Differentiate between malware <ul><li>Regional malware </li></ul><ul><li>Malware coming from a region </li></ul><ul><li>Payload (Banker vs GameThief trojan) </li></ul><ul><li>Automagically fabricated samples </li></ul><ul><ul><li>How many Zlobs do you want in the equation? </li></ul></ul>
  10. 10. Response time <ul><li>Global outbreak </li></ul><ul><li>Localized outbreak </li></ul><ul><li>Low priority malware </li></ul><ul><li>Infectors/trojanizers </li></ul>
  11. 11. Retrospective testing <ul><li>1 second is enough </li></ul><ul><li>Modified ‘droppers’ </li></ul><ul><li>Type of samples </li></ul>
  12. 12. Product technologies <ul><li>HIPS-like module </li></ul><ul><li>Components working together – AV vs IS </li></ul><ul><li>(Memory scanner) </li></ul><ul><li>Not so relevant (in this case): </li></ul><ul><ul><li>Malware removal </li></ul></ul><ul><ul><li>Registry cleanup </li></ul></ul><ul><ul><li>Malware detection on infected system </li></ul></ul>
  13. 13. Conclusions <ul><li>Other/nicer ways to check out the competition  </li></ul><ul><li>Product technologies make testing-life harder </li></ul><ul><li>Testing will always be flawed </li></ul>
  14. 14. The end <ul><li>Thank you for your attention! </li></ul><ul><li>Questions or comments? </li></ul><ul><li>[email_address] </li></ul>

×