About Anti-Virus Testing Dr. Vesselin Bontchev, anti–virus researcher FRISK Software International Thverholt 18, IS-105 Reykjavik, ICELAND National Laboratory of Computer Virology Bulgarian Academy of Sciences Acad. G. Bontchev Str., Bl. 2, 1113, BULGARIA E–mail: [email_address]

About Anti-Virus Testing Why Are You Here? You’re an AV tester But if you don’t already know how to do AV testing, how can you be an AV tester? You’re an AV developer But if you don’t already know how to test your own product… Well, let’s leave it at that You like Icelandic weather/food/beer But that is unlikely So, let’s assume that you want to become an AV tester, or to improve your work

Why Are You Here?

You’re an AV tester

But if you don’t already know how to do AV testing, how can you be an AV tester?

You’re an AV developer

But if you don’t already know how to test your own product… Well, let’s leave it at that

You like Icelandic weather/food/beer

But that is unlikely

So, let’s assume that you want to become an AV tester, or to improve your work

So, How Do You Test AV Products? First, you need AV products That’s less trivial than it sounds Then you have to decide what to test Product type Scanner, integrity checker, behavior blocker, suite… Operation method On-demand, on-access, both Platform Operating system, device, clean, infected, under attack, etc… Properties to test Detection, disinfection, identification, speed, false positives, usability, etc. Test set Not necessarily a virus collection!

First, you need AV products

That’s less trivial than it sounds

Then you have to decide what to test

Product type

Scanner, integrity checker, behavior blocker, suite…

Operation method

On-demand, on-access, both

Platform

Operating system, device, clean, infected, under attack, etc…

Properties to test

Detection, disinfection, identification, speed, false positives, usability, etc.

Test set

Not necessarily a virus collection!

More “How-to” Stuff Then, You Have to Figure Out How to Use the AV Products You Have That’s even less trivial than it sounds! Usually you do the wrong thing as early as during the installation phase Talk to the developers! Trust me, they know better than you how their products are supposed to be used Then You Have to Build Your Test Set Getting it from the WLO is not “it” Neither is downloading it from a Vx site More (much more) about this - later

Then, You Have to Figure Out How to Use the AV Products You Have

That’s even less trivial than it sounds!

Usually you do the wrong thing as early as during the installation phase

Talk to the developers! Trust me, they know better than you how their products are supposed to be used

Then You Have to Build Your Test Set

Getting it from the WLO is not “it”

Neither is downloading it from a Vx site

More (much more) about this - later

Even More “How-to” Stuff Then You Have to Publish Your Testing Methodology You have one, right? Right?! Prepare for it to be shot down by the more knowledgeable than you So that you’ll have to revise it completely And, no, you can’t do it right the first time Only Then You Can Begin Testing But That’s Not the End!

Then You Have to Publish Your Testing Methodology

You have one, right? Right?!

Prepare for it to be shot down by the more knowledgeable than you

So that you’ll have to revise it completely

And, no, you can’t do it right the first time

Only Then You Can Begin Testing

But That’s Not the End!

And Even More “How-to” Stuff Then You Have to Make Sense of the Results Which often don’t make sense Talk to the developers! They know better than you But don’t trust the developers! They are biased towards their own product. Always verify the things yourself Finally, Publish the Results And prepare to be accused of incompetence by developers, users, fellow testers – in other words, by just about anyone Fun, Isn’t It? But, hey, nobody ever said that AV product testing was easy …

Then You Have to Make Sense of the Results

Which often don’t make sense

Talk to the developers! They know better than you

But don’t trust the developers! They are biased towards their own product. Always verify the things yourself

Finally, Publish the Results

And prepare to be accused of incompetence by developers, users, fellow testers – in other words, by just about anyone

Fun, Isn’t It?

But, hey, nobody ever said that AV product testing was easy …

Questions?

Questions?