The benefits of using multiple antivirus engines, the disadvantages and how we can overcome them.

1. The value of Multi-Scanning
Benny Czarny
CEO OPSWAT, Inc.

2. Why Multi-Scanning ?
What are the threats we are up against ?
What is the capability of our solution?

3. What are the threats we are up against ?
Differences in reporting the total amount of threats
Source: McAfee
Source: Av-Test.org

4. What are the threats we are up against ?
Differences in detection rate for new malware
Source: McAfee
Source: Av-Test.org

5. What is the capability of our solution ?
Measuring the quality of anti-malware engines
 Detection coverage
 Response time
 Operating system compatibility
 Amount of false positive
 Other metrics

6. What is the capability of our solution ?
Measuring the quality of anti-malware engines
November 2010 February 2011 August 2011
AV Comparatives 97.6 % 95.8 % 92.1 %
AV Test 97 % 99 % 96 %
AMTSO mission is to develop and publish
standards and best practices for testing of anti-
malware products

7. Why Multi-Scanning ?
Conclusions
 No current answer about the amount of threats
 No clear answer about the quality of anti-malware
engines

8. Multi-Scanning
Can we quantify advantages and
disadvantages of multi-scanning?

9. Multi-Scanning
Advantages Disadvantages
 Improve malware  Increase amount of
detection False Positives
 Decrease detection time  Decrease
of an outbreak performance
 Increase resiliency to  Costly
antivirus engines
vulnerability

10. Advantages - Improve malware detection
Measuring detection coverage
100%
Antivirus 1
97.2%
Detection Rate:
Antivirus 2
92.1%
Detection Rate:
Source: www.av-comparatives.org

11. Advantages - Improve malware detection
Threats detected by Antivirus A and Antivirus B
 Malware sharing programs
between vendors
 In the wild
 3rd party sites e.g
 metascan-online.com
 virustotal.com
 jotti.com
Source: www.av-comparatives.org

12. Advantages - Improve malware detection
Factors affecting detection rate of a single antivirus
 Quality of software code
 Malware detection engine
 Signature database
 Update frequency
 Location of the analysts
 Other factors

13. Advantages - Improve malware detection
Software reliability models
Provide developers and managers with reasonably accurate quantitative
estimates of the software's reliability
Failure rate, N, can be made.
N = F*K* ( *Number of lines of source code)
When
F is the program's linear execution frequency
K is the defect exposure ratio

14. Advantages - Improve malware detection
Software reliability model
Source: AV-Test.org

15. Advantages - Improve malware detection
Defects
Assuming linear growth of malware

16. Advantages - Improve malware detection
AV-Test.org’s Malware Collection
Source: AV-Test.org

17. Advantages - Improve malware detection
Assuming exponential growth of malware
Defects

18. Advantages - Improve malware detection
Antivirus 1
QA defects not
detected by Antivirus 2
And unique samples
Shared samples
Antivirus 2
QA defects not
detected by Antivirus 1
And unique samples
Source: www.av-comparatives.org

19. Advantages - Improve malware detection
Probability
P( A ) = Probability of Antivirus A to Detect a virus
P( B ) = Probability of Antivirus B to Detect a virus
The probability that Antivirus A or Antivirus B detect a virus
P(A ∪ B) = P(A) + P(B) - P(A ∩ B)

20. Advantages - Decrease detection time of an outbreak
Source: AV-Test.org
Malware Name Malware Name Time Difference
From
AV 1 W32/Bredolab/Genreic2 Zero-hour - No detection
AV 2 Win32.Bredolab-BC [Trj] 24.28 hrs. Win32.Bredolab-BN(Trj) 2.10 hrs.
AV 3 Agent2.ABYO (Trojan horse) 10.18 hrs. Win32/Cryptor 3.52 hrs.
AV 4 - No detection Win32/Bredolab.Cgeneric Zero-hour
AV 5 Trojan.Agent-130266 40.82 hrs. - No detection
AV 6 Trojan.Botnetlog.II 3.68 hrs. Trojan.Botnetlog.140 13.17 hrs.
AV 7 Win32/TrojanDownloader.Bredolab.AA trojan 2.35 hrs. Win32/Kryptik.BHT trojan (variant) Zero-hour
AV 8 Gen:Trojan Heur.bqW@yzoXKwacdf Zero-hour Trojan Downloader.Bredolab CK 20.03 hrs.
AV 9 Trojan.Win32.Bredolab 2.55 hrs. Downloader Delphi 1.90 hrs.
AV 10 - No detection - No detection
AV 11 Backdoor.Win32.Bredolab.bge 6.70 hrs. Backdoor.Win32.Bredolab.btw 14.52 hrs.
AV 13 Generic Dropper.Ir(trojan) 28.83 hrs. - No detection
AV 14 TrojanDownloader:Win32/Bredolab X 11.62 hrs. - No detection
AV 15 W32/Obfuscated D Zero-hour - No detection
AV 16 Trj/Sinowal WRW 76.48 hrs. - No detection
AV 17 Trojan.Win32.GenericSIF369E9 71.27 hrs. - No detection
AV 18 - No detection - No detection
AV 19 - No detection Trojan.Win32.Bredolab.Gen2(v) Zero-hour
AV 20 Trojan.Fraudload.Gen!Pac.5(mutant) 4.05 hrs. TrojanFraudload Gen!Pac 5 (mutant) Zero-hour

21. Advantages - Decrease detection time of an outbreak
Theoretical average time to decrease the detection of an outbreak
Number of engines
Average time to respond
to an outbreak

22. Advantages - Decrease detection time of an outbreak
Example handling a specific outbreak with 1-30 antivirus engines
60
50
40
30
20
10 Amount of engines
Average time to respond
to an outbreak
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30

23. Advantages - Increase resiliency to antivirus engines vulnerability
Vulnerabilities of selected 4 engines
Number of advisories on the selected AVs. In a 3 years
2.5
2
1.5
1
0.5
0
AV 1 AV 2 AV 3 AV 4

24. Advantages - Increase resiliency to antivirus engines vulnerability
Known and Known and
unknown unknown
Vulnerabilities in Vulnerabilities in
Antivirus 1 Antivirus 2
Source: www.av-comparatives.org

25. Advantages - Increase resiliency to antivirus engines vulnerability
P(A) = the probability of one Antivirus A to encounter a vulnerability
P(B) = the probability of one Antivirus A to encounter a vulnerability
P(A ∩ B) = P(A)*P(B)
The Challenge - The vulnerability will not effect the multiscanner software

26. Disadvantages of Multi-Scanning
 Increased amount of false positives
 Decreased performance
 Costly

27. Disadvantages - Increased amount of false positives
Measuring detection coverage
Antivirus 1
False Positives:
Antivirus 2
False Positives
Source: www.av-comparatives.org

28. Disadvantages - Increased amount of false positives
Antivirus 1 Antivirus 2
8 false positives 10 false positives
14
AbsoluteBlue package AbsoluteBlue package
Azarus package Win32:Malware-gen
Trojan.Generic.6304836 Win32:Malware-gen
DateCalc package
Buchdruck package Azarus package
Win32:Trojan-gen
Gen:Variant.Zbot.29 Trojan.Generic.6304836 DB2EXE package
Intrapact package Buchdruck package Win32:Malware-gen
Gen:Trojan.Heur.VP2.fm0@a5Koffgi Gen:Variant.Zbot.29 Fiman package
Shellex package DateCalc package Win32:Malware-gen
Gen:Variant.Kazy.17493 Win32:Trojan-gen FTPcontrol package
Skriptum package DB2EXE package Win32:Malware-gen
Win32:Malware-gen Joshua package
Exploit.CVE-2011-0977.Gen
Fiman package Win32:Malware-gen
Virtualization package Sardu package
Gen:Trojan.Heur.KT.4.bq8@aqLITyf Win32:Malware-gen
Win32:Dropper-FRU
WinnerTw package FTPcontrol package
Shannel package
Gen:Variant.Kazy.18603 Win32:Malware-gen Win32:Fasec
WoodMahjongg package Intrapact package ShellPicture package
Gen:Variant.Kazy.14979 Gen:Trojan.Heur.VP2.fm0@a5Koffgi Win32:Malware-gen
Joshua package xComposer package
Win32:Malware-gen Win:32:SMorph
ShellPicture package
Win32:Malware-gen
Virtualization package
Gen:Trojan.Heur.KT.4.bq8@aqLITyf
WinnerTw package
Gen:Variant.Kazy.18603
WoodMahjongg package
Gen:Variant.Kazy.14979
xComposer package
Test performed August 2011
Win:32:SMorph Source: www.av-comparatives.org

29. Disadvantages - Increased amount of false positives
P(A) = Probability of Antivirus A to Detect a false positive
P(B) = Probability of Antivirus B to Detect a false positive
The probability that Antivirus A or Antivirus B reports a false
positive
P(A ∪ B) = P(A) + P(B) - P(A ∩ B)

30. Disadvantages - Increase amount of False Positives
How can white list engine help
P(A) = Probability of Antivirus A to Detect a virus
P(B) = Probability of Antivirus B to Detect a virus
P(C) = Probability of White list Engine to miss a threat
The probability that Antivirus A or Antivirus B detect a virus
P(A ∪ B) = P(A) + P(B) - P(A ∩ B)- P(C)

31. Disadvantages - Decreased performance
Assumption
Multi-scanning
Engine 1
Engine 2
Engine 3
Engine 4
Engine 5
Engine 6
Engine 7
0 5 10 15 20 25 30 35 40
►Time

32. Disadvantages - Decreased Performance
Way to increase performance
 Reduce Redundant tasks such as
 Open archives
 Detect file types
 Use different engines based on their capabilities to detect
threats in different file types
 Usage of distributed computing
 Usage of multicore processing
 Force scanning in memory

33. Disadvantages - Decreased performance
Reality
Presumed Speed
1 engine
3 engines
8 engines
Actual Speed
SYSTEM PROFILE
OS: Windows Server 2008 R2
CPU: Intel® Xeon® 2.13GHz
8cores
RAM: 8GB
PDF EXE JPG OTHER

34. Disadvantages - Costly
 Linear increased bandwidth consumption
 Increase in hardware requirements
 IT training
 Compliance checks is becoming more complex
 The solution cost more

35. Conclusion
The argument for multi-scanning is
clear though it is difficult to
measure its advantages.

36. References
AV-test.com
AV-Comperatives.com
www.metascan-online.com
AMTSO
Software system defect content prediction from development process
and product characteristics - Harris institute
McAfee