The value of Multi-Scanning Benny Czarny CEO OPSWAT, Inc.
Why Multi-Scanning ?What are the threats we are up against ?What is the capability of our solution?
What are the threats we are up against ? Differences in reporting the total amount of threatsSource: McAfeeSource: Av-Test.org
What are the threats we are up against ? Differences in detection rate for new malwareSource: McAfeeSource: Av-Test.org
What is the capability of our solution ? Measuring the quality of anti-malware engines Detection coverage Response time Operating system compatibility Amount of false positive Other metrics
What is the capability of our solution ? Measuring the quality of anti-malware engines November 2010 February 2011 August 2011AV Comparatives 97.6 % 95.8 % 92.1 %AV Test 97 % 99 % 96 % AMTSO mission is to develop and publish standards and best practices for testing of anti- malware products
Why Multi-Scanning ? Conclusions No current answer about the amount of threats No clear answer about the quality of anti-malware engines
Multi-ScanningCan we quantify advantages anddisadvantages of multi-scanning?
Multi-ScanningAdvantages Disadvantages Improve malware Increase amount of detection False Positives Decrease detection time Decrease of an outbreak performance Increase resiliency to Costly antivirus engines vulnerability
Advantages - Improve malware detection Threats detected by Antivirus A and Antivirus B Malware sharing programs between vendors In the wild 3rd party sites e.g metascan-online.com virustotal.com jotti.comSource: www.av-comparatives.org
Advantages - Improve malware detection Factors affecting detection rate of a single antivirus Quality of software code Malware detection engine Signature database Update frequency Location of the analysts Other factors
Advantages - Improve malware detection Software reliability modelsProvide developers and managers with reasonably accurate quantitativeestimates of the softwares reliabilityFailure rate, N, can be made.N = F*K* ( *Number of lines of source code)WhenF is the programs linear execution frequencyK is the defect exposure ratio
Advantages - Improve malware detection Antivirus 1 QA defects not detected by Antivirus 2 And unique samples Shared samples Antivirus 2 QA defects not detected by Antivirus 1 And unique samplesSource: www.av-comparatives.org
Advantages - Improve malware detection ProbabilityP( A ) = Probability of Antivirus A to Detect a virusP( B ) = Probability of Antivirus B to Detect a virusThe probability that Antivirus A or Antivirus B detect a virus P(A ∪ B) = P(A) + P(B) - P(A ∩ B)
Advantages - Decrease detection time of an outbreak Source: AV-Test.org Malware Name Malware Name Time Difference FromAV 1 W32/Bredolab/Genreic2 Zero-hour - No detectionAV 2 Win32.Bredolab-BC [Trj] 24.28 hrs. Win32.Bredolab-BN(Trj) 2.10 hrs.AV 3 Agent2.ABYO (Trojan horse) 10.18 hrs. Win32/Cryptor 3.52 hrs.AV 4 - No detection Win32/Bredolab.Cgeneric Zero-hourAV 5 Trojan.Agent-130266 40.82 hrs. - No detectionAV 6 Trojan.Botnetlog.II 3.68 hrs. Trojan.Botnetlog.140 13.17 hrs.AV 7 Win32/TrojanDownloader.Bredolab.AA trojan 2.35 hrs. Win32/Kryptik.BHT trojan (variant) Zero-hourAV 8 Gen:Trojan Heur.bqW@yzoXKwacdf Zero-hour Trojan Downloader.Bredolab CK 20.03 hrs.AV 9 Trojan.Win32.Bredolab 2.55 hrs. Downloader Delphi 1.90 hrs.AV 10 - No detection - No detectionAV 11 Backdoor.Win32.Bredolab.bge 6.70 hrs. Backdoor.Win32.Bredolab.btw 14.52 hrs.AV 13 Generic Dropper.Ir(trojan) 28.83 hrs. - No detectionAV 14 TrojanDownloader:Win32/Bredolab X 11.62 hrs. - No detectionAV 15 W32/Obfuscated D Zero-hour - No detectionAV 16 Trj/Sinowal WRW 76.48 hrs. - No detectionAV 17 Trojan.Win32.GenericSIF369E9 71.27 hrs. - No detectionAV 18 - No detection - No detectionAV 19 - No detection Trojan.Win32.Bredolab.Gen2(v) Zero-hourAV 20 Trojan.Fraudload.Gen!Pac.5(mutant) 4.05 hrs. TrojanFraudload Gen!Pac 5 (mutant) Zero-hour
Advantages - Decrease detection time of an outbreak Theoretical average time to decrease the detection of an outbreak Number of engines Average time to respond to an outbreak
Advantages - Decrease detection time of an outbreak Example handling a specific outbreak with 1-30 antivirus engines605040302010 Amount of engines Average time to respond to an outbreak 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Advantages - Increase resiliency to antivirus engines vulnerability Vulnerabilities of selected 4 engines Number of advisories on the selected AVs. In a 3 years2.5 21.5 10.5 0 AV 1 AV 2 AV 3 AV 4
Advantages - Increase resiliency to antivirus engines vulnerability Known and Known and unknown unknown Vulnerabilities in Vulnerabilities in Antivirus 1 Antivirus 2Source: www.av-comparatives.org
Advantages - Increase resiliency to antivirus engines vulnerabilityP(A) = the probability of one Antivirus A to encounter a vulnerabilityP(B) = the probability of one Antivirus A to encounter a vulnerabilityP(A ∩ B) = P(A)*P(B)The Challenge - The vulnerability will not effect the multiscanner software
Disadvantages of Multi-Scanning Increased amount of false positives Decreased performance Costly
Disadvantages - Increased amount of false positivesP(A) = Probability of Antivirus A to Detect a false positiveP(B) = Probability of Antivirus B to Detect a false positiveThe probability that Antivirus A or Antivirus B reports a falsepositiveP(A ∪ B) = P(A) + P(B) - P(A ∩ B)
Disadvantages - Increase amount of False Positives How can white list engine helpP(A) = Probability of Antivirus A to Detect a virusP(B) = Probability of Antivirus B to Detect a virusP(C) = Probability of White list Engine to miss a threatThe probability that Antivirus A or Antivirus B detect a virus P(A ∪ B) = P(A) + P(B) - P(A ∩ B)- P(C)
Disadvantages - Decreased Performance Way to increase performance Reduce Redundant tasks such as Open archives Detect file types Use different engines based on their capabilities to detect threats in different file types Usage of distributed computing Usage of multicore processing Force scanning in memory
Disadvantages - Decreased performance Reality Presumed Speed 1 engine 3 engines 8 engines Actual Speed SYSTEM PROFILE OS: Windows Server 2008 R2 CPU: Intel® Xeon® 2.13GHz 8cores RAM: 8GBPDF EXE JPG OTHER
Disadvantages - Costly Linear increased bandwidth consumption Increase in hardware requirements IT training Compliance checks is becoming more complex The solution cost more
ConclusionThe argument for multi-scanning isclear though it is difficult tomeasure its advantages.
ReferencesAV-test.comAV-Comperatives.comwww.metascan-online.comAMTSOSoftware system defect content prediction from development processand product characteristics - Harris instituteMcAfee