Avtest 2012 02-android_anti-malware_report_english


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Avtest 2012 02-android_anti-malware_report_english

  1. 1. Test Report: Anti-Malware solutions for Android Published: March, 9th 2012 Version: 1.0a
  2. 2. Anti-Malware solutions for AndroidUpdate 09th March 2012:It has been brought to our attention that certain parts in ourpaper and the testing methodology are consideredimprecise and/or flawed by third parties.Therefore we are now in contact with the reportingparties/vendors and performing additional tests to sort outany of those issues and will provide an updated version ofthe report as soon as possible.We would like to thank MYMobileSecurity(MYAndroidProtection), NQmobile (Netqin) and TotalDefense for their feedback on this.
  3. 3. Anti-Malware solutions for AndroidCopyright © 2012 AV-TEST GmbH. All rights reserved.Postal address: Klewitzstr. 7, 39112 Magdeburg, GermanyPhone +49 (0) 391 60754-60, Fax +49 (0) 391 60754-69For further details, please visit: http://www.av-test.org 1
  4. 4. Anti-Malware solutions for AndroidContent1. Introduction......................................................................................................................................... 32. Test report ........................................................................................................................................... 43. Test results .......................................................................................................................................... 54. Testing issues....................................................................................................................................... 85. Conclusion ........................................................................................................................................... 86. Product details..................................................................................................................................... 9 2
  5. 5. Anti-Malware solutions for Android1. IntroductionThe Smartphone market grew enormously over the last five years and the mobile malware evolvedrapidly, too. Right now there are over 450.000 apps in the Android market, where as there were lessthan 100.000 in July 2010 1. This makes it the fastest growing software market overall. With the riseof new apps, the number of malware increases as well. Figure 1 shows the growth of the AV-TESTAndroid malware collection. The increasing curve is similar to what weve seen for PC malware in thelast years. The threats for Android include Phishing- and Banking-Trojans, Spyware, Bots, RootExploits, SMS Fraud, Premium Dialers and Fake Installers. There have also been reports aboutDownload-Trojans – apps that download their malicious code after installation – which means thatthese apps cant be easily detected by Googles Bouncer technology 2 during publication in the GoogleAndroid Market. Our collection used for this test contains more than 20 different Android malwarefamilies, which cover each of the previously named threats. Android Malware Collection Growth 14000 12000 10000 8000 6000 4000 2000 0 New Samples Total # Samples Figure 1: Android malware collection growth since January 2011In November 2011 we’ve revealed that many Antivirus apps, which are available for free in GooglesAndroid Market, dont provide a sufficient malware protection for your Android mobile. This time weare trying to cover the good and the bad and started reviewing as many Android Anti-Malware appsas we could find, regardless whether an app requires a specific Android version or device. These appsinclude free and non-free programs, intended for personal use. This report aims to give animpression of the malware detection rates. As an independent test institute, we arent in the positionto recommend a specific product, but you can certainly use our report to find your personal favorite.However please bear in mind, that malware may not the only or the most important threat to your1 <http://en.wikipedia.org/wiki/Android_operating_system>2 Googles Bouncer technology checks apps for malware during publication in Googles Android Market<http://googlemobile.blogspot.com/2012/02/android-and-security.html> 3
  6. 6. Anti-Malware solutions for Androiddevice. Even if a product scores poorly in malware detection it may have other convenient features,such as remote lock and wipe, backup and phone locating, that make it useful for your purposes. It isalso possible to run two or more security apps on your device at the same time, using only the bestfeatures of the single apps. 4
  7. 7. Anti-Malware solutions for Android2. Test reportThe large number of tested apps required a scalable test environment, so we decided to use theAndroid emulator supplied by the Android SDK as basis for the review. The emulator has someadvantages in contrast to a real device. There is root-access without exploiting the device and youcan easily switch between API versions and screen sizes. It has also some disadvantages. You donthave a real phone number, which might be required to activate an app through SMS, and theemulated 3G connection may have a too high latency for querying the cloud of some vendors. Whilethe advantages of the emulator make testing more comfortable, the disadvantages limit the numberof apps, which could be properly tested. To get around this limitation, the apps, which didnt work inthe emulator, were tested on a real device and all emulator results were cross checked and verifiedon a real device. The emulator was set up with API level 10 (Gingerbread, Version 2.3) and for non-emulated testing we used a Samsung GalaxyTab (GT-P1010) with Froyo (Version 2.2) and a SamsungGalaxy Nexus (GT-I9250) with Ice Cream Sandwich (Version 4). The products were updated to theirlatest available versions/signature updates and were allowed to connect to their cloud during thetest. The real devices were flashed to factory default settings after every test to provide each productthe same clean environment.Among the tested apps we saw two different approaches for the on-demand scan. While many appssimply scan the complete device storage, some other apps scan installed apps and important filesonly. The latter were not able to scan the malware set with 618 malicious APK-files as it was storedon the SD card. Therefore, we tested the real-time protection feature of those apps instead. Thatmeans that all malware apps in our sample set were installed on a device or emulator one by one.After an app has been installed, the tester waited for feedback of the real-time protection, whichshould pop up if it finds a malicious app. In case of an undetected sample, it was uninstalledmanually. This is a time consuming approach and may not work in the future with larger sample sets(see Fig. 1).Regarding the detection rates, it makes no difference whether a malicious app is detected by an on-demand scan or by the real-time scan, when the app is installed. From the testers point of view, anon-demand scan with many samples is much easier to realize than an on-access scan. However fromthe user’s point of view the only criterion is protection, no matter at which point and how this takesplace.After an on-demand scan has been completed and all detections were removed the testers saved theremaining files, because the reporting abilities werent consistent among all apps. The files that wereleft over and have not been modified were flagged as "not detected". In case of the on-accesstesting, the testers wrote their own report since the samples were tested one by one. With theknowledge of which specific files have been detected by a scanner, we were able to analyze the scanresults based on malware families. The family based analysis can help vendors to improve theprotection for malware families with low detection rates. If the results would only provide a total,absolute detection rate, it would be impossible to notice if an app that scored well missed an entiremalware family or not. So this way of displaying the results gives both the reader and the vendormuch more insight. Furthermore this helps to decide whether a product that doesn’t score 100% isstill a good choice, e.g. because it misses on a malware family that is no threat to a specific usergroup or environment. 5
  8. 8. Anti-Malware solutions for AndroidIn this report no exact detection rates are given, instead the products are grouped into five differentcategories, referring to different ranges of detections (Fig. 2 and Fig. 3). The first category containsproducts that detected over 90%, the second category 90% to 65%, the third 65% to 40%, the fourtheverything less than 40% but above 0% and finally the last group contains the products that didn’tdetect anything. VERYGOOD GOOD SATISFYING SUFFICIENT NULL > 90% > 65% > 40% > 0% 0% Figure 2: Detection rate legendThere are several reasons for doing that: 1. The number of malware samples is still fairly small 2. Determining the prevalence of malware apps is difficult 3. Malware apps are quickly removed from the market (and even remotely from the device)This all comes down to one issue: It can happen very easily that a sample set is distorted by samplesthat are not really relevant anymore or were never at all. It is impossible for us to measure theprevalence of malware apps. It is also not possible to determine when and how long they have beena threat to the user. Therefore we identified the most widely known malware families and primarilyused those for the test. Only malicious apps that we have discovered between August and December2011 have been included in the test set. A few further malicious apps which don’t belong to thelisted families have been put in a category called “Other” and represent other families. Even withthose precautions it is possible that malware samples that are not suitable for this test are included.Already 30 wrongly chosen samples could change the result by 5%. In order to avoid too heavyeffects from these issues, the results are categorized. However, by looking at the individual familydetections it is still possible to get a fairly accurate picture of the absolute detection rate.The products were distributed over all detection ranges as shown in Figure 3. Detection rate distribution 6 7 12 10 6 > 90% > 65% > 40% > 0% 0% Figure 3: Detection rate distribution 6
  9. 9. Anti-Malware solutions for Android3. Test resultsDuring February 2012 we tested 41 Product Average Family Detectiondifferent Android Anti-Malware solutions. avast! Free Mobile Security Dr.Web anti-virus LightThe results are shown in Figure 4. Please F-Secure Mobile Security >90%note that the products in a certain category IKARUS mobile.security LITEare sorted alphabetically, so this listing is Kaspersky Mobile Security (Lite) Lookout Security & Antivirusnot a ranking! Mostly traditional anti-virus Zoner AntiVirus Freevendors are in the top range of the overall AegisLab Antivirus Freedetection results. Exceptions are Zoner and AVG Mobilation Anti-Virus Free Bitdefender Mobile SecurityLookout which also make it into the top ESET Mobile Securitygroup. Using these products you don’t have Norton Mobile Security Lite >65%to worry about your protection. Products Quick Heal Mobile Security Super Securitywith a detection rate between 90% and Trend Micro Mobile Security65% are still very good and could move to Vipre Mobile Security (BETA)the top range depending on changes to the Webroot SecureAnywhere Mobiletested malware set. Some of these products BullGuard Mobile Security Comodo Mobile Securityjust miss one or two malware families, G Data MobileSecurity >40%which might be not prevalent in certain McAfee Mobile Securityenvironments anyway. Again, there are only NQ Mobile Security Total Defense Mobile Securitytwo products from specialized mobile ALYac Androidsecurity vendors: AegisLab and Super Antivirus FreeSecurity. All other products in this group BlackBelt AntiVirus BluePoint Security Freecome from vendors well known in the CMC Mobile SecurityDesktop IT. Bullguard, Comodo, G Data, Fastscan Anti-Virus Free >0%McAfee, NetQin and Total Defense are in GuardX Antivirusthe third range. These vendors may not yet Kinetoo Malware Scan MobiShield Mobile Securityhave a sufficient infrastructure to collect a Privateer LITEbroad range of malware or they focus on a Snap Securelocal market. They provide reliable malware TrustGo Mobile Security Android Antivirusprotection against a few families, but have Android Defendertrouble with some others. It can be LabMSF Antivirus beta 0%expected that these products will improve MobileBot Antivirus MT Antivirusonce they broaden their sample acquiring. MYAndroid Protection AntivirusThe fourth group doesn’t contain any Figure 4: Average detection rate per malware familytraditional anti-virus vendor and include theproducts which also failed in our last report.We’ve reviewed six more products which are listed in the last category. We could not clearlydetermine whether they scanned the malware set correctly or not or whether they are able to detectanything at all. This means that we haven’t seen any detection, neither on our widely known samplesnor on the EICAR test file 3. Even in the on-access tests these products had no detections. So it is safeto assume that these products really don’t detect anything, but we still wanted to point out thepossibility of a flaw in our testing methodology.3 The EICAR test file can be used to determine whether an anti-malware software is operational or not and canbe obtained here <http://www.eicar.org/86-0-Intended-use.html> 7
  10. 10. Anti-Malware solutions for Android The malware family based analysis in Figure 5 shows that some products miss the top group only due to their low detection of one or two malware families. You can expect better signatures for these families to be added in the near future. The detection of specific families can also depend on each vendor’s definition of malware. Some families might only be annoying advertisement apps, while others include real malicious code, which can lead to monetary damage or data loss. Therefore some vendors may decide to not detect certain potentially unwanted, but not clearly malicious, apps. Average Family Detection Exploit.Lotoor Glodream BaseBrid DorDrae FakeInst Geinimi Nickspy KungFu Opfake Rooter Gonca Xsider SerBG Other Boxer Jifake Kmin Adrd Yzhcavast! Free Mobile SecurityDr.Web anti-virus LightF-Secure Mobile SecurityIKARUS mobile.security LITEKaspersky Mobile Security (Lite)Lookout Security & AntivirusZoner AntiVirus FreeAegisLab Antivirus FreeAVG Mobilation Anti-Virus FreeBitdefender Mobile SecurityESET Mobile SecurityNorton Mobile Security LiteQuick Heal Mobile SecuritySuper SecurityTrend Micro Mobile SecurityVipre Mobile Security (BETA)Webroot SecureAnywhere MobileBullGuard Mobile SecurityComodo Mobile SecurityG Data MobileSecurityMcAfee Mobile SecurityNQ Mobile SecurityTotal Defense Mobile SecurityALYac AndroidAntivirus FreeBlackBelt AntiVirusBluePoint Security FreeCMC Mobile SecurityFastscan Anti-Virus FreeGuardX AntivirusKinetoo Malware ScanMobiShield Mobile SecurityPrivateer LITESnap SecureTrustGo Mobile SecurityAndroid AntivirusAndroid DefenderLabMSF Antivirus betaMobileBot AntivirusMT AntivirusMYAndroid Protection Antivirus Figure 5: Detection by malware family 8
  11. 11. Anti-Malware solutions for Android4. Testing issuesDespite the fact that some apps weren’t able to scan our sample set on the SD card and thereforehave to be tested in a time consuming on-access test, we were also faced with apps which couldn’tdelete all detections automatically. They didn’t even provide a "Do it! And never ask me again!"option in the case of more than one malware detection. This fact led to testers clicking a "remove"-button several hundred times. While such options are very common in desktop applications, theyarent in the Android world yet. Also scan reports couldnt be saved within most of the tested apps.Some apps use SQLite databases to save their scan results and we were able to collect thecorresponding db-files from the emulators only. As accessing those files requires root privileges, theywerent collected from the real devices. The average user shouldnt miss such features, as its deviceshould never be infected with hundreds of malicious apps, but those simple functions would make atesters life much easier.As pointed out before, there are also apps which use their cloud to detect malware. While thisworked flawlessly with most products, both in emulated environments as well as on a real devicethere were a few exceptions. We have seen products that were not able to query their cloud in theemulator at all, even if full internet access was provided. There were also products that did havesome trouble on a real device. This might be due to latency issues and could only be resolved byrepeated tests until no further problems occurred. 9
  12. 12. Anti-Malware solutions for Android5. ConclusionEven if Google now checks all apps on its Android Market, you should consider installing a securityapp, because nowadays the malware authors are able to load their malicious code after a seeminglyclean app has been installed. Regarding the detection rates, you can trustfully choose from at least17 products to protect your Android device. What you should also have in mind when choosing yourmobile security app are additional functions such as backup and anti-theft protection (e.g. find yourlost device or wipe all data remotely).To keep your device free of malware even without a security app, you should install apps only fromtrusted sources, like the Google Android Market or the Amazon Appstore for Android. Read thecomments carefully and check whether the required permissions are reasonable (e.g. a game usuallyshouldnt need the permission to read or write SMS unless its description lists the specific featuresusing these permissions). As it may take between two to four weeks until Google removes maliciousapps from its Android Market, you should also be careful with new apps on the market. Wait untilapps are well-established, e.g. they were downloaded several thousand times and have many goodratings, or visit the developer’s website, which should at least provide contact information.In most cases when there is a free (often called Lite) and a paid version, the malware detectioncapabilities are the same. So if you are just looking at the detection rates, you can take the Lite resultand apply this to the paid version and vice versa. Another finding of the test is, that the well knownDesktop IT vendors perform above the average. Even the worst products from those vendors are stillbetter than most of the specialized mobile security software vendors. 10
  13. 13. Anti-Malware solutions for Android 6. Product details Product Vendor Android Package 4 VersionAegisLab Antivirus Free AegisLab com.aegislab.sd3prj.antivirus.free 1.0.4ALYac Android ESTsoft com.estsoft.alyac Antivirus Android Antivirus and.anti 1.6Antivirus Free Creative Apps com.zrgiu.antivirus 1.3.1Android Defender AndroidAppTools com.virusshield.android 1.1avast! Free Mobile Security AVAST com.avast.android.mobilesecurity 1.0.1282AVG Mobilation Anti-Virus Free AVG Mobilation com.antivirus 2.10Bitdefender Mobile Security BitDefender com.bitdefender.security 1.1.483BlackBelt AntiVirus BlackBelt SmartPhone Defence com.blackbelt.antivirus 2.2.0002BluePoint Security Free BluePoint Security bluepointfree.ad 4.0.17BullGuard Mobile Security BullGuard com.smobile.securityshield.android.bullgard Mobile Security CMC InfoSec com.cmcinfosec.mobilesec 2.1Comodo Mobile Security Comodo Security Solutions com.comodo.pimsecure 1.1.16984.2Dr.Web anti-virus Light Doctor Web com.drweb 6.01.5ESET Mobile Security ESET com.eset.emsw Anti-Virus Free K-TEC jp.ktinc.fastscan 1.1.5F-Secure Mobile Security F-Secure com.fsecure.browser 7.6.08787G Data MobileSecurity G Data de.gdata.mobilesecurity 23.2.17613GuardX Antivirus QStar org.qstar.guardx 2.3IKARUS mobile.security LITE IKARUS Security Software com.ikarus.mobile.security Mobile Security (Lite) Kaspersky Lab com.kms 9.10.106Kinetoo Malware Scan CPU Media SARL com.cpumedia.android.kinetoo 1.7.1LabMSF Antivirus beta LabMSF com.ReSync.RNGN 1.0Lookout Security & Antivirus Lookout Mobile Security com.lookout 7.1McAfee Mobile Security McAfee com.wsandroid.suite Antivirus Desktop Shark avm.defender 1.05MobiShield Mobile Security trustmobi com.trustmobi.MobiShield 3.1.5MT Antivirus KissDroid com.hot.free.defence.main 1.0.8MYAndroid Protection Antivirus MYMobileSecurity com.mymobileprotection20 Mobile Security Lite NortonMobile com.symantec.mobilesecurity Mobile Security NetQin Mobile com.nqmobile.antivirus20 LITE Privateer Labs com.privateer.lite 2.1.4Quick Heal Mobile Security Quick Heal Technologies com.quickheal.platform 1.01.017Snap Secure Exclaim Mobility com.exclaim.snapsecure.app 6.45Super Security Superdroid.net com.superdroid.security2 1.04Trend Micro Mobile Security Trend Micro com.trendmicro.tmmspersonal 2.1TrustGo Mobile Security TrustGo Mobile com.trustgo.security.beta 0.8.5Total Defense Mobile Security Total Defense com.tdi.security Mobile Security (BETA) GFI Software com.ssd.vipre 1.0.231Webroot SecureAnywhere Mobile Webroot com.webroot.security AntiVirus Free ZONER com.zoner.android.antivirus 1.2.10 Figure 6: Product details of all products listed in the test results 4 The Android package name is unique among all apps in the Google Android Market. You can use it as search term if you want to install a specific program from the Android Market. 11
  14. 14. Anti-Malware solutions for AndroidAegisLab Antivirus Free belongs ALYac Android is a free Mobile Android Antivirus showed noto the second range with its Security. It has a clear user detections in our tests anddetection rate between 65% and interface but the detection rates crashed several times. The90%. It has additional Anti-Theft need to improve. advertisements worked properly.functions in the Elite Version.Antivirus Free just detects a avast! Free Mobile Security is AVG Mobilation Anti-Virus Free ishandful of samples in the test set. available for free, easy to use and a good choice to secure yourIt shows advertisements at the has many features to protect your phone, being in the second groupbottom of the screen. device. With its very good of detection rates. It also provides detection rate it is one of the best Anti-Theft functions. security products for your Android device. 12
  15. 15. Anti-Malware solutions for AndroidThe premium version of BlackBelt AntiVirus is simple to BluePoint Security Free uses aBitdefender Mobile Security use. However the poor detection clear user interface but has a lowincludes a variety of other useful rate doesn’t excuse to pay for the detection rate with its cloud scanfunctions in addition to the good product after the trial period has engine.malware and privacy scanner. expired.BullGuard Mobile Security The free CMC Mobile Security Comodo Mobile Security providescontains Parental Control and seems to be out of date. The latest statistics at its home screen andBackup beside its virus scanner. signatures are several months old. provides fair malware detection. 13
  16. 16. Anti-Malware solutions for AndroidDr.Web anti-virus Light has very ESET Mobile Security provides a Fastscan Anti-Virus Free covers allgood detection rates. You need good to very good malware malware families but thethe premium version to use Anti- detection and extended Anti-Theft signatures still need to enhance.Theft and Anti-Spam features. functions.F-Secure Mobile Security has one G Data MobileSecurity scans on- GuardX Antivirus displaysof the best test results. F-Secure demand and periodically with a advertisements. It has no realoffers a comprehensive package satisfactory detection rate. You advantage over using no viruswith Anti-Theft and Safe Browsing. can also check apps for specific scanner. permissions. 14
  17. 17. Anti-Malware solutions for AndroidIKARUS mobile.security LITE is a Kaspersky Mobile Security (Lite) is Kinetoo Malware Scan offers aplain virus scanner and got top one of the best malware marginal detection rate. The freemarks in the malware detection protection solutions and contains version contains a regularlytest. Anti-Theft, Privacy Protection, updated database of mobile Parental Control and Data malware and spyware. Encryption.With LabMSF Antivirus we found Lookout Security & Antivirus McAfee Mobile Security offersneither any malware nor the achieved very good results for comprehensive security functionsEICAR test file. malware detection. Privacy with a 1-year subscription. Advisor, Safe Browsing, Remote Lock and Wipe and other functions are available in the premium version. 15
  18. 18. Anti-Malware solutions for AndroidMobileBot Antivirus couldn’t find MobiShield Mobile Security The only working feature of MTany malware sample, but it’s free contains free Antivirus, Backup, Antivirus seems to be theof ads. System Optimization, Anti-Theft, advertisements at the bottom. Traffic-Monitor and more. The malware detection test ends with moderate results.MYAndroid Protection Antivirus Norton Mobile Security Lite NQ Mobile Security provideslooks good, but it detected achieves good test results. The Antivirus, Network Manager,nothing. free version includes Anti- Privacy Advisor, Optimization and Malware and Anti-Theft. Backup in its free version. 16
  19. 19. Anti-Malware solutions for AndroidPrivateer LITE has no additional Quick Heal Mobile Security Snap Secure has a clear menu butfunctions to its scan feature, includes Anti-Malware detection, it detected less than 40 percent ofwhich didn’t detect too many Call Blocker, Anti-Theft and our malware test set.samples. Message Filtering.Super Security is a free solution Total Defense Mobile Security Trend Micro Mobile Securitywith a good detection rate. It has provides AntiVirus, Monitoring Personal Edition scored well in theseveral other functions. malware detection test. Safe and Backup. Browsing, Parental Control Call and Message Filter as well as Anti- Theft functions are integrated. 17
  20. 20. Anti-Malware solutions for AndroidTrustGo Mobile Security has to Vipre Mobile Security is available Virus Shield didn’t detect anythingimprove its detection rates. It for free. It’s a beta release but in our test. Every scan ended withoffers many functions for free. already shows good detection full screen advertisements. rates.Webroot SecureAnywhere Mobile Zoner AntiVirus Free surprisesshows good detection results in with very good test results andthe malware test. The premium many free functions such as Anti-version offers Secure Browsing, Theft, Task Manager, Call Filter,Lost Device Protection, Call and Parental Control and others.SMS Filter and an App Inspector. 18