Building & Leveraging White Database for Antivirus Testing

2,745 views

Published on

Presented at the International Antivirus Testing Workshop 2007 by Mario Vuksan, Director, Knowledgebase Services, Bit9

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,745
On SlideShare
0
From Embeds
0
Number of Embeds
46
Actions
Shares
0
Downloads
523
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • I'm Mario Vuksan, Director of Knowledgebase Services for Bit9. In case you haven't heard of Bit9, we're a leading application control and device control solution provider. Part of our offering is the Bit9 Knowledgebase, the  largest collection of actionable intelligence about the world's software and today I am here to talk with you about trust-based computing.
  • Building & Leveraging White Database for Antivirus Testing

    1. 1. Building and Leveraging a Whitelist Database for Anti-Virus Testing Mario Vuksan, Director, Knowledgebase Services
    2. 2. Agenda <ul><li>Growing Signature/Definition Problem </li></ul><ul><li>Building a Global Whitelist </li></ul><ul><li>Leveraging a Global Whitelist </li></ul><ul><li>QA </li></ul>
    3. 3. Growing Signature Problem <ul><li>Cumulative unique variants have grown ten-fold over last 5 years (Yankee Group) </li></ul><ul><li>“ Denial-Of-Service” Attacks: Malware changing signature every 10 minutes </li></ul><ul><li>Solutions </li></ul><ul><ul><li>Heuristic & Behavioral Detections </li></ul></ul><ul><li>New Problem: High “False Positive” Count </li></ul>
    4. 4. Whitelist: a Google-sized Project Sizing Software Universe <ul><li>Number of Files Released Daily by: </li></ul><ul><ul><li>Microsoft – 500K / IBM – 100K / Sourceforge – 500K / Mozilla.Org – 250K </li></ul></ul><ul><li>More Components, Daily Builds, Auto Updaters </li></ul><ul><li>2.7B Files Indexed, heading for 10B </li></ul><ul><li>30TB of Installers, heading for 100TB </li></ul><ul><li>Daily acquiring 50M File Records, ¼ of YouTube </li></ul><ul><li>Tracking 20,000 Software Companies </li></ul><ul><ul><li>E.g. DMOZ tracks 200,000+ Entities </li></ul></ul>
    5. 5. Mechanics of a Whitelist Collect Extract Analyze Software Infrastructure Hardware Infrastructure Publish (Interfaces) Consumers Outbound Metadata Inbound User Metadata
    6. 6. Building a Whitelist <ul><li>Trusted Partners </li></ul><ul><ul><li>Benefits </li></ul></ul><ul><ul><ul><li>Trusted Source of Binary Material </li></ul></ul></ul><ul><ul><ul><li>In-depth Information on the Binary Data Indexed </li></ul></ul></ul><ul><ul><li>Realities </li></ul></ul><ul><ul><ul><li>Expensive Partner Programs </li></ul></ul></ul><ul><ul><ul><li>Complicated Applications </li></ul></ul></ul><ul><ul><ul><li>Lack of Interest </li></ul></ul></ul><ul><ul><ul><li>Lack of Comprehensive Repositories </li></ul></ul></ul>
    7. 7. Certifying Software <ul><ul><li>Certificate Mechanism </li></ul></ul><ul><ul><ul><li>As a Component for Validation </li></ul></ul></ul><ul><ul><ul><li>Costly Process, Cumbersome for QA Departments </li></ul></ul></ul><ul><ul><ul><li>Great When Seen on Shareware Sites </li></ul></ul></ul><ul><ul><ul><li> Less than 10% Penetration </li></ul></ul></ul><ul><ul><li>First-Seen Date </li></ul></ul><ul><ul><ul><li>Microsoft & Shared Installer Components </li></ul></ul></ul><ul><ul><ul><li>Long Time & No Detection  Likely Good </li></ul></ul></ul>
    8. 8. Challenges of Software Acquisition <ul><li>Buying/Getting Physical Media </li></ul><ul><ul><li>Retail Prices vs. Ebay </li></ul></ul><ul><ul><li>How to process 35K DVDs? </li></ul></ul><ul><li>FTP Sites </li></ul><ul><li>Web Sites </li></ul><ul><ul><li>Simple: Links and Forms </li></ul></ul><ul><ul><li>Complicated: Javascript </li></ul></ul><ul><ul><li>Super Complicated: Frames and AJAX </li></ul></ul><ul><li>Shareware Sites </li></ul><ul><li>Warez </li></ul><ul><ul><li>Legal Ramifications </li></ul></ul><ul><ul><li>Users vs. Collectors </li></ul></ul>
    9. 9. Harvesting The Internet <ul><li>Order of Difficulty </li></ul><ul><ul><li>FTPs – Wget, Curl </li></ul></ul><ul><ul><li>Simple HTTPs – Open Source Spiders </li></ul></ul><ul><ul><li>Try Grabbing Download.com </li></ul></ul><ul><ul><li>Try Grabbing Downloads.microsoft.com </li></ul></ul><ul><ul><li>Try Grabbing Canon or any Driver Site </li></ul></ul><ul><li>Datacenter Requirements </li></ul>
    10. 10. Assuring Software is Trustworthy <ul><li>Anti-Malware Scanning </li></ul><ul><ul><li>Name and Type Normalization </li></ul></ul><ul><li>Behavior Scanning </li></ul><ul><li>Code Inspection </li></ul><ul><li>External Meta Data Collection and Matching </li></ul>
    11. 11. Software Analysis Results <ul><li>Basic Embedded Data </li></ul><ul><li>PE Header Analysis </li></ul><ul><ul><li>Processor, Language, Binary Type </li></ul></ul><ul><li>Packers and Protectors </li></ul><ul><ul><li>500+ Variants </li></ul></ul><ul><ul><li>ASPack and Adobe </li></ul></ul><ul><ul><li>PECompact and Google </li></ul></ul><ul><li>Install Formats </li></ul><ul><ul><li>Proprietary (like Skype) </li></ul></ul><ul><ul><li>Binary Diffs (Patch Factory, MS PSF) </li></ul></ul><ul><li>Runtime Analysis and Sandboxing </li></ul>
    12. 12. Software Classifications <ul><li>Classifying Source </li></ul><ul><ul><li>Trust-based vs. Type-based </li></ul></ul><ul><li>Classifying Files </li></ul><ul><ul><li>Functional (Font, Driver, Screensaver) vs. Descriptive </li></ul></ul><ul><li>Classifying Products </li></ul><ul><ul><li>Basic </li></ul></ul><ul><ul><ul><li>Open Source </li></ul></ul></ul><ul><ul><ul><li>Commercial: Driver vs. Application </li></ul></ul></ul><ul><ul><ul><li>IM / P2P / Games </li></ul></ul></ul><ul><ul><li>Better </li></ul></ul><ul><ul><ul><li>Malware Classifications </li></ul></ul></ul><ul><ul><li>Interesting </li></ul></ul><ul><ul><ul><li>Steganography/Watermarking/Hacking/Hiding </li></ul></ul></ul>
    13. 13. Industry & Government Certifications <ul><li>Government Certifications </li></ul><ul><ul><li>NIAP, FIPS, DCTS </li></ul></ul><ul><li>Vulnerability Reports </li></ul><ul><ul><li>CVE, CERT, SANS, MSB, etc. </li></ul></ul><ul><li>For Good Software: </li></ul><ul><ul><li>Certification Programs </li></ul></ul><ul><ul><ul><li>Built for Vista, Windows Certified, Java Approved </li></ul></ul></ul><ul><ul><li>eTrust Download </li></ul></ul><ul><li>For Malware: </li></ul><ul><ul><li>StopBadware, CME </li></ul></ul>
    14. 14. Leveraging the Whitelist
    15. 15. PE Header Subsystem
    16. 16. Other PE Header Data
    17. 17. What about False Positives? <ul><li>Typical Suspects: </li></ul><ul><ul><li>Internet Explorer </li></ul></ul><ul><ul><li>Drivers (Network, File Access) </li></ul></ul><ul><ul><li>OS Components </li></ul></ul><ul><ul><li>Universal Installer and Uninstaller Components </li></ul></ul><ul><li>Optimized Applications: </li></ul><ul><ul><li>Using Obscure Third-Party Software </li></ul></ul><ul><ul><li>ASPack, PECompact, Themida </li></ul></ul>
    18. 18. Archive Format Distribution <ul><li>Most popular archive/packer formats </li></ul>
    19. 19. Or Are They False Positives? (FTP Injection Attacks) <ul><li>HP </li></ul>
    20. 20. Or Are They False Positives? (FTP Injection Attacks) <ul><li>Nero AG </li></ul>
    21. 21. Vertical Detection <ul><li>Malware Sample Vertical File Detection Chart </li></ul><ul><li>Good File Vertical Analysis </li></ul><ul><li>Anti-Malware Reports per Web Site </li></ul><ul><ul><li>Bit9 ISV Safe Software Program </li></ul></ul>
    22. 22. Use Case: Anti-Malware <ul><li>Benefits </li></ul><ul><ul><li>R&D Tool </li></ul></ul><ul><ul><ul><li>Packers, Metadata, Sources </li></ul></ul></ul><ul><ul><li>QA Tool </li></ul></ul><ul><ul><ul><li>False Positives </li></ul></ul></ul><ul><ul><li>Performance Accelerator </li></ul></ul><ul><ul><ul><li>Robin Bloor’s AVID </li></ul></ul></ul><ul><ul><ul><li>Next Generation Anti-Malware </li></ul></ul></ul>
    23. 23. About Bit9 <ul><li>What We Do: </li></ul><ul><ul><li>Application and Device Control Solutions and Software Metadata Reporting </li></ul></ul><ul><li>What We Offer: </li></ul><ul><ul><li>Bit9 Parity Protects against Malicious Software and Data Leakage </li></ul></ul><ul><ul><li>The Bit9 Knowledgebase is the Largest Collection of Actionable Intelligence about the World’s Software </li></ul></ul><ul><li>Background </li></ul><ul><ul><li>Founded in 2002 by founders of Okena (Cisco) </li></ul></ul><ul><ul><li>$2 Million NIST ATP Grant in 2003 </li></ul></ul><ul><ul><li>Headquartered in Cambridge, Mass. </li></ul></ul><ul><ul><li>Venture Funded </li></ul></ul>

    ×