Building and Leveraging a Whitelist Database for Anti-Virus Testing Mario Vuksan, Director, Knowledgebase Services

Agenda Growing Signature/Definition Problem Building a Global Whitelist Leveraging a Global Whitelist QA

Growing Signature/Definition Problem

Building a Global Whitelist

Leveraging a Global Whitelist

QA

Growing Signature Problem Cumulative unique variants have grown ten-fold over last 5 years (Yankee Group) “ Denial-Of-Service” Attacks: Malware changing signature every 10 minutes Solutions Heuristic & Behavioral Detections New Problem: High “False Positive” Count

Cumulative unique variants have grown ten-fold over last 5 years (Yankee Group)

“ Denial-Of-Service” Attacks: Malware changing signature every 10 minutes

Solutions

Heuristic & Behavioral Detections

New Problem: High “False Positive” Count

Whitelist: a Google-sized Project Sizing Software Universe Number of Files Released Daily by: Microsoft – 500K / IBM – 100K / Sourceforge – 500K / Mozilla.Org – 250K More Components, Daily Builds, Auto Updaters 2.7B Files Indexed, heading for 10B 30TB of Installers, heading for 100TB Daily acquiring 50M File Records, ¼ of YouTube Tracking 20,000 Software Companies E.g. DMOZ tracks 200,000+ Entities

Number of Files Released Daily by:

Microsoft – 500K / IBM – 100K / Sourceforge – 500K / Mozilla.Org – 250K

More Components, Daily Builds, Auto Updaters

2.7B Files Indexed, heading for 10B

30TB of Installers, heading for 100TB

Daily acquiring 50M File Records, ¼ of YouTube

Tracking 20,000 Software Companies

E.g. DMOZ tracks 200,000+ Entities

Mechanics of a Whitelist Collect Extract Analyze Software Infrastructure Hardware Infrastructure Publish (Interfaces) Consumers Outbound Metadata Inbound User Metadata

Building a Whitelist Trusted Partners Benefits Trusted Source of Binary Material In-depth Information on the Binary Data Indexed Realities Expensive Partner Programs Complicated Applications Lack of Interest Lack of Comprehensive Repositories

Trusted Partners

Benefits

Trusted Source of Binary Material

In-depth Information on the Binary Data Indexed

Realities

Expensive Partner Programs

Complicated Applications

Lack of Interest

Lack of Comprehensive Repositories

Certifying Software Certificate Mechanism As a Component for Validation Costly Process, Cumbersome for QA Departments Great When Seen on Shareware Sites  Less than 10% Penetration First-Seen Date Microsoft & Shared Installer Components Long Time & No Detection  Likely Good

Certificate Mechanism

As a Component for Validation

Costly Process, Cumbersome for QA Departments

Great When Seen on Shareware Sites

 Less than 10% Penetration

First-Seen Date

Microsoft & Shared Installer Components

Long Time & No Detection  Likely Good

Challenges of Software Acquisition Buying/Getting Physical Media Retail Prices vs. Ebay How to process 35K DVDs? FTP Sites Web Sites Simple: Links and Forms Complicated: Javascript Super Complicated: Frames and AJAX Shareware Sites Warez Legal Ramifications Users vs. Collectors

Buying/Getting Physical Media

Retail Prices vs. Ebay

How to process 35K DVDs?

FTP Sites

Web Sites

Simple: Links and Forms

Complicated: Javascript

Super Complicated: Frames and AJAX

Shareware Sites

Warez

Legal Ramifications

Users vs. Collectors

Harvesting The Internet Order of Difficulty FTPs – Wget, Curl Simple HTTPs – Open Source Spiders Try Grabbing Download.com Try Grabbing Downloads.microsoft.com Try Grabbing Canon or any Driver Site Datacenter Requirements

Order of Difficulty

FTPs – Wget, Curl

Simple HTTPs – Open Source Spiders

Try Grabbing Download.com

Try Grabbing Downloads.microsoft.com

Try Grabbing Canon or any Driver Site

Datacenter Requirements

Assuring Software is Trustworthy Anti-Malware Scanning Name and Type Normalization Behavior Scanning Code Inspection External Meta Data Collection and Matching

Anti-Malware Scanning

Name and Type Normalization

Behavior Scanning

Code Inspection

External Meta Data Collection and Matching

Software Analysis Results Basic Embedded Data PE Header Analysis Processor, Language, Binary Type Packers and Protectors 500+ Variants ASPack and Adobe PECompact and Google Install Formats Proprietary (like Skype) Binary Diffs (Patch Factory, MS PSF) Runtime Analysis and Sandboxing

Basic Embedded Data

PE Header Analysis

Processor, Language, Binary Type

Packers and Protectors

500+ Variants

ASPack and Adobe

PECompact and Google

Install Formats

Proprietary (like Skype)

Binary Diffs (Patch Factory, MS PSF)

Runtime Analysis and Sandboxing

Software Classifications Classifying Source Trust-based vs. Type-based Classifying Files Functional (Font, Driver, Screensaver) vs. Descriptive Classifying Products Basic Open Source Commercial: Driver vs. Application IM / P2P / Games Better Malware Classifications Interesting Steganography/Watermarking/Hacking/Hiding

Classifying Source

Trust-based vs. Type-based

Classifying Files

Functional (Font, Driver, Screensaver) vs. Descriptive

Classifying Products

Basic

Open Source

Commercial: Driver vs. Application

IM / P2P / Games

Better

Malware Classifications

Interesting

Steganography/Watermarking/Hacking/Hiding

Industry & Government Certifications Government Certifications NIAP, FIPS, DCTS Vulnerability Reports CVE, CERT, SANS, MSB, etc. For Good Software: Certification Programs Built for Vista, Windows Certified, Java Approved eTrust Download For Malware: StopBadware, CME

Government Certifications

NIAP, FIPS, DCTS

Vulnerability Reports

CVE, CERT, SANS, MSB, etc.

For Good Software:

Certification Programs

Built for Vista, Windows Certified, Java Approved

eTrust Download

For Malware:

StopBadware, CME

Leveraging the Whitelist

PE Header Subsystem

Other PE Header Data

What about False Positives? Typical Suspects: Internet Explorer Drivers (Network, File Access) OS Components Universal Installer and Uninstaller Components Optimized Applications: Using Obscure Third-Party Software ASPack, PECompact, Themida

Typical Suspects:

Internet Explorer

Drivers (Network, File Access)

OS Components

Universal Installer and Uninstaller Components

Optimized Applications:

Using Obscure Third-Party Software

ASPack, PECompact, Themida

Archive Format Distribution Most popular archive/packer formats

Most popular archive/packer formats

Or Are They False Positives? (FTP Injection Attacks) HP

HP

Or Are They False Positives? (FTP Injection Attacks) Nero AG

Nero AG

Vertical Detection Malware Sample Vertical File Detection Chart Good File Vertical Analysis Anti-Malware Reports per Web Site Bit9 ISV Safe Software Program

Malware Sample Vertical File Detection Chart

Good File Vertical Analysis

Anti-Malware Reports per Web Site

Bit9 ISV Safe Software Program

Use Case: Anti-Malware Benefits R&D Tool Packers, Metadata, Sources QA Tool False Positives Performance Accelerator Robin Bloor’s AVID Next Generation Anti-Malware

Benefits

R&D Tool

Packers, Metadata, Sources

QA Tool

False Positives

Performance Accelerator

Robin Bloor’s AVID

Next Generation Anti-Malware

About Bit9 What We Do: Application and Device Control Solutions and Software Metadata Reporting What We Offer: Bit9 Parity Protects against Malicious Software and Data Leakage The Bit9 Knowledgebase is the Largest Collection of Actionable Intelligence about the World’s Software Background Founded in 2002 by founders of Okena (Cisco) $2 Million NIST ATP Grant in 2003 Headquartered in Cambridge, Mass. Venture Funded

What We Do:

Application and Device Control Solutions and Software Metadata Reporting

What We Offer:

Bit9 Parity Protects against Malicious Software and Data Leakage

The Bit9 Knowledgebase is the Largest Collection of Actionable Intelligence about the World’s Software

Background

Founded in 2002 by founders of Okena (Cisco)

$2 Million NIST ATP Grant in 2003

Headquartered in Cambridge, Mass.

Venture Funded