iConference Popovsky

Collision of events…

Typical Network Incident Response
Technicians must choose:
Expend effort collecting forensically sound data, or
Simply restore network as quickly as possible
Evidentiary files altered in the process
Forensic value limited
Expediency wins…and so do attackers!

New Zealand vs. Russian Cases Characteristics NZ Hacker Case Russian Hacker Case Type of attack Typical script kiddie intrusion scenario Online criminal automated auction scam Damages $400,000 $25 million Investigator time 417 hours 9 months Consequences Community service 3 & 4 years in Federal prison

Lack of interest in prosecution
Inordinate effort/cost of investigations
Poor legal outcomes
Investigations not scalable
Too expensive
Too labor intensive
Ties up brilliant technical minds
Little comes of it

Growing Threat Spectrum

The Escalation Tendency of the Hacker Arms Race

Fueling the "arms race"
The volume of cyber attacks continues to increase.
It takes less technical knowledge to launch increasingly sophisticated attacks, using increasingly sophisticated hacker tools.
Organizations are becoming increasingly reliant on public networks, often without tempering enthusiasm with a concern for security
Surveys continue to report increased organizational investments in tools and techniques that protect information systems and prevent intrusions in response, yet criminal intrusions are escalating in number and severity.

Expect the appetite for prosecution to change $$$$$$$$$$$$$$$$

The Problem Why this problem must be solved

Frye / Daubert Standards
Frye Standards :
Is the approach sufficiently established?
Has the technique gained general acceptance in its field?
Does it require study/experience to gain special knowledge?
Does expertise lie in common experience/knowledge?
Daubert/Kumho Factors :
Has the technique used to collect evidence been tested? (or, can it be tested?)
Has the theory underlying the procedure, or the technique itself been subjected to peer review and publication?
Does the scientific technique have a known or potential rate of error?
Do standards exist, along with maintenance standards, for controlling the technique’s operation?

Expert Witness Testimony
The challenge:
Collect/store forensic data
Present forensic data credibly in court
Admissibility standards
Frye v. United States. 293 F. 1013 (D.C. Cir. 1923)
Daubert v. Merrell Dow Pharmaceuticals, Inc. Daubert, 509 U.S. 579 (1993)
(further enunciated in Kumho Tire Co. v. Carmichael)
Rule 702 (Federal Rules of Evidence )

Foundation
Expert believability based on jury trust
Experts either
Explain evidence so a jury can understand or
It’s so complex, only an expert can understand
Opposing counsel discredits witness by challenging testimony's foundation—
'how do you know this?’;
'how can you say this?';
'how can we believe the validity of what you say?‘
Radar gun analogy
The Genuine Tipmra Speeding Ticket Defense
http://www.tipmra.com/new_tipmra/washington_state_speeding_ticket.htm

Computer Forensic Tool Testing Project (CFTT-NIST)
"… to establish a methodology for testing computer forensic tools by the development of functional specifications, test procedures, test criteria, test sets and test hardware .“
Scope : 'software and hardware tools used by law enforcement agencies to acquire data from digital storage media'
Gap : Network devices that collect/gather data

Problem
… the courts may begin to expect the same high standards to which they've become accustomed for the preservation of evidence on computer hard drives, when evidence is gathered on complex networks or captured in transmission .
(Sommer, September 2002)

Rationale
Experts must speak competently about forensic data reliability
Skills of data gatherer
Process used
Devices employed
Establishing soundness of network data gathering devices can
Support prosecution/defense
Assist pursuit of legal remedies
BUT manufacturers rarely provide conclusive information
Proprietary design
Expense of calibration
As yet no demand
FURTHER manufacturers specifications are not reliable
We expect this to change…..

Consequences
A justice system subject to confusion —as innocent individuals are wrongly convicted and those deserving of punishment get away with criminal acts,
Escalating growth in online crime —as prosecution cases fail due to inadmissible evidence and digital crimes go unpunished,
Growing liability for companies —as sensitive customer information and digital assets are vulnerable to increasing online theft and as internal misusers challenge employee disciplinary action supported by
questionable digital evidence,
Decreasing trust in the e-economy —as companies and customers reassess doing business over public networks, and
A general halt to the progress of the Information Age —as online business and communications are no longer viable [FH07].

In the meantime…
No standards
No testing labs
Unreliable specifications
Network evidence admitted anyway
First responders still responsible

Proposed Solution
Develop device calibration standards
Comparison of instrument performance to a standard of known accuracy in order to determine deviation from nominal and/or make adjustments to minimize error
Start with user verification tests
Use current network testing protocols
Establish calibration approach

Calibration
"I often say that when you can measure what you are speaking about and express it in numbers you know something about it; but when you cannot express it in numbers your knowledge is a meager and unsatisfactory kind; it may be the beginning of knowledge but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be."
Lord Kelvin lecture to the Institution of Civil Engineers 3 May 1883 [1] [4]
[ 1] Lord (William Thomson) Kelvin--scientist, engineer and pioneering metrologist--is associated with the development of the Kelvin temperature measurement scale

The Problem
"… the courts may begin to expect the same high standards to which they've become accustomed for the preservation of evidence on computer hard drives, when evidence is gathered on complex networks or captured in transmission ." [Som02]
Computer (disk) forensics – more developed science
Disks seized by law enforcement
Investigators trained in legal procedures
Tools, procedures
Data accepted in court
Network forensics – can’t “bag and tag!”
Crime scene a live network
“ Investigators” often untrained network administrators
Tools developed for other purposes – troubleshooting, tuning, etc.
Data admitted anyway
Sophistication on both sides of the bar is growing – expect challenges!

Consequences
A justice system subject to confusion —as innocent individuals are wrongly convicted and those deserving of punishment get away with criminal acts,
Escalating growth in online crime —as prosecution cases fail due to inadmissible evidence and digital crimes go unpunished,
Growing liability for companies —as sensitive customer information and digital assets are vulnerable to increasing online theft and as internal misusers challenge employee disciplinary action supported by
questionable digital evidence,
Decreasing trust in the e-economy —as companies and customers reassess doing business over public networks, and
A general halt to the progress of the Information Age —as online business and communications are no longer viable [FH07].

Rationale for Calibration Focus
Without calibration of network devices used to collect forensic data, the data is:
Subject to serious legal challenge and
At risk for inadmissibility in court proceedings [ECF07, Som02].
Calibration not currently performed:
Proprietary architecture and forwarding algorithms
Troubleshooting, network tuning functionality focus
Collecting admissible evidence not primary
No standards for device validation

Computer Forensic Tool Testing Project (CFTT-NIST) Established
Established in anticipation of legal challenge
Mission to develop testing methods to evaluate computer forensic tools
Scope limited to 'software and hardware tools used by law enforcement agencies to acquire data from digital storage media'
Gap : Enterprise network devices used collect forensic data out of scope

Rationale for Developing Network Device Calibration Methodology
Need to establish reliability of network data gathering devices
Need to provide conclusive information that manufacturers don’t provide
FURTHER manufacturer specifications are not reliable
Courtroom challenges to network devices used to collect evidence is expected
Yet, no calibration standards/third party labs exist
Network evidence admitted anyway
First responders still responsible

Proposed Solution
Develop network device calibration standards
Start with user verification tests
Use current network testing protocols
Establish calibration approach

Summary of Progress

iConference Popovsky

by Brian Rowe on Feb 08, 2011

Accessibility

Tags

Upload Details

Usage Rights

1 Embed 62

Statistics

iConference Popovsky — Presentation Transcript