iConference Popovsky

  • 631 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
631
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
3
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • Today's computing security environment demonstrates a pattern of escalation between hackers and targets. As information system defenses improve, hackers must improve their skills in order to continue to wage successful attacks. As hackers' skills improve, information system defenses improve in order to repulse these more effective attacks. This in turn challenges hackers to acquire even better skills and abilities in order to wage even more effective attacks, which then motivates organizations to improve their information system security further in order to survive these more devastating attacks, and so on, in a never-ending pattern of escalation.
  • Before I begin I have some acknowledgements. I’m convinced some habits never die, especially those disciplines you develop early in your working career—I began mine as a systems analyst and developed an abiding respect for the software engineering approach—so I began my research gathering requirements, spending quite a bit of time with the legal and forensic community of practitioners, the eventual user community, who patiently explained their challenges. This is my opportunity to say thank you publicly: Attfield Dittrich Fluckiger Huang Nelson Orton Phillips Pollitt Schroeder Simon Review from yesterday: Digital forensic expert challenge: Collecting and storing data in a manner bounded by legal constraints is only half the battle, it also must present that data credibly in court. Nothing I ever said in the classroom was as effective at emphasizing the differences between these two functions as setting up a mock courtroom experience for students. Example: mock courtroom at SU—taught skills but then had them testify.
  • How is this done? Attorneys establish foundation for the believability of the expert
  • Knowing that the tools we use as forensic analysts are subject to challenge for the soundness of their performance, NIST began the Computer Forensic Tool Testing Project (CFTT To….mission Scope….. The Gap….network devices—no bandwidth at the agency to go beyond the current mission. Disk/computer forensics is relatively new, while disk forensics is a more settled science.
  • Sommers, 2002 conference at U of Idaho where I had the pleasure to meet most of you, stated that he feared that the expectation would be that the same reliability we’ve placed on disk forensics, we will expect of network forensics.
  • Given that experts must speak competently about forensic data reliability—skill, process, devices Important to establish soundness of network data gathering devices But Manufacturers rarely provide conclusion information No demand Expense—what’s the payback? Proprietary design Further what manufacturers do provide is unreliable—RFC2544, Fluke Expect to change—harbingers--legal interest in network intrusions—count the belly buttons x hours x $100’s ---anecdote Ted Vosk
  • Nevertheless—no standard methodologies for testing these devices, no labs, evidence admitted anyway, first responders still responsible
  • Ultimate solution is to develop calibration standards, labs, etc. We start with verification testing—defined as verification of manufacturer’s specifications.
  • Knowing that the tools we use as forensic analysts are subject to challenge for the soundness of their performance, NIST began the Computer Forensic Tool Testing Project (CFTT To….mission Scope….. The Gap….network devices—no bandwidth at the agency to go beyond the current mission. Disk/computer forensics is relatively new, while disk forensics is a more settled science.
  • Given that experts must speak competently about forensic data reliability—skill, process, devices Important to establish soundness of network data gathering devices But Manufacturers rarely provide conclusion information No demand Expense—what’s the payback? Proprietary design Further what manufacturers do provide is unreliable—RFC2544, Fluke Expect to change—harbingers--legal interest in network intrusions—count the belly buttons x hours x $100’s ---anecdote Ted Vosk
  • Ultimate solution is to develop calibration standards, labs, etc. We start with verification testing—defined as verification of manufacturer’s specifications.

Transcript

  • 1. Collision of events…
  • 2. Typical Network Incident Response
    • Technicians must choose:
        • Expend effort collecting forensically sound data, or
        • Simply restore network as quickly as possible
            • Evidentiary files altered in the process
            • Forensic value limited
    • Expediency wins…and so do attackers!
  • 3. New Zealand vs. Russian Cases Characteristics NZ Hacker Case Russian Hacker Case Type of attack Typical script kiddie intrusion scenario Online criminal automated auction scam Damages $400,000 $25 million Investigator time 417 hours 9 months Consequences Community service 3 & 4 years in Federal prison
  • 4. Lack of interest in prosecution
    • Inordinate effort/cost of investigations
    • Poor legal outcomes
    • Investigations not scalable
        • Too expensive
        • Too labor intensive
        • Ties up brilliant technical minds
        • Little comes of it
  • 5. Growing Threat Spectrum
  • 6. The Escalation Tendency of the Hacker Arms Race
  • 7. Fueling the "arms race"
    • The volume of cyber attacks continues to increase.
    • It takes less technical knowledge to launch increasingly sophisticated attacks, using increasingly sophisticated hacker tools.
    • Organizations are becoming increasingly reliant on public networks, often without tempering enthusiasm with a concern for security
    • Surveys continue to report increased organizational investments in tools and techniques that protect information systems and prevent intrusions in response, yet criminal intrusions are escalating in number and severity.
  • 8. Expect the appetite for prosecution to change $$$$$$$$$$$$$$$$
  • 9. The Problem Why this problem must be solved
  • 10. Frye / Daubert Standards
    • Frye Standards :
    • Is the approach sufficiently established?
    • Has the technique gained general acceptance in its field?
    • Does it require study/experience to gain special knowledge?
    • Does expertise lie in common experience/knowledge?
    • Daubert/Kumho Factors :
    • Has the technique used to collect evidence been tested? (or, can it be tested?)
    • Has the theory underlying the procedure, or the technique itself been subjected to peer review and publication?
    • Does the scientific technique have a known or potential rate of error?
    • Do standards exist, along with maintenance standards, for controlling the technique’s operation?
  • 11. Expert Witness Testimony
    • The challenge:
      • Collect/store forensic data
      • Present forensic data credibly in court
    • Admissibility standards
        • Frye v. United States. 293 F. 1013 (D.C. Cir. 1923)
        • Daubert v. Merrell Dow Pharmaceuticals, Inc. Daubert, 509 U.S. 579 (1993)
        • (further enunciated in Kumho Tire Co. v. Carmichael)
        • Rule 702 (Federal Rules of Evidence )
  • 12. Foundation
    • Expert believability based on jury trust
    • Experts either
      • Explain evidence so a jury can understand or
      • It’s so complex, only an expert can understand
    • Opposing counsel discredits witness by challenging testimony's foundation—
      • 'how do you know this?’;
      • 'how can you say this?';
      • 'how can we believe the validity of what you say?‘
    • Radar gun analogy
      • The Genuine Tipmra Speeding Ticket Defense
      • http://www.tipmra.com/new_tipmra/washington_state_speeding_ticket.htm
  • 13. Computer Forensic Tool Testing Project (CFTT-NIST)
    • "… to establish a methodology for testing computer forensic tools by the development of functional specifications, test procedures, test criteria, test sets and test hardware .“
    • Scope : 'software and hardware tools used by law enforcement agencies to acquire data from digital storage media'
    • Gap : Network devices that collect/gather data
  • 14. Problem
    • … the courts may begin to expect the same high standards to which they've become accustomed for the preservation of evidence on computer hard drives, when evidence is gathered on complex networks or captured in transmission .
    • (Sommer, September 2002)
  • 15. Rationale
    • Experts must speak competently about forensic data reliability
      • Skills of data gatherer
      • Process used
      • Devices employed
    • Establishing soundness of network data gathering devices can
      • Support prosecution/defense
      • Assist pursuit of legal remedies
    • BUT manufacturers rarely provide conclusive information
      • Proprietary design
      • Expense of calibration
      • As yet no demand
    • FURTHER manufacturers specifications are not reliable
    • We expect this to change…..
  • 16. Consequences
    • A justice system subject to confusion —as innocent individuals are wrongly convicted and those deserving of punishment get away with criminal acts,
    • Escalating growth in online crime —as prosecution cases fail due to inadmissible evidence and digital crimes go unpunished,
    • Growing liability for companies —as sensitive customer information and digital assets are vulnerable to increasing online theft and as internal misusers challenge employee disciplinary action supported by
    • questionable digital evidence,
    • Decreasing trust in the e-economy —as companies and customers reassess doing business over public networks, and
    • A general halt to the progress of the Information Age —as online business and communications are no longer viable [FH07].
  • 17. In the meantime…
    • No standards
    • No testing labs
    • Unreliable specifications
    • Network evidence admitted anyway
    • First responders still responsible
  • 18. Proposed Solution
    • Develop device calibration standards
    • Comparison of instrument performance to a standard of known accuracy in order to determine deviation from nominal and/or make adjustments to minimize error
    • Start with user verification tests
      • Use current network testing protocols
      • Establish calibration approach
  • 19. Calibration
    • "I often say that when you can measure what you are speaking about and express it in numbers you know something about it; but when you cannot express it in numbers your knowledge is a meager and unsatisfactory kind; it may be the beginning of knowledge but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be."
    • Lord Kelvin lecture to the Institution of Civil Engineers 3 May 1883 [1] [4]
    • [ 1] Lord (William Thomson) Kelvin--scientist, engineer and pioneering metrologist--is associated with the development of the Kelvin temperature measurement scale
  • 20. The Problem
    • "… the courts may begin to expect the same high standards to which they've become accustomed for the preservation of evidence on computer hard drives, when evidence is gathered on complex networks or captured in transmission ." [Som02]
      • Computer (disk) forensics – more developed science
        • Disks seized by law enforcement
        • Investigators trained in legal procedures
        • Tools, procedures
        • Data accepted in court
      • Network forensics – can’t “bag and tag!”
        • Crime scene a live network
        • “ Investigators” often untrained network administrators
        • Tools developed for other purposes – troubleshooting, tuning, etc.
        • Data admitted anyway
      • Sophistication on both sides of the bar is growing – expect challenges!
  • 21. Consequences
    • A justice system subject to confusion —as innocent individuals are wrongly convicted and those deserving of punishment get away with criminal acts,
    • Escalating growth in online crime —as prosecution cases fail due to inadmissible evidence and digital crimes go unpunished,
    • Growing liability for companies —as sensitive customer information and digital assets are vulnerable to increasing online theft and as internal misusers challenge employee disciplinary action supported by
    • questionable digital evidence,
    • Decreasing trust in the e-economy —as companies and customers reassess doing business over public networks, and
    • A general halt to the progress of the Information Age —as online business and communications are no longer viable [FH07].
  • 22. Rationale for Calibration Focus
    • Without calibration of network devices used to collect forensic data, the data is:
        • Subject to serious legal challenge and
        • At risk for inadmissibility in court proceedings [ECF07, Som02].
    • Calibration not currently performed:
        • Proprietary architecture and forwarding algorithms
        • Troubleshooting, network tuning functionality focus
        • Collecting admissible evidence not primary
        • No standards for device validation
  • 23. Computer Forensic Tool Testing Project (CFTT-NIST) Established
    • Established in anticipation of legal challenge
    • Mission to develop testing methods to evaluate computer forensic tools
    • Scope limited to 'software and hardware tools used by law enforcement agencies to acquire data from digital storage media'
    • Gap : Enterprise network devices used collect forensic data out of scope
  • 24. Rationale for Developing Network Device Calibration Methodology
    • Need to establish reliability of network data gathering devices
    • Need to provide conclusive information that manufacturers don’t provide
    • FURTHER manufacturer specifications are not reliable
    • Courtroom challenges to network devices used to collect evidence is expected
    • Yet, no calibration standards/third party labs exist
    • Network evidence admitted anyway
    • First responders still responsible
  • 25. Proposed Solution
    • Develop network device calibration standards
    • Start with user verification tests
      • Use current network testing protocols
      • Establish calibration approach
  • 26. Summary of Progress