Dark Data and Missing Evidence
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Dark Data and Missing Evidence



Dark Data is created in everything that we create digitally. Valuable evidence may be hiding in these obscured areas.

Dark Data is created in everything that we create digitally. Valuable evidence may be hiding in these obscured areas.



Total Views
Views on SlideShare
Embed Views



1 Embed 1

http://www.linkedin.com 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • This presentation was provided for an ASDFED Indianapolis Chapter meeting.
  • How did I get the term “Dark Data”? Not from Darth Vader, but they do have some things in common.
  • I copied “Dark Matter”, because it also goes undetected yet still affects things (objects/solar systems) around it.This image was created by observing the gravitational effects on light and objects around the matter. No instrument can actually see the dark matter directly.
  • Dark Data is in everything digital that we create, yet we don’t see it.
  • Dark Data is hiding in the most unsuspecting places.
  • DCO – Used to reduce the disk size to exactly match the size of another hard drive. This makes it easier to clone hard drives.HPA – Used to store vendor utilities on a hard drive, where a user can’t delete them.These areas are difficult to access and add or remove.Unformatted Disk Space is the remaining space that has not been allocated to a disk volume that the user can access.
  • Many recovery tools falsely report their recovery success. Many of the successfully recovered files are actually corrupted with other file fragments.
  • Most Forensics Tools keep these files in the Exception Bin. Have you ever seen an investigation with an empty Exception Bin? What if the best evidence was hiding in that Exception Bin?!?Ex: Hidden TrueCrypt volume file, that looks like random data.
  • The list on the left was produced with Windows, as an extreme example. Although, many eDiscovery tools don’t do much better than this.The list on the right was produced by a tool that specializes in accurately identifying thousands of file types.Notice the 3 Alternate Data Streams identified on the right. They weren’t just detected, but analyzed to catch any hidden file types.
  • Many tools combine RAM slack with Drive Slack. This causes confusion when file carving for partial files, because these slacks come from different sources.
  • Common files may contain stowaways.Bpp = Bits Per Pixel
  • Step 1: Rename the file to be smuggled to ‘document.xml’ (I used a simple text file)Step 2: Rename Word.docx to Word.zipStep 3: Open Word.zip with WinZipStep 4: Add the new smuggled ‘document.xml’ to Word.zip (in the root)Step 5: Rename Word.zip to Word.docx
  • This example shows an MS Outlook Form Template that was edited to remove part of a sentence. The deleted content is still there!When the paragraph/object shrank, the Stream Slack inherited the end of the paragraph.Existing Redaction tools use Microsoft libraries that ignore the Stream Slack.
  • Smuggled data is broken down into bits and substituted for picture data that doesn’t effect the visible image enough to be noticed.May just change 1 bit per pixel, or fill the Field Slack.The smuggled data may also be encrypted before insertion.

Dark Data and Missing Evidence Presentation Transcript

  • 1. Dark DataandMissing Evidence
    Rob Zirnstein
    Forensic Innovations
    January 13th, 2011
  • 2. Darth Vader?
    No, “Dark Data”, but they both
    Are often associated with evil
    Keep secrets (“Luke, I’m your father”)
    Are potentially harmful
  • 3. Dark Matter?
    No, “Dark Data”! But they both
    Go undetected
    Are surrounded by
    detectable stuff
    Affect things around them
  • 4. What is Dark Data?
    Dark Data in our digital devices
    Everyone creates it (unintentionally)
    Criminals may hide it (Anti-Forensics)
    Forensic tools can’t see it
    But it is there!
    Data that we can’t see
    On our hard drives
    On out flash drives
    In our computer files
  • 5. Where is Dark Data?
    DCO & HPA
    Unformatted Disk Space
    Deleted Files
    Unknown Files
    Between Files
    Inside Common Files
    Deleted Data Objects
  • 6. Hard Drive Layout
    Device Configuration
    Overlay (DCO)
    Data Cleaner+ http://www.mp3cdsoftware.com/blancco---data-cleaner--download-16317.htm
    Host Protected
    Area (HPA)
    Forensic Duplicator
    HDD Capacity Restore Tool http://hddguru.com/software/2007.07.20-HDD-Capacity-Restore-Tool/
    Unformatted Disk Space
  • 7. Deleted Files
    Deleted Files aren’t really gone?
    Unused Disk Space (in a volume)
    Disk Caches / Swap Files
    Windows Recycle Bin
    Are they hard to recover?
    Fragmentation is deadly
    Large databases tend to be
    heavily fragmented
    Even DFRWS Researchers find
    that fragmentation can make
    some file types impossible to
    recover (http://www.dfrws.org/2007/challenge/results.shtml)
  • 8. Unknown Files (1)
    500 types of files handled by eDiscovery, Document Management & Computer Forensics Tools
    50,000+* types of files in the world
    5,000 types of files typically in use
  • 9. Unknown Files (2)
    Typical ToolsFI Tools
    (23 wrong files) (26 Correct Files)
  • 10. Between Files
    Alternate Data Streams (ADS)
    Files hiding behind files (on NTFS)
    RAM Slack
    Padding between the end of a file and the end of the current sector
    Typically zeros, sometimes random content
    File/Cluster/Residual/Drive Slack
    Padding between sectors used
    & the end of the current cluster
    Previous sector content that
    should be used in File Carving
  • 11. Inside Common Files
    Deleted Objects
    Ex: Adobe PDF & MS Office 2003 (OLE)
    not removing deleted data (change tracking)
    Smuggled Objects
    Ex: MS Office 2007 (Zip) and MS Wave
    (RIFF) formats ignore foreign objects
    Object / Stream Slack
    Ex: OLE objects have sector size issues,
    just like with disk sectors
    Field Slack
    Ex: Image files that don’t use the whole
    palette, and/or less than 8/16/32/48 bpp
  • 12. Smuggled Objects
    Some formats ignore
    foreign objects
    MS Office 2007 (Zip)
    MS Wave (RIFF)
    This example
    I added a file to a
    Word 2007 document.
    The document opens
    without any error.
  • 13. Deleted Data in Slack
    Deleted Data that evades Redaction
  • 14. Steganography
    Intentional Data Hiding
  • 15. Is Dark Data Important?
    Cases are won or lost based on the ability to find the evidence.
    The strongest evidence may be hidden accidentally or intentionally.
    Corporate Digital Assets may be lost, but recoverable.
    Employee misconduct is tracked by the hidden trail of improper acts.
    Intellectual Property theft can
    put a company out of business.
    Identify in-house criminals by detect-
    ing smuggled data before it leaves.
  • 16. Dark Data Can Be Fragile
    Live Forensics software tools run on the live system.
    The RAM that they use affects the memory cache files on the hard drive.
    The running computer deletes, fragments & over writes files on the hard drive constantly.
    Hard drive activity can destroy Dark Data!
    Dark Data must be collected first!
    Before other tools interfere with the data.
    Image RAM
    Image Hard Drive (when possible)
    Analyze Unallocated Disk Space
    Analyze File Slack Space
    Collect relevant file types
  • 17. What Does FI Do?
    Create Technologies to Capture Dark Data
    File Investigator
    File Expander
    File Harvester
    Equip Law Enforcement with Tools
    FI Object Explorer
    FI Data Profiler Portable
  • 18. FI Technologies
    File Investigator
    Discovers Files Masquerading as Other Types
    Identifies 3,953+ File Types
    High Accuracy & Speed
    File Expander
    Discovers Hidden Data within files
    Data missed by all forensic tools
    • File Harvester (Under Development)
    Recovers deleted/lost files the
    rest of the industry can’t
    Will eventually rebuild partial files
  • 19. Thank you
    Rob Zirnstein
    (317) 430-6891