Dark Data Hiding in your Records: Opportunity or Danger?
•Download as PPTX, PDF•
0 likes•950 views
There is Dark Data hiding in every document that we create. Does this Dark Data represent an opportunity or danger to us and our business?
Recommended
More Related Content
Similar to Dark Data Hiding in your Records: Opportunity or Danger?
Similar to Dark Data Hiding in your Records: Opportunity or Danger? (20)
Recently uploaded
Recently uploaded (20)
Dark Data Hiding in your Records: Opportunity or Danger?
- 1. Dark DataHiding in your RecordsOpportunity or Danger? Rob Zirnstein President Forensic Innovations January 19th, 2011
- 2. Darth Vader? No, “Dark Data”, but they both Are often associated with evil Keep secrets (“Luke, I’m your father”) Are potentially harmful
- 3. Dark Matter? No, “Dark Data”! But they both Go undetected Are surrounded by detectable stuff Affect things around them
- 4. What is Dark Data? Dark Data in our digital devices Everyone creates it (unintentionally) Criminals may hide it (Anti-Forensics) Forensic tools can’t see it But it is there! Data that we can’t see On our hard drives On out flash drives In our computer files
- 5. Where is Dark Data? DCO & HPA Unformatted Disk Space Deleted Files Unknown Files Between Files Inside Common Files Deleted Data Objects
- 6. Hard Drive Layout Device Configuration Overlay (DCO) http://www.forensicswiki.org/wiki/SAFE_Block_XP Data Cleaner+ http://www.mp3cdsoftware.com/blancco---data-cleaner--download-16317.htm http://www.utica.edu/academic/institutes/ecii/publications/articles/EFE36584-D13F-2962-67BEB146864A2671.pdf Host Protected Area (HPA) http://www.thinkwiki.org/wiki/Hidden_Protected_Area Forensic Duplicator http://www.tableau.com/pdf/en/Tableau_TD1_Product_Brief.pdf HDD Capacity Restore Tool http://hddguru.com/software/2007.07.20-HDD-Capacity-Restore-Tool/ Unformatted Disk Space
- 7. Deleted Files Deleted Files aren’t really gone? Unused Disk Space (in a volume) Disk Caches / Swap Files Windows Recycle Bin Are they hard to recover? Fragmentation is deadly Large databases tend to be heavily fragmented Even DFRWS Researchers find that fragmentation can make some file types impossible to recover (http://www.dfrws.org/2007/challenge/results.shtml)
- 8. Unknown Files (1) 500 types of files handled by eDiscovery, Document Management & Computer Forensics Tools 50,000+* types of files in the world 5,000 types of files typically in use *http://filext.com
- 9. Unknown Files (2) Typical ToolsFI Tools (23 wrong files) (26 Correct Files)
- 10. Between Files Alternate Data Streams (ADS) Files hiding behind files (on NTFS) RAM Slack Padding between the end of a file and the end of the current sector Typically zeros, sometimes random content File/Cluster/Residual/Drive Slack Padding between sectors used & the end of the current cluster Previous sector content that should be used in File Carving http://www.forensics-intl.com/def6.html
- 11. Inside Common Files Deleted Objects Ex: Adobe PDF & MS Office 2003 (OLE) not removing deleted data (change tracking) Smuggled Objects Ex: MS Office 2007 (Zip) and MS Wave (RIFF) formats ignore foreign objects Object / Stream Slack Ex: OLE objects have sector size issues, just like with disk sectors Field Slack Ex: Image files that don’t use the whole palette, and/or less than 8/16/32/48 bpp Steganography
- 12. Smuggled Objects Some formats ignore foreign objects MS Office 2007 (Zip) MS Wave (RIFF) This example I added a file to a Word 2007 document. The document opens without any error.
- 13. Deleted Data in Slack Deleted Data that evades Redaction
- 14. Steganography Intentional Data Hiding
- 15. Dark Data Can Be Fragile Deleting Files without using the Recycle Bin. SHIFT + DEL Defragmenting a hard drive. Installing Applications. Turning off “Track Changes” & “Fast Save” options. Using Redaction Tools. MS Word - http://redaction.codeplex.com PDF - http://www.appligent.com/redax PDF - http://www.rapidredact.com Using Data Wipers. SafeErase - http://www.oo-software.com CyberScrub - http://www.cyberscrub.com
- 16. Dangers You may loose a law suit if the other side finds what you missed. Corporate Digital Assets may be walking out the door. Intellectual Property theft can put a company out of business.
- 17. Opportunities Protect your company by being Aware of your Digital Assets. Illegal content may be hidden accidentally or intentionally. Recover lost Digital Assets by knowing where to look. Employee misconduct is tracked by the hidden trail of improper acts. Catch Intellectual Property theft before it walks out the door. Identify in-house criminals by detecting their smuggling methods.
- 18. What Does FI Do? Create Technologies to Capture Dark Data File Investigator File Expander File Harvester Equip Law Enforcement with Tools FI TOOLS FI Object Explorer FI Data Profiler Portable
- 20. Thank you Contact Rob Zirnstein Rob.Zirnstein@ForensicInnovations.com www.ForensicInnovations.com (317) 430-6891
Editor's Notes
- This presentation was provided for an ARMA Indianapolis Chapter meeting.
- How did I get the term “Dark Data”? Not from Darth Vader, but they do have some things in common.
- I copied “Dark Matter”, because it also goes undetected yet still affects things (objects/solar systems) around it.This image was created by observing the gravitational effects on light and objects around the matter. No instrument can actually see the dark matter directly.
- Dark Data is in everything digital that we create, yet we don’t see it.
- Dark Data is hiding in the most unsuspecting places.
- DCO – Used to reduce the disk size to exactly match the size of another hard drive. This makes it easier to clone hard drives.HPA – Used to store vendor utilities on a hard drive, where a user can’t delete them.These areas are difficult to access and add or remove.Unformatted Disk Space is the remaining space that has not been allocated to a disk volume that the user can access.
- Many recovery tools falsely report their recovery success. Many of the successfully recovered files are actually corrupted with other file fragments.
- Most Forensics Tools keep these files in the Exception Bin. Have you ever seen an investigation with an empty Exception Bin? What if the best evidence was hiding in that Exception Bin?!?Ex: Hidden TrueCrypt volume file, that looks like random data.
- The list on the left was produced with Windows, as an extreme example. Although, many eDiscovery tools don’t do much better than this.The list on the right was produced by a tool that specializes in accurately identifying thousands of file types.Notice the 3 Alternate Data Streams identified on the right. They weren’t just detected, but analyzed to catch any hidden file types.
- Many tools combine RAM slack with Drive Slack. This causes confusion when file carving for partial files, because these slacks come from different sources.
- Common files may contain stowaways.Bpp = Bits Per Pixel
- Step 1: Rename the file to be smuggled to ‘document.xml’ (I used a simple text file)Step 2: Rename Word.docx to Word.zipStep 3: Open Word.zip with WinZipStep 4: Add the new smuggled ‘document.xml’ to Word.zip (in the root)Step 5: Rename Word.zip to Word.docx
- This example shows an MS Outlook Form Template that was edited to remove part of a sentence. The deleted content is still there!When the paragraph/object shrank, the Stream Slack inherited the end of the paragraph.Existing Redaction tools use Microsoft libraries that ignore the Stream Slack.
- Smuggled data is broken down into bits and substituted for picture data that doesn’t effect the visible image enough to be noticed.May just change 1 bit per pixel, or fill the Field Slack.The smuggled data may also be encrypted before insertion.
- Here are some methods for cleaning out and preventing Dark Data.
- Researching tools that can track & redact metadata andDark Data artifacts is vital in your fight against misconduct. If your IT department isn’t doing this, then you are your company’s last line of defense.