Inves&ga&ng Coordinated Data Exﬁltra&on Golden G. Richard III University of New Orleans and Digital Forensics Solu9ons, LLC & Andrew Case Digital Forensics Solu9ons, LLC
2 Speaker’s Introduc&on (1) Golden G. Richard III • Professor of Computer Science and University Research Professor @ University of New Orleans • Director, Greater New Orleans Center for Informa9on Assurance (GNOCIA) • Co-‐founder, Digital Forensics Solu9ons, LLC • GCFA cert • United States Secret Service Cybercrime Taskforce • Member of the American Academy of Forensic Sciences (AAFS)
3 Speaker’s Introduc&on (2) Andrew Case • Senior Security Analyst • GCFA cert • Blackhat, DFRWS, and SOURCE speaker • Experienced digital forensics inves9gator, penetra9on tester, and reverse engineer
4 Digital Forensics Solu&ons / UNO • Digital Forensics Solu9ons, LLC – New Orleans company with oﬃces in the Garden District – Full service digital forensics, data recovery – Rela9onships for seamless digital forensics / e-‐discovery – Security assessment, secure erasure of media, security training – Research: new tools and techniques • GNOCIA / University of New Orleans – Pioneering curriculum in digital forensics and reverse engineering – Digital forensics research: new tools and techniques – Educa9on: Crea9ng a strong local tech workforce – Liaison with local, state, federal law enforcement to solve diﬃcult cases
5 The Purpose of This Talk • Provide some basic background on digital forensics techniques applicable to data exﬁltra9on cases • Illustrate the extent to which data exﬁltra9on can be performed in a straighZorward manner on a normal computer • And how data exﬁltra9on can be inves9gated • A recent case we inves9gated required analyzing almost every common data exﬁltra9on technique • We believe this case serves as a great learning example for other inves9gators
Digital Forensics? ★★ • (Benevolently) prey on mechanisms designed with performance (not privacy) in mind • Crea9ve uses of data intended mostly for other things • Correla9on of simplis9c data sources to create richer context • In some cases: logs, etc. actually meant to be used for forensic purposes
7 Agenda • Introduc9on to Data Exﬁltra9on Issues • Overview of our Recent Case • How to Inves9gate Exﬁltra9on • Wri9ng a Proper Case Report • Conclusion [Some brief background on various digital forensics issues and techniques as we go—please feel free to ask ques&ons to clarify anything that isn’t clear]
8 Data Exﬁltra&on Introduc&on • Data exﬁltra9on is the removal of sensi9ve informa9on from an owner’s control • Common examples include: – A rogue employee removing informa9on from a company’s computer systems – Aaackers stealing data aber they have gained access to an internal network – Malware stealing and expor9ng sensi9ve data
9 How Exﬁltra&on Occurs 1. A malicious user (or program) gets access to sensi9ve data 2. The data is then gathered and moved outside of the owner’s network 3. Commonly used methods • Removable Media (USB, CD/DVD, Smartphones) • Internet-‐Based (Email, File Uploads, Dropbox, FTP, SCP, etc.) • Malware (transmission via email, TCP, UDP, etc.)
10 Consequences of Exﬁltra&on • Consequences can be severe • Immediate eﬀect: – Loss of intellectual property and other sensi9ve informa9on – Expensive incident response process must begin – Possible requirements for disclosure to be made and compensa9on of aﬀected par9es • Long term eﬀect: – Loss of trust by clients – Liability / Lawsuits / Other legal issues
12 Preliminary Informa&on • A former employee of a ﬁnancial ins9tu9on (our client) was suspected of stealing sensi9ve informa9on and using it to bring business to his new employer • We were to inves9gate: 1. Was data stolen? 2. If so, how? 3. What data was taken 4. If other people were involved in the incident
13 Data/Equipment to Inves&gate • We were given the suspected user’s laptop • The user’s Blackberry was remote wiped upon his leaving the company as per-‐policy – No backups made before wiping – Never got access to this informa9on • We were supposed to receive a copy of the user’s archived Outlook email (PST ﬁle) – This was never provided
15 Ini&al Analysis • Imaged hard drive of laptop • The suspect’s laptop was running XP SP2 • Internet Explorer only browser installed • The user was not a local administrator • The machine had over 20 System Restore Points – We will be discussing the importance of this throughout
16 System Restore Points • System Restore Points are created to backup cri9cal ﬁles when de-‐stabilizing opera9ons are performed on the OS – System updates – 3rd Party sobware installa9ons – Installa9on of unsigned drivers – … • Good source for historical copies of the Windows registry • In our case, System Restore Points allowed orderly examina9on of data over ﬁve months old
17 Inves&ga&on Flow • Inves9gate Removable Media – Determine which removable media was used, which ﬁles were moved, when they moved, and to where • Inves9gate Web Based Ac9vity – Determine if ﬁles were transferred over network • Inves9gate Accessed Files – Find any ﬁles that were inappropriately accessed • Determine if other people were involved – Look for emails and other communica9on
19 First Steps • USB history analysis typically requires analyzing two sources: – USBSTOR registry informa9on – The setupapi.log ﬁle – Renamed and split under Win7: • setupapi.app.log and setupapi.dev.log • Details aber a brief discussion of the Windows registry
21 Windows Registry • Can be a forensics goldmine • Lots of informa9on, fairly diﬃcult to “clean” • Usernames • Internet history • Program installa9on informa9on • Recently accessed ﬁles • Devices (USB, et al) • Network conﬁgura9on
22 Registry: Windows 9x • On Windows 95/98: • “system.dat” and “user.dat” ﬁles • If mul9ple users, look in Windowsproﬁles<acct> for individual user.dat ﬁles • “system.dat” – System-‐wide informa9on • “user.dat” (one “original” one, then others as users are created) – User informa9on • Careful, because on Windows 9x, new user proﬁles are oben based on previously created proﬁles!
23 Registry: NT/Win2K/XP • “ntuser.dat” – List of most recently used ﬁles – Each user has a separate “ntuser.dat” ﬁle – documents and sesngsuser • “default” in <windowsdir>system32conﬁg – Ini9al system sesngs • “SAM” – User account sesngs, security sesngs • “security” – Security-‐related sesngs • “sobware” – Installed programs, sesngs, usernames, passwords • “system” – Misc. system sesngs
25 ** VERY IMPORTANT ** “Select” key chooses which control set is current, which is “last known good” conﬁgura9on SYSTEM ﬁle Copyright 2004-‐2011 by Golden G. Richard III.
26 What user accounts are on the machine? SAM ﬁle Copyright 2004-‐2011 by Golden G. Richard III.
27 Which &mezone does the computer use? SYSTEM ﬁle Copyright 2004-‐2011 by Golden G. Richard III.
28 Which ﬁles were recently accessed by a par&cular user? NTUSER.dat ﬁle Copyright 2004-‐2011 by Golden G. Richard III.
29 Which URLS were typed recently by a par&cular user? NTUSER.dat ﬁle Copyright 2004-‐2011 by Golden G. Richard III.
30 SOFTWARE ﬁle Which programs are installed on the machine? Which license keys are in use? Copyright 2004-‐2011 by Golden G. Richard III.
31 Which programs run automa&cally when a par&cular user logs in? NTUSER.dat ﬁle Copyright 2004-‐2011 by Golden G. Richard III.
32 Which programs run automa&cally when ANY user SOFTWARE ﬁle logs in? Copyright 2004-‐2011 by Golden G. Richard III.
33 Two Jumpdrive Elite thumbdrives 750GB USB hard drives (same type) What has been plugged in? SYSTEM ﬁle Copyright 2004-‐2011 by Golden G. Richard III.
34 Networking info SYSTEM ﬁle Copyright 2004-‐2011 by Golden G. Richard III.
35 Disk info SYSTEM ﬁle Copyright 2004-‐2011 by Golden G. Richard III.
36 Summary: Registry Forensics • Last write 9mes for individual registry keys can be used to infer useful informa9on • Overall, lots of informa9on, some of which can’t be obtained elsewhere • Extreme care is needed during analysis • Lots of mysterious data • Much of the informa&on is essen&ally undocumented and meaning is determined experimentally
37 USBSTOR • The SYSTEM registry hive contains a history of connected USB devices – Registry ﬁles backed up by System Restore Point facility • All of this informa9on is stored under the CurrentControlSetEnumUSBSTOR key • Contains an entry for each USB device that was connected to the machine • Also contains the “Friendly Name” and serial number of each aaached device • The only 9mestamp informa9on available is last wriaen 9me for key corresponding to par9cular USB device!
38 Analyzing the Registry Files • Aber gathering all of the SYSTEM ﬁles… – Current – Historical (via System Restore Points) • …Used Regripper  USBSTOR plugin to enumerate previously aaached USB devices • Then wrote a wrapper script to dump this informa9on into Excel • Now we had informa9on on connected USB devices going back many months
39 Results of USBSTOR Analysis • Eight USB drives were used during the target 9me range – Six were thumb drives with capacity ranging from 2 to 8GB – One USB device was the previously men9oned user’s Blackberry smartphone – Last was a digital camera • Next step was to determine the extent of use for the six thumb drives
40 Analyzing setupapi.log • Text ﬁle in c:Windows (under XP) • Tracks device installa9on, service-‐pack installa9on, hoZix installa9on, etc. for the setup applica9on • Reveals the ﬁrst 9me each device was plugged in, as Windows selects appropriate device drivers • USBSTOR registry key tells us the last 9me a device was connected • We used SetupAPI Extractor  to analyze the ﬁle rather than simply viewing it as a text ﬁle
41 Using setupapi.log Informa&on • Using the ﬁrst and last connect 9mes gives us a 9me range for each device • Use this informa9on to assign drive leaers to speciﬁc thumb drives – Discussed next • Also helped build a clearer 9meline of the suspected user’s ac9vity
42 Inves&ga&ng Individual Drives • Used procedure illustrated on next slide to determine: – Drive leaer mapped to a USB device – The ﬁrst and last 9me each device was connected • Have to be careful when assigning drive leaers – Mul9ple drives can be mapped to same leaer over 9me – Need to correlate 9me informa9on between drive and ﬁles accessed to substan9ate
45 Email Examina&on: Overview • Two email services were used to exﬁltrate ﬁles: – Gmail – Company Email (Exchange) • We were told during the pre-‐inves9ga9on phase that the IT team knew of a Gmail account for the user under inves9ga9on • Needed to ﬁnd all contact with suspect’s new employer while s9ll employed by our client • We didn’t have PST access, our only hope was web-‐ based email • Knew that only fragments would be recovered from Gmail
46 Inves&ga&ng Gmail • Two pieces of evidence were discovered from Gmail: – A number of ﬁle exﬁltra9on instances – Evidence of contact between suspect and new employer well before our client suspected • How did we ﬁnd this informa9on?
47 Gmail: Technical Details • Gmail makes a number of eﬀorts to avoid disk forensics of messages read and sent – Puts messages in separate iframes – Uses SSL and no-‐cache browser direc&ves • Uses similar techniques for other parts of the Gmail interface – Contacts, labels, searches, etc. • Essen9ally, simple examina9on of browser cache isn’t going to yield much
48 Scalpel Overview • File carving is typically used to recover deleted ﬁles, based on the structure of ﬁle types • Scalpel is a ﬁle carver , but can also be used as a very eﬃcient indexer for speciﬁc search terms – Latest version is mul9threaded and can use GPUs (CUDA) for high performance opera9on • The audit ﬁle created by Scalpel (audit.txt) contains loca9ons of every discovered instance of every search term
49 Using Scalpel • We ran Scalpel to ﬁnd all instances of the new employer’s email domain • We then used the Sleuthkit to quickly map these oﬀsets to ﬁles within the ﬁlesystem – See  for an updated method on how to do this • Produced hits in both web cache ﬁles and pageﬁle.sys, the Windows swap ﬁle
50 pageﬁle.sys Analysis • Hits in pageﬁle are on previously viewed Gmail Inbox indices (illustrated on the next slide) • These indices contain a number of useful ar9facts about email messages: – Time received – Message fragment – Sender – Aaachment names (if any)
51 Gmail Inbox View • The image above is a screenshot of the Inbox view • The default view shows 50 messages • We were able to recover a number of instances of these using Scalpel on the pageﬁle
52 U&lizing Message Fragments • Aber gathering all message indices discovered in the pageﬁle… • …We created a new Scalpel conﬁg ﬁle and carved again on the pageﬁle to try to recover message fragments • This produced fragments of en9re message bodies sent through Gmail by the rogue employee – This is where it got interes9ng!
53 Message Fragments: Gold Mine • The recovered message bodies revealed the employee under inves9ga9on had contacted his new employer a number of months before leaving the company • Well before our client had suspected • The uncovered messages were par9cularly damaging • Revealed precise details of plan to steal and later u9lize our client’s data
54 Gmail Aeachments • Aber discovering aaachment names in the fragments, we used this data to discover which ﬁles were transferred • Analysis revealed a number of ﬁles were emailed from the user’s local Outlook installa9on to his Gmail account • Filenames were matched to those in LNK ﬁles and MRU lists (discussed later)
57 Analyzing Browser History Using IEHistoryView
60 Flash Cookies • Flash applica9ons are provided client storage through local shared objects (LSOs) • Browsers are only recently giving users the ability to delete them – Previously had to ﬁnd LSOs within the ﬁlesystem and manually delete • Stored outside of the normal cookie/cache storage subsystem – “Private” browsing modes DO NOT aﬀect ﬂash cookies! • Analysis leads to informa9on about websites visited, when they were visited, etc.
61 Analyzing Flash Cookies • The loca9on of the ﬁles is opera9ng system dependent: – hap://en.wikipedia.org/wiki/ Local_Shared_Object#File_loca9ons • A few tools exist for analysis, but none seem completely stable: – Minerva -‐ hap://blog.coursevector.com/ minerva – SOLReader -‐ hap://www.sephiroth.it/python/ solreader.php
62 Using Browser Analysis • Browser analysis revealed many accesses to Gmail as well as informa9on related to the new employer • “9tle” and other URL informa9on recorded in the history ﬁle helped in analysis discussed later
Inves&ga&ng Coordinated Data Exﬁltra&on (2nd Hour) Golden G. Richard III University of New Orleans and Digital Forensics Solu9ons, LLC & Andrew Case Digital Forensics Solu9ons, LLC
65 Inves&ga&ng Files Transferred During the Exﬁltra&on
66 Recap • At this point in the inves9ga9on we have: – Shown that a number of thumb drives were previously aaached to the computer under inves9ga9on – That ﬁles were sent to an external Gmail address from a company Email address – That the target employee had contacted his new employer many months before leaving our client
67 Updated Workﬂow • We now had two goals: – Find out which ﬁles were accessed by the user – Find out which were then transferred onto USB drives – Determine the loca9on of the ﬁles sent via Gmail
68 Finding Accessed Files • Windows provides a number of forensics ar9facts related to historical ﬁle access • Three main ones were used in this inves9ga9on: – LNK Files – MRU Lists – File Access History
70 LNK Files • Link ﬁles (.lnk) are Windows shortcut ﬁles • Similar to symbolic links under Unix • The metadata contained in these ﬁles is very useful during forensics inves9ga9ons – MAC 9mes of target ﬁle – Full path to target ﬁle – Whether target is/was local or on the network – Network share informa9on – Volume serial number (used to match to speciﬁc drive)
71 lnk-‐parse  on a Local File MAC Times of Target File Target Hard Drive Target File
72 parse-‐lnk Output for Network Share MAC Time of Target File Size of File The network share related to the ﬁle, including path
73 Using LNK Files • The target computer had a large number of relevant LNK ﬁles • (Some) LNK ﬁles are backed up within System Restore Points! • These ﬁles were helpful for two purposes: 1. Iden9fying which ﬁles were moved to which USB drives 2. Iden9fying which ﬁles were downloaded from which network shares • More on this in a minute…
74 Automa&ng LNK File Analysis • Since there were so many LNK ﬁles, we needed to automate the process • Wrote a script to parse lnk-‐parse output and write contents to an Excel sheet • Could then quickly determine which ﬁles, network shares, and 9mes were involved in the exﬁltra9on
75 LNK File Research • There a few very good resources on LNK ﬁle analysis: – “The Meaning of Life”  • 21 page research paper on analysis with LNK ﬁles – Forensics Wiki Page  – Forensics Focus Ar9cle 
77 Most Recently Used (MRU) Lists • MRU lists store informa9on about the documents most recently accessed by a user for a par9cular applica9on • Stored in the Windows Registry – Again, System Restore Points give us access to historical MRU lists as well as current ones • Common examples are when you click ‘File’ in an applica9on’s menu and see a list of previously opened documents
78 Popular MRU Lists • Microsob Oﬃce – For all applica9ons (Word, Excel, PPT, etc) • Internet Explorer – Recently typed URLS (The URL dropdown) • Adobe – Recently accessed PDF ﬁles • An extensive list of over 30 MRU loca9ons and associated applica9ons can be found at 
79 Using MRU Lists • Gathered the current and historical SOFTWARE registry ﬁles • Used Regripper to acquire all of the relevant MRU lists – Most important were Oﬃce and Adobe • (Again) we wrote a script to parse output and write to an Excel sheet
80 Analyzing the MRU lists • The combined MRU lists provided ﬁlenames and paths to numerous ﬁles of interest to the case – Spread out across the local drive, thumb drives, and network shares • A number of these ﬁles were also duplicates of those found in the LNK ﬁles – Great for correla9on and soundness of ﬁndings
82 More File Accesses • Web browser history also revealed access to a number of internal web applica9ons that create reports • The ﬁlename of these reports contained the parameters (date, search, etc) used to create them – This was visible in the URL (GET parameter)
83 Web Applica&on Reports • We then found copies of these reports on the local machine • Contained informa9on on other employees that the target user was not oﬃcially authorized to view
84 “File” Accesses • The “browser” history ﬁles also keep records of access to speciﬁc ﬁles (ﬁle:///) – Including full path name and MAC 9me type informa9on • Analysis of these ﬁles on the target machine revealed access to more unauthorized ﬁles – Beyond what was found through LNK and MRU analysis
86 Recycle Bin Forensics • Windows trash can facility for dele9ng ﬁles • Files maintained in a hidden directory un9l the user emp9es the Recycle Bin, then insecurely deleted • The Recycle Bin maintains a history of ﬁles deleted within INFO2 ﬁles • INFO2 ﬁles contain: – The fullpath of the deleted ﬁle – The date the ﬁle was moved to the recycle bin – The sequence in which ﬁles were moved to the recycle bin • A great resource on INFO2 analysis can be found at 
87 Analyzing the Recycle Bin • Analysis of INFO2 ﬁles found on the target machine revealed that many of the ﬁles found through previous analysis had been deleted by the user • The 9mestamps of the dele9on were very close to the exﬁltra9on 9mes • Very damaging evidence
89 Network Share Access • In many corporate environments, including the one in this case, departments store all informa9on on network shares • Employees should technically only have access to speciﬁc ﬁles, but implemen9ng this properly is painful • This makes inves9ga9ng network share access a must in data exﬁltra9on cases
90 Analyzing Network Shares • CurrentControlSetServicesLanManagerShares contains informa9on about network shares on the computer – Again, historical records were also available through restore points – Allowed quick mapping of drive names to places on the network
91 Using Network Shares • Aber determining which drive leaers corresponded to which network shares, we gathered the ﬁlenames that were accessed • We then sent this informa9on to the IT security team – They were able to ﬁnd all these ﬁles and we subsequently used this informa9on in our report
93 Results So Far • At this point we had a wealth of informa9on: – We knew exﬁltra9on occurred over USB devices and Gmail – We knew which ﬁles were transferred and the 9me/date of transfer for some of them – We knew that contact was made with the future employer and exact details
94 Data to Correlate • We had drive leaers, ﬁlenames, and access 9mes from our evidence sources • Needed to create a 9meline of user ac9vity for each ﬁle of interest – File Access – File Transfer (if any) – File Dele9on (if deleted)
95 Performing the Correla&on • Used access 9mes from LNK ﬁles, browser history, etc. to determine when interac9on with a ﬁle started • Used LNK ﬁles related to USB drives to determine when copied • Used browser history and Gmail view index to determine when a ﬁle was emailed • Used INFO2/Recycle Bin to determine if/when a ﬁle was deleted
96 Correla&on Results • For many ﬁles of interest, we could show that, within a 5 minute 9me period, the ﬁle was accessed, exﬁltrated, and then deleted • We could also which ﬁles were simply viewed and then discarded • Made for compelling (and hard to refute) evidence
98 Next Steps • Our last step was to determine if other employees were involved • We requested a list of ﬁrst and last names, user logins, and email addresses from IT security for: – Close co-‐workers of the target – Other people who recently leb the company • We used this informa9on as our star9ng point…
99 Inves&ga&on Process • We took the informa9on given from IT to build a Scalpel conﬁgura9on ﬁle as previously described • This would (hopefully) ﬁnd all informa9on related to these other employees…
100 First Clue • Emails were found between the suspect and his secretary, related to the new company • We then requested the computer of the secretary • Analysis of her computer revealed sharing of USB thumb drives – Based on USB serial numbers and inves9ga9on of USBSTOR in the registries
101 Further Analysis of the Second PC • Similar evidence was found on the secretary’s PC as on the ini9al targets – Use of removable media – Downloading of unauthorized ﬁles from ﬁleservers – Emailing of ﬁles to outside accounts • Also found emails to a third person within the organiza9on 101
102 Analyzing Employee Three • Aber ﬁnding emails from secretary to employee three, we requested his computer as well • Analysis of this computer revealed sharing of USB drives by all three employees • Also revealed contact by employee three to new company
104 Mortal Sins of Repor&ng • Do NOT: • Include opinions (especially legal ones) – You weren’t asked to be a lawyer – Will hurt your credibility • Include informa9on you could not verify – Will come up in tes9mony and can hurt your credibility
105 Report Outline • Every report should contain at least these sec9ons: – Execu9ve Summary – Evidence Catalogue – Findings Sec9ons – Conclusion – Aaachments
106 Report -‐ Execu&ve Summary • Should contain a high level overview of the case results and be less than one page • Purpose is to allow execu9ves to quickly understand the outcome of the inves9ga9on • Should answer three ques9ons: – Was data exﬁltrated? – If so, were you able to conclude who was responsible for the exﬁltra9on? – If so, what data was taken and how much of it?
107 Report -‐ Evidence Catalogue • The rest of the report should be for managers and IT staﬀ who need technical details • The evidence catalogue should contain these: – A descrip9on of all evidence analyzed – A picture of each piece of evidence – Any unique informa9on (serial numbers) – Hashes of the data, if applicable – How copies of the evidence was acquired
108 Report -‐ Findings Sec&ons • The bulk of the report should be your ﬁndings • Should be broken into logical sec9ons – Similar to how this presenta9on ﬂowed • Needs to include: – Your exact inves9ga9on methodology – A lis9ng of tool(s) used – The relevance of each ﬁnding to the case
109 Report -‐ Conclusion • The conclusion should be a factual summary of the case – Again -‐ NO opinions • Can include recommenda9ons for further inves9ga9on – For example, our ini9al report recommended acquiring the computer of the secretary
110 Report -‐ Aeachments • All processed data from the case, such as the Excel sheets we men9oned, should be included as aaachments to the report – On digital media (CDs, DVDs, etc.) – Or printed, as appropriate • This makes handling the ﬁles (prin9ng, searching, etc) much easier for everyone involved
111 Conclusions • Data exﬁltra9on inves9ga9on is a labor-‐ intensive process • Requires a wide range of skills on part of the inves9gator – We only inves9gated Windows machines during this case, and s9ll needed a number of tools and skillsets • The resul9ng report must be carefully wriaen