First Responders Course - Session 7 - Incident Scope Assessment [2004]


Published on

The seventh session from a two day course I ran for potential first responders in a large financial services client.

Published in: Business, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

First Responders Course - Session 7 - Incident Scope Assessment [2004]

  1. 1. Phil HugginsFebruary 2004
  2. 2.  Description Strategy Meeting Documentation Debugging Log Processing Basic Host Analysis Rootkits Information Gathering
  3. 3.  The Scope Assessment Phase has the followinggoals: To confirm the existence of the incident To identify which systems (if any) are involved in the incident To estimate the damage (if any) done to involved systems To identify if the attack is still underway To identify the complexity of the incident To gather any other data needed to make decisions on how to respond Sources of data: Logs Network Monitoring AssessmentAnalysis
  4. 4.  Large or complex incidents may require an initialstrategy meeting to coordinate efforts This tends to be a more technically focused meetingthan the initial team meeting discussed yesterday This meeting will identify who is responsible for: Verifying the initial report Verifying that similar systems were not affected Watching for an additional incident Deploying additional monitoring tools
  5. 5.  Document everything (even mistakes) Trust nothing on the suspect system Suspect systems should be modified as littleas possible Chain of Custody forms should be generatedfor all evidence
  6. 6.  Debugging is simply “finding what’s wrongwith stuff” Obvious principles but MUST be applied Book Recommendation: Debugging by David J. Agans, ISBN 0-8144-7168-4
  7. 7.  Understand the system Make it fail Quit thinking and look Divide and conquer Change one thing at a time Keep an audit trail Check the plug Get a fresh view If you didn’t fix it, it ain’t fixed.
  8. 8.  The goal is to find new clues and validate other findings Using information that is already known about the incident,consult logs for additional clues Extract logs that reference suspect systems from devicesbetween the gateway and the suspect systems (using grep) If a time frame is known, extract all logs from that time ongateways and remote access devices (stone-steppingscenario) Identify additional hosts that have similar log activity or thatmay have been used as a stepping stone Generate MD5 values of extracted logs Ensure that logs from the incident timeframe are notoverwritten
  9. 9.  In some cases, an analysis needs to be performed on acompromised system before a forensic acquisition occurs The goal of this analysis is to identify the scope of damageand quickly gather additional clues The analysis may answer: Have hiding mechanisms, such as a rootkit, been installed Who recently logged on and from where Were log files modified What files were recently created or modified If it is suspected that there are “time bombs” or other“traps”, then the system should be unplugged and onlyexamined with a trusted kernel
  10. 10.  Document everything The “AccessTime” of files will be updated when you viewtheir contents, record which files you look at so thosetimes can be explained Send log files to an evidence server via netcat, calculatean MD5 value, and analyze that copy Trust nothing on the suspect system Use only trusted tools from an response kit CD-ROM Kernel Module rootkits will hide data even with originalbinaries
  11. 11.  Suspect systems should be modified as little as possible Use a tool such as mac-robber ( or mac-daddy ( to collect the MACtimes of files before they get modified during the analysis Use a tool such asThe Sleuth Kit ( to analyzethe file system from the raw device (the MAC times will not be modified) Use tools such as Afind fromThe Forensic ToolkitVersion 2.0( under resources and free tools) to search for recentlyedited files on Windows systems.. Stop schedulers from running commands on system Do not write files to the disk, it will overwrite deleted content. Instead pipe datausing netcat to the evidence server or to a floppy diskOn Evidence Server:# nc -l -p 9000 > wtmp.logOn Suspect system:# cat wtmp.log | nc -w 5 9000
  12. 12.  Volatile data acquisition procedures should be donefirst to collect the data before it could be modified(we will cover this later) netstat ps / pslist lsof / handle / fport etc. Examine the output (on the evidence server) forsuspicious processes, open ports, and logged onusers
  13. 13.  All files have at least 3 times associated with them(Modified, Access, and Change) Timelines can be created with file activity at anytime For UNIX hosts,The Sleuth Kit can collect the datafrom the raw device and not modify the file system An alternative is mac-robber or mac-daddy, whichwill modify the access times of directories Both approaches will send data to an evidenceserver where it is processed and analyzed
  14. 14.  Sleuth Kit:# fls -f solaris -m / -r/dev/rdsk/c0t0d0s0 | nc -w 510.0.0.1 9000# ils -f solaris -m /dev/rdsk/c0t0d0s0 |nc -w 5 9000…. (repeat for each partition) mac-robber:# mac-robber / | nc -w 5 9000 mac-daddy:# perl / | nc -w 5
  15. 15.  On the evidence server (a new file for each partitionwith the Sleuth Kit):# nc -l -p 9000 > mac_1.dat Sleuth Kit and mac-robber require a processing toolfrom the Sleuth Kit:# mactime -b mac_1.dat 01/01/2002 > Refer to the timeline.README document in theSleuth Kit for details (
  16. 16.  DIBS MycroftV3 Very fast and cheap
  17. 17.  Rootkits are installed by attackers to: Hide files and processes that they created Collect data (such as logins and passwords) from thenetwork or local system Provide a back-door method of gaining access to thesystem Remove evidence of previous attack There are two major varieties of data hiding: Classical binary modification Kernel Modules
  18. 18.  The original system binaries are modified to read aconfiguration file The configuration file contains a list of processes orfiles to hide These can be detected by comparing the MD5 valueof current binary with one from a non-compromisedsystem (change management) In basic versions of this, running ‘strings’ on thebinary will show the location of the configurationfile (/dev/ptx0)
  19. 19.  Contents of a process config file (LRK 4)2 slice22 snif2 pscan2 imp3 qd2 bs.sh3 nn3 egg.lin Contents of a file hiding config file (LRK 4)tcp.logslice2scanapaddy.awk.fakeid
  20. 20.  Strings of a trojaned (LRK 4) ps binary:<…>90t:u&Vh/dev/ptypNR PID STACK ESP EIPTMOUT ALARMPID TTY MAJFLT MINFLT TRS DRSSIZE SWAP<…> /dev/ptyp file is a regular file, not a device /dev/ptyp0, /dev/ptyp1, etc. are valid devices
  21. 21.  Compare MD5 values of binaries with: Trusted system with same patch level Solaris Fingerprint Database ( NIST NSRL ( Linux RPM (with -V a flag) Compare output of system binaries withtrusted binaries on a CD chkrootkit ( signatureanalysis
  22. 22.  Kernel Module rootkits modify the kernelsystem call table instead of modifying thebinaries These rootkits prevent the kernel from givinginformation on the processes and files thatare in a configuration file These are harder to detect because the MD5of the binaries remain constant
  23. 23.  Normally, tools like ‘ps’ and ‘ls’ use theAPI torequest a list of processes or files from theKernel
  24. 24.  A rootkit goes between the Kernel and API Now, the API requests a list of processes or filesfrom the Rootkit, which forwards the request to theKernel and then filters out the “hidden” data.
  25. 25.  Trojan ‘sshd’ and ‘tcpd’ servers also exist to allowaccess ‘ifconfig’ can be trojaned to hide the Promiscuousflag Padding can be added to the end of new binaries tomatch the CRC value of the original Use an accepted hashing algorithm such as MD5 or SHA-1
  26. 26.  New open network ports (nmap port scan) Promiscuous network interface (AntiSnif) Updated patch levels Modified logs AntiVirus software
  27. 27.  Different output from ‘nmap’ than ‘netstat’ Different output from ‘ls’ than the Sleuth Kitor Encase Carbonite chkrootkit Kstat Intrusion Prevention Systems
  28. 28. UNIX Windowst0rn NetBusAdore (LKM) Back OrificeSLKM (LKM) Sub SevenLinux RootKit(LRK)NT RootkitRomanian VanquishAcquatica HE4Hook
  29. 29.  Check MD5 values of ‘ls’, ‘ps’, ‘netstat’, ‘sshd’binaries Compare output of nmap port scan and netstat Look for text files in /dev/ or directories that startwith a ‘.’ in UNIX Compare output of ‘ls’ with that of the Sleuth Kit Examine a file activity timeline created by theSleuth Kit (not mac-robber or mac-daddy)
  30. 30.  Data can also be hidden while not using rootkits UNIX files and directories that start with a ‘.’ are notshown by default:# find / -name “.*” -print NTFS Alternate Data Streams are not shown bydefault:C:> echo “test” > file1.txtC:> echo “hidden test” >file1.txt:hidden Crucial ADS, sfind, and the Sleuth Kit will show theirexistence
  31. 31.  Copy logs to evidence server for analysis (usingnetcat as previously described) Look at wtmp logs on UNIX and run integrity checksto see if it has been modified Look at other logs and correlate entries withremotely stored copies or network device logs Copy Event Logs fromWindows and open in EventViewer (will be missing some application log text) Don’t forget to generate MD5 values
  32. 32.  To analyze a UNIX system, a CD withThe Sleuth Kit, Autopsy,and other utilities can be created for remote analysis. Autopsy is HTML-based, so it is run from the CD and listenson a given port The investigator connects to the port on the suspect systemand can browse the file system through the raw device This means that no files are modified and that rootkits willbe bypassed EnCase Preview offers a similar function forWindowssystems
  33. 33.  Internet-based Research Sanitize your location & be careful where youvisit Use a dial up account NOT the corporatenetwork Mailing lists may contain additionalinformation - google searches If IRC information or IP addresses are found,it is not recommended that you join the IRCchannel or do a port scan of the host
  34. 34.  Converting an IP address to a hostname or ahostname from an IP ‘dig’ collects data about domains and networksfrom DNS records
  35. 35.  ‘whois’ returns contact information for an IP address
  36. 36.  traceroute may show where a host is located(based on hostnames of back-bone devices)
  37. 37.  Powerful collection of ‘network detective’ tools run fromthe web site. Windows tool for download. American IP Allocation Database European IP Allocation Database Asia Pacific IP Allocation Database
  38. 38.  This phase should answer: Which systems are involved and to what extent? How critical is each involved system? Which systems do we need to acquire? Is the attack still in progress? Is there an ongoing threat? Do we want to prosecute? Are more monitoring and logging needed for theinvestigation? Are there any suspects? Is this from an insider?
  39. 39.  This phase collects data to identify the scope of theincident The types of activities of this phase will depend onthe type of incident The data collected will be used in the ResponsePhase, which will decide whether it is necessary touse additional monitoring or do an acquisition Documentation and non-intrusive analysis arecrucial Chain of Custody is important if prosecution is likely
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.