Reverse Engineering

1. NTXISSA.org
Reverse Engineering Binaries to
Detect Malware
John Podolanko, CISSP
Cybersecurity Manager
Tenet Healthcare
5 October 2018

2. NTXISSA.org
Malware Analysis Techniques
• Basic Static Analysis
• Examines binary executable without running it
• Can confirm if file is malicious
• Provide basic info but can miss important behavior
• Produce simple signatures
• Quick but ineffective
• Tools: VirusTotal, strings, BinText, UPX, PEiD, LordPE, PEView, Dependency Walker
• Basic Dynamic Analysis
• Run malware and monitor its effect
• Use a VM and take snapshots
• Produce more effective signatures
• Can be used by weak coders
• Still somewhat ineffective for more advanced malware
• Tools: RegShot, Process Monitor, Process Explorer, ApateDNS, Wireshark, INetSim,
netcat
2

3. NTXISSA.org
Malware Analysis Techniques
• Advanced Static Analysis
• Reverse-engineering with a disassembler
• Complex, requires understanding of assembly code
• Tools: IDA Pro
• Advanced Dynamic Analysis
• Run code in a debugger
• Examine internal state of a running malicious executable
• Tools: Process Hacker, CaptureBAT, Cuckoo Sandbox
• RAM Analysis: Mandant Redline and Volatility
3

4. NTXISSA.org
General Rules for Malware Analysis
• The Devil is in the Details
• You don’t need to understand 100% of the code
• Focus on key features
• Try Several Tools
• If one tool fails, try another
• Don’t get stuck on a hard issue, move along and
come back later
• Malware authors are constantly raising the bar
4

5. NTXISSA.org
Setting up Analysis Environment
• VMware Workstation preferred
• Windows 7 Professional/Enterprise VM
• x64 preferred, but x86 acceptable
• 40-60 GB Hard Disk Drive space
• 2-4 GB RAM
• Be sure to install VMware Tools!
• https://developer.microsoft.com/en-
us/microsoft-edge/tools/vms/
5

6. NTXISSA.org
Configuring VMware
• Create Custom Virtual Network
• Why shouldn’t you disable networking?
• It can cause the malware to fail
• Can’t analyze malicious network activity if disabled
• Setup Host-Only Networking
• Not connected to Internet
• NOTE: Nothing is 100% secure
6

7. NTXISSA.org
Using Multiple VMs
• Connect all VMs to same
Custom Network
• VMnet <#>
• Use Virtual Machine Teams
• Services VM
• Analysis VM
• Connected to each other
but not to host
7

8. NTXISSA.org
Using Your Dynamic Analysis VM
• Connecting to the Internet
• Bridged adapter
• Shares same physical interface
• Network Address Translation (NAT) mode
• Shares host IP and Internet connection
• Host acts as router
• Makes it difficult to connect VM adapter to host’s LAN
• Ideal if host connected via Wi-Fi
• Connecting/Disconnecting Peripherals
• Connect to either host or VM, but not both
• Examples
• CD-ROM
• USB storage
• Be wary when inserting USB storage devices
• Malware may spread if VM window active when inserted
8

9. NTXISSA.org
Using Your Dynamic Analysis VM
• Save VM’s current state and
restore later
• Always have a base snapshot to
revert to just in case
• Using multiple snapshots
encouraged
• Can be done to revert to middle
of malware analysis
• Snapshots can even be branched
• Transferred files can be lost
after revert
• Transfer via shared folders or
drag-and-drop (VMware and
VirtualBox only)
9

10. NTXISSA.org
Risks of VMware Malware Analysis
• Malware may behave differently if it detects VM
• VMware doesn’t consider this a vulnerability
• VMware occasionally has vulnerabilities
• Can cause host OS to crash
• Can run code on host OS
• Shared folders commonly found to be exploitable
• Tools exist to exploit drag-and-drop
• Keep VMware up-to-date
10

11. NTXISSA.org
VMware Record/Replay
• VMware allows you to record everything
• Replay the recording later
• Every instruction executed during original
recording is executed during replay
• Can replay unique, unrepeatable scenarios
• Allows you to catch things you may have missed
• Can setup breakpoints to interrupt replay
• Can restore to breakpoints
11

12. NTXISSA.org
Summary
• Start with clean snapshot with no malware
on it
• Transfer malware to VM
• Conduct analysis on the VM
• Take notes, screenshots, and data from the
VM and transfer it to host
• Revert VM to clean snapshot
12

13. NTXISSA.org
Why Perform Dynamic Analysis?
• Static analysis can reach a dead end due to
• Obfuscation
• Packing
• Analyst has exhausted available static techniques
• Dynamic analysis
• Efficient
• Shows exactly what malware does
• Examples
• Run keylogger to find where info is stored/sent
• Functions in binary may not be called/used
13

14. NTXISSA.org
Running Malware in Windows
• Executables may be run
• By double-clicking executable (.EXE) file
• Via command line
(‘C:> .af50a41e812cb0b0d367ded9f04a55e5’)
• DLLs cannot be directly executed
• Use rundll32.exe
• Via command line
(‘C:> rundll32.exe DLLname, ExportArgs’)
• The ExportArgs are functions found by using
Dependency Walker, PEview or PE Explorer
14

15. NTXISSA.org
Launching DLLs
• Use Dependency Walker, PEview or PE Explorer
to view the Export table
• rip.dll contains 2 exports
• Install
• Uninstall
• Execute by function name
‘C:> rundll32.exe rip.dll, Install’
• Execute by ordinal
‘C:> rundll32.exe rip.dll, #1’
• DLLMain often holds most malicious code
• It is executed whenever the DLL is loaded
• It is possible to modify the PE header and convert a DLL into an EXE
• Delete IMAGE_FILE_DLL (0x2000) flag from Characteristics field in
IMAGE_FILE_HEADER
• Execute a service
‘C:> rundll32 ipr32x.dll,InstallService ServiceName’
‘C:> net start ServiceName’
15

16. NTXISSA.org
Process Monitor (Procmon)
• Procmon monitors all activity in
• Registry
• File system
• Network
• Processes
• Threads
• All recorded events (system calls) are kept
• Filters make displaying items of interest easier
• Doesn’t capture everything though
• Device driver activity
• GUI calls like SetWindowsHookEx
• Don’t run it for too long
• Fills up RAM
• Crashes the system
• Running Procmon
• It starts collecting full system data when started
• Stop collection using
• File  Capture Events
• Apply filter
• Remove irrelevant data before restarting
• Edit  Clear Display
• Run collection again
• File  Capture Events
16

17. NTXISSA.org
Process Explorer
• Microsoft tool that lets you list
• Active processes
• DLLs loaded by a process
• Process properties
• Overall system information
• Other features and actions
• Kill processes
• Log out user
• Launch / validate processes
• Color coding
• Services are pink
• Processes are blue
• New processes are green briefly (temporary) ***
• Terminated processes are red (temporary) ***
• Refreshes every second
17

18. NTXISSA.org
Process Explorer – DLL Mode
• Open DLL information display window
• Click a process to see all DLLs it loaded into memory
• Handles window shows all file handles, mutexes,
and events of process
18

19. NTXISSA.org
Process Explorer – Properties
• Opens when you double-click a
process name
• Shows Data Execution Prevention
(DEP) and Address Space Layout
Randomization (ASLR) status
• Verify button checks the disk file’s
Windows signature
• Not the RAM image
• Process replacement not detected
this way
• Process in memory overwritten by
malware
19

20. NTXISSA.org
Process Explorer – Other Features
• Can be used to detect process replacement
• Compare Image and Memory strings
• Launch Dependency Walker
• Right-click process name
• Select Launch Depends
• Find DLL or Handle by name
• Find  Find Handle or DLL
• Useful to see if any running processes are using a malicious DLL that you found
• Verify button verifies EXE file on disk, but not DLLs loaded during runtime
• Find a malicious DLL by comparing DLL list in Process Explorer to the imports
shown in Dependency Walker
• Detect malicious documents (see if they spawn a new process)
• In practice, use older/unpatched versions of document viewers
20

21. NTXISSA.org
Regshot
• Use Regshot
• Take 1st shot
• Run malware
• Take 2nd shot
• Click cOmpare to
see which registry
keys were changed
21

22. NTXISSA.org
Faking the Network
• Create fake network to quickly obtain network
indicators
• DNS names
• IP addresses
• Packet signatures
• Faking network makes malware think it’s on a
physical machine
• ApateDNS
• See which DNS requests are made by malware
• Spoofs DNS responses
• Usage
• Set IP address to use in responses
• 127.0.0.1 (localhost)
• External IP for fake web server
• Select interface
• Click Start Server
• NXDOMAIN option can catch additional domains
22

23. NTXISSA.org
Faking the Network – INetSim
• Free, Linux-based suite for simulating common Internet services
• Allows you to analyze network behavior of malware
• To use with Windows
• Create a Linux VM and install INetSim on it
• Set up Linux VM on same virtual network as malware analysis VM
• Services emulated by default
• HTTP, HTTPS, DNS, SMTP, SMTPS, POP3, POP3S, IRC, NTP, TIME, DAYTIME, FINGER,
SYSLOG, TFTP, FTP, FTPS, IDENT, ECHO, DISCARD, QUOTD, CHARGEN, DUMMY
• Serves almost any file requested
• Doesn’t return 404 error message
• Can fool web browsers and NMAP
• Records all inbound requests / connections
• Highly customizable and configurable
• Can provide restricted Internet access if needs be
• Features Dummy service
• Logs all data received from client regardless of port
23

24. NTXISSA.org
Wireshark Sniffing
• Open-source packet capture tool
• Intercepts / logs network traffic
• Provides
• Visualization
• Packet stream analysis
• In-depth analysis of individual packets
• Capture packets
• Select Capture  Interfaces
• Select interface to use and options
• Good usage
• Analyze internal networks
• Analyze network usage
• Debug application issues
• Study protocols in action
• Bad usage
• Sniff passwords
• Reverse engineer network protocols
• Steal sensitive information
• Listen to online chatter on public networks
24

25. NTXISSA.org
Put It All Together
• Run your tools
• Run procmon
• Set your filter on the malware executable name
• Clear out all events
• Start Process Explorer
• Get a 1st shot of the registry with Regshot
• Setup your virtual network with INetSim and
ApateDNS
• Start Wireshark and begin capturing packets
• Run your malware
• Collect your data
• Stop procmon event collection
• Get a 2nd shot of the registry with Regshot
• Stop packet capture in Wireshark
25

26. NTXISSA.org
Analyzing the Data
• Examine ApateDNS to see if DNS requests were
performed
• Review procmon results for file system modifications
• Compare two snapshots taken with Regshot to identify
changes
• Use Process Explorer to determine if process creates
mutexes or listens for incoming connections
• Review INetSim logs for requests and attempted
connections
• Review Wireshark capture for network traffic
26

27. NTXISSA.org
27
Thank you