Anti-debugging techniques can be used to detect the presence of a debugger. There are three main categories of anti-debugging techniques: API based detection, exception based detection, and direct structure access based detection. API based techniques use functions like IsDebuggerPresent() and CheckRemoteDebuggerPresent() to check debugger flags. Exception based techniques check for differences in exception handling when exceptions like INT3 are triggered. Direct structure access techniques directly check debugger flags in the Process Environment Block structure and heap for differences when being debugged. The document provides examples of specific techniques in each category to detect the presence of a debugger.

1. ANTI|DEBUGGING
By
Adwiteeya Agrawal

2. REVERSE ENGINEERING
• Definition
“Software reverse engineering is about opening up a
program’s “box,” and looking inside. “

3. PROGRAMMERS vs REVERSERS
Logic
Code
Executable
Logic
Code

4. Example : Code Written
Ideally , the else block
should never be executed
since the value of var1 is
not changed.

5. After Disassembling
We can easily modify the binary to execute the else loop.

6. Problem ?
• A Google search for “Crack Torrent” returns
200 million results and “Software Crack”
around 45M.
• Reversing is extensively used to develop cracks
for proprietary software.

7. Possible Solutions
• Anti-Debugging (Detecting how a process is different from
when it is being debbugged and when its run.)
• Code Obfuscation(Encryption, pseudo execution flow, logic)
• However : The end result after exploring the various options
available is that we cannot totally prevent software reverse
engineering however we can slow down the process for a
dedicated reverse engineer.

8. Anti - Debugging
• Anti Debugging is done acknowledging the fact that when a
process is being debugged it is going to have a set of
properties that would be different from when it is not
debugged.
• Anti-debugging requires a thorough understanding of the
environment in which the program would be run.

9. Does Anti - Debugging help ?
Example : CCleaner
Software Vendors
Software Crackers
Malware Analysts

10. Types
API Based
Detection
Techniques
Direct
structure
access
Exception
Handling
Based
Detection

11. API Based Anti|Debugging
(11 Techniques)

12. 1.FindWindow API
• Scans memory for a process with the particular class name.
{ hnd = FindWindow("OLLYDBG", 0); }
• “OLLYDBG” is the class name for all windows that would be created and
have the same callback function.
• “0” scans irrespective of WindowName.
• Returns a handle if successful
• Spy++ utility can be used to enumerate the ClassName.
• DEMO - spy++

13. Spy++ on OllyDbg

14. 2. Registry Value
• RegOpenKeyEx and RegQueryValueEx open a
registry key and retrieve its value respectively
KEY Function
HKEY_CLASSES_ROOTexefileshellOpen with OllyDbg
Specifies the menu for
opening an exe file with
OllyDbg with a right click
HKEY_CLASSES_ROOTexefileshellOpen with
OllyDbgcommand
Path to OllyDbg
HKEY_CLASSES_ROOTdllfileshellOpen with OllyDbg
Specifies the menu for
opening a dll file with OllyDbg
with a right click
HKEY_CLASSES_ROOTdllfileshellOpen with
OllyDbgcommand
Path to OllyDbg
HKEY_LOCAL_MACHINESOFTWAREMicrosoftWindo
ws NTCurrentVersionAeDebugDebugger
Path to default Debugger

15. 3. IsDebuggerPresent API
• Looks for the BeingDebugged flag inside the PEB.
• Extremely simple to use and thwart.
• { IsDebuggerPresent() } returns true if the process is being
debugged false otherwise.
• Can be done manually
• Assembly dump :

16. 4. CheckRemoteDebuggerPresent API
• Function : Detects if a Debug Port has been set.
• Can be used for a “remote” process but is used locally mostly.
• Locally :
CheckRemoteDebuggerPresent(GetCurrentProcess(),&pblsPresent)
;
• Remotely for a PID 2800
{ hndle=OpenProcess(PROCESS_ALL_ACCESS, FALSE, 2800);
CheckRemoteDebuggerPresent(hndle,&pblsPresent); }

17. 5.OutputDebugString API
Set Random Value for ERROR
Call OutputDebugString with any string
Check Value Of ERROR
If(No Change) : Debugger Present
Else : No Debugger
Works in XP.

18. Debug String Display in OllyDbg

19. Run Time Dynamic Linking
LoadLibrary
Handle to dll
received
GetProcAddress
Address of
exported
function
Call
Example : ntdll.dll
Ex : NTSetInformationThread,
NTQueryInformationProcess

20. 6. ZwSetInformationThread Function
• Use run-time dynamic linking to get the address of
ZwSetInformationThread.
• Make the Call
• (_ZwSetInformationThread)(GetCurrentThread(),0×11,0,0);
• GetCurrentThread : Pseudo handle for the current thread
• 0x11 : ThreadHideFromDebugger , 17 in THREADINFOCLASS enum.
• 0 : Pointer of value that is to be set
• 0 : Size of the value.
• Different Approached followed.

21. 7. DebugActiveProcess based Self
Debugging
• Theoretically a process can be debugged only
by One Debugger.
• Working :
Create a child
process
Child Process
attempts to
debug parent
If Error :
Debugger
Detected

22. 8. OllyDbg Format String Vulnerability
for Debugger Messages
• More like a vulnerability based detection
• The internal function that handles the input from
OutputDebugString does so without using the correct number of
format specifiers, allowing a user to supply their own format
specifiers.
• OutputDebugString(TEXT("%s%s%s%s%s%s%s%s%s%s%s%s%s%s%
s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s%s"));
• Similar vulnerability based detection can be modeled around
various vulnerabilities, Reference : http://waleedassar.blogspot.in

23. 9. NTQueryInformationProcess to
detect ProcessDebugPort
• Run-time dynamic linking to get the pointer to NTqueryInformationProcess
• Make the call with PROCESSINFOCLASS = 0x07
• NTSTATUS WINAPI NtQueryInformationProcess(
_In_ HANDLE ProcessHandle,
_In_ PROCESSINFOCLASS ProcessInformationClass,
_Out_ PVOID ProcessInformation,
_In_ ULONG ProcessInformationLength,
_Out_opt_ PULONG ReturnLength
);
• The 0x07 value is the process debug port.
• Debug Port is used for communication of Debug_Event between ring 3 and the
kernel
• If set the process is being debugged.

24. • private enum PROCESSINFOCLASS: int
{
ProcessBasicInformation
ProcessQuotaLimits,
ProcessIoCounters
ProcessVmCounters,
ProcessTimes,
ProcessBasePriority,
ProcessRaisePriority,
ProcessDebugPort,
ProcessExceptionPort
ProcessAccessToken,
ProcessLdtInformation,
ProcessLdtSize,
.
.
.
ProcessDefaultHardErrorMode,
ProcessIoPortHandlers,
ProcessPooledUsageAndLimits,
ProcessWorkingSetWatch,
ProcessDynamicFunctionTableInformation,
ProcessHandleCheckingMode,
ProcessKeepAliveCount,
ProcessRevokeFileHandles,
MaxProcessInfoClass
};

25. 10. ProcessDebugFlags using NTQueryInformationProcess
11. ProcessDebugObject using NTQueryInformationProcess
•Run-time Dynamic Linking to get address of NTQuertInformationProcess
Function.
•Make the call with PROCESSINFOCLASS 0x1F.
•If Set Debugger is present.
•Run-time dynamic linking to get the address
•Call with PROCESSINFOCLASS 0x1E
•If set debugger is present

26. • private enum PROCESSINFOCLASS: int
{
ProcessBasicInformation
ProcessQuotaLimits,
ProcessIoCounters
ProcessVmCounters,
ProcessTimes,
ProcessBasePriority,
ProcessRaisePriority,
.
.
.
ProcessAffinityMask,
ProcessPriorityBoost,
ProcessDeviceMap,
ProcessSessionInformation,
ProcessForegroundInformation,
ProcessWow64Information,
ProcessImageFileName,
ProcessLUIDDeviceMapsEnabled,
ProcessBreakOnTermination,
ProcessDebugObjectHandle,
ProcessDebugFlags,
ProcessHandleTracing,
ProcessIoPriority,
ProcessExecuteFlags,
ProcessResourceManagement,
.
.
.
ProcessRevokeFileHandles,
MaxProcessInfoClass
};

27. Exception Based Anti|Debugging
(9 Techniques)

28. Exceptions | Windows
First Chance
VEH
SEH
Last Chance
Unhandled
1st
• EXCEPTION_DEBUG_EVENT
VEH
• VEH
SEH
• SEH, per thread, FS:0
LC
• Process Suspend
UE
• UnhandledExceptionFilter
• System Final Handler

29. Exception Handlers
typedef struct _EXCEPTION_RECORD
{
DWORD ExceptionCode;
DWORD ExceptionFlags;
struct _EXCEPTION_RECORD* ExceptionRecord;
PVOID ExceptionAddress;
DWORD NumberParameters;
ULONG_PTR ExceptionInformation[EXCEPTION_MAXIMUM_PARAMETERS];
} EXCEPTION_RECORD, *PEXCEPTION_RECORD;
typedef struct _EXCEPTION_POINTERS
{
PEXCEPTION_RECORD ExceptionRecord;
PCONTEXT ContextRecord;
} EXCEPTION_POINTERS, *PEXCEPTION_POINTERS;
LONG CALLBACK ExceptionHandler(
__in PEXCEPTION_POINTERS ExceptionInformation);

30. 1. INT 3 Break Point Exception.
• The INT 3 instruction generates a special one byte opcode (CC) that is intended for
calling the debug exception handler.
• To set INT3 breakpoint, a debugger replaces first byte of the 80x86 command by
0xCC
• Type : EXCEPTION_BREAKPOINT to the Debugger
• When manually inlined then the process would just halt at the next instruction if
run in a Debugger(thereby not triggering your catch block) or return 0x80000003
status code if run without a debugger.

31. 2. INT 2D Exception
• Int 2Dh is used by ntoskrnl.exe to interact with kernel debugging system
but we can use it also in user-mode or from ring 3 as well since the call will
eventually filter to a ring 3 debugger if no kernel debugger exists.
• When int 2Dh is called the system will skip one byte after the interrupt,
leading to opcode scission.
• Behaves exactly like int 3
• Based on weather our catch block is executed or not we declare presence
of a debugger.

32. 3. 0xF1 ICE Break Point
• This is identical to the functioning of 0xcc except the fact the this uses two
bytes 0xCD 0x3C to insert the breakpoint.
4. TWO byte INT 3 breakpoint.
•One of the Intel's undocumented instruction, opcode 0xF1.
•Executing this instruction will generate a SINGLE_STEP exception.
•The debugger will think it is the normal exception generated by executing
the instruction with the SingleStep bit set in the Flags registers.

33. 5. Close Handle API call
• The CloseHandle function throws an exception, if an invalid handle is
provided.
• The logic behind detection with Close Handle function is the opposite to
the ones used in INT3 or INT 2D exception based debugger detection.
• Exception is only thrown when in a debugger
• Usually in debugging sessions it is a common practice that the exception is
passed to the application

34. 6. Trap Flag exception based detection
• The EFLAGS is a 32-bit register used as a collection of bits representing
Boolean values to store the results of operations and the state of the
processor.
• By in-lining ASM we can modify the EFLAGS register.
• First bit of the second byte that is TF or the trap flag. If this flag is set the
execution halts after executing the current instruction.
• Also called as the single step exception

35. EGLAGS Register

36. 7. Memory breakpoint based
detection
• If a memory breakpoint is set up, any access to the page in which the breakpoint
exists, would result in the exception and the process would halt.
Allocate some 5mb
Fill with RET
Make it a GUARD PAGE Pointer to the PAGE
Call the Pointer
Detect Error

37. Technically
VirtualAlloc(NULL, 0x500000, MEM_COMMIT,
PAGE_READWRITE);
RtlFillMemory(memRegion, 0x10, 0xC3);
VirtualProtect(memRegion, 0x10,
PAGE_EXECUTE_READ | PAGE_GUARD,
&oldProt);
myproc = (FARPROC)
memRegion
myproc();
Detect Error
ProcessExplorer:Demo

38. Hack that ?

39. 8. Control C VEH
• If a console process is being debugged for a CTRL+C signals,
the system generates a DBG_CONTROL_C exception.
• If a debugger is not present CTRL+C event would require a
console control handler.
• The CCH is executed if the debugger is not present.
• However if the debugger is present and it passes the
exception to the application we can still detect it by adding
a vectored exception handler. ;)

40. 9. INVALID OPCODE exception based
debugging
• Invalid OPCODE can be formed by manually editing bytes.
• F0 0F C7 C8 popularly known as the "Pentium F00F bug.“
• This evaluates to LOCK CMPXCHNG8B EAX
• OPCODE since a LOCK on CMPXCHNG8B cannot be applied with the
destination operand as a register
• Now, before this OPCODE is executed an exception is created
ILLEGAL_INSTRUCTION (C000001D)
• we define an UnhandledExceptionFilter ,only executed when the program
is not being debugged. So even if the debugger passes the exception to
the application there isnt any handler. ;)

41. Direct Structure Access
Based Anti|Debugging
(4 Techniques)

42. Direct isDebuggerPressent via PEB
• In this method we manually do what the isdebuggerpresent function call
does.
• Run-Time Dynamic Linking to call NTQueryInformationProcess with
PROCESSINFOCLASS set to ProcessBasicInformation = 0.
• typedef struct _PROCESS_BASIC_INFORMATION {
PVOID Reserved1;
PPEB PebBaseAddress;
PVOID Reserved2[2];
ULONG_PTR UniqueProcessId;
PVOID Reserved3;
} PROCESS_BASIC_INFORMATION;
• Access ProcessExecutionBlock via PPEB (pointer) to check
BeingDebbugedFlag.

43. PEB |Structure
typedef struct _PEB {
BYTE Reserved1[2];
BYTE BeingDebugged;
BYTE Reserved2[1];
PVOID Reserved3[2];
PPEB_LDR_DATA Ldr;
PRTL_USER_PROCESS_PARAMETERS ProcessParameters;
BYTE Reserved4[104];
PVOID Reserved5[52];
PVOID Reserved8[312];
BYTE Reserved6[128];
PVOID Reserved7[1];
ULONG SessionId;
} PEB, *PPEB;
pPIB.PebBaseAddress->BeingDebugged

44. ProcessHeapFlag Debugger Detection
Heap Header User Allocation
Heap Header User Allocation Tail Checking Pattern Heap Extra
Multiple of 16 bytes16 bytes
16 bytes Multiple of 16 bytes 16 bytes 16–31 bytes
DEBUGGERWithout
With
{ HEAP }
{ /HEAP }
•In order to tell this to the OS before creating the HEAP two flags are set. “Flags” and
“ForceFlags”
•Present at the offset 0x14 and 0x18 for windows XP.
•PEBbase address + Offset to HEAPbaseAddress + Flag offset.

45. typedef struct _HEAP
{
HEAP_ENTRY Entry;
ULONG SegmentSignature;
ULONG SegmentFlags;
LIST_ENTRY SegmentListEntry;
PHEAP Heap;
.
.
.
ULONG NumberOfUnCommittedPages;
ULONG NumberOfUnCommittedRanges;
WORD SegmentAllocatorBackTraceIndex;
WORD Reserved;
LIST_ENTRY UCRSegmentList;
ULONG Flags;
ULONG ForceFlags;
ULONG CompatibilityFlags;
ULONG EncodeFlagMask;
HEAP_ENTRY Encoding;
.
.
HEAP_COUNTERS Counters;
HEAP_TUNING_PARAMETERS TuningParameters;
} HEAP, *PHEAP;

46. NTGlobalFlag Debugger Detection
• NTGlobalFlag is a DWORD value present at the offset 0x68 from the PEB
base address.
• When inside a debugger the following flags are set by the operating
system : FLG_HEAP_ENABLE_TAIL_CHECK (0x10)
FLG_HEAP_ENABLE_FREE_CHECK(0x20)
FLG_HEAP_VALIDATE_PARAMETERS(0x40)
• This equals to 0x70. We detect the value by the similar method and judge
if a debugger is present or not.

47. and many more… 
• Quick Discussion on minor.
• And MOST importantly if I have reached this
slide :D
• THANK YOU NULL !