The document presents an introduction to binary exploitation by Gábor Pék, who discusses his background in virtualization and malware security. It covers fundamental concepts including compilers, native binaries, program memory layout, and types of bugs, along with examples of function calls and memory management. The document also addresses memory corruption errors, common vulnerabilities, and various protections and exploitation techniques.

1. An introduction to binary
exploitation
Gábor Pék

2. About me
● Co-founder and CTO at Avatao
● PhD in virtualization and malware security (CrySyS Lab)
● Virtualization hacks (e.g., XSA-59)
● Research of advanced malware (e.g., Duqu, Flame)
● Co-founder of !SpamAndHex (3x DEFCON CTF Finalist team)

3. So why this topic?

4. https://www.theverge.com

5. Let’s begin
LSzekeres,MPayer,WeiTao,DSong:SoK–EternalWarinMemory,IEEE
SecurityandPrivacy,2013

6. Maybe at another time :)
LSzekeres,MPayer,WeiTao,DSong:SoK–EternalWarinMemory,IEEE
SecurityandPrivacy,2013
X

7. First, Understand the basics
● Compilers (native, JIT, AOT) vs. interpreters
● Native binaries
○ compiled from C, C++ or Rust for example
● Loaders
● Program memory layout
● CPU instructions and registers including Assembly
● Type of bugs

8. interpreters
“An interpreter executes some
representation of a program’s source
file at run- time.”
Randall Hyde

9. Compilers
“A compiler translates a source
program in text form into executable
machine code.”
Randall Hyde

10. COMPILATION PROCESS (SIMPLIFIED)
Source
code
Analysis
Optimization
Code generation
Machine
code

11. Example Source code
Machine (object) code

12. Basic x86 system
Source: [1]

13. Cpu registers (32-bit)
Source: [1]

14. Interested in architecture internals? :D
Intel® 64 and IA-32 Architectures
Software Developer Manuals
(4846 pages)

15. Assembly basics
Assembler language is the
representation of machine code in
human readable form

16. assembly instructions
AT&T syntax: <instruction (mnemonic)>
<source>, <destination>
Intel syntax: <instruction (mnemonic)>
<destination>, <source>

17. Machine code vs Assembly

18. So how to store data in memory?
Stack: Built up from stack frames of
functions, LIFO, grows downwards
Heap: Dynamically allocated
memory, grows upwards

19. ELF binary in memory … too much for today

20. Function call example
int main(void){
int a, b, c;
a=7;
b=3;
c = addnum(a,b);
printf("%d", c);
}
int addnum(int a, int b){
int c = 4;
c = a + b;
return c;
}

21. Stack frame example (x86)
Free memory
← ESP
Local variables
Saved EBP
Previous frame
Return address
Function parameters
← EBP
Higher memory
addresses

22. Stack frame before function call (x86)
Free memory
← ESP
Lower memory
addresses
stack pointer, points to
the top of the stack frame

23. second function parameter placed first
Free memory
← ESP
Lower memory
addresses
3 (b) int main(void){
int a, b, c;
a=7;
b=3;
c = addnum(a,b);
printf("%d", c);
}

24. First function parameter placed after
Free memory
← ESP
Lower memory
addresses
3 (b) int main(void){
int a, b, c;
a=7;
b=3;
c = addnum(a,b);
printf("%d", c);
}
7 (a)

25. Place return address (Address of next instruction)
Free memory
← ESP
Lower memory
addresses
3 (b) int main(void){
int a, b, c;
a=7;
b=3;
c = addnum(a,b);
printf("%d", c);
}
7 (a)
Return address
& __asm__ add esp, 8

26. Place the Base pointer of the previous stack frame
Free memory
← ESP
Lower memory
addresses
3 (b) int addnum(int a, int b){
int c = 4;
c = a + b;
return c;
}
7 (a)
Return address
Saved EBP
__asm__ push ebp
__asm__ mov ebp, esp
__asm__ sub esp, 0x10

27. Align the base pointer to the current stack frame
Free memory
← ESP,
Lower memory
addresses
3 (b) int addnum(int a, int b){
int c = 4;
c = a + b;
return c;
}
7 (a)
Return address
Saved EBP
__asm__ push ebp
__asm__ mov ebp, esp
__asm__ sub esp, 0x10
EBP

28. Create space for local variables
Free memory
← EBP
Lower memory
addresses
3 (b) int addnum(int a, int b){
int c = 4;
c = a + b;
return c;
}
7 (a)
Return address
Saved EBP
__asm__ push ebp
__asm__ mov ebp, esp
__asm__ sub esp, 0x10
← ESP
Space for local variables

29. Let’s return: delete local variables from stack frame
Free memory
← EBP
Lower memory
addresses
3 (b) int addnum(int a, int b){
int c = 4;
c = a + b;
return c;
}
7 (a)
Return address
Saved EBP
...
ESP
__asm__ mov esp, ebp
__asm__ pop ebp
__asm__ ret

30. restore ebp to its previous value
Free memory
←ESP
Lower memory
addresses
3 (b) int addnum(int a, int b){
int c = 4;
c = a + b;
return c;
}
7 (a)
Return address
...
__asm__ mov esp, ebp
__asm__ pop ebp
__asm__ ret

31. Return to the address we saved before
Free memory
←ESP
Lower memory
addresses
3 (b) int addnum(int a, int b){
int c = 4;
c = a + b;
return c;
}
7 (a) ...
__asm__ mov esp, ebp
__asm__ pop ebp
__asm__ ret

32. Caller cleans the stack from function parameters
Free memory
←ESP
Lower memory
addresses
int main(void){
int a, b, c;
a=7;
b=3;
c = addnum(a,b);
printf("%d", c);
}
__asm__ add esp, 8

33. What about 64-bit mode (e.g.,System V AMD64) ?
Function parameters are put into CPU
registers like
RDI,RSI,RDX,RCX,R8,R9,XMM0-7

34. Debuggers (E.g., GDB, WinDBG, IDA)
Understand the runtime behaviour of
a program

35. Memory corruption errors
Attackers can read or write
arbitrary memory locations of a
vulnerable program

36. Vulnerability Examples
Stack overflow, heap overflow,
integer overflow format string,
use-after-free, ...

37. Protections
Stack canaries, DEP, ASLR, Control
and Data Flow Integrity, using safe
libraries, ...

38. Some exploitation techniques
Return to LibC, ROP, JOP, GOT
hijacking, Off-by-one, ...

39. Maybe next time
LSzekeres,MPayer,WeiTao,DSong:SoK–EternalWarinMemory,IEEE
SecurityandPrivacy,2013

40. My two cents
Write less and more secure code :)

41. Readings
[1] Randall Hyde: Write Great Code, Volume 2 - Thinking
low-level, writing high-level, No Starch Press, 2006
[2] L Szekeres, M Payer , Wei Tao, D Song : SoK – Eternal
War in Memory, IEEE Security and Privacy, 2013
[3] Intel® 64 and IA-32 Architectures Software Developer
Manuals, 2016

42. Huge thanks for coming!
Questions? gabor.pek@avatao.com