Cgc2

從DARPA CGC及DEFCON CTF探討自動攻防技術
C.K. Chen
Twitter: Bletchley13
ckchen@cs.nctu.edu.tw
1

 陳仲寬(Bletchley)
 交通大學網路安全實驗室博士生
 Sandbox: https://github.com/GlacierW/MBA
 Malware, Vulnerability, Virtual Machine, Machine Learning
 BambooFox領隊/交大網樂安全策進會顧問
 CTF, CTF and more CTFs
 Rank 31 in ctftime 2016
 NCTU PT Team -> discover about 40 bugs in NCTU
 Synology bug bounty -> 7 bugs and about 40w NTD
 HackerCollege Member
 http://hackercollege.nctu.edu.tw/
 HITCON.KB Editor
 ….
2

 Introduction to CGC
 CQE
 CFE
 Automatic Vulnerability Discovery
 Fuzz
 Symbolic/Concolic Execution
 Symbolic-assist Fuzzing
 Software Hardeness
 General Protection
 Software Patch
 Conclusion
3

 A weakness of an asset or group of assets that can be exploited
by one or more threats -- ISO 27005
 Programmer may make some mistakes, the unpredictable
behavior may result in software vulnerability
 Buffer overflow
 Use-After-Free
 Type Confusion
 What attacker can do?
 Information leak – Heartbleed
 Arbitrary Code Execution - PHPMailer
5

 CVE is a dictionary of publicly known information security
vulnerabilities and exposures
 More than 10000 CVEs in 2016
6

 Malware, which may use vulnerability to attack/exploit, is
mostly active only 2~3 hours
 The vulnerability needs about 24 day to patch
 Between the time windows, it’s the 0 day.
7

void do_something(char
*Buffer)
{
char MyVar[128];
strcpy(MyVar,Buffer);
}
int main (int argc, char **argv)
{
do_something(argv[1]);
}
8

 The a prize competition organized by DARPA to make innovation
for next generation technique
 2007 Urban Challenge
 2012 Robotics Challenge
 Cyber Grand Challenge
 The first full machine attack-defense CTF
 Focus on develop automatic attack-defense system
10

 7 teams are qualified to compete the final champion
11

 Start from 2014
 Qualification round in June 3, 2015
12

 Modified Linux
 Customize ELF format
 only 7 syscalls
 terminate (exit)
 transmit (write)
 receive (read)
 fdwait (select)
 allocate (mmap)
 deallocate (munmap)
 Random
 no signal handling, no not-executable stack, no ASLR, …
13

 CGC format
 Minor modification to ELF
 32-bit, x86
 Inetd-style services
 Each connection create a new process
 IPC, communicate via controlled fd socket-pairs
 Userspace
 Statically linked
 Compiled Binaries only (not hand coded)
14

Text vs Code of trivial program

 Type 1
 Hi-jack control flow
 Control EIP and one register
 Type 2
 Information leak
 Leak information in the magic page
16

 Cyber Reasoning System
 Given Challenge Binary
CRS
CB
CB
CB
RCB
IDS
POV
17

 113 CBs are provided to participator’s CRS
 CRS – Cyber Reasoning System
 Produce crash
 Generate and apply patch
 2 Scored Events for testing the CRS
 基本理念是希望展現各隊的特長。若某個CB全部隊伍都解出/都沒
解出，這些題目分數就會較低。而只有部分隊伍解出的題目，將會
得到比較高的分數。這樣的計分方式可以鼓勵各隊提出不同獨特的
程式分析技術。
18

 VM Provided by DARPA
 Code and Data
 https://github.com/cybergrandchallenge
 https://repo.cybergrandchallenge.com/
 VirtualBox VM and Vagrant
19

 A small program with overflow
 Crash when we give it long input
20

 Based on regular expression
 Most teams in final doesn’t like IDS
22

 Testing if your POV crash the CB
23

 Poller : Check if your program remains it’s functionalities
 Cb-test can test if your
patch binary pass the poller
24

 SECUINSIDE CTF 2016
 Simple ECHO Server
 Ask user name
 Echo every thing from user
 Make it crash to get the flag
25

 Use radare2 to reverse
 Steps
 Become admin
 Give admin’s key
 Reach crash point
 How to find admin’s key
 Use symbolic execution
 We will talk about it later 
26
mov dword [ebp - local_454h], 0
mov eax, dword [ebp - local_454h]
mov byte [eax], 0xa

Team Research CTF Enterprise
CodeJitsu Berkeley BlueLotus Cyberhaven
CSDS University of
Idaho
Deep Red Some CTF Player Raytheon
disekt Different
university
disekt
ForAllSecure CMU CyLab PPP ForAllSecure
Shellphish UCSB Shellphish LastLine
TECHx University of
Virginia
Some White Hack
Students
GrammaTech
35

 CGC Final Event was held at DEF CON 2016
 Final Winner – ForAllSecure/Mayhem
 Startup company ForAllSecure
 Most member come from PPP CTF Team
 Researcher from CMU CyLab
 Next day to the CGC, Mayhem competed with top human
hackers in DEFCON CTF
 Mayhem get the last rank, but PPP win the game
37

 Cyber Reasoning System
 Given Challenge Binary
 Given OpRCB, IDS
CRS
CB
CB
CB
CB
CB
RCB
CB
CB
IDS
CB
CB
PCAP
RCB
IDS
CFE
POV
38

 Simulator for CGC Competition
 For CRS interface testing
39

 Client API
 Upload RCB, POV, IDS
 Download other team’s RCB, IDS
40

Availability
Security
Evaluation
除了會用不同方法檢查是
否符合功能完整性，還會
看time, memory usage,
and space efficiency，大
概共有2x-30個檢查，所
以分數是符合項/總檢查
數
0~1
1 or
2
1~2
X
X
被拿到flag or not
被拿到flag or not
Scoring 評分系統有三大指標
X
100

 DARPA provide the rich interface for demonstrate the
competition
 Cool visualization
43

• 2003年1月25日，SQL Slammer 蠕蟲短短10分鐘內，
感染七萬多台主機。
• 2010 MS LNK漏洞MS10-045，微軟Patch一次還不夠，
2015年又被挖出沒修好。

 HeartBleed
 Demonstrate how these technique can solve real world problem

 Determine if the program has a vulnerability is undicidable
 Assume we have a Machine M that can detect any vulnerability in the
program
 Halting Problem If M(P) has no bug:
do_some_bug()
Else:
do_nothing()
56

 If we have a execution trace, we can check if the bug appeared
in this path
 To testing software complete, we need to traversal all the code
inside the program
 Halting problem
 But we can still do something 
57

 Automatic generate the input to make the program crash
 Not inspect into program semantic
 Generate input randomly, or some heuristic
 Coverage-based
 AFL, Peach, BFF
58

 American Fuzzy Loop
 The easy-to-use fuzzer
 Efficiency
 low-level compile-time
instrumentation
 Coverage-based Fuzzer
 Effective Mutation Strategy
At least 4 team in CGC use AFL
59

 How AFL do?
1. Load user-supplied initial test cases into the queue
2. Take next input file from the queue
3. Attempt to trim the test case to the smallest size that doesn't
alter the measured behavior of the program,
4. Repeatedly mutate the file using a balanced and well-
researched variety of traditional fuzzing strategies
5. If any of the generated mutations resulted in a new state
transition recorded by the instrumentation, add mutated output
as a new entry in the queue.
6. Go to 2.
 Binary fuzzer -> QEMU(emulator) support
60

 Feed every thing in network PCAP into APF
 Using AFL as first layer checker
 Check if the input is worth for deep analysis
 This instance would run through each of the incoming PCAP files and
evaluate whether they brought anything "new" to the table.
Building an Autonomous Cyber Battle System: Our Experience in DARPA's Cyber
Grand Challenge
61

 Most test falls into some high frequency path
 Strategy to find the low frequency path
 Energy
 The number of inputs to be generated from that seed
 Strategy
 low energy to seeds exercising high-frequency paths
 high energy to seeds exercising low-frequency paths
 AFLFast Paper published in
 ACM Conference on Computer
and Communications Security
 Github
 https://github.com/mboehme/
aflfast
62

 Murphy is the directed fuzzer based on AFL
 Most improvement is on the efficiency
 Binary-only instrument
 Maybe base on BAP
63

 A mechanism to discover the code coverage
 Translate each instruction/code line into constraints
 Constraints: a formula define the operation functionality
 Collect all the constraints
 Solve when required condition is meet
 E.g. branch happened
64

1st 2nd 3rd
0 ebx = 0
1 ecx = 0
2 eax = 3
3 ebx = 3 ebx = 6 ebx = 9
4 ecx = 1 ecx = 2 ecx = 3
5 NE NE E
6 N N Y
7 Y Y
8 NE
9 N
0 mov ebx, 0
1 mov ecx, 0
2 mov eax, input
3 add ebx, eax
4 add ecx, 1
5 cmp ecx, 3
6 je final
7 jmp loop
8 cmp ebx, 15
9 je f1
f1 f2 65

1st 2nd 3rd
0 ebx_0 = 0
1 ecx_0 = 0
2 eax_0 = sym_0
3 ebx_1 =
ebx_0+eax_0
ebx_2 =
ebx_1+eax_0
ebx_3 =
ebx_2+eax_0
4 ecx_1 =
ecx_0+1
ecx_2 =
ecx_1+1
ecx_3 =
ecx_2+1
5 NE NE E
6 N N Y
7 Y Y
8 NE
9 N
66
0 mov ebx, 0
1 mov ecx, 0
2 mov eax, input
3 add ebx, eax
4 add ecx, 1
5 cmp ecx, 3
6 je final
7 jmp loop
8 cmp ebx, 15
9 je f1
f1 f2

 Can we jump to final block when loop 3
times?
1st 2nd 3rd
0 ebx_0 = 0
1 ecx_0 = 0
2 eax_0 = sym_0
3 ebx_1 =
ebx_0+eax_0
ebx_2 =
ebx_1+eax_0
ebx_3 =
ebx_2+eax_0
4 ecx_1 =
ecx_0+1
ecx_2 =
ecx_1+1
ecx_3 =
ecx_2+1
5 NE NE E
67
0 mov ebx, 0
1 mov ecx, 0
2 mov eax, input
3 add ebx, eax
4 add ecx, 1
5 cmp ecx, 3
6 je final
7 jmp loop
8 cmp ebx, 15
9 je f1
f1 f2

 Can we jump to final block when loop 3
times?
( = ecx_3 3)
( = ( + ecx_2 1 ) 3)
( = ( + (+ ecx_1 1 ) 1 ) 3)
( = ( + (+ (+ ecx_0 1) 1 ) 1 ) 3) and ( = ecx_0
0)
SMT Solver
SAT! This formula is
satisfiable.
68
0 mov ebx, 0
1 mov ecx, 0
2 mov eax, input
3 add ebx, eax
4 add ecx, 1
5 cmp ecx, 3
6 je final
7 jmp loop
8 cmp ebx, 15
9 je f1
f1 f2

 Can we enter f1?
1st 2nd 3rd
0 ebx_0 = 0
1 ecx_0 = 0
2 eax_0 = sym_0
3 ebx_1 =
ebx_0+eax_0
ebx_2 =
ebx_1+eax_0
ebx_3 =
ebx_2+eax_0
4 ecx_1 =
ecx_0+1
ecx_2 =
ecx_1+1
ecx_3 =
ecx_2+1
5 NE NE E
6 N N Y
7 Y Y
8 NE
9 N 69
0 mov ebx, 0
1 mov ecx, 0
2 mov eax, input
3 add ebx, eax
4 add ecx, 1
5 cmp ecx, 3
6 je final
7 jmp loop
8 cmp ebx, 15
9 je f1
f1 f2

 Tracking related instructions only?
1st 2nd 3rd
0 ebx = 0
1 ecx = 0
2 eax = 3
3 ebx = 3 ebx = 6 ebx = 9
4 ecx = 1 ecx = 2 ecx = 3
5 NE NE E
6 N N Y
7 Y Y
8 NE
9 N
70
0 mov ebx, 0
1 mov ecx, 0
2 mov eax, input
3 add ebx, eax
4 add ecx, 1
5 cmp ecx, 3
6 je final
7 jmp loop
8 cmp ebx, 15
9 je f1
f1 f2

 Number of possible path increasing exponentially
 In symbolic execution, every memory location is symbolize
 Too many symbole to solve
 Concolic Execution
 Only make the interesting memory symbolize
 Concrete value
71

1st 2nd 3rd
0 ebx_0 = 0
1 ecx_0 = 0
2 eax_0 = sym_0
3 ebx_1 =
ebx_0+eax_0
ebx_2 =
ebx_1+eax_0
ebx_3 =
ebx_2+eax_0
4 ecx_1 = 1 ecx_2 = 2 ecx_3 = 3
5 NE NE E
6 N N Y
7 Y Y
8 NE
9 N 72
0 mov ebx, 0
1 mov ecx, 0
2 mov eax, input
3 add ebx, eax
4 add ecx, 1
5 cmp ecx, 3
6 je final
7 jmp loop
8 cmp ebx, 15
9 je f1
f1 f2

 Which input make us arrive f1?
1st 2nd 3rd
0 ebx_0 = 0
1 ecx_0 = 0
2 eax_0 =
sym_0
3 ebx_1 =
ebx_0+eax_
0
ebx_2 =
ebx_1+eax_
0
ebx_3 =
ebx_2+eax_
0
4 ecx_1 = 1 ecx_2 = 2 ecx_3 = 3
5 NE NE E
6 N N Y
7 Y Y
8 NE
9 N
ebx_3
+
ebx_2
+
ebx_1
+
eax_0
eax_0
eax_0ebx_0
0 73

 Which input make us arrive f1?
Final
( = 15 (+ in ( + in (+ in 0) ) ) )
SMT Solver
SAT! This formula is
satisfiable when in = 5.
ebx_3
+
ebx_2
+
ebx_1
+
eax_0
eax_0
eax_0ebx_0
0
74

 In-house symbolic execution engine, called Grace
 Path Priority
 “Grace to focus on unique and interesting inputs, rather than
churning away at things that would likely lead down previously-
explored paths”
 Symbolize authentication/random token
 Powerful static
analysis
Grand Challenge
75

 Virtual Machine Symbolic Execution
Framework - S2E
 Selective symbolic execution/Concolic
Execution
 Execution consistency models
 state merging and prioritizing
Whole
System
QEMU
KLEEBinary
LLVM
76

 Angr
 Not only the symbolic execution engine, but a binary analysis
framework
 http://angr.io/
“Cyber Grand Shellphish”, shellphish, DEFCON 24
77

 binary-only symbolic execution
 Fast than S2E(whole system/LLVM), Angr(VEX Simulator)
 BAP-based binary instrument
 Veritesting
 A search strategy based on coverage
 Other feature
 fine-tuned process-based instrumentation and taint analysis
 access to an extensive set of tested x86 semantics
 several years of performance tuning for solvers (expression
rewriting, caches, etc)
 path merging
78

 Shellphish
 SoK: (State of) The Art of War:
Offensive Techniques in Binary
Analysis
 IEEE Symposium on Security
and Privacy 2016
79

 One of most important technique we learn from CGC is “How to
integrate efficiency fuzzer and sophisticated symbolic
execution”
80

 sharing seeds between Mayhem and our custom AFL.
https://blog.forallsecure.com/2016/02/09/unleashing-mayhem/
81

 Seed sharing: fuzzer + S2E + traffic replay
 Path exploration
 S2E helps Fuzzer to break through some branches
82

 Remind of AFL Gate-Keeper
83

 Driller
 Switch between fuzzer and symbolic execution
 Driller: Augmenting Fuzzing Through Selective Symbolic Execution
 Network and Distributed System Security Symposium 2016
84

 DEP
 ASLR
 Stack Guard
 CFI
 Pointer Integrity
 Shadow Stack
 Binary Patch
 Input filter
 IDS
87

 PS: Meyhem put most attention on attack, doing less on defense
88

 Patcherex
 https://github.com/shellphish/patcherex
 QEMU 0 Day for anti-analysis
89

 Return pointer encryption
 Protect indirect calls/jmps
 Extended Malloc allocations
 Randomly shift the stack (ASLR)
 Clean uninitialized stack space
90

91Automatic Binary Exploitation and Patching using Mechanical [Shell]Phish
Automatic Patching
● Prevent binary form being exploit
● Preserve binary functionality
● Preserve binary performance
○ speed
○ memory usage
○ disk space
● Prevent analysis from other teams

Patcherex
Patching
Backends
Patching
Techniques
Patches
Patched Binary
Original Binary

Patcherex
● Defensive Techniques
○ Return pointer encryption
○ Protect indirect calls/jmps
○ Extended Malloc allocations
○ Randomly shift the stack (ASLR)
○ ...

Patcherex
● Adversarial Techniques
○ Detect QEMU
mov eax, 0x1
push eax
push eax
push eax
fld TBYTE PTR [esp]
fsqrt
○ Backdoor
○ ...

Patcherex
● Making the original binary faster →
Our patches can be slower!
● Optimization Techniques:
○ Constant Propagation
○ Dead Assignment Elimination
○ ...

Patcherex – Backends
● Patching Backends
○ Inject code/data in an existing binary
○ No source code
○ No symbols

● How to inject code without breaking functionality?
0x0 : mov eax, 0x11
0x5 : jmp eax
0x7 : mov edx, 0x11223344
0xc : mov ebx, 0x55667788
0x11: mov ecx, ebx
0x0 : mov eax, 0x11
0x5 : jmp eax
0x7: call inserted_function
0xc : mov edx, 0x11223344
0x11: mov ebx, 0x55667788
0x16: mov ecx, ebx

● Detour Backend
○ Try to add code without moving the original one
○ Not always possible
○ Slow (requires a lot of additional jmp instructions)
0x0 : mov eax, 0x11
0x5 : jmp eax
0x7 : mov edx, 0x11223344
0xc : mov ebx, 0x55667788
0x11: mov ecx, ebx
0x0 : mov eax, 0x11
0x5 : jmp eax
0x7 : jmp out1
0xc : mov ebx, 0x55667788
0x11: mov ecx, ebx
mov edx, 0x11223344
call inserted_function
jmp 0xc

● Reassemler Backend
○ Recover original “program symbols”
○ More efficient code
○ (Slightly) less reliable
Ramblr: Making Reassembly Great Again.
R. Wang, Y. Shoshitaishvili, A. Bianchi, A. Machiry, J. Grosen, P. Grosen, C. Kruegel,
G. Vigna
In NDSS 2017

0x0 : mov eax, 0x11
0x5 : jmp eax
0x7 : mov edx, 0x11223344
0xc : mov ebx, 0x55667788
0x11: mov ecx, ebx
mov eax, _label1
jmp eax
mov edx, 0x11223344
mov ebx, 0x55667788
_label1:
mov ecx, ebx
mov eax, _label1
jmp eax
call inserted_function
mov edx, 0x11223344
mov ebx, 0x55667788
_label1:
mov ecx, ebx
0x0 : mov eax, 0x16
0x5 : jmp eax
0x7 : call inserted _function
0xc : mov edx, 0x11223344
0x11: mov ebx, 0x55667788
0x16: mov ecx, ebx

 CFI: control flow integrity
 Shadow stacks
 Maintain a duplicate stack
 Once the return address difference from the one in shadow stack,
then attack is detected
 DEP
 Randomization
 Data leakage defense
101

 Control Flow Integrity
https://www.trust.informatik.tu-darmstadt.de/research/projects/current-
projects/control-flow-integrity/
102

 TechX achieve the
first place about
security
 PEASOUP
 Code Sonar
Grand Challenge
103

 The other important aspect is “how to integrate many system in
large architecture”
 Handle with complicated system architecture
 Reliable is difficult
 Mayhem meets some problem and fails in half of the game
104

105

“Cyber Grand Challenge and CodeJitsu”, Chao Zhang
106

 DARPA CGC Introduction
 Most team have research, CTF and enterprise support
 Automatic Vulnerability Discovery
 Fuzzer and Symbolic Execution are widely used technique in CGC
 How to integrate fuzzer and symbolic execution
 Engineering Power: Integration many different software system
107

 https://cgc.darpa.mil/
 https://www.cybergrandchallenge.com/
 “DARPA’s Cyber Grand Challenge: Creating a League of Extra-Ordinary Machines”, Ben Price
and Michael Zhivich, ACSAC
 “Rise of the Machines: Cyber Grand Challenge 及 DEFCON 24 CTF 决赛介绍”, MaskRay
 “Cyber Grand Challenge and CodeJitsu”, Chao Zhang
 https://www.youtube.com/watch?v=xfgGZq86iWk
 Reddit IamA Mayhem, the Hacking Machine that won DARPA‘s Cyber Grand Challenge. AMA!
 Unleashing the Mayhem CRS, ForAllSecure
 “Cyber Grand Shellphish”, shellphish, DEFCON 24
 “The Cyber Grand Challenge”, GrammaTech Eric Rizzi
 “Hybrid Concolic Execution, Part 1 (Background)”, GrammaTech Ducson Nguyen
 “Hybrid Concolic Execution, Part 2”, GrammaTech Ducson Nguyen
 “Case Study: LEGIT_00004”, ForAllSecure
108

Cgc2

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Cgc2

Similar to Cgc2 (20)

More from Chong-Kuan Chen

More from Chong-Kuan Chen (16)

Recently uploaded

Recently uploaded (20)

Cgc2