Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© 2015 IBM Corporation
How to cook your own fast and scalable DBI-based security tool. A case study
Exploring billion stat...
© 2015 IBM Corporation
IBM Research - Haifa
About me
 PhD (Tomsk State University of Control Systems and Radioelectronics...
© 2015 IBM Corporation
IBM Research - Haifa
Outline
 Dynamic binary instrumentation technique
– General idea and techniqu...
© 2015 IBM Corporation
IBM Research - Haifa
Dynamic binary instrumentation (DBI)
DBI is a technique of analyzing the behav...
© 2015 IBM Corporation
IBM Research - Haifa
How does it work ?
Application
launcher.exe
core.dll
DBI Engine
CreateProcess ...
© 2015 IBM Corporation
IBM Research - Haifa
Frameworks Comparison
DynamoRIO Intel PIN
Redistribution
model
Open-source, BS...
© 2015 IBM Corporation
IBM Research - Haifa
Instructions Counting. Example
DynamoRIOIntel PIN
2017
© 2015 IBM Corporation
IBM Research - Haifa
Instrumentation Granularity
Instruction level
(instrument all executed instruc...
© 2015 IBM Corporation
IBM Research - Haifa
Application
 Software security analysis & testing
– Support fuzzing (code cov...
© 2015 IBM Corporation
IBM Research - Haifa
Example I. Bugs detection
2017
© 2015 IBM Corporation
IBM Research - Haifa
WinHeap Explorer Tool
 WinHeap Explorer is a system for heap-based bug detect...
© 2015 IBM Corporation
IBM Research - Haifa
Motivation Example
2017
© 2015 IBM Corporation
IBM Research - Haifa
Motivation Example
2017
© 2015 IBM Corporation
IBM Research - Haifa
Motivation Example
Heap memory block for a
Heap Layout
pMethod 1 pMethod 2 pMe...
© 2015 IBM Corporation
IBM Research - Haifa
Motivation Example
Heap memory block for a pEvilMet 1 pMethod 2 pMethod 3 pMet...
© 2015 IBM Corporation
IBM Research - Haifa
Detection of Heap-based Bugs. General Idea
Code Heap
instruction #1
instructio...
© 2015 IBM Corporation
IBM Research - Haifa
Detection of Heap-based Bugs. General Idea
Code Heap
instruction #1
instructio...
© 2015 IBM Corporation
IBM Research - Haifa
Detection of Heap-based Bugs. General Idea
Code
Correct instruction #1
Correct...
© 2015 IBM Corporation
IBM Research - Haifa
Detection of Heap-based Bugs. General Idea
Code
Correct instruction #1
Correct...
© 2015 IBM Corporation
IBM Research - Haifa
Detection of Heap-based Bugs. General Idea
Code
Correct instruction #1
Correct...
© 2015 IBM Corporation
IBM Research - Haifa
Detection of Heap-based Bugs. General Idea
Code
Correct instruction #1
Correct...
© 2015 IBM Corporation
IBM Research - Haifa
WinHeap Explorer. Implementation #1
Step 1. Instrument all heap management API...
© 2015 IBM Corporation
IBM Research - Haifa
WinHeap Explorer. Implementation #1
Step 1. Instrument all heap management API...
© 2015 IBM Corporation
IBM Research - Haifa
WinHeap Explorer. Implementation #1
Step 1. Instrument all heap management API...
© 2015 IBM Corporation
IBM Research - Haifa
Shadow Memory Approach
2017
© 2015 IBM Corporation
IBM Research - Haifa
Windows Heap Management APIs Architecture
kernel32.dll
HeapAlloc
GlobalAlloc
L...
© 2015 IBM Corporation
IBM Research - Haifa
Light-weight instrumentation. System DLLs instrumentation
2017
© 2015 IBM Corporation
IBM Research - Haifa
Light-weight instrumentation. Whole System Architecture
2017
© 2015 IBM Corporation
IBM Research - Haifa
WinHeap Explorer. Runtime overhead #2
2017
© 2015 IBM Corporation
IBM Research - Haifa
Example 2. Malware analysis
2017
© 2015 IBM Corporation
IBM Research - Haifa
Dynamic Malware Analysis. Motivation
2017
© 2015 IBM Corporation
IBM Research - Haifa
Dynamic Malware Analysis. Motivation
2017
© 2015 IBM Corporation
IBM Research - Haifa
Dynamic Malware Analysis. Idea & Solution
 Goal – transparently and efficient...
© 2015 IBM Corporation
IBM Research - Haifa
DrLtrace Tool
 DrLtrace is a standalone application for transparent API calls...
© 2015 IBM Corporation
IBM Research - Haifa
Output Examples
2017
© 2015 IBM Corporation
IBM Research - Haifa
Conclusion
 DBI is a powerful technique for transparent and efficient machine...
© 2015 IBM Corporation
IBM Research - Haifa
Links
 WinHeapExplorer – https://github.com/WinHeapExplorer/WinHeap-Explorer
...
© 2015 IBM Corporation
IBM Research - Haifa
Thank you for you attention!
PhD, Research Staff Member
(Cyber Security)
IBM R...
Upcoming SlideShare
Loading in …5
×

Изучаем миллиард состояний программы на уровне профи. Как разработать быстрый и масштабируемый DBI-инструмент для обеспечения безопасности. Пример из практики

184 views

Published on

Основная цель доклада — познакомить аудиторию с динамической бинарной инструментацией (DBI), углубиться в эту тему, продемонстрировать основные преимущества этой методики, а также рассмотреть типичные проблемы, связанные с ее применением на практике. Слушатели узнают об основных аспектах технологии DBI, поймут, в каких сферах ее можно использовать, а также познакомятся с потенциальными проблемами при написании собственной утилиты на основе DBI-фреймворков Intel PIN и DynamoRIO. Докладчик на реальных примерах покажет, как DBI может применяться для поиска ошибок типа переполнения кучи в «тяжеловесных» программах и для динамического анализа вредоносного кода.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Изучаем миллиард состояний программы на уровне профи. Как разработать быстрый и масштабируемый DBI-инструмент для обеспечения безопасности. Пример из практики

  1. 1. © 2015 IBM Corporation How to cook your own fast and scalable DBI-based security tool. A case study Exploring billion states of a program like a pro 2017
  2. 2. © 2015 IBM Corporation IBM Research - Haifa About me  PhD (Tomsk State University of Control Systems and Radioelectronics) – Vulnerabilities detection in machine code (x86)  Cyber Security Researcher at IBM Research Lab in Haifa, Israel – R&D in technologies for highly-evasive malware analysis and detection  A main contributor to DynamoRIO/DrMemory DBI frameworks 2017
  3. 3. © 2015 IBM Corporation IBM Research - Haifa Outline  Dynamic binary instrumentation technique – General idea and technique implementation – DynamoRIO and Intel PIN frameworks comparison – Possible application fields  Example I. Heap-based bug detection using DBI – General idea & motivation – Implementation & issues – Solutions for described issues  Example II. Dynamic malware analysis – General idea & motivation – Implementation – Tool demo  Conclusion 2017
  4. 4. © 2015 IBM Corporation IBM Research - Haifa Dynamic binary instrumentation (DBI) DBI is a technique of analyzing the behavior of a binary application at runtime through the injection of instrumentation code 2017
  5. 5. © 2015 IBM Corporation IBM Research - Haifa How does it work ? Application launcher.exe core.dll DBI Engine CreateProcess (suspended) (1) Inject core.dll (2) Application in memory Windows kernel Takebasicblock (4)(3) Hook entry point core.dll + user dll/dlls shared system dlls ins1 ins2 ins3 insN . . Code cache basic block transformation (5) ins1 inst_ins1 inst_ins2 ins2 inst_ins3 inst_ins4 ins3 inst_ins4 inst_ins5 insN inst_insM inst_insM+1 Execute & calculate addr of next basic block (6) Takenextbasicblock (7) . .
  6. 6. © 2015 IBM Corporation IBM Research - Haifa Frameworks Comparison DynamoRIO Intel PIN Redistribution model Open-source, BSD – license Proprietary, no-source code available Supported architectures x86, x86-64, ARM, AArch64 x86, x86-64 Supported Platforms Linux, Windows, MacOS, Android Linux, Windows, MacOS, Android Average runtime overhead 108% (no tool) 139% (BBs counter) 130% (no tool) 162% (BBs counter) Language C/C++ C/C++ (some Python wrappers available) Technology Binary code transformation callout/trampolines 2017
  7. 7. © 2015 IBM Corporation IBM Research - Haifa Instructions Counting. Example DynamoRIOIntel PIN 2017
  8. 8. © 2015 IBM Corporation IBM Research - Haifa Instrumentation Granularity Instruction level (instrument all executed instructions) Basic block level (instrument all executed basic blocks) Function call level (instrument all executed calls) Module level (instrument all load/unload module events) Events • exceptions/signals • syscalls • thread/process creation/deletion 2017
  9. 9. © 2015 IBM Corporation IBM Research - Haifa Application  Software security analysis & testing – Support fuzzing (code coverage assessment, tainted input tracking) – Bugs detection (overflows, use-after-free, uninitialized access and etc.) – Symbolic execution – Software bugs exploitability assessment  Malware analysis – Execution tracing – Automatic unpacking  Reverse-engineering – Control-flow graph visualization – Debugging – Taint-tracking  Other non-security fields – Performance evaluation – Memory leak detection – Optimization 2017
  10. 10. © 2015 IBM Corporation IBM Research - Haifa Example I. Bugs detection 2017
  11. 11. © 2015 IBM Corporation IBM Research - Haifa WinHeap Explorer Tool  WinHeap Explorer is a system for heap-based bug detection with the lowest runtime overhead build on top of Intel PIN framework.  Advantages: – Light-weight instrumentation support (shown further) – Lowest runtime-overhead – Open-source (BSD license) 2017
  12. 12. © 2015 IBM Corporation IBM Research - Haifa Motivation Example 2017
  13. 13. © 2015 IBM Corporation IBM Research - Haifa Motivation Example 2017
  14. 14. © 2015 IBM Corporation IBM Research - Haifa Motivation Example Heap memory block for a Heap Layout pMethod 1 pMethod 2 pMethod 3 pMethod 4 Virtual table for pFileOpen 2017
  15. 15. © 2015 IBM Corporation IBM Research - Haifa Motivation Example Heap memory block for a pEvilMet 1 pMethod 2 pMethod 3 pMethod 4 Virtual table for pFileOpen Heap Layout 2017
  16. 16. © 2015 IBM Corporation IBM Research - Haifa Detection of Heap-based Bugs. General Idea Code Heap instruction #1 instruction #2 instruction #3 instruction #4 instruction #5 0x0 – 0x3 0x4 – 0x7 0x8 – 0xB 0xC – 0xF 0x10 – 0x13 0x14 – 0x17 0x80 – 0x83 0x84 – 0x87 0x88 – 0x8B 0x8C – 0x8F 0x90 – 0x93 0x94 – 0x98 … 2017
  17. 17. © 2015 IBM Corporation IBM Research - Haifa Detection of Heap-based Bugs. General Idea Code Heap instruction #1 instruction #2 instruction #3 instruction #4 instruction #5 0x0 – 0x3 0x4 – 0x7 0x8 – 0xB 0xC – 0xF 0x10 – 0x13 0x14 – 0x17 0x80 – 0x83 0x84 – 0x87 0x88 – 0x8B 0x8C – 0x8F 0x90 – 0x93 0x94 – 0x98 … Allocated memory block #1 Freed memory block #2 2017
  18. 18. © 2015 IBM Corporation IBM Research - Haifa Detection of Heap-based Bugs. General Idea Code Correct instruction #1 Correct instruction #2 Incorrect instruction #3 Incorrect instruction #4 Incorrect instruction #5 [overflow] [access block #1] [access block #1] [underflow] [use after free] Heap 0x0 – 0x3 0x4 – 0x7 0x8 – 0xB 0xC – 0xF 0x10 – 0x13 0x14 – 0x17 0x80 – 0x83 0x84 – 0x87 0x88 – 0x8B 0x8C – 0x8F 0x90 – 0x93 0x94 – 0x98 … Allocated memory block #1 Freed memory block #2 2017
  19. 19. © 2015 IBM Corporation IBM Research - Haifa Detection of Heap-based Bugs. General Idea Code Correct instruction #1 Correct instruction #2 Incorrect instruction #3 Incorrect instruction #4 Incorrect instruction #5 [overflow] [access block #1] [access block #1] [underflow] [use after free] Heap 0x0 – 0x3 0x4 – 0x7 0x8 – 0xB 0xC – 0xF 0x10 – 0x13 0x14 – 0x17 0x80 – 0x83 0x84 – 0x87 0x88 – 0x8B 0x8C – 0x8F 0x90 – 0x93 0x94 – 0x98 … Allocated memory block #1 Freed memory block #2 redzone redzone marked as freed 2017
  20. 20. © 2015 IBM Corporation IBM Research - Haifa Detection of Heap-based Bugs. General Idea Code Correct instruction #1 Correct instruction #2 Incorrect instruction #3 Incorrect instruction #4 Incorrect instruction #5 [overflow] [access block #1] [access block #1] [underflow] [use after free] Heap 0x0 – 0x3 0x4 – 0x7 0x8 – 0xB 0xC – 0xF 0x10 – 0x13 0x14 – 0x17 0x80 – 0x83 0x84 – 0x87 0x88 – 0x8B 0x8C – 0x8F 0x90 – 0x93 0x94 – 0x98 … Allocated memory block #1 Freed memory block #2 redzone redzone marked as freed 𝑖𝑓 𝑝𝑜𝑖𝑛𝑡𝑒𝑟 ∈ 𝑟𝑒𝑑𝑧𝑜𝑛𝑒 𝑜𝑟 𝑝𝑜𝑖𝑛𝑡𝑒𝑟 ∈ 𝑓𝑟𝑒𝑒𝑑 𝑡ℎ𝑒𝑛 𝒃𝒖𝒈 2017
  21. 21. © 2015 IBM Corporation IBM Research - Haifa Detection of Heap-based Bugs. General Idea Code Correct instruction #1 Correct instruction #2 Incorrect instruction #3 Incorrect instruction #4 Incorrect instruction #5 [overflow] [access block #1] [access block #1] [underflow] [use after free] Heap 0x0 – 0x3 0x4 – 0x7 0x8 – 0xB 0xC – 0xF 0x10 – 0x13 0x14 – 0x17 0x80 – 0x83 0x84 – 0x87 0x88 – 0x8B 0x8C – 0x8F 0x90 – 0x93 0x94 – 0x98 … Allocated memory block #1 Freed memory block #2 redzone redzone marked as freed 2017
  22. 22. © 2015 IBM Corporation IBM Research - Haifa WinHeap Explorer. Implementation #1 Step 1. Instrument all heap management API calls (allocation/reallocation/freeing) Step 2. Save redzones and heap block statuses (freed or in use) in a hashtable Step 3. Instrument all executed instructions, dynamically check that an instruction doesn’t access redzone or previously freed memory block 2017
  23. 23. © 2015 IBM Corporation IBM Research - Haifa WinHeap Explorer. Implementation #1 Step 1. Instrument all heap management API calls (allocation/reallocation/freeing) Step 2. Save redzones and heap block statuses (freed or in use) in a hashtable Step 3. Instrument all executed instructions, dynamically check that an instruction doesn’t access redzone or previously freed memory block. Results:  Runtime overhead: x310-x850 – Launching Mozilla Firefox = ~498M instructions (or 8 minutes to show first window) – Launching Acrobat Reader = ~84M instructions (or 3 minutes to show first window) – Launching PowerPoint = ~860M instructions (or 12 minutes to show first window)  Memory overhead: x90-x120 2017
  24. 24. © 2015 IBM Corporation IBM Research - Haifa WinHeap Explorer. Implementation #1 Step 1. Instrument all heap management API calls (allocation/reallocation/freeing) Step 2. Save redzones and heap block statuses (freed or in use) in a hashtable Step 3. Instrument all executed instructions, dynamically check that an instruction doesn’t access redzone or previously freed memory block. Results:  Runtime overhead: x310-x850 – Launching Mozilla Firefox = ~498M instructions (or 8 minutes to show first window) – Launching Acrobat Reader = ~84M instructions (or 3 minutes to show first window) – Launching PowerPoint = ~860M instructions (or 12 minutes to show first window)  Memory overhead: x90-x120 Absolutely unacceptable 2017
  25. 25. © 2015 IBM Corporation IBM Research - Haifa Shadow Memory Approach 2017
  26. 26. © 2015 IBM Corporation IBM Research - Haifa Windows Heap Management APIs Architecture kernel32.dll HeapAlloc GlobalAlloc LocalAlloc Memory allocation msvcr*.dll malloc calloc new [] ole32.dll CoTaskMemAlloc kernel32.dll HeapReAlloc GlobalReAlloc LocalReAlloc Memory reallocation msvcr*.dll realloc ole32.dll CoTaskMemrealloc kernel32.dll HeapFree GlobalFree LocalFree Memory freeing msvcr*.dll free delete [] ole32.dll CoTaskMemFree ntdll.dll RtlAllocateHeap RtlReAllocateHeap RtlFreeHeap Windows Kernel 2017
  27. 27. © 2015 IBM Corporation IBM Research - Haifa Light-weight instrumentation. System DLLs instrumentation 2017
  28. 28. © 2015 IBM Corporation IBM Research - Haifa Light-weight instrumentation. Whole System Architecture 2017
  29. 29. © 2015 IBM Corporation IBM Research - Haifa WinHeap Explorer. Runtime overhead #2 2017
  30. 30. © 2015 IBM Corporation IBM Research - Haifa Example 2. Malware analysis 2017
  31. 31. © 2015 IBM Corporation IBM Research - Haifa Dynamic Malware Analysis. Motivation 2017
  32. 32. © 2015 IBM Corporation IBM Research - Haifa Dynamic Malware Analysis. Motivation 2017
  33. 33. © 2015 IBM Corporation IBM Research - Haifa Dynamic Malware Analysis. Idea & Solution  Goal – transparently and efficiently trace malware’s library calls  Solution – Instrument calls to exported library functions – Print a trace of each executed function along with some arguments information – Print a return address of each executed function (to be able to recognize calls from unpacked code) 2017
  34. 34. © 2015 IBM Corporation IBM Research - Haifa DrLtrace Tool  DrLtrace is a standalone application for transparent API calls tracing build on top of DynamoRIO framework.  Benefits: – Transparent (no API-hooking, no debugging) – High visibility (each API call, all arguments) – Open-source (BSD license) – Supported Windows, Linux, Android – Easy to use, just specify: drltrace.exe –logdir <log_name> -- <app_name> 2017
  35. 35. © 2015 IBM Corporation IBM Research - Haifa Output Examples 2017
  36. 36. © 2015 IBM Corporation IBM Research - Haifa Conclusion  DBI is a powerful technique for transparent and efficient machine code introspection  Numerous possible ways to use, especially for software security analysis, dynamic malware introspection, reverse-engineering and etc.  Traditional fast and scalable data structures may introduce significant overhead in case of DBI  Two open-source tools were introduced: – WinHeap Explorer – a tool for heap-based bug detection in Windows applications – Dr.Ltrace – a tool for transparent dynamic library calls tracing 2017
  37. 37. © 2015 IBM Corporation IBM Research - Haifa Links  WinHeapExplorer – https://github.com/WinHeapExplorer/WinHeap-Explorer  DrLtrace 1 - https://github.com/DynamoRIO/drmemory/tree/master/drltrace  Whitepaper - https://github.com/WinHeapExplorer/WinHeap-Explorer/tree/master/PHD 1 – please build from source code to have the latest functionality 2017
  38. 38. © 2015 IBM Corporation IBM Research - Haifa Thank you for you attention! PhD, Research Staff Member (Cyber Security) IBM Research Israel Maksim Shudrak maksims@il.ibm.com mxmssh@gmail.com 2017

×