Slides da minha apresentação no IC/UNICAMP, em 2017, referentes a defesa da minha dissertação de mestrado "Hardware-Assisted Malware Analysis"

1. Introduction Research Final Remarks
Hardware-Assisted Malware Analysis
Marcus Botacin1, Andr´e Gr´egio3, Paulo L´ıcio de Geus2
1Msc. Candidate
Institute of Computing - UNICAMP
{marcus}@lasca.ic.unicamp.br
2Advisor
Institute of Computing - UNICAMP
{paulo}@lasca.ic.unicamp.br
3Co-Advisor
Federal University of Paran´a (UFPR)
gregio@inf.ufpr.br
July 28th, 2017
Hardware-Assisted Malware Analysis IC-UNICAMP

2. Introduction Research Final Remarks
Topics
1 Introduction
Malware
Malware Analysis
Anti-Analysis
Transparency
2 Research
A Survey
Branch Monitoring
3 Final Remarks
Conclusions
Future Work
Acknowledgments
Hardware-Assisted Malware Analysis IC-UNICAMP

3. Introduction Research Final Remarks
Malware
Topics
1 Introduction
Malware
Malware Analysis
Anti-Analysis
Transparency
2 Research
A Survey
Branch Monitoring
3 Final Remarks
Conclusions
Future Work
Acknowledgments
Hardware-Assisted Malware Analysis IC-UNICAMP

4. Introduction Research Final Remarks
Malware
Malware Threats.
Figure: Washington Post: https://tinyurl.com/ljo7ekm
Figure: BBC: https://tinyurl.com/mfogjhe
Hardware-Assisted Malware Analysis IC-UNICAMP

5. Introduction Research Final Remarks
Malware Analysis
Topics
1 Introduction
Malware
Malware Analysis
Anti-Analysis
Transparency
2 Research
A Survey
Branch Monitoring
3 Final Remarks
Conclusions
Future Work
Acknowledgments
Hardware-Assisted Malware Analysis IC-UNICAMP

6. Introduction Research Final Remarks
Malware Analysis
Malware Analysis.
Figure: Imperva: https://tinyurl.com/zkbsnl2
Figure: Enigma: https://tinyurl.com/kydgwve
Hardware-Assisted Malware Analysis IC-UNICAMP

7. Introduction Research Final Remarks
Malware Analysis
More specifically...
Static Analysis
Analyzing without executing it.
Dynamic Analysis
Analyzing by executing it.
Hardware-Assisted Malware Analysis IC-UNICAMP

8. Introduction Research Final Remarks
Anti-Analysis
Topics
1 Introduction
Malware
Malware Analysis
Anti-Analysis
Transparency
2 Research
A Survey
Branch Monitoring
3 Final Remarks
Conclusions
Future Work
Acknowledgments
Hardware-Assisted Malware Analysis IC-UNICAMP

9. Introduction Research Final Remarks
Anti-Analysis
Malware Analysis.
Figure: Themerkle: https://tinyurl.com/kasuxcr
Figure: Forbes: https://tinyurl.com/l7ecrex
Hardware-Assisted Malware Analysis IC-UNICAMP

10. Introduction Research Final Remarks
Anti-Analysis
What happens to static analysis?
Control Flow Obfuscation
Listing 1: Control flow
obfuscation example.
1 entry : xor eax , eax
2 jnz %l 1
3 l 1 : xor eax , eax
4 jnz %l 2
5 l 2 : xor eax , eax
6 jnz %l 3
7 ln : . . .
Hardware-Assisted Malware Analysis IC-UNICAMP

11. Introduction Research Final Remarks
Anti-Analysis
What happens to static analysis?
Path Explosion
Figure: Source Image: https://tinyurl.com/n52fz5r
Hardware-Assisted Malware Analysis IC-UNICAMP

12. Introduction Research Final Remarks
Anti-Analysis
What happens to dynamic analysis?
Debugger Detection
Listing 2: Anti-debug example.
1 i f ( IsDebuggerPresent () )
2 p r i n t f (” debuggedn ”) ;
VM Detection
Listing 3: Anti-vm example.
1 cpuid (0 x40000000 , &eax , &ebx , &ecx , &edx ) ;
2 memcpy( h y p e r v e n d o r i d + 0 , &ebx , 4) ;
3 i f ( ! strcmp ( hyper vendor id , ”VMwareVMware”) )
4 p r i n t f (”VMn ”) ;
Hardware-Assisted Malware Analysis IC-UNICAMP

13. Introduction Research Final Remarks
Anti-Analysis
And then...
Figure: Real malware claiming a registry problem when an anti-analysis
trick succeeded.
Hardware-Assisted Malware Analysis IC-UNICAMP

14. Introduction Research Final Remarks
Anti-Analysis
And then...
Figure: Commercial solution armored with anti-debug technique.
Hardware-Assisted Malware Analysis IC-UNICAMP

15. Introduction Research Final Remarks
Anti-Analysis
And then...
Figure: Real malware impersonating a secure solution which cannot run
under an hypervisor.
Hardware-Assisted Malware Analysis IC-UNICAMP

16. Introduction Research Final Remarks
Transparency
Topics
1 Introduction
Malware
Malware Analysis
Anti-Analysis
Transparency
2 Research
A Survey
Branch Monitoring
3 Final Remarks
Conclusions
Future Work
Acknowledgments
Hardware-Assisted Malware Analysis IC-UNICAMP

17. Introduction Research Final Remarks
Transparency
Definitions
1 Higher privileged.
2 No non-privileged side-effects.
3 Identical Basic Instruction Semantics.
4 Transparent Exception Handling.
5 Identical Measurement of Time.
Hardware-Assisted Malware Analysis IC-UNICAMP

18. Introduction Research Final Remarks
A Survey
Topics
1 Introduction
Malware
Malware Analysis
Anti-Analysis
Transparency
2 Research
A Survey
Branch Monitoring
3 Final Remarks
Conclusions
Future Work
Acknowledgments
Hardware-Assisted Malware Analysis IC-UNICAMP

19. Introduction Research Final Remarks
A Survey
HVM
Figure: Type-1 Hypervisor. Source: https://tinyurl.com/ku7oho5
Hardware-Assisted Malware Analysis IC-UNICAMP

20. Introduction Research Final Remarks
A Survey
HVM
Figure: HVM operating layers
.
Hardware-Assisted Malware Analysis IC-UNICAMP

21. Introduction Research Final Remarks
A Survey
HVM
Figure: HVM memory operation.
Hardware-Assisted Malware Analysis IC-UNICAMP

22. Introduction Research Final Remarks
A Survey
HVM
Figure: Ether’s architecture.
Hardware-Assisted Malware Analysis IC-UNICAMP

23. Introduction Research Final Remarks
A Survey
HVM
Figure: Fattori’s architecture.
Hardware-Assisted Malware Analysis IC-UNICAMP

24. Introduction Research Final Remarks
A Survey
HVM
Figure: Fattori’s traceable events.
Hardware-Assisted Malware Analysis IC-UNICAMP

25. Introduction Research Final Remarks
A Survey
HVM
Figure: Fattori’s techniques.
Hardware-Assisted Malware Analysis IC-UNICAMP

26. Introduction Research Final Remarks
A Survey
HVM
Figure: MAVMM’s overhead.
Hardware-Assisted Malware Analysis IC-UNICAMP

27. Introduction Research Final Remarks
A Survey
SMM
Figure: Operation modes. Source: https://tinyurl.com/l2uqr8d
Hardware-Assisted Malware Analysis IC-UNICAMP

28. Introduction Research Final Remarks
A Survey
SMM
Figure: SMIs. Source: https://tinyurl.com/kuqlmau
Hardware-Assisted Malware Analysis IC-UNICAMP

29. Introduction Research Final Remarks
A Survey
Battle of the rings
Figure: Processor rings. Source: https://tinyurl.com/mzjezkg
Hardware-Assisted Malware Analysis IC-UNICAMP

30. Introduction Research Final Remarks
A Survey
AMT/ME
Figure: Arstechnica. Source: https://tinyurl.com/kc62fw8
Hardware-Assisted Malware Analysis IC-UNICAMP

31. Introduction Research Final Remarks
A Survey
SGX
Figure: SGX enclaves. Source: https://tinyurl.com/ln6eb4g
Hardware-Assisted Malware Analysis IC-UNICAMP

32. Introduction Research Final Remarks
A Survey
External Hardware
Figure: JTAG. Source: https://tinyurl.com/m8r79e8
Hardware-Assisted Malware Analysis IC-UNICAMP

33. Introduction Research Final Remarks
A Survey
Branch Monitor
Figure: Branch Stack.
Hardware-Assisted Malware Analysis IC-UNICAMP

34. Introduction Research Final Remarks
A Survey
Performance Monitor
Figure: PMU-based side-effect detection.
Source: https://tinyurl.com/m9zgphk
Hardware-Assisted Malware Analysis IC-UNICAMP

35. Introduction Research Final Remarks
A Survey
GPUs
Figure: GPU. Source: https://tinyurl.com/lnhufn9
Hardware-Assisted Malware Analysis IC-UNICAMP

36. Introduction Research Final Remarks
A Survey
TSX
Figure: TSX-based CFI. Source: https://tinyurl.com/l8jfycj
Hardware-Assisted Malware Analysis IC-UNICAMP

37. Introduction Research Final Remarks
Branch Monitoring
Topics
1 Introduction
Malware
Malware Analysis
Anti-Analysis
Transparency
2 Research
A Survey
Branch Monitoring
3 Final Remarks
Conclusions
Future Work
Acknowledgments
Hardware-Assisted Malware Analysis IC-UNICAMP

38. Introduction Research Final Remarks
Branch Monitoring
Proposed Framework
Figure: Proposed framework architecture.
Hardware-Assisted Malware Analysis IC-UNICAMP

39. Introduction Research Final Remarks
Branch Monitoring
Could I develop a performance-counter-based malware
analyzer?
Could I isolate processes’ actions?
Figure: Data Storage (DS) AREA.
Hardware-Assisted Malware Analysis IC-UNICAMP

40. Introduction Research Final Remarks
Branch Monitoring
Could I develop a performance-counter-based malware
analyzer?
Could I isolate processes’ actions?
Figure: Local Vector Table (LVT).
Hardware-Assisted Malware Analysis IC-UNICAMP

41. Introduction Research Final Remarks
Branch Monitoring
Could I develop a performance-counter-based malware
analyzer?
Is CG reconstruction possible?
Table: ASLR - Library placement after two consecutive reboots.
Library NTDLL KERNEL32 KERNELBASE
Address 1 0xBAF80000 0xB9610000 0xB8190000
Address 2 0x987B0000 0x98670000 0x958C0000
Hardware-Assisted Malware Analysis IC-UNICAMP

42. Introduction Research Final Remarks
Branch Monitoring
Could I develop a performance-counter-based malware
analyzer?
Is CG reconstruction possible?
Table: Function Offsets from ntdll.dll library.
Function Offset
NtCreateProcess 0x3691
NtCreateProcessEx 0x30B0
NtCreateProfile 0x36A1
NtCreateResourceManager 0x36C1
NtCreateSemaphore 0x36D1
NtCreateSymbolicLinkObject 0x36E1
NtCreateThread 0x30C0
NtCreateThreadEx 0x36F1
Hardware-Assisted Malware Analysis IC-UNICAMP

43. Introduction Research Final Remarks
Branch Monitoring
Could I develop a performance-counter-based malware
analyzer?
Is CG reconstruction possible?
Figure: Introspection Mechanism.
Hardware-Assisted Malware Analysis IC-UNICAMP

44. Introduction Research Final Remarks
Branch Monitoring
Could I develop a performance-counter-based malware
analyzer?
Is CG reconstruction possible?
_lock_file+0x90printf+0xe3__iob_funcprintf+0xcaprintfNewToy
Figure: Step Into.
scanf+0x3f NewToy printf
Figure: Step Over.
Hardware-Assisted Malware Analysis IC-UNICAMP

45. Introduction Research Final Remarks
Branch Monitoring
Is CFG reconstruction possible?
Figure: Code block identification.
Hardware-Assisted Malware Analysis IC-UNICAMP

46. Introduction Research Final Remarks
Branch Monitoring
Is the final solution transparent?
Identified tricks
Listing 4: Fake Conditional.
1 0x190 xor eax , eax
2 0x192 jnz 0x19c
Listing 5: Control Flow
Change.
1 0x180 push 0x10a
2 0x185 r e t
Hardware-Assisted Malware Analysis IC-UNICAMP

47. Introduction Research Final Remarks
Branch Monitoring
Is the final solution transparent?
Identified tricks
Listing 6: Hook Detection.
1 0x340 cmp eax ,0 xe9
2 0x345 jnz 0x347
Listing 7: Hardware Debugger Detection.
1 0x400 QWORD PTR f s :0 x0 , rsp
2 0x409 mov rax ,QWORD PTR [ rsp+0xc ]
3 0x40e cmp rbx ,QWORD PTR [ rax+0x4 ]
Hardware-Assisted Malware Analysis IC-UNICAMP

48. Introduction Research Final Remarks
Branch Monitoring
Is the final solution transparent?
Deviating Behavior
Figure: Deviating behavior identification.
Hardware-Assisted Malware Analysis IC-UNICAMP

49. Introduction Research Final Remarks
Branch Monitoring
Is the final solution transparent?
Deviating Behavior
Figure: Divergence: True Positive. Figure: Divergence: False Positive.
Hardware-Assisted Malware Analysis IC-UNICAMP

50. Introduction Research Final Remarks
Branch Monitoring
Could I develop a Debugger?
Inverted I/O
Figure: Debugger’s working mechanism.
Hardware-Assisted Malware Analysis IC-UNICAMP

51. Introduction Research Final Remarks
Branch Monitoring
Could I develop a Debugger?
Suspending Processes
EnumProcessThreads + SuspendThread.
DebugActiveProcess.
NtSuspendProcess.
Hardware-Assisted Malware Analysis IC-UNICAMP

52. Introduction Research Final Remarks
Branch Monitoring
Could I develop a Debugger?
Integration
Figure: GDB integration.
Hardware-Assisted Malware Analysis IC-UNICAMP

53. Introduction Research Final Remarks
Branch Monitoring
Does the solution handle ROP attacks?
ROP Attacks
Figure: ROP chain example.
Hardware-Assisted Malware Analysis IC-UNICAMP

54. Introduction Research Final Remarks
Branch Monitoring
Does the solution handle ROP attacks?
CALL-RET Policy
Figure: CALL-RET CFI policy.
Hardware-Assisted Malware Analysis IC-UNICAMP

55. Introduction Research Final Remarks
Branch Monitoring
Does the solution handle ROP attacks?
Gadget-size policy
Figure: KBouncer’s exploit stack.
Hardware-Assisted Malware Analysis IC-UNICAMP

56. Introduction Research Final Remarks
Branch Monitoring
Does the solution handle ROP attacks?
Exploit Analysis
Table: Excerpt of the branch window of the ROP payload.
FROM TO
—- 0x7c346c0a
0x7c346c0b 0x7c37a140
0x7c37a141 —-
Hardware-Assisted Malware Analysis IC-UNICAMP

57. Introduction Research Final Remarks
Branch Monitoring
Does the solution handle ROP attacks?
Exploit Analysis
Listing 8: Static disassembly of the MSVCR71.dll library.
1 7 c346c08 : f2 0 f 58 c3 addsd %xmm3,%xmm0
2 7 c346c0c : 66 0 f 13 44 24 04 movlpd %xmm0,0 x4(%esp )
Listing 9: Dynamic disassembly of the
MSVC71.dll executed code.
1 0x1000 ( s i z e =1) pop rax
2 0x1001 ( s i z e =1) r e t
Hardware-Assisted Malware Analysis IC-UNICAMP

58. Introduction Research Final Remarks
Branch Monitoring
Is the solution easy to implement?
Lines of Code comparison
100
1000
10000
100000
1×106
Ether:Comp
Ether:Xen
Ether:Patches
Ether:Patch
Ether:Ctl
MAVMM:Comp
MAVMM:Rep
Bit:Comp
Our:Comp
Our:Cli
Our:Drv
Our:Scr
Lines of Code by solution
Solution
Figure: Lines of Code by solution.
Hardware-Assisted Malware Analysis IC-UNICAMP

59. Introduction Research Final Remarks
Branch Monitoring
Is the solution portable?
Talking about Linux
Listing 10: Linux BTS support
(Source:https://tinyurl.com/mws6v8x
1 s t a t i c i n i t i n t b t s i n i t ( void )
2 bts pmu . c a p a b i l i t i e s = PERF PMU CAP AUX NO SG
| PERF PMU CAP ITRACE
3 bts pmu . t a s k c t x n r = p e r f s w c o n t e x t ;
4 bts pmu . e v e n t i n i t = b t s e v e n t i n i t ;
5 bts pmu . add = b t s e v e n t a d d ;
6 bts pmu . d e l = b t s e v e n t d e l ;
7 bts pmu . s t a r t = b t s e v e n t s t a r t ;
8 bts pmu . stop = b t s e v e n t s t o p ;
9 bts pmu . read = b t s e v e n t r e a d ;
10 r e t u r n p e r f p m u r e g i s t e r (&bts pmu ,
11 ” i n t e l b t s ”,−1)
Hardware-Assisted Malware Analysis IC-UNICAMP

60. Introduction Research Final Remarks
Branch Monitoring
Is the solution portable?
Talking about Linux
Listing 11: Linux BTS support
(Source:https://tinyurl.com/mx759d7)
1 p e r f i n i t (&pe , MMAP PAGES) ;
2 f c n t l ( g b l s t a t u s . fd evt , F SETOWN,
get pid () ) ;
3 monitor loop ( p i d c h i l d , s o u t f i l e ) ;
Hardware-Assisted Malware Analysis IC-UNICAMP

61. Introduction Research Final Remarks
Branch Monitoring
Is solution’s overhead acceptable?
Could the solution run in real-time?
Task Base value
System
monitoring Penalty
Benchmark
monitoring Penalty
Floating-point
operations (op/s) 101530464 99221196 2.27% 97295048 4.17%
Integer operations
(op/s) 285649964 221666796 22.40% 219928736 23.01%
MD5 Hashes
(hash/s) 777633 568486 26.90% 568435 26.90%
RAM transfer
(MB/s) 7633 6628 13.17% 6224 18.46%
HDD transfer
(MB/s) 90 80 11.11% 75 16.67%
Overall (benchm. pt) 518 470 9.27% 439 15.25%
Hardware-Assisted Malware Analysis IC-UNICAMP

62. Introduction Research Final Remarks
Conclusions
Topics
1 Introduction
Malware
Malware Analysis
Anti-Analysis
Transparency
2 Research
A Survey
Branch Monitoring
3 Final Remarks
Conclusions
Future Work
Acknowledgments
Hardware-Assisted Malware Analysis IC-UNICAMP

63. Introduction Research Final Remarks
Conclusions
Lessons learned
Transparency is essential.
Hardware-assisted approaches may fulfill transparency
requirements.
There are open problems on hardware monitoring.
Security, performance, and development efforts as trade-offs
(really?).
Performance monitors as lightweight alternatives.
Hardware-Assisted Malware Analysis IC-UNICAMP

64. Introduction Research Final Remarks
Future Work
Topics
1 Introduction
Malware
Malware Analysis
Anti-Analysis
Transparency
2 Research
A Survey
Branch Monitoring
3 Final Remarks
Conclusions
Future Work
Acknowledgments
Hardware-Assisted Malware Analysis IC-UNICAMP

65. Introduction Research Final Remarks
Future Work
And now?
Multi-core monitor.
Linux Monitor.
Malware clustering.
Hardware-Assisted Malware Analysis IC-UNICAMP

66. Introduction Research Final Remarks
Future Work
Solution’s Availability
Figure: Solution’s Availability. Solution is public on github.
https://github.com/marcusbotacin/BranchMonitoringProject
Hardware-Assisted Malware Analysis IC-UNICAMP

67. Introduction Research Final Remarks
Acknowledgments
Topics
1 Introduction
Malware
Malware Analysis
Anti-Analysis
Transparency
2 Research
A Survey
Branch Monitoring
3 Final Remarks
Conclusions
Future Work
Acknowledgments
Hardware-Assisted Malware Analysis IC-UNICAMP

68. Introduction Research Final Remarks
Acknowledgments
Questions?
Hardware-Assisted Malware Analysis IC-UNICAMP