My talk at the HackInTheBox security conference Amsterdam 2023 about the reverse engineering of AV engines, covering signatures, whitelists, blocklists, kernel drivers, hooking, and much more.

1. Marcus Botacin
Texas A&M University, USA
@MarcusBotacin
All You Always Wanted to Know
About AntiViruses
(and I had to hands-on to tell you!)
HITB 2023, HACK IN THE BOX SECURITY CONFERENCE
Apr 17-21, 2023, AMSTERDAM, NL
1
1

2. Publication
2

3. 0x0. AV products are not the same as AV
engines
3

4. Engine Sharing
4

5. 0x1. AVs have multiple components
5

6. AV Architecture
6

7. 0x2. Whitelists are still widely used
7

8. Whitelisting
8

9. Whitelisting
9

10. Whitelisting
10

11. 0x3. Companies make money selling
whitelisting data
11

12. Selling Whitelists
12

13. Selling Whitelists
13

14. 0x4. Signatures are still widely used
14

15. 15
Signatures in Practice

16. Signature Extraction Algorithm
16

17. The extracted signatures
17

18. Signature Extraction Algorithm in Practice
18

19. Signature Usage: Prevalence
19

20. Signature sizes
20

21. 0x5. (Packed Malware) Detection is also a
cost-benefit trade-off
21

22. Packers
22

23. UPX support
23

24. 0x6. AVs largely rely on userland hooking for
data collection
24

25. Injected Libraries and hooks
Try yourself!
25

26. Libs and Machine Learning: A Discussion
26

27. 0x7. AVs largely rely on kernel driver for
self-protection
27

28. Kernel Filters and Callbacks
28

29. Access Control
29

30. 0x8. AVs “spy” on your network traffic
30

31. Network Process: avp (Kaspersky)
31

32. Network Certificates
32

33. Snort Rules (VIPRE)
33

34. 0x9. AVs store files in quarentines
34

35. Quarantine: Encoded File Information
35

36. Quarantine
36

37. Quarantine: Original File
37

38. Quarantine: Encoded File
38

39. 0xA. AVs collect lots of data from you and
about you
39

40. Databases and Logs
40

41. AV Telemetry
41

42. File Cache
42

43. URL cache
43

44. 0xB. Caches can also be exploited
44

45. Attacks due to caching
45

46. 0xC. AVs updates definitions AND
components
46

47. AV update in a month
47

48. Downloads in plain HTTP
48

49. VPX structure
49

50. For Programmers
50

51. Extracting PE from VPX
51

52. 0xD. AVs security depends on their integrity
52

53. Patching in secure boot mode
53

54. 0xE. AV’s security depends on pristine
installations
54

55. Assume pristine installation
55

56. Pristine installation attempt
56

57. 0xF. Browser extensions are AV clients
57

58. Javascript Injection
58

59. Content Modification
59

60. 0x10. Android AVs are VERY weak
60

61. Android AVs: Static Filtering
61

62. Android AVs: Whitelisting
62

63. Android AVs: Exploiting accessibility services
63

64. Future Directions
64

65. What to do now?
● AV companies must be more transparent about their decisions.
● Researchers have many opportunities to be explored.
● AV evaluations should be multi-dimensional
65

66. What do we use this knowledge for?
66

67. Publications
Terminator: A Secure Coprocessor to Accelerate Real-Time AntiViruses
Using Inspection Breakpoints
67
HEAVEN: A Hardware-Enhanced AntiVirus ENgine to accelerate
real-time, signature-based malware detection
``VANILLA'' malware: vanishing antiviruses by interleaving layers and
layers of attacks

68. HITB 2023, HACK IN THE BOX SECURITY CONFERENCE
Apr 17-21, 2023, AMSTERDAM, NL
68
All You Always Wanted to Know About AntiViruses
(and I had to hands-on to tell you!)
Thank you!
Contact: botacin@tamu.edu or @MarcusBotacin
My Website: marcusbotacin.github.io
Marcus Botacin
Texas A&M University, USA
@MarcusBotacin