The document discusses techniques for capturing network traffic and system logs to detect security incidents in large networks. It describes how to capture traffic using libpcap, nfqueue, and DAQ. It also discusses challenges like fragmentation and the need to decode protocols. For logs, it highlights weaknesses like signature-based detection and the importance of normalized, unconfigurable logs. It introduces CUDA and NetGPU for GPU-accelerated traffic processing and visualization tools like SecViz and Circos for analyzing large datasets. The conclusion emphasizes that visualization can help solve the problem of events getting lost in noise and overcome technical limitations of current detection approaches.

1. Capture me if you can!
Sebastien Tricaud1
1 Picviz Labs
Hackito Ergu Sum (Paris, France) 2011
1/54

2. $ whoami
• Sebastien Tricaud
• Picviz Labs Director
• Picviz Labs is the editor of Picviz Inspector, a data-mining
software for security
• Honeynet Project CTO
• 15 years of various IDS implementations
2/54

3. 1 Introduction
2 Network Capture
3 Logs Capture
4 CUDA
5 Visualization
6 Conclusion
3/54

4. Context
Once upon a time. . .
4/54

5. Context
Once upon a time. . .
Two days ago, at CERIAS, M. Neal Ziring said:
The attack data is often lost in the noise of events
4/54

6. Context
Mr. Neal Ziring is currently a technical director in the
Information Assurance Directorate (IAD), at NSA. The IAD
provides cryptographic, network, and operational security
products and services to protect and defend national security
systems.
5/54

7. Talk objective
How capture can be performed and managed to effectively find
incidents1 in large networks.
1
attacks, documents leaks, etc.
6/54

8. Find incidents in large networks: Network traffic
1 Capture all the traffic
2 Someone reports an incident
3 Run Snort on the captured traffic
7/54

9. Find incidents in large networks: Network traffic
1 Capture all the traffic
2 Someone reports an incident
3 Run Snort on the captured traffic
• Two countries examples:
• 30 Gb Netflow Traffic for a 20 millions people country per
24 hours (about 1700 events/s; 510 000 events/5 mn)
7/54

10. Find incidents in large networks: Network traffic
1 Capture all the traffic
2 Someone reports an incident
3 Run Snort on the captured traffic
• Two countries examples:
• 30 Gb Netflow Traffic for a 20 millions people country per
24 hours (about 1700 events/s; 510 000 events/5 mn)
• 5 min Netflow Capture on the main backbone on a 45
millions people country: 3 millions events/5 mn
7/54

11. 1 Introduction
2 Network Capture
3 Logs Capture
4 CUDA
5 Visualization
6 Conclusion
8/54

12. Capture with libpcap
u_char ∗ packet ;
struct timeval packet_tv ;
s t r u c t pcap_pkthdr pheader ;
...
packet = ( u_char ∗ ) pcap_next ( pcaph , &pheader ) ;
while ( packet ) {
p a c k e t _ t v = pheader . t s ;
t = packet_tv . tv_sec ;
s t r t i m e = c t i m e (& t ) ;
i f ( ntohs ( e t h e r −>e t h _ t y p e ) == ETH_TYPE_IP ) {
i p = ( s t r u c t i p _ h d r ∗ ) ( packet + ETH_HDR_LEN ) ;
...
9/54

13. How does libpcap works?
• Layer 2
• Packet copied! (ahah)
• Apply a BPF filter
• Get the data
10/54

14. Netfilter QUEUE (nfqueue)
11/54

15. DAQ
(Awesome) Data Acquisition Library written by Sourcefire.
Available from http://www.snort.org
Unifies:
• AFPacket
• ipqueue
• netfilter_queue
• libpcap
12/54

16. Other ways to capture
• Daemonlogger: relies on libpcap
• Streams2 : relies on libpcap just for BPF
• Various works from Luca Deri with PF_RING
• using GPGPU
2
git clone git://git.carnivore.it/streams.git
13/54

17. Now you (perhaps) got your packet!
The packet is captured, fine! however:
• It can be fragmented
• If you run a signature maching, UTF-8 encoding can
bypass it
• A protocol like RPC need to be decoded
• The attack can be located at different DoD model levels
14/54

18. Fragmentation
Let’s have a look at Linux:
• IPV4: linux-src/net/ipv4/ip_fragment.c
• IPV6: linux-src/net/ipv6/reassembly.c
How it is performed in IPV4:
• Defragmentation happens with the function ip_defrag()
• Called only by:
• ip_local_deliver()
• ip_call_ra_chain: only if the socket is tied to an interface
15/54

19. • Linux does not defragment upon FORWARD
• Netfilter may do it
• modprobe nf_conntrack_ipv4
16/54

20. We captured, we want evils!
Snort gives up several ways to find the evil:
• Binary:
content:"|0A 00 00 01 85 04 00 00
80|root|00|" (sid:1775)
• Simple pattern:
content:"fuck fuck fuck" (sid:1316)
• PCRE:
pcre:"/ˆ x3c(REQIMG|RVWCFG) x3e/ism"
(sid:2460)
Problem: How Snort manages pattern matching algorithms
along with PCRE? Each PCRE is tried on each packet?
17/54

21. snort PCRE lookup
• Long patterns are easier to find
• PCRE and pattern matching within Snort:
• Search for the longest pattern in each signature
• function fpAddLongestContent() in fpcreate.c
• The traffic is prequalifed (MPSE)
• Rules aare sequentially tested
• The PCRE option is ignored until the complete rule test
after the prequalification
• PCRE uses its own DFA/NFA
⇒ Less we have PCRE, better we are.
18/54

22. Netflow
• It is easier to investigate with connection flow
• Looking at TCP SYN is better for understanding than the
whole SYN>SYN-ACK>ACK>PSH>PSH-ACK, etc.
• Streams was designed to help you there
19/54

23. 1 Introduction
2 Network Capture
3 Logs Capture
4 CUDA
5 Visualization
6 Conclusion
20/54

24. Logs
Logs highly used for forensic activity for cybercrime
investigation
21/54

25. Logs
Logs highly used for forensic activity for cybercrime
investigation
Question: who cares about logs? their weakness,
normalization, etc.?
21/54

26. SSH defaults accounts testing
sshd [ 6 5 7 4 ] : error : PAM: Authentication failure for r o o t from 1 9 2 . 1 6 8 . 1 2 . 2
sshd [ 6 5 7 4 ] : error : PAM: Authentication failure for guest from 1 9 2 . 1 6 8 . 1 2 . 2
sshd [ 6 5 7 4 ] : error : PAM: Authentication failure for p r i n t e r from 1 9 2 . 1 6 8 . 1 2 . 2
sshd [ 6 5 7 4 ] : error : PAM: Authentication failure for l p from 1 9 2 . 1 6 8 . 1 2 . 2
sshd [ 6 5 7 4 ] : error : PAM: Authentication failure for admin from 1 9 2 . 1 6 8 . 1 2 . 2
22/54

27. Detection dilemna
1 Detecting
• A user enumeration is more likely to get caught and
correlated
• Use tools like OSSEC and get it right in your mailbox
• OSSEC and any other tools like that need logs to analyze
and detect things
2 Log analyzers common weaknesses
• Signature based
• PCRE based (with PCRE weaknesses as well, but this is
for an other talk)
• Needs food == Needs logs
23/54

28. Know Your Enemy
Log analyzer enemy == Configurable log
24/54

29. Squid
Log Format configuration
l o g f o r m a t s q u i d %t s .%03 t u %6 t r %>a %Ss/%03>Hs %<s t %rm %r u %un %Sh/%<A %mt
Log Format options
...
[ h t t p : : ] rm Request method (GET/POST e t c )
[ h t t p : : ] ru Request URL
[ h t t p : : ] rp Request URL−Path e x c l u d i n g hostname
...
25/54

30. ProFTPd
Log with mod_log
Log Format configuration
LogFormat d e f a u l t "%h %l %u %t "% r " %s %b "
Log Format options
%A − Anonymous username ( password g i v e n )
%a − Remote c l i e n t IP address
%b − Bytes s e n t f o r r e q u e s t
26/54

31. Apache
Log with mod_log
Log Format configuration
LogFormat "%h %l %u %t "% r " %>s %b " % { R e f e r e r } i " " % { User−Agent } i " " combined
Cool options!
• %b did you see this %b?
• %b: Size of response in bytes, excluding HTTP headers.
In CLF format, i.e. a ’-’ rather than a 0 when no bytes are
sent.
• It is possible to exploit this weakness
27/54

32. Log misuse 0-day
A log misuse 0-day is:
• an application fails to properly log an information it could
• log injection
• incorrect logged information
There is NO log misuse 0-day database!
28/54

33. Simple Log misuse 0-day
Back on ProFTPd, remember:
Log Format options
%A − Anonymous username ( password g i v e n )
password given = gets anything
Code managing the password
# d e f i n e PR_TUNABLE_PATH_MAX 1024
char arg [ PR_TUNABLE_PATH_MAX+1] = { ’ 0 ’ } ;
case META_ANON_PASS:
argp = arg ;
pass = p r _ t a b l e _ g e t ( s e s s i o n . notes , " mod_auth . anon−passwd " , NULL ) ;
i f ( ! pass )
pass = "UNKNOWN" ;
s s t r n c p y ( argp , pass , s i z e o f ( arg ) ) ;
→ Remote log injection possible, in /var/log/proftpd/auth.log
29/54

34. Log misuse database
Actually there is CWE. . .
• Common Weakness Enumeration
• CWE-778: Insufficient Logging
"When a security-critical event occurs, the software either
does not record the event or omits important details about
the event when logging it."
30/54

35. CVE examples
• CVE-2003-1566: Microsoft IIS 5.0 does not log requests
that use the TRACK method, which allows remote
attackers to obtain sensitive information without detection.
• CVE-2007-3730: OpenVMS does not log the source IP.
• CVE-2008-1203: Adobe ColdFusion 8 and ColdFusion
MX7 do not log failed connection attempts on the
administrative interface.
• ...
Those CVE are still under review
31/54

36. YASA! (Yet Another Stealth Attack)
Ever seen this attack?
66.249.65.39 - - [28/Mar/2007:03:08:46 +0200] "GET /index.html
HTTP/1.1" 404 394 "-" "Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)"
32/54

37. 1 Introduction
2 Network Capture
3 Logs Capture
4 CUDA
5 Visualization
6 Conclusion
33/54

38. My laptop has a NVIDIA Geforce GT 420M
• 96 CUDA cores
• Memory Bandwidth 25.6 GB/sec
• A Thread block can run up to 512 threads
34/54

39. CUDA architecture
35/54

40. CUDA processing flow
36/54

41. Capture using CUDA: NetGPU
Available from http://code.google.com/p/netgpu
37/54

42. 1 Introduction
2 Network Capture
3 Logs Capture
4 CUDA
5 Visualization
6 Conclusion
38/54

43. Problems with SIEM and Intrusion Detection
• Capture is complex
• Rulesets are required: always after the problem
• Too many false positives
39/54

44. Why Visualization
Handle large data without extracting known events to correlate
yourself.
40/54

45. Secviz
Visualization community website: http://www.secviz.org
41/54

46. Circos
42/54

47. Limitation
Enough with limitations.
43/54

48. How many events are in this picture?
44/54

49. How many events are in this picture?
45/54

50. Discover a successful attack in less than one minute
46/54

51. Discover a successful attack in less than one minute
47/54

52. Discover a successful attack in less than one minute
48/54

53. Discover a successful attack in less than one minute
49/54

54. Discover a successful attack in less than one minute
50/54

55. Discover a successful attack in less than one minute
51/54

56. 1 Introduction
2 Network Capture
3 Logs Capture
4 CUDA
5 Visualization
6 Conclusion
52/54

57. Conclusion
• Data are obviously lost in the noise of events today
• If we are creative, we may be able to solve this issue
• We have some technical limitations, we need to find ways
to get around them
53/54

58. Conclusion
• Data are obviously lost in the noise of events today
• If we are creative, we may be able to solve this issue
• We have some technical limitations, we need to find ways
to get around them
• We have some technical solutions (hint: SIEM), we need to
find ways to get around them
• I strongly believe visualization has a great role to play in it
53/54

59. Questions?
• Email: stricaud@picviz.com
• Company website: http://www.picviz.com
• Twitter: @tricaud
• Blog: http://logviz.blogger.com
54/54