Slideshow Transcript

1. Slide 1: Intrusion Alert Correlation by Norazah Aziz (GCIA) Cyberspace Security Lab, MIMOS Berhad
2. Slide 2: Outline • Why Correlation? • Correlation Process • Correlation Techniques • Conclusion 2
3. Slide 3: Terminologies • Correlation – The degree to which or more attributes or measurements on the same group of elements show a tendency to vary together. Source: http://dictionary.reference.com/browse/correlation • Event – Low level entity that analyzed by IDS eg: Network packets • Alert – Generated by IDS to notify of the interesting events. • Alert Correlation – Multi step process that receives alerts from one or more intrusion detection systems as input and produces a high-level description of the malicious activity on the network. 3
4. Slide 4: Outline • Why Correlation? • Correlation Process • Correlation Techniques • Conclusion • Q&A 4
5. Slide 5: Correlation can address some if the IDS weaknesses • Alert Flooding – generate a large amount of alerts • Context – not group related alerts • False Alert – generate a false negative and false positive • Scalability – difficult to achieve large-scale deployment 5
6. Slide 6: With correlation.. • Can capture a high level view of the attack activity on the target network without losing security-relevant information. 6
7. Slide 7: Outline • Why Correlation? • Correlation Process • Correlation Techniques • Conclusion • Q&A 7
8. Slide 8: Main correlation process • Alert Collection • Alert Aggregation and Verification • High Level Alert Structures 8
9. Slide 9: Correlation Process Alert Collection Alert Aggregation & Verification Sensor Alerts Normalization Pre-Processing Alert Fusion Alert Verification Sensor Thread Ontology Reconstruction Databases Asset Database Attack Session Reconstruction Impact Multi-Step Focus Intrusion Reports Prioritization Analysis Correlation Recognition High-Level Alert Structures Sources: Kruegel et.al, Intrusion Detection and Correlation Challenges and Solution. 9
10. Slide 10: Attack Scenario Example NIDS2 INTERNET Switch HIDS NIDS1 Portscan Worm exploit Mic.IIS Attacker 10.1.9.2, Linux Failed 67.6.1.0 Running Apache Apache Buffer Overflow Exploit Running Application Local exploit against linuxconf Log IDS (APPL) 10
11. Slide 11: Attack Scenario Alerts ID Signature Start/End Src IPs Dest IPs Sensor Tag 1 Scanning 09:1/13:9 67.6.1.0 10.1.9.2 NIDS2 2 Portscan 09:0/14:1 67.6.1.0 10.1.9.2 NIDS1 3 IIS Exploit 11:5/11:5 20.9.0.1 10.1.9.2, NIDS1 port:80 4 Apache Exploit 19:0/19:0 67.6.1.0 10.1.9.2 NIDS1 port:80 5 Bad Request 19:1/19:1 Localhost, APPL Apache 6 Local Exploit 21:3/21:3 linuxconf HIDS 7 Local Exploit 21:4/21:4 linuxconf HIDS 11
12. Slide 12: Alert Normalization Alert Collection Alert Aggregation & Verification Sensor Alerts Normalization Pre-Processing Alert Fusion Alert Verification Sensor Thread Ontology Reconstruction Databases Asset Database Attack Session Reconstruction Impact Multi-Step Focus Intrusion Reports Prioritization Analysis Correlation Recognition High-Level Alert Structures Sources: Kruegel et.al, Intrusion Detection and Correlation Challenges and Solution. 12
13. Slide 13: Alert Normalization ID Signature Start/End Src IPs Dest IPs Sensor Tag 1 Scanning 09:1/13:9 67.6.1.0 10.1.9.2 NIDS2 2 Portscan 09:0/14:1 67.6.1.0 10.1.9.2 NIDS1 3 IIS Exploit 11:5/11:5 20.9.0.1 10.1.9.2, port:80 NIDS1 4 Apache Exploit 19:0/19:0 67.6.1.0 10.1.9.2 port:80 NIDS1 5 Bad Request 19:1/19:1 Localhost, APPL Apache 6 Local Exploit 21:3/21:3 linuxconf HIDS 7 Local Exploit 21:4/21:4 linuxconf HIDS Standardize the alert messages in different formats using CVE (Common Vulnerabilities and Exposures) ID Signature Start/End Src IPs Dest IPs Sensor Tag 1 Portscan 09:1/13:9 67.6.1.0 10.1.9.2 NIDS2 2 Portscan 09:0/14:1 67.6.1.0 10.1.9.2 NIDS1 13
14. Slide 14: Alert Preprocessing Alert Collection Alert Aggregation & Verification Sensor Alerts Normalization Pre-Processing Alert Fusion Alert Verification Sensor Thread Ontology Reconstruction Databases Asset Database Attack Session Reconstruction Impact Multi-Step Focus Intrusion Reports Prioritization Analysis Correlation Recognition High-Level Alert Structures Sources: Kruegel et.al, Intrusion Detection and Correlation Challenges and Solution. 14
15. Slide 15: Alert Preprocessing ID Signature Start/End Src IPs Dest IPs Sensor Tag 1 Portscan 09:1/13:9 67.6.1.0 10.1.9.2 NIDS2 2 Portscan 09:0/14:1 67.6.1.0 10.1.9.2 NIDS1 3 IIS Exploit 11:5/11:5 20.9.0.1 10.1.9.2, port:80 NIDS1 4 Apache Exploit 19:0/19:0 67.6.1.0 10.1.9.2 port:80 NIDS1 5 Bad Request 19:1/19:1 Localhost, Apache APPL 6 Local Exploit 21:3/21:3 linuxconf HIDS 7 Local Exploit 21:4/21:4 linuxconf HIDS Supply best-effort values information by determine the alert’s source and target ID Signature Start/End Src IPs Dest IPs Sensor Tag Bad Request 19:1/19:1 10.1.9.2 10.1.9.2, Apache APPL 5 Local Exploit 21:3/21:3 10.1.9.2 10.1.9.2, linuxconf HIDS 6 7 Local Exploit 21:4/21:4 10.1.9.2 10.1.9.2, linuxconf HIDS 15
16. Slide 16: Alert Fusion Alert Collection Alert Aggregation & Verification Sensor Alerts Normalization Pre-Processing Alert Fusion Alert Verification Sensor Thread Ontology Reconstruction Databases Asset Database Attack Session Reconstruction Impact Multi-Step Focus Intrusion Reports Prioritization Analysis Correlation Recognition High-Level Alert Structures Sources: Kruegel et.al, Intrusion Detection and Correlation Challenges and Solution. 16
17. Slide 17: Alert Fusion ID Signature Start/End Src IPs Dest IPs Sensor Tag 1 Portscan 09:1/13:9 67.6.1.0 10.1.9.2 NIDS2 2 Portscan 09:0/14:1 67.6.1.0 10.1.9.2 NIDS1 3 IIS Exploit 11:5/11:5 20.9.0.1 10.1.9.2, port:80 NIDS1 4 Apache Exploit 19:0/19:0 67.6.1.0 10.1.9.2 port:80 NIDS1 5 Bad Request 19:1/19:1 10.1.9.2 10.1.9.2, Apache APPL 6 Local Exploit 21:3/21:3 10.1.9.2 10.1.9.2, linuxconf HIDS 7 Local Exploit 21:4/21:4 10.1.9.2 10.1.9.2, linuxconf HIDS Identifies alerts that refer to the same underlying event ID Signature Start/End Src IPs Dest IPs Sensor Tag 7 Local Exploit 21:4/21:4 10.1.9.2 10.1.9.2, linuxconf HIDS Meta-Alert- 09:0/13:9 67.6.1.0 10.1.9.2 {NIDS1,NIDS2} 8 {1,2} 17
18. Slide 18: Alert Verification Alert Collection Alert Aggregation & Verification Sensor Alerts Normalization Pre-Processing Alert Fusion Alert Verification Sensor Thread Ontology Reconstruction Databases Asset Database Attack Session Reconstruction Impact Multi-Step Focus Intrusion Reports Prioritization Analysis Correlation Recognition High-Level Alert Structures Sources: Kruegel et.al, Intrusion Detection and Correlation Challenges and Solution. 18
19. Slide 19: Alert Verification ID Signature Start/End Src IPs Dest IPs Sensor Tag 1 Portscan 09:1/13:9 67.6.1.0 10.1.9.2 NIDS2 2 Portscan 09:0/14:1 67.6.1.0 10.1.9.2 NIDS1 3 IIS Exploit 11:5/11:5 20.9.0.1 10.1.9.2, port:80 NIDS1 4 Apache Exploit 19:0/19:0 67.6.1.0 10.1.9.2 port:80 NIDS1 5 Bad Request 19:1/19:1 10.1.9.2 10.1.9.2, Apache APPL 6 Local Exploit 21:3/21:3 10.1.9.2 10.1.9.2, linuxconf HIDS 7 Local Exploit 21:4/21:4 10.1.9.2 10.1.9.2, linuxconf HIDS 8 Meta-Alert- 09:0/13:9 67.6.1.0 10.1.9.2 {NIDS1,NIDS2} {1,2} Remove or ignore the irrelevant alerts, e.g. unsuccessful alert ID Signature Start/End Src IPs Dest IPs Sensor Tag 3 IIS Exploit 11:5/11:5 20.9.0.1 10.1.9.2, port:80 NIDS1 Irrelevant 4 Apache Exploit 19:0/19:0 67.6.1.0 10.1.9.2 port:80 NIDS1 19
20. Slide 20: Attack Thread Reconstruction Alert Collection Alert Aggregation & Verification Sensor Alerts Normalization Pre-Processing Alert Fusion Alert Verification Sensor Thread Ontology Reconstruction Databases Asset Database Attack Session Reconstruction Impact Multi-Step Focus Intrusion Reports Prioritization Analysis Correlation Recognition High-Level Alert Structures Sources: Kruegel et.al, Intrusion Detection and Correlation Challenges and Solution. 20
21. Slide 21: Thread Reconstruction ID Signature Start/End Src IPs Dest IPs Sensor Tag 1 Portscan 09:1/13:9 67.6.1.0 10.1.9.2 NIDS2 2 Portscan 09:0/14:1 67.6.1.0 10.1.9.2 NIDS1 3 IIS Exploit 11:5/11:5 20.9.0.1 10.1.9.2, port:80 NIDS1 irrelevant 4 Apache Exploit 19:0/19:0 67.6.1.0 10.1.9.2, port:80 NIDS1 correlated 5 Bad Request 19:1/19:1 10.1.9.2 10.1.9.2, Apache APPL 6 Local Exploit 21:3/21:3 10.1.9.2 10.1.9.2, linuxconf HIDS correlated 7 Local Exploit 21:4/21:4 10.1.9.2 10.1.9.2, linuxconf HIDS correlated 8 Meta-Alert 09:0/13:9 67.6.1.0 10.1.9.2 {NIDS1,NIDS2} {1,2} correlated 9 Meta-Alert 09:0/19:0 67.6.1.0 10.1.9.2, port:80 {NIDS1,NIDS2} {4,8} 10 Meta-Alert 21:3/21:4 10.1.9.2 10.1.9.2, linuxconf HIDS {6,7} Combines a series of alerts that refer to attacks launched by one attacker against a single target. 21
22. Slide 22: Attack Session Reconstruction Alert Collection Alert Aggregation & Verification Sensor Alerts Normalization Pre-Processing Alert Fusion Alert Verification Sensor Thread Ontology Reconstruction Databases Asset Database Attack Session Reconstruction Impact Multi-Step Focus Intrusion Reports Prioritization Analysis Correlation Recognition High-Level Alert Structures Sources: Kruegel et.al, Intrusion Detection and Correlation Challenges and Solution. 22
23. Slide 23: Session Reconstruction ID Signature Start/End Src IPs Dest IPs Sensor Tag 1 Portscan 09:1/13:9 67.6.1.0 10.1.9.2 NIDS2 2 Portscan 09:0/14:1 67.6.1.0 10.1.9.2 NIDS1 3 IIS Exploit 11:5/11:5 20.9.0.1 10.1.9.2, port:80 NIDS1 irrelevant 4 Apache Exploit 19:0/19:0 67.6.1.0 10.1.9.2, port:80 NIDS1 correlated 5 Bad Request 19:1/19:1 10.1.9.2 10.1.9.2, Apache APPL 6 Local Exploit 21:3/21:3 10.1.9.2 10.1.9.2, linuxconf HIDS correlated 7 Local Exploit 21:4/21:4 10.1.9.2 10.1.9.2, linuxconf HIDS correlated 8 Meta-Alert 09:0/13:9 67.6.1.0 10.1.9.2 {NIDS1,NIDS2} {1,2} correlated 9 Meta-Alert 09:0/19:0 67.6.1.0 10.1.9.2, port:80 {NIDS1,NIDS2} {4,8} 10 Meta-Alert 21:3/21:4 10.1.9.2 10.1.9.2, linuxconf HIDS {6,7} 11 Meta-Alert 09:0/19:1 {67.6.1.0 10.1.9.2, port:80, {NIDS1,NIDS2, {5,9} ,10.1.9.2} Apache APPL} Linking network-based to host-based alerts 23
24. Slide 24: Attack Focus Recognition Alert Collection Alert Aggregation & Verification Sensor Alerts Normalization Pre-Processing Alert Fusion Alert Verification Sensor Thread Ontology Reconstruction Databases Asset Database Attack Session Reconstruction Impact Multi-Step Focus Intrusion Reports Prioritization Analysis Correlation Recognition High-Level Alert Structures Sources: Kruegel et.al, Intrusion Detection and Correlation Challenges and Solution. 24
25. Slide 25: Attack Focus Recognition • To identify hosts that are either source or the target of a substantial amount of attacks. • Effective in reducing the number of alerts caused by DDoS and portscan activity. • Example: Several portscan alerts against multiple targets from the same source, alerts would be merged into a one2many meta-alert. 25
26. Slide 26: Multi Step Correlation Alert Collection Alert Aggregation & Verification Sensor Alerts Normalization Pre-Processing Alert Fusion Alert Verification Sensor Thread Ontology Reconstruction Databases Asset Database Attack Session Reconstruction Impact Multi-Step Focus Intrusion Reports Prioritization Analysis Correlation Recognition High-Level Alert Structures Sources: Kruegel et.al, Intrusion Detection and Correlation Challenges and Solution. 26
27. Slide 27: Multistep Correlation ID Signature Start/End Src IPs Dest IPs Sensor Tag 1 Portscan 09:1/13:9 67.6.1.0 10.1.9.2 NIDS2 2 Portscan 09:0/14:1 67.6.1.0 10.1.9.2 NIDS1 3 IIS Exploit 11:5/11:5 20.9.0.1 10.1.9.2, port:80 NIDS1 irrelevant 4 Apache Exploit 19:0/19:0 67.6.1.0 10.1.9.2, port:80 NIDS1 correlated 5 Bad Request 19:1/19:1 10.1.9.2 10.1.9.2, Apache APPL 6 Local Exploit 21:3/21:3 10.1.9.2 10.1.9.2, linuxconf HIDS correlated 7 Local Exploit 21:4/21:4 10.1.9.2 10.1.9.2, linuxconf HIDS correlated 8 Meta-Alert 09:0/13:9 67.6.1.0 10.1.9.2 {NIDS1,NIDS2} {1,2} correlated 9 Meta-Alert 09:0/19:0 67.6.1.0 10.1.9.2, port:80 {NIDS1,NIDS2} {4,8} 10 Meta-Alert 21:3/21:4 10.1.9.2 10.1.9.2, linuxconf HIDS {6,7} correlated 11 Meta-Alert 09:0/19:1 {67.6.1.0 10.1.9.2, port:80, {NIDS1,NIDS2,A {5,9} ,10.1.9.2} Apache PPL} correlated 12 Meta-Alert 09:0/21:4 {67.6.1.0 10.1.9.2, port:80, {NIDS1,NIDS2,A {10,11} ,10.1.9.2} Apache, linuxconf PPL,HIDS} Defined by using some form of expert knowledge (attack scenario) 27
28. Slide 28: Impact Analysis Alert Collection Alert Aggregation & Verification Sensor Alerts Normalization Pre-Processing Alert Fusion Alert Verification Sensor Thread Ontology Reconstruction Databases Asset Database Attack Session Reconstruction Impact Multi-Step Focus Intrusion Reports Prioritization Analysis Correlation Recognition High-Level Alert Structures Sources: Kruegel et.al, Intrusion Detection and Correlation Challenges and Solution. 28
29. Slide 29: Impact Analysis • To analyze alerts with regard to their impact on the network infrastructure and the attached resources. • Example: – Dependency between mail services that requires an operational domain name service (DNS) to work properly. 29
30. Slide 30: Alert Prioritization Alert Collection Alert Aggregation & Verification Sensor Alerts Normalization Pre-Processing Alert Fusion Alert Verification Sensor Thread Ontology Reconstruction Databases Asset Database Attack Session Reconstruction Impact Multi-Step Focus Intrusion Reports Prioritization Analysis Correlation Recognition High-Level Alert Structures Sources: Kruegel et.al, Intrusion Detection and Correlation Challenges and Solution. 30
31. Slide 31: Prioritization ID Signature Description Priority Tag Ref Tag 1 Portscan LOW correlated 2 Portscan LOW correlated 3 IIS Exploit LOW Irrelevant 4 Apache Exploit LOW correlated 5 Bad Request LOW correlated 6 Local Exploit LOW correlated 7 Local Exploit LOW correlated 8 Meta-Alert Fused Portscan LOW correlated {2,3} 9 Meta-Alert Remote Attack Thread LOW correlated {8,4} 10 Meta-Alert Local Attack Thread LOW correlated {6,7} 11 Meta-Alert Remote Attack Session LOW correlated {9,5} 12 Meta-Alert Multistep Attack Scenario HIGH {11,10} To define appropriate priorities related to the attack’s impact and the effect of a response by classify alerts (high, medium or low) 31
32. Slide 32: Outline • Why Correlation? • Correlation Process • Correlation Techniques • Conclusion • Q&A 32
33. Slide 33: Correlation Techniques • Similar Attack Attributes • Pre-defined Attack Scenarios • Pre & Post Condition of Individual Attack • Statistical Causality Analysis 33
34. Slide 34: Technique 1: Similar Attack Attributes • Alerts belonging to the same attack often have similar attributes • To recognize correlation – inspect alert attributes and finding similarities between them • Well-founded list of alert attributes such as sensor, alert thread, source and destination IPs, ports and time. • Emerald system is an example of such implementation 34
35. Slide 35: Technique 2: Pre-defined Attack Scenarios • Process of filling in attack scenarios templates and formulate them. • Attack scenarios are broken down to explicit correlation rules. • Restricted to known attacks and misuse detection only • Specified by human users or learned through training datasets • Example of implementation : Attack Specification Language (ASL) 35
36. Slide 36: Technique 3: Pre & Post Condition of Individual Attack • Powerful mechanism of expressing correlation criteria • Based on the preconditions and consequences of individual attacks; correlates alerts if the pre- condition of some later alerts are satisfied by the consequences of some earlier alerts. • Uncover the causal relationship between alerts, and is not restricted to known attack scenarios. 36
37. Slide 37: Technique 4: Statistical Causality Analysis • Misuse detection toward anomaly detection • Not feasible solution for the complete correlation process • But it can be utilized as a part of a larger system to pre-process alerts or to provide meta-alert signatures. • Example of implementation: Granger Causality Test (GCT) 37
38. Slide 38: Outline • Why Correlation? • Correlation Process • Correlation Techniques • Conclusion • Q&A 38
39. Slide 39: Conclusion • IDS is still needed and the correlation is the effective means to analyze the alert to high level view picture. • Research on better correlation with detectable unknown attack need to be done. 39
40. Slide 40: References • Christopher Kruegel, Fredrik Valeur, Giovanni Vigna (2005). Intrusion Detection and Correlation, Challenges and Solution, Springer Science+Business Media Inc, USA. • Antti Hatala, et al, “Event Data Exchange and Intrusion Alert Correlation in Heterogeneous Networks”, Proceeding of the 8th Collogquium for Information Systems Security Education West Point, NY, June 2004 • Xinzhou Qin, et al, “Statistical Causality of INFOSEC Alert Data”; in Proceedings of Recent Advances in Intrusion Detection 2003. • Hervé Debar and Andreas Wespi: “Aggregation and correlation of Intrusion- Detection Alerts”; in Proceedings of Recent Advances in Intrusion Detection 2001. • Peng Ning , et al, “Constructing attack scenarios through correlation of intrusion alerts”; in Proceedings of the 9th ACM conference on Computer and Communications security; 2002. • Hervé Debar and Andreas Wespi: “Aggregation and correlation of Intrusion- Detection Alerts”; in Proceedings of Recent Advances in Intrusion Detection 2001. 40
41. Slide 41: Thank You 41