.lusoftware verification & validation
VVS
Automatically Repairing Web
Application Firewalls based on
Successful SQL Injection Attacks
Dennis Appelt, Annibale Panichella, Lionel Briand
1
Code Injection
Manipulated data structures
Collect and analyze information
Indicator
Employ probabilistic techniques
Manipulate system resources
Subvert access control
Abuse existing functionality
Engage in deceptive interactions 2 %
2 %
3 %
3 %
3 %
4 %
9 %
32 %
42 %
X-Force Threat Intelligence Index 2017
2
https://www.ibm.com/security/xforce/
More than 40% of all
attacks were SQL
injection (SQLi)
Web Application Firewalls (WAFs)
3
Servermalicious
malicious
malicious
legitimate
WAF
WAF Rule Set
4
Rule set of Apache ModSecurity
https://github.com/SpiderLabs/ModSecurity
Misconfigured WAFs
5
BLOCKED
False Positive
ALLOWED
False Negative
Rule Set Customization
6
Customization is error-prone:
•Complex filter rules
•Limited time and resources
•Lack of automated tools
Rule customization is necessary:
•To avoid false positives
•To protect from new threats
Our Approach
Anatomy of SQLi attacks
8
‘ OR“a”=“a”#
Bypassing Attack
<START>
<sq> <wsp> <sqliAttack> <cmt>
<boolAttack>
<opOR> <boolTrueExpr>
OR <bynaryTrue>
<dq> <ch> <dq> <opEq> <dq> <ch> <dq>
“ a ” = “ a ”
<sQuoteContext>
‘ #_
Decomposition Tree
‘
_
OR”a”=“a”
#
S =
{
Attack Slices
Learning Attack Patterns
9
S1 S2 S3 S4 … Sn Outcome
A1 1 1 0 0 … 0 Passed
A2 0 1 0 0 … 0 Blocked
… … … … … … … …
Am 1 1 1 1 … 1 Blocked
Training Set
Sn
PassedBlocked
S4
YesNo
YesNo
YesNo
S3
S1
S2
…
Decision Tree
Learning Attack Patterns
10
S1 S2 S3 S4 … Sn Outcome
A1 1 1 0 0 … 0 Passed
A2 0 1 0 0 … 0 Blocked
… … … … … … … …
Am 1 1 1 1 … 1 Blocked
Sn
PassedBlocked
S4
YesNo
YesNo
YesNo
S3
S1
S2
…
Training Set Decision Tree
Regular Expression
S2 ∧ ¬ Sn ∧ S1
Fixing Vulnerable WAFs
11
SQLi Attacks
Attacks
Decomposition
Machine
Learning (DT)
New Regular
Expressions
Existing
Rule Set
Fixed
Rule Set
# Blocked
Attacks
# Blocked
Legitimate
Request
Multi-Objective Optimization
12
Problem: selecting a subset of the regular expressions produced
by Decision Tree such as to (1) maximizing the recall (blocked
attacks) and (2) minimizing the false positive rate.Recall
False Positive
Pareto
Front
Multi-Objective Genetic Algorithms
13
Evaluation
Selection
Crossover
Mutation
NSGA-II
Initial
Solutions
Multi-Objective Genetic Algorithms
14
R1 R2 R2 R4 … Rk
1 1 0 0 … 0
0 1 1 1 … 1
Initial Solutions
Evaluation
Selection
Crossover
Mutation
NSGA-II
Initial
Solutions
Solutions are evaluated
and selected according
to the Pareto Optimality
Empirical Evaluation
Research Questions
16
How effective are the found regular expressions in
identifying bypassing attacks?RQ1
To which extent do the found regular expressions
misclassify legitimate traffic as attacks?RQ2
How does NSGA-II compare to random search?RQ3
Case Studies
17
Study 1 (open-source)
1 http://www.cyclos.org
Protected System = Cyclos1
OWASP Core Rule Set =
actively maintained by
security experts
Study 2 (industrial)
Target WAF = Proprietary
SOA system processing
financial transactions
Target WAF = ModSecurity2
2 https://modsecurity.org
Rule set is maintained
by security experts
Case Studies
18
Study 1 (open-source) Study 2 (industrial)
Operations
# Benign
Requests
#Bypassing
Attacks
doPayment
1567
1234
expireTicket 1127
simulatePayment 1265
Operations
# Benign
Requests
# Bypassing
Attacks
Op1
2600
943
Op2 19957
Op3 169
Op4 11462
D. Appelt, C. Nguyen, and L. Briand. “Behind an application
firewall, are we safe from sql injection attacks?’ ICST 2015
Some Results
19
Target WAF:
ModeSecurity
OWASP Core Rule Set
Target Operation:
doPayment()
# Attacks = 1234
# Benign Req = 1567
Hypevolume(NSGAII) >Hypevolume(RS)
Hypervolume Results
20
Hypervolume
0,00
0,25
0,50
0,75
1,00
Op1 Op2 Op3 Op3
NSGA-II Random
Hypervolume
0,00
0,25
0,50
0,75
1,00
doPayment expireTicket simulate-

Payment
NSGA-II Random
ModSecurity Industrial WAF
Testing Against Unseen Requests
21
Proprietary WAF
Regexes with the
lowest false
positive rate
1 http://sqlmap.org/
New Requests
575 Benign requests (fun. tests)
222 New SQLi Attacks (SqlMap1)
Results
False posit. = 0.86% - 1.92%
Recall = 91.50% - 100%
Summary
22
Automatically Repairing Web Application Firewalls based on Successful SQL Injection Attacks

Automatically Repairing Web Application Firewalls based on Successful SQL Injection Attacks

  • 1.
    .lusoftware verification &validation VVS Automatically Repairing Web Application Firewalls based on Successful SQL Injection Attacks Dennis Appelt, Annibale Panichella, Lionel Briand 1
  • 2.
    Code Injection Manipulated datastructures Collect and analyze information Indicator Employ probabilistic techniques Manipulate system resources Subvert access control Abuse existing functionality Engage in deceptive interactions 2 % 2 % 3 % 3 % 3 % 4 % 9 % 32 % 42 % X-Force Threat Intelligence Index 2017 2 https://www.ibm.com/security/xforce/ More than 40% of all attacks were SQL injection (SQLi)
  • 3.
    Web Application Firewalls(WAFs) 3 Servermalicious malicious malicious legitimate WAF
  • 4.
    WAF Rule Set 4 Ruleset of Apache ModSecurity https://github.com/SpiderLabs/ModSecurity
  • 5.
  • 6.
    Rule Set Customization 6 Customizationis error-prone: •Complex filter rules •Limited time and resources •Lack of automated tools Rule customization is necessary: •To avoid false positives •To protect from new threats
  • 7.
  • 8.
    Anatomy of SQLiattacks 8 ‘ OR“a”=“a”# Bypassing Attack <START> <sq> <wsp> <sqliAttack> <cmt> <boolAttack> <opOR> <boolTrueExpr> OR <bynaryTrue> <dq> <ch> <dq> <opEq> <dq> <ch> <dq> “ a ” = “ a ” <sQuoteContext> ‘ #_ Decomposition Tree ‘ _ OR”a”=“a” # S = { Attack Slices
  • 9.
    Learning Attack Patterns 9 S1S2 S3 S4 … Sn Outcome A1 1 1 0 0 … 0 Passed A2 0 1 0 0 … 0 Blocked … … … … … … … … Am 1 1 1 1 … 1 Blocked Training Set Sn PassedBlocked S4 YesNo YesNo YesNo S3 S1 S2 … Decision Tree
  • 10.
    Learning Attack Patterns 10 S1S2 S3 S4 … Sn Outcome A1 1 1 0 0 … 0 Passed A2 0 1 0 0 … 0 Blocked … … … … … … … … Am 1 1 1 1 … 1 Blocked Sn PassedBlocked S4 YesNo YesNo YesNo S3 S1 S2 … Training Set Decision Tree Regular Expression S2 ∧ ¬ Sn ∧ S1
  • 11.
    Fixing Vulnerable WAFs 11 SQLiAttacks Attacks Decomposition Machine Learning (DT) New Regular Expressions Existing Rule Set Fixed Rule Set # Blocked Attacks # Blocked Legitimate Request
  • 12.
    Multi-Objective Optimization 12 Problem: selectinga subset of the regular expressions produced by Decision Tree such as to (1) maximizing the recall (blocked attacks) and (2) minimizing the false positive rate.Recall False Positive Pareto Front
  • 13.
  • 14.
    Multi-Objective Genetic Algorithms 14 R1R2 R2 R4 … Rk 1 1 0 0 … 0 0 1 1 1 … 1 Initial Solutions Evaluation Selection Crossover Mutation NSGA-II Initial Solutions Solutions are evaluated and selected according to the Pareto Optimality
  • 15.
  • 16.
    Research Questions 16 How effectiveare the found regular expressions in identifying bypassing attacks?RQ1 To which extent do the found regular expressions misclassify legitimate traffic as attacks?RQ2 How does NSGA-II compare to random search?RQ3
  • 17.
    Case Studies 17 Study 1(open-source) 1 http://www.cyclos.org Protected System = Cyclos1 OWASP Core Rule Set = actively maintained by security experts Study 2 (industrial) Target WAF = Proprietary SOA system processing financial transactions Target WAF = ModSecurity2 2 https://modsecurity.org Rule set is maintained by security experts
  • 18.
    Case Studies 18 Study 1(open-source) Study 2 (industrial) Operations # Benign Requests #Bypassing Attacks doPayment 1567 1234 expireTicket 1127 simulatePayment 1265 Operations # Benign Requests # Bypassing Attacks Op1 2600 943 Op2 19957 Op3 169 Op4 11462 D. Appelt, C. Nguyen, and L. Briand. “Behind an application firewall, are we safe from sql injection attacks?’ ICST 2015
  • 19.
    Some Results 19 Target WAF: ModeSecurity OWASPCore Rule Set Target Operation: doPayment() # Attacks = 1234 # Benign Req = 1567 Hypevolume(NSGAII) >Hypevolume(RS)
  • 20.
    Hypervolume Results 20 Hypervolume 0,00 0,25 0,50 0,75 1,00 Op1 Op2Op3 Op3 NSGA-II Random Hypervolume 0,00 0,25 0,50 0,75 1,00 doPayment expireTicket simulate-
 Payment NSGA-II Random ModSecurity Industrial WAF
  • 21.
    Testing Against UnseenRequests 21 Proprietary WAF Regexes with the lowest false positive rate 1 http://sqlmap.org/ New Requests 575 Benign requests (fun. tests) 222 New SQLi Attacks (SqlMap1) Results False posit. = 0.86% - 1.92% Recall = 91.50% - 100%
  • 22.