Though publicly known for a long time, SQL injection attacks do not yet seem to have reached their peak – the LulzSec activities in mid 2011 showed the overall presence of applications vulnerable to SQL attacks. Organizations like OWASP and Mitre rank SQL injections as the most dangerous threats to our (web) infrastructures and even SQL injections in SMS text messages have been reported. Vendors of Web Application Firewalls spend enormous e?orts to create patterns to detect SQL injections at the application protocol layer, but attackers spend even more e?orts ?nding evasions of these patterns using various encodings or polymorphic substitutions within SQL. In this talk we will have a look at SQL injections from the syntax level perspective of the SQL language. We exploit the parser component of the database system to produce a syntax tree of the command that has been passed to the database by the web frontend. The resulting tree provides a representation of the command that can be compared to a set of known commands expected to be used by the deployed web application.
Bio: Starting with Linux/network security in 1996, Christian Bockermann has been working in computer security for over 10 years. While working as a Java web-application developer for several years he started concentrating on web-security as primary subject. Since he graduated with a MSc in computer science with an emphasis on Anomaly Detection in Web-Applications, he is currently working on his Ph.D. combining methods of machine learning and artificial intelligence in web-application firewalls and system monitoring. A proposal of his intelligent web-application firewall project has been elected among the top-10 projects of the 2nd GermanIT-Security Award. Alongside to this Ph.D. research, Christian is working as a freelancer in web-security consulting, mostly focused on Apache and ModSecurity. He is also author of several Java tools supplementary to ModSecurity, most prominent being the AuditConsole log-management server for ModSecurity.
The Codex of Business Writing Software for Real-World Solutions 2.pptx
hashdays 2011: Christian Bockermann - Protecting Databases with Trees
1. Protecting Databases
with Trees
A syntax-based approach to detect SQL injections
Christian Bockermann - chris @ jwall.org
2. About Me
Researcher of the Artificial Intelligence Group
at the University of Dortmund, Germany
Computer Science
Department
Studying machine learning methods Artificial Intelligence Group
for web-security
Developer of several projects supplementory www.jwall.org
to ModSecurity @jwallorg
AuditViewer, AuditConsole
Web Policy Compiler, Web Application Profiler
jwall-tools
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
3. Outline
Are SQL injections still a threat?
Where to fight SQL injections?
Protecting Databases with Trees
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
4. We start like every
SQL injection talk...
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
6. $name = $_POST[‘name‘];
// $name = “Robert‘); DROP TABLE Students; --“
$insert = “INSERT INTO STUDENTS VALUES (‘$name‘);“;
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
7. $name = $_POST[‘name‘];
// $name = “Robert‘); DROP TABLE Students; --“
$insert = “INSERT INTO STUDENTS VALUES (‘$name‘);“;
INSERT INTO STUDENTS VALUES (‘Robert‘); DROP TABLE
Students; -- ‘);
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
8. It‘s over 5 years old!
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
9. The Web Hacking Incident
Database, June 2011
New Sony Hack Claims Over a
Million User Passwords
2011-06-02 FBI Partner Organization Website
Hacked
Sony Europe hacked by Lebanese 2011-06-06
hacker... Again
2011-06-04 Hacker breaks into MIT website
2011-06-08
LulzSec has compromised
SonyPictures.RU
Citigroup Card Customers’ Data
2011-06-05
Hacked
2011-06-09
DDoS attack takes down
Atlassian's SaaS platform Sony Portugal latest to fall to
2011-06-06 hackers
2011-06-09
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
10. The Web Hacking Incident
Database, June 2011
New Sony Hack Claims Over an
Million User Passwordsnje ctio
S QL I FBI Partner Organization Website
2011-06-02
tion
Hacked Injec
SQL 2011-06-06
Sony Europe hacked by Lebanese
hacker... Again tion
Injec2011-06-04 Hacker breaks into MIT website
SQL
tion
njec
QL I 2011-06-08
LulzSec has compromised S
SonyPictures.RU tion
Injec2011-06-05 Citigroup Card Customers’ Data
SQL Hacked
2011-06-09
DDoS attack takes down
Atlassian's SaaS platform Sony Portugal latest to fall to
2011-06-06 hackers
tion
Injec2011-06-09
SQL
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
11. The Web Hacking Incident
Database, June 2011
SQL Injection
dos/ddos
other
Web Hacking Incident Database, June 2011
http://projects.webappsec.org/
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
12. Imperva‘s Trend Report #4
Anatomy of a SQL attack
Monitoring 30 web applications:
on average 71 SQL injection attempts per hour
800-1300 injection attempts at peak times
Use of highly automated SQL injection tools,
e.g. sqlmap, Havij,...
Imperva Monthly Trend Report #4, September 2011
http://www.imperva.com/download.asp?id=352
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
13. Imperva‘s recent
Hack-Forum Analysis
spam
SQL Injection
dos/ddos
zero-day
shell code
brute force
Imperva Monthly Trend Report #5, October 2011
HTML injection http://www.imperva.com/download.asp?id=327
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
14. Just a couple of days ago...
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
15. Motivation - Top 10 Attacks
The Open Web Application Security Project lists the
OWASP - Open Web
Top-10 vulnerabilities: Application Security
Project
1.Injection Flaws (SQL-Injection, RFI, ...) http://www.owasp.org/
2.Cross Site Scripting (XSS)
3.Broken Authentication / Session Management
4.Insecure Direct Object Reference
5.Cross Site Request Forgery (CSRF)
6.Security Misconfiguration
7.Malicious File Execution (Remote File Inclusion)
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
16. Mitre Top-25
Improper Neutralization of SQL Elements SQL Injection
Improper Neutralization of OS commands Command Injection
Buffer copy without size-check Buffer overflow
Improper Neutralization of Input during page generation Cross-Site Scripting
Missing Authentication of critical functions
Missing Authorization
Use of hard-coded credentials
Missing Encryption of sensitive data
Unrestricted file uploads of dangerous file-types
...
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
17. Not only web...
A SQL injection vulnerability in
Symantec's Sygate Management
Server (SMS) version 4.1, build
1417 and earlier could potentially
allow a remote or local attacker to
gain administrative privileges to the
SMS server.
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
18. What makes a SQL injection?
/query?search=security
Web Server
Web Application
SELECT title,abstract FROM DOCS
WHERE
txt LIKE ‘%security%‘
database
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
19. What makes a SQL injection?
1. Attacker injects SQL code into the application
2. injection alters the statement that is executed
/query?search=security
Web Server
Web Application
SELECT title,abstract FROM DOCS
WHERE
txt LIKE ‘%security%‘
database
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
20. What makes a SQL injection?
1. Attacker injects SQL code into the application
2. injection alters the statement that is executed
/query?search=`+UNION+SELECT+LOGIN,PASSWORD+FROM
+USERS; --
Web Server
SELECT title,abstract FROM DOCS
Web Application WHERE
txt LIKE ‘%‘
UNION
SELECT LOGIN,PASS FROM USERS;
--%`;
database
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
21. Where to fight SQL injections?
/query?search=security
Web Server
Web Application
database
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
22. Where to fight SQL injections?
/query?search=security
Web Server
Web Application Within the app‘s code
database
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
23. Where to fight SQL injections?
/query?search=security
Within the HTTP traffic
Web Server
Web Application Within the app‘s code
database
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
24. Where to fight SQL injections?
/query?search=security
Within the HTTP traffic
Web Server
Web Application Within the app‘s code
Within the executed SQL
database
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
25. Fighting SQL injections (code)
Conceptual approaches to elevate OWASP Guides
the security of web applications Code Review Guide
Eoin Keary et.al.
Testing Guide
Specifications, Developer Trainings Matteo Meucci et.al.
Backend Security Project
Carlo Pelliccioni et.al.
Penetration testing, Code Reviews AppSensors Project
Michael Coates, Colin Watson
et.al.
Risk Management
Risk Management
Strategies
STRIDE / DREAD
J.D. Meier et. al. Microsoft 2005
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
26. Fighting SQL injections (code)
Prepared statements can help a lot
PreparedStatement p =
con.prepareStatement(
“SELECT * FROM USERS WHERE login = ?“ );
p.setParameter( 0, username );
p.executeQuery();
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
27. Fighting SQL injections (code)
PreparedStatement p =
con.prepareStatement(
“SELECT * FROM USERS WHERE login = “ + username );
p.executeQuery();
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
28. Fighting SQL injections (code)
PreparedStatement p =
con.prepareStatement(
“SELECT * FROM USERS WHERE login = “ + username );
p.executeQuery();
Proper use of prepared statements required!
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
29. Yes, the whole intrusion
detection (and prevention ...)
game is ‚just‘ a big attempt to
‚patch‘ bugged systems...
Damiano Bolzoni, focus-ids mailing list 10/2008
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
30. Fighting SQL injections (waf/ids)
External approaches to web security
Intrusion Detection Systems
PHPIDS, Snort
Web Application Firewalls
Web Server
ModSecurity,...
Web Application
database
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
31. Fighting SQL injections (waf/ids)
WAF / IDS usually check for SQL attack patterns in
HTTP requests, i.e. the user input
/query?search=`+UNION+SELECT+NAME,PASSWORD+FROM+USERS; --
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
32. Fighting SQL injections (waf/ids)
WAF / IDS usually check for SQL attack patterns in
HTTP requests, i.e. the user input
/query?search=`+UNION+SELECT+NAME,PASSWORD+FROM+USERS; --
UNION
Any SQL keywords SELECT
DROP TABLE
contained?? INSERT
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
33. Fighting SQL injections (waf/ids)
Almost(?) all WAF/IDS approaches follow this black-
listing or pattern based approach
The ModSecurity Core-Rules
IBM Web Application Firewall
Imperva SecureSphere
AQTRONIX Webknight
PHPIDS
Snort
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
34. Fighting SQL injections (waf/ids)
It‘s not just keywords, it‘s mostly regular expressions
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
36. Fighting SQL injections (waf/ids)
How do keyword/regex filters work with text?
„The following Cookie triggers [rule] 981248
LtpaToken2=x5Orq
(it didn't like "50r"?????)“
ModSecurity Core Rules Mailing list, 2.9.2011
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
37. Evading Pattern Detection
Some WAFs try to remove keywords from inputs
Replacements can easily be evaded:
id=1+UNunionION+SEselectLECT+1,2,3--
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
38. Evading Pattern Detection
Some WAFs try to remove keywords from inputs
Replacements can easily be evaded:
id=1+UNunionION+SEselectLECT+1,2,3--
This would likely result in a database error
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
39. Evading Pattern Detection
Some WAFs try to remove keywords from inputs
Replacements can easily be evaded:
id=1+UNunionION+SEselectLECT+1,2,3--
id=1+UNION+SELECT+1,2,3--
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
40. Evading Pattern Detection
Some WAFs try to remove keywords from inputs
Replacements can easily be evaded:
id=1+UNunionION+SEselectLECT+1,2,3--
id=1+UNION+SELECT+1,2,3--
This looks like a good candidate for a successful
SQL injection
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
41. Evading Pattern Detection
There are many approaches to evade
pattern detection:
SQLi filter evasion and
by encoding obfuscation
Johannes Dahse, RUB
at Conference Confidence 2.0
by obfuscation
Beyond SQLi: Obfuscate and
Bypass
by HTTP parameter pollution By CWH Underground
www.exploit-db.com/papers/17934
Bypassing PHPIDS 0.6.5
Michael Brooks (https://sitewat.ch)
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
42. Evading Pattern Detection
Some WAFs do only a single decoding before filtering
double encoding your attack will bypass the WAF
http://victim.com/news.php?id=1%252f
%252a*/union%252f%252a*/select%252f
%252a*/1,2,3%252f%252a*/from%252f
%252a*/users--
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
43. Evading Pattern Detection
Some WAFs do only a single decoding before filtering
double encoding your attack will bypass the WAF
http://victim.com/news.php?id=1%2f
%2a*/union%2f%2a*/select%2f%2a*/
1,2,3%2f%2a*/from%2f%2a*/users--
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
44. Evading Pattern Detection
Some WAFs do only a single decoding before filtering
double encoding your attack will bypass the WAF
http://victim.com/news.php?id=1/**/
union/**/select/**/1,2,3/**/from/**/
users--
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
45. Evading Pattern Detection
SQL injections possible in a lot of places
Example: ASP viewState variable
Stores client-side state
base64-encoded
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
46. Evading Pattern Detection
SQL injections possible in a lot of places
Example: ASP viewState variable
Stores client-side state
base64-encoded
aWQ9YCBPUiAxID4gMDtzdGF0ZT17d
mFyOiJBQkMiLGNvbHVtbnM6M30
What about this?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
47. Evading Pattern Detection
SQL injections possible in a lot of places
Example: ASP viewState variable
Stores client-side state
base64-encoded
id=` OR 1 > 0;
state={var:"ABC",columns:3}
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
48. Evading Pattern Detection
Catching /* */ comments is sometimes not enough
MySQL allows for 3 different types of comments
starting with # until end-of-line
starting with -- until end-of-line
C-style comments /* */
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
49. Evading Pattern Detection
Most DBMS are pretty tolerant regarding their SQL
MySQL does accept comments in a lot of places
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
50. Evading Pattern Detection
Most DBMS are pretty tolerant regarding their SQL
MySQL does accept comments in a lot of places
id=1/**/union/**/select/**/1,2/**/
from/**/users--
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
51. Evading Pattern Detection
The following comment based evasion was used to
bypass a simple ModSecurity CRS rule
http://victim.com/news.php?id=0+div
+1+union%23foo*%2F*bar%0D%0Aselect
%23foo%0D%0A1%2C2%2Ccurrent_user
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
52. Evading Pattern Detection
The following comment based evasion was used to
bypass a simple ModSecurity CRS rule
http://victim.com/news.php?id=0 div
1 union#foo*/*bar
select#foo
1,2,current_user
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
53. Evading Pattern Detection
Special format of MySQL comments allows inline-code
/*! MySQL code */
Can be extended for version specific code
CREATE /*!32302 TEMPORARY */
TABLE t ..
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
54. Evading Pattern Detection
This allows for another style of comments
/news.php?id=1/*!UnIoN*/SeLecT+1,2,3--
Used to bypass ModSecurity CRS and Wapple WAF:
1 ||1=1
1 /*!order by*/ 3
1 /*!union select*/ 1,table_name from
/*!information_schema.tables*/
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
55. Evading Pattern Detection
Patterns like ` OR 1 > 0` can be evaded by an endless
repertoire of variants
` OR 2*3 > 4
` OR @@version == @@version
` OR 1
` or round(pi(),1) + 1 + 1 = version()
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
56. Evading Pattern Detection
Exploiting different views of WAF and Application
/news.php?id=val1&id=val2
Web Server Interpretation Example
ASP.NET/IIS Concat by comma id=val1,val2
ASP/IIS Last parameter id=val2
PHP/Apache First parameter id=val1
JSP/Tomcat First parameter id=val1
DBMan Concat by tildes id=val1~~val2
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
57. Evading Pattern Detection
Bypassing ModSecurity CRS with HPP
/?id=select name&id=password from users
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
58. Evading Pattern Detection
Bypassing ModSecurity CRS with HPP
/?id=select name&id=password from users
ModSecurity filter view:
id=select name
id=password from users
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
59. Evading Pattern Detection
Bypassing ModSecurity CRS with HPP
/?id=select name&id=password from users
ModSecurity filter view:
id=select name
id=password from users
Backend ASP application:
id=select name,password from users
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
60. Evading Pattern Detection
Parameter pollution+variants have been used to evade
ModSecurity CRS
IBM Web Application Firewall
Imperva SecureSphere
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
61. Evading Pattern Detection
Parameter pollution+variants have been used to evade
ModSecurity CRS
IBM Web Application Firewall
Imperva SecureSphere
So, how to evade the evasion?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
62. Taking a different perspective
Web Server
Web Application
database
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
63. Taking a different perspective
Web Server
Web Application
database
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
64. Where to detect injections?
At this detection point
any encodings have been decoded by the web-
server and the application
no more „mangling“ is done
prior execution Web Server
any encoding-based evasions Web Application
do not apply anymore
database
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
65. What makes a successful SQLi?
1. SQL injection needs to modify existing statement
2. modified statement needs to be valid SQL
SELECT title,abstract FROM DOCS
WHERE
txt LIKE ‘DROP TABLE STUDENTS‘;
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
66. What makes a successful SQLi?
1. SQL injection needs to modify existing statement
2. modified statement needs to be valid SQL
SELECT title,abstract FROM DOCS
WHERE
txt LIKE ‘%‘
UNION
SELECT LOGIN,PASS FROM USERS;
--%`;
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
67. What makes a successful SQLi?
1. SQL injection needs to modify existing statement
2. modified statement needs to be valid SQL
SELECT title,abstract FROM DOCS
WHERE
txt LIKE ‘%‘
UNION
SELECT LOGIN,PASS FROM USERS;
--%`;
How do we „capture“ such modifications?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
68. Structure of SQL
SQL is a highly structured language
(ISO SQL-92, ISO/IEC9075:2003, ...)
statements parsed to abstract syntax tree
AST presents the structure of a statement
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
69. Structure of SQL
SQL is a highly structured language
(ISO SQL-92, ISO/IEC9075:2003, ...)
statements parsed to abstract syntax tree
AST presents the structure of a statement
SELECT
title,abstract
FROM DOCS
WHERE
txt LIKE ‘%security%‘;
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
70. Structure of SQL
SQL is a highly structured language
(ISO SQL-92, ISO/IEC9075:2003, ...)
statements parsed to abstract syntax tree
AST presents the structure of a statement
SELECT
SELECT
title,abstract
FROM DOCS
FROM COLS WHERE
WHERE
txt LIKE ‘%security%‘;
COL COL LIKE
`DOCS` `title` `abstr` `txt` `%`
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
71. Structure of an SQL injection
SELECT title,abstract FROM DOCS
WHERE
txt LIKE ‘%`;
SELECT
TABLE_REF COLUMN_LIST WHERE_COND
LIKE
COLUMN COLUMN
COLUMN CONST
`DOCS` `title` `abstr`
`txt` `%`
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
72. Structure of an SQL injection
SELECT title,abstract FROM DOCS
WHERE
txt LIKE ‘%‘
UNION
SELECT LOGIN,PASS FROM USERS; --%`;
UNION
SELECT SELECT
TABLE_REF COLUMN_LIST WHERE_COND COLUMN_LIST
TABLE_REF
LIKE
COLUMN COLUMN COLUMN COLUMN
COLUMN CONST
`DOCS` `title` `abstr`
`txt` `%` `USERS` `LOGIN` `PASS`
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
73. Structure of an SQL injection
INSERT INTO STUDENTS (NAME,CLASS,GRADE)
VALUES (`Robert`, ``, ``); DROP TABLE STUDENTS; --
`CS1`,`4`);
INSERT DROP
TABLE_REF COLUMN_LIST VALUE_LIST
TABLE_REF
COLUMN COLUMN COLUMN CONST CONST CONST
`STUDENTS` `NAME` `CLASS` `GRADE` `Robert` `` `` `STUDENTS`
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
74. Related Work
Parse Tree Validation to prevent SQL-Injections
injected snippets do change overall
structure of the query
Using Parse Tree Validation to
Prevent SQL Injection Attacks.
compare query trees BEFORE and Gregory T. Buehrer, Bruce W.
Weide, Paolo A.G. Sivilotti
AFTER inserting user-data SEM '05: Proceedings of the 5th
international workshop on
Software engineering and
implementation „SQLGuard“ extends middleware, ACM, 2005
Java‘s JDBC interface
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
75. Related Work
Parse Tree Validation to prevent SQL-Injections
injected snippets do change overall
structure of the query
Using Parse Tree Validation to
Prevent SQL Injection Attacks.
compare query trees BEFORE and Gregory T. Buehrer, Bruce W.
Weide, Paolo A.G. Sivilotti
AFTER inserting user-data SEM '05: Proceedings of the 5th
international workshop on
Software engineering and
implementation „SQLGuard“ extends middleware, ACM, 2005
Java‘s JDBC interface
Change in application code required, for
checking before and after user-data insertion
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
76. Related Work
Parse Tree Validation to prevent SQL-Injections
injected snippets do change overall
structure of the query
Using Parse Tree Validation to
Prevent SQL Injection Attacks.
compare query trees BEFORE and Gregory T. Buehrer, Bruce W.
Weide, Paolo A.G. Sivilotti
AFTER inserting user-data SEM '05: Proceedings of the 5th
international workshop on
Software engineering and
implementation „SQLGuard“ extends middleware, ACM, 2005
Java‘s JDBC interface
If you need to change the code, then
switch to prepared statements!!!!
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
77. How to detect structural changes?
What changes as SQL snippets are inserted into
SQL statements?
regular injected
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
78. How to detect structural changes?
What changes as SQL snippets are inserted into
SQL statements?
the number of inner tree nodes 9 : 15
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
79. How to detect structural changes?
What changes as SQL snippets are inserted into
SQL statements?
the number of inner tree nodes 9 : 15
the number of leave nodes 6 : 10
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
80. How to detect structural changes?
What changes as SQL snippets are inserted into
SQL statements?
the number of inner tree nodes 9 : 15
the number of leave nodes 6 : 10
the height of the tree 4:5
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
81. Effect of Evasions?
How does this scale in case of other WAF evasion
techniques?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
82. Effect of Evasions?
How does this scale in case of other WAF evasion
techniques?
replace or 1 = 1 with
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
83. Effect of Evasions?
How does this scale in case of other WAF evasion
techniques?
replace or 1 = 1 with
or round(pi(),1) + 1 + 1 = version()
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
84. Effect of Evasions?
How does this scale in case of other WAF evasion
techniques?
or round(pi(),1) + 1 + 1 = version()
OR
fn:equals
fn:add fn:version
fn:round fn:add
fn:pi 1 1 1
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
86. How can we use that to detect
attacks?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
87. A simple Demo-Shop
We implemented a simple Java Web Shop
uses MySQL backend
highly vulnerable to SQL injections
allows for simple definition of URL-to-SQL map
logs SQL statements along with request
allows logging ModSecurity anomaly scoring
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
88. A simple Demo-Shop
Multiple URLs that execute one or more SQL queries,
defined in a URL-to-SQL map:
GET /view-product: SELECT * FROM products
WHERE id = %{id};
POST /search: SELECT * FROM products
WHERE name LIKE ‘%%{query}%‘
OR desc LIKE ‘%%{query}%‘;
GET /cart/view: SELECT * FROM cart
WHERE id = ‘%{SESSION:ID}‘;
GET /cart/add: INSERT INTO cart VALUES
( %{SESSION:ID}, %{id}, 1 );
...
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
89. A simple Experiment
Generated some SQL logs
SQLMAP
Generated a „normal work load“ Bernardo Damele, Miroslav Stampar
http://www.sqlmap.org
attacked the shop with sqlmap
Recorded all HTTP traffic and SQL queries
test-client sqlmap Total
6251 147 6398
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
90. Distribution of number of inner nodes vs. total node
count for normal statements and SQL injections
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
91. Distribution of number of inner nodes vs. total node
count for normal statements and SQL injections
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
92. Distribution of number of inner nodes vs. total node
count for normal statements and SQL injections
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
93. Training a classifier
We‘re looking for a simple binary classification
Use some training data to find a function f that will
output „normal“ or „attack“ on new, unseen data
Recorded
Data
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
94. Training a classifier
We‘re looking for a simple binary classification
Use some training data to find a function f that will
output „normal“ or „attack“ on new, unseen data
Training
Recorded
Data
Test
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
95. Training a classifier
We‘re looking for a simple binary classification
Use some training data to find a function f that will
output „normal“ or „attack“ on new, unseen data
Training classifier
Recorded
Data
Test
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
96. Training a classifier
We‘re looking for a simple binary classification
Use some training data to find a function f that will
output „normal“ or „attack“ on new, unseen data
Training
Recorded
Data
Test classifier
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
97. Training a classifier
We‘re looking for a simple binary classification
Use some training data to find a function f that will
output „normal“ or „attack“ on new, unseen data
Training
Recorded
Data
Test classifier ??
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
98. Normal Attacks Total
Watching a single URL 514 65 579
We trained a simple classifier on the data to distinguish
a normal query and its modifications
normal sqlmap
pred normal 514 21 96,1 %
pred sqlmap 0 44 100 %
100 % 67,7 %
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
99. Normal Attacks Total
Watching a single URL 514 65 579
We trained a simple classifier on the data to distinguish
a normal query and its modifications
normal sqlmap
pred normal 514 21 96,1 %
pred sqlmap 0 44 100 %
100 % 67,7 %
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
100. Normal Attacks Total
Watching a single URL 514 65 579
We trained a simple classifier on the data to distinguish
a normal query and its modifications
normal sqlmap
pred normal 514 21 96,1 %
pred sqlmap 0 44 100 %
100 % 67,7 %
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
101. Normal Attacks Total
Watching a single URL 514 65 579
We trained a simple classifier on the data to distinguish
a normal query and its modifications
normal sqlmap
pred normal 514 21 96,1 %
pred sqlmap 0 44 100 %
100 % 67,7 %
Data labeled by User-Agent string, but sqlmap sends valid
requests at initial probe phase. These are no injections.
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
102. Normal Attacks Total
Watching a single URL 1245 55 1300
On a second data set with correctly labeled data, the
classifier perfectly detects all attacks with no false
positives
normal attack
pred normal 1245 0 100 %
pred attack 0 55 100 %
100 % 100 %
Results obtained by a 10-fold,
stratified cross validation
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
103. Watching multiple URLs
So we‘re able to learn how to tell a normal query and
its anomalous modification apart
Most web apps use more than a single query
How does our approach scale with multiple
queries?
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
104. A more complex Experiment
In this experiment, we checked detection capabilities with multiple
statements and their modified injection versions
normal attack
pred normal 6251 31 99,51 %
pred attack 0 116 100 %
100 % 78,91 %
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
105. A more complex Experiment
In this experiment, we checked detection capabilities with multiple
statements and their modified injection versions
normal attack
pred normal 6251 31 99,51 %
pred attack 0 116 100 %
100 % 78,91 %
The results above are obtained with a Support Vector Machine
(SVM) with linear kernel, C=1000.0
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
106. A more complex Experiment
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
107. The power of trees...
So far we explored classification using only the height and number of
nodes of a tree
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
108. The power of trees...
So far we explored classification using only the height and number of
nodes of a tree
What about using the complete tree?
SELECT name,SUM(PUNKTE)
FROM STUDENTS
WHERE
name = 'Marcin'
AND lvID = '42509'
SELECT
name SUM
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
109. The power of trees...
So far we explored classification using only the height and number of
nodes of a tree
What about using the complete tree?
SELECT name,SUM(PUNKTE)
Start --> SELECT
FROM STUDENTS
SELECT --> ResultCols From Where
WHERE
ResultCols --> ResultCol ResultCol
name = 'Marcin'
ResultCol --> ColRef
AND lvID = '42509'
ColRef --> 'NAME'
ResultCol --> ColRef
SELECT
AggregateNode --> SUM
ColRef --> 'PUNKTE'
FromList --> TableRef
TableRef --> STUDENTS
Where --> AndNode
AndNode --> BinOp BinOp
name SUM BinaryOp --> Eq ColRef Const
ColRef --> `name`
Const --> `Marcin`
...
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
110. The power of trees...
So far we explored classification using only the height and number of
nodes of a tree
What about using the complete tree?
SELECT name,SUM(PUNKTE)
Start --> SELECT
FROM STUDENTS
SELECT --> ResultCols From Where
WHERE
ResultCols --> ResultCol ResultCol
name = 'Marcin'
ResultCol --> ColRef
AND lvID = '42509'
ColRef --> 'NAME'
ResultCol --> ColRef
SELECT
AggregateNode --> SUM
ColRef --> 'PUNKTE'
FromList --> TableRef
TableRef --> STUDENTS
Where --> AndNode
AndNode --> BinOp BinOp
name SUM BinaryOp --> Eq ColRef Const
ColRef --> `name`
Const --> `Marcin`
...
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
111. The power of trees...
So far we explored classification using only the height and number of
nodes of a tree
What about using the complete tree?
.
SELECT name,SUM(PUNKTE) 0
Start --> SELECT 1
FROM STUDENTS
SELECT --> ResultCols From Where 1
WHERE 1
ResultCols --> ResultCol ResultCol
name = 'Marcin' 1
ResultCol --> ColRef 2
AND lvID = '42509'
ColRef --> 'NAME' 1
ResultCol --> ColRef 1
SELECT 1
AggregateNode --> SUM
1
ColRef --> 'PUNKTE' 1
FromList --> TableRef 1
TableRef --> STUDENTS 1
Where --> AndNode 1
1
AndNode --> BinOp BinOp
1
name SUM BinaryOp --> Eq ColRef Const 1
ColRef --> `name` 1
Const --> `Marcin` 1
... 0
.
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
112. The power of trees... Experiment
A high-dimensional feature space provides more chances to separate
between normal and attack - so let‘s see:
normal attack
pred normal 6251 11 99,82 %
pred attack 0 136 100 %
100 % 92,52 %
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
113. The power of trees... Experiment
A high-dimensional feature space provides more chances to separate
between normal and attack - so let‘s see:
normal attack
pred normal 6251 11 99,82 %
pred attack 0 136 100 %
100 % 92,52 %
The SVM classifier performs much better and is able to predict the attacks pretty
good, with a polynomial kernel of degree 3, gamma=100.0, C=1000.0
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
114. 11 attacks missed??
So what went wrong?
SELECT id,name,desc,price
FROM products
WHERE
name LIKE '%secret%'
OR desc LIKE '%secret%'
By accident labeled as „attack“ (User-Agent).
Just a „probe“ query of sqlmap
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
115. 11 attacks missed??
So what else went wrong?
SELECT id,name,desc,price
FROM products WHERE
name LIKE '%secret) AND 8579=8579 AND (7161=7161%'
OR
desc LIKE '%secret) AND 8579=8579 AND (7161=7161%'
This one was labeled as „attack“ in the test data.
The classifier said it is „normal“.
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
116. The power of trees... Experiment
After manual inspection - all missed attacks turned out
to be normal queries or unsuccessful SQL injections
normal attack
pred normal 6262 0 100 %
pred attack 0 136 100 %
100 % 100 %
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
117. The power of trees... Experiment
After manual inspection - all missed attacks turned out
to be normal queries or unsuccessful SQL injections
normal attack
pred normal 6262 0 100 %
pred attack 0 136 100 %
100 % 100 %
The SVM classifier perfectly distinguished attacks and normal queries
with a polynomial kernel of degree 3, gamma=100.0, C=1000.0
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
118. So what about a real application?
Good question!
Please upload your database-logs + web-logs to my
web-site and I will try :-)
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
119. So what about a real application?
Good question!
Please upload your database-logs + web-logs to my
web-site and I will try :-)
We checked out Typo-3
1000 queries, 15 artificial attacks
about 90% detection rate
too few training data
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
120. Trying to visualize SQL of Typo-3...
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
121. Trying to visualize SQL of Typo-3...
ISOM created from Typo3 1000 SQL queries with 15 artificial SQL
injections, a tree-kernel was used as similarity measure
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
122. Summary
A successful SQL injection needs to alter the query
Syntactical approach for detecting SQL injections
Escapes evasion attacks by inspecting the queries
just before they hit the database
Vectorization of trees for detection using machine
learning showed good results
Creating an SQL parser is the hardest part :-)
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org
123. Ingres SQL Parser Collection
References Part of the Ingres Migration Toolset
http://code.ingres.com/
jsqlparser
SQL parser libraries http://jsqlparser.sf.net
my fork of jsqlparser
jsqlparser (Java) github.com/cbockermann/jsqlparser
(generated with javacc)
Ingres SQL parser library (Java)
(based on antlr, conversion of mysql parser)
Machine Learning Tool
RapidMiner RapidMiner
http://rapid-i.com/
Hashdays 2011, Luzern Christian Bockermann - chris @ jwall.org