libinjection: new technique in detecting SQLi attacks, iSEC Partners Open Forum

6,548 views

Published on

Published in: Technology
1 Comment
5 Likes
Statistics
Notes
No Downloads
Views
Total views
6,548
On SlideShare
0
From Embeds
0
Number of Embeds
775
Actions
Shares
0
Downloads
57
Comments
1
Likes
5
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • libinjection: new technique in detecting SQLi attacks, iSEC Partners Open Forum

    1. libinjectionNew Techinques in Detecting SQLi AtttacksiSEC Partners Open ForumGilt Group, New York, September 6 2102Nick Galbreath @ngalbreath nickg@client9.com
    2. http://client9.com/20120906/ Follow along at your own pace
    3. First presented atBlack Hat USA 2012http://client9.com/20120725 iSEC Partners party at Bellagio
    4. The Next 15 Minutes• You know what SQLi is and why its important• Why detecting SQLi is a hard problem• Why current solutions arent so good• The libinjection algorithm and library
    5. Its Easy to Get Startedwith Regular Expressions s/UNIONs+(ALL)?/i ‣ At least two open source WAF use regular expressions. ‣ Failure cases in closed-source WAFs also indicate regexp.
    6. 1992 SQL Spec625 pages plain text bit.ly/10fmhZ
    7. 2003 SQL Spec128 pages pure BNF bit.ly/OB5vfW
    8. Integer Forms• [0-9]+• 0x[0-9a-fA-F]+ 0xDEADbeef MySQL, MSSQL 0x is case sensitive• 0x MSSQL only• xDEADbeef PgSQL• b10101010 MySQL, PgSQL• 0b010101 MySQL
    9. Floating Point• digits• digits[.] http://bit.ly/Qp6KTu• digits[.]digits • digits[.]digits[eE]digits• digits[eE]digits • [.]digits• digits[eE][+-]digits • [.]digits[eE]digits• digits[.][eE]digits • [.]digits[eE][+-]digits• digits[.]digits[eE][+-]digits • "binary_float_infinity" (O) Optional starts with [+-] Optional ending with [dDfF] (Oracle)
    10. Money Literals• MSSQL has a money type.• -$45.12• $123.0• +$1,000,000.00 Commas ignored• Havent experimented with this yet.• Does it auto-cast to a float or int type?
    11. Ridiculous Operators• != not equals, standard • ||/ cube root (pgsql)• <=> mysql • ** exponents (oracle)• <> mssql • # bitwise xor (pgsql conflicts with mysql• ^= oracle comment)• !>, !< not less than mssql• / oracle• !! factorial (pgsql)• |/ sqaure root (pgsql)
    12. Strings,Charsets,and Comments Such a cluster that I defer you to my DEFCON 20 talk. http://client9.com/20120727/
    13. (?:)s*whens*d+s*then)|(?:"s*(?:#|--|{))|(?:/*!s?d+)|(?:ch(?:a)?rs*(s*d)|(?:(?:(n?and|x?or|not)s+||||&&)s*w+()(?:[s()]cases*()|(?:)s*likes*()|(?:havings*[^s]+s*[^ws])|(?:ifs?([dw]s*[=<>~])(?:"s*ors*"?d)|(?:x(?:23|27|3d))|(?:^.?"$)|(?:(?:^["]*(?:[d"]+|[^"]+"))+s*(?:n?and|x?or|not||||&&)s*[w"[+&!@(),.-])|(?:[^ws]w+s*[|-]s*"s*w)|(?:@w+s+(and|or)s*["d]+)|(?:@[w-]+s(and|or)s*[^ws])|(?:[^ws:]s*dW+[^ws]s*".)|(?:Winformation_schema|table_nameW)(?:"s**.+(?:or|id)W*"d)|(?:^")|(?:^[ws"-]+(?<=ands)(?<=ors)(?<=xors)(?<=nands)(?<=nots)(?<=||)(?<=&&)w+()|(?:"[sd]*[^ws]+W*dW*.*["d])|(?:"s*[^ws?]+s*[^ws]+s*")|(?:"s*[^ws]+s*[Wd].*(?:#|--))|(?:".**s*d)|(?:"s*ors[^d]+[w-]+.*d)|(?:[()*<>%+-][w-]+[^ws]+"[^,])(?:d"s+"s+d)|(?:^admins*"|(/*)+"+s?(?:--|#|/*|{)?)|(?:"s*or[ws-]+s*[+<>=(),-]s*[d"])|(?:"s*[^ws]?=s*")|(?:"W*[+=]+W*")|(?:"s*[!=|][ds!=+-]+.*["(].*$)|(?:"s*[!=|][ds!=]+.*d+$)|(?:"s*likeW+[w"(])|(?:siss*0W)|(?:wheres[sw.,-]+s=)|(?:"[<>~]+")(?:unions*(?:all|distinct|[(!@]*)?s*[([]*s*select)|(?:w+s+likes+")|(?:likes*"%)|(?:"s*likeW*["d])|(?:"s*(?:n?and|x?or|not ||||&&)s+[sw]+=s*w+s*having)|(?:"s**s*w+W+")|(?:"s*[^?ws=.,;)(]+s*[(@"]*s*w+W+w)|(?:selects*[[]()sw.,"-]+from)|(?:find_in_sets*()(?:ins*(+s*select)|(?:(?:n?and|x?or|not ||||&&)s+[sw+]+(?:regexps*(|soundss+likes*"|[=d]+x))|("s*ds*(?:--|#))|(?:"[%&<>^=]+ds*(=|or))|(?:"W+[w+-]+s*=s*dW+")|(?:"s*iss*d.+"?w)|(?:"|?[w-]{3,}[^ws.,]+")|(?:"s*iss*[d.]+s*W.*")(?:[dW]s+ass*["w]+s*from)|(?:^[Wd]+s*(?:union|select|create|rename|truncate|load|alter|delete|update|insert|desc))|(?:(?:select|create|rename|truncate|load|alter|delete|update|insert|desc)s+(?:(?:group_)concat|char|load_file)s?(?)|(?:ends*);)|("s+regexpW)|(?:[s(]load_files*()(?:@.+=s*(s*select)|(?:d+s*ors*d+s*[-+])|(?:/w+;?s+(?:having|and|or|select)W)|(?:ds+groups+by.+()|(?:(?:;|#|--)s*(?:drop|alter))|(?:(?:;|#|--)s*(?:update|insert)s*w{2,})|(?:[^w]SETs*@w+)|(?:(?:n?and|x?or|not ||||&&)[s(]+w+[s)]*[!=+]+[sd]*["=()])(?:"s+ands*=W)|(?:(s*selects*w+s*()|(?:*/from)|(?:+s*d+s*+s*@)|(?:w"s*(?:[-+=|@]+s*)+[d(])|(?:coalesces*(|@@w+s*[^ws])|(?:W!+"w)|(?:";s*(?:if|while|begin))|(?:"[sd]+=s*d)|(?:orders+bys+ifw*s*()|(?:[s(]+cased*W.+[tw]hen[s(])(?:(select|;)s+(?:benchmark|if|sleep)s*?(s*(?s*w+)(?:creates+functions+w+s+returns)|(?:;s*(?:select|create|rename|truncate|load|alter|delete|update|insert|desc)s*[[(]?w{2,})(?:alters*w+.*characters+sets+w+)|(";s*waitfors+times+")|(?:";.*:s*goto)(?:procedures+analyses*()|(?:;s*(declare|open)s+[w-]+)|(?:creates+(procedure|function)s*w+s*(s*)s*-)|(?:declare[^w]+[@#]s*w+)|(execs*(s*@)(?:selects*pg_sleep)|(?:waitfors*delays?"+s?d)|(?:;s*shutdowns*(?:;|--|#|/*|{))(?:sexecs+xp_cmdshell)|(?:"s*!s*["w])|(?:fromW+information_schemaW)|(?:(?:(?:current_)?user|database|schema|connection_id)s*([^)]*)|(?:";?s*(?:select|union|having)s*[^s])|(?:wiifs*()|(?:execs+master.)|(?:union select @)|(?:union[w(s]*select)|(?:select.*w?user()|(?:into[s+]+(?:dump|out)files*")(?:merge.*usings*()|(executes*immediates*")|(?:W+d*s*havings*[^s-])|(?:matchs*[w(),+-]+
    14. Proven Fail‣ At Black Hat USA 2005, Hanson and Patterson presented: Guns and Butter: Towards Formal Axioms of Validation (http://bit.ly/OBe7mJ)‣ …formally proved that for any regex validator, we could construct either a safe query which would be flagged as dangerous, or a dangerous query which would be flagged as correct.‣ (summary from libdejector documentation)
    15. Can we do better?
    16. libinjection
    17. Key Insight‣ A SQLi attack must be parsed as SQL within the original query.‣ SQL has a rigid syntax ‣ it works, or its a syntax error. ‣ Compare this to HTML/XSS rules‣ "Is it a SQLi attack?" becomes "Could it be a SQL snippet?"
    18. Only 3 ContextsUser input is only "injected" into SQL in threeways:‣ As-Is‣ Inside a single quoted string‣ Inside a double quoted stringMeans we have to parse input three times.Compare to XSS
    19. Identification of SQL snippets without context is hard‣ 1-917-660-3400 my phone number or an arithmetic expression in SQL?‣ @ngalbreath my twitter account or a SQL variable?‣ English-like syntax and common keywords: union, group, natural, left, right, join, top, table, create, in, is, not, before, begin, between
    20. Existing SQL Parsers‣ Only parse their flavor of SQL‣ Not well designed to handle snippets‣ Hard to extend‣ Worried about correctness ... so I wrote my own!
    21. Tokenization‣ Converts input into a stream of tokens‣ Uses "master list" of keywords and functions across all databases.‣ Handles comments, string, literals, weirdos.
    22. 5000224 UNION USER_ID>0--[ (...500224, string), (UNION, union operator), (USER_ID, name), (>, operator), (0, number), (--....., comment) ]
    23. Meet the Tokens‣ none/name ‣ group-like operation‣ variable ‣ union-like operator‣ string ‣ logical operator‣ regular operator ‣ function‣ unknown ‣ comma‣ number ‣ semi-colon‣ comment ‣ left parens‣ keyword ‣ right parens
    24. Merging, Specialization, Disambiguation‣ "IS", "NOT" ==> "IS NOT" (single op)‣ "NATURAL", "JOIN" => "NATURAL JOIN"‣ ("+", operator) -> ("+", unary operator)‣ (COS, function), (1, number) ==> (COS, name), (1, number) functions must be followed with a parenthesis!
    25. Folding‣ This step actually isnt needed to detect, but is needed to reduce false positives.‣ Converts simple arithmetic expressions into a single value (dont try to evaluate them).‣ 1-917-660-3400 -> "1"
    26. Knows nothing about SQLi ‣ So far this is purely a parsing problem. ‣ Knows nothing about SQLi (which is evolving) ‣ Can be 100% tested against any SQL input (not SQLi) for correctness. ‣ Language independent test cases$ cat test-tokens-numbers-floats-003.txt--TEST--floating-point parsing test--INPUT--SELECT .0;--EXPECTED--k SELECT1 .0; ;
    27. Fingerprints‣ The token types of a user input form a hash or a fingerprint. ‣ -6270" UNION ALL SELECT 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594, 5594# AND "JWWQ"="JWWQ ‣ becomes "sUk1,1,1,1,1,1,1,1,&"‣ Now lets generate fingerprints from Real World Data.‣ Can we distinguish between SQLi and benign input?
    28. Training on SQLi‣ Parse known SQLi attacks from ‣ SQLi vulnerability scanners ‣ Published reports ‣ SQLI How-Tos‣ > 32,000 total‣ Since Black Hat, donations from ‣ modsecurity ‣ qualys‣ > 50,000 total
    29. Training on Real Input‣ 100s of Millions of user inputs from Etsys access logs were also parsed.‣ Large enough to get a good sample (Top 50 USA site)‣ Old enough to have lots of odd ways of query string formatting.‣ Full text search with an diverse subject domain
    30. How many tokens areneeded to determine ifuser input is SQLi or not?
    31. 5no matter how long the input
    32. 622 out of 1,048,576 are SQLi&1o1U &f()o &f(1) 1&((f 1&((k 1&(1, 1&(1o 1&(f( 1&(k( 1&(k1 1&(kf 1&(kk 1&(kn 1&(ko 1&(v) 1&1Bf 1&1Uk 1&1f( 1&1o( 1&1o11&1of 1&1ok 1&1on 1&1oo 1&1ov 1&f(( 1&f() 1&f(1 1&f(f 1&f(k 1&f(n 1&f(v 1&k(1 1&k(f 1&k1k 1&kUk 1&kk1 1&n() 1&no1 1&o(11&o1o 1&so1 1)&(1 1)&(f 1)&(k 1)&(n 1)&1B 1)&1U 1)&1f 1)&1o 1)&f( 1)&o( 1)()s 1))&( 1))&1 1))&f 1))&o 1)))& 1)))) 1)));1)))B 1)))U 1)))k 1)))o 1));k 1))B1 1))Uk 1))Un 1))k1 1))kk 1))o( 1))o1 1))of 1))ok 1))on 1),(1 1);k& 1);k( 1);kf 1);kk1);kn 1);ko 1)B1 1)B1& 1)B1c 1)B1o 1)U(k 1)Uk1 1)Ukf 1)Ukk 1)Ukn 1)Unk 1)k1 1)k1c 1)k1o 1)kks 1)o(1 1)o(k 1)o(n 1)o1B1)o1f 1)o1k 1)o1o 1)of( 1)ok( 1)ok1 1)on& 1)ono 1,(f( 1,(k( 1,(k1 1,(kf 1,1), 1,1)o 1,1B1 1,1Uk 1,f(1 1,s), 1;k&k 1;k((1;k(1 1;k(o 1;k1, 1;kf( 1;kks 1;kn( 1;kn, 1;knc 1;ko( 1;kok 1B1 1B1,1 1B1,n 1B1Uk 1B1c 1B1k1 1B1ks 1Bf(1 1Bf(f 1Bk(11Bn,n 1Bnk1 1U(k1 1U(kf 1U1,1 1Uc 1Uk 1Uk(1 1Uk(k 1Uk(n 1Uk1 1Uk1, 1Uk1c 1Uk1f 1Uk1k 1Uk1n 1Uk1o 1Ukf 1Ukf( 1Ukf,1Ukk( 1Ukk, 1Ukk1 1Ukkk 1Ukkn 1Ukn& 1Ukn( 1Ukn, 1Ukn1 1Uknc 1Uknk 1Ukno 1Ukns 1Uko1 1Ukok 1Uks, 1Uksc 1Ukv 1Ukv, 1Ukvc1Un,1 1Un1, 1Unk( 1Unk1 1Unkf 1Uon1 1f()k 1k1U( 1k1Uk 1k1c 1kU1, 1kf(1 1kk(1 1kksc 1knkn 1n&f( 1nUk1 1nUkn 1nk1c 1nkf(1o((( 1o((1 1o((f 1o(1) 1o(1o 1o(f( 1o(k( 1o(k1 1o(kf 1o(kn 1o(kv 1o(n) 1o(s) 1o1)& 1o1)o 1o1Bf 1o1Uk 1o1f( 1o1kf 1o1o(1o1o1 1o1of 1o1oo 1o1ov 1of() 1of(1 1of(f 1of(n 1of(s 1ok(1 1ok(k 1ok1 1ok1, 1ok1c 1ok1k 1okf( 1oks, 1oksc 1okv, 1onos1oso1 ;kknc Uk1,1 Uk1,f Uk1,n Ukkkn f((k( f((kf f()&f f()of f(1)& f(1)o f(1,f f(1o1 f(f() f(f(1 f(k() f(k,( f(k,n f(n()f(v,1 k()ok k(1)U k(ok( k(vv) k1,1, k1,1c k1,1k k1,f( k1k(k k1o(s k;non kf(1) kf(1, kf(f( kf(n, kf(v: kk(f( kk1kk kk1nkkk1vk kk1vn kn1kk kn1vk kn1vn ko(k( ko(kf kok(k kv) kvk(1 n&(1) n&(k1 n&(o1 n&1f( n&1o( n&1o1 n&1of n&1oo n&f(1 n&f(fn&k(1 n&o1o n)&(k n)&1f n)&1o n)&f( n))&( n))&1 n))&f n)))& n))); n)))k n)))o n));k n))kk n))o( n))o1 n))of n))ok n);k&n);k( n);kf n);kk n);kn n);ko n)k1o n)kks n)o(k n)o1& n)o1f n)o1o n)of( n)ok( n,(f( n,(k( n,(k1 n,(kf n,f(1 n:o1U n;k&kn;k(( n;k(1 n;kf( n;kks n;kn( n;ko( n;kok nUk(k nUk1, nUkn, nUnk( nk1Uk nkf(1 nkksc nnn)U nno1U no(k1 no(o1 no1&1 no1Ukno1f( no1o( no1o1 no1of no1oo nof(1 nok(1 nok(k o1kf( oUk1, of()o of(1) ok1o1 okkkn ook1, s&(1) s&(1, s&(1o s&(f( s&(k)s&(k1 s&(kf s&1Bf s&1Uk s&1c s&1f( s&1o( s&1o1 s&1of s&1on s&1oo s&1os s&1ov s&f(( s&f() s&f(1 s&f(f s&f(v s&k&s s&k(1s&k(o s&k1o s&kc s&knk s&ko( s&ko1 s&kok s&kos s&n&s s&no1 s&nos s&o(1 s&o(k s&o1o s&okc s&oko s&os s&sos s&v:o s&voss&vso s)&(1 s)&(k s)&1B s)&1f s)&1o s)&f( s)&o( s))&( s))&1 s))&f s))&n s))&o s)))& s))); s)))B s)))U s)))k s)))o s));ks))B1 s))Uk s))Un s))k1 s))kk s))o( s))o1 s))of s))ok s),(1 s);k& s);k( s);kf s);kk s);kn s);ko s)B1 s)B1& s)B1c s)B1os)U(k s)Uk1 s)Unk s)k1 s)k1c s)k1o s)kks s)o(1 s)o(k s)o1B s)o1f s)o1k s)o1o s)of( s)ok( s)ok1 s,1), s;k&k s;k(( s;k(1s;k(o s;k1, s;k1o s;k; s;k[k s;k[n s;kf( s;kkn s;kks s;kn( s;knk s;knn s;ko( s;kok s;kvc s;kvk s;n:k sB1 sB1&s sB1UksB1c sB1os sU((k sU(k( sU(kk sU(kn sU(ks sUk(k sUk1 sUk1& sUk1, sUk1c sUk1k sUk1o sUkf( sUkk1 sUkkk sUkn( sUkn, sUkn1sUknk sUkok sUkv, sUkvc sUn(k sUnk1 sUnkf sUno1 sf(1) sf(n, sf(s) sk)&( sk)&1 sk)&f sk);k sk)B1 sk)Uk sk)Un sk)k1 sk)kksk)o( sk)o1 sk)of sk)ok sk1&1 sk1Uk sk1c sk1o1 sk1os skU1, skks skksc sn,f( sno1U so((( so((k so((s so(1) so(1o so(f(so(k) so(k1 so(kk so(kn so(ko so(ks so(os so(s) so1&1 so1&o so1&s so1Bf so1Uk so1c so1f( so1kf so1o( so1o1 so1of so1okso1oo so1os so1ov sof() sof(1 sof(f sof(k sok&s sok(1 sok(k sok(o sok(s sok1 sok1, sok1c sok1o sokc sokf( sokn, soknksoko( soko1 sokok sokos sonk1 sono1 sonos sos sos&( soso( sosos sov&1 sov&s sov:o sovo1 sovok sovos sovov sovso v:o1)vUk1, vok1,
    33. The LibraryOn GitHub Now~500 Lines of CodeOne file + dataNo memory allocationNo threadsNo external dependenciesFixed stack size>100k checks a second
    34. tada#include "sqlparse.h"#include <string.h>int main(){ const char* ucg = "1 OR 1=1"; // input should be normalized, upper-cased // You can use sqli_normalize // if you dont have your own function sfilter sf; return is_sqli(&sf, ucg, strlen(ucg));}$ gcc -Wall -Wextra sample.c sqlparse.c$ ./a.out$ echo $?1
    35. Whats Next?• Change API to allow passing in fingerprint data or a function. Allows upgrades without code changes.• Can we reduce the number of tokens? String, variables, numbers are all just values.• Folding of comma-separated values? 1,2,3,4 => 1• Can we just eliminate all parenthesis?
    36. Help!• More SQLi from the field please!• False positives welcome• More test cases with exotic SQL to test parser.• Ports to other languages (the language- neutral test framework should make this easier).• Compiling on Windows (mostly tested on Mac OS X and Linux)
    37. Slides and Source Code:http://www.client9.com/libinjection/Nick Galbreath@ngalbreathnickg@client9.com Sept 20 OWASP NY SQL obfuscation and libinjection Oct 25, OWASP USA, Austin, Texas Continuous Deployment and Security Thanks

    ×