Heuristic methods
used in sqlmap
Miroslav Štampar
(dev@sqlmap.org)
Heuristic methods
used in sqlmap
Miroslav Štampar
(dev@sqlmap.org)
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 2
Heuristic method
Heuristic (/hjʉˈrɪstɨk/; Greek: “Εὑρίσκω”, “find” or
“discover”)
Educational shortcuts to ease the cognitive load of
making a decision
Speeding up the process of finding a satisfactory
solution
Serve as an aid to learning, discovery and problem
solving
Experimental and trial-and-error approach
Resulting with a solution which is not guaranteed to
be optimal
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 3
Heuristic SQL injection checks (1)
In some cases DBMS error reporting is enabled
Deliberately invalid parameter value is sent (e.g.
())'”(''”')
 ”SQL error: You have an error in your SQL syntax;
check the manual that corresponds to your MySQL
server version...”
Narrowing down tests to only those specific to
recognized/parsed DBMS
Automatically extending with DBMS specific tests that
would be done (only) on higher level
Switch --smart for finding targets in minutes
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 4
Heuristic SQL injection checks (2)
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 5
Heuristic SQL injection checks (3)
Particularly interesting cases when parameter
contains integer value (e.g. id=1)
Response is changing with value changed (e.g. id=2)
Response is the same for arithmetic operation resulting
in original value (e.g. id=1183-1182)
Warning the user that the processed parameter is
most probably prone to SQL injection attack(s)
Making user aware to persist in eventual further tests
(to use higher --level/--risk and/or --tamper)
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 6
Heuristic SQL injection checks (4)
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 7
Heuristic SQL injection checks (5)
In cases when error reporting is turned off
At least one generic (i.e. non-DBMS dependent) SQL
injection technique is found
Single request per supported DBMS is being sent
Using form supported by only the targeted one (e.g.
id=1 AND (SELECT 0x41597548)=0x41597548)
for MySQL)
If getting response as expected, narrowing and/or
expanding tests for particular DBMS
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 8
Heuristic SQL injection checks (6)
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 9
Type casting detection (1)
In some cases (preventive) hard type casting (e.g.
$id=intval($_REQUEST['id'])) is used
Especially noticeable on integer parameter values
(e.g. id=1)
Different responses for different integer values (e.g.
id=2)
Same response for appended random string values
(e.g. id=1vHxr)
High probability that the type casting is being used
(hence, not exploitable)
Warning the user of a potentially futile run
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 10
Type casting detection (2)
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 11
Length constraining detection (1)
Suhosin ( ) - popular open source PHP patch
“...protecting servers and users from known and
unknown flaws in PHP applications and PHP core...”
Constraining the length of request parameter values
(e.g. GET values constrained to less than 512 chars)
Problematic when longer payloads are used (e.g.
injecting of PHP shell through SELECT..INTO
OUTFILE statement)
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 12
Length constraining detection (2)
One lengthy payload containing a simple boolean
question (e.g. id=1 AND 3182=...3182) is sent
right after successful identification of SQL injection
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 13
WAF detection/fingeprinting (1)
Web Application Firewall (engl. abbr. WAF)
“...commonly used mechanism for prevention of
malicious web application attacks...”
Modsecurity returns HTTP error code 501 on detected
attack
F5 BIG-IP adds its own X-Cnection HTTP header to
the response
Generic detection (switch --check-waf)
Fingeprinting 28 different WAF products (switch
--identify-waf)
Sending deliberately suspicious payloads and
checking response(s) for unique characteristics
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 14
WAF detection/fingeprinting (2)
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 15
False positive detection (1)
“...term used for describing a result that indicates
that a given condition is present when it is not...”
Another way of saying “mistake”
Giving false sense of certainty while in reality there is
nothing exploitable at the other side
Especially noticeable in boolean-based blind and time-
based blind cases
Simple tests are being done after the detection phase
Inspecting responses to predetermined boolean
operations (e.g. id=1 AND 95>27)
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 16
False positive detection (2)
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 17
Delay detection (1)
Network latency (or lagging) is the main problem of
time-based blind technique
For example, used deliberate delay is 1 sec, normal
response times are >0.5 and <2.0 secs, what we can
conclude for 1.5 sec response?
Also, how to recognize delays in hard-queries (e.g.
BENCHMARK(5000000,MD5('foobar')))?
Mathematical statistics to the rescue
Gaussian bell-shaped curve for the normal distribution
is being calculated
Everything inside is considered as “normal”, outside
as “not normal”
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 18
Delay detection (2)
Everything that's normal (i.e. not deliberately
delayed) should fit under the curve
μ(t) represents a mean, while σ(t) represents a
standard deviation of response times
99.99% of normal response times fall under the
upper border value μ(t) + 7σ(t)
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 19
Delay detection (3)
sqlmap learns what's normal and what's not from
non-delay based payload responses (e.g. boolean-
based blind)
Also, few dummy requests are done if there is a need
for more
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 20
Delay detection (4)
In case of considerable network latency user is
warned to use as high delay value as possible
On the other hand, if negligible, sqlmap optimizes the
delay value (if applicable) to a more appropriate one
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 21
Finding number of query fields (1)
For successful usage of UNION-based technique,
number of fields has to be known
Foremost method used is ORDER BY clause
If response for an arbitrary large value (e.g. id=1
ORDER BY 9664#) drastically differs from neutral
(e.g. id=1 ORDER BY 1#)
sqlmap uses adapted (chunked) binary search
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 22
Finding number of query fields (2)
If ORDER BY method is not usable, similar approach
is used as in delay detection
Constant field value (e.g. NULL) used across a
(current chunk of) search space in UNION ALL
SELECT statement
Searching for the number of fields that, compared to
the original response, “sticks out” out of others in a
statistical manner
Responses for non-valid number of fields will not
much differ from each other
Response for the right number of fields will appear
not normal compared to others
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 23
Finding number of query fields (3)
FSec – FOI 2013, Varaždin (Croatia) September 19th, 2013 24
Questions?

Heuristic methods used in sqlmap

  • 1.
    Heuristic methods used insqlmap Miroslav Štampar (dev@sqlmap.org) Heuristic methods used in sqlmap Miroslav Štampar (dev@sqlmap.org)
  • 2.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 2 Heuristic method Heuristic (/hjʉˈrɪstɨk/; Greek: “Εὑρίσκω”, “find” or “discover”) Educational shortcuts to ease the cognitive load of making a decision Speeding up the process of finding a satisfactory solution Serve as an aid to learning, discovery and problem solving Experimental and trial-and-error approach Resulting with a solution which is not guaranteed to be optimal
  • 3.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 3 Heuristic SQL injection checks (1) In some cases DBMS error reporting is enabled Deliberately invalid parameter value is sent (e.g. ())'”(''”')  ”SQL error: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version...” Narrowing down tests to only those specific to recognized/parsed DBMS Automatically extending with DBMS specific tests that would be done (only) on higher level Switch --smart for finding targets in minutes
  • 4.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 4 Heuristic SQL injection checks (2)
  • 5.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 5 Heuristic SQL injection checks (3) Particularly interesting cases when parameter contains integer value (e.g. id=1) Response is changing with value changed (e.g. id=2) Response is the same for arithmetic operation resulting in original value (e.g. id=1183-1182) Warning the user that the processed parameter is most probably prone to SQL injection attack(s) Making user aware to persist in eventual further tests (to use higher --level/--risk and/or --tamper)
  • 6.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 6 Heuristic SQL injection checks (4)
  • 7.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 7 Heuristic SQL injection checks (5) In cases when error reporting is turned off At least one generic (i.e. non-DBMS dependent) SQL injection technique is found Single request per supported DBMS is being sent Using form supported by only the targeted one (e.g. id=1 AND (SELECT 0x41597548)=0x41597548) for MySQL) If getting response as expected, narrowing and/or expanding tests for particular DBMS
  • 8.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 8 Heuristic SQL injection checks (6)
  • 9.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 9 Type casting detection (1) In some cases (preventive) hard type casting (e.g. $id=intval($_REQUEST['id'])) is used Especially noticeable on integer parameter values (e.g. id=1) Different responses for different integer values (e.g. id=2) Same response for appended random string values (e.g. id=1vHxr) High probability that the type casting is being used (hence, not exploitable) Warning the user of a potentially futile run
  • 10.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 10 Type casting detection (2)
  • 11.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 11 Length constraining detection (1) Suhosin ( ) - popular open source PHP patch “...protecting servers and users from known and unknown flaws in PHP applications and PHP core...” Constraining the length of request parameter values (e.g. GET values constrained to less than 512 chars) Problematic when longer payloads are used (e.g. injecting of PHP shell through SELECT..INTO OUTFILE statement)
  • 12.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 12 Length constraining detection (2) One lengthy payload containing a simple boolean question (e.g. id=1 AND 3182=...3182) is sent right after successful identification of SQL injection
  • 13.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 13 WAF detection/fingeprinting (1) Web Application Firewall (engl. abbr. WAF) “...commonly used mechanism for prevention of malicious web application attacks...” Modsecurity returns HTTP error code 501 on detected attack F5 BIG-IP adds its own X-Cnection HTTP header to the response Generic detection (switch --check-waf) Fingeprinting 28 different WAF products (switch --identify-waf) Sending deliberately suspicious payloads and checking response(s) for unique characteristics
  • 14.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 14 WAF detection/fingeprinting (2)
  • 15.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 15 False positive detection (1) “...term used for describing a result that indicates that a given condition is present when it is not...” Another way of saying “mistake” Giving false sense of certainty while in reality there is nothing exploitable at the other side Especially noticeable in boolean-based blind and time- based blind cases Simple tests are being done after the detection phase Inspecting responses to predetermined boolean operations (e.g. id=1 AND 95>27)
  • 16.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 16 False positive detection (2)
  • 17.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 17 Delay detection (1) Network latency (or lagging) is the main problem of time-based blind technique For example, used deliberate delay is 1 sec, normal response times are >0.5 and <2.0 secs, what we can conclude for 1.5 sec response? Also, how to recognize delays in hard-queries (e.g. BENCHMARK(5000000,MD5('foobar')))? Mathematical statistics to the rescue Gaussian bell-shaped curve for the normal distribution is being calculated Everything inside is considered as “normal”, outside as “not normal”
  • 18.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 18 Delay detection (2) Everything that's normal (i.e. not deliberately delayed) should fit under the curve μ(t) represents a mean, while σ(t) represents a standard deviation of response times 99.99% of normal response times fall under the upper border value μ(t) + 7σ(t)
  • 19.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 19 Delay detection (3) sqlmap learns what's normal and what's not from non-delay based payload responses (e.g. boolean- based blind) Also, few dummy requests are done if there is a need for more
  • 20.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 20 Delay detection (4) In case of considerable network latency user is warned to use as high delay value as possible On the other hand, if negligible, sqlmap optimizes the delay value (if applicable) to a more appropriate one
  • 21.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 21 Finding number of query fields (1) For successful usage of UNION-based technique, number of fields has to be known Foremost method used is ORDER BY clause If response for an arbitrary large value (e.g. id=1 ORDER BY 9664#) drastically differs from neutral (e.g. id=1 ORDER BY 1#) sqlmap uses adapted (chunked) binary search
  • 22.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 22 Finding number of query fields (2) If ORDER BY method is not usable, similar approach is used as in delay detection Constant field value (e.g. NULL) used across a (current chunk of) search space in UNION ALL SELECT statement Searching for the number of fields that, compared to the original response, “sticks out” out of others in a statistical manner Responses for non-valid number of fields will not much differ from each other Response for the right number of fields will appear not normal compared to others
  • 23.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 23 Finding number of query fields (3)
  • 24.
    FSec – FOI2013, Varaždin (Croatia) September 19th, 2013 24 Questions?