Building a benchmark for SQL injection scanners

Intro

In couple of last years we have seen a lot of emerging projects aiming at web application vulnerability analysis automation. That's right, I mean security scanners. Just to name a few: w3af, skipfish, Grendel-Scan, arachni, wapiti, SecuBat, sqlMap, hexjector, SQLiX and many more.

I like to group security scanners according to their feature sets:

• general purpose vs special-purpose (testing for SQLi or XSS only);
• detection only vs detection + exploitation.

Naturally, several good question arise:

1. Is it true that special-purpose tools perform better than general-purpose ones?
2. Is it true that commercial tools perform better than free ones?
3. Is possible to find an ultimate champion for a certain vulnerability class (i.e. SQLi)? If no, which tools would produce the best result when combined together?

Our goal was to answer these question for a specific class of web application vulnerabilities, namely SQLi. The scope:

1. We are not interested in second order SQLi vulnerabilities. Reason: there are no feasible techniques to detect second order SQLi in black box.
2. We are not interested in crawling capabilities of the scanners.
3. We are not interested in measuring exploitation capabilities.

In order to answer the questions we created a benchmark - a comprehensive set of vulnerable and non-vulnerable test cases, and tested several scanners against it.
This is the point you should ask:
- "Why the hell I should care? There was a lot of similar efforts. Do you provide a proof that your test set is complete or something?"¹

In fact yes. Our approach is not an ad hoc. Let's take a look at it.
First of all we will outline the capabilities of the existing SQLi detection techniques. After that we will proceed to the discussion of problems arising during automation. Finally, we will present our approach for benchmarking of SQLi scanners.

Existing techniques for SQLi detections are not complete!

A black-box scanner does not have a lot of information channels to make its decisions about the presence or absence of vulnerabilities:

• HTTP status code;
• HTTP headers (i.e. Location, Set-cookie, etc.);
• HTTP body;
• HTTP response delay;
• out-of-band channels.

According to its definition, SQLi vulnerability is a possibility to alter a syntactic structure of an SQL statement. The main idea behind existing black-box SQLi detection techniques is to inject just that statements which, when evaluated by back-end, would produce a change measurable from outside (alter status code, response body, delay, issue a DNS request, etc).
To sum the thins up, the community has come up with several SQLi detection techniques:

• error-based;
• content-based (or blind);
• time-based;
• out-of-band.

Alas, this set is not complete. Consider a code snippet, which extracts a user-agent value from HTTP request and stores it into back-end DBMS without validation:

$ua = $_SERVER["HTTP_USER_AGENT"];
$rs = mysql_query("SELECT id from user_agents WHERE user_agent = '.$ua.'");
$num_rows = mysql_num_rows($result);
if ($num_rows == 0) {
//insert ua into database
}

Furthermore, all possible exceptions are caught and suppressed and DBMS does not support functions like sleep (MS Access, for instance). [Forget about heavy queries either ;)] This means that we cannot influence from outside neither response itself nor response delay. But this may well be exploitable vulnerability (e.g. consider web-shell installation).
Now after we know that there are certain SQLi instances, which could not be detected by existing techniques in general, it is time to survey a potential for their automation.

Automation challenges

SQLi scanner developers face two main challenges:

1. An injected input should leave the whole query syntactically correct. Thus, a scanner should somehow infer the injection point (e.g. tell a difference between an injection into column-name in a SELECT statement, and an injection after the LIMIT keyword).
2. An algorithm for page comparison should tell apart a true and a false page when doing a blind SQLi. This is an easy task for a human with cognitive comprehension, and almost impossible in general case for a machine. Current web applications may have irregular page structure and produce different content in response to identical http requests (ads, social widgets, etc.), which make the situation even worse.

Our approach

The idea behind our approach is to establish a classification of all possible implementations of a single web application module interacting with a database. After that we would be able to produce all possible modules (or scripts if you like), which would make a test set. Why do we need classification anyways? Well, to reason about test set completeness, of course.

At the current level of abstraction we define five steps which typically constitute a common web application module:

1. Get user input.
2. Validate user input.
3. Construct a query.
4. Perform a query and handle the result.
5. Construct and issue an HTTP response.

We then took a closer look into each step and enumerated all possible ways of its implementation.
Here's what we got.

Classification of environments

Criteria №1: DBMS and version.
Classes: all possible DBMSes with versions.
Reason: a set of features of an SQL dialect is determined by DBMS version. Almost every new major DBMS version is shipped with new built-in functions, which could be used in SQLi attack vectors.
Current implementation: supports only MySQL 5.0.

Criteria №2: exception suppression settings.
Classes: exceptions suppressed, exceptions not suppressed.
Comment: this is an environment setting; thus it does not relate to a possible try-catch block in a module.
Reason: this criteria allows to measure the implementation of error-based technique.
Current implementation: via PHP display_errors setting.

Criteria №3: execution time limit settings.
Classes: limited to 1 second, unlimited.
Reason: this criteria allows to check whether a scanner implements both time-based and blind SQLi detection techniques and is able to determine the correct one to use.
Current implementation: Not implemented. Planned: PHP max_execution_time setting.

Classification of the "Get user input" step

Criteria №4: location of the payload.
Classes: GET-parameter name, GET-parameter value, URL path component, POST-parameter name, POST-parameter value, cookie parameter name, cookie parameter value, header name, header value.
Reason: the scanner should be capable to inject payloads not only into GET and POST parameters, but also into cookies and other headers as well.
Current implementation: only one GET parameter is used to obtain user input.

Classification of the "Validate user input" step

Criteria №5: maximum length enforcement.
Classes: user-defined lengths, unlimited length.
Reason: a scanner should strive to make its probe vectors as tiny as possible.
Current implementation: a test bench operator can alter this limit accordingly. By default is set to 32.

Criteria №6: sanitization approach.
Classes:

proper escaping (prepared statement or built-in escaping function);
improper usage of built-in escaping facilities;
manual escaping:
- of single quotes;
- of double quotes;
- of slashes;
manual removal:
- of single quotes;
- of double quotes;
- of slashes;
- of SQL whitespaces;
- of SQL delimiters;
- of SQL keywords;
manual regexp-like check with error generation:
- of single quotes;
- of double quotes;
- of slashes;
- of SQL whitespaces;
- of SQL delimiters;
- of SQL keywords.

In fact, to obtain all classes from the latter three bullets (manual handling) we should get all permutations of them.
Reason: a perfect scanner should be able to bypass any flawed input validation.
Current implementation: proper escaping (mysql_real_escape_string inside quotes), improper usage of built-in escaping facilities (mysql_real_escape_string for numbers), manual escaping of single quotes, manual escaping of double quotes, manual escaping of all quotes, manual removal of all quotes, manual removal of SQL whitespaces.

Classification of the "Construct a query" step

Criteria №7: An injection point.
Classes: all possible injection points according to DBMS documentation.
Reason: a perfect scanner should be able to detect SQLi in all query types.
Current implementation:

- after the SELECT keyword in the field list:
- inside/without backquotes;
- inside brackets (nesting levels - 1, 2, 3);
- as a last or middle (including first) argument for the SQL function;
- in the WHERE clause:
- in a string/numeric literal;
- inside brackets (nesting levels - 1, 2, 3);
- in the left/right part of a condition ($id=id vs id=$id);
- as a last or middle (including first) argument for the SQL function;
- after the ORDER BY/GROUP BY in the field name
- inside/without backquotes;
- after the ORDER BY/GROUP BY in an expression (ORDER BY `price` * $discount*…) - in a string/numeric literal;
- inside brackets (nesting levels - 1, 2, 3);
- as a last or middle (including first) argument for the SQL function;
- after the ORDER BY/GROUP BY in a sort order ASC/DESC (ORDER BY `price` $style)

Classification of the "Perform a query and handle the result" step

Criteria №8: expected result.
Classes: result is not expected (DML queries), one field expected, one row expected, multiple rows expected.
Reason: an expected result type influences how a scanner should detect a potential SQLi. For example, of query result is discarded it is useless to alter the result set using boolean conditions (blind SQLi).
Current implementation: result is not expected (DML queries), one field expected, one row expected, multiple rows expected.

Criteria №9: error handling.
Classes: error results in a different page (no DBMS error message), error results in a page with DBMS error message, error is suppressed silently.
Reason: scanners should switch to blind or error-based technique accordingly.
Current implementation: implemented.

Classification of the "Construct and issue an HTTP response" step

Criteria №10: which response part depends on the SQL result.
Classes: status code, header, text within body, markup within body.
Reason: a perfect scanner should detect changes after successful injection in any part of HTTP response.
Current implementation: Location header, text within body, markup within body.

Criteria №11: response stability.
Classes: stable, unstable text, unstable initial DOM (before js evaluation), unstable resulting DOM (after js evaluation).
Reason: a perfect scanner should detect changes after successful injection in any part of HTTP response regardless of DOM stability (ads, social widgets, etc.).
Current implementation: stable, unstable text, unstable initial DOM.

The resulting test set would be a complete permutation of the classes. For example, there is among the others a test case with:
C1: MySQL 5.0 backend;
C2: suppressed exceptions;
C3: unlimited execution time;
C4: a test module would get input from the GET parameter value;
C5: without maximum length enforcement;
C6: with improper used mysql_real_escape_string (for numbers);
C7: as a first string argument inside a built-in function after ORDER BY keyword;
C8: one row would be expected;
C9: errors are silently suppressed;
C10: a query result determines a redirection destination (i.e. the value of the Location header);
C11: the response (redirection) is stable except the Date header.

Implementation

We have implemented our test bench as virtual machine containing web server, PHP interpreter, test generator, and result analyzer. It can be downloaded from here.
Currently the generator creates 27680 test cases (both vulnerable and not vulnerable). A test case is a PHP file, which receives one GET-parameter. There's also an index file, which links all test cases. This index file would be a starting point for scanners.
We have also implemented wrappers for several scanners: sqlmap, skipfish, wapiti, w3af. These wrappers with a specially designed scheduler allowed us to run several scanner instances in parallel, from the VM localhost.
Please, referer to the README files inside the archive for further information about the environment and its usage.

Evaluation

Here's just a small portion of the overall table.

	Positives (vulnerable)	False positives (not vulnerable)	Positives with unstable response	False positives with unstable response
Total	16224	11456	2016	840
sqlmap 0.8-1	1128	229	576	229
skipfish 1.81b	10937	37	82	37
wapiti 2.2.1	11151	58	1344	3
acunetix	7395	0	1003	0
w3af 1.0-rc5	13316	198	1572	126

Comments on the numbers:

• The score is not normalized; thus, it is not correct to determine the best scanner as a scanner with the highest score. For example, the most of the test cases are the ones with unsuppressed errors. Thus, a scanner, which would perform best with error-based SQLi detection technique would have a more chances to get the highest overall score. This is the case with Wapiti.
• The main feature of out test bench is that it allows to get scores for custom classes of test cases, once the scanner have done a run. For example, you could define a class "blind SQLi after ORDER BY keyword" and re-compute the results for this class. For the time being I refrained to dig into interpretation of more granular classes of the test cases.

Just a few facts.
Sqlmap 0.8:
- does not detect SQLi with output into HTTP headers;
- lacks intelligence with unstable output;
– fails on tests with SQLi into table fields surrounded in backquotes.

Skipfish 1.81b:
- does not detect SQLi with output into HTTP headers;
- lacks intelligence with unstable output;
- low false positive rate.

Wapiti 2.2.1:
- only error-based and time-based techniques are implemented;
– fails on tests with SQLi into table fields surrounded in backquotes;
– fails on tests with SQLi inside nested brackets.

Acunetix 7.0.0:
- zero false positives;
- often fails to detect SQLi with output into HTTP headers;
- often fails to detect time based SQLi even if it was successful;
- not so good with non error-based techniques.

w3af 1.0-rc5:
- best implementation of error-based technique;
- often fails if the normal query returns zero rows (as with login pages).

Outro

More analytics - in a few weeks.
Care to contribute in extending test classes? YOU ARE WELCOME! Contact us! There's so much to be done!

Side note #1: one of the most comprehensive surveys is one made by Shay Chen. If you haven't checked it out yet - you should!!!

Credits:
Karim Valiev - implementation of benchmark environment, classification.
Andrew Petukhov - main idea and classification.

Detecting Insufficient Access Control in Web Applications

Two weeks ago we attended the 1st SysSec Workshop and DIMVA conference at Amsterdam. We presented there our paper entitled "Detecting Insufficient Access Control in Web Applications". We were surprised to see so many people at the workshop (as far as I'm concerned this workshop received a larger audience than DIMVA itself!).
Well, great events and great people. Many thanks go to the organizers, especially to Herbert Bos, who had made this event happen.

Here comes some stuff we had prepared for the workshop:
- A presentation:

It can be downloaded here.

- A paper "Detecting Insufficient Access Control in Web Applications".

- Source code of our tool is available for checkout at Google Code.

The 1st SysSec Workshop

Our paper "Detecting Insufficient Access Control in Web Applications" was accepted for the First SysSec Workshop.

This work is a follow-up research based on the OWASP Access Control Rules Tester project, which was initiated during the OWASP Summer of Code 2008.

If any of you guys happen to attend DIMVA'11 at Amsterdam, I'd be very glad to meet for a beer :)

Acknowledgements. I'd like to thank George Noseevich, who have put so much effort into this project and this paper.

Comments on the talk made by Rafal Los (HP) about business logic flaws at Blackhat

Recently I've stumbled upon a talk on business logic vulnerabilites (whitepaper available here) made by Rafal Los at BlackHat Europe. Being a guy personally interested in the topic I decided to share my thoughts (rather critical) on the subject. I have divided my thoughts into two categories.

Inaccuracy in wordings and definitions.
1. The definition of business logic flaw is too general to be useful.
"A defect that exposes the component business processes or component flows to manipulation from the attacker perspective to achieve unintended and undesirable consequences from the design perspective; without disrupting the general function or continuity of the application."
Well, almost every application uses a so-called internal state to carry out its operations. In general, every web application workflow is a sequence of computational steps. Each computational step can be guarded by a predicate on the internal state. Hence, modification of internal state can alter a web application workflow by enabling other computational steps or disabling particular ones within it. Thus, every flaw in web application, which could be used to alter the internal state of web application may lead to attacks satisfying the definition presented by Rafal. SQL injection is the best example here.

2. 'Taxonomy' is a wrong word.
The confusion starts from the point where Rafal has introduced the term "taxonomy". Talking about taxonomies is dangerous. One have to prove the properties of the proposed taxonomy or at least try to. Rafal introduces a taxonomy of business logic flaws which consists of two classes. Neither criteria which were used to create those classes, nor a classification procedure were provided.
After all, Rafal is mixing several terms: a vulnerability (which is an enabling property of a software for a successful attack), an attack (one of the possible manifestations of an exploitable flaw) and a goal of an attack (what an attacker wishes to achieve). The same goal could be achieved through different vulns; a single vuln could be exploited for different goals. Rafal ties the first class of business logic flaws to "privilege manipulation", which is a goal. IMO this is wrong approach - this goal could be achieved through different attacks (and underlying vulns): SQLi (login bypass through injection), forceful browsing or even server misconfiguration. How does these relate to business logic?
The same applies to the second class, which is "Transaction Control Manipulation".
Finally, OWASP Top 10 is not a taxonomy and never meant to be. It is just an ordered list.

Doubts related to the proposed ideas
A method intended to find business logic flaws should operate in terms of the business domain and application logic. Hence, the first problem, which should be solved, is mapping of a set of possible interactions with web applications into use cases (which would be the basis for further analysis). It is not possible to automate this step without specification (not to mention technical issues like form submissions & captchas).
Ok, let's suppose we have solved the first problem. Now the second question is: how does the proposed approach differ from the known Q&A methods like fault injection, tests mutations, etc? Could you point out the added value of the proposed approach?

Final words
And in the end I'd like to point out the previous researches in the area, which Rafal hadn't mentioned in his white-paper:
- Differential analysis and its variants. This technique was described in several sources [1], [2], [3] and was designed for finding improper access control implementations. This means that from the methodological point of view the problem of detecting improper access control is already solved. Btw, some of these ideas were implemented in IBM AppScan, AFAIK.
- "Toward Automated Detection of Logic Vulnerabilities in Web Applications". A good example of how business logic flaws could be formalized. Not a black-box approach though.
- "Multi-Module Vulnerability Analysis of Web-based Applications". An example of how a notion of "intended workflow" could be formalized. Not a black-box approach though.

References
[1] J. Scambray and M. Shema, Hacking exposed: Web applications. McGraw-Hill Osborne Media, 2002.
[2] O. Segal, “Automated testing of privilege escalation in web applications,” Watchfire, 2006.
[3] D. Stuttard and M. Pinto, The Web Application Hacker’s Handbook: Discovering and Exploiting Security Flaws. Wiley Publishsing, 2007.

ruCTF'2011 Quals

Well, our CTF team Bushwhackers took the first place in the ruCTF'2011 Qualification game. I'd like to congratulate all the team members with this victory. Wishing the same luck in the Final.

I'd also like to thank the organization team for their efforts.

Here's some evidences of our team at work:

Web application scanner comparison efforts

It's been three months since we started a project, which aims at benchmarking SQLI scanners. Although our project is far from the finish, I've decided to share articles and postings by other researches who had undertaken similar efforts. Publications are sorted in order of appearance.

1. Andreas Wiegenstein, Frederik Weidemann, Dr. Markus Schumacher, Sebastian Schinzel. Web Application Vulnerability Scanners – a Benchmark. Published in October 2006.


2. Larry Suto. Analyzing the Effectiveness and Coverage of Web Application Security Scanners. Published in October 2007. And responses to it by Ory Segal (IBM) and by Jeff Forristal (HP).


3. Anantasec. Web Application Scanners Comparison. Published in January 2007.
   


4. Larry Suto. Analyzing the Accuracy and Time Costs of Web Application Security Scanners. And responses to it by Acunetix, NT Objectives, Jeremiah Grossman and HP. Published in February 2010.
   


5. Jason Bau, Elie Bursztein, Divij Gupta, John Mitchell. State of the Art: Automated Black-Box Web Application Vulnerability Testing. Published in May 2010.
   


6. Adam Doupe, Marco Cova, and Giovanni Vigna. Why Johnny Can’t Pentest: An Analysis of Black-box Web Vulnerability Scanners. Published in July 2010.
   


7. Shay Chen. Web Application Scanners Accuracy Assessment. Published in December 2010.
   

Out of list, but still related work.

• OWASP SQLiBench Project: a comparison of features for SQLI scanners. There was no effort to measure efficiency (precision, coverage) of features.


• Web Application Security Scanner Evaluation Criteria.


• OWASP Web Application Scanner Specification Project.

Deutsche Post Security Cup

Recently our team Bushwhackers participated in the Deutsche Post Security Cup.

The Cup Results are as follows:
- the first place took the RUB team from Ruhr University Bochum led by famous Mario Heiderich (see also FluxFingers team); I would recommend to check a site created and maintained by him - http://html5sec.org/. Congratulations! Good job, guys!
- we took the second place.
- the third place took UK Hax team.

I would like to thank Security Cup organization team including (but not limited to) Karsten Nohl, Ralph Zwierzina and personally Sascha May. Great job! See you in Moscow ;)

Enhancing black-box scanning with dynamic analysis

As its very name suggests a black-box scanner doesn't know how each input http parameter is handled by web application. From an attacker's perspective for each input parameter we would like to know:
- the list of sensitive operations, which are called with arguments derived from the input value (whether parameter "name" is passed to echo, to mysql_query or to both?)
- the syntactic structure of sensitive queries
- URLs, which return http-response with input parameter processing results (there we will check for evidences of successful exploitation).
In general case it is impossible to precisely derive this kind of information with black-box analysis. That's why in order to demonstrate good results black-box scanners have to:
- sequentially inject in each input parameter attacks of every type (XSS, SQLI, command injection, XPath injection, RFI, and many more)
- within a given attack type (e.g. SQLI) we have to guess the control characters (for SQL - quotes, number of brackets, etc), which we need to prepend to the injected attack vector so that the resulting query is left syntactically correct. * assuming web application handles exceptions properly *
- in order to detect multi-module vulnerabilities (which make second order attacks possible) we have to crawl every web application interface after each injection. Indeed, in general case we do not know which web application interface contains the results of poisoned parameter processing.

Let us call this way of testing - "undirected fuzzing". If we just could gain data indicated in the first list above we could perform a "directed fuzzing". We would inject in each input parameter only those attack vectors that are relevant to the processing path of the data. Besides, we would inject only such attack vectors that leave queries generated by web application syntactically correct. As such, we might focus on filter bypassing techniques. Finally, after each injection we will know which URLs to check for evidences of successful exploitation.

And all this kind of useful information is possible to gather by means of dynamic analysis. Let us suppose, we have two components: a scanner and a server-side module (implemented either as a part of an interpreter (Python, Perl, PHP, Ruby) or as a web application instrumentation/hooks (.NET, Java)). Here's one of the possible workflows:
1. A scanner component performs crawling with form submission and authentication. At the same time server-side module builds traces of web application execution for each http-request.
2. Once crawling completes, execution traces are sent to the scanner for further analysis. At this stage scanner builds dependence graphs and analyses them. The goal is to infer:
- the list of sensitive operations, which are called with arguments derived from the input value
- the syntactic structure of sensitive queries
- URLs, which return http-response that show input parameter processing results.
This information is used to create a plan for directed fuzzing.
3. Scanner performs directed fuzzing, while server-side module collects execution traces. Once fuzzing completes, collected traces are sent to the scanner for the final analysis. Scanner uses different checks to detect whether each attack was successful:
- response analysis (good for XSS detection);
- parse tree analysis (see explanation here);
- control flow analysis (detect exceptional conditions if any).

This is just an illustration of how one could leverage dynamic analysis to substantially enhance black-box scanning results (and time!).

I hope you have found this blog post useful and I'm always interested in hearing any feedback you have.

The need for differentiation of filter routines in multi-module taint analysis

Last time we talked about three limitations of the taint analysis vulnerability model. The first two of those limitations can be overcome by introduction of classes for tainted data and by tracking data dependencies through persistent storage (e.g. database).
Let us assume that we have implemented such multi-module class-aware taint analysis. Let us consider the following example of the inter-module data dependency:

addpost.php
1: $text = $_GET['text'];
2: $text = addslashes($text);
3: $author = $_GET['author'];
4: $author = addslashes($author);
5: mysql_connect();
6: mysql_select_db("myDB");
7: $query = "INSERT INTO posts (author, text) VALUES ('".$author."', '".$text."')";
8: mysql_query($query);
// do redirect

getstats.php
1: mysql_connect();
2: mysql_select_db("myDB");
3: $result = mysql_query("SELECT author FROM posts ORDER by author");
4: while ($row = mysql_fetch_assoc($result)) {
5: $author = $row['author'];
6: $count = mysql_query("SELECT count(text) FROM posts WHERE author = '".$author."'");
7: $row = mysql_fetch_array($count)
8: echo "Author ".$author." has ".$row[0]." posts; ";
}
//suboptimal schema, suboptimal queries, I know...

Let us apply taint analysis to this code. There are several paths that input data take to reach critical operations. Let us consider one of them:

addpost.php:3
$author = $_GET['author'];
taint analysis: data is received from the untrusted source, so we will associate all taint classes with it; $author is marked with (sqli, xss, osi, rfi).

addpost.php:4
$author = addslashes($author);
taint analysis: data is passed through filter function, so we will remove corresponding taint flag from it; $author is marked with (xss, osi, rfi).

addpost.php:7
$query = "INSERT INTO posts (author, text) VALUES ('".$author."', '".$text."')";
taint analysis: data is used to initialize another variable, so we will copy taint flags as well; $query is marked with (xss, osi, rfi).

addpost.php:8
mysql_query($query);
taint analysis: critical operation receives data with flags (xss, osi, rfi). These flags do not contain sqli flag, so everything is fine, no vulnerability. Also, database field posts.author is marked to have a dependency upon variable $author.

getstats.php:3
$result = mysql_query("SELECT author FROM posts ORDER by author");
taint analysis: critical operation is called with constant value, everything is fine. Also, variable $result is marked to have a dependence upon database field posts.author and, consequently, upon variable $author from module addpost.php.

getstats.php:4
while ($row = mysql_fetch_assoc($result)) {
taint analysis: Variable $row is marked to have a dependence upon variable $result and, consequently, upon variable $author from module addpost.php.

getstats.php:5
$author = $row['author'];
taint analysis: Variable $author is initialized from database field, which depends upon variable $author from module addpost.php. Thus, local variable $author is associated with taint flags (xss, osi, rfi).

getstats.php:6
$count = mysql_query("SELECT count(text) FROM posts WHERE author = '".$author."'");
taint analysis: critical operation receives data with flags (xss, osi, rfi). These flags do not contain sqli flag, so everything is fine, no vulnerability.

getstats.php:7
$row = mysql_fetch_array($count)
taint analysis: not interesting

getstats.php:8
echo "Author ".$author." has ".$row[0]." posts; ";
taint analysis: critical operation is called with flags (xss, osi, rfi). These flags do contain xss flag, so vulnerability (Stored XSS) is detected.

Alas, we missed a multi-module (second order) SQL injection vulnerability. This happened because taint analysis by default does not take care of the semantics of filter functions. Indeed, if we analyze methods of input validation, we might come to the 5 types of it:
1. Typecasting.
2. Encoding.
3. Removing of bad chars/keywords.
4. Escaping.
5. Branching. Cannot be addressed by taint analysis (see limitation #3).
The first three types of validation remove injection pattern once and for all. This mean that the common workflow get input -> validate input -> store in database -> get from database -> use in critical operation will not contain vulnerability.
But escaping of database queries is different. Escaping removes injection pattern only for the first query. In database data is stored as-is and consequently this data will contain injection pattern when retrieved from database. Hence, the workflow get input -> escape input -> store in database -> get from database -> use in sql query will contain a multi-module vulnerability.
As such, multi-module taint analysis should take account for the types of filter functions used in web applications, and assign taint flags accordingly.

Limitations of taint propagation vulnerability model

In the previous post we discussed the two most popular vulnerability models used by static and runtime analyzers to detect security flaws in web applications. At this time I decided to discuss some limitations of the models. I'd like to emphasize that these limitations come from the assumptions and definitions made at the basis of the models. Thus we will not consider limitations imposed by the approach (static or dynamic analysis) chosen to implement a model.

Let us start from the taint analysis vulnerability model. Here is the vulnerability definition used in the model.
1. All data originating from web application users is untrusted. To track this data most analyzers associate a special "taint" mark with it.
2. All local data (file system, database, etc.) is trusted (we do not want to address local threats).
3. Untrusted data can be made trusted through special kinds of processing, which we will call sanitization. Thus, if analyzer detects that certain data marked with "taint" flag is passed through a sanitization routine, the flag will be removed.
4. Untrusted data cannot reach security critical operations like database queries, HTTP response generation, evals, etc. The violation of this rule is called a vulnerability.

A security analyst who is going to use this model will be required to:
1. Compile a list of language constructs that return user input. In some technologies (i.e. PHP) these constructs are built-in, in other technologies (Python) they are framework-dependent (consider obtaining HTTP parameters in mod_python vs WSGI).
2. Compile a list of sanitization constructs (built-in, 3rd-party libraries, or even implemented by web application developer).
3. Compile a list of critical operations (these are mostly built-in).
This becomes a configuration for the taint analysis.

Let us point out some limitations of the model.
1. The fisrt is a minor limitation, which can be simply overcome. The basic definition does not support classes of untrusted data. This means that the following code snippet will not produce a vulnerability warning:

$skip = $_GET['skip'];
$skip = htmlspecialchars($skip);
mysql_connect();
mysql_select_db("myDB");
$query = "SELECT text FROM news LIMIT 10 OFFSET ".$skip;
$result = mysql_query($query);

Indeed, htmlspecialchars should be listed as a sanitization routine, so the "taint" flag associated with it at line 1 will be removed. But the the program does have an SQL injection vulnerability. This undetected vulnerability adds to false negatives and affects completeness of the analysis.
The limitation can be overcome by introduction of "taint" classes: "SQLI-untrusted", "XSS-untrusted", "Shell-injection untrusted", etc.

2. The second limitation lies within the assumption that all local data is trusted. This results in inability to detect multi-module vulnerabilities (aka second order injections). Let us consider the following example:

addpost.php
$text = $_GET['text'];
$text = addslashes($text);
mysql_connect();
mysql_select_db("myDB");
$query = "INSERT INTO posts (text) VALUES ('".$text."')";
mysql_query($query);
// do redirect

viewpost.php
$skip = $_GET['skip'] + 0;
mysql_connect();
mysql_select_db("myDB");
$query = "SELECT text FROM posts LIMIT 10 OFFSET ".$skip;
$result = mysql_query($query);
while ($row = mysql_fetch_assoc($result)) {
   echo $row['text'];
}

This code demonstrates the simplest stored XSS vulnerability. However, due to the second assumption of the model, the analyzer will treat text data returned from the database as trusted. This limitation also leads to undetected vulnerabilities and affects the completeness of the analysis.
The limitation can be overcome by introduction of inter-module data dependency analysis. This approach was described in some papers:

[2007] Multi-Module Vulnerability Analysis of Web-based Applications
[2008] Detecting Security Vulnerabilities in Web Applications Using Dynamic Analysis with Penetration Testing

It is likely that most commercial static analyzers have adopted this approach by now.

3. The third limitation lies within the third rule, which states that sanitization should be performed through special routines that always return "good" data. But what about input validation through conditioning? This limitation cannot be overcome without integration with other vulnerability models. Let us consider the following example:

$email = $_GET['email'];
$valid_email_pattern = "..."; //complex reg exp from RFC 822
if (preg_match($valid_email_pattern, $email)) {
   // do processing
} else {
   echo "You have entered an invalid email address: ".$email; //XSS
   exit;
}

It is unclear how to "untaint" variables, which are sanitized via such checks. In the code above variable $email should be "untainted" if the call to preg_match returns true. Could we use this as a rule to untaint all variables passed through preg_match? Obviously no! Let's take a look at the other example:

$email = $_GET['email'];
$invalid_email_pattern = "..."; //negated complex reg exp from RFC 822
if (preg_match($invalid_email_pattern, $email)) {
   echo "You have entered an invalid email address: ".$email; //XSS
   exit;
} else {
   // do processing
}

So, in general we cannot determine in which branch we should "untaint" variables validated via conditional statements. Let us examine how analyzers could handle this issue. Basically, there are two options:
- preserve "taint" flag in both branches. This leads to false positives and affects precision of the analysis.
- remove "taint" flag from both branches. This leads to false negatives and affects completeness of the analysis.

4. The last minor drawback is the implicit trust laid on sanitization routines and on operator compiling the configuration. If any sanitization routine contains an error (i.e. is incomplete, does not perform normalization, susceptible to bypassing techniques, etc.) the inherent vulnerability will not be detected. Also, analysis can become either incomplete or imprecise if configuration lists with sanitization, input and critical routines were compiled with errors/omissions.

I hope you have found this blog post useful and I’m always interested in hearing any feedback you have.

p.s. In the next post we will discuss limitations of the other vulnerability model, that is parse-tree model.

Vulnerability models in web applications

I've seen much confusion in understanding and using such terms as "taint propagation", "static analysis for vulnerabilities", "dynamic analysis". So I decided to clarify some important thing here (particularly with respect to web applications).
First of all, we should distinguish a model of program behavior and a model of vulnerability.
Every model of program behavior introduces its own terms for behavior specification. Depending on the level of abstraction, those terms could specify low-level (i.e. how program uses CPU registers) or high-level (i.e. dataflow dependencies) behavior.
The trivial models of program behavior are specifications in terms of source and byte- or binary code. Here are some other popular models: CFG (Control flow graph), various dependence graphs, program slices and pre- and postconditions, a set of possible values for each variable at every program point (hello, so-called string analysis).
What is most important is that these models have nothing to do with static or dynamic analysis. Static and dynamic analysis are the means to construct the model of program behavior. Of course, static analysis tends to yield complete but imprecise models while dynamic analysis yields precise but incomplete models. You can find concepts of both static and dynamic slices; there are concepts of pre and post conditions in static and runtime.

Now let us move to vulnerability models. Every vulnerability model is tightly bound to the model of program behavior. Indeed, the existence of vulnerability should be specified in terms of program behavior!
There is widely accepted model of vulnerabilities at the time - the non-interference model and its extensions. In simple words it tells us that untrusted input should not interfere (e.g. change the intended behavior) with critical operations. Okey, we have this requirement, so how do we check it in our program?

The simplest way is to follow the very definition of non-interference: let us mark untrusted input as taint source and critical operations as taint sinks. Now if a taint sink depends on a taint source, and input data is not sanitized (i.e. made trusted) we shall raise a vulnerability warning. We can see that this re-formulation of non-interference is perfectly expressed in terms of dependencies. Hence, automated tools that utilize this definition should be able to make program slices and/or build dependence graphs (does not matter in runtime or in static).

A more subtle (and in my opinion beautiful) way to express interference formally was proposed in early 2000s. Let us survey what SQL (and any other) injection is. It's when a user can mix control and data channels of a query, or in other terms, change its syntactic structure. So, the definition of non-interference could be re-formulated as follows: we should raise a vulnerability warning if syntactic structure of a critical query depends on the user input. We can see that this re-formulation of non-interference is perfectly expressed in terms of parse trees of string values, which program variables may hold at every execution point. Hence, automated tools that utilize this definition should be able to perform string analysis in static (at runtime there is no difficulty to obtain the values of the needed variable).

And finally, we can see now that taint propagation is not a method to detect vulnerabilities. Taint propagation is one of the means to determine data dependencies at runtime.

Some references:
Basic dependency model (aka taint propagation)
-Static
[2006] Pixy - Technical Report
[2006] Static Detection of Security Vulnerabilities in Scripting Languages
[2009] TAJ Effective Taint Analysis of Web Applications

-Runtime
[2004] Analysis of Perl Taint Mode
[2005] Automatically hardening web applications using precise tainting
[2005] Dynamic Taint Propagation for Java

Parse tree validation model:
-Static
[2005] Static Approximation of Dynamically Generated Web Pages
[2007] Sound and precise analysis of web applications for injection vulnerabilities

-Runtime
[2005] Combining Static Analysis and Runtime Monitoring to Counter SQL-Injection Attacks
[2005] Using Parse Tree Validation to Prevent SQL Injection Attacks
[2006] The Essence of Command Injection Attacks in Web Applications
[2006] Using Positive Tainting and SyntaxAware Evaluation to Counter SQL Injection Attacks

In the next post we will discuss some limitations of these models. Stay tuned!

Bushwhackers won bronze at RusCrypto CTF!

Congratulations! The team of students from our «Information Security» profile workshop at Moscow State University won bronze at RusCrypto CTF.
The final standings are:
1. CIT (Saint Petersburg State University of Information Technologies, Mechanics and Optics)
2. HackerDom (Ural State University)
3. Bushwhackers (Moscow State University)
4. SiBears (Tomsk State University)
5. [Censored] (Immanuel Kant State University of Russia)
6. Huge Ego Team (Moscow Engineering Physics Institute)

Bushwhackers qualified for two major CTF events

Congratulations! The team of students from our «Information Security» profile workshop at Moscow State University qualified^{1, 2} for ruCTF-2010 and RusCrypto CTF.

Thoughts on building a taxonomy of vulnerabilities, pt.1

Classifications, taxonomies... Web security...
I decided to make a number of posts discussing these topics.
The important questions are:
- what is a classification; what is a taxonomy;
- what to classify: flaws or vulnerabilities; and what for;
- what are the existing taxonomies/classifications;
- why it is easier to build a taxonomuy of attacks/threats than of flaws or vulnerabilities;
- finally, is it possible to build a useful taxonomy of web application vulnerabilities.

First of all, I'd like to introduce the notion of taxonomy, just to make sure those readers that might come across these posts are synchronized with me.
In common life the terms taxonomy and classification are usually used interchangeably. Further, often by word “classification” people refer to some process of breaking the set of items into groups. Let us introduce some order here.
Marradi (“Classification, typology, taxonomy”) defines the process of classification as one of the following:

• an intellectual operation, whereby the extension of a concept at a given level of generality is subdivided into several narrower extensions corresponding to as many concepts at a lower level of generality;


• an operation whereby the objects or events in a given set are divided into two or more subsets according to the perceived similarities of one or several properties; or


• an operation whereby objects or events are assigned to classes or types that have been previously defined.

As we see, there are two approaches to building a classification. I will call them a model-based (from the general to the particular, or some might say deductive) approach and an instance-based (from the particular to the general, or some might say inductive) approach respectively.
Ok, what's then a taxonomy anyways?
According to Simpson (“Principles of animal taxonomy”), a taxonomy is a “classification, including bases, principles, procedures and rules”. Thus, a taxonomy is more than a classification – it states the principles, according to which the classification is done, and procedures to be followed in order to classify new objects.
Taxonomies should have taxonomic categories with the following characteristics:

1. Mutually exclusive: the categories do not overlap.


2. Exhaustive: taken together, the categories include all the possibilities.


3. Unambiguous: clear and precise so that classification is not uncertain, regardless of who is classifying.


4. Repeatable: repeated applications result in the same classification, regardless of who is classifying.


5. Accepted: logical and intuitive so that categories could become generally approved.


6. Useful: could be used to gain insight into the field of inquiry.

As the authors of “A Vulnerability Taxonomy Methodology applied to the Web Services” point out, there is no such thing as the ultimate taxonomy. Rather, each taxonomy is designed for the specific intended usage. Hence, the value of each taxonomy and its usefulness should be considered along with the viewpoint and the scope that the authors thereof had intended. Moreover, the authors of the certain taxonomy should explicitly state the intended usage, the scope and the viewpoint thereof.

In the next post I am going to say a few words on taxonomies in application to web security. The important questions is: what to classify: flaws or vulnerabilities? And what for?

NULL pointer dereference flaw -> vulnerability

NULL pointer dereference is a common implementation-time flaw. Sometimes this flaw becomes a vulnerability. Strangely, but this is because of a bad design.


To illustrate this, let us consider a network application. Suppose, it dereferences a NULL pointer under certain circumstances. In order for this flaw to become a vulnerability, a single user should have an opportunity to interrupt application services exploiting NULL pointer dereference. Obviously, this is a bad design decision.


This is the way implementation-time flaw becomes a design-time vulnerability.

Hello, World!

I am going to post here some thoughts on web application security and oil prices (ha-ha). All comments are really welcome!