This document discusses security test automation. It provides examples of unit tests, static code analysis, and dynamic code analysis to test for vulnerabilities like XSS, input validation, TLS configuration, and known library vulnerabilities. Testing HTTP interactions, TLS settings, and library dependencies can be done programmatically through tools like RestAssured, SSL Labs, and OWASP Dependency Check. Automating security tests is important for projects to help ensure requirements are met continuously.

1. Security Test Automation
Marek Puchalski
marek.puchalski@capgemini.com
marek.puchalski@owasp.org
@marek_devsec

2. About me

3. Table of Contents
• Security in projects
• OWASP ASVS
• Examples:
– Unit tests
– Static code analysis
– Dynamic code analysis Sorry, maybe next time…
Only a bit

4. SECURITY IN PROJECTS

5. http://www.michaelmolloy.co.uk/construction-photography.html

6. Waterfall

7. http://www.letgoyourmind.com/wp-content/uploads/2017/03/buildingtall.jpg

8. Agile

9. Security in Waterfall

10. Security in Agile

11. Best idea so far
Automate security tests

12. OWASP ASVS

13. OWASP Application Security
Verification Standard (ASVS)
• Provides a list of requirements for secure
development
• Defines different security assurance levels
(Opportunistic, Standard, Advanced, also
called Level 1, 2, 3)

14. Example

15. # of requirements

16. EXAMPLES

17. Example 1, ASVS 11.8
Verify that the X-XSS-Protection: 1;
mode=block header is in place.
„
„

18. HTTP Communication Example
GET http://oasp-ci.cloudapp.net/oasp4j-
sample/services/rest/offermanagement/v1/offer HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:37.0)
Gecko/20100101 Firefox/37.0
Accept: application/json, text/plain, */*
Accept-Language: en-US,en;q=0.5
X-CSRF-TOKEN: fcbfc729-15d2-4f04-8e50-082f20cb2dfb
Referer: http://oasp-ci.cloudapp.net/oasp4j-
sample/jsclient/
Cookie: JSESSIONID=F340544E6AE9078812ECF61139D03C7B
Connection: keep-alive
Host: oasp-ci.cloudapp.net
HTTP request
HTTP/1.1 200 OK
Date: Sat, 11 Jul 2015 20:28:36 GMT
Server: Apache-Coyote/1.1
Content-Type: application/json;charset=UTF-8
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
[{"id":1,"modificationCounter":1,"revision":null,"name":null,"
description":"Schnitzel-
Menü","number":null,"mealId":1,"drinkId":10,"sideDishId":5,"
state":"NORMAL","price":"6.99"},{"id":2,"modificationCounte
r":1, (…)
HTTP response

19. Why do we have to deal with HTTP?
• It’s a trust boundary between the client and
the server
• It offers maximu flexibility by allowing request
manupulation on the text/byte level
• One can fabricate request the client side of
application would never generate
• This is what the hackers are doing :)

20. What does X-XSS-Protection do?
• Offers (reflected) XSS protection
• Turned on by default, but works in the
sanitization mode
• Turn the most rigorous mode on over X-XSS-
Protection: 1; mode=block

21. Preferred type of test
Source: https://blogs.msdn.microsoft.com/visualstudioalmrangers/2017/04/20/set-up-a-cicd-pipeline-to-run-automated-tests-efficiently/

22. Code
import static io.restassured.RestAssured.*;
(...)
when()
.get("https://haveibeenpwned.com/")
.then()
.statusCode(200)
.header("X-XSS-Protection", "1; mode=block");

23. Example 2, ASVS 5.5
Verify that input validation routines are
enforced on the server side.
„
„

24. Tested application
Source: http://demoqa.com/contact/

25. Request

26. Response

27. Code
import static io.restassured.RestAssured.*;
(...)
String body = TEMPLATE.replace("<name>", "Marek")
.replace("<email>", "marek@test");
given(new RequestSpecBuilder()
.addHeader("X-Requested-With", "XMLHttpRequest")
.addHeader("Accept", "application/json, text/javascript, */*; q=0.01")
.addHeader("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8")
.setBody(body)
.build()).
when()
.post("http://demoqa.com/contact/").
then()
.statusCode(200)
.body("mailSent", equalTo(false),
"invalids[0].message", equalTo("Email address seems invalid."));

28. Wait! Is this really this easy?

29. I would not call that „easy”
• Understanding security requirements takes time
• We need to deal with traffic on the HTTP level
• Some technologies are easier to automate than
others
• We didn't show how to deal with authentication
or CSRF protection
• But yes, in many cases the code can still be sexy!

30. Example 3, ASVS 10.16
Verify that the TLS settings are in line with
current leading practice, particularly as
common configurations, ciphers, and
algorithms become insecure.
„
„

31. What is TLS?
• It’s the „S” in HTTPS ;)
• It’s actually much more than this, but let’s not
complicate things, because…

32. Understanding TLS is hard
• Understand: PKI, CA, MAC, OCSP, CSR, cipher
suites, ...
• Use: RSA, ECDHE, DHE, AES, GCM, CBC, SHA,...
• Don't use: MD5, RC4, SSL, ...
• Prevent: Drown, BEAST, padding oracle,
CRIME, TIME, BREACH, Heartbleed,
DUAL_EC_DRBG, ...

33. How to deal with it?
Source: https://www.ssllabs.com/ssltest/

34. Code
import io.beekeeper.ssllabs.junit.BaseSSLLabsTest;
@RunWith(Parameterized.class)
public class AppTest extends BaseSSLLabsTest
{
public AppTest(String host) {
super(host);
}
@Parameters(name = "Host: {0}")
public static Iterable<String> data() {
return Arrays.asList("marek.puchal.ski", "www.poczta-polska.pl");
}
}
Source: https://github.com/beekpr/ssllabs

35. Example 4, ASVS 1.11
Verify that all application components, libraries,
modules, frameworks, platform, and operating
systems are free from known vulnerabilities.
„
„

36. Case Equifax
(STRUTS 2, CVE-2017-5638)
BTW: Struts 2 had 15 known vulnerabilities in 2016
Source: https://www.imperva.com/blog/2017/03/cve-2017-5638-new-remote-code-execution-rce-vulnerability-in-apache-struts-2/

37. How to deal with it?
• Update every library every release
• Or use a library scanning tool
– OWASP Dependency Check
– Victims
– Black Duck (Copilot)
– Many other

38. Code
.bindependency-check.bat --project
victim --scan victim*

39. OWASP Dependency Check

40. Static code analysis
public List<String> find(String mode) {
String sql = "select text from portfolio where mode = '" + mode + "'";
RowMapper<String> mapper = new RowMapper<String>() {
@Override
public String mapRow(ResultSet rs, int arg1) throws SQLException {
return rs.getString("text");
}
};
List<String> result = jdbcTemplate.query(sql, new Object[] {}, mapper);
return result;
}

41. Static Application Security Testing
(SAST)
Pros
• Good at searching after
certain weakness
categories
Cons
• Commercial tools are
very expensive
• Many false positives

42. Challenge
• Look at the tool list
https://www.owasp.org/index.php/Static_Cod
e_Analysis#OWASP_Tools
• Find the tool for your langauge to try out
• Go for it!

43. Summary
• First step to security assurance - know what is
to be done
• Don't fear HTTP – test implementation is not
necessary hard
• You can even deal with „special cases” like TLS
validation and software composition analysis
on the code level

44. QUESTIONS?
marek.puchalski@capgemini.com
marek.puchalski@owasp.org
@marek_devsec