Test-driven security involves writing security-focused test cases to test for vulnerabilities during the development process. This helps enable continuous deployment by ensuring new code does not introduce security bugs. The key aspects discussed are: 1) Having developers or security experts write test cases to validate common vulnerabilities like authentication failures, input validation, and authorization checks. 2) Involving non-technical team members like project managers in writing test cases using plain language to specify scenarios. 3) Integrating security testing into continuous integration pipelines to automatically catch issues during code reviews.

1. Test-Driven Security
Louis Nyffenegger
@snyff

2. About Me
• Security consultant working for Securus Global in
Melbourne
• 2 side projects:
– PentesterLab (.com): cool (web) training
– PNTSTR (.com): easy first round for interviews
• And today… I’m going to talk about Secure
Development… in a way ;)

3. Too often
when people talk about
secure development
they explain…

4. How most people do it…
Security?

5. How you should do it…
Security?

6. Agile??

7. Agile
Agile software development is a group of software
development methods based on iterative and incremental
development, where requirements and solutions evolve
through collaboration between self-organizing, cross-
functional teams. It promotes adaptive planning,
evolutionary development and delivery, a time-boxed
iterative approach, and encourages rapid and flexible
response to change. It is a conceptual framework that
promotes foreseen interactions throughout the
development cycle.

8. TL; DR;
• Projects evolved with clients’ needs, not based on
project managers’ fantasy ;)
• No formal list of functionality
• New code is push to production “all the time”
– Etsy: 20 times a day
• No predefined milestones

9. WHAT???

10. But how can people deploy all the
time?
 four-leafed clover and rabbit's foot on each
production servers
 Magic
 Super awesome developers who don’t do any
mistakes
 Coverage of everything using tests and all tests are
run before every push to production

11. But how can people deploy all the
time?
 four-leafed clover and rabbit's foot on each
production servers
 Magic
 Super awesome developers who don’t do any
mistakes
 Coverage of everything using tests and all tests are
run before every push to production

12. Example of tests
def test_can_see_exercises
get "/exercises"
assert last_response.status == 200
end
def test_can_access_login
get "/login“
assert last_response.body =~ /login/
assert last_response.body =~ /password/
assert last_response.body =~ /email/
end

13. Example of tests… more
• Some people even create test libraries that use plain
English:
Scenario: Regular numbers
* I have entered 3 into the calculator
* I have entered 2 into the calculator
* I press divide
* the result should be 1.5 on the screen/
• And a developer writes the logic behind each line
Given /I have entered (d+) into the calculator/ do |n|
@calc.push n.to_i
end

14. It can even be FUN

15. Summary
• Everyone can write test cases
• When a bug is found, a dedicated test is written…
-> A bug can only appears once
• New code can be deployed really quick
• All test cases written will be checked before each
push to production

16. As a security person, I can only say
one thing
10 points for
Gryffindor

17. Back to security… agenda()
• Test-Driven Security
• Create security champion
• Get other people to write test cases
• Pair programming/Peer review
• Continuous integration

18. Current test cases
• A lot of security related functions are tested:
• A user can log in ?
• A user can change his password?
• A user can access his profile
• But I never, ever see things like:
• A user can’t log in with an invalid password
• A user can’t log in with an empty password
• A user can’t log in without password
• A user can’t access other users’ profile

19. Functions needed
def login(user,password)
creds = { :email => user,
:password => password }
post("/login", creds)
end
def assert_redirect_to_login
assert last_response.header["Location"] =~ //login$/
assert last_response.status == 302
end

20. Functions needed
def test_cannot_secret_without_login
get "/secret"
assert_redirect_to_login
end
def test_cannot_login_with_blank_password
login("louis@pentesterlab.com", "")
assert_redirect_to_login
end

21. Functions needed
def test_cannot_login_with_wrong_password
login("louis@pentesterlab.com", "wrong")
assert_redirect_to_login
End
def test_logout_on_access_other_users_stuff
login("louis@pentesterlab.com", “password")
get "/other_users_stuff"
assert_redirect_to_login
End

22. It’s pretty simple and
straightforward, but not many
people are doing it :/
You can even go further…
and create more security checks

23. More test cases
• When I put a single quote in a field
• Do I get an error
• If it’s echoed back in the page, is it encoded?
• Same for ‘<‘
• Same for ‘>’
• Same for ‘”’
• If the application uses files, what happens if I put
“../” in the file path

24. But to do that you need developers
with security training…

25. Not necessarily,
Half of the test cases should be
based on business logic…
Modern frameworks take care of
the other half.
But it’s always good to have some
security champions.

26. FIRST RECIPE
• Steps:
• Take a developer
• Teach him everything about security: Top 10,
Detection, Exploitation, …
• Put him back in the development team
• Pros:
• You have now a good security person
• Cons:
• Likely to go away to do pentesting

27. SECOND RECIPE
• Steps:
• Take a developer
• Teach him how to detect potential bugs
• Put him back in the development team
• Pro:
• You don’t have a wannabe hacker in your team
• You have someone who can find and fix bugs
quickly
• Cons:
• The training was probably less interesting

28. Detecting potential bugs?
• Forget everything you know about security
• Aside from business logic bugs… most security issues
are based on: “Breaking the syntax”
• XSS: breaking JS or HTML syntax
• Code injection: breaking code syntax
• SQL injection: breaking SQL syntax
• …
• You just need to explain that correctly

29. Get non-devs involved
• Project managers:
• They are close to the business
• They can now write test cases in plain English
• Security people:
• Most of them should be able to write test cases
• They understand security
• Every time a bug is found they can write a test
case to make sure it will never happen again

30. As a process…
• Perform sensibility training when the project starts:
• To avoid things like SQL built on the client side
• Introduction to test driven security
• Architecture review (SSL, Session mgmt…)
• If you perform penetration test, write issues as new
test cases…
• Get a security person to review the “security test
cases”
• Get a project manager to review the “business logic”
security checks

31. Peer review
• Pair programming and security:
• junior/senior team
• dev/security team
• Peer review and security:
• Bug spotted earlier
• With modern versioning system (ie: git > 1.7.9),
you can even sign commits:

32. Continuous integration
• You can automatically setup code review tools to
scan your application
• You can automatically setup (free) web scanners to
scan your application
• Cons:
• Lot of time spent setting that up
• Need to filter all possible false positive
• Pros:
• Sleep like a baby

33. Good news

34. Limitations
• Production vs Testing
• You can’t prevent things like:
• Weak crypto
• Weak PRNG
• Cookies related issues (“user=admin”)
• Or can you?
• More testing…
• This is when security people should start writing
test cases.

35. Conclusion
• No rocket science here…
… Just simple things to test
• If your developers don’t use tests… I guess you have
other problems than security to take care of :/
• Reliable and simple way to increase your
applications’ security

36. Questions?