Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continuous Security Testing

866 views

Published on

Using Acceptance Test Driven Approach to Security Test, easy for stakeholder involvement, and a whole team approach to address security needs in software development process.

Examples in this session are based on RobotFramework, making it easy for people with limited programming skills.

Published in: Software
  • Be the first to comment

Continuous Security Testing

  1. 1. Continuous Security Testing Acceptance Test Driven Approach Sunday, 15 December, 13
  2. 2. Who am I? •Agile, TDD Coaching, Ugly Code Cleaning Dude •I love coding - Java, C#, Javascript, C/ C++, PHP, Perl, and some weird ones •I speak English, Cantonese, and Mandarin 2 Odd-e Pte. Ltd. Steven Mak 麥天志 Agile Coach Hong Kong Email: steven@odd-e.com Web: www.odd-e.com Twitter: stevenmak Sunday, 15 December, 13
  3. 3. Do you automate your tests? 3 Sunday, 15 December, 13
  4. 4. Is that what you feel? 4 Script Unreadable? Keep Changing? Time consuming to write? Sunday, 15 December, 13
  5. 5. Technical Activity Workflow Specification pyramid 5 RuleClarity Stability Specification Users can understand Automation Technical Sunday, 15 December, 13
  6. 6. Use Examples 6 With 3 judges giving scores 4, 20, and 18, the displayed score should be 42. When the first 2 judges have given their scores, e.g. 10 and 5, the intermediate score of 15 should be displayed already. No scores displayed as a dash (–), not zero. Maximum score from a judge is 20 points! Sunday, 15 December, 13
  7. 7. Examples, Tests, and Spec 7 Examples Tests Requirements can become elaborate verify Sunday, 15 December, 13
  8. 8. More ideas from • Threat Modelling • Session-Based Test Management / Exploratory Testing • Product Requirement • Experts 8 Sunday, 15 December, 13
  9. 9. Avoid handoff 9 Sunday, 15 December, 13
  10. 10. Avoid imperative • login • enter username • enter password • enter homepage • click category • choose product • put it on shopping cart • click generate order • ..... 10 Sunday, 15 December, 13
  11. 11. Avoid imperative • login • enter username • enter password • enter homepage • click category • choose product • put it on shopping cart • click generate order • ..... 11 Given I selected a doll in shopping cart When I generate order Then the order should contain doll and the price is 83.55 Sunday, 15 December, 13
  12. 12. Avoid imperative • login • enter username • enter password • enter homepage • click category • choose product • put it on shopping cart • click generate order • ..... 12 Given I selected a doll in shopping cart When I generate order Then the order should contain doll and the price is 83.55 This “Given When Then” is a common pattern called Gherkin Sunday, 15 December, 13
  13. 13. Good ones • Focus on business, not software design • Not coupled with code • Not coupled with UI • Concise • Use domain languages 13 Getting us towards Living Documentation and can be executed against existing system Sunday, 15 December, 13
  14. 14. Robot Framework www.robotframework.org 14 Sunday, 15 December, 13
  15. 15. Test Tools Robot Architecture 15 Test Data (Tables) Robot Framework Test Libraries System Under Test Test Library API application interfaces Robot comes with a number of built-in test libraries and you can (should!) add your own. Test libraries can use any test tool necessary to interact with the system under test. Sunday, 15 December, 13
  16. 16. It's all in the tables 16 Sunday, 15 December, 13
  17. 17. Test Cases are composed of keyword-driven actions 17 !"#$%&'()*+%),'-./()0 Sunday, 15 December, 13
  18. 18. Test Cases are composed of keyword-driven actions 17 !"#$%&'()*+%),'-./()0 this is the name of a test case Sunday, 15 December, 13
  19. 19. Test Cases are composed of keyword-driven actions 17 !"#$%&'()*+%),'-./()0 this is the name of a test case these keywords form the test case Sunday, 15 December, 13
  20. 20. Test Cases are composed of keyword-driven actions 17 !"#$%&'()*+%),'-./()0 this is the name of a test case these keywords form the test case keywords receive arguments Sunday, 15 December, 13
  21. 21. 2 types of keywords 18 Sunday, 15 December, 13
  22. 22. 2 types of keywords 18 We can import keyword libraries for a test case Sunday, 15 December, 13
  23. 23. 2 types of keywords 18 We can import keyword libraries for a test case ...and libraries may be configured, too. Sunday, 15 December, 13
  24. 24. 2 types of keywords 18 We can import keyword libraries for a test case ...and libraries may be configured, too. This keyword comes from the imported library. Sunday, 15 December, 13
  25. 25. 2 types of keywords 18 We can import keyword libraries for a test case ...and libraries may be configured, too. This keyword comes from the imported library. This is a user keyword, implemented in table format. (Think macros composed of other macros.) Sunday, 15 December, 13
  26. 26. 19 Data-driven test cases this is the name of a test case these keywords form the test case keywords receive arguments Sunday, 15 December, 13
  27. 27. 20 using Template *** Test Cases *** Email Delivered Acceptance Rule [Template] Confirm Email Delivered Workflow sender@mail.com user@example.com 3asyp3asy 1 sender@nonexist.com user@example.com 3asyp3asy 0 *** Keywords *** Confirm Email Delivered Workflow [Arguments] ${sender} ${recipient} ${password} ${number_of_emails_expected} Open Mail Box ${MAIL_SERVER} ${recipient} ${password} Count Mail Received ${sender} ${number_of_emails_expected} Keyword used as template test data feed as arguments Sunday, 15 December, 13
  28. 28. Given-when-then (BDD) 21 *** Test Cases *** Addition Given calculator has been cleared When user types "1 + 1" and user pushes equals Then result is "2" *** Keywords *** Calculator has been cleared Push button C User types "${expression}" Push buttons ${expression} User pushes equals Push button = Result is "${result}" Result should be ${result} this is the name of a test case these keywords form the test case Sunday, 15 December, 13
  29. 29. Variables 22 !"#$"%&'( )#*+,-*++"./,&$.'0 Sunday, 15 December, 13
  30. 30. Other choices • Cucumber • Fitnesse 23 Sunday, 15 December, 13
  31. 31. 24 An Example Sunday, 15 December, 13
  32. 32. 25 *** Settings *** Resource resource.txt *** Test Cases *** Checking Opened Ports [Template] Only these ports are opened 22 25 80 135 139 445 *** Keywords *** Only these ports are opened [Arguments] @{expected_ports} @{actual_ports_opened}= Scan with Fast Mode ${HOST} List Should Contain Sub List ${actual_ports_opened} ${expected_ports} *** Settings *** Library nmapLibrary Library Collections *** Variables *** ${HOST} www.scrumprimer.org import nmap class nmapLibrary: def scan_with_fast_mode(self, host): nm = nmap.PortScanner() nm.scan(str(host), arguments="-F") return [str(port) for port in nm[str(nm.all_hosts()[0].encode())].all_tcp()] resource.txt port_scanning.txt nmapLibrary.py (with python-nmap) Sunday, 15 December, 13
  33. 33. 26 pybot -d output nmap.txt ============================================================================== Port Scaning ============================================================================== Checking Openned Ports | PASS | ------------------------------------------------------------------------------ Nmap | PASS | 1 critical test, 1 passed, 0 failed 1 test total, 1 passed, 0 failed ============================================================================== Output: /Users/stevenmak/Work/robotframework/securityTests/2013.12.14VXCon/output/output.xml Log: /Users/stevenmak/Work/robotframework/securityTests/2013.12.14VXCon/output/log.html Report: /Users/stevenmak/Work/robotframework/securityTests/2013.12.14VXCon/output/report.html run the test: report: (also available in xml format for Jenkins integration) Sunday, 15 December, 13
  34. 34. 27 Sunday, 15 December, 13
  35. 35. More to wrap & integrate • w3af • garmr • arachni • dirb • sslyze • sqlmap 28 Sunday, 15 December, 13
  36. 36. Acceptance Test Driven Development 29 Discuss in workshop Develop in concurrence Deliver for acceptance Sunday, 15 December, 13
  37. 37. 30 Discuss in workshop Develop in concurrence Deliver for acceptance Focus on customer collaboration and user engagement. Try to get as many of these people attend as you can. Product Owner Dev Team Users IT operations Help Desk Tech Writers ? Sunday, 15 December, 13
  38. 38. 31 Discuss in workshop Develop in concurrence Deliver for acceptance With 3 judges giving scores 4, 20, and 18, the displayed score should be 42. When the first 2 judges have given their scores, e.g. 10 and 5, the intermediate score of 15 should be displayed already. No scores displayed as a dash (–), not zero. Maximum score from a judge is 20 points! Sunday, 15 December, 13
  39. 39. 31 Discuss in workshop Develop in concurrence Deliver for acceptance With 3 judges giving scores 4, 20, and 18, the displayed score should be 42. When the first 2 judges have given their scores, e.g. 10 and 5, the intermediate score of 15 should be displayed already. No scores displayed as a dash (–), not zero. Maximum score from a judge is 20 points! Robot tests are written in tables so that computers can read them Sunday, 15 December, 13
  40. 40. 32 Deliver for acceptance Discuss in workshop Develop in concurrence Sunday, 15 December, 13
  41. 41. Collaboration is key 33 team gets feedback earlier scope of work is clear and understood by all team understands what they're implementing shared language and vocabulary is built team collaborates closely with product owner Sunday, 15 December, 13
  42. 42. CITCON Hong Kong 34 • When: Apr 11 & 12, 2014 • Cost: Free • Registration: contact me • Sponsorship Welcome! http://citconf.com/hongkong2014/ Sunday, 15 December, 13
  43. 43. Thank you for spending time with me this evening. More feedback can be sent to: 35 Odd-e Hong Kong Ltd. Steven Mak 麥天志 Agile Coach Hong Kong Email: steven@odd-e.com Web: www.odd-e.com Twitter: stevenmak Sunday, 15 December, 13

×