Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Push Functional Testing Further

827 views

Published on

Slides for the UK London BCS Sigist March Keynote

Published in: Internet
  • Be the first to comment

Push Functional Testing Further

  1. 1. Push Your Functional Testing Further Alan Richardson EvilTester.com @EvilTester Technology and Security into
  2. 2. @EvilTester 2 Part the first, wherein we describe Functional Testing in terms of Systems and Models, and expand A model of testing to include Technical Testing.
  3. 3. @EvilTester 3 Functional Testing ● Testers learn how to test systems – Requirements – 'What' a system 'should' do
  4. 4. @EvilTester 4 Systems ● System under Development ● System of Development
  5. 5. @EvilTester 5 System Under Development ● System under Development – Requirements – Architecture – Environments – etc.
  6. 6. @EvilTester 6 System Of Development ● Methodological Context ● Social Constructs ● Model Different Systems of Testing – Systems of Feedback – Systems of Learning – Systems of Questioning – ...
  7. 7. @EvilTester 7 Requirement Example – RestMud
  8. 8. @EvilTester 8 A Model of Testing ● Modelling, ● Observation, ● Intent, ● Reflection, ● Manipulation
  9. 9. @EvilTester 9 We can push our functional testing further ● “What is it supposed to do?” – vs “What does it do?” ● Comparison to other models ● Is it viable? ● Precondition analysis ● Presupposition analysis
  10. 10. @EvilTester 10 We can push our functional testing further ● Explore 'How' the system does what it does ● Understand the technology used to build the system – Identify technology risks – Identify risks at different levels of the stack – Work at different levels of the stack
  11. 11. @EvilTester 11 A Model of Technical Testing ● Modelling, ● Observation, ● Interrogation, ● Reflection (includes intent), ● Manipulation
  12. 12. @EvilTester 12 Part the second, wherein two technical models are provided with a discussion of possible technical testing approaches.
  13. 13. @EvilTester 13 Example – a Java App ● HouseOfTest.se – /2016/02/testers-contest-crappy-little-datagenerator/
  14. 14. @EvilTester 14 Observation Example – a Java App ● Double click run – see GUI ● tail -f DataGeneration.txt ● Text Editor – Line endings – Refresh ● No exceptions shown
  15. 15. @EvilTester 15 Observation Example – a Java App ● java -jar crappy_little_datagenerator_v_1.0.jar – Now can see stdout written to command line, exceptions, errors etc.
  16. 16. @EvilTester 16 Interrogation Example – a Java App ● Decompile e.g. http://jd.benow.ca/ ● Load .jar as a library into a Project
  17. 17. @EvilTester 17 Interrogation Example – a Java App
  18. 18. @EvilTester 18 Observe - Multiple Entry Points
  19. 19. @EvilTester 19 Modelling Example – a Java App GUI CLI DataGeneration
  20. 20. @EvilTester 20 Manipulation Example – a Java App
  21. 21. @EvilTester 21 Manipulate Example – a Java App
  22. 22. @EvilTester 22 Risks ● Does this test approach add risk? – Because testing at a lower level in the stack? – Because working against a non-deployed version? – Because it is not how the user would run it?
  23. 23. @EvilTester 23 Risk ● Mitigating risk provides one reason for testing ● No test approach mitigates all risk ● Multiple test approaches required
  24. 24. @EvilTester 24 Modelling – a Web App HTML GUI Web App Browser Web Server App Server
  25. 25. @EvilTester 25 Observe & Interrogate – a Web App HTML & DOM Web App Web Server App Server HTTP via Proxies View Source Dev Tools HTTP via Dev Tools
  26. 26. @EvilTester 26 Manipulate – a Web App HTML & DOM Web App Web Server App Server HTTP via Proxies Dev Tools
  27. 27. @EvilTester 27 Modelling – this Web App Web App Web Server App Server Player GUI Admin GUI Rest API
  28. 28. @EvilTester 28 Part the third, wherein the overlap between technical testing and security testing are explored.
  29. 29. @EvilTester 29 Deeper Testing
  30. 30. @EvilTester 30 Technical Testing ● We will find defects and issues we would otherwise miss – Observing, Interrogating, Manipulating lower ● Some defects normally associated with security testing – I am not a security tester – Technically Informed Modelling
  31. 31. @EvilTester 31 Security Overlap Examples ● Code reviews find hard-coded security issues ● Form field inspection exposed emails, executable shell code, file names for survey answers ● HTML Commented URLs to 'secret' parts of the application ● User HTTP json contains more info than displayed as HTML
  32. 32. @EvilTester 32 Security Overlap Examples Because... ● Code reviews find hard-coded security issues ● Form field inspection exposed emails, executable shell code, file names for survey answers ● HTML Commented URLs to 'secret' parts of the application ● User HTTP json contains more info than displayed as HTML Observed More Deeply Interrogated More Deeply Interrogated More Deeply Observed, Interrogated, Manipulated More Deeply Because Modelled More Deeply
  33. 33. @EvilTester 33 Any Methodology : Any Tester ● Methodology context does not dictate – 'Process' context might dictate – 'Social' context might dictate ● Any Tester can do this – Limited by technology knowledge – Limited by technical skill – Limited by choice
  34. 34. @EvilTester 34 Part the fourth, wherein the steps to increase technical ability are made lay afore the public.
  35. 35. @EvilTester 35 I am fairly Technical ● Books ● SeleniumSimplified.com ● EvilTester.com ● JavaForTesters.com ● Online Training Courses ● Consultancy, work hands on with teams
  36. 36. @EvilTester 36 I grew up with computers... http://www.retrogamer.net/profiles/hardware/zx-spectrum-hardware-profile/
  37. 37. @EvilTester 37 I grew up reading computer books... http://www.usborne.com/catalogue/feature-page/computer-and-coding-books.aspx
  38. 38. @EvilTester 38 ...but not with the internet http://www.worldofspectrum.org/hardware/feat24.html
  39. 39. @EvilTester 39 I had to learn how to test the web
  40. 40. @EvilTester 40 How to learn to test the web ● Model What You Know – HTML? HTTP? Browsers? ● Increase your ability to Observe at the GUI – View Source – Inspect Element - Dev Tools – You will see things you don't understand (add to your model & research)
  41. 41. @EvilTester 41 How to learn to test the web ● Increase your ability to Manipulate at the GUI – Inspect Element - Dev Tools – Amend DOM prior to submitting a form – Inspect and manipulate URLs ● Cookies, Local Storage – Inspect – Figure out how to manipulate (plugins required?)
  42. 42. @EvilTester 42 Basic Web Challenges ● View Source and inspect Element of: – Your favourite web sites ● How do they do 'that'? ● Any free 'pdf' report that requires 'email' – find the download without adding your email ● Newspapers - 'you have read too many articles today' – how can they tell? Manipulate to bypass?
  43. 43. @EvilTester 43 How to learn to test the web ● Observe HTTP Traffic in Browser – Network tab in Dev Tools ● Observe HTTP Traffic outside Browser – HTTP Proxy – Fiddler, Charles, BurpSuite, Owasp ZAP – Interrogate and Manipulate Traffic with a Proxy
  44. 44. @EvilTester 44 How to learn to test the web ● Learn features in the browser – View Source, Users, Dev Tools – How can the feature help you test? ● Learn features in the proxies – Replay Message, Fuzzers, Auto Responders – How can the feature help you test?
  45. 45. @EvilTester 45 Technology Basics ● Model the Technology – Where are the gaps in your understanding? – These gaps are risks to your testing. ● How can I observe X? ● How can I interrogate X? ● How can I manipulate X? ● Repeat
  46. 46. @EvilTester 46 Application Basics ● Model the application ● What is it actually doing? – Not just what is it supposed to do ● How does it do X? – Observe, Interrogate, Manipulate
  47. 47. @EvilTester 47 Going Further
  48. 48. @EvilTester 48 Pushing Functional Testing Further Go Even Further: ● Explore and automate systems below the GUI ● GUI as API ● Quickly enter more combinations of input than would otherwise be feasible ● Fuzzers, Setup Test Data ● Test at an API level without specialist tools ● Proxy message creation Even
  49. 49. @EvilTester 49 Do the work! ● Research ● Experiment ● Learn ● Take Small Steps
  50. 50. @EvilTester 50 Do The Work! ● Alan Richardson – @EvilTester – www.EvilTester.com – www.JavaForTester.com – www.SeleniumSimplified.com – www.CompendiumDev.co.uk

×