Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Automated testing 101

2,740 views

Published on

Automated Testing for WordPress

Published in: Software
  • Be the first to comment

Automated testing 101

  1. 1. Automated Testing 101 By Tabby Chapman @tabbymarie http://www.tabbychapman.com 1
  2. 2. About Tabby Chapman @tabbymarie https://github.com/tabbymarie 2
  3. 3. About Me • Work for the University of California, Irvine • Computer Scientist • Developer Cheerleader • Education Junkie 3
  4. 4. About Me • WordPress Skillz – Reaching my 10th anniversary developing with WordPress – Started with 2.0.5, building a custom theme. – Ran my own company for four years – Joined Zeek for almost two years – Currently build highly specialized custom themes and plugins for UCI. 4
  5. 5. About This Presentation • Geared towards intermediate QA testers and Developers and also for stakeholders/clients/customers • Assumptions: – You know what WordPress is. – You know what PHP is. – You have written PHP code several times. Successfully. Or. – You test all the things. 5
  6. 6. About This Presentation • We’ll go over – The Developer Workflow(s) – Different Types of Tests – Automated Testing – Demo – Last Words 6
  7. 7. How it Currently Works 7 • Generic Software Development Lifecycle – In the dinosaur days of computers, the majority of its users were very technical people. – As computers became more accessible (and smaller), more and more users began to use it – Developers needed to create a process to think outside of their own processes to accommodate non-technical users
  8. 8. How it Currently Works 8 • Generic Software Development Lifecycle (cont) – Many various methods were created to attempt to meet this need – One of these is the Software Development Lifecycle (which was a similar process used in other disciplines) – It is a top down approach: generic to specific – Specification, Design, Coding
  9. 9. How it Currently Works 9 • Generic Software Development Lifecycle (cont) – Waterfall method is created in the ‘70s. – Structured analysis, Structured design, Structured programming (sounds a lot like specification, design, and coding) – Most developers tend to use Define, Design, Develop, Deploy, Maintain.
  10. 10. Generic Modern Development 10 Image credit: https://www.owasp.org/images/8/84/SDLC.jpg
  11. 11. Freelance Development Cycle 11 Image credit: http://mohamadsubhan.files.wordpress.com/2009/06/linear.jpg
  12. 12. How it Currently Works 12 • Full Software Development Lifecycle – Many developers on teams tend to use some form of the full development lifecycle, as there tends to need to be some form of accountability for teams. – This includes: feasibility study, requirements definition, software design, implementation and unit testing, integration and system testing, operation and maintenance.
  13. 13. Team Development Cycle 13 Image credit: http://www.softwaretestinghelp.com/unit-testing/
  14. 14. Types of Tests • Black box • White box • Unit • Integration • Functional • System • End-to-end • Sanity • Regression • Acceptance • Load • Stress • Performance • Usability 14 Source: http://www.softwaretestinghelp.com/types-of-software-testing/
  15. 15. Testing Players 15 • Quality Assurance (QA) Engineers • Developers • Product Owners • Clients • End-Users • Other Stakeholders
  16. 16. Black Box Testing? 16
  17. 17. Black Box Testing 17 • Tests against requirements and functionality. Is not concerned with the internal workings. • “If I put in 2+4, I expect 6. Does that work?”
  18. 18. White Box Testing? 18
  19. 19. White Box 19 • Tests against requirements and functionality with knowledge of the inner workings. • “I know that I this system has these conditionals, so I will test all possible outcomes.”
  20. 20. Unit Testing? 20
  21. 21. Unit Testing 21 • Tests specific methods or functions, created by a programmer • “Does function addTwoIntegers($num1,$num2) work as expected?”
  22. 22. Integration Testing? 22
  23. 23. Integration Testing 23 • Testing of whole application after new work has been integrated as they interact with the client/server • “Does this site’s individual modules still work as expected when we put it all together?”
  24. 24. Functional Testing? 24
  25. 25. Functional Testing 25 • Black Box testing for specific functional requirements. • “If I tell it to add five text inputs and send an email, does it add five inputs and send an email as expected?”
  26. 26. Systems Testing 26 • Black Box Testing based on overall requirements. • “Testing All the Things”. “Does this system meet all of the requirements as expected?”
  27. 27. End-to-end Testing 27 • Similar to Systems, but also tests interaction between various elements, protocols, layers and servers. • “Testing All the Things and Moar”. “Does this system meet all of the requirements as expected and communicate as expected to its various parts?”
  28. 28. Sanity Testing 28 • A pre-test to make sure the new development is done enough to fully test. • “Does it at least work a little bit?”
  29. 29. Regression Testing 29 • Testing the whole application after something has been modified to see if something was broken as a result of the modification. • “Does it STILL work as expected?”
  30. 30. Acceptance Testing 30 • User or Customer does a round of manual tests to make sure that it meets their requirements/needs • “Does the client think it works as expected?”
  31. 31. Stress Testing 31 • Testing Performance Requirements • “Can more than one interaction happen at the same time? How many more than one?”
  32. 32. Usability Testing 32 • Testing for user-friendliness. This can be done BEFORE and AFTER development phase. • “Yeah, it works, but is it easy to use?”
  33. 33. Which testing is important for developers? 33
  34. 34. Which testing is important for developers? • Black box • White box • Unit • Integration • Functional • System • End-to-end • Sanity • Regression • Acceptance • Load • Stress • Performance • Usability 34 Source: http://www.softwaretestinghelp.com/types-of-software-testing/
  35. 35. For the Developers 35 • Which testing is important for developers? – All of them (know thy tester and thy tester’s methods) – But especially • Unit • Functional • Systems • Integration
  36. 36. Which testing is important for testers? 36
  37. 37. Which testing is important for testers? • Black box • White box • Unit • Integration • Functional • System • End-to-end • Sanity • Regression • Acceptance • Load • Stress • Performance • Usability 37 Source: http://www.softwaretestinghelp.com/types-of-software-testing/
  38. 38. For the Testers 38 • Which testing is important for testers? – All of them (uphold they developer to unit tests) – But especially • Black Box • White Box • Integration • Systems • Sanity
  39. 39. Which testing is important for Clients/Stakeholders? 39
  40. 40. Which testing is important for Clients/Stakeholders? • Black box • White box • Unit • Integration • Functional • System • End-to-end • Sanity • Regression • Acceptance • Load • Stress • Performance • Usability 40 Source: http://www.softwaretestinghelp.com/types-of-software-testing/
  41. 41. For the Clients/Stakeholders 41 • Which testing is important for clients/stakeholders? • Black Box • Usability • Regression • Acceptance
  42. 42. For the Clients/Stakeholders 42 • Which testing is important for clients/stakeholders? • Black Box • Usability • Regression • Acceptance
  43. 43. What does Automation of Testing mean? 43
  44. 44. What does Automation of Testing mean? 44 • The traditional/easiest way to test what you develop is to make a change, and then load the app to see if that change was effective. Very Manual • Automated Testing means to write code to test code and use testing software to test the front-end. Don’t rely on the old manual check.
  45. 45. How does this apply to WordPress? 45 • WordPress utilizes a ‘test suite’ “which have multiple test cases and a library and command line tool that runs the test suite or suites” according to WordPress’ automated testing core handbook page • For this, they use phpUnit. https://phpunit.de
  46. 46. How does this apply to WordPress? 46 • PHPUnit has a series of commands that you can use to run all tests, specific tests, or specific types of tests.
  47. 47. Developer Workflow 47 • When you are ready to write your plugin: – Think of the outcomes you want for your plugin – Think of the individual methods you want to write. Should have one output. – Write a test that checks that the output is what you expect. – Write your method. – Lather. Rinse. Repeat.
  48. 48. Developer Workflow 48 • Testing Tools • phpUnit (official wordpress test suite) • WordPress Developer Branch – https://make.wordpress.org/core/handbook/automa • Terminal or Editor with Testing Integrated
  49. 49. Developer Workflow 49 Editors (not limited to…) – TextMate - https://macromates.com/download • (https://github.com/meritt/phpunit-tmbundle) – Sublime – http://www.sublimetext.com/ • https://packagecontrol.io/packages/PHPUnit – phpStorm - https://www.jetbrains.com/phpstorm/ • Supports all OS’ • Phpunit built in
  50. 50. Developer Workflow 50 • Put your tests IN your plugin • /plugins/[plugindir]/tests/test- [pluginslug].php • Add a phpunit.xml file: • /plugins/[plugindir]/tests/phpunit.xml
  51. 51. Developer Workflow phpunit.xml 51 <?xml version=“1.0” encoding=“UTF-8” ?> <phpunit bootstrap=“vendor/autoload.php” > <testsuites> <testsuite> <directory prefix="test-” suffix=".php"> ./tests/ </directory> </testsuite> </testsuites> </phpunit>
  52. 52. Developer Workflow phpunit.xml 52 <?xml version=“1.0” encoding=“UTF-8” ?> <phpunit bootstrap=“vendor/autoload.php” > <testsuites> <testsuite> <directory prefix="test-” suffix=".php"> ./tests/ </directory> </testsuite> </testsuites> </phpunit>
  53. 53. Developer Workflow phpunit.xml 53 <?xml version=“1.0” encoding=“UTF-8” ?> <phpunit bootstrap=“vendor/autoload.php” > <testsuites> <testsuite> <directory prefix="test-” suffix=".php"> ./tests/ </directory> </testsuite> </testsuites> </phpunit>
  54. 54. Developer Workflow phpunit.xml 54 <?xml version=“1.0” encoding=“UTF-8” ?> <phpunit bootstrap=“vendor/autoload.php” > <testsuites> <testsuite> <directory prefix="test-” suffix=".php"> ./tests/ </directory> </testsuite> </testsuites> </phpunit>
  55. 55. Tester Workflow 55 • When you are ready to test – Be familiar with the general requirements for the site as a whole – Read the requirements of the change request – Focus on specifically what is supposed to have changed. – Write a test while waiting for the change. – Run the test once you get the change from the developer
  56. 56. Tester Workflow 56 • Selenium IDE, Firefox Plugin – http://www.seleniumhq.org/download/ • Browser Specific – Chrome https://sites.google.com/a/chromium.org/chromed – Safari http://selenium-release.storage.googleapis.com/2
  57. 57. Tester Workflow 57 • Selenium – Installationhttp ://geekpad.ca/blog/post/automating-browser-testi • First you must download the Selenium server and the Internet Explorer Server from http://docs.seleniumhq.org/download/ • Then you must download the Chrome server from http://chromedriver.storage.googleapis.com/index .html
  58. 58. Tester Workflow 58 • Selenium – Installation (continued) • You do not need to download a Firefox driver since it’s builtin the main Selenium file. • Once all files have been downloaded you must add your selenium folder to your existing path see http://www.java.com/en/download/help/path.xml • Then you can launch the server using java. With something that looks like this – Mac - java -jar selenium-server-standalone-[version].jar
  59. 59. Tester Workflow 59 Why is Selenium cool? •Records your actions •Let’s you test expected outcomes •Let’s you test them in a series
  60. 60. Tester Workflow 60 Example •The developer was given a task to make sure that all instances of the word “Orange County” is replaced with the word “OC” on the website, for “legal reasons”. •Demo
  61. 61. Jenkins 61
  62. 62. Jenkins 62 • Jenkins is an award-winning application that monitors executions of repeated jobs, such as building a software project or jobs run by cron. Among those things, current Jenkins focuses on the following two jobs: – Building/testing software projects continuously – Monitoring executions of externally-run jobs https://wiki.jenkins-ci.org/display/JENKINS/Meet+Jenkins
  63. 63. Jenkins 63
  64. 64. Jenkins 64
  65. 65. Questions? 65
  66. 66. Resources • http://www.dcs.bbk.ac.uk/~sven/cdm06/cdm3. • https://unit-tests.trac.wordpress.org/ • http://www.slideshare.net/ptahdunbar/automat • http://johnregan3.co/2013/08/13/why-you-need • http://blog.aizatto.com/2010/05/15/beginni ng-wordpress-automated-testing/ 66
  67. 67. Resources • https://www.owasp.org/index.php/Testing_ Guide_Introduction • https://make.wordpress.org/core/handboo k/automated-testing/ • http://stackoverflow.com/questions/224141 34/different-types-of-testing-eg-unit- functional-integration-etc-document 67
  68. 68. Resources • http://www.seleniumhq.org/ • http://geekpad.ca/blog/post/automating-browser- testing-with-phpunit-and-selenium • https://wiki.jenkins- ci.org/display/JENKINS/Meet+Jenkins • https://wiki.jenkins- ci.org/display/JENKINS/Installing+Jenkins 68

×