Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Testing for fun in production Into The Box 2018

49 views

Published on

Mark Drew

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Testing for fun in production Into The Box 2018

  1. 1. TESTING IN PRODUCTION Fun* times getting your application tested
  2. 2. PREVIOUSLY ON THIS PRESENTATION…
  3. 3. QUICK RECAP…
  4. 4. CMDdevelop • deploy • deliver @markdrew mark@cmdhq.io http://cmdhq.io
  5. 5. WHAT WE ARE COVERING ➤ Background ➤ Workflow ➤ Unit Testing ➤ QA Testing ➤ Testing in Production ➤ Q&A (not QA)
  6. 6. BACKGROUND
  7. 7. BACKGROUND ➤ Complex Application ➤ Owned actual hardware at ISP ➤ Disaster Recovery (DR) ➤ WIDE range of platforms ➤ Range of usage patterns ➤ Lots of assets and data ➤ Issues with getting code to production ➤ 💩 code constipation ➤ 💥 Actual Disaster - ISP needs to move.
  8. 8. Web app Image Conv Meta indexing Image Conv Log
  9. 9. PLATFORM BEHAVIOUR A B siteA.domain.com siteB.domain.com
  10. 10. WORKFLOW
  11. 11. TIMETABLE Week1 Mon Tue Wed Thurs Fri Dev Dev Dev Dev Dev Dev QA Infra Week2 Mon Tue Wed Thurs Fri Dev Dev Dev Release Estimate Train/Est QA Test Test Test Infra Week3 Mon Tue Wed Thurs Fri Dev Dev Dev Dev Dev … QA Test Test Infra Prep Deploy
  12. 12. ENVIRONMENTS Development Unit Test QA Live
  13. 13. ENVIRONMENTS Development Unit Test QA Live
  14. 14. UNIT TESTING
  15. 15. TESTBOX ➤ Unit testing framework ➤ Not just for Coldbox apps! ➤ Also includes Mockbox ➤ Test individual logic of your code. ➤ Just had two sessions of it!
  16. 16. BASIC TEST component extends="testbox.system.BaseSpec" { function run() { describe( "Has a commentGateway", function() { beforeEach( function() { commentService = new core.objects.platform.comment.commentService(); entityId = 3078; } ); it( "should have a function called `getCommentGateway`", function() { makePublic(commentService, "getCommentGateway"); expect(commentService).toHaveKey("getCommentGateway"); } ); it( "should call `getCommentGateway` function and return a new instance", function() { makePublic(commentService, "getCommentGateway"); var commentGateway = commentService.getCommentGateway(); expect(commentGateway).toBeTypeOf("component"); } ); } ); }
  17. 17. MOCKBOX component extends="testbox.system.BaseSpec" { function run() { describe( "Can delete comments", function() { beforeEach( function() { commentService = new core.objects.platform.comment.commentService(); } ); it( "should return a error object", function() { var responseData = { "data": "Sorry, something went wrong.", "statusCode": "500", "statusText": "Internal server error" }; var commentService = prepareMock(new core.objects.platform.comment.commentService()); var commentGateway = prepareMock(new core.objects.platform.comment.commentGateway()); commentService.$("getCommentGateway", commentGateway); commentGateway.$("deleteComment"); var response = commentService.post(testData); expect(response).toBe(responseData) }); }); } }
  18. 18. TEST RESULTS
  19. 19. TEST INDEX
  20. 20. AUTOMATED TESTING (IN JENKINS) #!/usr/bin/env bash echo "Building the Unit Test image" container_name="unittests" image_name="unittests" docker build -t ${image_name} -f Dockerfile-UnitTests . mkdir -p reports echo "Starting..." docker run --rm --name ${container_name} -v "${PWD}/reports:/reports" ${image_name}
  21. 21. 😗 ❤
  22. 22. UNIT TESTING DOCKER IMAGE FROM ortussolutions/commandbox COPY . /app ENV IMAGE_TESTING_IN_PROGRESS true CMD cd $APP_DIR && $BUILD_DIR/run.sh && box testbox run runner=docker outputFile=/reports/ testresults.txt reporter=Text verbose=no && box server stop
  23. 23. ENVIRONMENTS Development Unit Test QA Live
  24. 24. QA TESTING
  25. 25. QA TESTING ➤ Manual testing ➤ Automated testing ➤ What cannot be automated?
  26. 26. MEET BOB…* * He’s a real person!
  27. 27. MANUAL TESTING
  28. 28. STANDARD PERFORMANCE AND FUNCTIONALITY TESTS
  29. 29. TICKETS BEING TESTED
  30. 30. AUTOMATED UI TESTING
  31. 31. CodeceptJS
  32. 32. INSTALLATION ➤ NPM ➤ Node (obviously) ➤ Selenium Server ➤ CodeceptJS
  33. 33. @markdrew > npm install -g selenium-standalone > selenium-standalone install > selenium-standalone start Selenium
  34. 34. BUT
  35. 35. @markdrew { "tests": "./*_test.js", "timeout": 10000, "output": "./output", "helpers": { "WebDriverIO": { "url": "https://www.google.com", "browser": "chrome" } }, "include": { "I": "./steps_file.js" }, "bootstrap": false, "mocha": {}, "name": "codecept" } codecept.json
  36. 36. @markdrew Feature('MainTest'); Scenario('test something', (I) => { I.amOnPage('/'); I.fillField('#lst-ib', 'Into The Box'); I.click('#tsf > div.tsf-p > div.jsb > center > input[type="submit"]:nth-child(1)'); I.see('Into the Box is a yearly web technologies and software craftsmanship conference'); I.click('Into The Box 2018'); I.see('April 25-27, 2018 | Texas') }); maintest_test.js
  37. 37. AUTOMATED TESTING ➤ Pros ➤ Can automate a lot of the work ➤ Takes pictures of failures ➤ Nice logs and results ➤ Cons ➤ Doesn’t support everything ➤ File uploads ➤ File Downloads ➤ Drag and Drop events
  38. 38. ENVIRONMENTS Development Unit Test QA Live
  39. 39. TESTING IN PRODUCTION
  40. 40. TESTING IN PRODUCTION ➤Can be done. ➤Don’t do load testing though 😂 ➤Critical Paths
  41. 41. PLATFORM BEHAVIOUR A B siteA.domain.com siteB.domain.com
  42. 42. GUIDELINES ➤ Create your own test data. ➤ The naming convention of test data should be realistic. ➤ Do not play with other existing user’s data. ➤ Create your credentials to access the application. ➤ Never try load test on a production environment. ➤ Test only if there is less load on the application. ➤ Change emails/mail provider if you can ➤ Add to the logs / tagged output
  43. 43. BENEFITS ➤ Monitor behaviour realtime ➤ Run edge cases (network failure? Interrupted calls) ➤ API responses are … real ➤ More bugs are detected
  44. 44. ENVIRONMENTS Development Development Unit Test Unit Test QA QA Live Live Tests
  45. 45. Q&A (NOT QA)
  46. 46. IN SUMMARY ➤ Put unit tests in place ➤ Jon Clausen is a 🌟 ➤ Make space for QA Testing ➤ Automate what you can ➤ Manually test what you can ➤ Performance Check ➤ Live testing ➤ with your own data ➤ Against metrics ➤ Log “invisible” actions
  47. 47. CMDdevelop • deploy • deliver @markdrew mark@cmdhq.io http://cmdhq.io

×