Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Serverless - how to speed up tests over 300 times and achieve continuous feedback?

6 views

Published on

Tomasz Konieczny @ Ro

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Serverless - how to speed up tests over 300 times and achieve continuous feedback?

  1. 1. Serverless: how to speed up tests over 300 times and achieve continuous feedback? Tomasz Konieczny @ Ro @tkoniecznyPL
  2. 2. @tkoniecznyPL About me Tomasz Konieczny Senior QA Engineer @ Ro Conference speaker
  3. 3. Why do we execute E2E regression tests? @tkoniecznyPL
  4. 4. Problem: time @tkoniecznyPL
  5. 5. Solution: automation @tkoniecznyPL
  6. 6. Execution speed? @tkoniecznyPL
  7. 7. Faster execution: Faster feedback @tkoniecznyPL
  8. 8. Faster feedback: Faster delivery @tkoniecznyPL
  9. 9. So… we automate tests @tkoniecznyPL
  10. 10. A few minutes instead of hours @tkoniecznyPL
  11. 11. Automated build process @tkoniecznyPL
  12. 12. And then… Test set grows... @tkoniecznyPL
  13. 13. 10 minutes… 30 minutes… Hours... @tkoniecznyPL
  14. 14. Same problem appeared again @tkoniecznyPL
  15. 15. Let’s assume: 500 Selenium tests, 20 seconds each @tkoniecznyPL
  16. 16. How to solve it this time? @tkoniecznyPL
  17. 17. Test set: redesign? @tkoniecznyPL
  18. 18. Tests Pyramid: more low level tests? @tkoniecznyPL
  19. 19. Less frequent builds? @tkoniecznyPL
  20. 20. More compute resources? @tkoniecznyPL
  21. 21. No. @tkoniecznyPL
  22. 22. Real solution: Cloud @tkoniecznyPL
  23. 23. Parallel test execution @tkoniecznyPL
  24. 24. Serverless @tkoniecznyPL
  25. 25. AWS Lambda @tkoniecznyPL
  26. 26. AWS Lambda initial concurrency: 500-3000 @tkoniecznyPL
  27. 27. Lambda Layers @tkoniecznyPL
  28. 28. Serverless Framework + AWS Lambda @tkoniecznyPL
  29. 29. Lambda Layers - runtime lambda-layers/ ├── chrome │ ├── chromedriver │ └── chrome/chromium ├── selenium │ └── python │ └── lib │ └── python3.7 │ └── site-packages │ └── selenium └── serverless.yml @tkoniecznyPL
  30. 30. Lambda Layers - runtime service: lambda-selenium-layer provider: name: aws runtime: python3.7 region: eu-west-1 timeout: 30 layers: selenium: path: selenium CompatibleRuntimes: [ "python3.7" ] chrome: path: chrome CompatibleRuntimes: [ "python3.7" ] @tkoniecznyPL
  31. 31. Deployment package limits ❏ 50 MB (zipped, for direct upload) ❏ 250 MB (unzipped, including layers) @tkoniecznyPL
  32. 32. Compression + Decompression at runtime ❏ /tmp directory storage (512MB) ❏ Compression Algorithm ❏ xz/LZMA ❏ Brotli ❏ Decompression at runtime @tkoniecznyPL
  33. 33. Test Runner Function lambda-test-runner/ ├── handler.py └── serverless.yml @tkoniecznyPL
  34. 34. Test Runner Function - serverless.yml service: lambda-test-runner app: lambda-test-runner provider: name: aws runtime: python3.7 region: eu-west-1 stage: dev memorySize: 1024 timeout: 30 @tkoniecznyPL functions: lambdarunner: handler: handler.lambdarunner events: - http: path: lambdarunner method: get layers: - ${cf:lambda-selenium-layer-dev.SeleniumLayerExport} - ${cf:lambda-selenium-layer-dev.ChromeLayerExport}
  35. 35. Test Runner Function - handler @tkoniecznyPL
  36. 36. Test Runner Function - handler @tkoniecznyPL
  37. 37. runner.py @tkoniecznyPL
  38. 38. runner.py - libraries, args @tkoniecznyPL
  39. 39. runner.py - test set @tkoniecznyPL
  40. 40. runner.py - boto3 client @tkoniecznyPL
  41. 41. runner.py - invoke test @tkoniecznyPL
  42. 42. runner.py - executor @tkoniecznyPL
  43. 43. runner.py - report @tkoniecznyPL
  44. 44. $ time python3 runner.py @tkoniecznyPL
  45. 45. Full Test set (500 tests): about 30 seconds @tkoniecznyPL
  46. 46. Execution cost: $0.17 @tkoniecznyPL
  47. 47. PoC: a few hundred lines of code @tkoniecznyPL
  48. 48. Limitations? @tkoniecznyPL
  49. 49. AWS API Gateway timeout: 30 seconds @tkoniecznyPL
  50. 50. Lambda invoke: event @tkoniecznyPL
  51. 51. Test execution state storage @tkoniecznyPL
  52. 52. How to extend this PoC? @tkoniecznyPL
  53. 53. “Cloud is powerful” @tkoniecznyPL
  54. 54. DevOps Methodology @tkoniecznyPL
  55. 55. Questions? tomasz.konieczny@ro.co @tkoniecznyPL

×