Tomasz Konieczny @ Ro

1. Serverless: how to speed up tests
over 300 times
and achieve continuous feedback?
Tomasz Konieczny @ Ro
@tkoniecznyPL

2. @tkoniecznyPL
About me
Tomasz Konieczny
Senior QA Engineer @ Ro
Conference speaker

3. Why do we execute
E2E regression tests?
@tkoniecznyPL

4. Problem: time
@tkoniecznyPL

5. Solution: automation
@tkoniecznyPL

6. Execution speed?
@tkoniecznyPL

7. Faster execution:
Faster feedback
@tkoniecznyPL

8. Faster feedback:
Faster delivery
@tkoniecznyPL

9. So… we automate tests
@tkoniecznyPL

10. A few minutes instead of hours
@tkoniecznyPL

11. Automated build process
@tkoniecznyPL

12. And then…
Test set grows...
@tkoniecznyPL

13. 10 minutes… 30 minutes… Hours...
@tkoniecznyPL

14. Same problem appeared again
@tkoniecznyPL

15. Let’s assume:
500 Selenium tests, 20 seconds each
@tkoniecznyPL

16. How to solve it this time?
@tkoniecznyPL

17. Test set:
redesign?
@tkoniecznyPL

18. Tests Pyramid:
more low level tests?
@tkoniecznyPL

19. Less frequent builds?
@tkoniecznyPL

20. More compute resources?
@tkoniecznyPL

21. No.
@tkoniecznyPL

22. Real solution: Cloud
@tkoniecznyPL

23. Parallel test execution
@tkoniecznyPL

24. Serverless
@tkoniecznyPL

25. AWS Lambda
@tkoniecznyPL

26. AWS Lambda initial concurrency:
500-3000
@tkoniecznyPL

27. Lambda Layers
@tkoniecznyPL

28. Serverless Framework + AWS Lambda
@tkoniecznyPL

29. Lambda Layers - runtime
lambda-layers/
├── chrome
│ ├── chromedriver
│ └── chrome/chromium
├── selenium
│ └── python
│ └── lib
│ └── python3.7
│ └── site-packages
│ └── selenium
└── serverless.yml
@tkoniecznyPL

30. Lambda Layers - runtime
service: lambda-selenium-layer
provider:
name: aws
runtime: python3.7
region: eu-west-1
timeout: 30
layers:
selenium:
path: selenium
CompatibleRuntimes: [
"python3.7"
]
chrome:
path: chrome
CompatibleRuntimes: [
"python3.7"
]
@tkoniecznyPL

31. Deployment package limits
❏ 50 MB (zipped, for direct upload)
❏ 250 MB (unzipped, including layers)
@tkoniecznyPL

32. Compression + Decompression at runtime
❏ /tmp directory storage (512MB)
❏ Compression Algorithm
❏ xz/LZMA
❏ Brotli
❏ Decompression at runtime
@tkoniecznyPL

33. Test Runner Function
lambda-test-runner/
├── handler.py
└── serverless.yml
@tkoniecznyPL

34. Test Runner Function - serverless.yml
service: lambda-test-runner
app: lambda-test-runner
provider:
name: aws
runtime: python3.7
region: eu-west-1
stage: dev
memorySize: 1024
timeout: 30
@tkoniecznyPL
functions:
lambdarunner:
handler: handler.lambdarunner
events:
- http:
path: lambdarunner
method: get
layers:
- ${cf:lambda-selenium-layer-dev.SeleniumLayerExport}
- ${cf:lambda-selenium-layer-dev.ChromeLayerExport}

35. Test Runner Function - handler
@tkoniecznyPL

36. Test Runner Function - handler
@tkoniecznyPL

37. runner.py
@tkoniecznyPL

38. runner.py - libraries, args
@tkoniecznyPL

39. runner.py - test set
@tkoniecznyPL

40. runner.py - boto3 client
@tkoniecznyPL

41. runner.py - invoke test
@tkoniecznyPL

42. runner.py - executor
@tkoniecznyPL

43. runner.py - report
@tkoniecznyPL

44. $ time python3 runner.py
@tkoniecznyPL

45. Full Test set (500 tests):
about 30 seconds
@tkoniecznyPL

46. Execution cost: $0.17
@tkoniecznyPL

47. PoC: a few hundred lines of code
@tkoniecznyPL

48. Limitations?
@tkoniecznyPL

49. AWS API Gateway timeout:
30 seconds
@tkoniecznyPL

50. Lambda invoke: event
@tkoniecznyPL

51. Test execution state storage
@tkoniecznyPL

52. How to extend this PoC?
@tkoniecznyPL

53. “Cloud is powerful”
@tkoniecznyPL

54. DevOps Methodology
@tkoniecznyPL

55. Questions?
tomasz.konieczny@ro.co
@tkoniecznyPL