1.
Инфраструктура
тестирования в
Kubernetes
Роман Орлов, Aerokube
2020

2.
# whoami
devops with ops background
Selenium master
bash one-liner jedi

3.
Aerokube
Painless Selenium
Golang
K8s for the win

4.
Selenium
Browser automation
Do actions in browser with a code
For Web testing

5.
Selenium
OK
Open browser
Go to URL
HTTP
OK
Put string inside
textinput
HTTP
OK
Press the button
HTTP
OK
Take a screenshot
HTTP
YOURTEST
SELENIUM
INFRASTRUCTURE

6.
Selenium infra
/usr/bin/firefox
/usr/bin/chrome
Selenium Grid
Test

7.
Selenium infra
C:Program Files
Internet Explorer
iexplore.exe
Selenium Grid
Test

8.
Problems
Can’t install 2 browsers on 1 server
Cache, history, temp files after tests
Unstable

9.
Docker era
/usr/bin/firefox
/usr/bin/chrome
Selenium Grid
Test
Selenoid
/usr/bin/firefox /usr/bin/chrome/usr/bin/firefox/usr/bin/firefox /usr/bin/chrome/usr/bin/chrome

10.
Docker era
Test
Selenoid
/usr/bin/firefox /usr/bin/chrome/usr/bin/firefox/usr/bin/firefox /usr/bin/chrome/usr/bin/chrome

11.
Problems solved
Can’t install 2 browsers on 1 server
Cache, history, temp files after tests
Unstable
Can’t install 2 browsers on 1 server
Cache, history, temp files after tests
Unstable

12.
Selenium infra
We want more

13.
Selenium in Kubernetes
Test
Moon

14.
Optimize infra. Case 1
Selenoid
/usr/bin/firefox
/usr/bin/chrome
Host 1
Dev1
Dev2
Dev3
Host 2
CPU Mem
CPU Mem

15.
Dev5Dev4
Host 2
CPU Mem
Optimize infra. Case 1
CPU Mem
Moon
Dev1
Dev3Dev2
Host 1

16.
Optimize infra. Case 2

17.
Better hardware utilization
Single runtime for everything
Easy to deploy
Selenium in Kubernetes

18.
How to run Windows inside K8s
Linux host
VM

19.
How to run Windows inside K8s
Linux host
VM
/dev/kvm

20.
How to run Windows inside K8s
https://github.com/aerokube/windows-images

21.
How to run Windows inside K8s
/dev/kvm
Moon
nodeSelector: hardware
{
"internet explorer": {
…
"nodeSelector": {
"node-type": "hardware"
}
…

22.
Stateless arch
Moon
chrome-81-SESSION-ID
Your
test
Open browser
Your browser id:
chrome-81-SESSION-ID
Domain name

23.
Stateless arch
Moon
chrome-81-SESSION-ID
Your
test URL opened
Open URL in browser:
chrome-81-SESSION-ID
Open URL
URL opened
Moon
Domain name

24.
Stateless arch
Moon
Moon
LB
Your
test

25.
Stateless arch. Demo

26.
Problems
Slow DNS
Kubernetes API overload

27.
Slow DNS
chrome-81-SESSION-ID
Domain name
Disable cache in CoreDNS
Wait until record is active

28.
Kubernetes API overload
Optimize requests to API~200 RPS to API

29.
Moon
https://github.com/aerokube/moon-deploy

30.
Aerokube@aerokube /aerokube
aerokube.com
Thanks
Roman Orlov, fote@aerokube.com
telegram: @fote2