This is my Seleniumconf 2017 talk slides. In this talk I describe some concepts helpful to build really scalable Selenium cluster using simple components like Selenoid server ang Ggr load balancer.

1. Scalable Selenium cluster: up & running
by Ivan Krutov

2. About me
Java & Golang developer
Allure Framework contributor
Devops
Big Selenium cluster

3. Selenium clusters
Typical Selenium Grid
50 browsers
10000 sessions / day
1 data center
5 rps
500 Kbit/s
Sometimes down
My cluster
5000+ browsers
1.5M sessions / day
5 data centers
4000 rps
800 Mbit/s
24/7/365

4. Available browsers

5. Selenium clusters

6. We do not rely on Selenium server

7. Using Selenium server
is a time bomb for scalability

8. Individual developer
Easy installation
Very simple usage
Reliable
Zero maintenance
Working out of the box

9. How Selenium works

10. Installing Selenium
Manually install Java
Manually download Selenium JAR
Manually download web-driver binaries
Manually unpack web-drivers from archives
Manually install required browsers
Manually type complicated start command

11. Selenium window focus problem

12. Selenium browser settings problem

13. Installing two Firefox versions
/usr/bin
FF 46
FF 45

14. Selenium compatibility issues
Chromedriver ???56

15. Bad Foundation

16. Linux covers 80% of browsers

17. Let’s use containers where possible!

18. How it should work
Selenium
API
Container

19. SelenoidSelenium

20. Selenoid runs isolated browsers

21. Images for all recent browsers
3.6+ 33.0+
12.16
48.0+

22. Custom screen resolution

23. Small Demo

24. Small team

25. Scaling Selenium for small team
Workstation Server or VM

26. Medium team

27. Fault Tolerance
DC1 DC2

28. Scalability
DC1 DC2 DC3 DC4

29. Standard API

30. Access control

31. Browser consumption control
x 100
x 50
x 10someuser

32. Selenium Grid

33. Selenium Grid
Not fault-tolerant
Not really scalable
No access control

34. Client-side Load Balancing

35. Hosts Information File

36. Client-side load balancing
Fault-tolerant
Scalable
No access control
Not standard Selenium

37. Big company = a lot of teams

38. Server-side Load Balancing

39. GoGridRouter (aka Ggr)
100% Golang
6 Mb binary
400 rps 200 Mb RAM→

40. How to run your tests
Standard Selenium server:
http://example.com:4444/wd/hub
GridRouter:
http://user:password@example.com:4444/wd/hub

41. Multiple Datacenter Problem
Ggr1
Ggr2
Host with browsers
New session
Open page
?

42. Original session ID:
e0dfdda0-1786-4e8f-987b-29a61f5c165e
md5(‘http://example.com:4444’):
798744d2b7036ba36c4ea285272bcd48
Extended session ID:
798744d2b7036ba36c4ea285272bcd48e0dfdda0-
1786-4e8f-987b-29a61f5c165e
Session ID magic

43. How to scale the cluster
SLB
Ggr
Hosts with browsers

44. Scaling with no downtime
GgrGgr
SIGHUP

45. One more demo

46. What to put behind Ggr
Low resources consumption
Ability to send logs & statistics
No downtime maintenance

47. Selenium vs Selenoid resources
Selenium Server
100% Java
20 Mb + 100 Mb JRE
500+ Mb RAM
Selenoid
100% Golang
7 Mb binary
60 Mb RAM

48. Fine-grained resources control
Limit total number of sessions per host
Queues
Limit CPUs per container
Limit RAM per container

49. Tmpfs support

50. Centralized logs support
Selenoid
E
L
K

51. Statistics as JSON
Selenoid
Graphite
InfluxDB
ElasticSearch Grafana

52. Some numbers

53. Well documented

54. Stop using Selenium server!

55. References
@aerokube
http://telegram.me/aerokube
http://aerokube.com/
vania-pooh@aerokube.com
https://github.com/aerokube/selenoid
https://github.com/aerokube/ggr