By Arik Lerner & Waseem Hamshawi
Kubernetes your Tests!
Testing with Docker on Google Cloud Platform
The JourneyBased On True Story!

{
“Name”: “Arik Lerner”,
“Role”: “Automation Infra Team Leader at LivePerson”,
“Data”: [
“3 years as Team Leader in Liveperson” ,
“2 years Java developer Reporting Platform Liveperson”
],
“Hobbies”: “Private pilot”
}
Active Prevention Department

{
“Name”: “Waseem Hamshawi”,
“Role”: “Senior Automation Infra Developer at Liveperson”,
“Data”: [
“2.5 years at Liveperson”,
“github: waseemh.github.io”,
],
“Hobbies”: “Running”
}
Waseem is Here!
Active Prevention Department

Short Story
From Manual Post Deployment Verification
TO
Automation

❏ Fast feedback
❏ Stable Environment
❏ Isolation
❏ Investigation tools - Hard to support (Video/HAR)
❏ Scale
❏ Efficiency
❏ Browser diversity
❏ Mobile Support
Requirements

Selenium Hub
Running Test
Static VMs
Supported Requirements
Browser diversity
Mobile Support
Parallelism
Traditional Selenium Grid

Ranking:
Tooling
Deployment/
Configuration
Feedback
Isolation
Efficiency
Scalability
❏ Less stable
❏ Crashes - require restarts
❏ Cancel build in progress leave open sessions
❏ Investigation tools - Hard to support (Video/HAR)
❏ Manual grid node Configuration/deployment
❏ Not scalable
❏ Slow feedback
❏ Not efficient - idle grid machines
Facing Issues

Cattlevs.Pets
VM
Container
● When they get ill you nurse them back
to health
● You name them! And take of them!
● When they get ill you get another one!
● They are almost identical to one another
● Don't have specific names. Given numbers
● Can be simply replaced if they die

What Docker gives us ?
● Build it once run it everywhere!
● Resource Isolation
● Test Isolation
● Package format
● Implementation of a container
Docker

Chrome FirefoxVNC Server
Selenium Server
Hub/Node JAR
Mobile Android
What is in a Selenium Container ?

VM
VM
VM
VM
Nodes
HUB
N Hub’s
Running Tests
Selenium Grid Architecture Makeover

Selenium Docker Images
hub.docker.com/r/selenium

DEMO

We need more than just Packaging and
Isolation:
● Orchestration - Distribute containers
● Health Checks
● Resilience and Self-healing
● Scaling - making jobs bigger or smaller
Kubernetes Orchestrator

VM
VM
VM
VM
Pods
HUB
N Hub’s
Running Tests
Selenium Grid Architecture Makeover

Testing on the Cloud!

Benefits of Cloud Platform
● Kubernetes cluster in a click of a button (GCP)
● No idle machines - Use when needed
● Costs per build/test - measurable
● Not part of Liveperson network
● Geolocation simulation

DEMO

Build Life Cycle

GCP - Selenium Grid lifecycle
Run tests
Kill Selenium Grid
Scale Down GCP Instances
Scale Out GCP Instances
Create Selenium Grid container cluster

Requirements

Requirements
❏ Fast feedback
❏ Stable Environment
❏ Isolation
❏ Investigation tools - Hard to support (Video/HAR)
❏ Scale
❏ Efficiency
❏ Browser diversity
❏ Mobile Support

Ranking:
Tooling
Deployment/
Configuration
Feedback
Isolation
Efficiency
Scalability
❏ Investigation tools - Video tools not reliable and not
stable.
❏ Feedback
❏ Improved BUT still needs to wait for nodes to
connect into Hub
❏ Point of failures
❏ Hub/Nodes is more failure points in the
architecture that not really necessary
Facing Issues

waseemh-mac$ java -jar selenium-server-standalone-2.53.1.jar -role
Selenium server can run in one of the following roles:
hub as a hub of a Selenium grid
node as a node of a Selenium grid
standalone as a standalone server not being a part of a grid
Test Hub
Selenium Server
(Node)
Selenium Standalone

Non-Grid Solution
● Fast → Deploy Selenium Standalone containers on-demand
● Short-lived → Selenium containers are destroyed once test ends
● Distribute and manage containers with Kubernetes
Test
createRemoteWebDriverSession(caps)
VM
VM
VM
VM
new RemoteWebDriver(External Endpoint)

Test Cluster
?

github.com/LivePersonInc/ephemerals
Ephemerals

Ephemerals
API
Deploy
Endpoint IP:Portnew RemoteWebDriver(Endpoint IP:Port)
createRemoteWebDriver
(desiredCapabilities)
TestTestTest
TestTestTestTestTest
Test

JUnit Integration
@Rule
public EphemeralResource<RemoteWebDriver> seleniumResource =
new EphemeralResource(
new SeleniumEphemeral.Builder(deploymentContext)
.withBrowser(FIREFOX_42)
.withVideoRecording(true)
.build());
@Test
public void test() {
RemoteWebDriver remoteWebDriver = seleniumResource.get();
.....
}

DEMO

Challenges
● Appium crashes/unresponsive, adb server restarts
● Android SDKs environment installation/maintenance
● Appium↔WebDriver API Versions Incompatibility
● Difficult to parallelise mobile tests
● Scale mobile tests ?!
Mobile Testing
Emulator

Ephemeral’s Way
Emulator
Appium
WebDriver
Test
Install
(APK URL)
Ephemerals
API
createMobileDriver
(capabilities) Deploy
Appium Endpoint
IP:Port
new AndroidDriver(Appium IP:Port)

DEMO

Requirements

Ranking:
Tooling
Deployment/
Configuration
Feedback
Isolation
Efficiency
Scalability
Requirements
❏ Fast feedback
❏ Stable Environment
❏ Isolation
❏ Investigation tools - Hard to support (Video/HAR)
❏ Scale
❏ Efficiency
❏ Browser diversity
❏ Mobile Support

Beyond Selenium...
Deploy any test dependency inside a container
● Web Servers (/modules/nginx)
● Temporary Databases (/modules/mysql)
● Mock Servers (/modules/wiremock)
● Any Docker Image (/modules/custom)

Talking Together...
Web Application
Nginx Ephemeral Selenium Ephemeral
EP
Test Script
EP
WireMock Ephemeral
EP
REST
API
HTTP

Ephemerals - Future Plans
● REST API (for non-Java tests)
● Support more cluster management systems:
○ Apache Mesos
○ Docker Swarm
● More modules

Takeaways
● Enhance traditional Selenium tools
● Containerization - Short lived, throwaway instances
● Scalable tests with Containers Orchestration
● Benefits of Testing on Cloud Platforms

THANK YOU!