Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Kubernetes your tests! automation with docker on google cloud platform

4,548 views

Published on

Arik Lerner, Automation Team Leader, and Waseem Hamshawi, Automation Infra Developer, present how to build a large scale automated testing platform by leveraging containers orchestration over GCP, with the ability to scale out and provide fast feedback while maintaining a highly reliable test infrastructure.

The presentation includes new approach of managing a scalable testing platform of distributed automated tests with Kubernetes and Docker over Google Cloud Platform.

Topics:

• GCP and Kubernetes introduction for automated testing

• Traditional Selenium Grid vs Selenium Standalone with Kubernetes and Docker for Web and Mobile tests

• Distributed and containerized testing environment over container cluster - different use cases

Ephemerals - "Short-lived Testing Endpoints". An Open Source by LivePerson which makes automation testing at large scale like a "Walk in the park".

Published in: Software
  • Be the first to comment

Kubernetes your tests! automation with docker on google cloud platform

  1. 1. By Arik Lerner & Waseem Hamshawi Kubernetes your Tests! Testing with Docker on Google Cloud Platform The JourneyBased On True Story!
  2. 2. { “Name”: “Arik Lerner”, “Role”: “Automation Infra Team Leader at LivePerson”, “Data”: [ “3 years as Team Leader in Liveperson” , “2 years Java developer Reporting Platform Liveperson” ], “Hobbies”: “Private pilot” } Active Prevention Department
  3. 3. { “Name”: “Waseem Hamshawi”, “Role”: “Senior Automation Infra Developer at Liveperson”, “Data”: [ “2.5 years at Liveperson”, “github: waseemh.github.io”, ], “Hobbies”: “Running” } Waseem is Here! Active Prevention Department
  4. 4. Short Story From Manual Post Deployment Verification TO Automation
  5. 5. ❏ Fast feedback ❏ Stable Environment ❏ Isolation ❏ Investigation tools - Hard to support (Video/HAR) ❏ Scale ❏ Efficiency ❏ Browser diversity ❏ Mobile Support Requirements
  6. 6. Selenium Hub Running Test Static VMs Supported Requirements Browser diversity Mobile Support Parallelism Traditional Selenium Grid
  7. 7. Ranking: Tooling Deployment/ Configuration Feedback Isolation Efficiency Scalability ❏ Less stable ❏ Crashes - require restarts ❏ Cancel build in progress leave open sessions ❏ Investigation tools - Hard to support (Video/HAR) ❏ Manual grid node Configuration/deployment ❏ Not scalable ❏ Slow feedback ❏ Not efficient - idle grid machines Facing Issues
  8. 8. Cattlevs.Pets VM Container ● When they get ill you nurse them back to health ● You name them! And take of them! ● When they get ill you get another one! ● They are almost identical to one another ● Don't have specific names. Given numbers ● Can be simply replaced if they die
  9. 9. What Docker gives us ? ● Build it once run it everywhere! ● Resource Isolation ● Test Isolation ● Package format ● Implementation of a container Docker
  10. 10. Chrome FirefoxVNC Server Selenium Server Hub/Node JAR Mobile Android What is in a Selenium Container ?
  11. 11. VM VM VM VM Nodes HUB N Hub’s Running Tests Selenium Grid Architecture Makeover
  12. 12. Selenium Docker Images hub.docker.com/r/selenium
  13. 13. DEMO
  14. 14. We need more than just Packaging and Isolation: ● Orchestration - Distribute containers ● Health Checks ● Resilience and Self-healing ● Scaling - making jobs bigger or smaller Kubernetes Orchestrator
  15. 15. VM VM VM VM Pods HUB N Hub’s Running Tests Selenium Grid Architecture Makeover
  16. 16. Testing on the Cloud!
  17. 17. Benefits of Cloud Platform ● Kubernetes cluster in a click of a button (GCP) ● No idle machines - Use when needed ● Costs per build/test - measurable ● Not part of Liveperson network ● Geolocation simulation
  18. 18. DEMO
  19. 19. Build Life Cycle
  20. 20. GCP - Selenium Grid lifecycle Run tests Kill Selenium Grid Scale Down GCP Instances Scale Out GCP Instances Create Selenium Grid container cluster
  21. 21. Requirements
  22. 22. Requirements ❏ Fast feedback ❏ Stable Environment ❏ Isolation ❏ Investigation tools - Hard to support (Video/HAR) ❏ Scale ❏ Efficiency ❏ Browser diversity ❏ Mobile Support
  23. 23. Ranking: Tooling Deployment/ Configuration Feedback Isolation Efficiency Scalability ❏ Investigation tools - Video tools not reliable and not stable. ❏ Feedback ❏ Improved BUT still needs to wait for nodes to connect into Hub ❏ Point of failures ❏ Hub/Nodes is more failure points in the architecture that not really necessary Facing Issues
  24. 24. waseemh-mac$ java -jar selenium-server-standalone-2.53.1.jar -role Selenium server can run in one of the following roles: hub as a hub of a Selenium grid node as a node of a Selenium grid standalone as a standalone server not being a part of a grid Test Hub Selenium Server (Node) Selenium Standalone
  25. 25. Non-Grid Solution ● Fast → Deploy Selenium Standalone containers on-demand ● Short-lived → Selenium containers are destroyed once test ends ● Distribute and manage containers with Kubernetes Test createRemoteWebDriverSession(caps) VM VM VM VM new RemoteWebDriver(External Endpoint)
  26. 26. Test Cluster ?
  27. 27. github.com/LivePersonInc/ephemerals Ephemerals
  28. 28. Ephemerals API Deploy Endpoint IP:Portnew RemoteWebDriver(Endpoint IP:Port) createRemoteWebDriver (desiredCapabilities) TestTestTest TestTestTestTestTest Test
  29. 29. JUnit Integration @Rule public EphemeralResource<RemoteWebDriver> seleniumResource = new EphemeralResource( new SeleniumEphemeral.Builder(deploymentContext) .withBrowser(FIREFOX_42) .withVideoRecording(true) .build()); @Test public void test() { RemoteWebDriver remoteWebDriver = seleniumResource.get(); ..... }
  30. 30. DEMO
  31. 31. Challenges ● Appium crashes/unresponsive, adb server restarts ● Android SDKs environment installation/maintenance ● Appium↔WebDriver API Versions Incompatibility ● Difficult to parallelise mobile tests ● Scale mobile tests ?! Mobile Testing Emulator
  32. 32. Ephemeral’s Way Emulator Appium WebDriver Test Install (APK URL) Ephemerals API createMobileDriver (capabilities) Deploy Appium Endpoint IP:Port new AndroidDriver(Appium IP:Port)
  33. 33. DEMO
  34. 34. Requirements
  35. 35. Ranking: Tooling Deployment/ Configuration Feedback Isolation Efficiency Scalability Requirements ❏ Fast feedback ❏ Stable Environment ❏ Isolation ❏ Investigation tools - Hard to support (Video/HAR) ❏ Scale ❏ Efficiency ❏ Browser diversity ❏ Mobile Support
  36. 36. Beyond Selenium... Deploy any test dependency inside a container ● Web Servers (/modules/nginx) ● Temporary Databases (/modules/mysql) ● Mock Servers (/modules/wiremock) ● Any Docker Image (/modules/custom)
  37. 37. Talking Together... Web Application Nginx Ephemeral Selenium Ephemeral EP Test Script EP WireMock Ephemeral EP REST API HTTP
  38. 38. Ephemerals - Future Plans ● REST API (for non-Java tests) ● Support more cluster management systems: ○ Apache Mesos ○ Docker Swarm ● More modules
  39. 39. Takeaways ● Enhance traditional Selenium tools ● Containerization - Short lived, throwaway instances ● Scalable tests with Containers Orchestration ● Benefits of Testing on Cloud Platforms
  40. 40. THANK YOU!

×