Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Quality automation at walmart scale

25 views

Published on

Implementing Quality Automation at the scale of Walmart

Published in: Software
  • Be the first to comment

  • Be the first to like this

Quality automation at walmart scale

  1. 1. Pranav Verma Sr. Engineering Manager Developer Experience Tools EMAIL: pverma@walmartlabs.com Quality Automation at Walmart Scale October 23, 2018 E X P E R I E N C E T O O L S CUSTO M ER E X P E R I E N C E T O O L S DEVELOPER
  2. 2. Agenda • Scope Across Walmart • Test Armada Infrastructure • Continuous Integration • Automation Cloud • Data Insights • Q&A
  3. 3. Scope Across Walmart TEAMS WE SUPPORT 85 projects in 2 years 200 projects by 2020 1.2M Development Hours Saved 19.5M Functional Tests Executed 196M Mocked API Calls LOCATIONS WE SUPPORT NUMBERS AT SCALE Bay Area San Diego Bangalore Canada Omaha Hoboken Bentonville UK Mexico Point of Sales RCT ASDA Growth USGM Jet.com Pharmacy GIF Store Systems OmniChannel Customer Promise Hayneedle USGR EXPO Portland
  4. 4. Infrastructure at Scale AUTOMATION CLOUD CONTINUOUS INTEGRATION DATA INSIGHTS
  5. 5. How to run automation tests fast at a massive scale reliably. CONTINUOUS INTEGRATION PROBLEM Tests need to run against multiple OS/Browsers, which multiplies the numbers of tests. Need a solution to run massive number of tests without compromising execution time. GOAL
  6. 6. SOLUTION Shard Master Homepage 64 TESTS Item Page 65 TESTS Cart 228 TESTS 64 TestsShard 1 64 TestsShard 2 64 TestsShard 3 65 TestsShard 1 65 TestsShard 2 228 TestsShard 1 228 TestsShard 2 228 TestsShard 3 228 TestsShard 4 Thread 2 Thread 1 Thread 3 Thread 4 Thread n
  7. 7. BENEFITS CART PR Verify (228 Tests on 2 Browsers) 6 HOURSAverage Serial Test Execution Time 25 MINUTESAverage Parallel Test Execution Time to 93% Savings =
  8. 8. How to test Walmart websites and native apps on all supported browsers and devices. AUTOMATION CLOUD PROBLEM Need a solution to support automation reliably at Walmart scale, and also allows team to configure as per their testing needs. GOAL
  9. 9. SOLUTION Browser Plugin Device Plugin Test Agent Plugin Resource Manager Reconciliation Engine Quota Management Priority Management ResourcesTest LabsPlugin Architecture Internal Cloud External Cloud Internal Cloud External Cloud Internal Cloud External Cloud
  10. 10. BENEFITS 1,000+ Unique combinations 1,150 Available Browser/OS combinations Devices in a private cloud 81 250KAutomated Developer Hours Per Day Unlimited Devices 30KEngineers Working Per Day
  11. 11. How to see health of your project in one place. DATA INSIGHTS PROBLEM The application quality information is scattered between JIRA, CI, Test frameworks and Automation clouds. Need a solution to visualize all information in one place to make informed decisions. GOAL
  12. 12. Data Insights Platform Managed Services SOLUTION Functional Test Agent Perf Test Agent CI … DB Functional Report Client Perf Report Backend Perf Report Self-Service Portal API H X HYGIEIA XRAY CLIENT
  13. 13. BENEFITS Messages per Second 1Million SCALABLE PROCESSING Report Retention 6Months TRENDING REPORTS Latency 1Minute REAL TIME REPORTS
  14. 14. Common Test Armada Questions Can Test Armada support Java Tests? Can I use Test Armada for FE testing if I am not using Nightwatch? How can I use Test Armada for BE Testing? Can I use Test Armada Data Insights with my custom test runner?
  15. 15. r Open Test Armada Platform Resource Manager Self ServiceData Insights TEST ARMADA FEATURES PROGRAMMING LANGUAGE Magellan TEST RUNNER Test Armada TDK TEST ARMADA LIBRERIES OUT OF THE BOX SOLUTION Any Language PROGRAMMING LANGUAGE Any Runner TEST RUNNER Test Armada Service TEST ARMADAAPI Resource Manager Self ServiceData Insights TEST ARMADA FEATURES CUSTOMIZED SOLUTION
  16. 16. Q&A
  17. 17. Contact Us @testarmada github.com/testarmada http://www.testarmada.io

×