Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Quality engineering challenges in e commerce v1.3


Published on

Published in: Technology
  • Be the first to comment

Quality engineering challenges in e commerce v1.3

  1. 1. QUALITY ENGINEERING CHALLENGESIN E-COMMERCEYotam Sharan,Quality Engineering ManagerMay 2013Ver. 1.3 for publish
  2. 2. WHAT CHALLENGE?WHAT’S THE BIG DEAL?WHAT CAN YOU TELL US?QA documentationRun sanity testsFull-regression timeCOTS test toolsTest AutomationMocks and toolsNoDevelopersMax 1 dayNoX-org F.W.QE does
  3. 3. Quality Engineering Challenges 4Enable merchantsto succeed in newcommerce env’tEnable consumersto buyanything, anywhere,anyhow110MACTIVE USERS25MSELLERS$175BCOMMERCE VOLUMEEBAY MARKETPLACES
  4. 4. Quality Engineering Challenges 5DATA CHALLENGE• Big Data• Various languages• Numerous processes• Subsystems dependencies• QA env. is partial & detached from live data• Daily ETL / deployments• Manual is too long
  5. 5. Quality Engineering Challenges 6AUTOMATED GOLDEN SET• Copy Production data once, as baseline• Use/reuse static dataset & manipulate variables (e.g. ID, timestamps)• Sample specific data, evaluate large sets/volumes• Automate predefined data input – check against expected output• Mocking• Monitors• Reporting: test results, logs, asserts, notifications• Smoke test and monitor ProductionTools we use: Selenium/WD, SoapUI, Jbehave, Jmeter
  6. 6. • Exploratory testing• Regression testing• Compatibility testing• Automation testing• Scenario testing• Unit testing• Code review• Paired testing• Test-driven development• Bug bashes• Capacity planning• Load and Performance testing• Accessibility testing• Mocks and stubs• Analytic testing tools (FindBugs, HTTPWatch, CAL, FireBug, Sonar)TEST INFECTION• Integration testing• E2E testing• Smoke testing• PRD review• Test case review• Dogfood testing• Sprint demo• Security testing• Usability testing• Definition of Done• CI monitors• Production monitors• Live Site Testing• Crowdsourcing
  7. 7. Automation testingAutomate everything, regression first. Plan the right automation approach.Business ValueProviding thin vertical slices of functionality with business value, every iteration.Continuous IntegrationConstant integration testing to assure build integrity and quality, every drop.Design for TestabilityEnsure that it is easier to develop and apply tests for the designed product.Early StartThe whole-team approach engages test engineers at early design phases.AGILE DEV & TESTING PRINCIPLES
  8. 8. Everyone needs to let go of titles and self-image and engage in a constantfeedback, open, transparent, punishment-free work environment.Developer test and Testers developAnalysts, programmers and PM write test scenariosAll engineers help define requirementsQuality is everyone’s responsibilityQUALITY DRIVEN APPROACH
  9. 9. Sequential ExecutionTest execution: 7000 minutesQuality Engineering Challenges 10WEBDRIVER GRIDParallel ExecutionTest execution: 60 minutes150 VMs/Browsers on WD Grid
  10. 10. Quality Engineering Challenges 11WEBDRIVER GRIDSeleniumGrid HubLogAnalyzer:testlogs, stacktrace, serverlogs, configurat’nWDListeners
  11. 11. Quality Engineering Challenges 12INTEGRATED QE PROCESSES• Automation Framework for component reuse• Generic test cases for data-driven testing• Fully-integrated env. for anyone to run tests easily
  12. 12. “In God we trust; all others must bring data” – W. Edwards DemingGut feelings…Engineering clockspeedAdopting/Adjusting best practicesInstant feedback: CI, Pair-x, RT dashboards, Quality indicators (bugs, code QI)Constant feedback: Stories, Planning, Test results, Reviews, Retrospective, SurveysFEEDBACK & FINE-TUNE
  14. 14. I’d love to hear from you  the draw!Work @eBay:jobs-il@ebay.comQuestions?Quality Engineering Challenges 15