QUALITY ENGINEERING CHALLENGESIN E-COMMERCEYotam Sharan,Quality Engineering ManagerMay 2013Ver. 1.3 for publish
WHAT CHALLENGE?WHAT’S THE BIG DEAL?WHAT CAN YOU TELL US?QA documentationRun sanity testsFull-regression timeCOTS test tool...
Quality Engineering Challenges 4Enable merchantsto succeed in newcommerce env’tEnable consumersto buyanything, anywhere,an...
Quality Engineering Challenges 5DATA CHALLENGE• Big Data• Various languages• Numerous processes• Subsystems dependencies• ...
Quality Engineering Challenges 6AUTOMATED GOLDEN SET• Copy Production data once, as baseline• Use/reuse static dataset & m...
• Exploratory testing• Regression testing• Compatibility testing• Automation testing• Scenario testing• Unit testing• Code...
Automation testingAutomate everything, regression first. Plan the right automation approach.Business ValueProviding thin v...
Everyone needs to let go of titles and self-image and engage in a constantfeedback, open, transparent, punishment-free wor...
Sequential ExecutionTest execution: 7000 minutesQuality Engineering Challenges 10WEBDRIVER GRIDParallel ExecutionTest exec...
Quality Engineering Challenges 11WEBDRIVER GRIDSeleniumGrid HubLogAnalyzer:testlogs, stacktrace, serverlogs, configurat’nW...
Quality Engineering Challenges 12INTEGRATED QE PROCESSES• Automation Framework for component reuse• Generic test cases for...
“In God we trust; all others must bring data” – W. Edwards DemingGut feelings…Engineering clockspeedAdopting/Adjusting bes...
CONTINUOUS INTEGRATION & CODE QUALITY
I’d love to hear from you www.ebay-m.com/yotamWin the draw!Work @eBay:jobs-il@ebay.comQuestions?Quality Engineering Chall...
Quality engineering challenges in e commerce v1.3
Upcoming SlideShare
Loading in...5
×

Quality engineering challenges in e commerce v1.3

377

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
377
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
15
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Quality engineering challenges in e commerce v1.3

  1. 1. QUALITY ENGINEERING CHALLENGESIN E-COMMERCEYotam Sharan,Quality Engineering ManagerMay 2013Ver. 1.3 for publish
  2. 2. WHAT CHALLENGE?WHAT’S THE BIG DEAL?WHAT CAN YOU TELL US?QA documentationRun sanity testsFull-regression timeCOTS test toolsTest AutomationMocks and toolsNoDevelopersMax 1 dayNoX-org F.W.QE does
  3. 3. Quality Engineering Challenges 4Enable merchantsto succeed in newcommerce env’tEnable consumersto buyanything, anywhere,anyhow110MACTIVE USERS25MSELLERS$175BCOMMERCE VOLUMEEBAY MARKETPLACES
  4. 4. Quality Engineering Challenges 5DATA CHALLENGE• Big Data• Various languages• Numerous processes• Subsystems dependencies• QA env. is partial & detached from live data• Daily ETL / deployments• Manual is too long
  5. 5. Quality Engineering Challenges 6AUTOMATED GOLDEN SET• Copy Production data once, as baseline• Use/reuse static dataset & manipulate variables (e.g. ID, timestamps)• Sample specific data, evaluate large sets/volumes• Automate predefined data input – check against expected output• Mocking• Monitors• Reporting: test results, logs, asserts, notifications• Smoke test and monitor ProductionTools we use: Selenium/WD, SoapUI, Jbehave, Jmeter
  6. 6. • Exploratory testing• Regression testing• Compatibility testing• Automation testing• Scenario testing• Unit testing• Code review• Paired testing• Test-driven development• Bug bashes• Capacity planning• Load and Performance testing• Accessibility testing• Mocks and stubs• Analytic testing tools (FindBugs, HTTPWatch, CAL, FireBug, Sonar)TEST INFECTION• Integration testing• E2E testing• Smoke testing• PRD review• Test case review• Dogfood testing• Sprint demo• Security testing• Usability testing• Definition of Done• CI monitors• Production monitors• Live Site Testing• Crowdsourcing
  7. 7. Automation testingAutomate everything, regression first. Plan the right automation approach.Business ValueProviding thin vertical slices of functionality with business value, every iteration.Continuous IntegrationConstant integration testing to assure build integrity and quality, every drop.Design for TestabilityEnsure that it is easier to develop and apply tests for the designed product.Early StartThe whole-team approach engages test engineers at early design phases.AGILE DEV & TESTING PRINCIPLES
  8. 8. Everyone needs to let go of titles and self-image and engage in a constantfeedback, open, transparent, punishment-free work environment.Developer test and Testers developAnalysts, programmers and PM write test scenariosAll engineers help define requirementsQuality is everyone’s responsibilityQUALITY DRIVEN APPROACH
  9. 9. Sequential ExecutionTest execution: 7000 minutesQuality Engineering Challenges 10WEBDRIVER GRIDParallel ExecutionTest execution: 60 minutes150 VMs/Browsers on WD Grid
  10. 10. Quality Engineering Challenges 11WEBDRIVER GRIDSeleniumGrid HubLogAnalyzer:testlogs, stacktrace, serverlogs, configurat’nWDListeners
  11. 11. Quality Engineering Challenges 12INTEGRATED QE PROCESSES• Automation Framework for component reuse• Generic test cases for data-driven testing• Fully-integrated env. for anyone to run tests easily
  12. 12. “In God we trust; all others must bring data” – W. Edwards DemingGut feelings…Engineering clockspeedAdopting/Adjusting best practicesInstant feedback: CI, Pair-x, RT dashboards, Quality indicators (bugs, code QI)Constant feedback: Stories, Planning, Test results, Reviews, Retrospective, SurveysFEEDBACK & FINE-TUNE
  13. 13. CONTINUOUS INTEGRATION & CODE QUALITY
  14. 14. I’d love to hear from you www.ebay-m.com/yotamWin the draw!Work @eBay:jobs-il@ebay.comQuestions?Quality Engineering Challenges 15
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×