How we Test at eBay Europe

1,426 views

Published on

Published in: Business, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,426
On SlideShare
0
From Embeds
0
Number of Embeds
39
Actions
Shares
0
Downloads
21
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

How we Test at eBay Europe

  1. 1. How we test at eBay Michael Palotas Head of Quality Engineering Europe Zürich, 17 August 2011
  2. 2. The Company 2 Produced by: Michael Palotas
  3. 3. eBay Inc. 3 Produced by: Michael Palotas
  4. 4. Setup Mobile Global Portland Global Product USA Product San Jose Chennai USA India EU Quality Engineering Global Local Product (Zürich, London, Berlin, Paris) Product Shanghai Zürich, China London Adjacencies (i.e. Brands for Friends) 4 Produced by: Michael Palotas
  5. 5. Not only the Web Windows Desktop Web Mobile 5 Produced by: Michael Palotas
  6. 6. LandscapeTeams all around the globe in all possible time zones 90.000 development days / year ina India + Ch our ced to ece s outs M ajor pi 300 test engineers around the globe nt t environme tes UnstableContinuou 1000+ new bugs found every week s Integratio n not cons istent eBay Inc Produced by: Michael Palotas 6
  7. 7. Waterfall...SCRUM…and everything in between 7 Produced by: Michael Palotas
  8. 8. Testing at eBay in the past Very manual Manual regression testing for 22 sites, every 2 weeks Teams did not rely on automation Skill set of QA Engineers mainly non-technical Sources: www.linesofflight.net www.generallyAwesome.com 8 Produced by: Michael Palotas
  9. 9. Testing Today – Ownership of Quality Manual Test Consulting + Coaching Automation 9 Produced by: Michael Palotas
  10. 10. Manual Test Still very important, mainly exploratory type All new features are tested manually first Edge cases Not automatable tests Manual testing still yields the highest number of bugs 10 Produced by: Michael Palotas
  11. 11. Automation Automation is not about eliminating jobs !!! Save time and reinvest into manual testing Regression Testing (fully) automated Programmatic approach (JAVA) Open source only (WebDriver/Selenium 2 GRID) 3500 end to end testcases running in less than 1 hour 11 Produced by: Michael Palotas
  12. 12. GRIDeBay Europe:-  3500 e2e regression testcases-  2-3 minutes execution time (each) executions Execution time 7000 minutes (~116 hours = ~5 days)Sequential ParallelExecution Test 1 Test 2 Test 3 Test … Test … Test 3500 Execution TimeParallel Test Test Test Test Test Parallel executions 150 Virtual Machines / Browsers inexecution in Test Test the WebDriver GRID Test Test Test Execution time: 60 minutesGrid Test Test Test Test Test Execution Time 12 Produced by: Michael Palotas
  13. 13. Coaching / Consulting EU team conducts project audits Improvement initiatives Consulting for dev and test teams 13 Produced by: Michael Palotas
  14. 14. Software Measurement - SONAR 14 Produced by: Michael Palotas
  15. 15. Project Audits 15 Produced by: Michael Palotas
  16. 16. EU Quality Engineering Team – Automation Impact Team % of global Bugs 2009"   Team makes up for 20.00% 18.00% Start ofonly 2% of global test 16.00% automation efforts inengineer headcount 14.00% early 2009 Contribution 12.00% Trendline 10.00% 8.00% 6.00%"   Clear correlation 4.00%between global bug 2.00% headcount sharecontribution and .00% 1 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 13 Weekautomationinvestment / coverage 18.0% Team % of global Bugs 2010 16.0% 14.0% 12.0% 10.0% 8.0% 6.0% 4.0% headcount share 2.0% headcount share 0.0% 16 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38Palotas 43 44 45 46 47 48 49 50 51 Produced by: Michael 39 40 41 42 Week
  17. 17. Bug QualityBug Quality = real bugs / submitted bugs 100 Bug Quality in % (2008-2010) 95 90 Start of automation efforts 85 Bug Quality 2008-2010 80 75 Q1 08 Q2 08 Q3 08 Q4 08 Q1 09 Q2 09 Q3 09 Q4 09 Q1 10 Q2 10 Q3 10 Q4 10 17 Produced by: Michael Palotas
  18. 18. eBay checkbox – vs. HTML checkbox<div align="center”><input type="checkbox" name=”myName" value=”eBay"> eBay<br></div> 18 Produced by: Michael Palotas
  19. 19. Summary Testing is changing Great career opportunities in testing More technical skill sets required Testautomation can have a big impact …. or make your life hell Essential to work with developers Measure and improve 19 Produced by: Michael Palotas
  20. 20. Contact: mpalotas@ebay.comTHANK YOU!QUESTIONS? 20 Produced by: Michael Palotas

×