SlideShare a Scribd company logo
How we test at eBay



            Michael Palotas
            Head of Quality Engineering Europe

            Zürich, 17 August 2011
The Company




                                             2
              Produced by: Michael Palotas
eBay Inc.




                                           3
            Produced by: Michael Palotas
Setup


                 Mobile
   Global         Portland           Global
   Product         USA               Product

   San Jose                             Chennai
     USA                                 India

                   EU
                 Quality
               Engineering
    Global                            Local
    Product
               (Zürich, London,
                 Berlin, Paris)
                                     Product

    Shanghai                            Zürich,
     China                              London


                 Adjacencies
                 (i.e. Brands
                 for Friends)
                                                                 4
                                  Produced by: Michael Palotas
Not only the Web


                        Windows
                        Desktop
               Web
                     Mobile




                                                                 5
                                  Produced by: Michael Palotas
Landscape

Teams all around the globe in all possible time zones


                                 90.000 development days / year


                                             ina
                                   India + Ch
                        our ced to
             ece s outs
    M ajor pi
               300 test engineers around the globe



                            nt
                t environme
          tes
 Unstable

Continuou                                  1000+ new bugs found every week
         s Integratio
                     n not cons
                                  istent
   eBay Inc                                              Produced by: Michael Palotas
                                                                                        6
Waterfall...SCRUM…and everything in between




                                                                             7
                                              Produced by: Michael Palotas
Testing at eBay in the past




                               Very manual
        Manual regression testing for 22 sites, every 2 weeks


         Teams did not rely on automation

 Skill set of QA Engineers mainly non-technical

           Sources: www.linesofflight.net
           www.generallyAwesome.com

                                                                                   8
                                                    Produced by: Michael Palotas
Testing Today – Ownership of Quality




                     Manual Test


         Consulting
             +
         Coaching


                       Automation


                                                                      9
                                       Produced by: Michael Palotas
Manual Test



        Still very important, mainly exploratory type

        All new features are tested manually first

                        Edge cases

                   Not automatable tests

   Manual testing still yields the highest number of bugs


                                                                            10
                                             Produced by: Michael Palotas
Automation



       Automation is not about eliminating jobs !!!

       Save time and reinvest into manual testing

             Regression Testing (fully) automated

               Programmatic approach (JAVA)

     Open source only (WebDriver/Selenium 2 GRID)

  3500 end to end testcases running in less than 1 hour
                                                                            11
                                             Produced by: Michael Palotas
GRID

eBay Europe:
-  3500 e2e regression testcases
-  2-3 minutes execution time (each)




                           executions
                                                    Execution time 7000 minutes (~116 hours = ~5 days)
Sequential

                           Parallel
Execution                                          Test
                                                    1
                                                          Test
                                                           2
                                                                 Test
                                                                  3
                                                                        Test
                                                                         …
                                                                               Test
                                                                                …
                                                                                        Test
                                                                                        3500

                                                                                        Execution Time




Parallel                                           Test   Test   Test   Test   Test
                             Parallel executions




                                                                 150 Virtual Machines / Browsers in
execution in                                       Test   Test
                                                                 the WebDriver GRID
                                                                 Test   Test  Test
                                                                 Execution time: 60 minutes
Grid
                                                   Test   Test   Test   Test   Test


                                                                                         Execution Time



                                                                                                                 12
                                                                                  Produced by: Michael Palotas
Coaching / Consulting




             EU team conducts project audits



                  Improvement initiatives



             Consulting for dev and test teams




                                                                           13
                                            Produced by: Michael Palotas
Software Measurement - SONAR




                                                              14
                               Produced by: Michael Palotas
Project Audits




                                                15
                 Produced by: Michael Palotas
EU Quality Engineering Team – Automation Impact


                                                                          Team % of global Bugs 2009
"   Team
       makes up for      20.00%
                         18.00%      Start of
only 2% of global test   16.00%      automation
                                     efforts in
engineer headcount       14.00%
                                     early 2009                                                                                                              Contribution
                         12.00%                                                                                                                              Trendline
                         10.00%
                         8.00%
                         6.00%
"   Clear correlation    4.00%
between global bug       2.00%            headcount share

contribution and          .00%     1 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 13

                                                                                                  Week
automation
investment / coverage    18.0%                                            Team % of global Bugs 2010
                         16.0%
                         14.0%
                         12.0%
                         10.0%
                          8.0%
                          6.0%
                          4.0%
                                       headcount share
                          2.0%          headcount share
                          0.0%                                                                                                                                 16
                                  3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38Palotas 43 44 45 46 47 48 49 50 51
                                                                                                 Produced by: Michael 39 40 41 42
                                                                                                  Week
Bug Quality



Bug Quality = real bugs / submitted bugs
       100
                                    Bug Quality in % (2008-2010)

        95




        90
                            Start of
                            automation
                            efforts
        85                                                                                   Bug Quality 2008-2010




        80




        75
             Q1 08 Q2 08 Q3 08 Q4 08 Q1 09 Q2 09 Q3 09 Q4 09 Q1 10 Q2 10 Q3 10 Q4 10


                                                                                                                      17
                                                                                       Produced by: Michael Palotas
eBay checkbox – vs. HTML checkbox




<div align="center”>
<input type="checkbox" name=”myName" value=”eBay"> eBay<br>
</div>




                                                                            18
                                             Produced by: Michael Palotas
Summary



                   Testing is changing
           Great career opportunities in testing
            More technical skill sets required
          Testautomation can have a big impact
                            ….
                  or make your life hell
            Essential to work with developers
                  Measure and improve

                                                                           19
                                            Produced by: Michael Palotas
Contact: mpalotas@ebay.com



THANK YOU!
QUESTIONS?
                                                 20
                  Produced by: Michael Palotas

More Related Content

What's hot

Uber Mobility Meetup: Mobile Testing
Uber Mobility Meetup:  Mobile TestingUber Mobility Meetup:  Mobile Testing
Uber Mobility Meetup: Mobile Testing
Apple Chow
 
CI in the mobile world
CI in the mobile worldCI in the mobile world
CI in the mobile world
Godfrey Nolan
 
Uber mobility - Build & Release
Uber mobility - Build & ReleaseUber mobility - Build & Release
Uber mobility - Build & Release
Dhaval Patel
 
Thucydides – next generation acceptance testing with WebDriver
Thucydides – next generation acceptance testing with WebDriverThucydides – next generation acceptance testing with WebDriver
Thucydides – next generation acceptance testing with WebDriver
Mikalai Alimenkou
 
Coding With JRebel - Java Forever Changed
Coding With JRebel - Java Forever ChangedCoding With JRebel - Java Forever Changed
Coding With JRebel - Java Forever ChangedElizabeth Quinn-Woods
 
Managing Continuous Delivery of Mobile Apps - for the Enterprise
Managing Continuous Delivery of Mobile Apps - for the EnterpriseManaging Continuous Delivery of Mobile Apps - for the Enterprise
Managing Continuous Delivery of Mobile Apps - for the Enterprise
Sauce Labs
 
Automate you Appium test like a pro!
Automate you Appium test like a pro!Automate you Appium test like a pro!
Automate you Appium test like a pro!
TestObject - Mobile Testing
 
Different Android Test Automation Frameworks - What Works You the Best?
Different Android Test Automation Frameworks - What Works You the Best?Different Android Test Automation Frameworks - What Works You the Best?
Different Android Test Automation Frameworks - What Works You the Best?
Bitbar
 
Selenium + Specflow
Selenium + SpecflowSelenium + Specflow
Selenium + Specflow
cromwellryan
 
Top Best Practices for Successful Mobile Test Automation
Top Best Practices for Successful Mobile Test AutomationTop Best Practices for Successful Mobile Test Automation
Top Best Practices for Successful Mobile Test Automation
Fred Beringer
 
WSO2Con US 2013 - Keynote: Developing Enterprise Apps In the Cloud
WSO2Con US 2013 - Keynote: Developing Enterprise Apps In the CloudWSO2Con US 2013 - Keynote: Developing Enterprise Apps In the Cloud
WSO2Con US 2013 - Keynote: Developing Enterprise Apps In the CloudWSO2
 
Web Application Software Testing
Web Application Software TestingWeb Application Software Testing
Web Application Software Testing
Andrew Kandels
 
How to explain what JRebel does to a developer
How to explain what JRebel does to a developerHow to explain what JRebel does to a developer
How to explain what JRebel does to a developer
Bogomil Shopov
 
Automation Open Source tools
Automation Open Source toolsAutomation Open Source tools
Automation Open Source toolsQA Club Kiev
 
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium
Srijan Technologies
 
Appium vs Appium with Perfecto
Appium vs Appium with PerfectoAppium vs Appium with Perfecto
Appium vs Appium with Perfecto
Lizzy Guido (she/her)
 
Zero redeployment with JRebel
Zero redeployment with JRebelZero redeployment with JRebel
Zero redeployment with JRebel
Minh Hoang
 
Mobile performance metrics and performance monitoring meetup 2017 05 10
Mobile performance metrics and performance monitoring meetup 2017 05 10Mobile performance metrics and performance monitoring meetup 2017 05 10
Mobile performance metrics and performance monitoring meetup 2017 05 10
Bitbar
 
An introduction to unit testing
An introduction to unit testingAn introduction to unit testing
An introduction to unit testing
Adam Stephensen
 
Continuous deployment
Continuous deploymentContinuous deployment
Continuous deploymentDaniel
 

What's hot (20)

Uber Mobility Meetup: Mobile Testing
Uber Mobility Meetup:  Mobile TestingUber Mobility Meetup:  Mobile Testing
Uber Mobility Meetup: Mobile Testing
 
CI in the mobile world
CI in the mobile worldCI in the mobile world
CI in the mobile world
 
Uber mobility - Build & Release
Uber mobility - Build & ReleaseUber mobility - Build & Release
Uber mobility - Build & Release
 
Thucydides – next generation acceptance testing with WebDriver
Thucydides – next generation acceptance testing with WebDriverThucydides – next generation acceptance testing with WebDriver
Thucydides – next generation acceptance testing with WebDriver
 
Coding With JRebel - Java Forever Changed
Coding With JRebel - Java Forever ChangedCoding With JRebel - Java Forever Changed
Coding With JRebel - Java Forever Changed
 
Managing Continuous Delivery of Mobile Apps - for the Enterprise
Managing Continuous Delivery of Mobile Apps - for the EnterpriseManaging Continuous Delivery of Mobile Apps - for the Enterprise
Managing Continuous Delivery of Mobile Apps - for the Enterprise
 
Automate you Appium test like a pro!
Automate you Appium test like a pro!Automate you Appium test like a pro!
Automate you Appium test like a pro!
 
Different Android Test Automation Frameworks - What Works You the Best?
Different Android Test Automation Frameworks - What Works You the Best?Different Android Test Automation Frameworks - What Works You the Best?
Different Android Test Automation Frameworks - What Works You the Best?
 
Selenium + Specflow
Selenium + SpecflowSelenium + Specflow
Selenium + Specflow
 
Top Best Practices for Successful Mobile Test Automation
Top Best Practices for Successful Mobile Test AutomationTop Best Practices for Successful Mobile Test Automation
Top Best Practices for Successful Mobile Test Automation
 
WSO2Con US 2013 - Keynote: Developing Enterprise Apps In the Cloud
WSO2Con US 2013 - Keynote: Developing Enterprise Apps In the CloudWSO2Con US 2013 - Keynote: Developing Enterprise Apps In the Cloud
WSO2Con US 2013 - Keynote: Developing Enterprise Apps In the Cloud
 
Web Application Software Testing
Web Application Software TestingWeb Application Software Testing
Web Application Software Testing
 
How to explain what JRebel does to a developer
How to explain what JRebel does to a developerHow to explain what JRebel does to a developer
How to explain what JRebel does to a developer
 
Automation Open Source tools
Automation Open Source toolsAutomation Open Source tools
Automation Open Source tools
 
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium
[Srijan Wednesday Webinar] Mastering Mobile Test Automation with Appium
 
Appium vs Appium with Perfecto
Appium vs Appium with PerfectoAppium vs Appium with Perfecto
Appium vs Appium with Perfecto
 
Zero redeployment with JRebel
Zero redeployment with JRebelZero redeployment with JRebel
Zero redeployment with JRebel
 
Mobile performance metrics and performance monitoring meetup 2017 05 10
Mobile performance metrics and performance monitoring meetup 2017 05 10Mobile performance metrics and performance monitoring meetup 2017 05 10
Mobile performance metrics and performance monitoring meetup 2017 05 10
 
An introduction to unit testing
An introduction to unit testingAn introduction to unit testing
An introduction to unit testing
 
Continuous deployment
Continuous deploymentContinuous deployment
Continuous deployment
 

Similar to How we Test at eBay Europe

ebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdamebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut PotsdamMichael Palotas
 
Att lyckas med integration av arbetet från flera scrum team - Christophe Acho...
Att lyckas med integration av arbetet från flera scrum team - Christophe Acho...Att lyckas med integration av arbetet från flera scrum team - Christophe Acho...
Att lyckas med integration av arbetet från flera scrum team - Christophe Acho...manssandstrom
 
Implementing Test Automation in Agile Projects
Implementing Test Automation in Agile ProjectsImplementing Test Automation in Agile Projects
Implementing Test Automation in Agile ProjectsMichael Palotas
 
Process Matters (Cloud2Days / Java2Days conference))
Process Matters (Cloud2Days / Java2Days conference))Process Matters (Cloud2Days / Java2Days conference))
Process Matters (Cloud2Days / Java2Days conference))
dev2ops
 
Releasing fast code - The DevOps approach
Releasing fast code - The DevOps approachReleasing fast code - The DevOps approach
Releasing fast code - The DevOps approach
Michael Kopp
 
Dr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex Environments
Dr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex EnvironmentsDr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex Environments
Dr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex EnvironmentsAgileSparks
 
Stop Being Lazy and Test Your Software
Stop Being Lazy and Test Your SoftwareStop Being Lazy and Test Your Software
Stop Being Lazy and Test Your Software
Laura Frank Tacho
 
Mark robinson what does lean mean for software testing
Mark robinson   what does lean mean for software testingMark robinson   what does lean mean for software testing
Mark robinson what does lean mean for software testingAGILEMinds
 
programs testing programs
programs testing programsprograms testing programs
programs testing programsICANS GmbH
 
Testing in an Open Source Middleware Platform Space The WSO2 Way.
Testing in an Open Source Middleware Platform Space  The WSO2 Way.Testing in an Open Source Middleware Platform Space  The WSO2 Way.
Testing in an Open Source Middleware Platform Space The WSO2 Way.WSO2
 
ICTSS 2010 - Iterative Software Testing Process for Scrum and Waterfall Projects
ICTSS 2010 - Iterative Software Testing Process for Scrum and Waterfall ProjectsICTSS 2010 - Iterative Software Testing Process for Scrum and Waterfall Projects
ICTSS 2010 - Iterative Software Testing Process for Scrum and Waterfall Projects
Eliane Collins
 
Continous testing for grails
Continous testing for grailsContinous testing for grails
Continous testing for grails
winkler1
 
Test parallelization using Jenkins
Test parallelization using JenkinsTest parallelization using Jenkins
Test parallelization using Jenkins
Rogue Wave Software
 
Testing in FrontEnd World by Nikita Galkin
Testing in FrontEnd World by Nikita GalkinTesting in FrontEnd World by Nikita Galkin
Testing in FrontEnd World by Nikita Galkin
Sigma Software
 
Containerize your Blackbox tests
Containerize your Blackbox testsContainerize your Blackbox tests
Containerize your Blackbox tests
Kevin Beeman
 
Visual Studio 2010 Testing & Lab Management Tools
Visual Studio 2010 Testing & Lab Management ToolsVisual Studio 2010 Testing & Lab Management Tools
Visual Studio 2010 Testing & Lab Management Tools
Ayman El-Hattab
 
Никита Галкин "Testing in Frontend World"
Никита Галкин "Testing in Frontend World"Никита Галкин "Testing in Frontend World"
Никита Галкин "Testing in Frontend World"
Fwdays
 
Battle for Code Quality - A Story of One Java Project
Battle for Code Quality - A Story of One Java ProjectBattle for Code Quality - A Story of One Java Project
Battle for Code Quality - A Story of One Java Project
GlobalLogic Ukraine
 

Similar to How we Test at eBay Europe (20)

ebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdamebay @ Hasso Plattner Institut Potsdam
ebay @ Hasso Plattner Institut Potsdam
 
Att lyckas med integration av arbetet från flera scrum team - Christophe Acho...
Att lyckas med integration av arbetet från flera scrum team - Christophe Acho...Att lyckas med integration av arbetet från flera scrum team - Christophe Acho...
Att lyckas med integration av arbetet från flera scrum team - Christophe Acho...
 
Implementing Test Automation in Agile Projects
Implementing Test Automation in Agile ProjectsImplementing Test Automation in Agile Projects
Implementing Test Automation in Agile Projects
 
Process Matters (Cloud2Days / Java2Days conference))
Process Matters (Cloud2Days / Java2Days conference))Process Matters (Cloud2Days / Java2Days conference))
Process Matters (Cloud2Days / Java2Days conference))
 
Releasing fast code - The DevOps approach
Releasing fast code - The DevOps approachReleasing fast code - The DevOps approach
Releasing fast code - The DevOps approach
 
Dr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex Environments
Dr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex EnvironmentsDr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex Environments
Dr. Ronen Bar-Nahor - Optimizing Agile Testing in Complex Environments
 
Stop Being Lazy and Test Your Software
Stop Being Lazy and Test Your SoftwareStop Being Lazy and Test Your Software
Stop Being Lazy and Test Your Software
 
test
testtest
test
 
test
testtest
test
 
Mark robinson what does lean mean for software testing
Mark robinson   what does lean mean for software testingMark robinson   what does lean mean for software testing
Mark robinson what does lean mean for software testing
 
programs testing programs
programs testing programsprograms testing programs
programs testing programs
 
Testing in an Open Source Middleware Platform Space The WSO2 Way.
Testing in an Open Source Middleware Platform Space  The WSO2 Way.Testing in an Open Source Middleware Platform Space  The WSO2 Way.
Testing in an Open Source Middleware Platform Space The WSO2 Way.
 
ICTSS 2010 - Iterative Software Testing Process for Scrum and Waterfall Projects
ICTSS 2010 - Iterative Software Testing Process for Scrum and Waterfall ProjectsICTSS 2010 - Iterative Software Testing Process for Scrum and Waterfall Projects
ICTSS 2010 - Iterative Software Testing Process for Scrum and Waterfall Projects
 
Continous testing for grails
Continous testing for grailsContinous testing for grails
Continous testing for grails
 
Test parallelization using Jenkins
Test parallelization using JenkinsTest parallelization using Jenkins
Test parallelization using Jenkins
 
Testing in FrontEnd World by Nikita Galkin
Testing in FrontEnd World by Nikita GalkinTesting in FrontEnd World by Nikita Galkin
Testing in FrontEnd World by Nikita Galkin
 
Containerize your Blackbox tests
Containerize your Blackbox testsContainerize your Blackbox tests
Containerize your Blackbox tests
 
Visual Studio 2010 Testing & Lab Management Tools
Visual Studio 2010 Testing & Lab Management ToolsVisual Studio 2010 Testing & Lab Management Tools
Visual Studio 2010 Testing & Lab Management Tools
 
Никита Галкин "Testing in Frontend World"
Никита Галкин "Testing in Frontend World"Никита Галкин "Testing in Frontend World"
Никита Галкин "Testing in Frontend World"
 
Battle for Code Quality - A Story of One Java Project
Battle for Code Quality - A Story of One Java ProjectBattle for Code Quality - A Story of One Java Project
Battle for Code Quality - A Story of One Java Project
 

More from Michael Palotas

Berlin Selenium Meetup - Galen Framework
Berlin Selenium Meetup -  Galen FrameworkBerlin Selenium Meetup -  Galen Framework
Berlin Selenium Meetup - Galen Framework
Michael Palotas
 
Selenium - The page object pattern
Selenium - The page object patternSelenium - The page object pattern
Selenium - The page object pattern
Michael Palotas
 
Berlin Selenium Meetup - A quick introduction to Selenium
Berlin Selenium Meetup - A quick introduction to SeleniumBerlin Selenium Meetup - A quick introduction to Selenium
Berlin Selenium Meetup - A quick introduction to Selenium
Michael Palotas
 
Zürich selenium meetup mobile and web automation under one umbrella
Zürich selenium meetup mobile and web automation under one umbrellaZürich selenium meetup mobile and web automation under one umbrella
Zürich selenium meetup mobile and web automation under one umbrella
Michael Palotas
 
Agile breakfast St. Gallen - Mindset. Skillset. Toolset
Agile breakfast St. Gallen - Mindset. Skillset. ToolsetAgile breakfast St. Gallen - Mindset. Skillset. Toolset
Agile breakfast St. Gallen - Mindset. Skillset. Toolset
Michael Palotas
 
Mobile Test Automation using one API and one infrastructure
Mobile Test Automation using one API and one infrastructureMobile Test Automation using one API and one infrastructure
Mobile Test Automation using one API and one infrastructure
Michael Palotas
 
Agile Bodensee - Testautomation & Continuous Delivery Workshop
Agile Bodensee - Testautomation & Continuous Delivery WorkshopAgile Bodensee - Testautomation & Continuous Delivery Workshop
Agile Bodensee - Testautomation & Continuous Delivery Workshop
Michael Palotas
 
Agile bodensee - Agile Testing: Bug prevention vs. bug detection
Agile bodensee - Agile Testing: Bug prevention vs. bug detectionAgile bodensee - Agile Testing: Bug prevention vs. bug detection
Agile bodensee - Agile Testing: Bug prevention vs. bug detectionMichael Palotas
 
Testing in the new world-bug prevention vs. bug detection
Testing in the new world-bug prevention vs. bug detectionTesting in the new world-bug prevention vs. bug detection
Testing in the new world-bug prevention vs. bug detectionMichael Palotas
 
Mobile test automation with Selenium, Selendroid and ios-driver
Mobile test automation with Selenium, Selendroid and ios-driverMobile test automation with Selenium, Selendroid and ios-driver
Mobile test automation with Selenium, Selendroid and ios-driverMichael Palotas
 
German Testing Day Keynote - Testing at ebay - a look at a rather unconvent...
German Testing Day Keynote  - Testing at ebay  - a look at a rather unconvent...German Testing Day Keynote  - Testing at ebay  - a look at a rather unconvent...
German Testing Day Keynote - Testing at ebay - a look at a rather unconvent...
Michael Palotas
 
Mobile WebDriver Selendroid
Mobile WebDriver SelendroidMobile WebDriver Selendroid
Mobile WebDriver SelendroidMichael Palotas
 
Scrum breakfast skillset_toolset_mindset
Scrum breakfast skillset_toolset_mindsetScrum breakfast skillset_toolset_mindset
Scrum breakfast skillset_toolset_mindsetMichael Palotas
 
EBAY - A LOOK BEHIND THE SCENES
EBAY -  A LOOK BEHIND THE SCENESEBAY -  A LOOK BEHIND THE SCENES
EBAY - A LOOK BEHIND THE SCENES
Michael Palotas
 
JAVA User Group Bern - Selenium
JAVA User Group Bern  - SeleniumJAVA User Group Bern  - Selenium
JAVA User Group Bern - Selenium
Michael Palotas
 
Mobile Testing and Mobile Automation at eBay
Mobile Testing and Mobile Automation at eBayMobile Testing and Mobile Automation at eBay
Mobile Testing and Mobile Automation at eBayMichael Palotas
 
Swiss Testing Day - Testautomation, 10 (sometimes painful) lessons learned
Swiss Testing Day - Testautomation, 10 (sometimes painful) lessons learnedSwiss Testing Day - Testautomation, 10 (sometimes painful) lessons learned
Swiss Testing Day - Testautomation, 10 (sometimes painful) lessons learnedMichael Palotas
 
Test Automation and Innovation with Open Source Tools
Test Automation and Innovation with Open Source ToolsTest Automation and Innovation with Open Source Tools
Test Automation and Innovation with Open Source Tools
Michael Palotas
 

More from Michael Palotas (18)

Berlin Selenium Meetup - Galen Framework
Berlin Selenium Meetup -  Galen FrameworkBerlin Selenium Meetup -  Galen Framework
Berlin Selenium Meetup - Galen Framework
 
Selenium - The page object pattern
Selenium - The page object patternSelenium - The page object pattern
Selenium - The page object pattern
 
Berlin Selenium Meetup - A quick introduction to Selenium
Berlin Selenium Meetup - A quick introduction to SeleniumBerlin Selenium Meetup - A quick introduction to Selenium
Berlin Selenium Meetup - A quick introduction to Selenium
 
Zürich selenium meetup mobile and web automation under one umbrella
Zürich selenium meetup mobile and web automation under one umbrellaZürich selenium meetup mobile and web automation under one umbrella
Zürich selenium meetup mobile and web automation under one umbrella
 
Agile breakfast St. Gallen - Mindset. Skillset. Toolset
Agile breakfast St. Gallen - Mindset. Skillset. ToolsetAgile breakfast St. Gallen - Mindset. Skillset. Toolset
Agile breakfast St. Gallen - Mindset. Skillset. Toolset
 
Mobile Test Automation using one API and one infrastructure
Mobile Test Automation using one API and one infrastructureMobile Test Automation using one API and one infrastructure
Mobile Test Automation using one API and one infrastructure
 
Agile Bodensee - Testautomation & Continuous Delivery Workshop
Agile Bodensee - Testautomation & Continuous Delivery WorkshopAgile Bodensee - Testautomation & Continuous Delivery Workshop
Agile Bodensee - Testautomation & Continuous Delivery Workshop
 
Agile bodensee - Agile Testing: Bug prevention vs. bug detection
Agile bodensee - Agile Testing: Bug prevention vs. bug detectionAgile bodensee - Agile Testing: Bug prevention vs. bug detection
Agile bodensee - Agile Testing: Bug prevention vs. bug detection
 
Testing in the new world-bug prevention vs. bug detection
Testing in the new world-bug prevention vs. bug detectionTesting in the new world-bug prevention vs. bug detection
Testing in the new world-bug prevention vs. bug detection
 
Mobile test automation with Selenium, Selendroid and ios-driver
Mobile test automation with Selenium, Selendroid and ios-driverMobile test automation with Selenium, Selendroid and ios-driver
Mobile test automation with Selenium, Selendroid and ios-driver
 
German Testing Day Keynote - Testing at ebay - a look at a rather unconvent...
German Testing Day Keynote  - Testing at ebay  - a look at a rather unconvent...German Testing Day Keynote  - Testing at ebay  - a look at a rather unconvent...
German Testing Day Keynote - Testing at ebay - a look at a rather unconvent...
 
Mobile WebDriver Selendroid
Mobile WebDriver SelendroidMobile WebDriver Selendroid
Mobile WebDriver Selendroid
 
Scrum breakfast skillset_toolset_mindset
Scrum breakfast skillset_toolset_mindsetScrum breakfast skillset_toolset_mindset
Scrum breakfast skillset_toolset_mindset
 
EBAY - A LOOK BEHIND THE SCENES
EBAY -  A LOOK BEHIND THE SCENESEBAY -  A LOOK BEHIND THE SCENES
EBAY - A LOOK BEHIND THE SCENES
 
JAVA User Group Bern - Selenium
JAVA User Group Bern  - SeleniumJAVA User Group Bern  - Selenium
JAVA User Group Bern - Selenium
 
Mobile Testing and Mobile Automation at eBay
Mobile Testing and Mobile Automation at eBayMobile Testing and Mobile Automation at eBay
Mobile Testing and Mobile Automation at eBay
 
Swiss Testing Day - Testautomation, 10 (sometimes painful) lessons learned
Swiss Testing Day - Testautomation, 10 (sometimes painful) lessons learnedSwiss Testing Day - Testautomation, 10 (sometimes painful) lessons learned
Swiss Testing Day - Testautomation, 10 (sometimes painful) lessons learned
 
Test Automation and Innovation with Open Source Tools
Test Automation and Innovation with Open Source ToolsTest Automation and Innovation with Open Source Tools
Test Automation and Innovation with Open Source Tools
 

Recently uploaded

Unveiling the Secrets How Does Generative AI Work.pdf
Unveiling the Secrets How Does Generative AI Work.pdfUnveiling the Secrets How Does Generative AI Work.pdf
Unveiling the Secrets How Does Generative AI Work.pdf
Sam H
 
amptalk_RecruitingDeck_english_2024.06.05
amptalk_RecruitingDeck_english_2024.06.05amptalk_RecruitingDeck_english_2024.06.05
amptalk_RecruitingDeck_english_2024.06.05
marketing317746
 
Digital Transformation and IT Strategy Toolkit and Templates
Digital Transformation and IT Strategy Toolkit and TemplatesDigital Transformation and IT Strategy Toolkit and Templates
Digital Transformation and IT Strategy Toolkit and Templates
Aurelien Domont, MBA
 
The Influence of Marketing Strategy and Market Competition on Business Perfor...
The Influence of Marketing Strategy and Market Competition on Business Perfor...The Influence of Marketing Strategy and Market Competition on Business Perfor...
The Influence of Marketing Strategy and Market Competition on Business Perfor...
Adam Smith
 
Brand Analysis for an artist named Struan
Brand Analysis for an artist named StruanBrand Analysis for an artist named Struan
Brand Analysis for an artist named Struan
sarahvanessa51503
 
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdfikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
agatadrynko
 
Exploring Patterns of Connection with Social Dreaming
Exploring Patterns of Connection with Social DreamingExploring Patterns of Connection with Social Dreaming
Exploring Patterns of Connection with Social Dreaming
Nicola Wreford-Howard
 
VAT Registration Outlined In UAE: Benefits and Requirements
VAT Registration Outlined In UAE: Benefits and RequirementsVAT Registration Outlined In UAE: Benefits and Requirements
VAT Registration Outlined In UAE: Benefits and Requirements
uae taxgpt
 
Meas_Dylan_DMBS_PB1_2024-05XX_Revised.pdf
Meas_Dylan_DMBS_PB1_2024-05XX_Revised.pdfMeas_Dylan_DMBS_PB1_2024-05XX_Revised.pdf
Meas_Dylan_DMBS_PB1_2024-05XX_Revised.pdf
dylandmeas
 
Enterprise Excellence is Inclusive Excellence.pdf
Enterprise Excellence is Inclusive Excellence.pdfEnterprise Excellence is Inclusive Excellence.pdf
Enterprise Excellence is Inclusive Excellence.pdf
KaiNexus
 
falcon-invoice-discounting-a-premier-platform-for-investors-in-india
falcon-invoice-discounting-a-premier-platform-for-investors-in-indiafalcon-invoice-discounting-a-premier-platform-for-investors-in-india
falcon-invoice-discounting-a-premier-platform-for-investors-in-india
Falcon Invoice Discounting
 
Putting the SPARK into Virtual Training.pptx
Putting the SPARK into Virtual Training.pptxPutting the SPARK into Virtual Training.pptx
Putting the SPARK into Virtual Training.pptx
Cynthia Clay
 
FINAL PRESENTATION.pptx12143241324134134
FINAL PRESENTATION.pptx12143241324134134FINAL PRESENTATION.pptx12143241324134134
FINAL PRESENTATION.pptx12143241324134134
LR1709MUSIC
 
anas about venice for grade 6f about venice
anas about venice for grade 6f about veniceanas about venice for grade 6f about venice
anas about venice for grade 6f about venice
anasabutalha2013
 
Attending a job Interview for B1 and B2 Englsih learners
Attending a job Interview for B1 and B2 Englsih learnersAttending a job Interview for B1 and B2 Englsih learners
Attending a job Interview for B1 and B2 Englsih learners
Erika906060
 
The Parable of the Pipeline a book every new businessman or business student ...
The Parable of the Pipeline a book every new businessman or business student ...The Parable of the Pipeline a book every new businessman or business student ...
The Parable of the Pipeline a book every new businessman or business student ...
awaisafdar
 
20240425_ TJ Communications Credentials_compressed.pdf
20240425_ TJ Communications Credentials_compressed.pdf20240425_ TJ Communications Credentials_compressed.pdf
20240425_ TJ Communications Credentials_compressed.pdf
tjcomstrang
 
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdfSearch Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Arihant Webtech Pvt. Ltd
 
CADAVER AS OUR FIRST TEACHER anatomt in your.pptx
CADAVER AS OUR FIRST TEACHER anatomt in your.pptxCADAVER AS OUR FIRST TEACHER anatomt in your.pptx
CADAVER AS OUR FIRST TEACHER anatomt in your.pptx
fakeloginn69
 
Maksym Vyshnivetskyi: PMO Quality Management (UA)
Maksym Vyshnivetskyi: PMO Quality Management (UA)Maksym Vyshnivetskyi: PMO Quality Management (UA)
Maksym Vyshnivetskyi: PMO Quality Management (UA)
Lviv Startup Club
 

Recently uploaded (20)

Unveiling the Secrets How Does Generative AI Work.pdf
Unveiling the Secrets How Does Generative AI Work.pdfUnveiling the Secrets How Does Generative AI Work.pdf
Unveiling the Secrets How Does Generative AI Work.pdf
 
amptalk_RecruitingDeck_english_2024.06.05
amptalk_RecruitingDeck_english_2024.06.05amptalk_RecruitingDeck_english_2024.06.05
amptalk_RecruitingDeck_english_2024.06.05
 
Digital Transformation and IT Strategy Toolkit and Templates
Digital Transformation and IT Strategy Toolkit and TemplatesDigital Transformation and IT Strategy Toolkit and Templates
Digital Transformation and IT Strategy Toolkit and Templates
 
The Influence of Marketing Strategy and Market Competition on Business Perfor...
The Influence of Marketing Strategy and Market Competition on Business Perfor...The Influence of Marketing Strategy and Market Competition on Business Perfor...
The Influence of Marketing Strategy and Market Competition on Business Perfor...
 
Brand Analysis for an artist named Struan
Brand Analysis for an artist named StruanBrand Analysis for an artist named Struan
Brand Analysis for an artist named Struan
 
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdfikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
 
Exploring Patterns of Connection with Social Dreaming
Exploring Patterns of Connection with Social DreamingExploring Patterns of Connection with Social Dreaming
Exploring Patterns of Connection with Social Dreaming
 
VAT Registration Outlined In UAE: Benefits and Requirements
VAT Registration Outlined In UAE: Benefits and RequirementsVAT Registration Outlined In UAE: Benefits and Requirements
VAT Registration Outlined In UAE: Benefits and Requirements
 
Meas_Dylan_DMBS_PB1_2024-05XX_Revised.pdf
Meas_Dylan_DMBS_PB1_2024-05XX_Revised.pdfMeas_Dylan_DMBS_PB1_2024-05XX_Revised.pdf
Meas_Dylan_DMBS_PB1_2024-05XX_Revised.pdf
 
Enterprise Excellence is Inclusive Excellence.pdf
Enterprise Excellence is Inclusive Excellence.pdfEnterprise Excellence is Inclusive Excellence.pdf
Enterprise Excellence is Inclusive Excellence.pdf
 
falcon-invoice-discounting-a-premier-platform-for-investors-in-india
falcon-invoice-discounting-a-premier-platform-for-investors-in-indiafalcon-invoice-discounting-a-premier-platform-for-investors-in-india
falcon-invoice-discounting-a-premier-platform-for-investors-in-india
 
Putting the SPARK into Virtual Training.pptx
Putting the SPARK into Virtual Training.pptxPutting the SPARK into Virtual Training.pptx
Putting the SPARK into Virtual Training.pptx
 
FINAL PRESENTATION.pptx12143241324134134
FINAL PRESENTATION.pptx12143241324134134FINAL PRESENTATION.pptx12143241324134134
FINAL PRESENTATION.pptx12143241324134134
 
anas about venice for grade 6f about venice
anas about venice for grade 6f about veniceanas about venice for grade 6f about venice
anas about venice for grade 6f about venice
 
Attending a job Interview for B1 and B2 Englsih learners
Attending a job Interview for B1 and B2 Englsih learnersAttending a job Interview for B1 and B2 Englsih learners
Attending a job Interview for B1 and B2 Englsih learners
 
The Parable of the Pipeline a book every new businessman or business student ...
The Parable of the Pipeline a book every new businessman or business student ...The Parable of the Pipeline a book every new businessman or business student ...
The Parable of the Pipeline a book every new businessman or business student ...
 
20240425_ TJ Communications Credentials_compressed.pdf
20240425_ TJ Communications Credentials_compressed.pdf20240425_ TJ Communications Credentials_compressed.pdf
20240425_ TJ Communications Credentials_compressed.pdf
 
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdfSearch Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdf
 
CADAVER AS OUR FIRST TEACHER anatomt in your.pptx
CADAVER AS OUR FIRST TEACHER anatomt in your.pptxCADAVER AS OUR FIRST TEACHER anatomt in your.pptx
CADAVER AS OUR FIRST TEACHER anatomt in your.pptx
 
Maksym Vyshnivetskyi: PMO Quality Management (UA)
Maksym Vyshnivetskyi: PMO Quality Management (UA)Maksym Vyshnivetskyi: PMO Quality Management (UA)
Maksym Vyshnivetskyi: PMO Quality Management (UA)
 

How we Test at eBay Europe

  • 1. How we test at eBay Michael Palotas Head of Quality Engineering Europe Zürich, 17 August 2011
  • 2. The Company 2 Produced by: Michael Palotas
  • 3. eBay Inc. 3 Produced by: Michael Palotas
  • 4. Setup Mobile Global Portland Global Product USA Product San Jose Chennai USA India EU Quality Engineering Global Local Product (Zürich, London, Berlin, Paris) Product Shanghai Zürich, China London Adjacencies (i.e. Brands for Friends) 4 Produced by: Michael Palotas
  • 5. Not only the Web Windows Desktop Web Mobile 5 Produced by: Michael Palotas
  • 6. Landscape Teams all around the globe in all possible time zones 90.000 development days / year ina India + Ch our ced to ece s outs M ajor pi 300 test engineers around the globe nt t environme tes Unstable Continuou 1000+ new bugs found every week s Integratio n not cons istent eBay Inc Produced by: Michael Palotas 6
  • 7. Waterfall...SCRUM…and everything in between 7 Produced by: Michael Palotas
  • 8. Testing at eBay in the past Very manual Manual regression testing for 22 sites, every 2 weeks Teams did not rely on automation Skill set of QA Engineers mainly non-technical Sources: www.linesofflight.net www.generallyAwesome.com 8 Produced by: Michael Palotas
  • 9. Testing Today – Ownership of Quality Manual Test Consulting + Coaching Automation 9 Produced by: Michael Palotas
  • 10. Manual Test Still very important, mainly exploratory type All new features are tested manually first Edge cases Not automatable tests Manual testing still yields the highest number of bugs 10 Produced by: Michael Palotas
  • 11. Automation Automation is not about eliminating jobs !!! Save time and reinvest into manual testing Regression Testing (fully) automated Programmatic approach (JAVA) Open source only (WebDriver/Selenium 2 GRID) 3500 end to end testcases running in less than 1 hour 11 Produced by: Michael Palotas
  • 12. GRID eBay Europe: -  3500 e2e regression testcases -  2-3 minutes execution time (each) executions Execution time 7000 minutes (~116 hours = ~5 days) Sequential Parallel Execution Test 1 Test 2 Test 3 Test … Test … Test 3500 Execution Time Parallel Test Test Test Test Test Parallel executions 150 Virtual Machines / Browsers in execution in Test Test the WebDriver GRID Test Test Test Execution time: 60 minutes Grid Test Test Test Test Test Execution Time 12 Produced by: Michael Palotas
  • 13. Coaching / Consulting EU team conducts project audits Improvement initiatives Consulting for dev and test teams 13 Produced by: Michael Palotas
  • 14. Software Measurement - SONAR 14 Produced by: Michael Palotas
  • 15. Project Audits 15 Produced by: Michael Palotas
  • 16. EU Quality Engineering Team – Automation Impact Team % of global Bugs 2009 "   Team makes up for 20.00% 18.00% Start of only 2% of global test 16.00% automation efforts in engineer headcount 14.00% early 2009 Contribution 12.00% Trendline 10.00% 8.00% 6.00% "   Clear correlation 4.00% between global bug 2.00% headcount share contribution and .00% 1 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 13 1 2 3 4 5 6 7 8 9 10 11 12 13 Week automation investment / coverage 18.0% Team % of global Bugs 2010 16.0% 14.0% 12.0% 10.0% 8.0% 6.0% 4.0% headcount share 2.0% headcount share 0.0% 16 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38Palotas 43 44 45 46 47 48 49 50 51 Produced by: Michael 39 40 41 42 Week
  • 17. Bug Quality Bug Quality = real bugs / submitted bugs 100 Bug Quality in % (2008-2010) 95 90 Start of automation efforts 85 Bug Quality 2008-2010 80 75 Q1 08 Q2 08 Q3 08 Q4 08 Q1 09 Q2 09 Q3 09 Q4 09 Q1 10 Q2 10 Q3 10 Q4 10 17 Produced by: Michael Palotas
  • 18. eBay checkbox – vs. HTML checkbox <div align="center”> <input type="checkbox" name=”myName" value=”eBay"> eBay<br> </div> 18 Produced by: Michael Palotas
  • 19. Summary Testing is changing Great career opportunities in testing More technical skill sets required Testautomation can have a big impact …. or make your life hell Essential to work with developers Measure and improve 19 Produced by: Michael Palotas
  • 20. Contact: mpalotas@ebay.com THANK YOU! QUESTIONS? 20 Produced by: Michael Palotas