Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Test Automation10 (sometimes painful) lessons learnedMichael Palotas & Dominik DaryQuality Engineering EuropeeBay Internat...
eBay Inc. Facts   Founded in 1995   Based in San Jose, California   27770 Employees worldwide
Platform Facts   104 million   active buyers and sellers worldwide   68 Billion USD   transaction volume 2011   2000 USD  ...
Data Facts   9 petabytes of data   In our Hadoop and Teradata clusters   2 billion page views   each day   75 billion data...
Mobile Facts   70,000,000 times   eBay Inc’s apps have been downloaded   5 billion USD   transacted 2011 on mobile devices...
Who we are             Michael Palotas             Head of Quality Engineering Europe             E-mail: mpalotas@ebay.co...
Lesson 1: Flip the testing triangle                                 Manual                                  Tests       Ma...
Lesson 2: Everyone needs to know what isautomatedUse Test Aspects**                   Advantages•  Test aspects briefly de...
Lesson 3: A Tool is not a Strategy    A test tool/testing framework wont    teach your testers how to test    A clear test...
Lesson 4: Write the right tests                           Source: http://farm1.static.flickr.com/119/269335955_03a0ff4b67_...
Lesson 4: Write the right tests    Automate the business critical use    cases of your customer    Focus on end-to-end tes...
Lesson 5: Instant Feedback is essential           Development                     Test D   T D   T D   T D     T D   T D  ...
Lesson 6: Automation is software developmentSkillset & Knowledge                 Engineering practice•  You need people wi...
Lesson 7: Maintainability is kingSource: http://geekandpoke.typepad.com/.a/6a00d8341d3df553ef01348602ac19970c-pi
Lesson 7: Maintainability is kingTechnical Debt                      Lean Test Automation•    Activities that result in   ...
Lesson 8: Speak the same language as thedevelopers     Use the same programming     language as development     Developers...
Lesson 9: Invest into the test infrastructure     Speed up and simplify     the creation of test     data     Offer and us...
Lesson 10: Manual testing is still very important    Automated tests only do what you have implemented:    at the same spe...
Conclusion of our ten lessons learned1.  Flip the testing triangle   6.  Automation is software2.  A tool is not a strateg...
Thank You!Questions or Comments?
Upcoming SlideShare
Loading in …5
×

Swiss Testing Day - Testautomation, 10 (sometimes painful) lessons learned

1,049 views

Published on

Published in: Technology, Education
  • Hello there! Get Your Professional Job-Winning Resume Here! http://bit.ly/topresum
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Swiss Testing Day - Testautomation, 10 (sometimes painful) lessons learned

  1. 1. Test Automation10 (sometimes painful) lessons learnedMichael Palotas & Dominik DaryQuality Engineering EuropeeBay InternationalMarch 2012
  2. 2. eBay Inc. Facts Founded in 1995 Based in San Jose, California 27770 Employees worldwide
  3. 3. Platform Facts 104 million active buyers and sellers worldwide 68 Billion USD transaction volume 2011 2000 USD transaction volume every second
  4. 4. Data Facts 9 petabytes of data In our Hadoop and Teradata clusters 2 billion page views each day 75 billion database calls each day
  5. 5. Mobile Facts 70,000,000 times eBay Inc’s apps have been downloaded 5 billion USD transacted 2011 on mobile devices 1 million items are listed with using mobile apps every week
  6. 6. Who we are Michael Palotas Head of Quality Engineering Europe E-mail: mpalotas@ebay.com Dominik Dary Staff Software Engineer in Test E-mail: ddary@ebay.com
  7. 7. Lesson 1: Flip the testing triangle Manual Tests Manual Tests E2E Tests E2E Automation Tests Integration Tests Unit Tests Unit Tests Inspiration: http://patrickwilsonwelsh.com/?p=32
  8. 8. Lesson 2: Everyone needs to know what isautomatedUse Test Aspects** Advantages•  Test aspects briefly describe •  Clear overview for all team “why” and “what” has to be members about test coverage tested, without describing •  Simplicity helps you to do “how” quick reviews and•  Enables for early testing walkthroughs with your product because tests can be written at owner the same time the user story is •  Enables you to identify manual created and automated test duplicates•  Test aspects are structurally •  Traceability between test derived cases and specification **Swiss Testing Day Presentation 2010: “Mit Testaspekten früh, intelligent und redundanzfrei testen” Melanie Späth Image Source: http://www.masternewmedia.org/images/social_software_impact_individual_organizations.jpg
  9. 9. Lesson 3: A Tool is not a Strategy A test tool/testing framework wont teach your testers how to test A clear testing process is mandatory for test automation Test Aspects and modeling of the Biz-Domain layer is the key success factor for your test strategy
  10. 10. Lesson 4: Write the right tests Source: http://farm1.static.flickr.com/119/269335955_03a0ff4b67_o.jpg
  11. 11. Lesson 4: Write the right tests Automate the business critical use cases of your customer Focus on end-to-end testing rather than UI component tests Design testability into your product
  12. 12. Lesson 5: Instant Feedback is essential Development Test D T D T D T D T D T D T D T D T
  13. 13. Lesson 6: Automation is software developmentSkillset & Knowledge Engineering practice•  You need people with software •  Task oriented development development skills and an helps you to get things done appetite for testing •  Follow the basic rules like lean•  Automation mindset is essential development principles•  Good understanding of the •  Design the test before coding it architecture of the application •  Do code reviews under test is required •  Your developers can give you very good feedback about your code quality
  14. 14. Lesson 7: Maintainability is kingSource: http://geekandpoke.typepad.com/.a/6a00d8341d3df553ef01348602ac19970c-pi
  15. 15. Lesson 7: Maintainability is kingTechnical Debt Lean Test Automation•  Activities that result in •  Eliminate Waste Definition of Done technical debt include •  Build Quality In Reviews •  Postponing documentation and •  Create Knowledge Document the essentials writing tests •  Defer Commitment Don’t build a framework •  attending to TODO •  Deliver Fast comments and •  Ignoring compiler and •  Respect People Long Leash static code analysis •  Optimize the Whole warnings•  Knowledge that isnt shared around the organization•  Code that is too confusing to be modified easily
  16. 16. Lesson 8: Speak the same language as thedevelopers Use the same programming language as development Developers can write or maintain tests if needed Enables for a cross functional team where everyone can work on nearly every task
  17. 17. Lesson 9: Invest into the test infrastructure Speed up and simplify the creation of test data Offer and use web services Web based test data creation tools can be used in test auto- mation and manual testing
  18. 18. Lesson 10: Manual testing is still very important Automated tests only do what you have implemented: at the same speed, same order, with exactly the same mouse clicks Manual testing allows you to go off the beaten path Test automation in combination with exploratory testing combine the best of both worlds Quality = Manual + Automated Testing
  19. 19. Conclusion of our ten lessons learned1.  Flip the testing triangle 6.  Automation is software2.  A tool is not a strategy development itself3.  Everyone knows 7.  Maintainability is king what’s automated 8.  Speak the same4.  Write the right tests language as the5.  Instant Feedback is developers essential 9.  Invest into the test Infrastructure 10. Manual testing is still very important
  20. 20. Thank You!Questions or Comments?

×