TEST ENGINEERING
TO
ENGINEERING PRODUCTIVITY
MICHAEL PALOTAS
EBAY INTERNATIONAL
29. MARCH 2014
Michael Palotas
Head of Productivity & Test Engineering
Pfingstweidstrasse 60
8005 Zürich
Switzerland
Email: mpalotas@ebay...
THE COMPANY
TESTING @ EBAY 3
EBAY INC.
TESTING @ EBAY 4
EBAY FACTS
Founded in 1995
Based in San Jose, California
28000 employees worldwide
>100 million active buyers and sellers ...
EBAY FACTS
TESTING @ EBAY 6
2 billion page views every day
75 billion database calls every day
>200 million downloads of e...
TESTERS
RAISE
YOUR
HANDS
TESTING @ EBAY 7
DEVELOPERS
RAISE
YOUR
HANDS
TESTING @ EBAY 8
AGILE PEOPLE
RAISE
YOUR
HANDS
TESTING @ EBAY 9
NOW SHAKE HANDS
TESTING @ EBAY 10
TESTING @ EBAY 11
LOOKING BACK
TESTING @ EBAY 12
AGILE?
TESTING @ EBAY 13
Development Test
Waterfall - 6 months
Development Test
Agile - 2 weeks
THE BI(U)G HUNT
TESTING @ EBAY 14
AUTOMATION
TESTING @ EBAY 15
OR IS THIS ACTUALLY THE PRESENT FOR YOU?
TESTING @ EBAY 16
HOW ABOUT PREVENTING BUGS?
TESTING @ EBAY 17
PRODUCTIVITY
&
TEST ENGINEERING
Make the team more
efficient
&
productive
TESTING @ EBAY 18
HORIZONTAL / VERTICAL SUPPORT
TESTING @ EBAY 19
TEAM 1 TEAM 2 TEAM 3
VERTICAL VERTICAL VERTICAL
HORIZONTAL SUPPORT
HORIZONTAL SUPPORT
TESTING @ EBAY
TDD
Automation
Advice
Architecture
Infrastructure
20
VERTICAL SUPPORT
Rapid testing / exploratory approach
Focus on fast feedback
Manual testing: Very very very very important...
Do we still need testers?
Or developers only?
TESTING @ EBAY 22
A FEW WORDS ON
(TEST) MANAGEMENT
A WORD ON MEASUREMENT & METRICS
TESTING @ EBAY 24
THE RACE TO ZERO
TESTING @ EBAY
Npr.org
25
OPEN SOURCE
TESTING @ EBAY
It is free
(it is not really free …)
Faster innovation cycles
Independence
Engagement / opportu...
THE LONG LEASH
TESTING @ EBAY 27
HYGIENE FACTORS
TESTING @ EBAY 28
THE BIG GOAL
TESTING @ EBAY 29
THANK YOU!
REFERENCES
• http://weekendreflection.blogspot.ch/
• http://perfectplate.com/
• http://www.jamshid-ibrahim.net/
• http://s...
Upcoming SlideShare
Loading in …5
×

CodeFest 2014. Michael Palotas — Test Engineering to Engineering Productivity: the road from bug detection to bug prevention

1,355 views

Published on

Published in: Internet
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,355
On SlideShare
0
From Embeds
0
Number of Embeds
515
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Expectation would have been that most people are working in testingSince this is the QA track
  • Depending on how many devs are in the room:Point out that there are test conferences vsdev conferencesVery few combined conferences like codefest for exampleI always ask myself why are these two disciplines so separateI sopke at German Testing Day and did the keynote: Requirements, Product management, development and test: quality only is created togetherAnyway, is QA something really so special? Do other people not understand us? Do we as testers try to separate ourselves from the rest of the organization?
  • Who in the room is a so called embedded tester? Who works in an agile team? (and why were you not able to convince your dev counterparts and product managers to attend the session)
  • Tell about Bret Rogers, Sydney KeynoteWhy does he think that way? The way product is built has fundamentally changedBetter tools, better programming languages, smaller incrementsEspecially in agile teams, traditional testers have a really hard time to find their spaceWhat about test driven development? In many agile teams there are no testers at allWhat about Google? Facebook has no testers at all (apparently) In many companies the full effect is not visible yet, because they haven’t gone the agile way yet fullyIn most places there is still a hybrid approach. So Brett Rogers is right, there is a huge wave of change coming to testing in the next 3-5 years
  • We are moving to agile, what are our testers going to do? We keep complaining about being left out etc.
  • Let’s look back: ebay but also the rest of the industryTeams are separated between dev and test Maybe even sometimes geographically (i.e. test factory) Everything done manuallyExtremely untechnicalLots of secondary artifacts: test plan, test cases, reviews,  but not a lot of testsDev and test has contrary goals: dev=few bugs, test=lots of bugsSimilar to DevOps  lots vs little change
  • Who was agile in the room? Everyone is agile but in my opinion very little has changed in so called agile teamsOk, there are embedded testersThey often do the work that devsdont want to doFirst development then testBefore it was 6 monthsAnd it is 2 weeks, we call it a sprintYet, testers still wait for devs to code something and then to test it
  • Main job of a tester used to be to find bugsMy job description used to be: responsible for software quality in europeBUT: I had no influence how the product was developed and how it was built, same for toolsTesters were looking for bugs and were super happy when they found oneBug bashes –> huge exitement about finding lots of bugsIsnt that reilly weird I was thinking about it for a long time: how about preventing bugs instead of finding them Testers have never tried to prevent a bug, instead they were rewarded if they found one
  • Probably one of the biggest failures in testingMillions were spent on tools that promised everything and kept nothingPeople automated because it was part of the processAutomation never in synch with developmentTools used that a dev would never touch Automation was the job of a testerAfter running the tests 50% failedNobody caredAfter 3 consecutive fails they were turned off, in oder to make the build green againAutomation not part of the sprintPeople declared autmation as failed
  • Always asked myself why we cannot not focus on avoiding and preventing bugs rather than just finding them
  • This is what we did at ebay:Renamed the team from quality engineering to productivity & test engineeringQuality is out of the nameTesting is still in thereWhy? Because the whole team is responsible for quality and not just the quality guyFocus is also on productivityWe try to prevent bugs by productive and efficient teamsGood tools, build pipelines, good automated tests
  • ConsultingHelp with i.e. automation tools, infrastructureCIVirtualizationTools, i.e. Selenium GridIos-driver, selendroid
  • There is no set processMay be different per project and per team Focus is on fast feedback No ceremonial stuffTesting often times does many things outside of testing, but not much actual testing BUT: it may be difficult to get the trust when you cannot show which testcases you have checked offManage coverageArticulate risk No independenceWork across SDLC
  • Yes absolutely do we need testersBut a very different kind of testerMch more technical Same language as developersVery versatileLook through user viewCode level  fix bugDo code reviewsReview user storiesSet up CI
  • We don’t have test managers, only testers
  • Maybe unconventional viewAre X bugs good or bad? Is 80% code coverage good or bad? Bug submission goals (15 per week) Testers were measured by how many bugs they submitWho is the better tester: 10 or 20 bugs? It is the change that matters and not the absolute number25% percent of the test cases are automated – huh? We look at metrics like in Sonar, but mot as goals but for information purposesStill important to collect metrics  example of 700 bugs, visibility, top management supportBUT: in agile environments it may be difficult to keep track of every bug. Testers can speak directly to dev and dont need to log a bug
  • Race to zero  products should not cost anything anymoreFor many orgs testing is a pure cost factorCompanies try to lower cost by offshoring, outsourcing Usually it is not cheaper at all actually Outsourcers try to place their engineers as long as possible, thats ok but I as a customer want the oppositeNever seen that an outsources can provide better quality as doing it in houseBrett Rogers: out of 130 outsourced people he would have hired oneWe completely stopped outsourcingReplayed 65 in india with 25 in houseFaster, cheaper and people are super motivated, loyal etc.
  • Job TitelWork hoursWork place WFHLet people fail Support them
  • WLBMoney, will nichtdassjemandwegen 10% woandershingehtbusiness classzero attrition in 4 yearsshower at work Nice dinnerstake taxi from airport
  • Quality needs to be in the DNA of a teamFocus on avoiding bugs instead of finding them afterwardsNo separate team Very technical skill set neededWe need to change on order to keep our jobsTest engineers should be able to fix the bugsMain job of tester: articulate risk, manage coverage, problem solver, mediator2. part: Long leash for peopleInnovation by trust What I told you works in EVERY company – if people want to make it workYou need to be ready for changeMost likely my job as it is today won’t exist in a year from nowThen there will be even more interesting things to do
  • CodeFest 2014. Michael Palotas — Test Engineering to Engineering Productivity: the road from bug detection to bug prevention

    1. 1. TEST ENGINEERING TO ENGINEERING PRODUCTIVITY MICHAEL PALOTAS EBAY INTERNATIONAL 29. MARCH 2014
    2. 2. Michael Palotas Head of Productivity & Test Engineering Pfingstweidstrasse 60 8005 Zürich Switzerland Email: mpalotas@ebay.com Twitter: @michael_palotas LinkedIn: http://ch.linkedin.com/in/michaelpalotas/ WHO AM I? TESTING @ EBAY 2
    3. 3. THE COMPANY TESTING @ EBAY 3
    4. 4. EBAY INC. TESTING @ EBAY 4
    5. 5. EBAY FACTS Founded in 1995 Based in San Jose, California 28000 employees worldwide >100 million active buyers and sellers worldwide 2500 USD transaction volume every second TESTING @ EBAY 5
    6. 6. EBAY FACTS TESTING @ EBAY 6 2 billion page views every day 75 billion database calls every day >200 million downloads of eBay Inc’s mobile apps
    7. 7. TESTERS RAISE YOUR HANDS TESTING @ EBAY 7
    8. 8. DEVELOPERS RAISE YOUR HANDS TESTING @ EBAY 8
    9. 9. AGILE PEOPLE RAISE YOUR HANDS TESTING @ EBAY 9
    10. 10. NOW SHAKE HANDS TESTING @ EBAY 10
    11. 11. TESTING @ EBAY 11
    12. 12. LOOKING BACK TESTING @ EBAY 12
    13. 13. AGILE? TESTING @ EBAY 13 Development Test Waterfall - 6 months Development Test Agile - 2 weeks
    14. 14. THE BI(U)G HUNT TESTING @ EBAY 14
    15. 15. AUTOMATION TESTING @ EBAY 15
    16. 16. OR IS THIS ACTUALLY THE PRESENT FOR YOU? TESTING @ EBAY 16
    17. 17. HOW ABOUT PREVENTING BUGS? TESTING @ EBAY 17
    18. 18. PRODUCTIVITY & TEST ENGINEERING Make the team more efficient & productive TESTING @ EBAY 18
    19. 19. HORIZONTAL / VERTICAL SUPPORT TESTING @ EBAY 19 TEAM 1 TEAM 2 TEAM 3 VERTICAL VERTICAL VERTICAL HORIZONTAL SUPPORT
    20. 20. HORIZONTAL SUPPORT TESTING @ EBAY TDD Automation Advice Architecture Infrastructure 20
    21. 21. VERTICAL SUPPORT Rapid testing / exploratory approach Focus on fast feedback Manual testing: Very very very very important Focus on primary work artifacts Domain Knowledge TESTING @ EBAY 21
    22. 22. Do we still need testers? Or developers only? TESTING @ EBAY 22
    23. 23. A FEW WORDS ON (TEST) MANAGEMENT
    24. 24. A WORD ON MEASUREMENT & METRICS TESTING @ EBAY 24
    25. 25. THE RACE TO ZERO TESTING @ EBAY Npr.org 25
    26. 26. OPEN SOURCE TESTING @ EBAY It is free (it is not really free …) Faster innovation cycles Independence Engagement / opportunities to grow Hiring advantage 26
    27. 27. THE LONG LEASH TESTING @ EBAY 27
    28. 28. HYGIENE FACTORS TESTING @ EBAY 28
    29. 29. THE BIG GOAL TESTING @ EBAY 29
    30. 30. THANK YOU!
    31. 31. REFERENCES • http://weekendreflection.blogspot.ch/ • http://perfectplate.com/ • http://www.jamshid-ibrahim.net/ • http://simplehomemade.net/ • http://npr.org • http://creatia2013.wordpress.com/ • www.kaushik.net • Sackmesser.ch • http://betterafter50.com • http://estonianfreepress.com/ TESTING @ EBAY 31

    ×