Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Crowdsourced Testing Magic

By: MAJA SCHREINER, Senior Test Manager at Swisscom
Aspresented at Spring OnlineTestConf June13-14, 2017

Would you like to hear and learn more about managing testing activities of different agile development teams, working in 1 or 2 week sprints, including involving crowdsourced testers. I can assure you that after some time investments and adjustments you can experience real crowd testing magic.

This presentation will offer testers and others interested in crowd testing the answers to questions like, what challenges you may face, how to adjust your testing strategy, what to expect from crowd testing, for what kind of tests it suited us the best and how to manage different platforms and tools.

What will attendees take away from this session?

– Crowdtesting test strategy

– Managing different Crowdtesting test cycles on a daily basis

– Tools and platforms Keywords Crowdtesting, agile, strategy, learnings, knowledge, sharing

  • Login to see the comments

  • Be the first to like this

Crowdsourced Testing Magic

  1. 1. Maja Schreiner OnlineTestConf, June 2017
  2. 2. Senior Test Master @ Swisscom, Switzerland Twitter: majaschreiner
  3. 3. Crowdsourcing is “Taking a service traditionally performed by a person or team and sourcing it to a large online group of people as an open project”. A unique way of involving crowd i.e. the real users/testers, into software testing under real world conditions. * Definition by Jeff Howe, Wired journalist & author of the book Crowdsourcing
  4. 4.  user-centric software -> success determined by users feedback.  testers experience and interact with the app without preconceptions, like their customers do, they don’t suffer from the confirmation biases that an in-house team would.  end users do not live in testing labs, but in the whole world.  Test coverage for  mobile iOS and Android devices  operating systems  internet browsers.  manual functional testing, cross-browser testing, and iOS and Android testing.  scaling the testing resources in a very short time period (testers test concurrently)  CT typically saves about 30% - 40 % of your budget
  5. 5.  4 Scrum development teams  test lead + in the meantime 3 junior testers employed temporarily  CT company: 2 test leads, lots of testers  Process:  Preparation  Trigger the test cycle  Analyze the results  Approve  Reject  Reward the testers (star the best ones)
  6. 6. inherited the CT contract + team time frame / deadline problems Not possible to scale the testing resources while having only 1 internal resource (tester / test lead) Developers liked the idea of having lots of testers Another big project in our company already used the same crowdsourced testing company
  7. 7. Problems: Developers rejecting lots of bugs Developers generally rejecting the value of CT Distrusting CT Communication taking lots of time Lots of administrative and coordination time needed Impacted everybody in the team Myself feeling a bit stressed and frustrated
  8. 8.  Decide where to start  Track improvements  Inspect and adapt  Never stop explaining and learning  The result  Improved process  Communication with the testers and their leads  Build trust  Add value
  9. 9.  Input  Test scope / products  Test plan  Internal release notes  Sprint review video demo & presentation  Roadmap  All other needed information  Output -> test results including featuring screenshots, video recordings, and crash reports -> test report (dashboard: results summary per Risk, Devices, Browsers, technical or functional etc.)
  10. 10. HOW TO SUCCEED  Know your testing goal & purpose  What  How  When do you want to test (every day, middle of sprint, end of sprint)  Integrate developers in the test cycle and even in the weekly sync calls  Foster the communication with crowdsourced testers  Star and reward the good testers  Keep the best testers in every cycle  Invite new testers – fresh point of view  Smoke test your products before giving them further to CT
  11. 11. FURTHER TIPS  Don’t expect CT to solve all your testing, QA or software development problems  Don’t forget to test internally as well  Beware the costs
  12. 12. Pros for crowdsourced testing Specialized skill sets Scale capacity to meet variable demand Increases fluidity of workforce Keeps fixed cost off balance sheet Good for functional end-user testing of mobile and web apps Contra for crowdsourced testing Not so applicable for the applications which require extensive business know-how
  13. 13. Why and how Strategy Problems How to succeed What to keep in mind
  14. 14. Please contact me for any further questions: Twitter: majaschreiner