Check out the webinar slides by team 99tests on the merits of real world testing. The file includes insights on stages of product development where crowd testing can be applied and detailed explanation on how crowd testing gives a wider coverage in terms of devices, configuration and geography. The recommended testing plan includes mind-mapping, exploratory testing, usability testing

1. Assure End User Experience by
Testing In Real World Scenarios
Kaushik Nath
Head of Product Strategy, 99tests
Bharath S
Director of Operations,
99tests

2. About 99tests

3. 99tests - In A Nutshell
Usability
Testing
Security
Testing
Functional Exploratory
Testing
Automation
Testing
Localisation
Testing
AuFn
Ux LnSc
150+
Countries
20000+
Testers
Test Case Driven
Testing
Tc

4. 99tests Advantage
Save money
& time
Access to top notch
professional testers
across globe
App testing in the
real world
Easy
integration
Flexible Engagement
Models
Testers with
Domain-Specific
Knowledge

5. How we work?
Requirement gathering & analysis +
Test strategy creation
( Through a personalized requirement
discussion, Testing needs & plans are
chalked out )
Project creation & launch Within 24 hours
( Testers & validators are assigned to the projects )
Monitoring Project progress
( Bug logging, Bug Validation & Testing
coverage analysis )
Project delivery
( Sharing bug reports, Test Summary
Reports & Recommendations )

6. Crowd Testing - Relevance

7. Testing For Customer Experience

8. Real World Network Conditions

9. Wide Variety in Screen Size

10. Device coverage

11. Massive variety of OS versions
+
Customization of OS by
manufacturers
100+ Companies

12. Difficulty to test in a distributed environment

13. Real People In Real Condition
Google map vs. Apple map

14. Challenges In Customer Experience Testing

15. The Process
•Clients give broad requirements.
•Users, from the crowd, as they are like real users, are able to expand
to large variety of end user expectations and requirements.
•How do we choose the crowd in such case – We carefully choose with
a broad demographics and expertise
•Exploratory → Test Case (Carefully Designed) → Usability Testing
with Non-Testers (Real Users).

16. The 99tests Process
Exploratory Test case
Carefully Designed - Maximum Coverage
Usability

17. Case Study - Increasing The Review Rating
A mid sized travel segment client
No testing team, agile process, with 15 days release cycle
Regular bug fixes and new features added to the application
Web and Mobile Web Application
What 99tests provided
25 testers across the globe with a mix of travel domain expertise
Coverage of up to 50 devices
Exploratory testing followed by regression cycles
Results:
Reduced the number of issues identified by their actual users
Increase the quality index of the application from 4.2 to 4.8
User Review Rating

18. Case Study - Enhanced End User Experience
For A Leading Airline Service Enterprise In The APJ Region
Large Enterprise Airline Booking Platform
Huge testing team
Web, Android and iOS application
Monthly release cycle
What 99tests provided
50 testers across the globe with a mix of travel and ticketing expertise
Coverage of more than 50 configurations
Exploratory testing followed by regression cycles
Results:
Critical bugs identified before release and reported to the client
Increase of (Internal) Quality Index from 3.8 to 4.4

19. Bugathon

20. THANK YOU