2018 State of Testing Survey Results

1. The 2018 State of Testing
Survey Results

2. A Quick Introduction
Gregg Hanson Bria Grangard

3. Page
Today’s Agenda
•
•
•
•

4. Create Great Software, Faster
Perform Code
and Doc Review
Collaborator
Design, Develop, and
Document APIs
SwaggerHub
UI
API
DEV TEST OPS
Code Script
Run Tests On Real Devices
in the Cloud
CrossBrowserTesting
Create Web Load Tests
LoadComplete
Spec
Create Automated API Functional Tests
(REST, SOAP, and more)
SoapUI Pro Script
Virtualize API and
Database Services
ServiceV Pro
Create API Load Tests
LoadUI Pro
Monitor Web and API Performance,
Availability, and Functional Correctness
AlertSite
Manage Manual and
Automated Tests
QAComplete
Integrations …100 +
Collaborate on an Idea using BDD, Test
Continuously, and Generate Living Docs
Hiptest
TestComplete
Create Automated UI
Functional Tests
(Web, Desktop, Mobile)
Create Automated UI
Tests in IDE Using BDD
TestLeft

5. Who is joining us
today?

6. Survey Methodology

7. Survey Methodology & Logistics
• 2,676responses
• 1,700completedresponsesincludingManualTesters,Automation
Engineers,Developers,Consultants,QAManagers,andAnalysts
• 45-questionsurvey
• July2018
• 16differentindustriesglobally

8. Demographic & Firmographic Data
Thousands of individuals globally responded to the Stateof Testing Survey

9. AWideVariety of Industries,…

10. … Team Sizes,and Roles

11. High Level Takeaways

12. Application Details &
Development Models

13. The API Economy is Booming
• Percentageof respondents who reported
testing APIs grew:
• 71%  80%
• All other typesof applications dropped from
2017 to2018
• Web, Desktop, Mobile, etc.
• “Other” popular responses
• Mainframe (6%)
• Embedded (6%)

14. What type of development
methodology are you practicing
today?

15. Agile holds as the dominant development methodology.
Respondents were asked toselect all models they follow
63% of
respondents
reported
following only
one model

16. Release cycles are getting shorter.

17. Jenkins is the main player when it comes to continuous integration.
100% of survey
respondents
reported using
a CI tool

18. Java reigns supreme.

19. Testing Approaches &
Tools

20. The same software professionals test both the front-
end and the back-end.

21. What types of tests are you
focused on automating?

22. Automation is growing,but not quickly.
Automated API testing is the fastest growing testing practice
• Automated APITesting grew:
• 49% to 54%
• Automated UItesting held at62% of
respondents
• Automated performancetesting
grew:
• 50% to54%
• Almost aquarter of respondents are
practicing TDD or BDD

23. API Testing

24. External API testing is growing.
In2017,71%ofrespondentsweretestingsomekindofAPIorwebservice.In2018,thatnumbergrewto80%.
• More developers and testers aretesting third-
partyand externalAPIs, rather than just internal
APIs
• The number of respondents doing nothing to
remedy externalAPIdependencies dropped
from 16%to12%

25. APIs are growing in numbers.
31% of survey respondents are testing between 51 and 500 endpoints.

26. Teams spend more time on Functional Testing than on any other
API testing practice.

27. UI Testing

28. Are you responsible for
both your API testing and
UI testing?

29. The majority of UI tests are still created using scripting.

30. Chrome,Firefox,and Internet Explorer remain as the three most
popular browsers to test applications.
Overhalf of QA Teams (51%) test exclusively on the latest browser.

31. You need to test on the right devices,browsers,and operating
systems to provide as close to a flawless user experience as
possible.

32. The number of testers executing tests in parallel has increased.
Running tests concurrently on multiple devices expands test coveragewhile reducing testing time.

33. Load Testing

34. Less than half of all QA professionals are load
testing their APIs.

35. Test Management

36. Teams are more likely to automate when they invest in a test
management tool.

37. The Future of Testing

38. What will the state of software testing look like two years from now?
Everyone expects automation to grow and release cadencesto shorten.

40. Would you like more information
on any testing tools and best
practices?

41. Question & Answer