Successfully reported this slideshow.

How (Much) Do Developers Test?

0

Share

Loading in …3
×
1 of 39
1 of 39

How (Much) Do Developers Test?

0

Share

Download to read offline

What do we know about software testing in the real world? It seems we know from Fred Brooks’ seminal work “The Mythical Man-Month” that 50% of project effort is spent on testing. However, due to the enormous advances in software engineering in the past 40 years, the question stands: Is this observation still true? In fact, was it ever true? The vision for our research is to settle the discussion about Brooks’ estimation once and for all: How much do developers test? Does developers’ estimation on how much they test match reality? How frequently do they execute their tests, and is there a relationship between test runtime and execution frequency? What are the typical reactions to failing tests? Do developers solve actual defects in the production code, or do they merely relax their test assertions? Emerging results from 40 software engineering students show that students overestimate their testing time threefold, and 50% of them test as little as 4% of their time, or less. Having proven the scalability of our infrastructure, we are now extending our case study with professional software engineers from open-source and industrial organizations.

What do we know about software testing in the real world? It seems we know from Fred Brooks’ seminal work “The Mythical Man-Month” that 50% of project effort is spent on testing. However, due to the enormous advances in software engineering in the past 40 years, the question stands: Is this observation still true? In fact, was it ever true? The vision for our research is to settle the discussion about Brooks’ estimation once and for all: How much do developers test? Does developers’ estimation on how much they test match reality? How frequently do they execute their tests, and is there a relationship between test runtime and execution frequency? What are the typical reactions to failing tests? Do developers solve actual defects in the production code, or do they merely relax their test assertions? Emerging results from 40 software engineering students show that students overestimate their testing time threefold, and 50% of them test as little as 4% of their time, or less. Having proven the scalability of our infrastructure, we are now extending our case study with professional software engineers from open-source and industrial organizations.

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Related Audiobooks

Free with a 14 day trial from Scribd

See all

How (Much) Do Developers Test?

  1. 1. How (Much) Do Developers Test? Moritz Beller @Inventitech Andy Zaidman Georgios Gousios
  2. 2. 50% … of Project Effort Software Testing takes ...
  3. 3. 50% … of Project Effort Brooks, 1975 Software Testing takes ...
  4. 4. Testing means all things to all men
  5. 5. How (Much) Do You Test?
  6. 6. 40
  7. 7. 4.2 yr
  8. 8. OMG!!!
  9. 9. Histogram of Testing Time
  10. 10. Histogram of Testing Time 50% of students test only 4% of their time, or less
  11. 11. Estimated 27%
  12. 12. Estimated 27% Reality 9%
  13. 13. >1,300
  14. 14. Estimated 48%
  15. 15. Estimated 48% Reality 25%
  16. 16. @Inventitech Moritz Beller, TU Delft How (Much) Do Developers Test? TestRoots.org
  17. 17. @Inventitech Moritz Beller, TU Delft How (Much) Do Developers Test? TestRoots.org Code Changes ~ Test Executions
  18. 18. @Inventitech Moritz Beller, TU Delft How (Much) Do Developers Test? TestRoots.org Code Changes ~ Test Executions
  19. 19. @Inventitech Moritz Beller, TU Delft How (Much) Do Developers Test? TestRoots.org Code Changes ~ Test Executions
  20. 20. @Inventitech Moritz Beller, TU Delft How (Much) Do Developers Test? TestRoots.org Code Changes ~ Test Executions Poster Session Thursday, 4pm, Passi Perduti

×