How (Much) Do Developers Test?

•

0 likes•1,453 views

What do we know about software testing in the real world? It seems we know from Fred Brooks’ seminal work “The Mythical Man-Month” that 50% of project effort is spent on testing. However, due to the enormous advances in software engineering in the past 40 years, the question stands: Is this observation still true? In fact, was it ever true? The vision for our research is to settle the discussion about Brooks’ estimation once and for all: How much do developers test? Does developers’ estimation on how much they test match reality? How frequently do they execute their tests, and is there a relationship between test runtime and execution frequency? What are the typical reactions to failing tests? Do developers solve actual defects in the production code, or do they merely relax their test assertions? Emerging results from 40 software engineering students show that students overestimate their testing time threefold, and 50% of them test as little as 4% of their time, or less. Having proven the scalability of our infrastructure, we are now extending our case study with professional software engineers from open-source and industrial organizations.

Software

How (Much) Do Developers
Test?
Moritz Beller
@Inventitech
Andy Zaidman
Georgios Gousios

50%
… of Project Effort
Software Testing takes ...

50%
… of Project Effort
Brooks, 1975
Software Testing takes ...

Histogram of Testing Time
50% of students test only 4% of their
time, or less

@Inventitech
Moritz Beller, TU Delft
How (Much) Do Developers Test?
TestRoots.org

@Inventitech
Moritz Beller, TU Delft
How (Much) Do Developers Test?
TestRoots.org
Code Changes
~
Test Executions

@Inventitech
Moritz Beller, TU Delft
How (Much) Do Developers Test?
TestRoots.org
Code Changes
~
Test Executions
Poster Session
Thursday, 4pm,
Passi Perduti

What's hot

A Happy Marriage between Context-Driven and AgileIlari Henrik Aegerter

Breathing the breath of the monster combining agile and context-drivenIlari Henrik Aegerter

The Will TestWilliam Hill

Devops its not about the toolingBram Vogelaar

What We Learned from Three Years of Sciencing the Crap Out of DevOpsSeniorStoryteller

Conversion Hotel 2018 Keynote: Aleksander FabijanWebanalisten .nl

Testing for everyoneMažvydas Skuodas

Bug debug keynote - Present problems and future solutionsRIA RUI Society

Claudiu Draghia ITEM 2018ITEM

Guess The Pm Gamejamesbb

Pertanyaan dan jawaban (graham et.al 2011) part 1ikhwana luthfi

Worst practices in software testing by the Testing trollViktor Slavchev

Hindsight lessons about API testingViktor Slavchev

Break to build - the mindset of the modern day testerViktor Slavchev

Agile Testers: Becoming a key asset for your teamgojkoadzic

Test automation – the bitter truthViktor Slavchev

Continuous Testing: The Path ForwardPerfecto by Perforce

Testing for everyone agile yorkshireAdy Stokes

Charity Majors - Bootstrapping an Ops TeamHeavybit

Automation vs. intelligence - "follow me if you want to live"Viktor Slavchev

What's hot (20)

A Happy Marriage between Context-Driven and Agile

Breathing the breath of the monster combining agile and context-driven

The Will Test

Devops its not about the tooling

What We Learned from Three Years of Sciencing the Crap Out of DevOps

Conversion Hotel 2018 Keynote: Aleksander Fabijan

Testing for everyone

Bug debug keynote - Present problems and future solutions

Claudiu Draghia ITEM 2018

Guess The Pm Game

Pertanyaan dan jawaban (graham et.al 2011) part 1

Worst practices in software testing by the Testing troll

Hindsight lessons about API testing

Break to build - the mindset of the modern day tester

Agile Testers: Becoming a key asset for your team

Test automation – the bitter truth

Continuous Testing: The Path Forward

Testing for everyone agile yorkshire

Charity Majors - Bootstrapping an Ops Team

Automation vs. intelligence - "follow me if you want to live"

Viewers also liked

Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Sour...Moritz Beller

Modern Code Reviews in Open Source Projects: Which Problems Do They Fix?Moritz Beller

The Last Line EffectMoritz Beller

Approach to a child with hematemesis or melenaAvijeet Mishra

Supply Chain Management: Technology's Role in Achieving Optimum PerformanceJump Technologies, Inc.

Spike Jonze Case StudyEl_Lester

Evaluation 4&5stephanie983

Codes and conventionstephanie983

대학생태환경의 급격 변화경선 이

Webinar: Mental Health Services in Schools For Students with Intellectual / D...USC UCEDD at Children's Hospital Los Angeles

Perceiving and believing 6th sessionTAUSIQUE SHEIKH

Hubungan etnik-di-malaysia-group-1Siti Norazlina

VPS Hosting Providers Mumbai | VPS HostingSwapnil Dighe

Bab 8 kls xiiFikri Pandoez

"BENVENUTI AD ECOLANDIA"Consorzio LaMMA - Corso UdC

Viewers also liked (15)

Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Sour...

Modern Code Reviews in Open Source Projects: Which Problems Do They Fix?

The Last Line Effect

Approach to a child with hematemesis or melena

Supply Chain Management: Technology's Role in Achieving Optimum Performance

Spike Jonze Case Study

Evaluation 4&5

Codes and convention

대학생태환경의 급격 변화

Webinar: Mental Health Services in Schools For Students with Intellectual / D...

Perceiving and believing 6th session

Hubungan etnik-di-malaysia-group-1

VPS Hosting Providers Mumbai | VPS Hosting

Bab 8 kls xii

"BENVENUTI AD ECOLANDIA"

Similar to How (Much) Do Developers Test?

Testing innovate or dieNigel Johnson

Testing In Production (TiP) Advances with Big Data and the CloudSOASTA

iOS Test-Driven DevelopmentPablo Villar

Making the business case for DevOpsMartin Croker

Gamification of Chaos TestingBram Vogelaar

Safety Bot Guaranteed -- Shmoocon 2017Richard Seymour

Can we induce change with what we measure?Michaela Greiler

Testing, Learning and Professionalism — 20171214David Rodenas

Trusting Your Ingredients - What Building Software And Cheesecake Have In CommonLeon Stigter

Keynote AST 2016Kim Herzig

Data Driven DevOpsLeon Stigter

What (and why) Admins need to know about Unit Testing, Julio Fernandez & Dori...CzechDreamin

TDD Anti-patterns (2022 edition)Ahmed Misbah

Solving the 3 Biggest Questions in Continuous TestingPerfecto by Perforce

How ANDROID TESTING changed how we think about DeathFernando Cejas

Agile2012 soccer witha_basketballteamdrewz lin

Automatic for the PeopleAndy Zaidman

Must.Kill.Mutants. Agile Testing Days 2017Gerald Muecke

DevOps: The Key to IT PerformanceNicole Forsgren

Technology-Driven Development: Using Automation and Development Techniques to...Hiroyuki Ito

Similar to How (Much) Do Developers Test? (20)

Testing innovate or die

Testing In Production (TiP) Advances with Big Data and the Cloud

iOS Test-Driven Development

Making the business case for DevOps

Gamification of Chaos Testing

Safety Bot Guaranteed -- Shmoocon 2017

Can we induce change with what we measure?

Testing, Learning and Professionalism — 20171214

Trusting Your Ingredients - What Building Software And Cheesecake Have In Common

Keynote AST 2016

Data Driven DevOps

What (and why) Admins need to know about Unit Testing, Julio Fernandez & Dori...

TDD Anti-patterns (2022 edition)

Solving the 3 Biggest Questions in Continuous Testing

How ANDROID TESTING changed how we think about Death

Agile2012 soccer witha_basketballteam

Automatic for the People

Must.Kill.Mutants. Agile Testing Days 2017

DevOps: The Key to IT Performance

Technology-Driven Development: Using Automation and Development Techniques to...

Recently uploaded

Software Quality Assurance Interview QuestionsArshad QA

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️Delhi Call girls

%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...masabamasaba

%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...masabamasaba

%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...masabamasaba

Introducing Microsoft’s new Enterprise Work Management (EWM) SolutionOnePlan Solutions

%in Durban+277-882-255-28 abortion pills for sale in Durbanmasabamasaba

%in Bahrain+277-882-255-28 abortion pills for sale in Bahrainmasabamasaba

AI Mastery 201: Elevating Your Workflow with Advanced LLM TechniquesVictorSzoltysek

The Ultimate Test Automation Guide_ Best Practices and Tips.pdfkalichargn70th171

W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...panagenda

TECUNIQUE: Success Stories: IT Service providermohitmore19

CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE9953056974 Low Rate Call Girls In Saket, Delhi NCR

Direct Style Effect Systems -The Print[A] Example- A Comprehension AidPhilip Schwarz

Right Money Management App For Your Financial GoalsJhone kinadey

%in tembisa+277-882-255-28 abortion pills for sale in tembisamasabamasaba

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️Delhi Call girls

Unlocking the Future of AI Agents with Large Language Modelsaagamshah0812

Exploring the Best Video Editing App.pdfproinshot.com

10 Trends Likely to Shape Enterprise Technology in 2024Mind IT Systems

Recently uploaded (20)

Software Quality Assurance Interview Questions

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️

%+27788225528 love spells in Vancouver Psychic Readings, Attraction spells,Br...

%+27788225528 love spells in new york Psychic Readings, Attraction spells,Bri...

%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...

Introducing Microsoft’s new Enterprise Work Management (EWM) Solution

%in Durban+277-882-255-28 abortion pills for sale in Durban

%in Bahrain+277-882-255-28 abortion pills for sale in Bahrain

AI Mastery 201: Elevating Your Workflow with Advanced LLM Techniques

The Ultimate Test Automation Guide_ Best Practices and Tips.pdf

W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...

TECUNIQUE: Success Stories: IT Service provider

CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE

Direct Style Effect Systems -The Print[A] Example- A Comprehension Aid

Right Money Management App For Your Financial Goals

%in tembisa+277-882-255-28 abortion pills for sale in tembisa

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️

Unlocking the Future of AI Agents with Large Language Models

Exploring the Best Video Editing App.pdf

10 Trends Likely to Shape Enterprise Technology in 2024

How (Much) Do Developers Test?

1. How (Much) Do Developers Test? Moritz Beller @Inventitech Andy Zaidman Georgios Gousios

3. 50% … of Project Effort Software Testing takes ...

4. 50% … of Project Effort Brooks, 1975 Software Testing takes ...

8. Testing means all things to all men

10. How (Much) Do You Test?

11.

12.

13.

14.

15.

16.

17.

18.

19.

20. 40

21. 4.2 yr

22. OMG!!!

23. Histogram of Testing Time

24. Histogram of Testing Time 50% of students test only 4% of their time, or less

25.

26.

27. Estimated 27%

28. Estimated 27% Reality 9%

29.

30. >1,300

31.

32. Estimated 48%

33. Estimated 48% Reality 25%

34.

35. @Inventitech Moritz Beller, TU Delft How (Much) Do Developers Test? TestRoots.org

36. @Inventitech Moritz Beller, TU Delft How (Much) Do Developers Test? TestRoots.org Code Changes ~ Test Executions

37. @Inventitech Moritz Beller, TU Delft How (Much) Do Developers Test? TestRoots.org Code Changes ~ Test Executions

38. @Inventitech Moritz Beller, TU Delft How (Much) Do Developers Test? TestRoots.org Code Changes ~ Test Executions

39. @Inventitech Moritz Beller, TU Delft How (Much) Do Developers Test? TestRoots.org Code Changes ~ Test Executions Poster Session Thursday, 4pm, Passi Perduti

How (Much) Do Developers Test?

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (15)

Similar to How (Much) Do Developers Test?

Similar to How (Much) Do Developers Test? (20)

Recently uploaded

Recently uploaded (20)

How (Much) Do Developers Test?