Download as PDF, PPTX
Uploaded byDelft University of Technology
To Mock or Not To Mock
This document describes a study on mocking practices used in software tests. The researchers investigated which dependencies developers commonly mock in tests, why they decide to mock or not mock certain dependencies, and what challenges they face when using mocks. They analyzed test code from several open source projects and conducted surveys and interviews with developers. The main findings were that databases, web services and external dependencies are most often mocked, while domain objects and Java libraries are less likely to be mocked. Developers consider whether the focus of the test is integration or isolation and the difficulty of implementing concrete dependencies when deciding to mock. Legacy systems, test-code coupling, and balancing production-quality code with mocks were challenges identified.
More Related Content
Similar to To Mock or Not To Mock
To Mock or Not To Mock
- 1. To Mock OrNot To Mock? An Empirical Study on Mocking Practices Davide Spadini, Mauricio Aniche, Magiel Bruntink, Alberto Bacchelli @DavideSpadini
- 2. How do Itest this?
- 3. With dependencies Withoutdependencies
- 4. Mock Unit under testUnitunder test
- 6.
- 7.
- 8.
- 9. • Faster •Easier Without dependencies
- 10. Without dependencies You losein reality, after all, it is just a simulation!
- 11. To Mock Notto Mock So, what should we do?
- 12. RQ1: What testdependencies do developers mock? RQ2: Why do developers decide to (not) mock specific dependencies? RQ3: Which are the main challenges experienced with testing using mocks? Research Questions
- 13.
- 14.
- 15. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies Alura TA P3 Mock P1 P2 1,333dependencies VRaptor TA P3 Mock P1 P2 21,768 dependencies Spring Framework TA P1 TA P1 TA P1 Methodology
- 16. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674dependencies TA P1 Tn Pn … … Sampling P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Data collection TA P1 TA P1 TA P1 Methodology
- 17. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674dependencies TA P1 Tn Pn … … Sampling Sample 1 P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Sample 2 Validation Manual analysis (2nd author) Manual analysis (1st author) Data analysisData collection Manual analysis TA P1 TA P1 TA P1 Methodology
- 18. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674dependencies TA P1 Tn Pn … … Sampling Commit Comments Commit Comments Commit Comments 132 categories Commit Comments7 Categories Sample 1 P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Sample 2 Validation Manual analysis (2nd author) Manual analysis (1st author) Discussion Data analysisData collection Manual analysis Categorisation TA P1 TA P1 TA P1 Methodology
- 19. RQ1 RQ1: What testdependencies do developers mock?
- 20. RQ1: What testdependencies do developers mock? RQ1 0% 25% 50% 75% 100% Database Web service External Dependencies Domain object Java libraries Test support 94%93%64%32%31%28% 6%7% 36% 68%69% 72% Mocked Not mocked
- 21. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674dependencies TA P1 Tn Pn … … Sampling Commit Comments Commit Comments Commit Comments 132 categories Commit Comments7 Categories Interview Guideline 3 developers Sample 1 P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Sample 2 Validation Manual analysis (2nd author) Manual analysis (1st author) Discussion Data analysis Interviews & Validation Data collection Manual analysis Categorisation TA P1 TA P1 TA P1 Methodology
- 22. RQ2 RQ2: Why dodevelopers decide to (not) mock a dependency?
- 23. RQ2 RQ2: Why dodevelopers decide to (not) mock a dependency? Not when the focus of the test is the integration Vehicle Car Plane Boat Mock interfaces rather than a specific implementation Concrete implementation is not easy
- 24. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674dependencies TA P1 Tn Pn … … Sampling Commit Comments Commit Comments Commit Comments 132 categories Commit Comments7 Categories Interview Guideline 3 developers Commit Comments Review Comment Commit Comments Interview Transcript Commit Comments Commit Comments Survey (105 respondents) Sample 1 P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Sample 2 Validation Manual analysis (2nd author) Manual analysis (1st author) Discussion Affinity Diagram Data analysis Interviews & Validation Data collection Manual analysis Categorisation TA P1 TA P1 TA P1 Methodology
- 25. RQ3 RQ3:Which are themain challenges experienced with testing using mocks?
- 26. RQ3 RQ3:Which are themain challenges experienced with testing using mocks? Legacy systems Dealing with coupling Production quality code vs mocks
- 27. Discussion Mockito developer • Heagreed on all the findings • How Mockito can help? • What Mockito is supposed to do more? Open questions • Software quality vs mocks • Are slow tests more mocked? • How faster are tests with mocks?