Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

To Mock or Not To Mock

13 views

Published on

MSR 2017

Davide Spadini

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

To Mock or Not To Mock

  1. 1. To Mock Or Not To Mock? An Empirical Study on Mocking Practices Davide Spadini, Mauricio Aniche, 
 Magiel Bruntink, Alberto Bacchelli @DavideSpadini
  2. 2. How do I test this?
  3. 3. With dependencies Without dependencies
  4. 4. Mock Unit under testUnit under test
  5. 5. With dependencies
  6. 6. With dependencies
  7. 7. With dependencies
  8. 8. • Faster • Easier Without dependencies
  9. 9. Without dependencies You lose in reality, after all, it is just a simulation!
  10. 10. To Mock Not to Mock So, what should we do?
  11. 11. RQ1: What test dependencies do developers mock? RQ2: Why do developers decide to (not) mock specific dependencies? RQ3: Which are the main challenges experienced with testing using mocks? Research Questions
  12. 12. Opensource Industrial Methodology
  13. 13. TA P1 Sonarqube Alura VRaptor Spring Framework TA P1 TA P1 TA P1 Methodology
  14. 14. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies Alura TA P3 Mock P1 P2 1,333 dependencies VRaptor TA P3 Mock P1 P2 21,768 dependencies Spring Framework TA P1 TA P1 TA P1 Methodology
  15. 15. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674 dependencies TA P1 Tn Pn … … Sampling P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Data collection TA P1 TA P1 TA P1 Methodology
  16. 16. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674 dependencies TA P1 Tn Pn … … Sampling Sample 1 P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Sample 2 Validation Manual analysis (2nd author) Manual analysis (1st author) Data analysisData collection Manual analysis TA P1 TA P1 TA P1 Methodology
  17. 17. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674 dependencies TA P1 Tn Pn … … Sampling Commit Comments Commit Comments Commit Comments 132 categories Commit Comments7 Categories Sample 1 P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Sample 2 Validation Manual analysis (2nd author) Manual analysis (1st author) Discussion Data analysisData collection Manual analysis Categorisation TA P1 TA P1 TA P1 Methodology
  18. 18. RQ1 RQ1: What test dependencies do developers mock?
  19. 19. RQ1: What test dependencies do developers mock? RQ1 0% 25% 50% 75% 100% Database Web service External 
 Dependencies Domain 
 object Java libraries Test support 94%93%64%32%31%28% 6%7% 36% 68%69% 72% Mocked Not mocked
  20. 20. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674 dependencies TA P1 Tn Pn … … Sampling Commit Comments Commit Comments Commit Comments 132 categories Commit Comments7 Categories Interview Guideline 3 developers Sample 1 P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Sample 2 Validation Manual analysis (2nd author) Manual analysis (1st author) Discussion Data analysis Interviews & Validation Data collection Manual analysis Categorisation TA P1 TA P1 TA P1 Methodology
  21. 21. RQ2 RQ2: Why do developers decide to (not) mock a dependency?
  22. 22. RQ2 RQ2: Why do developers decide to (not) mock a dependency? Not when the focus of the test is the integration Vehicle Car Plane Boat Mock interfaces rather than a specific implementation Concrete implementation is not easy
  23. 23. TA MockExtractor TA P3 Mock P1 P2 13,547 dependencies 674 dependencies TA P1 Tn Pn … … Sampling Commit Comments Commit Comments Commit Comments 132 categories Commit Comments7 Categories Interview Guideline 3 developers Commit Comments Review Comment Commit Comments Interview Transcript Commit Comments Commit Comments Survey (105 respondents) Sample 1 P1 Sonarqube TA P3 Mock P1 P2 1,665 dependencies 445 dependencies TA P1 Tn Pn … … Alura TA P3 Mock P1 P2 1,333 dependencies 438 dependencies TA P1 Tn Pn … … VRaptor TA P3 Mock P1 P2 21,768 dependencies 621 dependencies TA P1 Tn Pn … … Spring Framework Sample 2 Validation Manual analysis (2nd author) Manual analysis (1st author) Discussion Affinity Diagram Data analysis Interviews & Validation Data collection Manual analysis Categorisation TA P1 TA P1 TA P1 Methodology
  24. 24. RQ3 RQ3:Which are the main challenges experienced with testing using mocks?
  25. 25. RQ3 RQ3:Which are the main challenges experienced with testing using mocks? Legacy systems Dealing with coupling Production quality code vs mocks
  26. 26. Discussion Mockito developer • He agreed on all the findings • How Mockito can help? • What Mockito is supposed to do more? Open questions • Software quality vs mocks • Are slow tests more mocked? • How faster are tests with mocks?

×