Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Strategies for Avoiding TestFixture Smells duringSoftware EvolutionMichaela Greiler, Andy Zaidman, Arie van Deursen,Margar...
Test Fixture Smells2
What is a test smell?A symptom of a problem• Can originate from several causes• Refactoring needed3
Structure of a Test4
Test (method)Test Structure1. Statement2. Statement3. Statement4. Statement5. Statement6. StatementSetup of Test FixtureVe...
Test FixtureThe code that initializes and configures the system under test6
How to structure the test fixtureand where to place it?
Test – Inline FixtureSetup within methodTest Fixture8
Test – ImplicitSetupMethod9
Test – DelegateMethodcreateTestAirportOriginTest ()createTestAirportOriginTest(); explicit invocationdeleteTestAirportOrig...
“Most of the complexity of writing tests involveshow to write the Test Methods; what to includeinline and what to factor o...
Test Fixture Smells• General Fixture• Test Maverick• Dead Fields• Lack of Cohesion of Test Methods• Vague Header• Obscure ...
Developers recognize test fixture smells as aproblem(ICST’13)13
Developers recognize test fixture smells as aproblem(ICST’13)Resolving these smells after a long time can be problematic.I...
1. Investigate the evolution of test fixture smells and2. Understand which software changes lead to increased test smell d...
Azure39 KLOCs30 T-classes358 T-methods300 revisions1 yearJSoup20 KLOCs23 T-classes372 T-methods973 revisions2.5 yearsCheck...
TestEvoHound…is a static analysis tool that analyzes the testfixture smell evolution.
TestEvoHound…is a static analysis tool that analyzes the testfixture smell evolution.1.Revision Checkout2.Build Process3.T...
Fixture smells among projects0%10%20%30%40%50%60%70%80%90%100%Azure Checkstyle JSoup PMD VoldemortTest Mavericks General F...
Does Test Fixture Smell Density Increase?A general growth of test fixture smells over timedoes not occur.20Lehman’s law: w...
Increase in smell densityCheckstyle: dead fieldsRefactoring and forgotten functionality: “Added a helper method to create ...
Decrease in smell densityAzure: general fixtureNew, non-smelly tests added: “Table Client commit [...]”.22
Smell DispersionOver time, smellyclasses get smellierSmells cluster in a fewclasses23
Correlations?Number of t-methods per test class correlates with test fixture smell densityNumber of fields do not necessar...
Strategies and RecommendationsKeep test classes smallKeep inheritance structures flatLimit the scope of super classesUse c...
Findings0%20%40%60%80%100%Azure Checkstyle JSoup PMD VoldemortTest Mavericks General Fixture Vague HeaderDead Fields LCOTM...
0%20%40%60%80%100%Azure Checkstyle JSoup PMD VoldemortTest Mavericks General Fixture Vague HeaderDead Fields LCOTM Inline#...
Upcoming SlideShare
Loading in …5
×

Strategies to Avoid Test Fixture Smells durin Software Evolution

3,074 views

Published on

An important challenge in creating automated tests
is how to design test fixtures, i.e., the setup code that initializes the
system under test before actual automated testing can start. Test
designers have to choose between different approaches for the
setup, trading off maintenance overhead with slow test execution.
Over time, test code quality can erode and test smells can develop,
such as the occurrence of overly general fixtures, obscure inline
code and dead fields. In this paper, we investigate how
fixture-related test smells evolve over time by analyzing several
thousand revisions of five open source systems. Our findings
indicate that setup management strategies strongly influence the
types of test fixture smells that emerge in code, and that several
types of fixture smells often emerge at the same time. Based on
this information, we recommend important guidelines for setup
strategies, and suggest how tool support can be improved to help
in both avoiding the emergence of such smells as well as how to
refactor code when test smells do appear.

Published in: Technology, Business
  • Be the first to comment

Strategies to Avoid Test Fixture Smells durin Software Evolution

  1. 1. Strategies for Avoiding TestFixture Smells duringSoftware EvolutionMichaela Greiler, Andy Zaidman, Arie van Deursen,Margaret-Anne Storey
  2. 2. Test Fixture Smells2
  3. 3. What is a test smell?A symptom of a problem• Can originate from several causes• Refactoring needed3
  4. 4. Structure of a Test4
  5. 5. Test (method)Test Structure1. Statement2. Statement3. Statement4. Statement5. Statement6. StatementSetup of Test FixtureVerify OutcomeExercise SUTTeardown Test Fixture5
  6. 6. Test FixtureThe code that initializes and configures the system under test6
  7. 7. How to structure the test fixtureand where to place it?
  8. 8. Test – Inline FixtureSetup within methodTest Fixture8
  9. 9. Test – ImplicitSetupMethod9
  10. 10. Test – DelegateMethodcreateTestAirportOriginTest ()createTestAirportOriginTest(); explicit invocationdeleteTestAirportOriginTest()deleteTestAirportOriginTest();}10Delegate
  11. 11. “Most of the complexity of writing tests involveshow to write the Test Methods; what to includeinline and what to factor out into Test UtilityMethods, and so on.”(Meszaros)11
  12. 12. Test Fixture Smells• General Fixture• Test Maverick• Dead Fields• Lack of Cohesion of Test Methods• Vague Header• Obscure In-line setupICST 2013: Greiler, van Deursen, StoreyAutomatic Detection of Test FixtureStrategies and Smells12
  13. 13. Developers recognize test fixture smells as aproblem(ICST’13)13
  14. 14. Developers recognize test fixture smells as aproblem(ICST’13)Resolving these smells after a long time can be problematic.Immediately awareness of smells density important.14
  15. 15. 1. Investigate the evolution of test fixture smells and2. Understand which software changes lead to increased test smell densitiesto develop strategies to avoid fixture smells andto determine the best time to alert developers.15Resolving these smells after a long time can be problematic.Immediately awareness of smells density important.
  16. 16. Azure39 KLOCs30 T-classes358 T-methods300 revisions1 yearJSoup20 KLOCs23 T-classes372 T-methods973 revisions2.5 yearsCheckstyle66 KLOCs156 T-classes549 T-methods2251 revisions9.5 yearsPMD174 KLOCs118 T-classes739 T-methods1900 revisions5 yearsVoldemort130 KLOCs132 T-classes520 T-methods2900 revisions1.5 yearsCase studies16
  17. 17. TestEvoHound…is a static analysis tool that analyzes the testfixture smell evolution.
  18. 18. TestEvoHound…is a static analysis tool that analyzes the testfixture smell evolution.1.Revision Checkout2.Build Process3.Test Fixture Smell Analysis4.Trend Analysis
  19. 19. Fixture smells among projects0%10%20%30%40%50%60%70%80%90%100%Azure Checkstyle JSoup PMD VoldemortTest Mavericks General Fixture Vague Header Dead Fields LCOTM Inline19
  20. 20. Does Test Fixture Smell Density Increase?A general growth of test fixture smells over timedoes not occur.20Lehman’s law: when a system evolves, its complexity increases unless work isdone to maintain or reduce it.
  21. 21. Increase in smell densityCheckstyle: dead fieldsRefactoring and forgotten functionality: “Added a helper method to create aconfiguration for a check...”.21
  22. 22. Decrease in smell densityAzure: general fixtureNew, non-smelly tests added: “Table Client commit [...]”.22
  23. 23. Smell DispersionOver time, smellyclasses get smellierSmells cluster in a fewclasses23
  24. 24. Correlations?Number of t-methods per test class correlates with test fixture smell densityNumber of fields do not necessarily correlate with smell density# T-methods# Fields24
  25. 25. Strategies and RecommendationsKeep test classes smallKeep inheritance structures flatLimit the scope of super classesUse composition instead of inheritanceReconsider the “one test class per class” organization25
  26. 26. Findings0%20%40%60%80%100%Azure Checkstyle JSoup PMD VoldemortTest Mavericks General Fixture Vague HeaderDead Fields LCOTM Inline# T-methods# Fields26
  27. 27. 0%20%40%60%80%100%Azure Checkstyle JSoup PMD VoldemortTest Mavericks General Fixture Vague HeaderDead Fields LCOTM Inline# T-methods# FieldsThanks! Questions?Connect with meMichaela Greilermichaela.greiler@microsoft.commgreiler27

×