Strategies to Avoid Test Fixture Smells durin Software Evolution
An important challenge in creating automated tests
is how to design test fixtures, i.e., the setup code that initializes the
system under test before actual automated testing can start. Test
designers have to choose between different approaches for the
setup, trading off maintenance overhead with slow test execution.
Over time, test code quality can erode and test smells can develop,
such as the occurrence of overly general fixtures, obscure inline
code and dead fields. In this paper, we investigate how
fixture-related test smells evolve over time by analyzing several
thousand revisions of five open source systems. Our findings
indicate that setup management strategies strongly influence the
types of test fixture smells that emerge in code, and that several
types of fixture smells often emerge at the same time. Based on
this information, we recommend important guidelines for setup
strategies, and suggest how tool support can be improved to help
in both avoiding the emergence of such smells as well as how to
refactor code when test smells do appear.
Test – DelegateMethodcreateTestAirportOriginTest ()createTestAirportOriginTest(); explicit invocationdeleteTestAirportOriginTest()deleteTestAirportOriginTest();}10Delegate
“Most of the complexity of writing tests involveshow to write the Test Methods; what to includeinline and what to factor out into Test UtilityMethods, and so on.”(Meszaros)11
Test Fixture Smells• General Fixture• Test Maverick• Dead Fields• Lack of Cohesion of Test Methods• Vague Header• Obscure In-line setupICST 2013: Greiler, van Deursen, StoreyAutomatic Detection of Test FixtureStrategies and Smells12
Developers recognize test fixture smells as aproblem(ICST’13)13
Developers recognize test fixture smells as aproblem(ICST’13)Resolving these smells after a long time can be problematic.Immediately awareness of smells density important.14
1. Investigate the evolution of test fixture smells and2. Understand which software changes lead to increased test smell densitiesto develop strategies to avoid fixture smells andto determine the best time to alert developers.15Resolving these smells after a long time can be problematic.Immediately awareness of smells density important.
TestEvoHound…is a static analysis tool that analyzes the testfixture smell evolution.
TestEvoHound…is a static analysis tool that analyzes the testfixture smell evolution.1.Revision Checkout2.Build Process3.Test Fixture Smell Analysis4.Trend Analysis
Fixture smells among projects0%10%20%30%40%50%60%70%80%90%100%Azure Checkstyle JSoup PMD VoldemortTest Mavericks General Fixture Vague Header Dead Fields LCOTM Inline19
Does Test Fixture Smell Density Increase?A general growth of test fixture smells over timedoes not occur.20Lehman’s law: when a system evolves, its complexity increases unless work isdone to maintain or reduce it.
Increase in smell densityCheckstyle: dead fieldsRefactoring and forgotten functionality: “Added a helper method to create aconfiguration for a check...”.21
Decrease in smell densityAzure: general fixtureNew, non-smelly tests added: “Table Client commit [...]”.22
Smell DispersionOver time, smellyclasses get smellierSmells cluster in a fewclasses23
Correlations?Number of t-methods per test class correlates with test fixture smell densityNumber of fields do not necessarily correlate with smell density# T-methods# Fields24
Strategies and RecommendationsKeep test classes smallKeep inheritance structures flatLimit the scope of super classesUse composition instead of inheritanceReconsider the “one test class per class” organization25