Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Published on

  • Be the first to comment

  • Be the first to like this


  1. 1. M A N N I N GA guide for Java developersLASSE KOSKELA
  2. 2. Effective Unit TestingDownload from Wow! eBook <>
  3. 3. Download from Wow! eBook <>
  4. 4. Effective Unit TestingA GUIDE FOR JAVA DEVELOPERSLASSE KOSKELAM A N N I N GSHELTER ISLANDDownload from Wow! eBook <>
  5. 5. For online information and ordering of this and other Manning books, please The publisher offers discounts on this book when ordered in quantity.For more information, please contactSpecial Sales DepartmentManning Publications Co.20 Baldwin RoadPO Box 261Shelter Island, NY 11964Email:©2013 by Manning Publications Co. All rights reserved.No part of this publication may be reproduced, stored in a retrieval system, or transmitted, inany form or by means electronic, mechanical, photocopying, or otherwise, without prior writtenpermission of the publisher.Many of the designations used by manufacturers and sellers to distinguish their products areclaimed as trademarks. Where those designations appear in the book, and ManningPublications was aware of a trademark claim, the designations have been printed in initial capsor all caps.Recognizing the importance of preserving what has been written, it is Manning’s policy to havethe books we publish printed on acid-free paper, and we exert our best efforts to that end.Recognizing also our responsibility to conserve the resources of our planet, Manning booksare printed on paper that is at least 15 percent recycled and processed without the use ofelemental chlorine.Manning Publications Co. Development editor: Frank Pohlman20 Baldwin Road Copyeditor: Benjamin BergPO Box 261 Technical proofreader: Phil HannaShelter Island, NY 11964 Proofreader: Elizabeth MartinTypesetter: Dottie MarsicoCover designer: Marija TudorISBN 9781935182573Printed in the United States of America1 2 3 4 5 6 7 8 9 10 – MAL – 18 17 16 15 14 13Download from Wow! eBook <>
  6. 6. Few sights are as captivating as the pure joyof learning new things.Download from Wow! eBook <>
  7. 7. Download from Wow! eBook <>
  8. 8. viibrief contentsPART 1 FOUNDATIONS ..............................................................11 ■ The promise of good tests 32 ■ In search of good 153 ■ Test doubles 27PART 2 CATALOG....................................................................454 ■ Readability 475 ■ Maintainability 786 ■ Trustworthiness 115PART 3 DIVERSIONS ..............................................................1377 ■ Testable design 1398 ■ Writing tests in other JVM languages 1569 ■ Speeding up test execution 170Download from Wow! eBook <>
  9. 9. Download from Wow! eBook <>
  10. 10. ixcontentspreface xvacknowledgments xviiabout this book xixabout the cover illustration xxivPART 1 FOUNDATIONS...................................................11 The promise of good tests 31.1 State of the union: writing better tests 41.2 The value of having tests 5Factors of productivity 8 ■ The curve of design potential 101.3 Tests as a design tool 10Test-driven development 11 ■ Behavior-driven development 131.4 Summary 142 In search of good 152.1 Readable code is maintainable code 162.2 Structure helps make sense of things 182.3 It’s not good if it’s testing the wrong things 202.4 Independent tests run easily in solitude 21Download from Wow! eBook <>
  11. 11. CONTENTSx2.5 Reliable tests are reliable 232.6 Every trade has its tools and tests are no exception 252.7 Summary 263 Test doubles 273.1 The power of a test double 28Isolating the code under test 28 ■ Speeding up test execution 30Making execution deterministic 31 ■ Simulating specialconditions 32 ■ Exposing hidden information 323.2 Types of test doubles 33Test stubs are unusually short things 34 ■ Fake objects do itwithout side effects 35 ■ Test spies steal your secrets 36Mock objects object to surprises 383.3 Guidelines for using test doubles 39Pick the right double for the test 40 ■ Arrange, act, assert 40Check for behavior, not implementation 42 ■ Choose yourtools 42 ■ Inject your dependencies 433.4 Summary 44PART 2 CATALOG ........................................................454 Readability 474.1 Primitive assertions 48Example 48 ■ What to do about it? 49 ■ Summary 514.2 Hyperassertions 51Example 51 ■ What to do about it? 54 ■ Summary 554.3 Bitwise assertions 56Example 56 ■ What to do about it? 56 ■ Summary 574.4 Incidental details 57Example 57 ■ What to do about it? 58 ■ Summary 604.5 Split personality 60Example 61 ■ What to do about it? 61 ■ Summary 644.6 Split logic 64Example 65 ■ What to do about it? 66 ■ Summary 694.7 Magic numbers 70Example 70 ■ What to do about it? 71 ■ Summary 71Download from Wow! eBook <>
  12. 12. CONTENTS xi4.8 Setup sermon 72Example 72 ■ What to do about it? 73 ■ Summary 744.9 Overprotective tests 75Example 75 ■ What to do about it? 76 ■ Summary 764.10 Summary 765 Maintainability 785.1 Duplication 79Example 79 ■ What to do about it? 80 ■ Summary 825.2 Conditional logic 82Example 83 ■ What to do about it? 83 ■ Summary 845.3 Flaky test 85Example 85 ■ What to do about it? 86 ■ Summary 875.4 Crippling file path 88Example 88 ■ What to do about it? 89 ■ Summary 905.5 Persistent temp files 91Example 91 ■ What to do about it? 92 ■ Summary 935.6 Sleeping snail 94Example 94 ■ What to do about it? 95 ■ Summary 965.7 Pixel perfection 97Example 97 ■ What to do about it? 98 ■ Summary 1015.8 Parameterized mess 102Example 103 ■ What to do about it? 106 ■ Summary 1085.9 Lack of cohesion in methods 108Example 109 ■ What to do about it? 110 ■ Summary 1125.10 Summary 1136 Trustworthiness 1156.1 Commented-out tests 116Example 116 ■ What to do about it? 117 ■ Summary 1186.2 Misleading comments 118Example 118 ■ What to do about it? 119 ■ Summary 1206.3 Never-failing tests 121Example 121 ■ What to do about it? 122 ■ Summary 123Download from Wow! eBook <>
  13. 13. CONTENTSxii6.4 Shallow promises 123Example(s) 123 ■ What to do about it? 125 ■ Summary 1266.5 Lowered expectations 127Example 127 ■ What to do about it? 128 ■ Summary 1296.6 Platform prejudice 129Example 129 ■ What to do about it? 130 ■ Summary 1326.7 Conditional tests 133Example 133 ■ What to do about it? 134 ■ Summary 1356.8 Summary 135PART 3 DIVERSIONS...................................................1377 Testable design 1397.1 What’s testable design? 140Modular design 140 ■ SOLID design principles 141Modular design in context 143 ■ Test-driving towardmodular design 1437.2 Testability issues 143Can’t instantiate a class 144 ■ Can’t invoke a method 144Can’t observe the outcome 145 ■ Can’t substitute acollaborator 145 ■ Can’t override a method 1457.3 Guidelines for testable design 146Avoid complex private methods 146 ■ Avoid final methods 147Avoid static methods 148 ■ Use new with care 148 ■ Avoidlogic in constructors 149 ■ Avoid the Singleton 150 ■ Favorcomposition over inheritance 151 ■ Wrap external libraries 152Avoid service lookups 1527.4 Summary 1548 Writing tests in other JVM languages 1568.1 The premise of mixing JVM languages 157General benefits 157 ■ Writing tests 1598.2 Writing unit tests with Groovy 160Simplified setup for tests 161 ■ Groovier JUnit 4 tests 163Download from Wow! eBook <>
  14. 14. CONTENTS xiii8.3 Expressive power with BDD tools 163Groovy specs with easyb 164 ■ Spock Framework: steroids forwriting more expressive tests 165 ■ Spock Framework’s test doublesare on steroids, too 1678.4 Summary 1689 Speeding up test execution 1709.1 Looking for a speed-up 171The need for speed 171 ■ Approaching the situation 172Profiling a build 172 ■ Profiling tests 1759.2 Speeding up test code 178Don’t sleep unless you’re tired 178 ■ Beware the bloated baseclass 179 ■ Watch out for redundant setup and teardown 181Be picky about who you invite to your test 182 ■ Stay local, stayfast 183 ■ Resist the temptation to hit the database 184There’s no slower I/O than file I/O 1869.3 Speeding up the build 187Faster I/O with a RAM disk 188 ■ Parallelizing the build 189Offload to a higher-powered CPU 194 ■ Distribute the build 1969.4 Summary 200appendix A JUnit primer 202appendix B Extending JUnit 209index 217Download from Wow! eBook <>
  15. 15. Download from Wow! eBook <>
  16. 16. xvprefaceOn the night of June 10, 2009, I found an email in my inbox from Christina Rudlofffrom Manning, asking me if I knew anyone who might be a good candidate to write aJava edition of Roy Osherove’s book, The Art of Unit Testing in .NET. I told her I’d do it.That was a long time ago and what you’re looking at right now has very little incommon with Roy’s book. Let me explain.The project started as a straight translation from .NET to Java, only rewriting wherenecessary to match the changing technology platform, its tooling, and its audience. Ifinished the first chapter, the second chapter, the third chapter, and suddenly I foundmyself rewriting not just short passages but entire chapters. The tone of voice wasn’tmine; sometimes I would disagree or have preferences incompatible with Roy’s, andsometimes I simply felt strongly about saying something, setting things straight, andputting a stake into the ground.Eventually, I decided to start over.It was clear that we were not in a translation project. This was a brand new title ofits own—a book that helps a Java programmer improve his tests, gaining a deeperinsight into what makes a test good and what kind of pitfalls to look out for. You canstill see Roy’s thinking in many ways in this book. For instance, the chapter titles of thecatalog in part 2 I’ve blatantly borrowed from Roy and chapter 7 was written largelythanks to Roy’s counterpart in The Art of Unit Testing in .NET.This is a book for the Java programmer. Yet, I didn’t want to straitjacket the ideas inthis book artificially, so I tried to steer away from being too language-specific eventhough all of the code examples in the pattern catalog, for example, are Java. Writinggood tests is a language-agnostic problem and I heartily recommend you read thisDownload from Wow! eBook <>
  17. 17. PREFACExvibook thoughtfully, even if you happen to spend most of your office hours hacking onanother programming language.Along those same lines, I didn’t want to give you a tutorial on JUnit or my favoritemock object library. Aside from such technology being a constantly changing land-scape and bound to become stale information within months of publication, I wantedto write the kind of book that I would want to read. I like focused books that don’tforce me to lug around dead weight about a testing framework I already know byheart or a mock object library I don’t use. For these reasons, I’ve tried to minimize theamount of technology-specific advice. There is some but I want you to know that I’vedone my best to keep it to a minimum—just enough to have meaningful conversa-tions about the underlying concepts that I find essential in writing, running, maintain-ing, and improving tests.I tried to write the book I would’ve wanted to read. I hope you will enjoy it and,most importantly, integrate some of these ideas into your own practice.Download from Wow! eBook <>
  18. 18. xviiacknowledgmentsWhen I signed up to write this book I thought it’d be a short project. Everything wassupposed to be straightforward with no wildcards in sight. I should’ve known better.My wishful thinking was shattered as weeks turned to months and months turned toyears. Without the help of many, many people this book would definitely not be inyour hands and most likely it’d still be a work-in-progress.From the moment this project was initiated in a casual email exchange with Man-ning Publication’s Christina Rudloff, a massive amount of help has poured in and allof it has been very much appreciated—and needed.I’d like to thank the team at Manning for their support and persistence. In nospecific order, Michael Stephens, Elle Suzuki, Steven Hong, Nick Chase, KarenTegtmeyer, Sebastian Stirling, Candace M. Gillhoolley, Maureen Spencer, Ozren Har-lovic, Frank Pohlmann, Benjamin Berg, Elizabeth Martin, Dottie Marsico, Janet Vail,and Mary Piergies.A special thanks goes to the fine individuals that served as domain experts orreviewers and contributed their time to put their specific experience and expertise toimproving this book. Again, in no specific order, I’d like to extend my most sinceregratitude to Jeremy Anderson, Christopher Bartling, Jedidja Bourgeois, Kevin Con-away, Roger Cornejo, Frank Crow, Chad Davis, Gordon Dickens, Martyn Fletcher, PaulHolser, Andy Kirsch, Antti Koivisto, Paul Kronquist, Teppo Kurki, Franco Lombardo,Saicharan Manga, Dave Nicolette, Gabor Paller, J. B. Rainsberger, Joonas Reynders,Adam Taft, Federico Tomassetti, Jacob Tomaw, Bas Vodde, Deepak Vohra, Rick Wag-ner, Doug Warren, James Warren, Robert Wenner, Michael Williams, and Scott Sauyet.Download from Wow! eBook <>
  19. 19. ACKNOWLEDGMENTSxviiiSpecial thanks to Phil Hanna for his technical review of the manuscript just before itwent into production.And last, but definitely not least, I’d like to thank my family for their continuedsupport. I imagine it has at times felt like a never-ending endeavor to get this book toprint. Thank you for understanding all of those late nights with a computer on my lapand for carrying me through the rough spots.Download from Wow! eBook <>
  20. 20. xixabout this bookDeveloper testing has been increasing its mindshare significantly among Java develop-ers over the past 10 years or so. Today, no computer science student graduates withouthaving at least read about automated unit tests and their importance in softwaredevelopment. The idea is simple—to ensure that our code works and keeps working—but the skill takes significant effort to learn.Some of that effort goes to simply writing tests and learning the technology such asa test framework like JUnit. Some of that effort (and quite possibly most of it) that’srequired for truly mastering the practice of writing automated unit tests goes to read-ing test code and improving it. This constant refactoring of tests—trying out differentways of expressing your intent, structuring tests for various aspects of behavior, orbuilding the various objects used by those tests—is our pragmatic way of teaching our-selves and developing our sense for unit tests.That sense is as much about what good unit tests are like as it is about what not-so-good unit tests are like. There may be some absolute truths involved (such as that acode comment repeating exactly what the code says is redundant and should beremoved) but the vast majority of the collective body of knowledge about unit tests ishighly context-sensitive. What is generally considered good might be a terrible idea ina specific situation. Similarly, what is generally a bad idea and should be avoided cansometimes be just the right thing to do.It turns out that often the best way to find your way to a good solution is to try oneapproach that seems viable, identify the issues with that approach, and change theapproach to remove the icky parts. By repeating this process of constantly evaluatingand evolving what you have, eventually you reach a solution that works and doesn’tsmell all that bad. You might even say that it’s a pretty good approach!Download from Wow! eBook <>
  21. 21. ABOUT THIS BOOKxxWith this complexity in mind, we’ve adopted a style and structure for this bookwhere we don’t just tell you what to do and how to write unit tests. Instead, we aim togive you a solid foundation on what kind of properties we want our tests to exhibit(and why) and then give you as many concrete examples as we can to help youdevelop your sense for test smells—to help you notice when something about your testseems to be out of place.AudienceThis book is aimed at Java programmers of all experience levels who are looking toimprove the quality of the unit tests they write. While we do provide appendices thatteach you about a test framework (JUnit), our primary goal is to help Java program-mers who already know how to write unit tests with their test framework of choice towrite better unit tests. Regardless of how many unit tests you’ve written so far, we’re cer-tain that you can still get better at it, and reading a book like this might be just what youneed to stimulate a line of thought that you’ve struggled to put into words.RoadmapEffective Unit Testing takes on a multifaceted challenge that calls for a structure thatsupports each of those facets. In our wisdom (gained through several iterations offailed attempts) we’ve decided to divide this book into three parts.Part 1 begins our journey toward better tests by introducing what we’re trying toachieve and why those goals should be considered desirable in the first place. Thesethree chapters present the fundamental tools and simple guidelines for writing agood test.Chapter 1 starts off with the value proposition of automated unit tests. We establishthe value by considering the many things that factor into our productivity as program-mers and how well-written automated unit tests contribute to that productivity or pre-vent things from dragging us down.Chapter 2 sets the bar high and attempts to define what makes a test good. Theproperties and considerations in this chapter serve as the core foundation for part 2,touching on how we want our tests to be readable, maintainable, and reliable.Chapter 3 steps out of the line for a moment to introduce test doubles as an essen-tial tool for writing good tests. It’s not really using test doubles that we’re after butrather using them well and with consideration. (They’re not a silver bullet in case youwere wondering.)Part 2 turns the tables and offers a stark contrast to part 1 in its approach, present-ing a catalog of test smells you should watch for. Along with describing a suspect pat-tern in test code we’ll suggest solutions to try when you encounter such a smell. Thechapters in this part are divided into three themes: smells that suggest degraded read-ability, smells that indicate a potential maintenance nightmare, and smells that reek oftrust issues. Many of the smells in part 2 could be featured in any of these three chap-ters, but we’ve tried to arrange them according to their primary impact.Download from Wow! eBook <>
  22. 22. ABOUT THIS BOOK xxiChapter 4 focuses on test smells that are primarily related to the intent or imple-mentation of a test being unnecessarily opaque. We touch on things like illegible asser-tions, inappropriate levels of abstraction, and information scatter within our test code.Chapter 5 walks through test smells that might lead to late nights at the office,because it takes forever to update one mess of a unit test related to a small change orbecause making that small change means we need to change a hundred tests. We takeon code duplication and logic in our test code and we expound on the horrors oftouching the filesystem. And it’s not like we’re giving a free pass to slow tests eitherbecause time is money.Chapter 6 concludes our catalog of test smells with a sequence of gotchas aroundassumptions. Some of these assumptions are made because there’s an inconvenientcomment in our test code and some are the unfortunate products of a failure toexpress ourselves unambiguously.Part 3 could have been called “advanced topics.” It’s not, however, because the top-ics covered here don’t necessarily build on parts 1 or 2. Rather, these are topics that aJava programmer might stumble onto at any point on his or her test-writing journey.After all, almost everything about “good” unit tests is context-sensitive so it’s not sur-prising that a pressing topic for one programmer is a minor consideration foranother, whether it’s about inheritance between unit test classes, about the program-ming language we use for writing tests, or about the way our build infrastructure exe-cutes the tests we’ve written.Chapter 7 picks up on where chapter 2 left off, exploring what constitutes testabledesign. After a brief overview of useful principles and clarifying how we are essentiallylooking for modular designs, we study the fundamental testability issues that untest-able designs throw our way. The chapter concludes with a set of simple guidelines tokeep us on the righteous path of testable design.Chapter 8 throws a curveball by posing the question, what if we’d write our unittests in a programming language other than Java? The Java Virtual Machine allows themodern programmer to apply a number of alternative programming languages andintegrate it all with plain Java code.Chapter 9 returns to common reality by taking on the challenge of dealing withincreasingly slow build times and delayed test results. We look for solutions bothwithin our test code, considering ways of speeding up the code that’s being run aspart of our build, and in our infrastructure, pondering whether we could get thatextra bit of oomph from faster hardware or from a different way of allocating work tothe existing hardware.Despite JUnit’s popularity and status as the de facto unit test framework within theJava community, not all Java programmers are familiar with this great little open sourcelibrary. We’ve included two appendices to help those individuals and programmers whohaven’t squeezed all the horsepower out of JUnit’s more advanced features.Appendix A offers a brief introduction to writing tests with JUnit and how JUnitpokes and prods those tests when you tell it to run them. After skimming through thisDownload from Wow! eBook <>
  23. 23. ABOUT THIS BOOKxxiiappendix you’ll be more than capable of writing tests and making assertions withJUnit’s API.Appendix B takes a deeper dive into the JUnit API with the goal of extending itsbuilt-in functionality. While not trying to cover everything about JUnit to the last bit ofdetail, we’ve chosen to give you a brief overview of the two common ways of extendingJUnit—rules and runners—and devote this appendix to showcasing some of the built-in rules that are not only useful, but also give you an idea of what you can do with yourcustom extensions.Code conventionsThe code examples presented in this book consist of Java source code as well as a hostof markup languages and output listings. We present the longer pieces of code as list-ings with their own headers. Smaller bits of code are run inline with the text. In allcases, we present the code using a monospaced font like this, to differentiate it fromthe rest of the text. We frequently refer to elements in code listings taken from thetext. Such references are also presented using a monospaced font, to make themstand out. Many longer listings have numbered annotations that we refer to in thetext.Code downloadsThe Manning website page for this book at a source code package you can download to your computer. This includesselected parts of the source code shown in the book, should you want to take thingsfurther from where we left off.The download includes an Apache Maven 2 POM file and instructions for installingand using Maven ( to compile and run the examples. Notethat the download doesn’t include the various dependencies, and you need to have aninternet connection when running the Maven build for the first time—Maven willthen download all the required dependencies from the internet. After that, you’refree to disconnect and play with the examples offline.The code examples were written against Java 6, so you’ll need to have that installedin order to compile and run the examples. You can download a suitable Java environ-ment from (To compile the code, you’ll need to download the JDK,not the JRE.)We recommend installing a proper IDE as well. You may want to download andinstall the latest and greatest version of Eclipse ( or another main-stream tool like IntelliJ IDEA ( or NetBeans ( Allof these should work fine as long as you’re familiar with the tool.What’s next?This book should give you enough insight to start visibly improving your unit tests. It’sgoing to be a long journey, and there’s bound to be a question or two that we haven’tDownload from Wow! eBook <>
  24. 24. ABOUT THIS BOOK xxiiimanaged to predict or answer in full. Fortunately, you’ll have plenty of peers on thisjourney and many of them will be more than happy to share and discuss the nuancesof test code online.Manning has set up an online forum where you can talk to the authors of Manningtitles. That includes the book you’re reading right now so head over to the AuthorOnline forum for this book at’s also an active community of test-infected programmers over at the testdriv-endevelopment and extremeprogramming Yahoo! Groups. While these forums aren’t exclu-sively for discussions about unit tests, they are excellent places for holding thosediscussions. Besides, maybe you’ll manage to pick up some new ideas outside of testcode, too.If you’re looking for a more focused forum for having discussions about developertesting, head over to the CodeRanch at and the excellentTesting forum. Again, a lovely group of very helpful people over there.Most importantly, however, I suggest that you actively talk about your test code withyour peers at work. Some of the best insights I’ve had about my code have beenthrough having someone else look at it on the spot.Author OnlinePurchase of Effective Unit Testing includes free access to a private web forum runby Manning Publications, where you can make comments about the book, ask techni-cal questions, and receive help from the author and from other users. To access theforum and subscribe to it, point your web browser to This page provides information on how to get on the forum onceyou are registered, what kind of help is available, and the rules of conduct on theforum.Manning’s commitment to our readers is to provide a venue where a meaningfuldialogue between individual readers and between readers and the author can takeplace. It is not a commitment to any specific amount of participation on the part of theauthor, whose contribution to the book’s forum remains voluntary (and unpaid). Wesuggest you try asking the author some challenging questions, lest his interest stray!The Author Online forum and the archives of previous discussions will be accessi-ble from the publisher’s website as long as the book is in print.Download from Wow! eBook <>
  25. 25. xxivabout the cover illustrationThe figure on the cover of Effective Unit Testing is captioned “A Man from Drnis, Dal-matia, Croatia.” The illustration is taken from a reproduction of an album of Croatiantraditional costumes from the mid-nineteenth century by Nikola Arsenovic, publishedby the Ethnographic Museum in Split, Croatia, in 2003. The illustrations wereobtained from a helpful librarian at the Ethnographic Museum in Split, itself situatedin the Roman core of the medieval center of the town: the ruins of Emperor Diocle-tian’s retirement palace from around AD 304. The book includes finely colored illus-trations of figures from different regions of Croatia, accompanied by descriptions ofthe costumes and of everyday life.Drnis is a small town in inland Dalmatia, built on the ruins of an old medieval for-tress. The figure on the cover is wearing blue woolen trousers and, over a white linenshirt, a blue woolen vest which is richly trimmed with the colorful embroidery typicalfor this region. He is carrying a long pipe and a burgundy jacket is slung over hisshoulder. A red cap and leather moccasins complete the outfit.Dress codes and lifestyles have changed over the last 200 years, and the diversity byregion, so rich at the time, has faded away. It is now hard to tell apart the inhabitantsof different continents, let alone of different hamlets or towns separated by only a fewmiles. Perhaps we have traded cultural diversity for a more varied personal life—certainly for a more varied and fast-paced technological life.Manning celebrates the inventiveness and initiative of the computer business withbook covers based on the rich diversity of regional life of two centuries ago, broughtback to life by illustrations from old books and collections like this one.Download from Wow! eBook <>
  26. 26. Part 1FoundationsThis first part of the book aims to give you, the reader, and me, the author,a shared context to build upon throughout the chapters. With the ultimate pur-pose of this book being to help you improve your ability to write good tests,chapter 1 begins with an overview of what kind of value we can extract from writ-ing tests in the first place. Once we’ve discussed the dynamics of programmerproductivity and the kind of impact that our tests—and the quality of our tests—can have on it, we’ll conclude the chapter with a brief introduction to two meth-ods that are closely related to automated tests: test-driven development (TDD) andbehavior-driven development (BDD).Chapter 2 takes on the challenge of defining what makes for a good test. Inshort, we’d like to write tests that are readable, maintainable, and reliable. Part 2will go deeper into this rabbit hole by turning the question around and review-ing a collection of examples of what we don’t want to get.Part 1 concludes with chapter 3, which tackles one of the most essential toolsin the modern programmer’s tool belt—test doubles. We’ll establish the legiti-mate uses of test doubles, such as isolating code so that it can be tested properly,and we’ll make a distinction between the types of test doubles we might reachout to. Finally, we’ll throw in guidelines for making good use of test doubles tohelp you get the benefits without stumbling on common pitfalls.After reading these first three chapters you should have a good idea of whatkind of tests you want to write and why. You should also have a clear understand-ing of test doubles as a useful vehicle for getting there. The rest of this book willthen build on this foundation and give you more ammunition for your journeyin the real world.Download from Wow! eBook <>
  27. 27. Download from Wow! eBook <>
  28. 28. 3The promise of good testsWhen I started getting paid for programming, the world looked a lot different.This was more than 10 years ago and people used simple text editors like Vim andEmacs instead of today’s integrated development environments like Eclipse, Net-Beans, and IDEA. I vividly remember a senior colleague wielding his Emacs macrosto generate tons of calls to System.out.println as he was debugging our software.Even more vivid are my memories of deciphering the logs those printouts endedup in after a major customer had reported that their orders weren’t going throughlike they should.That was a time when “testing” for most programmers meant one of twothings—the stuff that somebody else does after I’m done coding, or the way yourun and poke your code before you say you’re done coding. And when a bug didslip through, you’d find yourself poking and prodding your code again—this timeadding a few more logging statements to see if you could figure out where thingswent wrong.In this chapter The value of having unit tests How tests contribute to a programmer’s productivity Using tests as a design toolDownload from Wow! eBook <>
  29. 29. 4 CHAPTER 1 The promise of good testsAutomation was a state-of-the-art concept for us. We had makefiles to compile andpackage our code in a repeatable manner, but running automated tests as part of thebuild wasn’t quite in place. We did have various shell scripts that launched one or two“test classes”—small applications that operated our production code and printed whatwas happening and what our code was returning for what kind of input. We were farfrom the kind of standard testing frameworks and self-verifying tests that report allfailures in our assertions.We’ve come a long way since those days.1.1 State of the union: writing better testsToday, it’s widely accepted that developers should write automated tests that fail thebuild if they find regressions. Furthermore, an increasing number of professionals isleaning on a test-first style of programming, using automated tests not for protectingagainst regression but for aiding them in design, specifying the behavior they expectfrom code before writing that code, thereby validating a design before verifying itsimplementation.Being a consultant, I get to see a lot of teams, organizations, products, and codebases. Looking at where we are today, it’s clear that automated tests have made theirway into the mainstream. This is good because without such automated tests, most soft-ware projects would be in a far worse situation than they are today. Automated testsimprove your productivity and enable you to gain and sustain development speed.Automated tests being mainstream doesn’t mean that our test coverage is as high as itshould be or that our productivity couldn’t improve. In fact, a significant part of mywork in the last five years or so has revolved around helping people write tests, writetests before code, and especially write better tests.Why is it so important to write better tests? What’ll happen if we don’t pay atten-tion to the quality of our tests? Let’s talk about what kind of value tests give us and whythe quality of those tests matters.Help! I’m new to unit testingIf you aren’t that familiar with writing automated tests, this would be a good time toget acquainted with that practice. Manning has released several books on JUnit, thede facto standard library for writing unit tests for Java, and the second edition of JUnitin Action (written by Petar Tahchiev, et al. and published in July 2010) is a goodprimer for writing tests for all kinds of Java code, from plain old Java objects to Enter-prise JavaBeans.In case you’re at home with writing unit tests but you’re new to Java or JUnit, perhapsall you need to get the most out of this book is to first check out appendix A, so thatyou won’t have trouble reading the examples.Download from Wow! eBook <>
  30. 30. 5The value of having tests1.2 The value of having testsMeet Marcus. Marcus is a prominent programming wiz who graduated two years agoand joined the IT group of a local investment bank, where he’s been developing thebank’s online self-service web application. Being the youngest member of the team,he kept a low profile at first and focused his energy on learning about the bankingdomain and getting to know “the way things are done here.”A few months on, Marcus started noticing that a lot of the team’s work was rework:fixing programmer mistakes.1He started paying attention to the types of mistakes theteam was fixing and noticed that most of them would’ve been caught fairly easily byunit tests. Marcus started writing unit tests here and there when he felt that the codewas particularly prone to having mistakes in it.Tests help us catch mistakes.Time went on and the rest of the team was writing unit tests, too, here and there. Mar-cus had become test-infected and rarely left a piece of code that he touched withoutfairly good coverage from automated tests.2They weren’t spending any more time fix-ing errors than they had before, but for the first time, their total number of opendefects was on the decline. The tests started having a clearly visible impact on thequality of the team’s work.Almost a year had passed since Marcus wrote the first test in their code base. Onhis way to the company Christmas party, Marcus realized how time had flown andstarted thinking back to the changes they’d seen. The team’s test coverage had grownquickly and had started stabilizing in the recent weeks and months, peaking at 98%branch coverage.For a while Marcus had thought that they should push that number all the way to100%. But in the last couple of weeks, he’d more or less made up his mind—thosemissing tests wouldn’t give them much more value, and putting any more effort intowriting tests wouldn’t yield additional benefit. A lot of the code that wasn’t coveredwith tests was there only because the APIs they used mandated the implementationof certain interfaces that Marcus’s team wasn’t using, so why test those emptymethod stubs?100% CODE COVERAGE ISN’T THE GOAL 100% sure sounds better than, say,95%, but the difference in how valuable those tests are to you may be negligi-ble. It really depends on what kind of code isn’t covered by tests and whetherthe tests you have are able to highlight programming mistakes. Having 100%coverage doesn’t guarantee the lack of defects—it only guarantees that all ofyour code was executed, regardless of whether your application behaves like itshould. So rather than obsessing about code coverage, focus your attentionon making sure that the tests you do write are meaningful.1For some reason, people referred to them as errors, defects, bugs, or issues.2The term test-infected was coined by Erich Gamma and Kent Beck in their 1998 Java Report article, “Test-Infected: Programmers Love Writing Tests.”Download from Wow! eBook <>
  31. 31. 6 CHAPTER 1 The promise of good testsThe team had reached the plateau—the flat partof the curve where additional investments havediminishing returns. In the local Java user groupmeetings, many teams had reported similar expe-riences and drawn sketches like figure 1.1.What changed Marcus’s thinking was Sebas-tian, a senior software architect who’d previouslyconsulted for the investment bank. Sebastianjoined the self-service team and quickly became amentor for the more junior team members,including Marcus. Sebastian was an old fox whoseemed to have worked with almost every majorprogramming language suitable for developingweb applications. The effect Sebastian had onMarcus was the way he wrote unit tests.Marcus had formed a habit of back-filling unittests for any code he’d written before checking itinto the team’s version control system. But Sebas-tian’s style focused on starting with a test thatwould fail (obviously), writing enough code tomake that test pass, and then writing another failing test. He’d work in this loop untilhe was done with the task.Working with Sebastian, Marcus had noticed how his own style of programmingwas starting to evolve. He’d structure his objects differently and his code looked some-how different, simply because he started looking at the design and behavior of hiscode from the caller’s perspective.Tests help us shape our design to actual use.Pointing out what he’d noticed, Marcus felt as if something clicked. As he tried to putinto words how his programming style had changed and what kind of effects it had onthe outcome, it dawned on Marcus that the tests he’d writen weren’t merely a qualityassurance tool and protection from mistakes and regression. The tests served as a wayof designing code such that it served a concrete purpose. The code Marcus wrote to passthose failing tests tended to be much simpler than what he used to write, and yet it dideverything it needed to.Tests help us avoid gold-plating by being explicitabout what the required behavior is.What Marcus, our protagonist in this story, experienced is not unlike what many test-infected programmers go through as their awareness and understanding about testsdevelops. We often start writing automated unit tests because we believe they’ll helpus avoid embarrassing, time-consuming mistakes. Later on, we may learn that havingthe tests as a safety net is just part of the equation and another—perhaps bigger—Effort and attention to writing testsValueoftestsFigure 1.1 The more tests we have,the less value an additional test is likelyto yield. The first tests we write mostlikely target some of the most crucialareas of the code base and as such tendto be high-value and high-risk. Whenwe’ve written tests for almosteverything, most likely the bits that arestill lacking test coverage are the leastcrucial and least likely to break.Download from Wow! eBook <>
  32. 32. 7The value of having testspart of the benefit is the thinking process we go through as we formulate those testsinto code.Earlier I suggested that it’s widely accepted that developers should write auto-mated tests. What’s much less widely accepted is how many and what kind of tests towrite. In my consulting work I’ve noticed that most programmers agree they shouldwrite some tests. Similarly, I’ve noticed that few programmers hold the opinion thatthey should aim for a 100% code coverage—meaning that their tests exercise everypossible execution path through the production code.So what can we make of this? We agree that writing some tests is a no-brainer, butour certainty decreases as we get closer to full code coverage—we’ve written tests forpretty much everything and the remaining code without any tests is almost too trivialto break. This is what some folks call a diminishing return and it’s often visualized with acurve similar to figure 1.1.In other words, adding a hundred carefully chosen automated tests to a code baseis a clear improvement, but when we already have 30,000 tests, those additional hun-dred tests aren’t as likely to make much difference. But as Marcus, our young pro-grammer in an investment bank, learned while working with his mentor, it’s not assimple as that. The preceding reasoning fails to acknowledge what I call the Law of theTwo Plateaus:The biggest value of writing a testlies not in the resulting testbut in what we learn from writing it.The Law of the Two Plateaus refers to the twolevels of thinking about tests that Marcus’sstory highlighted. In order to realize the fullpotential of tests, we need to climb not onebut two hills, shown in figure 1.2.As long as we perceive tests merely as aquality assurance tool, our potential isbounded by the lower arc in figure 1.2. Chang-ing our frame of mind toward tests as a pro-gramming tool—a design tool—is the catalystthat allows us to move past the quality plateautoward the second plateau, the test-as-design-tool plateau.Unfortunately, most code bases seem tostagnate somewhere along the first curve,with the developers not taking that catalyststep in using their tests to drive design. Corre-spondingly, the developers don’t focusenough of their attention on the cost of main-taining the constantly growing test suite.Using tests asa quality toolUsing tests asa design toolEffort and attention to writing testsValueoftestsFigure 1.2 The first plateau lies wherewriting more tests or better tests no longeryields additional value. The second plateau isfurther up the hill, where we uncover furtherpayoffs from changing our mindset—thinkingabout tests and testing as a resource richerthan a mere tool for verification.Download from Wow! eBook <>
  33. 33. 8 CHAPTER 1 The promise of good testsFrom this position, the big question is what are the factors influencing a program-mer’s productivity, and what role do unit tests have in that dynamic?1.2.1 Factors of productivityThe fastest way to write tests is to type them in as quickly as you can without payingattention to the health of an important part of your code base—the test code. But aswe’ll soon discuss, there are good reasons why you should take the time to nurtureyour test code, refactor away duplication, and generally pay attention to its structure,clarity, and maintainability.Test code naturally tends to be simpler than production code.3Nevertheless, if youcut corners and take on this technical debt of code quality within your tests, it’s goingto slow you down. In essence, test code ridden with duplication and superfluous com-plexity lowers your productivity, eating away some of the positive impact of havingthose tests in place.It’s not just about your tests’ readability, but also their reliability and general trust-worthiness as well as the time it takes to run those tests. Figure 1.3 illustrates the sys-tem of forces around tests that influences our productivity.Note how the two direct influences for productivity I’ve identified in this figure arethe length of the feedback cycle and debugging. In my experience these are the two mainculprits that consume a programmer’s time while on the keyboard.4Time spentdebugging is rework that could’ve largely been avoided if you’d learned about themistake soon after it happened—and the longer your feedback cycle is, the longer ittakes to even begin debugging.3For instance, few tests make use of conditional statements, loops, and so on.4There’s a lot to be said about meetings as the ultimate killer of productivity, but that’s another book.Speed of testexecutionReadability oftestsDefectsDebuggingDeveloperproductivityLength offeedback cycleAccuracy oftest results Reliability oftestsTrust-worthinessof testsshortenshelpsavoidhelpspreventhelpspreventreducesreduceslead toimprovesimprovesFigure 1.3 Multiple forces of influence related to tests directly or indirectly affect productivity.Download from Wow! eBook <>
  34. 34. 9The value of having testsWaiting for validation and verification of a change you’ve made is largely bound to thespeed of test execution, which is one of the root causes highlighted with bold in the dia-gram. The other three root causes all contribute to the amount of debugging a pro-grammer does.Lack of readability naturally makes you slower in your analysis and encourages youto pick up a debugger because reading the test code doesn’t make you any wiser—it’sso hard to understand what’s going on. Such a lack of readability also induces moredefects, as it’s difficult to see a mistake you’re making, and defects lead to more debug-ging sessions.Another factor that tends to increase the number of defects slipping through is theaccuracy of your test results. This accuracy is a fundamental requirement for you to be ableto rely on your test suite in identifying when you’ve made a mistake and introduced adefect. The two remaining test-related root causes that influence productivity are directcontributors to the tests’ accuracy, namely their trustworthiness and reliability. In order torely on your tests being accurate in their reports, the tests need to assert what theypromise to and they have to provide that assertion in a reliable, repeatable manner.It’s by paying attention to these influences to your productivity that you can scalethe quality plateau. These few root causes we’ve identified are the key for a program-mer to become more productive. The prerequisite to being productive is to knowyour tools well enough that they are not a constant distraction. Once you know yourprogramming language, you can navigate its core APIs. When you are familiar withthe problem domain, focus on the root causes—the readability, reliability, trustworthi-ness, and speed of your tests.It is by paying attention to these influences to your productivity that you can scalethe quality plateau.This is also the stuff that most of the rest of this book will revolve around—helpingyou improve your ability, awareness, and sensibility toward the test code’s readability,trustworthiness, and reliability, and making certain that you can sustain this way ofworking with the test code, ensuring its maintainability.Before we go there, let’s address that second curve from figure 1.2.The rising cost of fixing a bugWhat do you think fixing a bug would cost on average? At the XP Day 2009 conferencein London, Google’s Mark Striebeck reported on Google’s estimates around the costof delay in fixing defects.Google had estimated that it cost $5 to fix a bug immediately after a programmer hadintroduced it. Fixing the same defect would cost $50 if it escaped the programmer’seyes and was found only after running a full build of the project. The cost surged to$500 if it was found during an integration test, and to $5000 if it managed to find itsway to a system test.Considering these numbers, surely it’s better to find out about such issues as soonas you possibly can!Download from Wow! eBook <>
  35. 35. 10 CHAPTER 1 The promise of good tests1.2.2 The curve of design potentialLet’s assume you’ve written the most crucial tests first—for the most common andessential scenarios and for the most critical hotspots in the software’s architecture.Your tests are of high quality, and you’ve mercilessly refactored away duplication,keeping your tests lean and maintainable. Now imagine you’ve accumulated suchhigh coverage with your tests that the only ones that you haven’t written are those thatdirectly test trivial behaviors such as field accessors. It’s fair to say that there’s littlevalue to be added by writing those tests. Hence, the earlier gesture toward diminish-ing returns—you’ve reached an upper limit for the value you can extract from “just”writing tests.This is the quality plateau, caused by what we don’t do. I say that because to gobeyond this point and continue your journey toward a higher level of productivity, youneed to stop thinking about tests the way you’re thinking about them. In order torecover that lost potential I’m talking about, you need to harness a whole differentkind of value from the tests you write—creative, design-oriented value rather than thekind of protective, verification-oriented value of tests protecting you from regressionand defects.To summarize, in order to get across both of these two plateaus and reach the fullpotential of your tests, you need to:1 Treat your test code like you treat production code—refactoring it mercilessly,creating and maintaining high quality tests that you can trust and rely on.2 Start using your tests as a design tool that informs and guides the design of yourcode toward its actual purpose and use.As we established earlier, it’s the former of these two that most programmers stumbleover, struggling to produce the kind of high quality that we’re looking for from testcode while minimizing the operational cost of writing, maintaining, and runningthose tests. That’s also the focus of this book—having good tests.That means that we’re not going to devote much time and space to discussing theaspect of using tests as a design tool.5With that said, I do want to give you an overallunderstanding of the kind of dynamics and a way of working that I’m referring to, solet’s elaborate on that topic before moving on.1.3 Tests as a design toolAutomated tests written by programmers have traditionally been considered a qualityassurance effort targeted at the dual purposes of verifying the correctness of an imple-mentation at the time of its writing and verifying its correctness in the future as the codebase evolves. That’s using tests as a verification tool—you envision a design, write codeto implement the design, and write tests to verify that you’ve implemented it correctly.5It so happens that I’ve written an excellent book on that topic—Test Driven (Manning Publications, 2007).Another great pick would be Growing Object-Oriented Software, Guided by Tests by Steve Freeman and Nat Pryce(Addison-Wesley, 2009).Download from Wow! eBook <>
  36. 36. 11Tests as a design toolUsing automated tests as a design tool turns everything upside down. When you usetests to design your code, you transform the canonical sequence of “design, code, test”into “test, code, design.” Yes, that’s right. Test comes before code, and you concludewith a retroactive design activity. That concluding design activity is better known asrefactoring, making the sequence into test-code-refactor, visualized as figure 1.4.If this sounds familiar, you’ve probably heard of a practice called test-first program-ming or test-driven development (TDD).6That’s exactly what we’re talking about here solet’s call it what it is.1.3.1 Test-driven developmentTDD, illustrated in figure 1.5, is a disciplined programming technique built around asimple idea: no production code is written before there’s a failing test that justifies the existenceof that code. That’s why it’s sometimes referred to as test-first programming.It’s more than that; by writing a test first, you drive your production code toward adesign called for by the test. This has the following desirable consequences: Code becomes usable—The design and API of your code are suitable for the kind ofscenarios you’re going to use it for. Code becomes lean—The production code only implements what’s required bythe scenarios it’s used for.First, regardless of where in the system’s blueprints you’re currently working and whatother components, classes, or interfaces exist or don’t exist, you always have a con-crete scenario for which you design a solution. You encode that scenario into an exe-cutable example in the shape of an automated unit test. Running that test and seeingit fail gives you a clear goal of making that test pass by writing just enough productioncode—and no more.Sketching that scenario into compiling executable test code is a design activity.The test code becomes a client for the production code you’re developing and letsyou validate your design before you’ve written a single line. Expressing your needs inthe form of concrete examples is a powerful validation tool—value that we can onlyextract by using tests as a design tool.Second, by ruthlessly following the rule of only writing enough code to make thetests pass, you can keep your design simple and fit for a purpose. There’s no sign ofgold-plating anywhere because no line of code exists without a test—a scenario—that6Or you actually read the story about Marcus and his mentor, Sebastian, a few pages earlier...Test Code Refactor(repeat)Figure 1.4 The test-drivendevelopment process starts bywriting a failing test, writing code topass the test, and refactoring thecode base to improve its design.Download from Wow! eBook <>
  37. 37. 12 CHAPTER 1 The promise of good testswarrants it. One of the worst offenders of code quality and a major contributor todevelopers’ productivity crawling to a halt is accidental complexity.Accidental complexity is unnecessary complexity. It’s complexity that couldbe avoided by substituting a simpler design that still meets the requirements. Some-times we programmers like to demonstrate our mental capacity by producing suchcomplex designs that we have trouble understanding them ourselves. I will bet yourecognize that primal instinct in yourself, too. The problem with complex designs isthat complexity kills productivity and unnecessary complexity is, well, unnecessaryand counterproductive.The combination of a test specifying a flaw or missing functionality in the code, therule of writing only enough code to make the tests pass, and the ruthless refactoringtoward a simple design7are enormously powerful at keeping such accidental complex-ity at bay. It’s not a magic potion, and ultimately what you end up with is influenced bythe programmer’s design sense and experience.There’s much more to TDD than this—so much more that entire books have beenwritten about the method, including my previous book, Test Driven (Manning, 2007),and the more recent Growing Object-Oriented Software, Guided by Tests by Steve Freemanand Nat Pryce (Addison-Wesley, 2009). If you’d prefer a more compact introductory7Simple design isn’t the same as simplistic design.Write a newtestRun all testsMake testspass bywritingproductioncodeRun all testsStartRefactorproductionand test codeTests pass?Run all testsTests pass?Revert torefactoringTests pass?yesnonoyesnoyesFigure 1.5 Test-driven development is a cyclic process of writing a failing test, making it pass, andrefactoring the code to amplify clarity of intent while minimizing duplication. Every step along the way,you’re constantly running tests to make sure that you know where you stand.Download from Wow! eBook <>
  38. 38. 13Tests as a design tooltitle on TDD, I recommend picking up a copy of Kent Beck’s Test Driven Development: ByExample (Addison-Wesley, 2002).With such widely available and thorough descriptions of TDD around, we’re goingto keep it short and let you refer to those titles for a more in-depth exploration. But Ido want to discuss one aspect of TDD because it has close ties to our main topic ofgood tests—a style of programming called behavior-driven development.1.3.2 Behavior-driven developmentYou may have heard of something called BDD, which stands for behavior-driven devel-opment. Though people have programmed in some kind of a test-first manner for sev-eral decades, the method we know as test-driven development was described andgiven a name during the 1990s.Some 10 years later, a London-based consultant named Dan North stepped up tochampion the next evolution of test-first programming, after realizing that the vocab-ulary and mindset of TDD speaking about “tests” was leading people along the wrongpath. Dan gave this style of TDD the name behavior-driven development and, in hisintroductory article published in Better Software in 2006, he described BDD like this:It suddenly occurred to me that people’s misunderstandings about TDD almostalways came back to the word “test.”That’s not to say that testing isn’t intrinsic to TDD—the resulting set ofmethods is an effective way of ensuring your code works. However, if the meth-ods do not comprehensively describe the behaviour of your system, then theyare lulling you into a false sense of security.I started using the word “behaviour” in place of “test” in my dealings withTDD and found that not only did it seem to fit but also that a whole category ofcoaching questions magically dissolved. I now had answers to some of thoseTDD questions. What to call your test is easy—it’s a sentence describing thenext behaviour in which you are interested. How much to test becomes moot—you can only describe so much behaviour in a single sentence. When a testfails, simply work through the process described above—either you introduceda bug, the behaviour moved, or the test is no longer relevant.I found the shift from thinking in tests to thinking in behaviour so profoundthat I started to refer to TDD as BDD, or behaviour-driven development.After Dan started writing and talking about BDD, others in the global community ofsoftware developers stepped forward to further develop the ideas of example-driven,specification-of-behavior oriented thinking that was embodied in Dan North’s BDD. Asa result, people today refer to BDD in contexts and domains wider than that of code—most notably integrating BDD with business analysis and specification activities at therequirements level.A particular concept that BDD practitioners vouch for is outside-in developmentwith acceptance tests. This is how Matt Wynne and Aslak Hellesøy, authors of TheCucumber Book: Behaviour-Driven Development for Testers and Developers, describe it:Behaviour-Driven Development builds upon Test-Driven Development by for-malizing the good habits of the best TDD practitioners. The best TDD practi-Download from Wow! eBook <>
  39. 39. 14 CHAPTER 1 The promise of good teststioners work from the outside-in, starting with a failing customer acceptancetest that describes the behavior of the system from the customer’s point ofview. As BDD practitioners, we take care to write the acceptance tests as exam-ples that anyone on the team can read. We make use of the process of writingthose examples to get feedback from the business stakeholders aboutwhether we’re setting out to build the right thing before we get started.As testament to this development, a crop of BDD tools and frameworks has popped upseeking to embed the underlying ideas, practices, and conventions into the softwaredeveloper’s tool chain. We’ll see some of those tools in chapter 8.When we speak of “good tests” in the remainder of this book, keep in mind therealization Dan made and pay attention to words. Vocabulary matters.1.4 SummaryWe’ve come a long way from the days of “cowboy coding” with programmers hackingaway at code without any tests to catch them when they fall from the saddle. Developertesting and automated tests are commonplace or, if not exactly standard practice, aredefinitely a topic of active discussion. The value of having a thorough suite of auto-mated tests for your code is undeniable.In this chapter you familiarized yourself with the Law of the Two Plateaus. The firstplateau is where the value programmers are getting out of their tests is limitedbecause they already have full test coverage. But you can get further. Programmerswho pay attention to the quality of their tests zoom right past the first plateau, aimingfor the top. It’s one thing to have tests, and it’s a whole nother thing to have goodtests. That’s why you’re reading this book and that’s the challenge we’re going to takeup in the remaining chapters.After all, you get much more out of the climb if you go all the way to the top ratherthan stopping halfway.Download from Wow! eBook <>
  40. 40. 15In search of goodWe’re on a journey of learning about good tests. We want to learn to identify goodtests, write good tests, and improve not-so-good tests so they become good tests—orat least closer to being good tests. The question is, What makes a test “good”? Whatare the magic ingredients? There are several aspects to consider, including: The test code’s readability and maintainability How the code is organized both within the project and within a givensource file What kinds of things a test is checking for How reliable and repeatable a test is How a test makes use of test doublesWe’ll be taking a closer look at all of these aspects in this chapter.The preceding list is far from being comprehensive. The range of factors thatmay tip your test-quality scale either way is endless. Similarly, some of the factorsIn this chapter What makes a test “good”? Testing relevant behavior The importance of reliable testsDownload from Wow! eBook <>
  41. 41. 16 CHAPTER 2 In search of gooddon’t matter that much in all contexts. For some tests, their execution speed may becrucial, whereas for other tests, being extremely focused is key.Furthermore, some of the quality of test code is in the eye of the beholder. As is thecase with code in general, personal preference has a role in defining what “good” is—I’m not going to pretend that it wouldn’t. I’m also not going to pretend that I canavoid my bias and preference from coming through in this book. Though I’ve tried tosteer away from pure matter-of-taste questions, you’ll find numerous sections wheremy opinions clearly show through. I think that’s fine. After all, the best I can offer ismy honest (and opinionated) view of things based on my experience, shaped by thewonderful individuals and software professionals from whom I’ve learned about codeand, specifically, about test code.With that disclaimer out of the way, let’s discuss some of those aspects of test qual-ity and establish what about them makes them relevant to our interests.2.1 Readable code is maintainable codeYesterday I popped into our office on my way back from a consulting gig and struck upa conversation with a colleague about an upcoming 1K competition that my colleaguewas going to attend. Such competitions are an age-old tradition at demo parties —atype of geek gathering where hackers gather at a massive arena for a long weekendwith their computers, sleeping bags, and energy drinks. Starting from the first gather-ings, these hackers have faced off, wielding their mad skills at producing 3D anima-tions with what most people would today consider antiquated hardware.A typical constraint for such animations has been size. In the case of the competi-tion my colleague was preparing for, the name 1K refers to the maximum size of thecode compiled into a binary executable, which must be less than 1,024 bytes. Yes,that’s right—1,024 bytes. In order to squeeze a meaningful program into such a tinyspace, the competitors need to resort to all kinds of tricks. For example, one commontrick to pack your code more tightly is to use the same name for many variables—because the resulting code compresses a bit better like that. It’s crazy.What’s also crazy is the resulting code. When they’re done squeezing the codedown to 1,024 bytes, the source code is undecipherable. You can barely recognizewhich programming language they’ve used! It’s essentially a write-only code base—once you start squeezing and compressing, you can’t make functional changesbecause you wouldn’t be able to tell what to edit where and how.To give you a vivid taste of what such code might look like, here’s an actual submis-sion from a recent JS1k competition where the language of choice is JavaScript and itneeds to fit into 1,024 bytes:<script>with({margin="0px";overflow="hidden";}var w=window.innerWidth;var h=window.innerHeight;var ca=document.getElementById("c");ca.width=w;ca.height=h;var c=ca.getContext("2d");m=Math;fs=m.sin;fc=m.cos;fm=m.max;setInterval(d,30);function p(x,y,z){return{x:x,y:y,z:z};}function s(a,z){r=w/10;R=w/3;b=-20*fc(a*5+t);return p(w/2+(R*fc(a)+r*fs(z+2*t))/z+fc(a)*b,h/2+(R*fs(a))/z+fs(a)*b);}function q(a,da,z,dz){var v=[s(a,z),s(a+da,z),s(a+da,z+dz),s(a,z+dz)]Download from Wow! eBook <>
  42. 42. 17Readable code is maintainable code;c.beginPath();c.moveTo(v[0].x,v[0].y);for(i in v)c.lineTo(v[i].x,v[i].y);c.fill();}var Z=-0.20;var t=0;function d(){t+=1/30.0;c.fillStyle="#000";c.fillRect(0,0,w,h);c.fillStyle="#f00";var n=30;var a=0;var da=2*Math.PI/n;var dz=0.25;for(var z=Z+8;z>Z;z-=dz){for(var i=0;i<n;i++){fog=1/(fm((z+0.7)-3,1));if(z<=2){fog=fm(0,z/2*z/2);}var k=(205*(fog*Math.abs(fs(i/n*2*3.14+t))))>>0;k*=(0.55+0.45*fc((i/n+0.25)*Math.PI*5));k=k>>0;c.fillStyle="rgb("+k+","+k+","+k+")";q(a,da,z,dz);if(i%3==0){c.fillStyle="#000";q(a,da/10,z,dz);}a+=da;}}Z-=0.05;if(Z<=dz)Z+=dz;}</script>Granted, that is a couple of magnitudes more extreme a situation than what you’dfind at a typical software company. But we’ve all seen code at work that makes ourbrains hurt. Sometimes we call that kind of code legacy because we’ve inherited it fromsomeone else and now we’re supposed to maintain it—except that it’s so difficult thatour brains hurt every time we try to make sense of it. Maintaining such unreadablecode is hard work because we expend so much energy understanding what we’re look-ing at. It’s not just that. Studies have shown that poor readability correlates stronglywith defect density.1Automated tests are a useful protection against defects. Unfortunately, automatedtests are also code and are also vulnerable to bad readability. Code that’s difficult toread tends to be difficult to test, too, which leads to fewer tests being written. Further-more, the tests we do write often turn out to be far from what we consider good testsbecause we need to kludge our way around the awkwardly structured, difficult-to-understand code with APIs and structures that aren’t exactly test-friendly.We’ve established (almost to the point of a rant) that code readability has a direimpact on the code’s maintainability. Now what about the readability of test code?How is that different, or is it any different? Let’s take a look at a not-so-far-fetchedexample of unreadable test code, shown in this listing:@Testpublic void flatten() throws Exception {Env e = Env.getInstance();Structure k = e.newStructure();Structure v = e.newStructure();//int n = 10;int n = 10000;for (int i = 0; i < n; ++i) {k.append(e.newFixnum(i));v.append(e.newFixnum(i));}Structure t = (Structure),new IObject[] {v}, Block.NULL_BLOCK);v = (Structure) t.flatten(e.getCurrentContext());assertNotNull(v);}1Raymond P.L. Buse, Westley R. Weimer. “Learning a Metric for Code Readability.” IEEE Transactions on Soft-ware Engineering, 09 Nov. 2009. IEEE computer Society Digital Library. IEEE Computer Society, 2.1 Code doesn’t have to be complex to lack readabilityDownload from Wow! eBook <>
  43. 43. 18 CHAPTER 2 In search of goodWhat is this test checking? Would you say that it’s easy to decipher what’s going onhere? Imagine yourself being the new guy on this team—how long would it take you tofigure out the test’s intent? What kind of code forensics would you have to go throughto grok the situation if this test suddenly starts failing? Based on how I feel about thathideous snippet of code, I’ll wager that you immediately identified a handful of thingsthat could be improved about the poor little test—and that readability is a commontheme with those improvements.2.2 Structure helps make sense of thingsI’ve had the pleasure and horror of seeing numerous code bases that weren’t full ofbeautiful strides of genius flowing from one source file to the next. Some of themnever jumped to another source file because it was all there—all of the code and logictriggered by, say, the submission of a web form would reside in a single source file. I’vehad a text editor crash due to the size of a source file I foolishly tried to open. I’veseen a web application vomit an error because a JavaServer Pages file had grown so bigthat the resulting byte code violated the Java class file specification. It’s not just thatstructure would be useful—the lack of structure can be damaging.What’s common among most of these instances of never-ending source listings isthat nobody wanted to touch them. Even the simplest conceptual changes would betoo difficult to map onto the source code in front of you. There was no structure yourbrain could rely on. Divide and conquer wasn’t an option—you had to juggle thewhole thing in your head or be prepared for a lot of contact between your foreheadand the proverbial brick wall.As illustrated by figure 2.1 you don’t want just any structure to help make sense ofthings. You need structure that makes sense as such—one that’s aligned with the wayyour brain and your mental models are prepared to slice and dice the world. Blindlyexternalizing snippets of code into separate source files, classes, or methods doesreduce the amount of code you’re looking at a given point in time, thereby alleviatingthe problem of overloading your brain. But it doesn’t get you much closer to isolatingand understanding the one aspect of the program’s logic that we’re interested in rightnow. For that you need a structure that makes sense.Code has nostructureNobody wantsto touch it...Any change implies aghtConceptual change ishard to map into code...but you have tocode will havemore structure......but is it useful structure?Chop it up intosmaller chunksFigure 2.1 It’s not just about having structure—it needs to be a useful structure.Download from Wow! eBook <>
  44. 44. 19Structure helps make sense of thingsWhen faced with monolithic, never-ending source listings, the obvious solution is tochop them up into smaller pieces, extracting blocks of code into methods. You mightgo from a huge 500-line method in one class to dozens of methods on 10 classes withthe average length dropping below 10 lines per method. That would introduce morestructure into the code—at least if you ask a compiler. You’d also be able to see wholemethods on your screen at once instead of scrolling back and forth.But if the boundaries for splitting up that monolith don’t make sense—if theydon’t map to the domain and your abstractions—we might be doing more harm thangood because the concepts might now be physically scattered farther from each otherthan before, increasing the time you spend going back and forth between source files.It’s simple. What matters is whether the structure of your code helps you locate theimplementation of higher-level concepts quickly and reliably.Test code is an excellent example of this phenomenon. Let’s say you have an appli-cation that’s fairly well covered by an automated test—one automated test. Imaginethat this test exercises all of the application’s business logic and behavior through asingle monolithic test method that takes half an hour to execute. Now say that the testeventually fails, as illustrated in figure 2.2, because you’re making a change in howmailing addresses are represented internally in the application and you mess up some-thing while you’re at it. There’s a bug. What happens next?I’d imagine it’s going to take a while to pinpoint the exact place in the test codewhere your programming error manifests itself. There’s no structure in the test codeto help you see what affects what, where a certain object is instantiated, what the valueof a given variable is at the point where things fall apart, and so forth. Eventually,when you’ve managed to identify and correct your mistake, you have no choice but torun the whole test—all 30 minutes of it—to make sure that you really did fix the prob-lem and that you didn’t break something else in the process.Continuing this thought experiment, fast-forwarding an hour or so, you’re aboutto make another change. This time, having learned from your previous mistake,you’re careful to make sure you’ve understood the current implementation in orderto ensure that you’ll make the right kind of change. How would you do that? By read-ing the code, and especially perusing test code that shows you in concrete terms howthe production code is expected to behave. Except that you can’t find the relevantparts of the test code because there’s no structure in place.Monolithic test30-minute wait time begins...ProgrammererrorHey! It broke!Figure 2.2 Long delay in feedback is a real productivity-killerDownload from Wow! eBook <>
  45. 45. 20 CHAPTER 2 In search of goodWhat you need are focused tests that are readable, accessible, and comprehensible sothat you can: Find test classes that are relevant for the task at hand Identify the appropriate test methods from those classes Understand the lifecycle of objects in those test methodsThese are all things that you can get by paying attention to your tests’ structure andmaking sure that it’s useful. Having a useful structure is hardly enough, of course.2.3 It’s not good if it’s testing the wrong thingsMore than once I’ve concluded a journey of reading and debugging code to find thecause for an undesirable system behavior at almost the same place I started looking. Aparticularly annoying detail overlooked in such a bug-hunt is the contents of a test.The first thing I tend to do when digging into code is to run all the tests to tell mewhat’s working and what’s not. Sometimes I make the mistake of trusting what thetests’ names tell me they’re testing. Sometimes it turns out that those tests are testingsomething completely different.This is related to having a good structure—if a test’s name misrepresents what ittests, it’s akin to driving with all the road signs turned the wrong way. You should beable to trust your tests.A couple of years ago I was carrying out a code audit for a product that had beenunder development for more than a decade. It was a big codebase and I could tellfrom its structure that some parts were clearly newer than others. One of the thingsthat distinguished more recent code from the older was the presence of automatedtests. But I quickly found out that I couldn’t tell from the tests’ names what they weresupposed to verify and, looking closer, it turned out that the tests weren’t actually test-ing what they promised to test. It wasn’t a Java codebase but I’ve taken the freedom totranslate a representative example to Java:public class TestBmap {@Testpublic void mask() {Bmap bmap = new Bmap();bmap.addParameter(IPSEC_CERT_NAME);bmap.addParameter(IPSEC_ACTION_START_DAYS, 0);bmap.addParameter(IPSEC_ACTION_START_HOURS, 23);assertTrue(bmap.validate());}}Looking at this code you’ll immediately notice that the test’s name is less than perfect.But on a closer look it turns out that whatever “mask” means for a “Bmap,” the test isonly checking that certain parameters are a valid combination. Whether the valida-tion works is somewhat irrelevant if the actual behavior isn’t correct even when theinput is valid.Download from Wow! eBook <>
  46. 46. 21Independent tests run easily in solitudeThere’s a lot to be said about testing the right things, but it’s also crucial to testthose right things the right way. Of particular importance from a maintainability pointof view is that your tests are checking for the intended behavior and not a specificimplementation. That’s a topic we’ll touch on in the next chapter, so let’s leave it atthat for now.2.4 Independent tests run easily in solitudeThere’s a lot to be said about tests, what they should or shouldn’t contain, what theyshould or shouldn’t specify, and how they should be structured in the name of read-ability. What goes on around tests sometimes plays an equally vital role.Human beings—our brains to be more exact—are enormously powerful informa-tion processors. We can make seemingly instant evaluations of what’s going on in ourphysical surroundings and react in a blink. We dodge that incoming snowball beforewe even realize what we’re reacting to. These reactions are in our DNA. They’re behav-ioral recipes that instruct our body to move when our senses observe a familiar pat-tern. Over time our cookbook of these recipes grows in sophistication, and we’re soonlugging around a complex network of interconnected patterns and behaviors.This happens at work, too. Exploring a foreign code base for the first time, we’llhave formed a clear picture of the most prevalent conventions, patterns, code smells,and pitfalls within 15 minutes. What makes this possible is our ability to recognize afamiliar pattern and be able to tell what else we’re likely to see nearby.WHAT’S A CODE SMELL? A smell in code is a hint that something might bewrong with the code. To quote the Portland Pattern Repository’s Wiki, “Ifsomething smells, it definitely needs to be checked out, but it may not actu-ally need fixing or might have to just be tolerated.”For example, one of the first things I pay attention to when introducing myself to anew codebase is the size of methods. If methods are long I know that there are a bunchof other issues to deal with in those particular modules, components, or source files.Another signal I’m tuning to is how descriptive the names of the variables, methodsand classes are.Specifically in terms of test code, I pay attention to the tests’ level of independence,especially near architectural boundaries. I do this because I’ve found so many codesmells by taking a closer look at what’s going on in those boundaries, and I’ve learnedto be extra careful when I see dependencies to: Time Randomness Concurrency Infrastructure Pre-existing data Persistence NetworkingDownload from Wow! eBook <>
  47. 47. 22 CHAPTER 2 In search of goodWhat these things have in common is thatthey tend to complicate what I considerthe most basic litmus test for a project’stest infrastructure: can I check out a freshcopy from version control to a brand newcomputer I just unboxed, run a singlecommand, lean back, and watch a fullsuite of automated tests run and pass?Isolation and independence areimportant because without them it’smuch harder to run and maintain tests.Everything a developer needs to do totheir system in order to run unit testsmakes it that much more burdensome.Whether you need to create anempty directory in a specific location inyour filesystem, make sure that you havea specific version of MySQL running at aspecific port number, add a databaserecord for the user that the tests use forlogin, or set a bunch of environmentvariables—these are all things that adeveloper shouldn’t need to do. All ofthese small things add up to increasedeffort and weird test failures.2A characteristic of this type of depen-dency is that things like the system clockat the time of test execution or the nextvalue to come out from a random num-ber generator are not in your control. As arule of thumb, you want to avoid erratictest failures caused by such dependen-cies. You want to put your code into abench vise and control everything 3bypassing it test doubles and otherwise iso-lating the code to an environment thatbehaves exactly like you need it to.2If you can’t find a way to avoid such manual configuration, at least make sure developers need to do it onlyonce.3If this sounds far-fetched, you should know that JUnit doesn’t promise to run test methods in any particularorder. In fact, several tests in the NetBeans project started breaking when Java 7 changed the order in whichdeclared methods are returned through Class#getDeclaredMethods(). Oops. I guess they didn’t haveindependent tests...Don’t rely on test order within atest classThe general context for the advice ofnot letting tests depend on each otheris that you should not let tests in oneclass depend on the execution or out-come of tests in another class. But itreally applies to dependencies within asingle test class, too.The canonical example of this mistakeis when a programmer sets up the sys-tem in a starting state in a @Before-Class method and writes, say, threeconsecutive @Test methods, eachmodifying the system’s state, trustingthat the previous test has done itspart. Now, when the first test fails, allof the subsequent tests fail, but that’snot the biggest issue here—at leastyou’re alerted to something beingwrong, right?The real issue is when some of thosetests fail for the wrong reason. Forinstance, say that the test frameworkdecides to invoke the test methods ina different order. False alarm. The JVMvendor decides to change the order inwhich methods are returned throughthe Reflection API. False alarm. Thetest framework authors decide to runtests in alphabetical order. Falsealarm again.3You don’t want false alarms. You don’twant your tests failing when the behav-ior they’re checking isn’t broken.That’s why you shouldn’t intentionallymake your tests brittle by having themdepend on each other’s execution.Download from Wow! eBook <>
  48. 48. 23Reliable tests are reliableOne of the most unusual examples of a surprising test failure is a test that passes aspart of the whole test suite but fails miserably when it’s run alone (or vice versa).Those symptoms reek of interdependent tests. They assume that another test is runbefore they are, and that the other test leaves the system in a particular state. Whenthat assumption kicks you in the ankle, you have one hellish debugging session ahead.To summarize, you should be extra careful when writing tests for code that dealswith time, randomness, concurrency, infrastructure, persistence, or networking. As arule of thumb, you should avoid these dependencies as much as you can and localizethem into small, isolated units so that most of your test code doesn’t need to sufferfrom the complications and you don’t have to be on your toes all the time—just inthose few places where you tackle the tricky stuff.So how would that look in practice? What exactly should you do? For example, youcould see if you can find a way to: Substitute test doubles for third-party library dependencies, wrapping themwith your own adapters where necessary. The tricky stuff is then encapsulatedinside those adapters that you can test separately from the rest of applicationlogic. Keep test code and the resources they use together, perhaps in the samepackage. Let test code produce the resources it needs rather than keeping them separatefrom the source code. Have your tests set up the context they need. Don’t rely on any other tests beingrun before the one you’re writing. Use an in-memory database for integration tests that require persistence, as itgreatly simplifies the problem of starting tests with a clean data set. Plus, they’regenerally superfast to boot up. Split threaded code into asynchronous and synchronous halves, with all applica-tion logic in a regular, synchronous unit of code that you can easily test withoutcomplications, leaving the tricky concurrency stuff to a small, dedicated groupof tests.Achieving test isolation can be difficult when working with legacy code that wasn’tdesigned for testability and therefore doesn’t have the kind of modularity you’d liketo have. But even then, the gain is worth the effort of breaking those nasty dependen-cies and making your tests independent from their environment and from each other.After all, you need to be able to rely on your tests.2.5 Reliable tests are reliableIn the previous section I said that sometimes a test is testing something completely dif-ferent than what you thought it tests. What’s even more distracting is that sometimesthey don’t test a damn thing.Download from Wow! eBook <>
  49. 49. 24 CHAPTER 2 In search of goodA colleague used to call such tests happy tests, referring to a test happily executing apiece of production code—possibly all of its execution paths—without a single asser-tion being made. Yes, your test coverage reports look awesome as the tests are thor-oughly executing every bit of code you’ve written. The problem is that such tests canonly fail if the production code being invoked throws an exception. You can hardlyrely on such tests to watch your back, can you? Especially if the programmers have hada convention to encapsulate all of the test methods’ bodies into a try-catch.4Thislisting shows an example of one such bugger.@Testpublic void shouldRefuseNegativeEntries() {int total =;try {record.add(-1);} catch (IllegalArgumentException expected) {assertEquals(total,;}}Some tests are less likely to fail than others, and the previous listing is a prime exampleof the other extreme, where the test is likely to never fail (and likely never has in thepast either). If you look carefully, you’ll notice that the test won’t fail even if add(-1)doesn’t throw an exception as it’s supposed to.Tests that can hardly ever fail are next to useless. With that said, a test that passesor fails intermittently is an equally blatant violation toward fellow programmers; seefigure 2.3.Some years ago I was consulting on a project and spent most of my days pair pro-gramming with the client’s technical staff and other consultants. One morning I tookon a new task with my pair and ran the related test set as the first thing to do as usual.My pair was intimately familiar with the codebase, having written a significant part of it,and was familiar with its quirks, too. I noticed this when a handful of the tests failed onthe first run before we’d touched anything. What tipped me off was how my pairresponded to the test failure—he routinely started rerunning the tests again and againuntil after four or five times all of the tests had passed at least once. I’m not 100% sure,but I don’t think those particular tests all passed at the same time even once.4True story. I spent an hour removing them and the rest of the day fixing or deleting the failed tests that wereuncovered by my excavation.Listing 2.2 Can you spot the flaw in this test?TestneverfailsTestalwaysfailsDiminishing value Pure noise"Happy" tests Random testsWheres the sweet spot?Figure 2.3 Tests havelittle value if they’venever failed or if they’refailing all the time.Download from Wow! eBook <>
  50. 50. 25Every trade has its tools and tests are no exceptionAs flabbergasted as I was, I realized that I was looking at a bunch of tests that repre-sented a whole different cast of unreliable tests. These particular tests turned out to befailing randomly because the code under test incorporated nondeterministic logicthat screwed up the tests some 50% of the time. In addition to the use of pseudo-ran-dom generators in the code being tested, a common cause for such intermittently fail-ing behavior is the use of time-related APIs. My favorite is a call toSystem.currentTimeMillis(), but a close second is a ubiquitous Thread.sleep(1000) sprinkled throughout in an attempt to test asynchronous logic.In order to rely on your tests, they need to be repeatable. If I run a test twice, it mustgive me the same result. Otherwise, I’ll have to resort to manual arbitration after everybuild I make because there’s no way to tell whether 1250/2492 tests means that every-thing’s all right or that all hell’s broken loose with that last edit. There’s no way to tell.If your logic incorporates bits that are asynchronous or dependent on currenttime, be sure to isolate those bits behind an interface you can use for substituting a“test double” and make the test repeatable—a key ingredient of a test being reliable.2.6 Every trade has its tools and tests are no exceptionWhat’s this test double I speak of? If you don’t have test doubles in your programmer’stoolkit, you’re missing out on a lot of testing power. Test double is an umbrella term forwhat some programmers have come to know as stubs, fakes, or mocks (which is short formock object). Essentially they’re objects that you substitute for the real implementa-tion for testing purposes. See figure 2.4.You could say that test doubles are a test-infected programmer’s best friend. That’sbecause they facilitate many improvements and provide many new tools for our dis-posal, such as: Speeding up test execution by simplifying the code that would otherwise beexecuted Simulating exceptional conditions that would otherwise be difficult to create Observing state and interaction that would otherwise be invisible to your testcodeThere’s a lot more to test doubles than this and we’ll come back to this topic in moredetail in the next chapter. But test doubles aren’t the only tool of the trade for writinggood automated tests.DuckCampbell Duck Crested Duck Test Double DuckFigure 2.4 A test double for a duck looks just like a duck and quacks almost like aduck—but certainly isn’t a real duck.Download from Wow! eBook <>
  51. 51. 26 CHAPTER 2 In search of goodPerhaps the most essential tool of the trade is a test framework such as JUnit. I stillremember some of my first attempts at getting a piece of code to work like I wanted itto. When I messed up and got stuck, I’d uncomment half a dozen statements thatwould print to the console and relaunch the program so I could analyze the consoleoutput and figure out what I’d broken and how.It wasn’t more than a couple of months into my professional career when I firstbumped into this same practice being upheld by commercial software developers. Itrust that I don’t need to point out how incredibly unprofessional and wasteful such apractice is compared to writing automated, repeatable tests with tools like JUnit.In addition to a proper test framework and test doubles, my top three tools of thetrade for software developers writing automated tests include one more tool in thechain—the build tool. Whatever your build process looks like, whichever tool or tech-nology your build scripts use under the hood, there’s no good excuse for not integrat-ing your automated tests as part of that build.2.7 SummaryIn this chapter we’ve established several coarse properties for what a good test is. Wenoted that these things are dependent on context and that there are few absolutetruths when it comes to what makes a test “good.” We did identify a number of issuesthat generally have a major impact on how good or appropriate—how fit for its pur-pose—an automated test is.We began by noting that one of the essential virtues for a test is its readability,because lacking the ability to be read and understood, test code becomes a mainte-nance problem that solves itself very soon—by getting deleted because it’s too costly tomaintain.We then pointed out how test code’s structure makes it usable, allowing the pro-grammer to quickly find their way to the right pieces and helping the programmerunderstand what’s going on—a direct continuation on readability.Next we shed light on how tests sometimes test the wrong things and how that cancreate problems by leading you down the wrong path or by muddying the waters, hid-ing the test’s actual logic and making the test itself unreadable.To conclude the underlying theme of tests sometimes being unreliable, we identi-fied some of the common reasons for such unreliability and how important it is fortests to be repeatable.Lastly, we identified three essential tools of the trade for writing automated tests—a test framework, an automated build that runs tests written with that framework, andtest doubles for improving your tests and ability to test. This third topic is importantenough that we’ve dedicated the next chapter to discussing the use of test doubles forwriting good tests.Download from Wow! eBook <>
  52. 52. 27Test doublesThe concept of stubs or dummies has been around about as long as we’ve had theability to structure software code into classes and methods. Once the primary rea-son for creating such utilities was to serve as a placeholder until the real thingbecame available—to allow you to compile and execute one piece of code beforeits surrounding pieces were in place.These objects have much more diverse purposes in the context of moderndeveloper testing. Instead of merely allowing the compilation and execution ofcode without certain dependencies present, the test-infected programmer creates avariety of such “for testing only” facilities in order to isolate the code under test,speed up test execution, make random behavior deterministic, simulate specialconditions, and to give tests access to otherwise hidden information.These purposes are served by similar and yet different types of objects that wecollectively refer to as test doubles.1In this chapter What can we do with test doubles? What kind of test doubles do we have at our disposal? Guidelines for using test doubles1Though the term test double was first introduced to me by fellow Manning author J. B. Rainsberger. I creditGerard Meszaros and his book, xUnit Test Patterns: Refactoring Test Code (Addison Wesley, 2007), for popu-larizing the term and the related taxonomy within the software development community.Download from Wow! eBook <>