open source


Published on

1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

open source

  1. 1. open source Editor: Christof Ebert ■ Alcatel ■ JUnit: Unit Testing and Coding in Tandem Panagiotis Louridas Detecting and correcting defects either during or close to the phase where they originate is key for any fast, cost-effective software development. Unit testing, performed by the code writer, is still the most popular technique. Our contributor, Louridas Panagiotis, has summarized his experiences around JUnit, probably the most popular OSS tool for Java unit testing. Glad to hear from you, both readers and prospective authors, about this column and the products and tools you want to know more about. —Christof Ebert he prospect of debugging has haunted grammers write some lines of code, then write a T programmers ever since a moth was found trapped inside the Harvard Mark II on 9 September 1947. (This “first ac- tual” bug is now at the Smithsonian Na- tional Museum of American History.) Computer pioneer Maurice Wilkes described the moment he understood this prospect: “The realization came over me test for them; if the test fails, they correct the bug. If everything is fine, they write some more code and some more tests, and so on. All tests are kept with the code, so if the working code changes, it’s easy to perform regression tests to ensure that the changes didn’t break the software. Easy integration of coding and unit testing lies at the heart of Extreme Programming and the agile software development movement. In with full force that a good part agile programming, software is constructed by of the remainder of my life was implementing functionality incrementally, in going to be spent in finding er- short spurts of activity. To do this requires as- rors in my own programs.” surance that what we implement is correct so His realization rings true for that we can deliver it with confidence and con- many computer professionals. tinue building on it. Unit testing offers a way to keep bugs in check. First, it applies to individual JUnit concepts pieces of code—typically, functions or meth- In unit testing, we exercise some code—pro- ods—and unless these smallest pieces are cor- viding input, if needed, and examining the code’s rect, the whole software edifice is sure to crum- behavior, usually in the form of its output. A ble. Second, unit testing lets the people who simple way to do this, and one that program- write the code units also test them—if only pro- mers have long used, employs a series of If state- grammers would buy into it. Testing smokes ments that compare something to its expected out the bugs, but programmers are generally value. In JUnit, instead of writing a series of If loath to devote time to anything but coding. statements, we write assertions. An assertion is a JUnit is an open source Java library that pur- way of specifying the desired outcome and com- ports to make unit testing fun—so much fun, in paring it with the actual outcome. If the desired fact, that programmers will love writing tests. and actual outcomes are identical, the assertion Kent Beck and Erich Gamma created JUnit, de- succeeds; otherwise, it fails. riving it from Beck’s Smalltalk testing framework. A test usually needs some scaffolding—a The idea is to test and code simultaneously. Pro- way to initialize variables and create objects, for 12 IEEE SOFTWARE Published by the IEEE Computer Society 0740-7459/05/$20.00 © 2005 IEEE
  2. 2. OPEN SOURCE example. In JUnit, preparation is called setup. It’s independent of the assertions, import junit.framework.TestCase; import junit.framework.Test; so the same scaffolding can apply to sev- import junit.framework.TestSuite; eral individual tests. The setup is run be- fore each test. public class ComplexTest extends TestCase { When a test ends, the scaffolding private Complex a; private Complex b; might require some cleanup actions. In JUnit this is called teardown. It ensures protected void setUp() { that each test ends up with a clean a = new Complex(1, -1); slate. Then if another test ensues, the b = new Complex(2, 5); } setup will run correctly. Setup and tear- (a) } down are called a test’s fixture. Individual tests are called test cases. public void testComplexEquality() Test cases seldom exist in a vacuum. If a Complex expected = new Complex(1, -1); project undertakes serious unit testing, it assertEquals(expected, a); will accumulate a large test case collec- } tion. Often, programmers seem to exer- public void testComplexAddition() { cise some of these cases together. When Complex expected = new Complex(3, 4); this occurs, they can combine the cases assertEquals(expected, a.add(b)); into test suites that run as a single group. } public void testComplexMultiplication() { An example Complex expected = new Complex(1*2 - (-1)*5, This is easier to understand in prac- 1*5 + (-1)*2); tice. Let’s suppose that we need a com- assertEquals(expected, a.multiply(b)); (b) } plex number class and that, true to the “not invented here” syndrome, none of the libraries from a simple search en- public static Test suite() { gine query impressed us. Our Complex return new TestSuite(ComplexTest.class); (c) } class will contain methods for the known operations in complex num- bers, and we’d like to test them. Figure 1. Complex number class example. (a) JUnit test setup, (b) JUnit test We start by creating a TestCase sub- cases, and (c) dynamic creation of a test suite for all test cases in a class. class. In the subclass, we create an in- stance variable for each part of the fix- The assertEquals method is over- deed, in the test-directed development ture. We then override the setUp() loaded and defined for all Java primi- that the agile movement promotes, you method for initializing variables and the tives, apart from Java objects. We can can write tests first and code second, al- tearDown() method for cleaning up any also use assertTrue and assert- though you can always start with the resources the test uses and thus avoiding False to test a condition, assertNull code classes first. side effects. In this case, however, our op- and assertNotNull to test if some- JUnit is available at erations on complex numbers have no thing is a null reference, and assert- The site also contains some documenta- side effects, so we don’t need to override Same and assertNotSame to test if tion, including Beck and Gamma’s orig- tearDown(). Figure 1a shows how we two objects are the same. inal article introducing JUnit. To install might begin by creating a TestCase Next we want to run the tests. The JUnit, you need only unzip it in a direc- subclass and overriding setup(). easiest way is to let JUnit use reflection tory. During compilation, the classpath To create test cases, we add methods to find the defined test cases. Alterna- must include the library’s path, so if you to ComplexTest that contain the asser- tively, we can statically specify the test put the program in /usr/java/junit, tions we want to exercise. Test method cases we want to run, at the cost of the compilation command is javac names start with “test” and must be some more code. To dynamically create –classpath “.:/usr/java/junit/ public so that JUnit can call them using a test suite that contains all the test junit.jar” reflection. We want to test object equal- cases that JUnit found in a class by us- To run the tests, you start the J- ity (our Complex class overrides java. ing reflection, we need only add the Unit graphical user interface with lang.Object’s equals() method), ad- method in figure 1c to ComplexTest. java -classpath “.:/usr/java/ dition, and multiplication. junit/junit.jar” junit.swingui. Figure 1b shows test cases for test- Running tests TestRunner and enter the name of ing equality, complex number addition, To this point, we need not have writ- the class containing the test suite. Then and complex number multiplication. ten any code for the Complex class. In- JUnit performs the tests. If all pass, you July/August 2005 IEEE SOFTWARE 13
  3. 3. OPEN SOURCE that’s more than 100 times the size of its open source competitors. However, the additional features also reflect a different philosophy. JUnit is, at heart, a very sim- ple library. In fact, it would be easy to rewrite it from scratch, if you wished to. It’s designed to do only one thing and do it well. This philosophy, much like the Unix tools’ philosophy, requires devel- opers to add tools to their toolbox for extra functionality. Some notable exam- ples for JUnit include JTestCase (http://, JUnitDoclet (, JUnitPerf (www., MockObjects (, and EasyMock ( Commercial testing frameworks, on the other hand, are comprehensive en- vironments that try to maximize their offerings. Test coverage During the coding and testing process, you must make sure that your tests do not leave important parts of Figure 2. JUnit graphical interface. The green bar indicates that all three the code untested. JUnit helps write ComplexTest cases passed. tests, but selecting a complete test set is up to the individual programmer. get a green bar, as shown in figure 2 tests (for details on Ant, see Nicholas Ser- There are various metrics for test cov- (hence, JUnit’s motto, “Keep the bar rano and Ismael Ciordia’s column in the erage, such as statement coverage (also green to keep the code clean.”) If some- Nov./Dec. 2004 issue). The JUnit site known as line coverage), decision cov- thing goes wrong, the bar turns red and contains links and descriptions for exten- erage (also known as branch coverage), a message appears with details about sions and development environments. path coverage, and others. the test that failed. If you prefer the text- If your favorite cup isn’t Java, other If you’re working on an open source based interface, the command would be programming languages offer ports project, you can obtain Clover (www. java -classpath “.:/usr/java/ and frameworks similar to JUnit. The, a commercial test junit/junit.jar” junit.textui. moniker xUnit refers to the whole fam- coverage tool, under a noncommercial TestRunner ComplexTest, and JUnit ily. CppUnit manages unit testing for license. Another commercial coverage will display the results in the terminal C++, PerlUnit is the Perl port, PyUnit is tool, jcoverage (, window. the standard unit testing framework includes a GPL version, though its func- Now you can add more code to the for Python, and the Test::Unit frame- tionality is limited. GroboUtils (http:// Complex class, perhaps a method for work handles the Ruby community. includes a complex division. At the same time, you NUnit is written in C# and brings open package for test coverage analysis. Emma would add a test case in ComplexTest, source unit test support to the .NET ( is another perform the test, correct any bugs, and platform. easy-to-use tool. Two JUnit-specific cov- go on coding and testing. TestNG ( is erage packages are NoUnit (http:// a recent open source unit testing alter- and JUnit legacy and competition native that has attracted developer at- Jester ( The JUnit’s success has inspired several ex- tention. Parasoft’s Jtest (www.para- GNU compiler suite includes the Gcov tensions that facilitate the testing of spe- is a popular commercial coverage tool ( cific code. These extensions target data- testing tool. Table 1 compares them base code, Enterprise JavaBeans, and with JUnit. Unit tests in the servlets. JUnit is also integrated with sev- The table readily shows that Jtest of- development process eral development environments, and the fers additional features. You would ex- To quote from Glenford Myers’ clas- JUnit Ant task allows Ant to run JUnit pect this from a commercial application sic book on testing (see the “JUnit Re- 14 IEEE SOFTWARE w w w . c o m p u t e r. o r g / s o f t w a r e
  4. 4. OPEN SOURCE Table 1 Comparison of testing frameworks Feature JUnit TestNG Jtest Java Development Kit 1.1 1.4 1.3 Test classes extend framework class or Yes No Yes implement interface Test method discovery Reflection Annotations (J2SE 5.0), Javadoc (J2SE 1.4) Reflection Test setup methods Naming conventions Annotations (J2SE 5.0), Naming Javadoc (J2SE 1.4) conventions Test case selection Program code, XML with JTestCase XML and annotations (J2SE 5.0), Via GUI Javadoc (J2SE 1.4) Test and configuration method parameters Not allowed; workaround with XML and annotations (J2SE 5.0), Via GUI JTestCase extension Javadoc (J2SE 1.4) Dependent methods (methods that are tested No Yes No only if tests succeed) Automatic test case generation Test skeletons with JUnitDoclet extension No Yes Performance testing With JUnitPerf extension No Test time-outs Report generation With JUnitReport Ant task With JUnitReport Ant task Yes Support for generating stub and mock objects With MockObjects or EasyMock libraries With MockObjects or EasyMock libraries Yes Support for design by contract No No Yes Coding standards checking No No Yes Coverage checking See text See text Yes Distribution size (Linux) 432 Kbytes 1.3 Mbytes 120 Mbytes sources” sidebar): “Avoid throwaway JUnit Resources test cases unless the program is truly a throwaway program.” You should de- For an in-depth discussion of JUnit, see Vincent Massol’s book, JUnit in Action, velop test cases in tandem with the code written with Ted Husted, Manning, 2003. they check and keep tests and code in Two books by Andy Hunt and Dave Thomas explain JUnit and NUnit and ex- sync throughout development. This pound good unit testing practices: makes it easy to run a regression test whenever working code changes. ■ Pragmatic Unit Testing in Java with JUnit, Pragmatic Programmers, 2003 Carefully planned test sets also es- ■ Pragmatic Unit Testing in C# with NUnit, Pragmatic Programmers, 2004 tablish very effective project documen- tation. The best way to see how a class Kent Beck has written an introduction to unit testing in the context of the agile de- works is to see it in use, and unit test velopment movement: Test Driven Development: By Example, Addison-Wesley, 2002. cases contain code snippets that show Glenford Myers wrote the classic book on testing in 1979; recently updated, it is just that. now in its second edition, The Art of Software Testing, 2nd ed., John Wiley, 2004. A comprehensive test set doesn’t eat For more details on the various code coverage metrics, read Steve Cornett’s up runtime resources because the tests paper at do not interfere with the program’s source code per se. However, they do eat up compilation resources, so prolif- f course, there is more to testing ing). Nevertheless, adding enough ease erating test cases can slow down the build process. To avoid this, developers can customize the building procedure O than unit testing. We also need to test chunks of code on a bigger scale—say, a group of classes that a and even programming fun to testing the smallest software units builds a firm foundation for further testing. to run a complete system build, com- certain function requires. This is func- prising the test cases, on demand. For tional testing, and it is usually followed instance, they might use Ant to define a by testing the system as a whole (sys- set of test-related tasks that program- tem testing) before delivery to the final Panagiotis Louridas is an adjunct lecturer at the De- partment of Management Science and Technology, Athens Uni- mers can invoke at the end of each day, users, who will test to see whether it versity of Economics and Business, and a software developer for before the code is committed. meets their demands (acceptance test- Emporiki Bank. Contact him at July/August 2005 IEEE SOFTWARE 15