Software Testing - Invited Lecture at UNSW Sydney

Software testing COMP9321 Julien Ponge (julienp@cse.unsw.edu.au) Invited lecture: september 7th 2006 Computer School of Engineering The University of New South Wales, Sydney, Australia

• We will see: • why tests are important • what habits you should get rid of • how to create tests • tips to make your life easier with tests

Introduction 3

Testing... but what? • Unit testing • Functional testing • Conformance testing • Performance testing 4

Unit testing • Typically classes • Isolate as much as possible • Self-contained and automated • “Developers love writing tests” • Metrics: passed tests and code coverage 5

What to expect? • Less stupid bugs + test more often • Prevent regression bugs • Reveal bad design early • Easier deep refactorings 6

The cost of tests • Requires more 4,8 efforts in the early stages 4 3,2 • A code base without tests can 2,4 become too difﬁcult 1,6 to manage as it grows 0,8 0 0,4 0,8 1,2 1,6 2 2,4 2,8 3,2 • Test suites help when scaling! 7

Old-school testing 8

Typical approach • Write a main method (console or GUI) • Perform a few checkings and/or output values (System.out.println(...)) • You may feed data manually and/or interactively • Manually check if it works or not 9

Example 1 import my.package.NiceClass; public class OldSchoolTesting { public static void main(String[] args) { NiceClass nc = new NiceClass(); System.out.println(quot;Hey!quot;); nc.doSomething(); System.out.println(quot;It didn't crash!!! (yet)quot;); System.out.println(nc.getSomeValue()); } } 10

Example 2 public class MyGreatClass { // (...) public void doSomethingGreat() { String str = grabItSomewhereElse(); System.out.println(quot;=> str is quot; + str); if (!str.equals(ONE_CONSTANT)) { cry(); goToBed(); } else { enjoy(); haveOneBeerOrTwo(); } } // (...) } 11

Example 3 12

Problems • Not automated: • you check • you feed the data • you need to run it • System.out.println + messy outputs • Most of the time... you throw it away when it works! 13

To address this... • We will see an automated unit testing framework: JUnit • We will see how to properly output data when need be: Logging 14

JUnit 15

JUnit • Most famous for Java, many extensions • Very stupid set of classes simple • Composite pattern for test cases & test suites • Primarily for unit testing, but functional testing can be conducted as well • Derivatives: NUnit, PyUnit, CppUnit, ... 16

Running JUnit • It reports: • passed & failed tests • errors (exceptions) • It provides a text-mode and a Swing- based runners • Most IDEs embed JUnit and display a progress bar (green / red) 17

How it works • Base class is TestCase • Start testing methods names with test • Override setUp() & tearDown() if initialization and cleanups are needed • Check values with assertEquals, assertNotSame, assertNull, assertNotNull, ... 18

Basic test case canvas • Create a class that extends TestCase, and name it as MyClassTest for MyClass • Create a test method testFoo for each non-trivial method foo • Instantiate data, invoke methods • Use JUnit assertions 19

Example 1 public class DifferenceOperatorTest extends TestCase { private DifferenceOperator operator = new DifferenceOperator(new BusinessProtocolFactoryImpl()); public void testApply() throws DocumentException { BusinessProtocol p1 = TestUtils.loadProtocol(quot;difference/p1.wsprotocolquot;); BusinessProtocol p2 = TestUtils.loadProtocol(quot;difference/p2.wsprotocolquot;); BusinessProtocol result = operator.apply(p2, p1); BusinessProtocol expected = TestUtils.loadProtocol(quot;difference/p2-diff- p1.wsprotocolquot;); TestCase.assertEquals(expected, result); } public void testComputeComplement() throws DocumentException { BusinessProtocol p1 = TestUtils.loadProtocol(quot;difference/p1.wsprotocolquot;); BusinessProtocol expected = TestUtils.loadProtocol(quot;difference/compl-p1.wsprotocolquot;); BusinessProtocol result = operator.computeComplement(p1); TestCase.assertEquals(expected, result); } } 20

Example 2 public class OperationImplTest extends TestCase { public void testEqualsObject() { State s1 = new StateImpl(quot;s1quot;, false); State s2 = new StateImpl(quot;s2quot;, false); State s3 = new StateImpl(quot;s3quot;, false); State s4 = new StateImpl(quot;s4quot;, false); Message m1 = new MessageImpl(quot;aquot;, Polarity.POSITIVE); Message m2 = new MessageImpl(quot;aquot;, Polarity.NEGATIVE); OperationImpl o1 = new OperationImpl(quot;T1quot;, s1, s2, m1); OperationImpl o2 = new OperationImpl(quot;T2quot;, s3, s4, m2); TestCase.assertEquals(o1, o1); TestCase.assertNotSame(o1, o2); } public void testToString() { State s1 = new StateImpl(quot;s1quot;, false); State s2 = new StateImpl(quot;s2quot;, false); Message m1 = new MessageImpl(quot;aquot;, Polarity.POSITIVE); OperationImpl o1 = new OperationImpl(quot;T1quot;, s1, s2, m1); TestCase.assertEquals(quot;T1: ((s1),[a](+),(s2),explicit)quot;, o1.toString()); } } 21

What your tests should do • Test both good and bad cases • Challenge your code: push it with misuses and stupid values • Make sure you get a good coverage: • use dedicated tools • Eclipse since version 3.2 22

Separate tests from the rest of the code! 23

Build integration • Apache Ant is similar to Make, but for Java • There is a Ant task • Apache Maven is project-oriented, and expects JUnit tests • Tests must pass for many Maven goals 24

Ant + JUnit <target name=quot;testsquot; depends=quot;build,build.testsquot;> <junit printsummary=quot;yesquot; fork=quot;yesquot; haltonfailure=quot;yesquot;> <formatter type=quot;plainquot;/> <batchtest fork=quot;yesquot; todir=quot;${reports.tests}quot;> <fileset dir=quot;${src.tests}quot;> <include name=quot;**/*Test*.javaquot;/> <exclude name=quot;**/AllTests.javaquot;/> </fileset> </batchtest> </junit> </target> 25

IDE integration keep the bar green!

Logging 27

Why? • System.out and System.err are limited • Not all events are of the same importance: informations, debugging messages, errors, ... • Activate some messages on-demand, output to console, network, dbs, ﬁles, ... • Useful for server-side applications, but also standalone applications or libraries 28

Logging in Java • Log4J is a popular library • Java 1.4+ has java.util.logging • Apache commons-logging is a popular choice to abstract and dynamically discover the logging API • Very minor impact on performances • Never throw messages, keep them! 29

Logging levels • Fatal : critical errors • Error : errors that can be recovered • Warn : warnings • Info : general informations (start, stop, ...) • Debug : only-useful for debugging • Trace : only for critical debugging 30

Log4J outputs root logger formatters logger.A logger.B logger.B.a ﬁlters 31

Sample Log4J conﬁg log4.properties # Root logger log4j.rootLogger=INFO, consoleAppender # A console appender log4j.appender.consoleAppender=org.apache.log4j.ConsoleAppender log4j.appender.consoleAppender.layout=org.apache.log4j.PatternLayout log4j.appender.consoleAppender.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n 32

Example public class OperationImpl implements Operation { (...) /** Logger. */ private static Log log = LogFactory.getLog(OperationImpl.class); (...) public void setMessage(Message message) { Message oldMessage = this.message; this.message = message; listeners.firePropertyChange(MESSAGE_PROPERTY_CHANGE, oldMessage, message); if (log.isDebugEnabled()) { log.debug(quot;Changing the message from quot; + oldMessage + quot; to quot; + message); } } (...) } 33

Beyond just tests 34

Life of a tests suite • A test suite is like good wine: it gets better over time! • A good example is to reproduce bugs as they get identiﬁed: you will prevent regressions 35

Handling bugs (ah ah I’m back) (I am a bug) Test case Tests suite 36

Tests are also... • Useful to enforce API contracts • An excellent API usage documentation • A solid test suite allows deep refactorings: it acts as a safeguard against regressions • Tests reveal bad design early: it is difﬁcult to test, then you need to refactor! 37

Self-containment 38

Self-containment • Your test suites should be as self- contained as possible • Avoid environment setup requirements (databases servers, application servers, ...) • Eases automation in any context 39

Examples data access relational objects database HTTP HTTP client server 40

Self-contained DB public abstract class DAOTestBase extends TestCase { protected Map<String, String> queries; protected QueryRunner runner; base class protected Connection connection; public DAOTestBase() throws IOException { super(); DbUtils.loadDriver(quot;org.hsqldb.jdbcDriverquot;); runner = new QueryRunner(); queries = QueryLoader.instance().load(DBConstants.INIT_DB_SQL_RESOURCE); } protected void setUp() throws Exception { connection = DriverManager.getConnection(quot;jdbc:hsqldb:mem:CustomerDAOTestquot;, quot;saquot;, quot;quot;); runner.update(connection, queries.get(quot;create.customersquot;)); runner.update(connection, queries.get(quot;create.transactionsquot;)); runner.update(connection, queries.get(quot;data.1quot;)); runner.update(connection, queries.get(quot;data.2quot;)); runner.update(connection, queries.get(quot;data.3quot;)); runner.update(connection, queries.get(quot;data.4quot;)); runner.update(connection, queries.get(quot;data.5quot;)); } protected void tearDown() throws Exception { runner.update(connection, queries.get(quot;drop.transactionsquot;)); runner.update(connection, queries.get(quot;drop.customersquot;)); DbUtils.closeQuietly(connection); } 41 }

Self-contained DB public class CustomerDAOTest extends DAOTestBase { public void testSaveAndUpdate() throws IOException, SQLException { CustomerDAO dao = new CustomerDAO(connection); Customer customer = new Customer(); test class customer.setName(quot;Bernard Minetquot;); customer.setEmail(quot;bernard@les-muscles.comquot;); customer.setAddress(quot;Francequot;); dao.save(customer); Customer readCustomer = dao.loadCustomerByName(quot;Bernard Minetquot;); assertEquals(customer, readCustomer); customer.setName(quot;Framboisierquot;); customer.setEmail(quot;framboisier@les-muscles.comquot;); customer.setAddress(quot;Paris, Francequot;); customer.setUrl(quot;http://blog.les-muscles.com/framboisier/quot;); customer.creditBalance(50.0); dao.update(customer); readCustomer = dao.loadCustomerByName(quot;Bernard Minetquot;); assertNull(readCustomer); readCustomer = dao.loadCustomerByName(quot;Framboisierquot;); assertNotNull(readCustomer); assertEquals(customer, readCustomer); } 42

Self-contained HTTP public abstract class AbstractHttpSoapTransportTest extends TestCase { private Server server; base class private Servlet servlet; private ServletHandler servletHandler; protected AbstractHttpSoapTransportTest(Servlet servlet) { this.servlet = servlet; server = new Server(); SelectChannelConnector connector = new SelectChannelConnector(); connector.setPort(8085); server.setConnectors(new Connector[]{connector}); servletHandler = new ServletHandler(); servletHandler.addServletWithMapping(new ServletHolder(servlet), quot;/dtcquot;); server.setHandler(servletHandler); } protected void setUp() throws Exception { server.start(); } protected void tearDown() throws Exception { server.stop(); } } 43

Self-contained HTTP public class DummyServlet extends HttpServlet { protected void doPost(HttpServletRequest request, HttpServletResponse httpServletResponse) throws ServletException, IOException { InputStream in = request.getInputStream(); OutputStream out = httpServletResponse.getOutputStream(); byte[] buffer = new byte[1024]; servlet for int nbytes = 0; while ((nbytes = in.read(buffer, 0, 1024)) != -1) { out.write(buffer, 0, nbytes); testing } } protected void doGet(HttpServletRequest request, HttpServletResponse httpServletResponse) throws ServletException, IOException { InputStream in = SoapMessageTest.class.getResourceAsStream(quot;simple-soapmessage.xmlquot;); OutputStream out = httpServletResponse.getOutputStream(); byte[] buffer = new byte[1024]; int nbytes = 0; while ((nbytes = in.read(buffer, 0, 1024)) != -1) { out.write(buffer, 0, nbytes); } } } 44

Self-contained HTTP public class HttpSoapTransportWithDummyServletTest extends AbstractHttpSoapTransportTest { private NodeComparator nodeComparator = new NodeComparator(); public HttpSoapTransportWithDummyServletTest() { test class super(new DummyServlet()); } public void testSend() throws DocumentException, IOException, XmlPullParserException, SoapTransportException { HttpSoapTransport soapTransport = new HttpSoapTransport(); SoapMessage message = SoapMessageUtils.readSoapMessageFromStream (SoapMessageTest.class .getResourceAsStream(quot;simple-soap-message.xmlquot;)); SoapMessage response = soapTransport.send(quot;http://localhost:8085/dtcquot;, message, null); assertEquals(0, nodeComparator.compare(message, response)); } } 45

HttpUnit: testing webapps 46

HttpUnit • Functional testing • Works like a web browser / HTTP client (cookies, form submissions, ...) • Embeds a JavaScript engine (Rhino) • JUnit integration • Provides ServletUnit to unit-test servlets in isolation of servlet containers 47

Main classes • WebConversation: stateful client session • WebRequest: a single GET or POST HTTP request • WebResponse: HTTP response • WebForm, WebLink, (...): many helper classes to access the pages content 48

Example public void testGoodLogin() throws Exception { WebConversation conversation = new WebConversation(); WebRequest request = new GetMethodWebRequest( quot;http://www.meterware.com/servlet/TopSecretquot;); WebResponse response = conversation.getResponse(request); WebForm loginForm = response.getForms()[0]; request = loginForm.getRequest(); request.setParameter( quot;namequot;, quot;masterquot; ); response = conversation.getResponse( request ); assertTrue( quot;Login not acceptedquot;, response.getText().indexOf( quot;You made it!quot; ) != -1 ); assertEquals( quot;Page titlequot;, quot;Top Secretquot;, response.getTitle() ); } 49 http://fb2.hu/x10/Articles/HttpUnit.html

(from the cookbook) WebConversation wc = new WebConversation(); WebResponse resp = wc.getResponse( quot;http://www.httpunit.org/doc/cookbook.htmlquot; ); WebLink link = resp.getLinkWith( quot;responsequot; ); link.click(); WebResponse jdoc = wc.getCurrentPage(); WebTable table = resp.getTables()[0]; assertEquals( quot;rowsquot;, 4, table.getRowCount() ); assertEquals( quot;columnsquot;, 3, table.getColumnCount() ); assertEquals( quot;linksquot;, 1, table.getTableCell( 0, 2 ).getLinks().length ); WebForm form = resp.getForms()[0]; assertEquals( quot;La Cerentollaquot;, form.getParameterValue( quot;Namequot; ) ); assertEquals( quot;Chinesequot;, form.getParameterValue( quot;Foodquot; ) ); assertEquals( quot;Manayunkquot;, form.getParameterValue( quot;Locationquot; ) ); assertEquals( quot;onquot;, form.getParameterValue( quot;CreditCardquot; ) ); form.setParameter( quot;Foodquot;, quot;Italianquot; ); form.removeParameter( quot;CreditCardquot; ); form.submit(); 50

ServletUnit example ServletRunner sr = new ServletRunner(); sr.registerServlet( quot;myServletquot;, StatefulServlet.class.getName() ); ServletUnitClient sc = sr.newClient(); WebRequest request = new PostMethodWebRequest(quot;http://test.meterware.com/myServletquot;); request.setParameter( quot;colorquot;, quot;redquot; ); WebResponse response = sc.getResponse( request ); assertNotNull( quot;No response receivedquot;, response ); assertEquals( quot;content typequot;, quot;text/plainquot;, response.getContentType() ); assertEquals( quot;requested resourcequot;, quot;You selected redquot;, response.getText() ); public class StatefulServlet extends HttpServlet { protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException,IOException { resp.setContentType( quot;text/plainquot; ); writeSelectMessage( req.getParameter( quot;colorquot; ), resp.getWriter() ); setColor( req, req.getParameter( quot;colorquot; ) ); } protected void writeSelectMessage(String color, PrintWriter pw) throws IOException { pw.print( quot;You selected quot; + color ); pw.close(); } void setColor( HttpServletRequest req, String color ) throws ServletException { req.getSession().setAttribute( quot;colorquot;, color ); } 51 }

Tests-friendly patterns 52

Problem: dependencies AdressBook PersonDAO + loadPerson(...) + savePerson(...) 53

Hard-dependencies • Isolating the class to test is difﬁcult and/or impossible • Failures may be induced by a dependency • More generally, it reveals potential code evolution problems: strong coupling 54

Depending on interfaces <<interface>> AdressBook IPersonDAO + loadPerson(...) + savePerson(...) DBPersonDAO FilePersonDAO MockPersonDAO 55

Interfaces dependencies • Interfaces introduce loose coupling • At execution time: use the real class • At test time: use a mock class • Future evolution is easier • BTW, don’t over-use interfaces! 56

Mock objects • Give a simple behavior: ﬁxed values, random values, values passed by methods of the mock class, ... • They allow you to test even if the real class hasn’t been created yet • EasyMock, JMockIt allow to generate mocks on-the-ﬂy 57

Object.equals() public boolean equals(Object obj) { if (obj instanceof ComparisonNode) { ComparisonNode other = (ComparisonNode) obj; return symbol.equals(other.symbol) && leftChild.equals(other.leftChild) && rightChild.equals(other.rightChild); } return false; important for } TestCase.assertXXX(...) 58

Dependency injection • 2 types of injection: setter-based & constructor-based • Constructor injection is preferable, and make your objects ready right after their instanciation • Clariﬁes classes dependencies • Principle used in inversion of control frameworks: PicoContainer, Spring, ... 59

Example private IConstraintNode leftChild; private IConstraintNode rightChild; private String symbol; public ComparisonNode(String symbol, VariableNode var, ConstantNode cst) { this(var, cst, symbol); } public ComparisonNode(String symbol, ConstantNode cst, VariableNode var) { this(cst, var, symbol); } protected ComparisonNode(IConstraintNode leftChild, IConstraintNode rightChild, String symbol) { super(); this.symbol = symbol; this.leftChild = leftChild; this.rightChild = rightChild; } 60

A few tips 61

Do • Try to really write your tests ﬁrst • Test as soon as you create a new class • Test often • Start with sensible simple scenarios • Load data from resources (Class.getResourceAsStream(“/path/to”)) • Check for code coverage 62

Don’t • Write overly exhaustive test cases • Write tests for trivial methods (getters, setters, ...) even if it lowers coverage • Catch exceptions, unless you actually test that • Fix bugs without ﬁrst reproducing them in a test case 63

Traversing object graphs • Sometimes you will need to walk through deep object graphs to test values • This is really boring • JXPath uses XPath on objects, and replaces iterations/loops by requests 64

JXPath example Iterator it; String str; JXPathContext ctx = JXPathContext.newContext(bp2); TestCase.assertEquals(quot;s0quot;, bp2.getInitialState().getName()); it = ctx.iterate(quot;finalStates/namequot;); TestCase.assertTrue(it.hasNext()); TestCase.assertEquals(quot;s1quot;, (String) it.next()); TestCase.assertFalse(it.hasNext()); it = ctx.iterate(quot;states[name='s0']/successors/namequot;); TestCase.assertTrue(it.hasNext()); TestCase.assertEquals(quot;s1quot;, (String) it.next()); TestCase.assertEquals(quot;s0quot;, (String) it.next()); TestCase.assertFalse(it.hasNext()); (...) Operation toRemove = (Operation) ctx.getValue(quot;operations[message/name='a']quot;); bp2.removeOperation(toRemove); List remainingOps = (List) ctx.getValue(quot;states[name='s0']/incomingOperationsquot;); TestCase.assertTrue(remainingOps.size() == 1); 65

The case of ‘old’ software • Test cases can be added to existing applications • In many situations the design can make it difﬁcult to test / isolate... • You cannot break existing APIs • Don’t expect to introduce tests cases for 100% of the code base... • It will still allow you to spot new bugs! 66

Continuous integration 67

Problem • Developers work on different portions of the code base • Traditional integration may not be frequent enough (~ weekly) • It can be difﬁcult to ﬁnd who, when and how the build was broken • Continuous integration aims at detecting problems in nearly real-time 68

C.I. environment 4 email developers instant messaging 1 continuous integration changeset (Continuum, ...) 3 execute notification via hooks 2 source code tests suite repository 69 (SVN, CVS, ...)

• Demo: • project using Maven2 • Apache Continuum conﬁguration • launch the tests from Continuum 70

Thanks! 71

http://jpz-log.info	65
http://www.slideshare.net	3
http://www.techgig.com	1

Software Testing - Invited Lecture at UNSW Sydney

by julien.ponge

Accessibility

Categories

Upload Details

Usage Rights

3 Embeds 69

Statistics

Software Testing - Invited Lecture at UNSW Sydney Presentation Transcript