Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Software Testing - Invited Lecture at UNSW Sydney

2,380 views

Published on

Published in: Education, Technology
  • I have done a couple of papers through ⇒⇒⇒WRITE-MY-PAPER.net ⇐⇐⇐ they have always been great! They are always in touch with you to let you know the status of paper and always meet the deadline!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1lite.top/9jLjD ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • You can ask here for a help. They helped me a lot an i`m highly satisfied with quality of work done. I can promise you 100% un-plagiarized text and good experts there. Use with pleasure! ⇒ www.HelpWriting.net ⇐
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • D0WNL0AD FULL ▶ ▶ ▶ ▶ http://1lite.top/9jLjD ◀ ◀ ◀ ◀
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Your GCSE Maths program has helped me immensely in maths. I am much more confident with this subject and I'm striving for better grades. I really appreciate the time you took in making this program because it has boosted many students self-confidence with their exams. ▲▲▲ https://bit.ly/33W8jmf
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Software Testing - Invited Lecture at UNSW Sydney

  1. 1. Software testing COMP9321 Julien Ponge (julienp@cse.unsw.edu.au) Invited lecture: september 7th 2006 Computer School of Engineering The University of New South Wales, Sydney, Australia
  2. 2. • We will see: • why tests are important • what habits you should get rid of • how to create tests • tips to make your life easier with tests
  3. 3. Introduction 3
  4. 4. Testing... but what? • Unit testing • Functional testing • Conformance testing • Performance testing 4
  5. 5. Unit testing • Typically classes • Isolate as much as possible • Self-contained and automated • “Developers love writing tests” • Metrics: passed tests and code coverage 5
  6. 6. What to expect? • Less stupid bugs + test more often • Prevent regression bugs • Reveal bad design early • Easier deep refactorings 6
  7. 7. The cost of tests • Requires more 4,8 efforts in the early stages 4 3,2 • A code base without tests can 2,4 become too difficult 1,6 to manage as it grows 0,8 0 0,4 0,8 1,2 1,6 2 2,4 2,8 3,2 • Test suites help when scaling! 7
  8. 8. Old-school testing 8
  9. 9. Typical approach • Write a main method (console or GUI) • Perform a few checkings and/or output values (System.out.println(...)) • You may feed data manually and/or interactively • Manually check if it works or not 9
  10. 10. Example 1 import my.package.NiceClass; public class OldSchoolTesting { public static void main(String[] args) { NiceClass nc = new NiceClass(); System.out.println(quot;Hey!quot;); nc.doSomething(); System.out.println(quot;It didn't crash!!! (yet)quot;); System.out.println(nc.getSomeValue()); } } 10
  11. 11. Example 2 public class MyGreatClass { // (...) public void doSomethingGreat() { String str = grabItSomewhereElse(); System.out.println(quot;=> str is quot; + str); if (!str.equals(ONE_CONSTANT)) { cry(); goToBed(); } else { enjoy(); haveOneBeerOrTwo(); } } // (...) } 11
  12. 12. Example 3 12
  13. 13. Problems • Not automated: • you check • you feed the data • you need to run it • System.out.println + messy outputs • Most of the time... you throw it away when it works! 13
  14. 14. To address this... • We will see an automated unit testing framework: JUnit • We will see how to properly output data when need be: Logging 14
  15. 15. JUnit 15
  16. 16. JUnit • Most famous for Java, many extensions • Very stupid set of classes simple • Composite pattern for test cases & test suites • Primarily for unit testing, but functional testing can be conducted as well • Derivatives: NUnit, PyUnit, CppUnit, ... 16
  17. 17. Running JUnit • It reports: • passed & failed tests • errors (exceptions) • It provides a text-mode and a Swing- based runners • Most IDEs embed JUnit and display a progress bar (green / red) 17
  18. 18. How it works • Base class is TestCase • Start testing methods names with test • Override setUp() & tearDown() if initialization and cleanups are needed • Check values with assertEquals, assertNotSame, assertNull, assertNotNull, ... 18
  19. 19. Basic test case canvas • Create a class that extends TestCase, and name it as MyClassTest for MyClass • Create a test method testFoo for each non-trivial method foo • Instantiate data, invoke methods • Use JUnit assertions 19
  20. 20. Example 1 public class DifferenceOperatorTest extends TestCase { private DifferenceOperator operator = new DifferenceOperator(new BusinessProtocolFactoryImpl()); public void testApply() throws DocumentException { BusinessProtocol p1 = TestUtils.loadProtocol(quot;difference/p1.wsprotocolquot;); BusinessProtocol p2 = TestUtils.loadProtocol(quot;difference/p2.wsprotocolquot;); BusinessProtocol result = operator.apply(p2, p1); BusinessProtocol expected = TestUtils.loadProtocol(quot;difference/p2-diff- p1.wsprotocolquot;); TestCase.assertEquals(expected, result); } public void testComputeComplement() throws DocumentException { BusinessProtocol p1 = TestUtils.loadProtocol(quot;difference/p1.wsprotocolquot;); BusinessProtocol expected = TestUtils.loadProtocol(quot;difference/compl-p1.wsprotocolquot;); BusinessProtocol result = operator.computeComplement(p1); TestCase.assertEquals(expected, result); } } 20
  21. 21. Example 2 public class OperationImplTest extends TestCase { public void testEqualsObject() { State s1 = new StateImpl(quot;s1quot;, false); State s2 = new StateImpl(quot;s2quot;, false); State s3 = new StateImpl(quot;s3quot;, false); State s4 = new StateImpl(quot;s4quot;, false); Message m1 = new MessageImpl(quot;aquot;, Polarity.POSITIVE); Message m2 = new MessageImpl(quot;aquot;, Polarity.NEGATIVE); OperationImpl o1 = new OperationImpl(quot;T1quot;, s1, s2, m1); OperationImpl o2 = new OperationImpl(quot;T2quot;, s3, s4, m2); TestCase.assertEquals(o1, o1); TestCase.assertNotSame(o1, o2); } public void testToString() { State s1 = new StateImpl(quot;s1quot;, false); State s2 = new StateImpl(quot;s2quot;, false); Message m1 = new MessageImpl(quot;aquot;, Polarity.POSITIVE); OperationImpl o1 = new OperationImpl(quot;T1quot;, s1, s2, m1); TestCase.assertEquals(quot;T1: ((s1),[a](+),(s2),explicit)quot;, o1.toString()); } } 21
  22. 22. What your tests should do • Test both good and bad cases • Challenge your code: push it with misuses and stupid values • Make sure you get a good coverage: • use dedicated tools • Eclipse since version 3.2 22
  23. 23. Separate tests from the rest of the code! 23
  24. 24. Build integration • Apache Ant is similar to Make, but for Java • There is a Ant task • Apache Maven is project-oriented, and expects JUnit tests • Tests must pass for many Maven goals 24
  25. 25. Ant + JUnit <target name=quot;testsquot; depends=quot;build,build.testsquot;> <junit printsummary=quot;yesquot; fork=quot;yesquot; haltonfailure=quot;yesquot;> <formatter type=quot;plainquot;/> <batchtest fork=quot;yesquot; todir=quot;${reports.tests}quot;> <fileset dir=quot;${src.tests}quot;> <include name=quot;**/*Test*.javaquot;/> <exclude name=quot;**/AllTests.javaquot;/> </fileset> </batchtest> </junit> </target> 25
  26. 26. IDE integration keep the bar green!
  27. 27. Logging 27
  28. 28. Why? • System.out and System.err are limited • Not all events are of the same importance: informations, debugging messages, errors, ... • Activate some messages on-demand, output to console, network, dbs, files, ... • Useful for server-side applications, but also standalone applications or libraries 28
  29. 29. Logging in Java • Log4J is a popular library • Java 1.4+ has java.util.logging • Apache commons-logging is a popular choice to abstract and dynamically discover the logging API • Very minor impact on performances • Never throw messages, keep them! 29
  30. 30. Logging levels • Fatal : critical errors • Error : errors that can be recovered • Warn : warnings • Info : general informations (start, stop, ...) • Debug : only-useful for debugging • Trace : only for critical debugging 30
  31. 31. Log4J outputs root logger formatters logger.A logger.B logger.B.a filters 31
  32. 32. Sample Log4J config log4.properties # Root logger log4j.rootLogger=INFO, consoleAppender # A console appender log4j.appender.consoleAppender=org.apache.log4j.ConsoleAppender log4j.appender.consoleAppender.layout=org.apache.log4j.PatternLayout log4j.appender.consoleAppender.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n 32
  33. 33. Example public class OperationImpl implements Operation { (...) /** Logger. */ private static Log log = LogFactory.getLog(OperationImpl.class); (...) public void setMessage(Message message) { Message oldMessage = this.message; this.message = message; listeners.firePropertyChange(MESSAGE_PROPERTY_CHANGE, oldMessage, message); if (log.isDebugEnabled()) { log.debug(quot;Changing the message from quot; + oldMessage + quot; to quot; + message); } } (...) } 33
  34. 34. Beyond just tests 34
  35. 35. Life of a tests suite • A test suite is like good wine: it gets better over time! • A good example is to reproduce bugs as they get identified: you will prevent regressions 35
  36. 36. Handling bugs (ah ah I’m back) (I am a bug) Test case Tests suite 36
  37. 37. Tests are also... • Useful to enforce API contracts • An excellent API usage documentation • A solid test suite allows deep refactorings: it acts as a safeguard against regressions • Tests reveal bad design early: it is difficult to test, then you need to refactor! 37
  38. 38. Self-containment 38
  39. 39. Self-containment • Your test suites should be as self- contained as possible • Avoid environment setup requirements (databases servers, application servers, ...) • Eases automation in any context 39
  40. 40. Examples data access relational objects database HTTP HTTP client server 40
  41. 41. Self-contained DB public abstract class DAOTestBase extends TestCase { protected Map<String, String> queries; protected QueryRunner runner; base class protected Connection connection; public DAOTestBase() throws IOException { super(); DbUtils.loadDriver(quot;org.hsqldb.jdbcDriverquot;); runner = new QueryRunner(); queries = QueryLoader.instance().load(DBConstants.INIT_DB_SQL_RESOURCE); } protected void setUp() throws Exception { connection = DriverManager.getConnection(quot;jdbc:hsqldb:mem:CustomerDAOTestquot;, quot;saquot;, quot;quot;); runner.update(connection, queries.get(quot;create.customersquot;)); runner.update(connection, queries.get(quot;create.transactionsquot;)); runner.update(connection, queries.get(quot;data.1quot;)); runner.update(connection, queries.get(quot;data.2quot;)); runner.update(connection, queries.get(quot;data.3quot;)); runner.update(connection, queries.get(quot;data.4quot;)); runner.update(connection, queries.get(quot;data.5quot;)); } protected void tearDown() throws Exception { runner.update(connection, queries.get(quot;drop.transactionsquot;)); runner.update(connection, queries.get(quot;drop.customersquot;)); DbUtils.closeQuietly(connection); } 41 }
  42. 42. Self-contained DB public class CustomerDAOTest extends DAOTestBase { public void testSaveAndUpdate() throws IOException, SQLException { CustomerDAO dao = new CustomerDAO(connection); Customer customer = new Customer(); test class customer.setName(quot;Bernard Minetquot;); customer.setEmail(quot;bernard@les-muscles.comquot;); customer.setAddress(quot;Francequot;); dao.save(customer); Customer readCustomer = dao.loadCustomerByName(quot;Bernard Minetquot;); assertEquals(customer, readCustomer); customer.setName(quot;Framboisierquot;); customer.setEmail(quot;framboisier@les-muscles.comquot;); customer.setAddress(quot;Paris, Francequot;); customer.setUrl(quot;http://blog.les-muscles.com/framboisier/quot;); customer.creditBalance(50.0); dao.update(customer); readCustomer = dao.loadCustomerByName(quot;Bernard Minetquot;); assertNull(readCustomer); readCustomer = dao.loadCustomerByName(quot;Framboisierquot;); assertNotNull(readCustomer); assertEquals(customer, readCustomer); } 42
  43. 43. Self-contained HTTP public abstract class AbstractHttpSoapTransportTest extends TestCase { private Server server; base class private Servlet servlet; private ServletHandler servletHandler; protected AbstractHttpSoapTransportTest(Servlet servlet) { this.servlet = servlet; server = new Server(); SelectChannelConnector connector = new SelectChannelConnector(); connector.setPort(8085); server.setConnectors(new Connector[]{connector}); servletHandler = new ServletHandler(); servletHandler.addServletWithMapping(new ServletHolder(servlet), quot;/dtcquot;); server.setHandler(servletHandler); } protected void setUp() throws Exception { server.start(); } protected void tearDown() throws Exception { server.stop(); } } 43
  44. 44. Self-contained HTTP public class DummyServlet extends HttpServlet { protected void doPost(HttpServletRequest request, HttpServletResponse httpServletResponse) throws ServletException, IOException { InputStream in = request.getInputStream(); OutputStream out = httpServletResponse.getOutputStream(); byte[] buffer = new byte[1024]; servlet for int nbytes = 0; while ((nbytes = in.read(buffer, 0, 1024)) != -1) { out.write(buffer, 0, nbytes); testing } } protected void doGet(HttpServletRequest request, HttpServletResponse httpServletResponse) throws ServletException, IOException { InputStream in = SoapMessageTest.class.getResourceAsStream(quot;simple-soap- message.xmlquot;); OutputStream out = httpServletResponse.getOutputStream(); byte[] buffer = new byte[1024]; int nbytes = 0; while ((nbytes = in.read(buffer, 0, 1024)) != -1) { out.write(buffer, 0, nbytes); } } } 44
  45. 45. Self-contained HTTP public class HttpSoapTransportWithDummyServletTest extends AbstractHttpSoapTransportTest { private NodeComparator nodeComparator = new NodeComparator(); public HttpSoapTransportWithDummyServletTest() { test class super(new DummyServlet()); } public void testSend() throws DocumentException, IOException, XmlPullParserException, SoapTransportException { HttpSoapTransport soapTransport = new HttpSoapTransport(); SoapMessage message = SoapMessageUtils.readSoapMessageFromStream (SoapMessageTest.class .getResourceAsStream(quot;simple-soap-message.xmlquot;)); SoapMessage response = soapTransport.send(quot;http://localhost:8085/dtcquot;, message, null); assertEquals(0, nodeComparator.compare(message, response)); } } 45
  46. 46. HttpUnit: testing webapps 46
  47. 47. HttpUnit • Functional testing • Works like a web browser / HTTP client (cookies, form submissions, ...) • Embeds a JavaScript engine (Rhino) • JUnit integration • Provides ServletUnit to unit-test servlets in isolation of servlet containers 47
  48. 48. Main classes • WebConversation: stateful client session • WebRequest: a single GET or POST HTTP request • WebResponse: HTTP response • WebForm, WebLink, (...): many helper classes to access the pages content 48
  49. 49. Example public void testGoodLogin() throws Exception { WebConversation conversation = new WebConversation(); WebRequest request = new GetMethodWebRequest( quot;http://www.meterware.com/servlet/TopSecretquot;); WebResponse response = conversation.getResponse(request); WebForm loginForm = response.getForms()[0]; request = loginForm.getRequest(); request.setParameter( quot;namequot;, quot;masterquot; ); response = conversation.getResponse( request ); assertTrue( quot;Login not acceptedquot;, response.getText().indexOf( quot;You made it!quot; ) != -1 ); assertEquals( quot;Page titlequot;, quot;Top Secretquot;, response.getTitle() ); } 49 http://fb2.hu/x10/Articles/HttpUnit.html
  50. 50. (from the cookbook) WebConversation wc = new WebConversation(); WebResponse resp = wc.getResponse( quot;http://www.httpunit.org/doc/cookbook.htmlquot; ); WebLink link = resp.getLinkWith( quot;responsequot; ); link.click(); WebResponse jdoc = wc.getCurrentPage(); WebTable table = resp.getTables()[0]; assertEquals( quot;rowsquot;, 4, table.getRowCount() ); assertEquals( quot;columnsquot;, 3, table.getColumnCount() ); assertEquals( quot;linksquot;, 1, table.getTableCell( 0, 2 ).getLinks().length ); WebForm form = resp.getForms()[0]; assertEquals( quot;La Cerentollaquot;, form.getParameterValue( quot;Namequot; ) ); assertEquals( quot;Chinesequot;, form.getParameterValue( quot;Foodquot; ) ); assertEquals( quot;Manayunkquot;, form.getParameterValue( quot;Locationquot; ) ); assertEquals( quot;onquot;, form.getParameterValue( quot;CreditCardquot; ) ); form.setParameter( quot;Foodquot;, quot;Italianquot; ); form.removeParameter( quot;CreditCardquot; ); form.submit(); 50
  51. 51. ServletUnit example ServletRunner sr = new ServletRunner(); sr.registerServlet( quot;myServletquot;, StatefulServlet.class.getName() ); ServletUnitClient sc = sr.newClient(); WebRequest request = new PostMethodWebRequest(quot;http://test.meterware.com/myServletquot;); request.setParameter( quot;colorquot;, quot;redquot; ); WebResponse response = sc.getResponse( request ); assertNotNull( quot;No response receivedquot;, response ); assertEquals( quot;content typequot;, quot;text/plainquot;, response.getContentType() ); assertEquals( quot;requested resourcequot;, quot;You selected redquot;, response.getText() ); public class StatefulServlet extends HttpServlet { protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException,IOException { resp.setContentType( quot;text/plainquot; ); writeSelectMessage( req.getParameter( quot;colorquot; ), resp.getWriter() ); setColor( req, req.getParameter( quot;colorquot; ) ); } protected void writeSelectMessage(String color, PrintWriter pw) throws IOException { pw.print( quot;You selected quot; + color ); pw.close(); } void setColor( HttpServletRequest req, String color ) throws ServletException { req.getSession().setAttribute( quot;colorquot;, color ); } 51 }
  52. 52. Tests-friendly patterns 52
  53. 53. Problem: dependencies AdressBook PersonDAO + loadPerson(...) + savePerson(...) 53
  54. 54. Hard-dependencies • Isolating the class to test is difficult and/or impossible • Failures may be induced by a dependency • More generally, it reveals potential code evolution problems: strong coupling 54
  55. 55. Depending on interfaces <<interface>> AdressBook IPersonDAO + loadPerson(...) + savePerson(...) DBPersonDAO FilePersonDAO MockPersonDAO 55
  56. 56. Interfaces dependencies • Interfaces introduce loose coupling • At execution time: use the real class • At test time: use a mock class • Future evolution is easier • BTW, don’t over-use interfaces! 56
  57. 57. Mock objects • Give a simple behavior: fixed values, random values, values passed by methods of the mock class, ... • They allow you to test even if the real class hasn’t been created yet • EasyMock, JMockIt allow to generate mocks on-the-fly 57
  58. 58. Object.equals() public boolean equals(Object obj) { if (obj instanceof ComparisonNode) { ComparisonNode other = (ComparisonNode) obj; return symbol.equals(other.symbol) && leftChild.equals(other.leftChild) && rightChild.equals(other.rightChild); } return false; important for } TestCase.assertXXX(...) 58
  59. 59. Dependency injection • 2 types of injection: setter-based & constructor-based • Constructor injection is preferable, and make your objects ready right after their instanciation • Clarifies classes dependencies • Principle used in inversion of control frameworks: PicoContainer, Spring, ... 59
  60. 60. Example private IConstraintNode leftChild; private IConstraintNode rightChild; private String symbol; public ComparisonNode(String symbol, VariableNode var, ConstantNode cst) { this(var, cst, symbol); } public ComparisonNode(String symbol, ConstantNode cst, VariableNode var) { this(cst, var, symbol); } protected ComparisonNode(IConstraintNode leftChild, IConstraintNode rightChild, String symbol) { super(); this.symbol = symbol; this.leftChild = leftChild; this.rightChild = rightChild; } 60
  61. 61. A few tips 61
  62. 62. Do • Try to really write your tests first • Test as soon as you create a new class • Test often • Start with sensible simple scenarios • Load data from resources (Class.getResourceAsStream(“/path/to”)) • Check for code coverage 62
  63. 63. Don’t • Write overly exhaustive test cases • Write tests for trivial methods (getters, setters, ...) even if it lowers coverage • Catch exceptions, unless you actually test that • Fix bugs without first reproducing them in a test case 63
  64. 64. Traversing object graphs • Sometimes you will need to walk through deep object graphs to test values • This is really boring • JXPath uses XPath on objects, and replaces iterations/loops by requests 64
  65. 65. JXPath example Iterator it; String str; JXPathContext ctx = JXPathContext.newContext(bp2); TestCase.assertEquals(quot;s0quot;, bp2.getInitialState().getName()); it = ctx.iterate(quot;finalStates/namequot;); TestCase.assertTrue(it.hasNext()); TestCase.assertEquals(quot;s1quot;, (String) it.next()); TestCase.assertFalse(it.hasNext()); it = ctx.iterate(quot;states[name='s0']/successors/namequot;); TestCase.assertTrue(it.hasNext()); TestCase.assertEquals(quot;s1quot;, (String) it.next()); TestCase.assertEquals(quot;s0quot;, (String) it.next()); TestCase.assertFalse(it.hasNext()); (...) Operation toRemove = (Operation) ctx.getValue(quot;operations[message/name='a']quot;); bp2.removeOperation(toRemove); List remainingOps = (List) ctx.getValue(quot;states[name='s0']/incomingOperationsquot;); TestCase.assertTrue(remainingOps.size() == 1); 65
  66. 66. The case of ‘old’ software • Test cases can be added to existing applications • In many situations the design can make it difficult to test / isolate... • You cannot break existing APIs • Don’t expect to introduce tests cases for 100% of the code base... • It will still allow you to spot new bugs! 66
  67. 67. Continuous integration 67
  68. 68. Problem • Developers work on different portions of the code base • Traditional integration may not be frequent enough (~ weekly) • It can be difficult to find who, when and how the build was broken • Continuous integration aims at detecting problems in nearly real-time 68
  69. 69. C.I. environment 4 email developers instant messaging 1 continuous integration changeset (Continuum, ...) 3 execute notification via hooks 2 source code tests suite repository 69 (SVN, CVS, ...)
  70. 70. • Demo: • project using Maven2 • Apache Continuum configuration • launch the tests from Continuum 70
  71. 71. Thanks! 71

×