Testing in Java San Gabriel Valley Java Users Group http://www.sgvjug.org/ June 25, 2007 David Noble [email_address] Sr. Solutions Architect, Exist Global
Goal Motivate  excellence  in  testing with  concepts ,  practices,  and  tools
Scope requirements static code analysis debugging profiling continuous integration quality assurance Testing concepts, practices, tools
Agenda Test-Driven Development Terminology Tools Application Discussion
http://www.flickr.com/photos/h2oalchemist/342496829/ System Concept System Requirements System Design Detailed Design Implementation Testing
Demo A Taste of TDD
Test-Driven Development See also: http://www.agiledata.org/essays/tdd.html Fail Pass Pass Add test Make small change Commit changes
Test-Driven Development Fail Pass Add test Fix bug Commit changes
Test-Driven Development Fail Pass Pass Pass Add test Make small change Refactor Code Commit changes Commit changes
Test-Driven Development Small steps Frequent check-ins
Test-Driven Development Deeper than “test first” Design for testability
Test-Driven Development Unit tests as example code Interface before implementation Function drives form
Test-Driven Development Loose coupling Services with clear responsibilities Mock objects
http://flickr.com/photos/basak/227480302/ Taste the soup!
Reality Intent Requirements, Architecture, Design Implementation, Deployment
Gap! Reality Intent Requirements, Architecture, Design Implementation, Deployment
Reality Accountability Intent Requirements, Architecture, Design Testing, QA Implementation, Deployment
Intent Reality Check! Reality Accountability Requirements, Architecture, Design Testing, QA Implementation, Deployment
Terminology QA – quality assurance Do the right things Do things right Ensure quality standards
Terminology Test – success criteria Is it right? Is it done?
Terminology Test plan Test case Test suite Test report Test coverage
Terminology Unit test Functional test Integration test System test Acceptance test Regression test
Terminology Non-functional testing Performance test Load test Soak test
Terminology Continuous integration
Tools IDE Build Unit Tests Mock Objects Eclipse, NetBeans Ant, Maven JUnit, TestNG jMock, EasyMock
Tools Databases JEE Servers Web Apps DBUnit Cactus Selenium, HttpUnit
Tools Coverage Performance Acceptance Cobertura, Emma JMeter, Eclipse TPTP Selenium, FitNesse, Stiq
Tools Continuous Integration Cruise Control, Continuum, Hudson
Tools IDE Build Unit Tests Mock Objects Databases JEE Servers Web Apps Coverage Performance Acceptance Continuous Integration Unit testing Eclipse, NetBeans Ant, Maven JUnit, TestNG jMock, EasyMock DBUnit Cactus Selenium, HttpUnit Cobertura, Emma JMeter, Eclipse TPTP Selenium, FitNesse, Stiq Cruise Control, Continuum, Hudson
Demo Tools
Resources: SGV JUG Web Site http://sgvlug.net/mailman/listinfo/java-sig Mailing List (Hosted by SGV Linux Users Group) http://sgvlug.net/mailman/listinfo/java-sig [email_address]
Resources: TDD Articles http://www.agiledata.org/essays/tdd.html http://en.wikipedia.org/wiki/Test_driven_development http://www.testdriven.com/modules/xoopsfaq/ http://www.artima.com/intv/testdriven.html http://www.google.com/search?hl=en&q=test+driven+development   (really!)
Resources: IDE & Build Tools Eclipse IDE http://www.eclipse.org/ NetBeans IDE http://www.netbeans.org/ Ant http://ant.apache.org/ Maven 2 http://maven.apache.org/ http://www.devzuz.com/web/guest/products/resources#BBWM Maven & Eclipse http://m2eclipse.codehaus.org/ Maven & Netbean http://maven.apache.org/guides/mini/guide-ide-netbeans/guide-ide-netbeans.html
Resources: Unit Testing JUnit Framework http://junit.org/ JUnit FAQ http://junit.sourceforge.net/doc/faq/faq.htm JUnit with Ant http://ant.apache.org/manual/OptionalTasks/junit.html http://ant.apache.org/manual/OptionalTasks/junitreport.html JUnit with Maven http://maven.apache.org/plugins/maven-surefire-plugin/ TestNG Framework http://testng.org/doc/ Jester – Mutation Testing http://jester.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-jester/
Resources: Mock Objects Mock Objects Blog http://mockobjects.com/ EasyMock Library http://easymock.org/ jMock Library http://jmock.org/
Resources: Unit & Functional XmlUnit Library http://xmlunit.sourceforge.net/ DBUnit Library http://dbunit.sourceforge.net/ HttpUnit Library http://httpunit.sourceforge.net/ Cactus for JEE Server Code http://jakarta.apache.org/cactus/ Cargo for JEE Server Code http://cargo.codehaus.org/
Resources: Test Coverage Cobertura http://cobertura.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-cobertura/ Cobertura with Ant http://cobertura.sourceforge.net/anttaskreference.html Cobertura with Maven http://mojo.codehaus.org/cobertura-maven-plugin/usage.html Emma http://emma.sourceforge.net/
Resources: Perf. & Load JUnitPerf http://clarkware.com/software/JUnitPerf.html JMeter http://jakarta.apache.org/jmeter/ Eclipse TPTP (Test & Performance Tools Platform) http://www.eclipse.org/tptp/ Netbeans Profiler http://profiler.netbeans.org/
Resources: Acceptance Tests Selenium for Web Apps http://www.openqa.org/selenium/ http://www.openqa.org/selenium-ide/download.action Automating Selenium (“Remote Control”) http://www.openqa.org/selenium-rc/ http://wiki.openqa.org/display/SRC/Selenium-RC+and+Continuous+Integration Selenium & Maven http://raibledesigns.com/rd/entry/selenium_plugin_for_maven FitNesse http://fitnesse.org/ Stiq http://storytestiq.sourceforge.net/
Resources: Cont. Integration Cruise Control http://cruisecontrol.sourceforge.net/ Continuum http://maven.apache.org/continuum/ Hudson https://hudson.dev.java.net/
Resources: App used in Demo Open Source Medical Record System http://www.openmrs.org/ Prerequisites: SVN, MySQL 5, Tomcat 5 http://subclipse.tigris.org/update_1.2.x http://mysql.org/ http://tomcat.apache.org/ SVN Repository http://svn.openmrs.org/openmrs/branches/alpha Installation Edit build.properties and properties.xml to set tomcat.user/password, or add test/test to $CATALINA_HOME/conf/tomcat-users.xml file. Edit build.properties and/or properties.xml to set tomcat.home file. Comment out creation of "test" MySQL user in schema if it already exists. Default username/password is admin/test.
Thank you To all who participated!

Testing In Java4278

  • 1.
    Testing in JavaSan Gabriel Valley Java Users Group http://www.sgvjug.org/ June 25, 2007 David Noble [email_address] Sr. Solutions Architect, Exist Global
  • 2.
    Goal Motivate excellence in testing with concepts , practices, and tools
  • 3.
    Scope requirements staticcode analysis debugging profiling continuous integration quality assurance Testing concepts, practices, tools
  • 4.
    Agenda Test-Driven DevelopmentTerminology Tools Application Discussion
  • 5.
    http://www.flickr.com/photos/h2oalchemist/342496829/ System ConceptSystem Requirements System Design Detailed Design Implementation Testing
  • 6.
  • 7.
    Test-Driven Development Seealso: http://www.agiledata.org/essays/tdd.html Fail Pass Pass Add test Make small change Commit changes
  • 8.
    Test-Driven Development FailPass Add test Fix bug Commit changes
  • 9.
    Test-Driven Development FailPass Pass Pass Add test Make small change Refactor Code Commit changes Commit changes
  • 10.
    Test-Driven Development Smallsteps Frequent check-ins
  • 11.
    Test-Driven Development Deeperthan “test first” Design for testability
  • 12.
    Test-Driven Development Unittests as example code Interface before implementation Function drives form
  • 13.
    Test-Driven Development Loosecoupling Services with clear responsibilities Mock objects
  • 14.
  • 15.
    Reality Intent Requirements,Architecture, Design Implementation, Deployment
  • 16.
    Gap! Reality IntentRequirements, Architecture, Design Implementation, Deployment
  • 17.
    Reality Accountability IntentRequirements, Architecture, Design Testing, QA Implementation, Deployment
  • 18.
    Intent Reality Check!Reality Accountability Requirements, Architecture, Design Testing, QA Implementation, Deployment
  • 19.
    Terminology QA –quality assurance Do the right things Do things right Ensure quality standards
  • 20.
    Terminology Test –success criteria Is it right? Is it done?
  • 21.
    Terminology Test planTest case Test suite Test report Test coverage
  • 22.
    Terminology Unit testFunctional test Integration test System test Acceptance test Regression test
  • 23.
    Terminology Non-functional testingPerformance test Load test Soak test
  • 24.
  • 25.
    Tools IDE BuildUnit Tests Mock Objects Eclipse, NetBeans Ant, Maven JUnit, TestNG jMock, EasyMock
  • 26.
    Tools Databases JEEServers Web Apps DBUnit Cactus Selenium, HttpUnit
  • 27.
    Tools Coverage PerformanceAcceptance Cobertura, Emma JMeter, Eclipse TPTP Selenium, FitNesse, Stiq
  • 28.
    Tools Continuous IntegrationCruise Control, Continuum, Hudson
  • 29.
    Tools IDE BuildUnit Tests Mock Objects Databases JEE Servers Web Apps Coverage Performance Acceptance Continuous Integration Unit testing Eclipse, NetBeans Ant, Maven JUnit, TestNG jMock, EasyMock DBUnit Cactus Selenium, HttpUnit Cobertura, Emma JMeter, Eclipse TPTP Selenium, FitNesse, Stiq Cruise Control, Continuum, Hudson
  • 30.
  • 31.
    Resources: SGV JUGWeb Site http://sgvlug.net/mailman/listinfo/java-sig Mailing List (Hosted by SGV Linux Users Group) http://sgvlug.net/mailman/listinfo/java-sig [email_address]
  • 32.
    Resources: TDD Articleshttp://www.agiledata.org/essays/tdd.html http://en.wikipedia.org/wiki/Test_driven_development http://www.testdriven.com/modules/xoopsfaq/ http://www.artima.com/intv/testdriven.html http://www.google.com/search?hl=en&q=test+driven+development (really!)
  • 33.
    Resources: IDE &Build Tools Eclipse IDE http://www.eclipse.org/ NetBeans IDE http://www.netbeans.org/ Ant http://ant.apache.org/ Maven 2 http://maven.apache.org/ http://www.devzuz.com/web/guest/products/resources#BBWM Maven & Eclipse http://m2eclipse.codehaus.org/ Maven & Netbean http://maven.apache.org/guides/mini/guide-ide-netbeans/guide-ide-netbeans.html
  • 34.
    Resources: Unit TestingJUnit Framework http://junit.org/ JUnit FAQ http://junit.sourceforge.net/doc/faq/faq.htm JUnit with Ant http://ant.apache.org/manual/OptionalTasks/junit.html http://ant.apache.org/manual/OptionalTasks/junitreport.html JUnit with Maven http://maven.apache.org/plugins/maven-surefire-plugin/ TestNG Framework http://testng.org/doc/ Jester – Mutation Testing http://jester.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-jester/
  • 35.
    Resources: Mock ObjectsMock Objects Blog http://mockobjects.com/ EasyMock Library http://easymock.org/ jMock Library http://jmock.org/
  • 36.
    Resources: Unit &Functional XmlUnit Library http://xmlunit.sourceforge.net/ DBUnit Library http://dbunit.sourceforge.net/ HttpUnit Library http://httpunit.sourceforge.net/ Cactus for JEE Server Code http://jakarta.apache.org/cactus/ Cargo for JEE Server Code http://cargo.codehaus.org/
  • 37.
    Resources: Test CoverageCobertura http://cobertura.sourceforge.net/ http://www.ibm.com/developerworks/java/library/j-cobertura/ Cobertura with Ant http://cobertura.sourceforge.net/anttaskreference.html Cobertura with Maven http://mojo.codehaus.org/cobertura-maven-plugin/usage.html Emma http://emma.sourceforge.net/
  • 38.
    Resources: Perf. &Load JUnitPerf http://clarkware.com/software/JUnitPerf.html JMeter http://jakarta.apache.org/jmeter/ Eclipse TPTP (Test & Performance Tools Platform) http://www.eclipse.org/tptp/ Netbeans Profiler http://profiler.netbeans.org/
  • 39.
    Resources: Acceptance TestsSelenium for Web Apps http://www.openqa.org/selenium/ http://www.openqa.org/selenium-ide/download.action Automating Selenium (“Remote Control”) http://www.openqa.org/selenium-rc/ http://wiki.openqa.org/display/SRC/Selenium-RC+and+Continuous+Integration Selenium & Maven http://raibledesigns.com/rd/entry/selenium_plugin_for_maven FitNesse http://fitnesse.org/ Stiq http://storytestiq.sourceforge.net/
  • 40.
    Resources: Cont. IntegrationCruise Control http://cruisecontrol.sourceforge.net/ Continuum http://maven.apache.org/continuum/ Hudson https://hudson.dev.java.net/
  • 41.
    Resources: App usedin Demo Open Source Medical Record System http://www.openmrs.org/ Prerequisites: SVN, MySQL 5, Tomcat 5 http://subclipse.tigris.org/update_1.2.x http://mysql.org/ http://tomcat.apache.org/ SVN Repository http://svn.openmrs.org/openmrs/branches/alpha Installation Edit build.properties and properties.xml to set tomcat.user/password, or add test/test to $CATALINA_HOME/conf/tomcat-users.xml file. Edit build.properties and/or properties.xml to set tomcat.home file. Comment out creation of "test" MySQL user in schema if it already exists. Default username/password is admin/test.
  • 42.
    Thank you Toall who participated!

Editor's Notes

  • #2 My background: Mostly C & Perl from 1990 – 1997 Mostly Java since then. Telemetry processing, mission ops, EIS @ JPL. BPM, workflow, assorted biz apps.
  • #3 Goal falls within the larger context of pursuing excellence in software development.
  • #6 In traditional software development, testing is usually at the end of the schedule. If development runs late, the testing often suffers.
  • #15 An old man calls the waiter over to his table. "What would you like, sir?" asks the waiter. "Taste the soup," he says. "Is the soup too cold?" asks the waiter. "Taste the soup," says the old man. "Is it too salty?" asks the waiter. "Taste the soup!" says the old man. "Is there a fly in it?" asks the waiter. "Taste the soup!" the old man insists. The waiter looks down: "There's no spoon!" "Aha!" exclaims the old man.
  • #20 Miriam-Webster: “a program for the systematic monitoring and evaluation of the various aspects of a project, service, or facility to ensure that standards of quality are being met”
  • #22 Test plan: document Test case: implementation of specific test. Test suite: collection of test cases. Test report: results of running the tests. Test coverage: description of how much code (and which code) was exercised by tests.
  • #23 Unit test: typically at the granularity of a class. Functional test: a feature or use case. Integration test: multiple modules. System test: all the modules. Acceptance test: defined by user/customer.
  • #24 Non-functional testing: the “-ilities”. Performance test: how fast. Load test: how many. Soak test: how long.
  • #25 This is an alternative to “big bang integration”, otherwise known as “integration hell”.
  • #27 HttpUnit is better suited to situations where you want to examine the DOM at a low level. If you're looking for convenience, and don't need to get deep into the details, look at Selenium instead.
  • #28 Selenium IDE, a Firefox plugin, lets users or product managers create their own scenarios. Selenium is also useful during development, to quickly fast forward through actions to get the web application into a desired state.
  • #30 The items in green aren't really a stack, just a collection of technology-specific libraries that can be useful for creating test cases.
  • #31 Be sure to design Selenium tests so they can be repeatable. Either delete data when they are done, or always start from a fixed known state.