Slideshow Transcript

1. Slide 1: Testing Tools and Strategies Scott Ryan March 2007
2. Slide 2: Agenda • Business Drivers • Unit Testing • Coverage • Code Quality Management (CQM) • Integration Testing • Load Testing • Test Data • Automated Test Generation • Demo
3. Slide 3: Business Drivers • Errors found early in the development lifecycle cost less to fix • Cost varies by Industry and Software usage • Cost is not linear • Testing improves quality and reduces cost • Testing costs money too and there is a tradeoff
4. Slide 4: Business Drivers • Cost of running integration and system tests again. • Cost of recertification (if you're in that kind of environment). • Cost of deploying the software again. • Support costs when only half your customers deploy the new version. • Indirect costs caused by using resources to fix bugs rather than implement revenue-generating features. • Liability for damages caused by the bug.
5. Slide 5: Testing Levels • Unit Testing • Integration Testing • System Testing
6. Slide 6: Test Design • Black Box • White Box
7. Slide 7: Testing Techniques • Manual Testing • Automated Testing • Regression Testing • Environment Testing • Stress Testing • Load Testing • Performance Testing
8. Slide 8: Unit Testing Frameworks • Junit o HttpUnit o StrutsUnit o XMLUnit o DBUnit • TestNG
9. Slide 9: JUnit • Version 3 and 4 • Extend TestCase Class • Validate through Asssertions • Setup and TearDown Methods • Testing Suites • Lots of Automation
10. Slide 10: Test NG • Java 5 (Primarily) • Parameter Support • No need to extend base • Supports Test class Distribution • Good use of Java 5 Annotations • New Execution Model • DataDriven Tests o No Suites • Embedded Beanshell o Groups • Embedded Logging, JDK o Relationships • Application Server Testing support
11. Slide 11: TestNG Annotations • @Test • @BeforeMethod • @Parameters • @AfterMethod • @DataProvider • @BeforeClass • @BeforeSuite • @AfterClass • @AfterSuite • @BeforeGroup • @BeforeTest • @AfterGroup • @AfterTest
12. Slide 12: Unit Testing Tips • Keep tests in same package as tested code • Separate tests for packaging • Start small and work up • Give useful names and assertions • Run often and review the results • Build tests around bugs you are fixing
13. Slide 13: Coverage • Cobertura • Djunit • Clover • Hansel • Emma • Gretel • Jtest (Parasoft) • Jester • GroboUtils
14. Slide 14: Coverage Types • Statement Coverage • Branch Coverage (if and implied else) • Condition Coverage • Coverage Percentage
15. Slide 15: Coverage Hints and Tips • Use at the beginning of test development • Strive for branch over line coverage • Build tests to increase coverage • Hard to tell what is important so the developer is key • Review reports often as code base grows
16. Slide 16: Code Quality Management • Used to raise the quality of code base • Range from trivial to high impact • All improvements are important • Use a wide variation of tools and tune around your team’s best practice • Use to drive refactoring
17. Slide 17: CQM Tools • Checkstyle • Jalopy • Findbugs • Eclipse • PMD • NetBeans • CPD • JavaDoc • Jtest • JDepend
18. Slide 18: CQM Hints and Tips • Automate and run often • Make incremental improvements over time • Quality is important • Use tools to improve quality at the source • Use as a guide to refactoring
19. Slide 19: Integration Testing • Tests the assembled deployed unit • Usually requires automation • Developed after Unit tests • Least used technique but most effective • Fairly complex with many moving parts • Require mature development process
20. Slide 20: Integration Testing Frameworks • Selenium • Dumbster (SMTP) • Cactus • WinRunner (HP) • Canoo • SilkTest (Borland) • JWebUnit • HttpUnit • TeamTest (Rational) • Jmeter • Cargo • Eclipse • Maven
21. Slide 21: Integration Testing Hints and Tips • Start small • Automate as soon as practical • Strive for breadth over depth • Work with QA and UAT groups • Tie tests to use cases • Use as smoke test to insure proper deployments • Understand data needs and system impact
22. Slide 22: Test Data Hints and Tips • Understand the data needs of your tests • Start the system in a known state • Return system to known state when done • Watch out for protected data • Need to understand the impact of your tests • Keep data current • Understand system impact (i.e. database keys) • Leverage Automation
23. Slide 23: Test Data Tools • Setup and teardown methods • Database Scripts • DBUnit • XMLUnit • DDSteps • JTestcase
24. Slide 24: Mock Testing • Objects act as mediators • Simulates complex or unfinished systems • Allows access to “closed” systems • Quicker Testing • Mocks are not stubs they are dynamic • Mocks offer behavior verification • Stubs offer state verification
25. Slide 25: Mock Toolsets • Easy Mock • Spring Mock • JMock • Jmockit • MockRunner
26. Slide 26: Automated Test Generation • JunitDoclet • TestGen4J • Jtest (Parasoft) • Appfuse/Freemarker Templates
27. Slide 27: Load Testing • Grinder • Jmeter • JunitPerf • OpenSTA • Eclipse
28. Slide 28: Other Testing • Slim (JSP) • J2ME Unit • JFCUnit (Swing) • JSTester (Javascript) • TagUnit (JSP Tags) • SOATest (Parasoft) (Web Services) • XMLUnit (XML)
29. Slide 29: Demos
30. Slide 30: Demos • Junit • TestGen4J • TestNG • Grinder • Coverage • Selenium • Parasoft Jtest • Eclipse