0
Testing Tools and Strategies  Scott Ryan March 2007
Agenda <ul><ul><li>Business Drivers </li></ul></ul><ul><ul><li>Unit Testing </li></ul></ul><ul><ul><li>Coverage </li></ul>...
Business Drivers <ul><ul><li>Errors found early in the development lifecycle cost less to fix </li></ul></ul><ul><ul><li>C...
Business Drivers <ul><ul><li>Cost of running integration and system tests again.  </li></ul></ul><ul><ul><li>Cost of recer...
Testing Levels <ul><ul><li>Unit Testing </li></ul></ul><ul><ul><li>Integration Testing </li></ul></ul><ul><ul><li>System T...
Test Design <ul><ul><li>Black Box </li></ul></ul><ul><ul><li>White Box </li></ul></ul>
Testing Techniques <ul><ul><li>Manual Testing </li></ul></ul><ul><ul><li>Automated Testing </li></ul></ul><ul><ul><li>Regr...
Unit Testing Frameworks <ul><ul><li>Junit </li></ul></ul><ul><ul><ul><li>HttpUnit </li></ul></ul></ul><ul><ul><ul><li>Stru...
JUnit <ul><ul><li>Version 3 and 4 </li></ul></ul><ul><ul><li>Extend TestCase Class </li></ul></ul><ul><ul><li>Validate thr...
Test NG <ul><ul><li>Java 5 (Primarily) </li></ul></ul><ul><ul><li>No need to extend base class </li></ul></ul><ul><ul><li>...
TestNG Annotations <ul><ul><li>@Test </li></ul></ul><ul><ul><li>@Parameters </li></ul></ul><ul><ul><li>@DataProvider </li>...
Unit Testing Tips <ul><ul><li>Keep tests in same package as tested code </li></ul></ul><ul><ul><li>Separate tests for pack...
Coverage <ul><ul><li>Cobertura </li></ul></ul><ul><ul><li>Clover </li></ul></ul><ul><ul><li>Emma </li></ul></ul><ul><ul><l...
Coverage Types <ul><ul><li>Statement Coverage </li></ul></ul><ul><ul><li>Branch Coverage (if and implied else) </li></ul><...
Coverage Hints and Tips <ul><ul><li>Use at the beginning of test development </li></ul></ul><ul><ul><li>Strive for branch ...
Code Quality Management <ul><ul><li>Used to raise the quality of code base </li></ul></ul><ul><ul><li>Range from trivial t...
CQM Tools <ul><ul><li>Checkstyle </li></ul></ul><ul><ul><li>Findbugs </li></ul></ul><ul><ul><li>PMD </li></ul></ul><ul><ul...
CQM Hints and Tips <ul><ul><li>Automate and run often </li></ul></ul><ul><ul><li>Make incremental improvements over time <...
Integration Testing <ul><ul><li>Tests the assembled deployed unit </li></ul></ul><ul><ul><li>Usually requires automation <...
Integration Testing Frameworks <ul><ul><li>Selenium </li></ul></ul><ul><ul><li>Cactus </li></ul></ul><ul><ul><li>Canoo </l...
Integration Testing Hints and Tips <ul><ul><li>Start small  </li></ul></ul><ul><ul><li>Automate as soon as practical </li>...
Test Data Hints and Tips <ul><ul><li>Understand the data needs of your tests </li></ul></ul><ul><ul><li>Start the system i...
Test Data Tools <ul><ul><li>Setup and teardown methods </li></ul></ul><ul><ul><li>Database Scripts </li></ul></ul><ul><ul>...
Mock Testing <ul><ul><li>Objects act as mediators </li></ul></ul><ul><ul><li>Simulates complex or unfinished systems </li>...
Mock Toolsets <ul><ul><li>Easy Mock </li></ul></ul><ul><ul><li>Spring Mock </li></ul></ul><ul><ul><li>JMock </li></ul></ul...
Automated Test Generation <ul><ul><li>JunitDoclet </li></ul></ul><ul><ul><li>TestGen4J </li></ul></ul><ul><ul><li>Jtest (P...
Load Testing <ul><ul><li>Grinder </li></ul></ul><ul><ul><li>Jmeter </li></ul></ul><ul><ul><li>JunitPerf </li></ul></ul><ul...
Other Testing <ul><ul><li>Slim (JSP) </li></ul></ul><ul><ul><li>J2ME Unit </li></ul></ul><ul><ul><li>JFCUnit (Swing) </li>...
Demos
Demos <ul><ul><li>Junit </li></ul></ul><ul><ul><li>TestNG </li></ul></ul><ul><ul><li>Coverage </li></ul></ul><ul><ul><li>S...
Upcoming SlideShare
Loading in...5
×

Software Testing Tools and Strategies March 2007

2,644

Published on

A presentation on a Developers viewpoint of available testing tools and strategies for testing Java applications. The presentation was given to the Denver Open Source User\'s Group www.denveropensource.org

Published in: Business, Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,644
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
339
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Software Testing Tools and Strategies March 2007"

  1. 1. Testing Tools and Strategies Scott Ryan March 2007
  2. 2. Agenda <ul><ul><li>Business Drivers </li></ul></ul><ul><ul><li>Unit Testing </li></ul></ul><ul><ul><li>Coverage </li></ul></ul><ul><ul><li>Code Quality Management (CQM) </li></ul></ul><ul><ul><li>Integration Testing </li></ul></ul><ul><ul><li>Load Testing </li></ul></ul><ul><ul><li>Test Data </li></ul></ul><ul><ul><li>Automated Test Generation </li></ul></ul><ul><ul><li>Demo </li></ul></ul>
  3. 3. Business Drivers <ul><ul><li>Errors found early in the development lifecycle cost less to fix </li></ul></ul><ul><ul><li>Cost varies by Industry and Software usage </li></ul></ul><ul><ul><li>Cost is not linear </li></ul></ul><ul><ul><li>Testing improves quality and reduces cost </li></ul></ul><ul><ul><li>Testing costs money too and there is a tradeoff </li></ul></ul>
  4. 4. Business Drivers <ul><ul><li>Cost of running integration and system tests again. </li></ul></ul><ul><ul><li>Cost of recertification (if you're in that kind of environment). </li></ul></ul><ul><ul><li>Cost of deploying the software again. </li></ul></ul><ul><ul><li>Support costs when only half your customers deploy the new version. </li></ul></ul><ul><ul><li>Indirect costs caused by using resources to fix bugs rather than implement revenue-generating features. </li></ul></ul><ul><ul><li>Liability for damages caused by the bug. </li></ul></ul>
  5. 5. Testing Levels <ul><ul><li>Unit Testing </li></ul></ul><ul><ul><li>Integration Testing </li></ul></ul><ul><ul><li>System Testing </li></ul></ul>
  6. 6. Test Design <ul><ul><li>Black Box </li></ul></ul><ul><ul><li>White Box </li></ul></ul>
  7. 7. Testing Techniques <ul><ul><li>Manual Testing </li></ul></ul><ul><ul><li>Automated Testing </li></ul></ul><ul><ul><li>Regression Testing </li></ul></ul><ul><ul><li>Environment Testing </li></ul></ul><ul><ul><li>Stress Testing </li></ul></ul><ul><ul><li>Load Testing </li></ul></ul><ul><ul><li>Performance Testing </li></ul></ul>
  8. 8. Unit Testing Frameworks <ul><ul><li>Junit </li></ul></ul><ul><ul><ul><li>HttpUnit </li></ul></ul></ul><ul><ul><ul><li>StrutsUnit </li></ul></ul></ul><ul><ul><ul><li>XMLUnit </li></ul></ul></ul><ul><ul><ul><li>DBUnit </li></ul></ul></ul><ul><ul><li>TestNG </li></ul></ul>
  9. 9. JUnit <ul><ul><li>Version 3 and 4 </li></ul></ul><ul><ul><li>Extend TestCase Class </li></ul></ul><ul><ul><li>Validate through Asssertions </li></ul></ul><ul><ul><li>Setup and TearDown Methods </li></ul></ul><ul><ul><li>Testing Suites </li></ul></ul><ul><ul><li>Lots of Automation </li></ul></ul>
  10. 10. Test NG <ul><ul><li>Java 5 (Primarily) </li></ul></ul><ul><ul><li>No need to extend base class </li></ul></ul><ul><ul><li>Good use of Java 5 Annotations </li></ul></ul><ul><ul><li>DataDriven Tests </li></ul></ul><ul><ul><li>Embedded Beanshell </li></ul></ul><ul><ul><li>Embedded Logging, JDK </li></ul></ul><ul><ul><li>Parameter Support </li></ul></ul><ul><ul><li>Supports Test Distribution </li></ul></ul><ul><ul><li>New Execution Model </li></ul></ul><ul><ul><ul><li>No Suites </li></ul></ul></ul><ul><ul><ul><li>Groups </li></ul></ul></ul><ul><ul><ul><li>Relationships </li></ul></ul></ul><ul><ul><li>Application Server Testing support </li></ul></ul>
  11. 11. TestNG Annotations <ul><ul><li>@Test </li></ul></ul><ul><ul><li>@Parameters </li></ul></ul><ul><ul><li>@DataProvider </li></ul></ul><ul><ul><li>@BeforeSuite </li></ul></ul><ul><ul><li>@AfterSuite </li></ul></ul><ul><ul><li>@BeforeTest </li></ul></ul><ul><ul><li>@AfterTest </li></ul></ul><ul><ul><li>@BeforeMethod </li></ul></ul><ul><ul><li>@AfterMethod </li></ul></ul><ul><ul><li>@BeforeClass </li></ul></ul><ul><ul><li>@AfterClass </li></ul></ul><ul><ul><li>@BeforeGroup </li></ul></ul><ul><ul><li>@AfterGroup </li></ul></ul>
  12. 12. Unit Testing Tips <ul><ul><li>Keep tests in same package as tested code </li></ul></ul><ul><ul><li>Separate tests for packaging </li></ul></ul><ul><ul><li>Start small and work up </li></ul></ul><ul><ul><li>Give useful names and assertions </li></ul></ul><ul><ul><li>Run often and review the results </li></ul></ul><ul><ul><li>Build tests around bugs you are fixing </li></ul></ul>
  13. 13. Coverage <ul><ul><li>Cobertura </li></ul></ul><ul><ul><li>Clover </li></ul></ul><ul><ul><li>Emma </li></ul></ul><ul><ul><li>Jtest (Parasoft) </li></ul></ul><ul><ul><li>Jester </li></ul></ul><ul><ul><li>GroboUtils </li></ul></ul><ul><ul><li>Djunit </li></ul></ul><ul><ul><li>Hansel </li></ul></ul><ul><ul><li>Gretel </li></ul></ul>
  14. 14. Coverage Types <ul><ul><li>Statement Coverage </li></ul></ul><ul><ul><li>Branch Coverage (if and implied else) </li></ul></ul><ul><ul><li>Condition Coverage </li></ul></ul><ul><ul><li>Coverage Percentage </li></ul></ul>
  15. 15. Coverage Hints and Tips <ul><ul><li>Use at the beginning of test development </li></ul></ul><ul><ul><li>Strive for branch over line coverage </li></ul></ul><ul><ul><li>Build tests to increase coverage </li></ul></ul><ul><ul><li>Hard to tell what is important so the developer is key </li></ul></ul><ul><ul><li>Review reports often as code base grows </li></ul></ul>
  16. 16. Code Quality Management <ul><ul><li>Used to raise the quality of code base </li></ul></ul><ul><ul><li>Range from trivial to high impact </li></ul></ul><ul><ul><li>All improvements are important </li></ul></ul><ul><ul><li>Use a wide variation of tools and tune around your team’s best practice </li></ul></ul><ul><ul><li>Use to drive refactoring </li></ul></ul>
  17. 17. CQM Tools <ul><ul><li>Checkstyle </li></ul></ul><ul><ul><li>Findbugs </li></ul></ul><ul><ul><li>PMD </li></ul></ul><ul><ul><li>CPD </li></ul></ul><ul><ul><li>Jtest </li></ul></ul><ul><ul><li>JDepend </li></ul></ul><ul><ul><li>Jalopy </li></ul></ul><ul><ul><li>Eclipse </li></ul></ul><ul><ul><li>NetBeans </li></ul></ul><ul><ul><li>JavaDoc </li></ul></ul>
  18. 18. CQM Hints and Tips <ul><ul><li>Automate and run often </li></ul></ul><ul><ul><li>Make incremental improvements over time </li></ul></ul><ul><ul><li>Quality is important </li></ul></ul><ul><ul><li>Use tools to improve quality at the source </li></ul></ul><ul><ul><li>Use as a guide to refactoring </li></ul></ul>
  19. 19. Integration Testing <ul><ul><li>Tests the assembled deployed unit </li></ul></ul><ul><ul><li>Usually requires automation </li></ul></ul><ul><ul><li>Developed after Unit tests </li></ul></ul><ul><ul><li>Least used technique but most effective </li></ul></ul><ul><ul><li>Fairly complex with many moving parts </li></ul></ul><ul><ul><li>Require mature development process </li></ul></ul>
  20. 20. Integration Testing Frameworks <ul><ul><li>Selenium </li></ul></ul><ul><ul><li>Cactus </li></ul></ul><ul><ul><li>Canoo </li></ul></ul><ul><ul><li>JWebUnit </li></ul></ul><ul><ul><li>HttpUnit </li></ul></ul><ul><ul><li>Jmeter </li></ul></ul><ul><ul><li>Cargo </li></ul></ul><ul><ul><li>Eclipse </li></ul></ul><ul><ul><li>Maven </li></ul></ul><ul><ul><li>Dumbster (SMTP) </li></ul></ul><ul><ul><li>WinRunner (HP) </li></ul></ul><ul><ul><li>SilkTest (Borland) </li></ul></ul><ul><ul><li>TeamTest (Rational) </li></ul></ul>
  21. 21. Integration Testing Hints and Tips <ul><ul><li>Start small </li></ul></ul><ul><ul><li>Automate as soon as practical </li></ul></ul><ul><ul><li>Strive for breadth over depth </li></ul></ul><ul><ul><li>Work with QA and UAT groups </li></ul></ul><ul><ul><li>Tie tests to use cases </li></ul></ul><ul><ul><li>Use as smoke test to insure proper deployments </li></ul></ul><ul><ul><li>Understand data needs and system impact </li></ul></ul>
  22. 22. Test Data Hints and Tips <ul><ul><li>Understand the data needs of your tests </li></ul></ul><ul><ul><li>Start the system in a known state </li></ul></ul><ul><ul><li>Return system to known state when done </li></ul></ul><ul><ul><li>Watch out for protected data </li></ul></ul><ul><ul><li>Need to understand the impact of your tests </li></ul></ul><ul><ul><li>Keep data current </li></ul></ul><ul><ul><li>Understand system impact (i.e. database keys) </li></ul></ul><ul><ul><li>Leverage Automation </li></ul></ul>
  23. 23. Test Data Tools <ul><ul><li>Setup and teardown methods </li></ul></ul><ul><ul><li>Database Scripts </li></ul></ul><ul><ul><li>DBUnit </li></ul></ul><ul><ul><li>XMLUnit </li></ul></ul><ul><ul><li>DDSteps </li></ul></ul><ul><ul><li>JTestcase </li></ul></ul>
  24. 24. Mock Testing <ul><ul><li>Objects act as mediators </li></ul></ul><ul><ul><li>Simulates complex or unfinished systems </li></ul></ul><ul><ul><li>Allows access to “closed” systems </li></ul></ul><ul><ul><li>Quicker Testing </li></ul></ul><ul><ul><li>Mocks are not stubs they are dynamic </li></ul></ul><ul><ul><li>Mocks offer behavior verification </li></ul></ul><ul><ul><li>Stubs offer state verification </li></ul></ul>
  25. 25. Mock Toolsets <ul><ul><li>Easy Mock </li></ul></ul><ul><ul><li>Spring Mock </li></ul></ul><ul><ul><li>JMock </li></ul></ul><ul><ul><li>Jmockit </li></ul></ul><ul><ul><li>MockRunner </li></ul></ul>
  26. 26. Automated Test Generation <ul><ul><li>JunitDoclet </li></ul></ul><ul><ul><li>TestGen4J </li></ul></ul><ul><ul><li>Jtest (Parasoft) </li></ul></ul><ul><ul><li>Appfuse/Freemarker Templates </li></ul></ul>
  27. 27. Load Testing <ul><ul><li>Grinder </li></ul></ul><ul><ul><li>Jmeter </li></ul></ul><ul><ul><li>JunitPerf </li></ul></ul><ul><ul><li>OpenSTA </li></ul></ul><ul><ul><li>Eclipse </li></ul></ul>
  28. 28. Other Testing <ul><ul><li>Slim (JSP) </li></ul></ul><ul><ul><li>J2ME Unit </li></ul></ul><ul><ul><li>JFCUnit (Swing) </li></ul></ul><ul><ul><li>JSTester (Javascript) </li></ul></ul><ul><ul><li>TagUnit (JSP Tags) </li></ul></ul><ul><ul><li>SOATest (Parasoft) (Web Services) </li></ul></ul><ul><ul><li>XMLUnit (XML) </li></ul></ul>
  29. 29. Demos
  30. 30. Demos <ul><ul><li>Junit </li></ul></ul><ul><ul><li>TestNG </li></ul></ul><ul><ul><li>Coverage </li></ul></ul><ul><ul><li>Selenium </li></ul></ul><ul><ul><li>Parasoft Jtest </li></ul></ul><ul><ul><li>Eclipse </li></ul></ul><ul><ul><li>TestGen4J </li></ul></ul><ul><ul><li>Grinder </li></ul></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×