Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Quo pertentas, OSS?


Published on

Quo pertentas, OSS? - How Open Source can benefit from well-crafted Tests

This talk illustrates how a suite of well-written tests can benefit any Open Source project on multiple levels*:
- improve maintainability of the code-base
- help increase the truck factor** of the project
- "after-the-fact" tests help understand existing code and serve as documentation
- Behavior Driven Development (BDD) concepts can help create specification-like tests

The idea of adding BDD-style unit tests was introduced into the actively developed OWASP ZAP project end of 2012. It will be explained
- how the ZAP team approached this task initially
- what the improvements for the project were so far
- where we are going with automated testing in the future

Disclaimer: Some source code will definetely be shown during this talk, but you won't need to be a Java expert to follow the story! Having some general programming experience is totally sufficient!

*= surprisingly also works for proprietary software projects!
**= number of contributors that could be (fatally) run over by a truck without effectively killing the project

Published in: Technology
  • Be the first to comment

Quo pertentas, OSS?

  1. 1. Quo pertentas, OSS? How Open Source can benefit from well-crafted Tests Björn Kimminich Web: Twitter: @bkimminich v1.1
  2. 2. Let‘s start with some code…
  3. 3. …and a corresponding unit test!
  4. 4. It passes with flying colors…
  5. 5. … and achieves 100% code coverage!
  6. 6. Nothing could possibly go wrong!
  7. 7. How about adding another test?
  8. 8. Oops!
  9. 9. FindingBugsinOpen Source Software
  10. 10. Code Reviews Cartoon: Geek & Poke PairProgramming Infeasible with remote development Occasionally during Hackathons PeerReview Developers review each other Hard to organize properly CommitterReview Not everyone has commit rights Senior developers review contributions before merge into master
  11. 11. Static Code Analysis Cartoon: Geek & Poke Some commercial Tools might be more powerful… …but are typically not affordable for OSS projects Find code smells and potential programming errors… …but miss a lot as well …or produce false positives Popular Open Source Tools FindBugs CheckStyle PMD Sonar
  12. 12. Testing Test Types Unit Tests Integration Tests GUI Tests Manual Tests Load/Stress Tests Penetration Tests Cartoon: Geek & Poke
  13. 13. Bestvs. BadPracticesfor Testing
  14. 14. Test Pyramid Manual Tests GUI Tests Integration Tests Unit Tests Source: WatirMelon
  15. 15. Manual Tests GUI Tests Integration Tests Unit Tests Test Ice-Cream Cone Source: WatirMelon
  16. 16. Happy Path Testing Photo: Tortured Mind Photography
  17. 17. Testing Border & Exceptional Cases
  18. 18. No Assertions
  19. 19. API Tests
  20. 20. Scenario Tests with BDD
  21. 21. Benefits of well-craftedTests forOSS
  22. 22. Maintainability++ A suite of automated regression tests helps finding defects resulting from code changes New contributors do not have to fear touching old code… …neither do long-time committers after a longer vacation! Cartoon: Geek & Poke
  23. 23. Documentation++ External and Javadoc documentation tends to rot quickly and becomes obsolete or even misleading Tests that get outdated tend to break, so they have to be fixed resulting in updated documentation Well-written tests document the intended behavior of a class or component Even if the production code is hard to understand, a good test can help to fill this gap Cartoon: Geek & Poke
  24. 24. Specification++ Writing tests before the production code is even better than just documenting existing code Consequent TDD / BDD will let the Tests become the actual specification of the program's intended behavior Failing tests indicate that the specification is not met yet (or any more) Cartoon: Geek & Poke
  25. 25. Contribution++ Well maintained, documented and tested projects are safer and more fun to contribute to Nobody likes working on an untested piece of unreadable code (especially in their free time) Cartoon: Geek & Poke
  26. 26. Truck Factor++ How many project contributors could be fatally hit by a truck before the project perishes? The lower the number, the more volatile the project as it relies on individual experts The number can be increased by spreading knowledge and lowering entry barriers Cartoon: Geek & Poke
  27. 27. IntroducingUnitTeststo OWASP ZAP
  28. 28. OWASP Zed Attack Proxy (ZAP) Easy-to-use integrated penetration- testing tool Locates vulnerabilities in web applications Helps building secure apps OWASP Flagship Project Programmed in Java with javax.swing UI
  29. 29. How to contribute to ZAP? Develop core features Develop addons Help with translation Promote ZAP
  30. 30. ZAP Truck Factor ≤2 Source: Ohloh
  31. 31. Starting from zero Unit Tests No Unit Tests Some JUnit- based Integration tests
  32. 32. Separate Test Project
  33. 33. ZAPs first Unit Test
  34. 34. Adding some more Show Cases
  35. 35. Separation into Test Suites
  36. 36. Providing Test Guidelines Types of Tests Test Suites Test Libraries Naming Conventions Behavior Driven Development Code Quality Code Coverage
  37. 37. Pull vs. Push Pull Push Photos: One Man Think Tank
  38. 38. Commits to ZAP Tests over time
  39. 39. Measure Code Coverage
  40. 40. Move Tests close to Production Code
  41. 41. Instant execution from IDE
  42. 42. Run all Tests during Continuous Build...
  43. 43. ...and let it fail when any tests fail!
  44. 44. Future: Adding a GUI Testing Framework ZAP is very UI heavy which makes a lot of the code hard or impossible to unit test Right now there are no GUI Tests in place for ZAP Several free UI Testing Frameworks exist for Java Swing… …unfortunately none is actively maintained any more Manual Tests GUI Tests Integration Tests Unit Tests
  45. 45. Conclusion Testing is a crucial part of Software Development Good Tests are the better documentation Tests can make a difference between a prospering and a dead-end OSS project
  46. 46. Thank you! Björn Kimminich Web: Twitter: @bkimminich Background Image: Eikira