Quo pertentas, OSS?

  • 1,007 views
Uploaded on

Quo pertentas, OSS? - How Open Source can benefit from well-crafted Tests …

Quo pertentas, OSS? - How Open Source can benefit from well-crafted Tests

This talk illustrates how a suite of well-written tests can benefit any Open Source project on multiple levels*:
- improve maintainability of the code-base
- help increase the truck factor** of the project
- "after-the-fact" tests help understand existing code and serve as documentation
- Behavior Driven Development (BDD) concepts can help create specification-like tests

The idea of adding BDD-style unit tests was introduced into the actively developed OWASP ZAP project end of 2012. It will be explained
- how the ZAP team approached this task initially
- what the improvements for the project were so far
- where we are going with automated testing in the future

Disclaimer: Some source code will definetely be shown during this talk, but you won't need to be a Java expert to follow the story! Having some general programming experience is totally sufficient!

*= surprisingly also works for proprietary software projects!
**= number of contributors that could be (fatally) run over by a truck without effectively killing the project

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,007
On Slideshare
0
From Embeds
0
Number of Embeds
7

Actions

Shares
Downloads
5
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Quo pertentas, OSS? How Open Source can benefit from well-crafted Tests Björn Kimminich Web: http://kimminich.de Twitter: @bkimminich v1.1
  • 2. Let‘s start with some code…
  • 3. …and a corresponding unit test!
  • 4. It passes with flying colors…
  • 5. … and achieves 100% code coverage!
  • 6. Nothing could possibly go wrong!
  • 7. How about adding another test?
  • 8. Oops!
  • 9. FindingBugsinOpen Source Software
  • 10. Code Reviews Cartoon: Geek & Poke PairProgramming Infeasible with remote development Occasionally during Hackathons PeerReview Developers review each other Hard to organize properly CommitterReview Not everyone has commit rights Senior developers review contributions before merge into master
  • 11. Static Code Analysis Cartoon: Geek & Poke Some commercial Tools might be more powerful… …but are typically not affordable for OSS projects Find code smells and potential programming errors… …but miss a lot as well …or produce false positives Popular Open Source Tools FindBugs CheckStyle PMD Sonar
  • 12. Testing Test Types Unit Tests Integration Tests GUI Tests Manual Tests Load/Stress Tests Penetration Tests Cartoon: Geek & Poke
  • 13. Bestvs. BadPracticesfor Testing
  • 14. Test Pyramid Manual Tests GUI Tests Integration Tests Unit Tests Source: WatirMelon
  • 15. Manual Tests GUI Tests Integration Tests Unit Tests Test Ice-Cream Cone Source: WatirMelon
  • 16. Happy Path Testing Photo: Tortured Mind Photography
  • 17. Testing Border & Exceptional Cases
  • 18. No Assertions
  • 19. API Tests
  • 20. Scenario Tests with BDD
  • 21. Benefits of well-craftedTests forOSS
  • 22. Maintainability++ A suite of automated regression tests helps finding defects resulting from code changes New contributors do not have to fear touching old code… …neither do long-time committers after a longer vacation! Cartoon: Geek & Poke
  • 23. Documentation++ External and Javadoc documentation tends to rot quickly and becomes obsolete or even misleading Tests that get outdated tend to break, so they have to be fixed resulting in updated documentation Well-written tests document the intended behavior of a class or component Even if the production code is hard to understand, a good test can help to fill this gap Cartoon: Geek & Poke
  • 24. Specification++ Writing tests before the production code is even better than just documenting existing code Consequent TDD / BDD will let the Tests become the actual specification of the program's intended behavior Failing tests indicate that the specification is not met yet (or any more) Cartoon: Geek & Poke
  • 25. Contribution++ Well maintained, documented and tested projects are safer and more fun to contribute to Nobody likes working on an untested piece of unreadable code (especially in their free time) Cartoon: Geek & Poke
  • 26. Truck Factor++ How many project contributors could be fatally hit by a truck before the project perishes? The lower the number, the more volatile the project as it relies on individual experts The number can be increased by spreading knowledge and lowering entry barriers Cartoon: Geek & Poke
  • 27. IntroducingUnitTeststo OWASP ZAP
  • 28. OWASP Zed Attack Proxy (ZAP) Easy-to-use integrated penetration- testing tool Locates vulnerabilities in web applications Helps building secure apps OWASP Flagship Project Programmed in Java with javax.swing UI
  • 29. How to contribute to ZAP? Develop core features https://code.google.com/p/zaproxy/ Develop addons https://code.google.com/p/zap-extensions/ Help with translation https://crowdin.net/project/owasp-zap Promote ZAP https://code.google.com/p/zaproxy/wiki/ZapEvangelists
  • 30. ZAP Truck Factor ≤2 Source: Ohloh
  • 31. Starting from zero Unit Tests No Unit Tests Some JUnit- based Integration tests
  • 32. Separate Test Project
  • 33. ZAPs first Unit Test
  • 34. Adding some more Show Cases
  • 35. Separation into Test Suites
  • 36. Providing Test Guidelines Types of Tests Test Suites Test Libraries Naming Conventions Behavior Driven Development Code Quality Code Coverage
  • 37. Pull vs. Push Pull Push Photos: One Man Think Tank
  • 38. Commits to ZAP Tests over time
  • 39. Measure Code Coverage
  • 40. Move Tests close to Production Code
  • 41. Instant execution from IDE
  • 42. Run all Tests during Continuous Build...
  • 43. ...and let it fail when any tests fail!
  • 44. Future: Adding a GUI Testing Framework ZAP is very UI heavy which makes a lot of the code hard or impossible to unit test Right now there are no GUI Tests in place for ZAP Several free UI Testing Frameworks exist for Java Swing… …unfortunately none is actively maintained any more Manual Tests GUI Tests Integration Tests Unit Tests
  • 45. Conclusion Testing is a crucial part of Software Development Good Tests are the better documentation Tests can make a difference between a prospering and a dead-end OSS project
  • 46. Thank you! Björn Kimminich Web: http://kimminich.de Twitter: @bkimminich Background Image: Eikira