0
Quo pertentas, OSS?
How Open Source can benefit from well-crafted Tests
Björn Kimminich
Web: http://kimminich.de
Twitter: ...
Let‘s start with some code…
…and a corresponding unit test!
It passes with flying colors…
… and achieves 100% code coverage!
Nothing could possibly go wrong!
How about adding another test?
Oops!
FindingBugsinOpen Source Software
Code Reviews
Cartoon: Geek & Poke
PairProgramming
Infeasible
with remote
development
Occasionally
during
Hackathons
PeerRe...
Static Code Analysis
Cartoon: Geek & Poke
Some commercial Tools might be more powerful…
…but are typically not affordable ...
Testing
Test Types
Unit Tests
Integration
Tests
GUI Tests
Manual
Tests
Load/Stress
Tests
Penetration
Tests
Cartoon: Geek &...
Bestvs. BadPracticesfor Testing
Test Pyramid Manual
Tests
GUI Tests
Integration Tests
Unit Tests
Source: WatirMelon
Manual Tests
GUI Tests
Integration
Tests
Unit
Tests
Test Ice-Cream Cone
Source: WatirMelon
Happy Path Testing
Photo: Tortured Mind Photography
Testing Border & Exceptional Cases
No Assertions
API Tests
Scenario Tests with BDD
Benefits of well-craftedTests forOSS
Maintainability++
A suite of automated regression tests helps
finding defects resulting from code changes
New contributors...
Documentation++
External and Javadoc documentation tends to rot
quickly and becomes obsolete or even misleading
Tests that...
Specification++
Writing tests before the production code is even better
than just documenting existing code
Consequent TDD...
Contribution++
Well maintained, documented and
tested projects are safer and more
fun to contribute to
Nobody likes workin...
Truck Factor++
How many project contributors could be fatally
hit by a truck before the project perishes?
The lower the nu...
IntroducingUnitTeststo OWASP ZAP
OWASP Zed Attack Proxy (ZAP)
Easy-to-use
integrated
penetration-
testing tool
Locates vulnerabilities in web applications
...
How to contribute to ZAP?
Develop core features https://code.google.com/p/zaproxy/
Develop addons https://code.google.com/...
ZAP Truck Factor ≤2
Source: Ohloh
Starting from zero Unit Tests
No Unit Tests
Some JUnit-
based
Integration
tests
Separate Test Project
ZAPs first Unit Test
Adding some more Show Cases
Separation into Test Suites
Providing Test Guidelines
Types of Tests
Test Suites
Test Libraries
Naming
Conventions
Behavior
Driven
Development
Code Qu...
Pull vs. Push
Pull Push
Photos: One Man Think Tank
Commits to ZAP Tests over time
Measure Code Coverage
Move Tests close to Production Code
Instant execution from IDE
Run all Tests during Continuous Build...
...and let it fail when any tests fail!
Future: Adding a GUI Testing Framework
ZAP is very UI heavy which makes a lot of the
code hard or impossible to unit test
...
Conclusion
Testing is a crucial part of Software
Development
Good Tests are the better documentation
Tests can make a diff...
Thank you!
Björn Kimminich
Web: http://kimminich.de
Twitter: @bkimminich
Background Image: Eikira
Upcoming SlideShare
Loading in...5
×

Quo pertentas, OSS?

1,642

Published on

Quo pertentas, OSS? - How Open Source can benefit from well-crafted Tests

This talk illustrates how a suite of well-written tests can benefit any Open Source project on multiple levels*:
- improve maintainability of the code-base
- help increase the truck factor** of the project
- "after-the-fact" tests help understand existing code and serve as documentation
- Behavior Driven Development (BDD) concepts can help create specification-like tests

The idea of adding BDD-style unit tests was introduced into the actively developed OWASP ZAP project end of 2012. It will be explained
- how the ZAP team approached this task initially
- what the improvements for the project were so far
- where we are going with automated testing in the future

Disclaimer: Some source code will definetely be shown during this talk, but you won't need to be a Java expert to follow the story! Having some general programming experience is totally sufficient!

*= surprisingly also works for proprietary software projects!
**= number of contributors that could be (fatally) run over by a truck without effectively killing the project

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,642
On Slideshare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
6
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Quo pertentas, OSS?"

  1. 1. Quo pertentas, OSS? How Open Source can benefit from well-crafted Tests Björn Kimminich Web: http://kimminich.de Twitter: @bkimminich v1.1
  2. 2. Let‘s start with some code…
  3. 3. …and a corresponding unit test!
  4. 4. It passes with flying colors…
  5. 5. … and achieves 100% code coverage!
  6. 6. Nothing could possibly go wrong!
  7. 7. How about adding another test?
  8. 8. Oops!
  9. 9. FindingBugsinOpen Source Software
  10. 10. Code Reviews Cartoon: Geek & Poke PairProgramming Infeasible with remote development Occasionally during Hackathons PeerReview Developers review each other Hard to organize properly CommitterReview Not everyone has commit rights Senior developers review contributions before merge into master
  11. 11. Static Code Analysis Cartoon: Geek & Poke Some commercial Tools might be more powerful… …but are typically not affordable for OSS projects Find code smells and potential programming errors… …but miss a lot as well …or produce false positives Popular Open Source Tools FindBugs CheckStyle PMD Sonar
  12. 12. Testing Test Types Unit Tests Integration Tests GUI Tests Manual Tests Load/Stress Tests Penetration Tests Cartoon: Geek & Poke
  13. 13. Bestvs. BadPracticesfor Testing
  14. 14. Test Pyramid Manual Tests GUI Tests Integration Tests Unit Tests Source: WatirMelon
  15. 15. Manual Tests GUI Tests Integration Tests Unit Tests Test Ice-Cream Cone Source: WatirMelon
  16. 16. Happy Path Testing Photo: Tortured Mind Photography
  17. 17. Testing Border & Exceptional Cases
  18. 18. No Assertions
  19. 19. API Tests
  20. 20. Scenario Tests with BDD
  21. 21. Benefits of well-craftedTests forOSS
  22. 22. Maintainability++ A suite of automated regression tests helps finding defects resulting from code changes New contributors do not have to fear touching old code… …neither do long-time committers after a longer vacation! Cartoon: Geek & Poke
  23. 23. Documentation++ External and Javadoc documentation tends to rot quickly and becomes obsolete or even misleading Tests that get outdated tend to break, so they have to be fixed resulting in updated documentation Well-written tests document the intended behavior of a class or component Even if the production code is hard to understand, a good test can help to fill this gap Cartoon: Geek & Poke
  24. 24. Specification++ Writing tests before the production code is even better than just documenting existing code Consequent TDD / BDD will let the Tests become the actual specification of the program's intended behavior Failing tests indicate that the specification is not met yet (or any more) Cartoon: Geek & Poke
  25. 25. Contribution++ Well maintained, documented and tested projects are safer and more fun to contribute to Nobody likes working on an untested piece of unreadable code (especially in their free time) Cartoon: Geek & Poke
  26. 26. Truck Factor++ How many project contributors could be fatally hit by a truck before the project perishes? The lower the number, the more volatile the project as it relies on individual experts The number can be increased by spreading knowledge and lowering entry barriers Cartoon: Geek & Poke
  27. 27. IntroducingUnitTeststo OWASP ZAP
  28. 28. OWASP Zed Attack Proxy (ZAP) Easy-to-use integrated penetration- testing tool Locates vulnerabilities in web applications Helps building secure apps OWASP Flagship Project Programmed in Java with javax.swing UI
  29. 29. How to contribute to ZAP? Develop core features https://code.google.com/p/zaproxy/ Develop addons https://code.google.com/p/zap-extensions/ Help with translation https://crowdin.net/project/owasp-zap Promote ZAP https://code.google.com/p/zaproxy/wiki/ZapEvangelists
  30. 30. ZAP Truck Factor ≤2 Source: Ohloh
  31. 31. Starting from zero Unit Tests No Unit Tests Some JUnit- based Integration tests
  32. 32. Separate Test Project
  33. 33. ZAPs first Unit Test
  34. 34. Adding some more Show Cases
  35. 35. Separation into Test Suites
  36. 36. Providing Test Guidelines Types of Tests Test Suites Test Libraries Naming Conventions Behavior Driven Development Code Quality Code Coverage
  37. 37. Pull vs. Push Pull Push Photos: One Man Think Tank
  38. 38. Commits to ZAP Tests over time
  39. 39. Measure Code Coverage
  40. 40. Move Tests close to Production Code
  41. 41. Instant execution from IDE
  42. 42. Run all Tests during Continuous Build...
  43. 43. ...and let it fail when any tests fail!
  44. 44. Future: Adding a GUI Testing Framework ZAP is very UI heavy which makes a lot of the code hard or impossible to unit test Right now there are no GUI Tests in place for ZAP Several free UI Testing Frameworks exist for Java Swing… …unfortunately none is actively maintained any more Manual Tests GUI Tests Integration Tests Unit Tests
  45. 45. Conclusion Testing is a crucial part of Software Development Good Tests are the better documentation Tests can make a difference between a prospering and a dead-end OSS project
  46. 46. Thank you! Björn Kimminich Web: http://kimminich.de Twitter: @bkimminich Background Image: Eikira
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×