Introducing a new coding technique that helped PretaWeb deliver a large workflow system on time, on budget and most importantly, delivered what we expected to deliver. This will cover • unit and doctest in python, • test driven development, • usecase analysis, • automated functional web testing, • some practical examples using Grok and • a brief look at documentation driven development. Techniques covered are applicable to small and large web developments

1. dylan@pretaweb.comPloneConf 2009 Dylan Jay
TestBrowser Driven Development
How to get bulletproof code from lazy developers
Dylan Jay
djay@pretaweb.com

2. dylan@pretaweb.comPloneConf 2009 Dylan Jay
• Dylan Jay
• 6 years with Plone
• Large corporate Java/Rational Unified process
background
• Co-founder of PretaWeb in Australia
• Started pretaweb.funnelweb, collective.hostout,
Products.LoginLockout
(and way way back RemoteUserFolder)
About me

3. dylan@pretaweb.comPloneConf 2009 Dylan Jay
What this talk is about
• Story of how we delivered the difficult (previous
failures)
• Communication
• Users ↔ Business Analyst
• Business Analyst ↔ Developers
• For project managers, business analysts,
technical leads
• But also for solo development

4. dylan@pretaweb.comPloneConf 2009 Dylan Jay
The story begins...
• Gov. department
• Complex Workflow Application of sensitive info
– generating certificates
• BA 40% Dev 60% + support
• Fixed price + fixed deadline.
• Sold them on scrum … without mentioning
scrum
• Used hybrid SCUM + usecase analysis

5. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Roles: reference
• BA: Business Analyst - Extract requirements
from client and produce requirements
document. Traditionally job ends before code
begins
• PO: Product Owner - In SCRUM: Sort of like a
BA who sticks around. Go to person.
• Dev: Developer - People who want solve code
problems not user problems :)
• ST/QA: System Tester - Non-developers there
to break developers work

6. dylan@pretaweb.comPloneConf 2009 Dylan Jay
CRC Card sessions
• Class, Responsibility, Collaboration

7. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Use-case analysis
Great for workflow/edge cases
• Usecase:
– 1 main scenario (sunny day)
– ~30 alternate scenarions (rainy day)
• Challenge - level of detail
– Premature detail vs. not enough
• ~50 scenarios
• 43 page word doc including non-functional
requirements

8. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Paper Based Prototype
• http://www.balsamiq.com/products/mockups

9. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Stories vs Usecases
• We used SCRUM
• SCRUM uses Stories
• Stories are good for estimation
• Stories group functionality vs. Scenarios runs
across functionality
• We
– Broke large scenario into three stories
– Grouped 3-4 related scenarios into stories

10. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Unit Testing
1 Code
2 Write test
3 Run test: it succeeds
4 Refactor
5 Run test: it succeeds
• Tests are code
• Test functions/API
• Tests run fast
• Tests run often
• Tests run after every
change

11. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Unit Testing – scorecard
• Communication of requirements - x
• Validation of requirements – x
• Prevent regressions - ✓

12. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Doctests
• Structured text document
• Tells a story
• Tells a story with EXAMPLES THAT WORK!
• Code interleaved
• Output checked against actual == test
• docstring or standalone doctests

13. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Functional Testing
• Test system from outside – GUI after
development
• Written from user perspective
• Easy to write – write it as you would use it
• Usecase = test
• Often done by separate team – system testers
• Often done just before delivery
• Example: selenium

14. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Selenium

15. dylan@pretaweb.comPloneConf 2009 Dylan Jay
zc.testbrowser an api browser
Easy to keep in your head – fun - natural
browser.getControl(label=None, name=None, index=None)
/.options /.value /.click()
browser.getLink(text=None, url=None, id=None)
/.click()
browser.open(url)
brower.goBack()
browser.reload()
print browser.contents

16. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Usecases to tests
• Dev: Branch created
• Dev: Scenarios copied to StoryX.txt
• Dev: Code developed
• Dev: StoryX.txt augmented with testbrowser
• Dev: All Tests pass
• PO: Code + test reviewed
• PO: Branch merged

17. dylan@pretaweb.comPloneConf 2009 Dylan Jay
TestBrowser extras
• Doesn't do javascript
• But zc.testbrowser.real does
– Requires firefox
• Zope.testrecorder
– Generate tests from GUI
• With Plone use roadrunner

18. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Still not working...
• Implemented wrong thing – rework
• Usecases not detailed enough
• Lazy BA – didn't want tell developers how to do
it
• Lazy developers – didn't want at the user level

19. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Functional Test Driven Development
• PO: Branch created
• PO: Scenarios copied to StoryX.txt
• PO: StoryX.txt augmented with tests (sort of)
• Dev: Code developed
• Dev: Test finished off
• Dev: All tests pass
• PO: Code + test reviewed
• PO: Branch merged

20. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Our process – Bugs
• Dev: Branch created
• Dev: StoryX.txt add in bug condition
• Dev: Show test fails
• Dev: Code fixed
• Dev: All tests pass
• PO: Code + test reviewed
• PO: Branch merged

21. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Test Driven Development
1 Write test
2 Run test: it fails
3 Fix code
4 Run test: it succeeds
5 Refactor
6 Run test: it succeeds
7 Repeat
• Write tests before
code
• red/green/refactor
– First fail then fix
• Forces you to
understand
requirements before
writing code

22. dylan@pretaweb.comPloneConf 2009 Dylan Jay
TestBrowser Driven Development
• Communication of requirements - ✓
– Level of detail right
• Easy – ✓
– Technical PO can do it in reasonable time
• Validation of requirements – ✓
– Easy to see changes to tests
• Prevent regressions - ✓
– Test suite run after any change e.g. 1862 bug

23. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Example: URL Shortener

24. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Proposal: Screenshot gen
• Testbrowser statements → screenshot
• Highlight action or comparison in screenshot
• Doctests with pictures and no code
• A better pypi & Plone Software Center
• Documentation Driven Development

25. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Conclusion
• ~50 usecases
– all turned into doctests and extended
– Living requirements/documention
• Quality was very important
– delivered with peace of mind
• Tight deadline, tight budget
– delivered on time on budget
• Lazy developers
– less rework, less delays = $$$

26. dylan@pretaweb.comPloneConf 2009 Dylan Jay
Credits
http://docs.python.org/library/doctest.html
http://pypi.python.org/pypi/zc.testbrowser
http://plone.org/documentation/tutorial/testing/functional-tests