This document discusses best practices for testing code. It recommends writing unit, integration, functional, generated, and performance tests. Unit tests should be isolated, reproducible, fast, simple, and automated. Integration tests check interactions between components. Functional tests simulate user behaviors. Generated tests use random inputs to try breaking code. Performance tests measure timing and load capacity. The document provides examples of tools and techniques for writing different types of tests in Python.

1. Test your code
Cédric Krier
http://www.b2ck.com/

2. 2
Who am I ?
●
B2CK shareholder
●
Developers (since 2001)
●
Tryton developers (for 10 years)

3. 3
Tryton source code

4. 4
Why writing tests?
●
Correctness of code… now
●
Correctness of code… tomorrow
●
Prevent past bugs to haunt you again
●
Give you confidence when refactoring

5. 5
How to write tests?
➔
Unit
➔
Integration
➔
Functional
➔
Generated
➔
Performance

6. 6
Testing good practices
➔
Isolated
➔
Reproducible / Deterministic
➔
Fast (if possible)
➔
Simple
➔
Automated (with CI)

7. 7
Test should be isolated
●
No interference between each test
●
Can run only one at a time
●
Can be run in parallel

8. 8
Test should be
reproducible/deterministic
●
Avoid “It works on my computer”
●
Static input:
– No random
– Date/Time: FreezeGun
– PYTHONHASHSEED

9. 9
Test should be fast

10. 10
Test should be fast
●
Developers should not hesitate to run
them
●
Allow to run the subsets targeting the
changes
●
Avoid:
– heavy computation
– slow network connection

11. 11
Test should be simple
●
Test your code, not your dependencies
●
Simple… but not trivial
– Test the behavior not the implementation
●
Simple enough to allow to fix failure
without debugging
●
Easier to maintain

12. 12
Test should be automated
●
Continuous Integration (Buildbot,
Jenkins, Drone etc.)
– Run on each commit (or pull request)
– Test on multiple environments (tox, docker, …)
– Feedback to author about failure
(responsibility)

13. 13
Tools for testing in Python
●
Test Case:
– unittest, doctest, mock
●
Test runners:
– unittest, pytest, nose, tox
●
Behavior driven development ⚠
– behave, lettuce
●
Generated tests
– hypothesis

14. 14
How testing?
➔
Unit
➔
Integration
➔
Functional
➔
Generated
➔
Performance

15. 15
Unit test
●
Testing smallest unit of code → the
function
●
Check output from defined input
(like a black box)

16. 16
Unit test good practices
●
Triple-A
– Arrange
– Act
– Assert

17. 17
Unit test good practices

18. 18
Unit test good practices
●
Limit usage of Mock
●
Do not bloat API for dependency
injection

19. 19
Unit test good practices

20. 20
Unit test in Python
●
Use explicit docstring
●
Use:
– assertEqual instead of assert(a == b)
– assertRaises (context manager)
– assertWarns, assertLogs
– assertListEqual, assertDictEqual, etc.
– addCleanup (register tearDown in setUp)

21. 21
Integration tests
●
Test interactions between components
●
Instead of testing internal parts,
test public API

22. 22
Integration test good practices
●
Don’t forget to clean up
●
Check absence of error instead of the
result

23. 23
Integration test in Python
●
Use “setUp” and “tearDown”

24. 24
Functional test
●
Check behaviors as an end user
●
Run complete workflow, ex:
sale → shipment → invoice → payment

25. 25
Functional test good practices
●
Run with the full stack
– Database
– Network
– Access rights
●
Simulate user behavior
– Selenium
– RPC call

26. 26
Functional test for Python
●
Use doctest

27. 27
Functional test for Python
●
Selenium with Python

28. 28
Generated test
●
Test properties of output with random
input (but limited)
●
Try to break with fuzzy input

29. 29
Generated test good practices
●
Not included in CI
●
But run from time to time

30. 30
Generated test in Python
●
Hypothesis

31. 31
Performance test
●
Measure time to make an action
– Warm-up (setup, cache etc.)
– Compare mean
●
Test load capacity
– Fire a lot of requests

32. 32
Performance test in Python
●
timeit

33. 33
Performance test in Python
●
Boom

34. 34
Conclusion
Write enough tests to be confident
Do not write too much that will block
you