My talk at #Prague #Pyvo (Python meetup) https://pyvo.cz/praha-pyvo/ Overview of Python testing frameworks and how the compare.

1. What is doctest?
Python testing frameworks overview
1 / 26

2. What am I doing here?
2 / 26

3. What am I doing here?
Facts
Jachym Cepicky
GIS (maps in computer)
Open source software
Not facts
not a tester
not educated software engineer
nor GIS educated
3 / 26

4. What am I doing here?
Me: confused py.test with doctest
4 / 26

5. Let's get started
5 / 26

6. Testing motivation
6 / 26

7. Everybody loves to tests.
7 / 26

8. Everybody has heard, that using TDD is the only way how to write
code loves to tests.
8 / 26

9. TDD
9 / 26

10. The fact
Writing tests
...
does make sense
10 / 26

11. Summary
Making sure, that the system
meets the requirements that guided its design and development,
responds correctly to all kinds of inputs,
performs its functions within an acceptable time,
is sufficiently usable,
can be installed and run in its intended environments, and
achieves the general result its stakeholders desire.
https://en.wikipedia.org/wiki/Software_testing
11 / 26

12. Testing in Python - overview
12 / 26

13. It's pretty cool and easy to test in Python
13 / 26

14. Unittest
https://docs.python.org/3/library/unittest.html
originally inspired by JUnit (inspired by SUnit for Smalltalk, since 1998)
https://shebanator.com/2007/08/21/a-brief-history-of-test-frameworks/
Pros
Native framework
Testing methods
Supported everywhere
Auto discovery of test classes and functions
Cons
Classes
Testing methods
Complicated (for some people)
Lack of Group fixtures (specified environment for a whole group)
Still better to separate testing code from business logic
Unit-test oriented
14 / 26

15. import unittest
class TestStringMethods(unittest.TestCase):
def test_upper(self):
self.assertEqual('foo'.upper(), 'FOO')
def test_isupper(self):
self.assertTrue('FOO'.isupper())
self.assertFalse('Foo'.isupper())
def test_split(self):
s = 'hello world'
self.assertEqual(s.split(), ['hello', 'world'])
# check that s.split fails when the separator is not a string
with self.assertRaises(TypeError):
s.split(2)
if __name__ == '__main__':
unittest.main()
15 / 26

16. $ python my_unittest.py
.F.
======================================================================
FAIL: test_split (__main__.TestStringMethods)
----------------------------------------------------------------------
Traceback (most recent call last):
File "neco.py", line 14, in test_split
self.assertEqual(s.split(), ['hellox', 'world'])
AssertionError: Lists differ: ['hello', 'world'] != ['hellox', 'world']
First differing element 0:
'hello'
'hellox'
- ['hello', 'world']
+ ['hellox', 'world']
? +
----------------------------------------------------------------------
Ran 3 tests in 0.001s
FAILED (failures=1)
16 / 26

17. py.test
http://doc.pytest.org/en/latest/
Pros
Plugins
HTML output
Simpler and more straight-forward syntax
Auto discovery of test functions
Distributed
Cons
Not native
Can get complicated too (decorators)
17 / 26

18. def inc(x):
return x + 1
def test_answer():
assert inc(3) == 5
18 / 26

19. $ python3 -m pytest my_pytest.py
============================================ test session starts ============================================
platform linux -- Python 3.5.2+, pytest-3.0.6, py-1.4.32, pluggy-0.4.0
rootdir: /tmp, inifile:
collected 2 items
neco.py F.
================================================= FAILURES ==================================================
________________________________________________ test_answer ________________________________________________
def test_answer():
> assert inc(3) == 5
E assert 4 == 5
E + where 4 = inc(3)
neco.py:5: AssertionError
==================================== 1 failed, 1 passed in 0.03 seconds =====================================
19 / 26

20. Doctest
https://docs.python.org/3/library/doctest.html
Pros
Native
Tests are (can be) part of class and method documentation
Can be in separated text (e.g. README) files
Supported by Sphinx
Cons
Not suitable for larger testing systems
No fixtures
20 / 26

21. def unique_words(page):
''' Returns set of the unique words in list of lines of text
Example:
>>> from StringIO import StringIO
>>> fileText = """the cat sat on the mat
... the mat was ondur the cat
... one fish two fish red fish
... blue fish
... This fish has a yellow car
... This fish has a yellow star"""
>>> file = StringIO(fileText)
>>> page = file.readlines()
>>> words = unique_words(page)
>>> print sorted(list(words))
["This", "a", "blue", "car", "cat", "fish", "has", "mat",
"on", "ondur", "one", "red", "sat", "star", "the", "two",
"was", "yellow"]
>>>
'''
return set(word for line in page for word in line.split())
def _test():
import doctest
doctest.testmod()
if __name__ == "__main__":
_test()
21 / 26

22. $ python2 my_doctest.py
**********************************************************************
File "neco.py", line 16, in __main__.unique_words
Failed example:
print sorted(list(words))
Expected:
["This", "a", "blue", "car", "cat", "fish", "has", "mat",
"on", "ondur", "one", "red", "sat", "star", "the", "two",
"was", "yellow"]
Got:
['This', 'a', 'blue', 'car', 'cat', 'fish', 'has', 'mat', 'on', 'ondur', 'one', 'red', 'sat', 'star', 'the', 'two', 'was', 'yellow']
**********************************************************************
1 items had failures:
1 of 6 in __main__.unique_words
***Test Failed*** 1 failures.
22 / 26

23. nose (nose2)
nose extends unittest to make testing easier
smart developer should get familiar doctest, unittest, pytest, and nose
https://pypi.python.org/pypi/nose/ --> new projects should use py.test or
nose2
https://github.com/nose-devs/nose2
Pros
Build on top of unittests
Overpasses some of it's limits
Command line tool
Cons
nose not maintained any more --> nose2
Not native
Under development (nose2)
23 / 26

24. Python unittests overview
24 / 26

25. Conclusion
Doctests != py.test
Doctests are the natural way, how to test your code and how to document
it
Doctests should be used for documentation, not for testing
py.test seems to be current leading testing framework
Event /me knows, what is the difference between doctests and py.test
25 / 26

26. That's all folks
#jachymc
jachym.cepicky at opengeolabs dot cz
26 / 26