This document discusses various tools and techniques for testing Python code, including: - Unittest and doctest modules for writing and running tests - The mocker library for creating mock objects in tests - Nose for finding and running tests across files - Twill for testing web applications through a command line interface - Code coverage tools like Coverage for analyzing test coverage - Continuous integration tools like Buildbot for running tests automatically

1. PYTHON TESTING
John
Saturday, December 21, 2013

2. Type of testing
• Unit testing: Unit testing is testing of the
smallest possible pieces of a program. it's the
foundation upon which everything else is
based
• Integration testing: tests encompass
interactions between related units.
• System testing: system tests are an extreme
form of integration tests.

3. UNITTEST MODULE

4. unittest introduction
• the batteries-included test module standard
library.
• Similar usage as JUnit, nUnit, CppUnit series of
tools.

5. Unittest package
Import unittest
From unittest import TestCase,main
class two_failing_tests(TestCase):
def test_assertTrue(self):
self.assertTrue(1 == 1 + 1)
def test_assertEqual(self):
self.assertEqual(1, 1 + 1)
if __name__ == '__main__':
main()

6. Assert method
•
•
•
•
assertTrue: will succeed if expression is true
assertFalse: will succeed if expression is false
assertEqual, assertNotEqual
assertAlmostEqual: use this one compare
comparing floating point numbers. For example
3==3.00
• assertNotAlmostEqual
• assertRaises
• Final one: fail

7. PART 1:
INTRODUCE
DOCTEST MODULE

8. Doctest: the easiest Testing tool
• Doctest will be the mainstay of your testing
toolkit
• doctest tests are written in plain text.
• Doctest extracts the tests and ignores the
rest of the text
• So the tests can be embedded in humanreadable explanations or discussions.

9. creating and running your first
doctest
• Open a new text file in your editor, and name it test.txt.
• Insert the following text into the file:
This is a simple doctest that checks some of Python's arithmetic
operations.
>>> 2 + 2
4
>>> 3 * 3
10
• Run command: python -m doctest test.txt
• Or you can write a small python code file:
import doctest
doctest.testfile(“test.txt")
• You can simply add IDLE input and output as test here

10. Directives:
• +SKIP to skip one test
>>> 'This test would fail.' # doctest: +SKIP
• +ELLIPSIS: can use … match any substring in
the actual output
func(56, "hello") # doctest: +ELLIPSIS
<mocker.Mock object at ...>
• More directives see here

11. Embedding doctests in Python
docstrings
def testable(x):
r"""
The `testable` function returns the square root of its
parameter, or 3, whichever is larger.
>>> testable(7)
3.0
>>> testable(16)
4.0
>>> testable(9)
3.0
>>> testable(10) == 10 ** 0.5
True
"""
if x < 9:
return 3.0
return x ** 0.5
•Run command line: python ‑ m doctest ‑ v test.py
•If you want to run the test in code. Add cose like:
if __name__ == "__main__":
import doctest
doctest.testmod()

12. Tips
• Write your test before code development
• reload(module) if your module is changed.
reload array

13. PART 2: INTRO TO
PYTHON MOCKER
AND UNITTEST

14. Install mocker first
• pip install mocker
• Or python easy_install.py mocker
• Or download it by yourself
http://labix.org/mocker

15. What is Mock object?
• Mock objects imitate the real objects that
make up your program
• So we can decouple the multiplication class
from others

16. Step by step
1. Create the mocking context: mocker =
Mocker().
2. Create mocking object under this context: a =
mocker.mock()
3. demonstrate how you expect the mock objects
to be used :
func(56, "hello") # doctest: +ELLIPSIS
<mocker.Mock object at ...>
>>> mocker.result(11)
4. Here we expect func(56,”hello”) will output 11

17. 6. Replay mode: mocker.replay(). Once enter
this mode, the first time call
func(56,”hello”). It will return 11 instead of
call this func.
7. mocker.restore() : back the point not set
mock.
8. mocker.verify(): check that the actual usage
of the mocks was as expected. For example:
check if func(56,”hello”) is 11

18. Function test

19. Parameter name in Mock
• ANY: any single object.i.e.func(a)
• ARGS: any more arguments. i.e. func(a,b)
• KWARGS: any more keyword arguments i.e.
func(a=1,b=2)
• IS: func(7, IS(param)) # doctest: +ELLIPSIS
• IN:func(7, IN([45, 68, 19])) # doctest:
+ELLIPSIS

20. CONTAINS: if the list contain this
vlaue
>>> from mocker import Mocker, CONTAINS
>>> mocker = Mocker()
>>> func = mocker.mock()
>>> func(7, CONTAINS(45)) # doctest: +ELLIPSIS
<mocker.Mock object at ...>
>>> mocker.result(5)
>>> mocker.replay()
>>> func(7, [12, 31, 45, 18])
5
>>> mocker.restore()
>>> mocker.verify()

21. MATCH: if match,it return true
>>> from mocker import Mocker, MATCH
>>> def is_odd(val):
... return val % 2 == 1
>>> mocker = Mocker()
>>> func = mocker.mock()
>>> func(7, MATCH(is_odd)) # doctest: +ELLIPSIS
<mocker.Mock object at ...>
>>> mocker.result(5)
>>> mocker.replay()
>>> func(7, 1001)
5
>>> mocker.restore()
>>> mocker.verify()

22. mocker.count
• Mocker.count to specify the expected
number of repetitions
• count(3): repeat 3 times
• count(1,3): repeat times is between 1 and 3.
• count(1,None): repeat at least 1 time, no
maximum.

23. Package test
• Use replace to temporarily replace the lib
from time import time
>>> from mocker import Mocker
>>> mocker = Mocker()
>>> mock_time = mocker.replace('time.time')
>>> mock_time() # doctest: +ELLIPSIS
<mocker.Mock object at ...>
>>> mocker.result(1.3)
>>> mocker.replay()
>>> '%1.3g' % time()
'1.3‘
>>> mocker.restore()
>>> mocker.verify()

24. Class test
• Let self be a mock object
>>> from testable import testable
>>> from mocker import Mocker
>>> mocker = Mocker()
>>> target = mocker.mock()
>>> target.method1(12) # doctest: +ELLIPSIS
<mocker.Mock object at ...>
>>> mocker.result(5)
>>> target.method2(12) # doctest: +ELLIPSIS
<mocker.Mock object at ...>
>>> mocker.result(7)

25. PART 3: PYTHON
TOOL NOSE

26. What is nose?
•
•
•
•
A tool for finding and running all of your tests
presents with a nice report after tests finish.
Nose understands doctest and unittest tests
Install nose from PYPI (use easy_install or
pip)
• After install it, run command line: nosetests

27. Recommendation how to organize
the source code folder
• Nose recognizes test files based on their names
– Any file whose name
contains test or Test contain
unittest TestCases
– Nose find doctest tests either
embedded in docstrings or
separate test files
• Reorgnize your folder. Change
to the dir. Run command line:
nosetests --with-doctest
--doctest-extension=txt -v

28. Options for nosetests
• Create a configuration file called nose.cfg or
.noserc in $HOME dir
(For windows, the $HOME means C:Usersusers)
• Place following inside it:
[nosetests]
with-doctest=1
doctest-extension=txt
include="(?:^[Dd]oc)“
• Run nosetests –v.

29. PART 4: TESTING
WEB APPLICATION
FRONTENDS USING
TWILL

30. Brief intro
• twill allows users to browse the Web from a
command-line interface.
• twill supports automated Web testing
• Twill has a simple Python interface.
• Use pip install twill package first
• Use command: twill-sh to start the shell

31. A simple example
• Create slashdot.twill file contain code:
code 200
follow Science
code 200
formvalue 2 fhfilter "aardvark"
submit
code 200
find aardvark code 200
follow Science
code 200
formvalue 2 fhfilter "aardvark"
submit
code 200
find aardvark
• Run commandl line: twill-sh -u http://slashdot.org/
slashdot.twill

32. You can also test interactively
•
•
•
•
>> go http://slashdot.org/
>> code 200
>> follow Science
….

33. Use twill in python
Import twill
browser = twill.get_browser()
Browser methods include: go,
reload,back,get_code,get_html,get_title,get_ur
l,find_link,follow_link,set_agent_string,get_all_
forms,get_form,get_form_field,clicked,submit,
save_cookies,load_cookies,clear_cookies

34. PART 5: OTHER
TESTING TOOLS
AND TECHNIQUES

35. Code coverage
• A code coverage keeps track of which lines of
code are (and aren't) executed while tests
are running
• Give you a report describing how well your
tests cover the whole body of code
Attention: Code coverage is a tool to give you insight
into what your tests are doing, and what they may
be overlooking. It's not the definition of a good test
suite.

36. 1. Install PYPI package coverage (pip install
coverage)
2. When run your nose. Use option –withcoverage –-cover-erage
3. From the example, we saw line 16,19-20 of
toy.py do not executes.

37. Automated continuous integration
• automated continuous integration system
compiles your code (if need be) and runs
your tests many times, in many different
environments.
• Buildbot is a popular automated continuous
integration tool.

38. Reference
• Book << Python Testing:Beginner's Guide>>