Writing tests                                       Jonathan Fine                                  LTS, The Open Universit...
Pure functionsA pure function is one that      has no side effects.      always gives same output for same input.A pure fun...
Example: Split a string into piecesSuppose we’re writing a wiki language parser, a syntax highlighter, orsomething similar...
Writing tests with doctestPython’s interactive console is very useful for exploring and learning. Thedoctest module, part ...
Writing tests with unittestThe unittest module, part of the standard library, is modelled on Javaand Smalltalk testing fra...
Writing tests with pytestThe pytest package was developed to meet the needs of the PyPy project.Around 2010 there was a ma...
Writing tests with noseThe nose package seems to be a rewrite of pytest to meet magic (andother?) concerns. It’s available...
Aside: What’s all this about magic?To avoid verbosity, pytest writes its tests using assertions. This is finewhen the test ...
Splitting a string into pieces has a special propertyEnough of the assert stuff! It’s a distraction. Our problem is to writ...
Creating a sequence of strings from test dataRecall that to write a test all we have to do is specify the ouput. Wecan get...
Writing the testsWriting the tests is now easy. Write down a string and figure out wherethe splits should go. This gives yo...
Running the testsfrom testdata import TEST_DATAimport re; fn = re.compile(r’’’( +)’’’).splitdef doit(item):               ...
Porting the tests to unittest, pytest and noseThis file works with unittest, pytest and nose.import unittestfrom as_before ...
There’s something I forgot to tell you!Modern web applications are complex. Some code runs in the browserand other on the ...
Conclusions and Recommendations (for pure functions)Conclusions:      Pure functions are easier to test.      Functions th...
Upcoming SlideShare
Loading in...5
×

Writing tests

187

Published on

Slides for UK PyCon, September 2012

Published in: Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
187
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Writing tests

  1. 1. Writing tests Jonathan Fine LTS, The Open University Milton Keynes, UK Jonathan.Fine@open.ac.uk http://www.slideshare.net/jonathanfine http://jonathanfine.wordpress.com https://bitbucket.org/jfine 29 September 2012Jonathan Fine (Open University) Writing tests 29 September 2012 1 / 15
  2. 2. Pure functionsA pure function is one that has no side effects. always gives same output for same input.A pure function is one that can be safely cached, with no change exceptto performance.(The two hardest problems in computer science are naming things andcache invalidation.)Side-effects, especially changing global variables, make a program hard totest and hard to understand.The programmer who write a pure function should be rewarded for thiswhen it comes to testing. Jonathan Fine (Open University) Writing tests 29 September 2012 2 / 15
  3. 3. Example: Split a string into piecesSuppose we’re writing a wiki language parser, a syntax highlighter, orsomething similar. We’ll have to split the input string(s) into pieces.Writing the parser is not our present problem.Our problem here is to write the tests for the function that splitsthe string into pieces.For simplicity, we assume that we want to split the string on repeatedwhite space. We can write such a splitter using a regular expression.>>> import re>>> fn = re.compile(r’’’( +)’’’).split>>> fn(’this and that ’)[’this’, ’ ’, ’and’, ’ ’, ’that’, ’ ’]In fact, our task now is to make it easier to write tests for fn (whichin reality will be much more complicated than the above). Jonathan Fine (Open University) Writing tests 29 September 2012 3 / 15
  4. 4. Writing tests with doctestPython’s interactive console is very useful for exploring and learning. Thedoctest module, part of Python’s standard library, allows consolesessions to be used as tests. Here’s some examples.>>> import re; fn = re.compile(r’’’( +)’’’).split>>>>>> fn(’a b c’) # As we’d expect.[’a’, ’ ’, ’b’, ’ ’, ’c’]>>>>>> fn(’atb c’) # Tabs not counted as white space.[’atb’, ’ ’, ’c’]>>>>>> fn(’ ’) # Is this what we want?[’’, ’ ’, ’’]Good for examples, but what if we have 20 input-output pairs to test? Jonathan Fine (Open University) Writing tests 29 September 2012 4 / 15
  5. 5. Writing tests with unittestThe unittest module, part of the standard library, is modelled on Javaand Smalltalk testing frameworks. It’s a bit verbose.import fn # The function we want to test.class TestSplitter(unittest.TestCase): # Container. def test_1(self): arg = ’a b c’ expect = [’a’, ’ ’, ’b’, ’ ’, ’c’] actual = fn(arg) # Boilerplate. self.assertEqual(actual, expect) # Boilerplate.Problems: (1) It’s verbose (but why?). (2) Every test is a function.(3) arg and expect are not parameters to the test. (4) Nor is fn.(5) We have to give every test a name (even if we don’t want to).(6) We can’t easily loop over (arg, expect) pairs. Jonathan Fine (Open University) Writing tests 29 September 2012 5 / 15
  6. 6. Writing tests with pytestThe pytest package was developed to meet the needs of the PyPy project.Around 2010 there was a major cleanup refactoring. It’s available on PyPI.import fn # The function we want to test.def test_1(): arg = ’a b c’ expect = [’a’, ’ ’, ’b’, ’ ’, ’c’] actual = fn(arg) # Boilerplate. assert actual == expect # Boilerplate.Problems: (1) It’s less but still verbose . (2) Every test is a function.(3) arg and expect are not parameters to the test. (4) Nor is fn.(5) We have to give every test a name (even if we don’t want to).(6) We’re not looping over (arg, expect) pairs.(7) (Not shown): it used to use magic for error reporting.(8) (Not shown): it now uses rewrite of assert for this. Jonathan Fine (Open University) Writing tests 29 September 2012 6 / 15
  7. 7. Writing tests with noseThe nose package seems to be a rewrite of pytest to meet magic (andother?) concerns. It’s available on PyPI. Many tests will work for bothpackages. Our example (and most of the problems) are the same as before.import fn # The function we want to test.def test_1(): arg = ’a b c’ expect = [’a’, ’ ’, ’b’, ’ ’, ’c’] actual = fn(arg) # Boilerplate. assert actual == expect # Boilerplate.Problems: (1) It’s less but still verbose . (2) Every test is a function.(3) arg and expect are not parameters to the test. (4) Nor is fn.(5) We have to give every test a name (even if we don’t want to).(6) We’re not looping over (arg, expect) pairs.(7) (Not shown): it has opaque error reporting. Jonathan Fine (Open University) Writing tests 29 September 2012 7 / 15
  8. 8. Aside: What’s all this about magic?To avoid verbosity, pytest writes its tests using assertions. This is finewhen the test passes, but when it fails we want to know what went wrong.The problem is that in assert a == bthe values of a and b are lost when the AssertionError is raised.A non-magic workaround is to use assert a == b, get_traceback()but this goes against conciseness and was not used.Earlier versions of pytest reran failing tests in a special way to obtain atraceback, from which it produces a useful error message. Current versionsof pytest “explicitly rewrite assert statements in test modules”.It seems to me that the developers of nose liked the conciseness of pytestbut did not like this magic. So they wrote a non-magic variant of pytest. Jonathan Fine (Open University) Writing tests 29 September 2012 8 / 15
  9. 9. Splitting a string into pieces has a special propertyEnough of the assert stuff! It’s a distraction. Our problem is to writetests for a function that splits a string into pieces. In our example(below) no pieces are lost.>>> import re; fn = re.compile(r’’’( +)’’’).split>>> fn(’a b c’) # As we’d expect.[’a’, ’ ’, ’b’, ’ ’, ’c’]The input string is the join of the output (below). So all we have todo is specify the output (and join it to get the input).>>> arg = ’a b c’>>> actual = fn(arg)>>> ’’.join(actual) == argTrueLet’s use this special property to help write our tests! Jonathan Fine (Open University) Writing tests 29 September 2012 9 / 15
  10. 10. Creating a sequence of strings from test dataRecall that to write a test all we have to do is specify the ouput. Wecan get the input as the join of the output.Q: What’s an easy way to specify a sequence of strings?A: How about splitting on a character?>>> import re; fn = re.compile(r’’’( +)’’’).split>>>>>> test_data = ’’’a| |b| |c’’’>>> expect = test_data.split(’|’)>>> arg = ’’.join(expect)>>> arg’a b c’>>> expect[’a’, ’ ’, ’b’, ’ ’, ’c’]>>> fn(arg) == expectTrue Jonathan Fine (Open University) Writing tests 29 September 2012 10 / 15
  11. 11. Writing the testsWriting the tests is now easy. Write down a string and figure out wherethe splits should go. This gives you a test.TEST_DATA = ( # We use | to show where the splits go. ’’, ’a’, ’ab’, ’| |’, ’a| |’, ’| |a’, ’a| |b| |c’, ’a| |b| |c’, )What could be simpler? Something like this will work for anystring-splitting function (provided ‘|’ is not a special character). Jonathan Fine (Open University) Writing tests 29 September 2012 11 / 15
  12. 12. Running the testsfrom testdata import TEST_DATAimport re; fn = re.compile(r’’’( +)’’’).splitdef doit(item): # Split and join raw data. expect = item.split(’|’) arg = ’’.join(expect) return arg, expectTESTS = tuple(map(doit, TEST_DATA)) # Put into normal form.def runtest(test, fn): # Generic item runner. arg, expect = test actual = fn(arg) if actual != expect: # If equal return None. return expect, actualfor test in TESTS: # Generic multiple runner. outcome = runtest(test, fn) if outcome is not None: # Report failures. print ’Expect %snActual%s’ % outcome Jonathan Fine (Open University) Writing tests 29 September 2012 12 / 15
  13. 13. Porting the tests to unittest, pytest and noseThis file works with unittest, pytest and nose.import unittestfrom as_before import TESTS, fn# Use TEST to hide this function from nose!def TEST_factory(fn, arg, expect): def test_anon(self): # Using a closure. actual = fn(arg) self.assertEqual(actual, expect) return test_anon # Return inner function.# Create a class without using a class statement.TestSplitter = type( ’TestSplitter’, (unittest.TestCase,), dict( (’test_%s’ % i, TEST_factory(fn, *test)) for i, test in enumerate(TESTS) ))if __name__ == ’__main__’: unittest.main() Jonathan Fine (Open University) Writing tests 29 September 2012 13 / 15
  14. 14. There’s something I forgot to tell you!Modern web applications are complex. Some code runs in the browserand other on the server. Validation code might be run on both browserand server.Web frameworks based on nodejs have a big advantage here, because youcan write code without knowing the architecture of the application.We’re writing tests for a parser. I forgot to tell you that the parser willbe running in the browser (JavaScript) and on the server (we’ve not madeour mind up yet — what do you suggest?).Writing language neutral test data is a big advantage in today’s andtomorrow’s environment.To paraphrase Nikolaus Wirth Data Structures + Algorithms = Tests Jonathan Fine (Open University) Writing tests 29 September 2012 14 / 15
  15. 15. Conclusions and Recommendations (for pure functions)Conclusions: Pure functions are easier to test. Functions that satisfy ‘mathematical’ conditions are easier to test. It’s good to write tests that are programming language neutral. JSON is good.Recommendations: Write pure functions when you can. Write functions that are easy to write tests for. Write custom code that creates unittest tests. Put all together so it runs in unittest, pytest and nose.Finally: Encourage work for testing classes, objects etc. Jonathan Fine (Open University) Writing tests 29 September 2012 15 / 15
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×