Talk about where Testing is in Django, and how to get started doing it.

1. Django Testing
Eric Holscher
http://ericholscher.com
Tuesday, May 5, 2009 1

2. How do you know?
• First 4 months of my job was porting and testing
Ellington
• Going from Django r1290 to Django 1.0.
• Suite from 0 to 400 tests.
Tuesday, May 5, 2009 2

3. 30,000 Ft View
• State of testing in Django
• Why you should be testing
• How you start testing
• Useful tools
• Eventual Goals
Tuesday, May 5, 2009 3

4. State of Django Testing
Tuesday, May 5, 2009 4

5. assertTrue('Hello World', community.testing.status)
Tuesday, May 5, 2009 5

6. Django 1.1
Making Testing Possible since 2009
Tuesday, May 5, 2009 6

7. manage.py startapp
creates a tests.py
Tuesday, May 5, 2009 7

8. from django.test import TestCase
class SimpleTest(TestCase):
def test_basic_addition(self):
quot;quot;quot;
Tests that 1 + 1 always equals 2.
quot;quot;quot;
self.failUnlessEqual(1 + 1, 2)
__test__ = {quot;doctestquot;: quot;quot;quot;
Another way to test that 1 + 1 is equal to 2.
>>> 1 + 1 == 2
True
quot;quot;quot;}
Tuesday, May 5, 2009 8

9. Fast Tests
(Transactions)
Tuesday, May 5, 2009 9

10. Minutes (Lower is better)
Ellington Test Speedup
60
45
30
15
0
Django 1.0 Django 1.1
Tuesday, May 5, 2009 10

11. You now have no excuse.
Tuesday, May 5, 2009 11

12. Why to test
Tuesday, May 5, 2009 12

13. Scary
Tuesday, May 5, 2009 13

14. Less Scary
Tuesday, May 5, 2009 14

15. Not Scary
Tuesday, May 5, 2009 15

16. Peace of Mind
Tuesday, May 5, 2009 16

17. Code must adapt
Tuesday, May 5, 2009 17

18. “It is not the strongest of
the species that survives, nor
the most intelligent, but the
one most responsive to
change.”
- Charles Darwin
Tuesday, May 5, 2009 18

19. Won’t somebody please
think of the users?!
Tuesday, May 5, 2009 19

20. Tests as Documentation
Tuesday, May 5, 2009 20

21. Tests as Documentation
Test driven development
+
Document driven development
=
Test Driven Documentation
Tuesday, May 5, 2009 20

22. “Code without tests is
broken as designed”
- Jacob Kaplan-Moss
Tuesday, May 5, 2009 21

23. Dizzying Array of Testing
Options
Tuesday, May 5, 2009 22

24. What kind of test?
• doctest
• unittest
Tuesday, May 5, 2009 23

25. Doctests
• Inline documentation
• <Copy from terminal to test file>
• Can’t use PDB
• Hide real failures
• “Easy”
Tuesday, May 5, 2009 24

26. def parse_ttag(token, required_tags):
quot;quot;quot;
A function to parse a template tag.
It sets the name of the tag to 'tag_name' in the hash returned.
>>> from test_utils.templatetags.utils import parse_ttag
>>> parse_ttag('super_cool_tag for my_object as obj', ['as'])
{'tag_name': u'super_cool_tag', u'as': u'obj'}
>>> parse_ttag('super_cool_tag for my_object as obj', ['as', 'for'])
{'tag_name': u'super_cool_tag', u'as': u'obj', u'for': u'my_object'}
quot;quot;quot;
bits = token.split(' ')
tags = {'tag_name': bits.pop(0)}
for index, bit in enumerate(bits):
bit = bit.strip()
if bit in required_tags:
if len(bits) != index-1:
tags[bit] = bits[index+1]
return tags
Tuesday, May 5, 2009 25

27. Unit Tests
• Use for everything else
• More rubust
• setUp and tearDown
• Standard (XUnit)
Tuesday, May 5, 2009 26

28. import random
import unittest
class TestRandom(unittest.TestCase):
def setUp(self):
self.seq = range(10)
def testshuffle(self):
# make sure the shuffled sequence does not lose any elements
random.shuffle(self.seq)
self.seq.sort()
self.assertEqual(self.seq, range(10))
if __name__ == '__main__':
unittest.main()
Tuesday, May 5, 2009 27

29. Django TestCase
• Subclasses unittest
• Fixtures
• Assertions
• Mail
• URLs
Tuesday, May 5, 2009 28

30. Test Client
• Test HTTP Requests without server
• Test Views, Templates, and Context
Tuesday, May 5, 2009 29

31. from django.contrib.auth.models import User
from django.test import TestCase
from django.core import mail
class PasswordResetTest(TestCase):
fixtures = ['authtestdata.json']
urls = 'django.contrib.auth.urls'
def test_email_not_found(self):
quot;Error is raised if the provided email address isn't currently registeredquot;
response = self.client.get('/password_reset/')
self.assertEquals(response.status_code, 200)
response = self.client.post('/password_reset/', {'email': 'not_a_real_email@email.com'})
self.assertContains(response, quot;That e-mail address doesn&#39;t have an associated user
accountquot;)
self.assertEquals(len(mail.outbox), 0)
Tuesday, May 5, 2009 30

32. What flavor of test?
• Unit
• Functional
• Browser
Tuesday, May 5, 2009 31

33. Unit test
• Low level tests
• Small, focused, exercising one bit of functionality
Tuesday, May 5, 2009 32

34. Regression test
• Written when you find a bug
• Proves bug was fixed
• Django Tickets
Tuesday, May 5, 2009 33

35. Functional
• “Black Box Testing”
• Check High Level Functionality
Tuesday, May 5, 2009 34

36. Functional Testing Tools
• Twill
• Django Test Client
• ...
Tuesday, May 5, 2009 35

37. Ellington
• Lots of different clients
• Need to test deployment (Functional)
Tuesday, May 5, 2009 36

38. go {{ site.url }}/marketplace/search/
formvalue 1 q pizza
submit
code 200
Tuesday, May 5, 2009 37

39. Tuesday, May 5, 2009 38

40. Pretty Functional Tests
Tuesday, May 5, 2009 39

41. Can test
• All sites on a server
• All sites of a certain type
• A single problemed client
• A test across all sites
Tuesday, May 5, 2009 40

42. Browser tests
• Run tests in a web browser
• Check compatibility of design
• Basically an IE sanity check
• Only real way to test JS, AJAX, CSS
Tuesday, May 5, 2009 41

43. Browser Testing Tools
• Windmill
• Selenium
Tuesday, May 5, 2009 42

44. Tuesday, May 5, 2009 43

45. Other kinds of testing
• Spiders
• Fuzz testing
• Load testing
• Prayer
Tuesday, May 5, 2009 44

46. Where do I start?
Tuesday, May 5, 2009 45

47. Fixed a bug
Tuesday, May 5, 2009 46

48. Poking at code on the
command line
Tuesday, May 5, 2009 47

49. Pony turned Horse
Tuesday, May 5, 2009 48

50. Now what?
Tuesday, May 5, 2009 49

51. Start with a regression
test or a functional test
Tuesday, May 5, 2009 50

52. Use unittest unless you
have a reason not to!
Tuesday, May 5, 2009 51

53. Use the data, Luke
• Use data as a pivot
• Fixtures means you use Unit Tests
• Creation on the command line, Doctests
Tuesday, May 5, 2009 52

54. Creating Fixtures
• ./manage.py dumpdata <app>
• ./manage.py makefixture Model[x:y]
• Follows relations
• By Hand
Tuesday, May 5, 2009 53

55. TestShell
./manage.py testshell <fixture>
Tuesday, May 5, 2009 54

56. Making Functional tests
• Usually a relatively annoying process
• Testmaker makes it easy.
• ./manage.py testmaker <app>
• Simply browse and your session is recorded.
Tuesday, May 5, 2009 55

57. Testing your views
generally gets you the
most coverage.
Tuesday, May 5, 2009 56

58. 80% Case
Tuesday, May 5, 2009 57

59. When > Where
Tuesday, May 5, 2009 58

60. Tools
Tuesday, May 5, 2009 59

61. Coverage
Tuesday, May 5, 2009 60

62. Tuesday, May 5, 2009 61

63. Mock Objects
• http://www.voidspace.org.uk/python/mock/
Tuesday, May 5, 2009 62

64. import unittest
from mock import Mock
from templatetags.cms_tags import if_link_is_active, IsActiveNode
class TestIsActiveTag(unittest.TestCase):
def test_returns_correct_node_type(self):
token = Mock(methods=['split_contents'])
token.split_contents.return_value = ('if_link_is_active',
'bar')
self.assertEqual(type(if_link_is_active(Mock(), token)),
IsActiveNode)
Tuesday, May 5, 2009 63

65. Custom Test Runners
Tuesday, May 5, 2009 64

66. Django Test Extensions
• Gareth Rushgrove
• Extra Assertions
• Coverage and XML Test Runners
• http://github.com/garethr/django-test-extensions
Tuesday, May 5, 2009 65

67. Django Sane Testing
• Ella Folk, Lucas (Hi!)
• Based on nosetests
• Selenium
• Live server
• http://devel.almad.net/trac/django-sane-testing/
Tuesday, May 5, 2009 66

68. Django Test Utils
• Mine!
• Testmaker
• Crawler
• Random fanciness
• http://github.com/ericholscher/django-test-utils/tree/
master
Tuesday, May 5, 2009 67

69. Recording tests is
generally seen as bad.
Tuesday, May 5, 2009 68

70. My philosophy
• Write tests.
• Notice patterns and best practices
• Automate recording of tests with those patterns
• If you can’t automate, use tools to make it easier.
Tuesday, May 5, 2009 69

71. Process for testmaker
• Most view tests check status_code and response
context
• Write middleware that catches this info
• Records it to a file
• Err on the side of more data.
Tuesday, May 5, 2009 70

72. Goals
(That perfect world)
Tuesday, May 5, 2009 71

73. Some form of TDD
• Write tests as you write code
• Makes your code easy to test
Tuesday, May 5, 2009 72

74. Follow Django’s Model
• Tests with every commit
• Docs with every commit
• Run tests before commiting
Tuesday, May 5, 2009 73

75. Use a DVCS
• At work we have a central SVN repo
• Git feature branches
• Code is staged for documentation and testing
• Committed to SVN once it is “done”
Tuesday, May 5, 2009 74

76. Continuous Integration
Tuesday, May 5, 2009 75

77. NEVER LEAVE THE
BUILD BROKEN
Tuesday, May 5, 2009 76

78. Love Green
Tuesday, May 5, 2009 77

79. Fast(er) Tests
Tuesday, May 5, 2009 78

80. JSON
Tuesday, May 5, 2009 79

81. Profiling
python -m cProfile manage.py test
Tuesday, May 5, 2009 80

82. Mock Objects
http://www.voidspace.org.uk/python/mock/
Tuesday, May 5, 2009 81

83. Future and Ponies
Tuesday, May 5, 2009 82

84. Summer of Code
• Test-Only Models
• Coverage
• Windmill tests of the admin
Tuesday, May 5, 2009 83

85. Test Suites in Django
Tuesday, May 5, 2009 84

86. Central Testing
Repository
Tuesday, May 5, 2009 85

87. Nose Plugin Integration
Tuesday, May 5, 2009 86

88. Things to remember
• Testing is not hard, you just have to get started.
• If your code doesn’t have tests, it will be hard/
impossible to refactor
• Once you have tests, you need to run them!
Tuesday, May 5, 2009 87

89. Credits
• http://www.flickr.com/photos/tym/192416981/
• http://www.flickr.com/photos/seandreilinger/2459266781/
• http://www.homebabysafety.com/images/baby_crawl_suit.jpg
• http://www.flickr.com/photos/pinkypigs/960572985/
Tuesday, May 5, 2009 88