Django Testing
Eric Holscher
http://ericholscher.com
Tuesday, May 5, 2009 1

How do you know?
• First 4 months of my job was porting and testing
Ellington
• Going from Django r1290 to Django 1.0.
• Suite from 0 to 400 tests.
Tuesday, May 5, 2009 2

30,000 Ft View
• State of testing in Django
• Why you should be testing
• How you start testing
• Useful tools
• Eventual Goals
Tuesday, May 5, 2009 3

State of Django Testing
Tuesday, May 5, 2009 4

assertTrue('Hello World', community.testing.status)
Tuesday, May 5, 2009 5

Django 1.1
Making Testing Possible since 2009
Tuesday, May 5, 2009 6

manage.py startapp
creates a tests.py
Tuesday, May 5, 2009 7

from django.test import TestCase
class SimpleTest(TestCase):
def test_basic_addition(self):
\"\"\"
Tests that 1 + 1 always equals 2.
\"\"\"
self.failUnlessEqual(1 + 1, 2)
__test__ = {\"doctest\": \"\"\"
Another way to test that 1 + 1 is equal to 2.
>>> 1 + 1 == 2
True
\"\"\"}
Tuesday, May 5, 2009 8

Fast Tests
(Transactions)
Tuesday, May 5, 2009 9

Minutes (Lower is better)
Ellington Test Speedup
60
45
30
15
0
Django 1.0 Django 1.1
Tuesday, May 5, 2009 10

You now have no excuse.
Tuesday, May 5, 2009 11

Why to test
Tuesday, May 5, 2009 12

Scary
Tuesday, May 5, 2009 13

Less Scary
Tuesday, May 5, 2009 14

Not Scary
Tuesday, May 5, 2009 15

Peace of Mind
Tuesday, May 5, 2009 16

Code must adapt
Tuesday, May 5, 2009 17

“It is not the strongest of
the species that survives, nor
the most intelligent, but the
one most responsive to
change.”
- Charles Darwin
Tuesday, May 5, 2009 18

Won’t somebody please
think of the users?!
Tuesday, May 5, 2009 19

Tests as Documentation
Tuesday, May 5, 2009 20

Tests as Documentation
Test driven development
+
Document driven development
=
Test Driven Documentation
Tuesday, May 5, 2009 20

“Code without tests is
broken as designed”
- Jacob Kaplan-Moss
Tuesday, May 5, 2009 21

Dizzying Array of Testing
Options
Tuesday, May 5, 2009 22

What kind of test?
• doctest
• unittest
Tuesday, May 5, 2009 23

Doctests
• Inline documentation
• <Copy from terminal to test file>
• Can’t use PDB
• Hide real failures
• “Easy”
Tuesday, May 5, 2009 24

def parse_ttag(token, required_tags):
\"\"\"
A function to parse a template tag.
It sets the name of the tag to 'tag_name' in the hash returned.
>>> from test_utils.templatetags.utils import parse_ttag
>>> parse_ttag('super_cool_tag for my_object as obj', ['as'])
{'tag_name': u'super_cool_tag', u'as': u'obj'}
>>> parse_ttag('super_cool_tag for my_object as obj', ['as', 'for'])
{'tag_name': u'super_cool_tag', u'as': u'obj', u'for': u'my_object'}
\"\"\"
bits = token.split(' ')
tags = {'tag_name': bits.pop(0)}
for index, bit in enumerate(bits):
bit = bit.strip()
if bit in required_tags:
if len(bits) != index-1:
tags[bit] = bits[index+1]
return tags
Tuesday, May 5, 2009 25

Unit Tests
• Use for everything else
• More rubust
• setUp and tearDown
• Standard (XUnit)
Tuesday, May 5, 2009 26

import random
import unittest
class TestRandom(unittest.TestCase):
def setUp(self):
self.seq = range(10)
def testshuffle(self):
# make sure the shuffled sequence does not lose any elements
random.shuffle(self.seq)
self.seq.sort()
self.assertEqual(self.seq, range(10))
if __name__ == '__main__':
unittest.main()
Tuesday, May 5, 2009 27

Django TestCase
• Subclasses unittest
• Fixtures
• Assertions
• Mail
• URLs
Tuesday, May 5, 2009 28

Test Client
• Test HTTP Requests without server
• Test Views, Templates, and Context
Tuesday, May 5, 2009 29

from django.contrib.auth.models import User
from django.test import TestCase
from django.core import mail
class PasswordResetTest(TestCase):
fixtures = ['authtestdata.json']
urls = 'django.contrib.auth.urls'
def test_email_not_found(self):
\"Error is raised if the provided email address isn't currently registered\"
response = self.client.get('/password_reset/')
self.assertEquals(response.status_code, 200)
response = self.client.post('/password_reset/', {'email': 'not_a_real_email@email.com'})
self.assertContains(response, \"That e-mail address doesn&#39;t have an associated user
account\")
self.assertEquals(len(mail.outbox), 0)
Tuesday, May 5, 2009 30

What flavor of test?
• Unit
• Functional
• Browser
Tuesday, May 5, 2009 31

Unit test
• Low level tests
• Small, focused, exercising one bit of functionality
Tuesday, May 5, 2009 32

Regression test
• Written when you find a bug
• Proves bug was fixed
• Django Tickets
Tuesday, May 5, 2009 33

Functional
• “Black Box Testing”
• Check High Level Functionality
Tuesday, May 5, 2009 34

Functional Testing Tools
• Twill
• Django Test Client
• ...
Tuesday, May 5, 2009 35

Ellington
• Lots of different clients
• Need to test deployment (Functional)
Tuesday, May 5, 2009 36

go {{ site.url }}/marketplace/search/
formvalue 1 q pizza
submit
code 200
Tuesday, May 5, 2009 37

Tuesday, May 5, 2009 38

Pretty Functional Tests
Tuesday, May 5, 2009 39

Can test
• All sites on a server
• All sites of a certain type
• A single problemed client
• A test across all sites
Tuesday, May 5, 2009 40

Browser tests
• Run tests in a web browser
• Check compatibility of design
• Basically an IE sanity check
• Only real way to test JS, AJAX, CSS
Tuesday, May 5, 2009 41

Browser Testing Tools
• Windmill
• Selenium
Tuesday, May 5, 2009 42

Tuesday, May 5, 2009 43

Other kinds of testing
• Spiders
• Fuzz testing
• Load testing
• Prayer
Tuesday, May 5, 2009 44

Where do I start?
Tuesday, May 5, 2009 45

Fixed a bug
Tuesday, May 5, 2009 46

Poking at code on the
command line
Tuesday, May 5, 2009 47

Pony turned Horse
Tuesday, May 5, 2009 48

Now what?
Tuesday, May 5, 2009 49

Start with a regression
test or a functional test
Tuesday, May 5, 2009 50

Use unittest unless you
have a reason not to!
Tuesday, May 5, 2009 51

Use the data, Luke
• Use data as a pivot
• Fixtures means you use Unit Tests
• Creation on the command line, Doctests
Tuesday, May 5, 2009 52

Creating Fixtures
• ./manage.py dumpdata <app>
• ./manage.py makefixture Model[x:y]
• Follows relations
• By Hand
Tuesday, May 5, 2009 53

TestShell
./manage.py testshell <fixture>
Tuesday, May 5, 2009 54

Making Functional tests
• Usually a relatively annoying process
• Testmaker makes it easy.
• ./manage.py testmaker <app>
• Simply browse and your session is recorded.
Tuesday, May 5, 2009 55

Testing your views
generally gets you the
most coverage.
Tuesday, May 5, 2009 56

80% Case
Tuesday, May 5, 2009 57

When > Where
Tuesday, May 5, 2009 58

Tools
Tuesday, May 5, 2009 59

Coverage
Tuesday, May 5, 2009 60

Tuesday, May 5, 2009 61

Mock Objects
• http://www.voidspace.org.uk/python/mock/
Tuesday, May 5, 2009 62

import unittest
from mock import Mock
from templatetags.cms_tags import if_link_is_active, IsActiveNode
class TestIsActiveTag(unittest.TestCase):
def test_returns_correct_node_type(self):
token = Mock(methods=['split_contents'])
token.split_contents.return_value = ('if_link_is_active',
'bar')
self.assertEqual(type(if_link_is_active(Mock(), token)),
IsActiveNode)
Tuesday, May 5, 2009 63

Custom Test Runners
Tuesday, May 5, 2009 64

Django Test Extensions
• Gareth Rushgrove
• Extra Assertions
• Coverage and XML Test Runners
• http://github.com/garethr/django-test-extensions
Tuesday, May 5, 2009 65

Django Sane Testing
• Ella Folk, Lucas (Hi!)
• Based on nosetests
• Selenium
• Live server
• http://devel.almad.net/trac/django-sane-testing/
Tuesday, May 5, 2009 66

Django Test Utils
• Mine!
• Testmaker
• Crawler
• Random fanciness
• http://github.com/ericholscher/django-test-utils/tree/
master
Tuesday, May 5, 2009 67

Recording tests is
generally seen as bad.
Tuesday, May 5, 2009 68

My philosophy
• Write tests.
• Notice patterns and best practices
• Automate recording of tests with those patterns
• If you can’t automate, use tools to make it easier.
Tuesday, May 5, 2009 69

Process for testmaker
• Most view tests check status_code and response
context
• Write middleware that catches this info
• Records it to a file
• Err on the side of more data.
Tuesday, May 5, 2009 70

Goals
(That perfect world)
Tuesday, May 5, 2009 71

Some form of TDD
• Write tests as you write code
• Makes your code easy to test
Tuesday, May 5, 2009 72

Follow Django’s Model
• Tests with every commit
• Docs with every commit
• Run tests before commiting
Tuesday, May 5, 2009 73

Use a DVCS
• At work we have a central SVN repo
• Git feature branches
• Code is staged for documentation and testing
• Committed to SVN once it is “done”
Tuesday, May 5, 2009 74

Continuous Integration
Tuesday, May 5, 2009 75

NEVER LEAVE THE
BUILD BROKEN
Tuesday, May 5, 2009 76

Love Green
Tuesday, May 5, 2009 77

Fast(er) Tests
Tuesday, May 5, 2009 78

JSON
Tuesday, May 5, 2009 79

Profiling
python -m cProfile manage.py test
Tuesday, May 5, 2009 80

Mock Objects
http://www.voidspace.org.uk/python/mock/
Tuesday, May 5, 2009 81

Future and Ponies
Tuesday, May 5, 2009 82

Summer of Code
• Test-Only Models
• Coverage
• Windmill tests of the admin
Tuesday, May 5, 2009 83

Test Suites in Django
Tuesday, May 5, 2009 84

Central Testing
Repository
Tuesday, May 5, 2009 85

Nose Plugin Integration
Tuesday, May 5, 2009 86

Things to remember
• Testing is not hard, you just have to get started.
• If your code doesn’t have tests, it will be hard/
impossible to refactor
• Once you have tests, you need to run them!
Tuesday, May 5, 2009 87

Credits
• http://www.flickr.com/photos/tym/192416981/
• http://www.flickr.com/photos/seandreilinger/2459266781/
• http://www.homebabysafety.com/images/baby_crawl_suit.jpg
• http://www.flickr.com/photos/pinkypigs/960572985/
Tuesday, May 5, 2009 88