A rehash of my previous Django testing talk with a bit of content added.

1. Token Testing Talk
Eric Holscher
http://ericholscher.com
Djangocon 2009

2. Testing Talks per Conference
10.0
7.5
5.0
2.5
0
Pycon RailsConf DjangoCon

3. How do you know?
» First 4 months of my job was porting and
testing Ellington
» Going from Django r1290 to Django 1.0.
» Suite from 0 to 400 tests. (Now has 1715)

4. 30,000 Ft View
» State of testing in Django
» Why you should be testing
» How you start testing
» Useful tools
» Eventual Goals

5. State of Django Testing

6. assertTrue('Hello World',
community.testing.status)

7. Django 1.1
Making Testing Possible since 2009

8. manage.py startapp creates a
tests.py

9. Basic Test
from django.test import TestCase
class SimpleTest(TestCase):
def test_basic_addition(self):
"""
Tests that 1 + 1 always equals 2.
"""
self.failUnlessEqual(1 + 1, 2)
__test__ = {"doctest": """
Another way to test that 1 + 1 is equal to 2.
>>> 1 + 1 == 2
True
"""}

10. Fast Tests
(Transactions)

11. Minutes (Lower is better)
Ellington Test Speedup
60
45
30
15
0
Django 1.0 Django 1.1

12. You now have no
excuse.

13. Why to test

14. Scary

15. Less Scary

16. Not Scary

17. Peace of Mind

18. Code must adapt

19. “It is not the strongest of
the species that survives, nor
the most intelligent, but the
one most responsive to
change.
- Charles Darwin

20. Won’t somebody please think of
the users?!

21. Tests as Documentation

22. Tests as Documentation
Test driven development
+
Document driven development
=
Test Driven Documentation

23. “Code without tests is
broken as designed
- Jacob Kaplan-Moss

24. Dizzying Array of Testing
Options

25. What kind of test?
» doctest
» unittest

26. Doctests
» Inline documentation
» <Copy from terminal to test file>
» Easy

27. Awesome Documentation
def parse_ttag(token, required_tags):
"""
A function to parse a template tag.
It sets the name of the tag to 'tag_name' in the hash returned.
>>> from test_utils.templatetags.utils import parse_ttag
>>> parse_ttag('super_cool_tag for my_object as obj', ['as'])
{'tag_name': u'super_cool_tag', u'as': u'obj'}
>>> parse_ttag('super_cool_tag for my_object as obj', ['as', 'for'])
{'tag_name': u'super_cool_tag', u'as': u'obj', u'for': u'my_object'}
"""
bits = token.split(' ')
tags = {'tag_name': bits.pop(0)}
for index, bit in enumerate(bits):
bit = bit.strip()
if bit in required_tags:
if len(bits) != index-1:
tags[bit] = bits[index+1]
return tags

28. Awesome Documentation
>>> from test_utils.templatetags.utils import parse_ttag
>>> parse_ttag('super_cool_tag for my_object as obj', ['as'])
{'tag_name': u'super_cool_tag', u'as': u'obj'}
>>> parse_ttag('super_sweet for object as obj', ['as', 'for'])
{'tag_name': u'super_sweet', u'as': u'obj', u'for': u'object'}

29. Doctest problems
» Can’t use PDB
» Hide real failures

30. Unit Tests
» Standard (XUnit)
» More robust
» setUp and tearDown

31. Basic Unit Test
import random
import unittest
class TestRandom(unittest.TestCase):
def setUp(self):
self.seq = range(10)
def testshuffle(self):
# shuffled sequence does not lose any elements
random.shuffle(self.seq)
self.seq.sort()
self.assertEqual(self.seq, range(10))
if __name__ == '__main__':
unittest.main()

32. Django TestCase
» Subclasses unittest
» Fixtures
» Assertions
» Mail
» URLs

33. Test Client
» Test HTTP Requests without server
» Test Views, Templates, and Context

34. Django’s TestCase
from django.contrib.auth.models import User
from django.test import TestCase
from django.core import mail
class PasswordResetTest(TestCase):
fixtures = ['authtestdata.json']
urls = 'django.contrib.auth.urls'
def test_email_not_found(self):
"Error is raised if the provided email address isn't currently
registered"
response = self.client.post('/password_reset/', {'email':
'not_a_real_email@email.com'})
self.assertEquals(len(mail.outbox), 0)

35. What flavor of test?
» Unit
» Functional
» Browser

36. Unit test
» Low level tests
» Small, focused, exercising one bit of
functionality
» Great for libraries

37. Regression test
» Written when you find a bug
» Proves bug was fixed
» Django Tickets

38. Functional
» ‘Black Box Testing’
» Check High Level Functionality

39. Functional Testing Tools
» Twill
» Django Test Client

40. Browser tests
» Run tests in a web browser
» Check compatibility of design
» Basically an IE sanity check
» Only real way to test JS, AJAX, CSS
» Slow

41. Browser Testing Tools
» Windmill
» Selenium

43. Other kinds of testing
» Spiders
» Fuzz testing
» Load testing

44. Where do I start?

45. Use unittest unless you have a
reason not to!

46. Start with a regression test or a
functional test

47. Fixed a bug

48. Poking at code on the command
line

49. Pony turned Horse

50. Use the data, Luke
» Use data as a pivot
» Fixtures means you use Unit Tests
» Creation on the command line, Doctests

51. Creating Fixtures
»./manage.py dumpdata <app>
»./manage.py makefixture Model[x:y]
» Follows relations
» Slicing
»By Hand

52. Getting in Context
» ./manage.py testshell <fixture>
» (i)pdb

53. Making Functional tests
» Usually a relatively annoying process
» Testmaker makes it easy.
» ./manage.py testmaker [app]
» Simply browse and your session is
recorded.

54. Testing your views gets you the
most coverage.

55. 80% Case

56. When > Where

57. Enough to be useful

58. Tools

59. Summer of Code
» Test-Only Models
» Skipping tests
» Coverage
» Windmill tests of the admin

60. Test-Only Models
» Allow for models available only during
tests
» Don’t need test_project now!

61. Test-Only Models
#test_models.py
from django.db import models
class TestModel(models.Model):
name = models.CharField()
#tests.py
from django.test import TestCase
class TestMyViews(TestCase):
test_models = ['test_models']
def testIndexPageView(self):
from myapp.models import TestModel
TestModel.objects.get(name='daniellindsleyrocksdahouse')

62. Skipping Tests
» Views Required
» Models Required
» Specific database type
» Conditional on a Function

63. Skipping Tests
from django.tests.decorators import conditional_skip
import datetime
class TestUnderCondition(TestCase):
def _check_2009():
# Condition returning True if test should be run and False if it
# should be skipped.
if datetime.datetime.now() > datetime.datetime(2009, 01, 01):
return True
@conditional_skip(_check_2009, reason='This test only runs in 2009')
def testOnlyIn2009(self):
# Test to run if _my_condition evaluates to True

64. Skipping Tests
======================================================================
SKIPPED: test_email_found (django.contrib.auth.tests.basic.PasswordResetTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dnaquin/Dropbox/Sandbox/django/django/test/decorators.py", line 43, in _skip
raise SkippedTest(reason=reason)
SkippedTest: Required view for this test not found: django.contrib.auth.views.password_reset
----------------------------------------------------------------------
Ran 408 tests in 339.663s
FAILED (failures=1, skipped=2)

65. Coverage
» Basically the hotness
» django-admin.py test --coverage
» django-admin.py test --coverage --report

66. Basic Coverage

67. Pretty Coverage

68. Pretty Coverage

69. Windmill Tests
» Didn’t get finished
» Allow for functional tests of the admin
» Allow for custom Windmill tests of your
apps

70. Custom Test Runners
» Allow you to do anything you want
» Load custom fixtures
» nose or py.test runners

71. Django Test Extensions
» Gareth Rushgrove
» Extra Assertions
» Coverage and XML Test Runners
» http://github.com/garethr/django-test-
extensions

72. Django Sane Testing
» Ella Folks
» Based on nosetests
» Selenium
» Live server
» http://devel.almad.net/trac/django-
sane-testing/

73. Django Test Utils
» Mine!
» Testmaker
» Crawler
» Random fanciness
» http://github.com/ericholscher/django-
test-utils/tree/master

74. Goals

75. Some form of TDD
» Write tests as you write code
» Makes your code easy to test

76. Follow Django’s Model
» Tests with every commit
» Docs with every commit
» Run tests before commiting

77. Continuous Integration

78. NEVER LEAVE THE BUILD
BROKEN

79. Love Green

80. Profiling
» python -m cProfile manage.py test
» Allows you to profile a repeatable case
» Faster tests = Faster production

81. Things to remember

82. Testing is not hard, you just
have to get started.

83. If your code doesn’t have tests, it
will be hard/impossible to
refactor

84. Once you have tests, you need to
run them!

85. Teh Pretty
» Thanks to Idan Gazit for design help

86. Credits
» http://www.flickr.com/photos/tym/
192416981/
» http://www.flickr.com/photos/
seandreilinger/2459266781/
» http://www.homebabysafety.com/images/
baby_crawl_suit.jpg

87. Questions?
» twitter.com/ericholscher
» http://ericholscher.com
» eric@ericholscher.com