Token Testing Slides

Token Testing Talk
Eric Holscher
http://ericholscher.com
Djangocon 2009

Testing Talks per Conference
10.0

7.5

5.0

2.5

0
Pycon RailsConf DjangoCon

How do you know?

» First 4 months of my job was porting and
testing Ellington
» Going from Django r1290 to Django 1.0.

» Suite from 0 to 400 tests. (Now has 1715)

30,000 Ft View

» State of testing in Django
» Why you should be testing
» How you start testing
» Useful tools
» Eventual Goals

assertTrue('Hello World',
community.testing.status)

Django 1.1

Making Testing Possible since 2009

manage.py startapp creates a
tests.py

Basic Test
from django.test import TestCase

class SimpleTest(TestCase):
def test_basic_addition(self):
"""
Tests that 1 + 1 always equals 2.
"""
self.failUnlessEqual(1 + 1, 2)

__test__ = {"doctest": """
Another way to test that 1 + 1 is equal to 2.

>>> 1 + 1 == 2
True
"""}

Minutes (Lower is better)

Ellington Test Speedup
60

45

30

15

0
Django 1.0 Django 1.1

“It is not the strongest of
the species that survives, nor
the most intelligent, but the
one most responsive to
change.
- Charles Darwin

Won’t somebody please think of
the users?!

Tests as Documentation

Test driven development
+
Document driven development
=
Test Driven Documentation

“Code without tests is
broken as designed
- Jacob Kaplan-Moss

Dizzying Array of Testing
Options

What kind of test?

» doctest

» unittest

Doctests

» Inline documentation

» <Copy from terminal to test file>

» Easy

Awesome Documentation

def parse_ttag(token, required_tags):
"""
A function to parse a template tag.

It sets the name of the tag to 'tag_name' in the hash returned.

>>> from test_utils.templatetags.utils import parse_ttag
>>> parse_ttag('super_cool_tag for my_object as obj', ['as'])
{'tag_name': u'super_cool_tag', u'as': u'obj'}
>>> parse_ttag('super_cool_tag for my_object as obj', ['as', 'for'])
{'tag_name': u'super_cool_tag', u'as': u'obj', u'for': u'my_object'}

"""
bits = token.split(' ')
tags = {'tag_name': bits.pop(0)}
for index, bit in enumerate(bits):
bit = bit.strip()
if bit in required_tags:
if len(bits) != index-1:
tags[bit] = bits[index+1]
return tags

Awesome Documentation

>>> from test_utils.templatetags.utils import parse_ttag
>>> parse_ttag('super_cool_tag for my_object as obj', ['as'])
{'tag_name': u'super_cool_tag', u'as': u'obj'}
>>> parse_ttag('super_sweet for object as obj', ['as', 'for'])
{'tag_name': u'super_sweet', u'as': u'obj', u'for': u'object'}

Doctest problems

» Can’t use PDB

» Hide real failures

Unit Tests

» Standard (XUnit)

» More robust

» setUp and tearDown

Basic Unit Test
import random
import unittest

class TestRandom(unittest.TestCase):

def setUp(self):
self.seq = range(10)

def testshuffle(self):
# shuffled sequence does not lose any elements
random.shuffle(self.seq)
self.seq.sort()
self.assertEqual(self.seq, range(10))

if __name__ == '__main__':
unittest.main()

Django TestCase

» Subclasses unittest

» Fixtures

» Assertions

» Mail

» URLs

Test Client

» Test HTTP Requests without server

» Test Views, Templates, and Context

Django’s TestCase

from django.contrib.auth.models import User
from django.core import mail

class PasswordResetTest(TestCase):
fixtures = ['authtestdata.json']
urls = 'django.contrib.auth.urls'

def test_email_not_found(self):
"Error is raised if the provided email address isn't currently
registered"
response = self.client.post('/password_reset/', {'email':
'not_a_real_email@email.com'})
self.assertEquals(len(mail.outbox), 0)

What flavor of test?

» Unit

» Functional

» Browser

Unit test

» Low level tests

» Small, focused, exercising one bit of
functionality
» Great for libraries

Regression test

» Written when you find a bug

» Proves bug was fixed

» Django Tickets

Functional

» ‘Black Box Testing’

» Check High Level Functionality

Functional Testing Tools

» Twill

» Django Test Client

Browser tests

» Run tests in a web browser

» Check compatibility of design

» Basically an IE sanity check

» Only real way to test JS, AJAX, CSS

» Slow

Browser Testing Tools

» Windmill

» Selenium

Other kinds of testing

» Spiders

» Fuzz testing

» Load testing

Use unittest unless you have a
reason not to!

Start with a regression test or a
functional test

Poking at code on the command
line

Use the data, Luke

» Use data as a pivot

» Fixtures means you use Unit Tests

» Creation on the command line, Doctests

Creating Fixtures

»./manage.py dumpdata <app>
»./manage.py makefixture Model[x:y]
» Follows relations
» Slicing
»By Hand

Getting in Context

» ./manage.py testshell <fixture>

» (i)pdb

Making Functional tests

» Usually a relatively annoying process

» Testmaker makes it easy.

» ./manage.py testmaker [app]

» Simply browse and your session is
recorded.

Testing your views gets you the
most coverage.

Summer of Code

» Test-Only Models

» Skipping tests

» Coverage

» Windmill tests of the admin

Test-Only Models

» Allow for models available only during
tests
» Don’t need test_project now!

Test-Only Models

#test_models.py

from django.db import models

class TestModel(models.Model):
name = models.CharField()

#tests.py


class TestMyViews(TestCase):
test_models = ['test_models']

def testIndexPageView(self):
from myapp.models import TestModel
TestModel.objects.get(name='daniellindsleyrocksdahouse')

Skipping Tests

» Views Required

» Models Required

» Specific database type

» Conditional on a Function

Skipping Tests

from django.tests.decorators import conditional_skip
import datetime

class TestUnderCondition(TestCase):

def _check_2009():
# Condition returning True if test should be run and False if it
# should be skipped.
if datetime.datetime.now() > datetime.datetime(2009, 01, 01):
return True

@conditional_skip(_check_2009, reason='This test only runs in 2009')
def testOnlyIn2009(self):
# Test to run if _my_condition evaluates to True

Skipping Tests

======================================================================
SKIPPED: test_email_found (django.contrib.auth.tests.basic.PasswordResetTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/dnaquin/Dropbox/Sandbox/django/django/test/decorators.py", line 43, in _skip
raise SkippedTest(reason=reason)
SkippedTest: Required view for this test not found: django.contrib.auth.views.password_reset

----------------------------------------------------------------------
Ran 408 tests in 339.663s

FAILED (failures=1, skipped=2)

Coverage

» Basically the hotness

» django-admin.py test --coverage

» django-admin.py test --coverage --report

Windmill Tests

» Didn’t get finished

» Allow for functional tests of the admin

» Allow for custom Windmill tests of your
apps

Custom Test Runners

» Allow you to do anything you want

» Load custom fixtures

» nose or py.test runners

Django Test Extensions

» Gareth Rushgrove

» Extra Assertions

» Coverage and XML Test Runners

» http://github.com/garethr/django-test-
extensions

Django Sane Testing

» Ella Folks

» Based on nosetests

» Selenium

» Live server

» http://devel.almad.net/trac/django-
sane-testing/

Django Test Utils

» Mine!

» Testmaker

» Crawler

» Random fanciness

» http://github.com/ericholscher/django-
test-utils/tree/master

Some form of TDD

» Write tests as you write code

» Makes your code easy to test

Follow Django’s Model

» Tests with every commit

» Docs with every commit

» Run tests before commiting

NEVER LEAVE THE BUILD
BROKEN

Profiling

» python -m cProfile manage.py test

» Allows you to profile a repeatable case

» Faster tests = Faster production

Testing is not hard, you just
have to get started.

If your code doesn’t have tests, it
will be hard/impossible to
refactor

Once you have tests, you need to
run them!

Teh Pretty

» Thanks to Idan Gazit for design help

Credits

» http://www.flickr.com/photos/tym/
192416981/
» http://www.flickr.com/photos/
seandreilinger/2459266781/
» http://www.homebabysafety.com/images/
baby_crawl_suit.jpg

Questions?

» twitter.com/ericholscher

» http://ericholscher.com

» eric@ericholscher.com

Token Testing Slides

More Related Content

What's hot

Similar to Token Testing Slides

Recently uploaded

Token Testing Slides