Advanced Django
Simon Willison
http://simonwillison.net/
PyCon UK, 8th September 2007

Today’s topics
Unit testing
newforms
Ajax
And if we have time... OpenID

Unit testing

Hard core TDD
Test Driven Development as taught by Kent
Beck
“Write new code only if an automated test has
failed”
Revolutionises the way you write code
Pretty hard to adopt

“ I don’t do test driven
development. I do stupidity
driven testing... I wait until I do
something stupid, and then write
”
tests to avoid doing it again.
Titus Brown

What NOT to do
No matter how tough the deadline, don't let
your test suite start breaking and say “I’ll fix it
later”
This will hurt. A lot.

Testing in Django
Testing web apps is HARD, but Django helps out
with a bunch of features:
Fixtures
Doctests
Test Client
E-mail capture

Doctests
Used by Django for both ORM testing and
generated documentation
You are encouraged to add them to your own
models.py
manage.py test to execute them
Great for regression tests - just copy and paste
from an interpreter session

>>> p = Person(name=\"Bob\", dob=date(1980, 1, 1))
>>> p.age(date(1980, 1, 1))
0
>>> p.age(date(1979, 1, 1))
-1
>>> p.age(date(1981, 6, 1))
1
>>> p.age(date(2000, 1, 1))
20
>>> p.age(date(1999, 12, 31))
19
>>> p2 = Person(name=\"Barry\", dob=date(1981, 5, 5))
>>> p2.age(date(1981, 5, 5))
0
>>> p2.age(date(1982, 5, 4))
0
>>> p2.age(date(1982, 5, 5))
1
>>> p2.age(date(1982, 5, 6))
1

class Person(models.Model):
\"\"\"
... tests here ...
\"\"\"
name = models.CharField(maxlength=100)
dob = models.DateField()
def age(self, age=False):
return 1

$ python manage.py test
...F
======================================================================
FAIL: Doctest: peopleage.models.Person
----------------------------------------------------------------------
Traceback (most recent call last):
File \"/usr/lib/python2.5/site-packages/django/test/_doctest.py\", line 2161, in runTest
raise self.failureException(self.format_failure(new.getvalue()))
AssertionError: Failed doctest test for peopleage.models.Person
File \"/home/simon/Projects/Django/oscon07/peopleage/models.py\", line 4, in Person
----------------------------------------------------------------------
File \"/home/simon/Projects/Django/oscon07/peopleage/models.py\", line 7, in
peopleage.models.Person
Failed example:
p.age(date(1980, 1, 1))
Expected:
0
Got:
1

def age(self, age=False):
if not age:
age = date.today()
delta = age - self.dob
return int(math.floor(delta.days / float(365)))

File \"/home/simon/Projects/Django/oscon07/peopleage/models.py\", line 16,
in peopleage.models.Person
Failed example:
p.age(date(1999, 12, 31))
Expected:
19
Got:
20

def age(self, age=False):
if not age:
age = date.today()
years = age.year - self.dob.year
this_year_birthday = self.dob.replace(year=age.year)
birthday_has_passed = age >= this_year_birthday
if not birthday_has_passed:
years = years - 1
return years

Fixtures
It’s useful to be able to clear and populate your
test database in between tests
Fixtures let you do that (they also let you
populate your database with real data when you
deploy your application)

Denormalisation

“
Normalised data is
for sissies
”
Cal Henderson

A forum, where each thread can have one
or more replies
Maintain a separate counter in the Forum
table of number of replies, to speed up
queries

class Thread(models.Model):
subject = models.CharField(maxlength=100)
num_replies = models.IntegerField(default=0)
class Reply(models.Model):
thread = models.ForeignKey(Thread)
message = models.TextField()

[{ \"model\": \"forum.thread\",
\"pk\": \"1\",
\"fields\": {
\"num_replies\": 0,
\"subject\": \"First thread\" }
},{ \"model\": \"forum.thread\",
\"pk\": \"2\",
\"fields\": {
\"num_replies\": 1,
\"subject\": \"Second thread\" }
},{ \"model\": \"forum.reply\",
\"pk\": \"1\",
\"fields\": {
\"thread\": 2,
\"message\": \"First post!1\" }
}]

from django.test import TestCase
from models import Thread, Reply
class ThreadCountTestCase(TestCase):
fixtures = ['threadcount.json']
def test_add_reply(self):
thread = Thread.objects.get(pk=2)
self.assertEqual(thread.num_replies, 1)
thread.reply_set.create(message=\"Another post\")
thread = Thread.objects.get(pk=2)
self.assertEqual(thread.reply_set.count(), 2)
self.assertEqual(thread.num_replies, 2)
def test_delete_reply(self):
thread = Thread.objects.get(pk=2)
self.assertEqual(thread.num_replies, 1)
Reply.objects.get(pk=1).delete()
thread = Thread.objects.get(pk=2)
self.assertEqual(thread.reply_set.count(), 0)
self.assertEqual(thread.num_replies, 0)

======================================================
FAIL: test_add_reply (forum.tests.ThreadCountTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File \"/home/simon/Projects/Django/oscon07/forum/tests.py\", line 16, in
test_add_reply
self.assertEqual(thread.num_replies, 2)
AssertionError: 1 != 2
======================================================
FAIL: test_delete_reply (forum.tests.ThreadCountTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File \"/home/simon/Projects/Django/oscon07/forum/tests.py\", line 23, in
test_delete_reply
self.assertEqual(thread.num_replies, 0)
AssertionError: 1 != 0
----------------------------------------------------------------------

class Reply(models.Model):
...
def save(self):
super(Reply, self).save()
self.thread.num_replies = self.thread.reply_set.count()
self.thread.save()
def delete(self):
super(Reply, self).delete()
self.thread.num_replies = self.thread.reply_set.count()
self.thread.save()

.......
-----------------------------------------------------------
Ran 7 tests in 0.372s
OK

Testing views
Django’s TestClient lets you simulate a browser
interacting with your site
It also provides hooks in to the underlying
application, so you can test against the template
and context that was used to generate a page

from django.test import TestCase
class RegistrationTest(TestCase):
def test_slash(self):
response = self.client.get('/register')
self.assertEqual(response.status_code, 301)
response = self.client.get('/register/')
self.assertEqual(response.status_code, 200)
def test_register(self):
response = self.client.get('/register/')
self.assertEqual(response.template[0].name, 'register.html')
self.assertTemplateUsed(response, 'register.html')

def test_signup_done_page(self):
self.assertEqual(len(mail.outbox), 0)
data = {
'email': 'simon@example.com', 'username': 'example',
'firstname': 'Example', 'lastname': 'User',
'password': 'password1', 'password2': 'password1',
}
response = self.client.post('/signup/', data)
self.assertEquals(response.status_code, 302)
self.assertEquals(response['Location'], '/welcome/')
# Check that the confirmation e-mail was sent
self.assertEqual(len(mail.outbox), 1)
sent = mail.outbox[0]
self.assertEqual(sent.subject, 'Welcome to example.com')

More on testing with Django:
http://www.djangoproject.com/documentation/testing/

newforms

The perfect form
Display a form
User fills it in and submits it
Validate their entered data
If errors, redisplay form with previously
entered data and contextual error messages
Continue until their submission is valid
Convert submission to appropriate Python types

Manipulators

Manipulators
newforms

Forms are declarative
from django import newforms as forms
class UserProfileForm(forms.Form):
name = forms.CharField(max_length=100)
email = forms.EmailField()
bio = forms.CharField(widget=forms.Textarea)
dob = forms.DateField(required=False)
receive_newsletter = forms.BooleanField(required=False)

A form instance can...
validate a set of data against itself
render itself (and its widgets)
re-render itself with errors
convert to Python types

Simple form handling view
def create_profile(request):
if request.method == 'POST':
form = UserProfileForm(request.POST)
if form.is_valid():
# ... save the user’s profile
return HttpResponseRedirect('/profile/saved/')
else:
form = UserProfileForm()
return render_to_response('profile.html', {'form': form})

Initial data
form = UserProfileForm(
initial = {
'name': 'Simon Willison',
'email': 'simon@simonwillison.net',
}
)

Simple template
<style type=\"text/css\">
ul.errorlist { color: red; }
</style>
...
<form action=\"/profile/create/\" method=\"POST\">
{{ form.as_p }}
<input type=\"submit\" value=\"Submit\" />
</form>
...

Output
<form action=\"/profile/create/\" method=\"POST\">
<p><label for=\"id_name\">Name:</label> <input id=\"id_name\"
type=\"text\" name=\"name\" maxlength=\"100\" /></p>
<p><label for=\"id_email\">Email:</label> <input type=\"text\"
name=\"email\" id=\"id_email\" /></p>
<p><label for=\"id_bio\">Bio:</label> <textarea id=\"id_bio\" rows=\"10\"
cols=\"40\" name=\"bio\"></textarea></p>
<p><label for=\"id_dob\">Dob:</label> <input type=\"text\" name=\"dob\"
id=\"id_dob\" /></p>
<p><label for=\"id_receive_newsletter\">Receive newsletter:</label>
<input type=\"checkbox\" name=\"receive_newsletter\"
id=\"id_receive_newsletter\" /></p>
<input type=\"submit\" value=\"Submit\" />
</form>

Output
<form action=\"/profile/create/\" method=\"POST\">
<p><label for=\"id_name\">Name:</label> <input id=\"id_name\"
type=\"text\" name=\"name\" maxlength=\"100\" /></p>
<p><label for=\"id_email\">Email:</label> <input type=\"text\"
name=\"email\" id=\"id_email\" /></p>
<p><label for=\"id_bio\">Bio:</label> <textarea id=\"id_bio\" rows=\"10\"
cols=\"40\" name=\"bio\"></textarea></p>
<p><label for=\"id_dob\">Dob:</label> <input type=\"text\" name=\"dob\"
id=\"id_dob\" /></p>
<p><label for=\"id_receive_newsletter\">Receive newsletter:</label>
<input type=\"checkbox\" name=\"receive_newsletter\"
id=\"id_receive_newsletter\" /></p>
<input type=\"submit\" value=\"Submit\" />
</form>

Output
<form action=\"/profile/create/\" method=\"POST\">
<p><label for=\"id_name\">Name:</label> <input id=\"id_name\"
type=\"text\" name=\"name\" maxlength=\"100\" /></p>
<p><label for=\"id_email\">Email:</label> <input type=\"text\"
name=\"email\" id=\"id_email\" /></p>
<p><label for=\"id_bio\">Bio:</label> <textarea id=\"id_bio\" rows=\"10\"
cols=\"40\" name=\"bio\"></textarea></p>
<p><label for=\"id_dob\">Dob:</label> <input type=\"text\" name=\"dob\"
id=\"id_dob\" /></p>
<p><label for=\"id_receive_newsletter\">Receive newsletter:</label>
<input type=\"checkbox\" name=\"receive_newsletter\"
id=\"id_receive_newsletter\" /></p>
<input type=\"submit\" value=\"Submit\" />
</form>

Custom validation
from django import newforms as forms
from django.newforms.util import ValidationError
class UserProfileForm(forms.Form):
name = forms.CharField(max_length=100)
email = forms.EmailField()
bio = forms.CharField(widget=forms.Textarea)
dob = forms.DateField(required=False)
receive_newsletter = forms.BooleanField(required=False)
def clean_email(self):
if self.cleaned_data['email'].split('@')[1] == 'hotmail.com':
raise ValidationError, \"No hotmail.com emails, please.\"

Custom validation
from django import newforms as forms
from django.newforms.util import ValidationError
class UserProfileForm(forms.Form):
name = forms.CharField(max_length=100)
email = forms.EmailField()
bio = forms.CharField(widget=forms.Textarea)
dob = forms.DateField(required=False)
receive_newsletter = forms.BooleanField(required=False)
def clean_email(self):
if self.cleaned_data['email'].split('@')[1] == 'hotmail.com':
raise ValidationError, \"No hotmail.com emails, please.\"

Custom rendering
<ol class=\"formItems longForm\">
<li{% if form.email.errors %} class=\"errors\"{% endif %}>
<label for=\"id_email\">Email: </label>
{{ form.email }}
{{ form.email.errors }}
<p class=\"info\">Your e-mail address.</p>
</li>
...
</ol>

Model shortcuts
DRY: You’ve already declared your models; you
shouldn’t have to repeat yourself in your forms
UserForm = form_for_model(User)
###############################
page = Page.objects.get(pk=1)
PageForm = form_for_instance(page)
form = PageForm(request.POST)
...
if form.is_valid():
form.save()

Full documentation:
http://www.djangoproject.com/documentation/newforms/
django/trunk/tests/regressiontests/forms/tests.py

Ajax

First things first
If you're going to do Ajax, you need a JavaScript
library
You could use Yet Another XMLHttpRequest
abstraction... but the popular libraries offer fantastic
convenience
Good libraries include YUI, Dojo, MochiKit and
(controversial) Prototype...

.. and jQuery
I'm going to be using jQuery
Almost everything is done in terms of CSS
selectors and chained methods
It looks like a gimmick, but it isn't
http://simonwillison.net/2007/Aug/15/jquery/

Ajax formats...
Django has great support for any and every Ajax
format
HTML fragments
XML
JSON

Username available?
from django.contrib.auth.models import User
def check_username(request):
reply = \"\"
username = request.GET.get('username', '')
if username:
if User.objects.filter(username=username).count():
reply = 'Unavailable'
else:
reply = 'Available'
return HttpResponse(reply)

jQuery('span#msg').load(
'/check_username/?username=' + input.val()
);

$('span#msg').load(
'/check_username/?username=' + input.val()
);

var input = $('input#id_username')
input.keyup(function() {
$('span#msg').load(
'/check_username/?username=' + input.val()
);
});

$(document).ready(function() {
var input = $('input#id_username')
input.keyup(function() {
$('span#msg').load(
'/check_username/?username=' + input.val()
);
});
});

$(function() {
var input = $('input#id_username')
input.keyup(function() {
$('span#msg').load(
'/check_username/?username=' + input.val()
);
});
});

Recycling server-side
form validation

from django import newforms as forms
class ContactForm(forms.Form):
subject = forms.CharField(max_length=100)
message = forms.CharField(widget=forms.Textarea())
sender = forms.EmailField()

def validate_contact(request):
\"Validate post data, return errors as json\"
form = ContactForm(request.POST)
if (request.GET.has_key('field')):
# Validate a single field
errors = form.errors[request.GET['field']]
else:
errors = form.errors
return JsonResponse({
'valid': not errors,
'errors': errors
})

from django.utils import simplejson
class JsonResponse(HttpResponse):
def __init__(self, data):
HttpResponse.__init__(
self, simplejson.dumps(data),
mimetype='application/json'
)

function validateInput(input) {
$.post('/contact/validate/?field=' + input.attr('id').replace('id_', ''),
$('form').formToArray(), function(data) {
var json = eval('(' + data + ')');
showErrors(input, json.errors);
}
);
}
$(function() {
$(':input').blur(function() {
validateInput($(this));
});
});

function relatedErrorList(input) {
var prevUL = $(input).parent().prev();
if (prevUL && prevUL.attr('class') == 'errorlist') {
return prevUL;
}
var errorlist = $('<ul class=\"errorlist\"></ul>');
input.parent().before(errorlist);
return errorlist;
}
function showErrors(input, errors) {
var errorlist = relatedErrorList(input);
errorlist.empty();
$.each(errors, function(i, error) {
errorlist.append('<li>' + error + '</li>');
});
}

Django philosophy
Django often gets marked down in “framework
comparisons” due to the lack of built in Ajax
support
Personally I think that shipping without a
recommended library is a feature, not a bug

(bonus section)

What is OpenID?

OpenID is a decentralised
mechanism for Single Sign On

An OpenID is a URL
http://simonwillison.myopenid.com/
http://simonwillison.net/
http://swillison.livejournal.com/
http://openid.aol.com/simonwillison

How it works
You enter your OpenID on a site (instead of the
usual username and password)
It redirects you back to your OpenID provider
They authenticate you in some way
They redirect you back to the original site

Simple registration
(an optional but useful extension)

Consumers can also ask...
Your preferred username
Your e-mail address
Your first and last name
Your date of birth
Your language, country and timezone

How do you use
OpenID in a Django
application?

The hard way
Use the JanRain OpenID library
Pretty much a reference implementation for the
OpenID spec
Well written, well tested but takes a while to get
the hang of
www.openidenabled.com/openid/libraries/python/

The easy way
Use django-openid
A simple wrapper around JanRain
A middleware component, some models and a
few pre-written views
http://code.google.com/p/django-openid/

Installation
Add 'django_openidconsumer' to your
INSTALLED_APPS setting
manage.py syncdb
Add the OpenIDMiddleware to your
MIDDLEWARE_CLASSES setting
Add the views to your URLconf

In urls.py
...
(r'^openid/$',
'django_openidconsumer.views.begin'),
(r'^openid/complete/$',
'django_openidconsumer.views.complete'),
(r'^openid/signout/$',
'django_openidconsumer.views.signout'),
...

request.openid
The middleware adds an openid property to the
Django request object
If the user is not signed in, this will be None
Otherwise, it will be an OpenID object; the str()
representation will be the OpenID (or use
request.openid.openid)

def example_view(request):
if request.openid:
return HttpResponse(\"OpenID is %s\" % escape(request.openid))
else:
return HttpResponse(\"No OpenID\")

request.openids
The module supports users signing in with more
than one OpenID at a time
request.openids provides a list of all
authenticated OpenIDs
request.openid merely returns the most
recent from this list

For simple registration
(r'^openid/$', 'django_openidconsumer.views.begin', {
'sreg': 'email,nickname'
}),
def example_sreg(request):
if request.openid and request.openid.sreg.has_key('email'):
return HttpResponse(\"Your e-mail address is: %s\" % escape(
request.openid.sreg['email']
))
else:
return HttpResponse(\"No e-mail address\")

Coming soon
django_openidauth, providing tools to associate
OpenIDs with existing django.contrib.auth
accounts
django_openidserver, to make it easy to provide
OpenIDs for users of your Django application

More information
http://openid.net/
Also home to the OpenID mailing lists
http://www.openidenabled.com/
http://simonwillison.net/tags/openid/
http://code.google.com/p/django-openid/

Thank you!