Advanced Django

Slide 1: Advanced Django Simon Willison http://simonwillison.net/ PyCon UK, 8th September 2007

Slide 2: Today’s topics Unit testing newforms Ajax And if we have time... OpenID

Slide 3: Unit testing

Slide 4: Hard core TDD Test Driven Development as taught by Kent Beck “Write new code only if an automated test has failed” Revolutionises the way you write code Pretty hard to adopt

Slide 5: “ I don’t do test driven development. I do stupidity driven testing... I wait until I do something stupid, and then write ” tests to avoid doing it again. Titus Brown

Slide 6: What NOT to do No matter how tough the deadline, don't let your test suite start breaking and say “I’ll ﬁx it later” This will hurt. A lot.

Slide 7: Testing in Django Testing web apps is HARD, but Django helps out with a bunch of features: Fixtures Doctests Test Client E-mail capture

Slide 8: Doctests Used by Django for both ORM testing and generated documentation You are encouraged to add them to your own models.py manage.py test to execute them Great for regression tests - just copy and paste from an interpreter session

Slide 9: >>> p = Person(name=\"Bob\", dob=date(1980, 1, 1)) >>> p.age(date(1980, 1, 1)) 0 >>> p.age(date(1979, 1, 1)) -1 >>> p.age(date(1981, 6, 1)) 1 >>> p.age(date(2000, 1, 1)) 20 >>> p.age(date(1999, 12, 31)) 19 >>> p2 = Person(name=\"Barry\", dob=date(1981, 5, 5)) >>> p2.age(date(1981, 5, 5)) 0 >>> p2.age(date(1982, 5, 4)) 0 >>> p2.age(date(1982, 5, 5)) 1 >>> p2.age(date(1982, 5, 6)) 1

Slide 10: class Person(models.Model): \"\"\" ... tests here ... \"\"\" name = models.CharField(maxlength=100) dob = models.DateField() def age(self, age=False): return 1

Slide 11: $ python manage.py test ...F ====================================================================== FAIL: Doctest: peopleage.models.Person ---------------------------------------------------------------------- Traceback (most recent call last): File \"/usr/lib/python2.5/site-packages/django/test/_doctest.py\", line 2161, in runTest raise self.failureException(self.format_failure(new.getvalue())) AssertionError: Failed doctest test for peopleage.models.Person File \"/home/simon/Projects/Django/oscon07/peopleage/models.py\", line 4, in Person ---------------------------------------------------------------------- File \"/home/simon/Projects/Django/oscon07/peopleage/models.py\", line 7, in peopleage.models.Person Failed example: p.age(date(1980, 1, 1)) Expected: 0 Got: 1

Slide 12: def age(self, age=False): if not age: age = date.today() delta = age - self.dob return int(math.floor(delta.days / float(365)))

Slide 13: File \"/home/simon/Projects/Django/oscon07/peopleage/models.py\", line 16, in peopleage.models.Person Failed example: p.age(date(1999, 12, 31)) Expected: 19 Got: 20

Slide 14: def age(self, age=False): if not age: age = date.today() years = age.year - self.dob.year this_year_birthday = self.dob.replace(year=age.year) birthday_has_passed = age >= this_year_birthday if not birthday_has_passed: years = years - 1 return years

Slide 15: Fixtures It’s useful to be able to clear and populate your test database in between tests Fixtures let you do that (they also let you populate your database with real data when you deploy your application)

Slide 16: Denormalisation

Slide 17: “ Normalised data is for sissies ” Cal Henderson

Slide 18: A forum, where each thread can have one or more replies Maintain a separate counter in the Forum table of number of replies, to speed up queries

Slide 19: class Thread(models.Model): subject = models.CharField(maxlength=100) num_replies = models.IntegerField(default=0) class Reply(models.Model): thread = models.ForeignKey(Thread) message = models.TextField()

Slide 20: [{ \"model\": \"forum.thread\", \"pk\": \"1\", \"fields\": { \"num_replies\": 0, \"subject\": \"First thread\" } },{ \"model\": \"forum.thread\", \"pk\": \"2\", \"fields\": { \"num_replies\": 1, \"subject\": \"Second thread\" } },{ \"model\": \"forum.reply\", \"pk\": \"1\", \"fields\": { \"thread\": 2, \"message\": \"First post!1\" } }]

Slide 21: from django.test import TestCase from models import Thread, Reply class ThreadCountTestCase(TestCase): fixtures = ['threadcount.json'] def test_add_reply(self): thread = Thread.objects.get(pk=2) self.assertEqual(thread.num_replies, 1) thread.reply_set.create(message=\"Another post\") thread = Thread.objects.get(pk=2) self.assertEqual(thread.reply_set.count(), 2) self.assertEqual(thread.num_replies, 2) def test_delete_reply(self): thread = Thread.objects.get(pk=2) self.assertEqual(thread.num_replies, 1) Reply.objects.get(pk=1).delete() thread = Thread.objects.get(pk=2) self.assertEqual(thread.reply_set.count(), 0) self.assertEqual(thread.num_replies, 0)

Slide 22: ====================================================== FAIL: test_add_reply (forum.tests.ThreadCountTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File \"/home/simon/Projects/Django/oscon07/forum/tests.py\", line 16, in test_add_reply self.assertEqual(thread.num_replies, 2) AssertionError: 1 != 2 ====================================================== FAIL: test_delete_reply (forum.tests.ThreadCountTestCase) ---------------------------------------------------------------------- Traceback (most recent call last): File \"/home/simon/Projects/Django/oscon07/forum/tests.py\", line 23, in test_delete_reply self.assertEqual(thread.num_replies, 0) AssertionError: 1 != 0 ----------------------------------------------------------------------

Slide 23: class Reply(models.Model): ... def save(self): super(Reply, self).save() self.thread.num_replies = self.thread.reply_set.count() self.thread.save() def delete(self): super(Reply, self).delete() self.thread.num_replies = self.thread.reply_set.count() self.thread.save()

Slide 24: ....... ----------------------------------------------------------- Ran 7 tests in 0.372s OK

Slide 25: Testing views Django’s TestClient lets you simulate a browser interacting with your site It also provides hooks in to the underlying application, so you can test against the template and context that was used to generate a page

Slide 26: from django.test import TestCase class RegistrationTest(TestCase): def test_slash(self): response = self.client.get('/register') self.assertEqual(response.status_code, 301) response = self.client.get('/register/') self.assertEqual(response.status_code, 200) def test_register(self): response = self.client.get('/register/') self.assertEqual(response.template[0].name, 'register.html') self.assertTemplateUsed(response, 'register.html')

Slide 27: def test_signup_done_page(self): self.assertEqual(len(mail.outbox), 0) data = { 'email': 'simon@example.com', 'username': 'example', 'firstname': 'Example', 'lastname': 'User', 'password': 'password1', 'password2': 'password1', } response = self.client.post('/signup/', data) self.assertEquals(response.status_code, 302) self.assertEquals(response['Location'], '/welcome/') # Check that the confirmation e-mail was sent self.assertEqual(len(mail.outbox), 1) sent = mail.outbox[0] self.assertEqual(sent.subject, 'Welcome to example.com')

Slide 28: More on testing with Django: http://www.djangoproject.com/documentation/testing/

Slide 29: newforms

Slide 30: The perfect form Display a form User ﬁlls it in and submits it Validate their entered data If errors, redisplay form with previously entered data and contextual error messages Continue until their submission is valid Convert submission to appropriate Python types

Slide 31: Manipulators

Slide 32: Manipulators newforms

Slide 33: Forms are declarative from django import newforms as forms class UserProfileForm(forms.Form): name = forms.CharField(max_length=100) email = forms.EmailField() bio = forms.CharField(widget=forms.Textarea) dob = forms.DateField(required=False) receive_newsletter = forms.BooleanField(required=False)

Slide 34: A form instance can... validate a set of data against itself render itself (and its widgets) re-render itself with errors convert to Python types

Slide 35: Simple form handling view def create_profile(request): if request.method == 'POST': form = UserProfileForm(request.POST) if form.is_valid(): # ... save the user’s profile return HttpResponseRedirect('/profile/saved/') else: form = UserProfileForm() return render_to_response('profile.html', {'form': form})

Slide 36: Initial data form = UserProfileForm( initial = { 'name': 'Simon Willison', 'email': 'simon@simonwillison.net', } )

Slide 37: Simple template <style type=\"text/css\"> ul.errorlist { color: red; } </style> ... <form action=\"/profile/create/\" method=\"POST\"> {{ form.as_p }} <input type=\"submit\" value=\"Submit\" /> </form> ...

Slide 38: Output <form action=\"/profile/create/\" method=\"POST\"> <p><label for=\"id_name\">Name:</label> <input id=\"id_name\" type=\"text\" name=\"name\" maxlength=\"100\" /></p> <p><label for=\"id_email\">Email:</label> <input type=\"text\" name=\"email\" id=\"id_email\" /></p> <p><label for=\"id_bio\">Bio:</label> <textarea id=\"id_bio\" rows=\"10\" cols=\"40\" name=\"bio\"></textarea></p> <p><label for=\"id_dob\">Dob:</label> <input type=\"text\" name=\"dob\" id=\"id_dob\" /></p> <p><label for=\"id_receive_newsletter\">Receive newsletter:</label> <input type=\"checkbox\" name=\"receive_newsletter\" id=\"id_receive_newsletter\" /></p> <input type=\"submit\" value=\"Submit\" /> </form>

Slide 41: Custom validation from django import newforms as forms from django.newforms.util import ValidationError class UserProfileForm(forms.Form): name = forms.CharField(max_length=100) email = forms.EmailField() bio = forms.CharField(widget=forms.Textarea) dob = forms.DateField(required=False) receive_newsletter = forms.BooleanField(required=False) def clean_email(self): if self.cleaned_data['email'].split('@')[1] == 'hotmail.com': raise ValidationError, \"No hotmail.com emails, please.\"

Slide 42: Custom validation from django import newforms as forms from django.newforms.util import ValidationError class UserProfileForm(forms.Form): name = forms.CharField(max_length=100) email = forms.EmailField() bio = forms.CharField(widget=forms.Textarea) dob = forms.DateField(required=False) receive_newsletter = forms.BooleanField(required=False) def clean_email(self): if self.cleaned_data['email'].split('@')[1] == 'hotmail.com': raise ValidationError, \"No hotmail.com emails, please.\"

Slide 43: Custom rendering <ol class=\"formItems longForm\"> <li{% if form.email.errors %} class=\"errors\"{% endif %}> <label for=\"id_email\">Email: </label> {{ form.email }} {{ form.email.errors }} <p class=\"info\">Your e-mail address.</p> </li> ... </ol>

Slide 44: Model shortcuts DRY: You’ve already declared your models; you shouldn’t have to repeat yourself in your forms UserForm = form_for_model(User) ############################### page = Page.objects.get(pk=1) PageForm = form_for_instance(page) form = PageForm(request.POST) ... if form.is_valid(): form.save()

Slide 45: Full documentation: http://www.djangoproject.com/documentation/newforms/ django/trunk/tests/regressiontests/forms/tests.py

Slide 46: Ajax

Slide 47: First things ﬁrst If you're going to do Ajax, you need a JavaScript library You could use Yet Another XMLHttpRequest abstraction... but the popular libraries offer fantastic convenience Good libraries include YUI, Dojo, MochiKit and (controversial) Prototype...

Slide 48: .. and jQuery I'm going to be using jQuery Almost everything is done in terms of CSS selectors and chained methods It looks like a gimmick, but it isn't http://simonwillison.net/2007/Aug/15/jquery/

Slide 49: Ajax formats... Django has great support for any and every Ajax format HTML fragments XML JSON

Slide 50: Username available? from django.contrib.auth.models import User def check_username(request): reply = \"\" username = request.GET.get('username', '') if username: if User.objects.filter(username=username).count(): reply = 'Unavailable' else: reply = 'Available' return HttpResponse(reply)

Slide 51: jQuery('span#msg').load( '/check_username/?username=' + input.val() );

Slide 52: $('span#msg').load( '/check_username/?username=' + input.val() );

Slide 53: var input = $('input#id_username') input.keyup(function() { $('span#msg').load( '/check_username/?username=' + input.val() ); });

Slide 54: $(document).ready(function() { var input = $('input#id_username') input.keyup(function() { $('span#msg').load( '/check_username/?username=' + input.val() ); }); });

Slide 55: $(function() { var input = $('input#id_username') input.keyup(function() { $('span#msg').load( '/check_username/?username=' + input.val() ); }); });

Slide 56: Recycling server-side form validation

Slide 57: from django import newforms as forms class ContactForm(forms.Form): subject = forms.CharField(max_length=100) message = forms.CharField(widget=forms.Textarea()) sender = forms.EmailField()

Slide 58: def validate_contact(request): \"Validate post data, return errors as json\" form = ContactForm(request.POST) if (request.GET.has_key('field')): # Validate a single field errors = form.errors[request.GET['field']] else: errors = form.errors return JsonResponse({ 'valid': not errors, 'errors': errors })

Slide 59: from django.utils import simplejson class JsonResponse(HttpResponse): def __init__(self, data): HttpResponse.__init__( self, simplejson.dumps(data), mimetype='application/json' )

Slide 60: function validateInput(input) { $.post('/contact/validate/?field=' + input.attr('id').replace('id_', ''), $('form').formToArray(), function(data) { var json = eval('(' + data + ')'); showErrors(input, json.errors); } ); } $(function() { $(':input').blur(function() { validateInput($(this)); }); });

Slide 61: function relatedErrorList(input) { var prevUL = $(input).parent().prev(); if (prevUL && prevUL.attr('class') == 'errorlist') { return prevUL; } var errorlist = $('<ul class=\"errorlist\"></ul>'); input.parent().before(errorlist); return errorlist; } function showErrors(input, errors) { var errorlist = relatedErrorList(input); errorlist.empty(); $.each(errors, function(i, error) { errorlist.append('<li>' + error + '</li>'); }); }

Slide 62: Django philosophy Django often gets marked down in “framework comparisons” due to the lack of built in Ajax support Personally I think that shipping without a recommended library is a feature, not a bug

Slide 63: (bonus section)

Slide 64: What is OpenID?

Slide 65: OpenID is a decentralised mechanism for Single Sign On

Slide 66: An OpenID is a URL http://simonwillison.myopenid.com/ http://simonwillison.net/ http://swillison.livejournal.com/ http://openid.aol.com/simonwillison

Slide 67: How it works You enter your OpenID on a site (instead of the usual username and password) It redirects you back to your OpenID provider They authenticate you in some way They redirect you back to the original site

Slide 68: Simple registration (an optional but useful extension)

Slide 69: Consumers can also ask... Your preferred username Your e-mail address Your ﬁrst and last name Your date of birth Your language, country and timezone

Slide 70: How do you use OpenID in a Django application?

Slide 71: The hard way Use the JanRain OpenID library Pretty much a reference implementation for the OpenID spec Well written, well tested but takes a while to get the hang of www.openidenabled.com/openid/libraries/python/

Slide 72: The easy way Use django-openid A simple wrapper around JanRain A middleware component, some models and a few pre-written views http://code.google.com/p/django-openid/

Slide 73: Installation Add 'django_openidconsumer' to your INSTALLED_APPS setting manage.py syncdb Add the OpenIDMiddleware to your MIDDLEWARE_CLASSES setting Add the views to your URLconf

Slide 74: In urls.py ... (r'ôpenid/$', 'django_openidconsumer.views.begin'), (r'ôpenid/complete/$', 'django_openidconsumer.views.complete'), (r'ôpenid/signout/$', 'django_openidconsumer.views.signout'), ...

Slide 75: request.openid The middleware adds an openid property to the Django request object If the user is not signed in, this will be None Otherwise, it will be an OpenID object; the str() representation will be the OpenID (or use request.openid.openid)

Slide 76: def example_view(request): if request.openid: return HttpResponse(\"OpenID is %s\" % escape(request.openid)) else: return HttpResponse(\"No OpenID\")

Slide 77: request.openids The module supports users signing in with more than one OpenID at a time request.openids provides a list of all authenticated OpenIDs request.openid merely returns the most recent from this list

Slide 78: For simple registration (r'^openid/$', 'django_openidconsumer.views.begin', { 'sreg': 'email,nickname' }), def example_sreg(request): if request.openid and request.openid.sreg.has_key('email'): return HttpResponse(\"Your e-mail address is: %s\" % escape( request.openid.sreg['email'] )) else: return HttpResponse(\"No e-mail address\")

Slide 79: Coming soon django_openidauth, providing tools to associate OpenIDs with existing django.contrib.auth accounts django_openidserver, to make it easy to provide OpenIDs for users of your Django application

Slide 80: More information http://openid.net/ Also home to the OpenID mailing lists http://www.openidenabled.com/ http://simonwillison.net/tags/openid/ http://code.google.com/p/django-openid/

Slide 81: Thank you!

Slideshare.net (beta)