Here are the key approaches to caching in Django: - Per site caching (generic approach) - Cache entire pages without GET/POST parameters and only for anonymous users. - Per view caching (granular approach) - Cache responses of individual views by decorating them with @cache_page and setting timeouts. - Template fragment caching (micro manage approach) - Cache fragments of templates, both static and dynamic, by using the {% cache %} tag and specifying keys. - Low level caching (micro micro manage) - Cache individual parameters by storing and retrieving from the cache backend directly in the views. The level of caching ranges from generic site-wide caching to fine-grained fragment caching. Per view and

1. @KevinVanWilder

2. Virtualenv
• Isolated Python environment
• Virtualenvwrapper
$ which python
/home/kevin/.virtualenvs/inuittest/bin/python
$ pip install virtualenvwrapper
$ mkproject inuittest
...
(inuittest)$ which python
/home/kevin/.virtualenvs/inuittest/bin/python
(inuittest)$ deactivate
$ workon inuittest

3. Installing libraries
• Always work in a virtualenv
$ pip freeze
# ONE GAZILLION LIBRARIES!!!!
(inuittest)$ pip freeze
argparse==1.2.1
wsgiref==0.1.2
(inuittest)$ pip install yolk django south
(inuittest)$ yolk –l
Django - 1.4.2 - active
Python - 2.7.3 - active development
(/usr/lib/python2.7/lib-dynload)
South - 0.7.6 - active
argparse - 1.2.1 - active development (/usr/lib/python2.7)
pip - 1.2.1 - active
setuptools - 0.6c11 - active
wsgiref - 0.1.2 - active development (/usr/lib/python2.7)
yolk - 0.4.3 - active

4. introduction
@KevinVanWilder

5. Hi!
• Architecture
• Development Practices
• The Django Ecosystem
• Infrastructure
• Tools

6. WTH is this thing?
ARCHITECTURE OVERVIEW

7. Architecture
RDBMS
Storage ORM Fixtures
Model
Signals
Forms
Template
URL Resolver View Template Loader
Middleware Middleware
Request Response

8. Model
RDBMS
Storage ORM Fixtures
Model
Signals
Forms
Template
URL Resolver View Template Loader
Middleware Middleware
Request Response

9. Model
from django.db import models
class Poll(models.Model):
question = models.CharField(max_length=200)
pub_date = models.DateTimeField('date published')
class Choice(models.Model):
poll = models.ForeignKey(Poll)
choice = models.CharField(max_length=200)
votes = models.IntegerField()

10. Model
BEGIN;
CREATE TABLE "polls_poll" (
"id" serial NOT NULL PRIMARY KEY,
"question" varchar(200) NOT NULL,
"pub_date" timestamp with time zone NOT NULL
);
CREATE TABLE "polls_choice" (
"id" serial NOT NULL PRIMARY KEY,
"poll_id" integer NOT NULL REFERENCES "polls_poll" ("id") DEFERRABLE INITIALLY
DEFERRED,
"choice" varchar(200) NOT NULL,
"votes" integer NOT NULL
);
COMMIT;

11. Request Response Cycle
RDBMS
Storage ORM Fixtures
Model
Signals
Forms
Template
URL Resolver View Template Loader
Middleware Middleware
Request Response

12. Request Response Cycle
1. Determine which URLConf to use
2. Load that URLConf and look for
urlpatterns variable
3. Run through each pattern in order and stop
at the one that matches
4. Once a match is found, import and call the
view with the HttpRequest object
5. If no regex matches or an exception is raised,
invoke error-handling

13. Request Response Cycle: URLConf
/polls /polls/1
/polls/1/results /polls/1/vote
from django.conf.urls import patterns, include, url
from django.contrib import admin
admin.autodiscover()
urlpatterns = patterns('',
url(r'^polls/$', 'polls.views.index'),
url(r'^polls/(?P<poll_id>d+)/$', 'polls.views.detail'),
url(r'^polls/(?P<poll_id>d+)/results/$', 'polls.views.results'),
url(r'^polls/(?P<poll_id>d+)/vote/$', 'polls.views.vote'),
url(r'^admin/', include(admin.site.urls)),
)
polls.views.detail(request=<HttpRequest object>, poll_id=‘1')

14. Request Response Cycle: View
from django.shortcuts import render_to_response
from polls.models import Poll
def index(request):
latest_poll_list = Poll.objects.all().order_by('-pub_date')[:5]
return render_to_response('polls/index.html',
{'latest_poll_list': latest_poll_list})

15. Request Response Cycle: View
from django.http import Http404
def detail(request, poll_id):
p = get_object_or_404(Poll, pk=poll_id)
return render_to_response('polls/detail.html', {'poll': p})

16. Request Response Cycle: View
from django.shortcuts import get_object_or_404, render_to_response
from django.http import HttpResponseRedirect, HttpResponse
from django.core.urlresolvers import reverse
from django.template import RequestContext
from polls.models import Choice, Poll
def vote(request, poll_id):
p = get_object_or_404(Poll, pk=poll_id)
try:
selected_choice = p.choice_set.get(pk=request.POST['choice'])
except (KeyError, Choice.DoesNotExist):
return render_to_response('polls/detail.html', {
'poll': p,
'error_message': "You didn't select a choice.",
}, context_instance=RequestContext(request))
else:
selected_choice.votes += 1
selected_choice.save()
return HttpResponseRedirect(reverse('polls.views.results', args=(p.id,)))

17. Templating
RDBMS
Storage ORM Fixtures
Model
Signals
Forms
Template
URL Resolver View Template Loader
Middleware Middleware
Request Response

18. Template
{% if latest_poll_list %}
<ul>
{% for poll in latest_poll_list %}
<li><a href="/polls/{{ poll.id }}/">{{ poll.question }}</a></li>
{% endfor %}
</ul>
{% else %}
<p>No polls are available.</p>
{% endif %}

19. The django developer in its natural habitat
DJANGO ECOSYSTEM

20. Django Debug Toolbar
• https://github.com/dcramer/django-debug-
toolbar
• Must have for all your projects

22. South
• Data schema migration
– Automatic migration creation
– Database independent
– Migration conflict detection
– Under source control
$ ./manage schemamigration –-initial
--- update models
$ ./manage schemamigration --auto

23. Celery
• Asynchronous task queue
• Message Passing
• Green threads
• RabbitMQ, Redis, MongoDB, CouchDB, …

24. • Search for Django
• Solr, ElasticSearch, Whoosh, Xapian

25. Sentry
• getsentry.com
• Exception & error aggregator
• Open Source and SaaS options

26. Tastypie
• tastypieapi.org
• Webservice API framework
• REST
• HATEOAS Compliant
• Alternative to django-piston

27. Pinax
• pinaxproject.com
• Standard project layout
• Starter projects
• Default templates for prototyping
• Reusable apps collection
– Get ready...

28. Pinax Reusable Apps
• Agon (points & positions) • Partitions (querysets)
• Agon-ratings • Reminders
• Agora (forum) • Rubberstamp (permissions)
• Aiteo (q&a, cfr. Stackoverflow) • Timezones
• Anafero (referrals) • Wakawaka (wiki)
• Atom-format • Idios (profile app)
• Biblion (blog) • Kaleo (user to user join invites)
• Brabeion (badges) • Mailout (mail campaigns)
• Dialogos (flaggable comments) • Marturion (testimonials)
• announcements • Metron (analytics & metrics)
• Boxes (db driven regions) • Minesis (media manager)
• Contacts-import • Nashvegas (db migration)
• Email-confirmation • Phileo (likes)
• Flag (inappropriate/spam) • Pinax (rapid development)
• Friends (contact/invitation) • Pinax-shop
• Mailer (queuing) • Pinax-theme-bootstrap
• Notification • Pinax-theme-classic
• Oauth-access • Symposion (apps for conferences)
• Observer

29. django-extensions
• Shell_plus
– Autoloading of the apps database models
• Additional database fields
– AutoSlugField
– CreationDateTimeField
– UUIDField
– EncryptedCharField
• DB Model graphing
• …

30. INFRASTRUCTURE

32. Application Server (AS)
Web Server Gateway Interface (WSGI)
• PEP 333 [P. J. Eby, dec 2010]
• Common ground for portable web applications
– Routing, environment variables, load balancing,
forwarding, post-prosessing
• 2 parts:
– Gateway (= server)
– Framework (= application)

33. AS: gunicorn
• “green unicorn” (cfr. green threading)
• Pre-fork worker model
• 4-12 workers = 1000’s req/sec
• Each worker runs an instance of your django
project => memory

34. AS: uWSGI
• Protocol “uwsgi” (lowercase)
• Included in Nginx & Cherokee
• Faster than gunicorn

35. TOOLS

36. Tools
• IPython
– Tab completion in the interpreter!
• import ipdb; ipdb.set_trace()
– Just like pdb, but with added magic
• Fabric (fabfile.org)
– $ fab production deploy
– Like Capistrano but without the voodoo
• Supervisord (supervisord.org)
– Process monitoring & controlling

37. QUESTIONS?

38. Deja vu for websites
CACHING

39. Cache Systems
CACHE_BACKEND = ....
– Dummy caching
• ‘dummy:///’
• For development, dummy!
– Local-Memory Caching
• ‘locmem:///’
• In memory caching without memcache
– Database Caching
• ‘db://my_cache_table’
• When DB I/O is not a problem.
– Filesystem Caching
• ‘file:///var/tmp/django_cache’ # directory
• Cache values stored in seperate files in pickled format
– Memcache
• ‘memcached:///127.0.0.1:112211/’

40. Approaches to Caching
• Per site caching (generic approach)
– Only cache everything without GET or POST parameters
– Only cache anonymous requests
• Per view caching (granular approach)
– Cache response of a view
– Set cache timeouts per view
• Template Fragment caching (micro manage approach)
– Caching both static and dynamic fragments
• Low level caching (micro micro manage)
– Indivudually store parameters in the cache

41. MIDDLEWARE_CLASSES = (
'django.middleware.cache.UpdateCacheMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.cache.FetchFromCacheMiddleware',
)
CACHE_MIDDLEWARE_ANONYMOUS_ONLY = True # optional

42. Approaches to Caching
• Per site caching (generic approach)
– Only cache everything without GET or POST parameters
– Only cache anonymous requests
• Per view caching (granular approach)
– Cache response of a view
– Set cache timeouts per view
• Template Fragment caching (micro manage approach)
– Caching both static and dynamic fragments
• Low level caching (micro micro manage)
– Indivudually store parameters in the cache

43. @cache_page(60 * 15)
def my_view(request):
# ...

44. Approaches to Caching
• Per site caching (generic approach)
– Only cache everything without GET or POST parameters
– Only cache anonymous requests
• Per view caching (granular approach)
– Cache response of a view
– Set cache timeouts per view
• Template Fragment caching (micro manage approach)
– Caching both static and dynamic fragments
• Low level caching (micro micro manage)
– Indivudually store parameters in the cache

45. Approaches to Caching
• Per site caching (generic approach)
– Only cache everything without GET or POST parameters
– Only cache anonymous requests
• Per view caching (granular approach)
– Cache response of a view
– Set cache timeouts per view
• Template Fragment caching (micro manage approach)
– Caching both static and dynamic fragments
• Low level caching (micro micro manage)
– Indivudually store parameters in the cache

46. {% load cache %}
{% cache 500 sidebar request.user.username %}
.. sidebar for logged in user ..
{% endcache %}