A mystical journey through Django performance optimization techniques, tools and gotchas. Talk presented at Django Meetup Buenos Aires June 2014

1. This slide was intentionally left blank

2. Speed is a feature
A mystical journey through Django performance
optimization techniques, tools and gotchas

3. Roadmap
How to find performance problems
Sneak peak: Front-end performance
How to fix them in Django

4. Why?
[Google] found that Half a
second delay caused a 20% drop
in traffic.
[Amazon] found that even very
small delays would result in
substantial and costly drops in
revenue.
Users really respond to speed
2006

5. There’s nothing like data
Don’t start with the code. Profile and gather real usage data.

6. There’s nothing like real data
Identify bottlenecks
New Relic
Paid service
Very good
Google Analytics
Free of charge
Less detail
Your logs
No data to third parties
Harder to use
Profiling
Complex setup required
Overhead

7. New Relic

8. New Relic

9. Google Analytics Site Speed

10. There’s nothing like data
Let’s find the culprit!
django-debug-toolbar
django-debug-toolbar-template-timings

11. Time all the things!

12. Typical backend bottlenecks
Database
External Services
CPU Intensive task
Template Rendering

13. Database
Missing index
select_related()
prefetch_related()
Order

14. select_related()
>>> for c in Comment.objects.all():
print c.user.name
# select * from comments;
# select * from users where id = 1;
# select * from users where id = 2;
...
>>> comments = Comment.objects.all();
>>> for c in comments.select_related(“user”):
print c.user.name
# select comments.*, users.*
# from comments, users
# where comments.user_id = users.id;

15. prefech_related()
>>> for u in User.objects.filter(id__lt=10):
print len(u.comments.all())
# select * from users;
# select * from comments where user_id = 1;
...
>>> users = User.objects.filter(id__lt=10)
>>> for u in users.prefetch_related(“comments”):
print len(u.comments.all())
# select * from users where id < 10;
# select * from comments
# where user_id in (1,2,3,4,5,6,7,8,9);
## Joins them in python

16. Background Jobs
Celery
from celery import task
@task
def send_confirmation_email(user_id):
...
def login(req):
...
send_confirmation_email.delay(req.user.id)
return HttpResponseRedirect(“/home/”)

17. Caching!
DB
External Services
CPU intensive tasks

18. Caching!
Per view cache
from django.views.decorators.cache import
cache_page
@cache_page(60 * 15) # seconds
def my_view(request):
...

19. Caching!
Low level cache API
>>> from django.core.cache import get_cache
>>> cache = get_cache('default')
>>> cache.set('my_key', 'hello, world!', 30)
>>> cache.get('my_key')
>>> from django.core.cache import caches
>>> cache = caches['default']

20. Caching!
from django.core.cache import get_cache
cache = get_cache('default')
def last_comments(request):
comments_ids = cache.get('last_comments')
if comments_ids:
comments = Comment.objects.filter(id__in=comments_ids)
else:
comments = fetch_last_comments()
comments_ids = [c.id for c in comments]
cache.set('last_comments', comments_ids)
...
Low level cache API

21. Caching!
Django Cacheback
from cacheback.decorators import cacheback
@cacheback
def fetch_last_comments_ids():
...
def last_comments(request):
comments_ids = fetch_last_comments_ids()
comments = Comment.objects.filter(id__in=comments_ids)
...

22. Template Compilation
Use
django.template.loaders.cached.Loader
TEMPLATE_LOADERS = (
('django.template.loaders.cached.Loader', (
'django.template.loaders.filesystem.Loader',
'django.template.loaders.app_directories.Loader',
)),
)

23. Template Fragment Caching
{% load cache %}
{% cache 500 “last_comments” %}
.. last comments ..
{% endcache %}
{% load cache %}
{% cache 500 “my_recent_comments” user.id %}
.. user’s recent comments ..
{% endcache %}

24. Don’t forget about the browser
80% or more of the end-user response
time is spent in the front end
Combine
Compress
Cache
Less is more
Load slow things later
Focus on making the
important things faster

25. Don’t forget about the browser
Google PageSpeed Insights
webpagetest.org
Chrome & Firefox

26. Assets
Django Compressor
{% compress js %}
<script src="/static/js/one.js"></script>
<script>obj.value = "value";</script>
{% endcompress %}
Django Assets
{% assets "js_all" %}
<script async src="{{ ASSET_URL }}"></script>
{% endassets %}

27. Q & Maybe A
@polmuz

28. References
Performance is a feature - http://blog.codinghorror.com/performance-is-a-feature/
Marissa Mayer at Web 2.0 - http://glinden.blogspot.com.ar/2006/11/marissa-mayer-at-web-20.html
Psychology of Web Performance - http://www.websiteoptimization.com/speed/tweak/psychology-web-performance/
New Relic - http://newrelic.com
Google Analytics - http://www.google.com/analytics/
Tracking Application Response Time with Nginx - http://lincolnloop.com/blog/tracking-application-response-time-nginx/
Logging Apache response times - http://www.moeding.net/archives/33-Logging-Apache-response-times.html
Django Debug Toolbar - http://django-debug-toolbar.readthedocs.org/
DDT Template Timings - https://github.com/orf/django-debug-toolbar-template-timings
Django Database Optimizations - https://docs.djangoproject.com/en/1.7/topics/db/optimization/
Two Hard Things - http://martinfowler.com/bliki/TwoHardThings.html
Django Cache Docs - https://docs.djangoproject.com/en/1.7/topics/cache/
Django Cacheback - http://django-cacheback.readthedocs.org/en/latest/index.html
Cached Templates - https://docs.djangoproject.com/en/1.7/ref/templates/api/#django.template.loaders.cached.Loader
Google PageSpeed Insights - https://developers.google.com/speed/pagespeed/insights/
Web Page Test - http://www.webpagetest.org/
Django Compressor - http://django-compressor.readthedocs.org/en/1.3/
Django Assets - http://django-assets.readthedocs.org/en/0.8/