Successfully reported this slideshow.

Django Performance Recipes

2

Share

Upcoming SlideShare
Django as your Backbone
Django as your Backbone
Loading in …3
×
1 of 41
1 of 41

Django Performance Recipes

2

Share

Download to read offline

I've written a lot of Django code in my career, some slow, and some fast; my aim is to share a few of the techniques you can use to go from 300ms response times to 30ms. We'll travel up and down the stack, looking to identify, monitor and solve performance issues, while dodging those which just aren't worth solving.

Full video is available at: https://skillsmatter.com/skillscasts/6628-django-performance-recipes

I've written a lot of Django code in my career, some slow, and some fast; my aim is to share a few of the techniques you can use to go from 300ms response times to 30ms. We'll travel up and down the stack, looking to identify, monitor and solve performance issues, while dodging those which just aren't worth solving.

Full video is available at: https://skillsmatter.com/skillscasts/6628-django-performance-recipes

More Related Content

Related Books

Free with a 14 day trial from Scribd

See all

Django Performance Recipes

  1. 1. Django Performance Recipes September 2015 Jon Atkinson Technical Director FARM Digital
  2. 2. Me • I am not a very good programmer. • I’m quite a good problem solver. • I have made a lot of mistakes. • I prefer valuable solutions.
  3. 3. My Context • “Fast-moving agency environment”. • We often participate in launches; demand is spiky. • Complex software.
  4. 4. Shared Context • Traffic is unpredictable. • We probably host in the cloud. • The Django ecosystem.
  5. 5. Performance Matters Speed is all about perception, but: 0-100ms Instant! 100-300ms Small delay. 300-1000ms Something is happening. 1000ms+ Likely task switch. 10000ms+ Abandoned task.
  6. 6. Rule of thumb 50% of users will abandon a task after 3 seconds.
  7. 7. Environment A web server, talking to: A managed python process, talking to: A cache, and a database.
  8. 8. Request Cycle Web Server ORM Middleware Views + Templates Web Server Tools
  9. 9. !
  10. 10. ORM • Optimising your database is a separate presentation entirely. • There are no ‘slow’ databases any more. • In a read-heavy environment, caching Querysets is a huge advantage.
  11. 11. ORM django-cachalot Caches your Django ORM queries and automatically invalidates them.
  12. 12. ORM $ pip install django-cachalot INSTALLED_APPS += (‘cachalot’) settings.CACHALOT_ENABLED = True
  13. 13. ORM from django.conf import settings from django.test.utils import override_settings with override_settings(CACHALOT_ENABLED=False): # SQL queries are not cached in this block @override_settings(CACHALOT_CACHE=‘second_cache') def your_function(): # What’s in this function uses another cache # Globally disables SQL caching until you set it back to True settings.CACHALOT_ENABLED = False
  14. 14. ORM • A few other data access tips: • Is your database doing DNS lookups? • Do you have a connection timeout set? The default is 0, and setup/teardown costs time. settings.DATABASES[‘…’][‘CONN_MAX_AGE’] = 600
  15. 15. Middleware Middleware is dumb.
  16. 16. Middleware
  17. 17. Middleware • Think hard. Milliseconds add up. • Middleware (and context processors!) often becomes a dumping ground for common features.
  18. 18. Middleware Middleware is helpful.
  19. 19. Middleware django.middleware.http.ConditionalGetMiddleware Optimises GET requests from modern browsers django.middleware.http.GZipMiddleware Compresses responses. But be aware of BREACH!
  20. 20. Middleware $ pip install django-cprofile-middleware “Once you've installed it, log in as a user who has staff privileges and add ?prof to any URL to see the profiler's stats.” eg. http://localhost:8000/foo/?prof.
  21. 21. Middleware 7986 function calls (7850 primitive calls) in 1.725 CPU seconds Ordered by: internal time, call count List reduced from 392 to 20 due to restriction <20> ncalls tottime percall cumtime percall filename:lineno(function) 2 1.570 0.785 1.570 0.785 /…/django/db/backends/__init__.py:36(_commit) 15 0.043 0.003 0.043 0.003 /…/linecache.py:68(updatecache) 1 0.020 0.020 0.027 0.027 /…/django/contrib/auth/models.py:1(<module>) 12 0.014 0.001 0.030 0.002 /…/django/utils/importlib.py:18(import_module) 1013 0.010 0.000 0.010 0.000 /…/posixpath.py:56(join)
  22. 22. Views & Templates • View performance problems are usually obvious: • Avoid nested loops (especially when generating QuerySets!) • Cache where you can. • Always return at the earliest possible moment.
  23. 23. Views & Templates • Templates are more interesting. • It’s easy to duplicate ORM calls already made in the view. • It’s easy to traverse relationships in the template language. • Templates are loaded from disk by default.
  24. 24. Views & Templates class Person(models.Model): def friends(self): # Hit the database here for something complex… return friends # View: if person.friends(): # Do something here… # Template: {% for friend in person.friends %}
  25. 25. Views & Templates from django.utils.functional import cached_property @cached_property def friends(self): # Hit the database here for something complex… return friends
  26. 26. Views & Templates • Caching template fragments is very powerful. • Sometimes you need to do something expensive in a template, but: {% load cache %} {% cache 500 sidebar %} … do something expensive here … {% endcache %}
  27. 27. Views & Templates • Where do your templates actually live? • Cloud disk performance can be erratic.
  28. 28. Views & Templates # Default setting. TEMPLATE_LOADERS = ( 'django.template.loaders.filesystem.Loader', 'django.template.loaders.app_directories.Loader', )
  29. 29. Views & Templates TEMPLATE_LOADERS = ( ('django.template.loaders.cached.Loader', ( 'django.template.loaders.filesystem.Loader', 'django.template.loaders.app_directories.Loader', )), )
  30. 30. nginx This is my “one weird tip”… … with a trade-off.
  31. 31. uwsgi_cache_path /tmp/nginx levels=1:2 keys_zone=my_zone:10m; server { listen 80; server_name example.com; … uwsgi_cache_use_stale error timeout invalid_header http_500; uwsgi_cache_valid 10m; location / { include uwsgi_params; uwsgi_pass unix:///tmp/example.sock; uwsgi_cache cache; uwsgi_cache_key $scheme:$host$request_uri:$request_method; uwsgi_cache_bypass $http_pragma $http_authorization; uwsgi_no_cache $http_pragma $http_authorization; }
  32. 32. nginx
  33. 33. Tools • The tool ecosystem is richer now than ever before. $ pip install django-debug-toolbar $ pip install dogslow • Measure twice, cut once. • Remember, Schrödinger’s web app.
  34. 34. Tools
  35. 35. Tools
  36. 36. import newrelic.agent from django.core.wsgi import get_wsgi_application application = get_wsgi_application() # This needs to be called after we bootstrapped the application # otherwise the settings wouldn't be configured from django.conf import settings # noqa if hasattr(settings, 'NEWRELIC_CONFIG'): newrelic.agent.initialize(settings.NEWRELIC_CONFIG getattr(settings, 'NEWRELIC_ENVIRONMENT', None)) application = newrelic.agent.WSGIApplicationWrapper(application)
  37. 37. Meta 1 Performance can be personal.
  38. 38. Meta 2
  39. 39. Meta 2
  40. 40. Meta 3 Performance is of huge value. 10x programmers probably don’t exist. Keep your eyes up. Concentrate on where the value lies.
  41. 41. Thank You Questions? Resources & Credits: http://blog.wearefarm.com

×