Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Speed is a feature PyConAr 2014

699 views

Published on

A mystical journey through Django performance optimization techniques, tools and gotchas.

Talk presented with Martín Blech at PyConAr November 2014

Published in: Software
  • Be the first to comment

Speed is a feature PyConAr 2014

  1. 1. This slide was intentionally left blank
  2. 2. Speed is a feature A mystical journey through Django performance optimization techniques, tools and gotchas @martinblech @polmuz
  3. 3. Roadmap How to find performance problems Sneak peak: Front-end performance How to fix them in Django
  4. 4. Why? [Google] found that Half a second delay caused a 20% drop in traffic. [Amazon] found that even very small delays would result in substantial and costly drops in revenue. Users really respond to speed 2006
  5. 5. There’s nothing like data Don’t start with the code. Profile and gather real usage data.
  6. 6. There’s nothing like real data Identify bottlenecks New Relic Very good Very paid Google Analytics Free of charge Less detail Your logs No data to third parties Harder to use Profiling Complex setup required Overhead
  7. 7. New Relic
  8. 8. New Relic
  9. 9. Google Analytics Site Speed
  10. 10. There’s nothing like data Let’s find the culprit! django-debug-toolbar django-debug-toolbar-template-timings
  11. 11. Time all the things!
  12. 12. Typical backend bottlenecks Database External Services CPU Intensive task Template Rendering
  13. 13. Database Missing index Big, unused fields Excessive # of queries Order
  14. 14. Missing Index class Comment(Model): ... created_at = DateTimeField(db_index=True) blogpost = ForeignKey(Blogpost) class Meta: index_together = [ ["created_at", "blogpost"], ]
  15. 15. select_related() >>> for c in Comment.objects.all(): print c.user.name # select * from comments; # select * from users where id = 1; # select * from users where id = 2; ... >>> comments = Comment.objects.all(); >>> for c in comments.select_related(“user”): print c.user.name # select comments.*, users.* # from comments, users # where comments.user_id = users.id;
  16. 16. prefech_related() >>> for u in User.objects.filter(id__lt=10): print len(u.comments.all()) # select * from users; # select * from comments where user_id = 1; # select * from comments where user_id = 2; ... >>> users = User.objects.filter(id__lt=10) >>> for u in users.prefetch_related(“comments”): print len(u.comments.all()) # select * from users where id < 10; # select * from comments # where user_id in (1,2,3,4,5,6,7,8,9); ## Joins them in python
  17. 17. Demo
  18. 18. Background Jobs Celery from celery import task @task def send_confirmation_email(user_id): ... def signup(req): ... send_confirmation_email.delay(req.user.id) return HttpResponseRedirect(“/home/”)
  19. 19. Template Compilation Use django.template.loaders.cached.Loader TEMPLATE_LOADERS = ( ('django.template.loaders.cached.Loader', ( 'django.template.loaders.filesystem.Loader', 'django.template.loaders.app_directories.Loader', )), )
  20. 20. Template Fragment Caching {% load cache %} {% cache 500 “last_comments” %} .. last comments .. {% endcache %} {% load cache %} {% cache 500 “my_recent_comments” user.id %} .. user’s recent comments .. {% endcache %}
  21. 21. Caching! Per view cache from django.views.decorators.cache import cache_page @cache_page(60 * 15) # seconds def my_view(request): ...
  22. 22. Caching! Per view cache Gotchas: User-specific content CSRF token
  23. 23. Caching! Low level cache API >>> from django.core.cache import get_cache >>> cache = get_cache('default') >>> cache.set('my_key', 'hello, world!', 30) >>> cache.get('my_key') >>> from django.core.cache import caches >>> cache = caches['default']
  24. 24. Caching! from django.core.cache import get_cache cache = get_cache('default') def last_comments(request): comments_ids = cache.get('last_comments') if comments_ids: comments = Comment.objects.filter(id__in=comments_ids) else: comments = fetch_last_comments() comments_ids = [c.id for c in comments] cache.set('last_comments', comments_ids) ... Low level cache API
  25. 25. Cache Invalidation “There are two hard things in computer science: cache invalidation, naming things, and off-by-one errors.” post_save/post_delete are a good place to start, but it doesn’t end there! @receiver(post_save, sender=Comment) @receiver(post_delete, sender=Comment) def invalidate_last_comments(sender, **kwargs): cache.delete('last_comments')
  26. 26. Caching! Django Cacheback from cacheback.decorators import cacheback @cacheback def fetch_last_comments_ids(): ... def last_comments(request): comments_ids = fetch_last_comments_ids() comments = Comment.objects.filter(id__in=comments_ids) ...
  27. 27. Don’t forget about the browser 80% or more of the end-user response time is spent in the front end Combine Compress Cache Less is more Load slow things later Focus on making the important things faster
  28. 28. Don’t forget about the browser Google PageSpeed Insights webpagetest.org Chrome & Firefox
  29. 29. Assets Django Compressor {% compress js %} <script src="/static/js/one.js"></script> <script>obj.value = "value";</script> {% endcompress %} Django Assets {% assets "js_all" %} <script async src="{{ ASSET_URL }}"></script> {% endassets %}
  30. 30. Assets aload.js <script data-aload="http://foo.com/foo.js"></script> <link data-aload="http://foo.com/foo.css" rel="stylesheet">
  31. 31. Assets Shameless plug: django-critical {% critical %} <link rel="stylesheet" href="bootstrap.min.css"> {% endcritical %} Alpha stage, use at your own risk!
  32. 32. Q & Maybe A
  33. 33. References Performance is a feature - http://blog.codinghorror.com/performance-is-a-feature/ Marissa Mayer at Web 2.0 - http://glinden.blogspot.com.ar/2006/11/marissa-mayer-at-web-20.html Psychology of Web Performance - http://www.websiteoptimization.com/speed/tweak/psychology-web-performance/ New Relic - http://newrelic.com Google Analytics - http://www.google.com/analytics/ Tracking Application Response Time with Nginx - http://lincolnloop.com/blog/tracking-application-response-time-nginx/ Logging Apache response times - http://www.moeding.net/archives/33-Logging-Apache-response-times.html Django Debug Toolbar - http://django-debug-toolbar.readthedocs.org/ DDT Template Timings - https://github.com/orf/django-debug-toolbar-template-timings Django Database Optimizations - https://docs.djangoproject.com/en/1.7/topics/db/optimization/ Two Hard Things - http://martinfowler.com/bliki/TwoHardThings.html Django Cache Docs - https://docs.djangoproject.com/en/1.7/topics/cache/ Django Cacheback - http://django-cacheback.readthedocs.org/en/latest/index.html Cached Templates - https://docs.djangoproject.com/en/1.7/ref/templates/api/#django.template.loaders.cached.Loader Google PageSpeed Insights - https://developers.google.com/speed/pagespeed/insights/ Web Page Test - http://www.webpagetest.org/ Django Compressor - http://django-compressor.readthedocs.org/en/1.3/ Django Assets - http://django-assets.readthedocs.org/en/0.8/ aload.js - https://github.com/pazguille/aload django-critical - https://github.com/martinblech/django-critical Demo - https://github.com/martinblech/pyconar2014_perfdemo

×