The life of a web request - techniques for measuring and improving Django application performance

  • 915 views
Uploaded on

The connection between sub-second web application performance and revenue is becoming more and more apparent with established companies regularly reporting the benefits of reducing page load …

The connection between sub-second web application performance and revenue is becoming more and more apparent with established companies regularly reporting the benefits of reducing page load times.

This talk will cover:
* Designing for performance
* Approaches to instrumenting and measuring application performance
* Areas of focus for both front-end and back-end improvement
* Techniques, tools and modules available in Django-land for improving performance
* New and emerging technologies, for example SPDY protocol and Django 1.5's StreamingHttpResponse

More in: Technology , Design
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
915
On Slideshare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
9
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The Life of a Web Request Measuring and improving Django application performance PyCon Australia 2013 Roger Barnes @mindsocket roger@mindsocket.com.au http://slideshare.net/mindsocket/
  • 2. What are you here for Aspects of site performance Measuring everything Fixing the right thing AND HOW
  • 3. Performance focus for this talk ✔ user perspective ✔ seconds per request ✗ benchmarks ✗ users per server ✗ requests per second
  • 4. Why should we care? ● Web application performance affects – engagement
  • 5. Why should we care? ● Web application performance affects – engagement – bounce rate http://www.slideshare.net/joshfraz/sept-2012rumtalk
  • 6. Why should we care? ● Web application performance affects – engagement – bounce rate – conversion/abandonment
  • 7. Why should we care? ● Web application performance affects – engagement – bounce rate – conversion/abandonment – search engine ranking
  • 8. Why should we care? ● Web application performance affects – engagement – bounce rate – conversion/abandonment – search engine ranking – revenue
  • 9. Why should we care? Web sites are larger & than ever
  • 10. Why should we care? Average web page: 1.3Mb 85 requests - HTTPArchive.org
  • 11. Why should we care? Users are increasingly mobile Their expectations are up Many "responsive" sites are no smaller than their desktop version
  • 12. Let's look at a web request!
  • 13. There's a redirect!
  • 14. How about some CSS?
  • 15. Let's get some JavaScript and images Start Render: 1.6s
  • 16. Document complete: 8.6s Fully loaded: 9.0s Requests: 177 Bytes in: 2,689kb This is on a good connection
  • 17. The Performance Golden Rule "80-90% of the end-user response time is spent on the frontend. Start there." - Steve Souders Source: http://www.stevesouders.com/blog/2012/02/10/the-performance-golden-rule/
  • 18. Front-end Measurement ● Instrument your live application for Real User Monitoring (RUM) – Google Analytics – New Relic
  • 19. Front-end Measurement ● Synthetic testing ● Lots of tools available ● 3 in particular...
  • 20. Chrome Developer Tools
  • 21. webpagetest.org
  • 22. Browser navigation timing http://www.w3.org/TR/navigation-timing/
  • 23. django-debug-toolbar Not in PyPi yet, see GitHub version Back end Front end
  • 24. Front-end offenders ● Static resources – size and number ● Lack of caching and compression ● 3rd-party resources ● Overdownloading
  • 25. Static Resources ● Minification/combination of CSS/JavaScript – django-compressor django-pipeline ● Images – Appropriate format and compression level – Correct size – Remove metadata ● jpegoptim, optiPNG – Consider sprites
  • 26. Static Resources ● Web server/accelerator config – Caching/expires/compression headers ● Static files to CDN – see Django's storage hooks ● Deferred loading ● Silver bullet (?) – mod_pagespeed (apache) – ngx_pagespeed (nginx)
  • 27. Ok, now what?
  • 28. Back-end measurement Production instrumentation with New Relic
  • 29. django-statsd and graphite
  • 30. django-debug-toolbar
  • 31. runprofileserver in django-extensions
  • 32. Web server WSGI application (WSGIHandler.__call__) Initialise Request Get Response (BaseHandler.get_response) URL Resolution Request Middleware Response Middleware View Middleware Template Response Middleware View Template Rendering Backend Services Typical Django Request
  • 33. Caching Dynamic Requests ● Potentially easy wins ● Beware cache invalidation ● Different levels – Caching headers – Web Server/Web Accelerator – Caching middleware – Per view caching – Template fragment caching – Low level object caching
  • 34. Web server / Reverse proxy WSGI application (WSGIHandler.__call__) Initialise request Get Response (BaseHandler.get_response) URL Resolution Request Middleware Response Middleware View Middleware Template Response Middleware View Template rendering Backend services Typical Django Request Caching Browser
  • 35. Compressing Dynamic Content Web server Web accelerator gzip middleware
  • 36. Query Performance ● Use debug toolbar ● Avoid unnecessary queries – eg exists followed by separate get ● select_related to get FKs in same query – Be wary of optional foreign keys ● prefetch_related to get collections – 1 additional query instead of N+1
  • 37. Defer work ● Defer user-irrelevant workloads ● Target – Views – Signals (eg post_save) – Middleware
  • 38. Defer work with celery def login(request): # Your login code here # ... user = form.save() upload_to_nsa.delay(user.id) return HttpResponse(...)
  • 39. Ok, now what?
  • 40. Web server WSGI application (WSGIHandler.__call__) Initialise request Get Response (BaseHandler.get_response) URL Resolution Request Middleware Response Middleware View Middleware Template Response Middleware View Template rendering Backend services Django Works Then Responds Bytes on the wire (Time to First Byte) Back end
  • 41. Can we stream? ● StreamingHttpResponse in 1.5 ● Patch for streaming templates (#13910) ● Caveats – Middleware compatibility – Exception handling – YAGNI?
  • 42. What about eager streaming? Hack: View middleware that streams the top of a page before even calling the view Proof of concept: https://github.com/mindsocket/ django-perf-example
  • 43. Regular Streaming Eager streaming
  • 44. Visual appearance
  • 45. Future context ● APIs, websockets and JavaScript, oh my! ● SPDY ● Mobile ● Full stack ownership
  • 46. Summary ● Measure, measure, measure ● Don't ignore the front-end ● Beware diminishing returns ● Technology is changing
  • 47. Resources ● General web performance – http://www.slideshare.net/Strangeloopnet/37-lessons- ive-learned-on-the-performance-front-lines – http://www.webpagetest.org/ – https://developers.google.com/speed/pagespeed/ – http://www.mnot.net/cache_docs/ – http://newrelic.com/ – https://dvcs.w3.org/hg/webperf/raw- file/tip/specs/NavigationTiming/Overview.html – https://www.varnish-cache.org/
  • 48. Resources ● Django – https://docs.djangoproject.com/en/dev/topics/cache/ – https://code.djangoproject.com/wiki/ProfilingDjango – https://code.google.com/p/django-command-extensions/wiki/RunProfileServer – https://developers.google.com/chrome-developer-tools/ – https://docs.djangoproject.com/en/dev/howto/static-files/deployment/#staticfiles- from-cdn – https://docs.djangoproject.com/en/dev/howto/custom-file-storage/ – https://github.com/django-debug-toolbar/django-debug-toolbar/issues/319 – https://github.com/sorl/sorl- thumbnail/commit/e5f493a340dea8a33703c05fd66ee76d2f019881 – https://github.com/aaugustin/django-c10k-demo – https://github.com/mindsocket/django-perf-example – https://www.djangopackages.com/grids/g/asset-managers/
  • 49. Questions? http://slideshare.net/mindsocket Thank You!