Your SlideShare is downloading. ×
  • Like
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Now you can save presentations on your phone or tablet

Available for both IPhone and Android

Text the download link to your phone

Standard text messaging rates apply

The life of a web request - techniques for measuring and improving Django application performance

  • 975 views
Published

The connection between sub-second web application performance and revenue is becoming more and more apparent with established companies regularly reporting the benefits of reducing page load …

The connection between sub-second web application performance and revenue is becoming more and more apparent with established companies regularly reporting the benefits of reducing page load times.

This talk will cover:
* Designing for performance
* Approaches to instrumenting and measuring application performance
* Areas of focus for both front-end and back-end improvement
* Techniques, tools and modules available in Django-land for improving performance
* New and emerging technologies, for example SPDY protocol and Django 1.5's StreamingHttpResponse

Published in Technology , Design
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
975
On SlideShare
0
From Embeds
0
Number of Embeds
4

Actions

Shares
Downloads
12
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The Life of a Web Request Measuring and improving Django application performance PyCon Australia 2013 Roger Barnes @mindsocket roger@mindsocket.com.au http://slideshare.net/mindsocket/
  • 2. What are you here for Aspects of site performance Measuring everything Fixing the right thing AND HOW
  • 3. Performance focus for this talk ✔ user perspective ✔ seconds per request ✗ benchmarks ✗ users per server ✗ requests per second
  • 4. Why should we care? ● Web application performance affects – engagement
  • 5. Why should we care? ● Web application performance affects – engagement – bounce rate http://www.slideshare.net/joshfraz/sept-2012rumtalk
  • 6. Why should we care? ● Web application performance affects – engagement – bounce rate – conversion/abandonment
  • 7. Why should we care? ● Web application performance affects – engagement – bounce rate – conversion/abandonment – search engine ranking
  • 8. Why should we care? ● Web application performance affects – engagement – bounce rate – conversion/abandonment – search engine ranking – revenue
  • 9. Why should we care? Web sites are larger & than ever
  • 10. Why should we care? Average web page: 1.3Mb 85 requests - HTTPArchive.org
  • 11. Why should we care? Users are increasingly mobile Their expectations are up Many "responsive" sites are no smaller than their desktop version
  • 12. Let's look at a web request!
  • 13. There's a redirect!
  • 14. How about some CSS?
  • 15. Let's get some JavaScript and images Start Render: 1.6s
  • 16. Document complete: 8.6s Fully loaded: 9.0s Requests: 177 Bytes in: 2,689kb This is on a good connection
  • 17. The Performance Golden Rule "80-90% of the end-user response time is spent on the frontend. Start there." - Steve Souders Source: http://www.stevesouders.com/blog/2012/02/10/the-performance-golden-rule/
  • 18. Front-end Measurement ● Instrument your live application for Real User Monitoring (RUM) – Google Analytics – New Relic
  • 19. Front-end Measurement ● Synthetic testing ● Lots of tools available ● 3 in particular...
  • 20. Chrome Developer Tools
  • 21. webpagetest.org
  • 22. Browser navigation timing http://www.w3.org/TR/navigation-timing/
  • 23. django-debug-toolbar Not in PyPi yet, see GitHub version Back end Front end
  • 24. Front-end offenders ● Static resources – size and number ● Lack of caching and compression ● 3rd-party resources ● Overdownloading
  • 25. Static Resources ● Minification/combination of CSS/JavaScript – django-compressor django-pipeline ● Images – Appropriate format and compression level – Correct size – Remove metadata ● jpegoptim, optiPNG – Consider sprites
  • 26. Static Resources ● Web server/accelerator config – Caching/expires/compression headers ● Static files to CDN – see Django's storage hooks ● Deferred loading ● Silver bullet (?) – mod_pagespeed (apache) – ngx_pagespeed (nginx)
  • 27. Ok, now what?
  • 28. Back-end measurement Production instrumentation with New Relic
  • 29. django-statsd and graphite
  • 30. django-debug-toolbar
  • 31. runprofileserver in django-extensions
  • 32. Web server WSGI application (WSGIHandler.__call__) Initialise Request Get Response (BaseHandler.get_response) URL Resolution Request Middleware Response Middleware View Middleware Template Response Middleware View Template Rendering Backend Services Typical Django Request
  • 33. Caching Dynamic Requests ● Potentially easy wins ● Beware cache invalidation ● Different levels – Caching headers – Web Server/Web Accelerator – Caching middleware – Per view caching – Template fragment caching – Low level object caching
  • 34. Web server / Reverse proxy WSGI application (WSGIHandler.__call__) Initialise request Get Response (BaseHandler.get_response) URL Resolution Request Middleware Response Middleware View Middleware Template Response Middleware View Template rendering Backend services Typical Django Request Caching Browser
  • 35. Compressing Dynamic Content Web server Web accelerator gzip middleware
  • 36. Query Performance ● Use debug toolbar ● Avoid unnecessary queries – eg exists followed by separate get ● select_related to get FKs in same query – Be wary of optional foreign keys ● prefetch_related to get collections – 1 additional query instead of N+1
  • 37. Defer work ● Defer user-irrelevant workloads ● Target – Views – Signals (eg post_save) – Middleware
  • 38. Defer work with celery def login(request): # Your login code here # ... user = form.save() upload_to_nsa.delay(user.id) return HttpResponse(...)
  • 39. Ok, now what?
  • 40. Web server WSGI application (WSGIHandler.__call__) Initialise request Get Response (BaseHandler.get_response) URL Resolution Request Middleware Response Middleware View Middleware Template Response Middleware View Template rendering Backend services Django Works Then Responds Bytes on the wire (Time to First Byte) Back end
  • 41. Can we stream? ● StreamingHttpResponse in 1.5 ● Patch for streaming templates (#13910) ● Caveats – Middleware compatibility – Exception handling – YAGNI?
  • 42. What about eager streaming? Hack: View middleware that streams the top of a page before even calling the view Proof of concept: https://github.com/mindsocket/ django-perf-example
  • 43. Regular Streaming Eager streaming
  • 44. Visual appearance
  • 45. Future context ● APIs, websockets and JavaScript, oh my! ● SPDY ● Mobile ● Full stack ownership
  • 46. Summary ● Measure, measure, measure ● Don't ignore the front-end ● Beware diminishing returns ● Technology is changing
  • 47. Resources ● General web performance – http://www.slideshare.net/Strangeloopnet/37-lessons- ive-learned-on-the-performance-front-lines – http://www.webpagetest.org/ – https://developers.google.com/speed/pagespeed/ – http://www.mnot.net/cache_docs/ – http://newrelic.com/ – https://dvcs.w3.org/hg/webperf/raw- file/tip/specs/NavigationTiming/Overview.html – https://www.varnish-cache.org/
  • 48. Resources ● Django – https://docs.djangoproject.com/en/dev/topics/cache/ – https://code.djangoproject.com/wiki/ProfilingDjango – https://code.google.com/p/django-command-extensions/wiki/RunProfileServer – https://developers.google.com/chrome-developer-tools/ – https://docs.djangoproject.com/en/dev/howto/static-files/deployment/#staticfiles- from-cdn – https://docs.djangoproject.com/en/dev/howto/custom-file-storage/ – https://github.com/django-debug-toolbar/django-debug-toolbar/issues/319 – https://github.com/sorl/sorl- thumbnail/commit/e5f493a340dea8a33703c05fd66ee76d2f019881 – https://github.com/aaugustin/django-c10k-demo – https://github.com/mindsocket/django-perf-example – https://www.djangopackages.com/grids/g/asset-managers/
  • 49. Questions? http://slideshare.net/mindsocket Thank You!