What are you here for
Aspects of site performance
Measuring everything
Fixing the right thing
AND HOW
Performance focus for this talk
✔ user perspective
✔ seconds per request
✗ benchmarks
✗ users per server
✗ requests per se...
Why should we care?
● Web application performance affects
– engagement
Why should we care?
● Web application performance affects
– engagement
– bounce rate
http://www.slideshare.net/joshfraz/se...
Why should we care?
● Web application performance affects
– engagement
– bounce rate
– conversion/abandonment
Why should we care?
● Web application performance affects
– engagement
– bounce rate
– conversion/abandonment
– search eng...
Why should we care?
● Web application performance affects
– engagement
– bounce rate
– conversion/abandonment
– search eng...
Why should we care?
Web sites are
larger
&
than ever
Why should we care?
Average web page:
1.3Mb
85 requests
- HTTPArchive.org
Why should we care?
Users are increasingly mobile
Their expectations are up
Many "responsive" sites are
no smaller than th...
Let's look at a
web request!
There's a redirect!
How about
some
CSS?
Let's get
some
JavaScript
and images
Start Render:
1.6s
Document complete: 8.6s
Fully loaded: 9.0s
Requests: 177
Bytes in: 2,689kb
This is on a good connection
The Performance Golden Rule
"80-90% of the end-user response time
is spent on the frontend.
Start there."
- Steve Souders
...
Front-end Measurement
● Instrument your live application for Real User
Monitoring (RUM)
– Google Analytics
– New Relic
Front-end Measurement
● Synthetic testing
● Lots of tools available
● 3 in particular...
Chrome Developer Tools
webpagetest.org
Browser navigation timing
http://www.w3.org/TR/navigation-timing/
django-debug-toolbar
Not in PyPi yet,
see GitHub version
Back end Front end
Front-end offenders
● Static resources – size and number
● Lack of caching and compression
● 3rd-party resources
● Overdow...
Static Resources
● Minification/combination of CSS/JavaScript
– django-compressor django-pipeline
● Images
– Appropriate f...
Static Resources
● Web server/accelerator config
– Caching/expires/compression headers
● Static files to CDN
– see Django'...
Ok, now what?
Back-end measurement
Production instrumentation with New Relic
django-statsd and graphite
django-debug-toolbar
runprofileserver in
django-extensions
Web server
WSGI application (WSGIHandler.__call__)
Initialise Request
Get Response (BaseHandler.get_response)
URL Resoluti...
Caching Dynamic Requests
● Potentially easy wins
● Beware cache invalidation
● Different levels
– Caching headers
– Web Se...
Web server / Reverse proxy
WSGI application (WSGIHandler.__call__)
Initialise request
Get Response (BaseHandler.get_respon...
Compressing Dynamic Content
Web server
Web accelerator
gzip middleware
Query Performance
● Use debug toolbar
● Avoid unnecessary queries
– eg exists followed by separate get
● select_related to...
Defer work
● Defer user-irrelevant workloads
● Target
– Views
– Signals (eg post_save)
– Middleware
Defer work with celery
def login(request):
# Your login code here
# ...
user = form.save()
upload_to_nsa.delay(user.id)
re...
Ok, now what?
Web server
WSGI application (WSGIHandler.__call__)
Initialise request
Get Response (BaseHandler.get_response)
URL Resoluti...
Can we stream?
● StreamingHttpResponse in 1.5
● Patch for streaming templates (#13910)
● Caveats
– Middleware compatibilit...
What about eager streaming?
Hack: View middleware that streams the top of a
page before even calling the view
Proof of con...
Regular
Streaming
Eager streaming
Visual appearance
Future context
● APIs, websockets and JavaScript, oh my!
● SPDY
● Mobile
● Full stack ownership
Summary
● Measure, measure, measure
● Don't ignore the front-end
● Beware diminishing returns
● Technology is changing
Resources
● General web performance
– http://www.slideshare.net/Strangeloopnet/37-lessons-
ive-learned-on-the-performance-...
Resources
●
Django
– https://docs.djangoproject.com/en/dev/topics/cache/
– https://code.djangoproject.com/wiki/ProfilingDj...
Questions?
http://slideshare.net/mindsocket
Thank You!
The life of a web request - techniques for measuring and improving Django application performance
The life of a web request - techniques for measuring and improving Django application performance
The life of a web request - techniques for measuring and improving Django application performance
The life of a web request - techniques for measuring and improving Django application performance
Upcoming SlideShare
Loading in...5
×

The life of a web request - techniques for measuring and improving Django application performance

1,196

Published on

The connection between sub-second web application performance and revenue is becoming more and more apparent with established companies regularly reporting the benefits of reducing page load times.

This talk will cover:
* Designing for performance
* Approaches to instrumenting and measuring application performance
* Areas of focus for both front-end and back-end improvement
* Techniques, tools and modules available in Django-land for improving performance
* New and emerging technologies, for example SPDY protocol and Django 1.5's StreamingHttpResponse

Published in: Technology, Design
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,196
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
13
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

The life of a web request - techniques for measuring and improving Django application performance

  1. 1. The Life of a Web Request Measuring and improving Django application performance PyCon Australia 2013 Roger Barnes @mindsocket roger@mindsocket.com.au http://slideshare.net/mindsocket/
  2. 2. What are you here for Aspects of site performance Measuring everything Fixing the right thing AND HOW
  3. 3. Performance focus for this talk ✔ user perspective ✔ seconds per request ✗ benchmarks ✗ users per server ✗ requests per second
  4. 4. Why should we care? ● Web application performance affects – engagement
  5. 5. Why should we care? ● Web application performance affects – engagement – bounce rate http://www.slideshare.net/joshfraz/sept-2012rumtalk
  6. 6. Why should we care? ● Web application performance affects – engagement – bounce rate – conversion/abandonment
  7. 7. Why should we care? ● Web application performance affects – engagement – bounce rate – conversion/abandonment – search engine ranking
  8. 8. Why should we care? ● Web application performance affects – engagement – bounce rate – conversion/abandonment – search engine ranking – revenue
  9. 9. Why should we care? Web sites are larger & than ever
  10. 10. Why should we care? Average web page: 1.3Mb 85 requests - HTTPArchive.org
  11. 11. Why should we care? Users are increasingly mobile Their expectations are up Many "responsive" sites are no smaller than their desktop version
  12. 12. Let's look at a web request!
  13. 13. There's a redirect!
  14. 14. How about some CSS?
  15. 15. Let's get some JavaScript and images Start Render: 1.6s
  16. 16. Document complete: 8.6s Fully loaded: 9.0s Requests: 177 Bytes in: 2,689kb This is on a good connection
  17. 17. The Performance Golden Rule "80-90% of the end-user response time is spent on the frontend. Start there." - Steve Souders Source: http://www.stevesouders.com/blog/2012/02/10/the-performance-golden-rule/
  18. 18. Front-end Measurement ● Instrument your live application for Real User Monitoring (RUM) – Google Analytics – New Relic
  19. 19. Front-end Measurement ● Synthetic testing ● Lots of tools available ● 3 in particular...
  20. 20. Chrome Developer Tools
  21. 21. webpagetest.org
  22. 22. Browser navigation timing http://www.w3.org/TR/navigation-timing/
  23. 23. django-debug-toolbar Not in PyPi yet, see GitHub version Back end Front end
  24. 24. Front-end offenders ● Static resources – size and number ● Lack of caching and compression ● 3rd-party resources ● Overdownloading
  25. 25. Static Resources ● Minification/combination of CSS/JavaScript – django-compressor django-pipeline ● Images – Appropriate format and compression level – Correct size – Remove metadata ● jpegoptim, optiPNG – Consider sprites
  26. 26. Static Resources ● Web server/accelerator config – Caching/expires/compression headers ● Static files to CDN – see Django's storage hooks ● Deferred loading ● Silver bullet (?) – mod_pagespeed (apache) – ngx_pagespeed (nginx)
  27. 27. Ok, now what?
  28. 28. Back-end measurement Production instrumentation with New Relic
  29. 29. django-statsd and graphite
  30. 30. django-debug-toolbar
  31. 31. runprofileserver in django-extensions
  32. 32. Web server WSGI application (WSGIHandler.__call__) Initialise Request Get Response (BaseHandler.get_response) URL Resolution Request Middleware Response Middleware View Middleware Template Response Middleware View Template Rendering Backend Services Typical Django Request
  33. 33. Caching Dynamic Requests ● Potentially easy wins ● Beware cache invalidation ● Different levels – Caching headers – Web Server/Web Accelerator – Caching middleware – Per view caching – Template fragment caching – Low level object caching
  34. 34. Web server / Reverse proxy WSGI application (WSGIHandler.__call__) Initialise request Get Response (BaseHandler.get_response) URL Resolution Request Middleware Response Middleware View Middleware Template Response Middleware View Template rendering Backend services Typical Django Request Caching Browser
  35. 35. Compressing Dynamic Content Web server Web accelerator gzip middleware
  36. 36. Query Performance ● Use debug toolbar ● Avoid unnecessary queries – eg exists followed by separate get ● select_related to get FKs in same query – Be wary of optional foreign keys ● prefetch_related to get collections – 1 additional query instead of N+1
  37. 37. Defer work ● Defer user-irrelevant workloads ● Target – Views – Signals (eg post_save) – Middleware
  38. 38. Defer work with celery def login(request): # Your login code here # ... user = form.save() upload_to_nsa.delay(user.id) return HttpResponse(...)
  39. 39. Ok, now what?
  40. 40. Web server WSGI application (WSGIHandler.__call__) Initialise request Get Response (BaseHandler.get_response) URL Resolution Request Middleware Response Middleware View Middleware Template Response Middleware View Template rendering Backend services Django Works Then Responds Bytes on the wire (Time to First Byte) Back end
  41. 41. Can we stream? ● StreamingHttpResponse in 1.5 ● Patch for streaming templates (#13910) ● Caveats – Middleware compatibility – Exception handling – YAGNI?
  42. 42. What about eager streaming? Hack: View middleware that streams the top of a page before even calling the view Proof of concept: https://github.com/mindsocket/ django-perf-example
  43. 43. Regular Streaming Eager streaming
  44. 44. Visual appearance
  45. 45. Future context ● APIs, websockets and JavaScript, oh my! ● SPDY ● Mobile ● Full stack ownership
  46. 46. Summary ● Measure, measure, measure ● Don't ignore the front-end ● Beware diminishing returns ● Technology is changing
  47. 47. Resources ● General web performance – http://www.slideshare.net/Strangeloopnet/37-lessons- ive-learned-on-the-performance-front-lines – http://www.webpagetest.org/ – https://developers.google.com/speed/pagespeed/ – http://www.mnot.net/cache_docs/ – http://newrelic.com/ – https://dvcs.w3.org/hg/webperf/raw- file/tip/specs/NavigationTiming/Overview.html – https://www.varnish-cache.org/
  48. 48. Resources ● Django – https://docs.djangoproject.com/en/dev/topics/cache/ – https://code.djangoproject.com/wiki/ProfilingDjango – https://code.google.com/p/django-command-extensions/wiki/RunProfileServer – https://developers.google.com/chrome-developer-tools/ – https://docs.djangoproject.com/en/dev/howto/static-files/deployment/#staticfiles- from-cdn – https://docs.djangoproject.com/en/dev/howto/custom-file-storage/ – https://github.com/django-debug-toolbar/django-debug-toolbar/issues/319 – https://github.com/sorl/sorl- thumbnail/commit/e5f493a340dea8a33703c05fd66ee76d2f019881 – https://github.com/aaugustin/django-c10k-demo – https://github.com/mindsocket/django-perf-example – https://www.djangopackages.com/grids/g/asset-managers/
  49. 49. Questions? http://slideshare.net/mindsocket Thank You!

×