Web Performance Culture and Tools at Etsy

58,453 views

Published on

Presented at the NY Web Performance Meetup on Jan 18, 2012.

Published in: Technology, Design
7 Comments
65 Likes
Statistics
Notes
No Downloads
Views
Total views
58,453
On SlideShare
0
From Embeds
0
Number of Embeds
5,846
Actions
Shares
0
Downloads
300
Comments
7
Likes
65
Embeds 0
No embeds

No notes for slide

Web Performance Culture and Tools at Etsy

  1. Web PerformanceCulture and Toolsat Etsy Mike Brittain @mikebrittain Director of Eng., Infrastructure Etsy
  2. All of this started in the last year.
  3. Make PerformanceMatter
  4. $0.20 per item3.5% transaction fee
  5. Focus on the marketplace
  6. ecommerce
  7. ecommercead supported
  8. ecommercead supportedsubscription
  9. ecommercead supportedsubscription“free”
  10. $$$
  11. “Prove it.”
  12. “Our own years of testinghave conclusively shown that when speed of a feature or product improves, usage, quite simply, goes up.” http://googleblog.blogspot.com/2009/12/this-week-in-search-121809.html
  13. 100+ web, api, jobs40+ databasesMany more doing lots of other stuff...
  14. $$$
  15. Performance is fun
  16. I want you to tell me how you feel...
  17. 95th percentilemilliseconds Average
  18. e w r! en ste h a 95th percentile w f is go is it h e T dmilliseconds m a Average
  19. I am better than you.
  20. Make PerformanceMatter
  21. PerformanceCulture
  22. Our perf team is two people...
  23. Measure
  24. MeasureBuild tools
  25. MeasureBuild toolsTrain & Coordinate
  26. MeasureBuild toolsTrain & CoordinateMotivate
  27. MeasureBuild toolsTrain & CoordinateMotivateDedicated
  28. Our perf team is two people... But every engineer thinks about performance
  29. Everything is slow.Please fix.
  30. page generationtime
  31. 2011 Goal800 ms on five pages
  32. http://www.etsy.com/blog/news/2011/november-2011-site-performance-report/
  33. http://www.etsy.com/blog/news/2011/november-2011-site-performance-report/
  34. Performance degradationis a P2 bug
  35. WARNING: This part is hardPerformance degradationis a P2 bug
  36. 2012 Goal800 ms for all pages
  37. EngineeringDenormalize dataOptimize queriesRewrite from scratchCaching (memcache, app code)
  38. OperationsNetwork switchesOperating systemsNewer hardwareAdditional hardware capacity
  39. ProductIdentify concerns earlyKill lagging features
  40. PerformanceCulture
  41. Tools
  42. How do you makethe graphs?
  43. Perf-related data belongs in your server logs
  44. apache_note(“custom_field”, $whatever);LogFormat %l %t "%r" %>s %b "%{Referer}i""%{User-Agent}i" %{custom_field}n
  45. LogFormat "%{True-Client-IP}i %l %u %t "%r"%>s %b "%{Referer}i" "%{User-Agent}i"%{etsy_user_id}n %{php_bytes}n %{php_microsec}n%D" combined
  46. $timer_start = microtime(true); ...$timer_diff = microtime(true) - $timer_start;register_shutdown_function()apache_note(php_microsec, $timer_diff);
  47. web0060 66.249.71.110 - - [24/Aug/2011:04:16:52+0000] "GET /listing/12189259/tropical-etched-pair-of-lampwork-glass HTTP/1.1" 200 11034 "-""Mozilla/5.0 (compatible; Googlebot/2.1;+http://www.google.com/bot.html)"- 13399576 505780 554876
  48. Using something else?time, http method, request uri, response code, referer,user-agent, response time, response memory,custom segmentation fields...
  49. Quick averagesgrep "GET /listing/" access.log | awk {sum=sum+$(NF-1)} END {print sum/NR}
  50. Quick graphsgrep "GET /listing/" access.log | perl -pe "s/.*[.*d{4}:(d{2}):(d{2}):d{2}.*]/1:2/" | awk {print $1, $(NF-1)} > /tmp/pagetimes.dat gives you...
  51. Quick graphs# /tmp/pagetimes.dat18:37 251.018:38 252.118:39 253.518:40 251.018:45 250.0 and then...
  52. Quick graphs# GNUPLOTset terminal pngset output listings.pngset yrange [0:2000]set xdata timeset timefmt "%d/%B/%Y:%H:%M:%S"set format x "%H:%M"plot /tmp/pagetimes.dat using 1:2 with points
  53. Quick graphs
  54. LogsterRun by cronMaintains a cursor on log filesSimple parsing & aggregationOutput to Ganglia or Graphite github.com/etsy
  55. Logster & Graphite Signed-in vs. Signed-out
  56. Response timesresponses (pages) by bucket time (ms)
  57. “Look at your Data”John Rauser, Velocity 2011 http://www.youtube.com/watch?v=coNDCIMH8bk
  58. Idea lifted fromhttp://www.codinghorror.com/blog/2011/06/performance-is-a-feature.html
  59. Don’t forget the API
  60. Measureperformancedata in Production
  61. Profiling
  62. “I/O Profiler”Lightweight, inline profilingStart and end times wrapped around service calls databases, memcache, apc, 3rd party services, etc.Enable with query string
  63. “I/O Profiler”
  64. “I/O Profiler”
  65. XHProfgithub.com/facebook/xhprofDeployed on pre-prod machineDirect access to profilingEnable with query string
  66. Profile againstProduction
  67. Make performance matter
  68. Start small
  69. Make graphs and reports
  70. Create simple tools
  71. Celebrate performance wins
  72. Thank you codeascraft.etsy.comMike Brittainmike@etsy.com@mikebrittain
  73. Jobs at Etsy We are currently hiring for a number of positions in engineering and operations. http://www.etsy.com/careers

×