Web Performance Culture and Tools at Etsy

  • 48,594 views
Uploaded on

Presented at the NY Web Performance Meetup on Jan 18, 2012.

Presented at the NY Web Performance Meetup on Jan 18, 2012.

More in: Technology , Design
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
48,594
On Slideshare
0
From Embeds
0
Number of Embeds
17

Actions

Shares
Downloads
261
Comments
6
Likes
58

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Web PerformanceCulture and Toolsat Etsy Mike Brittain @mikebrittain Director of Eng., Infrastructure Etsy
  • 2. All of this started in the last year.
  • 3. Make PerformanceMatter
  • 4. $0.20 per item3.5% transaction fee
  • 5. Focus on the marketplace
  • 6. ecommerce
  • 7. ecommercead supported
  • 8. ecommercead supportedsubscription
  • 9. ecommercead supportedsubscription“free”
  • 10. $$$
  • 11. “Prove it.”
  • 12. “Our own years of testinghave conclusively shown that when speed of a feature or product improves, usage, quite simply, goes up.” http://googleblog.blogspot.com/2009/12/this-week-in-search-121809.html
  • 13. 100+ web, api, jobs40+ databasesMany more doing lots of other stuff...
  • 14. $$$
  • 15. Performance is fun
  • 16. I want you to tell me how you feel...
  • 17. 95th percentilemilliseconds Average
  • 18. e w r! en ste h a 95th percentile w f is go is it h e T dmilliseconds m a Average
  • 19. I am better than you.
  • 20. Make PerformanceMatter
  • 21. PerformanceCulture
  • 22. Our perf team is two people...
  • 23. Measure
  • 24. MeasureBuild tools
  • 25. MeasureBuild toolsTrain & Coordinate
  • 26. MeasureBuild toolsTrain & CoordinateMotivate
  • 27. MeasureBuild toolsTrain & CoordinateMotivateDedicated
  • 28. Our perf team is two people... But every engineer thinks about performance
  • 29. Everything is slow.Please fix.
  • 30. page generationtime
  • 31. 2011 Goal800 ms on five pages
  • 32. http://www.etsy.com/blog/news/2011/november-2011-site-performance-report/
  • 33. http://www.etsy.com/blog/news/2011/november-2011-site-performance-report/
  • 34. Performance degradationis a P2 bug
  • 35. WARNING: This part is hardPerformance degradationis a P2 bug
  • 36. 2012 Goal800 ms for all pages
  • 37. EngineeringDenormalize dataOptimize queriesRewrite from scratchCaching (memcache, app code)
  • 38. OperationsNetwork switchesOperating systemsNewer hardwareAdditional hardware capacity
  • 39. ProductIdentify concerns earlyKill lagging features
  • 40. PerformanceCulture
  • 41. Tools
  • 42. How do you makethe graphs?
  • 43. Perf-related data belongs in your server logs
  • 44. apache_note(“custom_field”, $whatever);LogFormat %l %t "%r" %>s %b "%{Referer}i""%{User-Agent}i" %{custom_field}n
  • 45. LogFormat "%{True-Client-IP}i %l %u %t "%r"%>s %b "%{Referer}i" "%{User-Agent}i"%{etsy_user_id}n %{php_bytes}n %{php_microsec}n%D" combined
  • 46. $timer_start = microtime(true); ...$timer_diff = microtime(true) - $timer_start;register_shutdown_function()apache_note(php_microsec, $timer_diff);
  • 47. web0060 66.249.71.110 - - [24/Aug/2011:04:16:52+0000] "GET /listing/12189259/tropical-etched-pair-of-lampwork-glass HTTP/1.1" 200 11034 "-""Mozilla/5.0 (compatible; Googlebot/2.1;+http://www.google.com/bot.html)"- 13399576 505780 554876
  • 48. Using something else?time, http method, request uri, response code, referer,user-agent, response time, response memory,custom segmentation fields...
  • 49. Quick averagesgrep "GET /listing/" access.log | awk {sum=sum+$(NF-1)} END {print sum/NR}
  • 50. Quick graphsgrep "GET /listing/" access.log | perl -pe "s/.*[.*d{4}:(d{2}):(d{2}):d{2}.*]/1:2/" | awk {print $1, $(NF-1)} > /tmp/pagetimes.dat gives you...
  • 51. Quick graphs# /tmp/pagetimes.dat18:37 251.018:38 252.118:39 253.518:40 251.018:45 250.0 and then...
  • 52. Quick graphs# GNUPLOTset terminal pngset output listings.pngset yrange [0:2000]set xdata timeset timefmt "%d/%B/%Y:%H:%M:%S"set format x "%H:%M"plot /tmp/pagetimes.dat using 1:2 with points
  • 53. Quick graphs
  • 54. LogsterRun by cronMaintains a cursor on log filesSimple parsing & aggregationOutput to Ganglia or Graphite github.com/etsy
  • 55. Logster & Graphite Signed-in vs. Signed-out
  • 56. Response timesresponses (pages) by bucket time (ms)
  • 57. “Look at your Data”John Rauser, Velocity 2011 http://www.youtube.com/watch?v=coNDCIMH8bk
  • 58. Idea lifted fromhttp://www.codinghorror.com/blog/2011/06/performance-is-a-feature.html
  • 59. Don’t forget the API
  • 60. Measureperformancedata in Production
  • 61. Profiling
  • 62. “I/O Profiler”Lightweight, inline profilingStart and end times wrapped around service calls databases, memcache, apc, 3rd party services, etc.Enable with query string
  • 63. “I/O Profiler”
  • 64. “I/O Profiler”
  • 65. XHProfgithub.com/facebook/xhprofDeployed on pre-prod machineDirect access to profilingEnable with query string
  • 66. Profile againstProduction
  • 67. Make performance matter
  • 68. Start small
  • 69. Make graphs and reports
  • 70. Create simple tools
  • 71. Celebrate performance wins
  • 72. Thank you codeascraft.etsy.comMike Brittainmike@etsy.com@mikebrittain
  • 73. Jobs at Etsy We are currently hiring for a number of positions in engineering and operations. http://www.etsy.com/careers