Successfully reported this slideshow.

Web Performance Culture and Tools at Etsy

25

Share

Upcoming SlideShare
Take My Logs. Please!
Take My Logs. Please!
Loading in …3
×
1 of 58
1 of 58

Web Performance Culture and Tools at Etsy

25

Share

Download to read offline

This talk covers some of the tools that Etsy uses for measuring performance, how to instill a culture of performance, how Etsy tracks performance wins and regressions, and where to get started if you don't have a formalized performance team in your company.

Originally presented at the Boston Web Performance Meetup on Aug 24, 2011.

This talk covers some of the tools that Etsy uses for measuring performance, how to instill a culture of performance, how Etsy tracks performance wins and regressions, and where to get started if you don't have a formalized performance team in your company.

Originally presented at the Boston Web Performance Meetup on Aug 24, 2011.

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Web Performance Culture and Tools at Etsy

  1. Web Performance Culture and Tools at Etsy Mike Brittain Dir. of Engineering, Infrastructure Etsy Boston Web Performance Meetup Aug 24, 2011
  2. Overview Etsy and Engineering Make Performance Matter Tools and Process
  3. http://qacreate.etsy.com
  4. Monthly Weather Report $38 MM in sales 1.9 MM items sold http://etsy.me/weather-report-june-2011
  5. Monthly Weather Report 990 MM page views http://etsy.me/weather-report-june-2011
  6. Engineering Technologies Linux, Apache, MySQL, PHP, Memcache Solr, Squid, Hadoop, Amazon S3, EC2, EMR Ganglia, Cacti, Nagios, Graphite, Splunk, and some of our own...
  7. Engineering Teams 90 Engineers 3-6 engineers + product developer and/or designer
  8. Engineering Continuous Deployment ~40 releases per day inc. app code and config changes
  9. 1. Write code 2. Code review 3. Automated tests
  10. 1. Write code 2. Code review 3. Automated tests 4. Dev ⇾ QA ⇾ Pre-Prod ⇾ Prod
  11. 1. Write code 2. Code review 3. Automated tests 4. Dev ⇾ QA ⇾ Pre-Prod ⇾ Prod 5. Monitor! 6. Monitor! 7. Monitor!
  12. Engineering Data-Driven Development 45,000+ metrics 50+ dashboards
  13. Make Performance Matter Have a story
  14. Business Impact Measure “Our own years of testing have conclusively shown that when speed of a feature or product improves, usage, quite simply, goes up.” ~Google Abundance of research from Google, Bing, AOL, Amazon, Shopzilla, etc. http://googleblog.blogspot.com/2009/12/this-week-in-search-121809.html
  15. Business Impact Measure Bounce rate Search conversion
  16. Business Impact Measure Purchase funnel Ad impressions, page views and tracking discrepancies Social, engagement
  17. Operations Site Stability Measure perf for discrete pieces of your infrastructure Contention for shared resources like database, memcache, solr, or even web server processes
  18. Morale Happy Engineers
  19. Getting to work Where to Start? Focus your efforts where it makes sense for your business
  20. Getting to work Where to Optimize? Tiers of service SLAs
  21. Getting to work Focus Beware of CTS: Constant Tweaking Syndrome
  22. Getting to work Focus Friends don’t let friends tweak without graphing
  23. Make Performance Matter Have your story
  24. Tools and Process at Etsy
  25. Process Etsy’s Perf Team Standardize patterns Create tools and reports Coordinate efforts
  26. Process Server-side Performance There is nothing worse than a blank page or a spinning thing-a-ma-jigger
  27. Process Server-side Performance 95th Percentile > 800 ms is a P2 bug
  28. Process Server-side Performance
  29. Tools Logging Perf-related data belongs in your server logs apache_note()
  30. Tools Logging $timer_start = microtime(true); ... $timer_diff = microtime(true) - $timer_start; register_shutdown_function() apache_note('php_microsec', $timer_diff);
  31. Tools Logging LogFormat "%{True-Client-IP}i %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" %{etsy_user_id}n %{php_bytes}n %{php_microsec}n %D" combined
  32. Tools Logging LogFormat "%{True-Client-IP}i %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" %{etsy_user_id}n %{php_bytes}n %{php_microsec}n %D" combined
  33. Tools Logging LogFormat "%{True-Client-IP}i %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" %{etsy_user_id}n %{php_bytes}n %{php_microsec}n %D" combined
  34. Tools Logging web0060 66.249.71.110 - - [24/Aug/2011:04:16:52 +0000] "GET /listing/12189259/tropical-etched- pair-of-lampwork-glass HTTP/1.1" 200 11034 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http:// www.google.com/bot.html)" - 13399576 505780 554876
  35. Tools Analyze grep ... access.log | awk ... grep "/listing/" access.log | awk '{sum=sum+$(NF-1)} END {print sum/NR}' grep "/listing/" access.log | awk 'BEGIN {max=0} {if ($(NF-1)>max) max=$(NF-1)} END {print max}'
  36. Tools Analyze
  37. Tools Analyze 1. Capture perf data in logs page gen., Boomerang, Gomez 2. Aggregate Splunk, Logster, StatsD 3. Record in Graphite http://graphite.wikidot.com
  38. Tools Analyze
  39. Tools Logster 1. Read new log entries every minute 2. Parse and aggregate useful numbers 3. Send to Graphite http://github.com/etsy
  40. web0001 [04:28:54 2011] [error] [client 10.101.x.x] Oh noooooo! web0001 [04:28:54 2011] [warning] [client 10.101.x.x] Gaaaaahhh! web0001 [04:28:54 2011] [error] [client 10.101.x.x] Help me, Rhonda. web0001 [04:28:54 2011] [error] [client 10.101.x.x] Oh noooooo! web0001 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh! web0001 [04:28:54 2011] [error] [client 10.101.x.x] Heeeeeeellllllllllllllpp web0001 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!! web1101 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!! web0201 [04:28:54 2011] [error] [client 10.101.x.x] You've been eaten by a g web0001 [04:28:54 2011] [error] [client 10.101.x.x] Oh noooooo! web0001 [04:28:54 2011] [fatal] [client 10.101.x.x] Gaaaaahhh! web0201 [04:28:54 2011] [warning] [client 10.101.x.x] Gaaaaahhh! web0034 [04:28:54 2011] [warning] [client 10.101.x.x] Oh nooooooooooo web0001 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!! web1101 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!! web0201 [04:28:54 2011] [error] [client 10.101.x.x] You've been eaten by a g web0055 [04:28:54 2011] [fatal] [client 10.101.x.x] Gaaaaahhh!!! web0002 [04:28:54 2011] [warning] [client 10.101.x.x] Sky is falling. web0089 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!! web0020 [04:28:54 2011] [error] [client 10.101.x.x] Sky is falling. web1101 [04:28:54 2011] [fatal] [client 10.101.x.x] Gaaaaahhh! web0055 [04:28:54 2011] [warning] [client 10.101.x.x] Gaaaaahhh! web0001 [04:28:54 2011] [warning] [client 10.101.x.x] Oh nooooooooooo web0001 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!! web0034 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!! web0087 [04:28:54 2011] [fatal] [client 10.101.x.x] Sky is falling. web0002 [04:28:54 2011] [error] [client 10.101.x.x] Oh noooooo! web0201 [04:28:54 2011] [fatal] [client 10.101.x.x] Gaaaaahhh! web0077 [04:28:54 2011] [warning] [client 10.101.x.x] Gaaaaahhh! web0355 [04:28:54 2011] [warning] [client 10.101.x.x] Oh nooooooooooo web0052 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!! web0001 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!! web0089 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!! web0020 [04:28:54 2011] [error] [client 10.101.x.x] Sky is falling. web1101 [04:28:54 2011] [fatal] [client 10.101.x.x] Gaaaaahhh! web0055 [04:28:54 2011] [warning] [client 10.101.x.x] Gaaaaahhh! web0001 [04:28:54 2011] [warning] [client 10.101.x.x] Oh nooooooooooo
  41. Fatals Errors Warnings
  42. :)
  43. Tools StatsD Collects metrics from your app code and sticks them in Graphite StatsD::increment("logins.success"); StatsD::timing("gearman.time", $msec);
  44. Tools StatsD 90th pct average lower
  45. :) :)
  46. Tools Perf Dashboard
  47. Tools I/O Profiler Lightweight, inline profiling Start and end times wrapped around service calls databases, memcache, apc, etc.
  48. Tools I/O Profiler
  49. Tools I/O Profiler
  50. Tools Client-side Testing Gomez (API), Boomerang, WebPagetest
  51. Tools ShowSlow Tracks front-end best practices
  52. Tools ShowSlow Internal instance Automated testing End-point for I/O Profiler Trending on individual rules http://showslow.com
  53. Other thoughts YSlow & Page Speed Mind your development process 3rd-party content and ga.js “Use a CDN”
  54. Other thoughts Device-Specific Design The mobile web is very much about designing for performance Screen size, pixel density, connection speed, wi-fi vs. cellular, browser cache size, local storage, connections per host, metered pricing, etc.
  55. Make Performance Matter Have your story Use tools and process to focus your efforts
  56. We are Hiring http://etsy.com/careers Software Engineering positions available in a number of teams, including Analytics, Operations, Web Performance, Payments, Core Platform, Front-End, Internal Apps, Search, Security, and more...
  57. Thank You Mike Brittain mike@etsy.com @mikebrittain CodeAsCraft.etsy.com github.com/etsy graphite.wikidot.com showslow.com

×