Web Performance Culture and Tools at Etsy
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Web Performance Culture and Tools at Etsy

on

  • 9,234 views

This talk covers some of the tools that Etsy uses for measuring performance, how to instill a culture of performance, how Etsy tracks performance wins and regressions, and where to get started if you ...

This talk covers some of the tools that Etsy uses for measuring performance, how to instill a culture of performance, how Etsy tracks performance wins and regressions, and where to get started if you don't have a formalized performance team in your company.

Originally presented at the Boston Web Performance Meetup on Aug 24, 2011.

Statistics

Views

Total Views
9,234
Views on SlideShare
7,655
Embed Views
1,579

Actions

Likes
20
Downloads
113
Comments
0

12 Embeds 1,579

http://www.mikebrittain.com 1517
http://www.linkedin.com 29
http://socialwebkurs.wikispaces.com 12
http://twitter.com 7
http://us-w1.rockmelt.com 5
http://coderwall.com 3
http://translate.googleusercontent.com 1
http://webcache.googleusercontent.com 1
http://www.mikebrittain.com. 1
http://www.slideshare.net 1
http://www.onlydoo.com 1
https://twitter.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Web Performance Culture and Tools at Etsy Presentation Transcript

  • 1. Web PerformanceCulture and Toolsat EtsyMike BrittainDir. of Engineering, InfrastructureEtsyBoston Web Performance MeetupAug 24, 2011
  • 2. OverviewEtsy and EngineeringMake PerformanceMatterTools and Process
  • 3. http://qacreate.etsy.com
  • 4. MonthlyWeather Report$38 MM in sales1.9 MM items soldhttp://etsy.me/weather-report-june-2011
  • 5. MonthlyWeather Report990 MM page viewshttp://etsy.me/weather-report-june-2011
  • 6. EngineeringTechnologiesLinux, Apache, MySQL,PHP, MemcacheSolr, Squid, Hadoop, Amazon S3, EC2, EMRGanglia, Cacti, Nagios, Graphite, Splunk, andsome of our own...
  • 7. EngineeringTeams90 Engineers3-6 engineers+ product developer and/or designer
  • 8. EngineeringContinuousDeployment~40 releases per day inc. app code and config changes
  • 9. 1. Write code2. Code review3. Automated tests
  • 10. 1. Write code2. Code review3. Automated tests4. Dev ⇾ QA ⇾ Pre-Prod ⇾ Prod
  • 11. 1. Write code2. Code review3. Automated tests4. Dev ⇾ QA ⇾ Pre-Prod ⇾ Prod5. Monitor!6. Monitor!7. Monitor!
  • 12. EngineeringData-DrivenDevelopment45,000+ metrics50+ dashboards
  • 13. MakePerformanceMatterHave a story
  • 14. Business ImpactMeasure“Our own years of testing have conclusively shown that when speed of a feature or product improves, usage, quite simply, goes up.” ~GoogleAbundance of research from Google, Bing,AOL, Amazon, Shopzilla, etc.http://googleblog.blogspot.com/2009/12/this-week-in-search-121809.html
  • 15. Business ImpactMeasureBounce rateSearch conversion
  • 16. Business ImpactMeasurePurchase funnelAd impressions, page views andtracking discrepanciesSocial, engagement
  • 17. OperationsSite StabilityMeasure perf for discrete piecesof your infrastructureContention for shared resources likedatabase, memcache, solr, or even webserver processes
  • 18. MoraleHappy Engineers
  • 19. Getting to workWhere toStart?Focus your efforts where it makessense for your business
  • 20. Getting to workWhere toOptimize?Tiers of serviceSLAs
  • 21. Getting to workFocusBeware of CTS:Constant Tweaking Syndrome
  • 22. Getting to workFocusFriends don’t let friendstweak without graphing
  • 23. MakePerformanceMatterHave your story
  • 24. Tools andProcess atEtsy
  • 25. ProcessEtsy’sPerf TeamStandardize patternsCreate tools and reportsCoordinate efforts
  • 26. ProcessServer-sidePerformanceThere is nothing worsethan a blank page or a spinningthing-a-ma-jigger
  • 27. ProcessServer-sidePerformance95th Percentile> 800 ms is a P2 bug
  • 28. ProcessServer-sidePerformance
  • 29. ToolsLoggingPerf-related data belongs inyour server logsapache_note()
  • 30. ToolsLogging$timer_start = microtime(true); ...$timer_diff = microtime(true) - $timer_start;register_shutdown_function()apache_note(php_microsec, $timer_diff);
  • 31. ToolsLoggingLogFormat "%{True-Client-IP}i %l %u %t "%r"%>s %b "%{Referer}i" "%{User-Agent}i"%{etsy_user_id}n %{php_bytes}n %{php_microsec}n%D" combined
  • 32. ToolsLoggingLogFormat "%{True-Client-IP}i %l %u %t "%r"%>s %b "%{Referer}i" "%{User-Agent}i"%{etsy_user_id}n %{php_bytes}n %{php_microsec}n%D" combined
  • 33. ToolsLoggingLogFormat "%{True-Client-IP}i %l %u %t "%r"%>s %b "%{Referer}i" "%{User-Agent}i"%{etsy_user_id}n %{php_bytes}n %{php_microsec}n%D" combined
  • 34. ToolsLoggingweb0060 66.249.71.110 - - [24/Aug/2011:04:16:52+0000] "GET /listing/12189259/tropical-etched-pair-of-lampwork-glass HTTP/1.1" 200 11034 "-""Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" - 13399576 505780554876
  • 35. ToolsAnalyzegrep ... access.log | awk ...grep "/listing/" access.log | awk {sum=sum+$(NF-1)} END {print sum/NR}grep "/listing/" access.log | awk BEGIN {max=0} {if ($(NF-1)>max) max=$(NF-1)}END {print max}
  • 36. ToolsAnalyze
  • 37. ToolsAnalyze1. Capture perf data in logs page gen., Boomerang, Gomez2. Aggregate Splunk, Logster, StatsD3. Record in Graphite http://graphite.wikidot.com
  • 38. ToolsAnalyze
  • 39. ToolsLogster1. Read new log entries every minute2. Parse and aggregate useful numbers3. Send to Graphite http://github.com/etsy
  • 40. web0001 [04:28:54 2011] [error] [client 10.101.x.x] Oh noooooo!web0001 [04:28:54 2011] [warning] [client 10.101.x.x] Gaaaaahhh!web0001 [04:28:54 2011] [error] [client 10.101.x.x] Help me, Rhonda.web0001 [04:28:54 2011] [error] [client 10.101.x.x] Oh noooooo!web0001 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!web0001 [04:28:54 2011] [error] [client 10.101.x.x] Heeeeeeellllllllllllllppweb0001 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!!web1101 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!!web0201 [04:28:54 2011] [error] [client 10.101.x.x] Youve been eaten by a gweb0001 [04:28:54 2011] [error] [client 10.101.x.x] Oh noooooo!web0001 [04:28:54 2011] [fatal] [client 10.101.x.x] Gaaaaahhh!web0201 [04:28:54 2011] [warning] [client 10.101.x.x] Gaaaaahhh!web0034 [04:28:54 2011] [warning] [client 10.101.x.x] Oh noooooooooooweb0001 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!!web1101 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!!web0201 [04:28:54 2011] [error] [client 10.101.x.x] Youve been eaten by a gweb0055 [04:28:54 2011] [fatal] [client 10.101.x.x] Gaaaaahhh!!!web0002 [04:28:54 2011] [warning] [client 10.101.x.x] Sky is falling.web0089 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!!web0020 [04:28:54 2011] [error] [client 10.101.x.x] Sky is falling.web1101 [04:28:54 2011] [fatal] [client 10.101.x.x] Gaaaaahhh!web0055 [04:28:54 2011] [warning] [client 10.101.x.x] Gaaaaahhh!web0001 [04:28:54 2011] [warning] [client 10.101.x.x] Oh noooooooooooweb0001 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!!web0034 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!!web0087 [04:28:54 2011] [fatal] [client 10.101.x.x] Sky is falling.web0002 [04:28:54 2011] [error] [client 10.101.x.x] Oh noooooo!web0201 [04:28:54 2011] [fatal] [client 10.101.x.x] Gaaaaahhh!web0077 [04:28:54 2011] [warning] [client 10.101.x.x] Gaaaaahhh!web0355 [04:28:54 2011] [warning] [client 10.101.x.x] Oh noooooooooooweb0052 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!!web0001 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!!web0089 [04:28:54 2011] [error] [client 10.101.x.x] Gaaaaahhh!!!web0020 [04:28:54 2011] [error] [client 10.101.x.x] Sky is falling.web1101 [04:28:54 2011] [fatal] [client 10.101.x.x] Gaaaaahhh!web0055 [04:28:54 2011] [warning] [client 10.101.x.x] Gaaaaahhh!web0001 [04:28:54 2011] [warning] [client 10.101.x.x] Oh nooooooooooo
  • 41. Fatals Errors Warnings
  • 42. :)
  • 43. ToolsStatsDCollects metrics from your appcode and sticks them in GraphiteStatsD::increment("logins.success");StatsD::timing("gearman.time", $msec);
  • 44. ToolsStatsD 90th pct average lower
  • 45. :) :)
  • 46. ToolsPerf Dashboard
  • 47. ToolsI/O ProfilerLightweight, inline profilingStart and end times wrappedaround service calls databases, memcache, apc, etc.
  • 48. ToolsI/O Profiler
  • 49. ToolsI/O Profiler
  • 50. ToolsClient-sideTestingGomez (API), Boomerang,WebPagetest
  • 51. ToolsShowSlowTracks front-end best practices
  • 52. ToolsShowSlowInternal instanceAutomated testingEnd-point for I/O ProfilerTrending on individual rules http://showslow.com
  • 53. Other thoughtsYSlow &Page SpeedMind your development process3rd-party content and ga.js“Use a CDN”
  • 54. Other thoughtsDevice-SpecificDesignThe mobile web is very muchabout designing for performanceScreen size, pixel density, connection speed,wi-fi vs. cellular, browser cache size, localstorage, connections per host, meteredpricing, etc.
  • 55. MakePerformanceMatterHave your storyUse tools and processto focus your efforts
  • 56. We areHiringhttp://etsy.com/careersSoftware Engineering positions available ina number of teams, including Analytics,Operations, Web Performance, Payments,Core Platform, Front-End, Internal Apps,Search, Security, and more...
  • 57. ThankYouMike Brittainmike@etsy.com@mikebrittainCodeAsCraft.etsy.comgithub.com/etsygraphite.wikidot.comshowslow.com