0
Web PerformanceCulture and Toolsat Etsy         Mike Brittain         @mikebrittain         Director of Eng., Infrastructu...
All of this started in the last year.
Make PerformanceMatter
$0.20 per item3.5% transaction fee
Focus on the marketplace
ecommerce
ecommercead supported
ecommercead supportedsubscription
ecommercead supportedsubscription“free”
$$$
“Prove it.”
“Our own years of testinghave conclusively shown that when speed of a feature or  product improves, usage,   quite simply,...
100+ web, api, jobs40+ databasesMany more doing lots of other stuff...
$$$
Performance is fun
I want you to tell me   how you feel...
95th percentilemilliseconds                 Average
e                                            w r!                                          en ste                         ...
I am better than you.
Make PerformanceMatter
PerformanceCulture
Our perf team is two people...
Measure
MeasureBuild tools
MeasureBuild toolsTrain & Coordinate
MeasureBuild toolsTrain & CoordinateMotivate
MeasureBuild toolsTrain & CoordinateMotivateDedicated
Our perf team is two people...  But every engineer thinks     about performance
Everything is slow.Please fix.
page generationtime
2011 Goal800 ms on five pages
http://www.etsy.com/blog/news/2011/november-2011-site-performance-report/
http://www.etsy.com/blog/news/2011/november-2011-site-performance-report/
Performance degradationis a P2 bug
WARNING: This part is hardPerformance degradationis a P2 bug
2012 Goal800 ms for all pages
EngineeringDenormalize dataOptimize queriesRewrite from scratchCaching (memcache, app code)
OperationsNetwork switchesOperating systemsNewer hardwareAdditional hardware capacity
ProductIdentify concerns earlyKill lagging features
PerformanceCulture
Tools
How do you makethe graphs?
Perf-related data belongs in      your server logs
apache_note(“custom_field”, $whatever);LogFormat %l %t "%r" %>s %b "%{Referer}i""%{User-Agent}i" %{custom_field}n
LogFormat "%{True-Client-IP}i %l %u %t "%r"%>s %b "%{Referer}i" "%{User-Agent}i"%{etsy_user_id}n %{php_bytes}n %{php_micro...
$timer_start = microtime(true); ...$timer_diff = microtime(true) - $timer_start;register_shutdown_function()apache_note(ph...
web0060 66.249.71.110 - - [24/Aug/2011:04:16:52+0000] "GET /listing/12189259/tropical-etched-pair-of-lampwork-glass HTTP/1...
Using something else?time, http method, request uri, response code, referer,user-agent, response time, response memory,cus...
Quick averagesgrep "GET /listing/" access.log | awk {sum=sum+$(NF-1)} END {print sum/NR}
Quick graphsgrep "GET /listing/" access.log | perl -pe "s/.*[.*d{4}:(d{2}):(d{2}):d{2}.*]/1:2/" | awk {print $1, $(NF-1)} ...
Quick graphs# /tmp/pagetimes.dat18:37   251.018:38   252.118:39   253.518:40   251.018:45   250.0          and then...
Quick graphs# GNUPLOTset terminal pngset output listings.pngset yrange [0:2000]set xdata timeset timefmt "%d/%B/%Y:%H:%M:%...
Quick graphs
LogsterRun by cronMaintains a cursor on log filesSimple parsing & aggregationOutput to Ganglia or Graphite                 ...
Logster & Graphite         Signed-in vs. Signed-out
Response timesresponses (pages)                         by bucket                    time (ms)
“Look at your Data”John Rauser, Velocity 2011   http://www.youtube.com/watch?v=coNDCIMH8bk
Idea lifted fromhttp://www.codinghorror.com/blog/2011/06/performance-is-a-feature.html
Don’t forget the API
Measureperformancedata in Production
Profiling
“I/O Profiler”Lightweight, inline profilingStart and end times wrapped around service calls   databases, memcache, apc, 3rd ...
“I/O Profiler”
“I/O Profiler”
XHProfgithub.com/facebook/xhprofDeployed on pre-prod machineDirect access to profilingEnable with query string
Profile againstProduction
Make performance matter
Start small
Make graphs and reports
Create simple tools
Celebrate performance wins
Thank you           codeascraft.etsy.comMike Brittainmike@etsy.com@mikebrittain
Jobs at Etsy We are currently hiring for a number of positions in engineering and operations. http://www.etsy.com/careers
Web Performance Culture and Tools at Etsy
Web Performance Culture and Tools at Etsy
Web Performance Culture and Tools at Etsy
Web Performance Culture and Tools at Etsy
Web Performance Culture and Tools at Etsy
Web Performance Culture and Tools at Etsy
Web Performance Culture and Tools at Etsy
Web Performance Culture and Tools at Etsy
Web Performance Culture and Tools at Etsy
Web Performance Culture and Tools at Etsy
Upcoming SlideShare
Loading in...5
×

Web Performance Culture and Tools at Etsy

49,350

Published on

Presented at the NY Web Performance Meetup on Jan 18, 2012.

Published in: Technology, Design
6 Comments
63 Likes
Statistics
Notes
No Downloads
Views
Total Views
49,350
On Slideshare
0
From Embeds
0
Number of Embeds
18
Actions
Shares
0
Downloads
278
Comments
6
Likes
63
Embeds 0
No embeds

No notes for slide

Transcript of "Web Performance Culture and Tools at Etsy"

  1. 1. Web PerformanceCulture and Toolsat Etsy Mike Brittain @mikebrittain Director of Eng., Infrastructure Etsy
  2. 2. All of this started in the last year.
  3. 3. Make PerformanceMatter
  4. 4. $0.20 per item3.5% transaction fee
  5. 5. Focus on the marketplace
  6. 6. ecommerce
  7. 7. ecommercead supported
  8. 8. ecommercead supportedsubscription
  9. 9. ecommercead supportedsubscription“free”
  10. 10. $$$
  11. 11. “Prove it.”
  12. 12. “Our own years of testinghave conclusively shown that when speed of a feature or product improves, usage, quite simply, goes up.” http://googleblog.blogspot.com/2009/12/this-week-in-search-121809.html
  13. 13. 100+ web, api, jobs40+ databasesMany more doing lots of other stuff...
  14. 14. $$$
  15. 15. Performance is fun
  16. 16. I want you to tell me how you feel...
  17. 17. 95th percentilemilliseconds Average
  18. 18. e w r! en ste h a 95th percentile w f is go is it h e T dmilliseconds m a Average
  19. 19. I am better than you.
  20. 20. Make PerformanceMatter
  21. 21. PerformanceCulture
  22. 22. Our perf team is two people...
  23. 23. Measure
  24. 24. MeasureBuild tools
  25. 25. MeasureBuild toolsTrain & Coordinate
  26. 26. MeasureBuild toolsTrain & CoordinateMotivate
  27. 27. MeasureBuild toolsTrain & CoordinateMotivateDedicated
  28. 28. Our perf team is two people... But every engineer thinks about performance
  29. 29. Everything is slow.Please fix.
  30. 30. page generationtime
  31. 31. 2011 Goal800 ms on five pages
  32. 32. http://www.etsy.com/blog/news/2011/november-2011-site-performance-report/
  33. 33. http://www.etsy.com/blog/news/2011/november-2011-site-performance-report/
  34. 34. Performance degradationis a P2 bug
  35. 35. WARNING: This part is hardPerformance degradationis a P2 bug
  36. 36. 2012 Goal800 ms for all pages
  37. 37. EngineeringDenormalize dataOptimize queriesRewrite from scratchCaching (memcache, app code)
  38. 38. OperationsNetwork switchesOperating systemsNewer hardwareAdditional hardware capacity
  39. 39. ProductIdentify concerns earlyKill lagging features
  40. 40. PerformanceCulture
  41. 41. Tools
  42. 42. How do you makethe graphs?
  43. 43. Perf-related data belongs in your server logs
  44. 44. apache_note(“custom_field”, $whatever);LogFormat %l %t "%r" %>s %b "%{Referer}i""%{User-Agent}i" %{custom_field}n
  45. 45. LogFormat "%{True-Client-IP}i %l %u %t "%r"%>s %b "%{Referer}i" "%{User-Agent}i"%{etsy_user_id}n %{php_bytes}n %{php_microsec}n%D" combined
  46. 46. $timer_start = microtime(true); ...$timer_diff = microtime(true) - $timer_start;register_shutdown_function()apache_note(php_microsec, $timer_diff);
  47. 47. web0060 66.249.71.110 - - [24/Aug/2011:04:16:52+0000] "GET /listing/12189259/tropical-etched-pair-of-lampwork-glass HTTP/1.1" 200 11034 "-""Mozilla/5.0 (compatible; Googlebot/2.1;+http://www.google.com/bot.html)"- 13399576 505780 554876
  48. 48. Using something else?time, http method, request uri, response code, referer,user-agent, response time, response memory,custom segmentation fields...
  49. 49. Quick averagesgrep "GET /listing/" access.log | awk {sum=sum+$(NF-1)} END {print sum/NR}
  50. 50. Quick graphsgrep "GET /listing/" access.log | perl -pe "s/.*[.*d{4}:(d{2}):(d{2}):d{2}.*]/1:2/" | awk {print $1, $(NF-1)} > /tmp/pagetimes.dat gives you...
  51. 51. Quick graphs# /tmp/pagetimes.dat18:37 251.018:38 252.118:39 253.518:40 251.018:45 250.0 and then...
  52. 52. Quick graphs# GNUPLOTset terminal pngset output listings.pngset yrange [0:2000]set xdata timeset timefmt "%d/%B/%Y:%H:%M:%S"set format x "%H:%M"plot /tmp/pagetimes.dat using 1:2 with points
  53. 53. Quick graphs
  54. 54. LogsterRun by cronMaintains a cursor on log filesSimple parsing & aggregationOutput to Ganglia or Graphite github.com/etsy
  55. 55. Logster & Graphite Signed-in vs. Signed-out
  56. 56. Response timesresponses (pages) by bucket time (ms)
  57. 57. “Look at your Data”John Rauser, Velocity 2011 http://www.youtube.com/watch?v=coNDCIMH8bk
  58. 58. Idea lifted fromhttp://www.codinghorror.com/blog/2011/06/performance-is-a-feature.html
  59. 59. Don’t forget the API
  60. 60. Measureperformancedata in Production
  61. 61. Profiling
  62. 62. “I/O Profiler”Lightweight, inline profilingStart and end times wrapped around service calls databases, memcache, apc, 3rd party services, etc.Enable with query string
  63. 63. “I/O Profiler”
  64. 64. “I/O Profiler”
  65. 65. XHProfgithub.com/facebook/xhprofDeployed on pre-prod machineDirect access to profilingEnable with query string
  66. 66. Profile againstProduction
  67. 67. Make performance matter
  68. 68. Start small
  69. 69. Make graphs and reports
  70. 70. Create simple tools
  71. 71. Celebrate performance wins
  72. 72. Thank you codeascraft.etsy.comMike Brittainmike@etsy.com@mikebrittain
  73. 73. Jobs at Etsy We are currently hiring for a number of positions in engineering and operations. http://www.etsy.com/careers
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×