Scaling PHP to 40 Million Uniques
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Scaling PHP to 40 Million Uniques

on

  • 5,307 views

This is a talk I gave at Northeast PHP 2013 in Boston, MA.

This is a talk I gave at Northeast PHP 2013 in Boston, MA.

Links: jkle.in/nephp

Statistics

Views

Total Views
5,307
Views on SlideShare
4,582
Embed Views
725

Actions

Likes
14
Downloads
42
Comments
0

4 Embeds 725

http://jkle.in 700
http://librosweb.es 12
https://twitter.com 12
http://news.google.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Scaling PHP to 40 Million Uniques Presentation Transcript

  • 1. Scaling PHP to 40 Million Uniques Northeast PHP 2013 Jonathan Klein @jonathanklein Saturday, August 17, 13
  • 2. Scaling PHP to 40 Million Uniques Northeast PHP 2013 Jonathan Klein @jonathanklein Saturday, August 17, 13
  • 3. Scaling PHP to 60 Million Uniques Northeast PHP 2013 Jonathan Klein @jonathanklein Saturday, August 17, 13
  • 4. Who Am I? Saturday, August 17, 13
  • 5. Who Am I? • Performance Engineer at Etsy Saturday, August 17, 13
  • 6. Who Am I? • Performance Engineer at Etsy • Boston Web Perf Meetup Organizer Saturday, August 17, 13
  • 7. Who Am I? • Performance Engineer at Etsy • Boston Web Perf Meetup Organizer • Previously at Wayfair Saturday, August 17, 13
  • 8. Who Am I? • Performance Engineer at Etsy • Boston Web Perf Meetup Organizer • Previously at Wayfair • Led team that converted to PHP Saturday, August 17, 13
  • 9. Slides, Links: jkle.in/nephp Saturday, August 17, 13
  • 10. codeascraft.com Saturday, August 17, 13
  • 11. Some Etsy Stats Saturday, August 17, 13
  • 12. Some Etsy Stats • 1.5 billion page views/month Saturday, August 17, 13
  • 13. Some Etsy Stats • 1.5 billion page views/month • Almost $1B in sales last year Saturday, August 17, 13
  • 14. Some Etsy Stats • 1.5 billion page views/month • Almost $1B in sales last year • Over 1M lines of PHP Saturday, August 17, 13
  • 15. Some Etsy Stats • 1.5 billion page views/month • Almost $1B in sales last year • Over 1M lines of PHP • 60M+ unique visitors/month Saturday, August 17, 13
  • 16. 2012 Traffic Saturday, August 17, 13
  • 17. August 2012 Traffic Saturday, August 17, 13
  • 18. August 2012 Traffic 50% Higher Saturday, August 17, 13
  • 19. Holidays: ~90M Uniques Saturday, August 17, 13
  • 20. Saturday, August 17, 13
  • 21. Saturday, August 17, 13
  • 22. A few others... Saturday, August 17, 13
  • 23. A few others... • Search (Solr) Saturday, August 17, 13
  • 24. A few others... • Search (Solr) • Gearman Saturday, August 17, 13
  • 25. A few others... • Search (Solr) • Gearman • Redis Saturday, August 17, 13
  • 26. A few others... • Search (Solr) • Gearman • Redis • Postgres (legacy) Saturday, August 17, 13
  • 27. Saturday, August 17, 13
  • 28. Hardware (Supermicro) Saturday, August 17, 13
  • 29. Hardware (Supermicro) 2x 8-core Intel E5-2960 CPUs Saturday, August 17, 13
  • 30. Hardware (Supermicro) 2x 8-core Intel E5-2960 CPUs 24GB of RAM Saturday, August 17, 13
  • 31. Hardware (Supermicro) 2x 8-core Intel E5-2960 CPUs 24GB of RAM 160GB SSD Saturday, August 17, 13
  • 32. Apache Saturday, August 17, 13
  • 33. Apache • Apache 2.2 Saturday, August 17, 13
  • 34. Apache • Apache 2.2 • Prefork MPM Saturday, August 17, 13
  • 35. Apache • Apache 2.2 • Prefork MPM • mod_php5 Saturday, August 17, 13
  • 36. Apache • Apache 2.2 • Prefork MPM • mod_php5 StartServers 30 MinSpareServers 30 MaxSpareServers 60 ServerLimit 60 MaxClients 60 MaxRequestsPerChild 0 Saturday, August 17, 13
  • 37. PHP Saturday, August 17, 13
  • 38. PHP • PHP 5.4 Saturday, August 17, 13
  • 39. PHP • PHP 5.4 • Zend OPCache (3GB Memory Segment) Saturday, August 17, 13
  • 40. PHP • PHP 5.4 • Zend OPCache (3GB Memory Segment) • memory_limit: 128M Saturday, August 17, 13
  • 41. PHP • PHP 5.4 • Zend OPCache (3GB Memory Segment) • memory_limit: 128M • max_execution_time: 30 Saturday, August 17, 13
  • 42. Optimizing PHP http://www.slideshare.net/jnklein/northeast-php-high-performance-php Saturday, August 17, 13
  • 43. Optimizing PHP • See last year’s talk http://www.slideshare.net/jnklein/northeast-php-high-performance-php Saturday, August 17, 13
  • 44. Optimizing PHP • See last year’s talk • Use an Opcode Cache http://www.slideshare.net/jnklein/northeast-php-high-performance-php Saturday, August 17, 13
  • 45. Optimizing PHP • See last year’s talk • Use an Opcode Cache • xhprof http://www.slideshare.net/jnklein/northeast-php-high-performance-php Saturday, August 17, 13
  • 46. Optimizing PHP • See last year’s talk • Use an Opcode Cache • xhprof • StatsD/Graphite http://www.slideshare.net/jnklein/northeast-php-high-performance-php Saturday, August 17, 13
  • 47. Optimizing PHP • See last year’s talk • Use an Opcode Cache • xhprof • StatsD/Graphite • Find hotspots http://www.slideshare.net/jnklein/northeast-php-high-performance-php Saturday, August 17, 13
  • 48. Optimizing PHP • See last year’s talk • Use an Opcode Cache • xhprof • StatsD/Graphite • Find hotspots • Upgrade it http://www.slideshare.net/jnklein/northeast-php-high-performance-php Saturday, August 17, 13
  • 49. Static Arrays Saturday, August 17, 13
  • 50. Translations class language_de { static $translations = array( "<hash>" => array("content" => 'Accessoires', "file" => "TAXONOMY"), "<hash>" => array("content" => 'Keramik', "file" => "CATEGORY"), "<hash>" => array("content" => 'Dekorieren', "file" => "CATEGORY"), "<hash>" => array("content" => 'Getaggt %s', "file" => "Foo.php"), "<hash>" => array("content" => 'Badezusatz', "file" => "CATEGORY"), "<hash>" => array("content" => 'Datum:{% $date %}', "file" => "bar.tpl") ... snip ... ); } Saturday, August 17, 13
  • 51. Saturday, August 17, 13
  • 52. Saturday, August 17, 13
  • 53. Atomic Deploys Saturday, August 17, 13
  • 54. Deploying PHP http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Saturday, August 17, 13
  • 55. Deploying PHP • mod_realdoc - Apache module http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Saturday, August 17, 13
  • 56. Deploying PHP • mod_realdoc - Apache module • incpath - PHP Extension http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Saturday, August 17, 13
  • 57. Deploying PHP • mod_realdoc - Apache module • incpath - PHP Extension • A/B symlink swap http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Saturday, August 17, 13
  • 58. Deploying PHP • mod_realdoc - Apache module • incpath - PHP Extension • A/B symlink swap • Avoid recompilation to opcodes http://codeascraft.com/2013/07/01/atomic-deploys-at-etsy/ Saturday, August 17, 13
  • 59. Upgrading PHP Saturday, August 17, 13
  • 60. PHP 5.4 vs. 5.3 - CPU PHP 5.3 PHP 5.4 Saturday, August 17, 13
  • 61. PHP 5.4 vs. 5.3 - Memory PHP 5.3 PHP 5.4 Saturday, August 17, 13
  • 62. Saturday, August 17, 13
  • 63. Understand Framework Overhead Saturday, August 17, 13
  • 64. http://systemsarchitect.net/performance-benchmark-of-popular-php-frameworks/ Saturday, August 17, 13
  • 65. Saturday, August 17, 13
  • 66. Saturday, August 17, 13
  • 67. Saturday, August 17, 13
  • 68. Frameworks are Harder to Scale Saturday, August 17, 13
  • 69. Saturday, August 17, 13
  • 70. Memcached Saturday, August 17, 13
  • 71. Memcached • Roughly same hardware as webs Saturday, August 17, 13
  • 72. Memcached • Roughly same hardware as webs • More memory (48GB) Saturday, August 17, 13
  • 73. Memcached • Roughly same hardware as webs • More memory (48GB) • Shard keys across servers Saturday, August 17, 13
  • 74. Memcached • Roughly same hardware as webs • More memory (48GB) • Shard keys across servers • mctop - https://github.com/etsy/mctop Saturday, August 17, 13
  • 75. Saturday, August 17, 13
  • 76. MySQL Saturday, August 17, 13
  • 77. MySQL • MySQL 5.5 (http://jkle.in/nephp) Saturday, August 17, 13
  • 78. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs Saturday, August 17, 13
  • 79. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs • innodb Saturday, August 17, 13
  • 80. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs • innodb • thread_cache=800 Saturday, August 17, 13
  • 81. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs • innodb • thread_cache=800 • max_connections=2500 Saturday, August 17, 13
  • 82. MySQL • MySQL 5.5 (http://jkle.in/nephp) • ~20 master/master pairs • innodb • thread_cache=800 • max_connections=2500 • [Almost] no joins Saturday, August 17, 13
  • 83. Theoretically we can serve ~9000 req/sec Saturday, August 17, 13
  • 84. Saturday, August 17, 13
  • 85. Static Content Saturday, August 17, 13
  • 86. Saturday, August 17, 13
  • 87. ~500 Million Images Saturday, August 17, 13
  • 88. Offloading Requests Saturday, August 17, 13
  • 89. Offloading Requests • Multiple CDNs Saturday, August 17, 13
  • 90. Offloading Requests • Multiple CDNs • Img hit rate > 90% Saturday, August 17, 13
  • 91. Offloading Requests • Multiple CDNs • Img hit rate > 90% • CSS/JS hit rate ~100% Saturday, August 17, 13
  • 92. Offloading Requests • Multiple CDNs • Img hit rate > 90% • CSS/JS hit rate ~100% • Edge serves ~7000 reqs/sec Saturday, August 17, 13
  • 93. What about misses? Saturday, August 17, 13
  • 94. Saturday, August 17, 13
  • 95. Change Management Saturday, August 17, 13
  • 96. Feature Flags Saturday, August 17, 13
  • 97. Saturday, August 17, 13
  • 98. // Owner: Username $server_config['foo']['bar'] = array( 'users' => array('jsmith', 'jdoe'), ); // Owner: Username $server_config['baz'] = [ 'enabled' => [ 'variant1' => 0, 'variant2' => 0, 'variant3' => 50 ], ]; // Owner: Username $server_config['qux'] = array( 'enabled' => 100, 'data' => array( 'somedata' ) ); Saturday, August 17, 13
  • 99. if (Feature::isEnabled('foo.bar')) { // Code } Saturday, August 17, 13
  • 100. Ramp-up: 1%, 5%, 25%, 100% Saturday, August 17, 13
  • 101. Deployinator Saturday, August 17, 13
  • 102. Saturday, August 17, 13
  • 103. Schemanator Saturday, August 17, 13
  • 104. Saturday, August 17, 13
  • 105. Recent Improvements Saturday, August 17, 13
  • 106. Saturday, August 17, 13
  • 107. Sandy Bridge Saturday, August 17, 13
  • 108. Baseline Performance (median) Saturday, August 17, 13
  • 109. Small code change Baseline Performance (median) Saturday, August 17, 13
  • 110. Small code change PHP 5.4 Baseline Performance (median) Saturday, August 17, 13
  • 111. Small code change PHP 5.4 Disabled Hyperthreading Baseline Performance (median) Saturday, August 17, 13
  • 112. Challenges Saturday, August 17, 13
  • 113. Saturday, August 17, 13
  • 114. Options Saturday, August 17, 13
  • 115. Options • Scale Horizontally (more shards) Saturday, August 17, 13
  • 116. Options • Scale Horizontally (more shards) • Scale Vertically (SSD, bigger drives) Saturday, August 17, 13
  • 117. Options • Scale Horizontally (more shards) • Scale Vertically (SSD, bigger drives) • Change architecture Saturday, August 17, 13
  • 118. Saturday, August 17, 13
  • 119. Architecture Changes Saturday, August 17, 13
  • 120. Architecture Changes • Ongoing conversation Saturday, August 17, 13
  • 121. Architecture Changes • Ongoing conversation • Separate logical/physical shards Saturday, August 17, 13
  • 122. Architecture Changes • Ongoing conversation • Separate logical/physical shards • Complex Saturday, August 17, 13
  • 123. Architecture Changes • Ongoing conversation • Separate logical/physical shards • Complex • Server failure Saturday, August 17, 13
  • 124. Architecture Changes • Ongoing conversation • Separate logical/physical shards • Complex • Server failure • Migrating data Saturday, August 17, 13
  • 125. Architecture Changes • Ongoing conversation • Separate logical/physical shards • Complex • Server failure • Migrating data • Consistency Saturday, August 17, 13
  • 126. Monitoring Saturday, August 17, 13
  • 127. Graphite Saturday, August 17, 13
  • 128. Stacked Search Timers Saturday, August 17, 13
  • 129. Ganglia Saturday, August 17, 13
  • 130. Saturday, August 17, 13
  • 131. Nagios Saturday, August 17, 13
  • 132. Saturday, August 17, 13
  • 133. Saturday, August 17, 13
  • 134. Takeaways Saturday, August 17, 13
  • 135. K.I.S.S. Saturday, August 17, 13
  • 136. Use Proven Technologies Saturday, August 17, 13
  • 137. Saturday, August 17, 13
  • 138. Understand Your Stack Saturday, August 17, 13
  • 139. Measure Everything Saturday, August 17, 13
  • 140. Work at Etsy ;-) Saturday, August 17, 13
  • 141. Connect http://www.meetup.com/Web-Performance-Boston/ www.etsy.com/careers jonathan@etsy.com @jonathanklein Saturday, August 17, 13