Escalando una PHP App con DB sharding - PHP Conference

672 views
569 views

Published on

Presentación del PHP Conference Argentina 2013 sobre escalabilidad horizontal a nivel web y bases de datos para aplicaciones escritas en PHP, trabajando sobre sharding en mysql, amazon web services, y modelo de datos no relacional.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
672
On SlideShare
0
From Embeds
0
Number of Embeds
27
Actions
Shares
0
Downloads
13
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Escalando una PHP App con DB sharding - PHP Conference

  1. 1. Escalando una PHP App con DB Sharding & Amazon Web Services Matias Paterlini Tuesday, November 5, 13
  2. 2. Objetivo de la charla • Contar quien soy y qué hago • Mostrar orígenes de la arquitectura, infraestructura y escalabilidad de TFM • Hablar de DB Sharding • Hablar de Amazon • Preguntas • Cof Cof reclutar Ejem! Tuesday, November 5, 13
  3. 3. • +11 años de experiencia en PHP. • 6 años desarrollando aplicaciones sobre Redes Sociales • 25 empleados, + de 250 aplicaciones desarrolladas • + 10K clientes, + 10M Usuarios • 2 compañías construidas sobre plataformas sociales Tuesday, November 5, 13
  4. 4. Tuesday, November 5, 13
  5. 5. Tuesday, November 5, 13
  6. 6. Tuesday, November 5, 13
  7. 7. Tuesday, November 5, 13
  8. 8. Tuesday, November 5, 13
  9. 9. Tuesday, November 5, 13
  10. 10. Tuesday, November 5, 13
  11. 11. Tuesday, November 5, 13
  12. 12. Tuesday, November 5, 13
  13. 13. Tuesday, November 5, 13
  14. 14. Tuesday, November 5, 13
  15. 15. Tuesday, November 5, 13
  16. 16. Tuesday, November 5, 13
  17. 17. Tuesday, November 5, 13
  18. 18. Tuesday, November 5, 13
  19. 19. Tuesday, November 5, 13
  20. 20. Tuesday, November 5, 13
  21. 21. Tuesday, November 5, 13
  22. 22. Tuesday, November 5, 13
  23. 23. Tuesday, November 5, 13
  24. 24. Tuesday, November 5, 13
  25. 25. Tuesday, November 5, 13
  26. 26. Tuesday, November 5, 13
  27. 27. Tuesday, November 5, 13
  28. 28. Tuesday, November 5, 13
  29. 29. Tuesday, November 5, 13
  30. 30. Tuesday, November 5, 13
  31. 31. Tuesday, November 5, 13
  32. 32. 20.6 M Audiencia de nuestros primeros 8 clientes. Tenemos 250 en simultáneo. Tuesday, November 5, 13
  33. 33. +300.000 Fans conseguidos con solo un concurso Tuesday, November 5, 13
  34. 34. Todos tenemos una historia con PHP Tuesday, November 5, 13
  35. 35. Tuesday, November 5, 13
  36. 36. Tuesday, November 5, 13
  37. 37. <?php echo 'Hello World'; ?> Tuesday, November 5, 13
  38. 38. ? Tuesday, November 5, 13
  39. 39. Apache HTTP Server Tuesday, November 5, 13
  40. 40. Tuesday, November 5, 13
  41. 41. Free Web Hosting Tuesday, November 5, 13
  42. 42. Tuesday, November 5, 13
  43. 43. Tuesday, November 5, 13
  44. 44. OOP Tuesday, November 5, 13
  45. 45. <? class Caja{     var $alto;     var $ancho;     var $largo;     var $contenido;     var $color; function introduce($cosa){     $this->contenido = $cosa; } function muestra_contenido(){     echo $this->contenido; } } ?> Tuesday, November 5, 13
  46. 46. Auto-complete Object class reference Tuesday, November 5, 13
  47. 47. Tuesday, November 5, 13
  48. 48. FRAMEWORKS Tuesday, November 5, 13
  49. 49. Tuesday, November 5, 13
  50. 50. 2010 Tuesday, November 5, 13
  51. 51. The Fan Machine Tuesday, November 5, 13
  52. 52. Esto tiene que escalar! Tuesday, November 5, 13
  53. 53. ¿Qué sabíamos de escalar algo en PHP? Tuesday, November 5, 13
  54. 54. ¿Qué sabíamos de escalar algo a MILLONES de usuarios en PHP? Tuesday, November 5, 13
  55. 55. 0 Tuesday, November 5, 13
  56. 56. CERO Tuesday, November 5, 13
  57. 57. Qué onda con el NoSQL? Tuesday, November 5, 13
  58. 58. Qué onda con el Sharding? Tuesday, November 5, 13
  59. 59. Cuales eran nuestros posibles problemas? Tuesday, November 5, 13
  60. 60. Escalar Rápido Tuesday, November 5, 13
  61. 61. Muchos writes Tuesday, November 5, 13
  62. 62. Mucha variación en el Tráfico Tuesday, November 5, 13
  63. 63. MUCHOS cambios en el modelo de datos Tuesday, November 5, 13
  64. 64. Mucho Storage Tuesday, November 5, 13
  65. 65. Todo bajo SSL Tuesday, November 5, 13
  66. 66. Startup = poco $$ Tuesday, November 5, 13
  67. 67. Como lo solucionamos??? Tuesday, November 5, 13
  68. 68. Possible solutions... Tuesday, November 5, 13
  69. 69. Web Server & Storage Tuesday, November 5, 13
  70. 70. HORIZONTAL SCALABILITY Tuesday, November 5, 13
  71. 71. Static File Caching Web Server 1 Web Server 2 Web Server 3 Data Caching Database 1 Database 1 (replicated DB) Tuesday, November 5, 13
  72. 72. Amazon Azure GoGrid Rackspace Vurbia - SoftLayer Akamai Tuesday, November 5, 13
  73. 73. AMAZON Tuesday, November 5, 13
  74. 74. • S3 • EC2 - Web & Memcache • CloudWatch • CloudFront • ELB - SSL termination • Route 53 Tuesday, November 5, 13
  75. 75. What about the DB? Tuesday, November 5, 13
  76. 76. MongoDB Membase Hbase Cassandra SimpleDB NoSQL BigData CouchDB Hipertable RaptorDB Tuesday, November 5, 13
  77. 77. MongoDB Membase Hbase Cassandra SimpleDB NoSQL BigData CouchDB Hipertable RaptorDB Tuesday, November 5, 13
  78. 78. NoSQL = No Trust! Tuesday, November 5, 13
  79. 79. Y MySQL qué onda? Tuesday, November 5, 13
  80. 80. Horizontal Scalability Tuesday, November 5, 13
  81. 81. Master/Slave Tuesday, November 5, 13
  82. 82. Static File Caching Web Server 1 (Transaccional, read-write) Master Database Web Server 2 Web Server 3 Data Caching Slave Database (read) Tuesday, November 5, 13 Slave Database (read)
  83. 83. DB scalling issues... Tuesday, November 5, 13
  84. 84. Static File Caching Web Server 1 (Transaccional, read-write) Master Database Web Server 2 Web Server 3 Data Caching Slave Database (read) Tuesday, November 5, 13 Slave Database (read)
  85. 85. Tuesday, November 5, 13
  86. 86. CPU Memory Disk Tuesday, November 5, 13
  87. 87. Static File Caching Web Server 1 (Transaccional, read-write) Master Database Web Server 2 Web Server 3 Data Caching Slave Database (read) Tuesday, November 5, 13 Slave Database (read)
  88. 88. Retraso Tuesday, November 5, 13
  89. 89. Tuesday, November 5, 13
  90. 90. • MAAAAANY Records - Storage issues Tuesday, November 5, 13
  91. 91. • MAAAAANY Records - Storage issues • Caching not enough Tuesday, November 5, 13
  92. 92. • MAAAAANY Records - Storage issues • Caching not enough • Alters lock tables for hours! Tuesday, November 5, 13
  93. 93. • MAAAAANY Records - Storage issues • Caching not enough • Alters lock tables for hours! • Maintenance Issues Tuesday, November 5, 13
  94. 94. • MAAAAANY Records - Storage issues • Caching not enough • Alters lock tables for hours! • Maintenance Issues • Too much response time Tuesday, November 5, 13
  95. 95. Cluster Computing Tuesday, November 5, 13
  96. 96. Tuesday, November 5, 13
  97. 97. Tuesday, November 5, 13
  98. 98. Table Partitioning Tuesday, November 5, 13
  99. 99. Tuesday, November 5, 13
  100. 100. THE approach Tuesday, November 5, 13
  101. 101. The Friendfeed approach Tuesday, November 5, 13
  102. 102. MySQL Tuesday, November 5, 13
  103. 103. Sharded MySQL databases Tuesday, November 5, 13
  104. 104. MySQL emulating NoSQL over Sharded Databases Tuesday, November 5, 13
  105. 105. Documents = Entities Tuesday, November 5, 13
  106. 106. One Table for entities, many tables for indexes. Tuesday, November 5, 13
  107. 107. The Model index_user index_promo entities index_participant index_vote index_fan_conversion Tuesday, November 5, 13
  108. 108. The Model index_user index_promo entities index_participant index_vote index_fan_conversion Tuesday, November 5, 13
  109. 109. entities table added_id id body class created updated deleted Tuesday, November 5, 13
  110. 110. Entities Table CREATE TABLE `entities` ( `added_id` int(11) NOT NULL AUTO_INCREMENT, `id` char(32) NOT NULL DEFAULT '', `updated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, `deleted` timestamp NULL DEFAULT NULL COMMENT `body` mediumtext, `class` varchar(50) DEFAULT NULL, `created` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00', PRIMARY KEY (`added_id`), UNIQUE KEY `id` (`id`), KEY `updated` (`updated`), KEY `class` (`class`), KEY `created` (`created`) ) ENGINE=InnoDB AUTO_INCREMENT=1692491 DEFAULT CHARSET=utf8 | Tuesday, November 5, 13
  111. 111. users index table entity_id fb_uid Tuesday, November 5, 13
  112. 112. user index table CREATE TABLE `index_user` ( `entity_id` char(32) NOT NULL DEFAULT '', `uid` bigint(20) NOT NULL, PRIMARY KEY (`entity_id`,`uid`), UNIQUE KEY `uid` (`uid`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8 Tuesday, November 5, 13
  113. 113. How do we scale it? Tuesday, November 5, 13
  114. 114. File Caching Web Server 1 Web Server 2 Web Server 3 Data Caching DB Server 1 Shard 1 Shard 4 Shard 7 Shard 10 Shard 2 Shard 5 Shard 8 Shard 11 Shard 3 Shard 6 Shard 9 Shard 12 Database Backups Tuesday, November 5, 13 } Databases
  115. 115. Static File Caching Web Server 1 Web Server 2 Web Server 3 Web Server 3 DB Server 1 DB Server 2 DB Server 3 DB Server 4 Shard 1 Shard 2 Shard 3 Shard 4 Shard 5 Shard 6 Shard 7 Shard 8 Shard 9 Shard 10 Shard 11 Shard 12 Database Backups per Shard Tuesday, November 5, 13
  116. 116. THE Benefits... Tuesday, November 5, 13
  117. 117. • Bases de datos más pequeñas son más fáciles de manejar • Bases de datos más chicas son más rápidas • El sharding puede reducir costos Tuesday, November 5, 13
  118. 118. • Server maintenance per db-server • Data Maintenance Per db-shard • NO ALTERS on tables • Easy recovery and replication with MultiAZ • Easy scaling • Easy migration to other No-SQL data engines Tuesday, November 5, 13
  119. 119. ¿When is DB sharding Appropiate? Tuesday, November 5, 13
  120. 120. High transaction Applications Write-intensive transactions Tuesday, November 5, 13
  121. 121. Cómo repartimos entre los shards? Tuesday, November 5, 13
  122. 122. Driver Programmatic Tuesday, November 5, 13 DB
  123. 123. DB Driver DB Server 1 DB Server 2 DB Server 3 DB Server 4 Shard 1 Shard 2 Shard 3 Shard 4 Shard 5 Shard 6 Shard 7 Shard 8 Shard 9 Shard 10 Shard 11 Shard 12 memory cache Driver DB Tuesday, November 5, 13
  124. 124. DB driver table shard_id email Tuesday, November 5, 13
  125. 125. Progamatic Driver Tuesday, November 5, 13
  126. 126. Cómo usamos el driver Fetchear la entidad de un usuario: Tuesday, November 5, 13
  127. 127. Cómo usamos el driver Crear o actualizar una promo: Tuesday, November 5, 13
  128. 128. otra forma de guardar... Tuesday, November 5, 13
  129. 129. y se utiliza así en un helper: Tuesday, November 5, 13
  130. 130. Amazon Web Services Tuesday, November 5, 13
  131. 131. Tuesday, November 5, 13
  132. 132. Tuesday, November 5, 13
  133. 133. Tuesday, November 5, 13
  134. 134. Tuesday, November 5, 13
  135. 135. Tuesday, November 5, 13
  136. 136. Tuesday, November 5, 13
  137. 137. Tuesday, November 5, 13
  138. 138. Como evolucionamos en Amazon... Tuesday, November 5, 13
  139. 139. Un servidor LAMP, S3 DNS externo Tuesday, November 5, 13
  140. 140. Amazon Linux AMI Tuesday, November 5, 13
  141. 141. Memcache Tuesday, November 5, 13
  142. 142. Balancer Tuesday, November 5, 13
  143. 143. Route 53 Tuesday, November 5, 13
  144. 144. Que aprendimos?? Tuesday, November 5, 13
  145. 145. OJO con las public AMIs!!! Tuesday, November 5, 13
  146. 146. Cuando estopeas una instancia.... CHAN Tuesday, November 5, 13
  147. 147. PERDES LA IP ELASTICA Tuesday, November 5, 13
  148. 148. SE TE CAMBIA LA DNS PUBLICA Tuesday, November 5, 13
  149. 149. Clonar es super simple pero no super rápido Tuesday, November 5, 13
  150. 150. Tener stopeada una instancia sale.... casi nada!! Tuesday, November 5, 13
  151. 151. DNS... Tuesday, November 5, 13
  152. 152. SSL Termination Tuesday, November 5, 13
  153. 153. Límite de certificados de ssl Tuesday, November 5, 13
  154. 154. Questions? Matias Paterlini matias@thefanmachine.com Te estamos buscando!! jobs@thefanmachine.com Tuesday, November 5, 13

×