How Flipkart scales PHP

107,700 views
94,068 views

Published on

Slides from the talk I gave a

Published in: Technology
11 Comments
288 Likes
Statistics
Notes
No Downloads
Views
Total views
107,700
On SlideShare
0
From Embeds
0
Number of Embeds
3,604
Actions
Shares
0
Downloads
0
Comments
11
Likes
288
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • How Flipkart scales PHP

    1. How Flipkart Scales PHP Tips & Tricks
    2. About Flipkart• Flipkart.com• India’s largest eCommerce website*• Millions of requests/day
    3. About Me• Siddhartha Reddy• Tech Lead at Flipkart• Taking care of Search & Browse experience
    4. Today• Flipkart.com’s Architecture • How each component addresses scale
    5. Flipkart.com’s Architecture Session Database Store CMS Load WebBalancer Server(s) Search Caches OMS
    6. What is Scaling?
    7. What is Scaling?• Handle large amount of traffic• High availability• Good response times
    8. How Flipkart Scales
    9. How Flipkart Scales• Logging and measuring• Horizontal Scaling • Load-balancing• High Availability setup• Web-server optimizations• Caching
    10. How Flipkart Scales
    11. How Flipkart Scales• Logging and measuring• Horizontal Scaling • Load-balancing• High Availability setup• Web-server optimizations• Caching
    12. Logging & Measuring
    13. Logging & Measuring
    14. How Flipkart Scales• Horizontal Scaling • Load-balancing• High Availability setup• Web-server optimizations• Caching
    15. Traffic
    16. Servers: Chuck Norris Style
    17. Servers: Spartan Style
    18. Horizontal Scaling
    19. Horizontal Scaling Traffic web-1 web-2 ... web-n
    20. Horizontal Scaling Load- Traffic balancer web-1 web-2 ... web-n
    21. Horizontal Scaling Load- Traffic balancer web-1 web-2 ... web-n
    22. Horizontal Scaling Load- Traffic balancer web-1 web-2 ... web-n
    23. Horizontal Scaling Load- Traffic balancer web-1 web-2 ... web-n
    24. Horizontal Scaling Load- Traffic balancer web-1 web-2 ... web-n
    25. How Flipkart Scales• Horizontal Scaling • Load-balancing• High Availability setup• Web-server optimizations• Caching
    26. Load Balancer
    27. Load Balancer• Internal binary services (Apache Thrift)• Internal web services (HTTP)• External web traffic (HTTP/HTTPS)
    28. Load Balancer• Internal binary services (Apache Thrift)• Internal web services (HTTP)• External web traffic (HTTP/HTTPS)
    29. Load Balancer• Internal binary services (Apache Thrift) • Software: HAProxy • Transport layer (TCP) load balancer • TCP connect for availability check• Internal web services (HTTP)• External web traffic (HTTP/HTTPS)
    30. Load Balancer• Internal binary services (Apache Thrift)• Internal web services (HTTP)• External web traffic (HTTP/HTTPS)
    31. Load Balancer• Internal binary services (Apache Thrift)• Internal web services (HTTP) • Software: Varnish • Application Layer (HTTP) load balancer • Configurable/scriptable caching • HTTP GET for availability check• External web traffic (HTTP/HTTPS)
    32. Load Balancer• Internal binary services (Apache Thrift)• Internal web services (HTTP)• External web traffic (HTTP/HTTPS)
    33. Load Balancer• Internal binary services (Apache Thrift)• Internal web services (HTTP)• External web traffic (HTTP/HTTPS) • Software: Nginx • Application Layer (HTTP/HTTPS) load balancer • Caches and serves static content (JS/images) • HTTP GET for availability check
    34. Load Balancer:Session Affinity? Load- balancer ... web-n web-1 web-2
    35. Load Balancer:Session Affinity? SID: 123 (1) Load- balancer ... web-n web-1 web-2
    36. Load Balancer:Session Affinity? SID: 123 (1) Load- balancer ... web-n web-1 web-2
    37. Load Balancer:Session Affinity? SID: 123 (1) Load- balancer SID: 123 (1) ... web-n web-1 web-2
    38. Load Balancer:Session Affinity? SID: 123 (1) Load- SID: 123 (2) balancer SID: 123 (1) ... web-n web-1 web-2
    39. Load Balancer:Session Affinity? SID: 123 (1) Load- SID: 123 (2) balancer SID: 123 (1) ... web-n web-1 web-2
    40. Load Balancer:Session Affinity? SID: 123 (1) Load- SID: 123 (2) balancer SID: 123 (1) SID: 123 (2) ... web-n web-1 web-2
    41. HTTP Load Balancing: Session Affinity
    42. HTTP Load Balancing: Session Affinity• (+) Sessions can be stored on the web server• (+) Different code on different web servers • Useful for testing new code on a subset of users• (-) If any web server goes down, need to migrate sessions• (-) Not easy to take a web server out-of-rotation
    43. HTTP Load Balancer:Session Non-affinity
    44. HTTP Load Balancer: Session Non-affinity• Sessions stored in a common session-store• (+) Easy to manage -- all web servers have identical state• (+) If a web server goes down -- no problem• (+) Can take any web server out-of-rotation• (-) Can’t deploy to a subset of servers for testing• (-) Central session-store could be slower
    45. HTTP Load Balancer:Session Some-affinity
    46. HTTP Load Balancer: Session Some-affinity• Sessions sticky to a subset (group) of machines• Use a common session-store
    47. How Flipkart Scales• Horizontal Scaling • Load-balancing• High Availability setup• Web-server optimizations• Caching
    48. Load Balancer: Hardware?
    49. Load Balancer: Hardware?• Expensive• Inflexible to manage• They run Linux anyway!
    50. Flipkart’s Load- Balancers
    51. Flipkart’s Load- Balancers
    52. Flipkart’s Load- Balancers
    53. Flipkart’s Load- Balancers
    54. Flipkart’s Load- Balancers
    55. Flipkart’s Load- Balancers
    56. Flipkart’s Load- Balancers
    57. Flipkart’s Load- Balancers
    58. Flipkart’s Load- balancers bond:1 bond:1(10.3.1.1) (10.3.1.2) bond:0 bond:0(10.3.0.1) (10.3.0.2)
    59. Flipkart’s Load- balancers bond:1 bond:1 (10.3.1.1) (10.3.1.2) bond:0 bond:0 (10.3.0.1) (10.3.0.2)Active
    60. Flipkart’s Load- balancers bond:1 bond:1 (10.3.1.1) (10.3.1.2) bond:0 bond:0 (10.3.0.1) (10.3.0.2)Active Hot Stand-by
    61. Flipkart’s Load- balancers bond:1 bond:1 (10.3.1.1) (10.3.1.2) bond:0 bond:0 (10.3.0.1) (10.3.0.2) bond:0:0 (10.3.0.0)Active Hot Stand-by
    62. Flipkart’s Load- balancers bond:1 bond:1 (10.3.1.1) (10.3.1.2) bond:0 bond:0 Heartbeat (10.3.0.1) (10.3.0.2) bond:0:0 (10.3.0.0)Active Hot Stand-by
    63. How Flipkart Scales• Horizontal Scaling • Load-balancing• High Availability setup• Web-server optimizations• Caching
    64. On the Web Servers
    65. On the Web Servers• PHP5
    66. On the Web Servers• PHP5• Apache/mod_php
    67. On the Web Servers• PHP5• Apache/mod_php
    68. On the Web Servers• PHP5• Apache/mod_php• Apache/mod_fcgid (FastCGI)
    69. On the Web Servers• PHP5• Apache/mod_php• Apache/mod_fcgid (FastCGI)
    70. On the Web Servers• PHP5• Apache/mod_php• Apache/mod_fcgid (FastCGI)• PHP-FPM
    71. On the Web Servers• PHP5• Apache/mod_php• Apache/mod_fcgid (FastCGI)• PHP-FPM • PHP FastCGI Process Manager • Great process management • Security • Configurability & control • Adaptive process spawning (5.3.3RC1+)
    72. On the Web Server: fk-w3-agent
    73. On the Web Server: fk-w3-agent• Simple Java “middleware” daemon
    74. On the Web Server: fk-w3-agent• Simple Java “middleware” daemon• Deployed on each web server
    75. On the Web Server: fk-w3-agent• Simple Java “middleware” daemon• Deployed on each web server• Communicates with PHP through local socket
    76. On the Web Server: fk-w3-agent• Simple Java “middleware” daemon• Deployed on each web server• Communicates with PHP through local socket• Hosts pluggable “handlers”
    77. On the Web Server: fk-w3-agent• Simple Java “middleware” daemon• Deployed on each web server• Communicates with PHP through local socket• Hosts pluggable “handlers”
    78. fk-w3-agent: Why?
    79. fk-w3-agent: Why?• “I wish PHP could ...”
    80. fk-w3-agent: Why?• “I wish PHP could ...”• Logging
    81. fk-w3-agent: Why?• “I wish PHP could ...”• Logging• Connection pools
    82. fk-w3-agent: Why?• “I wish PHP could ...”• Logging• Connection pools• Parallelization
    83. fk-w3-agent: Why?• “I wish PHP could ...”• Logging• Connection pools• Parallelization• Config & discovery
    84. fk-w3-agent: Why?• “I wish PHP could ...”• Logging• Connection pools• Parallelization• Config & discovery• Running statistics
    85. fk-w3-agent: Logging
    86. fk-w3-agent: Logging• We like to log, a lot
    87. fk-w3-agent: Logging• We like to log, a lot• Logs need to be flushed to disk
    88. fk-w3-agent: Logging• We like to log, a lot• Logs need to be flushed to disk• PHP resources wasted • waiting for disk I/O • waiting for lock on log files • or logs go missing
    89. fk-w3-agent: Logging
    90. fk-w3-agent: Logging• PHP processes send logs to fk-w3-agent• fk-w3-agent writes logs to files
    91. fk-w3-agent:Connection Pooling
    92. fk-w3-agent: Connection Pooling• Many service calls / request
    93. fk-w3-agent: Connection Pooling• Many service calls / request• Cost of recreating connections high
    94. fk-w3-agent: Connection Pooling• Many service calls / request• Cost of recreating connections high• PHP does not support connection pools*
    95. fk-w3-agent:Connection Pooling
    96. fk-w3-agent: Connection Pooling• fk-w3-agent maintains persistent connection pools to various services• PHP accesses services by communicating with fk- w3-agent
    97. fk-w3-agent:Parallelization
    98. fk-w3-agent: Parallelization• Service calls can be parallelized
    99. fk-w3-agent: Parallelization• Service calls can be parallelized• Multi-threading in PHP is ...
    100. fk-w3-agent:Parallelization
    101. fk-w3-agent: Parallelization• PHP sends request to fk-w3-agent• fk-w3-agent uses multiple threads to hit services in parallel
    102. fk-w3-agent:Config & Discovery
    103. fk-w3-agent: Config & Discovery• Change config without deployment?
    104. fk-w3-agent: Config & Discovery• Change config without deployment?• Simple: store config in a database
    105. fk-w3-agent: Config & Discovery• Change config without deployment?• Simple: store config in a database• But: reloading config on each request: expensive
    106. fk-w3-agent:Config & Discovery
    107. fk-w3-agent: Config & Discovery• fk-w3-agent loads config from database • Caches it in memory • Keeps it up-to-date• PHP gets config from fk-w3-agent
    108. fk-w3-agent:Running Statistics
    109. fk-w3-agent: Running Statistics• PHP sends basic stats to fk-w3-agent for every request• fk-w3-agent aggregates these stats• Used for monitoring
    110. How Flipkart Scales• Horizontal Scaling • Load-balancing• Web-server optimizations• Caching
    111. To Cache or not to Cache?
    112. To Cache or not to Cache?• “There are only two hard things in Computer Science: cache invalidation and naming things”. -- Tim Bray quoting Phil Karlton
    113. To Cache or not to Cache?• “There are only two hard things in Computer Science: cache invalidation and naming things”. -- Tim Bray quoting Phil Karlton• “There are only two hard problems in Computer Science: cache invalidation, naming things and off-by-1 errors.”
    114. Caches Invalidation
    115. Caches Invalidation• Objects updated => Notification Message
    116. Caches Invalidation• Objects updated => Notification Message• Remove updated objects from cache
    117. Caches Invalidation• Objects updated => Notification Message• Remove updated objects from cache• Replace updated objects in cache
    118. Caches Invalidation• Objects updated => Notification Message• Remove updated objects from cache• Replace updated objects in cache• Aggressively populate cold cache
    119. Caches at Flipkart
    120. Caches at Flipkart• Memcached -- in memory caching
    121. Caches at Flipkart• Memcached -- in memory caching• Redis -- persistent store/cache
    122. Caches at Flipkart• Memcached -- in memory caching• Redis -- persistent store/cache• Varnish -- HTTP caching
    123. Caches at Flipkart• Memcached -- in memory caching• Redis -- persistent store/cache• Varnish -- HTTP caching• fk-w3-agent -- Config caching
    124. Let’s continue the discussion• @sids on Twitter• siddhartha@flipkart.com• #phpcloud on irc.freenode.net

    ×