Flipkart scales its PHP infrastructure through techniques like horizontal scaling using load balancers, optimizing web servers through technologies like PHP-FPM, and employing a Java middleware called fk-w3-agent to offload tasks from PHP like logging, connection pooling, and running statistics. This improves performance by reducing PHP resource usage and allowing parallelization of tasks. High availability is ensured through techniques such as load balancer standby nodes and session affinity configurations.

1. How Flipkart
Scales PHP
Tips & Tricks

2. About Flipkart
• Flipkart.com
• India’s largest eCommerce website*
• Millions of requests/day

3. About Me
• Siddhartha Reddy
• Tech Lead at Flipkart
• Taking care of Search & Browse experience

4. Today
• Flipkart.com’s Architecture
• How each component addresses scale

5. Flipkart.com’s
Architecture
Session
Database
Store
CMS
Load Web
Balancer Server(s)
Search
Caches OMS

6. What is Scaling?

7. What is Scaling?
• Handle large amount of traffic
• High availability
• Good response times

8. How Flipkart Scales

9. How Flipkart Scales
• Logging and measuring
• Horizontal Scaling
• Load-balancing
• High Availability setup
• Web-server optimizations
• Caching

10. How Flipkart Scales

11. How Flipkart Scales
• Logging and measuring
• Horizontal Scaling
• Load-balancing
• High Availability setup
• Web-server optimizations
• Caching

12. Logging & Measuring

13. Logging & Measuring

14. How Flipkart Scales
• Horizontal Scaling
• Load-balancing
• High Availability setup
• Web-server optimizations
• Caching

15. Traffic

16. Servers: Chuck Norris Style

17. Servers: Spartan Style

18. Horizontal Scaling

19. Horizontal Scaling
Traffic
web-1 web-2 ... web-n

20. Horizontal Scaling
Load-
Traffic balancer
web-1 web-2 ... web-n

21. Horizontal Scaling
Load-
Traffic balancer
web-1 web-2 ... web-n

22. Horizontal Scaling
Load-
Traffic balancer
web-1 web-2 ... web-n

23. Horizontal Scaling
Load-
Traffic balancer
web-1 web-2 ... web-n

24. Horizontal Scaling
Load-
Traffic balancer
web-1 web-2 ... web-n

25. How Flipkart Scales
• Horizontal Scaling
• Load-balancing
• High Availability setup
• Web-server optimizations
• Caching

26. Load Balancer

27. Load Balancer
• Internal binary services (Apache Thrift)
• Internal web services (HTTP)
• External web traffic (HTTP/HTTPS)

28. Load Balancer
• Internal binary services (Apache Thrift)
• Internal web services (HTTP)
• External web traffic (HTTP/HTTPS)

29. Load Balancer
• Internal binary services (Apache Thrift)
• Software: HAProxy
• Transport layer (TCP) load balancer
• TCP connect for availability check
• Internal web services (HTTP)
• External web traffic (HTTP/HTTPS)

30. Load Balancer
• Internal binary services (Apache Thrift)
• Internal web services (HTTP)
• External web traffic (HTTP/HTTPS)

31. Load Balancer
• Internal binary services (Apache Thrift)
• Internal web services (HTTP)
• Software: Varnish
• Application Layer (HTTP) load balancer
• Configurable/scriptable caching
• HTTP GET for availability check
• External web traffic (HTTP/HTTPS)

32. Load Balancer
• Internal binary services (Apache Thrift)
• Internal web services (HTTP)
• External web traffic (HTTP/HTTPS)

33. Load Balancer
• Internal binary services (Apache Thrift)
• Internal web services (HTTP)
• External web traffic (HTTP/HTTPS)
• Software: Nginx
• Application Layer (HTTP/HTTPS) load balancer
• Caches and serves static content (JS/images)
• HTTP GET for availability check

34. Load Balancer:
Session Affinity?
Load-
balancer
... web-n
web-1 web-2

35. Load Balancer:
Session Affinity?
SID: 123 (1)
Load-
balancer
... web-n
web-1 web-2

36. Load Balancer:
Session Affinity?
SID: 123 (1)
Load-
balancer
... web-n
web-1 web-2

37. Load Balancer:
Session Affinity?
SID: 123 (1)
Load-
balancer
SID: 123 (1)
... web-n
web-1 web-2

38. Load Balancer:
Session Affinity?
SID: 123 (1)
Load-
SID: 123 (2) balancer
SID: 123 (1)
... web-n
web-1 web-2

39. Load Balancer:
Session Affinity?
SID: 123 (1)
Load-
SID: 123 (2) balancer
SID: 123 (1)
... web-n
web-1 web-2

40. Load Balancer:
Session Affinity?
SID: 123 (1)
Load-
SID: 123 (2) balancer
SID: 123 (1)
SID: 123 (2)
... web-n
web-1 web-2

41. HTTP Load Balancing:
Session Affinity

42. HTTP Load Balancing:
Session Affinity
• (+) Sessions can be stored on the web server
• (+) Different code on different web servers
• Useful for testing new code on a subset of users
• (-) If any web server goes down, need to migrate sessions
• (-) Not easy to take a web server out-of-rotation

43. HTTP Load Balancer:
Session Non-affinity

44. HTTP Load Balancer:
Session Non-affinity
• Sessions stored in a common session-store
• (+) Easy to manage -- all web servers have identical state
• (+) If a web server goes down -- no problem
• (+) Can take any web server out-of-rotation
• (-) Can’t deploy to a subset of servers for testing
• (-) Central session-store could be slower

45. HTTP Load Balancer:
Session Some-affinity

46. HTTP Load Balancer:
Session Some-affinity
• Sessions sticky to a subset (group) of machines
• Use a common session-store

47. How Flipkart Scales
• Horizontal Scaling
• Load-balancing
• High Availability setup
• Web-server optimizations
• Caching

48. Load Balancer:
Hardware?

49. Load Balancer:
Hardware?
• Expensive
• Inflexible to manage
• They run Linux anyway!

50. Flipkart’s Load-
Balancers

51. Flipkart’s Load-
Balancers

52. Flipkart’s Load-
Balancers

53. Flipkart’s Load-
Balancers

54. Flipkart’s Load-
Balancers

55. Flipkart’s Load-
Balancers

56. Flipkart’s Load-
Balancers

57. Flipkart’s Load-
Balancers

58. Flipkart’s Load-
balancers
bond:1 bond:1
(10.3.1.1) (10.3.1.2)
bond:0 bond:0
(10.3.0.1) (10.3.0.2)

59. Flipkart’s Load-
balancers
bond:1 bond:1
(10.3.1.1) (10.3.1.2)
bond:0 bond:0
(10.3.0.1) (10.3.0.2)
Active

60. Flipkart’s Load-
balancers
bond:1 bond:1
(10.3.1.1) (10.3.1.2)
bond:0 bond:0
(10.3.0.1) (10.3.0.2)
Active Hot Stand-by

61. Flipkart’s Load-
balancers
bond:1 bond:1
(10.3.1.1) (10.3.1.2)
bond:0 bond:0
(10.3.0.1) (10.3.0.2)
bond:0:0
(10.3.0.0)
Active Hot Stand-by

62. Flipkart’s Load-
balancers
bond:1 bond:1
(10.3.1.1) (10.3.1.2)
bond:0 bond:0
Heartbeat
(10.3.0.1) (10.3.0.2)
bond:0:0
(10.3.0.0)
Active Hot Stand-by

63. How Flipkart Scales
• Horizontal Scaling
• Load-balancing
• High Availability setup
• Web-server optimizations
• Caching

64. On the Web Servers

65. On the Web Servers
• PHP5

66. On the Web Servers
• PHP5
• Apache/mod_php

67. On the Web Servers
• PHP5
• Apache/mod_php

68. On the Web Servers
• PHP5
• Apache/mod_php
• Apache/mod_fcgid (FastCGI)

69. On the Web Servers
• PHP5
• Apache/mod_php
• Apache/mod_fcgid (FastCGI)

70. On the Web Servers
• PHP5
• Apache/mod_php
• Apache/mod_fcgid (FastCGI)
• PHP-FPM

71. On the Web Servers
• PHP5
• Apache/mod_php
• Apache/mod_fcgid (FastCGI)
• PHP-FPM
• PHP FastCGI Process Manager
• Great process management
• Security
• Configurability & control
• Adaptive process spawning (5.3.3RC1+)

72. On the Web Server:
fk-w3-agent

73. On the Web Server:
fk-w3-agent
• Simple Java “middleware” daemon

74. On the Web Server:
fk-w3-agent
• Simple Java “middleware” daemon
• Deployed on each web server

75. On the Web Server:
fk-w3-agent
• Simple Java “middleware” daemon
• Deployed on each web server
• Communicates with PHP through local socket

76. On the Web Server:
fk-w3-agent
• Simple Java “middleware” daemon
• Deployed on each web server
• Communicates with PHP through local socket
• Hosts pluggable “handlers”

77. On the Web Server:
fk-w3-agent
• Simple Java “middleware” daemon
• Deployed on each web server
• Communicates with PHP through local socket
• Hosts pluggable “handlers”

78. fk-w3-agent: Why?

79. fk-w3-agent: Why?
• “I wish PHP could ...”

80. fk-w3-agent: Why?
• “I wish PHP could ...”
• Logging

81. fk-w3-agent: Why?
• “I wish PHP could ...”
• Logging
• Connection pools

82. fk-w3-agent: Why?
• “I wish PHP could ...”
• Logging
• Connection pools
• Parallelization

83. fk-w3-agent: Why?
• “I wish PHP could ...”
• Logging
• Connection pools
• Parallelization
• Config & discovery

84. fk-w3-agent: Why?
• “I wish PHP could ...”
• Logging
• Connection pools
• Parallelization
• Config & discovery
• Running statistics

85. fk-w3-agent: Logging

86. fk-w3-agent: Logging
• We like to log, a lot

87. fk-w3-agent: Logging
• We like to log, a lot
• Logs need to be flushed to disk

88. fk-w3-agent: Logging
• We like to log, a lot
• Logs need to be flushed to disk
• PHP resources wasted
• waiting for disk I/O
• waiting for lock on log files
• or logs go missing

89. fk-w3-agent: Logging

90. fk-w3-agent: Logging
• PHP processes send logs to fk-w3-agent
• fk-w3-agent writes logs to files

91. fk-w3-agent:
Connection Pooling

92. fk-w3-agent:
Connection Pooling
• Many service calls / request

93. fk-w3-agent:
Connection Pooling
• Many service calls / request
• Cost of recreating connections high

94. fk-w3-agent:
Connection Pooling
• Many service calls / request
• Cost of recreating connections high
• PHP does not support connection pools*

95. fk-w3-agent:
Connection Pooling

96. fk-w3-agent:
Connection Pooling
• fk-w3-agent maintains persistent connection pools
to various services
• PHP accesses services by communicating with fk-
w3-agent

97. fk-w3-agent:
Parallelization

98. fk-w3-agent:
Parallelization
• Service calls can be parallelized

99. fk-w3-agent:
Parallelization
• Service calls can be parallelized
• Multi-threading in PHP is ...

100. fk-w3-agent:
Parallelization

101. fk-w3-agent:
Parallelization
• PHP sends request to fk-w3-agent
• fk-w3-agent uses multiple threads to hit services in
parallel

102. fk-w3-agent:
Config & Discovery

103. fk-w3-agent:
Config & Discovery
• Change config without deployment?

104. fk-w3-agent:
Config & Discovery
• Change config without deployment?
• Simple: store config in a database

105. fk-w3-agent:
Config & Discovery
• Change config without deployment?
• Simple: store config in a database
• But: reloading config on each request: expensive

106. fk-w3-agent:
Config & Discovery

107. fk-w3-agent:
Config & Discovery
• fk-w3-agent loads config from database
• Caches it in memory
• Keeps it up-to-date
• PHP gets config from fk-w3-agent

108. fk-w3-agent:
Running Statistics

109. fk-w3-agent:
Running Statistics
• PHP sends basic stats to fk-w3-agent for every
request
• fk-w3-agent aggregates these stats
• Used for monitoring

110. How Flipkart Scales
• Horizontal Scaling
• Load-balancing
• Web-server optimizations
• Caching

111. To Cache or not to
Cache?

112. To Cache or not to
Cache?
• “There are only two hard things in Computer
Science: cache invalidation and naming things”. --
Tim Bray quoting Phil Karlton

113. To Cache or not to
Cache?
• “There are only two hard things in Computer
Science: cache invalidation and naming things”. --
Tim Bray quoting Phil Karlton
• “There are only two hard problems in Computer
Science: cache invalidation, naming things and off-by-1
errors.”

114. Caches Invalidation

115. Caches Invalidation
• Objects updated => Notification Message

116. Caches Invalidation
• Objects updated => Notification Message
• Remove updated objects from cache

117. Caches Invalidation
• Objects updated => Notification Message
• Remove updated objects from cache
• Replace updated objects in cache

118. Caches Invalidation
• Objects updated => Notification Message
• Remove updated objects from cache
• Replace updated objects in cache
• Aggressively populate cold cache

119. Caches at Flipkart

120. Caches at Flipkart
• Memcached -- in memory caching

121. Caches at Flipkart
• Memcached -- in memory caching
• Redis -- persistent store/cache

122. Caches at Flipkart
• Memcached -- in memory caching
• Redis -- persistent store/cache
• Varnish -- HTTP caching

123. Caches at Flipkart
• Memcached -- in memory caching
• Redis -- persistent store/cache
• Varnish -- HTTP caching
• fk-w3-agent -- Config caching

124. Let’s continue the
discussion
• @sids on Twitter
• siddhartha@flipkart.com
• #phpcloud on irc.freenode.net