The problem of handling 10k users concurrently (cC120)

2.  Since the beginning of time, humans have
been finding ways to be more and more
efficient under all circumstances possible.

3.  1 GHz CPUs were released in the year
2000. 3 GHz CPUs were released in year
2002. So following that trend 10 GHz CPUs
were expected in 2005. And by today, 192
GHz CPUs were supposed to exist.
However, that's not the case. We have only,
over time, found out ways to make software
more and more efficient making them
delivering more on the same hardware.

4.  A typical
scenario
where this
performance
becomes
extremely
prominent is
web server.

5.  Conventional multi-threaded servers are
architecturally built to create one thread per
user to serve them.
 However there's a limit to the number of
system threads and I/O channels the OS
might allow an application to make (Limit of
the I/O channels is called OS limit of the file
descriptor).
 Under sufficient stress this system inherently
fails.

7.  When too many (let’s give that a number,
say 10000) users try to connect to one
server at a time, availability is reduced and it
starts dropping connections.
 Also the volatile memory becomes a
bottleneck
 This is (rather famously!) known as the
C10k problem.

9. When there is a need for high velocity data
transaction, for making real-time reactive
applications (long-polling reverse AJAX) or for
handling many socket connections
simultaneously.

10.  Basic concept of non-blocking I/O is
something like, instead of making users at a
roll corner to wait in long queue while their
rolls are being made, they are handed over a
token and released immediately. They're
called back by the roll corner to serve, when
their rolls are ready. This helps the roll
corner to serve all customers. Customers are
also freed from frustration of standing and
waiting in long queues.

12.  JVM based – Java NIO (Java library for non-
blocking I/O), Netty (Java NIO based framework
for building scalable applications), Akka (Actor
model based message driven framework),
Vert.x (Event driven non-blocking framework for
writing scalable web apps)
 JavaScript based – JavaScript runtime based
on Google’s V8 JavaScript engine. Natively
event-driven and non-blocking.

13.  Nginx – High performance static content
server and TCP load balancer. Can be
coupled with anything that supports FastCGI
like PHP-FPM or HHVM.
 HAProxy – High performance, lightweight,
efficient load balancer that supports almost
all protocols.
 G-WAN – Less known hero of the crowd with
amazing benchmarks.