In this presentation, we discuss various load balancing algorithms, their benefits, the places where they will screw you over and which one you should be choosing for your use case.
9. Random algorithm:
โข Can you โtruly randomlyโ distribute load?
โข It is simple to implement and scales well when you have a
distributed load balancer setup.
โข Latency is not being accounted for.
โข Servers which are underperforming can have more requests
being sent to them.
14. Round Robin
algorithm
โข Guarantees same number of requests
to all servers.
โข We again face the same latency
problem.
โข Does not scale well for distributed
Load balancers as the logic of where to
start from needs to be statically
hardcoded.
โข What if the backend servers have
different configs?
17. Weighted
Round Robin
algorithm
โข Servers can take load based on their
H/W configs.
โข Can introduce another service which
check for latency and update the
weights dynamically.
โข The weights are present on the load
balancer and not the servers, so if it is
distributed, all need to be in sync.
โข Also called โJoin shortest queueโ
algorithm.
20. Least
Connections
algorithm
โข Routes requests considering the
current load on each servers.
โข In a distributed setup, the LBs should
be in sync about the active
connections on each server.
โข We can have race condition of all LBs
sending to the same server if all
requests come to the LBs at once.
22. Each LB, when receives a request
chooses a set of random servers.
Then, the least connection algorithm
is applied to those servers and the
backend server is chosen for the
request.
23. Least
connections
with
randomness
โข We are trying to solve the problem of race
condition in Least connections algorithm by using
this approach.
โข This works only in a distributed LB setup.
โข Adds one more layer of complexity to choose
random set of servers.
โข The number of random servers that need to be
chosen needs to be hardcoded in each LB. This
number cannot be too close to the actual number
of servers ? And cannot be too less as well?
24. โข All the techniques that we have looked at so
far assume that the application is stateless
and any of the servers can serve requests.
โข Maintaining persistent connections is a
much more difficult task and shall be
discussed in the future videos.
25. โข Join idle queue algorithm โ Link
โข Source IP hash algorithm, based on consistent hashing โ
Link1, Link2
โข This is used for maintaining persistent connections.
โข Global Server Load Balancing โ Link
Further reading:
26. References
โข Comparison between basic LB algorithms: Link
โข Tyler McMullen - Load Balancing is Impossible: Link
โข โBalls into Binsโ โ A Simple and Tight Analysis - Link
โข The Power of Two Choices in Randomized Load Balancing - Link