Hybrid Architecture
for Networked Games
1

single
completely
centralized
decentralized
server
2

P2P with
Central Arbiter
3

A trusted central arbiter receives all game
traffic and resolve conflicts if necessary.
4

Benefits of
P2P + Arbiter
5

low latency among players
6

have trusted authority now
7

Drawbacks of
P2P + Arbiter
8

limited scalability
arbiter becomes bottleneck
9

Mirrored Servers
Architecture
10

Multiple servers, each replicating the complete game
states, serve clients from different geographical
regions. Each client connects to one server.
11

Servers may be connected with high speed,
provisioned networked, reducing synchronization
latency among the servers.
12

Updates from a client are sent to its server, which then
forwards it to other servers. The servers then forward
the updates to all relevant clients.
13

P2P-style state synchronization
among servers is easier, since
1. servers can be trusted
2. clocks can be synchronized
3. IP multicast may be used
4. higher availability
5. shorter delay
14

Benefits of
Mirrored Servers
15

no single point of failures
A client can use another replicated server
if their nearest server fails.
16

increase total network capacity
good if networking bandwidth is the
bottleneck
17

lower latency between
client and server
18

Drawbacks of
Mirrored Servers
19

latency between players increases
due to additional forwarding
20

cannot scale to large world
each server still keeps the complete states
21

how to assign
players to servers?
22

Assigning players to closest servers may lead to
overloaded servers.
23

Assigning players to remote servers may increase
end-to-end latency.
24

But we only care about latency between players
that are relevant to each other.
25

An Open Problem
26

let
AOI(p) be the set of players relevant to p
S(p) be the server of player p
d(i,j) be the delay between i and j
P(s) be the set of players assigned to server s
27

let
L(p) be the average latency experience by p,
considering only relevant players
28

want to find S() such that
maximum L(p) is minimized over all p
subjected to
maximum P(s) is constrained for all s
29

Zoned Servers
Architecture
30

Divide the game world into independent regions.
One server is in-charged of one region.
31

A client connects to the server serving its current
region, and is handed-off to another server when it
moves to another region.
32

Servers seldom communicate with each other.
33

Updates from a client are sent to its server, which then
forwards it to all relevant clients in the same region.
34

35

Advantages of
Zoned Servers
36

Can scale to large world
Just add more servers
37

Drawbacks of
Zoned Servers
38

Single point of failures
(But can deploy mirrors for each region)
39

Constrained Game Designs
40

Supporting Seamless
Game Worlds
41

Divide the game world into regions.
One server is in-charged of one region.
42

Player may move around the world seamlessly
43

A client may see events and objects
from neighboring regions.
44

Events occurring in one region may affect
objects in another region.
45

Advantage: Partitioning is transparent
to the players. No artificial constraints in
game design.
46

Advantage: We can now resize the
regions to load-balance among the servers,
improving server utilization.
47

A client connects to the server serving its current
region, and is handed-off to another server when it
moves to another region.
48

Server communicates with each other to transfer
states, update states etc.
49

Updates from a client are sent to its server, which then
forwards it to all relevant clients in the same region or
neighboring regions.
50

or
51

A client connects to the servers serving its current
region and regions it subscribes to. Updates from
neighboring server are sent directly.
52

Q: How to partition
the game world?
53

Ideally: each server
should handle the same
number of players
54

Ideally: minimize
communications
between servers
55

Ideally: minimize the
number of hand offs
56

Idea: divide the game
world into small cells
and assign group of
cells to servers.
57

58

I will simplify the representation to a grid of cells with
number indicating the number of players (or load
incurred on server) in this cell.
2 0 3
3 1 4
1 1 0
59

Centralized approach
to partitioning
60

Balanced Deployment
Algorithm
by Bart De Vleeschauwer et al
61

Idea: sort the cells and assign
them one-by-one, highest load
first, to the servers to minimize
the maximum server load
62

Suppose we have only two servers (blue and red).
2 0 3
3 1 4
1 1 0
63

Blue’s load = 4 Red’s load = 0
2 0 3
3 1 4
1 1 0
64

Blue’s load = 4 Red’s load = 3
2 0 3
3 1 4
1 1 0
65

Blue’s load = 4 Red’s load = 6
2 0 3
3 1 4
1 1 0
66

Blue’s load = 6 Red’s load = 6
2 0 3
3 1 4
1 1 0
67

Blue’s load = 8 Red’s load = 7
2 0 3
3 1 4
1 1 0
68

Problem: neighboring cells are not adjacent. There
are 8 “borders” in this solution. Communication
overhead is not considered.
2 0 3
3 1 4
1 1 0
69

Let’s define a cost function for
assigning a cell c to a server S:
cost(c,S) =
computation load increase in S +
communication load increase in S
70

For simplicity,
CPU load = players,
newtork load = borders
71

Clustered Deployment
Algorithm
by Bart De Vleeschauwer et al
72

Start with each cell as one
cluster
73

Repeatedly merge two adjacent
clusters with least overhead until
number of clusters equals to
number of servers.
74

2 0 3
3 1 4
1 1 0
75

Least cost = 1 + 3 = 4
2 0 3
3 1 4
1 1 0
76

Least cost = 2 + 3 = 5
2 0 3
3 1 4
1 1 0
77

Least cost = 1 + 5 = 6
2 0 3
3 1 4
1 1 0
78

Least cost = 3 + 5 = 8
2 0 3
3 1 4
1 1 0
79

2 0 3
3 1 4
1 1 0
80

Just a heuristic
the optimization problem
is NP-complete
81

Last few steps involve heavily
loaded cells and can create
uneven deployment.
82

We can tweak this heuristic in
many ways to improve it, but it’s
still a heuristic.
83

Simulated Annealing
by Bart De Vleeschauwer et al
84

A general method for finding
good solution in an optimization
problem (meta-heuristic)
85

At every step, move towards a nearby,
better configuration probabilistically
(allowing a move towards worse
solution, prevent stuck at local
minimum)
86

cost
config
87

Start with a solution from a
heuristic.
88

Move to a nearby config:
swap cells with an adjacent
server, or move cell from one
server to another
89

Is it a better solution?
If so, keep it as current solution
with some probability.
90

2 0 3
3 1 4
1 1 0
91

Experimental results show that
Simulated Annealing works best
among the methods.
92

Disadvantages of
centralized approach
93

Expensive to compute a
good solution
94

How frequent should
the world be
repartitioned?
95

Localized World
Partitioning
96

Idea: Only repartition locally if a
server is overloaded. Look for a
lightly loaded server to shed
some load to.
97

overload
safe
light
98

Overloaded server sheds load until it’s load is below a
safe level. The new load of a previously lightly loaded
should remain below safe level.
overload
safe
light
99

l+o-s<s
l < 2s - o
100

Who to shed to?
101

Minimize overhead: try to shed
to as few neighboring server as
possible
102

Migrate cells to neighboring
servers that are not overloaded
103

Overload = 10, Safe = 7, Light = 4
0 1 1
2 0 3
1 1 4
5 3 0
104

Green server is overloaded.
Shed some load to neighbors.
0 1 1
2 0 3
1 1 4
5 3 0
105

If still not enough, look at the list of
lightly loaded servers maintained.
Pick the least load and shed to it.
106

Green server is still overloaded.
Neighbor (purple) server has hit the safe threshold.
0 1 1
2 0 3
1 1 4
5 3 0
107

Shed load to remote server.
But this could lead to high communication cost.
0 1 1
2 0 3
1 1 4
5 3 0
108

When communication overhead is too high,
isolated cells can be migrated to a nearby
server, “merging” with neighboring regions to
form contiguous regions, as long as the load
remains below the safe threshold.
0 1 1
2 0 3
1 1 0
1 3 0
109

Another Open Problem
110

No consideration on hand offs
111

Idea: If traffic is high between
point A and point B, then A and B
should be managed by the same
server.
112

3 2
2 1 1
0 1
2 4 2
0 2
1 1 2
2 4
113

Minimizing hand-off means finding minimum cuts.
3 2
2 1 1
0 1
2 4 2
0 2
1 1 2
2 4
114

Minimizing hand-off means finding minimum cuts.
3 2
2
1
2 4 2
2
1 2
2 4
115

Minimizing hand-off means finding minimum cuts.
3 2
2
1
2 4 2
2
2 4
116

Question: how to combine the
three objectives:
1. limiting number of players
2. reduce communication cost
3. reduce number of hand offs
117