Hybrid Architecture
for Networked Games


                        1
single
               completely
centralized
              decentralized
  server




                              2
P2P with
Central Arbiter


                  3
A trusted central arbiter receives all game
traffic and resolve conflicts if necessary.




                                ...
Benefits of
P2P + Arbiter


                5
low latency among players




                            6
have trusted authority now




                             7
Drawbacks of
P2P + Arbiter


                8
limited scalability

arbiter becomes bottleneck




                             9
Mirrored Servers
  Architecture


                   10
Multiple servers, each replicating the complete game
states, serve clients from different geographical
regions. Each clien...
Servers may be connected with high speed,
provisioned networked, reducing synchronization
latency among the servers.




 ...
Updates from a client are sent to its server, which then
forwards it to other servers. The servers then forward
the update...
P2P-style state synchronization
among servers is easier, since

1. servers can be trusted
2. clocks can be synchronized
3....
Benefits of
Mirrored Servers


                   15
no single point of failures
A client can use another replicated server
        if their nearest server fails.




        ...
increase total network capacity
  good if networking bandwidth is the
              bottleneck




                       ...
lower latency between
   client and server




                        18
Drawbacks of
Mirrored Servers


                   19
latency between players increases
      due to additional forwarding




                                     20
cannot scale to large world
each server still keeps the complete states




                                              ...
how to assign
players to servers?


                      22
Assigning players to closest servers may lead to
overloaded servers.




                                                 ...
Assigning players to remote servers may increase
end-to-end latency.




                                                 ...
But we only care about latency between players
that are relevant to each other.




                                      ...
An Open Problem



                  26
let

AOI(p) be the set of players relevant to p
S(p) be the server of player p
d(i,j) be the delay between i and j
P(s) be...
let

L(p) be the average latency experience by p,
considering only relevant players




                                  ...
want to find S() such that

	 maximum L(p) is minimized over all p

subjected to

	 maximum P(s) is constrained for all s

...
Zoned Servers
 Architecture


                30
Divide the game world into independent regions.
     One server is in-charged of one region.




                         ...
A client connects to the server serving its current
region, and is handed-off to another server when it
moves to another r...
Servers seldom communicate with each other.




                                              33
Updates from a client are sent to its server, which then
forwards it to all relevant clients in the same region.




     ...
35
Advantages of
Zoned Servers


                36
Can scale to large world
   Just add more servers




                           37
Drawbacks of
Zoned Servers


                38
Single point of failures
(But can deploy mirrors for each region)




                                           39
Constrained Game Designs




                           40
Supporting Seamless
   Game Worlds


                      41
Divide the game world into regions.
One server is in-charged of one region.




                                          ...
Player may move around the world seamlessly




                                              43
A client may see events and objects
     from neighboring regions.




                                      44
Events occurring in one region may affect
       objects in another region.




                                          ...
Advantage: Partitioning is transparent
to the players. No artificial constraints in
game design.




                      ...
Advantage: We can now resize the
regions to load-balance among the servers,
improving server utilization.




            ...
A client connects to the server serving its current
region, and is handed-off to another server when it
moves to another r...
Server communicates with each other to transfer
states, update states etc.




                                           ...
Updates from a client are sent to its server, which then
forwards it to all relevant clients in the same region or
neighbo...
or



     51
A client connects to the servers serving its current
region and regions it subscribes to. Updates from
neighboring server ...
Q: How to partition
 the game world?


                      53
Ideally: each server
should handle the same
   number of players


                         54
Ideally: minimize
 communications
between servers


                    55
Ideally: minimize the
 number of hand offs



                        56
Idea: divide the game
world into small cells
 and assign group of
   cells to servers.


                         57
58
I will simplify the representation to a grid of cells with
number indicating the number of players (or load
incurred on se...
Centralized approach
  to partitioning


                       60
Balanced Deployment
      Algorithm
    by Bart De Vleeschauwer et al




                                    61
Idea: sort the cells and assign
 them one-by-one, highest load
first, to the servers to minimize
   the maximum server load...
Suppose we have only two servers (blue and red).



                      2   0   3
                      3   1   4
      ...
Blue’s load = 4       Red’s load = 0



           2      0    3
           3      1    4
           1      1    0




   ...
Blue’s load = 4       Red’s load = 3



           2      0    3
           3      1    4
           1      1    0




   ...
Blue’s load = 4       Red’s load = 6



           2      0    3
           3      1    4
           1      1    0




   ...
Blue’s load = 6       Red’s load = 6



           2      0    3
           3      1    4
           1      1    0




   ...
Blue’s load = 8       Red’s load = 7



           2      0    3
           3      1    4
           1      1    0




   ...
Problem: neighboring cells are not adjacent. There
 are 8 “borders” in this solution. Communication
           overhead is...
Let’s define a cost function for
assigning a cell c to a server S:
          cost(c,S) =
computation load increase in S +
c...
For simplicity,
  CPU load = players,
newtork load = borders




                         71
Clustered Deployment
      Algorithm
    by Bart De Vleeschauwer et al




                                    72
Start with each cell as one
          cluster



                              73
Repeatedly merge two adjacent
clusters with least overhead until
  number of clusters equals to
       number of servers.
...
2   0   3
3   1   4
1   1   0




            75
Least cost = 1 + 3 = 4



       2   0   3
       3   1   4
       1   1   0




                         76
Least cost = 2 + 3 = 5



       2   0   3
       3   1   4
       1   1   0




                         77
Least cost = 1 + 5 = 6



       2   0   3
       3   1   4
       1   1   0




                         78
Least cost = 3 + 5 = 8



       2   0   3
       3   1   4
       1   1   0




                         79
2   0   3
3   1   4
1   1   0




            80
Just a heuristic

the optimization problem
     is NP-complete


                           81
Last few steps involve heavily
 loaded cells and can create
     uneven deployment.



                                 82
We can tweak this heuristic in
many ways to improve it, but it’s
       still a heuristic.



                            ...
Simulated Annealing
   by Bart De Vleeschauwer et al




                                   84
A general method for finding
good solution in an optimization
   problem (meta-heuristic)



                              ...
At every step, move towards a nearby,
better configuration probabilistically
(allowing a move towards worse
solution, preve...
cost




       config



               87
Start with a solution from a
         heuristic.



                               88
Move to a nearby config:
 swap cells with an adjacent
server, or move cell from one
      server to another


             ...
Is it a better solution?
If so, keep it as current solution
      with some probability.



                              ...
2   0   3
3   1   4
1   1   0




            91
Experimental results show that
Simulated Annealing works best
     among the methods.



                                 ...
Disadvantages of
centralized approach


                       93
Expensive to compute a
    good solution


                         94
How frequent should
   the world be
  repartitioned?


                      95
Localized World
  Partitioning



                  96
Idea: Only repartition locally if a
 server is overloaded. Look for a
   lightly loaded server to shed
            some lo...
overload
safe


light




           98
Overloaded server sheds load until it’s load is below a
safe level. The new load of a previously lightly loaded
should rem...
l+o-s<s
 l < 2s - o



              100
Who to shed to?




                  101
Minimize overhead: try to shed
to as few neighboring server as
           possible



                                  102
Migrate cells to neighboring
servers that are not overloaded



                                  103
Overload = 10, Safe = 7, Light = 4



             0   1   1
             2   0   3
             1   1   4
             5 ...
Green server is overloaded.
Shed some load to neighbors.


          0   1   1
          2   0   3
          1   1   4
   ...
If still not enough, look at the list of
lightly loaded servers maintained.
Pick the least load and shed to it.



       ...
Green server is still overloaded.
Neighbor (purple) server has hit the safe threshold.


                      0   1   1
 ...
Shed load to remote server.
But this could lead to high communication cost.


                   0   1   1
               ...
When communication overhead is too high,
isolated cells can be migrated to a nearby
server, “merging” with neighboring reg...
Another Open Problem



                       110
No consideration on hand offs




                                111
Idea: If traffic is high between
point A and point B, then A and B
 should be managed by the same
             server.



 ...
3       2

2       1       1
    0       1

2       4       2
    0       2

1       1       2
    2       4




         ...
Minimizing hand-off means finding minimum cuts.

                    3       2

                2       1       1
         ...
Minimizing hand-off means finding minimum cuts.

                    3       2

                2
                         ...
Minimizing hand-off means finding minimum cuts.

                    3       2

                2
                         ...
Question: how to combine the
         three objectives:
  1. limiting number of players
 2. reduce communication cost
 3. ...
Upcoming SlideShare
Loading in...5
×

CS4344 Lecture 8: Hybrid Architecture

451

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
451
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

CS4344 Lecture 8: Hybrid Architecture

  1. 1. Hybrid Architecture for Networked Games 1
  2. 2. single completely centralized decentralized server 2
  3. 3. P2P with Central Arbiter 3
  4. 4. A trusted central arbiter receives all game traffic and resolve conflicts if necessary. 4
  5. 5. Benefits of P2P + Arbiter 5
  6. 6. low latency among players 6
  7. 7. have trusted authority now 7
  8. 8. Drawbacks of P2P + Arbiter 8
  9. 9. limited scalability arbiter becomes bottleneck 9
  10. 10. Mirrored Servers Architecture 10
  11. 11. Multiple servers, each replicating the complete game states, serve clients from different geographical regions. Each client connects to one server. 11
  12. 12. Servers may be connected with high speed, provisioned networked, reducing synchronization latency among the servers. 12
  13. 13. Updates from a client are sent to its server, which then forwards it to other servers. The servers then forward the updates to all relevant clients. 13
  14. 14. P2P-style state synchronization among servers is easier, since 1. servers can be trusted 2. clocks can be synchronized 3. IP multicast may be used 4. higher availability 5. shorter delay 14
  15. 15. Benefits of Mirrored Servers 15
  16. 16. no single point of failures A client can use another replicated server if their nearest server fails. 16
  17. 17. increase total network capacity good if networking bandwidth is the bottleneck 17
  18. 18. lower latency between client and server 18
  19. 19. Drawbacks of Mirrored Servers 19
  20. 20. latency between players increases due to additional forwarding 20
  21. 21. cannot scale to large world each server still keeps the complete states 21
  22. 22. how to assign players to servers? 22
  23. 23. Assigning players to closest servers may lead to overloaded servers. 23
  24. 24. Assigning players to remote servers may increase end-to-end latency. 24
  25. 25. But we only care about latency between players that are relevant to each other. 25
  26. 26. An Open Problem 26
  27. 27. let AOI(p) be the set of players relevant to p S(p) be the server of player p d(i,j) be the delay between i and j P(s) be the set of players assigned to server s 27
  28. 28. let L(p) be the average latency experience by p, considering only relevant players 28
  29. 29. want to find S() such that maximum L(p) is minimized over all p subjected to maximum P(s) is constrained for all s 29
  30. 30. Zoned Servers Architecture 30
  31. 31. Divide the game world into independent regions. One server is in-charged of one region. 31
  32. 32. A client connects to the server serving its current region, and is handed-off to another server when it moves to another region. 32
  33. 33. Servers seldom communicate with each other. 33
  34. 34. Updates from a client are sent to its server, which then forwards it to all relevant clients in the same region. 34
  35. 35. 35
  36. 36. Advantages of Zoned Servers 36
  37. 37. Can scale to large world Just add more servers 37
  38. 38. Drawbacks of Zoned Servers 38
  39. 39. Single point of failures (But can deploy mirrors for each region) 39
  40. 40. Constrained Game Designs 40
  41. 41. Supporting Seamless Game Worlds 41
  42. 42. Divide the game world into regions. One server is in-charged of one region. 42
  43. 43. Player may move around the world seamlessly 43
  44. 44. A client may see events and objects from neighboring regions. 44
  45. 45. Events occurring in one region may affect objects in another region. 45
  46. 46. Advantage: Partitioning is transparent to the players. No artificial constraints in game design. 46
  47. 47. Advantage: We can now resize the regions to load-balance among the servers, improving server utilization. 47
  48. 48. A client connects to the server serving its current region, and is handed-off to another server when it moves to another region. 48
  49. 49. Server communicates with each other to transfer states, update states etc. 49
  50. 50. Updates from a client are sent to its server, which then forwards it to all relevant clients in the same region or neighboring regions. 50
  51. 51. or 51
  52. 52. A client connects to the servers serving its current region and regions it subscribes to. Updates from neighboring server are sent directly. 52
  53. 53. Q: How to partition the game world? 53
  54. 54. Ideally: each server should handle the same number of players 54
  55. 55. Ideally: minimize communications between servers 55
  56. 56. Ideally: minimize the number of hand offs 56
  57. 57. Idea: divide the game world into small cells and assign group of cells to servers. 57
  58. 58. 58
  59. 59. I will simplify the representation to a grid of cells with number indicating the number of players (or load incurred on server) in this cell. 2 0 3 3 1 4 1 1 0 59
  60. 60. Centralized approach to partitioning 60
  61. 61. Balanced Deployment Algorithm by Bart De Vleeschauwer et al 61
  62. 62. Idea: sort the cells and assign them one-by-one, highest load first, to the servers to minimize the maximum server load 62
  63. 63. Suppose we have only two servers (blue and red). 2 0 3 3 1 4 1 1 0 63
  64. 64. Blue’s load = 4 Red’s load = 0 2 0 3 3 1 4 1 1 0 64
  65. 65. Blue’s load = 4 Red’s load = 3 2 0 3 3 1 4 1 1 0 65
  66. 66. Blue’s load = 4 Red’s load = 6 2 0 3 3 1 4 1 1 0 66
  67. 67. Blue’s load = 6 Red’s load = 6 2 0 3 3 1 4 1 1 0 67
  68. 68. Blue’s load = 8 Red’s load = 7 2 0 3 3 1 4 1 1 0 68
  69. 69. Problem: neighboring cells are not adjacent. There are 8 “borders” in this solution. Communication overhead is not considered. 2 0 3 3 1 4 1 1 0 69
  70. 70. Let’s define a cost function for assigning a cell c to a server S: cost(c,S) = computation load increase in S + communication load increase in S 70
  71. 71. For simplicity, CPU load = players, newtork load = borders 71
  72. 72. Clustered Deployment Algorithm by Bart De Vleeschauwer et al 72
  73. 73. Start with each cell as one cluster 73
  74. 74. Repeatedly merge two adjacent clusters with least overhead until number of clusters equals to number of servers. 74
  75. 75. 2 0 3 3 1 4 1 1 0 75
  76. 76. Least cost = 1 + 3 = 4 2 0 3 3 1 4 1 1 0 76
  77. 77. Least cost = 2 + 3 = 5 2 0 3 3 1 4 1 1 0 77
  78. 78. Least cost = 1 + 5 = 6 2 0 3 3 1 4 1 1 0 78
  79. 79. Least cost = 3 + 5 = 8 2 0 3 3 1 4 1 1 0 79
  80. 80. 2 0 3 3 1 4 1 1 0 80
  81. 81. Just a heuristic the optimization problem is NP-complete 81
  82. 82. Last few steps involve heavily loaded cells and can create uneven deployment. 82
  83. 83. We can tweak this heuristic in many ways to improve it, but it’s still a heuristic. 83
  84. 84. Simulated Annealing by Bart De Vleeschauwer et al 84
  85. 85. A general method for finding good solution in an optimization problem (meta-heuristic) 85
  86. 86. At every step, move towards a nearby, better configuration probabilistically (allowing a move towards worse solution, prevent stuck at local minimum) 86
  87. 87. cost config 87
  88. 88. Start with a solution from a heuristic. 88
  89. 89. Move to a nearby config: swap cells with an adjacent server, or move cell from one server to another 89
  90. 90. Is it a better solution? If so, keep it as current solution with some probability. 90
  91. 91. 2 0 3 3 1 4 1 1 0 91
  92. 92. Experimental results show that Simulated Annealing works best among the methods. 92
  93. 93. Disadvantages of centralized approach 93
  94. 94. Expensive to compute a good solution 94
  95. 95. How frequent should the world be repartitioned? 95
  96. 96. Localized World Partitioning 96
  97. 97. Idea: Only repartition locally if a server is overloaded. Look for a lightly loaded server to shed some load to. 97
  98. 98. overload safe light 98
  99. 99. Overloaded server sheds load until it’s load is below a safe level. The new load of a previously lightly loaded should remain below safe level. overload safe light 99
  100. 100. l+o-s<s l < 2s - o 100
  101. 101. Who to shed to? 101
  102. 102. Minimize overhead: try to shed to as few neighboring server as possible 102
  103. 103. Migrate cells to neighboring servers that are not overloaded 103
  104. 104. Overload = 10, Safe = 7, Light = 4 0 1 1 2 0 3 1 1 4 5 3 0 104
  105. 105. Green server is overloaded. Shed some load to neighbors. 0 1 1 2 0 3 1 1 4 5 3 0 105
  106. 106. If still not enough, look at the list of lightly loaded servers maintained. Pick the least load and shed to it. 106
  107. 107. Green server is still overloaded. Neighbor (purple) server has hit the safe threshold. 0 1 1 2 0 3 1 1 4 5 3 0 107
  108. 108. Shed load to remote server. But this could lead to high communication cost. 0 1 1 2 0 3 1 1 4 5 3 0 108
  109. 109. When communication overhead is too high, isolated cells can be migrated to a nearby server, “merging” with neighboring regions to form contiguous regions, as long as the load remains below the safe threshold. 0 1 1 2 0 3 1 1 0 1 3 0 109
  110. 110. Another Open Problem 110
  111. 111. No consideration on hand offs 111
  112. 112. Idea: If traffic is high between point A and point B, then A and B should be managed by the same server. 112
  113. 113. 3 2 2 1 1 0 1 2 4 2 0 2 1 1 2 2 4 113
  114. 114. Minimizing hand-off means finding minimum cuts. 3 2 2 1 1 0 1 2 4 2 0 2 1 1 2 2 4 114
  115. 115. Minimizing hand-off means finding minimum cuts. 3 2 2 1 2 4 2 2 1 2 2 4 115
  116. 116. Minimizing hand-off means finding minimum cuts. 3 2 2 1 2 4 2 2 2 4 116
  117. 117. Question: how to combine the three objectives: 1. limiting number of players 2. reduce communication cost 3. reduce number of hand offs 117
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×