Downloaded 11 times
Uploaded byJames Munro
Real-Time Load Balancing of an Interactive Mutliplayer Game Server
This document summarizes research on implementing real-time load balancing in the game QuakeWorld. The researchers tested four load balancing algorithms (LPTF, SPTF, RR, SRR) across multiple servers and clients. Their initial results showed that LPTF provided the best workload distribution and server throughput, while reducing intra-frame wait times compared to the other algorithms. The researchers concluded that effective load balancing requires optimizing multiple performance metrics to balance workloads and minimize waiting.
More Related Content
Similar to Real-Time Load Balancing of an Interactive Mutliplayer Game Server
Real-Time Load Balancing of an Interactive Mutliplayer Game Server
- 1. QUAKEWORLD: REAL-TIME LOADBALANCING James Munro & Dr. Patrick Dickinson University of Lincoln EUROSIS GAME-ON 2010 17th – 19th November
- 2. INTRODUCTION Parallel/multi-threaded programming: Emerging trend of game development. Crucial to leverage power of modern hardware. Can be difficult: we’re still learning! Two approaches in research: Client-side (directly effects end-user). Server-side (indirectly effects end-user).
- 3. AN EXISTING SYSTEM Multi-threaded QuakeWorld: Exploits COW semantics of Linux kernel. Entity pre-processing allows lockless execution. Dynamic load balancing strategy. Improved performance over previous work. We used this as the starting point for our work.
- 4. PARALLEL QUAKEWORLD FRAME We’remostly interested in this step
- 5. EXPERIMENTAL SETUP Sevenquad-core PCs. One server (32-bit Linux). Six clients (32-bit Windows). Using headless automated player “bot”. Up to 32 bots per-machine. Four load balancing algorithms: Longest Processing Time First (LPTF) – good performance. Shortest Processing Time First (SPTF) – worse performance. Round-Robin (RR) - unsure. Sorted Round-Robin (SRR) - unsure.
- 6. METRICS / INITIALRESULTS We use the following set of metrics: FPS. Workload distribution. Server throughput. Intra-frame wait time. Performed an initial benchmark: Serial, 2, 3, 4 threads. Discovered that 4 threads is best. Cordeiro et al needed to consider further metrics. Had to artificially limit performance due to bandwidth. Uncapped, could support 400-600 players.
- 7. WORKLOAD DISTRIBUTION (LPTF) 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 12 3 4 AccumulatedWorkgroupWeight Thread ID
- 8. WORKLOAD DISTRIBUTION (SPTF) 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 12 3 4 AccumulatedWorkgroupWeight Thread ID
- 9. WORKLOAD DISTRIBUTION (RR) 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 12 3 4 AccumulatedWorkgroupWeight Thread ID
- 10. WORKLOAD DISTRIBUTION (SRR) 0 200,000 400,000 600,000 800,000 1,000,000 1,200,000 1,400,000 12 3 4 AccumulatedWorkgroupWeight Thread ID
- 11. SERVER THROUGHPUT –ALL ALGORITHMS 2,750 3,250 3,750 4,250 4,750 5,250 96 112 128 144 160 176 Throughput(RPP/s) Number of Clients LPTF SPTF RR SRR
- 12. IFWT– ALL ALGORITHMS 0 0.5 1 1.5 2 2.5 3 3.5 4 96112 128 144 160 176 WaitTime(ms) Number of Clients LPTF SPTF RR SRR
- 13. STANDARD DEVIATION (LPTF,SPTF, SRR) 0 2 4 6 8 10 12 14 1 2 3 4 Std.DevofProcessedEntity Groups Thread ID LPTF SPTF SRR
- 14. CONCLUSIONS Concurrency willbe crucial in game development. A full set of metrics is required to quantify performance. Best load balance is a trade-off: Consistently producing a decent distribution. Reducing IFWT, less time spent waiting.
Editor's Notes
- #2 Welcome and introduction: Good afternoon/morning. My name. University of Lincoln. I did this project for my MSc qualification. Motivation for undertaking the work: I’m really interested in parallel programming. With multi-core CPUs being so popular, there’s never been a better time. There’s a fairly small body of existing research specific to game development.
- #3 Parallel programming is not a new area of computer science. However, it is new in the context of video game development. Due to the current adoption of multi-core technology, its essential to improve game performance. PlayStation 3 is multi-core. XBOX 360 is multi-core. Your home PC and laptop is quite likely multi-core. GPGPU and CUDA – graphics cards for general purpose computing. There are two main approaches in academia: Client-side usually involves enhancing the complexity of: Artificial intelligence. Simulation. On-screen graphics. Server-side targets the multiplayer component of games: Usually to increase maximum player capacity. This is what we’re interested in.
- #4 Abdelkhalek et al perform an investigation into the performance of interactive multiplayer game servers. They identify that the performance characteristics are similar to web servers. When the server is overloaded with players, the CPU usage is 100% and performance starts to degrade. There are potential gains to be found by leveraging parallel execution. The same group continued the investigation and developed a parallel version of the QuakeWorld server. This was a traditional multi-threaded approach using the pthreads library. Once of the biggest difficulties was synchronising access to shared data structures: Threads can all read/write the same data. Undefined results if 2 threads write to the same data simultaneously Mutexes are used to access shared data in a serial fashion. This reduces the time spent executing concurrently: can drastically reduce performance. Static load balancing: Players are assigned to threads based on their position within the game world. What happens if all the players go into the same corner of a level? We’re no longer using the available threads effectively.
- #5 The same group continued the investigation and developed a parallel version of the QuakeWorld server. This was a traditional multi-threaded approach using the pthreads library. Once of the biggest difficulties was synchronising access to shared data structures: Threads can all read/write the same data. Undefined results if 2 threads write to the same data simultaneously Mutexes are used to access shared data in a serial fashion. This reduces the time spent executing concurrently: can drastically reduce performance. Static load balancing: Players are assigned to threads based on their position within the game world. What happens if all the players go into the same corner of a level? We’re no longer using the available threads effectively.
- #6 This approach tries to tackle the complexities experienced in Abdelkhalek’s version. Using a unique Linux kernel feature, it simplifies the sychronisation of shared data: We selectively choose data that we wish to keep. Child threads do not modify the global data, unless we let them (specifically). Still has quite high overhead, for different reasons. Performs pre-processing to identify groups of entities which can be processed in parallel. Attempts dynamic load balancing to ensure optimum performance. We’re mostly interested in this. To what extend does load balancing effect performance?
- #7 This is a fairly simple game loop. Client input is received via network: Clients send a request to move or perform an action. Prediction is used in the client program to provide uninterrupted gameplay. Client requests are processed: Request may have been illegal (i.e. moving too fast). Otherwise perform movement/action. Create necessary entities such as projectiles, remove pick-ups etc. Response packets are then sent via the network connection. IMPORTANT: time between request/response MUST be quick! Otherwise the players will experience “lag”, gameplay will degrade. Players are suceptible to just a few tens of Ms.
- #9 This is a fairly simple game loop. Client input is received via network: Clients send a request to move or perform an action. Prediction is used in the client program to provide uninterrupted gameplay. Client requests are processed: Request may have been illegal (i.e. moving too fast). Otherwise perform movement/action. Create necessary entities such as projectiles, remove pick-ups etc. Response packets are then sent via the network connection. IMPORTANT: time between request/response MUST be quick! Otherwise the players will experience “lag”, gameplay will degrade. Players are suceptible to just a few tens of Ms. We’ve added more complexity to the game loop. Entity pre-processing is a very important step: Identifies groups of clients that can be processed in parallel. Creates groups based on a range of 256 game units. Based on their potential movement/actions, they cannot affect entities outside their groups. See video (if there’s time). Load balancing distributes groups between threads based on algorithm (LPTF by default). To what extent does this effect server performance?
- #10 We’re using the Linux version of the QuakeWorld server. A Windows version is available: But this isn’t compatible with COW. We call it a “bot” in the loosest definition. It performs random inputs every 10 frames. Not representative of human behaviour, but enough to simulate/benchmark the server.
- #11 We’re interested in whether the initial weight penalty is justified. Their justification is that the main thread must perform some other work in addition to entity processing.
- #15 This is Cordeiro’s version of LPTF with the imbalance on Thread 1. First few iterations are identical.
- #19 This is the first step where the results are different. With 8 threads there are more spaces to distribute groups.
- #27 Which one of these is best? Technically 4 threads provides the best balance. The cost of using 8 threads may overshadow the benefit.
- #28 FPS is just a general measure of performance. Workload distribution helps us visualize the profile of the load balancing algorithm. Server throughput is perhaps the most important: Shows us the server’s capacity and ability to respond to incoming requests. Responses-per-seconds increases linearly with the number of players/clients. Dips when it has become saturated. Intra-frame wait time measures how long the main thread waits for the supplementary threads to complete: It cannot continue until they have done so. Any time spent waiting is wasted performance.
- #29 A pretty even distribution across all threads. Notice the imbalance on Thread 2, caused by initial weight penalty. We wanted to determine whether this was justified.
- #30 A pretty even distribution across all threads. Notice the imbalance on Thread 2, caused by initial weight penalty. We wanted to determine whether this was justified.
- #31 This algorithm appears to produce a better balance than LPTF. We didn’t expect this: an indication that a better metric may be required. Does not correlate with the server throughput results (in a few slides time).
- #32 * Produces a fairly even distribution, but apparently not as good as the previous algorithms.
- #33 * Pretty large imbalance on the first thread, but what effect will this have on overall performance?
- #34 Server FPS does not really tell us much about the overall performance of the server. Cordeiro bases his conclusions that 3 threads is better on this.
- #35 The number of response packets sent per-second, goes up linearly with player count. Point out the linear increase, until the server peaks. Its worth noting that LPTF and SRR exhibit very similar performance. This was confusing, because they have very different workload distributions. Intra-frame wait time needs to be considered.
- #36 Interestingly, wait time for SRR decreases instead of increasing! LPTF does spend time waiting (doing nothing) and so the imbalance is not required. SRR performs well because we’re reducing the time spent doing nothing. Performance likely to degrade when wait reaches zero. Advantage will be lost.
- #37 We can see that the deviation is high across all threads for SPTF. This demonstrates that the accumulated distribution was not a good measure of algorithm performance. This also backs up the results we saw in in the server throughput graph.
- #39 SRR did not perform as well on this machine. 6 threads *does* perform better, but the improvement is small at best.
- #40 Due to the current hardware trend, there’s essentially no way for game developers to avoid the need for parallel programming. A full set of metrics is required, Cordeiro’s work did not take into account the bigger picture. The imbalance on the main thread is not required: Further work could investigate this. In the context of the QuakeWorld server, the best load balancing strategy is a trade off between: Consistently producing the same distribution on a frame-by-frame basis. The main thread must spend as little time waiting as possible; else we’re wasting performance! Thanks: questions?