From Simulation to Online Gaming: the need for adaptive solutions


Published on

In many fields such as distributed simulation and online gaming the missing piece is adaptivity. There is a strong need for dynamic and adaptive solutions that can improve performances and react to problems.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

From Simulation to Online Gaming: the need for adaptive solutions

  1. 1. Gabriele D’Angelo <> Department of Computer Science University of Bologna From Simulation to Online Gaming: the need for adaptive solutions
  2. 2. Presentation outline <ul><li>Simulation
  3. 3. Parallel and Distributed Simulation (PADS)
  4. 4. Adaptive migration-based approach
  5. 5. Simulation of scale-free networks
  6. 6. Gossip protocols
  7. 7. Scale-free networks and gossip protocols for online gaming
  8. 8. Conclusions and future work </li></ul>
  9. 9. Simulation and much more <ul><li>“ A simulation is a system that represents or emulates the behavior of another system over time. In a computer simulation the system doing the emulating is a computer program ” (Richard M. Fujimoto)
  10. 10. The simulation is a widely used technique in the performance evaluation, risk assessment and security analysis of physical systems (or during the design phase)
  11. 11. It is fundamental for the creation of Digital Virtual Environments ( DVE s) and Online Games </li></ul>
  12. 12. Limitations of the monolithic approach <ul><li>Many of the systems of interest are composed by a very large number of entities , with a really dynamic nature and interaction pattern
  13. 13. For example, w ireless networks, under the simulation viewpoint, have very strict requirements in terms of level of detail
  14. 14. Monolithic simulators are unable to fulfill these scalability requirements due to memory constrains and excessive amount of time required to obtain the results
  15. 15. A Parallel And Distributed Simulation ( PADS ) can be used to aggregate memory and computational resources </li></ul>
  16. 16. P arallel A nd D istributed S imulation ( PADS ) <ul><li>In this case, the state of the simulation and its evolution is shared among a set Logical Processes ( LP s)
  17. 17. Each LP is responsible to manage only a subset of Simulated Entities and is allocated on a different Physical Execution Unit ( PEU )
  18. 18. The PEU s are interconnected together using tightly coupled (parallel simulation) loosely coupled (distributed simulation) communication networks
  19. 19. To obtain correct results it is necessary that the causal order of events is respected, two main approaches have been proposed: </li><ul><li>Conservative (Chandy-Misra-Bryant, CMB)
  20. 20. Optimistc (Time-Warp) </li></ul></ul>
  21. 21. The main drawbacks of PADS <ul><li>The PADS approach can aggregate many memory and computational resources: it has potential for very good scalability
  22. 22. This approach is not free from drawbacks , the distributed execution architecture needs appropriate mechanisms for: </li><ul><li>Synchronization (and more in general, communication )
  23. 23. Load-balancing </li></ul><li>Many approaches and techniques have been proposed to cope with these problems: in particular to reduce the communication overhead and to build-up efficient synchronization schemes </li></ul>
  24. 24. Partitioning problem <ul><li>A simulation can be seen as a set of Simulation Entities ( SE ) that interact together
  25. 25. In our vision, the reduction of the communication overhead and the computational load-balancing problems have to be considered at the same time
  26. 26. In practice, it is an instance of a dynamic partitioning problem in which there are many SE s, a (variable size) set of heterogeneous PEU s (with the corresponding LP s) and a communication network (with performance varying in time) </li></ul>
  27. 27. Migration -based approach <ul><li>All the static approaches are suboptimal given the variability in the load generated by the simulated mode, the execution platform and the communication network
  28. 28. It is necessary to employ a dynamic and adaptive approach that reacts to imbalances in the architecture and proactively enhances the model partitioning
  29. 29. We propose a migration-based approach : each entity in the simulation can be dynamically relocated (migrated), moving from a source PEU to a new destination PEU </li></ul>
  30. 30. Migration -based approach <ul><li>Clustering in the same LP the highly interacting entities it is possible to reduce the costly inter -LP communication and conversely increasing the rate of low cost intra -LP communication
  31. 31. This can be done considering the communication pattern of each model entity that is in the simulation
  32. 32. Given the unpredictable nature of parts the simulated model and of many parameters in the execution architecture it is not feasible to compute the optimal solution at runtime: </li><ul><li>we have to rely on heuristics (simple, fast and scalable) </li></ul></ul>
  33. 33. An example: communication overhead reduction <ul>A </ul>B <ul>network delay </ul><ul>Physical Execution Units </ul>A set of Simulated Entities allocated on PEUs A and B The entity x (PEU A ) produces a “ transmission-event” that involves some other Simulated Entities <ul>A </ul>B <ul>network delay </ul><ul>X </ul>X <ul><li>X </li></ul>The Simulated Entity x is Migrated from Host A to B with the aim to reduce the communication overhead <ul>Physical Execution Units </ul>The most of the interactions experience a (quite large) network delay Clustering the entities it is possible to increase the amount of memory-to-memory interactions
  34. 34. Migration-based approach: load balancing <ul><li>This approach can be also used to improve the computation load-balancing in the execution architecture
  35. 35. The synchronization points in the distributed architecture are exploited to tag each LP as “ fast ” or “ slow ”
  36. 36. A Logical Process is “slow” if: </li><ul><li>its PEU (i.e. CPU) is overloaded
  37. 37. its communication network has a higher delay with respect to other parts of the execution architecture </li></ul><li>In both cases, to speed up the simulation, the solution is to migrate some entities from the slow LPs to faster LPs, therefore acting to reduce imbalances </li></ul>
  38. 38. Migration-based approach: load balancing <ul><li>This description of the mechanism is very high level, the “real world” implementation has to take in account many subtle details such as: </li><ul><li>each migration has a cost that depends on many factors (e.g. the state size of entities, the network bandwidth and latency)
  39. 39. the algorithm has to be “ fully distributed ” (without any centralization point) and therefore has to take decisions without any “global vision” of the distributed system
  40. 40. the mechanism has to react very quickly to internal ( i.e. the creation of new simulated entities ) and external events ( i.e. a burst of CPU or network load ) but without introducing oscillatory behaviors </li></ul></ul>
  41. 41. Positive side-effects <ul><li>This approach can control the communication overhead and can be used to enhance the partitioning of simulated entities </li><ul><li>the overhead introduced by the mechanism can be controlled very easily: tuning the rate of migrations
  42. 42. it adaptively reacts to the behavior of the simulated model, such as hotspots in the simulated area
  43. 43. the execution platform can be very heterogeneous and shared with other tasks . The system is able to adaptively reconfigure at runtime looking for the better allocation pattern (this is very well suited for cloud computing and “on demand resources”) </li></ul></ul>
  44. 44. ARTÌS + GAIA As part of our research effort we have implemented a PADS middleware called ARTÌS in which the GAIA module implements the migration mechanism previously described
  45. 45. Wireless network simulation <ul><li>The migration-based approach works very well for the simulation of wireless networks
  46. 46. This is due to the nature of the wireless propagation and the locality of most wireless transmissions
  47. 47. In the last years we have tested the ARTÌS middleware in many case studies and simulating wireless network with different degrees of detail in the simulated model
  48. 48. In all cases the results have confirmed that it is possible to obtain a very good speed-up also using simple and general heuristics
  49. 49. In this way, massively populated networks (e.g. 1 million of nodes) using detailed MAC protocols (e.g. IEEE 802.11) can be simulated using Commodity-Off-The-Shelf (COTS) hardware </li></ul>
  50. 50. <ul>A more challenging scenario : scale-Free networks </ul><ul><li>A graph can be used to represent a network and its connectivity </li></ul><ul><li>Degree of a node = number of neighbor nodes attached to them </li></ul><ul><li>A scale-free network is a network whose degree distribution follows a power law </li></ul><ul><li>If p k is the probability that a node has a degree equal to k then: </li></ul><ul><ul><ul><li>p k ~ k - α , for some constant value α
  51. 51. ( usually: 2 < α < 3 ) </li></ul></ul></ul>
  52. 52. <ul>Scale-Free networks : meaning and examples </ul><ul><li>Quite good to model several types of real networks: </li></ul><ul><ul><li>computer networks (e.g. Web, Internet (?))
  53. 53. evolving networks in biology
  54. 54. transmission of diseases
  55. 55. citation graphs
  56. 56. social networks, etc. </li></ul></ul><ul><li>This means: </li></ul><ul><ul><li>a few highly connected nodes, called hubs
  57. 57. a very large number of poorly connected nodes
  58. 58. a very small network diameter </li></ul></ul>
  59. 59. <ul>Simulation of scale-free networks </ul><ul><li>Real networks are usually composed of a huge number of nodes
  60. 60. Under the simulation point of view : </li></ul><ul><ul><li>large amount of memory used to represent the massively populated models
  61. 61. huge amount of communication when simulating real-world protocols on top of such models
  62. 62. nodes in real-world networks are very heterogeneous (i.e. hubs leaf nodes, different hardware and software characteristics) </li></ul></ul><ul><li>Lack of powerful tools to simulate such complex networks </li></ul>
  63. 63. <ul>Gossiping protocols : fixed probability </ul><ul><li>Gossiping protocols have been implemented on top of the simulated scale-free networks, to build realistic case-studies </li></ul><ul><li>Gossip #1: Fixed Probability </li></ul><ul>For each message, the node randomly selects those edges through which the message must be propagated ( v = threshold value ) </ul><ul>Algorithm function INITIALIZATION() v ← CHOOSE_PROBABILITY() function GOSSIP(msg) for all n j in Π j do if RANDOM() < v then SEND(msg, n j ) end if end for </ul>
  64. 64. <ul>Gossiping protocols : fixed fanout </ul><ul><li>Gossip #2: Fixed Fanout </li></ul><ul>Each message is sent to a fixed number of nodes ( fanout ), the receivers are selected at random among the neighbors </ul><ul>Algorithm function INITIALIZATION() fanout ← RETRIEVE_FANOUT() function GOSSIP(msg) if fanout ≥ | Π j | then toSend ← Π j else SELECT_NODES() end if for all n j in toSend do SEND(msg, n j ) end for </ul>
  65. 65. <ul>Gossiping protocols : probabilistic broadcast </ul><ul><li>Gossip #3: Probabilistic Broadcast </li></ul><ul>If the message is locally generated then it is always broadcasted to all neighbors, otherwise it is randomly decided if it has to be broadcasted or ignored </ul><ul><ul><li>( p b = probability of dissemination ) </li></ul></ul><ul>Algorithm function INITIALIZATION() p b ← PROBABILITY_BROADCAST() function GOSSIP(msg) if (RANDOM() < p b or FIRST_TRANSMISSION()) then for all n j in Π j do SEND(msg, n j ) end for end if </ul>
  66. 66. <ul>PaScaS : the scale-free network simulator </ul><ul><li>Parallel and distributed Scale-free network Simulator (PaScaS) </li></ul><ul><ul><li>Implements the building of the scale-free networks (based on the preferential attachment algorithm ), the gossiping protocols and some data analysis utilities
  67. 67. Based on the Advanced RTI System ( ARTÌS ), a middleware used to implement sequential/parallel/distributed simulations that follows an event-based approach
  68. 68. It can exploit the adaptive simulation features provided by the Generic Adaptive Interaction Architecture ( GAIA )
  69. 69. PaScaS will be soon freely available: </li></ul></ul>
  70. 70. <ul>Parameter </ul><ul>Value </ul><ul>number of nodes </ul><ul>3000, 6000, 9000, 12000 </ul><ul>message generation </ul><ul>exponential distribution mean = 50 time-steps </ul><ul>cache size (local to each node) </ul><ul>10 slots </ul><ul>message Time To Live (TTL) </ul><ul>6 (fixed prob. and fanout) 4 (conditional broadcast) </ul><ul>probability of dissemination (v) </ul><ul>0.5 (i.e. 50%) </ul><ul>fanout value </ul><ul>5 </ul><ul>probability of broadcast ( p b ) </ul><ul>0.5 (i.e. 50%) </ul><ul>simulated time </ul><ul>1000 time-steps (after building) </ul><ul>Experimental evaluation : model parameters </ul><ul>Model parameters and simulation scenario </ul>
  71. 71. <ul>Experimental evaluation : execution architecture </ul><ul><li>Hardware : </li></ul><ul><ul><li>Dual processor unit, each processor equipped with a </li></ul></ul><ul><ul><ul><ul><li>single-core Intel Xeon “Gallatin” CPU 2.80 GHz Hyper-Threading, 2 GB RAM </li></ul></ul></ul></ul><ul>Software : </ul><ul><ul><li>Debian GNU/Linux, Kernel
  72. 72. PaScaS 0.9, ARTÌS 1.9.3 </li></ul></ul><ul><li>Note : in case of monolithic (sequential) execution, PaScaS is forced to use only a single processor (also in presence of multiprocessors or multi-cores) </li></ul>
  73. 73. <ul>Experimental evaluation : sequential execution </ul><ul>A single process is responsible to manage the whole simulation. The fixed probability has computational requirements higher than other gossip protocols </ul>
  74. 74. <ul>Experimental evaluation : parallel execution </ul><ul><li>The simulation is obtained through the coordinated execution of a set of components ( Logical Processes, LPs )
  75. 75. Each LP manages the evolution of a part of the model and it is usually run by a different CPU
  76. 76. Each node in the scale-free net is modeled as a Simulated Entity (SE) , therefore each LP manages a set of SE s
  77. 77. SE s are randomly allocated in the LP s </li></ul><ul><li>It is worth noting that the performance of a Parallel And Distributed Simulation (PADS) is a trade-off between: a) load parallelization; b) communication overhead in the parallel/distributed architecture (also including synchronization) </li></ul>
  78. 78. <ul>Experimental evaluation : parallel execution (fixed probability) </ul>
  79. 79. <ul>Experimental evaluation : parallel execution (fixed fanout) </ul>
  80. 80. <ul>Experimental evaluation : parallel execution (cond. broadcast) </ul>
  81. 81. <ul>Nodes </ul><ul>Gossip #1 </ul><ul>Gossip #2 </ul><ul>Gossip #3 </ul><ul>3000 </ul><ul>3.46 </ul><ul>-12.22 </ul><ul>-9.1 </ul><ul>6000 </ul><ul>0.19 </ul><ul>-4.49 </ul><ul>-6.23 </ul><ul>9000 </ul><ul>-5.35 </ul><ul>-0.63 </ul><ul>-3.36 </ul><ul>12000 </ul><ul>-9.07 </ul><ul>-0.25 </ul><ul>-2.42 </ul><ul>Experimental evaluation : monolithic vs. parallel execution </ul><ul><li>Performance gap (%) between LP=1 (monolithic) and LP=4 </li></ul><ul><li>The parallel execution gives very unsatisfactory results
  82. 82. The communication overhead introduced by the parallel execution does not balance the gain given by the load parallelization </li></ul><ul>green = parallel faster than monolithic </ul>
  83. 83. <ul>Experimental evaluation : adaptive parallel execution </ul>
  84. 84. <ul>Experimental evaluation : adaptive parallel execution </ul>
  85. 85. <ul>Experimental evaluation : adaptive parallel execution </ul>
  86. 86. <ul>Nodes </ul><ul>Gossip #1 </ul><ul>Gossip #2 </ul><ul>Gossip #3 </ul><ul>3000 </ul><ul>34.18 </ul><ul>-1.33 </ul><ul>19.75 </ul><ul>6000 </ul><ul>38.63 </ul><ul>6.87 </ul><ul>23.37 </ul><ul>9000 </ul><ul>30.97 </ul><ul>11.07 </ul><ul>24.59 </ul><ul>12000 </ul><ul>26.47 </ul><ul>9.65 </ul><ul>22.56 </ul><ul>Parallel execution vs. adaptive parallel execution </ul><ul><li>Performance gap (%) between LP=1 and LP=4 GAIA ON </li></ul><ul><li>The results obtained by adaptive parallel execution are quite good
  87. 87. The best performances are obtained for gossip #1 and gossip #3, because both protocols are communication-bounded </li></ul><ul>green = parallel faster than monolithic </ul>
  88. 88. <ul>Putting it all together </ul><ul><li>Parallel approach gives unsatisfactory results in the simulation of such complex networks
  89. 89. An approach based on dynamic and adaptive clustering of the simulated entities can give valuable results
  90. 90. From the simulator to the simulation model : are gossip protocols a good choice to build dissemination mechanisms on scale-free networks? </li></ul>
  91. 91. MOG s: scalability and responsiveness <ul><li>Scalability and responsiveness are open problems in Multiplayer Online Games ( MOGs ) </li></ul><ul><li>Several architectures have been proposed to support MOGs: </li><ul><li>client /server
  92. 92. mirrored servers
  93. 93. peer-to-peer </li></ul></ul><ul><li>Given the distributed nature of this kind of applications, the dissemination of game events can be very costly
  94. 94. Under the scalability viewpoint the peer-to-peer approach is very promising </li></ul>
  95. 95. MOGs: a peer-to-peer approach <ul><li>Each peer locally manages its copy of the game state
  96. 96. The peers are organized in some form of overlay network
  97. 97. The dissemination of game events is obtained by passing messages through the overlay
  98. 98. What is the best form of overlay ? </li></ul><ul><ul><li>tree
  99. 99. fully connected graph
  100. 100. random graph
  101. 101. scale-free network (?) </li></ul></ul>
  102. 102. Scale-free networks for MOG s <ul><li>The idea is to implement MOGs using a peer-to-peer architecture that is partially “unstructured” and “spontaneous”
  103. 103. Some properties of scale-free networks (e.g. the very small diameter: d ~ ln ln n, with a network of n nodes ) are very valuable in supporting scalability and responsiveness
  104. 104. In our view, the dissemination of game events will be obtained through probabilistic approaches, for example using gossip protocols
  105. 105. The game events generated at peers are disseminated to the whole network, using very simple gossip protocols and without any form of centralization or predefined routing </li></ul>
  106. 106. Parameter Value number of nodes 0-500 message generation exponential distribution mean = 50 time-steps cache size (local to each node) 256 slots message Time To Live ( ttl ) 6 probability of dissemination ( v ) 0.5, 0.8, 1 (i.e. 50-80-100%) fanout value 5 (# of nodes) probability of broadcast ( p b ) 0.5, 0.8, 1 (i.e. 50-80-100%) simulated time (gaming time) 1000 time-steps (after building) Performance evaluation : simulation-based
  107. 107. Performance evaluation: metrics <ul><li>Coverage </li></ul><ul><ul><li>percentage of nodes that have received all the messages that have been produced during the whole game execution “ are the game events received by all gamers? ” </li></ul></ul><ul><li>Delay </li></ul><ul><ul><li>average number of hops (that is time-steps) necessary to receive a message after its creation “ is the data dissemination really responsive? ” </li></ul></ul><ul><li>Messages </li></ul><ul><ul><li>total number of messages routed in the game execution “ what is the overhead due to the events dissemination? ” </li></ul></ul>
  108. 108. Evaluation: coverage rate (%)
  109. 109. Evaluation: number of hops
  110. 110. Evaluation: total number of messages
  111. 111. Evaluation: coverage rate (%)
  112. 112. Evaluation: number of hops
  113. 113. Evaluation: total number of messages
  114. 114. Conclusions and Future work <ul><li>The low diameter of scale-free networks is very good for fast data dissemination
  115. 115. Common gossip protocols are unable to disseminate the whole event trace and their overhead is very high
  116. 116. Simple mechanisms such as caching of packets, ttl and protocols tweaking are quite ineffective or with limited impact on performances ( e.g. # of routed messages )
  117. 117. Smarter / adaptive protocols are necessary: </li></ul><ul><ul><li>push / pull approaches for data dissemination
  118. 118. adaptive protocols and behaviors
  119. 119. more information shared among network nodes </li></ul></ul>
  120. 120. Further Conclusions and Future work <ul><li>Adaptive mechanisms based on entities migration are very effective in improving the scalability of simulators
  121. 121. The benefits are not limited to wireless models: also more general environments (such as scale-free networks) can obtain very satisfactory results
  122. 122. The next steps are: </li><ul><li>more complex aggregation mechanisms in the migration of model entities
  123. 123. the implementation of simulators based on this paradigm and taking advantage of cloud-computing (on-demand) resources </li></ul></ul>
  124. 124. References <ul><li>G. D'Angelo, M. Bracuto. Distributed Simulation of Large-Scale and Detailed Models . International Journal of Simulation and Process Modelling 2009 - Vol. 5, No. 2. Special issue on &quot;Parallel and Distributed Simulation&quot;, Inderscience Publishers
  125. 125. L. Bononi, M. Di Felice, G. D'Angelo, M. Bracuto, L. Donatiello. MoVES: A framework for parallel and distributed simulation of wireless vehicular ad hoc networks . ComNet Journal, Special issue on &quot;Emerging Wireless Networks: Performance Modeling and Analysis&quot;, Volume 52, Issue 1, Elsevier
  126. 126. S. Ferretti, G. D'Angelo. Multiplayer Online Games over Scale-Free Networks: a Viable Solution? Workshop on DIstributed SImulation and Online gaming (DISIO 2010)
  127. 127. G. D'Angelo, S. Ferretti. Simulation of Scale-Free Networks . Proceedings of 2nd ACM/ICST International Conference on Simulation Tools and Techniques (SIMUTools 2009)
  128. 128. M. Bracuto, G. D'Angelo. Detailed Simulation of Large-Scale Wireless Networks . Proceedings of the 11-th ACM/IEEE International Symposium on Distributed Simulation and Real Time Applications (DS-RT 2007)
  129. 129. L. Bononi, M. Bracuto, G. D'Angelo, L. Donatiello. Exploring the Effects of Hyper-threading on Parallel Simulation . Proceedings of the 10-th ACM/IEEE International Symposium on Distributed Simulation and Real Time Applications (DS-RT 2006)
  130. 130. L. Bononi, M. Bracuto, G. D'Angelo, L. Donatiello. An Adaptive Load Balancing Middleware for Distributed Simulation . Proceedings of Frontiers of High Performance Computing and Networking - ISPA 2006 Workshops
  131. 131. L. Bononi, G. D'Angelo, L. Donatiello. HLA-based adaptive distributed simulation of wireless mobile systems . Proceedings of the 17th ACM/IEEE/SCS Workshop on Parallel and Distributed Simulation (PADS '03) </li></ul>
  132. 132. Online References, Preprints and Software Gabriele D'Angelo Department of Computer Science, University of Bologna <ul><li>E-mail: < [email_address] >
  133. 133. </li></ul>Parallel and Distributed Simulation (PADS) research group: <ul><li> </li></ul>Personal homepage: <ul><li> </li></ul>
  134. 134. Gabriele D’Angelo <> Department of Computer Science University of Bologna From Simulation to Online Gaming: the need for adaptive solutions