線上遊戲與雲端運算

4,734 views

Published on

  • Be the first to comment

線上遊戲與雲端運算

  1. 1. 線上遊戲與雲端運算 陳昇瑋 中央研究院 資訊科學研究所
  2. 2.
  3. 3. Cloud for Games? 1. Computation cloud 2. Computation + rendering cloud
  4. 4. Is Server Consolidation Beneficial to MMOG? Yeng-Ting Lee and Sheng-Wei (Kuan-Ta) Chen Appeared in IEEE CLOUD 2010
  5. 5. Motivations <ul><li>Cost down </li></ul><ul><ul><li>Hardware investment (servers, network devices, cooling, space) </li></ul></ul><ul><ul><li>Adminstration (labor) costs </li></ul></ul><ul><ul><li>Energey saving </li></ul></ul><ul><li>Elasticity & Agility </li></ul><ul><ul><li>Equipment sharing among different game shards and even game titles </li></ul></ul>
  6. 6. Current Situation -- Overprovisioning <ul><li>Nonstandard architectures Different games may have totally different hardware and software (e.g., operating systems, middle ware, database systems) requirements. </li></ul><ul><li>Sharded design Most games adopt a “sharded” architecture by providing multiple identical game worlds (such a world is called a “realm” in World of Warcraft). </li></ul>
  7. 7. Sharded Architecture World 1 World 20 ...... At maximum 5,000 players in a world 100,000 players 20 duplicated worlds 500 players 3 servers
  8. 8. Realms in WoW IIS, Academia Sinica, Taiwan
  9. 9. Virtualization <ul><li>Virtualization is the creation of a virtual (rather than actual) version of something , such as an operating system, a server, a storage device or network resources. </li></ul>
  10. 10. Server Consolidation <ul><li>… is an approach to the efficient usage of computer server resources in order to reduce the total number of servers . </li></ul><ul><li>… in response to the problem of server sprawl , a situation in which multiple, under-utilized servers take up more space and consume more resources than can be justified by their workload. </li></ul>
  11. 11. Benefits of Server Consolidation <ul><li>Statically </li></ul><ul><ul><li>Aggregate workload on certain physical servers  fewer physical servers </li></ul></ul><ul><li>Dynamically </li></ul><ul><ul><li>Aggregation and distribution workload on the fly  fewer ACTIVE physical servers </li></ul></ul>
  12. 12. Server Consolidation is Good to MMORPGs: 3 Reasons <ul><li>Spatial locality property in players’ interaction  naturally partitionable </li></ul><ul><li>Workload is highly variable but predictable  potential to aggregate workload in off-peak periods </li></ul><ul><li>Operators normally run multiple games  possibility to share infrastructure </li></ul>
  13. 13. Spatial Locality Property <ul><li>Players tend to interact with others (by chatting, trading, fighting, or teaming up to fight others, etc) in their vicinity in game. </li></ul><ul><li>We consider such a zone a perfect unit for workload dispatching based on virtualization. </li></ul>
  14. 14. Workload Analysis IIS, Academia Sinica, Taiwan Variability Regularity Predictability
  15. 15. World of Warcraft <ul><li>The most popular MMOG for now </li></ul>
  16. 16. World of Warcraft <ul><li>WOW is the most popular MMORPG in the world, designed by BLIZZARD Entertainment. </li></ul><ul><li>The figures are in Winter Grasp Lake battle field. </li></ul><ul><li>There are many players/avatars interacting, combating, chatting and dying within the battle field. </li></ul>IIS, Academia Sinica, Taiwan
  17. 17. Data Collection Methodology <ul><li>Create a game character </li></ul><ul><li>Use the command ‘ who ’ </li></ul><ul><li>The command asks the game server to reply with a list of players who are currently online </li></ul><ul><li>Write a specialized data-collection program (using C#, VBScript, and Lua) </li></ul>
  18. 18. The Limitation of WoW API <ul><li>WoW returns at most 50 users in one query </li></ul><ul><li>We narrow down our query ranges by dividing all the users into different races, professions, and levels </li></ul>Level: 50+ Level: 30~39 Level: 40~49 Monster Human 100 users 45 users 15 users 60 users
  19. 19. Trace Summary
  20. 20. 福克斯大神之謎?? (1) ref. http://forum.gamebase.com.tw/content.jsp?no=4715&cno=47150002&sno=75201947 ref. http://www.wings-of-narnia.com/viewtopic.php?t=3012 網友 A :不知道在聖光之願部落的玩家有沒有發現到,在新手村薩滿訓練師的後面,永遠都會站著一個叫「福克斯大神」的獵人玩家!在半年前我到聖光定居時我在新手村見到他,到現在他仍然還是留守在那個地方……不會暫離、而且可以觀察他 = =&quot; 這種事該回報給 GM 嗎?創新手看到他的時候都覺得好恐佈啊囧 網友 B : me too 看到的一瞬間 突然起雞皮疙瘩 ..... 網友 C : &quot; 已離去 &quot; 玩家的怨念 ( 怨魂 @@) 嗎 ? 還是在悲傷愛情故事裡 , 癡等所愛的另一人 ? ^^^^^^^^QQ 網友 D :哈 線在好多人在看噢 旁邊為了一大群人 @@ 觀光景點呀 XD
  21. 21. 福克斯大神之謎?? (2) 網友 E :我剛剛也有去看了一下 開了一個 ID 叫做“聽說有鬼”的獸人戰士 坐在他面前的桶子一直望著他 ~ 忽然 ! < 暫離 > 福克斯大神 他蹲下了 ... 隔一分鐘 .. 消失 =ˇ=&quot; .. .. 現在我心裡也是毛毛的 .. 網友 F :好猛鬼啊 !!!!!! 大神的力量好可怕啊 , 一堆信眾死在他之前!!!!!! 網友 G :我上次有開過去看,還遇到了兩位同好,看的時候真的蠻不可思議的 ... 可以列入魔獸 10 大世界奇觀吧 !
  22. 22. 福克斯大神與祂的信眾們 -_-
  23. 23.
  24. 24.
  25. 25. Hourly Online Avatar Number
  26. 26. Variability <ul><li>Strong fluctuations over (0, 600) regularly </li></ul><ul><li>Coldest: 7am </li></ul><ul><li>Hottest: 11pm </li></ul>
  27. 27. Regularity - CDF <ul><li>Highly regular except weekends, where Sunday is more popular than Saturday </li></ul>
  28. 28. Periodicity - Autocorrelations <ul><li>Strong weekly and daily periodicity </li></ul><ul><li>Large coefficients with a lag shorter than 2 hours indicates high predictability within the future few hours </li></ul>
  29. 29. Predictability Analysis <ul><li>Highly predictable based on the last hour </li></ul><ul><li>Slightly less predictable mainly due to difference in weekdays and weekends </li></ul><ul><li>Prediction power is still high over adjacent weeks </li></ul>
  30. 30. Implications <ul><li>Avatar numbers are highly variable  potential benefits from server consolidations </li></ul><ul><li>Avatar number process is highly regular predictable  zone allocation algorithm needs not to be performed frequently and can even be pre-planned </li></ul>
  31. 31. Zone Allocation <ul><li>A classic bin-packing problem </li></ul><ul><li>One well-known heuristic algorithm is First-Fit Decreasing (FFD) </li></ul>
  32. 32. First-Fit Decreasing (FFD) IIS, Academia Sinica, Taiwan For each zone, it attempts to place the zone in the first server that can accommodate the zone. If no such a server is found, it boots a new server and puts the zone on the new server. (assuming each machine can support 150 units)
  33. 33. Simulation Setup <ul><li>An operator owns s = 100 servers hosting r = 100 realms of a game </li></ul><ul><li>Each realm contains r = 83 zones </li></ul><ul><li>Supposing that a server is capable of serving 7, 500 avatars </li></ul><ul><li>Modeling the avatar number in a realm as a normal distribution with mean 2640 and standard deviation 1500, which is derived from the data set on Warcraft Census and Wow Database </li></ul><ul><li>The avatar number in a zone is normalized using our traces from the TW-Light’s Hope realm </li></ul>
  34. 34. Three Policies Evaluated <ul><li>Fixed: state-of-the-art strategy </li></ul><ul><li>Dynamic (day): predict future one day workload and reallocate VMs on the servers per day (6 am). </li></ul><ul><li>Dynamic (hour): predict future one hour workload and reallocate VMs on the servers per hour. </li></ul>
  35. 35. # Server Used IIS, Academia Sinica, Taiwan
  36. 36. Results - Single Game
  37. 37. Results - Multiple Games
  38. 38. Summary <ul><li>Server consolidation is beneficial to MMOGs </li></ul><ul><li>However, how do MMOG operators adopt server consolidation? </li></ul>
  39. 39. A Scalable Virtualized Infrastructure for MMOGs on Cloud Scalability in terms of online user number and server number
  40. 40. Interactive Applications <ul><li>Definition </li></ul><ul><ul><li>NOT stateful </li></ul></ul><ul><ul><li>NOT cacheable </li></ul></ul><ul><ul><li>NOT request-response based </li></ul></ul><ul><ul><li>Sub-second real-time interaction requirements </li></ul></ul><ul><li>Features </li></ul><ul><ul><li>Persistent states </li></ul></ul><ul><ul><li>Long session time (e.g., a typical of 2 hours for gaming) </li></ul></ul><ul><li>Examples </li></ul><ul><ul><li>Networked virtual environments </li></ul></ul>
  41. 41. Side-by-side Comparison Web app Interactive app Connection time Sub-seconds hours Real-time requirement A few seconds 250 ms (typical) Connection inter-dependency Independent Frequent, real-time interactions between users Data writes (relatively) Few updates Continuous updates Data reads Many location-independent reads Few location-correlated reads
  42. 42. Server Required and Energy Consumed
  43. 43. Architecture Overview Server 1 VM - Game C Zone 1 Process Zone 2 Process Zone 3 Process Zone 4 Process Directory Service Mem DB (proxy) VM – Game A VM – Game B Disk DB Node 1 Disk DB Node 2 Disk DB Node X Server 2 VM - Game C Zone 5 Process Zone 6 Process Zone 7 Process Zone 8 Process Directory Service Mem DB (proxy) VM – Game A VM – Game B Server 3 Server 4 Server N Server 5 Directory Service Master Node
  44. 44. Research Challenges <ul><li>QoE-aware dynamic provisioning </li></ul><ul><ul><li>Both system performance metrics and user perceptions are multi-dimensional </li></ul></ul><ul><li>Live migrations </li></ul><ul><ul><li>With service uninterrupted </li></ul></ul><ul><li>Data placement </li></ul><ul><ul><li>Minimize data access latency and reduce data replication overhead </li></ul></ul>
  45. 45. Research Challenge #1: QoE-aware Dynamic Provisioning <ul><li>Ideally </li></ul><ul><ul><li>Move a process out of server A to another server B while A is overloaded </li></ul></ul><ul><li>In practice </li></ul><ul><ul><li>How to minimize # migrations? </li></ul></ul><ul><ul><li>Can we do workload prediction? </li></ul></ul><ul><ul><li>Processes may be correlated . We should put correlated processes on a server as possible </li></ul></ul><ul><ul><li>How to determine if a server is overloaded or not? (i.e., in terms of user perception) </li></ul></ul>Yeng-Ting Lee and Kuan-Ta Chen, &quot;Is Server Consolidation Beneficial to MMORPG? A Case Study of World of Warcraft,&quot; IEEE International Conference on Cloud Computing, July 2010.
  46. 46. Research Challenge #2: Live Migrations <ul><li>User migration across processes </li></ul><ul><ul><li>∵ Users may move from a zone to another zone </li></ul></ul><ul><ul><li>Processes may also reside at different VMs </li></ul></ul><ul><ul><li>Users may interact with others at any time </li></ul></ul><ul><li>Process migration across VMs </li></ul><ul><ul><li>∵ Server load balancing & consolidation </li></ul></ul><ul><ul><li>Keep # actives servers small but service quality maintained </li></ul></ul>Key challenges: maintain uninterrupted real-time server quality
  47. 47. Research Challenge #3: Data Placement <ul><li>Data are stored in distributed, replicated DBMS </li></ul><ul><ul><li>Key-value pair based </li></ul></ul><ul><li>FULL replication may not be needed (i.e., m DB nodes, k replicas, k < m) </li></ul><ul><ul><li>Data partitioning </li></ul></ul><ul><ul><li>Strategies to decide how to allocate k replicas of data (nearly uniformly) over m DB nodes </li></ul></ul><ul><ul><ul><li>Requesters are close to DB nodes </li></ul></ul></ul><ul><ul><ul><li>DB nodes are nearly load-balanced </li></ul></ul></ul>Jan-Jan Wu, Pangfeng Liu, Yi-Chien Chung, &quot;Metadata Partitioning for Large-Scale Distributed Storage Systems,&quot; IEEE International Conference on Cloud Computing, July 2010.
  48. 48. Practical Challenges <ul><li>Real-time data flow </li></ul><ul><ul><li>Data need to be transferred between processes and memory DB in nearly real time due to </li></ul></ul><ul><ul><ul><li>Users’ interactions in different processes </li></ul></ul></ul><ul><ul><ul><li>Fault tolerance </li></ul></ul></ul><ul><li>Application programming framework </li></ul><ul><ul><li>Transparent data update/access to/from in-memory and permanent DBMS </li></ul></ul><ul><li>Many yet to be identified… </li></ul>
  49. 49. SOME OTHER PROJECTS IN MMNET
  50. 50. Understanding the Performance of Thin-Client Gaming Yu-Chun Chang and Sheng-Wei (Kuan-Ta) Chen Working in progress
  51. 51. Thin Client Client Server User’s inputs Display update
  52. 52. Challenges <ul><li>Most thin-client products are proprietary </li></ul><ul><ul><li>Image compression, data-transmission protocol, and display update mechanism </li></ul></ul>
  53. 53. Our Focus
  54. 54. Experiment Methodology
  55. 55. Ms. Pac-Man & Bot <ul><li>Ms. Pac-Man </li></ul><ul><ul><li>Score  player’s QoE </li></ul></ul><ul><ul><li>Save score after the pacman ran out of 3 lives </li></ul></ul><ul><li>Bot: ICE Pambush3 (published in IEEE CIG 2009) </li></ul><ul><ul><li>Java-based controller to move the pacman </li></ul></ul><ul><ul><li>Capture the screen of the game and determine the position of the pacman, ghosts, and pills </li></ul></ul>Number Score Pill 220 10 Power pill 4 50 Ghost 4 200 (after eating power pills)
  56. 56. <ul><li>Three thin-client systems </li></ul><ul><ul><li>Logmein </li></ul></ul><ul><ul><li>Ultravnc </li></ul></ul><ul><ul><li>Teamviewer </li></ul></ul><ul><li>Network conditions </li></ul>Pipe Settings Delay Uplink / downlink 0 ms, 100 ms, 200 ms Loss rate Uplink / downlink 0%, 2.5%, 5% Bandwidth Uplink Unlimited, 50 kbps, 25 kbps Downlink Unlimited, 600 kbps, 300 kbps
  57. 57. <ul><li>Video recording in 200 fps </li></ul>
  58. 58. Thin Clients are Different!
  59. 59. Visual Difference Really Matters!
  60. 60. Statistical Regression Regression Model Frame delay Frame loss Frame loss burst length Frame quality QoE Frame rate Independent factors
  61. 61. Linear Model <ul><li>5 performance metrics + 3 apps </li></ul>Estimate Std. Error t value Pr(>|t|) (Intercept) 3487.2711 485.7793 7.179 4.53e-08 *** app1 135.7758 231.5055 0.586 0.5618 app2 -742.7495 298.9143 -2.485 0.0186 * Frame loss -13.5998 8.8013 -1.545 0.1324 Frame delay -0.4880 0.5754 -0.848 0.4029 Frame quality 3.1658 2.6756 1.183 0.2457 Frame loss burst length -28.9870 59.9977 -0.483 0.6324 Frame rate 9.3110 63.2724 0.147 0.8840 Adjusted R-squared: 0.8
  62. 62. Impact of Performance Metrics
  63. 63. Linear Model
  64. 64. Choosing IaaS Providers for MMOG Hosting Yeng-Ting Lee, Ya-Xuan Hong, and Sheng-Wei (Kuan-Ta) Chen Working in progress
  65. 65. *aaS
  66. 66. *aaS Instances
  67. 67. Amazon EC2 <ul><li>Amazon </li></ul><ul><li>Windows Azure </li></ul><ul><li>Rackspace </li></ul><ul><li>GoGrid </li></ul><ul><li>Flexiscale </li></ul>
  68. 68. Microsoft Azure
  69. 69. Rackspace
  70. 70. GoGrid
  71. 71. FlexScale
  72. 72. It is confusing!
  73. 73. Motivation <ul><li>No standard pricing framework </li></ul><ul><li>No standard SLA (service level agreement) framework for cloud services </li></ul>
  74. 74. Which cloud service provider and instance type is the best choice to online game operators?
  75. 75. Experiment <ul><li>Real game server and clients </li></ul><ul><li>eAthena: http://eathena.ws/wiki/index.php/Main_Page </li></ul><ul><li>OpenKore: http://www.openkore.com/index.php/Main_Page </li></ul>
  76. 76. Experiment flowchart How many avatars can be accommodated on each instance type (capacity / Instance)
  77. 77. # Avatar vs Processing Delay
  78. 78. Welcome to join us on the research of cloud gaming! Move your game to Cloud Today!
  79. 79. 謝謝聆聽,請多指教。 陳昇瑋 中央研究院 資訊科學研究所 http://www.iis.sinica.edu.tw/~swc

×