Successfully reported this slideshow.

On the Sensitivity of Online Game Playing Time to Network QoS


Published on

Online gaming is one of the most profitable businesses on the Internet. Among various threats to continuous player subscriptions, network lags are particularly notorious. It is widely known that frequent and long lags frustrate game players, but whether the players actually take action and leave a game is unclear. Motivated to answer this question, we apply survival analysis to a 1, 356-million-packet trace from a sizeable MMORPG, called ShenZhou Online.

We find that both network delay and network loss significantly affect a player’s willingness to continue a game. For ShenZhou Online, the degrees of player “intolerance” of minimum RTT, RTT jitter, client loss rate, and server loss rate are in the proportion of 1:2:11:6. This indicates that 1) while many network games provide “ping time,” i.e., the RTT, to players to facilitate server selection, it would be more useful to provide information about delay jitters; and 2) players are much less tolerant of network loss than delay. This is due to the game designer’s decision to transfer data in TCP, where packet loss not only results in additional packet delays due to in-order delivery and retransmission, but also a lower sending rate.

  • Be the first to comment

On the Sensitivity of Online Game Playing Time to Network QoS

  1. 1. On the Sensitivity of Online Game Playing Time to Network QoS Kuan-Ta Chen National Taiwan University Polly Huang Guo-Shiuan Wang Chun-Ying Huang Chin-Laung Lei Collaborators: INFOCOM 2006
  2. 2. Talk Outline <ul><li>Overview </li></ul><ul><li>Trace collection </li></ul><ul><li>Analysis and modeling of the relationship between session times and QoS </li></ul><ul><li>Implications and applications </li></ul><ul><li>Summary </li></ul>
  3. 3. Motivation <ul><li>Real-time interactive online games are generally considered QoS-sensitive </li></ul><ul><li>Gamers always complain about high “ping-times” or network lags </li></ul><ul><li>Online gaming is increasly popular depsite the best-effort Internet </li></ul>Q1: Are players really sensitive to network quality as they claim? Q2: If so, how do they react to poor network quality?
  4. 4. Assessment of User Satisfaction Internet Which path can provide the best user experience? ? ? ? Satisfaction Good Average Poor 3 Poor Good Average 2 Average Loss Poor Delay Jitter Good Latency 1 Path
  5. 5. Previous Work <ul><li>Evaluating the enjoyment of game playing in a controlled network environment </li></ul>Real-life user behavior is not measurable in a controlled experiment RTT = X Delay Jitter = Y Packet Loss = Z <ul><li>Subjective evaluation is costly and not scalable </li></ul><ul><li>Objective evaluation is not generalizable </li></ul><ul><ul><li>Shooting games: number of kills </li></ul></ul><ul><ul><li>Racing games: time taken to complete each lap </li></ul></ul><ul><ul><li>Strategy games: capital accumulated </li></ul></ul>
  6. 6. Our Conjecture Poor Network Quality Unstable Game Play Less Fun Shorter Game Play Time affects Verified by real-life game traces
  7. 7. Key Contributions / Findings <ul><li>Session time as a means to measure users’ feeling about network quality </li></ul><ul><li>Players are sensitive to network conditions (in terms of game playing time) </li></ul><ul><li>Proposed a time-QoS model to quantify the impact of network quality </li></ul>l o g ( d e p a r t u r e r a t e ) / 1 : 2 7 £ l o g ( r t t ) + 0 : 6 8 £ l o g ( j i t t e r ) + 0 : 1 2 £ l o g ( c l o s s ) + 0 : 0 9 £ l o g ( s l o s s )
  8. 8. 神州 Online <ul><li>A commercial MMORPG in Taiwan </li></ul><ul><li>Thousands of players online at anytime </li></ul><ul><li>TCP-based client-server architecture </li></ul>It’s me ShenZhou Online
  9. 9. Trace Collection (20 hours and 1,356 million packets) < 40 min Bottom 20% > 8 hours Top 20% 100 min Avg. Time 15,140 Session #
  10. 10. Round-Trip Times vs. Session Time y-axis is logarithmic
  11. 11. Delay Jitter vs. Session Time (std. dev. of the round-trip times)
  12. 12. Hypothesis Testing -- Effect of Loss Rate Null Hypothesis: All the survival curves are equivalent Log-rank test: P < 1e-20 We have > 99.999% confidence claiming loss rates are correlated with game playing times high loss low loss med loss The CCDF of game session times
  13. 13. Effect of QoS Factors -- Overview (significant: p-value < 0.01) N/A No Queueing Delay Negative Yes Server Packet Loss Negative Yes Client Packet Loss Negative Yes Delay Jitter Negative Correlation Yes Significant? Average RTT QoS Factor
  14. 14. Regression Modeling <ul><li>Linear regression is not adequate </li></ul><ul><ul><li>Violating the assumptions (normal errors, equal variance, …) </li></ul></ul><ul><li>The Cox regression model provides a good fit </li></ul><ul><ul><li>Log-hazard function is proportional to the weighted sum of factors </li></ul></ul>(our aim is to compute ) where each session has factors Z (RTT=x, jitter=y, …) The instantaneous rate of quitting a game for a player (session) Hazard function (conditional failure rate) h ( t ) = l i m ¢ t ! 0 P r [ t · T < t + ¢ t j T ¸ t ] ¢ t ¯ l o g h ( t j Z ) / ¯ t Z Commonly used to model the effect of treatment on the survival time or relapse time of patients
  15. 15. Model Fitting <ul><li>must be conformed </li></ul><ul><li>Explore true functional forms of factors by goodness-of-fit and Poisson regression </li></ul>A standard Poisson regression equation:  <ul><li>All factors are better describing user departure rate in the logarithmic form </li></ul>E ( X ) = e x p ( ¯ t Z ) e x p ( i n t e r c e p t ) h ( t j Z ) = e x p ( ¯ t Z ) h 0 ( t ) E [ s i ] = e x p ( ¯ t s ( Z ) ) Z 1 0 I ( t i > s ) h 0 ( s ) d s h ( t j Z ) / e x p ( ¯ t Z ) <ul><li>Human beings are known sensitive to the scale of physical magnitude rather than the magnitude itself </li></ul><ul><ul><li>Scale of sound (decibels vs. intensity) </li></ul></ul><ul><ul><li>Musical staff for notes (distance vs. frequency) </li></ul></ul><ul><ul><li>Star magnitudes (magnitude vs. brightness) </li></ul></ul>
  16. 16. The Logarithm Fits Better (client packet loss rate)
  17. 17. Final Model & Interpretation A: RTT = 200 ms B: RTT = 100 ms, other factors same as A Hazard ratio between A and B: exp((log(0.2) – log(0.1)) × 1.27 ) ≈ 2.4 A will more likely leave a game ( 2.4 times probability ) than B at any moment Interpretation < 1e-20 0.04 1.27 log(RTT) 0.01 0.01 0.03 Std. Err. 7e-13 0.09 log(sloss) < 1e-20 0.12 log(closs) < 1e-20 Signif. 0.68 Coef log(jitter) Variable
  18. 18. How good does the model fit? Observed average session times are almost within the 95% confidence band
  19. 19. Relative Influence of QoS Factors Server pakce loss = 15% Delay jitter = 45% Client packet loss = 20% Latency = 20% <ul><li>Earlier studies neglected the effect of delay jitters </li></ul><ul><li>Current games rely on only “ping times” to choose the best server </li></ul><ul><li>Client packets convey user commands </li></ul><ul><li>Server packets convey response and state updates </li></ul>
  20. 20. Applications of the Time-QoS Model <ul><li>An index to quantify user intolerance of network quality: </li></ul>(Best choice) <ul><li>[Application 1] Optimizing user experience </li></ul><ul><ul><li>Allocate more resources to players experience poor QoS </li></ul></ul><ul><li>[Application 2] Design tradeoffs </li></ul><ul><ul><li>Is it worth to sacrifice 20ms latency for reducing 10ms jitters? </li></ul></ul><ul><li>[Application 3] Path selection </li></ul>-5.4 -6.0 -5.6 Risk 30 ms (A) 20 ms (G) 50 ms (P) Jitter 1% (A) 200 ms (P) 3 1% (A) 150 ms (A) 2 5% (P) Loss rate 100 ms (G) Latency 1 Path l o g ( d e p a r t u r e r a t e ) / 1 : 2 7 £ l o g ( r t t ) + 0 : 6 8 £ l o g ( j i t t e r ) + 0 : 1 2 £ l o g ( c l o s s ) + 0 : 0 9 £ l o g ( s l o s s )
  21. 21. Summary <ul><li>Session time as a means to assess the impact of network quality on users in real-time applications </li></ul><ul><li>Game players are not only sensitive , but also reactive , to network conditions they experience </li></ul><ul><li>Proposed a time-QoS model as a utility function to optimize user experience and network infrastructure design </li></ul>
  22. 22. Thank You! Kuan-Ta Chen