Quantifying User Satisfaction in Mobile Cloud Games


Published on

We conduct real experiments to quantify user satisfaction in mobile cloud games using a real cloud gaming system built on the open-sourced GamingAnywhere. We share our experiences in porting GamingAnywhere client to Android OS and perform extensive experiments on both the mobile and desktop clients. The experiment results reveal several new insights: (1) gamers are more satisfied with the graphics quality on mobile devices, while they are more satisfied with the control quality on desktops, (2) the bitrate, frame rate, and network delay significantly affect the graphics and smoothness quality, and (3) the control quality only depends on the client type (mobile versus desktop). To the best of our knowledge, such user studies have never been done in the literature.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Quantifying User Satisfaction in Mobile Cloud Games

  1. 1. Quantifying User Satisfaction in Mobile Cloud Games Chun-Ying Huang, Cheng-Hsin Hsu, De-Yu Chen, and Kuan-Ta Chen ACM MoVid 2014, Singapore 1
  2. 2. Mobile Games • Mobile games are hot! • in 2011, 59% smartphone users played mobile games [1] • by 2016, mobile game market will grow to 16 billion USD [2] • Mobile games are less visually appealing, because of the limitations on • CPU/GPU power • memory space/speed • network bandwidth • battery capacity • Possible solution: mobile cloud gaming 2 [1] http://www.infosolutionsgroup.com/popcapmobile2012.pdf [2] https://www.abiresearch.com/research/product/1006313-mobile-gaming
  3. 3. What is Mobile Cloud Gaming 3 Real-time game playing using light-weight mobile clients
  4. 4. Cloud Games on Mobile Devices • Mobile cloud gaming has many benefits • better visual quality  attract serious gamers • lower porting effort/cost  more games • lower battery consumption  longer play time • But, most cloud games are played on PCs and TV set- top boxes • steep development cost  most SDKs [CloudCom’13, NOSSDAV’13, MM’11] are proprietary • high bars on gamer satisfaction  high-quality + low latency • We address these two challenges in this work 4
  5. 5. GamingAnywhere: An Open Source Project • We, researchers, have tons of ideas to improve cloud gaming services, but all cloud gaming systems are proprietary and closed • GamingAnywhere is the first cloud gaming platform for researchers, developers, and gamer 5
  6. 6. 6 http://gaminganywhere.org/ • In 10 months • Web: 31,897 visits, 20,019 unique visitors • Forum: 106 topics, 357 posts
  7. 7. Visitor Distribution 7
  8. 8. Our Two Contributions • First, we optimize GamingAnywhere client on Android device • the first transparent cloud gaming platform  researchers, developers, and gamers may run any PC games using our client • Second, we conduct extensive user studies • various GamingAnywhere configurations with diverse resolutions, frame rates, bitrates, and network delays are applied to desktops and mobile devices  many new insights 8 Cloud Server Mobile Client Networks
  9. 9. Porting Client to Android • Challenges • short system delay: wireless networks incur longer latency • efficient implementation: limited computation power and battery life • user-friendly controller: no physical inputs (buttons and joysticks) and small screen size • Solution approaches • enable hardware A/V decoders  faster decoding and lower energy consumption • realize proof-of-concept controllers as overlays  the best controller design is out-of-scope 9
  10. 10. Mobile Client Architecture • Implemented by leveraging open-source packages • Support S/W and H/W decoders 10 Built-in H/W Decoders S/W Decoders
  11. 11. Controllers • Implement three proof-of-concept controllers, designed for • Nintendo 64 • Nintendo DS • Limbo 11 Nintendo 64 Controller Limbo Controller
  12. 12. User Interface • Setting profiles and start games 12
  13. 13. Demo 13
  14. 14. Testbed for User Studies 14 GamingAnywhere ServerLAN GamingAnywhere Desktop Client WiFi APGamingAnywhere Mobile Client • To understand how device type, game genre, resolution, bitrate, frame rate, and network delay affect user experiences
  15. 15. Experiment Settings • Limbo, Mario Kart, Super Mario, Super Smash Bros • 10 male and 5 female subjects between 21-34 years old • Configurations (each subject try all 68 configurations) • Resolution: 640x480, 960x720, 1280x960 • Bitrate: 1, 3, 5 Mbps • Frame rate: 5, 20, 50 fps • Network delay: 0, 150, 300 ms • MOS score (1-5) on • Graphics • Smoothness • Control 15
  16. 16. Mobile versus PC 16 PCs have many physical keys The implementations are efficient Really? Mobile is better?
  17. 17. Why Mobile Performs Better in Graphics? • First, subjects may have lower expectation on graphics of mobile devices • Second, smaller screen sizes make graphics imperfection less noticeable • Observation: The satisfaction levels are based on observed flaws than absolute quality! 17
  18. 18. Impacts of Different Game Genres • Subjects are more sensitive to graphics quality in Limbo than in Mario Kart • Mario Kart is a fast-paced racing game, while Limbo is rather static • Subjects are less sensitive to controls in platform games (Limbo and Super Mario) than in fighting (Super Smash Bros.) and racing (Mario Kart) games • Gamers face AI opponents in fighting and racing games • Gamers have enough time to prepare in platform games 18
  19. 19. Different Configurations • Graphics quality is affected by bitrate (dominating) and frame rate (weaker) • Resolution has no impact on graphics quality (surprising) • We suspect: (1) games are not too complex and (2) mobile client always up-scales the video  Through analysis is our future task • Smoothness is affected by network delay, frame rate, and bitrate  We suspect low graphics quality leads to low MOS score, more analysis is our future work • Control is only affected by client type (PC versus mobile) 19
  20. 20. Conclusion • We presented the optimized Android Gaming- Anywhere client • We conducted extensive mobile cloud gaming user studies, which reveal three main insights 1. Gamers are more satisfied with the graphics quality on mobile devices 2. The bitrate, frame rate, and network delay affect the graphics and smoothness quality the most 3. The control quality is only affected by client type (PC versus mobile) 20
  21. 21. QUESTIONS? Join us at http://gaminganwhere.org 21
  22. 22. BACKUP 22