Games on Demand: Are We There Yet?

Kuan-Ta Chen
Institute of Information Science
Academia Sinica
Games on Demand:
Are We There Yet?

Academia Sinica
31 research institutes in 3 major divisions
1) mathematics, physics, and applied sciences;
2) life sciences;
3) humanities and social sciences.
1000 tenure-tracked researchers
5,000 research associates and technicians

Games on Demand / Sheng-Wei “Kuan-Ta” Chen 3
Institute of Information Science
Members
40 Principal Investigators
40 post-doctoral researchers
300 technicians and RAs
Research Areas
•Bioinformatics •Network System and Service
•Data Management and Information Discovery •MultimediaTechnologies
•Natural Language and Knowledge Processing •Computer System
•Programming Languages and Formal Methods •ComputationTheory and Algorithms
Multimedia Networking and Systems Lab

Multimedia Networking and Systems Lab
Research Areas
Multimedia Systems
Quality of Experience Management
Computational Social Science
http://mmnet.iis.sinica.edu.tw

Area 2: Quality of Experience
Using physiological measurements to predict the
market performance of online games
6
[1] Jing-Kai Lou, Kuan-Ta Chen, Hwai-Jung Hsu, and Chin-Laung Lei, Forecasting Online Game
Addictiveness, IEEE/ACM NetGames 2012.

Area 3: Computation Social Science
“The emerging intersection of the social and computational
sciences, an intersection that includes analysis of web-scale
observational data, virtual lab–style experiments, and
computational modeling” [1].
[1] Duncan J.Watts, Computational Social Science Exciting Progress and Future Directions, Frontiers of
Engineering, Winter 2013.

Area 3: CSS (cont.)
Help people reduce weight by providing visual
incentives
lost 5 kg lost 4 kg

GAMES ON DEMAND

Tough Life of Gamers
Games are becoming way too complex
The overhead of setting up a game is significant
Often locked on a specific computer
Games may not be incompatible with some
software/hardware
Computer hardware constantly
demands upgrading

On-demand services

Games on Demand: Approaches
Painless game
installation
e.g., on Xbox 360
Cloud gaming
Cloud-supported instant
game play

Cloud Gaming: File Streaming
Instant game play supported by a minimal, playable
code base (~ 5%)
Progressive downloading of game code and data during
game play
3D mesh streaming can be seen a special instance
(Figure courtesy of WeiTsang Ooi from “ScalableView-Dependent Progressive Mesh Streaming”)

Cloud Gaming: Video Streaming
Video-based remote desktop specialized for
Games running in cloud
High-definition real-time game play
Game servers
Internet
Streaming
Streaming
Streaming
PC
Laptop
Mobile

The Selling Points
Gamers’ perspectives
Frees gamers from indefinitely upgrading their computers
Enables gamers to play games anywhere, anytime
Game manufacturers’ perspectives
Allows developers to support more platforms
Reduces the production cost
Prevents pirating

Cloud gaming is expected to lead the future
growth of computer games: 9 times in 6 years
Cloud Gaming is Hot
[CGR] http://www.cgconfusa.com/report/documents/Content-5minCloudGamingReportHighlights.pdf

Challenges

Challenge #1
Unavoidable extra delays
Video encoding at the server
Video decoding and playout buffering at the client
Less opportunities for delay compensation
Game states (e.g., game objects’ positions and velocity) sare
not available at the client side
A Comparison with “Traditional”
Online Games

Challenge #1 (cont.)
OnLive dictates a server rendering/processing latency of
nearly 100 ms, and partially copes with it by setting up 7
data centers merely in North America
Only people who live in 1000 mile radius from a data center are encouraged to play
Similarly, Sony/Gaikai has 8 data centers in NA
(Figure courtesy of Mark Claypool from “Latency and Player Actions in Online Games)

Challenge #2
For a regular x264 zerolatency implementation,
3--5 Mbps is required for a quality 720p cloud gaming
session (on desktop / TV)
Playout buffering is commonly used to absorb packet
delivery disorders (loss, re-orders)
 not applied here as short latency is critical

Challenge #3
Investing thousands of cloud servers was partly the
reason for OnLive’s bankruptcy in 2012.
GPU virtualization is getting more
mature, but the degree of
multiplexity is still around 10—20
i.e., to support 10000 current users, 500—1000
servers are required

Challenge #3 (cont.)
The state of the practice
OnLive Sony NVIDIA ODM
Specification 2 MB in 2U 4 PS4 MB in 1U 2U with 6 Graphic cards 2 MB in 1U
# GPU 2 4 12 8
GPU/U 1 4 6 8

Outline
An Open-Source Cloud Gaming Testbed
Quantifying the Susceptibility of Games to Latency
Quantifying User Satisfaction in Mobile Cloud Games
QoE-aware Auto-Reconfiguration
Placing Virtual Machines to Optimize Cloud Games
Future Perspectives

An Open-Source Implementation
Researchers have tons of ideas to improve cloud
gaming services, but all existing cloud gaming
systems are proprietary and closed

System Architecture
The client and the server, with many
components
Implement by leveraging open-source packages

Process Video Frames in Parallel
Suppose the targeted inter-frame delay is ∆t
The response delay may greater than ∆t
frame capture + color space conversion + encoding
It could degrade encoding bitrate
Process in parallel

Video Playout Buffering
The 1-frame buffering strategy
Based on the RTP marker bit
An H.264 frame can be split into different numbers of packets
The marker bit (with a value of 1) indicates the last packet of a
frame

GA Has Lower Response Delay
Low response delay * network delay has been excluded for
FAIR comparisons

GA Provides (Relatively) Better
Video Quality

http://gaminganywhere.org/
56k+ visitors, 100k+ downloads since April 2013

Visitor Distribution
Geo-distribution
/ Day
July
2015

Outline
Future Perspectives

The Question
Are games equally
susceptible to
latency?

Definition
Real-time strictness (RS)
The degree a game’s QoE degrades when the latency is higher
Cloud-gaming friendliness
A cloud game’s susceptibility to latency in terms of its QoE

Selected Games
ACT
LEGO Batman (Batman)
Devil May Cry (DMC)
Sangoku Musou 5 (Dynasty Warriors 6) (SM5)
FPS
Call of Duty: World at War (COD)
F.E.A.R 2 (FEAR)
Unreal Tournament 3 (Unreal)
RPG
Ys Origin (Ys)
Loki: Heroes of Mythology (Loki)
Torchlight (Torch)

Facial EMG approach
1. Continuous emotion measures (can be at a rate of 1000
Hz or even higher)
2. Does not disturb game play
3. Objective since the emotional indicators are directly
measured rather than told by subjects
(EMG: Electromyography)

Facial EMG Measurement Setup
The corrugator
supercilii muscle
Negative emotions
The amount of annoyance
caused by latency

Measurement devices
PowerLab 16/30
Electrodes
Wires

During game play…

Trace Summary
Subjects
Trace

Overall EMG potentials

EMG Potentials for each game
1. Diverse baseline EMG potentials for each game
2. The increasing rates of EMG potential are game-dependent as
well

Deriving real-time strictness (RS)

RS of the studied games
In general, FPS > RPG > ACT in terms of RS
Game pace↑, RS↑, latency-critical↑

Our conjecture
How a game responds to players’ commands is
associated with its real-time strictness
If its commands are “lightweight”
Simple, fast, local moves
Timing is important  higher RS
If its commands are “heavy”
Associated with long and large amounts of animations
Timing is not critical  lower RS

Illustrations for “light” commands
https://www.youtube.com/watch?v=ycYDDBKrv4I

Illustrations for “heavy” commands
https://www.youtube.com/watch?v=GGm1YNJNWbo

RS prediction

Application #1: Balance games’ QoE
degradation due to latency
Scenario
N users are playing different games at the same time
Users experience different latencies and games have different RS
 Each player experiences different levels of QoE degradation
Usage
Use the model to infer which players are having a worse gaming experience
than others
Prioritize the server’s resources, such as CPU and GPU, to reduce those
players’ latencies and thereby mitigate QoE degradation they would
otherwise experience

Application #2: Co-optimizing data center
cost and gaming experience
Scenario
N data centers, each has distinct operation cost (electricity and labor)
Whenever a user signs in, we need to assign a data center to him for
real-time game play
Question: Which data center should we assign to the player?
Usage
Use the model to predict users’ QoE in all the cases and choose the data
center which provide a “just good enough” gaming experience
Data center A:
Lower cost, longer delay
Data center B:
Higher cost, shorter delay

Outline
Future Perspectives

Mobile games are !
in 2011, 59% smartphone users played mobile games [1]
by 2016, mobile game market will grow to 16 billion USD [2]
Mobile games are less visually appealing, because of the
limitations on
CPU/GPU power
memory space/speed
battery capacity
Possible solution: mobile cloud gaming
Mobile Games
[1] http://www.infosolutionsgroup.com/popcapmobile2012.pdf
[2] https://www.abiresearch.com/research/product/1006313-mobile-gaming

Testbed for User Studies
Nintendo 64 Limbo

Demo
Online

Questions
Is mobile cloud gaming energy efficient?
How to tune video parameters in an energy-
conserving way?
What components are energy-hungry?
Mobile gaming experience comparable to PC?

Cloud gaming is energy efficient
Independent of game genres Energy saving
(50% in CPU and 30% in energy)

Energy consumptions
Impact of tunable parameters
Frame rate > Bit rate > Resolution
3G consumes 30%--45% more energy than WiFi
Input event processing incurs non-trivial energy consumption

Comparison on Gaming Experience
PCs have many
physical keys
Implementations are
efficient
Really? Mobile is
better?

Why Mobile Performs Better in Graphics?
First, subjects may have lower expectation on
graphics of mobile devices
Second, smaller screen sizes make graphics
imperfection less noticeable
Observation: The satisfaction levels
are based on observed flaws than
absolute quality!

Outline
Future Perspectives

The need for auto reconfiguration
The provided QoE is normally poor when our video packets
experience loss events
 We will have to voluntarily reduce bandwidth usage when
network is (temporarily) overloaded
Due to network dynamics, the provisioning of network bandwidth
may vary in sub-seconds
 An automatic reconfiguration mechanism is required that can
respond to changes in run time

Our Goal
Assuming N users playing different games
A mechanism to select the best (bitrate, frame rate)
configuration for each user given the current game
he/she is playing
Two explicit objectives
Maximize the average gaming experience (i.e., utilitarian)
Maximize the worst gaming experience (i.e., fairness)

Crowdsourced user study

QoE vs. QoS factors
Our intuitions
Bitrate , frame rate  graphics quality
Frame rate  interactivity

Game Genre Matters
Action
Game
Car Racing
Game

Many cloud gaming users share
a bottleneck link to a data center
Maximize average MOS by choosing
bitrate and frame rate for each user
Problem Formulation

The proposed system
A passive bandwidth estimator for 802.11 network
A quadratic QoE model for each game
An approximate algorithm for solving the optimization
problem efficiently

Achieved Performance
(Efficiency = MOS score / bandwidth consumed)
(Running time in seconds)

Outline
Future Perspectives

The Research Problem
Assuming each VM handles one game session
Consolidating VMs in different ways results in different
profits and gaming quality
For example, different data centers have different prices and offer different
quality of service
Hence, we propose VM placement policies to maximize
the profits or gamer QoE

Notations
• Frame per Second:
• Processing Delay:
• Network Latency:
• CPU Utilization:
• GPU Utilization:
• Hourly fee:
• Operational Cost:
• Memory of Server:
• Uplink of Datacenter:

Problem Formulation:
Provider Version
Objective Function: Maximize Profits
Constraint: QoE Degradation
Frame Per Second
Delay
 Decision variable:
……

Quality-Driven Heuristic (QDH):
Provider Version
Intuition: put as many VMs on a server as possible
Condition: Do not exceed the user-specified maximal
tolerable QoE degradation

QDH’: Gamer Version
 A similar formulation but here we minimize QoE
degradation as possible
 Objective function:

Our Testbed
Physical Servers
CPU: i5
GPU: NVIDIA Quadro 6000
Memory: 16GB
Broker
CPU: i7 3.2 GHz
Memory: 16GB
Clients
CPU: i5
Memory: 4 GB

Baseline Algorithm
Location Based Placement (LBP) algorithm
places each VM on a random game server that is not
fully loaded and the data center geographically closest
to the gamer

QDH Increases Profits
 Save money (by shutting down more servers and
relocating servers to a less expensive data center)
 Always satisfy the specified QoE requirement

QDH’ Improves QoE
Outperforms LBP algo. by providing much higher QoE

Both Algorithms Run in Real Time
Both algorithms terminate in < 2.5 sec on a commodity
PC even for large services with 20,000 servers and
40,000 gamers

Outline
Are We There Yet?

Technical Reasons

Technical Reason #1
Explore possible next states
Render possible frames and send to user
User chooses one based on input
Manage to hide latency up to 384 ms at
the cost of 4.5x higher bandwidth (and
extra computation/rendering cost)
[1] Chu, K. L. D., Cuervo, E., Kopf, J., Grizan, S.,Wolman, A., & Flinn, J. Outatime: Using
Speculation to Enable Low-Latency Continuous Interaction for Cloud Gaming,ACM
MobiSys 2015.
Pre-render future frames seems possible

Technical Reason #2
Objects that are far away or near
peripheral vision can be coded with
fewer bits
Leads to ~50% bit rate reduction with
4.75% MOS reduction
[1] Ahmadi, H., Khoshnood, S., Hashemi, M. R., & Shirmohammadi, S., Efficient bitrate
reduction using a Game Attention Model in cloud gaming. In IEEE HAVE 2013.
Game info (e.g., camera and object
positions) can be used to better encode

Technical Reason #3
GPU virtualization is getting more mature
NVIDIA and AMD design specialized
GPUs and drivers for cloud gaming
Cloud-gaming-friendly game engines
would further boost the scalability (by
planned GPU & VRAM sharing, etc)
Degree of multiplexity keeps increasing

Marketing Reasons
As a complement, rather than a replacement solution
E.g., Playstation Now uses cloud gaming to provide backward compatibility
and cross-platform support
As a playable ad
Startups such as mNectar, Agawi, Voxel,
provide playable ad services
(mainly for mobile apps)

Marketing Reasons (cont.)
B2B2C business model
e.g., G-cluster Global provide turnkey solutions to telecom
operators around the world
to solution providers: almost risk-free and more scalable
to local service providers: low-cost investment as they can use
existing infrastructures
Seems a sustainable model which is key to success

Game Integration
Video Codec
Virtualization
User Interface
QoE Measurement and
Modeling
Server Selection
Parameter Adaptation
Resource Scheduling
[1] Kuan-Ta Chen, Chung-Ying Huang, and Cheng-Hsin Hsu, "Cloud Gaming Onward:
Research Opportunities and Outlook," Proceedings of IEEE C-Game 2014, July 2014.

Conclusion
Cloud gaming shares similar fundamental
problems with many interesting applications
Screencasting
Mobile smart lens
Tele medicine
Immersive remote
communications
Thus, cloud gaming seems a rewarding entrance to
fundamental multimedia system challenges!

My special thanks to…
GamingAnywhere team
Dr. Chun-Ying Huang Dr. Cheng-Hsin Hsu Chih-Fang Hsu
Hua-Jun Hong Ching-Ling Fang Tsung-HanTsai

Kuan-Ta Chen
Academia Sinica
cloud gaming
rocks!
Thank You!
http://www.iis.sinica.edu.tw/~swc

Games on Demand: Are We There Yet?

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Games on Demand: Are We There Yet?

Similar to Games on Demand: Are We There Yet? (20)

More from Academia Sinica

More from Academia Sinica (20)

Recently uploaded

Recently uploaded (20)

Games on Demand: Are We There Yet?