Research on cloud gaming: status and perspectives

Cloud Gaming
Status and Perspectives
Gwendal Simon

About gaming
2 / 36 Gwendal Simon Cloud Gaming

About gaming
59% of Americans play video games
39% of gamers are 36+ years old
48% of gamers are female

About gaming
The global revenue will hit $100 billion by 2018
More than Hollywood movie industry
Gaming is 20% of iPhone downloaded apps

About gaming
The global revenue will hit $100 billion by 2018
More than Hollywood movie industry
Gaming is 20% of iPhone downloaded apps
147.000 jobs are related to gaming in 2014 in US

Gaming modular architecture
Gamer Command
Interface
Game Engine
Graphical Output
Game
Server

Gamer Command
Interface
Game Engine
Graphical Output
Game
Server
Console/PC market based on power-
ful hardware. Target hardcore gamers
PC + software

Gamer Command
Interface
Game Engine
Graphical Output
Game
Server
Smartphone market based on app
store. Target casual gamers
smartphone + app

Gamer Command
Interface
Game Engine
Graphical Output
Smartphone market based on app
store. Target casual gamers
Cloud gaming based on the cloud and
lightweight devices. Target everybody
cloud
home

Cloud gaming architecture in a nutshell
Data-Center
Internet

Data-Center
Internet
user actions

Data-Center
Internet
user actions
video stream

Why does it make sense ?
For game developers :
Better piracy control

Support more client platforms

Easier to upgrade

Easier to upgrade
New business oﬀers (e.g. subscription-based)

Easier to upgrade
For gamers :
No more hardware

Easier to upgrade
For gamers :
No more hardware
Play games on thin clients

Easier to upgrade
For gamers :
No more hardware
Play games on thin clients
Follow-me gaming

Business status
The pioneers have shut down :
OnLive bankrupted in 2013. Too early
StreamMyGame never took oﬀ.

Business status
Last game consoles integrate cloud gaming solutions :
Sony acquired Gaikai (PS3 backward compatibility)
Microsoft launched XBox Live Cloud a year later

Business status
Last game consoles integrate cloud gaming solutions :
Sony acquired Gaikai (PS3 backward compatibility)
Microsoft launched XBox Live Cloud a year later
New actors enter the market :
NVidia released Nvidia Grid and Shield

Dissecting the
Latency

The main reasons for latency
Response time is the delay from an
action done at the user and the result of
this action on the display

gateway
t1

gateway
t1
Internet
t2

gateway
t1
Internet
t2
DC router
t3

gateway
t1
Internet
t2
DC router
t3 user input
processing
t4

gateway
t1
Internet
t2
DC router
t3 user input
processing
t4
game state
update
t5

gateway
t1
Internet
t2
DC router
t3 user input
processing
t4
game state
update
t5
graphics
rendering
t6

gateway
t1
Internet
t2
DC router
t3 user input
processing
t4
game state
update
t5
graphics
rendering
t6
video
encoding
t7

gateway
t1
Internet
t2
DC router
t3 user input
processing
t4
game state
update
t5
graphics
rendering
t6
video
encoding
t7
network
delivery
t1 + t2 + t3

gateway
t1
Internet
t2
DC router
t3 user input
processing
t4
game state
update
t5
graphics
rendering
t6
video
encoding
t7
network
delivery
t1 + t2 + t3video
decoding
t8

gateway
t1
Internet
t2
DC router
t3 user input
processing
t4
game state
update
t5
graphics
rendering
t6
video
encoding
t7
network
delivery
t1 + t2 + t3video
decoding
t8
latency due to network

gateway
t1
Internet
t2
DC router
t3 user input
processing
t4
game state
update
t5
graphics
rendering
t6
video
encoding
t7
network
delivery
t1 + t2 + t3video
decoding
t8
latency due to game engine

gateway
t1
Internet
t2
DC router
t3 user input
processing
t4
game state
update
t5
graphics
rendering
t6
video
encoding
t7
network
delivery
t1 + t2 + t3video
decoding
t8
latency due to game engine
latency due to video encoding and decoding

Acceptable latency
target

Game genre
game genre
avatar-based games
ﬁrst-person games third-person games
omnipresent games

Genre, latency, and performance
The Claypool law of latency acceptance :
genre ﬁrst-person third-person omnipresent
lag in ms 100 500 1000

Genre, latency, and performance
The Claypool law of latency acceptance :
genre ﬁrst-person third-person omnipresent
lag in ms 100 500 1000
Various other factors to consider :
Limitations in the classiﬁcation (e.g. StarCraft)
User control games (more or less immersive)
Gamer skill level (from casual to professional)

Game action taxonomy

Game action taxonomy
. . .add the impact of the action
Toward priority classes for actions ?

Latency target
The shorter the better

Latency target
The best range is from 100 to 150 ms

Latency target
Up to 300 ms is enough for some games

Latency target
Up to 300 ms is enough for some games
Action prioritization may be standardized

Chasing the ms

Video
Gamers have high expectations :
High-resolution (1080p possible since PS3)
High frame rate (60 fps is common)
Complex, dynamic scenes

Video
Gamers have high expectations :
High-resolution (1080p possible since PS3)
High frame rate (60 fps is common)
Complex, dynamic scenes
Screencasting technologies are booming :
Capture, encode, and packetize
e.g., Open Broadcast Software

Video decoding/encoding
In the GamingAnywhere platform :

In the Rhizome platform :
encoding capture im. convert im. encode frame stream frame total
720p 6.4ms 4.4ms 2.8ms 0.2ms 13.8ms
1080p 11.0ms 4.9ms 3.0ms 0.3ms 19.2ms

720p 6.4ms 4.4ms 2.8ms 0.2ms 13.8ms
1080p 11.0ms 4.9ms 3.0ms 0.3ms 19.2ms
Use info from the game engine to encode video
Reduce Motion Estimation (ME) using
Game object information
Depth map

720p 6.4ms 4.4ms 2.8ms 0.2ms 13.8ms
1080p 11.0ms 4.9ms 3.0ms 0.3ms 19.2ms
Use info from the game engine to encode video
Reduce Motion Estimation (ME) using
Game object information
Depth map
Reduce encoding complexity using
Object priority (adaptive object encoding)

Game engine latency

Virtualization on GPU
Cloud management is based on Virtual Machines
Mature technologies to create, shut down, migrate
Hardware consolidation : several VMs per machine

Poor performances on traditional GPU
Degrades memory transfer (GPU bottleneck)
Prevents Single-Instruction-Multiple-Data

Poor performances on traditional GPU
Degrades memory transfer (GPU bottleneck)
Prevents Single-Instruction-Multiple-Data
Pass-through GPU enables better performances :
One-to-one mapping between device and VM
New technologies coming (e.g., NVidia Grid)

Virtualization on GPU : performances
Bare-Metal Xen 4.0 Bare-Metal Xen 4.1
0
100
200
300
2011 2014
126.2
39.8
274.7
266.9
framespersecond
Doom3

Virtualization on GPU : performances
Bare-Metal Xen 4.0 Bare-Metal Xen 4.1
0
100
200
300
2011 2014
126.2
39.8
274.7
266.9
84.5
51
153.7 153.4
framespersecond
Doom3
Unigine Sanctuary

Game virtualization : consolidation
nb. of instances Bare-metal (in fps) VM (in fps)
1 30.2 29.2
2 30.0 28.9
3 29.8 28.9

Game virtualization : consolidation
nb. of instances Bare-metal (in fps) VM (in fps)
1 30.2 29.2
2 30.0 28.9
3 29.8 28.9
1 2 3
0
200
400
600
800
1,000
287
475
662
296
485
704
Nb. of game instances
Watts
Bare-metal
Virtual machines

Latency : summary
Incompressible latency (without network) :

Latency : summary
Video encoding and decoding : 40 ms

Latency : summary
Game engine : from 15 to 50 ms

Latency : summary
A lot of works for key latency gains here and there :

Latency : summary
A lot of works for key latency gains here and there :
Adaptive streaming in both resolution and fps
Prioritize actions
Provide metadata and information on game objects
Shared processes for multiple game instances

To contribute to the research eﬀort
Some open platforms :
GamingAnywhere, the most mature

Uniquitous based on Unity 3D

Uniquitous based on Unity 3D
Rhizome with integration of NVidia Grid

Focus on network
latency

Our objectives
Can the current cloud support cloud gaming ?

Our objectives
Which percentage of users can play cloud games ?

Our objectives
1. In excellent conditions (40 ms network latency)

Our objectives
1. In excellent conditions (40 ms network latency)
2. In good conditions (80 ms network latency)

Our population sample
A modiﬁed BitTorrent crawler in Dec. 2012 :
Collected 2,504 IP addresses in US
Most likely machines for home entertainment

Latency to EC2 : experiment
Amazon datacenters (Portland, Virginia, North Calif.)

For each client (IP address)
1. Measure latency from each DC

2. Pick the shortest latency

3. Do it ten times

Latency to EC2 : results
0 20 40 60 80 100 120 140 160
0
0.2
0.4
0.6
0.8
1
median latency (in ms)
ratioofcoveredusers

Latency to EC2 : results
0 20 40 60 80 100 120 140 160
0
0.2
0.4
0.6
0.8
1
two thirds of gamers in
good conditions
one third of gamers in
excellent conditions
one tenth of gamers
cannot play at all
ratioofcoveredusers

What if more DCs are used
44 PlanetLabs nodes (all over the US)

For each client (IP address) :
1. Measure latency from each PlanetLabs DC

2. Do it ten times

2. Do it ten times
For each k-subset of DCs :

2. Do it ten times
1. Choose for each client the shortest latency
2. Sum all shortest latencies

2. Do it ten times
1. Choose for each client the shortest latency
2. Sum all shortest latencies
Then pick the best k-subset of DCs

More DCs : results
5 10 15 20 25
0
0.2
0.4
0.6
0.8
1
EC2 80 ms
EC2 40 ms
number of datacenters
ratioofcoveredusers

More DCs : results
5 10 15 20 25
0
0.2
0.4
0.6
0.8
1
EC2 80 ms
EC2 40 ms
number of datacenters
ratioofcoveredusers
80 ms latency threshold
40 ms latency threshold

Toward a hybrid DC-P2P infrastructure
Out of the 2,504 clients :
1,000 are edge servers
1,504 are clients

1,504 are clients
For each client :
1. Measure latency to each Amazon DC

1,504 are clients
For each client :
2. Measure latency to all edge-servers

1,504 are clients
For each client :

1,504 are clients
For each client :
4. Do it ten times

Hybrid DC-P2P : results
0 20 40 60 80 100 120 140 160
0
0.2
0.4
0.6
0.8
1
ratioofcoveredusers
Amazon multi-DC infra.

0 20 40 60 80 100 120 140 160
0
0.2
0.4
0.6
0.8
1
ratioofcoveredusers
Hybrid DC-P2P infra.

+105%
+28%
+12%
0 20 40 60 80 100 120 140 160
0
0.2
0.4
0.6
0.8
1
ratioofcoveredusers
Hybrid DC-P2P infra.

Network latency : summary
Not all users will be served

100 ms overall lag needs a lot of servers at the edges

100 ms overall lag needs a lot of servers at the edges
Game placement on servers will be key

Conclusion

Main researchers in the area
Kuan-Ta Chen (Sinica - Taiwan)

Shervin Shirmohammadi (Ottawa - Canada)

Mark Claypool (Worcester - UK)

Jiangchuan Liu (Simon Fraser - Canada)

Jiangchuan Liu (Simon Fraser - Canada)
Carsten Griwodz (Simula - Norway)

Takeaway
Business attractive, but techno challenging

Takeaway
Many research areas to explore :
Delivery network

Takeaway
Delivery network
Game virtualization

Takeaway
Delivery network
Game virtualization
Priority actions and adaptive encoding

Takeaway
Delivery network
Game virtualization
... and everything to chase the ms everywhere

Takeaway
Delivery network
Game virtualization
... and everything to chase the ms everywhere
And what about multi-player games ?

Research on cloud gaming: status and perspectives

More Related Content

What's hot

Similar to Research on cloud gaming: status and perspectives

More from Gwendal Simon

Recently uploaded

Research on cloud gaming: status and perspectives