This document discusses using Amazon Web Services analytics tools to analyze game player data and behaviors. It provides examples of using Amazon Mobile Analytics to collect player event data and exporting it to Amazon Redshift for storage and analysis. It then demonstrates how to generate business metrics and perform analyses like segmentation, retention, and cohorts using SQL queries in Redshift. The overall message is that AWS offers affordable, scalable services to ingest, store, and analyze mobile game data to improve player engagement and monetization.
4. AWS Gaming Solutions
Analytics at Supercell
“You cannot predict success.
You will only find out after releasing it.”
- Ilkka Paananen, CEO, Supercell
5. AWS Gaming Solutions
Analyze What?
Emotions
• Enjoying game
• Engaged
• Like/dislike new content
• Stuck on a level
• Bored
• Giving up
Behaviors
• Hours played day/week
• Number of sessions/day
• Level progression
• Friend invites/referrals
• Response to mobile push
• Money spent/week
16. AWS Gaming Solutions
Ok, A Real Business Plan
Ingest
• S3 PUT
• Analytics SDK
• Kinesis
Store
• S3
• DynamoDB
• HDFS
Process
• Redshift
• EMR (Hadoop)
• Spark
Analyze
• Tableau
• Pentaho
• Jaspersoft
17. AWS Gaming Solutions
• Collect Events on Device
• Periodically Store in S3
• Process Data into Redshift
• Analyze with GUI Visualization Tool
Start Simple
2015-‐03-‐03,nateware,e4df,login
2015-‐03-‐03,nateware,e4df,gamestart
2015-‐03-‐03,nateware,e4df,gameend
2015-‐03-‐03,nateware,a88c,login
2015-‐03-‐03,nateware,a88c,friendlist
2015-‐03-‐03,nateware,a88c,gamestart
Profit!
18. AWS Gaming Solutions
Redshift at a Glance
10 GigE
(HPC)
Ingestion
Backup
Restore
SQL Clients/BI Tools
128GB RAM
16TB disk
16 cores
Amazon S3/DynamoDB
JDBC/ODBC
128GB RAM
16TB disk
16 coresCompute
Node
128GB RAM
16TB disk
16 coresCompute
Node
128GB RAM
16TB disk
16 coresCompute
Node
Leader
Node
• Leader Node
– SQL endpoint
– Stores metadata
– Coordinates query execution
• Compute Nodes
– Columnar table storage
– Load, backup, restore via Amazon S3
– Parallel load from Amazon DynamoDB
• Single node version available
20. AWS Gaming Solutions
Plumbing
① Create S3 bucket ("mygame-analytics-events")
② Request a security token for your mobile app:
http://docs.aws.amazon.com/STS/latest/UsingSTS/Welcome.html
③ Upload data from your users' devices
④ Run a scheduled copy to Redshift
⑤ Setup Tableau to access Redshift
⑥ Go to the Beach
21. AWS Gaming Solutions
Loading Redshift from S3
copy
events
from
's3://mygame-‐analytics-‐events'
credentials
'aws_access_key_id=<access-‐key-‐id>;
aws_secret_access_key=<secret-‐access-‐key>'
delimiter=',';
Scheduled Redshift Load using Data Pipeline:
http://aws.amazon.com/articles/1143507459230804
Search Slideshare for "BDT303" – re:Invent 2014
23. AWS Gaming Solutions
• Collect Server Logs
• Periodically Send to S3
• Process into Redshift
• External Analytics Data Too
More Data Sources
EC2
External
Analytics
25. AWS Gaming Solutions
• Different File Formats
• Device vs Apache vs CDN
• Cleanup with EMR Job
• Output to Clean Bucket
• Load into Redshift
Dealing With Messy Data
EC2
26. AWS Gaming Solutions
Redshift vs Elastic MapReduce
Redshift
• Columnar DB
• Familiar SQL
• Structured Data
• Batch Load
• Faster to Query
• Long-term Storage
Elastic MapReduce
• Hadoop
• Custom Java / Python
• Unstructured Data
• Streaming Loop
• Scales > PB's
• Transient
27. AWS Gaming Solutions
• Game Servers Uses DynamoDB
• Directly Export to Redshift
• Or Stage to S3
Integrate With Your Game Database
EC2
DynamoDB
33. AWS Gaming Solutions
# of Likes /Shares
Player Abort
Rates per Level
# of Plays per Hour
In-app Item Popularity
Track Custom Events
34. AWS Gaming Solutions
Custom Events
public
void
onLevelComplete(String
levelName,
String
difficulty,
double
timeToComplete,
int
playerState)
{
//
Create
a
Level
Complete
event
with
some
attributes
and
metrics
AnalyticsEvent
levelCompleteEvent
=
analytics.getEventClient().createEvent("LevelComplete")
.withAttribute("LevelName",
levelName)
.withAttribute("Difficulty",
difficulty)
.withAttribute("EndState",
playerState)
.withMetric("TimeToComplete",
timeToComplete);
//Record
the
Level
Complete
event
analytics.getEventClient().recordEvent(levelCompleteEvent);
}
35. AWS Gaming Solutions
Amazon
Mobile
Analytics
Fast
Event data is processed in < 60 minutes
Affordable
Free 100 MM Events per Month
$1 per MM Events beyond the free tier
Private
You own your data
We do not use your data, or share with 3rd parties
37. AWS Gaming Solutions
• Collect Events w/ Mobile Analytics SDK
• Auto-Export to S3
• Process Data into Redshift
• Analyze with GUI Visualization Tool
Mobile Analytics Auto-Export to S3
38. AWS Gaming Solutions
• Collect Events w/ Mobile Analytics SDK
• Auto-Export Directly to Redshift
• Analyze with GUI Visualization Tool
Start Simple
42. AWS Gaming Solutions
Exporting to Amazon Redshift
172.16.0.0/20
Public Subnet 172.16.0.0/22
172.16.0.0/20 Local
0.0.0.0/0 IGW
Amazon
Mobile
Analytics
EC2
45. AWS Gaming Solutions
DAU and MAU in SQL
select
date,
count(*)
over
(partition
by
date_trunc('day',
date)
order
by
date)
as
dau,
count(*)
over
(partition
by
date_trunc('month',
date)
order
by
date)
as
mau
from
user_sessions
order
by
date;
46. AWS Gaming Solutions
Measure Retention: Repeated Plays
create
view
events_by_user_by_month
as
select
user_id,
date_trunc('month',
event_date)
as
month_active,
count(*)
as
total_events
from
events
group
by
user_id,
month_active;
48. AWS Gaming Solutions
Cohorts & Cambria
• Enables calculating relative metrics
• Group users by a common attribute
– Month game installed
– Demographics
• Run analysis by cohort
– Join with metrics
• Use Redshift as it's SQL
– Example of where SQL is a good fit
49. AWS Gaming Solutions
Creating Cohorts with Redshift
create
view
cohort_by_first_event_date
as
select
user_id,
date_trunc('month',
min(event_date))
as
first_month
from
events
group
by
user_id;
http://snowplowanalytics.com/analytics/customer-
analytics/cohort-analysis.html
51. AWS Gaming Solutions
Moar Cohorts
• Define multiple cohorts
– By activity, time, demographics
– As many as you like
• Change cohort depending on analysis
• Join same metrics with different cohorts
– Retention by date
– Retention by demographic
– Retention by average plays/month quartile
52.
53. AWS Gaming Solutions
5-9 notifications retain 1.5x better than 10-14 notifications
Forza: Push Notifications vs Retention
55. AWS Gaming Solutions
Data Collection
• Mobile Devices
• Game Servers
• Ad Networks
• Size of event ~ 1 KB
• 500M+ events/day
• 500G+/day & growing
• JSON format
Source of Data Data Size & Growth
56. AWS Gaming Solutions
Redshift Schema
• Every game has its own database
• Each game event = table (e.g., battle_fight, iap)
• 40-50 tables per DB schema
• All game titles ~ 1000 tables in DW
57. AWS Gaming Solutions
{"player_id":"323726381807586881","player_level":169,"device":"iPhone
5","version":"iOS 7.1.2”,"platfrom":"ios","client_build":"440”,
"db":”mw_dw_ios","table":"player_login",
"uuid":"1414566719-rsl3hvhu7o","time_created":"2014-10-29 00:11:59”}
{"fight_time":"2014-10-29 00:11:59","attacker_id":"413142642356365377”,
"attacker_clan_size":500,"attacker_level":270,
"db":"mw_dw_ios","table":"battle_fight",
"uuid":"1414566719-p0oogk0bep","time_created":"2014-10-29 00:11:59"}
• PHP/Python Web Services
• Client side analytics SDK
• JSON data format
Data Stream
58. AWS Gaming Solutions
Analytics Store Architecture
S3
S3
Consumer
Game DB
Game
Servers
Kinesis
Amazon
Redshift
Elastic
MapReduce
DSV
JSON