Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Currently on the Cloud
1 Our situation in 2014
2 How we improved
3 Sweet things
4 More sweet things
5 Future
1 Our situation in 2014
2 How we improved
3 Sweet things
4 More sweet things
5 Future
Our situation in 2014
1st gen
“HSP” – (2014) from cybercafe/漫画木さ/PC방 platform
2nd gen
ECDH-based key exchange
Platform
Bil...
Globalization
Issues abroad
Loading…
Loading…
Loading…
Loading…
Fail!
Process
Dev/QA/Sandbox/REAL…
Get VM
Get L4 binding
ACLs/storage, etc.
From hours to days…
Thus,
the Game Cloud project began…
1 Our situation in 2014
2 How we improved
3 Sweet things
4 More sweet things
5 Future
Global
In our case
GSLB (Global Server Load Balancing)
HAProxy instead of hardware L4
Multi-team effort
client, server, cl...
LINE Global POP
New York
Tokyo
Seoul
HK
Singapole
Beijing
Frankfurt
a
a
Network layer control
High latency
Fit for cloud
Global
Testing in Thailand
TH  KR
GSLB
TH  SG  KR
HAProxy
500
1000
150...
Process
Dev
Ops
Government
The structure of your organization affects the structure of your software.
And vice-versa!
DevO...
Process
Progressive / Easy, simple
Requirements for our new platform :
We have many third parties and technology stacks in...
Process
For ourselves
simple reliable future proof
Process
Why not…
Docker Swarm
or CoreOS
Mesos Kubernetes
Process
KEEP THINGS SIMPLE AND RELIABLE!
For distributed systems, minimize coordination
A good paper: https://blog.acolyer...
LGC Story
Games planned for release were suddenly canceled
but we needed to show results!
Strong “sales” efforts to releas...
Putting Out Fires
The release was a success,followed by a
quick scaling-up, and then our first fires…
TECHNICAL
Riak fire ...
1 Our situation in 2014
2 How we improved
3 Sweet things
4 More sweet things
5 Future
Domain HAP
Launch
service
Configure/
load balance
Expose
ports
Bind
URL
In one click!
Monitoring
1 Our situation in 2014
2 How we improved
3 Sweet things
4 More sweet things
5 Future
Gearbox
Auto Scaling System
High
availability
Low cost
Why Do We Need It?
How Does It Work?
How Did We Build it?
Data Collector
Monitoring API
Predicator
Metrics
Raw Metrics
Scaler
States
Game Cloud API
Excute Scal...
Challenges
Complex query
Plenty of metrics
records
millions of records
per day
Scalability of the
auto-scaling
system itse...
Solutions
Data Collector
Monitoring API
Predicator
Metrics
Metrics
Scaler
States
Game Cloud API
Excute
Scaling
Gearbox
Mod...
Knife
Admin Site
Deploying a
New Service
Upgrade
Configuring the
Auto-Scaling
Policy
Back to Jojo
What’s coming next!
1 Our situation in 2014
2 How we improved
3 Sweet things
4 More sweet things
5 Future
Future
QUIC
SDN
ACL, IP by container, VLAN etc.
Cloud storage
TDI
Distributed GC – link paper
DCTCP
Image GC
Future
UX
Helpers/presets
Speed
Doc/tests/guides…
Reliability
QUIC
Quick
UDP
Internet
Connections
Cloud Storage
SDN- Software-Defined Networking
Container
specific IP
ACLs VLANs
TDI- Test-Driven Infrastructure
hardware OS
configuration
images
backup/
restore
Automated testing for
Etc.
firmware,
vers...
Distributed GC
Max
Avg.GC pause
http://arxiv.org/pdf/1504.02578.pdf
Median
Std.Dev.
Mean
7.847
0.0
2.296
0.579
2.312
GC of...
Added in Linux 3.18
https://kernelnewbies.org/
Linux_3.18
http://simula.stanford.edu/
~alizade/Site/DCTCP.html
DCTCP- Data...
Distributed GC
Because we generate
tons of
Docker images
And more and more…
• AP
• Optional CP
• Index/search
• CRDT
• Multiple backends
• User ACL support
RIAK/Choose a Safe and Simple Friend
Make ...
DataScript /
Maintain
Queries
B 2 line game cloud - our personal ec2
B 2 line game cloud - our personal ec2
B 2 line game cloud - our personal ec2
B 2 line game cloud - our personal ec2
B 2 line game cloud - our personal ec2
B 2 line game cloud - our personal ec2
B 2 line game cloud - our personal ec2
B 2 line game cloud - our personal ec2
B 2 line game cloud - our personal ec2
B 2 line game cloud - our personal ec2
Upcoming SlideShare
Loading in …5
×

B 2 line game cloud - our personal ec2

8,166 views

Published on

TBD

Published in: Technology
  • Be the first to comment

B 2 line game cloud - our personal ec2

  1. 1. Currently on the Cloud
  2. 2. 1 Our situation in 2014 2 How we improved 3 Sweet things 4 More sweet things 5 Future
  3. 3. 1 Our situation in 2014 2 How we improved 3 Sweet things 4 More sweet things 5 Future
  4. 4. Our situation in 2014 1st gen “HSP” – (2014) from cybercafe/漫画木さ/PC방 platform 2nd gen ECDH-based key exchange Platform Billing/AAA/Monitoring, etc. & Game servers “LGC” – (cloud release) 3rd gen: “Trident” – (current)
  5. 5. Globalization Issues abroad Loading… Loading… Loading… Loading… Fail!
  6. 6. Process Dev/QA/Sandbox/REAL… Get VM Get L4 binding ACLs/storage, etc. From hours to days…
  7. 7. Thus, the Game Cloud project began…
  8. 8. 1 Our situation in 2014 2 How we improved 3 Sweet things 4 More sweet things 5 Future
  9. 9. Global In our case GSLB (Global Server Load Balancing) HAProxy instead of hardware L4 Multi-team effort client, server, cloud We get more flexibility less latency
  10. 10. LINE Global POP New York Tokyo Seoul HK Singapole Beijing Frankfurt a a
  11. 11. Network layer control High latency Fit for cloud Global Testing in Thailand TH  KR GSLB TH  SG  KR HAProxy 500 1000 1500 2 12 206 16
  12. 12. Process Dev Ops Government The structure of your organization affects the structure of your software. And vice-versa! DevOps Small StartupCHOOSE
  13. 13. Process Progressive / Easy, simple Requirements for our new platform : We have many third parties and technology stacks involved... Etc.
  14. 14. Process For ourselves simple reliable future proof
  15. 15. Process Why not… Docker Swarm or CoreOS Mesos Kubernetes
  16. 16. Process KEEP THINGS SIMPLE AND RELIABLE! For distributed systems, minimize coordination A good paper: https://blog.acolyer.org/2016/01/19/dcft/ • Polling • 1way dataflow • Idempotency • Commutativity • limited trust
  17. 17. LGC Story Games planned for release were suddenly canceled but we needed to show results! Strong “sales” efforts to release other games on the LGC platform
  18. 18. Putting Out Fires The release was a success,followed by a quick scaling-up, and then our first fires… TECHNICAL Riak fire : the system works with Riak down OE fire: the system works with OE down Hardware and conf fires (TDI! Soon to come!) Full container reboot improved our design through limited trust
  19. 19. 1 Our situation in 2014 2 How we improved 3 Sweet things 4 More sweet things 5 Future
  20. 20. Domain HAP Launch service Configure/ load balance Expose ports Bind URL In one click!
  21. 21. Monitoring
  22. 22. 1 Our situation in 2014 2 How we improved 3 Sweet things 4 More sweet things 5 Future
  23. 23. Gearbox Auto Scaling System
  24. 24. High availability Low cost Why Do We Need It?
  25. 25. How Does It Work?
  26. 26. How Did We Build it? Data Collector Monitoring API Predicator Metrics Raw Metrics Scaler States Game Cloud API Excute Scaling Gearbox
  27. 27. Challenges Complex query Plenty of metrics records millions of records per day Scalability of the auto-scaling system itself
  28. 28. Solutions Data Collector Monitoring API Predicator Metrics Metrics Scaler States Game Cloud API Excute Scaling Gearbox Module Storage ElasticSearchAdmin Site Strategy Scaler Log Metrics 1.Strategy 2.Metrics { } Predicator Log 1.Strategy 2.Scaler Log
  29. 29. Knife Admin Site
  30. 30. Deploying a New Service
  31. 31. Upgrade
  32. 32. Configuring the Auto-Scaling Policy
  33. 33. Back to Jojo What’s coming next!
  34. 34. 1 Our situation in 2014 2 How we improved 3 Sweet things 4 More sweet things 5 Future
  35. 35. Future QUIC SDN ACL, IP by container, VLAN etc. Cloud storage TDI Distributed GC – link paper DCTCP Image GC
  36. 36. Future UX Helpers/presets Speed Doc/tests/guides… Reliability
  37. 37. QUIC Quick UDP Internet Connections
  38. 38. Cloud Storage
  39. 39. SDN- Software-Defined Networking Container specific IP ACLs VLANs
  40. 40. TDI- Test-Driven Infrastructure hardware OS configuration images backup/ restore Automated testing for Etc. firmware, version, etc.
  41. 41. Distributed GC Max Avg.GC pause http://arxiv.org/pdf/1504.02578.pdf Median Std.Dev. Mean 7.847 0.0 2.296 0.579 2.312 GC off 7.743 12.243 2.294 0.582 2.311 Blade 164.206 12.339 2.297 3.395 2.403 GC on
  42. 42. Added in Linux 3.18 https://kernelnewbies.org/ Linux_3.18 http://simula.stanford.edu/ ~alizade/Site/DCTCP.html DCTCP- Data Center TCP - high burst tolerance - low latency - high throughput http://simula.stanford.edu/~alizade/Site/DCTCP.html
  43. 43. Distributed GC Because we generate tons of Docker images
  44. 44. And more and more…
  45. 45. • AP • Optional CP • Index/search • CRDT • Multiple backends • User ACL support RIAK/Choose a Safe and Simple Friend Make a deliberate choice of consistency model SQL NoSQL But actually… With the authorization of Kingsbury Kyle (Aphyr)
  46. 46. DataScript / Maintain Queries

×