SlideShare a Scribd company logo

Scaling Social Games

Short talk given at the Berlin hadoop get together on the 27th of january 2011

1 of 49
Download to read offline
Scaling social games
   “the order of magnitude
          challenge”
  Paolo Negri @hungryblank
Order of magnitude
                                      DAU

                     1000000

                      750000
DAU:
                      500000
daily active users
                      250000

                           0
                               July         December
Social Games
Flash client (game)    HTTP API




                      http://www.flickr.com/photos/stars6/4381851322
Social Games
Flash client


               • Game actions need to be
                 persisted and validated

               • 1 API call every few secs
Social Games
                          HTTP API



• 5000 HTTP reqs/sec
• more than 90% writes
• 60K queries/sec


                         http://www.flickr.com/photos/stars6/4381851322
July 2010
                                HAproxy
• ~ 170 000 daily users
• Plain Ruby on Rails app
• Persistency 100% SQL        Ruby on Rails




                                MySQL

Recommended

Mongrel2, a short introduction
Mongrel2, a short introductionMongrel2, a short introduction
Mongrel2, a short introductionPaolo Negri
 
Pulsar Summit Asia - Structured Data Stream with Apache Pulsar
Pulsar Summit Asia - Structured Data Stream with Apache PulsarPulsar Summit Asia - Structured Data Stream with Apache Pulsar
Pulsar Summit Asia - Structured Data Stream with Apache PulsarShivji Kumar Jha
 
lessons from managing a pulsar cluster
 lessons from managing a pulsar cluster lessons from managing a pulsar cluster
lessons from managing a pulsar clusterShivji Kumar Jha
 
大規模環境でRailsと4年間付き合ってきて@ クックパッド * 食べログ合同勉強会
大規模環境でRailsと4年間付き合ってきて@ クックパッド * 食べログ合同勉強会大規模環境でRailsと4年間付き合ってきて@ クックパッド * 食べログ合同勉強会
大規模環境でRailsと4年間付き合ってきて@ クックパッド * 食べログ合同勉強会Takayuki Kyowa
 
Xen_and_Rails_deployment
Xen_and_Rails_deploymentXen_and_Rails_deployment
Xen_and_Rails_deploymentAbhishek Singh
 
How pulsar stores data at Pulsar-na-summit-2021.pptx (1)
How pulsar stores data at Pulsar-na-summit-2021.pptx (1)How pulsar stores data at Pulsar-na-summit-2021.pptx (1)
How pulsar stores data at Pulsar-na-summit-2021.pptx (1)Shivji Kumar Jha
 
Cassandra: An Alien Technology That's not so Alien
Cassandra: An Alien Technology That's not so AlienCassandra: An Alien Technology That's not so Alien
Cassandra: An Alien Technology That's not so AlienBrian Hess
 
XMPP/Jingle(VoIP)/Perl Ocean 2012/03
XMPP/Jingle(VoIP)/Perl Ocean 2012/03XMPP/Jingle(VoIP)/Perl Ocean 2012/03
XMPP/Jingle(VoIP)/Perl Ocean 2012/03Lyo Kato
 

More Related Content

What's hot

Training Slides: 101 - Basics: Tungsten Clustering - Under The Hood
Training Slides: 101 - Basics: Tungsten Clustering - Under The HoodTraining Slides: 101 - Basics: Tungsten Clustering - Under The Hood
Training Slides: 101 - Basics: Tungsten Clustering - Under The HoodContinuent
 
How you can contribute to Apache Cassandra
How you can contribute to Apache CassandraHow you can contribute to Apache Cassandra
How you can contribute to Apache CassandraYuki Morishita
 
A multi-tenant architecture for Apache Axis2
A multi-tenant architecture for Apache Axis2A multi-tenant architecture for Apache Axis2
A multi-tenant architecture for Apache Axis2Afkham Azeez
 
Maria DB Galera Cluster for High Availability
Maria DB Galera Cluster for High AvailabilityMaria DB Galera Cluster for High Availability
Maria DB Galera Cluster for High AvailabilityOSSCube
 
Cassandra Summit 2014: Deploying Cassandra for Call of Duty
Cassandra Summit 2014: Deploying Cassandra for Call of DutyCassandra Summit 2014: Deploying Cassandra for Call of Duty
Cassandra Summit 2014: Deploying Cassandra for Call of DutyDataStax Academy
 
Client Drivers and Cassandra, the Right Way
Client Drivers and Cassandra, the Right WayClient Drivers and Cassandra, the Right Way
Client Drivers and Cassandra, the Right WayDataStax Academy
 
Achieving Infrastructure Portability with Chef
Achieving Infrastructure Portability with ChefAchieving Infrastructure Portability with Chef
Achieving Infrastructure Portability with ChefMatt Ray
 
under the covers -- chef in 20 minutes or less
under the covers -- chef in 20 minutes or lessunder the covers -- chef in 20 minutes or less
under the covers -- chef in 20 minutes or lesssarahnovotny
 
Introduction to Apache Kafka
Introduction to Apache KafkaIntroduction to Apache Kafka
Introduction to Apache KafkaShiao-An Yuan
 
Redis everywhere - PHP London
Redis everywhere - PHP LondonRedis everywhere - PHP London
Redis everywhere - PHP LondonRicard Clau
 
Data Models and Consumer Idioms Using Apache Kafka for Continuous Data Stream...
Data Models and Consumer Idioms Using Apache Kafka for Continuous Data Stream...Data Models and Consumer Idioms Using Apache Kafka for Continuous Data Stream...
Data Models and Consumer Idioms Using Apache Kafka for Continuous Data Stream...Erik Onnen
 
Master master vs master-slave database
Master master vs master-slave databaseMaster master vs master-slave database
Master master vs master-slave databaseWipro
 
События, шины и интеграция данных в непростом мире микросервисов / Валентин Г...
События, шины и интеграция данных в непростом мире микросервисов / Валентин Г...События, шины и интеграция данных в непростом мире микросервисов / Валентин Г...
События, шины и интеграция данных в непростом мире микросервисов / Валентин Г...Ontico
 
Speed up your Symfony2 application and build awesome features with Redis
Speed up your Symfony2 application and build awesome features with RedisSpeed up your Symfony2 application and build awesome features with Redis
Speed up your Symfony2 application and build awesome features with RedisRicard Clau
 
Developing with the Go client for Apache Kafka
Developing with the Go client for Apache KafkaDeveloping with the Go client for Apache Kafka
Developing with the Go client for Apache KafkaJoe Stein
 
Hashicorp: Delivering the Tao of DevOps
Hashicorp: Delivering the Tao of DevOpsHashicorp: Delivering the Tao of DevOps
Hashicorp: Delivering the Tao of DevOpsRamit Surana
 
Introduction Apache Kafka
Introduction Apache KafkaIntroduction Apache Kafka
Introduction Apache KafkaJoe Stein
 

What's hot (17)

Training Slides: 101 - Basics: Tungsten Clustering - Under The Hood
Training Slides: 101 - Basics: Tungsten Clustering - Under The HoodTraining Slides: 101 - Basics: Tungsten Clustering - Under The Hood
Training Slides: 101 - Basics: Tungsten Clustering - Under The Hood
 
How you can contribute to Apache Cassandra
How you can contribute to Apache CassandraHow you can contribute to Apache Cassandra
How you can contribute to Apache Cassandra
 
A multi-tenant architecture for Apache Axis2
A multi-tenant architecture for Apache Axis2A multi-tenant architecture for Apache Axis2
A multi-tenant architecture for Apache Axis2
 
Maria DB Galera Cluster for High Availability
Maria DB Galera Cluster for High AvailabilityMaria DB Galera Cluster for High Availability
Maria DB Galera Cluster for High Availability
 
Cassandra Summit 2014: Deploying Cassandra for Call of Duty
Cassandra Summit 2014: Deploying Cassandra for Call of DutyCassandra Summit 2014: Deploying Cassandra for Call of Duty
Cassandra Summit 2014: Deploying Cassandra for Call of Duty
 
Client Drivers and Cassandra, the Right Way
Client Drivers and Cassandra, the Right WayClient Drivers and Cassandra, the Right Way
Client Drivers and Cassandra, the Right Way
 
Achieving Infrastructure Portability with Chef
Achieving Infrastructure Portability with ChefAchieving Infrastructure Portability with Chef
Achieving Infrastructure Portability with Chef
 
under the covers -- chef in 20 minutes or less
under the covers -- chef in 20 minutes or lessunder the covers -- chef in 20 minutes or less
under the covers -- chef in 20 minutes or less
 
Introduction to Apache Kafka
Introduction to Apache KafkaIntroduction to Apache Kafka
Introduction to Apache Kafka
 
Redis everywhere - PHP London
Redis everywhere - PHP LondonRedis everywhere - PHP London
Redis everywhere - PHP London
 
Data Models and Consumer Idioms Using Apache Kafka for Continuous Data Stream...
Data Models and Consumer Idioms Using Apache Kafka for Continuous Data Stream...Data Models and Consumer Idioms Using Apache Kafka for Continuous Data Stream...
Data Models and Consumer Idioms Using Apache Kafka for Continuous Data Stream...
 
Master master vs master-slave database
Master master vs master-slave databaseMaster master vs master-slave database
Master master vs master-slave database
 
События, шины и интеграция данных в непростом мире микросервисов / Валентин Г...
События, шины и интеграция данных в непростом мире микросервисов / Валентин Г...События, шины и интеграция данных в непростом мире микросервисов / Валентин Г...
События, шины и интеграция данных в непростом мире микросервисов / Валентин Г...
 
Speed up your Symfony2 application and build awesome features with Redis
Speed up your Symfony2 application and build awesome features with RedisSpeed up your Symfony2 application and build awesome features with Redis
Speed up your Symfony2 application and build awesome features with Redis
 
Developing with the Go client for Apache Kafka
Developing with the Go client for Apache KafkaDeveloping with the Go client for Apache Kafka
Developing with the Go client for Apache Kafka
 
Hashicorp: Delivering the Tao of DevOps
Hashicorp: Delivering the Tao of DevOpsHashicorp: Delivering the Tao of DevOps
Hashicorp: Delivering the Tao of DevOps
 
Introduction Apache Kafka
Introduction Apache KafkaIntroduction Apache Kafka
Introduction Apache Kafka
 

Viewers also liked

A Documentation Crash Course, LinuxCon 2016
A Documentation Crash Course, LinuxCon 2016A Documentation Crash Course, LinuxCon 2016
A Documentation Crash Course, LinuxCon 2016Chris Ward
 
Content Management Systems and Refactoring - Drupal, WordPress and eZ Publish
Content Management Systems and Refactoring - Drupal, WordPress and eZ PublishContent Management Systems and Refactoring - Drupal, WordPress and eZ Publish
Content Management Systems and Refactoring - Drupal, WordPress and eZ PublishJani Tarvainen
 
Electron - Solving our cross platform dreams?
Electron - Solving our cross platform dreams?Electron - Solving our cross platform dreams?
Electron - Solving our cross platform dreams?Chris Ward
 
Entrez dans le mouvement Maker à l’aide des technologies Microsoft
Entrez dans le mouvement Maker à l’aide des technologies MicrosoftEntrez dans le mouvement Maker à l’aide des technologies Microsoft
Entrez dans le mouvement Maker à l’aide des technologies MicrosoftFabrice BARBIN
 
SimpleDb, an introduction
SimpleDb, an introductionSimpleDb, an introduction
SimpleDb, an introductionPaolo Negri
 
Why you should come to DrupalSouth
Why you should come to DrupalSouthWhy you should come to DrupalSouth
Why you should come to DrupalSouthChris Ward
 
Offre développeur Javascript Back-end
Offre développeur Javascript Back-endOffre développeur Javascript Back-end
Offre développeur Javascript Back-endSite Analyzer
 
Automate your docs, automate yourself
Automate your docs, automate yourselfAutomate your docs, automate yourself
Automate your docs, automate yourselfChris Ward
 
Erlang introduction geek2geek Berlin
Erlang introduction geek2geek BerlinErlang introduction geek2geek Berlin
Erlang introduction geek2geek BerlinPaolo Negri
 
Contentful Berlin Offices
Contentful Berlin OfficesContentful Berlin Offices
Contentful Berlin OfficesIrina Botea
 
The Anatomy of Content Management (workshop by J Gollner at Intelligent Conte...
The Anatomy of Content Management (workshop by J Gollner at Intelligent Conte...The Anatomy of Content Management (workshop by J Gollner at Intelligent Conte...
The Anatomy of Content Management (workshop by J Gollner at Intelligent Conte...Joe Gollner
 
AWS Lambda in infrastructure
AWS Lambda in infrastructureAWS Lambda in infrastructure
AWS Lambda in infrastructurePaolo Negri
 
Le futur de Drupal et des applications web
Le futur de Drupal et des applications webLe futur de Drupal et des applications web
Le futur de Drupal et des applications webJulien Dubreuil
 
Distributed and concurrent programming with RabbitMQ and EventMachine Rails U...
Distributed and concurrent programming with RabbitMQ and EventMachine Rails U...Distributed and concurrent programming with RabbitMQ and EventMachine Rails U...
Distributed and concurrent programming with RabbitMQ and EventMachine Rails U...Paolo Negri
 
Devoxx France 2015 - Se préparer à l'arrivée d'Angular 2
Devoxx France 2015 - Se préparer à l'arrivée d'Angular 2Devoxx France 2015 - Se préparer à l'arrivée d'Angular 2
Devoxx France 2015 - Se préparer à l'arrivée d'Angular 2Romain Linsolas
 
Back to the future with static site generators
Back to the future with static site generatorsBack to the future with static site generators
Back to the future with static site generatorsChris Ward
 
ParisJS #10 : PhantomJs
ParisJS #10 : PhantomJs ParisJS #10 : PhantomJs
ParisJS #10 : PhantomJs Maurice Svay
 
Erlang as a cloud citizen, a fractal approach to throughput
Erlang as a cloud citizen, a fractal approach to throughputErlang as a cloud citizen, a fractal approach to throughput
Erlang as a cloud citizen, a fractal approach to throughputPaolo Negri
 
Google : Prise en charge de l'Ajax et de l'Angular JS
Google : Prise en charge de l'Ajax et de l'Angular JSGoogle : Prise en charge de l'Ajax et de l'Angular JS
Google : Prise en charge de l'Ajax et de l'Angular JSPeak Ace
 
API Days Australia - Automatic Testing of (RESTful) API Documentation
API Days Australia  - Automatic Testing of (RESTful) API DocumentationAPI Days Australia  - Automatic Testing of (RESTful) API Documentation
API Days Australia - Automatic Testing of (RESTful) API DocumentationRouven Weßling
 

Viewers also liked (20)

A Documentation Crash Course, LinuxCon 2016
A Documentation Crash Course, LinuxCon 2016A Documentation Crash Course, LinuxCon 2016
A Documentation Crash Course, LinuxCon 2016
 
Content Management Systems and Refactoring - Drupal, WordPress and eZ Publish
Content Management Systems and Refactoring - Drupal, WordPress and eZ PublishContent Management Systems and Refactoring - Drupal, WordPress and eZ Publish
Content Management Systems and Refactoring - Drupal, WordPress and eZ Publish
 
Electron - Solving our cross platform dreams?
Electron - Solving our cross platform dreams?Electron - Solving our cross platform dreams?
Electron - Solving our cross platform dreams?
 
Entrez dans le mouvement Maker à l’aide des technologies Microsoft
Entrez dans le mouvement Maker à l’aide des technologies MicrosoftEntrez dans le mouvement Maker à l’aide des technologies Microsoft
Entrez dans le mouvement Maker à l’aide des technologies Microsoft
 
SimpleDb, an introduction
SimpleDb, an introductionSimpleDb, an introduction
SimpleDb, an introduction
 
Why you should come to DrupalSouth
Why you should come to DrupalSouthWhy you should come to DrupalSouth
Why you should come to DrupalSouth
 
Offre développeur Javascript Back-end
Offre développeur Javascript Back-endOffre développeur Javascript Back-end
Offre développeur Javascript Back-end
 
Automate your docs, automate yourself
Automate your docs, automate yourselfAutomate your docs, automate yourself
Automate your docs, automate yourself
 
Erlang introduction geek2geek Berlin
Erlang introduction geek2geek BerlinErlang introduction geek2geek Berlin
Erlang introduction geek2geek Berlin
 
Contentful Berlin Offices
Contentful Berlin OfficesContentful Berlin Offices
Contentful Berlin Offices
 
The Anatomy of Content Management (workshop by J Gollner at Intelligent Conte...
The Anatomy of Content Management (workshop by J Gollner at Intelligent Conte...The Anatomy of Content Management (workshop by J Gollner at Intelligent Conte...
The Anatomy of Content Management (workshop by J Gollner at Intelligent Conte...
 
AWS Lambda in infrastructure
AWS Lambda in infrastructureAWS Lambda in infrastructure
AWS Lambda in infrastructure
 
Le futur de Drupal et des applications web
Le futur de Drupal et des applications webLe futur de Drupal et des applications web
Le futur de Drupal et des applications web
 
Distributed and concurrent programming with RabbitMQ and EventMachine Rails U...
Distributed and concurrent programming with RabbitMQ and EventMachine Rails U...Distributed and concurrent programming with RabbitMQ and EventMachine Rails U...
Distributed and concurrent programming with RabbitMQ and EventMachine Rails U...
 
Devoxx France 2015 - Se préparer à l'arrivée d'Angular 2
Devoxx France 2015 - Se préparer à l'arrivée d'Angular 2Devoxx France 2015 - Se préparer à l'arrivée d'Angular 2
Devoxx France 2015 - Se préparer à l'arrivée d'Angular 2
 
Back to the future with static site generators
Back to the future with static site generatorsBack to the future with static site generators
Back to the future with static site generators
 
ParisJS #10 : PhantomJs
ParisJS #10 : PhantomJs ParisJS #10 : PhantomJs
ParisJS #10 : PhantomJs
 
Erlang as a cloud citizen, a fractal approach to throughput
Erlang as a cloud citizen, a fractal approach to throughputErlang as a cloud citizen, a fractal approach to throughput
Erlang as a cloud citizen, a fractal approach to throughput
 
Google : Prise en charge de l'Ajax et de l'Angular JS
Google : Prise en charge de l'Ajax et de l'Angular JSGoogle : Prise en charge de l'Ajax et de l'Angular JS
Google : Prise en charge de l'Ajax et de l'Angular JS
 
API Days Australia - Automatic Testing of (RESTful) API Documentation
API Days Australia  - Automatic Testing of (RESTful) API DocumentationAPI Days Australia  - Automatic Testing of (RESTful) API Documentation
API Days Australia - Automatic Testing of (RESTful) API Documentation
 

Similar to Scaling Social Games

Getting started with Riak in the Cloud
Getting started with Riak in the CloudGetting started with Riak in the Cloud
Getting started with Riak in the CloudInes Sombra
 
Chirp 2010: Scaling Twitter
Chirp 2010: Scaling TwitterChirp 2010: Scaling Twitter
Chirp 2010: Scaling TwitterJohn Adams
 
Scaling the Web: Databases & NoSQL
Scaling the Web: Databases & NoSQLScaling the Web: Databases & NoSQL
Scaling the Web: Databases & NoSQLRichard Schneeman
 
Scaling with swagger
Scaling with swaggerScaling with swagger
Scaling with swaggerTony Tam
 
Scaling MySQL using Fabric
Scaling MySQL using FabricScaling MySQL using Fabric
Scaling MySQL using FabricKarthik .P.R
 
Scaling MySQL Using Fabric
Scaling MySQL Using FabricScaling MySQL Using Fabric
Scaling MySQL Using FabricRemote MySQL DBA
 
Stack Exchange Infrastructure - LISA 14
Stack Exchange Infrastructure - LISA 14Stack Exchange Infrastructure - LISA 14
Stack Exchange Infrastructure - LISA 14GABeech
 
High Performance WordPress II
High Performance WordPress IIHigh Performance WordPress II
High Performance WordPress IIBarry Abrahamson
 
A closer look to locaweb IaaS
A closer look to locaweb IaaSA closer look to locaweb IaaS
A closer look to locaweb IaaSGleicon Moraes
 
Webinar - DreamObjects/Ceph Case Study
Webinar - DreamObjects/Ceph Case StudyWebinar - DreamObjects/Ceph Case Study
Webinar - DreamObjects/Ceph Case StudyCeph Community
 
Service-Oriented Design and Implement with Rails3
Service-Oriented Design and Implement with Rails3Service-Oriented Design and Implement with Rails3
Service-Oriented Design and Implement with Rails3Wen-Tien Chang
 
Mindtalk Tech - Behind the scenes
Mindtalk Tech - Behind the scenesMindtalk Tech - Behind the scenes
Mindtalk Tech - Behind the scenesrobin_sy
 
RedisDay London 2018 - Stack Overflow's Next Steps in Redis
RedisDay London 2018 - Stack Overflow's Next Steps in RedisRedisDay London 2018 - Stack Overflow's Next Steps in Redis
RedisDay London 2018 - Stack Overflow's Next Steps in RedisRedis Labs
 
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #4: MS Azure Database MySQL
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #4: MS Azure Database MySQLWebinar Slides: MySQL HA/DR/Geo-Scale - High Noon #4: MS Azure Database MySQL
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #4: MS Azure Database MySQLContinuent
 
Big Data Developers Moscow Meetup 1 - sql on hadoop
Big Data Developers Moscow Meetup 1  - sql on hadoopBig Data Developers Moscow Meetup 1  - sql on hadoop
Big Data Developers Moscow Meetup 1 - sql on hadoopbddmoscow
 

Similar to Scaling Social Games (20)

Getting started with Riak in the Cloud
Getting started with Riak in the CloudGetting started with Riak in the Cloud
Getting started with Riak in the Cloud
 
Chirp 2010: Scaling Twitter
Chirp 2010: Scaling TwitterChirp 2010: Scaling Twitter
Chirp 2010: Scaling Twitter
 
Scaling the Web: Databases & NoSQL
Scaling the Web: Databases & NoSQLScaling the Web: Databases & NoSQL
Scaling the Web: Databases & NoSQL
 
Scaling with swagger
Scaling with swaggerScaling with swagger
Scaling with swagger
 
Scaling MySQL using Fabric
Scaling MySQL using FabricScaling MySQL using Fabric
Scaling MySQL using Fabric
 
Scaling MySQL Using Fabric
Scaling MySQL Using FabricScaling MySQL Using Fabric
Scaling MySQL Using Fabric
 
Stack Exchange Infrastructure - LISA 14
Stack Exchange Infrastructure - LISA 14Stack Exchange Infrastructure - LISA 14
Stack Exchange Infrastructure - LISA 14
 
High Performance WordPress II
High Performance WordPress IIHigh Performance WordPress II
High Performance WordPress II
 
redis
redisredis
redis
 
A closer look to locaweb IaaS
A closer look to locaweb IaaSA closer look to locaweb IaaS
A closer look to locaweb IaaS
 
Webinar - DreamObjects/Ceph Case Study
Webinar - DreamObjects/Ceph Case StudyWebinar - DreamObjects/Ceph Case Study
Webinar - DreamObjects/Ceph Case Study
 
Why ruby and rails
Why ruby and railsWhy ruby and rails
Why ruby and rails
 
KeyValue Stores
KeyValue StoresKeyValue Stores
KeyValue Stores
 
Service-Oriented Design and Implement with Rails3
Service-Oriented Design and Implement with Rails3Service-Oriented Design and Implement with Rails3
Service-Oriented Design and Implement with Rails3
 
High availability
High availabilityHigh availability
High availability
 
MySQL in the Cloud
MySQL in the CloudMySQL in the Cloud
MySQL in the Cloud
 
Mindtalk Tech - Behind the scenes
Mindtalk Tech - Behind the scenesMindtalk Tech - Behind the scenes
Mindtalk Tech - Behind the scenes
 
RedisDay London 2018 - Stack Overflow's Next Steps in Redis
RedisDay London 2018 - Stack Overflow's Next Steps in RedisRedisDay London 2018 - Stack Overflow's Next Steps in Redis
RedisDay London 2018 - Stack Overflow's Next Steps in Redis
 
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #4: MS Azure Database MySQL
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #4: MS Azure Database MySQLWebinar Slides: MySQL HA/DR/Geo-Scale - High Noon #4: MS Azure Database MySQL
Webinar Slides: MySQL HA/DR/Geo-Scale - High Noon #4: MS Azure Database MySQL
 
Big Data Developers Moscow Meetup 1 - sql on hadoop
Big Data Developers Moscow Meetup 1  - sql on hadoopBig Data Developers Moscow Meetup 1  - sql on hadoop
Big Data Developers Moscow Meetup 1 - sql on hadoop
 

More from Paolo Negri

Turning the web stack upside down rethinking how data flows through systems
Turning the web stack upside down  rethinking how data flows through systemsTurning the web stack upside down  rethinking how data flows through systems
Turning the web stack upside down rethinking how data flows through systemsPaolo Negri
 
Getting real with erlang
Getting real with erlangGetting real with erlang
Getting real with erlangPaolo Negri
 
Erlang factory 2011 london
Erlang factory 2011 londonErlang factory 2011 london
Erlang factory 2011 londonPaolo Negri
 
Erlang factory SF 2011 "Erlang and the big switch in social games"
Erlang factory SF 2011 "Erlang and the big switch in social games"Erlang factory SF 2011 "Erlang and the big switch in social games"
Erlang factory SF 2011 "Erlang and the big switch in social games"Paolo Negri
 
RabbitMQ with python and ruby RuPy 2009
RabbitMQ with python and ruby RuPy 2009RabbitMQ with python and ruby RuPy 2009
RabbitMQ with python and ruby RuPy 2009Paolo Negri
 
%w(map reduce).first - A Tale About Rabbits, Latency, and Slim Crontabs
%w(map reduce).first - A Tale About Rabbits, Latency, and Slim Crontabs%w(map reduce).first - A Tale About Rabbits, Latency, and Slim Crontabs
%w(map reduce).first - A Tale About Rabbits, Latency, and Slim CrontabsPaolo Negri
 

More from Paolo Negri (6)

Turning the web stack upside down rethinking how data flows through systems
Turning the web stack upside down  rethinking how data flows through systemsTurning the web stack upside down  rethinking how data flows through systems
Turning the web stack upside down rethinking how data flows through systems
 
Getting real with erlang
Getting real with erlangGetting real with erlang
Getting real with erlang
 
Erlang factory 2011 london
Erlang factory 2011 londonErlang factory 2011 london
Erlang factory 2011 london
 
Erlang factory SF 2011 "Erlang and the big switch in social games"
Erlang factory SF 2011 "Erlang and the big switch in social games"Erlang factory SF 2011 "Erlang and the big switch in social games"
Erlang factory SF 2011 "Erlang and the big switch in social games"
 
RabbitMQ with python and ruby RuPy 2009
RabbitMQ with python and ruby RuPy 2009RabbitMQ with python and ruby RuPy 2009
RabbitMQ with python and ruby RuPy 2009
 
%w(map reduce).first - A Tale About Rabbits, Latency, and Slim Crontabs
%w(map reduce).first - A Tale About Rabbits, Latency, and Slim Crontabs%w(map reduce).first - A Tale About Rabbits, Latency, and Slim Crontabs
%w(map reduce).first - A Tale About Rabbits, Latency, and Slim Crontabs
 

Recently uploaded

Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...Product School
 
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...DianaGray10
 
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro KozhevinFwdays
 
Automate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center ExcellenceAutomate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center ExcellencePrecisely
 
How to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response PlanHow to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response PlanDatabarracks
 
"Platform Engineering with Development Containers", Igor Fesenko
"Platform Engineering with Development Containers", Igor Fesenko"Platform Engineering with Development Containers", Igor Fesenko
"Platform Engineering with Development Containers", Igor FesenkoFwdays
 
Confoo 2024 Gettings started with OpenAI and data science
Confoo 2024 Gettings started with OpenAI and data scienceConfoo 2024 Gettings started with OpenAI and data science
Confoo 2024 Gettings started with OpenAI and data scienceSusan Ibach
 
Bringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxBringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxMaarten Balliauw
 
The Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product SchoolThe Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product SchoolProduct School
 
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Adrian Sanabria
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build PolandGDSC PJATK
 
How AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptxHow AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptxInfosec
 
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...MarcovanHurne2
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaISPMAIndia
 
"Testing of Helm Charts or There and Back Again", Yura Rochniak
"Testing of Helm Charts or There and Back Again", Yura Rochniak"Testing of Helm Charts or There and Back Again", Yura Rochniak
"Testing of Helm Charts or There and Back Again", Yura RochniakFwdays
 
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...htrindia
 
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17Ana-Maria Mihalceanu
 
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERNRonnelBaroc
 
Artificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdfArtificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdfIsidro Navarro
 

Recently uploaded (20)

Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...
 
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
 
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
"DevOps Practisting Platform on EKS with Karpenter autoscaling", Dmytro Kozhevin
 
Automate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center ExcellenceAutomate Your Master Data Processes for Shared Service Center Excellence
Automate Your Master Data Processes for Shared Service Center Excellence
 
How to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response PlanHow to write an effective Cyber Incident Response Plan
How to write an effective Cyber Incident Response Plan
 
"Platform Engineering with Development Containers", Igor Fesenko
"Platform Engineering with Development Containers", Igor Fesenko"Platform Engineering with Development Containers", Igor Fesenko
"Platform Engineering with Development Containers", Igor Fesenko
 
Confoo 2024 Gettings started with OpenAI and data science
Confoo 2024 Gettings started with OpenAI and data scienceConfoo 2024 Gettings started with OpenAI and data science
Confoo 2024 Gettings started with OpenAI and data science
 
Bringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxBringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptx
 
The Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product SchoolThe Future of Product, by Founder & CEO, Product School
The Future of Product, by Founder & CEO, Product School
 
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
Early Tech Adoption: Foolish or Pragmatic? - 17th ISACA South Florida WOW Con...
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build Poland
 
How AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptxHow AI and ChatGPT are changing cybersecurity forever.pptx
How AI and ChatGPT are changing cybersecurity forever.pptx
 
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
 
"Testing of Helm Charts or There and Back Again", Yura Rochniak
"Testing of Helm Charts or There and Back Again", Yura Rochniak"Testing of Helm Charts or There and Back Again", Yura Rochniak
"Testing of Helm Charts or There and Back Again", Yura Rochniak
 
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
HBR SERIES METAL HOUSED RESISTORS POWER ELECTRICAL ABSORBS HIGH CURRENT DURIN...
 
In sharing we trust. Taking advantage of a diverse consortium to build a tran...
In sharing we trust. Taking advantage of a diverse consortium to build a tran...In sharing we trust. Taking advantage of a diverse consortium to build a tran...
In sharing we trust. Taking advantage of a diverse consortium to build a tran...
 
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
 
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
21ST CENTURY LITERACY FROM TRADITIONAL TO MODERN
 
Artificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdfArtificial-Intelligence-in-Marketing-Data.pdf
Artificial-Intelligence-in-Marketing-Data.pdf
 

Scaling Social Games

  • 1. Scaling social games “the order of magnitude challenge” Paolo Negri @hungryblank
  • 2. Order of magnitude DAU 1000000 750000 DAU: 500000 daily active users 250000 0 July December
  • 3. Social Games Flash client (game) HTTP API http://www.flickr.com/photos/stars6/4381851322
  • 4. Social Games Flash client • Game actions need to be persisted and validated • 1 API call every few secs
  • 5. Social Games HTTP API • 5000 HTTP reqs/sec • more than 90% writes • 60K queries/sec
 http://www.flickr.com/photos/stars6/4381851322
  • 6. July 2010 HAproxy • ~ 170 000 daily users • Plain Ruby on Rails app • Persistency 100% SQL Ruby on Rails MySQL
  • 7. July 2010 • 1 haproxy server HAproxy • multiple RoR servers • 4 mysql servers Ruby on Rails (sharded dataset) MySQL
  • 8. July 2010 HAproxy Ruby on Rails Slow down MySQL
  • 9. July 2010 HAproxy High queries/request Ruby on Rails ratio Slow down MySQL
  • 10. Queries/request • Which code is triggering extra queries? • Why in our test environment the ratio is lower than live?
  • 11. Queries/request Running code of live system Application Plugins Ruby on Rails
  • 12. Queries/request Source of extra queries • sharding plugin “breaks” std Rails query cache Plugins • Flash wire protocol plugin generates extra queries
  • 13. Plugins • Deceiving “feature for free” • Might provide the right feature • But might not meet scaling need
  • 14. Plugins • Instant code legacy, for new projects also! • Once added it’s your code • Even if it’s maintained, will it follow your needs?
  • 15. Plugins • Assess code quality when you add it • Can you afford to maintain/change it?
  • 16. Plugins • We fixed it! • Query cut up to 40% on some requests
  • 17. Early August 30 22.5 15 7.5 0 6:00 6:10 6:20 6:30 6:40 6:50 7:00 7:10 7:20 7:30 7:40 7:50 8:00 8:10 query time in ms • The MySQL hiccup • every 70 mins query time spikes x7
  • 18. Hiccup causes Who is periodically blocking MySQL • Code (app + plugins + Rails)? • Some periodic job? • The devil (AWS)?
  • 19. Hiccup quick fix • We shard out the top queried table (40% of all queries) MySQL servers shard 1 shard 2 shard 3 shard 4
  • 20. Hiccup quick fix • We shard out the top queried table (40% of all queries) Top table Top table Top table Top table shard 1 shard 2 shard 3 shard 4 Other tables Other tables Other tables Other tables shard 1 shard 2 shard 3 shard 4
  • 21. Hiccup quick fix • Mysql likes it • “top table” shards will go a long way in the scaling process Top table Top table Top table Top table shard 1 shard 2 shard 3 shard 4 Other tables Other tables Other tables Other tables shard 1 shard 2 shard 3 shard 4
  • 22. Hiccup causes Who is periodically blocking MySQL • Code (app + plugins + Rails)? • Some periodic job? • The devil (AWS)? None of the Above
  • 23. Hiccup real cause • Emerging MySQL internal at high volume • MySQL flushes its buffer • Under heavy write IO it’s blocking
  • 24. Hiccup solution • Percona MySQL patches (XtraDB) avoid blocking behavior • Query time profile gets smooth • IO capacity limit manifested with gradual performance decay
  • 25. Write through cache • Memcache in front of MySQL • Evaluated before sharding • Was discarded • Because of our read/write reatio
  • 26. Write through cache 90% of the times we read data in order to modify it
  • 27. Write through cache It means 90% of the times 1. read cache 2. write cache 3. write SQL
  • 28. Write through cache Bound to Read heavy Write heavy • Mysql write (unless async) • memcache perfs • Write through lib optimized for writes?
  • 29. MySQL • Sharding SQL is a painful way to scale • Data migrations at high load imply downtime • ACID benefits all lost because of sharding or in name of performance
  • 30. Redis • A persistent cache • Fast 60000 qps on AWS hardware • Interesting data structures, not only KV • Already some small scale experince in house
  • 31. Redis adoption • Which data to start from? • How do we migrate without downtime? • Which Ruby object - Redis structure lib?
  • 32. Redis adoption • Which data to start from? • Best data fit for Redis hashes • Top 3rd queried table • a collection of integer fields that need only increment / decrement
  • 33. Redis adoption • How do we migrate without downtime? • Migrate one user at a time • Use a Redis set to keep note of migrated/ non migrated • No downtime, transparent to users
  • 34. Redis adoption • How do we migrate without downtime? MySQL User 123 RoR Server Redis
  • 35. Redis adoption • How do we migrate without downtime? read original data MySQL User 123 RoR Server Redis
  • 36. Redis adoption • How do we migrate without downtime? MySQL User 123 RoR Server Redis write migrated data
  • 37. Redis adoption • How do we migrate without downtime? • Migration might never complete • SQL + Redis set information to generate final batch migration
  • 38. Redis 1st result 10% query load from 4 MySQL server is moved to 1 Redis server Redis server load is 0.05
  • 39. Redis • Becomes the tool to use • Migration plan for all write intensive data • Migrate one “class” at a time
  • 40. Redis honeymoon end • Memory usage grows more than data • Snapshot to disk causes spikes in query time • Starting new slaves eats memory on the master node
  • 41. Redis honeymoon end Russian Roulette Feeling • Redis machine sized with overabundant RAM • Rigorous slave/master starting plan
  • 42. Redis • Redis team acknowledges persistency/ replication problems • Redis 2.4 diskstore plan starts
  • 44. 1.000.000 painless scaling HAproxy Ruby on Rails Persistency
  • 45. 1.000.000 HAproxy just add servers Ruby on Rails as load grows Peristency
  • 46. 1.000.000 HAproxy Ruby on Rails Painful and Peristency troublesome
  • 47. Infrastructure • AWS • Chef - through Scalarium • Ganglia
  • 49. wooga Is looking for Business Intelligence Engineer http://wooga.com/jobs