SlideShare a Scribd company logo
1 of 45
Scalable Event Analytics with Ruby on Rails & MongoDB Ruby Conf China 2010 Jared Rosoff (@forjared)  jrosoff@yottaa.com
Yottaa!!!! (www.yottaa.com)
Overview Ruby at Scale What is Event Analytics?  What are the different ways you could do it?  How we did it
Ruby At Scale? http://www.flickr.com/photos/laughingsquid
Event Analytics Data Source Event User Event Event Query Event Analytics Data Source Report Event Event Event Event User Query Data Source Event Event Report Event Event Event Data Source
Oh and we are a startup
Our requirements:
Rails default architecture Collection Server Data Source MySQL User Reporting Server
Rails default architecture Collection Server Data Source MySQL User Reporting Server “Just” a Rails App
Rails default architecture 	Performance Bottleneck: Too much load Collection Server Data Source MySQL User Reporting Server “Just” a Rails App
Let’s add replication! MySQL Master Collection Server Data Source Replication MySQL Master User Reporting Server MySQL Master MySQL Master
Let’s add replication! MySQL Master Collection Server Data Source Replication MySQL Master User Reporting Server MySQL Master MySQL Master Off the shelf! Scalable Reads!
Let’s add replication! 	Performance Bottleneck: Still can’t scale writes MySQL Master Collection Server Data Source Replication MySQL Master User Reporting Server MySQL Master MySQL Master Off the shelf! Scalable Reads!
What about sharding? Collection Server Data Source Sharding MySQL Master MySQL Master MySQL Master User Reporting Server Sharding
What about sharding? Collection Server Data Source Sharding MySQL Master MySQL Master MySQL Master User Reporting Server Sharding Scalable Writes!
What about sharding? 	Development Bottleneck: Need to write custom code Collection Server Data Source Sharding MySQL Master MySQL Master MySQL Master User Reporting Server Sharding Scalable Writes!
Key Value stores to the rescue? Collection Server Data Source MySQL Master MySQL Master Cassandra or Voldemort User Reporting Server
Key Value stores to the rescue? Collection Server Data Source MySQL Master MySQL Master Cassandra or Voldemort User Reporting Server Scalable Writes!
Key Value stores to the rescue? 	Development Bottleneck: Reporting is limited / hard Collection Server Data Source MySQL Master MySQL Master Cassandra or Voldemort User Reporting Server Scalable Writes!
Can I Hadoop my way out of this? MySQL Master MySQL Master Cassandra or Voldemort Collection Server Data Source Hadoop MySQL Master MySQL Master User Reporting Server MySQL Master MySQL Slave
Can I Hadoop my way out of this? MySQL Master MySQL Master Cassandra or Voldemort Collection Server Data Source Hadoop MySQL Master Scalable Writes! MySQL Master User Reporting Server MySQL Master MySQL Slave
Can I Hadoop my way out of this? MySQL Master MySQL Master Cassandra or Voldemort Collection Server Data Source Hadoop MySQL Master Scalable Writes! Flexible Reports! MySQL Master User Reporting Server MySQL Master MySQL Slave
Can I Hadoop my way out of this? MySQL Master MySQL Master Cassandra or Voldemort Collection Server Data Source Hadoop MySQL Master Scalable Writes! Flexible Reports! “Just” a Rails App MySQL Master User Reporting Server MySQL Master MySQL Slave
Can I Hadoop my way out of this? 	Development Bottleneck: Too many systems! MySQL Master MySQL Master Cassandra or Voldemort Collection Server Data Source Hadoop MySQL Master Scalable Writes! Flexible Reports! “Just” a Rails App MySQL Master User Reporting Server MySQL Master MySQL Slave
MongoDB!  Collection Server Data Source MySQL Master MySQL Master MongoDB User Reporting Server
MongoDB!  Collection Server Data Source MySQL Master MySQL Master MongoDB User Reporting Server Scalable Writes!
MongoDB!  Collection Server Data Source MySQL Master MySQL Master MongoDB User Reporting Server Scalable Writes! Flexible Reporting!
MongoDB!  Collection Server Data Source MySQL Master MySQL Master MongoDB User Reporting Server Scalable Writes! “Just” a rails app Flexible Reporting!
MongoD App Server Data Source Collection MongoD Load Balancer Passenger Nginx Mongos Reporting User MongoD
MongoD App Server Data Source Collection MongoD Load Balancer Passenger Nginx Mongos Reporting User MongoD Sharding!
MongoD App Server Data Source Collection MongoD Load Balancer Passenger Nginx Mongos Reporting User MongoD Sharding! High Concurrency
MongoD App Server Data Source Collection MongoD Load Balancer Passenger Nginx Mongos Reporting User MongoD Sharding! High Concurrency Scale-Out
MongoDBSharding
MongoDBSharding Replica Sets let us scale storage & transaction capacity for each shard
MongoDBSharding Replica Sets let us scale storage & transaction capacity for each shard Mongos routes transactions to shards based on “shard key”
MongoDBSharding Replica Sets let us scale storage & transaction capacity for each shard Mongos routes transactions to shards based on “shard key” Config servers store information about which shards exist
Inserting 3 2 Shard key == name bob  Shard 2 1 insert { ‘name’ : bob } Insert { ‘name’ : bob }
Querying 3 2 Shard key == name bob  Shard 2 1 Query { ‘name’ : bob } Query { ‘name’ : bob }
Map Reduce 2 2 2 2 1 Map-reduce( … )  Map-reduce(…)
Working with Mongo MongoMapper makes it look like ActiveRecord Documents are more natural than rows in many cases Map-Reduce rocks (but needs better support in rails) http://www.flickr.com/photos/elhamalawy/2526783078/
Ruby Mongo
Runs over all the objects in the views table, counting how many times a page was viewed Adds up all the counts for a unique url / date combination Run the map reduce job and return a collection containing the results
Results Version 1 of our analytics system took 2 weeks with 1 engineer  We have since added a lot more complexity, but we did it incrementally We replaced MySQL entirely with MongoDB No need for joins, transactions  Every table is now a document collection It’s fast!  63ms – Average response time for sending data to server 93ms – Average response time for displaying reports

More Related Content

What's hot

Webinar: Developing with the modern App Stack: MEAN and MERN (with Angular2 a...
Webinar: Developing with the modern App Stack: MEAN and MERN (with Angular2 a...Webinar: Developing with the modern App Stack: MEAN and MERN (with Angular2 a...
Webinar: Developing with the modern App Stack: MEAN and MERN (with Angular2 a...MongoDB
 
Back to Basics Spanish 4 Introduction to sharding
Back to Basics Spanish 4 Introduction to shardingBack to Basics Spanish 4 Introduction to sharding
Back to Basics Spanish 4 Introduction to shardingMongoDB
 
Real time fulltext search with sphinx
Real time fulltext search with sphinxReal time fulltext search with sphinx
Real time fulltext search with sphinxAdrian Nuta
 
Fluentd - Flexible, Stable, Scalable
Fluentd - Flexible, Stable, ScalableFluentd - Flexible, Stable, Scalable
Fluentd - Flexible, Stable, ScalableShu Ting Tseng
 
High Performance Applications with MongoDB
High Performance Applications with MongoDBHigh Performance Applications with MongoDB
High Performance Applications with MongoDBMongoDB
 
Async and Non-blocking IO w/ JRuby
Async and Non-blocking IO w/ JRubyAsync and Non-blocking IO w/ JRuby
Async and Non-blocking IO w/ JRubyJoe Kutner
 
Null Bachaav - May 07 Attack Monitoring workshop.
Null Bachaav - May 07 Attack Monitoring workshop.Null Bachaav - May 07 Attack Monitoring workshop.
Null Bachaav - May 07 Attack Monitoring workshop.Prajal Kulkarni
 
Distributed Logging Architecture in Container Era
Distributed Logging Architecture in Container EraDistributed Logging Architecture in Container Era
Distributed Logging Architecture in Container EraSATOSHI TAGOMORI
 
Using Sphinx for Search in PHP
Using Sphinx for Search in PHPUsing Sphinx for Search in PHP
Using Sphinx for Search in PHPMike Lively
 
mongoDB Performance
mongoDB PerformancemongoDB Performance
mongoDB PerformanceMoshe Kaplan
 
«Scrapy internals» Александр Сибиряков, Scrapinghub
«Scrapy internals» Александр Сибиряков, Scrapinghub«Scrapy internals» Александр Сибиряков, Scrapinghub
«Scrapy internals» Александр Сибиряков, Scrapinghubit-people
 
Back to Basics 2017: Mí primera aplicación MongoDB
Back to Basics 2017: Mí primera aplicación MongoDBBack to Basics 2017: Mí primera aplicación MongoDB
Back to Basics 2017: Mí primera aplicación MongoDBMongoDB
 
Mobile Analytics mit Elasticsearch und Kibana
Mobile Analytics mit Elasticsearch und KibanaMobile Analytics mit Elasticsearch und Kibana
Mobile Analytics mit Elasticsearch und Kibanainovex GmbH
 
Presto - Hadoop Conference Japan 2014
Presto - Hadoop Conference Japan 2014Presto - Hadoop Conference Japan 2014
Presto - Hadoop Conference Japan 2014Sadayuki Furuhashi
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDBJustin Smestad
 
Back to Basics Spanish Webinar 3 - Introducción a los replica sets
Back to Basics Spanish Webinar 3 - Introducción a los replica setsBack to Basics Spanish Webinar 3 - Introducción a los replica sets
Back to Basics Spanish Webinar 3 - Introducción a los replica setsMongoDB
 
Back to Basics Webinar 6: Production Deployment
Back to Basics Webinar 6: Production DeploymentBack to Basics Webinar 6: Production Deployment
Back to Basics Webinar 6: Production DeploymentMongoDB
 
Efficient cluster resource management by using Cook and Mesos / Li Jin (Two S...
Efficient cluster resource management by using Cook and Mesos / Li Jin (Two S...Efficient cluster resource management by using Cook and Mesos / Li Jin (Two S...
Efficient cluster resource management by using Cook and Mesos / Li Jin (Two S...Ontico
 
Conceptos básicos. Seminario web 2: Su primera aplicación MongoDB
 Conceptos básicos. Seminario web 2: Su primera aplicación MongoDB Conceptos básicos. Seminario web 2: Su primera aplicación MongoDB
Conceptos básicos. Seminario web 2: Su primera aplicación MongoDBMongoDB
 

What's hot (20)

Webinar: Developing with the modern App Stack: MEAN and MERN (with Angular2 a...
Webinar: Developing with the modern App Stack: MEAN and MERN (with Angular2 a...Webinar: Developing with the modern App Stack: MEAN and MERN (with Angular2 a...
Webinar: Developing with the modern App Stack: MEAN and MERN (with Angular2 a...
 
Back to Basics Spanish 4 Introduction to sharding
Back to Basics Spanish 4 Introduction to shardingBack to Basics Spanish 4 Introduction to sharding
Back to Basics Spanish 4 Introduction to sharding
 
Real time fulltext search with sphinx
Real time fulltext search with sphinxReal time fulltext search with sphinx
Real time fulltext search with sphinx
 
Fluentd - Flexible, Stable, Scalable
Fluentd - Flexible, Stable, ScalableFluentd - Flexible, Stable, Scalable
Fluentd - Flexible, Stable, Scalable
 
High Performance Applications with MongoDB
High Performance Applications with MongoDBHigh Performance Applications with MongoDB
High Performance Applications with MongoDB
 
Async and Non-blocking IO w/ JRuby
Async and Non-blocking IO w/ JRubyAsync and Non-blocking IO w/ JRuby
Async and Non-blocking IO w/ JRuby
 
Null Bachaav - May 07 Attack Monitoring workshop.
Null Bachaav - May 07 Attack Monitoring workshop.Null Bachaav - May 07 Attack Monitoring workshop.
Null Bachaav - May 07 Attack Monitoring workshop.
 
Distributed Logging Architecture in Container Era
Distributed Logging Architecture in Container EraDistributed Logging Architecture in Container Era
Distributed Logging Architecture in Container Era
 
Using Sphinx for Search in PHP
Using Sphinx for Search in PHPUsing Sphinx for Search in PHP
Using Sphinx for Search in PHP
 
mongoDB Performance
mongoDB PerformancemongoDB Performance
mongoDB Performance
 
«Scrapy internals» Александр Сибиряков, Scrapinghub
«Scrapy internals» Александр Сибиряков, Scrapinghub«Scrapy internals» Александр Сибиряков, Scrapinghub
«Scrapy internals» Александр Сибиряков, Scrapinghub
 
Back to Basics 2017: Mí primera aplicación MongoDB
Back to Basics 2017: Mí primera aplicación MongoDBBack to Basics 2017: Mí primera aplicación MongoDB
Back to Basics 2017: Mí primera aplicación MongoDB
 
Mobile Analytics mit Elasticsearch und Kibana
Mobile Analytics mit Elasticsearch und KibanaMobile Analytics mit Elasticsearch und Kibana
Mobile Analytics mit Elasticsearch und Kibana
 
Presto - Hadoop Conference Japan 2014
Presto - Hadoop Conference Japan 2014Presto - Hadoop Conference Japan 2014
Presto - Hadoop Conference Japan 2014
 
The tale of 100 cve's
The tale of 100 cve'sThe tale of 100 cve's
The tale of 100 cve's
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 
Back to Basics Spanish Webinar 3 - Introducción a los replica sets
Back to Basics Spanish Webinar 3 - Introducción a los replica setsBack to Basics Spanish Webinar 3 - Introducción a los replica sets
Back to Basics Spanish Webinar 3 - Introducción a los replica sets
 
Back to Basics Webinar 6: Production Deployment
Back to Basics Webinar 6: Production DeploymentBack to Basics Webinar 6: Production Deployment
Back to Basics Webinar 6: Production Deployment
 
Efficient cluster resource management by using Cook and Mesos / Li Jin (Two S...
Efficient cluster resource management by using Cook and Mesos / Li Jin (Two S...Efficient cluster resource management by using Cook and Mesos / Li Jin (Two S...
Efficient cluster resource management by using Cook and Mesos / Li Jin (Two S...
 
Conceptos básicos. Seminario web 2: Su primera aplicación MongoDB
 Conceptos básicos. Seminario web 2: Su primera aplicación MongoDB Conceptos básicos. Seminario web 2: Su primera aplicación MongoDB
Conceptos básicos. Seminario web 2: Su primera aplicación MongoDB
 

Viewers also liked

Mongoose v3 :: The Future is Bright
Mongoose v3 :: The Future is BrightMongoose v3 :: The Future is Bright
Mongoose v3 :: The Future is Brightaaronheckmann
 
Rainbird: Realtime Analytics at Twitter (Strata 2011)
Rainbird: Realtime Analytics at Twitter (Strata 2011)Rainbird: Realtime Analytics at Twitter (Strata 2011)
Rainbird: Realtime Analytics at Twitter (Strata 2011)Kevin Weil
 
Becoming Node.js ninja on Cloud Foundry
Becoming Node.js ninja on Cloud FoundryBecoming Node.js ninja on Cloud Foundry
Becoming Node.js ninja on Cloud FoundryRaja Rao DV
 
Testing nodejs apps
Testing nodejs appsTesting nodejs apps
Testing nodejs appsfelipefsilva
 
[C5]deview 2012 nodejs
[C5]deview 2012 nodejs[C5]deview 2012 nodejs
[C5]deview 2012 nodejsNAVER D2
 
소셜게임 서버 개발 관점에서 본 Node.js의 장단점과 대안
소셜게임 서버 개발 관점에서 본 Node.js의 장단점과 대안소셜게임 서버 개발 관점에서 본 Node.js의 장단점과 대안
소셜게임 서버 개발 관점에서 본 Node.js의 장단점과 대안Jeongsang Baek
 
Module, AMD, RequireJS
Module, AMD, RequireJSModule, AMD, RequireJS
Module, AMD, RequireJS偉格 高
 
Asynchronous Module Definition (AMD)
Asynchronous Module Definition (AMD)Asynchronous Module Definition (AMD)
Asynchronous Module Definition (AMD)xMartin12
 
HTML5 Real-Time and Connectivity
HTML5 Real-Time and ConnectivityHTML5 Real-Time and Connectivity
HTML5 Real-Time and ConnectivityPeter Lubbers
 
Veritabani Programlama MySQL I. Egitimi
Veritabani Programlama MySQL I. EgitimiVeritabani Programlama MySQL I. Egitimi
Veritabani Programlama MySQL I. EgitimiSevdanur Genc
 
Mongokit presentation mongofr-2010
Mongokit presentation mongofr-2010Mongokit presentation mongofr-2010
Mongokit presentation mongofr-2010namlook
 
A SOA approximation on symfony
A SOA approximation on symfonyA SOA approximation on symfony
A SOA approximation on symfonyJoseluis Laso
 
How We Use MongoDB in Our Advertising System
How We Use MongoDB in Our Advertising SystemHow We Use MongoDB in Our Advertising System
How We Use MongoDB in Our Advertising SystemMongoDB
 
Telephony with OpenShift Twilio and MongoDB
Telephony with OpenShift Twilio and MongoDBTelephony with OpenShift Twilio and MongoDB
Telephony with OpenShift Twilio and MongoDBMark Atwood
 
Build Your Own Custom Mobile Analytics with Node and MongoDB
Build Your Own Custom Mobile Analytics with Node and MongoDBBuild Your Own Custom Mobile Analytics with Node and MongoDB
Build Your Own Custom Mobile Analytics with Node and MongoDBMongoDB
 
10gen MongoDB Video Presentation at WebGeek DevCup
10gen MongoDB Video Presentation at WebGeek DevCup10gen MongoDB Video Presentation at WebGeek DevCup
10gen MongoDB Video Presentation at WebGeek DevCupWebGeek Philippines
 
Rapid Application Development using Ruby on Rails
Rapid Application Development using Ruby on RailsRapid Application Development using Ruby on Rails
Rapid Application Development using Ruby on RailsSimobo
 

Viewers also liked (20)

Mongoose v3 :: The Future is Bright
Mongoose v3 :: The Future is BrightMongoose v3 :: The Future is Bright
Mongoose v3 :: The Future is Bright
 
Grid FS
Grid FSGrid FS
Grid FS
 
Rainbird: Realtime Analytics at Twitter (Strata 2011)
Rainbird: Realtime Analytics at Twitter (Strata 2011)Rainbird: Realtime Analytics at Twitter (Strata 2011)
Rainbird: Realtime Analytics at Twitter (Strata 2011)
 
The SPDY Protocol
The SPDY ProtocolThe SPDY Protocol
The SPDY Protocol
 
Becoming Node.js ninja on Cloud Foundry
Becoming Node.js ninja on Cloud FoundryBecoming Node.js ninja on Cloud Foundry
Becoming Node.js ninja on Cloud Foundry
 
Testing nodejs apps
Testing nodejs appsTesting nodejs apps
Testing nodejs apps
 
[C5]deview 2012 nodejs
[C5]deview 2012 nodejs[C5]deview 2012 nodejs
[C5]deview 2012 nodejs
 
소셜게임 서버 개발 관점에서 본 Node.js의 장단점과 대안
소셜게임 서버 개발 관점에서 본 Node.js의 장단점과 대안소셜게임 서버 개발 관점에서 본 Node.js의 장단점과 대안
소셜게임 서버 개발 관점에서 본 Node.js의 장단점과 대안
 
RESTful API Design, Second Edition
RESTful API Design, Second EditionRESTful API Design, Second Edition
RESTful API Design, Second Edition
 
Module, AMD, RequireJS
Module, AMD, RequireJSModule, AMD, RequireJS
Module, AMD, RequireJS
 
Asynchronous Module Definition (AMD)
Asynchronous Module Definition (AMD)Asynchronous Module Definition (AMD)
Asynchronous Module Definition (AMD)
 
HTML5 Real-Time and Connectivity
HTML5 Real-Time and ConnectivityHTML5 Real-Time and Connectivity
HTML5 Real-Time and Connectivity
 
Veritabani Programlama MySQL I. Egitimi
Veritabani Programlama MySQL I. EgitimiVeritabani Programlama MySQL I. Egitimi
Veritabani Programlama MySQL I. Egitimi
 
Mongokit presentation mongofr-2010
Mongokit presentation mongofr-2010Mongokit presentation mongofr-2010
Mongokit presentation mongofr-2010
 
A SOA approximation on symfony
A SOA approximation on symfonyA SOA approximation on symfony
A SOA approximation on symfony
 
How We Use MongoDB in Our Advertising System
How We Use MongoDB in Our Advertising SystemHow We Use MongoDB in Our Advertising System
How We Use MongoDB in Our Advertising System
 
Telephony with OpenShift Twilio and MongoDB
Telephony with OpenShift Twilio and MongoDBTelephony with OpenShift Twilio and MongoDB
Telephony with OpenShift Twilio and MongoDB
 
Build Your Own Custom Mobile Analytics with Node and MongoDB
Build Your Own Custom Mobile Analytics with Node and MongoDBBuild Your Own Custom Mobile Analytics with Node and MongoDB
Build Your Own Custom Mobile Analytics with Node and MongoDB
 
10gen MongoDB Video Presentation at WebGeek DevCup
10gen MongoDB Video Presentation at WebGeek DevCup10gen MongoDB Video Presentation at WebGeek DevCup
10gen MongoDB Video Presentation at WebGeek DevCup
 
Rapid Application Development using Ruby on Rails
Rapid Application Development using Ruby on RailsRapid Application Development using Ruby on Rails
Rapid Application Development using Ruby on Rails
 

Similar to Scalable Event Analytics with MongoDB & Ruby on Rails

Ibm_interconnect_restapi_workshop
Ibm_interconnect_restapi_workshopIbm_interconnect_restapi_workshop
Ibm_interconnect_restapi_workshopShubhra Kar
 
No SQL at The Guardian
No SQL at The GuardianNo SQL at The Guardian
No SQL at The GuardianMat Wall
 
NoSql presentation
NoSql presentationNoSql presentation
NoSql presentationMat Wall
 
Battle of NoSQL stars: Amazon's SDB vs MongoDB vs CouchDB vs RavenDB
Battle of NoSQL stars: Amazon's SDB vs MongoDB vs CouchDB vs RavenDBBattle of NoSQL stars: Amazon's SDB vs MongoDB vs CouchDB vs RavenDB
Battle of NoSQL stars: Amazon's SDB vs MongoDB vs CouchDB vs RavenDBJesse Wolgamott
 
Connect js nodejs_api_shubhra
Connect js nodejs_api_shubhraConnect js nodejs_api_shubhra
Connect js nodejs_api_shubhraShubhra Kar
 
Modern Web: Trends der Webentwicklung
Modern Web: Trends der WebentwicklungModern Web: Trends der Webentwicklung
Modern Web: Trends der WebentwicklungOPEN KNOWLEDGE GmbH
 
Presto + Alluxio on steroids a romantic drama on Production with happy end
Presto + Alluxio on steroids a romantic drama on Production with happy endPresto + Alluxio on steroids a romantic drama on Production with happy end
Presto + Alluxio on steroids a romantic drama on Production with happy endAlluxio, Inc.
 
Why we chose mongodb for guardian.co.uk
Why we chose mongodb for guardian.co.ukWhy we chose mongodb for guardian.co.uk
Why we chose mongodb for guardian.co.ukGraham Tackley
 
2010 07-20 TDD with ActiveResource
2010 07-20 TDD with ActiveResource2010 07-20 TDD with ActiveResource
2010 07-20 TDD with ActiveResourceWolfram Arnold
 
Rapid API Development with LoopBack/StrongLoop
Rapid API Development with LoopBack/StrongLoopRapid API Development with LoopBack/StrongLoop
Rapid API Development with LoopBack/StrongLoopRaymond Camden
 
Viridians on Rails
Viridians on RailsViridians on Rails
Viridians on RailsViridians
 
Architecting Multitenant SaaS Applications with Azure - Microsoft Ignite The ...
Architecting Multitenant SaaS Applications with Azure - Microsoft Ignite The ...Architecting Multitenant SaaS Applications with Azure - Microsoft Ignite The ...
Architecting Multitenant SaaS Applications with Azure - Microsoft Ignite The ...Eran Stiller
 
Ruby and Rails short motivation
Ruby and Rails short motivationRuby and Rails short motivation
Ruby and Rails short motivationjistr
 
Christian Mladenov @ Intuitics
Christian Mladenov @ IntuiticsChristian Mladenov @ Intuitics
Christian Mladenov @ IntuiticsPAPIs.io
 
.NET Architects Day - DNAD 2011
.NET Architects Day - DNAD 2011.NET Architects Day - DNAD 2011
.NET Architects Day - DNAD 2011Fabio Akita
 
09 - Fábio Akita - Além do rails
09 - Fábio Akita - Além do rails09 - Fábio Akita - Além do rails
09 - Fábio Akita - Além do railsDNAD
 
Seattle StrongLoop Node.js Workshop
Seattle StrongLoop Node.js WorkshopSeattle StrongLoop Node.js Workshop
Seattle StrongLoop Node.js WorkshopJimmy Guerrero
 
Rapid, Scalable Web Development with MongoDB, Ming, and Python
Rapid, Scalable Web Development with MongoDB, Ming, and PythonRapid, Scalable Web Development with MongoDB, Ming, and Python
Rapid, Scalable Web Development with MongoDB, Ming, and PythonRick Copeland
 
Isomorphic javascript - Uppsala.js #8
Isomorphic javascript - Uppsala.js #8Isomorphic javascript - Uppsala.js #8
Isomorphic javascript - Uppsala.js #8Alexander Aivars
 
Moving from Relational to Document Store
Moving from Relational to Document StoreMoving from Relational to Document Store
Moving from Relational to Document StoreGraham Tackley
 

Similar to Scalable Event Analytics with MongoDB & Ruby on Rails (20)

Ibm_interconnect_restapi_workshop
Ibm_interconnect_restapi_workshopIbm_interconnect_restapi_workshop
Ibm_interconnect_restapi_workshop
 
No SQL at The Guardian
No SQL at The GuardianNo SQL at The Guardian
No SQL at The Guardian
 
NoSql presentation
NoSql presentationNoSql presentation
NoSql presentation
 
Battle of NoSQL stars: Amazon's SDB vs MongoDB vs CouchDB vs RavenDB
Battle of NoSQL stars: Amazon's SDB vs MongoDB vs CouchDB vs RavenDBBattle of NoSQL stars: Amazon's SDB vs MongoDB vs CouchDB vs RavenDB
Battle of NoSQL stars: Amazon's SDB vs MongoDB vs CouchDB vs RavenDB
 
Connect js nodejs_api_shubhra
Connect js nodejs_api_shubhraConnect js nodejs_api_shubhra
Connect js nodejs_api_shubhra
 
Modern Web: Trends der Webentwicklung
Modern Web: Trends der WebentwicklungModern Web: Trends der Webentwicklung
Modern Web: Trends der Webentwicklung
 
Presto + Alluxio on steroids a romantic drama on Production with happy end
Presto + Alluxio on steroids a romantic drama on Production with happy endPresto + Alluxio on steroids a romantic drama on Production with happy end
Presto + Alluxio on steroids a romantic drama on Production with happy end
 
Why we chose mongodb for guardian.co.uk
Why we chose mongodb for guardian.co.ukWhy we chose mongodb for guardian.co.uk
Why we chose mongodb for guardian.co.uk
 
2010 07-20 TDD with ActiveResource
2010 07-20 TDD with ActiveResource2010 07-20 TDD with ActiveResource
2010 07-20 TDD with ActiveResource
 
Rapid API Development with LoopBack/StrongLoop
Rapid API Development with LoopBack/StrongLoopRapid API Development with LoopBack/StrongLoop
Rapid API Development with LoopBack/StrongLoop
 
Viridians on Rails
Viridians on RailsViridians on Rails
Viridians on Rails
 
Architecting Multitenant SaaS Applications with Azure - Microsoft Ignite The ...
Architecting Multitenant SaaS Applications with Azure - Microsoft Ignite The ...Architecting Multitenant SaaS Applications with Azure - Microsoft Ignite The ...
Architecting Multitenant SaaS Applications with Azure - Microsoft Ignite The ...
 
Ruby and Rails short motivation
Ruby and Rails short motivationRuby and Rails short motivation
Ruby and Rails short motivation
 
Christian Mladenov @ Intuitics
Christian Mladenov @ IntuiticsChristian Mladenov @ Intuitics
Christian Mladenov @ Intuitics
 
.NET Architects Day - DNAD 2011
.NET Architects Day - DNAD 2011.NET Architects Day - DNAD 2011
.NET Architects Day - DNAD 2011
 
09 - Fábio Akita - Além do rails
09 - Fábio Akita - Além do rails09 - Fábio Akita - Além do rails
09 - Fábio Akita - Além do rails
 
Seattle StrongLoop Node.js Workshop
Seattle StrongLoop Node.js WorkshopSeattle StrongLoop Node.js Workshop
Seattle StrongLoop Node.js Workshop
 
Rapid, Scalable Web Development with MongoDB, Ming, and Python
Rapid, Scalable Web Development with MongoDB, Ming, and PythonRapid, Scalable Web Development with MongoDB, Ming, and Python
Rapid, Scalable Web Development with MongoDB, Ming, and Python
 
Isomorphic javascript - Uppsala.js #8
Isomorphic javascript - Uppsala.js #8Isomorphic javascript - Uppsala.js #8
Isomorphic javascript - Uppsala.js #8
 
Moving from Relational to Document Store
Moving from Relational to Document StoreMoving from Relational to Document Store
Moving from Relational to Document Store
 

More from Jared Rosoff

MongoDB Advanced Schema Design - Inboxes
MongoDB Advanced Schema Design - InboxesMongoDB Advanced Schema Design - Inboxes
MongoDB Advanced Schema Design - InboxesJared Rosoff
 
Mongosv 2011 - Sharding
Mongosv 2011 - ShardingMongosv 2011 - Sharding
Mongosv 2011 - ShardingJared Rosoff
 
Mongosv 2011 - Replication
Mongosv 2011 - ReplicationMongosv 2011 - Replication
Mongosv 2011 - ReplicationJared Rosoff
 
Mongosv 2011 - MongoDB on Amazon EC2
Mongosv 2011 - MongoDB on Amazon EC2Mongosv 2011 - MongoDB on Amazon EC2
Mongosv 2011 - MongoDB on Amazon EC2Jared Rosoff
 
MongoDB Deployment Tips
MongoDB Deployment TipsMongoDB Deployment Tips
MongoDB Deployment TipsJared Rosoff
 
Scaling with mongo db - SF Mongo User Group 7-19-2011
Scaling with mongo db - SF Mongo User Group 7-19-2011Scaling with mongo db - SF Mongo User Group 7-19-2011
Scaling with mongo db - SF Mongo User Group 7-19-2011Jared Rosoff
 
MongoDB on EC2 and EBS
MongoDB on EC2 and EBSMongoDB on EC2 and EBS
MongoDB on EC2 and EBSJared Rosoff
 
Indexing & query optimization
Indexing & query optimizationIndexing & query optimization
Indexing & query optimizationJared Rosoff
 
Web performance meetup bos 11 18-2010
Web performance meetup bos 11 18-2010Web performance meetup bos 11 18-2010
Web performance meetup bos 11 18-2010Jared Rosoff
 

More from Jared Rosoff (9)

MongoDB Advanced Schema Design - Inboxes
MongoDB Advanced Schema Design - InboxesMongoDB Advanced Schema Design - Inboxes
MongoDB Advanced Schema Design - Inboxes
 
Mongosv 2011 - Sharding
Mongosv 2011 - ShardingMongosv 2011 - Sharding
Mongosv 2011 - Sharding
 
Mongosv 2011 - Replication
Mongosv 2011 - ReplicationMongosv 2011 - Replication
Mongosv 2011 - Replication
 
Mongosv 2011 - MongoDB on Amazon EC2
Mongosv 2011 - MongoDB on Amazon EC2Mongosv 2011 - MongoDB on Amazon EC2
Mongosv 2011 - MongoDB on Amazon EC2
 
MongoDB Deployment Tips
MongoDB Deployment TipsMongoDB Deployment Tips
MongoDB Deployment Tips
 
Scaling with mongo db - SF Mongo User Group 7-19-2011
Scaling with mongo db - SF Mongo User Group 7-19-2011Scaling with mongo db - SF Mongo User Group 7-19-2011
Scaling with mongo db - SF Mongo User Group 7-19-2011
 
MongoDB on EC2 and EBS
MongoDB on EC2 and EBSMongoDB on EC2 and EBS
MongoDB on EC2 and EBS
 
Indexing & query optimization
Indexing & query optimizationIndexing & query optimization
Indexing & query optimization
 
Web performance meetup bos 11 18-2010
Web performance meetup bos 11 18-2010Web performance meetup bos 11 18-2010
Web performance meetup bos 11 18-2010
 

Recently uploaded

Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...CzechDreamin
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfFIDO Alliance
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftshyamraj55
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty SecureFemke de Vroome
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...FIDO Alliance
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?Mark Billinghurst
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceSamy Fodil
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform EngineeringMarcus Vechiato
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxJennifer Lim
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfUK Journal
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...FIDO Alliance
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessUXDXConf
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfFIDO Alliance
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...FIDO Alliance
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsStefano
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireExakis Nelite
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...CzechDreamin
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024Stephanie Beckett
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...panagenda
 

Recently uploaded (20)

Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
ECS 2024 Teams Premium - Pretty Secure
ECS 2024   Teams Premium - Pretty SecureECS 2024   Teams Premium - Pretty Secure
ECS 2024 Teams Premium - Pretty Secure
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Working together SRE & Platform Engineering
Working together SRE & Platform EngineeringWorking together SRE & Platform Engineering
Working together SRE & Platform Engineering
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdfBreaking Down the Flutterwave Scandal What You Need to Know.pdf
Breaking Down the Flutterwave Scandal What You Need to Know.pdf
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
Structuring Teams and Portfolios for Success
Structuring Teams and Portfolios for SuccessStructuring Teams and Portfolios for Success
Structuring Teams and Portfolios for Success
 
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdfHow Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
How Red Hat Uses FDO in Device Lifecycle _ Costin and Vitaliy at Red Hat.pdf
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
Microsoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - QuestionnaireMicrosoft CSP Briefing Pre-Engagement - Questionnaire
Microsoft CSP Briefing Pre-Engagement - Questionnaire
 
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
Integrating Telephony Systems with Salesforce: Insights and Considerations, B...
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 

Scalable Event Analytics with MongoDB & Ruby on Rails

  • 1. Scalable Event Analytics with Ruby on Rails & MongoDB Ruby Conf China 2010 Jared Rosoff (@forjared) jrosoff@yottaa.com
  • 3. Overview Ruby at Scale What is Event Analytics? What are the different ways you could do it? How we did it
  • 4. Ruby At Scale? http://www.flickr.com/photos/laughingsquid
  • 5. Event Analytics Data Source Event User Event Event Query Event Analytics Data Source Report Event Event Event Event User Query Data Source Event Event Report Event Event Event Data Source
  • 6.
  • 7. Oh and we are a startup
  • 9. Rails default architecture Collection Server Data Source MySQL User Reporting Server
  • 10. Rails default architecture Collection Server Data Source MySQL User Reporting Server “Just” a Rails App
  • 11. Rails default architecture Performance Bottleneck: Too much load Collection Server Data Source MySQL User Reporting Server “Just” a Rails App
  • 12. Let’s add replication! MySQL Master Collection Server Data Source Replication MySQL Master User Reporting Server MySQL Master MySQL Master
  • 13. Let’s add replication! MySQL Master Collection Server Data Source Replication MySQL Master User Reporting Server MySQL Master MySQL Master Off the shelf! Scalable Reads!
  • 14. Let’s add replication! Performance Bottleneck: Still can’t scale writes MySQL Master Collection Server Data Source Replication MySQL Master User Reporting Server MySQL Master MySQL Master Off the shelf! Scalable Reads!
  • 15. What about sharding? Collection Server Data Source Sharding MySQL Master MySQL Master MySQL Master User Reporting Server Sharding
  • 16. What about sharding? Collection Server Data Source Sharding MySQL Master MySQL Master MySQL Master User Reporting Server Sharding Scalable Writes!
  • 17. What about sharding? Development Bottleneck: Need to write custom code Collection Server Data Source Sharding MySQL Master MySQL Master MySQL Master User Reporting Server Sharding Scalable Writes!
  • 18. Key Value stores to the rescue? Collection Server Data Source MySQL Master MySQL Master Cassandra or Voldemort User Reporting Server
  • 19. Key Value stores to the rescue? Collection Server Data Source MySQL Master MySQL Master Cassandra or Voldemort User Reporting Server Scalable Writes!
  • 20. Key Value stores to the rescue? Development Bottleneck: Reporting is limited / hard Collection Server Data Source MySQL Master MySQL Master Cassandra or Voldemort User Reporting Server Scalable Writes!
  • 21. Can I Hadoop my way out of this? MySQL Master MySQL Master Cassandra or Voldemort Collection Server Data Source Hadoop MySQL Master MySQL Master User Reporting Server MySQL Master MySQL Slave
  • 22. Can I Hadoop my way out of this? MySQL Master MySQL Master Cassandra or Voldemort Collection Server Data Source Hadoop MySQL Master Scalable Writes! MySQL Master User Reporting Server MySQL Master MySQL Slave
  • 23. Can I Hadoop my way out of this? MySQL Master MySQL Master Cassandra or Voldemort Collection Server Data Source Hadoop MySQL Master Scalable Writes! Flexible Reports! MySQL Master User Reporting Server MySQL Master MySQL Slave
  • 24. Can I Hadoop my way out of this? MySQL Master MySQL Master Cassandra or Voldemort Collection Server Data Source Hadoop MySQL Master Scalable Writes! Flexible Reports! “Just” a Rails App MySQL Master User Reporting Server MySQL Master MySQL Slave
  • 25. Can I Hadoop my way out of this? Development Bottleneck: Too many systems! MySQL Master MySQL Master Cassandra or Voldemort Collection Server Data Source Hadoop MySQL Master Scalable Writes! Flexible Reports! “Just” a Rails App MySQL Master User Reporting Server MySQL Master MySQL Slave
  • 26. MongoDB! Collection Server Data Source MySQL Master MySQL Master MongoDB User Reporting Server
  • 27. MongoDB! Collection Server Data Source MySQL Master MySQL Master MongoDB User Reporting Server Scalable Writes!
  • 28. MongoDB! Collection Server Data Source MySQL Master MySQL Master MongoDB User Reporting Server Scalable Writes! Flexible Reporting!
  • 29. MongoDB! Collection Server Data Source MySQL Master MySQL Master MongoDB User Reporting Server Scalable Writes! “Just” a rails app Flexible Reporting!
  • 30. MongoD App Server Data Source Collection MongoD Load Balancer Passenger Nginx Mongos Reporting User MongoD
  • 31. MongoD App Server Data Source Collection MongoD Load Balancer Passenger Nginx Mongos Reporting User MongoD Sharding!
  • 32. MongoD App Server Data Source Collection MongoD Load Balancer Passenger Nginx Mongos Reporting User MongoD Sharding! High Concurrency
  • 33. MongoD App Server Data Source Collection MongoD Load Balancer Passenger Nginx Mongos Reporting User MongoD Sharding! High Concurrency Scale-Out
  • 35. MongoDBSharding Replica Sets let us scale storage & transaction capacity for each shard
  • 36. MongoDBSharding Replica Sets let us scale storage & transaction capacity for each shard Mongos routes transactions to shards based on “shard key”
  • 37. MongoDBSharding Replica Sets let us scale storage & transaction capacity for each shard Mongos routes transactions to shards based on “shard key” Config servers store information about which shards exist
  • 38. Inserting 3 2 Shard key == name bob  Shard 2 1 insert { ‘name’ : bob } Insert { ‘name’ : bob }
  • 39. Querying 3 2 Shard key == name bob  Shard 2 1 Query { ‘name’ : bob } Query { ‘name’ : bob }
  • 40. Map Reduce 2 2 2 2 1 Map-reduce( … ) Map-reduce(…)
  • 41. Working with Mongo MongoMapper makes it look like ActiveRecord Documents are more natural than rows in many cases Map-Reduce rocks (but needs better support in rails) http://www.flickr.com/photos/elhamalawy/2526783078/
  • 42.
  • 44. Runs over all the objects in the views table, counting how many times a page was viewed Adds up all the counts for a unique url / date combination Run the map reduce job and return a collection containing the results
  • 45. Results Version 1 of our analytics system took 2 weeks with 1 engineer We have since added a lot more complexity, but we did it incrementally We replaced MySQL entirely with MongoDB No need for joins, transactions Every table is now a document collection It’s fast! 63ms – Average response time for sending data to server 93ms – Average response time for displaying reports