SlideShare a Scribd company logo
1 of 13
Download to read offline
Notes from the field:
Different NoSQL tools in
       production
       Matthew Ford
We've used:

Key Value Stores:   Document Stores:

  Redis               MongoDB
  Tokyo Cabinet       CouchDB
Redis
Project: Gnip Filter

Requirements:

Users can filter realtime data streams by keywords.

Solution:

redis.set_add('ipad', 'client_id')

In the background we filter the data stream, when a keyword
matches, we push that piece of data to all the client ids.

redis.set_members('ipad')
Redis

Benefits:                 Drawbacks:

Fast                      Not fault tolerant

Simple model              No replication

Sets are useful           Whole data set in memory

O(1) for set add/remove
Tokyo Cabinet
Project: tweetarium

Requirement:

Store tweets from a user to build up twitter lists

Solution:

Serialized JSON from the twitter API, and arrays of twitter ids
for the lists. Running on a small slicehost slice (poor mans
memcached)


Live at http://tweetarium.com/
Tokyo Cabinet

Benefits:         Drawbacks:

Still Fast        Documentation mostly in
                  Japanese
Persistent
                  Kyoto Cabinet (successor)
Space efficient
CouchDB
Project: Project management app

Requirements:

Everything needed to have version tracking (snapshots of the
entire project and all of it's associations).

Solution:

Hybrid system, removed version control from SQL and pass a
JSON representation of the project data to CouchDB.
CouchDB

Benefits:               Drawbacks:

Flexible                Library support (at the time)

Simplify architecture   JS Views, with changing
                        schema
Robust
                        HTTP not a fast as binary
                        protocol
MongoDB
Project: feesability

Problem:

Case estimation and planning tool, with the need to 'clone
cases'.

Solution:

All cases data is embedded within the case document (with
some references to other documents). This enables us to easily
clone an entire case.
MongoDB

Benefits:                  Drawbacks:

Flexible                   Replication not here yet

Fast                       Libraries are playing catch up

Ad-Hoc queries             Durability Trade-off

Easy to
query embedded documents
Conclusion




   Use the right tools for the job
Thanks!

More info

http://blog.bitzesty.com

General NoSQL news

http://nosql.mypopescu.com/

More Related Content

What's hot

Streaming sql and druid
Streaming sql and druid Streaming sql and druid
Streaming sql and druid arupmalakar
 
NoSQL on the move
NoSQL on the moveNoSQL on the move
NoSQL on the moveCodemotion
 
SQL o NoSQL? Progettare applicazioni 'Big Data-ready' attraverso l'utilizzo d...
SQL o NoSQL? Progettare applicazioni 'Big Data-ready' attraverso l'utilizzo d...SQL o NoSQL? Progettare applicazioni 'Big Data-ready' attraverso l'utilizzo d...
SQL o NoSQL? Progettare applicazioni 'Big Data-ready' attraverso l'utilizzo d...Codemotion
 
Collaborative environment with data science notebook
Collaborative environment with data science notebook Collaborative environment with data science notebook
Collaborative environment with data science notebook Moon Soo Lee
 
The evolving container landscape
The evolving container landscapeThe evolving container landscape
The evolving container landscapeNilesh Trivedi
 
Introduction to GraphQL
Introduction to GraphQLIntroduction to GraphQL
Introduction to GraphQLRodrigo Prates
 
Евгений Варфоломеев "Hibernate vs my batis vs jdbc: is there a silver bullet?"
Евгений Варфоломеев "Hibernate vs my batis vs jdbc: is there a silver bullet?"Евгений Варфоломеев "Hibernate vs my batis vs jdbc: is there a silver bullet?"
Евгений Варфоломеев "Hibernate vs my batis vs jdbc: is there a silver bullet?"Anna Shymchenko
 
This Week in Neo4j - 24th November 2018
This Week in Neo4j - 24th November 2018This Week in Neo4j - 24th November 2018
This Week in Neo4j - 24th November 2018Neo4j
 
Shaping serverless architecture with domain driven design patterns
Shaping serverless architecture with domain driven design patternsShaping serverless architecture with domain driven design patterns
Shaping serverless architecture with domain driven design patternsAsher Sterkin
 
OpenStack MagnetoDB. Atlanta Summit 2014
OpenStack MagnetoDB. Atlanta Summit 2014OpenStack MagnetoDB. Atlanta Summit 2014
OpenStack MagnetoDB. Atlanta Summit 2014Ilya Sviridov
 
Serverless Code Components
Serverless Code Components Serverless Code Components
Serverless Code Components Shimon Tolts
 
In Memory Databases: A Real Time Analytics Solution
In Memory Databases: A Real Time Analytics SolutionIn Memory Databases: A Real Time Analytics Solution
In Memory Databases: A Real Time Analytics SolutionAdaryl "Bob" Wakefield, MBA
 
Implementing GraphQL API in Elixir – Victor Deryagin
Implementing GraphQL API in Elixir – Victor DeryaginImplementing GraphQL API in Elixir – Victor Deryagin
Implementing GraphQL API in Elixir – Victor DeryaginElixir Club
 
GraphQL in an Age of REST
GraphQL in an Age of RESTGraphQL in an Age of REST
GraphQL in an Age of RESTYos Riady
 
ResourceSpace: Recent pains and future gains
ResourceSpace: Recent pains and future gainsResourceSpace: Recent pains and future gains
ResourceSpace: Recent pains and future gainsResourceSpace
 

What's hot (20)

Streaming sql and druid
Streaming sql and druid Streaming sql and druid
Streaming sql and druid
 
NoSQL on the move
NoSQL on the moveNoSQL on the move
NoSQL on the move
 
GraphQL
GraphQLGraphQL
GraphQL
 
SQL o NoSQL? Progettare applicazioni 'Big Data-ready' attraverso l'utilizzo d...
SQL o NoSQL? Progettare applicazioni 'Big Data-ready' attraverso l'utilizzo d...SQL o NoSQL? Progettare applicazioni 'Big Data-ready' attraverso l'utilizzo d...
SQL o NoSQL? Progettare applicazioni 'Big Data-ready' attraverso l'utilizzo d...
 
Collaborative environment with data science notebook
Collaborative environment with data science notebook Collaborative environment with data science notebook
Collaborative environment with data science notebook
 
The evolving container landscape
The evolving container landscapeThe evolving container landscape
The evolving container landscape
 
Introduction to GraphQL
Introduction to GraphQLIntroduction to GraphQL
Introduction to GraphQL
 
Евгений Варфоломеев "Hibernate vs my batis vs jdbc: is there a silver bullet?"
Евгений Варфоломеев "Hibernate vs my batis vs jdbc: is there a silver bullet?"Евгений Варфоломеев "Hibernate vs my batis vs jdbc: is there a silver bullet?"
Евгений Варфоломеев "Hibernate vs my batis vs jdbc: is there a silver bullet?"
 
This Week in Neo4j - 24th November 2018
This Week in Neo4j - 24th November 2018This Week in Neo4j - 24th November 2018
This Week in Neo4j - 24th November 2018
 
Shaping serverless architecture with domain driven design patterns
Shaping serverless architecture with domain driven design patternsShaping serverless architecture with domain driven design patterns
Shaping serverless architecture with domain driven design patterns
 
OpenStack MagnetoDB. Atlanta Summit 2014
OpenStack MagnetoDB. Atlanta Summit 2014OpenStack MagnetoDB. Atlanta Summit 2014
OpenStack MagnetoDB. Atlanta Summit 2014
 
Serverless Code Components
Serverless Code Components Serverless Code Components
Serverless Code Components
 
In Memory Databases: A Real Time Analytics Solution
In Memory Databases: A Real Time Analytics SolutionIn Memory Databases: A Real Time Analytics Solution
In Memory Databases: A Real Time Analytics Solution
 
GraphQL
GraphQLGraphQL
GraphQL
 
Graphql
GraphqlGraphql
Graphql
 
Implementing GraphQL API in Elixir – Victor Deryagin
Implementing GraphQL API in Elixir – Victor DeryaginImplementing GraphQL API in Elixir – Victor Deryagin
Implementing GraphQL API in Elixir – Victor Deryagin
 
GraphQL in an Age of REST
GraphQL in an Age of RESTGraphQL in an Age of REST
GraphQL in an Age of REST
 
Hands On - GraphQL
Hands On - GraphQLHands On - GraphQL
Hands On - GraphQL
 
ResourceSpace: Recent pains and future gains
ResourceSpace: Recent pains and future gainsResourceSpace: Recent pains and future gains
ResourceSpace: Recent pains and future gains
 
Kafka Streams
Kafka StreamsKafka Streams
Kafka Streams
 

Similar to NoSQLEU: Different NoSQL tools in Production

Storing eBay's Media Metadata on MongoDB, by Yuri Finkelstein, Architect, eBay
Storing eBay's Media Metadata on MongoDB, by Yuri Finkelstein, Architect, eBayStoring eBay's Media Metadata on MongoDB, by Yuri Finkelstein, Architect, eBay
Storing eBay's Media Metadata on MongoDB, by Yuri Finkelstein, Architect, eBayMongoDB
 
MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB present...
MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB  present...MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB  present...
MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB present...MongoDB
 
GlobalsDB: Its significance for Node.js Developers
GlobalsDB: Its significance for Node.js DevelopersGlobalsDB: Its significance for Node.js Developers
GlobalsDB: Its significance for Node.js DevelopersRob Tweed
 
Building Microservices with .NET (speaker Anton Vasilenko, Binary Studio)
Building Microservices with .NET (speaker Anton Vasilenko, Binary Studio)Building Microservices with .NET (speaker Anton Vasilenko, Binary Studio)
Building Microservices with .NET (speaker Anton Vasilenko, Binary Studio)Binary Studio
 
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)QAware GmbH
 
Overhauling a database engine in 2 months
Overhauling a database engine in 2 monthsOverhauling a database engine in 2 months
Overhauling a database engine in 2 monthsMax Neunhöffer
 
Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010
Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010
Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010Bhupesh Bansal
 
Hadoop and Voldemort @ LinkedIn
Hadoop and Voldemort @ LinkedInHadoop and Voldemort @ LinkedIn
Hadoop and Voldemort @ LinkedInHadoop User Group
 
Strategies for Context Data Persistence
Strategies for Context Data PersistenceStrategies for Context Data Persistence
Strategies for Context Data PersistenceFIWARE
 
Node Js, AngularJs and Express Js Tutorial
Node Js, AngularJs and Express Js TutorialNode Js, AngularJs and Express Js Tutorial
Node Js, AngularJs and Express Js TutorialPHP Support
 
Massively Scaled High Performance Web Services with PHP
Massively Scaled High Performance Web Services with PHPMassively Scaled High Performance Web Services with PHP
Massively Scaled High Performance Web Services with PHPDemin Yin
 

Similar to NoSQLEU: Different NoSQL tools in Production (20)

Storing eBay's Media Metadata on MongoDB, by Yuri Finkelstein, Architect, eBay
Storing eBay's Media Metadata on MongoDB, by Yuri Finkelstein, Architect, eBayStoring eBay's Media Metadata on MongoDB, by Yuri Finkelstein, Architect, eBay
Storing eBay's Media Metadata on MongoDB, by Yuri Finkelstein, Architect, eBay
 
MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB present...
MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB  present...MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB  present...
MongoDB San Francisco 2013: Storing eBay's Media Metadata on MongoDB present...
 
Beginners Node.js
Beginners Node.jsBeginners Node.js
Beginners Node.js
 
GlobalsDB: Its significance for Node.js Developers
GlobalsDB: Its significance for Node.js DevelopersGlobalsDB: Its significance for Node.js Developers
GlobalsDB: Its significance for Node.js Developers
 
Building Microservices with .NET (speaker Anton Vasilenko, Binary Studio)
Building Microservices with .NET (speaker Anton Vasilenko, Binary Studio)Building Microservices with .NET (speaker Anton Vasilenko, Binary Studio)
Building Microservices with .NET (speaker Anton Vasilenko, Binary Studio)
 
Node js
Node jsNode js
Node js
 
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
Kubernetes One-Click Deployment: Hands-on Workshop (Mainz)
 
Nodejs overview
Nodejs overviewNodejs overview
Nodejs overview
 
NoSql Databases
NoSql DatabasesNoSql Databases
NoSql Databases
 
Overhauling a database engine in 2 months
Overhauling a database engine in 2 monthsOverhauling a database engine in 2 months
Overhauling a database engine in 2 months
 
Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010
Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010
Voldemort & Hadoop @ Linkedin, Hadoop User Group Jan 2010
 
Hadoop and Voldemort @ LinkedIn
Hadoop and Voldemort @ LinkedInHadoop and Voldemort @ LinkedIn
Hadoop and Voldemort @ LinkedIn
 
Strategies for Context Data Persistence
Strategies for Context Data PersistenceStrategies for Context Data Persistence
Strategies for Context Data Persistence
 
Node Js, AngularJs and Express Js Tutorial
Node Js, AngularJs and Express Js TutorialNode Js, AngularJs and Express Js Tutorial
Node Js, AngularJs and Express Js Tutorial
 
Massively Scaled High Performance Web Services with PHP
Massively Scaled High Performance Web Services with PHPMassively Scaled High Performance Web Services with PHP
Massively Scaled High Performance Web Services with PHP
 
Meet with Meteor
Meet with MeteorMeet with Meteor
Meet with Meteor
 
Meteor Angular
Meteor AngularMeteor Angular
Meteor Angular
 
Documentation
DocumentationDocumentation
Documentation
 
CouchDB
CouchDBCouchDB
CouchDB
 
Serverless Go at BuzzBird
Serverless Go at BuzzBirdServerless Go at BuzzBird
Serverless Go at BuzzBird
 

NoSQLEU: Different NoSQL tools in Production

  • 1. Notes from the field: Different NoSQL tools in production Matthew Ford
  • 2. We've used: Key Value Stores: Document Stores: Redis MongoDB Tokyo Cabinet CouchDB
  • 3. Redis Project: Gnip Filter Requirements: Users can filter realtime data streams by keywords. Solution: redis.set_add('ipad', 'client_id') In the background we filter the data stream, when a keyword matches, we push that piece of data to all the client ids. redis.set_members('ipad')
  • 4. Redis Benefits: Drawbacks: Fast Not fault tolerant Simple model No replication Sets are useful Whole data set in memory O(1) for set add/remove
  • 5. Tokyo Cabinet Project: tweetarium Requirement: Store tweets from a user to build up twitter lists Solution: Serialized JSON from the twitter API, and arrays of twitter ids for the lists. Running on a small slicehost slice (poor mans memcached) Live at http://tweetarium.com/
  • 6.
  • 7. Tokyo Cabinet Benefits: Drawbacks: Still Fast Documentation mostly in Japanese Persistent Kyoto Cabinet (successor) Space efficient
  • 8. CouchDB Project: Project management app Requirements: Everything needed to have version tracking (snapshots of the entire project and all of it's associations). Solution: Hybrid system, removed version control from SQL and pass a JSON representation of the project data to CouchDB.
  • 9. CouchDB Benefits: Drawbacks: Flexible Library support (at the time) Simplify architecture JS Views, with changing schema Robust HTTP not a fast as binary protocol
  • 10. MongoDB Project: feesability Problem: Case estimation and planning tool, with the need to 'clone cases'. Solution: All cases data is embedded within the case document (with some references to other documents). This enables us to easily clone an entire case.
  • 11. MongoDB Benefits: Drawbacks: Flexible Replication not here yet Fast Libraries are playing catch up Ad-Hoc queries Durability Trade-off Easy to query embedded documents
  • 12. Conclusion Use the right tools for the job