The document discusses the rapid growth of data on the web and how NoSQL databases provide an alternative to traditional relational databases by being able to handle massive amounts of unstructured and semi-structured data across a large number of servers in a simple and scalable way. It reviews different types of NoSQL databases like key-value stores, document databases, and graph databases and provides examples of popular NoSQL databases like MongoDB, CouchDB, HBase, and Neo4j that are being used by large companies to store and query large datasets.

1. NoSQL
In the Context of Social Web
Summer of Web 2010
Bogdan Gaza

2. About me
• Student at Faculty of Computer Science -
first year
• Ruby & Rails fan
• Building RailsAdmin for RubySOC 2010
• Interested in Scalability and High-Availability
• http://twitter.com/hurrycane

3. Data, data Everywhere

4. Data growth on the web
• Facebook Photos +25TB/week
• Twitter +7TB/day
• Flickr +21GB/hour
• +150GB of tweets while I give this talk
• All this amounts multiply every year

5. Data size
988.00
1000.00
750.00
623.00
500.00
397.00
253.00
250.00
161.00
0
2006 2007 2008 2009 2010
ExaBytes of data stored on the web 1ExaByte = 1018 bytes (source IDC)

6. How to store this huge amount of data?

7. Databases
More T RDBMS
MySQL, Oracle
Online transaction
processing (OLTP)
NoSQL
Mongodb, couchdb
Less T

8. RDBMS
• Relational database management system
• Based on E.F. Codd - relational model - 1969
• Dominant for transactional & analytical
applications
• Most popular and easy to use RDBMS is
MySQL

9. RDBMS performance
SalaryList
Majority of
Web Apps
Performance
Social networks
Trend analysis
Data complexity

10. NoSQL
• Doesn’t mean No to SQL
• It actually means Not Only SQL
• A class of data stores that may not require
fixed table schemas (wikipedia)
• Majority of them based on Google’s BigTable

11. NoSQL Categories
• Key-Value stores: Voldemort
• BigTable clones: HBase
• Document Databases: Mongodb, Couchdb
• Graph Databases: Neo4j

12. Common grounds
• They all support huge amount of data
• The majority of them supports replication
and sharding
• Have some sort of failure detection
mechanism
• Can scale to the complexity of data they
store

13. Key-Value stores
• Scales to huge amount of data
• Can handle massive load
• Based on Amazon’s Dynamo
• A big persistent associative-array
• Examples: Voldermort, Tokyo Tyrant/Cabinet

14. BigTable clones
• Tables similar to RDBMS but semi-
structured
• Based on Google’s BigTable paper
• Column oriented
• Examples: HBase, Cassandra

15. Document databases
• Similar to Key-Value Stores but the DB
knows what the Value is
• Collection of key-value collections
• Documents are often versioned
• Example: CouchDB, MongoDB, Redis

16. Graph databases
• Focus in structure of data
• Scales to the complexity of data
• Data stored in nodes
• Lots of cool Graph algorithms can be
implemented
• Examples: Neo4J, FlockDB

17. But how do I query it?
• RESTful interface
• QueryAPIs
• SPARQL
• Gremlin - graph traversal database
• GQL - SQL-like Query Lange for Google BT

18. Who uses NoSQL?

19. Who uses NoSQL?
BigTable Cassandra
FlockDB
Dynamo Voldemort
MongoDB

20. • Scalable, high-performance, open-source,
document oriented database
• Somewhere between key-value stores and
document databases
• Big community
• Tons of features

21. • JSON-style documents
{ author: 'joe', created : new
Date('03-28-2009'), title : 'Yet another blog
post' }
• Schema free
• Flexibility
• Data is stored in collections

22. • Dynamic queries
db.people.update( { name:"Joe" }, { $inc: { n : 1 } } );
• Replication - very easy to set up
• Indexing
• Sharding
• GeoSpatial Index - location based queries

23. • Map Reduce search
res = db.events.mapReduce(m, r, { query : {type:'sale'} });
• Simple querying
db.users.find({'last_name': 'Smith'})
• GridFS - for storing large files
• Support for many programming languages

24. Don’t search for:
One database to rule them all!

25. Use the best suited storage for each kind of data

27. Thanks!

28. One more thing!

30. Hummingbird
• Real time Web Traffic Visualiser
• Let’s you see visitors interacting with your
site in real time
• Did a say real time?

31. Hummingbird technology
• Node.js
Evented TCP server written in JavaScript
powered by the V8 JS engine.
• WebSocks
• SVG Graphs
• Real time = 20 times per second

32. Hummingbird technology
• Tracking pixel to gather data
• Build over