This document summarizes a presentation on using non-relational databases (NOSQL) for data storage in the cloud. It discusses typical motivations for cloud solutions like low costs and scalability. It then covers issues with using relational databases to scale horizontally and vertically. Various NOSQL database models are introduced, including key-value, document, columnar, and graph databases. Tradeoffs between consistency, availability, and partition tolerance are discussed in relation to Brewer's CAP theorem. Code examples are provided for different NOSQL databases.

1. 19. und 20. Oktober 2010 in Nürnbergwww.prioconference.de
NOSQLDatenspeicherung in der Cloud
Sergey Shishkin
sergey.shishkin@mt-ag.com
http://shishkin.org
@sshishkin

3. Setting Expectations
• Motivators for cloud solutions
• Distributed database issues
• NOSQL solutions
• Code examples
• It’s not a deep dive session

5. Typical Motivators
• Low cost start-up
• Grow fast
• Peak loads
• Security
• Maintenance costs

7. Commodity Infrastructure
• Infrastructure-, Platform as a Service
• Raise the level of abstraction
– Computational power
instead of CPU, GHz and L2 Cache
– Persistent storage
instead of SATA, RAID and SCSI
– Communication channels
instead of Ethernet, DHCP and TCP
• Everything will be in the cloud

9. Who can benefit from Cloud?
• Web 2.0 start-ups
• Social media

10. Source: http://www.appdata.com/leaderboard/apps

11. Casual Online Games
• Huge opportunity for start-ups
• Low cost, low risk
• Grow fast

13. 60 Million Active Monthly Users
• Don’t know how FarmVille actually works
• But their scalability requirements are huge

15. Classical Web-farm Scale

17. Data persistence is a bottleneck
• ACID Transactions
• Locks are expensive
• RDBMS power of choice
– Optimized for massive writes
– Or for ad-hoc structured queries
– What a surprise!
• How does RDBMS scale?

19. Vertical Scale
• Costs grow exponentially

21. Horizontal Scale
• Master-Slave Replication
– Still bad for massive writes
• Master-Master Replication
– Resolving conflicts is hard in a normalized model
• Sharding/Partitioning
– Give up unique constraints, foreign keys, joins

23. What was the point of all that again?
Pretty much nothing left over of RDBMS.

24. Consistency
All clients always
have the same
view of the data
Availability
Each client can
always read and
write
Partition
Tolerance
The system works
well despite
physical network
partitions
Brewer’s CAP-
Theorem

25. Meet CAP Theorem
• You can choose only two!

26. Visual Guide
to NOSQL
SimpleDB
AzureTS
CouchDBRDBMS
Consistency
All clients always
have the same
view of the data
Availability
Each client can
always read and
write
Partition
Tolerance
The system works
well despite
physical network
partitions
MongoDB

27. Key-Value Store
Collection:
BlogPosts
key: 1
ew0KICAgIHRpdGxlOiC
TQ291Y2hEQpQsDQogIC
AgY29udGVudDogk4WUD
Qp9DQo=
key: 2
ew0KICAgIHRpdGxlOiC
TTW9uZ29EQpQsDQogIC
AgY29udGVudDogk4WUD
Qp9DQo=

29. Document Store
Collection:
BlogPosts
key: 1
{
title: “CouchDB”,
content: “…”
links: [“…”, “…”]
}
key: 2
{
title: “MongoDB”,
content: “…”
}

30. Domain:
BlogPosts
Item: 1
Column:
Title
Value:
“SimpleDB”
Column:
Content
Value: “…”
Colulmn:
Links
Value:
amazonwebservices.com
Value: amazon.com
Item: 2
...
Column Store

32. Data Storage Ecosystem
• NOSQL = Not Only SQL
• RDBMS still has its strengths
– Unique constraints
– But they don’t scale horizontally
• You don’t need a hammer when you have the
right tools ;)

33. Düsseldorf 2010
25. November 2010, Ratingen
http://cloudcamp.org/dusseldorf

34. Beratung Projekte
Schulungen
Application Lifecycle Design Entwicklung
Architektur SOA Cloud Computing
BalckeBalcke-Dürr-Allee 9, 40882 Ratingen www.mt-ag.com info@mt-ag.com

35. Links
• Code Examples
– MongoDB
• http://github.com/shishkin/MyBlog/tree/mongodb/MyBlog.Web/Data
– CouchDB
• http://github.com/shishkin/MyBlog/tree/couchdb/MyBlog.Web/Data
– SimpleDB
• http://github.com/shishkin/MyBlog/tree/simpledb/MyBlog.Web/Data
– Azure Table Service
• http://github.com/shishkin/MyBlog/tree/azure/MyBlog.Web/Data
• Full Visual Guide to NOSQL
– http://blog.nahurst.com/visual-guide-to-nosql-systems

36. Images
• http://www.flickr.com/photos/brianauer/2599299352/ (swimmingpool)
• http://www.flickr.com/photos/orangeacid/227642583/ (cloud)
• http://www.flickr.com/photos/runningclouds/3220810175/ (city in clouds)
• http://isparade.jp (twitter parade)
• http://www.flickr.com/photos/seeminglee/4090890825/ (farmville)
• http://www.flickr.com/photos/icatus/2992269179/ (bottleneck)
• http://www.flickr.com/photos/theplanetdotcom/4879421740/ (data center)
• http://www.flickr.com/photos/redbullfanclub/3788029453/ (f1)
• http://www.flickr.com/photos/stevendepolo/4536694260/ (broken glass)
• http://www.flickr.com/photos/starstreak007/3232853321/ (toy car)
• http://browsertoolkit.com/fault-tolerance.png
• http://www.flickr.com/photos/ebarney/3348965007/ (tools)