N-O-SQL, new database technologies on the rise

N-O-SQL
new database technologies on the rise

http://www.ﬂickr.com/photos/wolfgangstaudt/2215246206/

IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org

Who am I

» Steven Noels - stevenn@outerthought.org

» Outerthought : scalable content applications

» makers of Daisy and Lily open source CMS

IIC » TECHNOLOGIEPARK 3 » B-9052 ZWIJNAARDE (GENT) » www.outerthought.org 2

Agenda

» raison d’être: what brought us here

» concepts: required theory readings

» market overview: trees & the forest

» experiences and (h)in(d)sights


Raison d’être


History

2. simpliﬁcation

1. standardization
hierarchical databases

IMS
XMLDB RDBMS

OODBMS


Inconsistency through slave lag

John Quinn (Digg)

Scaling writes (1)

John Quinn (Digg)

Scaling writes (2)

John Quinn (Digg)

Issues with partitioning

» lose the ability to make arbitrary queries

» have to predict data access patterns when
formulating partitioning strategy
» complex and fragile systems


Replication complexity


Scaling relational systems

» When scaling relational systems you loose
their advantages but retain their overhead


History

4. rethinking
the problem

RDBMS NOSQL

caching
denormalisation
sharding
replication ...
3. scaling


Moore vs Kryder

» seek time is
constant (network
latency as well?)
» transfer rate ! spindles !

» as a principle, writes are
hard to scale


Cambrian Explosion


Buzz-oriented
development

?

Cambrian Explosion

N-O-SQL


The Perspective of Cost


Common themes

» SCALE SCALE SCALE

» new datamodels

» devops

» N-O-SQL

» The Cloud :
technology is of no interest anymore


Numbers of scale

http://qos.doubleclick.net/counters/


Types of scaling
» scaling for usage » scaling types of ops
» volume of users » concurrent read
» volume of data » concurrent write

availability partioning
replication consistency

distribution


Distributed
systems are
hard !

8 fallacies of distributed computing
» The network is reliable.

» Latency is zero.

» Bandwidth is inﬁnite.

Peter Deutsch and James Gosling
» The network is secure.

» Topology doesn't change.

» There is one administrator.

» Transport cost is zero.

» The network is homogeneous.


New Data

» sparse structures

» weak schemas

» graphs

» semi-structured

» document-oriented


N-O-SQL =
not only SQL !


The NOSQL footprint
free-structured or sparse data

NOSQL

MongoDB
CouchDB
neo4j

Cassandra

available (complexity)
simple operational

HBase

highly scalable and
constraints
ACID,

SQL

referential integrity,
typed data


NOSQL, if you need ...

» horizontal scaling (out rather than up)

» unusually common data (aka free-structured)

» speed (especially for writes)

» the bleeding edge


SQL/RDBMS, if you need ...

» SQL

» ACID

» normalisation

» a deﬁned liability


Theory


Robust systems


Academic background

» Amazon Dynamo

» Google BigTable

» Eric Brewer CAP theorem


Amazon Dynamo
» coined the term ‘eventual consistency’

» consistent hashing


Consistent hashing

http://horicky.blogspot.com/2009/11/nosql-patterns.html


Consistent hashing

- node C
+ node D

http://www.lexemetech.com/2007/11/consistent-hashing.html


Google BigTable
» multi-dimensional column-oriented database

» on top of GoogleFileSystem

» object versioning


CAP theorem

strong high
consistency availability

partition-
tolerance


CAP
» Strong Consistency: all clients see the
same view, even in the presence of updates
» High Availability: all clients can ﬁnd some
replica of the data, even in the presence of
failures
» Partition-tolerance: the system
properties hold even when the system is
partitioned


Consistency

» Where is my data I just updated?

» Ideal world :

The result of every write-operation is
reﬂected by subsequent read-operations.


Consistency


Sunny-day scenario


Network partioning


Culture Clash
» Classic distributed systems: focus on ACID
» atomic
» consistent
» isolated
» durable

» Modern internet systems: focus on BASE
» basically available
» soft-state (or scalable)
» eventually consistent


Culture Clash
» ACID » BASE
» highest priority: strong » availability and scaling
consistency for highest priorities
transactions » weak consistency
» availability less important
» optimistic
» pessimistic
» best effort
» rigorous analysis
» simple and fast
» complex mechanisms

spectrum


Building for failure

» defensive programming

» creating replicas

» disk ﬂushing

» watch out for failure of utility infrastructure

» conscious sync/async decisions


Possible storage failures
» Application errors

» Repeatable DB failures

» Unrepeatable DB failures

» OS errors

» Local cluster HW failure

Michael Stonebreaker
» Local cluster network partitioning

» Disaster

» WAN network failure between remote clusters


Availability ≠
total async !


✘
The Enterprise Service Bus

bus =

congestion


Bus systems

» objects don’t ﬁt in a pipe

» object ➙ message

» serialization / de-serialization cost

» message size

» queuing = cost


Use a mixture of both

»async + sync

stuff which matters !


Numbers of scale

http://qos.doubleclick.net/counters/


Processing large datasets :

Map/Reduce


Smart Data
» sparse as a feature

» weak schemas

» ad-hoc indexing

» organic analytics

» near-data processing

» live(ly) datawarehouse

» distribution ➙ parallellization ➙ performance


Hadoop: HDFS + MapReduce
» single ﬁlesystem + single execution-space


MapReduce example: WordCount


MapReduce


MapReduce and HDFS

© lars george


Physical architecture


Processing large datasets with MR

» Beneﬁt from parallellisation

» Less modelling upfront (ad-hoc processing)

» Compartmentalized approach reduces
operational risks
» AsterData et al. have SQL/MR hybrids for
huge-scale BI


Market
overview


Categories

» key-value stores

» column stores

» document stores

» graph databases


Key-value stores

» Redis

» Voldemort

» Tokyo Cabinet


Redis

» REmote DIctionary Server

» http://code.google.com/p/redis/

» vmware


Redis Features
» persisted memcache, ‘awesome’

» RAM-based + persistable

» key ➙ values: string, list, set

» higher-level ops
» i.e. push/pop and sort for lists
» fast (very)

» conﬁgurable durability

» client-managed sharding


Voldemort

» http://project-voldemort.com/

» LinkedIn


Voldemort

» persistent

» distributed

» fault-tolerant

» hash table


Voldemort

API: GET, PUT,
DELETE


Voldemort

routing logic moving up the stack,
smaller latency


Voldemort data format

» key+values = arrays of bytes

» So how do we objects ⬌ bytes ?

» json
» string
» java-serialization
» protobuf
» identity


Tokyo Cabinet

» http://1978th.net/tokyocabinet/

» mixi.jp (i.e. Facebook Japan)


Product Family


Tokyo Cabinet
» memory or ﬁlesystem

» hash, b-tree, ﬁxed-length, table


Column stores

» BigTable

» HBase

» Cassandra


BigTable

» http://labs.google.com/papers/bigtable.html

» Google

» layered on top of GFS


HBase

» http://hadoop.apache.org/hbase/

» StumbleUpon / Adobe / Cloudera


HBase
» sorted » persisted
» distributed » storage system
» column-oriented
» multi-dimensional
» highly-available » adds random access
» high-performance reads and writes atop
HDFS


HBase data model
» Distributed multi-dimensional sparse map

» Multi-dimensional keys:
(table, row, family:column, timestamp) → value

» Keys are arbitrary strings

» Access to row data is atomic

Storage architecture

© lars george


Cassandra

» http://cassandra.apache.org/

» Rackspace / Facebook


Cassandra
» Key-value store (with added structure)

» Reliability (identical nodes)

» Eventual consistent

» Distributed
A
C
» Tunable
» Partitioning
P
» Replication


Cassandra write pattern


Cassandra applicability
FIT NO FIT

» Scalable reliability » Flexible indexing
(through identical » Only PK-based
nodes) querying
» Linear scaling » Big Binary Data
» Write throughput » 1 Row must ﬁt in
» Large Data Sets RAM entirely


Document stores

» CouchDB

» MongoDB

» Riak

» MarkLogic


CouchDB

» http://couchdb.apache.org/

» couch.io


CouchDB

» fault-tolerant

» schema-free

» document-oriented

» accessible via a RESTful HTTP/JSON API


CouchDB documents

{
“_id”: ”BCCD12CBB”,
“_rev”: ”AB764C”,
“type”: ”person”,
“name”: ”Darth Vader”,
“age”: 63,
“headware”: [“Helmet”, “Sombrero”],
“dark_side”: true
}


CouchDB REST API

» HTTP
» PUT /db/docid
» GET /db/docid
» POST /db/docid
» DELETE /db/docid


CouchDB Views
» MapReduce-based

» Filter, Collate, Aggregate

» Javascript

map reduce
function (doc) { function (Key, Values) {
for(var i in doc.tags) var sum = 0;
emit(doc.tags[i], 1); for(var i in Values)
} sum += Values[i];
return sum;
}


CouchDB

» be careful on semantics
» replication ≠ partioning/sharding !
» distributed database = distributable database

» sharded / distributed deployment
requires proxy layer


MongoDB

» http://www.mongodb.org/

» 10gen


MongoDB
» cfr. CouchDB, really

» except for:
» C++
» performance focus
» runtime queries (mapreduce still available)
» native drivers (no REST/HTTP layering)
» no MVCC: update-in-place
» auto sharding (alpha)


Riak

» http://riak.basho.com/

» Basho Technologies


Riak

» buckets/keys, links

» values/content = bucket + metadata

» pluggable storage engines (fs, (D)ETS, InnoDB)

» HTTP/REST API

» automatic distribution

» mapreduce using Javascript


Jackrabbit

» http://jackrabbit.apache.org/

» Day Software


Jackrabbit

» reference
implementation for
JSR 170 & 283
» remoting: WebDAV &
RMI
» persistence: RDBMS,
fs, memory


Jackrabbit

» Java-centric (duh)

» complex repository model (nodes+properties)
» mixins, inheritance

» workspaces

» query language

» no partioning/sharding


JCR API levels


Graph databases

» Neo4j

» AllegroGraph (RDF)


Neo4j

» http://neo4j.org/

» Neo Technology


Neo4j
» data = nodes + relationships + key/value properties


Neo4j
» many language bindings, little remoting

» ‘whiteboard’ friendly

» scaling to complexity (rather than volume?)

» lots of focus on domain modelling

» SPARQL/SAIL impl for triple geeks

» mostly RAM centric (with disk swapping &
persistence)


Experiences &
(h)in(d)sights


NOSQL applicability

» Horizontal scaling

» Multi-Master

» Data representation
» search of simplicity
» data that doesn’t ﬁt the E-R model
(graphs, trees, versions)
» Speed


Tools for the trade

» non-relational data: Couch, Mongo, Riak

» massive quantities: Cassandra, HBase

» persistent caching: Redis, Voldemort

» graphs: neo4j


Tool selection
» be careful on the marketeese:
smoke and mirrors beware!
» monitor dev list, IRC, Twitter, blogs

» monitor project ‘sponsors’

» mix-and-match

» DON’T NOSQL WITHOUT INTERNAL SYS
ARCHS & DEV(OP)S !


}
aptness

NOSQL
internet
enterprise

}

SQL
corporate
community

complexity

Our NOSQL-based project: Lily

» (open source)

» scalable store (Apache HBase)

» and search (Apache SOLR)

» content repository

» α due mid 2010

» www.lilycms.org or @outerthought


Lily architecture
distributed process coordination
and conﬁguration (ZooKeeper)

}
query update indexer
Lily
Lily Lily Store Server
store
client
node WAL MQ M/R

client

}
store
node 2ary WAL / HBase Region Server
documents
indexes MQ
client

store
node

} Hadoop DFS

REST

index
replica
inverted index

replica replica
} SOLR


When combining store
and search, make sure
your (search) index
doesn’t become the
store.


Key lessons learned

» importance of keyspace design

» secondary indexing

» data de-normalization

» schema vs. code ﬂexibility?

» distribution is everywhere
and you shouldn’t forget about it


Reading material

» Amazon Dynamo, Google BigTable, CAP

» http://nosql.mypopescu.com/

» http://nosql-database.org/

» http://twitter.com/nosqlupdate

» http://highscalability.com/


Questions?

http://www.ﬂickr.com/photos/leehaywood/4237636853/


Thanks for your
attention !

» stevenn@outerthought.org

» @stevenn


N-O-SQL, new database technologies on the rise

More Related Content

Similar to N-O-SQL, new database technologies on the rise

More from NGDATA

Recently uploaded

N-O-SQL, new database technologies on the rise