Topic 12: NoSQL in Action

12: NoSQL in Action
Zubair Nabi
zubair.nabi@itu.edu.pk
April 20, 2013
Zubair Nabi 12: NoSQL in Action April 20, 2013 1 / 33

Outline
1 Amazon’s Dynamo
2 MongoDB
3 Google BigTable
4 Cassandra

Introduction
At the forefront of the NoSQL movement and has inﬂuenced the design
of many subsequent systems

Introduction
At the forefront of the NoSQL movement and has inﬂuenced the design
of many subsequent systems
Design considerations are two-fold: 1) Infrastructure and 2) Business

Infrastructure Considerations
Tens of thousands of servers and network elements distributed across
the globe

the globe
Commodity off-the-shelf hardware
Failure is normal

the globe
Commodity off-the-shelf hardware
Failure is normal
Hundreds of services, all decentralized and loosely coupled

Business Considerations
Strict, internal SLAs regarding performance, reliability, and efﬁciency

Reliability is of paramount importance because an outage means loss
in revenue and customer trust

The platform needs to be highly scalable, to support continuous growth

Most services only store and retrieve data by primary key, such as best
sellers lists, shopping carts, etc.

Most services only store and retrieve data by primary key, such as best
sellers lists, shopping carts, etc.
No need for complex querying and management afforded by RDBMS

Design
1 Implemented as a partitioned system with replication and consistency
windows

Design
windows
2 Targets applications that require weaker consistency

Design
windows
3 Gives high availability

Design
windows
4 Possibility for write operations even in the presence of partitioning
amongst replicas

Design
windows
4 Possibility for write operations even in the presence of partitioning
amongst replicas
5 Always writeable so conﬂict resolution needs to happen during reads

Conﬂict Resolution
A datastore can only perform simple conﬂict resolution

Passes the buck to the application

The application is aware of the data schema and hence better suited to
choose a conﬂict resolution mechanism

The application is aware of the data schema and hence better suited to
choose a conﬂict resolution mechanism
If the application does not want to implement conﬂict resolution, simple
mechanisms, such as “last write wins” provided by the framework

Interface
1 Simple key/value interface storing values as BLOBs

Interface
2 Operations limited to one key/value pair at a time

Interface
2 Operations limited to one key/value pair at a time
3 No support for hierarchichal namespaces (like those in ﬁlesystems)

Node Assignment
Completely decentralized so all nodes have equal responsibilities

Node Assignment
Completely decentralized so all nodes have equal responsibilities
As nodes can be heterogeneous, work is distributed proportional to the
capabilities of a node

Operations
Provides two operations:

Operations
1 get(key), returns a list of objects and a context

Operations
2 put(key, context, object)

Operations
get can return more than one object if more than one conﬂicting
versions

Operations
versions
The context contains system metadata such as the object version

Operations
versions
The context contains system metadata such as the object version
Keys and values are stored as an array of bytes, and only interpreted
by the application

Partitioning
MD5 hash of keys determines their storage nodes

Partitioning
Consistent hashing to provide incremental scalability

Partitioning
Consistent hashing to provide incremental scalability
Partitioning done across virtual nodes instead of physical ones to take
hardware heterogeneity into account

Outline
1 Amazon’s Dynamo
2 MongoDB
3 Google BigTable
4 Cassandra

Introduction
Schemaless document database in C++

Introduction
Used by a large number of organizations including SourceForge.net.
foursquare, the New York Times, bit.ly, Craigslist, SAP, MTV, EA
Sports, github, etc.

Introduction
Used by a large number of organizations including SourceForge.net.
foursquare, the New York Times, bit.ly, Craigslist, SAP, MTV, EA
Sports, github, etc.
Databases are distributed over multiple servers

Databases and Collections
Databases contain collections (“named groupings”) of documents

Documents within a collection might be heterogeneous

But a good strategy is to create a database collection for each object
type

But a good strategy is to create a database collection for each object
type
A collection is created automatically whenever the ﬁrst document is
inserted into the database

Hierarchical Namespaces
Documents can be organized into a hierarchical structure using a
dot-notation

dot-notation
For instance, the collections wiki.articles, wiki.categories
and wiki.authors exist within the namespace wiki

dot-notation
For instance, the collections wiki.articles, wiki.categories
and wiki.authors exist within the namespace wiki
The collection namespace itself is ﬂat, hierarchical structure only for
the user

Documents
Unit of data storage

Documents
Conceptually similar to an XML document, JSON document, etc.

Documents
Documents are persisted in Binary JSON (BSON)

Documents
Easy to convert between BSON and JSON and between BSON and
other programming language structures

Documents
Easy to convert between BSON and JSON and between BSON and
other programming language structures
Possible to insert (insert), search (find), and update a document
(save)

Datatypes
Scalar: boolean, integer, double

Datatypes
Character sequence: string, code, etc.

Datatypes
BSON-objects: object

Datatypes
Object ID: To identify documents within a collection

Datatypes
Object ID: To identify documents within a collection
Misc: null, array, date

References
No mechanism for foreign keys

References
No mechanism for foreign keys
References between documents need to be resolved by client
applications

Transaction Properties
Atomicity for only update and delete operations

Allows code to be executed locally on database nodes (server-side
code execution)

code execution)
Three different strategies for server-side execution:

code execution)
1 Execution of arbitrary code on a single node via eval operator

code execution)
2 Aggregation via count, group, and distinct

code execution)
2 Aggregation via count, group, and distinct
3 MapReduce code execution on multiple nodes

Outline
1 Amazon’s Dynamo
2 MongoDB
3 Google BigTable
4 Cassandra

Introduction
Supports a relaxed relational model that is dynamically controlled by
the clients

Introduction
the clients
Clients can reason about the locality properties of the data

Introduction
the clients
Data indexing can be row-wise as well as column-wise

Introduction
the clients
Data can be delivered either out of memory or from disk

Introduction
the clients
Data can be delivered either out of memory or from disk
Used internally by Google for more than 60 projects including Google
Earth, Google Analytics, Orkut, and Google Docs

Data Model
Values stored as arrays of bytes which need to be interpreted by the
clients

Data Model
clients
Values are addressed by a 3-tuple (row-key, column-key,
timestamp)

Data Model
clients
timestamp)
Row keys are strings of up to 64KB

Data Model
clients
timestamp)
Rows are maintained in lexicographic order and are dynamically
partitioned into tablets

Data Model
clients
timestamp)
The unit of distribution and load balancing

Data Model
clients
timestamp)
Reads can be made efﬁcient (only having to access a small number of
servers) by wisely choosing row keys

Data Model
clients
timestamp)
Row ranges with small lexicographic distances are partitioned into fewer
tablets

Data Model
clients
timestamp)
Row ranges with small lexicographic distances are partitioned into fewer
tablets
For instance storing URLs in reverse order: com.cnn.blogs,
com.cnn.www, etc.

Columns
No limit on the number of columns per table

Columns
Columns grouped into sets called column families based on their key
preﬁx

Columns
preﬁx
Basic unit of access control

Columns
preﬁx
Expected to store the same or similar type of data so that it can be
compressed

Columns
preﬁx
Expected to store the same or similar type of data so that it can be
compressed
Need to be created before data can be stored in a column

Timestamps
64-bit integers that represent different versions of a cell value

Timestamps
Value assigned by either the datastore or the client

Timestamps
Cells ordered in decreasing order of their timestamp

Timestamps
Cells ordered in decreasing order of their timestamp
Automatic garbage collection can be used to remove revisions

API
Read operations for lookup, selection, etc.

API
Write operations for creation, update, and deletion of values

API
Write operations for tables and column families for creation and
deletion

API
deletion
Administrative operations to modify store conﬁguration and metadata

API
deletion
MapReduce hooks

API
deletion
MapReduce hooks
Transactions are atomic at the single-row level

Architecture
Implemented atop GFS

Architecture
Implemented atop GFS
Multiple tablet servers and a single master

HBase
Open source clone of HBase in Java

HBase
Implemented atop HDFS

HBase
HBase can be the source and/or the sink of Hadoop jobs

HBase
HBase can be the source and/or the sink of Hadoop jobs
Facebook Chat implemented using HBase

Outline
1 Amazon’s Dynamo
2 MongoDB
3 Google BigTable
4 Cassandra

Introduction
Borrows concepts from both Dynamo and BigTable

Introduction
Originally developed by Facebook but now an Apache open source
project

Introduction
Originally developed by Facebook but now an Apache open source
project
Designed for Facebook Chat for efﬁciently storing, indexing, and
searching messages

Design Goals
Processing of a large amount of data

Design Goals
Highly scalable

Design Goals
Highly scalable
Reliability at a massive scale

Design Goals
Highly scalable
Reliability at a massive scale
High throughput writes without sacriﬁcing read efﬁciency

Data Model
A table is a distributed multidimensional map indexed by a key

Data Model
Rows are identiﬁed by a string-key and operations over them are
atomic per replica regardless of the number of columns

Data Model
Column families encapsule columns and super columns

Data Model
Columns have a name and store a number of values per row, each with
a timestamp

Data Model
a timestamp
Super columns are columns with sub columns

Data Model
a timestamp
Super columns are columns with sub columns
Only three operations to get, insert, and delete

References
1 NoSQL Databases: https:
//oak.cs.ucla.edu/cs144/handouts/nosqldbs.pdf

Topic 12: NoSQL in Action

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (6)

Similar to Topic 12: NoSQL in Action

Similar to Topic 12: NoSQL in Action (20)

More from Zubair Nabi

More from Zubair Nabi (20)

Recently uploaded

Recently uploaded (20)

Topic 12: NoSQL in Action