Topic 10: Taxonomy of Data and Storage

10: Taxonomy of Data and Storage
Zubair Nabi
zubair.nabi@itu.edu.pk
April 20, 2013
Zubair Nabi 10: Taxonomy of Data and Storage April 20, 2013 1 / 27

Outline
1 Datasets
2 Storage
3 Beyond RDBMS
4 NoSQL Taxonomy
5 NewSQL

Introduction
Data is everywhere and is the driving force behind our lives

Introduction
The address book on your phone is data

Introduction
So is the newspaper that you read every morning

Introduction
Everything you see around you is a potential source of data which
might be useful for a certain application

Introduction
We use this data to share information and make a more informed
decision about different events

Introduction
Datasets can easily be classiﬁed on the basis of their structure

Introduction
Datasets can easily be classiﬁed on the basis of their structure
1 Structured
2 Unstructured
3 Semi-structured

Structured Data
Formatted in a universally understandable and identiﬁable way

Structured Data
In most cases, structured data is formally speciﬁed by a schema

Structured Data
Your phone address phone is structured because it has a schema
consisting of name, phone number, address, email address, etc.

Structured Data
Most traditional databases contain structured data revolving around
data laid out across columns and rows

Structured Data
Each ﬁeld also has an associated type

Structured Data
Each ﬁeld also has an associated type
Possible to search for items based on their data types

Unstructured Data
Data without any conceptual deﬁnition or type

Unstructured Data
Can vary from raw text to binary data

Unstructured Data
Processing unstructured data requires parsing and tagging on the ﬂy

Unstructured Data
Processing unstructured data requires parsing and tagging on the ﬂy
In most cases, consists of simple log ﬁles

Semi-structured Data
Occupies the space between the structured and unstructured data
spectrum

spectrum
For instance, while binary data has no structure, audio and video ﬁles
have meta-data which has structure, such as author, time of creation,
etc.

spectrum
For instance, while binary data has no structure, audio and video ﬁles
have meta-data which has structure, such as author, time of creation,
etc.
Can also be labelled as self-describing structure

Outline
1 Datasets
2 Storage
3 Beyond RDBMS
4 NoSQL Taxonomy
5 NewSQL

Database Management Systems (DBMS)
Used to store and manage data

Support for large amounts of data

Ensure concurrency, sharing, and locking

Security is useful too; to enable ﬁne-grained access control

Security is useful too; to enable ﬁne-grained access control
Ability to keep working in the face of failure

Relational Database Management Systems (RDBMS)
The most popular and predominant storage system in use

Data in different ﬁles is connected by using a key ﬁeld

Data is laid out in different tables, with a key ﬁeld that identiﬁes each
row

row
The same key ﬁeld is used to connect one table to another

row
For instance, a relation might have customer ID as key and her details
as data; another table might have the same key but different data, say
her purchases; yet another table with the same key might have a
breakdown of her preferences

row
For instance, a relation might have customer ID as key and her details
as data; another table might have the same key but different data, say
her purchases; yet another table with the same key might have a
breakdown of her preferences
Examples include Oracle Database, MS SQL Server, MySQL, IBM
DB2, and Teradata

Structured Query Language (SQL)
Non-procedural language used for data retrieval and manipulation in
RDBMS

RDBMS
Adds a layer of abstraction over relational algebra, which enables set
operations, selections, etc.

RDBMS
Due to its declarative nature, users operate in terms of their expected
output while the underlying system decides the actual query execution
plan

RDBMS
plan
Instructions consist of a speciﬁc SQL statement and additional
parameters and operands

RDBMS
plan
Instructions consist of a speciﬁc SQL statement and additional
parameters and operands
For instance, the SELECT operator retrieves certain records, INSERT
adds a record, and so on

RDBMS and Structured Data
As structured data follows a predeﬁned schema, it naturally maps on to
a relational database system

The schema deﬁnes the type and structure of the data and its relations

Schema design is an arduous process and needs to be done before
the database can be populated

Another consequence of a strict schema is that it is non-trivial to
extend it

extend it
For instance, adding a new attribute to an existing row necessitates
adding a new column to the entire table

extend it
For instance, adding a new attribute to an existing row necessitates
adding a new column to the entire table
Extremely suboptimal in tables with millions of rows

RDBMS and Semi- and Un-structured Data
Unstructured data has no notion of schema while semi-structured data
only has a weak one

only has a weak one
Data within such datasets also has an associated type

only has a weak one
In fact, types are application-centric: It might be possible to interpret a
ﬁeld as a ﬂoat in one application and as a string in another

only has a weak one
While it is possible, with human intervention, to glean structure from
unstructured data, it is an extremely expensive task

only has a weak one
Structureless data generated by real-time sources can change the
number of attributes and their types on the ﬂy

only has a weak one
RDBMS would require the creation of a new table each time such a
change takes place

only has a weak one
RDBMS would require the creation of a new table each time such a
change takes place
Therefore, unstructured and semi-structured data does not ﬁt the
relational model

Outline
1 Datasets
2 Storage
3 Beyond RDBMS
4 NoSQL Taxonomy
5 NewSQL

Motivation
Different semantics:
RDBMS provide ACID semantics:
1 Atomic: The entire transaction either succeeds or fails

Motivation
2 Consistent: Data within the database remains consistent after each
transaction

Motivation
transaction
3 Isolation: Transactions are sandboxed from each other

Motivation
transaction
4 Durable: Transactions are persistent across failures and restarts

Motivation
transaction
Overkill in case of most user-facing applications

Motivation
transaction
Most applications are more interested in availability and willing to
sacriﬁce consistency leading to eventual consistency

Motivation
transaction
This basically available, soft state, eventually consistent (BASE) model
enables applications to function even in the face of partial failure

Motivation
transaction
This basically available, soft state, eventually consistent (BASE) model
enables applications to function even in the face of partial failure
High Throughput: Most NoSQL databases sacriﬁce consistency for
availability leading to higher throughput (in some cases an order of
magnitude)

Motivation (2)
Horizontal Scalability: To cater for more data, NoSQL stores can be
scaled up by just adding more machines and the underlying system
automatically re-distributes the data

Motivation (2)
Commodity Hardware: A large number of RDBMS require specialized
and proprietary hardware for operation. In contrast, NoSQL databases
function over commodity off-the-shelf hardware

Motivation (2)
Commodity Hardware: A large number of RDBMS require specialized
and proprietary hardware for operation. In contrast, NoSQL databases
function over commodity off-the-shelf hardware
Programming Language Support: Over the years programming
languages have started providing abstractions for database support
(LINQ, etc.) while bypassing SQL. NoSQL databases provide
abstractions that directly map onto the language abstractions leading
to tighter coupling

Motivation (3)
The Rise of Cloud Computing: Cloud Computing applications require
horizontal scalability and low administration overhead. Both
requirements are naturally satisﬁed by NoSQL stores

Outline
1 Datasets
2 Storage
3 Beyond RDBMS
4 NoSQL Taxonomy
5 NewSQL

Introduction
NoSQL databases can be classiﬁed on the basis of:
1 Data Model: How data is represented

Introduction
2 Scalability: How scalable the system is

Introduction
3 Query Model: What type of API it exposes

Introduction
3 Query Model: What type of API it exposes
4 Persistence: How persistent the data is

Classiﬁcation by Data Model
Based on the data model, NoSQL databases can roughly be categorized
into three categories:
1 Key/value Stores: A map/dictionary allowing put/get semantics per
key

key
2 Document Stores: Complex data structures to encapsulate document
key/value pairs

key
2 Document Stores: Complex data structures to encapsulate document
key/value pairs
3 Column-Oriented Stores: Data laid out by column

Key/value Stores
Data is stored within a large hash map

Key/value Stores
Simple get/put API

Key/value Stores
Simple get/put API
Favour scalability over consistency

Key/value Stores
Simple get/put API
Limit on the size of the key

Key/value Stores
Simple get/put API
Limit on the size of the key
Examples include Amazon’s Dynamo, LinkedIn’s Voldemort, Redis,
and Memcached

Document Stores
Key/value semantics but based on documents

Document Stores
A document encapsulates data in a standard format, such as JSON,
XML, PDF, etc.

Document Stores
XML, PDF, etc.
Documents themselves can be heterogeneous

Document Stores
XML, PDF, etc.
Documents can also be retrieved based on their content

Document Stores
XML, PDF, etc.
Documents can also be retrieved based on their content
Examples include Apache CouchDB and MongoDB

Column-Oriented Stores
Data is stored and processed by column

Useful for read-mostly and read-intensive data

Data within the same column is of the same type enabling
opportunities for efﬁcient compression

Columns are stored separately so they can be loaded in parallel

Columns are stored separately so they can be loaded in parallel
Examples include Google’s BigTable (Apache HBase is its open source
clone) and Facebook’s Cassandra

Outline
1 Datasets
2 Storage
3 Beyond RDBMS
4 NoSQL Taxonomy
5 NewSQL

Introduction
A hybrid of traditional RDBMS and NoSQL

Introduction
Scalability and performance of NoSQL and ACID guarantees of RDBMS

Introduction
Use SQL as the primary language

Introduction
Ability to scale out and run over commodity hardware

Introduction
Classiﬁed into:
1 New Databases: Designed from scratch

Introduction
Classiﬁed into:
2 New MySQL Storage Engines: Keep MySQL as interface but replace
the storage engine

Introduction
Classiﬁed into:
2 New MySQL Storage Engines: Keep MySQL as interface but replace
the storage engine
3 Transparent Clustering: Add pluggable features to existing databases
to ensure scalability

New Databases
1 Query Distribution:
Each node holds a subset of the data

New Databases
Queries are split and shipped to nodes that own the data

New Databases
Examples include Google’s Spanner and NuoDB

New Databases
2 Pull Data:
A central node (possibly replicated) holds all data

New Databases
2 Pull Data:
A set of processing nodes receives queries and pulls in required data
from the central node

New Databases
2 Pull Data:
A set of processing nodes receives queries and pulls in required data
from the central node
Examples include VMware’s SQLFire

References
1 NoSQL Databases: https:
//oak.cs.ucla.edu/cs144/handouts/nosqldbs.pdf
2 NewSQL – The New Way to Handle Big Data: http://www.
linuxforu.com/2012/01/newsql-handle-big-data/

Topic 10: Taxonomy of Data and Storage

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Topic 10: Taxonomy of Data and Storage

Similar to Topic 10: Taxonomy of Data and Storage (20)

More from Zubair Nabi

More from Zubair Nabi (20)

Recently uploaded

Recently uploaded (20)

Topic 10: Taxonomy of Data and Storage