Ensuring Technical Readiness For Copilot in Microsoft 365
NoSQL Seminer
1.
Presented by:
Partha Pratim Das
5th Semester
M.Tech. in Computer Sc. And Applications
Registration no.:053792 of 2006-07
Roll no.: 97/CSA/111002
No SQL
Feb 19, 2014
1
2. Introduction to NOSQL
SQL v/s NoSQL
Architecture of NoSQL
ACID v/s BASE
Examples of NOSQL databases
NOSQL vs SQL
Conclusion
No SQL
Feb 19, 2014
2
3.
Database – is a organized collection of inter-related data.
Data base Management System (DBMS)- is a software
package with computer program that controls the
creation , maintenance & use of a database in a
convenient and efficient way.
◦ for DBMS , we use structured language to interact with it
◦ Ex. Oracle , IBM DB2 , Ms Access , MySQL , FoxPro etc.
Relational DBMS - A relational database is a
collection of data items organized as a set of formally
described tables from which data can be accessed
easily. A relational database is created using
the relational model. The software used in a relational
database is called a relational database management
system (RDBMS).
No SQL
Feb 19, 2014
3
4.
Structured Query Language
Special purpose programming language designed for
managing data in Relational DBMS.
Originaly based upon relational algebra & tuple relation
calculus.
SQl’s scope include data insertion, updation & deletion,
schema creation and modification , data access control.
It is static and strongly used in database.
Most widely used database language.
Query is the most important operation in SQL.
Ex. SELECT *
FROM Book
WHERE price > 100.00
ORDER BY title;
No SQL
Feb 19, 2014
4
5. Stands for Not Only SQL
Class of non-relational data storage systems
Usually do not require a fixed table schema or the
concept of joins
All NOSQL offerings relax one or more of the
ACID properties .
◦ Atomicity , Consistency , Isolation , Durability ( ACID )
“NOSQL” = “Not Only SQL” =
Not Only using traditional relational DBMS
No SQL
Feb 19, 2014
5
6. •
Alternative to traditional relational DBMS
• Flexible schema
• Quicker/cheaper to set up
• Massive scalability
• Relaxed consistency → higher performance &
availability
* No declarative query language → more programming
* Relaxed consistency → fewer guarantees
No SQL
Feb 19, 2014
6
7. Every problem cannot be solved by traditional
relational database system exclusively.
Handles huge databases.
Redundancy, data is pretty safe on commodity
hardware
Super flexible queries using map/reduce
Rapid development (no fixed schema)
Very fast for common use cases
No SQL
Feb 19, 2014
7
8. Inspired
by Distributed Data Storage problems
Scale easily by adding servers
Not suited to all problem types, but super-suited to
certain large problem types
High-write situations (eg activity tracking or timeline
rendering for millions of users)
A lot of relational uses are really dumbed down (eg
fetch by PK with update)
No SQL
Feb 19, 2014
8
10.
Clients know how to:
Send items to servers (consistent hashing)
What to do when a server fails
How to fetch keys from servers
Can “weigh” to server capacities
Servers know how to:
Store items they receive
Expire them from the cache
No inter-server communications – everything is
unaware
No SQL
Feb 19, 2014
10
11.
RDBMS tries to ensure ACID properties
NoSQL does not guarantee ACID and is therefore
much faster
We don’t need ACID everywhere
NoSQL follows BASE properties
No SQL
Feb 19, 2014
11
12. Basic availability
The store appears to work most of the
time.
Soft-state
Stores don’t have to be write-consistent, nor
do different replicas have to be mutually
consistent all the time.
Eventual consistency
Stores exhibit consistency at some later
point (e.g., lazily at read time).
No SQL
Feb 19, 2014
12
13.
Simple web application with not much traffic
◦ Application server, database server all on one machine
No SQL
Feb 19, 2014
13
14. More traffic comes in
Application server
Database server
Even more traffic comes in
Load balancer
Application server x2
Database server
No SQL
Feb 19, 2014
14
15. Even more traffic comes in
Load balancer x N
easy
Application server x N
easy
Database server xN
hard for SQL databases
No SQL
Feb 19, 2014
15
17. NoSQL Scalling Need more storage?
Add more servers!
Need higher performance?
Add more servers!
Need better reliability?
Add more servers!
No SQL
Feb 19, 2014
17
18.
You can scale SQL databases (Oracle, MySQL,
SQL Server…)
◦ This will cost you dearly
◦ If you don’t have a lot of money, you will reach limits
quickly
You can scale NoSQL databases
◦ Very easy horizontal scaling
◦ Lots of open-source solutions
◦ Scaling is one of the basic incentives for design, so it is
well handled
◦ Scaling is the cause of trade-offs causing you to have to
use map/reduce
No SQL
Feb 19, 2014
18
19. Almost infinite horizontal scaling
Very fast
Performance doesn’t deteriorate with growth
(much)
No fixed table schemas
No join operations
Ad-hoc queries difficult or impossible
Structured storage
Almost everything happens in RAM
No SQL
Feb 19, 2014
19
23.
Lineage: Amazon's Dynamo paper and Distributed
HashTables.
Data model: A global collection of key-value pairs
Example systems
◦ Google BigTable , Amazon Dynamo, Cassandra,
Voldemort , Hbase etc.
Implementation: efficiency, scalability, fault-tolerance,
load balancing
◦ Records distributed to nodes based on key
◦ Replication (R= 2*F+1) where F stands for fault
tolerence
◦ Single-record transactions, “eventual consistency”
No SQL
Feb 19, 2014
23
24. Lineage: Inspired by Lotus Notes.
Data model: Collections of documents, which
contain key-value collections (called
"documents").
Example: CouchDB, MongoDB, Riak
No SQL
Feb 19, 2014
24
27. Lineage: Draws from Euler and graph theory.
Data model: Nodes & relationships, both which
can hold key-value pairs
Example: AllegroGraph, InfoGrid, Neo4j
No SQL
Feb 19, 2014
27
28. Property
Graph:
• It contains nodes and relationships
• Nodes contain properties (key-value pairs)
• Relationships are named and directed, and always
have a start and end node
• Relationships can also contain properties
No SQL
Feb 19, 2014
28
31.
Google’s framework for processing highly
distributable problems across huge datasets
using a large number of computers
Let’s define large number of computers
◦ Cluster if all of them have same hardware
◦ Grid unless Cluster (if !Cluster for old-style programmers)
Process split into two phases
◦ Map
Take the input, partition it delegate to other machines
Other machines can repeat the process, leading to tree structure
Each machine returns results to the machine who gave it the task
No SQL
Feb 19, 2014
31
32. ◦ Reduce
collect results from machines you gave the tasks
combine results and return it to requester
◦ Slower than sequential data processing, but massively
parallel
◦ Sort petabyte of data in a few hours
◦ Input, Map, Shuffle, Reduce, Output
No SQL
Feb 19, 2014
32
34.
Cassandra
◦
◦
◦
◦
◦
◦
Facebook (original developer, used it till late 2010)
Twitter
Digg
Reddit
Rackspace
Cisco
BigTable
◦ Google (open-source version is HBase)
MongoDB
◦
◦
◦
◦
◦
Foursquare
Craigslist
Bit.ly
SourceForge
GitHub
No SQL
Feb 19, 2014
34
35. Written in: Java
Protocol: Custom, binary (Thrift)
Tunable trade-offs for distribution and replication
(N, R, W)
Querying by column, range of keys
BigTable-like features: columns, column families
Writes are much faster than reads (!)
◦ Constant write time regardless of database size
Map/reduce possible with Apache Hadoop
No SQL
Feb 19, 2014
35
36.
Cassandra is open source DBMS from Appache
software foundation.
Cassandra provides a structured key-value
store with tunable consistency
Cassandra is a distributed storage system for
managing structured data that is designed to scale to
a very large size across many commodity servers,
with no single point of failure
It is a NoSQL solution that was initially developed
by Facebook and powered their Inbox Search feature
until late 2010
No SQL
Feb 19, 2014
36
37.
Written in: Java
Main point: Billions of rows X millions of columns
Modeled after BigTable
Map/reduce with Hadoop
Query predicate push down via server side scan and get filters
Optimizations for real time queries
A high performance Thrift gateway
HTTP supports XML, Protobuf, and binary
Cascading, hive, and pig source and sink modules
No single point of failure
While Hadoop streams data efficiently, it has overhead for starting
map/reduce jobs. HBase is column oriented key/value store and
allows for low latency read and writes.
Random access performance is like MySQL
No SQL
Feb 19, 2014
37
38.
Written in: Erlang
Main point: DB consistency, ease of use
Bi-directional (!) replication, continuous or ad-hoc, with
conflict detection, thus, master-master replication. (!)
MVCC - write operations do not block reads
Previous versions of documents are available
Crash-only (reliable) design
Needs compacting from time to time
Views: embedded map/reduce
Formatting views: lists & shows
Server-side document validation possible
Authentication possible
Real-time updates via _changes (!)
Attachment handling
CouchApps (standalone JS apps)
No SQL
Feb 19, 2014
38
39.
Apache project
A framework that allows for the distributed processing of large
data sets across clusters of computers
Designed to scale up from single servers to thousands of
machines
Designed to detect and handle failures at the application layer,
instead of relying on hardware for it
Created by Doug Cutting, who named it after his son's toy
elephant
Hadoop subprojects
◦ Cassandra
◦ HBase
◦ Pig
Hive was a Hadoop subproject, but is now a top-level Apache project
No SQL
Feb 19, 2014
39
40.
Scales to hundreds or thousands of computers, each with
several processor cores
Designed to efficiently distribute large amounts of work across
a set of machines
Hundreds of gigabytes of data constitute the low end of
Hadoop-scale
Built to process "web-scale" data on the order of hundreds of
gigabytes to terabytes or petabytes
Uses Java, but allows streaming so other languages can
easily send and accept data items to/from Hadoop
No SQL
Feb 19, 2014
40
41.
Uses distributed file system (HDFS)
◦ Designed to hold very large amounts of data (terabytes
or even petabytes)
◦ Files are stored in a redundant fashion across multiple
machines to ensure their durability to failure and high
availability to very parallel applications
◦ Data organized into directories and files
◦ Files are divided into block (64MB by default) and
distributed across nodes
Design of HDFS is based on the design of the
Google File System
No SQL
Feb 19, 2014
41
42. A petabyte-scale data warehouse system for
Hadoop
Easy data summarization, ad-hoc queries
Query the data using a SQL-like language called
HiveQL
Hive compiler generates map-reduce jobs for
most queries
No SQL
Feb 19, 2014
42
43. NoSQL is a great problem solver if you need it
Choose your NoSQL platform carefully as each is
designed for specific purpose
Get used to Map/Reduce
It’s not a sin to use NoSQL alongside (yes)SQL
database
No SQL
Feb 19, 2014
43
44. Graph Databases by Ian Robinson,Jim Webber
and Emil Eifrem
http://www.mongodb.com/learn/nosql
http://www.couchbase.com/nosql-database
http://en.wikipedia.org/wiki/Apache_Cassandra
http://en.wikipedia.org/wiki/SQL
http://en.wikipedia.org/wiki/NoSQL
www.slideshare.com
No SQL
Feb 19, 2014
44