Dr. Awase Khirni Syed, profile picture
Uploaded byDr. Awase Khirni Syed
743 views

Mongo db groundup-0-nosql-intro-syedawasekhirni

The document provides an overview of NoSQL databases, including their history, characteristics, and categories. It discusses the evolution of database systems from hierarchical and network models to relational and object-oriented databases. It also covers the limitations of relational databases that led to the rise of NoSQL databases, including lack of scalability, availability, and flexibility. The key advantages of NoSQL databases like schema flexibility and horizontal scalability are outlined. Popular categories of NoSQL databases like key-value stores, document databases, and graph databases are also described.

Embed presentation

Downloaded 21 times
Ground Up Syed Awase
0.NO SQL DATA STORES SYED AWASE KHIRNI © Syed Awase 2015-16 – MongoDB Ground Up 2
Outline • Evolution and History of Database Management Systems • Outline of various database types. • Traditional Scalability • Common Problems • Non-traditional Databases • Non-traditional DB characteristics • Why NoSQL? – Design Goals • NoSQL vs RDBMS • Choosing a NoSQL Solution • Categories of NoSQL Database © Syed Awase 2015-16 – MongoDB Ground Up 3
Database Design © Syed Awase 2015-16 – MongoDB Ground Up 4
Database Management Systems © Syed Awase 2015-16 – MongoDB Ground Up 5
Evolution and History © Syed Awase 2015-16 – MongoDB Ground Up 6
NoSQL past and Present © Syed Awase 2015-16 – MongoDB Ground Up 7
Hierarchical Databases • Data organized in a tree – A parent can have many children – A child can have only one parent • Records described by entity type • 1:N (one-to-many) relationships • Query by path navigation • Examples – File system – LDAP – Windows Registry and Active Directory – XML documents and Xquery. © Syed Awase 2015-16 – MongoDB Ground Up 8
Network Databases • Data organized in graph(lattice0 – A parent can have many children – A child can have many parents • Bachmann diagrams • Record types define properties • Set types defined relationships – Parent-child, (double) linked list.. • Query by graph navigation • Examples – CODASYL © Syed Awase 2015-16 – MongoDB Ground Up 9
Relational Databases • Data organized as tuples in relations • Link between data tuples – Primary and foreign keys • Relational algebra – Project, select, join • Relational normal forms • Declarative language – Data definition, consistency, manipulation and querying • Examples – Oracle, Microsoft SQL Server, IBM DB2 – PostgreSQL, MySQL © Syed Awase 2015-16 – MongoDB Ground Up 10
Relational Databases • Relational model is very simple – Only basic concepts – references need to be simulated – Restricted type system – no user-defined types • Lack of semantic modeling – Complex data, versioning, roles • Little support of data and schema evolution • Object-relational impedance mismatch. © Syed Awase 2015-16 – MongoDB Ground Up 11
RDBMS Problem • Design for ACID – Atomicity, Consistency, Isolation and Durability • Hard to Scale • Availability? • Flexibility © Syed Awase 2015-16 – MongoDB Ground Up 12
Traditional Scalability • Scale-up – Memory and hardware has limitations • Scale-out – Scaling reads • Cache is the King – Query cache, Memcache, OLAP • Prefetching • Replication • Scaling Writes – Redundant disk arrays, RAID – Sharding © Syed Awase 2015-16 – MongoDB Ground Up 13
Common Problems • Relation model is heavy – Parsing – Locking – Logging – Buffer pool and thread • Not every case can work within a single node SMP • Sharding does not solve all problems – Cross shard or join between shards – Need to update across multiple shards within a transaction. – Shard failure – Online Schema changes without taking the shard offiline – Add or replace shards in-line. © Syed Awase 2015-16 – MongoDB Ground Up 14
Lack of Semantic Modeling • All advantages of the object-oriented paradigm which enable to produce models of an application domain that capture more of its aspects, e.g. using – Inheritance – Relationships • Inheritance and relationships can be mapped to the relational model, however, they result in a complex database schema. Such complex schemata are difficult to develop, maintain and decrease runtime performance. © Syed Awase 2015-16 – MongoDB Ground Up 15
Object-Relational Impedance Mismatch •Object-oriented application development and relational data management results in clash of two incompatible models. •Code to map between models is considerable overhead, costly and hard to maintain. © Syed Awase 2015-16 – MongoDB Ground Up 16
Database Evolution • Object-relational mapping (ORM) layers don’t really help – Writing manual code for ORM poses problem – Evolution code can usually not be generated automatically • Automatic evolution is not possible for complex cases – Semantic changes – Certain type changes ( String to Person) – Ambiguity of attribute add-remove vs. rename – Class hierarchy changes © Syed Awase 2015-16 – MongoDB Ground Up 17
Object-Oriented Databases • Data represented as objects – Object Identity – Attributes and methods – Reference, relationships, associations • Extensible type hierarchy – User-defined types, abstract data types – single or multiple inheritance – Overloading, overriding, late binding • Declarative language for ad hoc purposes • Binding for Object-oriented programming language © Syed Awase 2015-16 – MongoDB Ground Up 18
Object-Relational Database • Relational model extended – Nested relations – References – Sets – Row types, abstract types – Functions • Declarative language extended – Computationally complete • Fundamental impedance mismatch remains © Syed Awase 2015-16 – MongoDB Ground Up 19
Non-traditional Databases • XML Database • Graph Databases • Document Databases • Key-value Stores • Column Store • Others – Geospatial – File System NoSQL Non-Relational, Web- scale database First used by Carlo Strozzi in 1998 Reintroduced by Eric Evan in 2009 © Syed Awase 2015-16 – MongoDB Ground Up 20
Non-traditional Database Characteristics • Non-relational in Nature • Distributed • Horizontal Scalability • Schema-less/Schema-free • Eventual Consistency © Syed Awase 2015-16 – MongoDB Ground Up 21
Gartner Hype Cycle NoSQL © Syed Awase 2015-16 – MongoDB Ground Up 22
NO-SQL Next Generation Databases mostly addressing some of the points: being non-relational, distributed, open-source and horizontal scalable.. Schema- free, easy replication support, simple API, eventually consistent” © Syed Awase 2015-16 – MongoDB Ground Up -nosql-database.org • Non-Relational • Distributed • Open-Source • Horizontally Scalable • Schema-Free • Replication Support • Eventually Consistent Classification • Wide Column Store/Column Families • Document Store • Key value/Tuple Store • Graph Database • Multimodel Database • Object Database • Grid and Cloud Database Solutions • XML Databases • Multidimensional Databases • Multivalue databases • Time Series/Streaming Databases •Scientific and Specialized DB 23
Why NoSQL? © Syed Awase 2015-16 – MongoDB Ground Up 24
Why NoSQL? © Syed Awase 2015-16 – MongoDB Ground Up 25
Why NoSQL? Sharding: A database shard is a horizontal partition in a database or search engine. Each individual partition is referred to as a shard or database shard. © Syed Awase 2015-16 – MongoDB Ground Up 26
Why NoSQL? DESIGN GOALS • Semi Structured >> • Big Data >> • Real-time >> • Ubiquity >> • Schema-Free • Scalable reads/writes – Horizontal Scalability • To increase the no-of machines but maintaining proportional performance. – Vertical Scalability • To add more resources to your single machine to optimize performance. • High-performance • High-availability • Reliability, Fault tolerant. © Syed Awase 2015-16 – MongoDB Ground Up 27
NoSQL vs RDBMS NoSQL RDBMS • Schema-free • Scalable writes/reads • Auto high-availability • Limited queries • Eventual Consistency – applies to most NoSQL systems • BASE : Basic Availability, Soft state, Eventual consistency • Relational Schema • Scalable reads • Custom high-availability • Flexible • Consistent • ACID : Atomic, Consistent, Isolated and Durable. © Syed Awase 2015-16 – MongoDB Ground Up 28
RDBMS vs NoSQL • Traditional distributed data management solutions focus on ACID semantics – Atomicity – Consistency – Isolation – Durability • Modern Web-scale data management focuses on – Basic Availability – Soft-state – Eventual Consistent – CAP (Consistency, Availability, Partition and Tolerance) © Syed Awase 2015-16 – MongoDB Ground Up 29
ACID • Atomicity : All of the operations in the transaction will complete or none will • Consistency: the database will be in a consistent state when the transaction begins and ends. • Isolation: the transaction will behave as if it is the only operation being performed upon the database. (No interference of transaction). • Durability : Upon completion of the transaction, the operation will not be reversed. © Syed Awase 2015-16 – MongoDB Ground Up 30
CAP• Consistency: No contradiction b/w data. • Availability: Every operation must terminate in an intended response. • Partition tolerance: Operations will complete, even If individual components are unavailable. • NoSQL are based on CAP. © Syed Awase 2015-16 – MongoDB Ground Up 31
ACID vs BASE ACID BASE • Strong consistency • Isolation • Focus on “commit” • Nested Transactions • Availability? • Pessimistic/Conservative • Difficult evolution (e.g. schema) • Weak consistency • Availability first • Best effort • Approximate answers (OK) • Optimistic (aggressive) • Fast and simple • Easier evolution © Syed Awase 2015-16 – MongoDB Ground Up 32
Who’s using NoSQL? © Syed Awase 2015-16 – MongoDB Ground Up 33
Who’s using NoSQL? © Syed Awase 2015-16 – MongoDB Ground Up 34
Growth © Syed Awase 2015-16 – MongoDB Ground Up 35
NoSQL Use Cases • Consumer Use Cases • Facebook • Twitter • NetFlix • Enterprise Use Cases • Rackspace • TrendMicro • NewsCred Facebook •Hbase – Facebook messages •Scribe- Real-time click logs •Hive- SQL Queries -> MapReduce jobs •Hadoop •Web analytics warehouse •Distributed datastore •MySQL backups © Syed Awase 2015-16 – MongoDB Ground Up 36
Choosing a NoSQL Solution © Syed Awase 2015-16 – MongoDB Ground Up 37
Consistent, Available (CA) • CA- systems have trouble with partitions and deal with it with replication • Examples – MySQL (relational) – AsterData (relational) – Greenplum(relational) – Vertica (Column) © Syed Awase 2015-16 – MongoDB Ground Up 38
Availability, Partition-Tolerant (AP) • AP-systems have trouble with consistency, achieve “eventual consistency” through replication. • Examples – Cassandra(column/tabular) – Dynamo (Key-value) – Voldemort (Key-value) – Tokyo Cabinet (Key-value) – CouchDB(document) – SimpleDB(document) – Riak(document) © Syed Awase 2015-16 – MongoDB Ground Up 39
Consistent, Partition-Tolerant (CP) • CP-systems have trouble with availability while keep data consistent across partitioned nodes. • Examples – MongoDB(document) – BigTable(Column/tabular) – HyperTable(Column/tabular) – Hbase (Column/tabular) – Redis(key-value) – Scalaris(key-value) – MemcacheDB (Key-value) © Syed Awase 2015-16 – MongoDB Ground Up 40
Categories of NoSQL databases 1) Key Value stores  Don’t have any schema  Fast lookups facility  Can be use in a forum software where user’s statistic and messages are recorded. User’s id will serve as a key and will retrieve a string that represents all the relevant info of the user. And a background process recalculates the information and writes to the store independently after fixed interval of time.  Example  Redis : Redis is an open source, advanced key-value store. It is often referred to as a data structure server since keys (data types) can contain strings, hashes, lists, sets and sorted sets. © Syed Awase 2015-16 – MongoDB Ground Up 41
Categories (cont…)  API: Tons of languages, Written in: C,  Concurrency: in memory and saves asynchronous disk after a defined time. 2) Document databases  web application  tolerance of incomplete data  low query performance  no standard query syntax  Used when we don’t have complete data about all the entities of database but we still need to create database. Example CouchDB: API: JSON, Protocol: REST, Query Method: MapReduceR of JavaScript Funcs, Replication: Master Master, Written in: Erlang. © Syed Awase 2015-16 – MongoDB Ground Up 42
Categories (cont…)  MongoDB: API: BSON, Protocol: lots of langs, Query Method: dynamic object-based language, Replication: Master Slave, Written in: C++. 3) Graph databases  Social networking  graph algorithms, connectedness, degree of relationships  has to traverse the entire graph to get definitive answer.  not easy to cluster.  used in a situation where we want to analyze the on going trends and take decision on the basis of those trends. © Syed Awase 2015-16 – MongoDB Ground Up 43
Categories (cont…)  Example: Neo4J: API: lots of langs, Protocol: Java embedded / REST, Query Method: SparQL, nativeJavaAPI, JRuby, Replication: typical MySQL style master/slave, Written in: Java, Concurrency: non-block reads, writes locks involved nodes/relationships until commit, Misc: ACID possible 4) XML databases  Publishing  mature search technologies  schema validation  re-writing is easier than updating © Syed Awase 2015-16 – MongoDB Ground Up 44
Categories (cont…)  Used in a situation where one wants to produce documents of articles etc from a huge amount of documents but the format of those article doesn’t allow the publisher to perform search on it. Those articles are converted into xml database and wrap it in a readable-URL web service for the document production systems. 5) Distributed Peer Stores  distributed file systems  Fast lookups  Good distributed storage of data  Very low level API  Best for voting system. In such a situation one store/user and one store/piece of content is created. The user store will hold all the votes they have ever casted and the content will store a copy of the content on which vote was casted. © Syed Awase 2015-16 – MongoDB Ground Up 45
Categories (cont…) Example Cassandra: API: many Thrift languages, Query Method: MapReduce, Written in: Java, Concurrency: eventually consistent , Misc: like "Big-Table on Amazon Dynamo alike", initiated by Facebook. © Syed Awase 2015-16 – MongoDB Ground Up 46
Object Databases • Versant • Db40 • Objectivity • Startcounter • Perst • VelocityDB • HSS Database • ZODB • Magma • NEO • Ninja Database • PicoLisp • Acid-state • ObjectDB • Siaqodb • Morantex • EyeDB • FrameD © Syed Awase 2015-16 – MongoDB Ground Up 47
Key-Value Stores • HASH • Memcached • DynamoDB • Redis • Riak • Tokyo Cabinet • Azure Table Storage • Riak • Aerospike • FoundationDB • LevelDB • BerkeleyDB • GenieDB • BangDB • Chordless • Allegro-C • nessDB • HyperDex • TokyoCabinet/Tyrant • Scalien • Voldemort • Dynomite • KAI • MemcahcheDB • Faircom C-Tree • LSM • KitaroDB • HamsterDB • STSdb • Tarantool/Box • MaxTable • Quasardb • Pincaster • RaptorDB • TIBCO Active Spaces • Sophia © Syed Awase 2015-16 – MongoDB Ground Up 48
XML Database • EMC Documentum xDB • eXist • Sedna • BaseX • Qizx • Berkeley DB XML © Syed Awase 2015-16 – MongoDB Ground Up 49
Multidimensional databases • Globals • Intersystems Cache • GT.M • SciDB • MiniM DB • Rasdaman © Syed Awase 2015-16 – MongoDB Ground Up 50
Document Store (Semi-Structured) • IBM Lotus • MongoDB • CouchDB • MongoDB • ElasticSearch • RethinkDB • RavenDB • MarkLogic Server • Clusterpoint Server • ThruDB • Terrastore • JasDB • RaptorDB • Djondb • EJDB • Amisa Server • SisoDB • SDB • NoSQL embedded Db • Document Stores •Supports complex data model than Key Value Pairs •Good at handling content management, session, profile data •Multi index support •Dynamic schemas, Nested schemas •Auto distributed, eventual consistency •MVCC (CouchDB) or automic (MongoDB) • MongoDB, SimpleDB: widely adopted in this space • Use Case: Search by complex patterns & CRUD apps © Syed Awase 2015-16 – MongoDB Ground Up 51
Column-Oriented Stores • Semi-Structured • BigTable(inspiration Source) • Hbase • Apache Cassandra • Hypertable • Accumulo • Amazon Simple DB • Cloudata • Cloudera • HPCC • Stratosphere Hbase (Apache), Cassanda (Facebook) and HyperTable (Bidu) Hbase – CA Cassandra – AP Model consists of rows and columns Scalability: Splitting of both rows and columns Rows are split across nodes using primary key, range Columns are distributed using groups Horizonal and vertical partitioning can be used simultaneous Extension of document store HBase uses HDFS; Pig, Hive, Cascading can help Use case: Grouping of frequently used and un-used over data centers / stream of writes © Syed Awase 2015-16 – MongoDB Ground Up 52
Graph Database • InfoGrid • Neo4j • InfiniteGraph • DEX • Titan • InforGrid • HyperGraphDB • GraphBase • Trinity • AllegroGraph • BrightstarDB • BigData • Meronymy • WhiteDB • Openlink Virtuoso • VertexDB • FlockDB Social Graph Relationship between entities Data modeling on social networks Common Use Cases List of friends Recommendation system Following Followers Common Connections FAN IN/OUT © Syed Awase 2015-16 – MongoDB Ground Up 53
Multimodel databases • ArangoDB • OrientDB • Datanomic • FatDB • AlchemyDB • CortexDB © Syed Awase 2015-16 – MongoDB Ground Up 54
NoSQL (Not ONLY SQL) • Fits very well for volatile data • High read or write throughput. • Automatic horizontal scalability (Consistent Hashing) • Simple to implement, no investment for developers to design and implement relational model • Application logic defines object model • Support of MVCC (MultiVersion Concurrency Control) in some form • Compaction and un-compaction happens at top tier • In-memory or disk based or combination © Syed Awase 2015-16 – MongoDB Ground Up 55
NoSQL - Disadvantages • Packing and Un-packing of each key • Lack of relation from one key to another • Need whole value from the key; to read/write any practical information. • No Security or authentication • Data store is merely a storage layer, can’t be used for – Analytics – Reporting – Aggregation – Ordered Values © Syed Awase 2015-16 – MongoDB Ground Up 56
Some Popular NoSQL Databases © Syed Awase 2015-16 – MongoDB Ground Up 57
MongoDB 3.2 • Written in: C++ • Main point: Retains some friendly properties of SQL. (Query, index) • License: AGPL (Drivers: Apache) • Protocol: Custom, binary (BSON) • Master/slave replication (auto failover with replica sets) • Sharding built-in • Queries are javascript expressions • Run arbitrary javascript functions server-side • Better update-in-place than CouchDB • Uses memory mapped files for data storage • Performance over features • Journaling (with --journal) is best turned on • On 32bit systems, limited to ~2.5Gb • An empty database takes up 192Mb • GridFS to store big data + metadata (not actually an FS) • Has geospatial indexing • Data center aware Best used: If you need dynamic queries. If you prefer to define indexes, not map/reduce functions. If you need good performance on a big DB. If you wanted CouchDB, but your data changes too much, filling up disks. For example: For most things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you back. © Syed Awase 2015-16 – MongoDB Ground Up 58
Riak 1.2v • Written in: Erlang & C, some JavaScript • Main point: Fault tolerance • License: Apache • Protocol: HTTP/REST or custom binary • Stores blobs • Tunable trade-offs for distribution and replication • Pre- and post-commit hooks in JavaScript or Erlang, for validation and security. • Map/reduce in JavaScript or Erlang • Links & link walking: use it as a graph database • Secondary indices: but only one at once • Large object support (Luwak) • Comes in "open source" and "enterprise" editions • Full-text search, indexing, querying with Riak Search • In the process of migrating the storing backend from "Bitcask" to Google's "LevelDB" • Masterless multi-site replication replication and SNMP monitoring are commercially licensed Best used: If you want something Dynamo-like data storage, but no way you're gonna deal with the bloat and complexity. If you need very good single-site scalability, availability and fault-tolerance, but you're ready to pay for multi-site replication. For example: Point-of-sales data collection. Factory control systems. Places where even seconds of downtime hurt. Could be used as a well-update-able web server. © Syed Awase 2015-16 – MongoDB Ground Up 59
CouchDB 1.2v • Written in: Erlang • Main point: DB consistency, ease of use • License: Apache • Protocol: HTTP/REST • Bi-directional (!) replication, • continuous or ad-hoc, • with conflict detection, • thus, master-master replication. (!) • MVCC - write operations do not block reads • Previous versions of documents are available • Crash-only (reliable) design • Needs compacting from time to time • Views: embedded map/reduce • Formatting views: lists & shows • Server-side document validation possible • Authentication possible • Real-time updates via '_changes' (!) • Attachment handling • thus, CouchApps (standalone js apps) Best used: For accumulating, occasionally changing data, on which pre-defined queries are to be run. Places where versioning is important. For example: CRM, CMS systems. Master-master replication is an especially interesting feature, allowing easy multi-site deployments. © Syed Awase 2015-16 – MongoDB Ground Up 60
Redis 2.8v • Written in: C • Main point: Blazing fast • License: BSD • Protocol: Telnet-like, binary safe • Disk-backed in-memory database, • Dataset size limited to computer RAM (but can span multiple machines' RAM with clustering) • Master-slave replication, automatic failover • Simple values or data structures by keys • but complex operations like ZREVRANGEBYSCORE. • INCR & co (good for rate limiting or statistics) • Bit operations (for example to implement bloom filters) • Has sets (also union/diff/inter) • Has lists (also a queue; blocking pop) • Has hashes (objects of multiple fields) • Sorted sets (high score table, good for range queries) • Lua scripting capabilities (!) • Has transactions (!) • Values can be set to expire (as in a cache) • Pub/Sub lets one implement messaging Best used: For rapidly changing data with a foreseeable database size (should fit mostly in memory). For example: Stock prices. Analytics. Real- time data collection. Real- time communication. And wherever you used memcached before © Syed Awase 2015-16 – MongoDB Ground Up 61

More Related Content

PPTX
Scaling SQL and NoSQL Databases in the Cloud
byRightScale
 
PDF
Mongo db model relationships with documents
byDr. Awase Khirni Syed
 
PPTX
NoSQL Architecture Overview
byChristopher Foot
 
PPTX
What is NoSQL and CAP Theorem
byRahul Jain
 
PPTX
Rdbms vs. no sql
byAmar Jagdale
 
PDF
Polyglot Persistence - Two Great Tastes That Taste Great Together
byJohn Wood
 
PPTX
The Rise of NoSQL and Polyglot Persistence
byAbdelmonaim Remani
 
PDF
Scaling the Web: Databases & NoSQL
byRichard Schneeman
 
Scaling SQL and NoSQL Databases in the Cloud
byRightScale
 
Mongo db model relationships with documents
byDr. Awase Khirni Syed
 
NoSQL Architecture Overview
byChristopher Foot
 
What is NoSQL and CAP Theorem
byRahul Jain
 
Rdbms vs. no sql
byAmar Jagdale
 
Polyglot Persistence - Two Great Tastes That Taste Great Together
byJohn Wood
 
The Rise of NoSQL and Polyglot Persistence
byAbdelmonaim Remani
 
Scaling the Web: Databases & NoSQL
byRichard Schneeman
 

What's hot

DOCX
Sql vs NO-SQL database differences explained
bySatya Pal
 
PPTX
Sql vs nosql
byNick Verschueren
 
PPT
SQL, NoSQL, BigData in Data Architecture
byVenu Anuganti
 
PPTX
NoSQL Data Architecture Patterns
byMaynooth University
 
PPT
SQL/NoSQL How to choose ?
byVenu Anuganti
 
PPTX
Chapter1: NoSQL: It’s about making intelligent choices
byMaynooth University
 
PPTX
Demystfying nosql databases
byMike King
 
PPTX
NoSql - mayank singh
byMayank Singh
 
PDF
The Future of Distributed Databases
byNuoDB
 
PPTX
SQL vs. NoSQL. It's always a hard choice.
byDenis Reznik
 
PPTX
Introduction to NuoDB
bySandun Perera
 
PPTX
NoSQL Consepts
byMaynooth University
 
PDF
Application Development with Apache Cassandra as a Service
byWSO2
 
PDF
SQL vs NoSQL: Big Data Adoption & Success in the Enterprise
byAnita Luthra
 
PPTX
Lviv EDGE 2 - NoSQL
byzenyk
 
PDF
HPTS 2011: The NoSQL Ecosystem
byAdam Marcus
 
PPTX
To SQL or NoSQL, that is the question
byKrishnakumar S
 
PDF
NoSQL-Database-Concepts
byBhaskar Gunda
 
PDF
Nosql data models
byViet-Trung TRAN
 
PPT
No sql landscape_nosqltips
byimarcticblue
 
Sql vs NO-SQL database differences explained
bySatya Pal
 
Sql vs nosql
byNick Verschueren
 
SQL, NoSQL, BigData in Data Architecture
byVenu Anuganti
 
NoSQL Data Architecture Patterns
byMaynooth University
 
SQL/NoSQL How to choose ?
byVenu Anuganti
 
Chapter1: NoSQL: It’s about making intelligent choices
byMaynooth University
 
Demystfying nosql databases
byMike King
 
NoSql - mayank singh
byMayank Singh
 
The Future of Distributed Databases
byNuoDB
 
SQL vs. NoSQL. It's always a hard choice.
byDenis Reznik
 
Introduction to NuoDB
bySandun Perera
 
NoSQL Consepts
byMaynooth University
 
Application Development with Apache Cassandra as a Service
byWSO2
 
SQL vs NoSQL: Big Data Adoption & Success in the Enterprise
byAnita Luthra
 
Lviv EDGE 2 - NoSQL
byzenyk
 
HPTS 2011: The NoSQL Ecosystem
byAdam Marcus
 
To SQL or NoSQL, that is the question
byKrishnakumar S
 
NoSQL-Database-Concepts
byBhaskar Gunda
 
Nosql data models
byViet-Trung TRAN
 
No sql landscape_nosqltips
byimarcticblue
 

Viewers also liked

PDF
Geospatial Big Data - Foss4gNA
bynormanbarker
 
ODP
MongoDB - A Document NoSQL Database
byRuben Inoto Soto
 
PDF
C* Summit 2013: Real-time Analytics using Cassandra, Spark and Shark by Evan ...
byDataStax Academy
 
PPTX
An Intro to NoSQL Databases
byRajith Pemabandu
 
PPT
6 Data Modeling for NoSQL 2/2
byFabio Fumarola
 
PPT
5 Data Modeling for NoSQL 1/2
byFabio Fumarola
 
PDF
Cassandra NoSQL Tutorial
byMichelle Darling
 
Geospatial Big Data - Foss4gNA
bynormanbarker
 
MongoDB - A Document NoSQL Database
byRuben Inoto Soto
 
C* Summit 2013: Real-time Analytics using Cassandra, Spark and Shark by Evan ...
byDataStax Academy
 
An Intro to NoSQL Databases
byRajith Pemabandu
 
6 Data Modeling for NoSQL 2/2
byFabio Fumarola
 
5 Data Modeling for NoSQL 1/2
byFabio Fumarola
 
Cassandra NoSQL Tutorial
byMichelle Darling
 

Similar to Mongo db groundup-0-nosql-intro-syedawasekhirni

PPTX
NoSQL Basics and MongDB
byShamima Yeasmin Mukta
 
PDF
Nosql part1 8th December
byRuru Chowdhury
 
PPTX
NoSQL databases - An introduction
byPooyan Mehrparvar
 
PPTX
Unit-10.pptx
byGhanashyamBK1
 
PPTX
NoSQL.pptx
byShamrizSadat
 
PDF
the rising no sql technology
byINFOGAIN PUBLICATION
 
PPTX
unit2_mongodb_1.pptxhlhhlhhjkhjkhjkhjkhjj
byzmulani8
 
PPTX
Introduction to Data Science NoSQL.pptx
bytarakesh7199
 
PPTX
No sq lv2
byNusrat Sharmin
 
PPTX
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
byLucas Jellema
 
PPTX
No SQL- The Future Of Data Storage
byBethmi Gunasekara
 
PPTX
UNIT I Introduction to NoSQL.pptx
byRahul Borate
 
PPTX
NoSQL A brief look at Apache Cassandra Distributed Database
byJoe Alex
 
PPTX
Master.pptx
byKarthikR780430
 
PPTX
Nosql
byROXTAD71
 
PPTX
Nosql
byRoxana Tadayon
 
PDF
NoSQL Databases Introduction - UTN 2013
byFacundo Farias
 
PPTX
NOSQL
byakbarashaikh
 
PDF
Big Data technology Landscape
byShivanandaVSeeri
 
PPTX
2.Introduction to NOSQL (Core concepts).pptx
byRushikeshChikane2
 
NoSQL Basics and MongDB
byShamima Yeasmin Mukta
 
Nosql part1 8th December
byRuru Chowdhury
 
NoSQL databases - An introduction
byPooyan Mehrparvar
 
Unit-10.pptx
byGhanashyamBK1
 
NoSQL.pptx
byShamrizSadat
 
the rising no sql technology
byINFOGAIN PUBLICATION
 
unit2_mongodb_1.pptxhlhhlhhjkhjkhjkhjkhjj
byzmulani8
 
Introduction to Data Science NoSQL.pptx
bytarakesh7199
 
No sq lv2
byNusrat Sharmin
 
Introducing NoSQL and MongoDB to complement Relational Databases (AMIS SIG 14...
byLucas Jellema
 
No SQL- The Future Of Data Storage
byBethmi Gunasekara
 
UNIT I Introduction to NoSQL.pptx
byRahul Borate
 
NoSQL A brief look at Apache Cassandra Distributed Database
byJoe Alex
 
Master.pptx
byKarthikR780430
 
Nosql
byROXTAD71
 
Nosql
byRoxana Tadayon
 
NoSQL Databases Introduction - UTN 2013
byFacundo Farias
 
NOSQL
byakbarashaikh
 
Big Data technology Landscape
byShivanandaVSeeri
 
2.Introduction to NOSQL (Core concepts).pptx
byRushikeshChikane2
 

More from Dr. Awase Khirni Syed

PDF
Require js training
byDr. Awase Khirni Syed
 
PDF
Powershell training material
byDr. Awase Khirni Syed
 
PDF
Es2015 training material-syedawase
byDr. Awase Khirni Syed
 
PDF
R programming groundup-basic-section-i
byDr. Awase Khirni Syed
 
PDF
Knockout js
byDr. Awase Khirni Syed
 
PDF
C# ASP.NET WEB API APPLICATION DEVELOPMENT
byDr. Awase Khirni Syed
 
PDF
SycliQ-AgribusinessInfographik001
byDr. Awase Khirni Syed
 
Require js training
byDr. Awase Khirni Syed
 
Powershell training material
byDr. Awase Khirni Syed
 
Es2015 training material-syedawase
byDr. Awase Khirni Syed
 
R programming groundup-basic-section-i
byDr. Awase Khirni Syed
 
Knockout js
byDr. Awase Khirni Syed
 
C# ASP.NET WEB API APPLICATION DEVELOPMENT
byDr. Awase Khirni Syed
 
SycliQ-AgribusinessInfographik001
byDr. Awase Khirni Syed
 

Recently uploaded

PDF
TNG_Architecting Green Software - An Introduction_Nils-Oliver Linden&Alexande...
byQAware GmbH
 
PPTX
SNG460-CNG489_Week8_17-21Nov25_Chp8-OdtuClass.pptx
byberayseray382
 
PDF
Database Management Systems(DBMS):UNIT-II Relational Data Model BCA SEP SEM ...
byKuvempu University
 
PDF
DSD-INT 2025 Hydrological Modelling in Society Assumptions, Impacts, and Appl...
byDeltares
 
PDF
Constraints First - Why Our On-Prem Ticketing System Starts With Limits, Not ...
bySpirit Face
 
PDF
From Experiment to Enterprise: Scaling AI Coding Assistants Across Engineerin...
byMaxim Salnikov
 
PPTX
Membershipanywhere - Digital Membership Card - USA , Canada, Australia.pptx
bymembershipanywhere
 
PPTX
Modern P&C Insurance Software for Smarter, Faster Operations.pptx
byInsurance Tech Services
 
PDF
Design and Analysis of Algorithms(DAA): Unit-II Asymptotic Notations and Basi...
byKuvempu University
 
PDF
DSD-INT 2025 Latest Developments for HydroMT and wflow - Meshgi
byDeltares
 
PPTX
AI Agent Overview - Why we need AI Agent
byAlokGope
 
PDF
DSD-INT 2025 Introduction to Hydrology Suite - Slootjes
byDeltares
 
PPTX
Computer_Virus_Presentation_With_Slide7.pptx
byHEVISLIVE
 
PPTX
Lecture 3 - Scheduling - Operating System
bynurulalomador
 
PDF
Database Management Systems(DBMS):UNIT-I Introduction to Database(DBMS) BCA S...
byKuvempu University
 
PDF
DSD-INT 2025 Hydrological response of the Puget Sound area to a changing clim...
byDeltares
 
PDF
Manual vs Automated Accessibility Testing – What to Choose in 2025.pdf
bykalichargn70th171
 
PPTX
application security presentation 2 by harman
byharmanideapad
 
PPTX
Code Assessment Tools .pptx
byCodexpro
 
PDF
DSD-INT 2025 Managing low flows in the International Meuse Basin with Nature-...
byDeltares
 
TNG_Architecting Green Software - An Introduction_Nils-Oliver Linden&Alexande...
byQAware GmbH
 
SNG460-CNG489_Week8_17-21Nov25_Chp8-OdtuClass.pptx
byberayseray382
 
Database Management Systems(DBMS):UNIT-II Relational Data Model BCA SEP SEM ...
byKuvempu University
 
DSD-INT 2025 Hydrological Modelling in Society Assumptions, Impacts, and Appl...
byDeltares
 
Constraints First - Why Our On-Prem Ticketing System Starts With Limits, Not ...
bySpirit Face
 
From Experiment to Enterprise: Scaling AI Coding Assistants Across Engineerin...
byMaxim Salnikov
 
Membershipanywhere - Digital Membership Card - USA , Canada, Australia.pptx
bymembershipanywhere
 
Modern P&C Insurance Software for Smarter, Faster Operations.pptx
byInsurance Tech Services
 
Design and Analysis of Algorithms(DAA): Unit-II Asymptotic Notations and Basi...
byKuvempu University
 
DSD-INT 2025 Latest Developments for HydroMT and wflow - Meshgi
byDeltares
 
AI Agent Overview - Why we need AI Agent
byAlokGope
 
DSD-INT 2025 Introduction to Hydrology Suite - Slootjes
byDeltares
 
Computer_Virus_Presentation_With_Slide7.pptx
byHEVISLIVE
 
Lecture 3 - Scheduling - Operating System
bynurulalomador
 
Database Management Systems(DBMS):UNIT-I Introduction to Database(DBMS) BCA S...
byKuvempu University
 
DSD-INT 2025 Hydrological response of the Puget Sound area to a changing clim...
byDeltares
 
Manual vs Automated Accessibility Testing – What to Choose in 2025.pdf
bykalichargn70th171
 
application security presentation 2 by harman
byharmanideapad
 
Code Assessment Tools .pptx
byCodexpro
 
DSD-INT 2025 Managing low flows in the International Meuse Basin with Nature-...
byDeltares
 

Mongo db groundup-0-nosql-intro-syedawasekhirni

  • 1.
  • 2.
    0.NO SQL DATASTORES SYED AWASE KHIRNI © Syed Awase 2015-16 – MongoDB Ground Up 2
  • 3.
    Outline • Evolution andHistory of Database Management Systems • Outline of various database types. • Traditional Scalability • Common Problems • Non-traditional Databases • Non-traditional DB characteristics • Why NoSQL? – Design Goals • NoSQL vs RDBMS • Choosing a NoSQL Solution • Categories of NoSQL Database © Syed Awase 2015-16 – MongoDB Ground Up 3
  • 4.
    Database Design © SyedAwase 2015-16 – MongoDB Ground Up 4
  • 5.
    Database Management Systems ©Syed Awase 2015-16 – MongoDB Ground Up 5
  • 6.
    Evolution and History ©Syed Awase 2015-16 – MongoDB Ground Up 6
  • 7.
    NoSQL past andPresent © Syed Awase 2015-16 – MongoDB Ground Up 7
  • 8.
    Hierarchical Databases • Dataorganized in a tree – A parent can have many children – A child can have only one parent • Records described by entity type • 1:N (one-to-many) relationships • Query by path navigation • Examples – File system – LDAP – Windows Registry and Active Directory – XML documents and Xquery. © Syed Awase 2015-16 – MongoDB Ground Up 8
  • 9.
    Network Databases • Dataorganized in graph(lattice0 – A parent can have many children – A child can have many parents • Bachmann diagrams • Record types define properties • Set types defined relationships – Parent-child, (double) linked list.. • Query by graph navigation • Examples – CODASYL © Syed Awase 2015-16 – MongoDB Ground Up 9
  • 10.
    Relational Databases • Dataorganized as tuples in relations • Link between data tuples – Primary and foreign keys • Relational algebra – Project, select, join • Relational normal forms • Declarative language – Data definition, consistency, manipulation and querying • Examples – Oracle, Microsoft SQL Server, IBM DB2 – PostgreSQL, MySQL © Syed Awase 2015-16 – MongoDB Ground Up 10
  • 11.
    Relational Databases • Relationalmodel is very simple – Only basic concepts – references need to be simulated – Restricted type system – no user-defined types • Lack of semantic modeling – Complex data, versioning, roles • Little support of data and schema evolution • Object-relational impedance mismatch. © Syed Awase 2015-16 – MongoDB Ground Up 11
  • 12.
    RDBMS Problem • Designfor ACID – Atomicity, Consistency, Isolation and Durability • Hard to Scale • Availability? • Flexibility © Syed Awase 2015-16 – MongoDB Ground Up 12
  • 13.
    Traditional Scalability • Scale-up –Memory and hardware has limitations • Scale-out – Scaling reads • Cache is the King – Query cache, Memcache, OLAP • Prefetching • Replication • Scaling Writes – Redundant disk arrays, RAID – Sharding © Syed Awase 2015-16 – MongoDB Ground Up 13
  • 14.
    Common Problems • Relationmodel is heavy – Parsing – Locking – Logging – Buffer pool and thread • Not every case can work within a single node SMP • Sharding does not solve all problems – Cross shard or join between shards – Need to update across multiple shards within a transaction. – Shard failure – Online Schema changes without taking the shard offiline – Add or replace shards in-line. © Syed Awase 2015-16 – MongoDB Ground Up 14
  • 15.
    Lack of SemanticModeling • All advantages of the object-oriented paradigm which enable to produce models of an application domain that capture more of its aspects, e.g. using – Inheritance – Relationships • Inheritance and relationships can be mapped to the relational model, however, they result in a complex database schema. Such complex schemata are difficult to develop, maintain and decrease runtime performance. © Syed Awase 2015-16 – MongoDB Ground Up 15
  • 16.
    Object-Relational Impedance Mismatch •Object-oriented applicationdevelopment and relational data management results in clash of two incompatible models. •Code to map between models is considerable overhead, costly and hard to maintain. © Syed Awase 2015-16 – MongoDB Ground Up 16
  • 17.
    Database Evolution • Object-relationalmapping (ORM) layers don’t really help – Writing manual code for ORM poses problem – Evolution code can usually not be generated automatically • Automatic evolution is not possible for complex cases – Semantic changes – Certain type changes ( String to Person) – Ambiguity of attribute add-remove vs. rename – Class hierarchy changes © Syed Awase 2015-16 – MongoDB Ground Up 17
  • 18.
    Object-Oriented Databases • Datarepresented as objects – Object Identity – Attributes and methods – Reference, relationships, associations • Extensible type hierarchy – User-defined types, abstract data types – single or multiple inheritance – Overloading, overriding, late binding • Declarative language for ad hoc purposes • Binding for Object-oriented programming language © Syed Awase 2015-16 – MongoDB Ground Up 18
  • 19.
    Object-Relational Database • Relationalmodel extended – Nested relations – References – Sets – Row types, abstract types – Functions • Declarative language extended – Computationally complete • Fundamental impedance mismatch remains © Syed Awase 2015-16 – MongoDB Ground Up 19
  • 20.
    Non-traditional Databases • XMLDatabase • Graph Databases • Document Databases • Key-value Stores • Column Store • Others – Geospatial – File System NoSQL Non-Relational, Web- scale database First used by Carlo Strozzi in 1998 Reintroduced by Eric Evan in 2009 © Syed Awase 2015-16 – MongoDB Ground Up 20
  • 21.
    Non-traditional Database Characteristics • Non-relationalin Nature • Distributed • Horizontal Scalability • Schema-less/Schema-free • Eventual Consistency © Syed Awase 2015-16 – MongoDB Ground Up 21
  • 22.
    Gartner Hype Cycle NoSQL ©Syed Awase 2015-16 – MongoDB Ground Up 22
  • 23.
    NO-SQL Next Generation Databasesmostly addressing some of the points: being non-relational, distributed, open-source and horizontal scalable.. Schema- free, easy replication support, simple API, eventually consistent” © Syed Awase 2015-16 – MongoDB Ground Up -nosql-database.org • Non-Relational • Distributed • Open-Source • Horizontally Scalable • Schema-Free • Replication Support • Eventually Consistent Classification • Wide Column Store/Column Families • Document Store • Key value/Tuple Store • Graph Database • Multimodel Database • Object Database • Grid and Cloud Database Solutions • XML Databases • Multidimensional Databases • Multivalue databases • Time Series/Streaming Databases •Scientific and Specialized DB 23
  • 24.
    Why NoSQL? © SyedAwase 2015-16 – MongoDB Ground Up 24
  • 25.
    Why NoSQL? © SyedAwase 2015-16 – MongoDB Ground Up 25
  • 26.
    Why NoSQL? Sharding: Adatabase shard is a horizontal partition in a database or search engine. Each individual partition is referred to as a shard or database shard. © Syed Awase 2015-16 – MongoDB Ground Up 26
  • 27.
    Why NoSQL? DESIGN GOALS •Semi Structured >> • Big Data >> • Real-time >> • Ubiquity >> • Schema-Free • Scalable reads/writes – Horizontal Scalability • To increase the no-of machines but maintaining proportional performance. – Vertical Scalability • To add more resources to your single machine to optimize performance. • High-performance • High-availability • Reliability, Fault tolerant. © Syed Awase 2015-16 – MongoDB Ground Up 27
  • 28.
    NoSQL vs RDBMS NoSQLRDBMS • Schema-free • Scalable writes/reads • Auto high-availability • Limited queries • Eventual Consistency – applies to most NoSQL systems • BASE : Basic Availability, Soft state, Eventual consistency • Relational Schema • Scalable reads • Custom high-availability • Flexible • Consistent • ACID : Atomic, Consistent, Isolated and Durable. © Syed Awase 2015-16 – MongoDB Ground Up 28
  • 29.
    RDBMS vs NoSQL •Traditional distributed data management solutions focus on ACID semantics – Atomicity – Consistency – Isolation – Durability • Modern Web-scale data management focuses on – Basic Availability – Soft-state – Eventual Consistent – CAP (Consistency, Availability, Partition and Tolerance) © Syed Awase 2015-16 – MongoDB Ground Up 29
  • 30.
    ACID • Atomicity :All of the operations in the transaction will complete or none will • Consistency: the database will be in a consistent state when the transaction begins and ends. • Isolation: the transaction will behave as if it is the only operation being performed upon the database. (No interference of transaction). • Durability : Upon completion of the transaction, the operation will not be reversed. © Syed Awase 2015-16 – MongoDB Ground Up 30
  • 31.
    CAP• Consistency: No contradiction b/wdata. • Availability: Every operation must terminate in an intended response. • Partition tolerance: Operations will complete, even If individual components are unavailable. • NoSQL are based on CAP. © Syed Awase 2015-16 – MongoDB Ground Up 31
  • 32.
    ACID vs BASE ACIDBASE • Strong consistency • Isolation • Focus on “commit” • Nested Transactions • Availability? • Pessimistic/Conservative • Difficult evolution (e.g. schema) • Weak consistency • Availability first • Best effort • Approximate answers (OK) • Optimistic (aggressive) • Fast and simple • Easier evolution © Syed Awase 2015-16 – MongoDB Ground Up 32
  • 33.
    Who’s using NoSQL? ©Syed Awase 2015-16 – MongoDB Ground Up 33
  • 34.
    Who’s using NoSQL? ©Syed Awase 2015-16 – MongoDB Ground Up 34
  • 35.
    Growth © Syed Awase2015-16 – MongoDB Ground Up 35
  • 36.
    NoSQL Use Cases •Consumer Use Cases • Facebook • Twitter • NetFlix • Enterprise Use Cases • Rackspace • TrendMicro • NewsCred Facebook •Hbase – Facebook messages •Scribe- Real-time click logs •Hive- SQL Queries -> MapReduce jobs •Hadoop •Web analytics warehouse •Distributed datastore •MySQL backups © Syed Awase 2015-16 – MongoDB Ground Up 36
  • 37.
    Choosing a NoSQLSolution © Syed Awase 2015-16 – MongoDB Ground Up 37
  • 38.
    Consistent, Available (CA) •CA- systems have trouble with partitions and deal with it with replication • Examples – MySQL (relational) – AsterData (relational) – Greenplum(relational) – Vertica (Column) © Syed Awase 2015-16 – MongoDB Ground Up 38
  • 39.
    Availability, Partition-Tolerant (AP) •AP-systems have trouble with consistency, achieve “eventual consistency” through replication. • Examples – Cassandra(column/tabular) – Dynamo (Key-value) – Voldemort (Key-value) – Tokyo Cabinet (Key-value) – CouchDB(document) – SimpleDB(document) – Riak(document) © Syed Awase 2015-16 – MongoDB Ground Up 39
  • 40.
    Consistent, Partition-Tolerant (CP) •CP-systems have trouble with availability while keep data consistent across partitioned nodes. • Examples – MongoDB(document) – BigTable(Column/tabular) – HyperTable(Column/tabular) – Hbase (Column/tabular) – Redis(key-value) – Scalaris(key-value) – MemcacheDB (Key-value) © Syed Awase 2015-16 – MongoDB Ground Up 40
  • 41.
    Categories of NoSQLdatabases 1) Key Value stores  Don’t have any schema  Fast lookups facility  Can be use in a forum software where user’s statistic and messages are recorded. User’s id will serve as a key and will retrieve a string that represents all the relevant info of the user. And a background process recalculates the information and writes to the store independently after fixed interval of time.  Example  Redis : Redis is an open source, advanced key-value store. It is often referred to as a data structure server since keys (data types) can contain strings, hashes, lists, sets and sorted sets. © Syed Awase 2015-16 – MongoDB Ground Up 41
  • 42.
    Categories (cont…)  API:Tons of languages, Written in: C,  Concurrency: in memory and saves asynchronous disk after a defined time. 2) Document databases  web application  tolerance of incomplete data  low query performance  no standard query syntax  Used when we don’t have complete data about all the entities of database but we still need to create database. Example CouchDB: API: JSON, Protocol: REST, Query Method: MapReduceR of JavaScript Funcs, Replication: Master Master, Written in: Erlang. © Syed Awase 2015-16 – MongoDB Ground Up 42
  • 43.
    Categories (cont…)  MongoDB:API: BSON, Protocol: lots of langs, Query Method: dynamic object-based language, Replication: Master Slave, Written in: C++. 3) Graph databases  Social networking  graph algorithms, connectedness, degree of relationships  has to traverse the entire graph to get definitive answer.  not easy to cluster.  used in a situation where we want to analyze the on going trends and take decision on the basis of those trends. © Syed Awase 2015-16 – MongoDB Ground Up 43
  • 44.
    Categories (cont…)  Example: Neo4J:API: lots of langs, Protocol: Java embedded / REST, Query Method: SparQL, nativeJavaAPI, JRuby, Replication: typical MySQL style master/slave, Written in: Java, Concurrency: non-block reads, writes locks involved nodes/relationships until commit, Misc: ACID possible 4) XML databases  Publishing  mature search technologies  schema validation  re-writing is easier than updating © Syed Awase 2015-16 – MongoDB Ground Up 44
  • 45.
    Categories (cont…)  Usedin a situation where one wants to produce documents of articles etc from a huge amount of documents but the format of those article doesn’t allow the publisher to perform search on it. Those articles are converted into xml database and wrap it in a readable-URL web service for the document production systems. 5) Distributed Peer Stores  distributed file systems  Fast lookups  Good distributed storage of data  Very low level API  Best for voting system. In such a situation one store/user and one store/piece of content is created. The user store will hold all the votes they have ever casted and the content will store a copy of the content on which vote was casted. © Syed Awase 2015-16 – MongoDB Ground Up 45
  • 46.
    Categories (cont…) Example Cassandra: API:many Thrift languages, Query Method: MapReduce, Written in: Java, Concurrency: eventually consistent , Misc: like "Big-Table on Amazon Dynamo alike", initiated by Facebook. © Syed Awase 2015-16 – MongoDB Ground Up 46
  • 47.
    Object Databases • Versant •Db40 • Objectivity • Startcounter • Perst • VelocityDB • HSS Database • ZODB • Magma • NEO • Ninja Database • PicoLisp • Acid-state • ObjectDB • Siaqodb • Morantex • EyeDB • FrameD © Syed Awase 2015-16 – MongoDB Ground Up 47
  • 48.
    Key-Value Stores • HASH •Memcached • DynamoDB • Redis • Riak • Tokyo Cabinet • Azure Table Storage • Riak • Aerospike • FoundationDB • LevelDB • BerkeleyDB • GenieDB • BangDB • Chordless • Allegro-C • nessDB • HyperDex • TokyoCabinet/Tyrant • Scalien • Voldemort • Dynomite • KAI • MemcahcheDB • Faircom C-Tree • LSM • KitaroDB • HamsterDB • STSdb • Tarantool/Box • MaxTable • Quasardb • Pincaster • RaptorDB • TIBCO Active Spaces • Sophia © Syed Awase 2015-16 – MongoDB Ground Up 48
  • 49.
    XML Database • EMCDocumentum xDB • eXist • Sedna • BaseX • Qizx • Berkeley DB XML © Syed Awase 2015-16 – MongoDB Ground Up 49
  • 50.
    Multidimensional databases • Globals •Intersystems Cache • GT.M • SciDB • MiniM DB • Rasdaman © Syed Awase 2015-16 – MongoDB Ground Up 50
  • 51.
    Document Store (Semi-Structured) •IBM Lotus • MongoDB • CouchDB • MongoDB • ElasticSearch • RethinkDB • RavenDB • MarkLogic Server • Clusterpoint Server • ThruDB • Terrastore • JasDB • RaptorDB • Djondb • EJDB • Amisa Server • SisoDB • SDB • NoSQL embedded Db • Document Stores •Supports complex data model than Key Value Pairs •Good at handling content management, session, profile data •Multi index support •Dynamic schemas, Nested schemas •Auto distributed, eventual consistency •MVCC (CouchDB) or automic (MongoDB) • MongoDB, SimpleDB: widely adopted in this space • Use Case: Search by complex patterns & CRUD apps © Syed Awase 2015-16 – MongoDB Ground Up 51
  • 52.
    Column-Oriented Stores • Semi-Structured •BigTable(inspiration Source) • Hbase • Apache Cassandra • Hypertable • Accumulo • Amazon Simple DB • Cloudata • Cloudera • HPCC • Stratosphere Hbase (Apache), Cassanda (Facebook) and HyperTable (Bidu) Hbase – CA Cassandra – AP Model consists of rows and columns Scalability: Splitting of both rows and columns Rows are split across nodes using primary key, range Columns are distributed using groups Horizonal and vertical partitioning can be used simultaneous Extension of document store HBase uses HDFS; Pig, Hive, Cascading can help Use case: Grouping of frequently used and un-used over data centers / stream of writes © Syed Awase 2015-16 – MongoDB Ground Up 52
  • 53.
    Graph Database • InfoGrid •Neo4j • InfiniteGraph • DEX • Titan • InforGrid • HyperGraphDB • GraphBase • Trinity • AllegroGraph • BrightstarDB • BigData • Meronymy • WhiteDB • Openlink Virtuoso • VertexDB • FlockDB Social Graph Relationship between entities Data modeling on social networks Common Use Cases List of friends Recommendation system Following Followers Common Connections FAN IN/OUT © Syed Awase 2015-16 – MongoDB Ground Up 53
  • 54.
    Multimodel databases • ArangoDB •OrientDB • Datanomic • FatDB • AlchemyDB • CortexDB © Syed Awase 2015-16 – MongoDB Ground Up 54
  • 55.
    NoSQL (Not ONLYSQL) • Fits very well for volatile data • High read or write throughput. • Automatic horizontal scalability (Consistent Hashing) • Simple to implement, no investment for developers to design and implement relational model • Application logic defines object model • Support of MVCC (MultiVersion Concurrency Control) in some form • Compaction and un-compaction happens at top tier • In-memory or disk based or combination © Syed Awase 2015-16 – MongoDB Ground Up 55
  • 56.
    NoSQL - Disadvantages •Packing and Un-packing of each key • Lack of relation from one key to another • Need whole value from the key; to read/write any practical information. • No Security or authentication • Data store is merely a storage layer, can’t be used for – Analytics – Reporting – Aggregation – Ordered Values © Syed Awase 2015-16 – MongoDB Ground Up 56
  • 57.
    Some Popular NoSQLDatabases © Syed Awase 2015-16 – MongoDB Ground Up 57
  • 58.
    MongoDB 3.2 • Writtenin: C++ • Main point: Retains some friendly properties of SQL. (Query, index) • License: AGPL (Drivers: Apache) • Protocol: Custom, binary (BSON) • Master/slave replication (auto failover with replica sets) • Sharding built-in • Queries are javascript expressions • Run arbitrary javascript functions server-side • Better update-in-place than CouchDB • Uses memory mapped files for data storage • Performance over features • Journaling (with --journal) is best turned on • On 32bit systems, limited to ~2.5Gb • An empty database takes up 192Mb • GridFS to store big data + metadata (not actually an FS) • Has geospatial indexing • Data center aware Best used: If you need dynamic queries. If you prefer to define indexes, not map/reduce functions. If you need good performance on a big DB. If you wanted CouchDB, but your data changes too much, filling up disks. For example: For most things that you would do with MySQL or PostgreSQL, but having predefined columns really holds you back. © Syed Awase 2015-16 – MongoDB Ground Up 58
  • 59.
    Riak 1.2v • Writtenin: Erlang & C, some JavaScript • Main point: Fault tolerance • License: Apache • Protocol: HTTP/REST or custom binary • Stores blobs • Tunable trade-offs for distribution and replication • Pre- and post-commit hooks in JavaScript or Erlang, for validation and security. • Map/reduce in JavaScript or Erlang • Links & link walking: use it as a graph database • Secondary indices: but only one at once • Large object support (Luwak) • Comes in "open source" and "enterprise" editions • Full-text search, indexing, querying with Riak Search • In the process of migrating the storing backend from "Bitcask" to Google's "LevelDB" • Masterless multi-site replication replication and SNMP monitoring are commercially licensed Best used: If you want something Dynamo-like data storage, but no way you're gonna deal with the bloat and complexity. If you need very good single-site scalability, availability and fault-tolerance, but you're ready to pay for multi-site replication. For example: Point-of-sales data collection. Factory control systems. Places where even seconds of downtime hurt. Could be used as a well-update-able web server. © Syed Awase 2015-16 – MongoDB Ground Up 59
  • 60.
    CouchDB 1.2v • Writtenin: Erlang • Main point: DB consistency, ease of use • License: Apache • Protocol: HTTP/REST • Bi-directional (!) replication, • continuous or ad-hoc, • with conflict detection, • thus, master-master replication. (!) • MVCC - write operations do not block reads • Previous versions of documents are available • Crash-only (reliable) design • Needs compacting from time to time • Views: embedded map/reduce • Formatting views: lists & shows • Server-side document validation possible • Authentication possible • Real-time updates via '_changes' (!) • Attachment handling • thus, CouchApps (standalone js apps) Best used: For accumulating, occasionally changing data, on which pre-defined queries are to be run. Places where versioning is important. For example: CRM, CMS systems. Master-master replication is an especially interesting feature, allowing easy multi-site deployments. © Syed Awase 2015-16 – MongoDB Ground Up 60
  • 61.
    Redis 2.8v • Writtenin: C • Main point: Blazing fast • License: BSD • Protocol: Telnet-like, binary safe • Disk-backed in-memory database, • Dataset size limited to computer RAM (but can span multiple machines' RAM with clustering) • Master-slave replication, automatic failover • Simple values or data structures by keys • but complex operations like ZREVRANGEBYSCORE. • INCR & co (good for rate limiting or statistics) • Bit operations (for example to implement bloom filters) • Has sets (also union/diff/inter) • Has lists (also a queue; blocking pop) • Has hashes (objects of multiple fields) • Sorted sets (high score table, good for range queries) • Lua scripting capabilities (!) • Has transactions (!) • Values can be set to expire (as in a cache) • Pub/Sub lets one implement messaging Best used: For rapidly changing data with a foreseeable database size (should fit mostly in memory). For example: Stock prices. Analytics. Real- time data collection. Real- time communication. And wherever you used memcached before © Syed Awase 2015-16 – MongoDB Ground Up 61