§ Focus
    § Raising awareness
    § Trends
    § High level

§ Questions
   § Why are non-relational databases increasin...
§ Trend 1: Data is becoming more and more connected
    § Joins, joins, and more joins (relationships are exploding)

§ Tr...
§ “Trend” 4: Cloud Computing
   § ..and perhaps more specifically, the scaling and fault tolerance needs.
   § For cloud p...
§ What is ACID?
   § A promise ring your RDBMS wears.
   § Atomic, Consistent, Isolated, Durable
   § ACID trips when:
   ...
Key Value Databases                        Column-Oriented Databases
Stores entities as key value                     Stor...
Database System       Type                Open Source/Commercial/Proprietary
Dynamo                Key Value           Pro...
§ Concepts
    § Domains: similar to table concept except schema-less.
    § Keys: arbitrary value.
    § Values: arbitrar...
§ Memcached
   § Originally developed to speed up LiveJournal.com.
    § Generic in nature but intended for use in allevia...
§ SimpleDB
    § Written in Erlang (luckily you don’t need to know it to use it).
    § Eventually consistency is a key fe...
§ SimpleDB Limitations




                Key Value Databases: SimpleDB
§ Overview
                        EmployeeID   Name    Position
                        1            Moe     Director
   ...
§ BigTable (and HBase, and Hypertable)
   § BigTable == Google
   § HBase == Interpretation of BigTable (Java) + Hadoop
  ...
content
    “www.cnn.com”       content      language    anchor: bms.com
                            Contents
  “www.cnn.c...
§ Overview
   § Similar to key value stores
   § Most employ JSON.
   § Inherently schema-less
   § Most are denormalized....
“… is a distributed, fault tolerant, and schema-free
              document-oriented database accessible via a RESTful
   ...
Document Databases: CouchDB
Document Databases: CouchDB
§ Overview
    § Nodes represent entities.
    § Edges represent relationships.
    § Nodes and edges can have associated ...
§ Overview
    § Open source.
    § Java based.
    § Lightweight (single <500k JAR with minimal dependencies).
    § Stil...
§ If you’re in the cloud, you’re going to use them.
   § Amazon Web Services: SimpleDB
   § Google App Engine: BigTable
  ...
Upcoming SlideShare
Loading in …5
×

Non Relational Databases

5,913 views

Published on

High level presentation of non-relational databases focusing on key vale, document, column oriented, and graph.

Published in: Technology
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
5,913
On SlideShare
0
From Embeds
0
Number of Embeds
522
Actions
Shares
0
Downloads
0
Comments
0
Likes
11
Embeds 0
No embeds

No notes for slide

Non Relational Databases

  1. 1. § Focus § Raising awareness § Trends § High level § Questions § Why are non-relational databases increasing in usage? § What types or categories exist? § What are some examples in each category? § Why should I [the developer, the administrator, etc.] care? A View of the Non-Relational Database Landscape
  2. 2. § Trend 1: Data is becoming more and more connected § Joins, joins, and more joins (relationships are exploding) § Trend 2: Data sets are becoming larger and larger § Instruments dump massive amounts of data in the lab § Trend 3: Data is becoming less and less structured Why Are Non-Relational DBs Increasing In Usage?
  3. 3. § “Trend” 4: Cloud Computing § ..and perhaps more specifically, the scaling and fault tolerance needs. § For cloud providers, these are required hence addressed from the outset. § Backing up is replaced with having multiple active copies… § Data sets exist over multiple machines… § Nodes can crash and applications live to see another day… § Nodes can be added (or removed) at any point in time… vs. Why Are Non-Relational DBs Increasing In Usage?
  4. 4. § What is ACID? § A promise ring your RDBMS wears. § Atomic, Consistent, Isolated, Durable § ACID trips when: § Downtime is unacceptable § Reliability is >= 2 nodes § Challenging over networks § What is CAP Theorem? § Distributed systems can have two: § Consistency (data is correct all the time) § Availability (read and write all the time) § Partition Tolerance (plug and play nodes) § What is BASE? § More people much smarter than me came up with an ACID alternative: § Basically Available (appears to work all the time) § Soft state (doesn’t have to be consistent all the time…) § Eventually consistent (…but eventually it will be) Turn Up The BASE
  5. 5. Key Value Databases Column-Oriented Databases Stores entities as key value Stores entities by column pairs in large hash tables (versus row) Document Databases Graph Databases Stores documents (JSON) Stores entities as nodes and edges Distributed Databases More attribute than type! Non-Relational Database Landscape
  6. 6. Database System Type Open Source/Commercial/Proprietary Dynamo Key Value Proprietary (Amazon) SimpleDB Key Value Commercial (Amazon Web Services) Project Voldemort Key Value Open Source (started @ LinkedIn) Memcached Key Value Open Source Redis Key Value Open Source Tokyo Cabinet Key Value Open Source Cassandra Column-oriented * Open Source (started @ Facebook) BigTable Column-oriented * Proprietary (Google), Commercial (AppEngine) Hypertable Column-oriented * Open Source (implementation of BigTable) Hbase Column-oriented * Open Source (implementation of BigTable) CouchDB Document Open Source MongoDB Document Open Source Neo4j Graph Open Source Notable Non-Relational Databases
  7. 7. § Concepts § Domains: similar to table concept except schema-less. § Keys: arbitrary value. § Values: arbitrary blobs. § No explicit relationships between domains or within a domain. § Access § API (often SOAP or RESTful). § Some provide SQL-like syntax. § Basic filter predicates (=, !=, <, >, <=, >=). Ke Attributes y § Integrity 1 Make: Nissan § Often contained in application code Model: Pathfinder Color: Green Year: 2003 2 Make: Nissan Model: Pathfinder Color: Green Year: 2003 Transmission: Auto Key Value Databases
  8. 8. § Memcached § Originally developed to speed up LiveJournal.com. § Generic in nature but intended for use in alleviating database load. § Lightening fast, distributed, RAM only, no persistence. § “Everyone” uses it: Facebook, Digg, Slashdot, Twitter, YouTube, SourceForge, … function get_foo(int userid) { result = db_select("SELECT * FROM users WHERE userid = ?", userid); return result; } function get_foo(int userid) { result = memcached_fetch("userrow:" + userid); if (!result) { result = db_select("SELECT * FROM users WHERE userid = ?", userid); memcached_add("userrow:" + userid, result); } return result; } Key Value Databases: Memcached
  9. 9. § SimpleDB § Written in Erlang (luckily you don’t need to know it to use it). § Eventually consistency is a key feature (concurrency!!) § Available via Amazon Web Services at very low cost. § Very common to use it in conjunction with other AWS offerings (EC2, S3, SQS). Key Value Databases: SimpleDB
  10. 10. § SimpleDB Limitations Key Value Databases: SimpleDB
  11. 11. § Overview EmployeeID Name Position 1 Moe Director 2 Larry Developer 3 Curly Analyst A gross (emphasis on gross) simplification of what this serializes too… ROW: 1,Moe,Director;2,Larry,Developer;3,Curly,Analyst COLUMN: 1,2,3;Moe,Larry,Curly;Director,Developer,Analyst § Where It Shines § Querying many rows for smaller subsets of data (not all columns) § Maximizes disk performance (read scans) § Where It Is Outperformed § Querying all columns of a single row § Writing a new row if all of the column data is supplied at the same time Column Oriented Databases
  12. 12. § BigTable (and HBase, and Hypertable) § BigTable == Google § HBase == Interpretation of BigTable (Java) + Hadoop § Hypertable == Interpretation of BigTable (C++) + Hadoop § Collections of “Multi-dimensional Sparse Maps” A–y cell => row, column, timestamp A–n A Contents B … A’ B’ … § Rows § Columns § Name is an arbitrary string. § Two level naming structure § Ordered lexicographically. § family:optional_qualifier § Atomic access. § Families are a unit of access. § Creation is implicit. § Few column families in a table § Families can be marked with attributes. § Families can be assigned to locality groups Column Like Databases: BigTable & Co.
  13. 13. content “www.cnn.com” content language anchor: bms.com Contents “www.cnn.com/...” content language … Contents “www.cnn.com/.../...” content language … “Application A” jonest: settings schmoej: settings “Application B” … … “Application C” … … assay:a “Sample X” assay:a assay:b Contents “Sample Y” … … Contents “Sample Z” … … Column Like Databases: BigTable & Co.
  14. 14. § Overview § Similar to key value stores § Most employ JSON. § Inherently schema-less § Most are denormalized. § Often composed of collections (akin to tables w/o schema) Document Databases
  15. 15. “… is a distributed, fault tolerant, and schema-free document-oriented database accessible via a RESTful HTTP/JSON API…” § Other Tidbits § Believe it or not, idea was inspired by Lotus Notes. § Hosted with Apache, written in Erlang. § Futon: clean, stream-lined administrator interface. § Basic API § Create: HTTP PUT § Read: HTTP GET § Update: HTTP POST § Delete: HTTP DELETE § Adding Structure To Semi-Structured Data § Views are the method of aggregating and reporting on documents. § Built on-demand, dynamically, and do affect underlying documents. § Views are persisted. Document Databases: CouchDB
  16. 16. Document Databases: CouchDB
  17. 17. Document Databases: CouchDB
  18. 18. § Overview § Nodes represent entities. § Edges represent relationships. § Nodes and edges can have associated attributes (key values). § Most anything can be described as a graph. § Key value store with full support for relationships. Graph Databases
  19. 19. § Overview § Open source. § Java based. § Lightweight (single <500k JAR with minimal dependencies). § Still very early in development but looks promising. § Can handle graphs of several billion nodes/relationships/properties. § Disk based, solid state drive (SSD) ready. § Optional layers to expose it as an RDF store (OWL, SPARQL). § Has RDBMS features (ACID, durable persistence) Graph Databases: Neo4j
  20. 20. § If you’re in the cloud, you’re going to use them. § Amazon Web Services: SimpleDB § Google App Engine: BigTable § Open Source: Memcached, HBase, Hypertable, Cassandra, and more… § Break the habit; relational databases do not fit every problem. § Stuffing files into a RDBMS, maybe there’s something better? § Using a RDBMS for caching, perhaps a lighter-weight solution is better? § Cramming log data into a RDBMS, perhaps a key value store is better? § Despite the hype, relational databases are not doomed. § Though in my opinion their role and place will certainly change. § Scaling is a real challenge for relational databases. § Sharding is a band-aid, not feasible beyond a few nodes. § There is a hit in overcoming the initial learning curve § It changes how you build applications Parting Thoughts & Musings

×