0

NO SQL: What, Why, How

1,791

Published on

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,791
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
76
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Why are we here?
    What is wrong with the status quo?
  • What is wrong with the scale up? It worked before? Moore theorem is still valid and according to predictions, will be for some time to come.
  • CAP theorem, also known as (Eric) Brewer's theorem
    Consistency: All clients always have the same view of the data
    Availability: Each client can always read and write
    Partitioning: The system works well despite physical network partitions
  • Commodity hardware – in fact the real commodity hardware.
    But the industry acknowledges the need of packaging a lot of “commodity” machines in a small space – new computers on a chip models are coming.
  • According to NOSQL-databases.org lists 122+ databases:
    Next Generation Databases address some of the following points: being non-relational, distributed, open-source and horizontal scalable. The original intention has been modern web-scale databases. The movement began early 2009 and is growing rapidly. Often more characteristics apply as: schema-free, replication support, easy API, eventually consistency, and more. So the misleading term "NOSQL" (the community now translates it mostly with "not only sql") should be seen as an alias to something like the definition above.
  • Twitter generates 7TB/day (2PB+ year) – Hadoop for data analysis, Scribe for logging
    LinkedIn - Voldemort
  • Scalability:  relational databases were not designed to handle and do not generally cope well with Internet-scale, “big data” applications.  Most of the big Internet companies (e.g., Google, Yahoo, Facebook) do not rely on RDBMS technology for this reason.
  • Cassandra – Facebook Inbox Search
    Amazon Dynamo: not open source Voldemort: Open-Source implementation of Amazons Dynamo Key-Value Store. 
    Google Big Table: a sparse, distributed multi-dimensional sorted map
  • Distributed Storage Consistency Models - http://horicky.blogspot.com/2008/08/distributed-storage.html
    There is a number of client consistency models (http://horicky.blogspot.com/2009/11/nosql-patterns.html)Strict Consistency (one copy serializability): This provides the semantics as if there is only one copy of data. Any update is observed instantaneously.
    Read your write consistency: The allows the client to see his own update immediately (and the client can switch server between requests), but not the updates made by other clients
    Session consistency: Provide the read-your-write consistency only when the client is issuing the request under the same session scope (which is usually bind to the same server)
    Monotonic Read Consistency: This provide the time monotonicity guarantee that the client will only see more updated version of the data in future requests.
    Eventual Consistency: This provides the weakness form of guarantee. The client can see an inconsistent view as the update are in progress. This model works when concurrent access of the same data is very unlikely, and the client need to wait for some time if he needs to see his previous update.
  • The Simple Magic of Consistent Hashing - http://www.paperplanes.de/2011/12/9/the-magic-of-consistent-hashing.html
    Consistent Hashing - http://michaelnielsen.org/blog/consistent-hashing/
  • Transcript of "NO SQL: What, Why, How"

    1. 1. NO-SQL: WHY, WHAT, HOW Igor Moochnick Director, Cloud Platforms BlueMetal Architects igorm@bluemetal.com Blog: igorshare.wordpress.com
    2. 2. What is wrong with SQL?  Is it answering your needs?  Does it fit your solution?  Do you rip the benefits of the relational storage?  Will it support the needs of your projects in the future?  More users?  More data?
    3. 3. Gov't data stored in the US (2009): more than 800 petabytes
    4. 4. Assumptions  The data doesn’t fit on one node  The data may not fit one rack  Each machine operates independently with minimal coordination between themselves  Conclusion:  There is a need to partition data across lots of machines
    5. 5. Scale up or out ?
    6. 6. There is a limit to RDBMS scale  Scaling up doesn't work  Scaling out with traditional RDBMSs isn't so hot either  Sharding scales, but you lose all the features that make RDBMSs useful!  Sharding and Table partitioning are operationally heavy  If we don't need relational features, we want a distributed NRDBMS.
    7. 7. Fallacies of Distributed Computing 1. The network is reliable 2. Latency is zero 3. Bandwidth is infinite 4. The network is secure 5. Topology doesn’t change 6. There is one administrator 7. Transport cost is zero 8. The network is homogeneous http://en.wikipedia.org/wiki/Fallacies_of_Distributed_Computing
    8. 8. The magic CAP CA CP AP
    9. 9. Commodity Hardware Here Calxeda's EnergyCard atop a HP Redstone server prototype. Source: Jon Snyder CNET: Google uncloaks once-secret server RAM is new Disk, Disk is new Tape - Jim Gray, (former) manager of Microsoft Research’s eScience Group
    10. 10. Why it is important  New levels of scalability  Rapid development  Cloud ready  Distributed by nature  There’s no need for DBA, no need for complicated SQL queries and it is fast. Hooray, freedom for the people!  WORLD, PEACE!
    11. 11. Battle of the viewpoints
    12. 12. Who is using NOSQL?
    13. 13. Beware!  Data models are still important  Data duplication  Interfaces and interoperability - nonexistent  Understand limitations of the technology  OPS are screwed
    14. 14. Advantages of NOSQL  Cheap, easy to implement  Removes impedance mismatch between objects and tables  Quickly process large amounts of data  Data Modeling Flexibility (including schema evolution) Disadvantages of NOSQL  New Technology  Data is generally duplicated, potential for inconsistency  No standard language or format for queries  Depends on the application layer to enforce data integrity
    15. 15.  Document Databases  Based loosely on documents / POCO  Data model – collections of documents  Graph Databases  Based on Graph theory  Data model – graph, nodes, edges, properties NOSQL categories
    16. 16. NOSQL categories  Key Value Stores  Based on DHT (Distributed Hash Table), Amazon’s Dynamo design  Data model – collection of key value pairs  Column Stores  Based on Google’s BigTable design  Data model - big table, column families
    17. 17. Types of NOSQL Databases  Document (examples: MongoDB, CouchDB, RavenDB)  Graph (examples: Neo4J, Sones, TinkerGraph)  Key/Value (examples: Cassandra, SimpleDB, Dynamo, Voldemort, Riak, Redis)  Tabular/Wide Column (examples: BigTable, Hbase, Cassandra)  Search (example: Lucene) http://NOSQL-databases.org
    18. 18. Consistency Models  Full Consistency  Read-what-I-wrote  Session Consistency  Monotonic Read Consistency  Eventual Consistency
    19. 19. Write collision resolutions  Timestamps  Vector Clocks  HiLo algorithm
    20. 20. Quorum
    21. 21. Sharding  Is NOT  Replication  Clustering  Backup  It is a smart way of splitting data across databases  Requires aggregation  Enables parallelization
    22. 22. Where is my data?  Lookup tables  (Consistent) Hash functions Node A Node B Node C
    23. 23. Gossip NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode
    24. 24. Gossip (round 1) NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode
    25. 25. Gossip (round 2) NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode
    26. 26. Gossip (round 3) NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode
    27. 27. Gossip (round 3) NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode
    28. 28. Gossip (round 4) NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode NodeNodeNodeNode
    29. 29. Don’t forget backups !!! Replication ≠ Backups
    30. 30. Modeling  Stop thinking relational  Start thinking about how your data will be used  Usage scenarios  Optimize for reads? Writes?  Think about your domain objects and business logic in native .Net (POCO) classes  Deformalize if needed  Reference entities to other entities, or collections of them  Identify aggregate root(s)
    31. 31. Map-Reduce  First impressions, but it’s get better over time Original
    32. 32. Play Nice  Know and use the tools you need for the job at hand
    33. 33. Extra Links and References  NOSQL debrief  Distributed Storage - http://horicky.blogspot.com/2008/08/distributed- storage.html  Images from here
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×