Non-relational databases were developed to address the problems that traditional relational databases have in handling web-scale applications with massive amounts of data and users. They sacrifice consistency to gain availability and partition tolerance. Examples include BigTable, HBase, Dynamo, and Cassandra. They provide benefits like massive scalability, high availability, and elasticity through techniques like consistent hashing, replication, and MapReduce processing.

1. Non-relational Databases A new kind of Databases for handling Web Scale

2. Agenda The problem

3. The solution

4. Benefits

5. Cost

6. Example: Cassandra

7. The problem The Web introduces a new scale for applications, in terms of: Concurrent users (millions of reqs/second)

8. Data (peta-bytes generated daily)

9. Processing (all this data needs processing)

10. Exponential growth (surging unpredictable demands)

11. The problem (contd.) Web sites with very large traffic have no way to deal with this using existing RDBMS solutions: Oracle

12. MS SQL

13. Sybase

14. MySQL

15. PostgreSQL Even with their high-end clustering solutions

16. The problem (contd.) Why? Applications using normalized database schema require the use of join's, which doesn't perform well under lots of data and/or nodes

17. Existing RDBMS clustering solutions require scale-up, which is limited & not really scalable when dealing with exponential growth

18. Machines have upper limits on capacity, & sharding the data & processing across machines is very complex & app-specific

19. The problem (contd.) Why not just use sharding? Very problematic when adding/removing nodes

20. Basically, you end up denormalizing everything & loosing all benefits of relational databases

21. Who faced this problem? Web applications dealing with high traffic, massive data, large user-base & user-generated content, such as: Google

22. Yahoo!

23. Amazon

24. Facebook

25. Twitter

26. Linked-In

27. & many more

28. 1 difference though Compared to traditional large applications (telco, financial, &c), these web applications are usually free & therefore: can sacrifice data integrity / consistency No one will sue them if he doesn't receive the most current: status of their friends (Facebook/Twitter)

29. Web search result (Google /Yahoo!)

30. Item added to cart (Amazon)

31. The solution These companies had to come up with a new kind of DBMS, capable of handling web scale Possibly sacrificing some level of consistency or some other feature

32. Must we sacrifice something? In 2000, Eric Brewer (co-founder of Inktomi) formulated the CAP theorem, claiming that you can only optimize 2 out of these 3: C onsistency

33. A vailability

34. P artition-tolerance BTW, the theorem was later proved by MIT scientists in 2002

35. Simple example When you have a lot of data which needs to be highly available, you'll usually need to p artition it across machines & also replicate it to be more fault-tolerant

36. This means, that when writing a record, all replica's must be updated too

37. Now you need to choose between: Lock all relevant replica's during update => be less a vailable