[Anti]Patterns with NoSQL         Luca Bonmassar
NoSQL vs. SQL            NOSQL DAY ’11
NoSQL / NoAgendaNo specific NoSQL technologies (document, key/value, ...)No specific NoSQL db features (sharding, replicatio...
Does it solve my problem?                    NOSQL DAY ’11
Does it solve my problem?                    NOSQL DAY ’11
Does it solve my problem?But what is your problem?                            NOSQL DAY ’11
Does it solve my problem?But what is your problem?NoSQL fragmentation (like NoCloud, NoJava,NoMicrosoft, ...)             ...
[Anti]PatternsThe “dynamic” websiteData mappingThe AlienThe binary AlienThe SQLQueueOvernormalizedUnpredictable tomorrow  ...
Who am I?Passionate abouttechnologyFavorite topics: Cloud,Virtualization, NoSQLGamification fan! (or fun?)                 ...
The “dynamic” website
The “dynamic” website                   NOSQL DAY ’11
The “dynamic” websiteI have a website                   NOSQL DAY ’11
The “dynamic” websiteI have a websiteDynamic contents everywhere                              NOSQL DAY ’11
The “dynamic” websiteI have a websiteDynamic contents everywhereHigh DB load                              NOSQL DAY ’11
The “dynamic” website                   NOSQL DAY ’11
The “dynamic” website                   NOSQL DAY ’11
The “dynamic” website               SELECT username                                                                       ...
The “dynamic” website               SELECT username                                                                       ...
The “dynamic” website                   NOSQL DAY ’11
The “dynamic” website               SELECT username                                                                       ...
The “dynamic” websiteIs it that dynamic?Are there any datastructures that fit betterthan relational?                       ...
Cache, Cache, CacheUse Ad-Hoc Caching (e.g. Rails Caching)SQL Table as cache                                          NOSQ...
SQL Table as CacheWhen you use SQL Table asCache, a kitten somewherediesUse Memcache,MemcacheDB, Redis                    ...
Better tools?                NOSQL DAY ’11
Rethink data structuresSELECT COUNT(*) ...SELECT COUNT(*) ...SELECT COUNT(*) ...submitted++passed++failed++               ...
Rethink data structuresSELECT *, count(distinct ...)FROM submissions JOINusers on ... wheresubmissions.current_state= "suc...
Data mapping
Data mappingAn original idea: we need a newsfeed!Users can comment on feed itemsUsers can reply to comments               ...
What the customer is looking for                         NOSQL DAY ’11
What the developer is thinking about                           NOSQL DAY ’11
But how to ...
Eureka!The item has a feed_id anda parent_idYou have now a navigableSQL data structure!                             NOSQL ...
Data mapping - alternativesDo not try this at home! (or any office)A document db can help    { feed : 42, user : ‘luca.bonm...
The Alien
The AlienA type of object stored in a relational databaseNo (or weak) relations with any other tableStored in the db becau...
User SettingsUser preferencesuser_id, dataData as BLOB or TEXT                       NOSQL DAY ’11
The AlienThe Alien is a key/value entityUse a key/value storage to store it                                      NOSQL DAY...
The binary Alien
The binary AlienLike the Alien, but the payload is pure binary dataCommon solution to store “small” imagesEven worse: Base...
The binary AlienMove the binary Alien to a binary content provider   S3 or Filesystem are good onesLet the webserver acces...
The SQLQueue
The SQLQueueDistributed components need to exchange data   Producers / Consumers   Backlog of work to be completedSQL data...
Mail delivery                NOSQL DAY ’11
Mail delivery                NOSQL DAY ’11
Mail delivery                NOSQL DAY ’11
Mail delivery                NOSQL DAY ’11
The SQLQueueNo “relations”, like the AlienSimulating a Queue using AUTO_INCREMENT ids andtransactions                     ...
SQLQueueIn some countries,SQLQueue is considered acrime against softwareUse message queues(Amazon SQS, MemcacheQ,StormMQ, ...
Overnormalized
Overnormalized                 NOSQL DAY ’11
OvernormalizedThe process of organizing data to minimize redundancy                                        NOSQL DAY ’11
OvernormalizedThe process of organizing data to minimize redundancyA larger schema is broken into smaller ones            ...
OvernormalizedThe process of organizing data to minimize redundancyA larger schema is broken into smaller ones   user_id, ...
OvernormalizedThe process of organizing data to minimize redundancyA larger schema is broken into smaller ones   user_id, ...
OvernormalizedThe process of organizing data to minimize redundancyA larger schema is broken into smaller ones   user_id, ...
OvernormalizedPRO:  reduce redundancy  less overhead  each table scale separately                                NOSQL DAY...
OvernormalizedCons:                 NOSQL DAY ’11
OvernormalizedCons:... JOIN... JOIN... JOIN... JOIN... JOIN... JOIN... JOIN...                                            ...
OvernormalizedCache the normalized dataDenormalize / keep a replica of the denormalized viewUse document db or key/value s...
Unpredictable tomorrow
Unpredictable tomorrowYou are now part of a new Agile(TM) projectYou are Agile(TM), so:   No complete specs   No complete ...
How many wonderful things around meA mobile App w/ internet backend“Simple” use cases   User can log in   User can update ...
How many wonderful things around meData Model:   User   PlacesAliens!!! Aliens!!! Aliens!!!                               ...
...but sooner or later...                    NOSQL DAY ’11
How many wonderful things around meUsers can send/receive friendship invitationsUsers can import FB friends, Twitter follo...
Unpredictable tomorrowNo silver bulletsMix technologies   E.g. relational databases to handle relationsMigrations are pain...
One more thing
RecapDoes it make sense with the relational paradigm?Do I need a persistent storage or a relational database?             ...
Thanks! Any Questions?                 NOSQL DAY ’11
Contactsluca@coderloop.comlinkedin.com/in/lucabonmassartwitter.com/lucabonmassarjoind.in/talk/view/2939                   ...
Upcoming SlideShare
Loading in …5
×

Patterns / Antipatterns with NoSQL

7,991 views
7,696 views

Published on

NoSQL vs. SQL: practical patterns (and antipatterns to avoid) in real projects. Talk at NoSQL Day 2011.

Published in: Technology
2 Comments
16 Likes
Statistics
Notes
No Downloads
Views
Total views
7,991
On SlideShare
0
From Embeds
0
Number of Embeds
39
Actions
Shares
0
Downloads
153
Comments
2
Likes
16
Embeds 0
No embeds

No notes for slide

Patterns / Antipatterns with NoSQL

  1. 1. [Anti]Patterns with NoSQL Luca Bonmassar
  2. 2. NoSQL vs. SQL NOSQL DAY ’11
  3. 3. NoSQL / NoAgendaNo specific NoSQL technologies (document, key/value, ...)No specific NoSQL db features (sharding, replication, ...)No specific ...... NOSQL DAY ’11
  4. 4. Does it solve my problem? NOSQL DAY ’11
  5. 5. Does it solve my problem? NOSQL DAY ’11
  6. 6. Does it solve my problem?But what is your problem? NOSQL DAY ’11
  7. 7. Does it solve my problem?But what is your problem?NoSQL fragmentation (like NoCloud, NoJava,NoMicrosoft, ...) NOSQL DAY ’11
  8. 8. [Anti]PatternsThe “dynamic” websiteData mappingThe AlienThe binary AlienThe SQLQueueOvernormalizedUnpredictable tomorrow NOSQL DAY ’11
  9. 9. Who am I?Passionate abouttechnologyFavorite topics: Cloud,Virtualization, NoSQLGamification fan! (or fun?) NOSQL DAY ’11
  10. 10. The “dynamic” website
  11. 11. The “dynamic” website NOSQL DAY ’11
  12. 12. The “dynamic” websiteI have a website NOSQL DAY ’11
  13. 13. The “dynamic” websiteI have a websiteDynamic contents everywhere NOSQL DAY ’11
  14. 14. The “dynamic” websiteI have a websiteDynamic contents everywhereHigh DB load NOSQL DAY ’11
  15. 15. The “dynamic” website NOSQL DAY ’11
  16. 16. The “dynamic” website NOSQL DAY ’11
  17. 17. The “dynamic” website SELECT username SELECT COUNT(###) WHERE ... FROM users WHERE sessionid = ### SELECT username FROM users WHERE sessionid = ###SELECT COUNT("submissions"."puzzle_id")FROM "submissions" WHERE (puzzle_id IS NOT NULL) AND (user_id IS NOT NULL) SELECT COUNT(###) WHERE ... SELECT COUNT(###) WHERE ... GROUP BY puzzle_id ... SELECT users.*, count(distinct puzzle_id) FROM submissions JOIN users on submissions.user_id SELECT * FROM submissions = users.id where submissions.current_state = WHERE ... SELECT avatar.* FROM users "succeeded" ... WHERE ... NOSQL DAY ’11
  18. 18. The “dynamic” website SELECT username SELECT COUNT(###) WHERE ... FROM users WHERE sessionid = ### SELECT username FROM users WHERE sessionid = ###SELECT COUNT("submissions"."puzzle_id")FROM "submissions" WHERE (puzzle_id IS NOT NULL) AND (user_id IS NOT NULL) SELECT COUNT(###) WHERE ... SELECT COUNT(###) WHERE ... GROUP BY puzzle_id ... SELECT users.*, count(distinct puzzle_id) FROM submissions JOIN users on submissions.user_id SELECT * FROM submissions = users.id where submissions.current_state = WHERE ... SELECT avatar.* FROM users "succeeded" ... WHERE ... NOSQL DAY ’11
  19. 19. The “dynamic” website NOSQL DAY ’11
  20. 20. The “dynamic” website SELECT username SELECT COUNT(###) WHERE ... FROM users WHERE sessionid = ### SELECT username FROM users WHERE sessionid = ###SELECT COUNT("submissions"."puzzle_id")FROM "submissions" WHERE (puzzle_id IS NOT NULL) AND (user_id IS NOT NULL) SELECT COUNT(###) WHERE ... SELECT COUNT(###) WHERE ... GROUP BY puzzle_id ... SELECT users.*, count(distinct puzzle_id) FROM submissions JOIN users on submissions.user_id SELECT * FROM submissions = users.id where submissions.current_state = WHERE ... SELECT avatar.* FROM users "succeeded" ... WHERE ... NOSQL DAY ’11
  21. 21. The “dynamic” websiteIs it that dynamic?Are there any datastructures that fit betterthan relational? NOSQL DAY ’11
  22. 22. Cache, Cache, CacheUse Ad-Hoc Caching (e.g. Rails Caching)SQL Table as cache NOSQL DAY ’11
  23. 23. SQL Table as CacheWhen you use SQL Table asCache, a kitten somewherediesUse Memcache,MemcacheDB, Redis NOSQL DAY ’11
  24. 24. Better tools? NOSQL DAY ’11
  25. 25. Rethink data structuresSELECT COUNT(*) ...SELECT COUNT(*) ...SELECT COUNT(*) ...submitted++passed++failed++ NOSQL DAY ’11
  26. 26. Rethink data structuresSELECT *, count(distinct ...)FROM submissions JOINusers on ... wheresubmissions.current_state= "succeeded" ...List(5)List.add / List.replace NOSQL DAY ’11
  27. 27. Data mapping
  28. 28. Data mappingAn original idea: we need a newsfeed!Users can comment on feed itemsUsers can reply to comments NOSQL DAY ’11
  29. 29. What the customer is looking for NOSQL DAY ’11
  30. 30. What the developer is thinking about NOSQL DAY ’11
  31. 31. But how to ...
  32. 32. Eureka!The item has a feed_id anda parent_idYou have now a navigableSQL data structure! NOSQL DAY ’11
  33. 33. Data mapping - alternativesDo not try this at home! (or any office)A document db can help { feed : 42, user : ‘luca.bonmassar’, data : ‘living the dream!’, comments : [{user : ‘jonny’ : ‘make sense!’ }] } NOSQL DAY ’11
  34. 34. The Alien
  35. 35. The AlienA type of object stored in a relational databaseNo (or weak) relations with any other tableStored in the db because the db == persistency NOSQL DAY ’11
  36. 36. User SettingsUser preferencesuser_id, dataData as BLOB or TEXT NOSQL DAY ’11
  37. 37. The AlienThe Alien is a key/value entityUse a key/value storage to store it NOSQL DAY ’11
  38. 38. The binary Alien
  39. 39. The binary AlienLike the Alien, but the payload is pure binary dataCommon solution to store “small” imagesEven worse: Base64 encoded binary NOSQL DAY ’11
  40. 40. The binary AlienMove the binary Alien to a binary content provider S3 or Filesystem are good onesLet the webserver access/serve them directly NOSQL DAY ’11
  41. 41. The SQLQueue
  42. 42. The SQLQueueDistributed components need to exchange data Producers / Consumers Backlog of work to be completedSQL database (== persistency) as persistent queue NOSQL DAY ’11
  43. 43. Mail delivery NOSQL DAY ’11
  44. 44. Mail delivery NOSQL DAY ’11
  45. 45. Mail delivery NOSQL DAY ’11
  46. 46. Mail delivery NOSQL DAY ’11
  47. 47. The SQLQueueNo “relations”, like the AlienSimulating a Queue using AUTO_INCREMENT ids andtransactions NOSQL DAY ’11
  48. 48. SQLQueueIn some countries,SQLQueue is considered acrime against softwareUse message queues(Amazon SQS, MemcacheQ,StormMQ, RabbitMQ, ... ) NOSQL DAY ’11
  49. 49. Overnormalized
  50. 50. Overnormalized NOSQL DAY ’11
  51. 51. OvernormalizedThe process of organizing data to minimize redundancy NOSQL DAY ’11
  52. 52. OvernormalizedThe process of organizing data to minimize redundancyA larger schema is broken into smaller ones NOSQL DAY ’11
  53. 53. OvernormalizedThe process of organizing data to minimize redundancyA larger schema is broken into smaller ones user_id, email NOSQL DAY ’11
  54. 54. OvernormalizedThe process of organizing data to minimize redundancyA larger schema is broken into smaller ones user_id, email user_id, phone_num NOSQL DAY ’11
  55. 55. OvernormalizedThe process of organizing data to minimize redundancyA larger schema is broken into smaller ones user_id, email user_id, phone_num user_id, badge NOSQL DAY ’11
  56. 56. OvernormalizedPRO: reduce redundancy less overhead each table scale separately NOSQL DAY ’11
  57. 57. OvernormalizedCons: NOSQL DAY ’11
  58. 58. OvernormalizedCons:... JOIN... JOIN... JOIN... JOIN... JOIN... JOIN... JOIN... NOSQL DAY ’11
  59. 59. OvernormalizedCache the normalized dataDenormalize / keep a replica of the denormalized viewUse document db or key/value storage for the replica NOSQL DAY ’11
  60. 60. Unpredictable tomorrow
  61. 61. Unpredictable tomorrowYou are now part of a new Agile(TM) projectYou are Agile(TM), so: No complete specs No complete use cases NOSQL DAY ’11
  62. 62. How many wonderful things around meA mobile App w/ internet backend“Simple” use cases User can log in User can update their location User can get all the many wonderful things they have around themselves NOSQL DAY ’11
  63. 63. How many wonderful things around meData Model: User PlacesAliens!!! Aliens!!! Aliens!!! NOSQL DAY ’11
  64. 64. ...but sooner or later... NOSQL DAY ’11
  65. 65. How many wonderful things around meUsers can send/receive friendship invitationsUsers can import FB friends, Twitter followersUsers can interact with messages, pokesUsers can check-in into placesUsers can share their checkins with friends... NOSQL DAY ’11
  66. 66. Unpredictable tomorrowNo silver bulletsMix technologies E.g. relational databases to handle relationsMigrations are painful, but always an option NOSQL DAY ’11
  67. 67. One more thing
  68. 68. RecapDoes it make sense with the relational paradigm?Do I need a persistent storage or a relational database? NOSQL DAY ’11
  69. 69. Thanks! Any Questions? NOSQL DAY ’11
  70. 70. Contactsluca@coderloop.comlinkedin.com/in/lucabonmassartwitter.com/lucabonmassarjoind.in/talk/view/2939 NOSQL DAY ’11

×