Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Polyglot Persistence - Confoo 2016

9,998 views

Published on

NoSQL databases are every where and they are here to stay. For a successful scalable web application it is vital to understand capabilities of NoSQL databases.

Also relational databases are not dead and they will not be. Understanding abilities of each, would help making better architectural decisions.

Hybrid database solutions could use bests of the both worlds. We would go in to the important decision making factors for each.

Published in: Internet
  • Follow the link, new dating source: ❤❤❤ http://bit.ly/2u6xbL5 ❤❤❤
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Dating direct: ❶❶❶ http://bit.ly/2u6xbL5 ❶❶❶
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Polyglot Persistence - Confoo 2016

  1. 1. Polyglot Persistence
  2. 2. Majid Fatemian R E D V E N T U R E S @majidfn
  3. 3. Relational Non-Relational Key-Value Big Data
  4. 4. Application Relational
  5. 5. Application Relational Relational Non-Relational
  6. 6. Relational
  7. 7. BIG DATA
  8. 8. Capture Search Storage Q u e r y Privacy Visualize Analysis
  9. 9. Non-Relational
  10. 10. 1 2 3 Non-Relational Cluster Schemaless
  11. 11. 2 x 1 x 1 x
  12. 12. 2011 2012 2013 2014
  13. 13. 2011 2012 2013 2014
  14. 14. Application
  15. 15. ACID Normalized {Consistency Relational
  16. 16. Player ID Map ID Difficulty Score 1000 223 E 5 2000 673 H 4.5 1000 451 M 4 2000 980 H 3 Activity Player ID Name Location 1000 John US 2000 Catherine CA 3000 Paul CA 4000 Kylee US Profile
  17. 17. Replication Fault Tolerance Availability {Relational
  18. 18. OPs / DBAs Tools Devs{DevOps Relational
  19. 19. 2011 2012 2013 2014
  20. 20. Scalability Relational
  21. 21. Distribution Relational Data C Data B Data A Data Access Layer
  22. 22. Schema Relational {
 player_id: 1234,
 * level: 40,
 * duration: 75, * activity: activity,
 game_id: 4001,
 platform: WiiU, difficulty_level: medium,
 repetition: 178,
 score: 450,
 endurance:3
 } {
 player_id: 1234,
 level: 12,
 duration: 120, activity: activity
 }
  23. 23. Total Detailed stats stats Players Profiles Individual Activities Aggregated stats Categorized
  24. 24. ACID Transaction Partitioning Replication Tools OPs Data Volume Fixed Schema Scale Up Manual Sharding + -
  25. 25. NoSQL ...
  26. 26. mongoDB 1 2 3 OPs Devs Community
  27. 27. Schema-less Non-Relational player_id: 1234,
 * level: 40,
 * duration: 75, * activity: activity, game_id: 4001,
 platform: WiiU, difficulty_level: 2,
 repetition: 178,
 score: 450,
 endurance:3 player_id: 1234,
 level: 75,
 duration: 120, activity: activity

  28. 28. Scalability Non-Relational
  29. 29. Scalability Non-Relational +
  30. 30. Sharding Non-Relational Data A {key}
  31. 31. Sharding Non-Relational Data A A Data B B Data C C {key}
  32. 32. Map/Reduce Non-Relational Query K, V K,V1 K,V2 K,V3
  33. 33. Map/Reduce Non-Relational Map K,V1 K,V2 K,V3
  34. 34. Map/Reduce Non-Relational Result ReduceMap K,V1 K,V2 K,V3
  35. 35. Denormalized Non-Relationalread / }write Disk Space
  36. 36. Player ID Map ID Difficulty Score 1000 223 E 5 2000 673 H 4.5 1000 451 M 4 2000 980 H 3 Activity Player ID Name Location 1000 John US 2000 Catherine CA 3000 Paul CA 4000 Kylee US Profile
  37. 37. {Name: John, game:1, activity: Mission01, Score: 1245} {Name: John, game:3, activity: Mission04, score: 5431} {Name: John, game:1, activity: Mission01, Score: 1245}
  38. 38. Data Model Non-RelationalQuery Aggregation }
  39. 39. Eventual- Consistency Non-Relational X2 W X1 X1 X2
  40. 40. Eventual- Consistency Non-Relational X2 W X1 X1 R X1 X2
  41. 41. Eventual- Consistency Non-Relational X2 W X2 X2 R X2 X2
  42. 42. Balancer Non-Relational
  43. 43. Balancer Non-Relational +
  44. 44. Balancer Non-Relational
  45. 45. Non-Relational Resilience A B C
  46. 46. Non-Relational Resilience A C
  47. 47. Locking Non-RelationalDB Level (2.2) }Document Level (3.x)
  48. 48. Backup Non-Relational Point-in-time
  49. 49. Denormalized data Disk Space Expertise Complex Querying
 Eventual Consistency Resource Usage DB-Level Locking - Schema-Less Aggregated Data Large Scale Data Sharding Map/Reduce Memory Storage Journaling +
  50. 50. Random Profile
 Search
 (1M) Full History Retrieval
 (6M) Insert
 (6M) Aggregation
 Map/Reduce
 (6M) 0 0.003 0.005 0.008 0.01 MySQL MongoDB
  51. 51. Persistence Polyglot
  52. 52. Players Profile Leaderboards Activity Details Relational Data Schema-less Sharding
  53. 53. Total Sum Activity 1 Activity n Activity m ……
  54. 54. Shard1 Shard2 Config
  55. 55. Data Access Layer Application
  56. 56. The Challenge
  57. 57. Inconsistency1
  58. 58. Data Access Layer W Pending Commit 2-Phase
  59. 59. Data Access Layer Pending W Commit 2-Phase
  60. 60. Data Access Layer Done Commit 2-Phase
  61. 61. 2 Latency vs. Consistency
  62. 62. Operations3 Exponential complexity
  63. 63. Availability4 Plan for failure
  64. 64. Data Access Layer
  65. 65. Data Access Layer
  66. 66. Data Access Layer
  67. 67. Data Integrity5 Backward compatibility
  68. 68. Unit {TEST Integration Functional
  69. 69. Data Migration6
  70. 70. Profile History } { profile: { GUID:…, platform: { name:’xbox’, xuid:… } }, activity:…, duration:…, datetime:… } 3rd Party
  71. 71. 1 One-Shot Migration 4 hours !! Live !! 2 Profile versioning
  72. 72. V +1 1
  73. 73. V +1 1
  74. 74. V +1 2
  75. 75. Final Results
  76. 76. Scalability 2011 2012 2013 2014
  77. 77. 9x response time
  78. 78. 3 years uptime
  79. 79. Go Hybrid! Thoughtfully >
  80. 80. Thank YOU! @majidfn
  81. 81. References Polyglot Persistence NoSQL Distilled

×