Database Scalability Patterns

2,527 views
2,403 views

Published on

This is a copy of the slides for my talk "Database Scalability Patterns", given in March at PGEast 2010.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,527
On SlideShare
0
From Embeds
0
Number of Embeds
479
Actions
Shares
0
Downloads
0
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Database Scalability Patterns

  1. 1. Database Scalability {Patterns} / Robert Treat Thursday, April 8, 2010
  2. 2. robert treat omniti postgres oracle - mysql mssql - sqlite -nosql Thursday, April 8, 2010
  3. 3. What are “Database Scalability Patterns?” Thursday, April 8, 2010
  4. 4. Part Design Patterns Part Application Life-Cycle Thursday, April 8, 2010
  5. 5. MyFirstDatabase Thursday, April 8, 2010
  6. 6. Vertical Partitioning Thursday, April 8, 2010
  7. 7. Vertical Scaling Thursday, April 8, 2010
  8. 8. Vertical Scaling Thursday, April 8, 2010
  9. 9. Federated Data Storage “sharding” Horizontal Partitioning Read Slaves Multi-Master Horizontal Scaling Thursday, April 8, 2010
  10. 10. Read Slaves / Master - Slave Scale Read Load Thursday, April 8, 2010
  11. 11. Read Slaves / Master - Slave Scale Read Load • Typically • Full Copy of Data On Each Node • Asynchronous Thursday, April 8, 2010
  12. 12. Read Slaves / Master - Slave Scale Read Load • Typically • Consider • Full Copy of Data • Partial Copy On Each Node • Synchronous • Asynchronous • Don’t use a RDBMS? Thursday, April 8, 2010
  13. 13. Read Slaves / Master - Slave Scale Read Load • Typically • Consider • Full Copy of Data • Partial Copy On Each Node • Synchronous • Asynchronous • Don’t use a RDBMS? Requires Application Changes Thursday, April 8, 2010
  14. 14. Read Slaves / Master - Slave Scale Read Load • Typically • Consider • Full Copy of Data • Partial Copy On Each Node • Synchronous • Asynchronous • Don’t use a RDBMS? Requires Application Changes “easy” Thursday, April 8, 2010
  15. 15. Scaling Writes “not easy” Thursday, April 8, 2010
  16. 16. Multi-Master many different ways to implement this, few that actually work in production Thursday, April 8, 2010
  17. 17. Multi-Master many different ways to implement this, few that actually work in production write to any node, database syncs data Thursday, April 8, 2010
  18. 18. Multi-Master many different ways to implement this, few that actually work in production write to any node, database syncs data can reduce cpu, doesn’t reduce i/o Thursday, April 8, 2010
  19. 19. Multi-Master many different ways to implement this, few that actually work in production write to any node, database syncs data can reduce cpu, doesn’t reduce i/o failover solution not a scalability solution Thursday, April 8, 2010
  20. 20. Horizontal Partitioning • Divide schema by job operations • Move each piece to own server • Duplicate some data as needed Thursday, April 8, 2010
  21. 21. Horizontal Partitioning • Divide schema by job operations • Move each piece to own server • Duplicate some data as needed • You must separate dependencies in the app code first! Thursday, April 8, 2010
  22. 22. Horizontal Partitioning • Divide schema by job operations • Move each piece to own server • Duplicate some data as needed • You must separate dependencies in the app code first! Each node is a new instance of vertical scaling Thursday, April 8, 2010
  23. 23. Horizontal Scaling • data split across servers based on algorithm • data dropped into buckets (multiple?) • someone must keep track of data, and provide lookup services Thursday, April 8, 2010
  24. 24. Universal Truths of Scaling Databases Vertical Scalability is Helpful for Every Pattern Even in a horizontally scaled, fully distributed database, the number of nodes needed is affected by vertical scalability Thursday, April 8, 2010
  25. 25. Universal Truths of Scaling Databases New Nodes Are Never Free • Add points of failure • Add management costs • Add complexity to architecture • Add complexity to your app code Thursday, April 8, 2010
  26. 26. MyFirstDB V Vertical Partitioning V Vertical Scaling V Read Slaves V Horizontal Partitioning Thursday, April 8, 2010
  27. 27. plan for layered data sources read / write connections in code use schemas to separate services Thursday, April 8, 2010
  28. 28. plan for layered data sources read / write connections in code use schemas to separate services use postgres :-) Thursday, April 8, 2010
  29. 29. THANKS! more: xzilla.net @robtreat2 omniti.com Thursday, April 8, 2010

×