Scaling with Postgres

1,853 views

Published on

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,853
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
47
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Scaling with Postgres

  1. 1. Scaling with Postgres Robert Treat Percona Performance Conference 2009 Wednesday, April 22, 2009
  2. 2. Disclaimer Every “really big” site uses custom solutions. Don’t expect to get there using out of the box software. You don’t have to reinvent the wheel. But you may have to build the car. Wednesday, April 22, 2009
  3. 3. Who am I ? Wednesday, April 22, 2009
  4. 4. Who am I ? OmniTI TB+ databases top 100 websites Oracle/MySQL/Postgres Not necessarily in that order :-) Wednesday, April 22, 2009
  5. 5. Scaling Databases 101 Wednesday, April 22, 2009
  6. 6. Scaling Databases 101 Implementation Wednesday, April 22, 2009
  7. 7. Scaling Databases 101 Implementation Tuning Optimize Queries Optimize Schema Wednesday, April 22, 2009
  8. 8. Scaling Databases 101 Implementation Tuning Optimize Queries Optimize Schema Throw Hardware At It Caching Wednesday, April 22, 2009
  9. 9. Scaling Databases 101 Implementation Tuning Optimize Queries Optimize Schema Throw Hardware At It Caching Replication Wednesday, April 22, 2009
  10. 10. Scaling Databases 101 Implementation Tuning Optimize Queries Optimize Schema Throw Hardware At It Caching Replication Federation Wednesday, April 22, 2009
  11. 11. Implementation Normalization Foreign Keys Relational Modeling Wednesday, April 22, 2009
  12. 12. Implementation Normalization Foreign Keys Relational Modeling Mostly Irrelevant Wednesday, April 22, 2009
  13. 13. Implementation Wednesday, April 22, 2009
  14. 14. Implementation Monitoring nagios, munin, etc.. check_postgres Wednesday, April 22, 2009
  15. 15. Implementation Monitoring nagios, munin, etc.. check_postgres Trending cacti, noit watch your hardware! Wednesday, April 22, 2009
  16. 16. Tuning Wednesday, April 22, 2009
  17. 17. Tuning Get the Basics Right http://wiki.postgresql.org/wiki/ Tuning_Your_PostgreSQL_Server Wednesday, April 22, 2009
  18. 18. Tuning Get the Basics Right http://wiki.postgresql.org/wiki/ Tuning_Your_PostgreSQL_Server Someday Xlogs, Checkpoints Vacuum Strategy Costs and other Minutia Wednesday, April 22, 2009
  19. 19. Optimize Queries explain explain analyze explain analyze verbose Wednesday, April 22, 2009
  20. 20. Optimize Queries writing queries tuned to your databases planner Wednesday, April 22, 2009
  21. 21. Optimize Schema Wednesday, April 22, 2009
  22. 22. Optimize Schema indexing partial, functional fix data types add/remove tables de-normalize (maybe) Wednesday, April 22, 2009
  23. 23. Why is this important? Postgres is good at scaling vertically Flawed database schema hides the real problems Wednesday, April 22, 2009
  24. 24. Why is this unimportant? Optimizing is often a trade off of time vs. money It is not an ultimate solution Wednesday, April 22, 2009
  25. 25. Throw Hardware At It Find your bottlenecks Memory / RAM CPU I/O, DISKS usually the biggest problem Wednesday, April 22, 2009
  26. 26. Can you take me higher? 32 Cores 128 GB+ RAM Spindles out the wazoo tablespaces Wednesday, April 22, 2009
  27. 27. Caching Wednesday, April 22, 2009
  28. 28. Caching Internal materialized views Wednesday, April 22, 2009
  29. 29. Caching Internal materialized views External memcached, et al. static files Wednesday, April 22, 2009
  30. 30. Oh, the places you’ll go Nothing Fancy or Exciting Solid Engineering Understanding needs / limits Wednesday, April 22, 2009
  31. 31. Oh, the places you’ll go Nothing Fancy or Exciting Solid Engineering Understanding needs / limits Top 1000+ Website Wednesday, April 22, 2009
  32. 32. Replication Ignore the hype Triggers won’t destroy your site Scripting solves a lot of config issues Telling your DBA to do his !@#$ job solves config issues too Add-on is FUD for Modular Wednesday, April 22, 2009
  33. 33. Replication Truth in advertising Oracle does it better MySQL does it simpler Wednesday, April 22, 2009
  34. 34. Replication Wednesday, April 22, 2009
  35. 35. Replication Scaling Reads Master, many slaves Slony, Londiste, Bucardo, PGPoolII Wednesday, April 22, 2009
  36. 36. Replication Scaling Reads Master, many slaves Slony, Londiste, Bucardo, PGPoolII Scaling Writes Mostly impossible Bucardo, PGPoolII Wednesday, April 22, 2009
  37. 37. What is Federation? A process for eliminating performance and/or scaleability as a requirement for your database Wednesday, April 22, 2009
  38. 38. Federation Wednesday, April 22, 2009
  39. 39. Federation MySQL software has no federation advantage Wednesday, April 22, 2009
  40. 40. Federation MySQL software has no federation advantage MySQL community has large federation advantage Wednesday, April 22, 2009
  41. 41. Federation Things to consider Horizontal Partitioning Application Based Partitioning BASE golconde, gearman Wednesday, April 22, 2009
  42. 42. El Fin www.xzilla.net www.omniti.com www.postgresql.us Wednesday, April 22, 2009
  43. 43. Postgres BOF Tonight @ 7:30 PM Location @ Camino Real Wednesday, April 22, 2009

×