Developing PostgreSQL Performance, Simon Riggs

723 views

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
723
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Developing PostgreSQL Performance, Simon Riggs

  1. 1. © 2ndQuadrant Limited 2010 Developing PostgreSQL Performance
  2. 2. © 2ndQuadrant Limited 2010 Performance Comparisons
  3. 3. © 2ndQuadrant Limited 2010 PostgreSQL Performance Gain: Single Node Improvements 7.3.x (est) 7.4.x (est) 8.0.21 8.1.17 8.2.13 8.3.7 8.4.1 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% 120.00% Peak RO TPS Peak RW TPS
  4. 4. © 2ndQuadrant Limited 2010 What this talk is about • History of performance analysis & performance development • Lessons learned • Where next?
  5. 5. © 2ndQuadrant Limited 2010 Who Am I? • Simon Riggs • Major Developer on PostgreSQL project • CTO, 2ndQuadrant • Database Architect at Abbey National Bank Technical Advisor RDBMS Procurement
  6. 6. © 2ndQuadrant Limited 2010 Why performance matters • User Experience (Response time) • Headroom for growth (Selection Risk) • Total Cost of Ownership (Cost profile)
  7. 7. © 2ndQuadrant Limited 2010 Dramatis Personae • Simon Riggs • Mark Wong • Jonah Harris • Pavan Deolassee • Manfred Koizar • Tom Lane • Greg Smith • Heikki Linnakangas • Greg Stark • Jan Wieck
  8. 8. © 2ndQuadrant Limited 2010 Performance Analysis 7.3.x (est) 7.4.x (est) 8.0.21 8.1.17 8.2.13 8.3.7 8.4.1 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% 120.00% Peak RO TPS Peak RW TPS Laptop Server-based No re-analysis!
  9. 9. © 2ndQuadrant Limited 2010 Lessons • Need proper workload analysis • Laptop-only analysis is never good enough • Regular re-analysis is needed for each release
  10. 10. © 2ndQuadrant Limited 2010 Target Workloads • Various workloads, all different – High Volume Transactions • Read Only • Read Write – Data Warehouse • Very Large Query • Data Loading • Different approaches required • Different development areas
  11. 11. © 2ndQuadrant Limited 2010 Data Warehouse • Optimizer Improvements (all) • Partitioning (8.1, 9.1) • Sort Improvements (8.2-8.3) • WAL avoidance (8.3-9.0) • Hash Join skew avoidance (8.4)
  12. 12. © 2ndQuadrant Limited 2010 Real transactional benchmark! • Unisys sponsored benchmarks on 32-way • No way to analyse SQL execution time • EXPLAIN won't work on parameterised statements • Bgwriter gave negative performance benefit • Context switch storms • Massively untuned!
  13. 13. © 2ndQuadrant Limited 2010 Breakthroughs • Full stack analysis • Event targeting • Scalability Analysis
  14. 14. © 2ndQuadrant Limited 2010 Full stack analysis • Algorithms • Libraries/Modules • Compiler Optimisation • OS Layer Optimisation • Physical Layer • Hardware
  15. 15. © 2ndQuadrant Limited 2010 Event targeting • Reject the smoothness assumption: performance is not lost by constant friction • Events – Traffic Jams – Phase changes – Black Swans • Don't look at the totals and averages • Look at the behaviour
  16. 16. © 2ndQuadrant Limited 2010 Scalability Analysis • Resource sharing rules • Fine-grained Locking • Lock avoidance • Minimise • Partition • Dependency Removal
  17. 17. © 2ndQuadrant Limited 2010 Developments Buffer Alignment (7.4) Buffer Management (8.0) Buffer Manager (8.1) Lock Manager (8.2) Cache Line optimisation (8.2) Snapshot Management (8.3) Buffer recycling (8.3)
  18. 18. © 2ndQuadrant Limited 2010 Results
  19. 19. © 2ndQuadrant Limited 2010 Conclusions To conclude the above, I think it's safe to say that if we have known PostgreSQL as a slow beast, it's time to re-think (or measure) that, because it has gained quite much performance and scalability in the last three years. Not to speak about its features. György Vilmos http://suckit.blog.hu/2009/09/29/postgresql_history
  20. 20. © 2ndQuadrant Limited 2010 Where next? • Transactional Workloads – Smoothness (+10-20%) – Indexing (+10-20%) • Data Warehouse – Indexing – Partitioning – Many available techniques
  21. 21. © 2ndQuadrant Limited 2010 Performance Future 7.3.x (est) 7.4.x (est) 8.0.21 8.1.17 8.2.13 8.3.7 8.4.1 0.00% 20.00% 40.00% 60.00% 80.00% 100.00% 120.00% Peak RO TPS Peak RW TPS
  22. 22. © 2ndQuadrant Limited 2010 Radical Steps • Architecture is Critical • New options in the DBMS • Mixed mode • Plus, changes for multi-node scalability – Otherwise ignored during this talk
  23. 23. © 2ndQuadrant Limited 2010 PostgreSQL Books • http://www.2ndQuadrant.com/books/

×