Your SlideShare is downloading. ×
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Check Please!
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Check Please!

1,307

Published on

Check Please! What your Postgres database wishes you would monitor. Presented at PGCon 2010.

Check Please! What your Postgres database wishes you would monitor. Presented at PGCon 2010.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,307
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Check Please! What your Postgres database wishes you would monitor / Presentation Friday, May 21, 2010
  • 2. Who am I? • Lead Database Operations at OmniTI • Database Consulting / Management • Postgres? • TB+ OLAP/DSS • multiple 1000+ tps OLTP • custom built, private label • long time user (6.5-9.x) • community member • major contributor Friday, May 21, 2010
  • 3. Check Yourself! • Basic Tuning Is Job #1 • shared buffers, effective cache size, checkpoints • Resources • http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server • http://www.slideshare.net/xzilla/the-essential-postgresqlconf-presentation Friday, May 21, 2010
  • 4. Before You Wreck Yourself! Friday, May 21, 2010
  • 5. Before You Wreck Yourself! • Monitoring • If a server crashes in the woods • Pain is a great motivator Friday, May 21, 2010
  • 6. Before You Wreck Yourself! Friday, May 21, 2010
  • 7. Before You Wreck Yourself! • Trending • Knowing what things look like when they’re good helps determine when things are bad • You can often tell where you’re going by looking at where you came from Friday, May 21, 2010
  • 8. Tools? • Tools cannot replace experience and discipline • But they can help you maintain that discipline • Popular tools • nagios / munin • cacti / mrtg • circonus.com / reconnoiter • { check_postgres } Friday, May 21, 2010
  • 9. Connections • Hard limit on allowed connections • Game Over • Large numbers of concurrent users • Internet facing systems - Paul RJ Muller, beached whale Friday, May 21, 2010
  • 10. Disk Space • Data, clog, xlogs, log files • Game Over • (planning) • Everybody - Squiggle, Overloaded? Friday, May 21, 2010
  • 11. WAL Files • pg_xlog directory, transaction logs • maintain database consistency • excessive disk space • excessive recovery time • heavy write transactions • pg start/stop backup (buggy systems) - Jazzmasterson, Workspace 3.0 - Noguchi File Friday, May 21, 2010
  • 12. Size Matters • more data == more disk space • only grow if you should • large tables, i/o issues • unbounded growth? • fast paced development • everyone else (eventually) - elmada, Size Matters Friday, May 21, 2010
  • 13. Bloating • mvcc leaves dead rows • unused space in tables, indexes • i/o issues • disk space (eventually) • heavy updates, data churn • untuned systems - Joe Alterio, burpalurpa Friday, May 21, 2010
  • 14. Transactions • every statement is in a transaction • select/insert/update/deletes • underlying effects • load spikes • OLTP Systems • Logging/Internet Facing Systems - laurieofindy , Walmart on Black Friday 2009 Friday, May 21, 2010
  • 15. All Stats • pg_stat tables • tables, indexes... scans, tuples • underlying effects • load spikes • OLTP Systems • Logging/Internet Facing Systems - Inju, Statistics for the Utterly Confused Friday, May 21, 2010
  • 16. Free Space Map • Tracks unused space • Keeps vacuum effective • Table / Index Bloat • Medium to Large Systems • High Update / Data Churn DB - SkyTruth, Deepwater Horizon Oil Spill - RADARSAT-2, May 8, 2010 Friday, May 21, 2010
  • 17. Autovacuum Max Freeze Age • Ensures all tables get vacuumed • Prevents XID wrap-around • Heavy I/O • Locking Issues • High TPS / OLTP • pg_dump - Joe Marinaro, Good Morning! Friday, May 21, 2010
  • 18. Long Running Queries • postgres can’t freeze query plans • pg_stat_activity • Can cause issues for vacuum • Uh, response time obligations • active data collection • developers write queries ;-) - _Tawcan, Spiral Out..keep going Friday, May 21, 2010
  • 19. Idle Transactions • BEGIN; zzz... • pg_stat_activity • Can cause issues for vacuum • Connections holding memory • pretty much everyone - psd, Canadian Cashpoints, Bah! Friday, May 21, 2010
  • 20. Sequence limits • sequences limited to 2 billion • non-transactional • Can break inserts • Heavy insert (update?) systems - gavinzac, Rise and Fall in Donegal Friday, May 21, 2010
  • 21. Wrap-around • Postgres must vacuum every table within 2 Billion transactions • Catastrophic data loss • pretty much everyone - Jurvetson, Wrapped Around the Axle Friday, May 21, 2010
  • 22. Settings • postgresql.conf doesn’t always reflect reality • temporary changes can lead to long term trouble • pretty much everyone - denovich, P9220453.jpg Friday, May 21, 2010
  • 23. Thanks! • PGCon • PGCommunity • OmniTI • Want more? • xzilla@users.sourceforge.net, http://www.xzilla.net • @robtreat2 • robert@omniti.com, http://www.omniti.com/is/hiring Friday, May 21, 2010

×