This document discusses the importance of monitoring a Postgres database to prevent issues and ensure high performance. It provides examples of key things to monitor such as connections, disk space, transactions, queries, and settings. Regular monitoring helps identify problems early before they seriously impact the database or lead to data loss or downtime. Tools can help support monitoring but experience is also important.

1. Check Please!
What your Postgres database wishes you would monitor
/ Presentation
Friday, May 21, 2010

2. Who am I?
• Lead Database Operations at OmniTI
• Database Consulting / Management
• Postgres?
• TB+ OLAP/DSS
• multiple 1000+ tps OLTP
• custom built, private label
• long time user (6.5-9.x)
• community member
• major contributor
Friday, May 21, 2010

3. Check Yourself!
• Basic Tuning Is Job #1
• shared buffers, effective cache size, checkpoints
• Resources
• http://wiki.postgresql.org/wiki/Tuning_Your_PostgreSQL_Server
• http://www.slideshare.net/xzilla/the-essential-postgresqlconf-presentation
Friday, May 21, 2010

4. Before You Wreck Yourself!
Friday, May 21, 2010

5. Before You Wreck Yourself!
• Monitoring
• If a server crashes in the woods
• Pain is a great motivator
Friday, May 21, 2010

6. Before You Wreck Yourself!
Friday, May 21, 2010

7. Before You Wreck Yourself!
• Trending
• Knowing what things look like when they’re good helps determine
when things are bad
• You can often tell where you’re going by looking at where you came
from
Friday, May 21, 2010

8. Tools?
• Tools cannot replace experience and discipline
• But they can help you maintain that discipline
• Popular tools
• nagios / munin
• cacti / mrtg
• circonus.com / reconnoiter
• { check_postgres }
Friday, May 21, 2010

9. Connections
• Hard limit on allowed connections
• Game Over
• Large numbers of concurrent
users
• Internet facing systems
- Paul RJ Muller, beached whale
Friday, May 21, 2010

10. Disk Space
• Data, clog, xlogs, log files
• Game Over
• (planning)
• Everybody
- Squiggle, Overloaded?
Friday, May 21, 2010

11. WAL Files
• pg_xlog directory, transaction logs
• maintain database consistency
• excessive disk space
• excessive recovery time
• heavy write transactions
• pg start/stop backup (buggy systems)
- Jazzmasterson, Workspace 3.0 - Noguchi File
Friday, May 21, 2010

12. Size Matters
• more data == more disk space
• only grow if you should
• large tables, i/o issues
• unbounded growth?
• fast paced development
• everyone else (eventually)
- elmada, Size Matters
Friday, May 21, 2010

13. Bloating
• mvcc leaves dead rows
• unused space in tables, indexes
• i/o issues
• disk space (eventually)
• heavy updates, data churn
• untuned systems
- Joe Alterio, burpalurpa
Friday, May 21, 2010

14. Transactions
• every statement is in a transaction
• select/insert/update/deletes
• underlying effects
• load spikes
• OLTP Systems
• Logging/Internet Facing Systems
- laurieofindy , Walmart on Black Friday 2009
Friday, May 21, 2010

15. All Stats
• pg_stat tables
• tables, indexes... scans, tuples
• underlying effects
• load spikes
• OLTP Systems
• Logging/Internet Facing Systems
- Inju, Statistics for the Utterly Confused
Friday, May 21, 2010

16. Free Space Map
• Tracks unused space
• Keeps vacuum effective
• Table / Index Bloat
• Medium to Large Systems
• High Update / Data Churn DB
- SkyTruth, Deepwater Horizon Oil Spill - RADARSAT-2, May 8, 2010
Friday, May 21, 2010

17. Autovacuum Max Freeze Age
• Ensures all tables get vacuumed
• Prevents XID wrap-around
• Heavy I/O
• Locking Issues
• High TPS / OLTP
• pg_dump
- Joe Marinaro, Good Morning!
Friday, May 21, 2010

18. Long Running Queries
• postgres can’t freeze query plans
• pg_stat_activity
• Can cause issues for vacuum
• Uh, response time obligations
• active data collection
• developers write queries ;-)
- _Tawcan, Spiral Out..keep going
Friday, May 21, 2010

19. Idle Transactions
• BEGIN; zzz...
• pg_stat_activity
• Can cause issues for vacuum
• Connections holding memory
• pretty much everyone
- psd, Canadian Cashpoints, Bah!
Friday, May 21, 2010

20. Sequence limits
• sequences limited to 2 billion
• non-transactional
• Can break inserts
• Heavy insert (update?) systems
- gavinzac, Rise and Fall in Donegal
Friday, May 21, 2010

21. Wrap-around
• Postgres must vacuum every table
within 2 Billion transactions
• Catastrophic data loss
• pretty much everyone
- Jurvetson, Wrapped Around the Axle
Friday, May 21, 2010

22. Settings
• postgresql.conf doesn’t always
reflect reality
• temporary changes can lead
to long term trouble
• pretty much everyone
- denovich, P9220453.jpg
Friday, May 21, 2010

23. Thanks!
• PGCon
• PGCommunity
• OmniTI
• Want more?
• xzilla@users.sourceforge.net, http://www.xzilla.net
• @robtreat2
• robert@omniti.com, http://www.omniti.com/is/hiring
Friday, May 21, 2010