3. Table of Contents
Preface ................................................................................................................................................lx
1. What is PostgreSQL?.............................................................................................................lx
2. A Brief History of PostgreSQL.............................................................................................lxi
2.1. The Berkeley POSTGRES Project ...........................................................................lxi
2.2. Postgres95.................................................................................................................lxi
2.3. PostgreSQL..............................................................................................................lxii
3. Conventions..........................................................................................................................lxii
4. Further Information.............................................................................................................lxiii
5. Bug Reporting Guidelines...................................................................................................lxiii
5.1. Identifying Bugs.....................................................................................................lxiv
5.2. What to Report .......................................................................................................lxiv
5.3. Where to Report Bugs ............................................................................................lxvi
I. Tutorial.............................................................................................................................................1
1. Getting Started ........................................................................................................................1
1.1. Installation..................................................................................................................1
1.2. Architectural Fundamentals........................................................................................1
1.3. Creating a Database....................................................................................................2
1.4. Accessing a Database .................................................................................................3
2. The SQL Language .................................................................................................................5
2.1. Introduction ................................................................................................................5
2.2. Concepts .....................................................................................................................5
2.3. Creating a New Table .................................................................................................5
2.4. Populating a Table With Rows ...................................................................................6
2.5. Querying a Table ........................................................................................................7
2.6. Joins Between Tables..................................................................................................9
2.7. Aggregate Functions.................................................................................................11
2.8. Updates.....................................................................................................................12
2.9. Deletions...................................................................................................................13
3. Advanced Features................................................................................................................14
3.1. Introduction ..............................................................................................................14
3.2. Views ........................................................................................................................14
3.3. Foreign Keys.............................................................................................................14
3.4. Transactions..............................................................................................................15
3.5. Window Functions....................................................................................................17
3.6. Inheritance ................................................................................................................20
3.7. Conclusion................................................................................................................21
II. The SQL Language......................................................................................................................22
4. SQL Syntax...........................................................................................................................24
4.1. Lexical Structure.......................................................................................................24
4.1.1. Identifiers and Key Words............................................................................24
4.1.2. Constants......................................................................................................26
4.1.2.1. String Constants ..............................................................................26
4.1.2.2. String Constants with C-style Escapes............................................26
4.1.2.3. String Constants with Unicode Escapes..........................................28
4.1.2.4. Dollar-quoted String Constants.......................................................28
4.1.2.5. Bit-string Constants.........................................................................29
4.1.2.6. Numeric Constants ..........................................................................29
4.1.2.7. Constants of Other Types ................................................................30
iii
4. 4.1.3. Operators......................................................................................................31
4.1.4. Special Characters........................................................................................31
4.1.5. Comments ....................................................................................................32
4.1.6. Operator Precedence....................................................................................32
4.2. Value Expressions.....................................................................................................33
4.2.1. Column References......................................................................................34
4.2.2. Positional Parameters...................................................................................34
4.2.3. Subscripts.....................................................................................................35
4.2.4. Field Selection .............................................................................................35
4.2.5. Operator Invocations....................................................................................36
4.2.6. Function Calls..............................................................................................36
4.2.7. Aggregate Expressions.................................................................................36
4.2.8. Window Function Calls................................................................................38
4.2.9. Type Casts....................................................................................................39
4.2.10. Collation Expressions ................................................................................40
4.2.11. Scalar Subqueries.......................................................................................41
4.2.12. Array Constructors.....................................................................................41
4.2.13. Row Constructors.......................................................................................43
4.2.14. Expression Evaluation Rules .....................................................................44
4.3. Calling Functions......................................................................................................45
4.3.1. Using Positional Notation............................................................................45
4.3.2. Using Named Notation ................................................................................46
4.3.3. Using Mixed Notation..................................................................................46
5. Data Definition......................................................................................................................48
5.1. Table Basics..............................................................................................................48
5.2. Default Values ..........................................................................................................49
5.3. Constraints................................................................................................................50
5.3.1. Check Constraints........................................................................................50
5.3.2. Not-Null Constraints....................................................................................52
5.3.3. Unique Constraints.......................................................................................53
5.3.4. Primary Keys................................................................................................53
5.3.5. Foreign Keys................................................................................................54
5.3.6. Exclusion Constraints ..................................................................................57
5.4. System Columns.......................................................................................................57
5.5. Modifying Tables......................................................................................................59
5.5.1. Adding a Column.........................................................................................59
5.5.2. Removing a Column ....................................................................................59
5.5.3. Adding a Constraint.....................................................................................60
5.5.4. Removing a Constraint ................................................................................60
5.5.5. Changing a Column’s Default Value............................................................60
5.5.6. Changing a Column’s Data Type .................................................................61
5.5.7. Renaming a Column ....................................................................................61
5.5.8. Renaming a Table ........................................................................................61
5.6. Privileges ..................................................................................................................61
5.7. Schemas....................................................................................................................62
5.7.1. Creating a Schema .......................................................................................63
5.7.2. The Public Schema ......................................................................................64
5.7.3. The Schema Search Path..............................................................................64
5.7.4. Schemas and Privileges................................................................................65
5.7.5. The System Catalog Schema .......................................................................65
5.7.6. Usage Patterns..............................................................................................66
5.7.7. Portability.....................................................................................................66
iv
5. 5.8. Inheritance ................................................................................................................67
5.8.1. Caveats.........................................................................................................69
5.9. Partitioning ...............................................................................................................70
5.9.1. Overview......................................................................................................70
5.9.2. Implementing Partitioning ...........................................................................71
5.9.3. Managing Partitions.....................................................................................74
5.9.4. Partitioning and Constraint Exclusion .........................................................74
5.9.5. Alternative Partitioning Methods.................................................................76
5.9.6. Caveats.........................................................................................................76
5.10. Foreign Data ...........................................................................................................77
5.11. Other Database Objects ..........................................................................................78
5.12. Dependency Tracking.............................................................................................78
6. Data Manipulation.................................................................................................................80
6.1. Inserting Data ...........................................................................................................80
6.2. Updating Data...........................................................................................................81
6.3. Deleting Data............................................................................................................82
7. Queries ..................................................................................................................................83
7.1. Overview ..................................................................................................................83
7.2. Table Expressions.....................................................................................................83
7.2.1. The FROM Clause..........................................................................................84
7.2.1.1. Joined Tables ...................................................................................84
7.2.1.2. Table and Column Aliases...............................................................87
7.2.1.3. Subqueries .......................................................................................88
7.2.1.4. Table Functions ...............................................................................89
7.2.1.5. LATERAL Subqueries.......................................................................89
7.2.2. The WHERE Clause........................................................................................91
7.2.3. The GROUP BY and HAVING Clauses...........................................................91
7.2.4. Window Function Processing ......................................................................94
7.3. Select Lists................................................................................................................94
7.3.1. Select-List Items ..........................................................................................94
7.3.2. Column Labels.............................................................................................95
7.3.3. DISTINCT ....................................................................................................95
7.4. Combining Queries...................................................................................................96
7.5. Sorting Rows ............................................................................................................96
7.6. LIMIT and OFFSET...................................................................................................97
7.7. VALUES Lists ............................................................................................................98
7.8. WITH Queries (Common Table Expressions) ...........................................................99
7.8.1. SELECT in WITH...........................................................................................99
7.8.2. Data-Modifying Statements in WITH .........................................................102
8. Data Types...........................................................................................................................105
8.1. Numeric Types........................................................................................................106
8.1.1. Integer Types..............................................................................................107
8.1.2. Arbitrary Precision Numbers.....................................................................107
8.1.3. Floating-Point Types..................................................................................109
8.1.4. Serial Types................................................................................................110
8.2. Monetary Types......................................................................................................111
8.3. Character Types......................................................................................................111
8.4. Binary Data Types ..................................................................................................113
8.4.1. bytea Hex Format.....................................................................................114
8.4.2. bytea Escape Format................................................................................114
8.5. Date/Time Types.....................................................................................................116
8.5.1. Date/Time Input.........................................................................................117
v
6. 8.5.1.1. Dates..............................................................................................118
8.5.1.2. Times.............................................................................................118
8.5.1.3. Time Stamps..................................................................................119
8.5.1.4. Special Values ...............................................................................120
8.5.2. Date/Time Output ......................................................................................121
8.5.3. Time Zones ................................................................................................122
8.5.4. Interval Input..............................................................................................123
8.5.5. Interval Output...........................................................................................125
8.6. Boolean Type..........................................................................................................126
8.7. Enumerated Types ..................................................................................................127
8.7.1. Declaration of Enumerated Types..............................................................127
8.7.2. Ordering.....................................................................................................127
8.7.3. Type Safety ................................................................................................128
8.7.4. Implementation Details..............................................................................128
8.8. Geometric Types.....................................................................................................129
8.8.1. Points .........................................................................................................129
8.8.2. Line Segments............................................................................................129
8.8.3. Boxes..........................................................................................................130
8.8.4. Paths...........................................................................................................130
8.8.5. Polygons.....................................................................................................130
8.8.6. Circles ........................................................................................................131
8.9. Network Address Types..........................................................................................131
8.9.1. inet...........................................................................................................131
8.9.2. cidr...........................................................................................................132
8.9.3. inet vs. cidr............................................................................................132
8.9.4. macaddr ....................................................................................................132
8.10. Bit String Types....................................................................................................133
8.11. Text Search Types.................................................................................................134
8.11.1. tsvector ................................................................................................134
8.11.2. tsquery ..................................................................................................135
8.12. UUID Type ...........................................................................................................136
8.13. XML Type ............................................................................................................137
8.13.1. Creating XML Values ..............................................................................137
8.13.2. Encoding Handling ..................................................................................138
8.13.3. Accessing XML Values............................................................................139
8.14. JSON Type............................................................................................................139
8.15. Arrays ...................................................................................................................139
8.15.1. Declaration of Array Types......................................................................139
8.15.2. Array Value Input.....................................................................................140
8.15.3. Accessing Arrays .....................................................................................142
8.15.4. Modifying Arrays.....................................................................................143
8.15.5. Searching in Arrays..................................................................................146
8.15.6. Array Input and Output Syntax................................................................146
8.16. Composite Types ..................................................................................................148
8.16.1. Declaration of Composite Types..............................................................148
8.16.2. Composite Value Input.............................................................................149
8.16.3. Accessing Composite Types ....................................................................150
8.16.4. Modifying Composite Types....................................................................150
8.16.5. Composite Type Input and Output Syntax...............................................151
8.17. Range Types .........................................................................................................152
8.17.1. Built-in Range Types ...............................................................................152
8.17.2. Examples..................................................................................................152
vi
7. 8.17.3. Inclusive and Exclusive Bounds ..............................................................153
8.17.4. Infinite (Unbounded) Ranges...................................................................153
8.17.5. Range Input/Output..................................................................................153
8.17.6. Constructing Ranges................................................................................154
8.17.7. Discrete Range Types ..............................................................................155
8.17.8. Defining New Range Types .....................................................................155
8.17.9. Indexing ...................................................................................................156
8.17.10. Constraints on Ranges............................................................................156
8.18. Object Identifier Types .........................................................................................157
8.19. Pseudo-Types........................................................................................................159
9. Functions and Operators .....................................................................................................161
9.1. Logical Operators ...................................................................................................161
9.2. Comparison Operators............................................................................................161
9.3. Mathematical Functions and Operators..................................................................163
9.4. String Functions and Operators..............................................................................166
9.4.1. format.......................................................................................................180
9.5. Binary String Functions and Operators ..................................................................182
9.6. Bit String Functions and Operators........................................................................184
9.7. Pattern Matching ....................................................................................................185
9.7.1. LIKE...........................................................................................................185
9.7.2. SIMILAR TO Regular Expressions............................................................186
9.7.3. POSIX Regular Expressions......................................................................187
9.7.3.1. Regular Expression Details ...........................................................190
9.7.3.2. Bracket Expressions ......................................................................192
9.7.3.3. Regular Expression Escapes..........................................................193
9.7.3.4. Regular Expression Metasyntax....................................................196
9.7.3.5. Regular Expression Matching Rules.............................................197
9.7.3.6. Limits and Compatibility ..............................................................198
9.7.3.7. Basic Regular Expressions............................................................199
9.8. Data Type Formatting Functions ............................................................................199
9.9. Date/Time Functions and Operators.......................................................................206
9.9.1. EXTRACT, date_part...............................................................................210
9.9.2. date_trunc..............................................................................................213
9.9.3. AT TIME ZONE..........................................................................................214
9.9.4. Current Date/Time .....................................................................................215
9.9.5. Delaying Execution....................................................................................217
9.10. Enum Support Functions......................................................................................217
9.11. Geometric Functions and Operators.....................................................................218
9.12. Network Address Functions and Operators..........................................................222
9.13. Text Search Functions and Operators...................................................................224
9.14. XML Functions ....................................................................................................228
9.14.1. Producing XML Content..........................................................................228
9.14.1.1. xmlcomment ...............................................................................228
9.14.1.2. xmlconcat .................................................................................229
9.14.1.3. xmlelement ...............................................................................229
9.14.1.4. xmlforest .................................................................................231
9.14.1.5. xmlpi ..........................................................................................231
9.14.1.6. xmlroot......................................................................................232
9.14.1.7. xmlagg........................................................................................232
9.14.2. XML Predicates .......................................................................................233
9.14.2.1. IS DOCUMENT .............................................................................233
9.14.2.2. XMLEXISTS .................................................................................233
vii
8. 9.14.2.3. xml_is_well_formed..............................................................233
9.14.3. Processing XML ......................................................................................234
9.14.4. Mapping Tables to XML..........................................................................235
9.15. JSON Functions and Operators ............................................................................239
9.16. Sequence Manipulation Functions .......................................................................242
9.17. Conditional Expressions.......................................................................................245
9.17.1. CASE.........................................................................................................245
9.17.2. COALESCE ................................................................................................246
9.17.3. NULLIF.....................................................................................................247
9.17.4. GREATEST and LEAST..............................................................................247
9.18. Array Functions and Operators ............................................................................247
9.19. Range Functions and Operators............................................................................250
9.20. Aggregate Functions.............................................................................................252
9.21. Window Functions................................................................................................255
9.22. Subquery Expressions ..........................................................................................257
9.22.1. EXISTS.....................................................................................................258
9.22.2. IN .............................................................................................................258
9.22.3. NOT IN.....................................................................................................259
9.22.4. ANY/SOME .................................................................................................259
9.22.5. ALL...........................................................................................................260
9.22.6. Row-wise Comparison.............................................................................260
9.23. Row and Array Comparisons ...............................................................................260
9.23.1. IN .............................................................................................................261
9.23.2. NOT IN.....................................................................................................261
9.23.3. ANY/SOME (array) .....................................................................................261
9.23.4. ALL (array) ...............................................................................................262
9.23.5. Row-wise Comparison.............................................................................262
9.24. Set Returning Functions .......................................................................................263
9.25. System Information Functions .............................................................................266
9.26. System Administration Functions ........................................................................276
9.26.1. Configuration Settings Functions.............................................................276
9.26.2. Server Signaling Functions......................................................................277
9.26.3. Backup Control Functions .......................................................................278
9.26.4. Recovery Control Functions ....................................................................280
9.26.5. Snapshot Synchronization Functions.......................................................282
9.26.6. Database Object Management Functions.................................................282
9.26.7. Generic File Access Functions.................................................................284
9.26.8. Advisory Lock Functions.........................................................................285
9.27. Trigger Functions .................................................................................................287
9.28. Event Trigger Functions .......................................................................................288
10. Type Conversion................................................................................................................290
10.1. Overview ..............................................................................................................290
10.2. Operators ..............................................................................................................291
10.3. Functions ..............................................................................................................294
10.4. Value Storage........................................................................................................297
10.5. UNION, CASE, and Related Constructs..................................................................298
11. Indexes ..............................................................................................................................300
11.1. Introduction ..........................................................................................................300
11.2. Index Types...........................................................................................................301
11.3. Multicolumn Indexes............................................................................................303
11.4. Indexes and ORDER BY.........................................................................................304
11.5. Combining Multiple Indexes................................................................................304
viii
9. 11.6. Unique Indexes.....................................................................................................305
11.7. Indexes on Expressions ........................................................................................306
11.8. Partial Indexes ......................................................................................................306
11.9. Operator Classes and Operator Families ..............................................................309
11.10. Indexes and Collations........................................................................................310
11.11. Examining Index Usage......................................................................................310
12. Full Text Search ................................................................................................................312
12.1. Introduction ..........................................................................................................312
12.1.1. What Is a Document?...............................................................................313
12.1.2. Basic Text Matching ................................................................................313
12.1.3. Configurations..........................................................................................314
12.2. Tables and Indexes................................................................................................315
12.2.1. Searching a Table.....................................................................................315
12.2.2. Creating Indexes ......................................................................................316
12.3. Controlling Text Search........................................................................................317
12.3.1. Parsing Documents ..................................................................................317
12.3.2. Parsing Queries ........................................................................................318
12.3.3. Ranking Search Results ...........................................................................320
12.3.4. Highlighting Results ................................................................................322
12.4. Additional Features ..............................................................................................323
12.4.1. Manipulating Documents.........................................................................323
12.4.2. Manipulating Queries...............................................................................324
12.4.2.1. Query Rewriting..........................................................................325
12.4.3. Triggers for Automatic Updates ..............................................................326
12.4.4. Gathering Document Statistics ................................................................327
12.5. Parsers...................................................................................................................328
12.6. Dictionaries...........................................................................................................330
12.6.1. Stop Words...............................................................................................331
12.6.2. Simple Dictionary....................................................................................331
12.6.3. Synonym Dictionary ................................................................................333
12.6.4. Thesaurus Dictionary...............................................................................334
12.6.4.1. Thesaurus Configuration .............................................................335
12.6.4.2. Thesaurus Example .....................................................................336
12.6.5. Ispell Dictionary.......................................................................................337
12.6.6. Snowball Dictionary ................................................................................338
12.7. Configuration Example.........................................................................................338
12.8. Testing and Debugging Text Search.....................................................................340
12.8.1. Configuration Testing...............................................................................340
12.8.2. Parser Testing...........................................................................................342
12.8.3. Dictionary Testing....................................................................................343
12.9. GiST and GIN Index Types..................................................................................344
12.10. psql Support........................................................................................................345
12.11. Limitations..........................................................................................................347
12.12. Migration from Pre-8.3 Text Search...................................................................348
13. Concurrency Control.........................................................................................................349
13.1. Introduction ..........................................................................................................349
13.2. Transaction Isolation ............................................................................................349
13.2.1. Read Committed Isolation Level .............................................................350
13.2.2. Repeatable Read Isolation Level..............................................................351
13.2.3. Serializable Isolation Level......................................................................352
13.3. Explicit Locking...................................................................................................355
13.3.1. Table-level Locks.....................................................................................355
ix
10. 13.3.2. Row-level Locks ......................................................................................357
13.3.3. Deadlocks.................................................................................................358
13.3.4. Advisory Locks........................................................................................359
13.4. Data Consistency Checks at the Application Level..............................................360
13.4.1. Enforcing Consistency With Serializable Transactions...........................360
13.4.2. Enforcing Consistency With Explicit Blocking Locks............................361
13.5. Locking and Indexes.............................................................................................361
14. Performance Tips ..............................................................................................................363
14.1. Using EXPLAIN ....................................................................................................363
14.1.1. EXPLAIN Basics.......................................................................................363
14.1.2. EXPLAIN ANALYZE .................................................................................369
14.1.3. Caveats.....................................................................................................372
14.2. Statistics Used by the Planner ..............................................................................373
14.3. Controlling the Planner with Explicit JOIN Clauses............................................374
14.4. Populating a Database ..........................................................................................376
14.4.1. Disable Autocommit................................................................................376
14.4.2. Use COPY..................................................................................................376
14.4.3. Remove Indexes.......................................................................................377
14.4.4. Remove Foreign Key Constraints ............................................................377
14.4.5. Increase maintenance_work_mem........................................................377
14.4.6. Increase checkpoint_segments ..........................................................377
14.4.7. Disable WAL Archival and Streaming Replication .................................378
14.4.8. Run ANALYZE Afterwards........................................................................378
14.4.9. Some Notes About pg_dump...................................................................378
14.5. Non-Durable Settings ...........................................................................................379
III. Server Administration .............................................................................................................381
15. Installation from Source Code ..........................................................................................383
15.1. Short Version ........................................................................................................383
15.2. Requirements........................................................................................................383
15.3. Getting The Source...............................................................................................385
15.4. Installation Procedure...........................................................................................385
15.5. Post-Installation Setup..........................................................................................395
15.5.1. Shared Libraries.......................................................................................395
15.5.2. Environment Variables.............................................................................396
15.6. Supported Platforms.............................................................................................396
15.7. Platform-specific Notes ........................................................................................397
15.7.1. AIX ..........................................................................................................397
15.7.1.1. GCC Issues..................................................................................398
15.7.1.2. Unix-Domain Sockets Broken.....................................................398
15.7.1.3. Internet Address Issues................................................................398
15.7.1.4. Memory Management .................................................................399
References and Resources ................................................................400
15.7.2. Cygwin.....................................................................................................400
15.7.3. HP-UX .....................................................................................................401
15.7.4. IRIX .........................................................................................................402
15.7.5. MinGW/Native Windows ........................................................................402
15.7.5.1. Collecting Crash Dumps on Windows ........................................403
15.7.6. SCO OpenServer and SCO UnixWare.....................................................403
15.7.6.1. Skunkware...................................................................................403
15.7.6.2. GNU Make ..................................................................................403
15.7.6.3. Readline.......................................................................................403
x
11. 15.7.6.4. Using the UDK on OpenServer...................................................404
15.7.6.5. Reading the PostgreSQL Man Pages...........................................404
15.7.6.6. C99 Issues with the 7.1.1b Feature Supplement .........................404
15.7.6.7. Threading on UnixWare ..............................................................404
15.7.7. Solaris ......................................................................................................404
15.7.7.1. Required Tools ............................................................................404
15.7.7.2. Problems with OpenSSL .............................................................405
15.7.7.3. configure Complains About a Failed Test Program ....................405
15.7.7.4. 64-bit Build Sometimes Crashes.................................................405
15.7.7.5. Compiling for Optimal Performance...........................................406
15.7.7.6. Using DTrace for Tracing PostgreSQL.......................................406
16. Installation from Source Code on Windows .....................................................................407
16.1. Building with Visual C++ or the Microsoft Windows SDK.................................407
16.1.1. Requirements ...........................................................................................408
16.1.2. Special Considerations for 64-bit Windows ............................................409
16.1.3. Building ...................................................................................................410
16.1.4. Cleaning and Installing ............................................................................410
16.1.5. Running the Regression Tests..................................................................411
16.1.6. Building the Documentation....................................................................411
16.2. Building libpq with Visual C++ or Borland C++.................................................411
16.2.1. Generated Files ........................................................................................412
17. Server Setup and Operation ..............................................................................................413
17.1. The PostgreSQL User Account ............................................................................413
17.2. Creating a Database Cluster .................................................................................413
17.2.1. Network File Systems..............................................................................414
17.3. Starting the Database Server.................................................................................414
17.3.1. Server Start-up Failures ...........................................................................416
17.3.2. Client Connection Problems ....................................................................417
17.4. Managing Kernel Resources.................................................................................417
17.4.1. Shared Memory and Semaphores ............................................................417
17.4.2. Resource Limits .......................................................................................423
17.4.3. Linux Memory Overcommit....................................................................424
17.5. Shutting Down the Server.....................................................................................425
17.6. Upgrading a PostgreSQL Cluster .........................................................................426
17.6.1. Upgrading Data via pg_dump..................................................................427
17.6.2. Non-Dump Upgrade Methods..................................................................428
17.7. Preventing Server Spoofing..................................................................................428
17.8. Encryption Options...............................................................................................429
17.9. Secure TCP/IP Connections with SSL .................................................................430
17.9.1. Using Client Certificates..........................................................................431
17.9.2. SSL Server File Usage.............................................................................431
17.9.3. Creating a Self-signed Certificate............................................................431
17.10. Secure TCP/IP Connections with SSH Tunnels.................................................432
17.11. Registering Event Log on Windows...................................................................433
18. Server Configuration.........................................................................................................434
18.1. Setting Parameters................................................................................................434
18.1.1. Parameter Names and Values...................................................................434
18.1.2. Setting Parameters via the Configuration File .........................................434
18.1.3. Other Ways to Set Parameters..................................................................434
18.1.4. Examining Parameter Settings.................................................................435
18.1.5. Configuration File Includes......................................................................435
18.2. File Locations.......................................................................................................437
xi
12. 18.3. Connections and Authentication...........................................................................438
18.3.1. Connection Settings .................................................................................438
18.3.2. Security and Authentication.....................................................................440
18.4. Resource Consumption.........................................................................................442
18.4.1. Memory....................................................................................................442
18.4.2. Disk..........................................................................................................443
18.4.3. Kernel Resource Usage............................................................................444
18.4.4. Cost-based Vacuum Delay.......................................................................444
18.4.5. Background Writer...................................................................................445
18.4.6. Asynchronous Behavior...........................................................................446
18.5. Write Ahead Log ..................................................................................................447
18.5.1. Settings.....................................................................................................447
18.5.2. Checkpoints..............................................................................................450
18.5.3. Archiving .................................................................................................450
18.6. Replication............................................................................................................451
18.6.1. Sending Server(s).....................................................................................451
18.6.2. Master Server...........................................................................................452
18.6.3. Standby Servers .......................................................................................453
18.7. Query Planning.....................................................................................................454
18.7.1. Planner Method Configuration.................................................................454
18.7.2. Planner Cost Constants ............................................................................455
18.7.3. Genetic Query Optimizer.........................................................................457
18.7.4. Other Planner Options..............................................................................458
18.8. Error Reporting and Logging ...............................................................................459
18.8.1. Where To Log ..........................................................................................459
18.8.2. When To Log ...........................................................................................462
18.8.3. What To Log ............................................................................................463
18.8.4. Using CSV-Format Log Output ...............................................................466
18.9. Run-time Statistics................................................................................................468
18.9.1. Query and Index Statistics Collector .......................................................468
18.9.2. Statistics Monitoring................................................................................469
18.10. Automatic Vacuuming........................................................................................469
18.11. Client Connection Defaults ................................................................................471
18.11.1. Statement Behavior................................................................................471
18.11.2. Locale and Formatting ...........................................................................474
18.11.3. Other Defaults........................................................................................476
18.12. Lock Management..............................................................................................477
18.13. Version and Platform Compatibility...................................................................478
18.13.1. Previous PostgreSQL Versions ..............................................................478
18.13.2. Platform and Client Compatibility.........................................................480
18.14. Error Handling....................................................................................................480
18.15. Preset Options.....................................................................................................480
18.16. Customized Options ...........................................................................................482
18.17. Developer Options..............................................................................................482
18.18. Short Options......................................................................................................485
19. Client Authentication ........................................................................................................486
19.1. The pg_hba.conf File........................................................................................486
19.2. User Name Maps ..................................................................................................492
19.3. Authentication Methods .......................................................................................494
19.3.1. Trust Authentication ................................................................................494
19.3.2. Password Authentication .........................................................................494
19.3.3. GSSAPI Authentication...........................................................................494
xii
13. 19.3.4. SSPI Authentication.................................................................................495
19.3.5. Kerberos Authentication ..........................................................................495
19.3.6. Ident Authentication.................................................................................497
19.3.7. Peer Authentication..................................................................................498
19.3.8. LDAP Authentication ..............................................................................498
19.3.9. RADIUS Authentication..........................................................................500
19.3.10. Certificate Authentication......................................................................501
19.3.11. PAM Authentication ..............................................................................501
19.4. Authentication Problems ......................................................................................502
20. Database Roles..................................................................................................................503
20.1. Database Roles .....................................................................................................503
20.2. Role Attributes......................................................................................................504
20.3. Role Membership .................................................................................................505
20.4. Function and Trigger Security..............................................................................506
21. Managing Databases .........................................................................................................508
21.1. Overview ..............................................................................................................508
21.2. Creating a Database..............................................................................................508
21.3. Template Databases..............................................................................................509
21.4. Database Configuration ........................................................................................510
21.5. Destroying a Database..........................................................................................511
21.6. Tablespaces...........................................................................................................511
22. Localization.......................................................................................................................514
22.1. Locale Support......................................................................................................514
22.1.1. Overview..................................................................................................514
22.1.2. Behavior...................................................................................................515
22.1.3. Problems ..................................................................................................516
22.2. Collation Support..................................................................................................516
22.2.1. Concepts...................................................................................................516
22.2.2. Managing Collations................................................................................518
22.3. Character Set Support...........................................................................................519
22.3.1. Supported Character Sets.........................................................................519
22.3.2. Setting the Character Set..........................................................................522
22.3.3. Automatic Character Set Conversion Between Server and Client...........523
22.3.4. Further Reading .......................................................................................525
23. Routine Database Maintenance Tasks...............................................................................526
23.1. Routine Vacuuming ..............................................................................................526
23.1.1. Vacuuming Basics....................................................................................526
23.1.2. Recovering Disk Space............................................................................527
23.1.3. Updating Planner Statistics......................................................................528
23.1.4. Updating The Visibility Map ...................................................................529
23.1.5. Preventing Transaction ID Wraparound Failures.....................................529
23.1.5.1. Multixacts and Wraparound........................................................531
23.1.6. The Autovacuum Daemon .......................................................................532
23.2. Routine Reindexing..............................................................................................533
23.3. Log File Maintenance...........................................................................................534
24. Backup and Restore ..........................................................................................................536
24.1. SQL Dump............................................................................................................536
24.1.1. Restoring the Dump.................................................................................537
24.1.2. Using pg_dumpall....................................................................................537
24.1.3. Handling Large Databases .......................................................................538
24.2. File System Level Backup....................................................................................539
24.3. Continuous Archiving and Point-in-Time Recovery (PITR)................................540
xiii
14. 24.3.1. Setting Up WAL Archiving......................................................................541
24.3.2. Making a Base Backup ............................................................................543
24.3.3. Making a Base Backup Using the Low Level API ..................................543
24.3.4. Recovering Using a Continuous Archive Backup ...................................545
24.3.5. Timelines..................................................................................................547
24.3.6. Tips and Examples...................................................................................548
24.3.6.1. Standalone Hot Backups .............................................................548
24.3.6.2. Compressed Archive Logs ..........................................................548
24.3.6.3. archive_command Scripts ........................................................548
24.3.7. Caveats.....................................................................................................549
25. High Availability, Load Balancing, and Replication.........................................................551
25.1. Comparison of Different Solutions.......................................................................551
25.2. Log-Shipping Standby Servers.............................................................................554
25.2.1. Planning ...................................................................................................555
25.2.2. Standby Server Operation........................................................................555
25.2.3. Preparing the Master for Standby Servers ...............................................556
25.2.4. Setting Up a Standby Server....................................................................556
25.2.5. Streaming Replication..............................................................................557
25.2.5.1. Authentication .............................................................................558
25.2.5.2. Monitoring...................................................................................558
25.2.6. Cascading Replication .............................................................................559
25.2.7. Synchronous Replication .........................................................................559
25.2.7.1. Basic Configuration.....................................................................559
25.2.7.2. Planning for Performance............................................................560
25.2.7.3. Planning for High Availability ....................................................561
25.3. Failover.................................................................................................................561
25.4. Alternative Method for Log Shipping ..................................................................562
25.4.1. Implementation ........................................................................................563
25.4.2. Record-based Log Shipping.....................................................................563
25.5. Hot Standby..........................................................................................................564
25.5.1. User’s Overview.......................................................................................564
25.5.2. Handling Query Conflicts ........................................................................566
25.5.3. Administrator’s Overview........................................................................568
25.5.4. Hot Standby Parameter Reference...........................................................570
25.5.5. Caveats.....................................................................................................570
26. Recovery Configuration ....................................................................................................572
26.1. Archive Recovery Settings ...................................................................................572
26.2. Recovery Target Settings......................................................................................573
26.3. Standby Server Settings........................................................................................573
27. Monitoring Database Activity...........................................................................................575
27.1. Standard Unix Tools.............................................................................................575
27.2. The Statistics Collector.........................................................................................576
27.2.1. Statistics Collection Configuration ..........................................................576
27.2.2. Viewing Collected Statistics ....................................................................576
27.2.3. Statistics Functions ..................................................................................589
27.3. Viewing Locks......................................................................................................591
27.4. Dynamic Tracing ..................................................................................................591
27.4.1. Compiling for Dynamic Tracing..............................................................592
27.4.2. Built-in Probes .........................................................................................592
27.4.3. Using Probes............................................................................................600
27.4.4. Defining New Probes ...............................................................................601
28. Monitoring Disk Usage.....................................................................................................603
xiv
15. 28.1. Determining Disk Usage ......................................................................................603
28.2. Disk Full Failure...................................................................................................604
29. Reliability and the Write-Ahead Log................................................................................605
29.1. Reliability .............................................................................................................605
29.2. Write-Ahead Logging (WAL) ..............................................................................607
29.3. Asynchronous Commit.........................................................................................607
29.4. WAL Configuration ..............................................................................................609
29.5. WAL Internals ......................................................................................................611
30. Regression Tests................................................................................................................613
30.1. Running the Tests .................................................................................................613
30.1.1. Running the Tests Against a Temporary Installation...............................613
30.1.2. Running the Tests Against an Existing Installation .................................614
30.1.3. Additional Test Suites..............................................................................614
30.1.4. Locale and Encoding................................................................................615
30.1.5. Extra Tests................................................................................................615
30.1.6. Testing Hot Standby.................................................................................615
30.2. Test Evaluation .....................................................................................................616
30.2.1. Error Message Differences.......................................................................616
30.2.2. Locale Differences...................................................................................617
30.2.3. Date and Time Differences ......................................................................617
30.2.4. Floating-Point Differences.......................................................................617
30.2.5. Row Ordering Differences.......................................................................618
30.2.6. Insufficient Stack Depth...........................................................................618
30.2.7. The “random” Test...................................................................................618
30.2.8. Configuration Parameters.........................................................................618
30.3. Variant Comparison Files .....................................................................................619
30.4. Test Coverage Examination..................................................................................619
IV. Client Interfaces .......................................................................................................................621
31. libpq - C Library ...............................................................................................................623
31.1. Database Connection Control Functions..............................................................623
31.1.1. Connection Strings...................................................................................629
31.1.1.1. Keyword/Value Connection Strings ............................................629
31.1.1.2. Connection URIs.........................................................................629
31.1.2. Parameter Key Words ..............................................................................630
31.2. Connection Status Functions................................................................................634
31.3. Command Execution Functions ...........................................................................638
31.3.1. Main Functions ........................................................................................638
31.3.2. Retrieving Query Result Information ......................................................645
31.3.3. Retrieving Other Result Information .......................................................648
31.3.4. Escaping Strings for Inclusion in SQL Commands.................................649
31.4. Asynchronous Command Processing...................................................................652
31.5. Retrieving Query Results Row-By-Row ..............................................................656
31.6. Canceling Queries in Progress..............................................................................657
31.7. The Fast-Path Interface.........................................................................................658
31.8. Asynchronous Notification...................................................................................659
31.9. Functions Associated with the COPY Command ..................................................659
31.9.1. Functions for Sending COPY Data............................................................660
31.9.2. Functions for Receiving COPY Data.........................................................661
31.9.3. Obsolete Functions for COPY ...................................................................662
31.10. Control Functions...............................................................................................664
31.11. Miscellaneous Functions ....................................................................................665
xv
16. 31.12. Notice Processing ...............................................................................................667
31.13. Event System......................................................................................................668
31.13.1. Event Types............................................................................................668
31.13.2. Event Callback Procedure......................................................................670
31.13.3. Event Support Functions........................................................................671
31.13.4. Event Example.......................................................................................672
31.14. Environment Variables .......................................................................................674
31.15. The Password File ..............................................................................................676
31.16. The Connection Service File ..............................................................................676
31.17. LDAP Lookup of Connection Parameters..........................................................677
31.18. SSL Support........................................................................................................678
31.18.1. Client Verification of Server Certificates ...............................................678
31.18.2. Client Certificates...................................................................................679
31.18.3. Protection Provided in Different Modes ................................................679
31.18.4. SSL Client File Usage............................................................................681
31.18.5. SSL Library Initialization ......................................................................681
31.19. Behavior in Threaded Programs.........................................................................682
31.20. Building libpq Programs.....................................................................................683
31.21. Example Programs..............................................................................................684
32. Large Objects ....................................................................................................................693
32.1. Introduction ..........................................................................................................693
32.2. Implementation Features ......................................................................................693
32.3. Client Interfaces....................................................................................................693
32.3.1. Creating a Large Object...........................................................................694
32.3.2. Importing a Large Object.........................................................................694
32.3.3. Exporting a Large Object.........................................................................695
32.3.4. Opening an Existing Large Object...........................................................695
32.3.5. Writing Data to a Large Object................................................................695
32.3.6. Reading Data from a Large Object ..........................................................696
32.3.7. Seeking in a Large Object........................................................................696
32.3.8. Obtaining the Seek Position of a Large Object........................................696
32.3.9. Truncating a Large Object .......................................................................697
32.3.10. Closing a Large Object Descriptor ........................................................697
32.3.11. Removing a Large Object ......................................................................697
32.4. Server-side Functions ...........................................................................................698
32.5. Example Program .................................................................................................698
33. ECPG - Embedded SQL in C............................................................................................704
33.1. The Concept..........................................................................................................704
33.2. Managing Database Connections .........................................................................704
33.2.1. Connecting to the Database Server..........................................................704
33.2.2. Choosing a Connection............................................................................705
33.2.3. Closing a Connection...............................................................................707
33.3. Running SQL Commands.....................................................................................707
33.3.1. Executing SQL Statements ......................................................................707
33.3.2. Using Cursors...........................................................................................708
33.3.3. Managing Transactions............................................................................708
33.3.4. Prepared Statements.................................................................................709
33.4. Using Host Variables ............................................................................................710
33.4.1. Overview..................................................................................................710
33.4.2. Declare Sections.......................................................................................710
33.4.3. Retrieving Query Results.........................................................................711
33.4.4. Type Mapping ..........................................................................................712
xvi
17. 33.4.4.1. Handling Character Strings .........................................................713
33.4.4.2. Accessing Special Data Types.....................................................713
33.4.4.2.1. timestamp, date...............................................................714
33.4.4.2.2. interval ............................................................................714
33.4.4.2.3. numeric, decimal.............................................................715
33.4.4.3. Host Variables with Nonprimitive Types ....................................716
33.4.4.3.1. Arrays .............................................................................716
33.4.4.3.2. Structures........................................................................717
33.4.4.3.3. Typedefs..........................................................................718
33.4.4.3.4. Pointers ...........................................................................719
33.4.5. Handling Nonprimitive SQL Data Types.................................................719
33.4.5.1. Arrays..........................................................................................719
33.4.5.2. Composite Types .........................................................................721
33.4.5.3. User-defined Base Types.............................................................722
33.4.6. Indicators..................................................................................................723
33.5. Dynamic SQL.......................................................................................................724
33.5.1. Executing Statements without a Result Set .............................................724
33.5.2. Executing a Statement with Input Parameters .........................................724
33.5.3. Executing a Statement with a Result Set .................................................725
33.6. pgtypes Library.....................................................................................................726
33.6.1. The numeric Type ....................................................................................726
33.6.2. The date Type...........................................................................................729
33.6.3. The timestamp Type.................................................................................732
33.6.4. The interval Type .....................................................................................736
33.6.5. The decimal Type.....................................................................................736
33.6.6. errno Values of pgtypeslib .......................................................................737
33.6.7. Special Constants of pgtypeslib...............................................................738
33.7. Using Descriptor Areas ........................................................................................738
33.7.1. Named SQL Descriptor Areas .................................................................738
33.7.2. SQLDA Descriptor Areas ........................................................................740
33.7.2.1. SQLDA Data Structure................................................................741
33.7.2.1.1. sqlda_t Structure.............................................................741
33.7.2.1.2. sqlvar_t Structure............................................................742
33.7.2.1.3. struct sqlname Structure .................................................743
33.7.2.2. Retrieving a Result Set Using an SQLDA ..................................743
33.7.2.3. Passing Query Parameters Using an SQLDA..............................745
33.7.2.4. A Sample Application Using SQLDA ........................................746
33.8. Error Handling......................................................................................................751
33.8.1. Setting Callbacks .....................................................................................751
33.8.2. sqlca .........................................................................................................753
33.8.3. SQLSTATE vs. SQLCODE...........................................................................755
33.9. Preprocessor Directives ........................................................................................758
33.9.1. Including Files .........................................................................................758
33.9.2. The define and undef Directives ..............................................................759
33.9.3. ifdef, ifndef, else, elif, and endif Directives.............................................759
33.10. Processing Embedded SQL Programs................................................................760
33.11. Library Functions ...............................................................................................761
33.12. Large Objects......................................................................................................762
33.13. C++ Applications ...............................................................................................763
33.13.1. Scope for Host Variables........................................................................764
33.13.2. C++ Application Development with External C Module ......................765
33.14. Embedded SQL Commands ...............................................................................767
xvii