1. BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA
HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH
PostgreSQL Best Practices
Overview from the initial setup to an OLTP performance benchmark
against Oracle.
Emiliano Fusaglia Principal Consultant
Jacques Kostic Principal Consultant

2. 2018 © Trivadis
Exadata X7-2 POC with OVM
2
TechEvent September 2018
Specialties:
• Database Cloud computing (DBaaS)
• Oracle RAC
• Grid Infrastructure (CRS, ASM)
• Data Guard
• Instance and SQL Performance & Tuning
• Linux & Virtualization
Certifications:
• Oracle Certified Professional 9i, 10g, 11g & 12c
• Oracle Exadata Administrator X3 –X4 Certified Expert
Teaching Courses at Trivadis:
• Oracle 11g & 12c Grid Infrastructure & RAC
• Oracle 11g & 12c Data Guard
• Oracle Exadata
• Oracle 12c New Features
About me…
@EFusaglia

3. PostgreSQL Best Practices3 14/09/2018
Experience:
• Initially C/C++ developer
• In touch with Oracle since 1990 from version 4 on SCO Unix!
• High Availability and Backup & Recovery Architect
• SQL and Instance Performance & Tuning
• License Audit and Consolidation
Certifications:
• Oracle Certified Master 11g & 12c
• Oracle 11g Performance Tuning Certified Expert
• Oracle RAC 11g and Grid Infrastructure Administration
• Oracle Exadata Administrator Certified Expert
• Oracle Certified SQL Expert 11g
Teaching Courses at Trivadis:
• Oracle 11g & 12c Grid Infrastructure & RAC
• Oracle 11g & 12c Data Guard
• Oracle 11g & 12c Performance & Tuning
• Oracle 11g & 12c Administration
• SQL & PL-SQL
• OEM – 12 & 13
About me…
@JKOFR

4. Agenda
PostgreSQL Best Practices9/14/2018
1. PostgreSQL
Introduction & Architecture
OS Requirements
Installation Options
Securing PostgreSQL ClusterDB
Main parameters to configure
Backup and Recovery
2. OLTP performance benchmark PostgreSQL vs Oracle
Configuration
Results
3. Conclusion
Takeaway
4

5. PostgreSQL Best Practices9/14/2018
PostgreSQL Introduction &
Architecture
5

6. Introduction to PostgreSQL
PostgreSQL is an opensource and independent Object-RDBMS developed and
maintained by the PostgreSQL Global Development Group.
The first version was released in 1996 as INGRES development, and it included support
for Object orientation and SQL.
Main Characteristics:
ACID (Atomicity, Consistency, Isolation, Durability)
Multiversion concurrency control (MVCC)
Foreign keys, Indexes, Views, Trigger, Functions, Procedural Languages (PL), etc..
Streaming Replication (as of 9.0)
Hot Standby (as of 9.0)
PostgreSQL Best Practices9/14/20186

7. PostgreSQL Architecture
PostgreSQL Best Practices9/14/20187
pg_crl start
Source PostgreSQL documentation

8. Database Cluster
PostgreSQL Best Practices9/14/20188
Source PostgreSQL documentation
A cluster is an instance of postgreSQL containing one or many databases
– Conceptually similar to MySQL, SQL Server and Oracle Pluggable Databases
Server
Cluster pgclu01 (port 5438) Cluster pgclu02 (port 5439)
postgres template0 template1 postgres template0 template1
ecom01 erp01 sales01 dwh01 hr01 supp01

9. Database Cluster
PostgreSQL Best Practices9/14/20189
postgres
template0
template1
System or Master database, it contains system tables, views, procedures,
metadata, user and role definitions.
Template0 it is a read-only empty database used as seed database.
Template1 it is a read-write database, which allows customizations before
to be used as default seed database.
App
Application Database it contains application objects like tables, indexes,
views, procedures, constraints etc..

10. Before Image and Vacuum Process
PostgreSQL has no rollback segments, and it guarantees read consistency in the
following way:
Writing new image in a new location
Marking the initial image as OLD, keeping intact the data.
Adding a pointer to the OLD image pointing the new one.
PostgreSQL Best Practices9/14/201810

11. Before Image and Vacuum Process
PostgreSQL Best Practices9/14/201811
Page x
Case 1 The new image remains on the
same page.
1,’blue’
2,’green’
0,’red’
Case 2 The new image migrates
on a new page.
1,’white’ NEW
OLD
UPDATE app_tab SET col2=‘white’ WHERE col0=1;
Page x Page y
1,’bb’,test4
OLD
NEW
1,’blue’
2,’green’
0,’red’
1,’white’
UPDATE app_tab SET col2=‘white’ WHERE col0=1;

12. Before Image and Vacuum Process
PostgreSQL Best Practices9/14/201812
VACUUM Process
Reclaims space occupied by old tuple images
Updates data statistics used by the query planner
Updates the visibility map
Resets the transaction ID of old blocks to prevent wraparound
The standard VACUUM is executed regularly by default
Manual VACUUMing is possible

13. PostgreSQL Best Practices9/14/2018
Hands on set up
13

14. OS Optimization 1/2
PostgreSQL14 14/09/2018
Kernel optimization
– /etc/sysctl.d/90-postgres-sysctl.conf
fs.file-max = 6815744
fs.aio-max-nr = 1048576
kernel.sem = 250 32768 1000 128
kernel.shmall = 1073741824
kernel.shmmni = 4096
kernel.shmmax = 4398046511104
#Huge Pages 80GB
vm.nr_hugepages= 40960
vm.min_free_kbytes=524288

15. OS Optimization 2/2
PostgreSQL15 14/09/2018
– /etc/security/limits.d/postgres-limits.conf
postgres soft nofile 16384
postgres hard nofile 16384
postgres soft memlock 83886080
postgres hard memlock 83886080
Storage
– Binaries, ClusterDB, External Tablespaces and WAL files on T1 Storage
– Backups on T2 Storage

16. Installation Options
List of Supported Platforms: Linux (all recent distributions), Windows (Win2000 SP4
and later), FreeBSD, OpenBSD, NetBSD, Mac OS X, AIX, HP/UX, IRIX, Solaris, Tru64
Unix, and UnixWare.
P.S.: this presentation focuses on Linux 64-bit
PostgreSQL can be installed using one of the following method :
Source Code
Binary Package
Binaries Archive without installer (Advanced users) TVD Recommended Option
PostgreSQL Best Practices9/14/201816

17. Securing PostgreSQL ClusterDB
After the installation one of the first tasks to perform is securing the local and remote
database connections, defining the open ports and the authentication methods.
Those settings can be defined in two different configuration files:
postgresql.conf section CONNECTIONS AND AUTHENTICATION
pg_hba.conf
PostgreSQL Best Practices9/14/201817

18. Securing PostgreSQL ClusterDB - postgresql.conf 1/2
PostgreSQL Best Practices9/14/201818
#------------------------------------------------------------------------------
# CONNECTIONS AND AUTHENTICATION
#------------------------------------------------------------------------------
# - Connection Settings -
listen_addresses = '192.168.1.129,localhost' # what IP address(es) to listen on;
# comma-separated list of addresses;
# defaults to 'localhost'; use '*' for all
# (change requires restart)
port = 5544 # (change requires restart)
max_connections = 100 # (change requires restart)
#superuser_reserved_connections = 3 # (change requires restart)
#unix_socket_directories = '/tmp' # comma-separated list of directories
# (change requires restart)
#unix_socket_group = '' # (change requires restart)
unix_socket_permissions = 0770 # begin with 0 to use octal notation
# (change requires restart)
#bonjour = off # advertise server via Bonjour
# (change requires restart)
#bonjour_name = '' # defaults to the computer name
# (change requires restart)
# - Security and Authentication -
#authentication_timeout = 1min # 1s-600s
#ssl = off
#ssl_ciphers = 'HIGH:MEDIUM:+3DES:!aNULL' # allowed SSL ciphers
#ssl_prefer_server_ciphers = on
#ssl_ecdh_curve = 'prime256v1'
#ssl_dh_params_file = ''
#ssl_cert_file = 'server.crt'
...

19. Securing PostgreSQL ClusterDB - postgresql.conf 2/2
PostgreSQL Best Practices9/14/201819
# - Security and Authentication -
#authentication_timeout = 1min # 1s-600s
#ssl = off
#ssl_ciphers = 'HIGH:MEDIUM:+3DES:!aNULL' # allowed SSL ciphers
#ssl_prefer_server_ciphers = on
#ssl_ecdh_curve = 'prime256v1'
#ssl_dh_params_file = ''
#ssl_cert_file = 'server.crt'
#ssl_key_file = 'server.key'
#ssl_ca_file = ''
#ssl_crl_file = ‚‘
password_encryption = scram-sha-256 # md5 or scram-sha-256
#db_user_namespace = off
#row_security = on
# GSSAPI using Kerberos
#krb_server_keyfile = ''
#krb_caseins_users = off
# - TCP Keepalives -
# see "man 7 tcp" for details
#tcp_keepalives_idle = 0 # TCP_KEEPIDLE, in seconds;
# 0 selects the system default
#tcp_keepalives_interval = 0 # TCP_KEEPINTVL, in seconds;
# 0 selects the system default
#tcp_keepalives_count = 0 # TCP_KEEPCNT;
# 0 selects the system default
...

20. Securing PostgreSQL ClusterDB - pg_hba.conf
PostgreSQL Best Practices9/14/201820
# TYPE DATABASE USER ADDRESS METHOD
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
# host all all 127.0.0.1/32 trust
host all all 192.168.1.1/24 trust
# IPv6 local connections:
# host all all ::1/128 trust
# Allow replication connections from localhost, by a user with the
# replication privilege.
# local replication all trust
# host replication all 127.0.0.1/32 trust
# host replication all ::1/128 trust

21. Main parameters to configure - 1/2
PostgreSQL Best Practices9/14/201821
postgres=# select name,setting from pg_file_settings;
name settings
------------------------------------------------------------------------------------------------
external_pid_file | extra_pid.info
listen_addresses | 192.168.1.129,localhost
port | 5544
max_connections | 100
unix_socket_permissions | 0770
password_encryption | scram-sha-256
shared_buffers | 4096MB
huge_pages | on
max_stack_depth | 6MB
dynamic_shared_memory_type | posix
effective_io_concurrency | 80
max_worker_processes | 150
max_parallel_workers | 24
wal_level | replica
wal_compression | on
archive_mode | on
archive_command | test ! -f /home/postgres01/backup_dir/clusterTEST/archives/%f && cp %p
/home/postgres01/backup_dir/clusterTEST/archives/%f
archive_timeout | 14400
log_destination | stderr,syslog
logging_collector | on
log_directory | /home/postgres01/clusterTEST/logs

22. Main parameters to configure - 2/2
PostgreSQL Best Practices9/14/201822
name settings
------------------------------------------------------------------------------------------------
log_filename | alert_cluster_test-%Y-%m-%d_%H%M%S.log
log_file_mode | 0600
log_rotation_age | 30d
log_rotation_size | 100MB
syslog_facility | LOCAL0
syslog_ident | postgres_cluster_test
log_lock_waits | on
log_timezone | Europe/Vaduz
cluster_name | cluster_test
log_autovacuum_min_duration | 0
default_tablespace | TS_data01
datestyle | iso, mdy
timezone | Europe/Vaduz
extra_float_digits | 3
lc_messages | en_US.UTF-8
lc_monetary | en_US.UTF-8
lc_numeric | en_US.UTF-8
lc_time | en_US.UTF-8
default_text_search_config | pg_catalog.english
(40 rows)
postgres=#

23. Backup and Recovery
PostgreSQL provides the following options regarding the backup/recovery strategy:
Logical Backup
– Single database dump pg_dump
– Cluster database dump pg_dumpall
Physical Backup
– File System Level Backup
– Continuous Archiving and Point-in-Time Recovery (PITR)
PostgreSQL Best Practices9/14/201823

24. Backup and Recovery - Logical Backup
PostgreSQL provides the following options regarding the backup/recovery strategy:
Logical Backup
pg_dump create a TEXT file that can be restored by psql
Restore Dump File on database dbtest01_restore
PostgreSQL Best Practices9/14/201824
$ pg_dump dbtest01 > /backup_dir/dbtest01_dump_20180716.dmp
$ psql --set ON_ERROR_STOP=on dbtest01_restore <
/backup_dir/dbtest01_dump_20180716.dmp

25. Backup and Recovery - Physical Backup
PostgreSQL provides the following options regarding the backup/recovery strategy:
Physical Backup
Continuous Archiving and Point-in-Time Recovery (PITR)
PostgreSQL Best Practices9/14/201825
#!/bin/bash
db_cluster_base="/u01/PosgreSQL"
db_cluster_dir="/u01/PosgreSQL/clusterTEST"
bckup_start=$(date +"%Y%m%d%H%M")
logfile="/u01/PosgreSQL/backup_dir/logs/clusterTEST_backup_$bckup_start.log"
backup_dir="/u01/PosgreSQL/backup_dir/clusterTEST"
bck_label="Start_Backup_$bckup_start“
...

26. Backup and Recovery - Physical Backup
PostgreSQL Best Practices9/14/201826
...
psql postgres -L $logfile << EOF
SELECT pg_start_backup('$bck_label', false, false);
! tar zcvf $backup_dir/backup_$bckup_start.tar.gz --warning=no-file-changed
--warning=no-file-removed -C /u01/PosgreSQL clusterTEST --exclude='pg_wal/*‘
--exclude='postmaster*' --exclude='pg_replslot/*'
SELECT * FROM pg_stop_backup(false, true);
q
EOF

27. 9/14/2018
OLTP Performance Benchmark
PostgreSQL vs Oracle
Oracle DBaaS27

28. OLTP Test: PostGreSQL vs Oracle
PostgreSQL Best Practices9/14/201828
Goal
Use the same type of machine
Test the same OLTP workload on both databases
Test different CPU allocation
Compare the results

29. OLTP Test: PostGreSQL vs Oracle: Configuration
PostgreSQL Best Practices9/14/201829
Server details
Main host:
• 2 * 8 cores CPU E5-2680 0 @ 2.70GHz
• OEL 7.2
• 192 GB
• Flash Storage volumes on PCIe cards (no NVMe)
VMs:
• 8 vCPU
• 8 GB RAM
Concurrent sessions:
• 100

30. OLTP Test: PostGreSQL vs Oracle : Configuration
PostgreSQL Best Practices9/14/201830
Hammerdbcli PostGreSQL Test Setup
dbset db pg
diset tpcc pg_defaultdbase hammerdb
loadscript
vudestroy
vuset delay 5
vuset repeat 5
vuset showoutput 1
vuset timestamps 1
vuset logtotemp 1
vuset vu 100
vucreate
vurun

31. OLTP Test: PostGreSQL vs Oracle : Configuration
PostgreSQL Best Practices9/14/201831
Hammerdbcli Oracle Test Setup
dbset db ora
diset connection instance jko
loadscript
vudestroy
vuset delay 5
vuset repeat 5
vuset showoutput 1
vuset timestamps 1
vuset logtotemp 1
vuset vu 100
vucreate
vurun

32. OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices9/14/201832
PostGreSQL
Time to complete the full test  4.23 mn
Average CPU Usage  88 %
Transaction per minutes max  156’222

33. OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices9/14/201833
Oracle
Time to complete the full test  Time 4.12 mn
Average CPU Usage  74 %
Transaction per minutes max  172’268

34. OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices9/14/201834
select sum(value) from v$sysstat where name = 'user commits' or name = 'user rollbacks'
select sum(xact_commit + xact_rollback) from pg_stat_database

35. OLTP Test: PostGreSQL vs Oracle : Configuration
PostgreSQL Best Practices9/14/201835
Lets Scale!
Main host:
• 2 * 8 cores CPU E5-2680 0 @ 2.70GHz
• OEL 7.2
• 192 GB
• Flash Storage volumes on PCIe cards (no NVMe)
VMs:
• 16 vCPU
• 8 GB RAM
Concurrent sessions:
• 100

36. OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices9/14/201836
PostGreSQL
Time to complete the full test  Time 3.27 mn
Average CPU Usage  65 %
Transaction per minutes max  194’904

37. OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices9/14/201837
Oracle
Time to complete the full test  Time 3.16 mn
Average CPU Usage  57 %
Transaction per minutes max  251’292

38. OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices9/14/201838
select sum(value) from v$sysstat where name = 'user commits' or name = 'user rollbacks'
select sum(xact_commit + xact_rollback) from pg_stat_database

39. OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices9/14/201839
8 vCPU
2.6% Faster 16% Less CPU 9.3% More TPM

40. OLTP Test: PostGreSQL vs Oracle : Results
PostgreSQL Best Practices9/14/201840
16 vCPU
3.4% Faster 12.3% Less CPU 22.43% More TPM

41. 41 9/14/2018
Conclusion
Oracle DBaaS

42. Conclusions
PostgreSQL Best Practices9/14/201842
 PostgreSQL is a mature, enterprise RDBMS
 PostgreSQL offers good performances
 PostgreSQL offers High Availability solution

43. Emilian Fusaglia, Principal Consultant
Tel. +41-79-909 7213 Emiliano.Fusaglia@trivadis.com
Jacques Kostic, Principal Consultant
Tel. +41-79-909 7263 Jacques.Kostic@trivadis.com
9/14/201843 TechEvent September 2018

44. Session Feedback – now
TechEvent September 201844 14.09.2018
Please use the Trivadis Events mobile app to give feedback on each session
Use "My schedule" if you have registered for a session
Otherwise use "Agenda" and the search function
If the mobile app does not work (or if you have a Windows smartphone), use your
smartphone browser
– URL: http://trivadis.quickmobileplatform.eu/
– User name: <your_loginname> (such as "svv")
– Password: sent by e-mail...