Basic Cluster Database operations Configuring autostart of the Postgres database Configuring Huge Pages

1. POSTGRES DATABASE CLUSTER OPERATIONS.
Topics covered:
1. Creating new POSTGRES DB Cluster
2. Configuring AutoStart
3. Configuring Hugepages
If you have been following along in continuation to the previous post, Today we will see how to create
Postgres Database in a new directory location , By Default Postgres gets installed in the following location
Postgres 12 Home:
/usr/pgsql-12
Postgres Database:
/var/lib/pgsql/12/data
Since /var mountpoint is OS partition , we will install Postgres Database Cluster in a new location , This new
mountpoint can be SSD/Flash disk.
In this case /u01/db01 will be referred as data directory or data area.

2. [postgres@centos usr]$ mkdir -p /u01/pgsql/data
[post_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/usr/pgsql-12/bin/pg_ctl -D /u01/pgsql/data -l logfile gres@centos usr]$
export PGDATA=/u01/pgsql/data
[postgres@centos usr]$ /usr/pgsql-12/bin/initdb -D $PGDATA
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_NZ.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /u01/pgsql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Asia/Dubai
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg
Start
/usr/pgsql-12/bin/pg_ctl -D /u01/pgsql/data -l logfile start
NOTE:
1. initdb command will fail if the data directory exists and already contains files
2. Enabling or disabling group access on an existing cluster requires the cluster to be shut
down and the appropriate mode to be set on all directories and files before
restarting PostgreSQL.
3. Permission combinations for Datafile Directory and Files
a. Directory 0700
b. Files 0600
c. Groups 0640
4. Supported File Systems are
a. NFS
b. Secondary File Systems

3. Alternative way to create the database is using the pg_ctl utility :
pg_ctl -D /u01/pgsql/data initdb
[postgres@centos ~]$ pg_ctl -D /u01/pgsql/data/ initdb
The files belonging to this database system will be owned by user
"postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_NZ.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /u01/pgsql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Asia/Dubai
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/usr/pgsql-12/bin/pg_ctl -D /u01/pgsql/data -l logfile start
Now let’s start the new database cluster:
[postgres@centos ~]$ /usr/pgsql-12/bin/pg_ctl -D /u01/pgsql/data -l logfile
start
waiting for server to start.... stopped waiting
pg_ctl: could not start server
Examine the log output.
[postgres@centos ~]$ cat logfile
2020-04-18 02:26:33.967 +04 [3486] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by gcc
(GCC) 4.8.5 20150623 (Red Hat 4.8.5-39), 64-bit
2020-04-18 02:26:33.967 +04 [3486] LOG: could not bind IPv6 address "::1": Address already in use
2020-04-18 02:26:33.967 +04 [3486] HINT: Is another postmaster already running on port 5432? If not, wait a
few seconds and retry.
2020-04-18 02:26:33.967 +04 [3486] LOG: could not bind IPv4 address "127.0.0.1": Address already in use
2020-04-18 02:26:33.967 +04 [3486] HINT: Is another postmaster already running on port 5432? If not, wait a
few seconds and retry.
2020-04-18 02:26:33.967 +04 [3486] WARNING: could not create listen socket for "localhost"
2020-04-18 02:26:33.967 +04 [3486] FATAL: could not create any TCP/IP sockets
2020-04-18 02:26:33.967 +04 [3486] LOG: database system is shut down

4. #Expected error , To fix this we Add or edit the following lines :
Location : /u01/pgsql/data
File : postgresql.conf
[postgres@centos data]$ cat postgresql.conf |egrep "listen|5450"
listen_addresses = '*' # what IP address(es) to listen on;
port = 5450 # (change requires restart)
File : pg_hba.conf
[postgres@centos data]$ cat pg_hba.conf |grep md5
# METHOD can be "trust", "reject", "md5", "password", "scram-sha-256",
# Note that "password" sends passwords in clear text; "md5" or
host all all 0.0.0.0/0 md5
[postgres@centos data]$ /usr/pgsql-12/bin/pg_ctl -D /u01/pgsql/data -l
logfile start
waiting for server to start.... done
server started
[postgres@centos data]$ psql -p 5450 postgres
psql (12.2)
Type "help" for help.
postgres=# du
List of roles
Role name | Attributes | Member of
-----------+------------------------------------------------------------+-----------
postgres | Superuser, Create role, Create DB, Replication, Bypass RLS | {}
postgres=# x
Expanded display is on.
postgres=# l+
List of databases
-[ RECORD 1 ]-----+-------------------------------------------
Name | postgres
Owner | postgres
Encoding | UTF8
Collate | en_NZ.UTF-8
Ctype | en_NZ.UTF-8
Access privileges |
Size | 8193 kB
Tablespace | pg_default
Description | default administrative connection database
-[ RECORD 2 ]-----+-------------------------------------------
Name | template0
Owner | postgres
Encoding | UTF8
Collate | en_NZ.UTF-8
Ctype | en_NZ.UTF-8
Access privileges | =c/postgres +
| postgres=CTc/postgres
Size | 8049 kB
Tablespace | pg_default
Description | unmodifiable empty database

5. -[ RECORD 3 ]-----+-------------------------------------------
Name | template1
Owner | postgres
Encoding | UTF8
Collate | en_NZ.UTF-8
Ctype | en_NZ.UTF-8
Access privileges | =c/postgres +
| postgres=CTc/postgres
Size | 8049 kB
Tablespace | pg_default
Description | default template for new databases
Note:
Database Cluster, Collection of databases managed by single server instance.
Database Cluster has two main components Data Directory , Port number.
Default Databases created are
a. Template0
b. Template1
c. Postgres
Cluster can be started or stopped , DB cannot be started or stopped individually.
Each cluster has two default tablespace : pg_default and pg_global.
Local connection methods are:
1. psql
2. psql -p <port number> [DBNAME] [USERNAME]
Viewing current database information:
postgres=# select current_Database();
current_database
------------------
postgres
(1 row)
postgres=# select datname ,oid from pg_database ;
datname | oid
-----------+-------
postgres | 14187
template1 | 1
template0 | 14186
(3 rows)
Cluster Database Start/Status/Stop Operations

6. [postgres@centos data]$ #Ensure PGDATA variable is exported
[postgres@centos data]$ echo $PGDATA
/u01/pgsql/data
[postgres@centos data]$ pg_ctl status -D /u01/pgsql/data/
pg_ctl: server is running (PID: 4246)
/usr/pgsql-12/bin/postgres "-D" "/u01/pgsql/data/"
[postgres@centos data]$ pg_ctl stop -m immediate -D /u01/pgsql/data/
waiting for server to shut down.... done
server stopped
[1]+ Done nohup postgres -D /u01/pgsql/data/ > logfile 2>&1
[postgres@centos data]$ pg_ctl status -D /u01/pgsql/data/
pg_ctl: no server running
##MAKE SURE YOU RUN IT ALWAYS IN BACKGROUD
[postgres@centos data]$ nohup postgres -D /u01/pgsql/data/ >logfile 2>&1 &
[1] 4312
[postgres@centos data]$ pg_ctl status -D /u01/pgsql/data/
pg_ctl: server is running (PID: 4312)
/usr/pgsql-12/bin/postgres "-D" "/u01/pgsql/data/"
Configuring AutoStart of Postgres Database Cluster
#Create the following file
[root@centos data]# cat /etc/systemd/system/postgresql.service
[Unit]
Description=PostgreSQL database server
Documentation=man:postgres(1)
After=network.target
[Service]
Type=forking
User=postgres
Group=postgres
# Where to send early-startup messages from the server (before the logging
# options of postgresql.conf take effect)
# This is normally controlled by the global default set by systemd
# StandardOutput=syslog
# Disable OOM kill on the postmaster
OOMScoreAdjust=-1000
# ... but allow it still to be effective for child processes
# (note that these settings are ignored by Postgres releases before 9.5)
Environment=PG_OOM_ADJUST_FILE=/proc/self/oom_score_adj
Environment=PG_OOM_ADJUST_VALUE=0
# Maximum number of seconds pg_ctl will wait for postgres to start. Note
that

7. # PGSTARTTIMEOUT should be less than TimeoutSec value.
Environment=PGSTARTTIMEOUT=270
Environment=PGDATA=/u01/pgsql/data
ExecStart=/usr/pgsql-12/bin/pg_ctl start -D ${PGDATA} -s -w -t
${PGSTARTTIMEOUT}
ExecStop=/usr/pgsql-12/bin/pg_ctl stop -D ${PGDATA} -s -m fast
ExecReload=/usr/pgsql-12/bin/pg_ctl reload -D ${PGDATA} -s
# Give a reasonable amount of time for the server to start up/shut down.
# Ideally, the timeout for starting PostgreSQL server should be handled more
# nicely by pg_ctl in ExecStart, so keep its timeout smaller than this
value.
TimeoutSec=300
[Install]
WantedBy=multi-user.target
[root@centos data]# systemctl enable postgresql.service
Created symlink from /etc/systemd/system/multi-
user.target.wants/postgresql.service to
/etc/systemd/system/postgresql.service.
[root@centos data]# systemctl start postgresql.service
[root@centos data]# systemctl status postgresql
● postgresql.service - PostgreSQL database server
Loaded: loaded (/etc/systemd/system/postgresql.service; enabled; vendor
preset: disabled)
Active: active (running) since Sat 2020-04-18 03:37:19 +04; 3s ago
Docs: man:postgres(1)
Process: 5598 ExecStart=/usr/pgsql-12/bin/pg_ctl start -D ${PGDATA} -s -w
-t ${PGSTARTTIMEOUT} (code=exited, status=0/SUCCESS)
Main PID: 5601 (postgres)
Tasks: 8
CGroup: /system.slice/postgresql.service
├─5601 /usr/pgsql-12/bin/postgres -D /u01/pgsql/data
├─5602 postgres: logger
├─5604 postgres: checkpointer
├─5605 postgres: background writer
├─5606 postgres: walwriter
├─5607 postgres: autovacuum launcher
├─5608 postgres: stats collector
└─5609 postgres: logical replication launcher
Apr 18 03:37:19 centos systemd[1]: Starting PostgreSQL database server...
Apr 18 03:37:19 centos pg_ctl[5598]: 2020-04-18 03:37:19.150 +04 [5601] LOG:
starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by gcc (G...9),
64-bit
Apr 18 03:37:19 centos pg_ctl[5598]: 2020-04-18 03:37:19.150 +04 [5601] LOG:
listening on IPv4 address "0.0.0.0", port 5450

8. Apr 18 03:37:19 centos pg_ctl[5598]: 2020-04-18 03:37:19.150 +04 [5601] LOG:
listening on IPv6 address "::", port 5450
Apr 18 03:37:19 centos pg_ctl[5598]: 2020-04-18 03:37:19.151 +04 [5601] LOG:
listening on Unix socket "/var/run/postgresql/.s.PGSQL.5450"
Apr 18 03:37:19 centos pg_ctl[5598]: 2020-04-18 03:37:19.152 +04 [5601] LOG:
listening on Unix socket "/tmp/.s.PGSQL.5450"
Apr 18 03:37:19 centos pg_ctl[5598]: 2020-04-18 03:37:19.158 +04 [5601] LOG:
redirecting log output to logging collector process
Apr 18 03:37:19 centos pg_ctl[5598]: 2020-04-18 03:37:19.158 +04 [5601]
HINT: Future log output will appear in directory "log".
Apr 18 03:37:19 centos systemd[1]: Started PostgreSQL database server.
Hint: Some lines were ellipsized, use -l to show in full
Steps to Configure HugePages for Postgres, To leverage 2MB upto 1GB page Size.
#Get the current configuration of Postgres.
[postgres@centos data]$ pwd
/u01/pgsql/data
#Current Huge pages avaialbe
[postgres@centos data]$ cat /proc/meminfo |grep -i Huge
AnonHugePages: 81920 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
#Lets Understand how much memory we have , How much postgres is using
[root@centos sysctl.d]# free -g
total used free shared buff/cache
available
Mem: 7 0 7 0 0
7
[root@centos sysctl.d]# export PGDATA=/u01/pgsql/data
[root@centos sysctl.d]# cat /tmp/hp.sh
#set -x
#!/bin/bash
pid=`head -1 $PGDATA/postmaster.pid`
echo "Pid: $pid"
peak=`grep ^VmPeak /proc/$pid/status | awk '{ print $2 }'`
echo "VmPeak: $peak kB"
hps=`grep ^Hugepagesize /proc/meminfo | awk '{ print $2 }'`
echo "Hugepagesize: $hps kB"
hp=$((peak/hps))
echo Set Huge Pages: $hp
[root@centos sysctl.d]# /tmp/hp.sh
Pid: 1439

9. VmPeak: 397356 kB
Hugepagesize: 2048 kB
Set Huge Pages: 194
#So currently postgres has peaked to 388 MB , Our Huge Page size is 2MB
#Let configure postgres Database cluster to use 4GB of Memory with huge page
of 2MB
#Configure Kernel Parameters
File : /etc/sysctl.conf
[postgres@centos data]$ sudo vi /etc/sysctl.conf
vm.nr_hugepages = 2100 #Equals 4.1 GB
kernel.shmmax = 4294967296 #Equals 4 GB
kernel.shmall = 4194304 #Equals 4 MB
[postgres@centos data]$ sudo sysctl -p
vm.nr_hugepages = 2100 # Equals 4.1 GB
kernel.shmmax = 4294967296 #Equals 4 GB
kernel.shmall = 4194304 #Equals 4 MB
[postgres@centos data]$ free -g
total used free shared buff/cache
available
Mem: 7 4 2 0 0
2
Swap: 9 0 9
#stop Postgresql service
[postgres@centos data]$ sudo systemctl stop postgresql
[postgres@centos data]$ pg_ctl status -D /u01/pgsql/data
pg_ctl: no server running
#Edit the Shared buffer parameter to 4GB
[postgres@centos data]$ cat postgresql.conf|grep "shared_buffers = 4GB"
shared_buffers = 4GB # min 128Kb
#Start the Postgresql service
[postgres@centos data]$ sudo systemctl start postgresql
[postgres@centos data]$ pg_ctl status -D /u01/pgsql/data
pg_ctl: server is running (PID: 3830)
/usr/pgsql-12/bin/postgres "-D" "/u01/pgsql/data"
[postgres@centos data]$ sudo /tmp/hp.sh
Pid: 3830
VmPeak: 4571684
Hugepagesize: 2048 kB
Set Huge Pages: 2232