This document discusses automating the provisioning and management of a highly available PostgreSQL database on Amazon EC2 using Terraform and Ansible. It covers designing the infrastructure with EC2 instances, installing and configuring PostgreSQL for replication and high availability, setting up backups, monitoring, and automating failover using PgPool and watchdogs. The tools discussed help automate the entire lifecycle including initial provisioning, upgrades, backups, and disaster recovery for a scalable and reliable PostgreSQL deployment on AWS.

1. Provisioning and Automating High
Availability Postgres on AWS EC2
Payal Singh
DBA@OmniTI
1

2. Who am I?
DBA@OmniTI
Github: payals
Blog: http://penningpence.blogspot.com
Twitter: @pallureshu
Email: payal@omniti.com
2

3. Agenda
RDS vs EC2
Design
Provisioning Tools
Automate
3

4. Why EC2 over RDS?
Control
•Debugging
•Third party tools
•Resources
Flexibility
•Migration - globals
•Compatibility with non-AWS resources
•Monitoring
4

5. If you’re going from RDS to EC2...
More administration
Less handholding
Security
Basic monitoring
5

6. Automation Tools
6

7. Popular Automation Tools
7

8. Types of Tools
8

9. What is Terraform?
Tool for building, changing and versioning infrastructure
Specializes in cloud infrastructure management
9

10. 10

11. Why I chose Ansible
Simple
Learning curve
Galaxy and modules
Push - more convenient
Clarity - YAML
11

12. Design
12

13. What to Design?
EC2 Setup
Installs and reinstalls, dependencies
Tuning and customization
Replication
Backups
Monitoring
Failover
13

14. 14
Is there any such thing as a
In Postgres automation?

15. 15
YES

16. 16

17. 17

18. 18

19. 19

20. 20

21. Setup
and
Initialization
21

22. EC2 Setup - variables
22

23. EC2 Setup - main.tf
23

24. EC2 Setup - main.tf optionals
24

25. Terraform apply

26. Terraform applied!

27. Installation

28. Main YAML File

29. Installs
Get the repository:
Installing postgres and supporting packages:
postgresql_version: "9.6" 29

30. Re-installs
30

31. Customization
and Tuning

32. Customization
$PGDATA
Extensions
32

33. Tuning - postgresql.conf
Template files:
Attributes:
33

34. Tuning - pg_hba.conf
Template:
34

35. Tuning pg_hba.conf
Values:
File:
35

36. Ansible-playbook main.yml

39. Replication

40. Replication
40

41. Replication – recovery.conf

42. Ansible and replication
Groups:
42

43. Pgpass file
43

46. Slave Rebuild
Switch
Base backup
46

47. pg_rewind
Synchronize a PostgreSQL cluster with another copy of the same cluster,
after the clusters' timelines have diverged
9.5 and above
Superuser access
wal_log_hints = 'on' or enable data checksums
47

48. Backups

49. Backups and restores - Setup
PITR:
S3:
49

50. Backups
50

51. Restores
51

52. Configure PITR
52

53. WAL-e dependencies
53

54. Cronjobs
54

55. Monitoring
and
Alerts

56. Custom Monitoring
http://docs.ansible.com/ansible/list_of_monitoring_modules.html

57. CloudWatch Monitoring

58. Failover

59. Failover
Boolean parameter:
promote_slave: False
59

61. PgPool
and

62. Completely Automated Failover
62

63. Pgpool

64. PgPool

65. Watchdog

66. Watchdog

67. PgPool + Watchdog

68. Virtual IPs
Elastic IP in AWS:
68

69. Avoid
a Disaster

70. Useful Links
https://github.com/payals/postgresql_automation
https://galaxy.ansible.com/
http://docs.ansible.com/
https://www.terraform.io/docs/
70

71. Questions?
71