From PGConf Asia 2019. How did PostgreSQL become the primary database we think of for new projects? What makes it different? This talk goes over the history of the project and what makes it different.

1. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
How PostgreSQL Became King
chris.travers@adjust.com
August 2019

2. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
About Me
My name is Chris Travers. I am a
long-time user of PostgreSQL. I have
worked as a database administrator at
least in part for over a decade. I have
even contributed a bug fix. I help
manage an environment where 5-10PB
of data is stored in PostgreSQL.

3. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
About Adjust
Adjust is an mobile advertisement
attribution company, focusing on
transparency and fairness. Our
attribution service is the basis of our
analytics offerings. (Offer to give a 30
sec intro to what we do.)
We are also a big user of PostgreSQL,
mostly on analytic workloads.

4. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
The Promise of Inferential Data Banks
Gathering data is good but what do you do with it?
• Make decisions
• Do statistical analysis
• Make correlations
Hence Relational Databases were born

5. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Taking this Further
Relational Databases are great but why be bound to relational
math?
• Extend data derivations in the database.
• Extend relational operations in the database.
• Example: Show me all pictures of sunsets within 50 km of
some landmark.
This idea takes off in Oracle, DB2, SQL Server, SQLite, and
Interbase, but not MySQL.

6. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
The Sales Pitch
At the time most developers were using MySQL and MyISAM
tables. I heard about PostgreSQL and the magical features it
offered in version 6.5:
• Transactions
• Triggers
• Concurrency
MySQL offered none of these things so I was intrigued.

7. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Hidden Presence
What I found out was that there was a huge presence already of
Postgres users.
• Business apps
• Research apps
• Government apps
I was intrigued....... I started writing my own application that
supported PostgreSQL

8. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
A Different Paradigm
What I found was significant:
PostgreSQL offered major advanced functionality that MySQL
didn’t and was easier to use than Oracle.
PostgreSQL also was a database for many applications rather than
a dumb application-specific data store.

9. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Major improvements in usability over next few years
PostgreSQL became easier to use:
• ALTER TABLE DROP COLUMN
• Declarative foreign keys
• Crash safety
• Composite Types as first-class types
• Better robustness of backups and recovery
• Beginnings of PostGIS

10. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Rise of Web Frameworks
The rise of web applications lead to a number of challenges:
• Datababase migrations must be robust and testable
• Recovering from a half-way successful migration is not
acceptable
• Transactional DDL separates PostgreSQL from MySQL

11. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
MySQL vs PostgreSQL
2005 Oracle acquires InnoBase
2005 EnterpriseDB founded to help migrate Oracle to
Postgres
2008 Sun Microsystems acquires MySQL AB
2010 Oracle acquires Sun Microsystems, MariaDB forked
Uncertainty with MySQL’s future drives a lot of PostgreSQL
growth.

12. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Beginnings of Industry 4.0
2005 PostGIS 1.0 released, opens up proximity searches
2006 PostgreSQL gets native array types
2007 First-class tuple types and arrays of tuples, allowing
N1NF designs
2008 Role-based security
2010 Native JSON data type

13. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
What about Oracle?
• Bad reputation as a vendor.
• Immense legacy problem (only major RDBMS without
transactional DDL)
• Limited set of areas where they can and want to innovate.

14. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Standard Definition of Big Data
One or more of the following making it difficult to scale within a
standard RDBMS on one system:
• Volume of data (shift from low TB into PBs)
• Velocity of ingestion
• Variety of data, meaning a lot of semi-structured data.
Many of PostgreSQL’s features let you handle variety issues in a
single server.

15. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Behind the Big Data Shift: Progressive Automation
1. Traditional database system with lots of manual entry
2. Data coming in now from external sources (EDI invoices and
orders)
3. Shift from data entry to automated entry with human
oversight
4. Eventually a shift to massive automated ingestion from a
variety of sources.
5. Many fields from scientific research to manufacturing and
delivery is now partially automated.

16. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Stonebreaker’s Triangle
In his sales pitches for VoltDB, Stonebreaker suggested:

17. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Big Data Approaches
• PostgresXC/XL/etc.
• Roll your own federated storage
• Data storage for data pipelines
• More exotic solutions using FDWs

18. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
The Future
Important things on the horizon

19. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Pluggable Storage
• Allows for storage optimized for different workloads
• In particular columnar storage (Z-store) helps some workloads
• Not extremely useful yet due to many things not being
abstracted
• Expect this to get much better in future releases

20. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Better Distributed Storage
• Two Phase Commit in Foreign Data Wrappers
• Better streaming support in Foreign Data Wrappers
• Asynchronous Append/merge-append across foreign tables

21. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Development State
• Impressive feature set
• Pace of development accelerating
• Platform-in-a-box features getting better quickly
• Approachable, easy, and safe codebase to work with.

22. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Benefits
• Lots of great help on IRC/StackOverflow/Lists
• A lot of industry expertise to help newcomers get started
• Awesome, helpful community
• For professionals, easy to find work/employment

23. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Some Drawbacks
• Hiring (We are hiring but so is literally everyone else)
• Sometimes people migrate without thinking about differences.
• That’s about it.....

24. Introduction Early Years 1999 1999-2003 Rails/DDL Market Shifts Big Data The Future It’s Good to Be King
Thanks
Thank you for coming. Feel free to send
comments/questions/feedback to
mailto:chris.travers@adjust.com