The document outlines key developments in PostgreSQL from its early versions to the latest updates, including features and improvements introduced in version 9.3 such as materialized views, updatable views, and JSON support. It also touches on market trends in cloud computing, NoSQL, and big data, highlighting the operational challenges and innovations in managing large datasets. Additionally, there is a call for continued focus on fault tolerance and analytics in future PostgreSQL developments.

1. Past, Present, and Pachyderm
/ Robert Treat
Friday, October 25, 13

2. Hello
@robtreat2
Friday, October 25, 13

3. whoami?
COO @ OMNITI
“for companies who need to build scalable and efficient
web services, OmniTI is the leading cross-disciplinary
provider of consulting, development, and operational
resources to help your company grow”
10% of Alexa Top 100
Friday, October 25, 13

4. whoami?
Postgres 6.x
Friday, October 25, 13

5. whoami?
Major Contributor
advocacy, tools, web
Friday, October 25, 13

6. Past
Postgres
Friday, October 25, 13

7. Past
Major Themes
open source
extensibility
data safety
SQL
“bsd” licensed
Friday, October 25, 13

8. Past
Berkely POSTGRES
Postgres95
PostgreSQL (6.x)
Friday, October 25, 13

9. Past
7.0 - Foreign Keys
7.1 - WAL, Outer Joins
7.2 - Non-Blocking Vacuum
7.3 - Schemas
7.4 - Aggregates
Friday, October 25, 13

10. Past
8.0 - Windows, Point-In-Time Recovery
8.1 - autovacuum, partitioning
8.2 - warm standby
8.3 - remove xid wrap issue for read only work flow
8.4 - Windowing Functions / CTE / Recursive
Friday, October 25, 13

11. Past
9.0 - Streaming Replication / Hot Standby
9.1 - FDW, synch-rep
9.2 - cascading slaves, index oriented scans
Friday, October 25, 13

12. Present
What’s New In 9.3?
ASIDE: I will be speaking on this topic at PGConf.EU next week in Dublin. If you are there please come and say hi.
Friday, October 25, 13

13. Present
What’s New In 9.3?
September 9th, 2013
ASIDE: I will be speaking on this topic at PGConf.EU next week in Dublin. If you are there please come and say hi.
Friday, October 25, 13

14. Present
materialized views
combining rule definitions of view (for updating) with heap
properties of a table (used by planner)
Friday, October 25, 13

15. Present
updatable views
views that reference some set of columns from a single base
table are now updatable by default.
more complex views can be made updatable using INSTEAD
OF triggers or INSTEAD rules.
Friday, October 25, 13

16. Present
LATERAL Queries
SELECT
customer_id, rentals
FROM
customer c,
LATERAL(select count(distinct(rental_date)) as rentals
from rental where customer_id = c.customer_id) as x
WHERE
store_id = 1
ORDER BY rentals desc LIMIT 5
first_name | last_name | rentals
------------+-----------+--------ELEANOR
| HUNT
|
46
MARCIA
| DEAN
|
42
CLARA
| SHAW
|
42
TIM
| CARY
|
39
DAISY
| BATES
|
38
Friday, October 25, 13

17. Present
Postgres FDW
Friday, October 25, 13

18. Present
Foreign Data Wrappers
•
•
•
•
•
•
•
•
Friday, October 25, 13
csv
oracle
mysql
redis
mongodb
odbc
hadoop
twitter

19. Present
Postgres FDW
• access remote postgres databases
• supports remote cost estimates
• basic qual pushdown support
• reference implementation
Friday, October 25, 13

20. Present
writeable FDW
• ability to write to foreign tables
• insert / update / delete
• transaction aware(!)
• speed can still be an issue
Friday, October 25, 13

21. Present
improved FK locking
OLD:
• UPDATE to parent required “for update” row lock
• UPDATE to child required “for share” row lock
NEW:
• UPDATE to parent of non-keys == “for no-key update”
• UPDATE to child == “for key share”
no blocking unless a foreign-key column is changed
Friday, October 25, 13

22. Present
“timeline switching”
allows streaming replicants (slaves)
to follow new primary nodes (masters)
without need for log shipping
Friday, October 25, 13

23. Present
JSON
• Add operators and functions to extract
elements from JSON values
• Allow JSON values to be converted into
records
• Add functions to convert scalars, records,
and hstore values to JSON
Friday, October 25, 13

24. Present
JSON
• json_agg(anyrecord) -> json
• to_json(any) -> json
• hstore_to_json(hstore) -> json (also used as a cast)
• hstore_to_json_loose(hstore) -> json
• SELECT b->’f1’AS f1, b->’f3’AS f3 FROM aa WHERE a = 1
• json_each, json_each_text
• json_extract_path, json_extract_path_text
• json_object_keys
• json_populate_record, json_populate_recordset
• json_array_length
• json_array_elements
Friday, October 25, 13

25. Present
background worker processes
allows users to build custom background processes
access to C code level constructs
access to shared memory
use backend database connections
Friday, October 25, 13

26. Present
background worker processes
allows users to build custom background processes
access to C code level constructs
access to shared memory
use backend database connections
disconnect idle connections
custom statistics / logging
automated reindex / cluster
Friday, October 25, 13

27. Present
“mongres”
background workers listening for mongo wire protocol
storage and interaction with json data
https://github.com/umitanuki/mongres
Friday, October 25, 13

28. Pachyderm
Market Trends
*CLOUD*
aws
joyent
etc...
chef
puppet
etc...
best practices:
bad i/o
nodes go away
Friday, October 25, 13
distributed
system?

29. Pachyderm
Market Trends
*NoSQL*
200+ Options == Operational Overload
SQL-like languages being added
specialized use cases
Friday, October 25, 13

30. Pachyderm
Market Trends
*BIG DATA*
Scaling Postgres : 10TB
workload vs hardware
federation aka “sharding”
Friday, October 25, 13

31. Pachyderm
Market Trends
*BIG DATA*
BIG is not about size :: “Business Impact Generating”
Data Analytics <=> Advanced SQL
Friday, October 25, 13

32. Pachyderm
9.4 / 9.5
logical replication
master-master replication
Friday, October 25, 13

33. Pachyderm
What Should We Be Working On?
Fault Tolerance
Column Store
Analytics
GUI’s
Friday, October 25, 13

34. Thanks!
more:
@robtreat2
http://xzilla.net
@omniti
http://omniti.com
Postgres Services
24x7 Monitoring
RemoteDBA Services
Oracle -> Postgres Migration
Custom Postgres Consulting
Friday, October 25, 13