Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Past, Present, and Pachyderm

/ Robert Treat

Friday, October 25, 13
Hello

@robtreat2

Friday, October 25, 13
whoami?

COO @ OMNITI
“for companies who need to build scalable and efficient
web services, OmniTI is the leading cross-dis...
whoami?

Postgres 6.x

Friday, October 25, 13
whoami?

Major Contributor
advocacy, tools, web

Friday, October 25, 13
Past

Postgres

Friday, October 25, 13
Past

Major Themes
open source

extensibility
data safety
SQL

“bsd” licensed

Friday, October 25, 13
Past

Berkely POSTGRES
Postgres95
PostgreSQL (6.x)

Friday, October 25, 13
Past

7.0 - Foreign Keys
7.1 - WAL, Outer Joins
7.2 - Non-Blocking Vacuum
7.3 - Schemas
7.4 - Aggregates

Friday, October ...
Past

8.0 - Windows, Point-In-Time Recovery
8.1 - autovacuum, partitioning
8.2 - warm standby
8.3 - remove xid wrap issue ...
Past

9.0 - Streaming Replication / Hot Standby
9.1 - FDW, synch-rep
9.2 - cascading slaves, index oriented scans

Friday,...
Present

What’s New In 9.3?

ASIDE: I will be speaking on this topic at PGConf.EU next week in Dublin. If you are there pl...
Present

What’s New In 9.3?

September 9th, 2013

ASIDE: I will be speaking on this topic at PGConf.EU next week in Dublin...
Present

materialized views

combining rule definitions of view (for updating) with heap
properties of a table (used by pla...
Present

updatable views

views that reference some set of columns from a single base
table are now updatable by default.
...
Present

LATERAL Queries
SELECT
customer_id, rentals
FROM
customer c,
LATERAL(select count(distinct(rental_date)) as renta...
Present

Postgres FDW

Friday, October 25, 13
Present

Foreign Data Wrappers
•
•
•
•
•
•
•
•

Friday, October 25, 13

csv
oracle
mysql
redis
mongodb
odbc
hadoop
twitter
Present

Postgres FDW

• access remote postgres databases
• supports remote cost estimates
• basic qual pushdown support
•...
Present

writeable FDW

• ability to write to foreign tables
• insert / update / delete
• transaction aware(!)
• speed can...
Present

improved FK locking
OLD:
• UPDATE to parent required “for update” row lock
• UPDATE to child required “for share”...
Present

“timeline switching”
allows streaming replicants (slaves)
to follow new primary nodes (masters)
without need for ...
Present

JSON
• Add operators and functions to extract
elements from JSON values
• Allow JSON values to be converted into
...
Present

JSON
• json_agg(anyrecord) -> json
• to_json(any) -> json
• hstore_to_json(hstore) -> json (also used as a cast)
...
Present

background worker processes
allows users to build custom background processes
access to C code level constructs
a...
Present

background worker processes
allows users to build custom background processes
access to C code level constructs
a...
Present

“mongres”

background workers listening for mongo wire protocol
storage and interaction with json data

https://g...
Pachyderm

Market Trends
*CLOUD*
aws
joyent
etc...

chef
puppet
etc...
best practices:
bad i/o
nodes go away

Friday, Octo...
Pachyderm

Market Trends
*NoSQL*
200+ Options == Operational Overload
SQL-like languages being added
specialized use cases...
Pachyderm

Market Trends
*BIG DATA*
Scaling Postgres : 10TB
workload vs hardware
federation aka “sharding”

Friday, Octobe...
Pachyderm

Market Trends
*BIG DATA*
BIG is not about size :: “Business Impact Generating”
Data Analytics <=> Advanced SQL
...
Pachyderm

9.4 / 9.5

logical replication
master-master replication

Friday, October 25, 13
Pachyderm

What Should We Be Working On?
Fault Tolerance
Column Store
Analytics
GUI’s

Friday, October 25, 13
Thanks!

more:
@robtreat2
http://xzilla.net

@omniti
http://omniti.com

Postgres Services
24x7 Monitoring
RemoteDBA Servic...
Upcoming SlideShare
Loading in …5
×

Past, Present, and Pachyderm - All Things Open - 2013

6,871 views

Published on

Slides based on my talk at the All Things Open conference, held in Raleigh, North Carolina. This talk covers some basic history on Postgres, new features in the 9.3 release, and some thoughts on what might be in the future for Postgres.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Past, Present, and Pachyderm - All Things Open - 2013

  1. 1. Past, Present, and Pachyderm / Robert Treat Friday, October 25, 13
  2. 2. Hello @robtreat2 Friday, October 25, 13
  3. 3. whoami? COO @ OMNITI “for companies who need to build scalable and efficient web services, OmniTI is the leading cross-disciplinary provider of consulting, development, and operational resources to help your company grow” 10% of Alexa Top 100 Friday, October 25, 13
  4. 4. whoami? Postgres 6.x Friday, October 25, 13
  5. 5. whoami? Major Contributor advocacy, tools, web Friday, October 25, 13
  6. 6. Past Postgres Friday, October 25, 13
  7. 7. Past Major Themes open source extensibility data safety SQL “bsd” licensed Friday, October 25, 13
  8. 8. Past Berkely POSTGRES Postgres95 PostgreSQL (6.x) Friday, October 25, 13
  9. 9. Past 7.0 - Foreign Keys 7.1 - WAL, Outer Joins 7.2 - Non-Blocking Vacuum 7.3 - Schemas 7.4 - Aggregates Friday, October 25, 13
  10. 10. Past 8.0 - Windows, Point-In-Time Recovery 8.1 - autovacuum, partitioning 8.2 - warm standby 8.3 - remove xid wrap issue for read only work flow 8.4 - Windowing Functions / CTE / Recursive Friday, October 25, 13
  11. 11. Past 9.0 - Streaming Replication / Hot Standby 9.1 - FDW, synch-rep 9.2 - cascading slaves, index oriented scans Friday, October 25, 13
  12. 12. Present What’s New In 9.3? ASIDE: I will be speaking on this topic at PGConf.EU next week in Dublin. If you are there please come and say hi. Friday, October 25, 13
  13. 13. Present What’s New In 9.3? September 9th, 2013 ASIDE: I will be speaking on this topic at PGConf.EU next week in Dublin. If you are there please come and say hi. Friday, October 25, 13
  14. 14. Present materialized views combining rule definitions of view (for updating) with heap properties of a table (used by planner) Friday, October 25, 13
  15. 15. Present updatable views views that reference some set of columns from a single base table are now updatable by default. more complex views can be made updatable using INSTEAD OF triggers or INSTEAD rules. Friday, October 25, 13
  16. 16. Present LATERAL Queries SELECT customer_id, rentals FROM customer c, LATERAL(select count(distinct(rental_date)) as rentals from rental where customer_id = c.customer_id) as x WHERE store_id = 1 ORDER BY rentals desc LIMIT 5 first_name | last_name | rentals ------------+-----------+--------ELEANOR | HUNT | 46 MARCIA | DEAN | 42 CLARA | SHAW | 42 TIM | CARY | 39 DAISY | BATES | 38 Friday, October 25, 13
  17. 17. Present Postgres FDW Friday, October 25, 13
  18. 18. Present Foreign Data Wrappers • • • • • • • • Friday, October 25, 13 csv oracle mysql redis mongodb odbc hadoop twitter
  19. 19. Present Postgres FDW • access remote postgres databases • supports remote cost estimates • basic qual pushdown support • reference implementation Friday, October 25, 13
  20. 20. Present writeable FDW • ability to write to foreign tables • insert / update / delete • transaction aware(!) • speed can still be an issue Friday, October 25, 13
  21. 21. Present improved FK locking OLD: • UPDATE to parent required “for update” row lock • UPDATE to child required “for share” row lock NEW: • UPDATE to parent of non-keys == “for no-key update” • UPDATE to child == “for key share” no blocking unless a foreign-key column is changed Friday, October 25, 13
  22. 22. Present “timeline switching” allows streaming replicants (slaves) to follow new primary nodes (masters) without need for log shipping Friday, October 25, 13
  23. 23. Present JSON • Add operators and functions to extract elements from JSON values • Allow JSON values to be converted into records • Add functions to convert scalars, records, and hstore values to JSON Friday, October 25, 13
  24. 24. Present JSON • json_agg(anyrecord) -> json • to_json(any) -> json • hstore_to_json(hstore) -> json (also used as a cast) • hstore_to_json_loose(hstore) -> json • SELECT b->’f1’AS f1, b->’f3’AS f3 FROM aa WHERE a = 1 • json_each, json_each_text • json_extract_path, json_extract_path_text • json_object_keys • json_populate_record, json_populate_recordset • json_array_length • json_array_elements Friday, October 25, 13
  25. 25. Present background worker processes allows users to build custom background processes access to C code level constructs access to shared memory use backend database connections Friday, October 25, 13
  26. 26. Present background worker processes allows users to build custom background processes access to C code level constructs access to shared memory use backend database connections disconnect idle connections custom statistics / logging automated reindex / cluster Friday, October 25, 13
  27. 27. Present “mongres” background workers listening for mongo wire protocol storage and interaction with json data https://github.com/umitanuki/mongres Friday, October 25, 13
  28. 28. Pachyderm Market Trends *CLOUD* aws joyent etc... chef puppet etc... best practices: bad i/o nodes go away Friday, October 25, 13 distributed system?
  29. 29. Pachyderm Market Trends *NoSQL* 200+ Options == Operational Overload SQL-like languages being added specialized use cases Friday, October 25, 13
  30. 30. Pachyderm Market Trends *BIG DATA* Scaling Postgres : 10TB workload vs hardware federation aka “sharding” Friday, October 25, 13
  31. 31. Pachyderm Market Trends *BIG DATA* BIG is not about size :: “Business Impact Generating” Data Analytics <=> Advanced SQL Friday, October 25, 13
  32. 32. Pachyderm 9.4 / 9.5 logical replication master-master replication Friday, October 25, 13
  33. 33. Pachyderm What Should We Be Working On? Fault Tolerance Column Store Analytics GUI’s Friday, October 25, 13
  34. 34. Thanks! more: @robtreat2 http://xzilla.net @omniti http://omniti.com Postgres Services 24x7 Monitoring RemoteDBA Services Oracle -> Postgres Migration Custom Postgres Consulting Friday, October 25, 13

×