PostgreSQL 9.5
New capabilities, new features, new year
James Hanson
jhanson@freedomconsultinggroup.com
jamesphanson@yahoo.com
@jamey_hanson
Freedom Consulting Group
http://www.freedomconsultinggroup.com
11-Feb-16
PostgreSQL: mission capable database platform
PostgreSQL MigrationTeam 11-Feb-162
v.
PostgreSQL retains its position as the most advanced relational
database platform*
with unique NoSQL,
text, geospatial,
analytic, network,
and development
language capabilities.
* For data on a single-
server scale of a few TB.
} JSON CRUD operations and pretty-print
} Row-level security … saved for a later presentation
} UPSERT “UPDATE if matching, otherwise INSERT”
} Password complexity, expiration and management (EDB*)
} Re-sync replication partners after failover
} Foreign table constraints and inheritance
} New analytics for CUBE and ROLLUP with GROUP BY
} New versions of PostGIS with added geospatial tools
} Lots and lots of administrative improvements
*EnterpriseDB PostgreSQL Plus Advanced Server
PostgreSQL and PPAS* new capabilities
PostgreSQL MigrationTeam 11-Feb-163
} PostgreSQL now speaks
JSON* fluently.
You have two choices
for full JSON support:
MongoDB
and PostgreSQL
We are JSONic!
PostgreSQL MigrationTeam 11-Feb-164
* and GeoJSON too!
} Why is this such a really, really big deal?
} From Gartner Group’s 2015 Magic Quadrant on Operational Databases
By 2017,the“NoSQL” label will cease to distinguish DBMSs,
which will reduce its value and result in it falling out of use. By
2017,all leading operational DBMSs will offer multiple data
models,relational and NoSQL, in a single platform.
}
* Oracle and MySQL include a read-only JSON data type, but no UPDATE functions. MS
SQLServer can present SQL results as JSON, but there is no JSON data type.
JSON CRUD operations and pretty-print
PostgreSQL MigrationTeam 11-Feb-165
PostgreSQL is for first, best and
only RDBMS* to support CRUD
and pretty-print on JSON.
} JSON documents in RDBMSs read-only and indexable.
The model was …
} Create a JSON document
} Store and index it in the database
} Retreive the JSON document
} Make changes to the document in the application
} Put the changed JSON back in the database
} PostgreSQL 9.5 adds ...
} UPDATE values,included array elements and nested JSON
} REMOVE nulls,which are common when converting from
RDBMS
} ADD elements or UPDATE if the element already exists
JSON CRUD operations
PostgreSQL MigrationTeam 11-Feb-166
} JSON is intended to be both machine and human readable
} But a JSON document is all on one line, without returns … not
very human readable
} Traditionally,JSON is “prettied” in the applicaiton code of with
MongoDB
} PostgreSQL 9.5 includes JSONB_PRETTY(), which
presents JSON in human-readable form
} Example from a table of airport JSON documents:
SELECT JSONB_PRETTY(airports)
FROM airports_json
WHERE airports ->> 'ident' = 'KCGS';
JSON pretty-print
PostgreSQL MigrationTeam 11-Feb-167
College Park Airport, KCGS
8
{
"name": "College Park Airport",
"type": "small_airport",
"ident": "KCGS",
"region": "Maryland",
"country": "United States",
"elevation_ft": 48,
"airport_keywords": [
"DC3",
"ADIZ"
],
"location": {
"type": "Point",
"coordinates": [
-76.9223022461,
38.9805984497
]
},
"airport_home_link": "http://www.collegeparkairport.aero/",
"runways": [
{
"he_ident": "33",
"length_ft": 2607,
"he_location": {
"type": "Point",
"coordinates": [
-76.9192,
38.9779
} Database user accounts can have a security-function
applied to all of their SQL
} “tables can have row security policies that restrict,on a per-user
basis,which rows can be returned by normal queries or
inserted,updated,or deleted by data modification commands”
PostgreSQL 9.5 documentation, 5.7 Row Security Policies
Row-level security … for a later presentation
PostgreSQL MigrationTeam 11-Feb-169
} INSERT operator adds optional ON CONFLICT clause
} DO NOTHING
} DO UPDATE SET …
COLUMN_NAME = <value_expression_or_subselect>
WHERE ...
} Ex.
CREATE TABLE distributors (
id BIGSERIAL PRIMARY KEY,
name TEXT);
INSERT INTO distributors (name) VALUES
('Kirkland'),
('Peapod');
UPSERT: “UPDATE if matching, otherwise INSERT”
PostgreSQL MigrationTeam 11-Feb-1610
} Option I: if the record exists, do nothing
INSERT INTO distributors (id, name) VALUES
(2, 'Nature''s Promise')
ON CONFLICT (id) DO NOTHING;
} Option II: if the id exists, update the name
INSERT INTO distributors (id, name) VALUES
(2, 'Nature''s Promise')
ON CONFLICT (id) DO UPDATE
SET name = 'Nature''s Promise';
UPSERT: “UPDATE if matching, otherwise INSERT”
PostgreSQL MigrationTeam 11-Feb-1611
} Accounts get PROFILEs, which can define …
} Failed login
attempts
} Password
complexity
function,
re-use,
expiration,
etc.
Password PROFILES for complexity and mgmt. (EDB)
PostgreSQL MigrationTeam 11-Feb-1612
} DEFAULT profile specifies
FAILED_LOGIN_ATTEMPTS UNLIMITED
PASSWORD_LOCK_TIME UNLIMITED
PASSWORD_LIFE_TIME UNLIMITED
PASSWEORD_GRACE_TIME UNLIMITED
PASSWORD_REUSE_TIME UNLIMITED
PASSWORD_REUSE_MAX UNLIMITED
PASSWORD_VERIFY_FUNCTION NULL
} Database superusers can ALTER PROFILE
} Functional equivalent and 100% syntactically compatible
with Oracle RDBMS
Password PROFILES for complexity and mgmt. (EDB)
PostgreSQL MigrationTeam 11-Feb-1613
} Previously, PostgreSQL audit logs only included the
database username, such as JBOSS_USER
} Logged the database user,not the person using the application
} edb_audit_tagallows the application to “tag” each
database session with an identifier that is written to the
csv or xml audit log
} Similar to Oracle application_context
Audit application user (EDB)
PostgreSQL MigrationTeam 11-Feb-1614
vs.
} UTL_RAW package
UTL_HTTP
.WRITE_LINE &
.WRITE_TEXT
DBMS_SESSION.
.SET_ROLE
} WAIT n seconds in FOR UPDATE statements
} Limits statement failures caused by locks
(Even) more Oracle compatibility (EDB)
PostgreSQL MigrationTeam 11-Feb-1615
} PostgreSQL already has simple, stable and fast replication
} Automatic or manual failover in MASTER => SLAVE
partnerships
} Now, the original MASTER database can be re-
synchronized after a failover without a full database copy
using pg_rewind
} Makes replication failover a much smaller deal
} Suitable for testing,patches,upgrades and other new business
cases
Re-sync replication partners after failover
PostgreSQL MigrationTeam 11-Feb-1616
} External data sources can appear as PostgreSQL tables,
including constraints and inheritance
} OS files,including JSON!
} Other RDBMSs (Oracle, MySQL, etc.) via O/JDBC
} Anything you have a connector for (pgbson for MongoDB)
} Constraints are not enforced, but it means that a foreign
table can be part of a foreign key
Foreign table constraints and inheritance
PostgreSQL MigrationTeam 11-Feb-1617
} GROUP BY multiple columns …
SELECT * FROM items_sold;
brand | size | sales
Foo | L | 10
Foo | M | 20
Bar | M | 15
Bar | L | 5
SELECT brand, size, sum(sales)
FROM items_sold
GROUP BY GROUPING SETS ((brand), (size), ());
brand | size | sum
Foo | | 30
Bar | | 20
| L | 15
| M | 35
| | 50
New analytics for CUBE & ROLLUP with GROUP BY
PostgreSQL MigrationTeam 11-Feb-1618
1`
} More capabilities for aggregate functions using all sorts of
combinations of columns and values …
} Plus native support for many languages ...
New analytics for CUBE & ROLLUP with GROUP BY
PostgreSQL MigrationTeam 11-Feb-1619
} PostGIS 2.2 installed with PostgreSQL and PPAS 9.5
included with PostgreSQL. Simpler &
cheaper the Oracle and ESRI
} ST_SubDivide divides
large shapes
} 3D operators for Known Nearest
Neighbor (KNN)
} Compressed output toTiny
Well Known Text (TWKB)
} GeoJSON generator
} More & faster functions for geometry data type
New PostGIS with added geospatial tools
PostgreSQL MigrationTeam 11-Feb-1620
} New index type for sorted ranges of data (like
timestamps) BRIN
} Select sample data from
large tables
} Speed up CREATE INDEX
} SSL diagnostic view
} View into progress of
replication
} Hint suggestions for
mistyped column names
} SELECT with skip locked option
Lots and lots of administrative improvements
PostgreSQL MigrationTeam 11-Feb-1621
} Example
SELECT elevation
FROM airports
LIMIT 10;
… but the actual column is airports.elevation_ft
ERROR: column "elevation" does not exist
HINT: Perhaps you meant to reference the
column "airports.location" or the column
"airports.elevation_ft".
Hint suggestions for mistyped column names
PostgreSQL MigrationTeam 11-Feb-1622
} JSON CRUD operations and pretty-print
} Row-level security … saved for a later presentation
} UPSERT“UPDATE if matching, otherwise INSERT”
} Password complexity, expiration and management (EDB*)
} Re-sync replication partners after failover
} Foreign table constraints and inheritance
} New analytics for CUBE and ROLLUP with GROUP BY
} New versions of PostGIS with added geospatial tools
} Lots and lots of administrative improvements
*EnterpriseDB PostgreSQL Plus Advanced Server
PostgreSQL and PPAS new capabilities
PostgreSQL MigrationTeam 11-Feb-1623
Are
there
any
questions or
follow up?
24

Pg 95 new capabilities

  • 1.
    PostgreSQL 9.5 New capabilities,new features, new year James Hanson jhanson@freedomconsultinggroup.com jamesphanson@yahoo.com @jamey_hanson Freedom Consulting Group http://www.freedomconsultinggroup.com 11-Feb-16
  • 2.
    PostgreSQL: mission capabledatabase platform PostgreSQL MigrationTeam 11-Feb-162 v. PostgreSQL retains its position as the most advanced relational database platform* with unique NoSQL, text, geospatial, analytic, network, and development language capabilities. * For data on a single- server scale of a few TB.
  • 3.
    } JSON CRUDoperations and pretty-print } Row-level security … saved for a later presentation } UPSERT “UPDATE if matching, otherwise INSERT” } Password complexity, expiration and management (EDB*) } Re-sync replication partners after failover } Foreign table constraints and inheritance } New analytics for CUBE and ROLLUP with GROUP BY } New versions of PostGIS with added geospatial tools } Lots and lots of administrative improvements *EnterpriseDB PostgreSQL Plus Advanced Server PostgreSQL and PPAS* new capabilities PostgreSQL MigrationTeam 11-Feb-163
  • 4.
    } PostgreSQL nowspeaks JSON* fluently. You have two choices for full JSON support: MongoDB and PostgreSQL We are JSONic! PostgreSQL MigrationTeam 11-Feb-164 * and GeoJSON too!
  • 5.
    } Why isthis such a really, really big deal? } From Gartner Group’s 2015 Magic Quadrant on Operational Databases By 2017,the“NoSQL” label will cease to distinguish DBMSs, which will reduce its value and result in it falling out of use. By 2017,all leading operational DBMSs will offer multiple data models,relational and NoSQL, in a single platform. } * Oracle and MySQL include a read-only JSON data type, but no UPDATE functions. MS SQLServer can present SQL results as JSON, but there is no JSON data type. JSON CRUD operations and pretty-print PostgreSQL MigrationTeam 11-Feb-165 PostgreSQL is for first, best and only RDBMS* to support CRUD and pretty-print on JSON.
  • 6.
    } JSON documentsin RDBMSs read-only and indexable. The model was … } Create a JSON document } Store and index it in the database } Retreive the JSON document } Make changes to the document in the application } Put the changed JSON back in the database } PostgreSQL 9.5 adds ... } UPDATE values,included array elements and nested JSON } REMOVE nulls,which are common when converting from RDBMS } ADD elements or UPDATE if the element already exists JSON CRUD operations PostgreSQL MigrationTeam 11-Feb-166
  • 7.
    } JSON isintended to be both machine and human readable } But a JSON document is all on one line, without returns … not very human readable } Traditionally,JSON is “prettied” in the applicaiton code of with MongoDB } PostgreSQL 9.5 includes JSONB_PRETTY(), which presents JSON in human-readable form } Example from a table of airport JSON documents: SELECT JSONB_PRETTY(airports) FROM airports_json WHERE airports ->> 'ident' = 'KCGS'; JSON pretty-print PostgreSQL MigrationTeam 11-Feb-167
  • 8.
    College Park Airport,KCGS 8 { "name": "College Park Airport", "type": "small_airport", "ident": "KCGS", "region": "Maryland", "country": "United States", "elevation_ft": 48, "airport_keywords": [ "DC3", "ADIZ" ], "location": { "type": "Point", "coordinates": [ -76.9223022461, 38.9805984497 ] }, "airport_home_link": "http://www.collegeparkairport.aero/", "runways": [ { "he_ident": "33", "length_ft": 2607, "he_location": { "type": "Point", "coordinates": [ -76.9192, 38.9779
  • 9.
    } Database useraccounts can have a security-function applied to all of their SQL } “tables can have row security policies that restrict,on a per-user basis,which rows can be returned by normal queries or inserted,updated,or deleted by data modification commands” PostgreSQL 9.5 documentation, 5.7 Row Security Policies Row-level security … for a later presentation PostgreSQL MigrationTeam 11-Feb-169
  • 10.
    } INSERT operatoradds optional ON CONFLICT clause } DO NOTHING } DO UPDATE SET … COLUMN_NAME = <value_expression_or_subselect> WHERE ... } Ex. CREATE TABLE distributors ( id BIGSERIAL PRIMARY KEY, name TEXT); INSERT INTO distributors (name) VALUES ('Kirkland'), ('Peapod'); UPSERT: “UPDATE if matching, otherwise INSERT” PostgreSQL MigrationTeam 11-Feb-1610
  • 11.
    } Option I:if the record exists, do nothing INSERT INTO distributors (id, name) VALUES (2, 'Nature''s Promise') ON CONFLICT (id) DO NOTHING; } Option II: if the id exists, update the name INSERT INTO distributors (id, name) VALUES (2, 'Nature''s Promise') ON CONFLICT (id) DO UPDATE SET name = 'Nature''s Promise'; UPSERT: “UPDATE if matching, otherwise INSERT” PostgreSQL MigrationTeam 11-Feb-1611
  • 12.
    } Accounts getPROFILEs, which can define … } Failed login attempts } Password complexity function, re-use, expiration, etc. Password PROFILES for complexity and mgmt. (EDB) PostgreSQL MigrationTeam 11-Feb-1612
  • 13.
    } DEFAULT profilespecifies FAILED_LOGIN_ATTEMPTS UNLIMITED PASSWORD_LOCK_TIME UNLIMITED PASSWORD_LIFE_TIME UNLIMITED PASSWEORD_GRACE_TIME UNLIMITED PASSWORD_REUSE_TIME UNLIMITED PASSWORD_REUSE_MAX UNLIMITED PASSWORD_VERIFY_FUNCTION NULL } Database superusers can ALTER PROFILE } Functional equivalent and 100% syntactically compatible with Oracle RDBMS Password PROFILES for complexity and mgmt. (EDB) PostgreSQL MigrationTeam 11-Feb-1613
  • 14.
    } Previously, PostgreSQLaudit logs only included the database username, such as JBOSS_USER } Logged the database user,not the person using the application } edb_audit_tagallows the application to “tag” each database session with an identifier that is written to the csv or xml audit log } Similar to Oracle application_context Audit application user (EDB) PostgreSQL MigrationTeam 11-Feb-1614 vs.
  • 15.
    } UTL_RAW package UTL_HTTP .WRITE_LINE& .WRITE_TEXT DBMS_SESSION. .SET_ROLE } WAIT n seconds in FOR UPDATE statements } Limits statement failures caused by locks (Even) more Oracle compatibility (EDB) PostgreSQL MigrationTeam 11-Feb-1615
  • 16.
    } PostgreSQL alreadyhas simple, stable and fast replication } Automatic or manual failover in MASTER => SLAVE partnerships } Now, the original MASTER database can be re- synchronized after a failover without a full database copy using pg_rewind } Makes replication failover a much smaller deal } Suitable for testing,patches,upgrades and other new business cases Re-sync replication partners after failover PostgreSQL MigrationTeam 11-Feb-1616
  • 17.
    } External datasources can appear as PostgreSQL tables, including constraints and inheritance } OS files,including JSON! } Other RDBMSs (Oracle, MySQL, etc.) via O/JDBC } Anything you have a connector for (pgbson for MongoDB) } Constraints are not enforced, but it means that a foreign table can be part of a foreign key Foreign table constraints and inheritance PostgreSQL MigrationTeam 11-Feb-1617
  • 18.
    } GROUP BYmultiple columns … SELECT * FROM items_sold; brand | size | sales Foo | L | 10 Foo | M | 20 Bar | M | 15 Bar | L | 5 SELECT brand, size, sum(sales) FROM items_sold GROUP BY GROUPING SETS ((brand), (size), ()); brand | size | sum Foo | | 30 Bar | | 20 | L | 15 | M | 35 | | 50 New analytics for CUBE & ROLLUP with GROUP BY PostgreSQL MigrationTeam 11-Feb-1618
  • 19.
    1` } More capabilitiesfor aggregate functions using all sorts of combinations of columns and values … } Plus native support for many languages ... New analytics for CUBE & ROLLUP with GROUP BY PostgreSQL MigrationTeam 11-Feb-1619
  • 20.
    } PostGIS 2.2installed with PostgreSQL and PPAS 9.5 included with PostgreSQL. Simpler & cheaper the Oracle and ESRI } ST_SubDivide divides large shapes } 3D operators for Known Nearest Neighbor (KNN) } Compressed output toTiny Well Known Text (TWKB) } GeoJSON generator } More & faster functions for geometry data type New PostGIS with added geospatial tools PostgreSQL MigrationTeam 11-Feb-1620
  • 21.
    } New indextype for sorted ranges of data (like timestamps) BRIN } Select sample data from large tables } Speed up CREATE INDEX } SSL diagnostic view } View into progress of replication } Hint suggestions for mistyped column names } SELECT with skip locked option Lots and lots of administrative improvements PostgreSQL MigrationTeam 11-Feb-1621
  • 22.
    } Example SELECT elevation FROMairports LIMIT 10; … but the actual column is airports.elevation_ft ERROR: column "elevation" does not exist HINT: Perhaps you meant to reference the column "airports.location" or the column "airports.elevation_ft". Hint suggestions for mistyped column names PostgreSQL MigrationTeam 11-Feb-1622
  • 23.
    } JSON CRUDoperations and pretty-print } Row-level security … saved for a later presentation } UPSERT“UPDATE if matching, otherwise INSERT” } Password complexity, expiration and management (EDB*) } Re-sync replication partners after failover } Foreign table constraints and inheritance } New analytics for CUBE and ROLLUP with GROUP BY } New versions of PostGIS with added geospatial tools } Lots and lots of administrative improvements *EnterpriseDB PostgreSQL Plus Advanced Server PostgreSQL and PPAS new capabilities PostgreSQL MigrationTeam 11-Feb-1623
  • 24.