Postgres 9.4 First Look

/ Robert Treat
9.4 On The Floor
A Talk About Upcoming Features in Postgres 9.4
Thursday, May 22, 14

Hello
@robtreat2
xzilla.net

Hello
Postgres 6.x
Major Contributor
advocacy, tools, web

Hello
Postgres 6.x
Major Contributor
advocacy, tools, web
root beer : crocs : shawarma

Hello
“for companies who need to build scalable and efﬁcient
web services, OmniTI is the leading cross-disciplinary
provider of consulting, development, and operational
resources to help your company grow”
15% of Alexa Top 100
postgres consulting | web development

The Development Cycle
Postgres

• June14th,2013-Branch9.4-devel
• June2013-CF1
• September2013-CF2
• November2013-CF3
• January2014-CF4

• June14th,2013-Branch9.4-devel
• June2013-CF1
• September2013-CF2
• November2013-CF3
• January2014-CF4
• May15th,2014-Postgres9.4Beta1

https://wiki.postgresql.org/wiki/HowToBetaTest
Beta Release == Test Please!

So Whats New?
Performance
Administration
SQL
Backend
Future

Performance
reduce WAL for updates
Reduce the size of WAL records generated for UPDATE
operations
Less WAL volume should mean less I/O & resource usage

Performance
xlog insert scaling
Reduce lock contention for WAL insert operations,
improving scalability of WAL system

Performance
pg_prewarm
Contrib module for loading relation data into memory.
Can choose from OS cache (async or sync) or
shared_buffers.
SELECT pg_prewarm('actor');

Performance
GIN Index improvements
friday @ 4:15 | GIN stronger than ever
compact page format
2x - 10x reduced index size
fast scan improvements
queries w/ some freq. some rare items

Performance
separate planning / exec time in EXPLAIN ANALYZE
pagila=#explain analyze select count(*) from payment where payment_date<='2007-02-01'::date;
QUERY PLAN
--------------------------------------------------------------------------------------------
Aggregate (cost=72.26..72.27 rows=1 width=0) (actual time=11.712..11.712 rows=1 loops=1)
-> Append (cost=0.00..69.36 rows=1159 width=0) (actual time=2.256..11.416 rows=1157
loops=1)
-> Seq Scan on payment (cost=0.00..0.00 rows=1 width=0) (actual time=0.002..0.002
rows=0 loops=1)
Filter: (payment_date <= '2007-02-01'::date)
-> Seq Scan on payment_p2007_01 (cost=0.00..23.46 rows=1157 width=0) (actual
time=2.253..5.728 rows=1157 loops=1)
-> Seq Scan on payment_p2007_02 (cost=0.00..45.90 rows=1 width=0) (actual
time=5.380..5.380 rows=0 loops=1)
Rows Removed by Filter: 2312
Planning time: 140.555 ms
Execution time: 12.562 ms
(11 rows)

Performance
add “grouping columns” to EXPLAIN output
pagila=# explain select postal_code, count(*)
pagila-# from address group by postal_code;
QUERY PLAN
----------------------------------------------------------------
HashAggregate (cost=17.05..23.02 rows=597 width=5)
Group Key: postal_code
-> Seq Scan on address (cost=0.00..14.03 rows=603 width=5)

Performance
lock wait context
LOG: process 11367 still waiting for ShareLock on transaction 717 after 1000.108 ms
DETAIL: Process holding the lock: 11366. Wait queue: 11367.
CONTEXT: while updating tuple (0,2) in relation "foo"
STATEMENT: UPDATE foo SET value = 3;

Administration
ALTER SYSTEM
provide a mechanism to alter system level
conﬁguration from SQL
ALTER SYSTEM SET work_mem='42MB';
SELECT pg_reload_conf();

Administration
pset improvements
pagila=# pset
Border style (border) is 1.
Target width (columns) unset.
Expanded display (expanded) is off.
Field separator (fieldsep) is "|".
Default footer (footer) is on.
Output format (format) is aligned.
Line style (linestyle) is ascii.
Null display (null) is "".
Locale-adjusted numeric output (numericlocale) is off.
Pager (pager) is used for long output.
Record separator (recordsep) is <newline>.
Table attributes (tableattr) unset.
Title (title) unset.
Tuples only (tuples_only) is off.
also now works with tab completion

Administration
set tablespace options at create time
CREATE TABLESPACE rmd
LOCATION '/ramdisk'
WITH (random_page_cost = 0.9);
use to require create + alter

Administration
TABLESPACE SHUFFLING
move objects from one tablespace to another
{tables|indexes|all}
ALTER TABLESPACE pg_default
MOVE INDEXES TO rmd;

Administration
pg_stat_archiver statistics view
$ select * from pg_stat_archiver ;
-[ RECORD 1 ]------+------------------------------
archived_count | 21
last_archived_wal | 00000001000000000000000E
last_archived_time | 2014-04-29 11:37:45.337841+01
failed_count | 12
last_failed_wal | 00000001000000000000000F
last_failed_time | 2014-04-29 11:43:24.732469+01
stats_reset | 2014-04-29 11:37:16.995943+01

Administration
remove kerberos 5 authentication
deprecated since 8.3
use GSSAPI instead

Administration
SSL / TLS improvements
‣ add TLS v1.1/1.2 support
‣ remove SSL v3 support
‣ change default cipher suite to HIGH:!aNULL
‣ HIGH: Contains only secure and well-research algorithms
‣ !aNULL: Needed to disable suites that do not
authenticate server

Administration
conﬁg params
autovacuum_work_mem
session_preload_libraries
wal_log_hints

SQL
make time
• make_interval(years, months, weeks, days, hours, mins, secs)
• make_timestamp(year, month, day, hour, min, sec)
• make_timestamptz(year, month, day, hour, min, sec, timezone)
pagila=# select make_interval(hours := 2, mins := 1, secs := 1.2);
make_interval
---------------
02:01:01.2
(1 row)

SQL
“moving aggregates”
‣ optimization for aggregates called within moving window of
window function call
‣ initial support for count(), sum(), avg(), stddev(), variance(),
and boolean aggregates

SQL
ordered-set aggregates
‣ new class of aggregates
‣“offset in group”
‣WITHIN GROUP
‣SQL 2008

SQL
pagila=# select amount, count(*) from payment group by amount order by 1;
amount | count
--------+-------
0.00 | 24
0.99 | 2979
1.98 | 1
1.99 | 640
2.99 | 3542
3.98 | 8
3.99 | 1109
4.99 | 3789
5.98 | 7
5.99 | 1299
6.99 | 1119
7.98 | 5
7.99 | 670
8.97 | 1
8.99 | 485
9.98 | 1
9.99 | 256
10.99 | 104
11.99 | 10
(19 rows)

SQL
pagila=# SELECT
count(*), min(amount), max(amount),
percentile_cont(0.5) WITHIN GROUP (ORDER BY
amount)
FROM payment;
count | min | max | percentile_disc
-------+------+-------+-----------------
16049 | 0.00 | 11.99 | 3.99
(1 row)
median amount

SQL
pagila=# SELECT
count(*), min(amount), max(amount),
percentile_cont(array[0.5,0.9])
WITHIN GROUP (ORDER BY amount)
FROM payment;
count | min | max | percentile_disc
-------+------+-------+-----------------
16049 | 0.00 | 11.99 | {3.99,6.99}
(1 row)

SQL
pagila=# SELECT
count(*),min(amount), max(amount),
mode() within group (order by amount)
FROM payment;
count | min | max | mode
-------+------+-------+------
16049 | 0.00 | 11.99 | 4.99
(1 row)

SQL
percentile_cont()
percentile_disc()
rank()
dense_rank()
percent_rank()
cume_dist()
mode()

SQL
FILTER aggregates
pagila=# SELECT customer_id, count(*),
count(*) FILTER (WHERE amount > 5)
FROM payment WHERE customer_id = 148 GROUP BY
customer_id;
customer_id | count | count
-------------+-------+-------
148 | 46 | 13
(1 row)
‣ add support for FILTER clause in aggregates
‣ no more case-then-null / case

Backend
plpgsql stack traces
CREATE OR REPLACE FUNCTION
public.inner_func() RETURNS integer AS $$
DECLARE
stack text;
BEGIN
GET DIAGNOSTICS stack = PG_CONTEXT;
RAISE NOTICE E'--- Call Stack ---n%',
stack;
RETURN 1;
END;
$$ LANGUAGE plpgsql;

Backend
refresh materialized views concurrently
REFRESH MATERIALIZED VIEW CONCURRENTLY <view>
* requires unique index

Backend
“with check” option for auto-updateable views
‣ given an updateable view
‣ only allow visible rows within the view
‣ LOCAL
‣ checking conditions only on the view itself
‣ CASCADE
‣ recursively check on parents (default)

Backend
jsonb
new storage format for json text data
“binary json”

Backend
jsonb
“binary json”
pros:
✓no need to re-parse data
✓improved indexing support
✓allows for equality checking

Backend
jsonb
“binary json”
pros:
✓no need to re-parse data
✓improved indexing support
✓allows for equality checking
cons:
- whitespace munging
- json/jsonb quirkiness
- more disk space on small
json objects
tomorrow afternoon - morissette 256

Backend
jsonb
tomorrow afternoon - morissette 256
json jsonb mongodb
table size 1322MB 1375MB 1666MB
index size 630MB 283MB N/A
single key
index
79MB 58MB 96MB
speed 18.759ms 1.111ms 2ms
https://plus.google.com/+ThomBrownUK/posts/1JizRBGPYBq

Backend
recovery target “immediate”
recovery target tells postgres when to launch
- target time, target xid, target name, all
immediate allows start up as soon as
consistent state reached

Future
dynamic background workers
9.3 added background workers
available only at startup
9.4 gives you the ability to start
background workers dynamically, from
SQL

Future
dynamic shared memory
not dynamic shared_buffers! :-(

Future
dynamic shared memory
allows the server to create new shared memory
segments on the ﬂy, for example when
requested by a background worker. also adds
some message passing infrastructure.
not dynamic shared_buffers! :-(

Future
extensible TOAST support
added infrastructure to support external, alternative
compression schemes

Future
replication improvements
wal_level = logical
replica identity
replication slots
logical decoding
greatly expands capabilities of 3rd party replication tools
with an eye on “logical replication”
Friday @ 4:15 | Andres Freund | Changeset Extraction

Thanks!
Postgres Services
Health Audits
24x7 Monitoring
RemoteDBA Services
Postgres Migrations
more:
@robtreat2
http://xzilla.net
@omniti
http://omniti.com

Postgres 9.4 First Look

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Postgres 9.4 First Look

Similar to Postgres 9.4 First Look (20)

More from Robert Treat

More from Robert Treat (20)

Recently uploaded

Recently uploaded (20)

Postgres 9.4 First Look