SlideShare a Scribd company logo

Looking ahead at PostgreSQL 15

This talk explores PostgreSQL 15 enhancements (along with some history) and looks at how they improve developer experience (MERGE and SQL/JSON), optimize support for backups and compression, logical replication improvements, enhanced security and performance, and more.

1 of 48
Download to read offline
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Looking ahead at PostgreSQL 15
Jonathan Katz, Principal Product Manager Technical
Jim Mlodgenski, Principal Database Engineer
AWS RDS Open Source
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon Relational Database Service (Amazon RDS)
Fully managed relational database service
Spend time innovating and building new apps,
not managing infrastructure
• Schema design
• Query construction
• Query optimization
Automatic failover
Backup and recovery
Isolation and security
Industry compliance
Push-button scaling
Automated patching and
upgrades
Advanced monitoring
Routine maintenance
You AWS
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Amazon RDS
Set up, operate, and scale a relational database in the cloud
with just a few clicks
Available and
durable
Automatic Multi-AZ
data replication, with
automated backup,
snapshots, and failover
Easy to
administer
Easily deploy and
maintain hardware, OS,
and DB software,
with built-in monitoring
Performant and
scalable
Scale compute
and storage with a few clicks,
plus minimal downtime for
your application
Secure and
compliant
Data encryption at rest
and in transit,
with industry compliance
and assurance programs
PostgreSQL-Compatible
Edition
MySQL-Compatible
Edition
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• A PostgreSQL “major release” is an annual feature release
• PostgreSQL 15 release cycle:
• Release cycle begins: July 2021
• Feature freeze: March/April 2022
• Beta: May 2022
• General availability: Late Q3 / Early Q4 2022
• During beta:
• Very unlikely that new functionality is added
• Some functionality may be removed
4
PostgreSQL major release process
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
• PostgreSQL releases average ~175 new features
• Review beta release notes:
https://www.postgresql.org/docs/15/release-15.html
• As we go through the new PostgreSQL features in this talk, we will:
• Look at use cases for each feature
• Review previous work
• Describe the new functionality
• See examples
How to explore PostgreSQL 15
© 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Evolution of Conditional SQL
Ad

Recommended

Mastering PostgreSQL Administration
Mastering PostgreSQL AdministrationMastering PostgreSQL Administration
Mastering PostgreSQL AdministrationEDB
 
Introduction VAUUM, Freezing, XID wraparound
Introduction VAUUM, Freezing, XID wraparoundIntroduction VAUUM, Freezing, XID wraparound
Introduction VAUUM, Freezing, XID wraparoundMasahiko Sawada
 
ClickHouse Features for Advanced Users, by Aleksei Milovidov
ClickHouse Features for Advanced Users, by Aleksei MilovidovClickHouse Features for Advanced Users, by Aleksei Milovidov
ClickHouse Features for Advanced Users, by Aleksei MilovidovAltinity Ltd
 
Build a Complex, Realtime Data Management App with Postgres 14!
Build a Complex, Realtime Data Management App with Postgres 14!Build a Complex, Realtime Data Management App with Postgres 14!
Build a Complex, Realtime Data Management App with Postgres 14!Jonathan Katz
 
Enterprise Manager: Write powerful scripts with EMCLI
Enterprise Manager: Write powerful scripts with EMCLIEnterprise Manager: Write powerful scripts with EMCLI
Enterprise Manager: Write powerful scripts with EMCLIGokhan Atil
 
Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.Deep dive into PostgreSQL statistics.
Deep dive into PostgreSQL statistics.Alexey Lesovsky
 
High Availability and Disaster Recovery in PostgreSQL - EQUNIX
High Availability and Disaster Recovery in PostgreSQL - EQUNIXHigh Availability and Disaster Recovery in PostgreSQL - EQUNIX
High Availability and Disaster Recovery in PostgreSQL - EQUNIXJulyanto SUTANDANG
 

More Related Content

What's hot

MySQL Document Store를 활용한 NoSQL 개발
MySQL Document Store를 활용한 NoSQL 개발MySQL Document Store를 활용한 NoSQL 개발
MySQL Document Store를 활용한 NoSQL 개발Oracle Korea
 
patroni-based citrus high availability environment deployment
patroni-based citrus high availability environment deploymentpatroni-based citrus high availability environment deployment
patroni-based citrus high availability environment deploymenthyeongchae lee
 
Understanding oracle rac internals part 1 - slides
Understanding oracle rac internals   part 1 - slidesUnderstanding oracle rac internals   part 1 - slides
Understanding oracle rac internals part 1 - slidesMohamed Farouk
 
MySQL GTID 시작하기
MySQL GTID 시작하기MySQL GTID 시작하기
MySQL GTID 시작하기I Goo Lee
 
PostgreSQL Deep Internal
PostgreSQL Deep InternalPostgreSQL Deep Internal
PostgreSQL Deep InternalEXEM
 
Ash masters : advanced ash analytics on Oracle
Ash masters : advanced ash analytics on Oracle Ash masters : advanced ash analytics on Oracle
Ash masters : advanced ash analytics on Oracle Kyle Hailey
 
Operating PostgreSQL at Scale with Kubernetes
Operating PostgreSQL at Scale with KubernetesOperating PostgreSQL at Scale with Kubernetes
Operating PostgreSQL at Scale with KubernetesJonathan Katz
 
Data Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UI
Data Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UIData Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UI
Data Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UIAltinity Ltd
 
Patroni: Kubernetes-native PostgreSQL companion
Patroni: Kubernetes-native PostgreSQL companionPatroni: Kubernetes-native PostgreSQL companion
Patroni: Kubernetes-native PostgreSQL companionAlexander Kukushkin
 
Tanel Poder - Scripts and Tools short
Tanel Poder - Scripts and Tools shortTanel Poder - Scripts and Tools short
Tanel Poder - Scripts and Tools shortTanel Poder
 
UKOUG - 25 years of hints and tips
UKOUG - 25 years of hints and tipsUKOUG - 25 years of hints and tips
UKOUG - 25 years of hints and tipsConnor McDonald
 
Performance Stability, Tips and Tricks and Underscores
Performance Stability, Tips and Tricks and UnderscoresPerformance Stability, Tips and Tricks and Underscores
Performance Stability, Tips and Tricks and UnderscoresJitendra Singh
 
[Pgday.Seoul 2020] SQL Tuning
[Pgday.Seoul 2020] SQL Tuning[Pgday.Seoul 2020] SQL Tuning
[Pgday.Seoul 2020] SQL TuningPgDay.Seoul
 
High Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando PatroniHigh Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando PatroniZalando Technology
 
Oracle 19c initialization parameters
Oracle 19c initialization parametersOracle 19c initialization parameters
Oracle 19c initialization parametersPablo Echeverria
 
Tricks every ClickHouse designer should know, by Robert Hodges, Altinity CEO
Tricks every ClickHouse designer should know, by Robert Hodges, Altinity CEOTricks every ClickHouse designer should know, by Robert Hodges, Altinity CEO
Tricks every ClickHouse designer should know, by Robert Hodges, Altinity CEOAltinity Ltd
 
Linux Performance Analysis: New Tools and Old Secrets
Linux Performance Analysis: New Tools and Old SecretsLinux Performance Analysis: New Tools and Old Secrets
Linux Performance Analysis: New Tools and Old SecretsBrendan Gregg
 
Adapting and adopting spm v04
Adapting and adopting spm v04Adapting and adopting spm v04
Adapting and adopting spm v04Carlos Sierra
 
Stop the Chaos! Get Real Oracle Performance by Query Tuning Part 1
Stop the Chaos! Get Real Oracle Performance by Query Tuning Part 1Stop the Chaos! Get Real Oracle Performance by Query Tuning Part 1
Stop the Chaos! Get Real Oracle Performance by Query Tuning Part 1SolarWinds
 
My first 90 days with ClickHouse.pdf
My first 90 days with ClickHouse.pdfMy first 90 days with ClickHouse.pdf
My first 90 days with ClickHouse.pdfAlkin Tezuysal
 

What's hot (20)

MySQL Document Store를 활용한 NoSQL 개발
MySQL Document Store를 활용한 NoSQL 개발MySQL Document Store를 활용한 NoSQL 개발
MySQL Document Store를 활용한 NoSQL 개발
 
patroni-based citrus high availability environment deployment
patroni-based citrus high availability environment deploymentpatroni-based citrus high availability environment deployment
patroni-based citrus high availability environment deployment
 
Understanding oracle rac internals part 1 - slides
Understanding oracle rac internals   part 1 - slidesUnderstanding oracle rac internals   part 1 - slides
Understanding oracle rac internals part 1 - slides
 
MySQL GTID 시작하기
MySQL GTID 시작하기MySQL GTID 시작하기
MySQL GTID 시작하기
 
PostgreSQL Deep Internal
PostgreSQL Deep InternalPostgreSQL Deep Internal
PostgreSQL Deep Internal
 
Ash masters : advanced ash analytics on Oracle
Ash masters : advanced ash analytics on Oracle Ash masters : advanced ash analytics on Oracle
Ash masters : advanced ash analytics on Oracle
 
Operating PostgreSQL at Scale with Kubernetes
Operating PostgreSQL at Scale with KubernetesOperating PostgreSQL at Scale with Kubernetes
Operating PostgreSQL at Scale with Kubernetes
 
Data Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UI
Data Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UIData Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UI
Data Warehouses in Kubernetes Visualized: the ClickHouse Kubernetes Operator UI
 
Patroni: Kubernetes-native PostgreSQL companion
Patroni: Kubernetes-native PostgreSQL companionPatroni: Kubernetes-native PostgreSQL companion
Patroni: Kubernetes-native PostgreSQL companion
 
Tanel Poder - Scripts and Tools short
Tanel Poder - Scripts and Tools shortTanel Poder - Scripts and Tools short
Tanel Poder - Scripts and Tools short
 
UKOUG - 25 years of hints and tips
UKOUG - 25 years of hints and tipsUKOUG - 25 years of hints and tips
UKOUG - 25 years of hints and tips
 
Performance Stability, Tips and Tricks and Underscores
Performance Stability, Tips and Tricks and UnderscoresPerformance Stability, Tips and Tricks and Underscores
Performance Stability, Tips and Tricks and Underscores
 
[Pgday.Seoul 2020] SQL Tuning
[Pgday.Seoul 2020] SQL Tuning[Pgday.Seoul 2020] SQL Tuning
[Pgday.Seoul 2020] SQL Tuning
 
High Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando PatroniHigh Availability PostgreSQL with Zalando Patroni
High Availability PostgreSQL with Zalando Patroni
 
Oracle 19c initialization parameters
Oracle 19c initialization parametersOracle 19c initialization parameters
Oracle 19c initialization parameters
 
Tricks every ClickHouse designer should know, by Robert Hodges, Altinity CEO
Tricks every ClickHouse designer should know, by Robert Hodges, Altinity CEOTricks every ClickHouse designer should know, by Robert Hodges, Altinity CEO
Tricks every ClickHouse designer should know, by Robert Hodges, Altinity CEO
 
Linux Performance Analysis: New Tools and Old Secrets
Linux Performance Analysis: New Tools and Old SecretsLinux Performance Analysis: New Tools and Old Secrets
Linux Performance Analysis: New Tools and Old Secrets
 
Adapting and adopting spm v04
Adapting and adopting spm v04Adapting and adopting spm v04
Adapting and adopting spm v04
 
Stop the Chaos! Get Real Oracle Performance by Query Tuning Part 1
Stop the Chaos! Get Real Oracle Performance by Query Tuning Part 1Stop the Chaos! Get Real Oracle Performance by Query Tuning Part 1
Stop the Chaos! Get Real Oracle Performance by Query Tuning Part 1
 
My first 90 days with ClickHouse.pdf
My first 90 days with ClickHouse.pdfMy first 90 days with ClickHouse.pdf
My first 90 days with ClickHouse.pdf
 

Similar to Looking ahead at PostgreSQL 15

NoSQL and MySQL: News about JSON
NoSQL and MySQL: News about JSONNoSQL and MySQL: News about JSON
NoSQL and MySQL: News about JSONMario Beck
 
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1MariaDB plc
 
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1MariaDB plc
 
Modernizing SQL Server the Right Way
Modernizing SQL Server the Right WayModernizing SQL Server the Right Way
Modernizing SQL Server the Right WayJuan Fabian
 
Confoo 2021 -- MySQL New Features
Confoo 2021 -- MySQL New FeaturesConfoo 2021 -- MySQL New Features
Confoo 2021 -- MySQL New FeaturesDave Stokes
 
SQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginners
SQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginnersSQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginners
SQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginnersTobias Koprowski
 
Building Advanced Workflows with AWS Glue (ANT372) - AWS re:Invent 2018
Building Advanced Workflows with AWS Glue (ANT372) - AWS re:Invent 2018Building Advanced Workflows with AWS Glue (ANT372) - AWS re:Invent 2018
Building Advanced Workflows with AWS Glue (ANT372) - AWS re:Invent 2018Amazon Web Services
 
Building Advanced Workflows with AWS Glue (ANT333) - AWS re:Invent 2018
Building Advanced Workflows with AWS Glue (ANT333) - AWS re:Invent 2018Building Advanced Workflows with AWS Glue (ANT333) - AWS re:Invent 2018
Building Advanced Workflows with AWS Glue (ANT333) - AWS re:Invent 2018Amazon Web Services
 
SQL Server End Of Support
SQL Server End Of SupportSQL Server End Of Support
SQL Server End Of SupportMariano Kovo
 
Die Neuheiten in MariaDB 10.2 und MaxScale 2.1
Die Neuheiten in MariaDB 10.2 und MaxScale 2.1Die Neuheiten in MariaDB 10.2 und MaxScale 2.1
Die Neuheiten in MariaDB 10.2 und MaxScale 2.1MariaDB plc
 
Managing Application Lifecycle using Jira and Bitbucket Cloud and AWS Tooling
Managing Application Lifecycle using Jira and Bitbucket Cloud and AWS ToolingManaging Application Lifecycle using Jira and Bitbucket Cloud and AWS Tooling
Managing Application Lifecycle using Jira and Bitbucket Cloud and AWS ToolingAtlassian
 
Oracle to Azure PostgreSQL database migration webinar
Oracle to Azure PostgreSQL database migration webinarOracle to Azure PostgreSQL database migration webinar
Oracle to Azure PostgreSQL database migration webinarMinnie Seungmin Cho
 
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023Matt Houghton
 
What’s New in MariaDB Server 10.2
What’s New in MariaDB Server 10.2What’s New in MariaDB Server 10.2
What’s New in MariaDB Server 10.2MariaDB plc
 
Postgres_9.0 vs MySQL_5.5
Postgres_9.0 vs MySQL_5.5Postgres_9.0 vs MySQL_5.5
Postgres_9.0 vs MySQL_5.5Trieu Dao Minh
 
Amazon RDS for PostgreSQL: What's New and Lessons Learned - NY 2017
Amazon RDS for PostgreSQL: What's New and Lessons Learned - NY 2017Amazon RDS for PostgreSQL: What's New and Lessons Learned - NY 2017
Amazon RDS for PostgreSQL: What's New and Lessons Learned - NY 2017Grant McAlister
 

Similar to Looking ahead at PostgreSQL 15 (20)

PostgreSQL
PostgreSQLPostgreSQL
PostgreSQL
 
NoSQL and MySQL: News about JSON
NoSQL and MySQL: News about JSONNoSQL and MySQL: News about JSON
NoSQL and MySQL: News about JSON
 
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
 
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
What's New in MariaDB Server 10.2 and MariaDB MaxScale 2.1
 
PostgreSQL
PostgreSQLPostgreSQL
PostgreSQL
 
PostgreSQL
PostgreSQL PostgreSQL
PostgreSQL
 
Modernizing SQL Server the Right Way
Modernizing SQL Server the Right WayModernizing SQL Server the Right Way
Modernizing SQL Server the Right Way
 
Confoo 2021 -- MySQL New Features
Confoo 2021 -- MySQL New FeaturesConfoo 2021 -- MySQL New Features
Confoo 2021 -- MySQL New Features
 
SQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginners
SQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginnersSQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginners
SQLSaturday#290_Kiev_AdHocMaintenancePlansForBeginners
 
Building Advanced Workflows with AWS Glue (ANT372) - AWS re:Invent 2018
Building Advanced Workflows with AWS Glue (ANT372) - AWS re:Invent 2018Building Advanced Workflows with AWS Glue (ANT372) - AWS re:Invent 2018
Building Advanced Workflows with AWS Glue (ANT372) - AWS re:Invent 2018
 
Building Advanced Workflows with AWS Glue (ANT333) - AWS re:Invent 2018
Building Advanced Workflows with AWS Glue (ANT333) - AWS re:Invent 2018Building Advanced Workflows with AWS Glue (ANT333) - AWS re:Invent 2018
Building Advanced Workflows with AWS Glue (ANT333) - AWS re:Invent 2018
 
SQL Server End Of Support
SQL Server End Of SupportSQL Server End Of Support
SQL Server End Of Support
 
Die Neuheiten in MariaDB 10.2 und MaxScale 2.1
Die Neuheiten in MariaDB 10.2 und MaxScale 2.1Die Neuheiten in MariaDB 10.2 und MaxScale 2.1
Die Neuheiten in MariaDB 10.2 und MaxScale 2.1
 
Managing Application Lifecycle using Jira and Bitbucket Cloud and AWS Tooling
Managing Application Lifecycle using Jira and Bitbucket Cloud and AWS ToolingManaging Application Lifecycle using Jira and Bitbucket Cloud and AWS Tooling
Managing Application Lifecycle using Jira and Bitbucket Cloud and AWS Tooling
 
Oracle to Azure PostgreSQL database migration webinar
Oracle to Azure PostgreSQL database migration webinarOracle to Azure PostgreSQL database migration webinar
Oracle to Azure PostgreSQL database migration webinar
 
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023
Advanced Monitoring for Amazon RDS - AWS 4D Event Manchester 16th June 2023
 
Oracle SQL Tuning
Oracle SQL TuningOracle SQL Tuning
Oracle SQL Tuning
 
What’s New in MariaDB Server 10.2
What’s New in MariaDB Server 10.2What’s New in MariaDB Server 10.2
What’s New in MariaDB Server 10.2
 
Postgres_9.0 vs MySQL_5.5
Postgres_9.0 vs MySQL_5.5Postgres_9.0 vs MySQL_5.5
Postgres_9.0 vs MySQL_5.5
 
Amazon RDS for PostgreSQL: What's New and Lessons Learned - NY 2017
Amazon RDS for PostgreSQL: What's New and Lessons Learned - NY 2017Amazon RDS for PostgreSQL: What's New and Lessons Learned - NY 2017
Amazon RDS for PostgreSQL: What's New and Lessons Learned - NY 2017
 

More from Jonathan Katz

Vectors are the new JSON in PostgreSQL
Vectors are the new JSON in PostgreSQLVectors are the new JSON in PostgreSQL
Vectors are the new JSON in PostgreSQLJonathan Katz
 
High Availability PostgreSQL on OpenShift...and more!
High Availability PostgreSQL on OpenShift...and more!High Availability PostgreSQL on OpenShift...and more!
High Availability PostgreSQL on OpenShift...and more!Jonathan Katz
 
Get Your Insecure PostgreSQL Passwords to SCRAM
Get Your Insecure PostgreSQL Passwords to SCRAMGet Your Insecure PostgreSQL Passwords to SCRAM
Get Your Insecure PostgreSQL Passwords to SCRAMJonathan Katz
 
Safely Protect PostgreSQL Passwords - Tell Others to SCRAM
Safely Protect PostgreSQL Passwords - Tell Others to SCRAMSafely Protect PostgreSQL Passwords - Tell Others to SCRAM
Safely Protect PostgreSQL Passwords - Tell Others to SCRAMJonathan Katz
 
Building a Complex, Real-Time Data Management Application
Building a Complex, Real-Time Data Management ApplicationBuilding a Complex, Real-Time Data Management Application
Building a Complex, Real-Time Data Management ApplicationJonathan Katz
 
Using PostgreSQL With Docker & Kubernetes - July 2018
Using PostgreSQL With Docker & Kubernetes - July 2018Using PostgreSQL With Docker & Kubernetes - July 2018
Using PostgreSQL With Docker & Kubernetes - July 2018Jonathan Katz
 
An Introduction to Using PostgreSQL with Docker & Kubernetes
An Introduction to Using PostgreSQL with Docker & KubernetesAn Introduction to Using PostgreSQL with Docker & Kubernetes
An Introduction to Using PostgreSQL with Docker & KubernetesJonathan Katz
 
Developing and Deploying Apps with the Postgres FDW
Developing and Deploying Apps with the Postgres FDWDeveloping and Deploying Apps with the Postgres FDW
Developing and Deploying Apps with the Postgres FDWJonathan Katz
 
On Beyond (PostgreSQL) Data Types
On Beyond (PostgreSQL) Data TypesOn Beyond (PostgreSQL) Data Types
On Beyond (PostgreSQL) Data TypesJonathan Katz
 
Accelerating Local Search with PostgreSQL (KNN-Search)
Accelerating Local Search with PostgreSQL (KNN-Search)Accelerating Local Search with PostgreSQL (KNN-Search)
Accelerating Local Search with PostgreSQL (KNN-Search)Jonathan Katz
 
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Webscale PostgreSQL - JSONB and Horizontal Scaling StrategiesWebscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Webscale PostgreSQL - JSONB and Horizontal Scaling StrategiesJonathan Katz
 
Indexing Complex PostgreSQL Data Types
Indexing Complex PostgreSQL Data TypesIndexing Complex PostgreSQL Data Types
Indexing Complex PostgreSQL Data TypesJonathan Katz
 

More from Jonathan Katz (12)

Vectors are the new JSON in PostgreSQL
Vectors are the new JSON in PostgreSQLVectors are the new JSON in PostgreSQL
Vectors are the new JSON in PostgreSQL
 
High Availability PostgreSQL on OpenShift...and more!
High Availability PostgreSQL on OpenShift...and more!High Availability PostgreSQL on OpenShift...and more!
High Availability PostgreSQL on OpenShift...and more!
 
Get Your Insecure PostgreSQL Passwords to SCRAM
Get Your Insecure PostgreSQL Passwords to SCRAMGet Your Insecure PostgreSQL Passwords to SCRAM
Get Your Insecure PostgreSQL Passwords to SCRAM
 
Safely Protect PostgreSQL Passwords - Tell Others to SCRAM
Safely Protect PostgreSQL Passwords - Tell Others to SCRAMSafely Protect PostgreSQL Passwords - Tell Others to SCRAM
Safely Protect PostgreSQL Passwords - Tell Others to SCRAM
 
Building a Complex, Real-Time Data Management Application
Building a Complex, Real-Time Data Management ApplicationBuilding a Complex, Real-Time Data Management Application
Building a Complex, Real-Time Data Management Application
 
Using PostgreSQL With Docker & Kubernetes - July 2018
Using PostgreSQL With Docker & Kubernetes - July 2018Using PostgreSQL With Docker & Kubernetes - July 2018
Using PostgreSQL With Docker & Kubernetes - July 2018
 
An Introduction to Using PostgreSQL with Docker & Kubernetes
An Introduction to Using PostgreSQL with Docker & KubernetesAn Introduction to Using PostgreSQL with Docker & Kubernetes
An Introduction to Using PostgreSQL with Docker & Kubernetes
 
Developing and Deploying Apps with the Postgres FDW
Developing and Deploying Apps with the Postgres FDWDeveloping and Deploying Apps with the Postgres FDW
Developing and Deploying Apps with the Postgres FDW
 
On Beyond (PostgreSQL) Data Types
On Beyond (PostgreSQL) Data TypesOn Beyond (PostgreSQL) Data Types
On Beyond (PostgreSQL) Data Types
 
Accelerating Local Search with PostgreSQL (KNN-Search)
Accelerating Local Search with PostgreSQL (KNN-Search)Accelerating Local Search with PostgreSQL (KNN-Search)
Accelerating Local Search with PostgreSQL (KNN-Search)
 
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Webscale PostgreSQL - JSONB and Horizontal Scaling StrategiesWebscale PostgreSQL - JSONB and Horizontal Scaling Strategies
Webscale PostgreSQL - JSONB and Horizontal Scaling Strategies
 
Indexing Complex PostgreSQL Data Types
Indexing Complex PostgreSQL Data TypesIndexing Complex PostgreSQL Data Types
Indexing Complex PostgreSQL Data Types
 

Recently uploaded

Bringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxBringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxMaarten Balliauw
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build PolandGDSC PJATK
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfSafe Software
 
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...ISPMAIndia
 
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...MarcovanHurne2
 
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre..."Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...shaiyuvasv
 
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17Ana-Maria Mihalceanu
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaISPMAIndia
 
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...UiPathCommunity
 
How we think about an advisor tech stack
How we think about an advisor tech stackHow we think about an advisor tech stack
How we think about an advisor tech stackSummit
 
Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...Product School
 
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...Product School
 
Confoo 2024 Gettings started with OpenAI and data science
Confoo 2024 Gettings started with OpenAI and data scienceConfoo 2024 Gettings started with OpenAI and data science
Confoo 2024 Gettings started with OpenAI and data scienceSusan Ibach
 
Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...
Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...
Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...Product School
 
"Running Open-Source LLM models on Kubernetes", Volodymyr Tsap
"Running Open-Source LLM models on Kubernetes",  Volodymyr Tsap"Running Open-Source LLM models on Kubernetes",  Volodymyr Tsap
"Running Open-Source LLM models on Kubernetes", Volodymyr TsapFwdays
 
"Testing of Helm Charts or There and Back Again", Yura Rochniak
"Testing of Helm Charts or There and Back Again", Yura Rochniak"Testing of Helm Charts or There and Back Again", Yura Rochniak
"Testing of Helm Charts or There and Back Again", Yura RochniakFwdays
 
Act Like an Owner, Challenge Like a VC by former CPO, Tripadvisor
Act Like an Owner,  Challenge Like a VC by former CPO, TripadvisorAct Like an Owner,  Challenge Like a VC by former CPO, Tripadvisor
Act Like an Owner, Challenge Like a VC by former CPO, TripadvisorProduct School
 
Apex Replay Debugger and Salesforce Platform Events.pptx
Apex Replay Debugger and Salesforce Platform Events.pptxApex Replay Debugger and Salesforce Platform Events.pptx
Apex Replay Debugger and Salesforce Platform Events.pptxmohayyudin7826
 
My sample product research idea for you!
My sample product research idea for you!My sample product research idea for you!
My sample product research idea for you!KivenRaySarsaba
 
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...DianaGray10
 

Recently uploaded (20)

Bringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptxBringing nullability into existing code - dammit is not the answer.pptx
Bringing nullability into existing code - dammit is not the answer.pptx
 
Bit N Build Poland
Bit N Build PolandBit N Build Poland
Bit N Build Poland
 
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdfIntroducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
Introducing the New FME Community Webinar - Feb 21, 2024 (2).pdf
 
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
AI MODELS USAGE IN FINTECH PRODUCTS: PM APPROACH & BEST PRACTICES by Kasthuri...
 
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
Digital Transformation Strategy & Plan Templates - www.beyondthecloud.digital...
 
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre..."Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
"Journey of Aspiration: Unveiling the Path to Becoming a Technocrat and Entre...
 
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17Enhancing Productivity and Insight  A Tour of JDK Tools Progress Beyond Java 17
Enhancing Productivity and Insight A Tour of JDK Tools Progress Beyond Java 17
 
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish GuptaBuilding Products That Think- Bhaskaran Srinivasan & Ashish Gupta
Building Products That Think- Bhaskaran Srinivasan & Ashish Gupta
 
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
Dev Dives: Leverage APIs and Gen AI to power automations for RPA and software...
 
How we think about an advisor tech stack
How we think about an advisor tech stackHow we think about an advisor tech stack
How we think about an advisor tech stack
 
Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...Relationship Counselling: From Disjointed Features to Product-First Thinking ...
Relationship Counselling: From Disjointed Features to Product-First Thinking ...
 
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
Harnessing the Power of GenAI for Exceptional Product Outcomes by Booking.com...
 
Confoo 2024 Gettings started with OpenAI and data science
Confoo 2024 Gettings started with OpenAI and data scienceConfoo 2024 Gettings started with OpenAI and data science
Confoo 2024 Gettings started with OpenAI and data science
 
Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...
Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...
Synergy in Leadership and Product Excellence: A Blueprint for Growth by CPO, ...
 
"Running Open-Source LLM models on Kubernetes", Volodymyr Tsap
"Running Open-Source LLM models on Kubernetes",  Volodymyr Tsap"Running Open-Source LLM models on Kubernetes",  Volodymyr Tsap
"Running Open-Source LLM models on Kubernetes", Volodymyr Tsap
 
"Testing of Helm Charts or There and Back Again", Yura Rochniak
"Testing of Helm Charts or There and Back Again", Yura Rochniak"Testing of Helm Charts or There and Back Again", Yura Rochniak
"Testing of Helm Charts or There and Back Again", Yura Rochniak
 
Act Like an Owner, Challenge Like a VC by former CPO, Tripadvisor
Act Like an Owner,  Challenge Like a VC by former CPO, TripadvisorAct Like an Owner,  Challenge Like a VC by former CPO, Tripadvisor
Act Like an Owner, Challenge Like a VC by former CPO, Tripadvisor
 
Apex Replay Debugger and Salesforce Platform Events.pptx
Apex Replay Debugger and Salesforce Platform Events.pptxApex Replay Debugger and Salesforce Platform Events.pptx
Apex Replay Debugger and Salesforce Platform Events.pptx
 
My sample product research idea for you!
My sample product research idea for you!My sample product research idea for you!
My sample product research idea for you!
 
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
Automation Ops Series: Session 1 - Introduction and setup DevOps for UiPath p...
 

Looking ahead at PostgreSQL 15

  • 1. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Looking ahead at PostgreSQL 15 Jonathan Katz, Principal Product Manager Technical Jim Mlodgenski, Principal Database Engineer AWS RDS Open Source
  • 2. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon Relational Database Service (Amazon RDS) Fully managed relational database service Spend time innovating and building new apps, not managing infrastructure • Schema design • Query construction • Query optimization Automatic failover Backup and recovery Isolation and security Industry compliance Push-button scaling Automated patching and upgrades Advanced monitoring Routine maintenance You AWS
  • 3. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Amazon RDS Set up, operate, and scale a relational database in the cloud with just a few clicks Available and durable Automatic Multi-AZ data replication, with automated backup, snapshots, and failover Easy to administer Easily deploy and maintain hardware, OS, and DB software, with built-in monitoring Performant and scalable Scale compute and storage with a few clicks, plus minimal downtime for your application Secure and compliant Data encryption at rest and in transit, with industry compliance and assurance programs PostgreSQL-Compatible Edition MySQL-Compatible Edition
  • 4. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • A PostgreSQL “major release” is an annual feature release • PostgreSQL 15 release cycle: • Release cycle begins: July 2021 • Feature freeze: March/April 2022 • Beta: May 2022 • General availability: Late Q3 / Early Q4 2022 • During beta: • Very unlikely that new functionality is added • Some functionality may be removed 4 PostgreSQL major release process
  • 5. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • PostgreSQL releases average ~175 new features • Review beta release notes: https://www.postgresql.org/docs/15/release-15.html • As we go through the new PostgreSQL features in this talk, we will: • Look at use cases for each feature • Review previous work • Describe the new functionality • See examples How to explore PostgreSQL 15
  • 6. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Evolution of Conditional SQL
  • 7. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. What is “conditional SQL?” • “Conditional SQL” is when a statement takes an additional action based on the outcome of a previous statement • Prior to PostgreSQL 9.5, two ways of doing this both prone to race conditions • Application layer • Procedural Language (e.g. PL/pgSQL) • Additional syntax + overhead • Ease of use with ORMs “Add a new credit card to this account. However, if the credit card number already exists, update the expiration date.”
  • 8. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. PostgreSQL 9.5: INSERT … ON CONFLICT CREATE TABLE active_accounts ( account_id int PRIMARY KEY, last_active_at timestamptz NOT NULL ); INSERT INTO active_accounts VALUES (1, CURRENT_TIMESTAMP) ON CONFLICT (account_id) DO UPDATE SET last_active_at = CURRENT_TIMESTAMP; TABLE active_accounts; account_id | last_active_at ------------+------------------------------- 1 | 2022-04-29 12:12:25.626644-04
  • 9. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. PostgreSQL 9.5: INSERT … ON CONFLICT INSERT INTO active_accounts VALUES (1, CURRENT_TIMESTAMP) ON CONFLICT (account_id) DO UPDATE SET last_active_at = CURRENT_TIMESTAMP; TABLE active_accounts; account_id | last_active_at ------------+------------------------------- 1 | 2022-05-02 11:45:06.317415-04 (1 row)
  • 10. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. PostgreSQL 15: MERGE TRUNCATE TABLE active_accounts; -- clear out the data for this demo -- store each account activity in a log table CREATE TABLE activity_log ( account_id int NOT NULL, active_at timestamptz NOT NULL ); INSERT INTO activity_log VALUES (1, CURRENT_TIMESTAMP - '15 days'::interval), (1, CURRENT_TIMESTAMP - '10 days'::interval), (1, CURRENT_TIMESTAMP - '8 days'::interval);
  • 11. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. PostgreSQL 15: MERGE MERGE INTO active_accounts USING ( SELECT account_id, max(active_at) AS last_active_at FROM activity_log GROUP BY account_id ) alog ON active_accounts.account_id = alog.account_id WHEN NOT MATCHED AND alog.last_active_at >= CURRENT_TIMESTAMP - '14 days'::interval THEN INSERT VALUES (alog.account_id, alog.last_active_at) WHEN MATCHED AND alog.last_active_at < CURRENT_TIMESTAMP - '14 days'::interval THEN DELETE WHEN MATCHED AND active_accounts.last_active_at < alog.last_active_at THEN UPDATE SET last_active_at = alog.last_active_at;
  • 12. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. PostgreSQL 15: MERGE MERGE INTO active_accounts Specifies target table that is acted on by the MERGE conditions
  • 13. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. PostgreSQL 15: MERGE MERGE INTO active_accounts USING ( SELECT account_id, max(active_at) AS last_active_at FROM activity_log GROUP BY account_id ) alog ON active_accounts.account_id = alog.account_id Source data set and its relation to target. This example finds the most recent account activity. Specifies target table that is acted on by the MERGE conditions
  • 14. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. PostgreSQL 15: MERGE WHEN NOT MATCHED AND alog.last_active_at >= CURRENT_TIMESTAMP - '14 days'::interval THEN INSERT VALUES (alog.account_id, alog.last_active_at) WHEN MATCHED AND alog.last_active_at < CURRENT_TIMESTAMP - '14 days'::interval THEN DELETE WHEN MATCHED AND active_accounts.last_active_at < alog.last_active_at THEN UPDATE SET last_active_at = alog.last_active_at; Conditions and actions. If no row is found and the most recent activity was within 14 days: INSERT If a row is found but most recent activity older than 14 days: DELETE If a row is found and most recent activity is newer: UPDATE
  • 15. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. PostgreSQL 15: MERGE -- execute MERGE command TABLE active_accounts; account_id | last_active_at ------------+------------------------------- 1 | 2022-04-24 11:20:59.445867-04 INSERT INTO activity_log VALUES (1, CURRENT_TIMESTAMP); -- execute MERGE command TABLE active_accounts; account_id | last_active_at ------------+------------------------------- 1 | 2022-05-02 11:21:36.55613-04
  • 16. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. PostgreSQL 15: MERGE DELETE FROM activity_log WHERE active_at >= CURRENT_TIMESTAMP - '14 days'::interval; -- execute MERGE command TABLE active_accounts; account_id | last_active_at ------------+---------------- (0 rows)
  • 17. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 17 INSERT … ON CONFLICT vs Merge INSERT INTO active_accounts SELECT g, CURRENT_TIMESTAMP FROM generate_series(1, 10000000) g ON CONFLICT (account_id) DO UPDATE SET last_active_at = CURRENT_TIMESTAMP; INSERT 0 10000000 Time: 36295.022 ms (00:36.295) INSERT 0 10000000 Time: 66868.616 ms (01:06.869) MERGE INTO active_accounts USING (SELECT g, CURRENT_TIMESTAMP as t FROM generate_series(1, 10000000) g) AS a ON active_accounts.account_id = a.g WHEN NOT MATCHED THEN INSERT VALUES (a.g, a.t) WHEN MATCHED THEN UPDATE SET last_active_at = a.t; MERGE 10000000 Time: 17938.168 ms (00:17.938) MERGE 10000000 Time: 495383.872 ms (08:15.384) SET work_mem = '256MB’; MERGE 10000000 Time: 54970.733 ms (00:54.971)
  • 18. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. JSON
  • 19. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • 9.2 (2012): Support for JSON stored as text type • 9.3: Introspection / extraction functionality • 9.4: Support for JSON as binary type (JSONB) and indexing • 9.5: JSONB building functions (2017: SQL/JSON standard published) • 10: Fulltext search for JSON(B) documents • 11: JSON(B) transforms from PL/Python / PL/Perl • 12: SQL/JSON path language • 13: jsonpath.datetime • 14 (2021): JSON subscripting syntax (e.g. a[‘b’][‘c’]) A brief history of PostgreSQL and JSON
  • 20. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • Published as SQL standard extension with SQL:2016 • SQL standard for interfacing with JSON • Provide better interoperability between database engines • Previous PostgreSQL releases have supported functionality similar to SQL/JSON 20 SQL/JSON
  • 21. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • Constructors • JSON(), JSON_SCALAR(), JSON_ARRAY(), JSON_ARRAYAGG(), JSON_OBJECT(), JSON_OBJECTAGG() • Return "json" type by default; for "jsonb" use "RETURNING jsonb" • Query functions • JSON_EXISTS, JSON_VALUE, JSON_QUERY • Helpful for introspection and writing constraints • JSON table • Converts JSON into a PostgreSQL table • IS JSON • "SELECT value IS JSON;" 21 SQL/JSON and PostgreSQL 15
  • 22. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 22 SQL/JSON constructors =$ SELECT pg_typeof(JSON('{"a": 1}')); -- returns "json" =$ SELECT pg_typeof(JSON('{"a": 1}' RETURNING jsonb)); -- returns "jsonb" =$ SELECT JSON(1); ERROR: cannot cast type integer to json =$ SELECT JSON(JSON_SCALAR(1)); json ------ 1
  • 23. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 23 SQL/JSON constructors =$ SELECT JSON_ARRAYAGG(x ORDER BY x DESC RETURNING jsonb) FROM generate_series(1,5) x; json_arrayagg ----------------- [5, 4, 3, 2, 1] =$ SELECT JSON_OBJECT(x: x+1, x*2: x^2 RETURNING jsonb) FROM generate_series(1,5); json_object -------------------- {"1": 2, "2": 1} {"2": 3, "4": 4} {"3": 4, "6": 9} {"4": 5, "8": 16} {"5": 6, "10": 25}
  • 24. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 24 SQL/JSON query functions =$ SELECT JSON_EXISTS(JSON('{"a": { "aa": 1 } }'), '$.a.aa'); ERROR: JSON_EXISTS() is not yet implemented for json type =$ SELECT JSON_EXISTS(JSON('{"a": { "aa": 1 } }' RETURNING jsonb), '$.a.aa'); json_exists ------------- t =$ SELECT JSON_EXISTS(JSON('{"a": { "aa": 1 } }' RETURNING jsonb), '$.b'); json_exists ------------- f
  • 25. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 25 SQL/JSON query functions =$ SELECT JSON_QUERY( JSON_ARRAY( JSON_OBJECT('id': 1), JSON_OBJECT('id': 2), JSON_OBJECT('id': 3) RETURNING jsonb ), '$[$i].id' PASSING x AS i ) FROM generate_series(0, 2) x; json_query ------------ 1 2 3
  • 26. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 26 SQL/JSON table CREATE TABLE users AS SELECT * FROM JSON_TABLE( JSON_ARRAY( JSON_OBJECT('id': 1, 'name': 'abc', 'created_on': '2022-04-30'), JSON_OBJECT('id': 2, 'name': 'def', 'created_on': '2022-05-01'), JSON_OBJECT('id': 3, 'name': 'ghi', 'created_on': '2022-05-02') RETURNING jsonb ), '$[*]' COLUMNS ( id int, name text, created_on date ) );
  • 27. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 27 SQL/JSON table TABLE users; id | name | created_on ----+------+------------ 1 | abc | 2022-04-30 2 | def | 2022-05-01 3 | ghi | 2022-05-02
  • 28. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Backups, Recovery, Archiving
  • 29. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • "recovery_prefetch" parameter • During recovery, enables prefetching of blocks that are referenced in the WAL that are not yet in the buffer pool • Works for crash recovery, replication, and PITR • Extensibility • Custom WAL resource managers • Allows table access method extensions to participate in logical decoding • Remove exclusive backup mode Note: Some custom backup scripts may need to change 30 PostgreSQL 15 Backup and recovery improvements
  • 30. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 31 pg_walinspect =# SELECT pg_current_wal_lsn(); pg_current_wal_lsn -------------------- 16/F2392B60 (1 row) =# INSERT INTO active_accounts VALUES (-1, now()); INSERT 0 1 =# SELECT start_lsn, xid, resource_manager, record_type, record_length FROM pg_get_wal_records_info_till_end_of_wal('16/F2392B60'); start_lsn | xid | resource_manager | record_type | record_length -------------+-----+------------------+-------------------+--------------- 16/F2392B60 | 789 | Heap | INSERT | 71 16/F2392BA8 | 789 | Btree | INSERT_LEAF | 64 … (7 rows) • Provides SQL functions that allow view access to the contents of write-ahead log
  • 31. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • Allows for WAL archiving to be handled by a custom library instead of a shell command using “archive_command” • Will often be considerably more robust and performant • Example module provided by contrib/basic_archive # postgresql.conf archive_mode = 'on' archive_library = 'basic_archive' basic_archive.archive_directory = '/path/to/archive/directory' 32 Archive modules
  • 32. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • WAL compression (zstd + lz4) • Server-side compression for pg_basebackup • gzip, zstd, lz4 • Add zstd + lz4 for client-side • pg_receivewal (lz4 only) 33 Compression
  • 33. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 20000 1 2 4 8 16 32 64 128 256 512 transaction per second connections WAL compression off pglz lz4 zstd • PostgreSQL has the ability to compress full page writes • lz4 and zstd added to the existing pglz algorithm 34 WAL compression - performance 31% 41%
  • 34. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 0 20 40 60 80 100 120 140 160 1 2 4 8 16 32 64 128 256 512 size (GB) connections WAL compression off pglz lz4 zstd • PostgreSQL has the ability to compress full page writes • lz4 and zstd added to the existing pglz algorithm 35 WAL compression – size-on-disk 71% 79%
  • 35. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Logical Replication
  • 36. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • Logical replication allows for changes to be streamed to another database independent of the physical filesystem • Several use cases: • Change-data capture (CDC) • Streaming analytics / BI • Major version upgrades • Multi-writeable databases • Logical replication currently limited to data changes 37 Logical replication overview
  • 37. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • 9.4: Logical decoding, output plugins, replication slots • 10: Logical replication of INSERT/UPDATE/DELETE • 13: Logical replication of partitions • 14: Performance improvements; stream in-flight transactions A brief history of PostgreSQL and logical replication
  • 38. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • ALTER SUBSCRPTION … SKIP • Skip a transaction (e.g. on a conflict) and then resume replay • Can specify log sequence number (LSN) • Requires superuser privileges • Temporarily disable streaming replication • ALTER SUBSCRIPTION … SET disable_on_error = true; • Fix conflict on publisher side • ALTER SUBSCRIPTION ... ENABLE; 39 PostgreSQL 15 logical replication improvements
  • 39. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • Support for two-phase commit / prepared transactions • Sends transaction to subscriber when PREPARED TRANSACTION called • Publish all tables in schema • Previous was all tables in a database • Publish a subset of rows/columns in a table 40 PostgreSQL 15 logical replication improvements
  • 40. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. 41 PostgreSQL 15 Logical replication examples -- publish all tables in schema "app" =# CREATE SCHEMA app; =# CREATE TABLE app.users ( id int GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, username text); =# CREATE PUBLICATION pub FOR ALL TABLES IN SCHEMA app; -- publish only a subset of columns on a table =# CREATE TABLE abc(x int, y int, z int); =# ALTER TABLE abc REPLICA IDENTITY FULL; =# CREATE PUBLICATION pub2 FOR TABLE abc (x); -- publish only non-test data =# CREATE TABLE logs (id uuid PRIMARY KEY, log jsonb, is_test bool); =# CREATE PUBLICATION pub3 FOR TABLE logs WHERE (NOT is_test);
  • 41. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Other PostgreSQL 15 highlights
  • 42. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • Improved in-memory sort performance • Parallel SELECT DISTINCT • postgres_fdw parallel commit • Commit transactions executed on remote PostgreSQL asynchronously • Optimization for high CPU core arm64 processors (e.g. Graviton) • Tests show 10-20% speedup on 48+ cores 43 PostgreSQL 15 performance highlights
  • 43. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • CREATE revoked from PUBLIC in public schema in new databases • By default only database owners can create objects in default schema • Mitigation for CVE-2018-1058 Note: This may break some scripts run as a non-superuser • SECURITY INVOKER views • Uses permissions of user executing view, not owner 44 PostgreSQL 15 security highlights
  • 44. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • jsonlog log format • Structured logging and compatibility with log analysis tools • More regular expression functions • regexp_count: counts instances of pattern matches • regexp_instr: returns position that matches pattern • regexp_like: returns true if pattern matched; similar to ~ operator • dconfig • Displays non-default PostgreSQL configurations • Search for configuration names/value 45 PostgreSQL 15 general highlights
  • 45. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Summary
  • 46. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • Significant work on simplifying developer experience • More space-saving options for backups and archives • More flexibility with logical replication • General improvements that help with daily tasks 47 PostgreSQL 15 early takes
  • 47. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. • PostgreSQL commit logs • Blogs • Hubert "depesz" Lubaczewski – "Waiting for PostgreSQL 15" series • Presentations • Magnus Hagander – "What's new in PostgreSQL 15" • pgPedia • https://pgpedia.info/postgresql-versions/postgresql-15.html 48 References
  • 48. © 2022, Amazon Web Services, Inc. or its affiliates. All rights reserved. Thank you for attending Jonathan Katz jkatz@amazon.com 49 Jim Mlodgenski mlodj@amazon.com