SlideShare a Scribd company logo
1 of 16
Download to read offline
Scaling Data: Postgres,
                        The Stack and the
                      Future of Replication
                              Selena Deckelmann
                            selena@postgresql.org
                                @selenamarie
So
  mL
   SeC
    CA
      CL
       Ao
         E0
         2 f1
         ne
             1r2
              0e
               xn
                e c
1ce
               1n
            10e
            2r2
         ne
         2f
        N0
      OAo
     CC
   LeC
 OmS
So
Postgres disrupts
                         the rest of the
                      commercial relational
                        database industry.
So
 OmS
   LeC
     CC
      OAo
        N0
         2f
         ne
            2r2
            10e
               1n
                1ce
Disruptive forces

                      Licensing

                      “Enterprise” Web development

                      Proprietary DBA Career Path
So
 OmS
   LeC
     CC
      OAo
        N0
         2f
         ne
            2r2
            10e
               1n
                1ce
Licensing

                      Drop-in replacement for custom application
                      development on proprietary DBs.

                      Cheap Data Warehousing

                      Ecosystem!
So
 OmS
   LeC
     CC
      OAo
        N0
         2f
         ne
            2r2
            10e
               1n
                1ce
“Enterprise”
                        Web development
                      Postgres is a “Developers database” -
                      Extension is encouraged

                      Database of choice for Django

                      Oracle wants PHP developers
So
 OmS
   LeC
     CC
      OAo
        N0
         2f
         ne
            2r2
            10e
               1n
                1ce
“Visit the Ruby and Python Topic
                        for more information pertaining
                         to Ruby/RoR, Python and other
                      non-PHP scripting languages.”
So
 OmS
   LeC
     CC
      OAo
        N0
         2f
         ne
            2r2
            10e
               1n
                1ce
DBA Career path
                      Companies can’t hire Postgres DBAs fast
                      enough.

                      Postgres is close enough to Oracle, that
                      Oracle DBAs can jump in.

                      What we do better: Data loading,
                      automation, TUI and GUI admin tools
So
 OmS
   LeC
     CC
      OAo
        N0
         2f
         ne
            2r2
            10e
               1n
                1ce
And... Stability

                      Support major releases for at least 5 years.

                      Commits to HEAD are stable.

                      Bugs reported are fixed in <48 hrs.
So
 OmS
   LeC
     CC
      OAo
        N0
         2f
         ne
            2r2
            10e
               1n
                1ce
Demo!


                      • Demo here!
                        https://github.com/selenamarie/
                        pg_replication_demo
So
  mL
   SeC
    CA
      CL
       Ao
         E0
         2 f1
         ne
             1r2
              0e
               xn
                e c
Old problems

                      • Replication is hard
                      • Sharding is hard
                      • Slow in “the cloud”
So
  mL
   SeC
    CA
      CL
       Ao
         E0
         2 f1
         ne
             1r2
              0e
               xn
                e c
Solutions

                      • Streaming, cascaded replication
                      new!




                      • Sharding is still hard
                      • Index-only scans
                      new!
So
  mL
   SeC
    CA
      CL
       Ao
         E0
         2 f1
         ne
             1r2
              0e
               xn
                e c
The Good
                      • Community starting to really address Ops/
                        performance concerns:
                       • AVOIDING REINDEXING ON ALTER
                          TYPE OF INDEXED COLUMN
                       • NOT VALID CHECKS
                       • pg_basebackup
So
  mL
   SeC
    CA
      CL
       Ao
         E0
         2 f1
         ne
             1r2
              0e
               xn
                e c
The Bad

                      • Installation of Postgres harder than SQLite
                      • psql is powerful, not very friendly
                      • HA Ops support could be way better
So
  mL
   SeC
    CA
      CL
       Ao
         E0
         2 f1
         ne
             1r2
              0e
               xn
                e c
What’s happening now

                      • Heroku -> postgres.heroku.com
                        400 million write-transactions/day
                      • VMWare -> for managing lots of instances
                      • More interest in PaaS (like what MySQL
                        support has been for a long time)
So
  mL
   SeC
    CA
      CL
       Ao
         E0
         2 f1
         ne
             1r2
              0e
               xn
                e c
Next in HA?

                      • Continued usability improvements
                      • Cascaded Failover
                      • Multi-master with synchronous rep
So
  mL
   SeC
    CA
      CL
       Ao
         E0
         2 f1
         ne
             1r2
              0e
               xn
                e c

More Related Content

Viewers also liked (14)

Cmb
CmbCmb
Cmb
 
Natuk
NatukNatuk
Natuk
 
Kõnepuue
KõnepuueKõnepuue
Kõnepuue
 
Porla Paz
Porla PazPorla Paz
Porla Paz
 
1. imagen y discurso säarbrucken
1. imagen y discurso säarbrucken1. imagen y discurso säarbrucken
1. imagen y discurso säarbrucken
 
Shadow Fighter The Movie
Shadow Fighter The MovieShadow Fighter The Movie
Shadow Fighter The Movie
 
Challenging employeesmodernsamplefinal
Challenging employeesmodernsamplefinalChallenging employeesmodernsamplefinal
Challenging employeesmodernsamplefinal
 
El Vagón Del Amor (Carabajal, cáceres, Vasquez y Rovaretti)
El Vagón Del Amor (Carabajal, cáceres, Vasquez y Rovaretti)El Vagón Del Amor (Carabajal, cáceres, Vasquez y Rovaretti)
El Vagón Del Amor (Carabajal, cáceres, Vasquez y Rovaretti)
 
Src Voc
Src VocSrc Voc
Src Voc
 
Matkalla metaverseen?
Matkalla metaverseen?Matkalla metaverseen?
Matkalla metaverseen?
 
морфология
морфологияморфология
морфология
 
2007p&o1milieu
2007p&o1milieu2007p&o1milieu
2007p&o1milieu
 
Durga
DurgaDurga
Durga
 
Lamai Si Zahar
Lamai Si ZaharLamai Si Zahar
Lamai Si Zahar
 

Similar to Pg92 HA, LCA 2012, Ballarat

Harder, better, faster, stronger: PostgreSQL 9.1
Harder, better, faster, stronger: PostgreSQL 9.1Harder, better, faster, stronger: PostgreSQL 9.1
Harder, better, faster, stronger: PostgreSQL 9.1
Selena Deckelmann
 
Drilling Deep Into Exadata Performance
Drilling Deep Into Exadata PerformanceDrilling Deep Into Exadata Performance
Drilling Deep Into Exadata Performance
Enkitec
 
Mistakes were made - LCA 2012
Mistakes were made - LCA 2012Mistakes were made - LCA 2012
Mistakes were made - LCA 2012
Selena Deckelmann
 
OOW09 Ebs Tuning Final
OOW09 Ebs Tuning FinalOOW09 Ebs Tuning Final
OOW09 Ebs Tuning Final
jucaab
 
Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...
Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...
Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...
Ontico
 
Databases and the Cloud
Databases and the CloudDatabases and the Cloud
Databases and the Cloud
Henrik Ingo
 

Similar to Pg92 HA, LCA 2012, Ballarat (20)

Harder, better, faster, stronger: PostgreSQL 9.1
Harder, better, faster, stronger: PostgreSQL 9.1Harder, better, faster, stronger: PostgreSQL 9.1
Harder, better, faster, stronger: PostgreSQL 9.1
 
Tanel Poder - Performance stories from Exadata Migrations
Tanel Poder - Performance stories from Exadata MigrationsTanel Poder - Performance stories from Exadata Migrations
Tanel Poder - Performance stories from Exadata Migrations
 
Drilling Deep Into Exadata Performance
Drilling Deep Into Exadata PerformanceDrilling Deep Into Exadata Performance
Drilling Deep Into Exadata Performance
 
Mistakes were made - LCA 2012
Mistakes were made - LCA 2012Mistakes were made - LCA 2012
Mistakes were made - LCA 2012
 
Five Years of EC2 Distilled
Five Years of EC2 DistilledFive Years of EC2 Distilled
Five Years of EC2 Distilled
 
Oracle 11G SCAN: Concepts and Implementation Experience Sharing
Oracle 11G SCAN: Concepts and Implementation Experience SharingOracle 11G SCAN: Concepts and Implementation Experience Sharing
Oracle 11G SCAN: Concepts and Implementation Experience Sharing
 
SQL Server: Now It's Everywhere You Want to Be
SQL Server: Now It's Everywhere You Want to BeSQL Server: Now It's Everywhere You Want to Be
SQL Server: Now It's Everywhere You Want to Be
 
Cómo construimos Oracle TimesTen
Cómo construimos Oracle TimesTenCómo construimos Oracle TimesTen
Cómo construimos Oracle TimesTen
 
AWS to Bare Metal: Motivation, Pitfalls, and Results
AWS to Bare Metal: Motivation, Pitfalls, and ResultsAWS to Bare Metal: Motivation, Pitfalls, and Results
AWS to Bare Metal: Motivation, Pitfalls, and Results
 
OOW09 Ebs Tuning Final
OOW09 Ebs Tuning FinalOOW09 Ebs Tuning Final
OOW09 Ebs Tuning Final
 
OSCON Data 2011 -- NoSQL @ Netflix, Part 2
OSCON Data 2011 -- NoSQL @ Netflix, Part 2OSCON Data 2011 -- NoSQL @ Netflix, Part 2
OSCON Data 2011 -- NoSQL @ Netflix, Part 2
 
Performance Tuning: Pulling a Rabbit From a Hat - Atlassian Summit 2010
Performance Tuning: Pulling a Rabbit From a Hat - Atlassian Summit 2010Performance Tuning: Pulling a Rabbit From a Hat - Atlassian Summit 2010
Performance Tuning: Pulling a Rabbit From a Hat - Atlassian Summit 2010
 
Introducing JSR-283
Introducing JSR-283Introducing JSR-283
Introducing JSR-283
 
What's new in JSR-283?
What's new in JSR-283?What's new in JSR-283?
What's new in JSR-283?
 
Managing terabytes
Managing terabytesManaging terabytes
Managing terabytes
 
Engineered Systems: Oracle’s Vision for the Future
Engineered Systems: Oracle’s Vision for the FutureEngineered Systems: Oracle’s Vision for the Future
Engineered Systems: Oracle’s Vision for the Future
 
Building Asynchronous Applications
Building Asynchronous ApplicationsBuilding Asynchronous Applications
Building Asynchronous Applications
 
Performance Oriented Design
Performance Oriented DesignPerformance Oriented Design
Performance Oriented Design
 
Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...
Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...
Cистема распределенного, масштабируемого и высоконадежного хранения данных дл...
 
Databases and the Cloud
Databases and the CloudDatabases and the Cloud
Databases and the Cloud
 

More from Selena Deckelmann

While we're here, let's fix computer science education
While we're here, let's fix computer science educationWhile we're here, let's fix computer science education
While we're here, let's fix computer science education
Selena Deckelmann
 
Own it: working with a changing open source community
Own it: working with a changing open source communityOwn it: working with a changing open source community
Own it: working with a changing open source community
Selena Deckelmann
 
Managing terabytes: When Postgres gets big
Managing terabytes: When Postgres gets bigManaging terabytes: When Postgres gets big
Managing terabytes: When Postgres gets big
Selena Deckelmann
 

More from Selena Deckelmann (20)

While we're here, let's fix computer science education
While we're here, let's fix computer science educationWhile we're here, let's fix computer science education
While we're here, let's fix computer science education
 
Algorithms are Recipes
Algorithms are RecipesAlgorithms are Recipes
Algorithms are Recipes
 
Hire the right way
Hire the right wayHire the right way
Hire the right way
 
Mistakes were made
Mistakes were madeMistakes were made
Mistakes were made
 
Postgres needs an aircraft carrier
Postgres needs an aircraft carrierPostgres needs an aircraft carrier
Postgres needs an aircraft carrier
 
Mistakes were made
Mistakes were madeMistakes were made
Mistakes were made
 
How to ask for money
How to ask for moneyHow to ask for money
How to ask for money
 
Letters from the open source trenches - Postgres community
Letters from the open source trenches - Postgres communityLetters from the open source trenches - Postgres community
Letters from the open source trenches - Postgres community
 
Own it: working with a changing open source community
Own it: working with a changing open source communityOwn it: working with a changing open source community
Own it: working with a changing open source community
 
Managing terabytes: When Postgres gets big
Managing terabytes: When Postgres gets bigManaging terabytes: When Postgres gets big
Managing terabytes: When Postgres gets big
 
Managing terabytes: When PostgreSQL gets big
Managing terabytes: When PostgreSQL gets bigManaging terabytes: When PostgreSQL gets big
Managing terabytes: When PostgreSQL gets big
 
Pdxpugday2010 pg90
Pdxpugday2010 pg90Pdxpugday2010 pg90
Pdxpugday2010 pg90
 
Making Software Communities
Making Software CommunitiesMaking Software Communities
Making Software Communities
 
Illustrated buffer cache
Illustrated buffer cacheIllustrated buffer cache
Illustrated buffer cache
 
Bucardo
BucardoBucardo
Bucardo
 
How a bunch of normal people Used Technology To Repair a Rigged Election
How a bunch of normal people Used Technology To Repair a Rigged ElectionHow a bunch of normal people Used Technology To Repair a Rigged Election
How a bunch of normal people Used Technology To Repair a Rigged Election
 
Open Source Bridge Opening Day
Open Source Bridge Opening DayOpen Source Bridge Opening Day
Open Source Bridge Opening Day
 
What Assumptions Make: Filesystem I/O from a database perspective
What Assumptions Make: Filesystem I/O from a database perspectiveWhat Assumptions Make: Filesystem I/O from a database perspective
What Assumptions Make: Filesystem I/O from a database perspective
 
Leading Without Being In Charge
Leading Without Being In ChargeLeading Without Being In Charge
Leading Without Being In Charge
 
Fun with PostgreSQL User Groups
Fun with PostgreSQL User GroupsFun with PostgreSQL User Groups
Fun with PostgreSQL User Groups
 

Recently uploaded

Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 

Recently uploaded (20)

ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemkeProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
ProductAnonymous-April2024-WinProductDiscovery-MelissaKlemke
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
ICT role in 21st century education and its challenges
ICT role in 21st century education and its challengesICT role in 21st century education and its challenges
ICT role in 21st century education and its challenges
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
Vector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptxVector Search -An Introduction in Oracle Database 23ai.pptx
Vector Search -An Introduction in Oracle Database 23ai.pptx
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
CNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In PakistanCNIC Information System with Pakdata Cf In Pakistan
CNIC Information System with Pakdata Cf In Pakistan
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot ModelMcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
Mcleodganj Call Girls 🥰 8617370543 Service Offer VIP Hot Model
 
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​Elevate Developer Efficiency & build GenAI Application with Amazon Q​
Elevate Developer Efficiency & build GenAI Application with Amazon Q​
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, AdobeApidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
Apidays New York 2024 - Scaling API-first by Ian Reasor and Radu Cotescu, Adobe
 

Pg92 HA, LCA 2012, Ballarat

  • 1. Scaling Data: Postgres, The Stack and the Future of Replication Selena Deckelmann selena@postgresql.org @selenamarie So mL SeC CA CL Ao E0 2 f1 ne 1r2 0e xn e c
  • 2. 1ce 1n 10e 2r2 ne 2f N0 OAo CC LeC OmS So
  • 3. Postgres disrupts the rest of the commercial relational database industry. So OmS LeC CC OAo N0 2f ne 2r2 10e 1n 1ce
  • 4. Disruptive forces Licensing “Enterprise” Web development Proprietary DBA Career Path So OmS LeC CC OAo N0 2f ne 2r2 10e 1n 1ce
  • 5. Licensing Drop-in replacement for custom application development on proprietary DBs. Cheap Data Warehousing Ecosystem! So OmS LeC CC OAo N0 2f ne 2r2 10e 1n 1ce
  • 6. “Enterprise” Web development Postgres is a “Developers database” - Extension is encouraged Database of choice for Django Oracle wants PHP developers So OmS LeC CC OAo N0 2f ne 2r2 10e 1n 1ce
  • 7. “Visit the Ruby and Python Topic for more information pertaining to Ruby/RoR, Python and other non-PHP scripting languages.” So OmS LeC CC OAo N0 2f ne 2r2 10e 1n 1ce
  • 8. DBA Career path Companies can’t hire Postgres DBAs fast enough. Postgres is close enough to Oracle, that Oracle DBAs can jump in. What we do better: Data loading, automation, TUI and GUI admin tools So OmS LeC CC OAo N0 2f ne 2r2 10e 1n 1ce
  • 9. And... Stability Support major releases for at least 5 years. Commits to HEAD are stable. Bugs reported are fixed in <48 hrs. So OmS LeC CC OAo N0 2f ne 2r2 10e 1n 1ce
  • 10. Demo! • Demo here! https://github.com/selenamarie/ pg_replication_demo So mL SeC CA CL Ao E0 2 f1 ne 1r2 0e xn e c
  • 11. Old problems • Replication is hard • Sharding is hard • Slow in “the cloud” So mL SeC CA CL Ao E0 2 f1 ne 1r2 0e xn e c
  • 12. Solutions • Streaming, cascaded replication new! • Sharding is still hard • Index-only scans new! So mL SeC CA CL Ao E0 2 f1 ne 1r2 0e xn e c
  • 13. The Good • Community starting to really address Ops/ performance concerns: • AVOIDING REINDEXING ON ALTER TYPE OF INDEXED COLUMN • NOT VALID CHECKS • pg_basebackup So mL SeC CA CL Ao E0 2 f1 ne 1r2 0e xn e c
  • 14. The Bad • Installation of Postgres harder than SQLite • psql is powerful, not very friendly • HA Ops support could be way better So mL SeC CA CL Ao E0 2 f1 ne 1r2 0e xn e c
  • 15. What’s happening now • Heroku -> postgres.heroku.com 400 million write-transactions/day • VMWare -> for managing lots of instances • More interest in PaaS (like what MySQL support has been for a long time) So mL SeC CA CL Ao E0 2 f1 ne 1r2 0e xn e c
  • 16. Next in HA? • Continued usability improvements • Cascaded Failover • Multi-master with synchronous rep So mL SeC CA CL Ao E0 2 f1 ne 1r2 0e xn e c