EnterpriseDB’s
xDB Replication solution
Setup and Success stories by Ashnik in ASEAN
25/8/2014
Confidentialinformation,forinternaluseonly
Confidentialinformation,forinternaluseonly
2
POSTGRES
innovation
ENTERPRISE
reliability
24/7
support
Services
& training
Enterprise-class
features & tools
Indemnification
Product
road-map
Control
Thousands
of developers
Fast
development
cycles
Low cost
No vendor
lock-in
Advanced
features
Get The Best
Of Both Worlds
• Asynchronous cascading master to multiple replicas
• Asynchronous multi-master
• Can be used for:
• Improved performance for geographically
dispersed users
• High availability
• Load distribution (OLTP vs. reporting)
xDB Replication Server
3
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
• High Availability uses
• Geographic distribution of load
• For creation of Testing/staging env using snapshot replication
• Segregate OLTP and reporting
xDB Replication Use Cases
4
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Replication
xDB Replication Server (Single Master)
5
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Supports EDB and Oracle near real-time heterogeneous replication
Supports Postgres to Postgres replication now
Trigger-based system with less than 10% performance impact
Applications
• For customers “stuck” on an Oracle database
• Offload reporting to EDB
• Keep Oracle database running fast for transactions
xDB Single-Master Replication (SMR)
6
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Table
D
Table
C
Table
B
Table
A
Table
C
Table
D
Procs
Objects
ReportsQueries
Continuous
or Scheduled
----------------
Filtered
or All Rows
Transaction
Replication
Improved
OLTP
Performance
Inexpensive
Query /
Reporting
Oracle Server Postgres Plus Advanced Server
7
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
xDB Replication – Heterogeneous DB Support
Source  Target Oracle
Microsoft SQL
Server
2005/2008
PostgreSQL
Postgres Plus
Advanced
Server (Oracle
mode)
Postgres Plus
Advanced
Server
(PostgreSQL
mode)
Oracle Yes Yes Yes
Microsoft SQL Server Yes Yes Yes
PostgreSQL Yes Yes* Yes Yes*
Postgres Plus Advanced
Server (Oracle mode)
Yes Yes Yes*
Postgres Plus Advanced
Server (PostgreSQL mode)
Yes Yes* Yes Yes*
* Supported in Multi-Master mode
8
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Master-to-Master Replication for 2 or more nodes
Trigger-based asynchronous implementation
• Delay of a couple of seconds to possibly minutes depending
on configuration and load
Automatic Conflict Detection
• Uniqueness
• Update
• Delete
Multiple Conflict Resolution
• Earliest or Latest Timestamp
• Node priority – one node always wins
• Manual – Administrator reviews and actions
xDB Replication Server (Multi-Master)
9
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Ideal for Geo-dispersed Servers
Each geography updates a local master faster than to a single common
master
Allows failover of one geography to another geography
Provides write availability
• Java-based Replication Server and Replication Console
• Delta changes recorded in shadow tables via post Insert/Update/Delete triggers
• Data Replication through JDBC channel
xDB MMR Architecture
10
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
• Multi Master Replication – State Farms Insurance, USA
• DR Site Replication – ToT, Thailand
• Reporting Server Replication- Packet1, Malaysia
Production Implementation of xDB
11
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
12
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
SUCCESS STORIES in ASEAN
Issues in Environment
- Delayed Reports
- Additional Oracle Server for
reporting = Extra Cost
- Extra Add-on component needed
for Replication between OLTP and
Reporting DB
- Reports being pulled from two
Databases- ArchiveDB and LiveDB
- No Primary Keys in database to
implement change data capture
Solution Suggested
- Migrate Archive DB to Postgres Plus
- Do daily snapshot of changes from Oracle to
Postgres Plus
- Use xDB replication for Snapshot
Implementation
- Migrated 1.2TB of Archive and 700MB of
Live DB
- 500MB of Differential data replicated on
daily basis
- Result – Reports are generated on time!
- OLTP Performance improved due to
offloading
Packet1- A Major Telco in Malaysia improves Reporting
Performance with EnterpriseDB
13
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
ToT, Thailand plans to replace Oracle Exadata-TimesTen In-
memory database with HP-EnterpriseDB stack
14
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Background
• Largest public sector company in telecom in Thailand
• Has very stringent response time SLA (20ms response time for user requests ~ 3-5 read queries)
• Current setup has Oracle Exadata and four TimeTen In-memory databases
Current Challenges in Oracle Stack
• Issues in sync of Exadata with
In-memory databases
• Full refresh locks the tables
• Current system can go upto 1500 tps
• Dynamic Partitioning for huge table
• Current system load is 750 TPS with
write operations
• 5000 user requests (16.5K read queries)
per second with 4 servers
HP-EnterpriseDB Stack Proposed to be DR for Exadata
• HP DL-980 (4 CPU * 10 cores, 256GB RAM)
• 1TB VMA Disks for faster access
• Simulation tests show 2800 write-TPS achieved
• Flexible Partitioning scheme in PPAS (add partition on the fly for
each new date, no manual intervention)
• Currently this server is being tested as DR for Exadata
• Replication using xDB near-relatime replication for 1000 tps
• Scalability achieved using cascaded replication and Infinite Cache
• Setup can handle 5500 user requests (18K queries per second)
with 1.5-2.5ms response time per query
PostgreSQL – Postgres Plus Users, Globally
15
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
16
PostgreSQL – Postgres Plus Users, across ASEAN
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.
Malaysia
Philippines
Singapore
Vietnam Thailand
Indonesia
Confidentialinformation,forinternaluseonly
17

X-DB Replication Server and MMR

  • 1.
    EnterpriseDB’s xDB Replication solution Setupand Success stories by Ashnik in ASEAN 25/8/2014 Confidentialinformation,forinternaluseonly
  • 2.
    Confidentialinformation,forinternaluseonly 2 POSTGRES innovation ENTERPRISE reliability 24/7 support Services & training Enterprise-class features &tools Indemnification Product road-map Control Thousands of developers Fast development cycles Low cost No vendor lock-in Advanced features Get The Best Of Both Worlds
  • 3.
    • Asynchronous cascadingmaster to multiple replicas • Asynchronous multi-master • Can be used for: • Improved performance for geographically dispersed users • High availability • Load distribution (OLTP vs. reporting) xDB Replication Server 3 CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved.
  • 4.
    • High Availabilityuses • Geographic distribution of load • For creation of Testing/staging env using snapshot replication • Segregate OLTP and reporting xDB Replication Use Cases 4 CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved. Replication
  • 5.
    xDB Replication Server(Single Master) 5 CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved. Supports EDB and Oracle near real-time heterogeneous replication Supports Postgres to Postgres replication now Trigger-based system with less than 10% performance impact Applications • For customers “stuck” on an Oracle database • Offload reporting to EDB • Keep Oracle database running fast for transactions
  • 6.
    xDB Single-Master Replication(SMR) 6 CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved. Table D Table C Table B Table A Table C Table D Procs Objects ReportsQueries Continuous or Scheduled ---------------- Filtered or All Rows Transaction Replication Improved OLTP Performance Inexpensive Query / Reporting Oracle Server Postgres Plus Advanced Server
  • 7.
    7 CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved. xDB Replication –Heterogeneous DB Support Source Target Oracle Microsoft SQL Server 2005/2008 PostgreSQL Postgres Plus Advanced Server (Oracle mode) Postgres Plus Advanced Server (PostgreSQL mode) Oracle Yes Yes Yes Microsoft SQL Server Yes Yes Yes PostgreSQL Yes Yes* Yes Yes* Postgres Plus Advanced Server (Oracle mode) Yes Yes Yes* Postgres Plus Advanced Server (PostgreSQL mode) Yes Yes* Yes Yes* * Supported in Multi-Master mode
  • 8.
    8 CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved. Master-to-Master Replication for2 or more nodes Trigger-based asynchronous implementation • Delay of a couple of seconds to possibly minutes depending on configuration and load Automatic Conflict Detection • Uniqueness • Update • Delete Multiple Conflict Resolution • Earliest or Latest Timestamp • Node priority – one node always wins • Manual – Administrator reviews and actions xDB Replication Server (Multi-Master)
  • 9.
    9 CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved. Ideal for Geo-dispersedServers Each geography updates a local master faster than to a single common master Allows failover of one geography to another geography Provides write availability
  • 10.
    • Java-based ReplicationServer and Replication Console • Delta changes recorded in shadow tables via post Insert/Update/Delete triggers • Data Replication through JDBC channel xDB MMR Architecture 10 CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved.
  • 11.
    • Multi MasterReplication – State Farms Insurance, USA • DR Site Replication – ToT, Thailand • Reporting Server Replication- Packet1, Malaysia Production Implementation of xDB 11 CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved.
  • 12.
  • 13.
    Issues in Environment -Delayed Reports - Additional Oracle Server for reporting = Extra Cost - Extra Add-on component needed for Replication between OLTP and Reporting DB - Reports being pulled from two Databases- ArchiveDB and LiveDB - No Primary Keys in database to implement change data capture Solution Suggested - Migrate Archive DB to Postgres Plus - Do daily snapshot of changes from Oracle to Postgres Plus - Use xDB replication for Snapshot Implementation - Migrated 1.2TB of Archive and 700MB of Live DB - 500MB of Differential data replicated on daily basis - Result – Reports are generated on time! - OLTP Performance improved due to offloading Packet1- A Major Telco in Malaysia improves Reporting Performance with EnterpriseDB 13 CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved.
  • 14.
    ToT, Thailand plansto replace Oracle Exadata-TimesTen In- memory database with HP-EnterpriseDB stack 14 CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved. Background • Largest public sector company in telecom in Thailand • Has very stringent response time SLA (20ms response time for user requests ~ 3-5 read queries) • Current setup has Oracle Exadata and four TimeTen In-memory databases Current Challenges in Oracle Stack • Issues in sync of Exadata with In-memory databases • Full refresh locks the tables • Current system can go upto 1500 tps • Dynamic Partitioning for huge table • Current system load is 750 TPS with write operations • 5000 user requests (16.5K read queries) per second with 4 servers HP-EnterpriseDB Stack Proposed to be DR for Exadata • HP DL-980 (4 CPU * 10 cores, 256GB RAM) • 1TB VMA Disks for faster access • Simulation tests show 2800 write-TPS achieved • Flexible Partitioning scheme in PPAS (add partition on the fly for each new date, no manual intervention) • Currently this server is being tested as DR for Exadata • Replication using xDB near-relatime replication for 1000 tps • Scalability achieved using cascaded replication and Infinite Cache • Setup can handle 5500 user requests (18K queries per second) with 1.5-2.5ms response time per query
  • 15.
    PostgreSQL – PostgresPlus Users, Globally 15 CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved.
  • 16.
    16 PostgreSQL – PostgresPlus Users, across ASEAN CONFIDENTIAL ©2011EnterpriseDB.Allrightsreserved. Malaysia Philippines Singapore Vietnam Thailand Indonesia
  • 17.