Technical Introduction to PostgreSQL and PPAS

A Technical introduction
to PostgreSQL and PPAS
Enterprise Class PostgreSQL Database from EDB
10/6/2014
CONFIDENTIAL
©2011EnterpriseDB.Allrightsreserved.

• Introduction
• About Ashnik
• PostgreSQL and buzz it has created
• Postgres Plus Advanced Server
• Architecture
• Oracle Compatibility
• Performance Features
• Security Features
• High Availability Features
• DBA Tools
• User Stories
• What’s coming up in v9.3
• How to start adopting
Agenda
2
CONFIDENTIAL

• Independent & Thriving Development Community for over 20 years
• Thousands of active deployments worldwide in public and private
sector organizations of all sizes
PostgreSQL and the Community
3
CONFIDENTIAL

Postgres’ IMPACT
4
CONFIDENTIAL

PostgreSQL – Postgres Plus Users, Globally
5
CONFIDENTIAL

6
Malaysia
Philippines
Singapore
Vietnam Thailand
Indonesia
PostgreSQL – Postgres Plus Users, across ASEAN
CONFIDENTIAL

EnterpriseDB – The Company
7
• The Enterprise PostgreSQL company was founded in 2004,
first product GA in 2005
• 2,000+ customers across all market segments
• 70,000+ downloads/week of PostgreSQL and related products
• Enabling database consolidation using PostgreSQL and
advanced Oracle compatibility
• Saving customers millions through the power of open source
• Strong financial backing:
CONFIDENTIAL
Postgres Plus - Recognized
by Gartner’s Magic
Quadrant as a challenger
in the Database Market

The EnterpriseDB Advantage
8
• Products and Tools
• Advanced database server software
• Deep Oracle compatibility
• Bundled development and management tools
• Technical Support and Services
• Around the clock support
• Expert consulting
• Oracle migration services
• Remote management and monitoring
• Professional Training
• Learn PostgreSQL from the experts
• Web and on-site training
• Training for developers and DBAs
CONFIDENTIAL

• Fast development cycles
• Thousands of developers
• Better code
• Lower cost
• 24/7 support
• Services and training
• Certification
• Indemnification
• Product strategy
Bringing the Advantage to Enterprises
9
Open Source
Software
Commercial
SoftwareEnterpriseDB
CONFIDENTIAL

About PPAS
10
CONFIDENTIAL

Architectural Overview
11
CONFIDENTIAL

Process Architecture
Postmaster
BGWRITERBGWRITER STATS
COLLECTOR
STATS
COLLECTOR
ARCHIVERARCHIVERAUTO--VACUUMAUTO--VACUUM
BGWRITERBGWRITER STATS
COLLECTOR
STATS
COLLECTOR
Data
Files
WAL
Segments
Archived
WAL
Shared Memory
Shared Buffers Process ArrayWAL Buffers
WAL WriterWAL Writer LOG WRITERLOG WRITER
Error Log Files
12
CONFIDENTIAL

Database Limitations
13
CONFIDENTIAL
• Limitations are generally
defined by
• Operating System Limits
• Compile-Time Parameters
– Data Type Usage
General Database
Limitations
Limit Value
Maximum Database Size Unlimited
Maximum Table Size 32 TB
Maximum Row Size 1.6 TB
Maximum Field Size 1 GB (4TB in v9.3 Beta)
Maximum Rows per Table Unlimited
Maximum Columns per Table 250-1600 (Depending on
Column types)
Maximum Indexes per Table Unlimited

Oracle Compatibility
14
• Run applications written for Oracle
virtually unchanged
• No need to re-train Oracle DBAs and
developers
• Support for PL/SQL language and OCI
interoperability
• Replication for easy sharing of data
• Dramatic Cost Savings
• No Vendor Lock-in
CONFIDENTIAL

Survey: Re-Use of Oracle DBA Skills
15
CONFIDENTIAL

Compatibility Means
16
• SQL Extension support
• Decode, NVL, Substr, NVL2,replace,translate, table()
• Date/time functions: add_months, extract, next_day,months_between,trunc, current_date support
• Support for cascade option in drop table
• PL/SQL support
• REF Cursors, Implicit and explicit cursors
• Looping, variable declarations, conditional statements
• Collections: Associative Arrays, Varrays, Nested tables
• Bulk binding
• Named parameters
• User Defined Exceptions
• Explicit Transaction Control - within a stored procedure
• Tools
• EDB*Plus – SQL*Plus look-a-like
• EDB*Loader – SQL*Loader equivalent
• EDB*Wrap – similar to the PL/SQL wrapper
CONFIDENTIAL

Compatibility Means (cont.)
17
• Packages
• Stored procedures
• Functions
• Triggers
• Optimizer Hints
• Database Links
• Hierarchical Queries
• Synonyms – Public and
Private
• Sequences
• Materialized Views
CONFIDENTIAL
• Rownum
• Object types
o Create type … as object
o Create type … as table
o Create type …as varray
o Constructor and collection
methods
• PL/SQL like SUBTYPE, which
inherits form base type
• Users/Roles
• Dynamic SQL
• Features

Compatibility Means (cont.)
18
• Data Types
• Integer, number, char, double precision, float, varchar2, blob, clob, xmltype, rowid, boolean
• Built-in Packages
• DBMS_:
o SQL, LOB, JOB, PIPE, ALERT, OUTPUT, UTILITY, PROFILER, RLS
• UTL_:
o FILE, MAIL, SMTP, ENCODE, TCP
• Oracle-like Data Dictionary
• ALL_, DBA_, USER_ views
• Most commonly accessed views
• Diagnostics - DRITA
• System and session waits
o Not exposed in PostgreSQL
o Part of Advanced Server
• Statspack-like reporting
CONFIDENTIAL

Flexible Partitioning Scheme in PostgreSQL
19
• Partitioning Example in PostgreSQL
o CREATE TABLE employees
o Create Child Tables
o create table emp_mgmt (check (job in (MANAGER'))) inherits (employees);
o create table emp_sales (check (job in ('SALESMAN'))) inherits (employees);
o Create the partitioning function
o CREATE OR REPLACE FUNCTION partition_emp();
o Create the trigger for partition
o CREATE TRIGGER employees_partition_trigger BEFORE INSERT ON employees
FOR EACH ROW EXECUTE PROCEDURE partition_emp();
• Advantages
• Flexible
• Customize your Partitioning Policy
• Disadvantages
• Complex
• Difficult to add/delete/split Partitions
CONFIDENTIAL

• Partitioning Example in PPAS
CREATE TABLE employees
(empno numeric(4,0),ename varchar(10),job varchar(9),hiredate timestamp)
PARTITION BY LIST (job)
(PARTITION emp_mgmt VALUES ('MANAGER', 'PRESIDENT') tablespace tbsp_1, PARTITION
emp_sales VALUES ('SALESMAN') tablespace tbsp_2, PARTITION emp_ops VALUES ('CLERK')
tablespace tbsp_3
);
• PPAS Offers
• LIST and RANGE Partitioning Syntax
• Easy to manage and maintain partitions
• You can still take benefit of PostgreSQL syntax for complex partitioning e.g. Dynamic Partitions
• Easy to Add/Delete/Split/Swap/Detach/Attach partitions
• Brings the best of both the world!
Compatible and Flexible Partitioning Scheme in PPAS
20
CONFIDENTIAL

Scaling with Partitioning in PPAS v9.3
21
CONFIDENTIAL
0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
100
250
500
1000
Time in Seconds
NumberofTablePartitions
Partitioning Performance: PPAS 9.3 v. PostgreSQL 9.3 v. PPAS 9.2 • Postgres Plus
Advanced Server 9.3
is up to 460 times
faster with row
insertions into
partitioned tables
than PostgreSQL 9.3
or PPAS 9.2
• Better performance
for bulk loading and
disaster recovery

• Online Migration Toolkit enables point and click migration from Oracle
• Automatically Migrates:
Database Migration Toolkit
22
CONFIDENTIAL
• Data
• Schemas
• Stored Procedures
• Triggers
• Functions
• Sequences
• Packages
• Views
• Database Links
• Synonyms

• Developed for High Performance Transaction
Environments (OLTP)
• DynaTune:
• Dynamic tuning of the database server to make the
optimal usage of the system resources available on
the host machine
• Index Advisor:
• Helps determine which columns you should index
to improve performance in a given workload.
• Query Optimization Hints, Hi-Speed Bulk Loader,
Multi-Threaded Replication
Performance enhancement in PPAS
23
CONFIDENTIAL

• Infinite Cache
• High performance
horizontal scaling
architecture for
cache memory
• Cache expands with
inexpensive
commodity
hardware
Scalability with Infinite Cache
24
CONFIDENTIAL

25
• Architecture: How it Works
• Disk access for data is slower than RAM access
• Infinite Cache puts disk data into the RAM cache
• Additional requests for that data are read from the faster cache
• The cache scales as needed with inexpensive hardware
CONFIDENTIAL
Application
High
speed
pg_cach
e buffers
in RAM
Slow speed
disk
High
Speed
Network
Update
d Data
Blocks
Update
Read
If not in buffer
Database Server
InfiniteCache
RAMblades
High
speed

26
• Single Machine Performance
CONFIDENTIAL
Advanced Server is 16X
faster on a single machine
with large amounts of
memory
(e.g. greater than 2 GB)
Infinite Cache can be used
on a single machine!

27
• Infinite Cache - Features
• Expands and Distributes buffer cache across multiple machines
• Designed for Read-Mostly applications (e.g. Content Management, Query Intensive, Business
Intelligence, Data Warehouse)
• Cache is transparent to client applications (no cache coding needed)
• Compression feature enables caching entire databases (e.g. put a 250 GB database into 32 GB
RAM Cache)
• Cache can be pre-warmed for immediate results
• Cache scales using inexpensive commodity hardware
• Infinite Cache can be run to boost single machine performance!
• Created For
• DBAs and data managers overseeing large amounts of data requiring fast response times for
queries and reporting loads
• Developers who don’t want to write specialized caching code16
CONFIDENTIAL

• Red Hat Reference Architecture Series:
 Comparing BenchmarkSQL Performance
on Red Hat Enterprise Linux 5 to Windows
Server Enterprise (2010)
Performance Benchmark to SQL Server
28
CONFIDENTIAL

Scalability MySQL Vs PostgreSQL
29
CONFIDENTIAL

Security
30
• Object level privileges assigned to roles and users
• Virtual Private Database
• Kerberos and LDAP authentication
• Host base authentication
• SSL communication
• Data Level Encryption (AES, 3DES, etc)
• Ability to utilize 3rd party Key Stores in a full PKI Infrastructure
• Foundation for full compliance with the strictest of security standards
(PCI Data Security Standard)
• Flexible field level encryption and row level security
CONFIDENTIAL
“By default, PostgreSQL is probably the most
security-aware database available ...”
Database Hacker's Handbook- David Litchfield

PL/Secure and EDB*Wrap
31
• Encrypt your Pl/pgsql stored programs with PL/Secure
• Encrypt your SPL procedures with EDB*Wrap
CONFIDENTIAL

SQL/Protect
32
• DBA Managed SQL Injection Protection
a) Preventing attacks is normally the responsibility of the application developer, but
with SQL/Protect, DBAs can now provide another layer of protection to prevent
corruption or co-opting of the database.
• Multiple Prevention Techniques
a) Unauthorized Relations
b) Utility Commands (e.g. DDL)
c) SQL Tautology
d) Unbounded DML
CONFIDENTIAL

• Multi-Master Replication – Active-Active Technology
• Near real-time Replication
• Multi-Version Concurrency Control (MVCC)
• Point-in-Time Recovery
• Log Shipping for Standby (~ Oracle® Data Guard)
High Availability
33
CONFIDENTIAL
Availability

Backup and Recovery Options
34
CONFIDENTIAL
• Physical and Logical Backup
• Logical Backup using pg_dump
• Instance level logical backup using pg_dumpall
• Table level, Schema level or DB level logical backup
• pg_restore
• Physical Backups
• pg_basebackup
• Compressed backup
• Recovery with WAL and archived WAL
• Point in Time Recover

• Using OS Clustering
• Using a shared disk for data
Active – Passive OS HA Clustering
35
CONFIDENTIAL

• Uses WAL (Write Ahead Log)
• WAL is continuously shipped through
an opened TCP channel to Hot
Standby Node
• Hot Standby node can be used for
read scalability
Hot Streaming Replication (Hot Standby Mode)
36
CONFIDENTIAL

• Uses WAL (Write Ahead Log)
• Archived WAL is shipped to Hot
Standby Node
• Hot Standby node can be used for
read scalability
Log Shipping Replication (Hot Standby Mode)
37
CONFIDENTIAL

EnterpriseDB Postgres Plus Failover Manager
38
CONFIDENTIAL
Agent Agent

• Using pg-pool
• Using redundant disk for data
• Hot Standby node can be used
for read scalability
HA with read scaling (with pg-pool)
39
CONFIDENTIAL

• Trigger based replication
• Publication – Subscription
• Snapshot and Continuous
• Cascaded Replication
• Read Scalability
• Master/Slave DB can be:
• Oracle
• MSSQL
• Postgres Plus Advanced Server
• Either Master or Slave should be
PPAS/PostgreSQL
xDB Single Master Replication (SMR)
40
CONFIDENTIAL

• High Availability uses
• Geographic distribution of load
• For creation of Testing/staging env using snapshot replication
• Segregate OLTP and reporting
xDB Replication Use Cases
41
CONFIDENTIAL
Replication

xDB Single-Master Replication (SMR)
42
CONFIDENTIAL
Table
D
Table
C
Table
B
Table
A
Table
C
Table
D
Procs
Objects
ReportsQueries
Continuous
or Scheduled
----------------
Filtered
or All Rows
Transaction
Replication
Improved
OLTP
Performance
Inexpensive
Query /
Reporting
Oracle Server Postgres Plus Advanced Server

xDB SMR Heterogeneous Replication Support
43
CONFIDENTIAL
Source/Target Oracle MS SQL
Server
2005/2008
PostgreSQL
Advanced
Server
(Oracle
mode)
Advanced
Server
(PostgreSQL
mode)
Oracle No No Yes Yes Yes
MS SQL
Server
2005/2008
No No Yes Yes Yes
PostgreSQL No Yes Yes Yes Yes
Advanced
Server (Oracle
mode)
Yes Yes No
Yes
No
Advanced
Server
(PostgreSQL
mode)
No Yes Yes Yes Yes

• Trigger based replication
• 2 or more Masters can be Sync
• Auto Conflict Detection & Resolution
• Read & Write Scalability
xDB Multi Master Replication (MMR)
44
CONFIDENTIAL

• Java-based Replication Server and Replication Console
• Delta changes recorded in shadow tables via post Insert/Update/Delete triggers
• Data Replication through JDBC channel
xDB MMR Architecture
45
CONFIDENTIAL

Enterprise Class DBA Tools
46
CONFIDENTIAL

Remotely Manage, Monitor And Tune
• Management En-Mass Design
• Enterprise Performance Monitoring
• Proactive Alert Management
• Graphical Administration
• SQL Workload Profiling
• Simplified Capacity Planning
• Full SQL IDE
Postgres Enterprise Manager
47
CONFIDENTIAL

PEM Distributed Architecture
48
CONFIDENTIAL

• Performance
Monitoring Dashboards
• Capacity Manager
• Postgres Expert
• Alert Management
• Browser based console
and dashboard
• Audit Manager
• Team Support
• Distributed Architecture
• Convenient Access
PEM For DBAs: Centralized Tool
49
CONFIDENTIAL

Helps you apply patches and
updates for all the
products under PPAS:
Stackbuilder Plus
50
CONFIDENTIAL

• A solution to aid in the creation of highly
available configurations of Postgres
• Monitors the health of a Postgres HA
configuration
• Automates the failover process in the event
of a failure
• Used in conjunction with Streaming
Replication
Postgres Plus Failover Manager
51
CONFIDENTIAL
Slave
Witness
Master

Failover Manager Architecture
52
CONFIDENTIAL
Agent Agent

• Automatic Failover from master to replica node
• Configurable fencing operation
1. By default uses VIP
2. Parameter to specify alternative operation
a) Ex: reconfigure a load balancer
• Manual failover configuration possible
• Email notifications when cluster status changes
• Witness node provides protection against ‘split brain’ scenarios
• User configurable wait times
• Built on PPCD/JGroups technology
Failover Manager Features
53
CONFIDENTIAL

Replicate below objects from MS SQL Server to
Postgres Plus Advanced Server:
• Schemas
• Tables
• Constraints
• Indexes
• Table data
• Views
Replicating & Migrating Microsoft SQL Server
54
CONFIDENTIAL

• Split input files into multiple files
• Load single tables in parallel
• Magnitudes better load time performance
• Parallel load capabilities also in Migration Toolkit and Migration Studio
Parallel Data Loader
55
CONFIDENTIAL

FTD Replicates Oracle Data for Volume Peaks
Background
FTD is the world’s leading provider of floral-
related products and services
Processes 15M+ orders annually through 20,000
retail florists in the US
The ARGO System Project
FTD deployed new shipping administration
system (ARGO)
ARGO’s performance deteriorated during peak
holiday loads
Oracle-based system couldn’t handle peak
reporting volumes
Off-load reporting data form Oracle to Postgres
using Postgres Plus Replication Server
Why Postgres
No major changes required to run Oracle apps
on Postgres
Project completed in 6 weeks (Valentines Day
 Mother’s Day)
Out of the Box Compatibility with existing
database
Improved Vendor Service – response times
decreased dramatically
Performance of order processing systems
improved 400% by offloading reporting to
Postgres
“When someone comes in at a sixth of the cost [of Oracle]
and 500 times the customer service, that makes it very
easy.”
Jason Weiss, Software Architect

InMobi Delivers Mobile Ads with Postgres
57
CONFIDENTIAL
Background
• One of the fastest growing mobile advertising networks
• Huge scalability and performance requirements, billions of ads per month
• Uses HP Proliant server
Mobile Ad Platform Project
• Postgres the best fit for a scalable open-source
database
• Community support not sufficient to meet business
requirements, EnterpriseDB a partner
• Initial database install supported organic growth
• 25 instances across 4 data centers with replication
• Database sizes upwards of 600 GB
• 200K txns/minute with 3 ms response time
Why Postgres
• Most scalable open-source database
• Lower cost than traditional databases
• Performance and reliability up to the task at
internet scale, supporting the most extreme
workloads
• Customizations by EnterpriseDB rapidly
incorporated into the product ensuring costly
customization maintenance is avoided

Sony Reduces TCO of database environment by 80%
58
CONFIDENTIAL
Free Realms Project
• Lower TCO and improve licensing flexibility
• Leverage existing Oracle DBA and Developer talent
• Migrate key Oracle apps to Postgres
• Commercial-grade quality and reliability, including
backup and recovery standards, to support mission-
critical applications
• Scalable, high performance execution
Why Postgres
• Reduced TCO by 80%
• East of Migration – majority of Oracle apps
could run unmodified on Postgres Plus
• Skills re-use – Oracle – trained staff could
immediately work with Postgres Plus
• Exceptional technical support from
EnterpriseDB
“Postgres Plus has proven to be a cost-effective
database that can accommodate Free Realms’
massive growth.”
- Andy Sites, Free Realms Senior Producer, SOE
Background
• Free Realms massively multiplayer online game
• >9M registered players since April 2009
• IT Oracle scale-up was cost-prohibitive

Background
• Multi-terabyte Oracle DB and needed to
reduce DB costs
• Running old, unsupported version of
Oracle and had to pay to upgrade
The EPBS Project
• Engaged EnterpriseDB consulting
services to migrate
• Unique migration process developed
and implemented
Why Postgres
• 2.2 TeraByte system migrated and
now running on Postgres Plus
Advanced Server
• Current performance is comparable
to Oracle
• Costs reduced significantly by
migrating to EnterpriseDB solutions
Achieved Comparable Performance Replacing Oracle
59
CONFIDENTIAL

ToT, Thailand plans to replace Oracle Exadata-TimesTen In-
memory database with HP-EnterpriseDB stack
60
CONFIDENTIAL
Background
• Largest public sector company in telecom in Thailand
• Has very stringent response time SLA (20ms response time for user requests ~ 3-5 read queries)
• Current setup has Oracle Exadata and four TimeTen In-memory databases
Current Challenges in Oracle Stack
• Issues in sync of Exadata with
In-memory databases
• Full refresh locks the tables
• Current system can go upto 1500 tps
• Dynamic Partitioning for huge table
• Current system load is 750 TPS with
write operations
• 5000 user requests (16.5K read queries)
per second with 4 servers
HP-EnterpriseDB Stack Proposed to be DR for Exadata
• HP DL-980 (4 CPU * 10 cores, 256GB RAM)
• 1TB VMA Disks for faster access
• Simulation tests show 2800 write-TPS achieved
• Flexible Partitioning scheme in PPAS (add partition on the fly for
each new date, no manual intervention)
• Currently this server is being tested as DR for Exadata
• Replication using xDB near-relatime replication for 1000 tps
• Scalability achieved using cascaded replication and Infinite Cache
• Setup can handle 5500 user requests (18K queries per second)
with 1.5-2.5ms response time per query

v.9.3
• Switch master without archive logs in cascaded replication
• Faster recovery
• Enhanced partitioning feature- Better performance
• More extended JSON support
• Materialized Views
• Updatable Views
• Recursive Views
• Replication between Heterogeneous System Architectures – Facilitates central backup server
• pg_xlogdump to decode WAL
• 4TB Large Object Support
• Better memory managements
What’s New?
61
CONFIDENTIAL

v.9.4
• Logical Replication
• Time Lagging DR
• Enhanced JSON features
• Enhancements in Materialized Views
• Moving aggregates
• Pg_prewarm to warm up the cache
• Backup throttle
• ALTER SYSTEM command to set parameters
What’s Coming up?
62
CONFIDENTIAL

How to start adoption
63
CONFIDENTIAL
REPLICATION SERVER
BACKUP / DR SERVER
NEW APPLICATION
MIGRATION

You have a ‘Real’ alternative to Oracle or other
conventional proprietary Databases
Conclusion
64
CONFIDENTIAL
Economical
Technically
Sound
Easy to
Adopt

65
CONFIDENTIAL

Technical Introduction to PostgreSQL and PPAS

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (16)

Similar to Technical Introduction to PostgreSQL and PPAS

Similar to Technical Introduction to PostgreSQL and PPAS (20)

More from Ashnikbiz

More from Ashnikbiz (20)

Recently uploaded

Recently uploaded (20)

Technical Introduction to PostgreSQL and PPAS