Ashnik EnterpriseDB PostgreSQL - A real alternative to Oracle

  • 1,462 views
Uploaded on

A Technical introduction to PostgreSQL and Postgres Plus - …

A Technical introduction to PostgreSQL and Postgres Plus -
Enterprise Class PostgreSQL Database from EDB - You have a ‘Real’ alternative to Oracle and other conventional proprietary Databases

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
1,462
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
46
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Enterprise Class PostgreSQL Database from EDB CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. A Technical introduction to PostgreSQL and PPAS
  • 2. Agenda Introduction About Ashnik PostgreSQL and buzz it has created Postgres Plus Advanced Server • • • • • • Architecture Oracle Compatibility Performance Feature Security Features High Availability Features DBA Tools • User Stories • What’s coming up in v9.3 • How to start adopting CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • • • • 2
  • 3. PostgreSQL and the Community • Independent & Thriving Development Community for over 20 years CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Thousands of active deployments worldwide in public and private sector organizations of all sizes 3
  • 4. CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Postgres’ IMPACT 4
  • 5. CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. PostgreSQL – Postgres Plus Users, Globally 5
  • 6. PostgreSQL – Postgres Plus Users, across ASEAN Malaysia Singapore Vietnam Indonesia Thailand CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Philippines 6
  • 7. EnterpriseDB – The Company • The Enterprise PostgreSQL company was founded in 2004, first product GA in 2005 Postgres Plus - Recognized • 2,000+ customers across all market segments Quadrant as a challenger • 70,000+ downloads/week of PostgreSQL and related products in the Database Market • Saving customers millions through the power of open source • Strong financial backing: CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Enabling database consolidation using PostgreSQL and advanced Oracle compatibility by Gartner’s Magic 7
  • 8. The EnterpriseDB Advantage Products and Tools • Advanced database server software • Deep Oracle compatibility • Bundled development and management tools • Technical Support and Services • Around the clock support • Expert consulting • Oracle migration services • Remote management and monitoring Professional Training • Learn PostgreSQL from the experts • Web and on-site training • Training for developers and DBAs • CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • 8
  • 9. Bringing the Advantage to Enterprises • • • • Fast development cycles Thousands of developers Better code Lower cost Commercial Software EnterpriseDB • • • • • 24/7 support Services and training Certification Indemnification Product strategy CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Open Source Software 9
  • 10. About PPAS SECURITY PostgreSQL and Cloud PERFORMANCE and Manageability xDB REPLICATION Database MIGRATION Toolkit Server CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. HIGH AVAILABILITY and Scalability 10
  • 11. CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Architectural Overview 11
  • 12. Process Architecture Postmaster Shared Memory BGWRITER Process Array WAL Buffers STATS COLLECTOR AUTO--VACUUM ARCHIVER WAL Writer Data Files LOG WRITER WAL Segments Archived WAL CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Shared Buffers Error Log Files 12
  • 13. Database Limitations Limit Value Maximum Database Size Unlimited Maximum Table Size 32 TB Maximum Row Size 1.6 TB Maximum Field Size 1 GB (4TB in v9.3 Beta) Maximum Rows per Table Unlimited Maximum Columns per Table 250-1600 (Depending on Column types) Maximum Indexes per Table Unlimited CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Limitations are generally defined by • Operating System Limits • Compile-Time Parameters – Data Type Usage General Database Limitations 13
  • 14. Oracle Compatibility • Run applications written for Oracle virtually unchanged • Support for PL/SQL language and OCI interoperability • Replication for easy sharing of data • Dramatic Cost Savings • No Vendor Lock-in • Same technology which is used by IBM in DB2 since v9.7 CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • No need to re-train Oracle DBAs and developers 14
  • 15. Compatibility Means • SQL Extension support • Decode, NVL, Substr, NVL2,replace,translate, table() • Date/time functions: add_months, extract, next_day,months_between,trunc, current_date support • Support for cascade option in drop table • • • • • • • REF Cursors, Implicit and explicit cursors Looping, variable declarations, conditional statements Collections: Associative Arrays, Varrays, Nested tables Bulk binding Named parameters User Defined Exceptions Explicit Transaction Control - within a stored procedure • Tools • EDB*Plus – SQL*Plus look-a-like • EDB*Loader – SQL*Loader equivalent • EDB*Wrap – similar to the PL/SQL wrapper CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • PL/SQL support 15
  • 16. Compatibility Means (cont.) • • • • • • • • • • • Packages Stored procedures Functions Triggers Optimizer Hints Database Links Hierarchical Queries Synonyms – Public and Private Sequences Rownum Object types o o o o Create type … as object Create type … as table Create type …as varray Constructor and collection methods • PL/SQL like SUBTYPE, which inherits form base type • Users/Roles • Dynamic SQL CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Features 16
  • 17. Compatibility Means (cont.) • Data Types • Integer, number, char, double precision, float, varchar2, blob, clob, xmltype, rowid, boolean • Built-in Packages • DBMS_: o SQL, LOB, JOB, PIPE, ALERT, OUTPUT, UTILITY, PROFILER, RLS • UTL_: • Oracle-like Data Dictionary • ALL_, DBA_, USER_ views • Most commonly accessed views • Diagnostics - DRITA • System and session waits o Not exposed in PostgreSQL o Part of Advanced Server • Statspack-like reporting CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. o FILE, MAIL, SMTP, ENCODE, TCP 17
  • 18. Flexible Partitioning Scheme in PostgreSQL o o o o o o o o CREATE TABLE employees Create Child Tables create table emp_mgmt (check (job in (MANAGER'))) inherits (employees); create table emp_sales (check (job in ('SALESMAN'))) inherits (employees); Create the partitioning function CREATE OR REPLACE FUNCTION partition_emp(); Create the trigger for partition CREATE TRIGGER employees_partition_trigger BEFORE INSERT ON employees FOR EACH ROW EXECUTE PROCEDURE partition_emp(); • Advantages • • Flexible Customize your Partitioning Policy • Disadvantages • • Complex Difficult to add/delete/split Partitions CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Partitioning Example in PostgreSQL 18
  • 19. Compatible and Flexible Partitioning Scheme in PPAS Partitioning Example in PPAS CREATE TABLE employees (empno numeric(4,0),ename varchar(10),job varchar(9),hiredate timestamp) PARTITION BY LIST (job) (PARTITION emp_mgmt VALUES ('MANAGER', 'PRESIDENT') tablespace tbsp_1, PARTITION emp_sales VALUES ('SALESMAN') tablespace tbsp_2, PARTITION emp_ops VALUES ('CLERK') tablespace tbsp_3 ); • PPAS Offers • • • • • LIST and RANGE Partitioning Syntax Easy to manage and maintain partitions You can still take benefit of PostgreSQL syntax for complex partitioning e.g. Dynamic Partitions Easy to Add/Delete/Split/Swap/Detach/Attach partitions Brings the best of both the world! CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • 19
  • 20. Database Migration Toolkit • Online Migration Toolkit enables point and click migration from Oracle • • • • • • • • • • Data Schemas Stored Procedures Triggers Functions Sequences Packages Views Database Links Synonyms CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Automatically Migrates: 20
  • 21. Performance enhancement in PPAS • Developed for High Performance Transaction Environments (OLTP) • DynaTune: • Index Advisor: • Helps determine which columns you should index to improve performance in a given workload. • Query Optimization Hints, Hi-Speed Bulk Loader, Multi-Threaded Replication CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Dynamic tuning of the database server to make the optimal usage of the system resources available on the host machine 21
  • 22. • Infinite Cache • High performance horizontal scaling architecture for cache memory • Cache expands with inexpensive commodity hardware CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Scalability with Infinite Cache 22
  • 23. Scalability with Infinite Cache Update If not in buffer Read Database Server High Speed Network High speed InfiniteCache RAM blades Application High speed pg_cach e buffers in RAM Update d Data Blocks Slow speed disk CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Architecture: How it Works • Disk access for data is slower than RAM access • Infinite Cache puts disk data into the RAM cache • Additional requests for that data are read from the faster cache • The cache scales as needed with inexpensive hardware 23
  • 24. Scalability with Infinite Cache • Single Machine Performance Advanced Server is 16X faster on a single machine with large amounts of memory (e.g. greater than 2 GB) CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Infinite Cache can be used on a single machine! 24
  • 25. Scalability with Infinite Cache • Expands and Distributes buffer cache across multiple machines • Designed for Read-Mostly applications (e.g. Content Management, Query Intensive, Business Intelligence, Data Warehouse) • Cache is transparent to client applications (no cache coding needed) • Compression feature enables caching entire databases (e.g. put a 250 GB database into 32 GB RAM Cache) • Cache can be pre-warmed for immediate results • Cache scales using inexpensive commodity hardware • Infinite Cache can be run to boost single machine performance! • Created For • DBAs and data managers overseeing large amounts of data requiring fast response times for queries and reporting loads • Developers who don’t want to write specialized caching code CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Infinite Cache - Features 25
  • 26. Performance Benchmark to SQL Server • Red Hat Reference Architecture Series: • Comparing BenchmarkSQL Performance on Red Hat Enterprise Linux 5 to Windows CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Server Enterprise (2010) 26
  • 27. CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Scalability MySQL Vs PostgreSQL 27
  • 28. • • • • • • • • Object level privileges assigned to roles and users Virtual Private Database Kerberos and LDAP authentication Host base authentication SSL communication Data Level Encryption (AES, 3DES, etc) Ability to utilize 3rd party Key Stores in a full PKI Infrastructure Foundation for full compliance with the strictest of security standards (PCI Data Security Standard) • Flexible field level encryption and row level security “By default, PostgreSQL is probably the most security-aware database available ...” Database Hacker's Handbook- David Litchfield CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Security 28
  • 29. PL/Secure and EDB*Wrap CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Encrypt your Pl/pgsql stored programs with PL/Secure • Encrypt your SPL procedures with EDB*Wrap 29
  • 30. SQL/Protect • DBA Managed SQL Injection Protection • Preventing attacks is normally the responsibility of the application developer, but with SQL/Protect, DBAs can now provide another layer of protection to prevent corruption or co-opting of the database. Multiple Prevention Techniques • • • • Unauthorized Relations Utility Commands (e.g. DDL) SQL Tautology Unbounded DML CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • 30
  • 31. High Availability Multi-Master Replication – Active-Active Technology Near real-time Replication Multi-Version Concurrency Control (MVCC) Point-in-Time Recovery Log Shipping for Standby (~ Oracle® Data Guard) Availability CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • • • • • 31
  • 32. Backup and Recovery Options • Physical and Logical Backup • Logical Backup using pg_dump • Instance level logical backup using pg_dumpall • pg_restore • Physical Backups • pg_basebackup • Compressed backup • Recovery with WAL and archived WAL • Point in Time Recover CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Table level, Schema level or DB level logical backup 32
  • 33. Active – Passive OS HA Clustering Using OS Clustering • Using a shared disk for data CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • 33
  • 34. Hot Streaming Replication (Hot Standby Mode) Uses WAL (Write Ahead Log) • WAL is continuously shipped through an opened TCP channel to Hot Standby Node • Hot Standby node can be used for read scalability CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • 34
  • 35. Log Shipping Replication (Hot Standby Mode) Uses WAL (Write Ahead Log) • Archived WAL is shipped to Hot Standby Node • Hot Standby node can be used for read scalability CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • 35
  • 36. HA with read scaling (with pg-pool) Using pg-pool • Using redundant disk for data • Hot Standby node can be used for read scalability CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • 36
  • 37. xDB Single Master Replication (SMR) Trigger based replication • Publication – Subscription • Snapshot and Continuous • Cascaded Replication • Read Scalability • Master/Slave DB can be: • Oracle • MSSQL • Postgres Plus Advanced Server • Either Master or Slave should be PPAS/PostgreSQL CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • 37
  • 38. xDB Replication Use Cases High Availability uses Geographic distribution of load For creation of Testing/staging env using snapshot replication Segregate OLTP and reporting Replication CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • • • • 38
  • 39. xDB Single-Master Replication (SMR) Objects Transaction Replication Table A Table C Table B Table D Oracle Server Continuous or Scheduled ---------------Filtered or All Rows Queries Reports Table Table C D Postgres Plus Advanced Server CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Procs Inexpensive Query / Reporting Improved OLTP Performance 39
  • 40. xDB Multi Master Replication (MMR) Trigger based replication • 2 or more Masters can be Sync • Auto Conflict Detection & Resolution • Read & Write Scalability CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • 40
  • 41. xDB MMR Architecture CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Java-based Replication Server and Replication Console • Delta changes recorded in shadow tables via post Insert/Update/Delete triggers • Data Replication through JDBC channel 41
  • 42. CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Enterprise Class DBA Tools 42
  • 43. Postgres Enterprise Manager Remotely Manage, Monitor And Tune • Management En-Mass Design • Enterprise Performance Monitoring • Graphical Administration • SQL Workload Profiling • Simplified Capacity Planning • Full SQL IDE CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • Proactive Alert Management 43
  • 44. CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. PEM Distributed Architecture 44
  • 45. • Performance Monitoring Dashboards • Capacity Manager • Postgres Expert • Alert Management • Browser based console and dashboard • Audit Manager • Team Support • Distributed Architecture • Convenient Access CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. PEM For DBAs: Centralized Tool 45
  • 46. Stackbuilder Plus CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Helps you apply patches and updates for all the products under PPAS: 46
  • 47. Replicating & Migrating Microsoft SQL Server Replicate below objects from MS SQL Server to • • • • • • Schemas Tables Constraints Indexes Table data Views CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Postgres Plus Advanced Server: 47
  • 48. Parallel Data Loader Split input files into multiple files Load single tables in parallel Magnitudes better load time performance Parallel load capabilities also in Migration Toolkit and Migration Studio CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • • • • 48
  • 49. InMobi Delivers Mobile Ads with Postgres Background • One of the fastest growing mobile advertising networks • Huge scalability and performance requirements, billions of ads per month • Uses HP Proliant server Postgres the best fit for a scalable open-source Why Postgres • Most scalable open-source database • • Lower cost than traditional databases database • Community support not sufficient to meet business requirements, EnterpriseDB a partner • Initial database install supported organic growth • • 25 instances across 4 data centers with replication • • Database sizes upwards of 600 GB • 200K txns/minute with 3 ms response time Performance and reliability up to the task at internet scale, supporting the most extreme workloads Customizations by EnterpriseDB rapidly incorporated into the product ensuring costly customization maintenance is avoided CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Mobile Ad Platform Project 49
  • 50. Sony Reduces TCO of database environment by 80% Why Postgres • Free Realms massively multiplayer online game • >9M registered players since April 2009 • • • IT Oracle scale-up was cost-prohibitive • Free Realms Project • Lower TCO and improve licensing flexibility • Leverage existing Oracle DBA and Developer talent • • Reduced TCO by 80% East of Migration – majority of Oracle apps could run unmodified on Postgres Plus Skills re-use – Oracle – trained staff could immediately work with Postgres Plus Exceptional technical support from EnterpriseDB Migrate key Oracle apps to Postgres • • Commercial-grade quality and reliability, including backup and recovery standards, to support missioncritical applications Scalable, high performance execution “Postgres Plus has proven to be a cost-effective database that can accommodate Free Realms’ massive growth.” - Andy Sites, Free Realms Senior Producer, SOE CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Background 50
  • 51. Achieved Comparable Performance Replacing Oracle Background Multi-terabyte Oracle DB and needed to reduce DB costs • Running old, unsupported version of Oracle and had to pay to upgrade Why Postgres • 2.2 TeraByte system migrated and now running on Postgres Plus Advanced Server The EPBS Project • Engaged EnterpriseDB consulting services to migrate • Current performance is comparable to Oracle • Unique migration process developed and implemented • Costs reduced significantly by migrating to EnterpriseDB solutions CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • 51
  • 52. ToT, Thailand plans to replace Oracle Exadata-TimesTen Inmemory database with HP-EnterpriseDB stack Background • Largest public sector company in telecom in Thailand • Has very stringent response time SLA (20ms response time for user requests ~ 3-5 read queries) • Current setup has Oracle Exadata and four TimeTen In-memory databases HP-EnterpriseDB Stack Proposed to be DR for Exadata Issues in sync of Exadata with In-memory databases • • Full refresh locks the tables • • Current system can go upto 1500 tps • Dynamic Partitioning for huge table • • • Current system load is 750 TPS with write operations 5000 user requests (16.5K read queries) per second with 4 servers • • • • • • HP DL-980 (4 CPU * 10 cores, 256GB RAM) 1TB VMA Disks for faster access Simulation tests show 2800 write-TPS achieved Flexible Partitioning scheme in PPAS (add partition on the fly for each new date, no manual intervention) Currently this server is being tested as DR for Exadata Replication using xDB near-relatime replication for 1000 tps Scalability achieved using cascaded replication and Infinite Cache Setup can handle 5500 user requests (18K queries per second) with 1.5-2.5ms response time per query CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. Current Challenges in Oracle Stack 52
  • 53. What’s Coming Up? v.9.3 Beta is out Switch master without archive logs in cascaded replication • Faster recovery • More extended JSON support • Materialized Views • Updatable Views • Recursive Views • Replication between Heterogeneous System Architectures • pg_xlogdump to decode WAL • 4TB Large Object Support • Better memory managements CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. • 53
  • 54. How to start adoption BACKUP / DR SERVER NEW APPLICATION MIGRATION CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. REPLICATION SERVER 54
  • 55. Conclusion Economical Technically Easy to Sound Adopt CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved. You have a ‘Real’ alternative to Oracle or other conventional proprietary Databases 55
  • 56. 56 CONFIDENTIAL © 2011 EnterpriseDB. All rights reserved.