Migrating an Oracle database to Postgres is never an automated operation. And it rarely (never?) involve just the database. Experience brought us to develop an agile methodology for the migration process, involving schema migration, data import, migration of procedures and queries up to the generation of unit tests for QA. Pitfalls, technologies and main migration opportunities will be outlined, focusing on the reduction of total costs of ownership and management of a database solution in the middle-long term (without reducing quality and business continuity requirements).

1. Higher impact of license costs

2. Agile Oracle to PostgreSQL
migrations
PGConf.EU 2013, Dublin, Nov 1st
Gabriele Bartolini
2ndQuadrant Italia / PostgreSQL Europe
gabriele.bartolini@2ndQuadrant.it
@_GBartolini_

3. Gabriele Bartolini
• Co-Founder and Manager of 2ndQuadrant Italia
• Data Architect, Business critical environments
• Data warehousing
• Co-Founder Italian PostgreSQL Users Group
• Co-Founder PostgreSQL Europe
• PostgreSQL Contributor and Advocate

4. Outline
• Introduction
• The migration project
• Open source tools
• Conclusions

5. Target
• Chief Technical Officers
• Chief Information Officers
• Oracle users
• Evaluate moving to PostgreSQL 9.3

6. Goals
• Describe the migration process
• Spot the major sources of costs
• Help you plan a migration in the mid/long term

7. Part I
Introduction

8. Oracle vs Postgres
(*) not sure it is in the TODO List, yet

9. Upgrade automatically
Would you ever do an upgrade to a higher version of
Oracle in a completely automated way?

10. PostgreSQL
• Hot Standby (~ Data Guard)
• Memcache integration (~ InMemory Database Cache)
• Barman (~ RMAN)
• Partitioning (*)
• Londiste/PgQ (GoldenGate)
• PostGIS (Spatial)
• SE Linux integration (~
Database Vault)
• Future: Multi-master
replication (~ RAC)

11. License
• Very simple and clear
• Applies to every part
(module, product, option) of
Postgres
• server, client, libraries,
additional modules
• 100% open source
• No need to hire a lawyer to
understand it
• Highly Permissive
• Not subject to monopoly
nor acquisitions
• Fosters market competition
in professional services
offer

12. Oracle DBA, do not be afraid!
• Transferrable knowledge
• PostgreSQL focused professional training
• 100% open source

13. Part II
The Migration project

14. Migration = project
budget, scope, quality, time

15. Budget
• Crucial variable
• Usually triggers the migration process
• Cost analysis drives the migration project
• short term
• mid term

16. Quality
• PostgreSQL reliability
• Tests

17. Components
• Applications
• Database
• Processes
• Human resources

18. Migration cost components
Testing
Processes
Training
Licensing
fictitious data
Dev
25,00%
50,00%

19. Comparison with Oracle upgrade
Testing
Processes
Training
Licensing
fictitious data
Dev
25,00%
50,00%

20. External applications
• Force your supplier to provide Postgres support for their
application
• You set the rules
• Typical case of the public sector
• e.g.: Land registry
• Financial crisis and competition are on your side

21. Agile approach
• Requirements change
• Every context is different from the other
• No one-size-fits-all scenario
• Be dynamic:
• SCRUM (with the customer)
• Kanban (internally)
• DevOps

23. Focus
• Applications
• Database
• Processes
• Human resources

24. Database migration
• Schema
• Queries
• Data
• Stored procedures
• Tests

25. Schema conversion
• Tables
• Data type mapping
• Indexes
• Views
• Constraints

26. Schema major differences
• USER = schema/database
• UPPERCASE = lowercase
• Partitioning (*)
• Tablespaces
• SYNONYMs
• search_path / auto updatable views / foreign tables

27. Queries
• ANSI/ISO SQL
• PostgreSQL has no hints
(optimiser)
• OUTER JOINs
• (+)
• use LEFT/RIGHT JOIN
• Major differences:
• DUAL
• NULL = NULL
• ROWNUM and ROWID
• Postgres: NULL
• ...
• Oracle: TRUE

28. Data
• Might require specialised
consulting activities
• Major solutions:
• ETL tools (e.g. Kettle)
• External scripts (e.g.
ora2pg or custom)
• COPY t FROM stdin;
• Foreign tables with Oracle
FDW
• INSERT INTO t SELECT *
FROM ft;
• Cut-over time
•>0
•~0

29. Stored Procedures
• Most critical and complex component of the whole migration
• Package = extension (schema)
• Two major ways:
• 1:1 conversion (not always possible)
• complete rewriting (optimised for PostgreSQL)
• It might required specialised consulting activities

30. PL/SQL vs PL/pgSQL
• Syntactical differences
• PROCEDURE = FUNCTION that returns VOID
• TRIGGER = TRIGGER FUNCTION + TRIGGER
• Lack of autonomous transactions
• Lack of global variables
• ...

31. Test
• Vital component for a successful migration process
• It measures the QUALITY of the project
• It can be reused in the future for PostgreSQL upgrades
• strategic investment in the long period

32. Part III
Open source tools

33. ora2pg
• URL: http://ora2pg.darold.net/
• Written in Perl - GNU GPL 3
• Reads the information schema from Oracle
• Generates DDL instructions for PostgreSQL
• Highly configurable (mapping data type conversions)
• Basic support for PL/SQL (regexp)

34. orafce
• URL: http://orafce.projects.pgfoundry.org/
• C and SQL - BSD licensed
• Set of functions, objects and modules that are part of Oracle:
• NVL function (COALESCE)
• DBMS_ALERT, DBMS_OUTPUT, UTL_FILE, ...
• DUAL

35. Oracle FDW
• URL: http://oracle-fdw.projects.pgfoundry.org/
• SQL/MED
• uses the API for Foreign Data Wrapper
• CREATE FOREIGN TABLE
• Currently READ ONLY
• Useful for ETL

36. Example of Oracle FDW
...
CREATE FOREIGN TABLE oracle_table (
id integer NOT NULL,
...
) SERVER oradb OPTIONS
(schema 'ORAUSER', table 'ORATAB');
...
-- pushes down the WHERE clause
SELECT * FROM oracle_table WHERE id=10;

37. PgTap
• Unit tests suite for PostgreSQL
• Extensions (functions)
• Produces TAP output
• Test Driven Development in databases

38. Vagrant
• Open Source software
• Creation and configuration of virtual environments
• Provisioning: Puppet, Chef
• Useful for local testing

39. PostgreSQL
• Every major release of Postgres reduces the technology gap
with Oracle
• 5 years ago, Postgres did not have hot standby, streaming/
synchronous/cascading replication, window functions, CTEs,
column permissions, foreign tables, serialisable isolation
level, XML, JSON, updatable and materialised views ...
• Is it called “OraGIS” or “PostGIS”?
• Postgres does have transactional DDL

40. Business Continuity stack
App
repmgr
Master
Standby
server A
server B
Barman
DR
server C

41. If you really miss it ...
CREATE USER scott WITH PASSWORD ‘tiger’;
CREATE DATABASE scott WITH OWNER scott;
c scott scott
CREATE SCHEMA scott;

42. Part IV
Conclusions

43. Cost analysis of a database solution on a
period of 3 years for 1 server
License
Support
300.000
270.000
240.000
210.000
64.000
150.000
Costs in Euro
180.000
140.000
120.000
32.000
90.000
60.000
30.000
0
70.000
16.000
35.000
1 CPU Socket
2 CPU Socket
4 CPU Socket
* PostgreSQL with 24/7 support from us
7.500
PostgreSQL*

44. Cost analysis of a migration to PostgreSQL
in a 3 year period
Oracle
PostgreSQL
150.000
135.000
120.000
50.000
105.000
Costs in euro
90.000
75.000
60.000
45.000
70.000
30.000
2.500
0
Anno 1
2.500
16.000
15.000
16.000
Anno 2
Anno 3
* PostgreSQL with 24/7 support from us

45. Conclusions
• Final solution: 100% open
source
• reduced TCO
• No vendor lock-in
• Migration must be focused
on costs evaluation
• Supervised process (not
automated)
• If a feature is missing, you
can simply sponsor it
• If needed, professional
companies can help you
before, during and after the
migration process (you
won’t be alone)

46. Questions
E-mail: gabriele.bartolini@2ndQuadrant.it
Twitter: @_GBartolini_

47. Thank you!
Licenza Creative Commons BY-NC-SA 3.0
http://creativecommons.org/licenses/by-nc-sa/3.0/it/deed.it