Oracle Data Integration And Data Governance

1. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Oracle Data Integration & Data Governance
Business Analytics & Big Data OPN Forum Dubai 15.11.2015.
EMEA Central Team - Oracle Data Integration Solutions
Milomir Vojvodic, Business Development Director

2. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Oracle GoldenGate
Real Time Replica Between Any Databases
Oracle Data Integrator
ETL Platform
Active Data Guard
Disaster Recovery Of Oracle DB EE
Enterprise Data Quality
Data Quality Platform
Enterprise Metadata Manager
Data Governance Platform
ODI Management Pack
new SKU
EDQ Real Time Processing
new SKU
EDQ Audit & Dashboard
new SKU
OEMM
new SKU
Recent additions of DIS SKUs in pricelist
Big Data Prep CS (Product Code IMI)
Data Wrangling on the cloud
From GoldenGate SW
Acquisition in 2009 to OGG 12c
From Sunopsis Acquisition in
2007 to OD 12c
From Datanomics Acquisition in
2011 to EDQ 12c 2014
New DI Product announced at
OOW 2014
New DI Product announced FY
2015
New DI Product to be
announced CY 2016
Oracle GoldenGate Cloud Service
CDC on cloud
Recent additions of DIS products
, ODI Advanced Big Data Option
new SKU
Oracle Data Integration Solutions Dynamics
OGG for Big Data
new SKU
BDP CS
new subscription based SKU
OGG CS
new subscription based SKU

3. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Oracle GoldenGate For Big Data

4. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Target DB
OGG
Source DB
First OGG Differentiator Accessing
directly transaction logs
Second OGG Differentiator Moving only
committed transactions
Why Is OGG Different?
After OGG Before OGG
Reduce source system overhead
(and costs for stronger HW) by
70%
After OGG Before OGG
Reduce costs and efforts of
data loss by 80%

5. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
• Unlocking the most important data due to non-invasive changed data
capture at source
Hadoop As Target DBSource DB
What Is So Special About OGG for Big Data?

6. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Oracle Data Integrator For Big
Data

7. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
New DB/HW/OS/APP
Fully Active DB
Reporting Database and/or DR database
Data Warehouse
OGG
OGG
OGG ADG
OGG
Zero Downtime
Migrations & Upgrades
Active/Active High
Availability
Query Off-Loading
and Disaster Recovery
BI&DW
Synchronization and Loading
Migrations & Consolidations
ODI
ODI
Use Cases Of ODI

8. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
OLTP & ODS
Systems Data
Warehouse, Data Mart
Oracle
PeopleSoft, Siebel, SAP
Custom Apps
Files
Excel
XML
Enterprise
Performance
Custom Reporting Packaged
Applications
Business
Intelligence
Analytics
Data
Federation
Data
Warehousing
Custom
Data Marts
Data Access
Data Silos
SQL
Java
Batch Scripts
Data Hubs
Data
Migration
Data
Replication
OLAP
Replacing Manual Coding

9. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
OLTP & ODS
Systems Data
Warehouse, Data Mart
Oracle
PeopleSoft, Siebel, SAP
Custom Apps
Files
Excel
XML
Enterprise
Performance
Custom Reporting Packaged
Applications
Business
Intelligence
Analytics
OLAP
Oracle Data Integrator
Replacing Manual Coding
After ODI Before ODI
Reduce data transformation
maintenance costs by 80% (hard
to change, every script contains special
rules, code stored in many machines)

10. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
ODI E-LT
Staging Server
Data Warehouse
Second ODI Differentiator
ODI Declarative Design and
ODI Knowledge Modules for
reusing already written down
level SQL code
First ODI Differentiator
Transformations using
the power of the Target
Database – no staging
server
Why Is ODI Different?
After ODI Before ODI
Reduce ETL
development
costs by 30% (no
prebuilt code, need
to learn various
languages, need to
write and tune SQL)
After ODI Before ODI
Decrease the cost o of
ETL HW by 100%
ODI

11. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
• ODI Is Writing Big Data Code
• SW Project Lifecycle + Data Semantic Related Functionalities
• Reusability Principle
What Is So Special About ODI for Big Data?
Hadoop As Target DBSource DB
ODI

12. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Enterprise Data Quality

13. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
New DB/HW/OS/APP
Fully Active DB
Reporting Database and/or DR database
Data Warehouse
OGG
OGG
OGG ADG
OGG
Zero Downtime
Migrations & Upgrades
Active/Active High
Availability
Query Off-Loading
and Disaster Recovery
BI&DW
Synchronization and Loading
Migrations & Consolidations
ODI
ODI
Use Cases Of EDQ
EDQ
EDQ

14. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Customer ID Customer Name Address 1 Address 2 City State Zip Country Birth Date Gender
AD23298 Mr Peter Mayhew 9407 Main St Fairfax VA 22031-4001 USA 02/23/61 M
VS38611 Dr Ellen Van Der Heijde 144 E Grove St Kingston PA 18704 US 07/12/57
DC18223 Jalila Abdul-Alim (Do Not Call) 4548 Pennsylvania Ave Apt 205 Kansas City MO 64111-3349 USA 02/23/63 F
CO9387A Tayside Computers Inc. 4912 E 41st N Idaho Falls ID 83401 USA 31/03/2007 N/A
TZ35019 Mr Zachary P Jahn 98-1731 Ipuala Loop Aiea Hawaii 96701 1710 United States 06/12/86 Male
CB27843 Mrs Edith Y Baba Junior Baba Real Est. Corp. 209 Stony Point Trl Webster NY USA 11/17/1971 M
OX80306 Andrew & Mary Baxter 14 Oxbridge Way Milfrod NH 03055-4614 US 05/28/67 F
JP70210 Mr RJ & Mrs FB MacDonald 57 Hadleigh Close Westlea Swindon SN5 9BZ MA - USA - Y
RD48107 Mr Andy Baxter 14 Oxbridge Wy Milford NH 3056 USA 01/01/01 M
Inconsistent formats
Abbreviations
(often ambiguous)
Attributes non-standard, missing
or invalid
Widespread
duplication
(often hard
to spot)
Compound Names
Embedded Additional Information
Mixed Business & Personal Names
Multiple Names
Mis-Fielded Data
Erroneous Data
International Date Formats
Default or Dummy Data
Why EDQ?

15. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

16. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

17. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

18. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

19. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

20. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

21. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

22. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

23. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

24. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

25. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
• A Match Rule is simply the combination of comparison results
• Rules are evaluated in order and if one hits, we stop
• Rules can be ‘negative’ to eliminate pairs that are too different with a
‘No Match’ rule
• Rules can easily be turned on & off during the tuning process

26. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

27. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

28. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

29. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
• Let’s try the ‘brute force’ approach to
de-duplicating 10 million records:
• Start at record 1 and compare it with:
– Record 2, Record 3, … , Record 10
million
• Now move onto record 2 and compare
it with:
– Record 3, Record 4, … , Record 10
million
Why EDQ?
• This could take some time…
• The number of comparisons is about
– Half of 10 million x 10 million, which is
– 50,000,000,000,000!
• If a server can do 100,000 per second it
will take
– 500,000,000 seconds OR
– 138889 hours OR
– 15.85 years
• Which is rather too long to wait!
• So we need to work a bit smarter…

30. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Metadata Management As Data
Governance Foundation

31. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
New DB/HW/OS/APP
Fully Active DB
Reporting Database and/or DR database
Data Warehouse
OGG
OGG
OGG ADG
OGG
Zero Downtime
Migrations & Upgrades
Active/Active High
Availability
Query Off-Loading
and Disaster Recovery
BI&DW
Synchronization and Loading
Migrations & Consolidations
ODI
Use Cases Of OEMM
OEDQ
Oracle GoldenGate
Real Time Replica Between Any Databases
Oracle Data Integrator
ETL Platform
Active Data Guard
Disaster Recovery Of Oracle DB EE
Enterprise Data Quality
Data Quality Platform
Enterprise Metadata Manager
Data Quality Platform
OEMM
ODI OEDQ
OEMM

32. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
ETL
BI
Dashboards
App
ETL
ETL
How was sales figure
calculated?
What will happen if I
change this table?
What reports use
the mainframe
data?
Sys Admin
Executive
BI Developer
Where did this
data come
from?
Application User
Which reports use
this customer
data?
CDC
Hadoop
Data Lake
Data Steward
Can I trust the
sources of this
customer data?
ETL
Developer
I want to design an experiment to
measure the success of a
signup page. What data do I
have?
Data Scientist
GG
Data Warehouse
OGG
BI&DW
Synchronization and Loading
ODI OEDQ
OEMM
After OEMM
Before OEMM
OPEX - Reduce data maintenance costs
by 80% (not anymore.. hard to
change, long to find)
Value Of Metadata Management
After OEMM
Before OEMM
CAPEX - Reduce analytical project costs
by 30% (not anymore ..post
remediation costs, unnecessary
mistakes )

33. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

34. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

35. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

36. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

37. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

38. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

39. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

40. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

41. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

42. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

43. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Metadata Repository
Any Web
Browser
OEMM Architecture

44. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
1. Title - the title of the data object
2. Creator - the person or entity responsible for creating the data object
3. Subject - subject terms or keywords that describe the data object
4. Description - a brief description, or abstract, of the data object
5. Publisher - the entity responsible for making the data object available
6. Contributor - a person or entity who contributed to the creation of the data object
7. Date - data of creation, publication, or revision of the data object
8. Type - the type of object. For data this would typically be "dataset"
9. Format - a description of the format or file type(s) of the data object
10. Identifier - a permanent identifier used to locate and identify the data object
11. Language - the language(s) used within the data object (if applicable)
12. Source - a relational element describing the lineage of the data object
13. Relation - a relational element describing the relationship of this data object to other objects,
collections, or entities
14. Coverage - describes the spatial and temporal context of the data object
15. Rights - describes any rights, restrictions, or terms of use
Value Of Metadata Management

45. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Business Glossary Stewardship Workflow

46. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Business Glossary Manages Definitions

47. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Mapping of Business Term to Rules

48. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Capture Business Rule Definitions

49. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Semantic Linking of Business Rules

50. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Allowable Values of Business Terms

51. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
 Both : Big Data and Cloud introduces a new data silo, adding complexity to
existing lack of formal definition. This makes data hard to report, study and
reuse, unless they can be referenced from common semantic metadata
repository.
What Is So Special About OEMM for Big Data & Cloud?
OEMM

52. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |
Oracle Big Data Prep CS

53. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

54. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

55. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

56. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

57. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

58. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

59. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

60. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

61. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

62. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

63. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

64. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

65. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

66. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

67. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

68. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

69. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

70. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

71. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

72. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |

73. Copyright © 2015 Oracle and/or its affiliates. All rights reserved. |