More Related Content Similar to Oracle Data Integration - Overview (20) More from Jeffrey T. Pollock (11) Oracle Data Integration - Overview1. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle Data Integration
INTRODUCTION, CORE CAPABILITIES AND INVESTMENT ROADMAP
Jeff Pollock
Vice President of Products
Oracle Data Integration
September, 2016
#OpenWorld 2016
2. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
#OpenWorld 2016 2
3. Five Core
Capabilities
1. Business ContinuityDATA ALWAYS AVAILABLE
2. Data Movement
DATA ANYWHERE IT’S NEEDED
3. Data TransformationDATA ACCESSIBLE IN ANY FORMAT
4. Data GovernanceDATA THAT CAN BE TRUSTED
5. Streaming Data
DATA IN MOTION OR AT RES
4. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016 4
Eight Core
Products
Cloud or On-
Premise
5. Most
Innovative
Technology
#1
#1
Realtime / Streaming
Data Integration Tool
Pushdown / E-LT
Data Integration Tool
1st to certify replication with
Streaming Big Data
1st to certify E-LT tool with
Apache Spark/Python
1st to power Data Preparation
w/ML + NLP + Graph Data
1st to offer Self-Service &
Hybrid Cloud solution
6. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016
Data Integration Products
GOLDENGATE – DATA INTEGRATOR – DATA PREPARATION –
STREAM ANALYTICS – DATAFLOW ML – METADATA
MANAGEMENT – DATA QUALITY
7. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle GoldenGate
Realtime Performance
Extensible & Flexible
Proven & Reliable
Oracle GoldenGate provides low-impact capture, routing,
transformation, and delivery of database transactions
across homogeneous and heterogeneous environments in
real-time with no distance limitations.
Most
Databases
Data
Events
Transaction Streams
Cloud
DBs
Big
Data
Supports Databases, Big Data and NoSQL:
* The most popular enterprise integration tool in history
7
8. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Oracle Data Integrator
Bulk Data Performance
Non Invasive Footprint
Future Proof IT Skills
Oracle Data Integrator provides high performance bulk
data movement, massively parallel data transformation
using database or big data technologies, and block-level
data loading that leverages native data utilities
Bulk Data
Transformation
Most Apps,
Databases
& Cloud Bulk Data Movement
Cloud
DBs
Big
Data
1000’s of
customers –
more than other
ETL tools
Flexible ELT
workloads run
anywhere: DBs,
Big Data, Cloud
Up to 2x faster
batch processes
and 3x more
efficient tooling
8
9. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Self-Service
Better Recommendations
Built-in Data Graph Zero software
to install, easy
to use browser
based interface
Better
automation and
less grunt work
for humans
Graph database
of real-world
facts used for
enrichment
Oracle Data Preparation
ReportingApps
Files
ETL
Oracle Data Preparation is a self-service tool that makes
it simple to transform, prepare, enrich and standardize
business data – it can help IT accelerate solutions for the
Business by giving control of data formatting directly to
data analysts.
9
10. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Business Friendly
Extreme Performance
Spatial Awareness
Oracle Stream Analytics
DB
Web / Devices
Data
Event
Data & Transaction Streams
Downstream
(eg; Hadoop)
Data
Event
Oracle Stream Analytics is a powerful analytic toolkit
designed to work directly on data in motion – simple data
correlations, complex event processing, geo-fencing, and
advanced dashboards run on millions of events per
second.
Innovative dual
model for
Apache Spark or
Coherence grid
Simple to use
spatial and geo-
fencing features
an industry first
Includes Oracle
GoldenGate for
streaming
transactions
10
11. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Stream or Batch Data
Spark based Pipelines
ML-powered Profiling
Oracle Dataflow ML
Oracle Dataflow ML is big data solution for stream and
batch processing in a single environment – Lambda based
applications that can run streaming ETL for cloud based
analytic solutions.
Batch and
stream
processing at
the same time
Machine
learning guides
users for data
profiling
Data movement
across Oracle
PaaS services
Most Apps,
Databases
& Cloud
Bulk Data
Movement
Streaming Data Cloud
DBs
Big
Data
Big Data
Pipeline
11
12. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Business Glossary
End-to-End Lineage
100+ Supported Systems
Oracle Metadata Management
Oracle Metadata Management provides an integrated
toolkit that combines business glossary, workflow,
metadata harvesting and rich data steward collaboration
features.
Supports Databases, Big Data, ETL Tools, BI Tools etc:
BI Report Lineage
Taxonomy Lineage
Data Model Lineage
12
13. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Simple to Use
High Performance Matching
Powerful DQ Rules
Oracle Enterprise Data Quality
Oracle Enterprise Data Quality establishes trusted
business data by providing a foundation for data
profiling, data standardization, match and merge
capabilities and data cleansing.
Profile, Standardize,
Match, Merge and
Cleanse your data
DWMDM
Apps ETL
Health check for
your data; quick
& easy profiling
and cleansing
Intuitive
business user
friendly toolkit
for quality rules
Suitable for high
performance
Applications or
Master Data
13
14. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016
Powerful Cloud Solutions
For Data Integration
15. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016 15
Self-Service, Streaming or Batch Data Integration
Data Integration Cloud
Easy Self Service Data Preparation
Machine Learning and Semantic Technology
Streaming or Batch Integrations
• Modern Lambda and Kappa style streaming
applications for real-time data pipelines
• ETL integrations for classical Data Warehousing, Data
Marts and Operational Databases
• Real-time Replication with Changed Data Capture
• Point and click deployment of Replication services, ETL,
and Dataflow pipeline services
• Self-Service Data Preparation designed for non-
programmers to prepare rich data transformations
• Spark-based ML platform for Dataflow deployments
• High value Natural Language Processing (NLP) and
built-in data graph for Recommendation Service
Data
Preparation
Streaming
Applications
ETL Data
Processing
Ingest from Sources Automate Publish
16. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016 16
Comprehensive and Differentiated Data Integration in the Public Cloud
Data Integration Cloud – Core Capabilities
Ground to Cloud Data Streams Pushdown ELT/ETL
• GoldenGate realtime Replication
from on-premise to Cloud
• Data Integrator bulk data
movement into Cloud
• From on-premise to pipelines to
analytics in sub-seconds
• Ingest, move, transform and
analyze data 100% in memory
• Ultra-high performance ELT
processing in Hadoop or DBaaS
• Simple to choose which PaaS
you want the workload to run
streaming
batch
Pipelines Analytics
batch
data transformation
17. Five Cloud
Solutions
1. Data MigrationsREPEATABLE PROVEN DATA MIGRATION TOOLS
2. DW Integration
ENABLE CLOUD DATA WAREHOUSE STRATEGIES
3. Dev & Test CloudOPERATE DEV-TEST DB’S IN CLOUD
4. Data High AvailabilityDATA RELIABILITY AT 99.999% SERVICE LEVELS
5. Heterogeneous Cloud
BEST OF BREED FOR AMAZON AWS ETC.
18. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Key Benefits – Online/Offline Migrations
3rd Party Cloud Support
Database Migrations
• Seamless data migration from older databases or
poorly optimized Cloud services
• Take advantage of newest Oracle DB versions in a
simple subscription model
• No data loss for phased / online migrations
• Strong automation around provisioning and billing
• Reliable and proven core GoldenGate technology
• Migrate older Amazon RDS instances to Oracle
• Data delivery from on-premise to 3rd party Clouds
#OpenWorld 2016 18
Solution 1: Database Migrations in to Cloud
On Premise
RDS
Oracle DBaaS
I need to migrate my
DBs to newer and less
expensive systems
19. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Key Capabilities: Comprehensive Solution
3rd Party Cloud Support
Data Warehouse Integration
• Operational or Advanced Analytics in the Cloud
requires reliable data feeds
• Work with data coming from on-premise data sources
or other cloud sources
• Prepare and Ingest data in realtime; built in support for
replication and streaming ingestion to DW/Marts
• Transform and Cleanse data; innovative approach
allows workloads to run on different PaaS resources
• Supports Amazon RDS databases and Redshift
• Big data support with Kinesis and EMR
#OpenWorld 2016 19
Solution 2: Data Warehouse Integration in the Cloud
DW on Exadata
Cloud Service
UNSTRUCTUREDSTRUCTURED
BI & Data
Visualization
RDS & REDSHIFT
Help me run
my analytics
from the Cloud
20. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Key Benefits: Automate Data Movement
Oracle Database as a Service
Database Dev-Test Environments
• Keep development and test environments continuously
in sync with minimum infrastructure or process
overhead
• Leverage automation rather than manual syncs
• Low Overhead reduces manual and complex scripting
• Repeatable approach standardizes DBA processes
• Simple framework can be used across a variety of DBs
• All Supported Oracle Versions regardless of what you
have running on premise
• Mix and match versions or patch levels
#OpenWorld 2016 20
Solution 3: Dev & Test Systems in the Cloud
On Premise
Oracle DBaaS
12.1 & 12.2
10g & 11g
Make my DBA job
easier and more
productive
21. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Key Benefits for DBaaS
Powerful Cloud Solution
Regional High Availability
• When you can’t afford to have your mission critical
databases take downtime
• Particularly necessary when disaster recover is
necessary across regional datacenters
• Standby Databases can be always on with Active Data
Guard in the Cloud
• Active-Active can enable you to take read/write
transactions on all databases.
• Strong Automation of service provisioning and billing
• Oracle or Amazon Cloud operate active-active Oracle
Database in any Cloud across regional data centers
#OpenWorld 2016 21
Solution 4: High Availability for DBaaS in the Cloud
Oracle DBaaS
Datacenter Region 1
Active / Active
Oracle DBaaS
or Amazon RDS
Datacenter Region 2
On Premise
My business
applications
can not go
down!
22. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Heterogeneous and Complete Solution
Bring Enterprise-strength to AWS
• Advanced data integration and availability capabilities
for Kinesis, EMR, Redshift and RDS
• Proven data transfer rates of 10’s of thousands of DB
transactions per second
• Security controls work correctly across Amazon secure
data centers
• Flexible solution can leverage DB or Big Data from
within the AWS services
• Most Trusted data replication engine over all others
• Top most popular ETL/ELT tool in worldwide market
• Fastest growing data integration for big data
• Oracle Data Integration is a heterogeneous capability
that works across different data platforms and different
cloud platforms.
#OpenWorld 2016 22
Solution 5: Make Amazon AWS Enterprise Grade
Oracle Database
Active-Active
Realtime Big Data
GoldenGate
Datacenter A Datacenter B
GoldenGate
Data Integrator
23. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016
Big Data Solutions
For Data Integration & Governance
24. Pragmatic
Big Data
Solutions
1. Data IngestionINDUSTRIAL STRENGTH DATA INGESTION LAYER
2. Transformations
FUTURE-PROOF TOOLING FOR CODE GENERATION
3. GovernanceMETADATA LINEAGE AND BUSINESS GLOSSARY
4. ConnectorsHIGH SPEED PARALLEL LOADERS FOR BULK DATA
5. Streaming Big Data
INGEST & ANALYTICS 100% IN-MEMORY
25. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016 25
Business
Data
Serving
Layer
Apps
Analytics
EDWsRaw
Data
Staging
Data Discovery
ETL Offload
Batch Layer
Data Streams
Social and Logs
Enterprise Data
Highly Available
Databases
Databus
(Pub/Sub)
Streaming Analytics
Streaming Data Pipelines
...Speed Layer data broadcasting & streaming data processing
...Batch Layer data processing for huge data volumes
...Serving Layer for consumption
Pub / Sub
REST APIs
NoSQL
Bulk Data
Vision = Lambda/Kappa
Databus
(Pub/Sub)
Speed Layer
26. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016 26
Hybrid Open-Source
...Open Source at the core of speed & batch processing engines
...Enterprise Vendor tools for connecting to existing IT system and
...Cloud Platforms for data fabric
Business
Data
Serving
Layer
Apps
Analytics
Batch Layer
Data Streams
Social and Logs
Enterprise Data
Highly Available
Databases
Pub / Sub
REST APIs
NoSQL
Bulk Data
Speed Layer
Raw Data Stream Processing
Batch Processing
Prepared Data
27. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Examples
#OpenWorld 2016 27
Reference Architecture
Business
Data
Serving
Layer
Apps
Analytics
Batch Layer
Data Streams
Social and Logs
Enterprise Data
Highly Available
Databases
Pub / Sub
REST APIs
NoSQL
Bulk Data
Speed Layer
GoldenGate
Data Preparation
Data Quality, Metadata Management & Business Glossary
Oracle Data Integrator
Active DataGuard
Comprehensive architecture covers key areas – #1. Data Ingestion, #2. Data Preparation &
Transformation, #3. Streaming Big Data, #4. Parallel Connectivity, and #5. Data Governance –
and Oracle Data Integration has it covered.
Dataflow ML
Stream Analytics
Connectors
28. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
ApplicationsApplications DatabusApplications Speed Layer
Batch Layer
Capture
Trail
Route
Deliver
Pump
#OpenWorld 2016 28
Streaming Analytics
Application
Serving
Layer
REST
Services
Visualization
Tools
Reporting
Tools
Data Marts
User
Updates
DBMS
Updates
GoldenGate for Ingest
GG GG
Applications Serving
Layer
Speed Layer
Batch Layer
Platforms
29. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016 29
ODI for Transformations
ETL Engines
Big Data Frameworks
Speed Layer
Batch Layer
Serving
Layer
ApplicationsApplications DatabusApplications
Application
REST
Services
Visualization
Tools
Reporting
Tools
Data Marts
User
Updates
DBMS
Updates
Applications Serving
Layer
Speed Layer
Batch Layer
Oracle Data Integrator
Spark Streaming
Spark SQLSqoop
ERP
Oozie
Pig
Hive
Loaders
Kafka
NoSQL
OGG
SQL
30. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Data Catalog
Speed Layer
Batch Layer
Serving
Layer
#OpenWorld 2016 30
OEMM for Data Governance
ApplicationsApplications DatabusApplications
Application
REST
Services
Visualization
Tools
Reporting
Tools
Data Marts
User
Updates
DBMS
Updates
Applications Serving
Layer
Speed Layer
Batch Layer
Kafka
Generated Streaming
Generated ETL CodeSqoop
OLTP Databases
HDFS Files
HCatalog
Hive
NoSQL
ETL
Tools
Data Warehouses
BI Models
ER Models
Oracle Enterprise Metadata Management
140+ Supported Tools
31. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
from Devices
Batch Layer
#OpenWorld 2016 31
Streaming Data
ApplicationsApplications
Databus
Applications
Speed Layer Serving
Layer
REST
Services
Visualization
Tools
Reporting
Tools
Data Marts
Applications
Serving
Layer
Speed Layer
Batch Layer
Oracle Stream Analytics
Oracle Dataflow ML
Oracle GoldenGate
Application
ApplicationsApplicationsDevices
from Databases
32. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016
World Class Solutions
For Data Integration & Governance
33. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016 33
140+ Open Source, Standards and Third Party Integrations
Adaptive
Altova
Apache HCatalog
Apache Hive/HQL
Borland
CA ERwin
Cloudera Impala
COBOL Copybook
DataStax
Embarcadero
EMC ProActivity
GentleWare
Google BigQuery
Grandite
Hadapt Hive
Hortonworks Hive
IBM Cognos
IBM DB2
IBM DataStage
IBM Discovery
IBM Federation Server
IBM Lotus Notes
IBM Netezza
IBM Rational Rose
IBM Rational Architect
Informatica Metadata Manager
Informatica PowerCenter
CoSORT
ISO SQL Standard (DDL)
MapR Hadoop Hive
MicroFocus
Microsoft Access
Microsoft Office Excel
Microsoft Visio
Microsoft SQL Server
Microsoft SSIS
Microsoft Visual Studio
Microstrategy
Magic Draw
OMG CWM Standard
OMG UML Standard
Oracle BI Answers
Oracle BI Enterprise Edition
Oracle BI Server
Oracle DAC
Oracle Data Integrator
Oracle Data Modeler
Oracle Database
Oracle Designer
Oracle Hyperion Applications
Oracle Hyperion Essbase
Oracle Warehouse Builder
Pivotal Greenplum
PostgreSQL
QlikView
SAP BO Crystal Reports
SAP BO Designer
SAP BO Desktop Intelligence
SAP BO Repository
SAP BO Data Integrator
SAP BO Data Steward
SAP Master Data Management
SAP Sybase PowerDesigner
SAP Sybase ASE Database
SAS Data Integration Studio
SAS BI Server
SAS Information Map
SAS Metadata Management
SAS OLAP Server
Select
Sparx Architect
Syncsort
Tableau
Talend
Teradata
Tigris
Visible
W3C DTD & XSD Schema
Metadata Harvesting (Glossary, Lineage & Impact Analysis)Key Standards
34. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016 34
Data Integration Cloud
Data Integration
SaaS Data
Quality
DaaS Data
Quality
ODI on Big Data
Cloud Service
Data
Preparation
ODI Cloud
Service
Dataflow ML
Cloud Service
EDQ Cloud
Service
GoldenGate
Cloud Service
35. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016 35
• Use Starbucks loyalty card to buy a Latte
• Pay for a purchase with PayPal
• Withdraw cash at an ATM from Bank of America, or Citibank, or Wells Fargo,
or CapitalOne, or American Express, or JP Morgan Chase, or… most other
banks in the world
• Click on an Google AdWords link
• Select a recommended item from EBay.com
• Buy something from Amazon
• Use a store coupon from Safeway
• Browse the catalog from Overstock.com
• Check flight availability on American Airlines
• Use the Cartwheel App at Target stores
• Take your General Motors car in for service
• Go shopping at Macy’s online store on Black Friday
• Deposit your payroll check issued by Paychex
• Take care of your small business accounting on Quickbooks.com
• Buy a song on Apple iTunes
• Change your profile or add a skill on LinkedIn.com
Everyday Moments
36. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Presen-
tations
on:
#OpenWorld 2016 36
Data Integration Solutions Program - tinyurl.com/DISOOW16
Demo
Stations:
Hands-
on labs:
Oracle
Enterprise
Metadata
Management
Oracle
Enterprise
Data Quality
Oracle
GoldenGate
Oracle
Data
Integrator
Oracle
Big Data
Preparation
Cloud Service
Oracle
Enterprise
Data Quality
HOL7466
Oracle
GoldenGate
Deep Dive
HOL7528
ODI and OGG
for Big Data
HOL7434
Oracle Big Data
Preparation
Cloud Service
HOL7432
Middleware
Demoground
- Moscone South
Big Data
Showcase
- Moscone South
Database
Demoground
- Moscone South
37. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | #OpenWorld 2016 37
Data Integration Solutions Program - tinyurl.com/DISOOW16
Monday, Sept 19
• Oracle Data Integration Solutions – Platform Overview and Roadmap
[CON6619 ]
• Oracle Data Integration: the Foundation for Cloud Integration [CON6620 ]
• A Practical Path to Enterprise Data Governance with Cummins [CON6621]
• Oracle Data Integrator Product Update and Strategy [CON6622]
• Deep Dive into Oracle GoldenGate 12.3 New Features for the Oracle 12.2
Database [CON6555]
Tuesday, Sept 20
• Oracle Big Data Integration in the Cloud [CON7472]
• Oracle Data Integration Platform: a Cornerstone for Big Data [CON6624]
• Oracle Data Integrator and Oracle GoldenGate for Big Data [HOL7434]
• Oracle Enterprise Data Quality – Product Overview and Roadmap
[CON6627]
• Self Service Data Preparation for Domain Experts – No Programming
Required [CON6630]
• Oracle Big Data Preparation Cloud Service: Self-Service Data Prep for
Business Users [HOL7432]
• Oracle GoldenGate 12.3 Product Update and Strategy [CON6631]
• New GoldenGate 12.3 Services Architecture [CON6551]
• Meet the Experts: Oracle GoldenGate Cloud Service [MTE7119]
Wednesday, Sept 21
• Data Quality for the Cloud: Enabling Cloud Applications with Trusted Data
[CON6629]
• Transforming Streaming Analytical Business Intelligence to Business
Advantage [CON7352]
• Oracle Enterprise Data Quality for All Types of Data [HOL7466]
• Oracle GoldenGate for Big Data [CON6632]
• Accelerate Cloud On-Boarding using Oracle GoldenGate Cloud Service
[CON6633]
• Oracle GoldenGate Deep Dive and Oracle GoldenGate Cloud Service for Cloud
Onboarding [HOL7528]
Thursday, Sept 22
• Best Practices for Migrating to Oracle Data Integrator [CON6623]
• Best Practices for Oracle Data Integrator: Hear from the Experts [CON6625]
• Dataflow, Machine Learning and Streaming Big Data Preparation [CON6626]
• Data Governance with Oracle Enterprise Data Quality and Metadata
Management [CON6628]
• Faster Design, Development and Deployment with Oracle GoldenGate Studio
[CON6634]
• Getting started with Oracle GoldenGate [CON7318]
• Best Practice for High Availability and Performance Tuning for Oracle
GoldenGate [CON6558]
38. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Connect with Oracle Data Integration
@OracleDI
Blogs.oracle.com/DataIntegration/
Oracle Data Integration
Oracle Data Integration
#OpenWorld 2016 38
40. Copyright © 2016, Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The preceding is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, and timing of any features or
functionality described for Oracle’s products remains at the sole discretion of Oracle.
#OpenWorld 2016 40