SlideShare a Scribd company logo
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
OpenWorld 2017
Intelligent Integration and Governance
Next-Gen Enterprise Data Management
Greg Pavlik
Senior Vice President & CTO of PaaS
Jeff Pollock
Vice President, Product Management
Confidential – Oracle Internal/Restricted/Highly Restricted
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Data Management is going through a
major transformation…
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential
The future is already here; it’s just not very
evenly distributed. - Gibson
Enterprise data architecture
is a hub-and-spoke Kimball-
style solution
Big data technologies are
mostly for analytic use cases
that demand the 3 V’s
IT will provide the single
source of truth of the data
TRADITIONALAPPROACH
EMERGINGSOLUTION
Data Streams in Motion
Big Data is for Analytic and
Operational IT Use Cases for
Any Type of Data
Trusted Data means
Raw Data
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Discovery
RESTful API for Producers and Subscribers (events are pushed)
Raw Data
Topics
Schema
Event Topics
Data
Pipeline
(ETL)
Prepared
Data Topics
Master Data
Topics
Data
Pipeline
(ETL)
1,000’s 100’s 10’s
Oracle Open World 2015 4
App
DB
App
DB
App
DB
ERP
Operational
Data Store
EDW
Staging Prod
ETL
ETL
ETL
ETL
ETL
Mart
Mart
Mart
ETL
Enterprise BI
Mart
Mart
Mart
ETL
Departmental BIDiscovery
App
DB
App
DB
App
DB
ERP
WebApps
Mobile
EDW
NoSQL
Hadoop / Spark
Marts Marts
Less Governed --------------------------------------------------------------- More Governed
Enterprise BI
Departmental BI
Apps / Mobile
Classical Data Management: Hub and Spoke
• Invasive on Sources
• High Latency / SLA
• Mainly Relational Views
• Heavy IT process overhead
• Vendor-centric software
Next-Gen: Streaming Databus/Kappa
• Low impact on Sources
• Low Latency (< 1 second)
• Variety of Data Formats
• More Agile DevOps processes
• Open source centric software
GoldenGate
MDM
Hub
After 20yrs reign… hub-and-spoke is now a legacy
• ODS & ETL Hubs
• EDW/Mart Hubs
• MDM/RDM Hubs
• Static Data Lake Hubs
• Pub/Sub for Staging
• ETL in Pipelines
• Analytics/CEP in Stream
• Data is in Motion
NoSQL / APIs
LEGACY:
NEXT-GEN:
Less Governed ---------------------------------------------------- More Governed
Physical Layer for ETL Pipelines = MPP Streaming (eg; Apache Spark Streaming)
Physical Layer for Events = MPP Messaging (eg; Apache Kafka)
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 5
In the beginning there was MapReduce …and it was slow
• Original purpose for workloads on
peta-scale search indices
• Rise of Enterprise use cases, focus
shifted to Analytic Data
• 1st gen of big data in enterprise
reused old approach (hub-and-
spoke) using HDFS with MR2
• Operational Applications can use big
data for low-latency use cases
• Streaming for event correlation and
IoT style integrations (real-time)
• Analytics for traditional use cases
• Cloud/Object Storage is “the lake”
Next-Gen: Kappa is the Platform
Multi-
Structured
Data
Structured
Data
Applications
Analytics
Reporting
Batch Layer
Speed Layer
Raw Data
Layer
Serving
Layer
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 6
Trusted data used to mean …single source of truth
Extract Cleanse Stage
Summarize Create Views
Hello, is
this the IT
team? Yes, I need a
single source
of truth for
my data
Um, 2
years?
• IT heavy waterfall delivery cycle
creates drag on the business
• Over-optimizing data distorts truth
and diminishes fidelity of data
• Data is like food, trust it more
when it is minimally processed
• Raw Data accessible to business
• Subscriptions to data objects
• API Access REST API vs. SQL
• Agile DevOps process to deliver
• Schema/less can handle all data
Next-Gen: Sushi Principle of Data
1990’s
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Data
Staging
or Archive
Data Discovery
ETL Offload
Batch Layer
Oracle Confidential 7
Business
Data
Analytics
EDWs
Data Streams
Social and Logs
Enterprise Data
Highly Available
Databases
Databus
(topic modeling)
Stream Analytics
ETL Data Pipelines
Speed Layer
Our Vision is to enable the modern ‘Kappa style' data architecture for Enterprise Strength solutions
• Raw Data Layer common ingestion point for all enterprise data sources
• Speed Layer data processing for streaming data and ETL data pipelines, in-memory
• Batch Layer data processing for huge data volumes, that may span long time periods, using MPP
• Serving Layer technologies for easy access to any data, at any latency
Raw Data
Layer
Raw Events
Changed Data
Schema Events
Core Design Pattern: Kappa-style Databus
Pub / Sub
REST APIs
NoSQL
Bulk Data
Serving
Layer
Apps
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 8
Business
Data
Serving
Layer
Apps
Analytics
EDWs
Batch Layer
Data Streams
Social and Logs
Enterprise Data
Highly Available
Databases
Analytics
Speed Layer
Pub / Sub
REST APIs
NoSQL
Bulk Data
Raw Data
Layer
Oracle Approach: Blend of Commercial + Open Source
Modern Architecture will be a ‘Hybrid Open-Source’ pattern:
• Open Source at the core of speed and batch processing engines for general purpose data workloads
• Enterprise Vendors for connecting to legacy systems, strong governance, and for highly optimized workloads
• Cloud Platforms for Dev-Test (at least), rapid prototyping and eventually all production workloads
• SaaS & Applications are key data “producers” and will remain largely proprietary and/or highly customized
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential
Proof this is a Pattern: Many Instantiations
Kafka
Storm | Spark
| Apex | Flink
MapReduce | Pig
| Hive | Spark
Cassandra
| HBase
Hive
EventHubs
Stream Analytics
Data Lake
Table
Storage
SQL Server
Data
Factory
Kinesis
Firehose
EMR
Dynamo
Redshift
DMS
Pub/Sub
Dataflow
Dataproc
BigTable
BigQuery
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 10
Business
Data
Serving
Layer
Apps
Analytics
EDWs
Batch Layer
Data Streams
Social and Logs
Enterprise Data
Highly Available
Databases
Analytics
Speed Layer
Pub / Sub
REST APIs
NoSQL
Bulk Data
Raw Data
Layer
Best-of-Breed: Oracle Platform for Kappa-style Architecture
Oracle Software can help customers Accelerate & Reduce Risk around adoption:
• Ingest Data with lower latency, greater reliability and from any database using Oracle GoldenGate
• ETP Pipelines for Data automate pipeline creation with zero-footprint using Oracle Data Integrator
• Analyze Data In-Motion run temporal, spatial and predictive algorithms with Oracle Stream Analytics
• Foundation Services for hosting Kafka (Event Hub) Spark/Hadoop (Big Data Cloud) or Relational (Database)
• Govern the data flowing through Kappa architecture with Oracle Metadata Management
GoldenGate
Data Integrator
Stream Analytics
Event Hub Big Data Database
Metadata Management (for Data Governance)
Cassandra
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential
Oracle Big Data Platform Differentiators
• Industrial strength data ingest
• Streaming data pipelines
• ETL pipelines
• Analytic pipelines
• World class infrastructure
• Event hub cloud – Kafka
• Big data cloud – Spark
• Enterprise data governance
• Compare to sqoop / ETL tools
• Compare to old-style:
• Batch ETL processing
• Immature open-source
• Compare to:
• 1st Gen Cloud – AWS etc.
• Proprietary services
• Compare to Hadoop only
Complete
Simplified
Open
TECHNICALDIFFERENTIATION
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Raw
Data
Layer
Apps Layer Speed Layer
Batch Layer
Oracle Confidential 12
Industrial Strength Data Ingest: GoldenGate + Kappa
Streaming Analytics
Application
Serving
Layer
REST
Services
Visualization
Tools
Reporting
Tools
Data Marts
Capture
Trail
Route
Deliver
Pump
GG GG
User
Updates
DBMS
Updates
GoldenGate
for Big Data
Supported
Platforms
Kafka
HDFS
Fastest, most scalable and non-invasive way to ingest data into Apache. Benefits of
low-impact on Sources, micro-second access to transactions and ability to replicate
schema (DDL) events for downstream automation of change impact.
GG used with 4 of top 5 largest Kafka clusters in the world…
From user update
to serving layer in
<1 second & no
impact on Source
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 13
De-Coupling of the Database: Downstream Processing
Mid-Tier HostPrimary Site
Primary
Secondary Log Mine
GoldenGate
Capture
Trail
Route
Deliver
Pump
Business Apps
Active
DataGuard
WAN
REDO
Transport
Remote DR HostPrimary Site
Primary
Secondary
Remote
Standby
GoldenGate
Capture
Trail
Route
Deliver
Pump
Business Apps
AlwaysOn
WAN
AlwaysOn
Primary Site
DB2/z
Business Apps
LPAR / Linux Host
SP GoldenGate
Capture
Trail
Route
Deliver
Pump
OLD Approach: “Just the Facts!”
• On-host installation of Change Data
Capture (CDC) tools
• For typical DW use cases, only partial
data sets were replicated
• Focus was on summary data in the
analytic model, not the detail data
NEW Approach: “All the Data!”
• Mid-tier installation of replication
tools is necessary (non-invasive)
• Focus is now on moving all the data
into the data lake/data stream
• Raw data means detail data
• New use cases demand access to
more expansive (raw) data sets Primary Site
MySQL
Business Apps
Linux Host
GoldenGate
Capture
Trail
Route
Deliver
Pump
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Raw
Data
Layer
Speed Layer
Batch Layer
Oracle Confidential 14
ETL Pipelines with Data Integrator
Streaming ETL
Serving
Layer
REST
Services
Visualization
Tools
Reporting
Tools
Data Marts
Oracle Data Integrator
Capture
Trail
Route
Deliver
Pump
GG
SQOOP
API/File
SQOOP
+ Native Loaders
Data Integrator for Big Data
✓ Batch data ingestion with Sqoop,
native loaders & Oozie
✓ Generate data transformations in
Hive, Pig, Spark & Spark
Streaming
✓ Extract data into external DBs,
Files or Cloud
Compare to Informatica / Talend
✓ NoETL Engine native E-LT
execution, 1000’s of references
✓ Zero Footprint does not require
any Oracle install on cluster
✓ Loosely Coupled design time
means you can reuse mapping
logic in many big data languages
ETL
ETL
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Raw
Data
Layer
Speed Layer
Batch Layer
Streaming Analytics
Serving
Layer
REST
Services
Visualization
Tools
Reporting
Tools
Data Marts
Oracle Confidential 15
Analytic Pipelines with Stream Analytics
Capture
Trail
Route
Deliver
Pump
GoldenGate
CQL
Oracle Stream AnalyticsEvents &
Cloud Apps
Business
Dashboard
Native GoldenGate
Stream Type
Analytics
EDWs
AppsML
Geo
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 16
World-class Infrastructure for Big Data
• Elastic workload fabric to lower TCO
and right-size cluster based on
workload/SLA requirements
• A seamless storage fabric across
Cloud Storage, Block Storage, Local
NVMe and Memory for your Big Data
Clusters
• A broad choice of Storage engines
including relational databases (Oracle,
MySQL), NoSQL Databases
(Cassandra, HBase, Oracle NoSQL)
• REST or Native APIs deliver an
extremely low latency for real-time
processing needs.
• Start with a single partition and scale
up to hundreds of them as your
needs grow.
• Complexity of managing Kafka is done
by Oracle.
• Node-Failure and Cluster-Failure
Tolerance to ensures continuous
availability
• Big Data Cloud Machine for PaaS-like
experience in customer data center
• Same benefits and user
experience as Oracle PaaS
• Data stays onsite with customer
• Big Data Appliance for customer-
owned solutions.
• Customer owns full solution
• Very low startup costs
• Highly optimized platform
Oracle Event Hub Cloud Oracle Big Data Cloud
Oracle Bare Metal Cloud
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential
Kappa Data Flow Pattern using Oracle Tech Stack
GoldenGate
Raw Data (LCR)
Schema Events
(DDL)
Prepared Data Topics
Master Data
ETL ETL
1 Topic : 1 Table
Data Consumers
<subscribe>
Applications
Streaming Analytics
ODS (Data Store)
Big Data Lake
Data Warehouses
CQL & Spatial
Analytic Data
Oracle Event Hub
DBMS
Updates
Data Producers
Entire Enterprise
Database Estate
Stream Analytics
Data Integrator
Dev / Test Env.
Oracle Big Data
<generate>
<generate>
APIManagement
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
All Enterprise
Data Sources
Oracle Confidential 18
Data Views from a Pipeline: Subscribe to Canonical Topics
Poly-
Structured
Relational
Data
Science
Data
Analysts
Business
Analyst
DBAs
RAW
DATA
PREPARED
DATA
MASTER
DATA
SCHEMA
EVENTS
ETL ETL
<subscribe>
<subscribe>
<subscribe>
<subscribe>
<produce>
<produce>
<produce>
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Automate & Transform
• Intuit Migrated 100’s of DBs to
Cloud and Loads TB’s of data into
Streaming data fabric (Kafka)
• Tesla replicates databases and
links their SaaS Applications to
Enterprise Data Warehouse
Stream & Replicate
• SFDC trickle Feeds Data to its
corporate Data Warehouse
ensures 24x7x365 availability
• Starbucks Streams Data from POS
Systems into DW and Data Lake
• E-Bay streams 100B transactions
per day into private cloud
Cleanse & Govern
• Allianz harvests from non-Oracle
tools into enterprise data catalog
for complete data lineage
• Cummins operates global
Governance council with data
quality and metadata tools
>15,000 Oracle Data Integration Customers Worldwide
Key Product Announcements
Data Integration & Governance
OOW 2017
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 21
NEW: Oracle GoldenGate 12.3 is Available
New GoldenGate Micro-Services Architecture
REST interfaces for Configuration, Administration and Monitoring with included HTML5 Web applications
Deploy at cloud scale with fully secure HTTPS interfaces and Secure Web Sockets for streaming data
Oracle Database Sharding Support
Embedded & fully automated active-active replication within and across Database Shards
Automatic Conflict Detection and Resolution (CDR)
Conflict detection and resolution built directly into Oracle Database Kernel
Heterogeneous Enhancements
Generic JDBC Support. MySQL DDL replication. Remote capture for z/OS & SQL Server
Cloud – Data Integration
Integrated UI for Design, Development, Management and Monitoring for GoldenGate Cloud Service
Parallel Apply
Highly scalable client side apply engine with gains of over 5x throughput on Oracle
Enhanced Big Data, Kafka and NoSQL Support
Performance improvements, Adapters for Oracle NoSQL, MongoDB. Support for Hive Metadata
Expanded Oracle Database 12.2 Support
Long identifiers, Expanded SCN, Local Undo for PDBs, Top-level VARRAYs, REFs.
Procedural replication of Advanced Queues, Virtual Private Database, Online Redefinition, ….
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 22Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 22
NEW: Oracle Data Integration Platform
Integrate Cloud and On Premise Data Lakes and Data Warehouses
…a Unified solution …that’s Easy to use …for Powerful data-driven solutions
Key Capabilities
1. Data High Availability
2. Data Migrations
3. Data Warehouse
Automation
4. Databus & Stream
Integration
5. Data Governance
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 23Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 23
Applications Infrastructure Analytics
Integration for… Integration for… Integration for…
Unified Technology Platform (PaaS)
Application
Integration
API
Management
Process
Integration
Stream
Processing
Data
Replication
Bulk Data
ETL & E-LT
Metadata
Management
Data
Quality
New: Unified Integration Capabilities
Converged Solution for All Integration Needs
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |10/9/2017 24Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |
Complete
Simplified
Open
DATA
GOVERNANCE
PROCESS
AUTOMATION
STREAM
ANALYTICS
API
MANAGEMENT
APPLICATION
INTEGRATION
DATA
QUALITY
BULK DATA
TRANSFORMATION
REAL TIME DATA
STREAMING AND DATA
REPLICATION
Oracle Cloud Platform for Integration
24
Intelligent Integration OOW2017 - Jeff Pollock

More Related Content

What's hot

Microservices Patterns with GoldenGate
Microservices Patterns with GoldenGateMicroservices Patterns with GoldenGate
Microservices Patterns with GoldenGate
Jeffrey T. Pollock
 
Expand a Data warehouse with Hadoop and Big Data
Expand a Data warehouse with Hadoop and Big DataExpand a Data warehouse with Hadoop and Big Data
Expand a Data warehouse with Hadoop and Big Data
jdijcks
 
Unlocking Big Data Silos in the Enterprise or the Cloud (Con7877)
Unlocking Big Data Silos in the Enterprise or the Cloud (Con7877)Unlocking Big Data Silos in the Enterprise or the Cloud (Con7877)
Unlocking Big Data Silos in the Enterprise or the Cloud (Con7877)
Jeffrey T. Pollock
 
Webinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafkaWebinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafka
Jeffrey T. Pollock
 
One Slide Overview: ORCL Big Data Integration and Governance
One Slide Overview: ORCL Big Data Integration and GovernanceOne Slide Overview: ORCL Big Data Integration and Governance
One Slide Overview: ORCL Big Data Integration and Governance
Jeffrey T. Pollock
 
Oracle PL/SQL 12c and 18c New Features + RADstack + Community Sites
Oracle PL/SQL 12c and 18c New Features + RADstack + Community SitesOracle PL/SQL 12c and 18c New Features + RADstack + Community Sites
Oracle PL/SQL 12c and 18c New Features + RADstack + Community Sites
Steven Feuerstein
 
Inside open metadata—the deep dive
Inside open metadata—the deep diveInside open metadata—the deep dive
Inside open metadata—the deep dive
DataWorks Summit
 
Time to Talk about Data Mesh
Time to Talk about Data MeshTime to Talk about Data Mesh
Time to Talk about Data Mesh
LibbySchulze
 
The Hidden Value of Hadoop Migration
The Hidden Value of Hadoop MigrationThe Hidden Value of Hadoop Migration
The Hidden Value of Hadoop Migration
Databricks
 
IBM THINK 2018 - IBM Cloud SQL Query Introduction
IBM THINK 2018 - IBM Cloud SQL Query IntroductionIBM THINK 2018 - IBM Cloud SQL Query Introduction
IBM THINK 2018 - IBM Cloud SQL Query Introduction
Torsten Steinbach
 
Presentacin webinar move_up_to_power8_with_scale_out_servers_final
Presentacin webinar move_up_to_power8_with_scale_out_servers_finalPresentacin webinar move_up_to_power8_with_scale_out_servers_final
Presentacin webinar move_up_to_power8_with_scale_out_servers_final
Diego Alberto Tamayo
 
IlOUG Tech Days 2016 - Unlock the Value in your Data Reservoir using Oracle B...
IlOUG Tech Days 2016 - Unlock the Value in your Data Reservoir using Oracle B...IlOUG Tech Days 2016 - Unlock the Value in your Data Reservoir using Oracle B...
IlOUG Tech Days 2016 - Unlock the Value in your Data Reservoir using Oracle B...
Mark Rittman
 
Swimming Across the Data Lake, Lessons learned and keys to success
Swimming Across the Data Lake, Lessons learned and keys to success Swimming Across the Data Lake, Lessons learned and keys to success
Swimming Across the Data Lake, Lessons learned and keys to success
DataWorks Summit/Hadoop Summit
 
Machine Learning for z/OS
Machine Learning for z/OSMachine Learning for z/OS
Machine Learning for z/OS
Cuneyt Goksu
 
Harnessing the Power of Apache Hadoop
Harnessing the Power of Apache Hadoop Harnessing the Power of Apache Hadoop
Harnessing the Power of Apache Hadoop
Cloudera, Inc.
 
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Databricks
 
Tame Big Data with Oracle Data Integration
Tame Big Data with Oracle Data IntegrationTame Big Data with Oracle Data Integration
Tame Big Data with Oracle Data Integration
Michael Rainey
 
Verizon Centralizes Data into a Data Lake in Real Time for Analytics
Verizon Centralizes Data into a Data Lake in Real Time for AnalyticsVerizon Centralizes Data into a Data Lake in Real Time for Analytics
Verizon Centralizes Data into a Data Lake in Real Time for Analytics
DataWorks Summit
 
Data Engineer, Patterns & Architecture The future: Deep-dive into Microservic...
Data Engineer, Patterns & Architecture The future: Deep-dive into Microservic...Data Engineer, Patterns & Architecture The future: Deep-dive into Microservic...
Data Engineer, Patterns & Architecture The future: Deep-dive into Microservic...
Igor De Souza
 
Priyank Patel, Teradata, Hadoop & SQL
Priyank Patel, Teradata, Hadoop & SQLPriyank Patel, Teradata, Hadoop & SQL
Priyank Patel, Teradata, Hadoop & SQL
The Hive
 

What's hot (20)

Microservices Patterns with GoldenGate
Microservices Patterns with GoldenGateMicroservices Patterns with GoldenGate
Microservices Patterns with GoldenGate
 
Expand a Data warehouse with Hadoop and Big Data
Expand a Data warehouse with Hadoop and Big DataExpand a Data warehouse with Hadoop and Big Data
Expand a Data warehouse with Hadoop and Big Data
 
Unlocking Big Data Silos in the Enterprise or the Cloud (Con7877)
Unlocking Big Data Silos in the Enterprise or the Cloud (Con7877)Unlocking Big Data Silos in the Enterprise or the Cloud (Con7877)
Unlocking Big Data Silos in the Enterprise or the Cloud (Con7877)
 
Webinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafkaWebinar future dataintegration-datamesh-and-goldengatekafka
Webinar future dataintegration-datamesh-and-goldengatekafka
 
One Slide Overview: ORCL Big Data Integration and Governance
One Slide Overview: ORCL Big Data Integration and GovernanceOne Slide Overview: ORCL Big Data Integration and Governance
One Slide Overview: ORCL Big Data Integration and Governance
 
Oracle PL/SQL 12c and 18c New Features + RADstack + Community Sites
Oracle PL/SQL 12c and 18c New Features + RADstack + Community SitesOracle PL/SQL 12c and 18c New Features + RADstack + Community Sites
Oracle PL/SQL 12c and 18c New Features + RADstack + Community Sites
 
Inside open metadata—the deep dive
Inside open metadata—the deep diveInside open metadata—the deep dive
Inside open metadata—the deep dive
 
Time to Talk about Data Mesh
Time to Talk about Data MeshTime to Talk about Data Mesh
Time to Talk about Data Mesh
 
The Hidden Value of Hadoop Migration
The Hidden Value of Hadoop MigrationThe Hidden Value of Hadoop Migration
The Hidden Value of Hadoop Migration
 
IBM THINK 2018 - IBM Cloud SQL Query Introduction
IBM THINK 2018 - IBM Cloud SQL Query IntroductionIBM THINK 2018 - IBM Cloud SQL Query Introduction
IBM THINK 2018 - IBM Cloud SQL Query Introduction
 
Presentacin webinar move_up_to_power8_with_scale_out_servers_final
Presentacin webinar move_up_to_power8_with_scale_out_servers_finalPresentacin webinar move_up_to_power8_with_scale_out_servers_final
Presentacin webinar move_up_to_power8_with_scale_out_servers_final
 
IlOUG Tech Days 2016 - Unlock the Value in your Data Reservoir using Oracle B...
IlOUG Tech Days 2016 - Unlock the Value in your Data Reservoir using Oracle B...IlOUG Tech Days 2016 - Unlock the Value in your Data Reservoir using Oracle B...
IlOUG Tech Days 2016 - Unlock the Value in your Data Reservoir using Oracle B...
 
Swimming Across the Data Lake, Lessons learned and keys to success
Swimming Across the Data Lake, Lessons learned and keys to success Swimming Across the Data Lake, Lessons learned and keys to success
Swimming Across the Data Lake, Lessons learned and keys to success
 
Machine Learning for z/OS
Machine Learning for z/OSMachine Learning for z/OS
Machine Learning for z/OS
 
Harnessing the Power of Apache Hadoop
Harnessing the Power of Apache Hadoop Harnessing the Power of Apache Hadoop
Harnessing the Power of Apache Hadoop
 
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
Data Mesh in Practice: How Europe’s Leading Online Platform for Fashion Goes ...
 
Tame Big Data with Oracle Data Integration
Tame Big Data with Oracle Data IntegrationTame Big Data with Oracle Data Integration
Tame Big Data with Oracle Data Integration
 
Verizon Centralizes Data into a Data Lake in Real Time for Analytics
Verizon Centralizes Data into a Data Lake in Real Time for AnalyticsVerizon Centralizes Data into a Data Lake in Real Time for Analytics
Verizon Centralizes Data into a Data Lake in Real Time for Analytics
 
Data Engineer, Patterns & Architecture The future: Deep-dive into Microservic...
Data Engineer, Patterns & Architecture The future: Deep-dive into Microservic...Data Engineer, Patterns & Architecture The future: Deep-dive into Microservic...
Data Engineer, Patterns & Architecture The future: Deep-dive into Microservic...
 
Priyank Patel, Teradata, Hadoop & SQL
Priyank Patel, Teradata, Hadoop & SQLPriyank Patel, Teradata, Hadoop & SQL
Priyank Patel, Teradata, Hadoop & SQL
 

Similar to Intelligent Integration OOW2017 - Jeff Pollock

Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
StampedeCon
 
Meetup Oracle Database MAD: 2.1 Data Management Trends: SQL, NoSQL y Big Data
Meetup Oracle Database MAD: 2.1 Data Management Trends: SQL, NoSQL y Big Data Meetup Oracle Database MAD: 2.1 Data Management Trends: SQL, NoSQL y Big Data
Meetup Oracle Database MAD: 2.1 Data Management Trends: SQL, NoSQL y Big Data
avanttic Consultoría Tecnológica
 
Meetup Oracle Database BCN: 2.1 Data Management Trends
Meetup Oracle Database BCN: 2.1 Data Management TrendsMeetup Oracle Database BCN: 2.1 Data Management Trends
Meetup Oracle Database BCN: 2.1 Data Management Trends
avanttic Consultoría Tecnológica
 
Bring your SAP and Enterprise Data to Hadoop, Apache Kafka and the Cloud
Bring your SAP and Enterprise Data to Hadoop, Apache Kafka and the CloudBring your SAP and Enterprise Data to Hadoop, Apache Kafka and the Cloud
Bring your SAP and Enterprise Data to Hadoop, Apache Kafka and the Cloud
DataWorks Summit/Hadoop Summit
 
Cardinality-HL-Overview
Cardinality-HL-OverviewCardinality-HL-Overview
Cardinality-HL-Overview
Harry Frost
 
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Impetus Technologies
 
Stream based Data Integration
Stream based Data IntegrationStream based Data Integration
Stream based Data Integration
Jeffrey T. Pollock
 
Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...
Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...
Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...
DataWorks Summit/Hadoop Summit
 
Bring Your SAP and Enterprise Data to Hadoop, Kafka, and the Cloud
Bring Your SAP and Enterprise Data to Hadoop, Kafka, and the CloudBring Your SAP and Enterprise Data to Hadoop, Kafka, and the Cloud
Bring Your SAP and Enterprise Data to Hadoop, Kafka, and the Cloud
DataWorks Summit
 
MDS ap_OEM Product Portfolio Intorduction to the DT & Analytics
MDS ap_OEM Product Portfolio Intorduction to the DT & AnalyticsMDS ap_OEM Product Portfolio Intorduction to the DT & Analytics
MDS ap_OEM Product Portfolio Intorduction to the DT & Analytics
MDS ap
 
Hortonworks Oracle Big Data Integration
Hortonworks Oracle Big Data Integration Hortonworks Oracle Big Data Integration
Hortonworks Oracle Big Data Integration
Hortonworks
 
Whither the Hadoop Developer Experience, June Hadoop Meetup, Nitin Motgi
Whither the Hadoop Developer Experience, June Hadoop Meetup, Nitin MotgiWhither the Hadoop Developer Experience, June Hadoop Meetup, Nitin Motgi
Whither the Hadoop Developer Experience, June Hadoop Meetup, Nitin Motgi
Felicia Haggarty
 
Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platform
Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platformPivotal deep dive_on_pivotal_hd_world_class_hdfs_platform
Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platform
EMC
 
Horses for Courses: Database Roundtable
Horses for Courses: Database RoundtableHorses for Courses: Database Roundtable
Horses for Courses: Database Roundtable
Eric Kavanagh
 
Teradata - Presentation at Hortonworks Booth - Strata 2014
Teradata - Presentation at Hortonworks Booth - Strata 2014Teradata - Presentation at Hortonworks Booth - Strata 2014
Teradata - Presentation at Hortonworks Booth - Strata 2014
Hortonworks
 
Trafodion overview
Trafodion overviewTrafodion overview
Trafodion overview
Rohit Jain
 
What's Planned for SAP HANA SPS10
What's Planned for SAP HANA SPS10What's Planned for SAP HANA SPS10
What's Planned for SAP HANA SPS10
SAP Technology
 
Insights into Real-world Data Management Challenges
Insights into Real-world Data Management ChallengesInsights into Real-world Data Management Challenges
Insights into Real-world Data Management Challenges
DataWorks Summit
 
Streaming Data and Stream Processing with Apache Kafka
Streaming Data and Stream Processing with Apache KafkaStreaming Data and Stream Processing with Apache Kafka
Streaming Data and Stream Processing with Apache Kafka
confluent
 
Insights into Real World Data Management Challenges
Insights into Real World Data Management ChallengesInsights into Real World Data Management Challenges
Insights into Real World Data Management Challenges
DataWorks Summit
 

Similar to Intelligent Integration OOW2017 - Jeff Pollock (20)

Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
Best Practices For Building and Operating A Managed Data Lake - StampedeCon 2016
 
Meetup Oracle Database MAD: 2.1 Data Management Trends: SQL, NoSQL y Big Data
Meetup Oracle Database MAD: 2.1 Data Management Trends: SQL, NoSQL y Big Data Meetup Oracle Database MAD: 2.1 Data Management Trends: SQL, NoSQL y Big Data
Meetup Oracle Database MAD: 2.1 Data Management Trends: SQL, NoSQL y Big Data
 
Meetup Oracle Database BCN: 2.1 Data Management Trends
Meetup Oracle Database BCN: 2.1 Data Management TrendsMeetup Oracle Database BCN: 2.1 Data Management Trends
Meetup Oracle Database BCN: 2.1 Data Management Trends
 
Bring your SAP and Enterprise Data to Hadoop, Apache Kafka and the Cloud
Bring your SAP and Enterprise Data to Hadoop, Apache Kafka and the CloudBring your SAP and Enterprise Data to Hadoop, Apache Kafka and the Cloud
Bring your SAP and Enterprise Data to Hadoop, Apache Kafka and the Cloud
 
Cardinality-HL-Overview
Cardinality-HL-OverviewCardinality-HL-Overview
Cardinality-HL-Overview
 
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
Apache Spark – The New Enterprise Backbone for ETL, Batch Processing and Real...
 
Stream based Data Integration
Stream based Data IntegrationStream based Data Integration
Stream based Data Integration
 
Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...
Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...
Modernizing Business Processes with Big Data: Real-World Use Cases for Produc...
 
Bring Your SAP and Enterprise Data to Hadoop, Kafka, and the Cloud
Bring Your SAP and Enterprise Data to Hadoop, Kafka, and the CloudBring Your SAP and Enterprise Data to Hadoop, Kafka, and the Cloud
Bring Your SAP and Enterprise Data to Hadoop, Kafka, and the Cloud
 
MDS ap_OEM Product Portfolio Intorduction to the DT & Analytics
MDS ap_OEM Product Portfolio Intorduction to the DT & AnalyticsMDS ap_OEM Product Portfolio Intorduction to the DT & Analytics
MDS ap_OEM Product Portfolio Intorduction to the DT & Analytics
 
Hortonworks Oracle Big Data Integration
Hortonworks Oracle Big Data Integration Hortonworks Oracle Big Data Integration
Hortonworks Oracle Big Data Integration
 
Whither the Hadoop Developer Experience, June Hadoop Meetup, Nitin Motgi
Whither the Hadoop Developer Experience, June Hadoop Meetup, Nitin MotgiWhither the Hadoop Developer Experience, June Hadoop Meetup, Nitin Motgi
Whither the Hadoop Developer Experience, June Hadoop Meetup, Nitin Motgi
 
Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platform
Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platformPivotal deep dive_on_pivotal_hd_world_class_hdfs_platform
Pivotal deep dive_on_pivotal_hd_world_class_hdfs_platform
 
Horses for Courses: Database Roundtable
Horses for Courses: Database RoundtableHorses for Courses: Database Roundtable
Horses for Courses: Database Roundtable
 
Teradata - Presentation at Hortonworks Booth - Strata 2014
Teradata - Presentation at Hortonworks Booth - Strata 2014Teradata - Presentation at Hortonworks Booth - Strata 2014
Teradata - Presentation at Hortonworks Booth - Strata 2014
 
Trafodion overview
Trafodion overviewTrafodion overview
Trafodion overview
 
What's Planned for SAP HANA SPS10
What's Planned for SAP HANA SPS10What's Planned for SAP HANA SPS10
What's Planned for SAP HANA SPS10
 
Insights into Real-world Data Management Challenges
Insights into Real-world Data Management ChallengesInsights into Real-world Data Management Challenges
Insights into Real-world Data Management Challenges
 
Streaming Data and Stream Processing with Apache Kafka
Streaming Data and Stream Processing with Apache KafkaStreaming Data and Stream Processing with Apache Kafka
Streaming Data and Stream Processing with Apache Kafka
 
Insights into Real World Data Management Challenges
Insights into Real World Data Management ChallengesInsights into Real World Data Management Challenges
Insights into Real World Data Management Challenges
 

More from Jeffrey T. Pollock

Data Mesh Part 4 Monolith to Mesh
Data Mesh Part 4 Monolith to MeshData Mesh Part 4 Monolith to Mesh
Data Mesh Part 4 Monolith to Mesh
Jeffrey T. Pollock
 
Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3
Jeffrey T. Pollock
 
Flash session -streaming--ses1243-lon
Flash session -streaming--ses1243-lonFlash session -streaming--ses1243-lon
Flash session -streaming--ses1243-lon
Jeffrey T. Pollock
 
Flash session -goldengate--lht1053-lon
Flash session -goldengate--lht1053-lonFlash session -goldengate--lht1053-lon
Flash session -goldengate--lht1053-lon
Jeffrey T. Pollock
 
Version Control Training - First Lego League
Version Control Training - First Lego LeagueVersion Control Training - First Lego League
Version Control Training - First Lego League
Jeffrey T. Pollock
 
GoldenGate and Stream Processing with Special Guest Rakuten
GoldenGate and Stream Processing with Special Guest RakutenGoldenGate and Stream Processing with Special Guest Rakuten
GoldenGate and Stream Processing with Special Guest Rakuten
Jeffrey T. Pollock
 
CDO - Chief Data Officer Momentum and Trends
CDO - Chief Data Officer Momentum and TrendsCDO - Chief Data Officer Momentum and Trends
CDO - Chief Data Officer Momentum and Trends
Jeffrey T. Pollock
 
Big Data at Oracle - Strata 2015 San Jose
Big Data at Oracle - Strata 2015 San JoseBig Data at Oracle - Strata 2015 San Jose
Big Data at Oracle - Strata 2015 San Jose
Jeffrey T. Pollock
 
Oracle Big Data Governance Webcast Charts
Oracle Big Data Governance Webcast ChartsOracle Big Data Governance Webcast Charts
Oracle Big Data Governance Webcast Charts
Jeffrey T. Pollock
 
Brief lessons from the greatest product managers
Brief lessons from the greatest product managersBrief lessons from the greatest product managers
Brief lessons from the greatest product managers
Jeffrey T. Pollock
 
Klarna Tech Talk - Mind the Data!
Klarna Tech Talk - Mind the Data!Klarna Tech Talk - Mind the Data!
Klarna Tech Talk - Mind the Data!
Jeffrey T. Pollock
 
Accelerate Return on Data
Accelerate Return on DataAccelerate Return on Data
Accelerate Return on Data
Jeffrey T. Pollock
 
Semantic Web For Dummies
Semantic Web For DummiesSemantic Web For Dummies
Semantic Web For Dummies
Jeffrey T. Pollock
 

More from Jeffrey T. Pollock (13)

Data Mesh Part 4 Monolith to Mesh
Data Mesh Part 4 Monolith to MeshData Mesh Part 4 Monolith to Mesh
Data Mesh Part 4 Monolith to Mesh
 
Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3
 
Flash session -streaming--ses1243-lon
Flash session -streaming--ses1243-lonFlash session -streaming--ses1243-lon
Flash session -streaming--ses1243-lon
 
Flash session -goldengate--lht1053-lon
Flash session -goldengate--lht1053-lonFlash session -goldengate--lht1053-lon
Flash session -goldengate--lht1053-lon
 
Version Control Training - First Lego League
Version Control Training - First Lego LeagueVersion Control Training - First Lego League
Version Control Training - First Lego League
 
GoldenGate and Stream Processing with Special Guest Rakuten
GoldenGate and Stream Processing with Special Guest RakutenGoldenGate and Stream Processing with Special Guest Rakuten
GoldenGate and Stream Processing with Special Guest Rakuten
 
CDO - Chief Data Officer Momentum and Trends
CDO - Chief Data Officer Momentum and TrendsCDO - Chief Data Officer Momentum and Trends
CDO - Chief Data Officer Momentum and Trends
 
Big Data at Oracle - Strata 2015 San Jose
Big Data at Oracle - Strata 2015 San JoseBig Data at Oracle - Strata 2015 San Jose
Big Data at Oracle - Strata 2015 San Jose
 
Oracle Big Data Governance Webcast Charts
Oracle Big Data Governance Webcast ChartsOracle Big Data Governance Webcast Charts
Oracle Big Data Governance Webcast Charts
 
Brief lessons from the greatest product managers
Brief lessons from the greatest product managersBrief lessons from the greatest product managers
Brief lessons from the greatest product managers
 
Klarna Tech Talk - Mind the Data!
Klarna Tech Talk - Mind the Data!Klarna Tech Talk - Mind the Data!
Klarna Tech Talk - Mind the Data!
 
Accelerate Return on Data
Accelerate Return on DataAccelerate Return on Data
Accelerate Return on Data
 
Semantic Web For Dummies
Semantic Web For DummiesSemantic Web For Dummies
Semantic Web For Dummies
 

Recently uploaded

一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理
一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理
一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理
74nqk8xf
 
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdfUdemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Fernanda Palhano
 
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Kiwi Creative
 
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
sameer shah
 
Everything you wanted to know about LIHTC
Everything you wanted to know about LIHTCEverything you wanted to know about LIHTC
Everything you wanted to know about LIHTC
Roger Valdez
 
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Aggregage
 
Influence of Marketing Strategy and Market Competition on Business Plan
Influence of Marketing Strategy and Market Competition on Business PlanInfluence of Marketing Strategy and Market Competition on Business Plan
Influence of Marketing Strategy and Market Competition on Business Plan
jerlynmaetalle
 
Analysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performanceAnalysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performance
roli9797
 
Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......
Sachin Paul
 
End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024
Lars Albertsson
 
University of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma TranscriptUniversity of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma Transcript
soxrziqu
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
manishkhaire30
 
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
v7oacc3l
 
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
Timothy Spann
 
A presentation that explain the Power BI Licensing
A presentation that explain the Power BI LicensingA presentation that explain the Power BI Licensing
A presentation that explain the Power BI Licensing
AlessioFois2
 
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
Social Samosa
 
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
Walaa Eldin Moustafa
 
Challenges of Nation Building-1.pptx with more important
Challenges of Nation Building-1.pptx with more importantChallenges of Nation Building-1.pptx with more important
Challenges of Nation Building-1.pptx with more important
Sm321
 
Intelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicineIntelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicine
AndrzejJarynowski
 
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
74nqk8xf
 

Recently uploaded (20)

一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理
一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理
一比一原版(Coventry毕业证书)考文垂大学毕业证如何办理
 
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdfUdemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
 
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
 
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
 
Everything you wanted to know about LIHTC
Everything you wanted to know about LIHTCEverything you wanted to know about LIHTC
Everything you wanted to know about LIHTC
 
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...
 
Influence of Marketing Strategy and Market Competition on Business Plan
Influence of Marketing Strategy and Market Competition on Business PlanInfluence of Marketing Strategy and Market Competition on Business Plan
Influence of Marketing Strategy and Market Competition on Business Plan
 
Analysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performanceAnalysis insight about a Flyball dog competition team's performance
Analysis insight about a Flyball dog competition team's performance
 
Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......
 
End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024
 
University of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma TranscriptUniversity of New South Wales degree offer diploma Transcript
University of New South Wales degree offer diploma Transcript
 
Learn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queriesLearn SQL from basic queries to Advance queries
Learn SQL from basic queries to Advance queries
 
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
在线办理(英国UCA毕业证书)创意艺术大学毕业证在读证明一模一样
 
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Dat...
 
A presentation that explain the Power BI Licensing
A presentation that explain the Power BI LicensingA presentation that explain the Power BI Licensing
A presentation that explain the Power BI Licensing
 
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
 
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
 
Challenges of Nation Building-1.pptx with more important
Challenges of Nation Building-1.pptx with more importantChallenges of Nation Building-1.pptx with more important
Challenges of Nation Building-1.pptx with more important
 
Intelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicineIntelligence supported media monitoring in veterinary medicine
Intelligence supported media monitoring in veterinary medicine
 
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
一比一原版(牛布毕业证书)牛津布鲁克斯大学毕业证如何办理
 

Intelligent Integration OOW2017 - Jeff Pollock

  • 1. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | OpenWorld 2017 Intelligent Integration and Governance Next-Gen Enterprise Data Management Greg Pavlik Senior Vice President & CTO of PaaS Jeff Pollock Vice President, Product Management Confidential – Oracle Internal/Restricted/Highly Restricted
  • 2. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Data Management is going through a major transformation…
  • 3. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential The future is already here; it’s just not very evenly distributed. - Gibson Enterprise data architecture is a hub-and-spoke Kimball- style solution Big data technologies are mostly for analytic use cases that demand the 3 V’s IT will provide the single source of truth of the data TRADITIONALAPPROACH EMERGINGSOLUTION Data Streams in Motion Big Data is for Analytic and Operational IT Use Cases for Any Type of Data Trusted Data means Raw Data
  • 4. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Discovery RESTful API for Producers and Subscribers (events are pushed) Raw Data Topics Schema Event Topics Data Pipeline (ETL) Prepared Data Topics Master Data Topics Data Pipeline (ETL) 1,000’s 100’s 10’s Oracle Open World 2015 4 App DB App DB App DB ERP Operational Data Store EDW Staging Prod ETL ETL ETL ETL ETL Mart Mart Mart ETL Enterprise BI Mart Mart Mart ETL Departmental BIDiscovery App DB App DB App DB ERP WebApps Mobile EDW NoSQL Hadoop / Spark Marts Marts Less Governed --------------------------------------------------------------- More Governed Enterprise BI Departmental BI Apps / Mobile Classical Data Management: Hub and Spoke • Invasive on Sources • High Latency / SLA • Mainly Relational Views • Heavy IT process overhead • Vendor-centric software Next-Gen: Streaming Databus/Kappa • Low impact on Sources • Low Latency (< 1 second) • Variety of Data Formats • More Agile DevOps processes • Open source centric software GoldenGate MDM Hub After 20yrs reign… hub-and-spoke is now a legacy • ODS & ETL Hubs • EDW/Mart Hubs • MDM/RDM Hubs • Static Data Lake Hubs • Pub/Sub for Staging • ETL in Pipelines • Analytics/CEP in Stream • Data is in Motion NoSQL / APIs LEGACY: NEXT-GEN: Less Governed ---------------------------------------------------- More Governed Physical Layer for ETL Pipelines = MPP Streaming (eg; Apache Spark Streaming) Physical Layer for Events = MPP Messaging (eg; Apache Kafka)
  • 5. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 5 In the beginning there was MapReduce …and it was slow • Original purpose for workloads on peta-scale search indices • Rise of Enterprise use cases, focus shifted to Analytic Data • 1st gen of big data in enterprise reused old approach (hub-and- spoke) using HDFS with MR2 • Operational Applications can use big data for low-latency use cases • Streaming for event correlation and IoT style integrations (real-time) • Analytics for traditional use cases • Cloud/Object Storage is “the lake” Next-Gen: Kappa is the Platform Multi- Structured Data Structured Data Applications Analytics Reporting Batch Layer Speed Layer Raw Data Layer Serving Layer
  • 6. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Open World 2015 6 Trusted data used to mean …single source of truth Extract Cleanse Stage Summarize Create Views Hello, is this the IT team? Yes, I need a single source of truth for my data Um, 2 years? • IT heavy waterfall delivery cycle creates drag on the business • Over-optimizing data distorts truth and diminishes fidelity of data • Data is like food, trust it more when it is minimally processed • Raw Data accessible to business • Subscriptions to data objects • API Access REST API vs. SQL • Agile DevOps process to deliver • Schema/less can handle all data Next-Gen: Sushi Principle of Data 1990’s
  • 7. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Data Staging or Archive Data Discovery ETL Offload Batch Layer Oracle Confidential 7 Business Data Analytics EDWs Data Streams Social and Logs Enterprise Data Highly Available Databases Databus (topic modeling) Stream Analytics ETL Data Pipelines Speed Layer Our Vision is to enable the modern ‘Kappa style' data architecture for Enterprise Strength solutions • Raw Data Layer common ingestion point for all enterprise data sources • Speed Layer data processing for streaming data and ETL data pipelines, in-memory • Batch Layer data processing for huge data volumes, that may span long time periods, using MPP • Serving Layer technologies for easy access to any data, at any latency Raw Data Layer Raw Events Changed Data Schema Events Core Design Pattern: Kappa-style Databus Pub / Sub REST APIs NoSQL Bulk Data Serving Layer Apps
  • 8. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 8 Business Data Serving Layer Apps Analytics EDWs Batch Layer Data Streams Social and Logs Enterprise Data Highly Available Databases Analytics Speed Layer Pub / Sub REST APIs NoSQL Bulk Data Raw Data Layer Oracle Approach: Blend of Commercial + Open Source Modern Architecture will be a ‘Hybrid Open-Source’ pattern: • Open Source at the core of speed and batch processing engines for general purpose data workloads • Enterprise Vendors for connecting to legacy systems, strong governance, and for highly optimized workloads • Cloud Platforms for Dev-Test (at least), rapid prototyping and eventually all production workloads • SaaS & Applications are key data “producers” and will remain largely proprietary and/or highly customized
  • 9. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential Proof this is a Pattern: Many Instantiations Kafka Storm | Spark | Apex | Flink MapReduce | Pig | Hive | Spark Cassandra | HBase Hive EventHubs Stream Analytics Data Lake Table Storage SQL Server Data Factory Kinesis Firehose EMR Dynamo Redshift DMS Pub/Sub Dataflow Dataproc BigTable BigQuery
  • 10. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 10 Business Data Serving Layer Apps Analytics EDWs Batch Layer Data Streams Social and Logs Enterprise Data Highly Available Databases Analytics Speed Layer Pub / Sub REST APIs NoSQL Bulk Data Raw Data Layer Best-of-Breed: Oracle Platform for Kappa-style Architecture Oracle Software can help customers Accelerate & Reduce Risk around adoption: • Ingest Data with lower latency, greater reliability and from any database using Oracle GoldenGate • ETP Pipelines for Data automate pipeline creation with zero-footprint using Oracle Data Integrator • Analyze Data In-Motion run temporal, spatial and predictive algorithms with Oracle Stream Analytics • Foundation Services for hosting Kafka (Event Hub) Spark/Hadoop (Big Data Cloud) or Relational (Database) • Govern the data flowing through Kappa architecture with Oracle Metadata Management GoldenGate Data Integrator Stream Analytics Event Hub Big Data Database Metadata Management (for Data Governance) Cassandra
  • 11. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential Oracle Big Data Platform Differentiators • Industrial strength data ingest • Streaming data pipelines • ETL pipelines • Analytic pipelines • World class infrastructure • Event hub cloud – Kafka • Big data cloud – Spark • Enterprise data governance • Compare to sqoop / ETL tools • Compare to old-style: • Batch ETL processing • Immature open-source • Compare to: • 1st Gen Cloud – AWS etc. • Proprietary services • Compare to Hadoop only Complete Simplified Open TECHNICALDIFFERENTIATION
  • 12. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Raw Data Layer Apps Layer Speed Layer Batch Layer Oracle Confidential 12 Industrial Strength Data Ingest: GoldenGate + Kappa Streaming Analytics Application Serving Layer REST Services Visualization Tools Reporting Tools Data Marts Capture Trail Route Deliver Pump GG GG User Updates DBMS Updates GoldenGate for Big Data Supported Platforms Kafka HDFS Fastest, most scalable and non-invasive way to ingest data into Apache. Benefits of low-impact on Sources, micro-second access to transactions and ability to replicate schema (DDL) events for downstream automation of change impact. GG used with 4 of top 5 largest Kafka clusters in the world… From user update to serving layer in <1 second & no impact on Source
  • 13. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 13 De-Coupling of the Database: Downstream Processing Mid-Tier HostPrimary Site Primary Secondary Log Mine GoldenGate Capture Trail Route Deliver Pump Business Apps Active DataGuard WAN REDO Transport Remote DR HostPrimary Site Primary Secondary Remote Standby GoldenGate Capture Trail Route Deliver Pump Business Apps AlwaysOn WAN AlwaysOn Primary Site DB2/z Business Apps LPAR / Linux Host SP GoldenGate Capture Trail Route Deliver Pump OLD Approach: “Just the Facts!” • On-host installation of Change Data Capture (CDC) tools • For typical DW use cases, only partial data sets were replicated • Focus was on summary data in the analytic model, not the detail data NEW Approach: “All the Data!” • Mid-tier installation of replication tools is necessary (non-invasive) • Focus is now on moving all the data into the data lake/data stream • Raw data means detail data • New use cases demand access to more expansive (raw) data sets Primary Site MySQL Business Apps Linux Host GoldenGate Capture Trail Route Deliver Pump
  • 14. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Raw Data Layer Speed Layer Batch Layer Oracle Confidential 14 ETL Pipelines with Data Integrator Streaming ETL Serving Layer REST Services Visualization Tools Reporting Tools Data Marts Oracle Data Integrator Capture Trail Route Deliver Pump GG SQOOP API/File SQOOP + Native Loaders Data Integrator for Big Data ✓ Batch data ingestion with Sqoop, native loaders & Oozie ✓ Generate data transformations in Hive, Pig, Spark & Spark Streaming ✓ Extract data into external DBs, Files or Cloud Compare to Informatica / Talend ✓ NoETL Engine native E-LT execution, 1000’s of references ✓ Zero Footprint does not require any Oracle install on cluster ✓ Loosely Coupled design time means you can reuse mapping logic in many big data languages ETL ETL
  • 15. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Raw Data Layer Speed Layer Batch Layer Streaming Analytics Serving Layer REST Services Visualization Tools Reporting Tools Data Marts Oracle Confidential 15 Analytic Pipelines with Stream Analytics Capture Trail Route Deliver Pump GoldenGate CQL Oracle Stream AnalyticsEvents & Cloud Apps Business Dashboard Native GoldenGate Stream Type Analytics EDWs AppsML Geo
  • 16. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential 16 World-class Infrastructure for Big Data • Elastic workload fabric to lower TCO and right-size cluster based on workload/SLA requirements • A seamless storage fabric across Cloud Storage, Block Storage, Local NVMe and Memory for your Big Data Clusters • A broad choice of Storage engines including relational databases (Oracle, MySQL), NoSQL Databases (Cassandra, HBase, Oracle NoSQL) • REST or Native APIs deliver an extremely low latency for real-time processing needs. • Start with a single partition and scale up to hundreds of them as your needs grow. • Complexity of managing Kafka is done by Oracle. • Node-Failure and Cluster-Failure Tolerance to ensures continuous availability • Big Data Cloud Machine for PaaS-like experience in customer data center • Same benefits and user experience as Oracle PaaS • Data stays onsite with customer • Big Data Appliance for customer- owned solutions. • Customer owns full solution • Very low startup costs • Highly optimized platform Oracle Event Hub Cloud Oracle Big Data Cloud Oracle Bare Metal Cloud
  • 17. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential Kappa Data Flow Pattern using Oracle Tech Stack GoldenGate Raw Data (LCR) Schema Events (DDL) Prepared Data Topics Master Data ETL ETL 1 Topic : 1 Table Data Consumers <subscribe> Applications Streaming Analytics ODS (Data Store) Big Data Lake Data Warehouses CQL & Spatial Analytic Data Oracle Event Hub DBMS Updates Data Producers Entire Enterprise Database Estate Stream Analytics Data Integrator Dev / Test Env. Oracle Big Data <generate> <generate> APIManagement
  • 18. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | All Enterprise Data Sources Oracle Confidential 18 Data Views from a Pipeline: Subscribe to Canonical Topics Poly- Structured Relational Data Science Data Analysts Business Analyst DBAs RAW DATA PREPARED DATA MASTER DATA SCHEMA EVENTS ETL ETL <subscribe> <subscribe> <subscribe> <subscribe> <produce> <produce> <produce>
  • 19. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Automate & Transform • Intuit Migrated 100’s of DBs to Cloud and Loads TB’s of data into Streaming data fabric (Kafka) • Tesla replicates databases and links their SaaS Applications to Enterprise Data Warehouse Stream & Replicate • SFDC trickle Feeds Data to its corporate Data Warehouse ensures 24x7x365 availability • Starbucks Streams Data from POS Systems into DW and Data Lake • E-Bay streams 100B transactions per day into private cloud Cleanse & Govern • Allianz harvests from non-Oracle tools into enterprise data catalog for complete data lineage • Cummins operates global Governance council with data quality and metadata tools >15,000 Oracle Data Integration Customers Worldwide
  • 20. Key Product Announcements Data Integration & Governance OOW 2017
  • 21. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 21 NEW: Oracle GoldenGate 12.3 is Available New GoldenGate Micro-Services Architecture REST interfaces for Configuration, Administration and Monitoring with included HTML5 Web applications Deploy at cloud scale with fully secure HTTPS interfaces and Secure Web Sockets for streaming data Oracle Database Sharding Support Embedded & fully automated active-active replication within and across Database Shards Automatic Conflict Detection and Resolution (CDR) Conflict detection and resolution built directly into Oracle Database Kernel Heterogeneous Enhancements Generic JDBC Support. MySQL DDL replication. Remote capture for z/OS & SQL Server Cloud – Data Integration Integrated UI for Design, Development, Management and Monitoring for GoldenGate Cloud Service Parallel Apply Highly scalable client side apply engine with gains of over 5x throughput on Oracle Enhanced Big Data, Kafka and NoSQL Support Performance improvements, Adapters for Oracle NoSQL, MongoDB. Support for Hive Metadata Expanded Oracle Database 12.2 Support Long identifiers, Expanded SCN, Local Undo for PDBs, Top-level VARRAYs, REFs. Procedural replication of Advanced Queues, Virtual Private Database, Online Redefinition, ….
  • 22. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 22Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 22 NEW: Oracle Data Integration Platform Integrate Cloud and On Premise Data Lakes and Data Warehouses …a Unified solution …that’s Easy to use …for Powerful data-driven solutions Key Capabilities 1. Data High Availability 2. Data Migrations 3. Data Warehouse Automation 4. Databus & Stream Integration 5. Data Governance
  • 23. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 23Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 23 Applications Infrastructure Analytics Integration for… Integration for… Integration for… Unified Technology Platform (PaaS) Application Integration API Management Process Integration Stream Processing Data Replication Bulk Data ETL & E-LT Metadata Management Data Quality New: Unified Integration Capabilities Converged Solution for All Integration Needs
  • 24. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |10/9/2017 24Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Complete Simplified Open DATA GOVERNANCE PROCESS AUTOMATION STREAM ANALYTICS API MANAGEMENT APPLICATION INTEGRATION DATA QUALITY BULK DATA TRANSFORMATION REAL TIME DATA STREAMING AND DATA REPLICATION Oracle Cloud Platform for Integration 24