DATA VIRTUALIZATION PACKED LUNCH
WEBINAR SERIES
Sessions Covering Key Data Integration Challenges
Solved with Data Virtualization
Data Virtualization: An Introduction
Paul Shimabukuro
Sales Engineer, Denodo
Paul Moxon
SVP Data Architectures & Chief Evangelist, Denodo
Agenda
1. Data Virtualization: An Introduction
2. Data Virtualization – How it works
3. Product Demo
4. Q&A
5. Next Steps
Data Virtualization: An Introduction
4
Data Integration – A Classic Approach
5
Operational
Data Stores
Staging Area Data Warehouse Data Marts Analytics and
Reporting
ETLETLETL
Data Integration – A Modern Data Ecosystem
6
The Data Integration Challenge
7
Manually access different
systems
IT responds with point-to-
point data integration
Takes too long to get
answers to business users
MarketingSales ExecutiveSupport
Database
Apps
Warehouse Cloud
Big Data
Documents AppsNo SQL
Businesses are reporting that integrating data from
silos to support real-time insights has become a
nightmare, especially when supporting large and
complex data sets
Big Data Fabric 2.0 Drives Data Democratization, May 9, 2019
8
The Data Integration Challenge
It is difficult to integrate numerous
on-premises and cloud data sources.
Traditional tools cannot integrate streaming
data and data-at-rest in real time.
It is difficult to maintain consistent data access
and governance policies across data siloes.
Traditional data integration is extremely
resource intensive.
The Solution – A Data Abstraction Layer
9
Abstracts access to
disparate data sources
Acts as a single repository
(virtual)
Makes data available in
real-time to consumers
DATA ABSTRACTION LAYER
“Enterprise architects are finding that traditional
data architectures are failing to meet new business
requirements, especially around data integration for
streaming analytics and real-time analytics.”
The Forrester Wave: Enterprise Data Virtualization, Jan 12, 2018
DATA VIRTUALIZATION PLATFORM
Source: “Gartner Market Guide for Data Virtualization, November 16, 2018”
Data virtualization can be used to create virtualized and
integrated views of data in-memory rather than executing
data movement and physically storing integrated views in a
target data structure. It provides a layer of abstraction
above the physical implementation of data, to simplify
query logic.
Data Virtualization
11
Consume
in business applications
Combine
related data into views
Connect
to disparate data sources
2
3
1
DATA CONSUMERS
DISPARATE DATA SOURCES
Enterprise Applications, Reporting, BI, Portals, ESB, Mobile, Web, Users
Databases & Warehouses, Cloud/Saas Applications, Big Data, NoSQL, Web, XML, Excel, PDF, Word...
Analytical Operational
Less StructuredMore Structured
CONNECT COMBINE PUBLISH
Multiple Protocols,
Formats
Query, Search,
Browse
Request/Reply,
Event Driven
Secure
Delivery
SQL,
MDX
Web
Services
Big Data
APIs
Web Automation
and Indexing
CONNECT COMBINE CONSUME
Share, Deliver,
Publish, Govern,
Collaborate
Discover, Transform,
Prepare, Improve
Quality, Integrate
Normalized views of
disparate data
“Data virtualization
integrates disparate
data sources in real
time or near-real
time to meet
demands for
analytics and
transactional data.”
– Create a Road Map For A
Real-time, Agile, Self-
Service Data Platform,
Forrester Research, Dec 16,
2015
12
How Does It Work?
Development
Lifecycle Mgmt
Monitoring &
Audit
Governance
Security
Development
Tools and SDK
Scheduled Tasks
Data Caching
Query Optimizer
JDBC/ODBC/ADO.Net SOAP / REST WS
U
Business
View
Data Mart
View
J
Application
Layer
Business
Layer
Unified
View
Unified
View
Unified
View
Unified
View
A
J
J
Derived
View
Derived
View
J
JS
Transformation
& Cleansing
Data
Source
Layer
Base
View
Base
View
Base
View
Base
View
Base
View
Base
View
Base
View
Abstraction
Data Virtualization Connects the Users to the Data That They Need
1. Data Virtualization allows you to connect to (almost) any data source
2. You can combine and transform that data into the format needed by the
consumer
3. The data can be exposed to the consumers in a format and interface
that is usable by them
• Typically consumers use the tools that they already use – they don’t have to learn new
tools and skills to access the data
4. All of this can be done without copying or moving the data
• The data stays in the original sources (databases, applications, files, etc.) and is
retrieved, in real-time, on demand
13
Cliffs Notes version (TL;DR)
Gartner – The Evolution of Analytical Environments
14
Operational Application
Operational Application
Operational Application
IoT Data
Other NewData
Operational
Application
Operational
Application
Cube
Operational
Application
Cube
? Operational Application
Operational Application
Operational Application
IoT Data
Other NewData
1980s
Pre EDW
1990s
EDW
2010s2000s
Post EDW
Time
LDW
Operational
Application
Operational
Application
Operational
Application
Data
Warehouse
Data
Warehouse
Data
Lake
?
LDW
Data Warehouse
Data Lake
Marts
ODS
Staging/Ingest
Unified analysis
› Consolidated data
› "Collect the data"
› Single server, multiple nodes
› More analysis than any
one server can provide
©2018 Gartner, Inc.
Unified analysis
› Logically consolidated view of all data
› "Connect and collect"
› Multiple servers, of multiple nodes
› More analysis than any one system can provide
ID: 342254
Fragmented/
nonexistent analysis
› Multiple sources
› Multiple structured sources
Fragmented analysis
› "Collect the data" (Into
› different repositories)
› New data types,
› processing, requirements
› Uncoordinated views
15
Gartner – Logical Data Warehouse
“Adopt the Logical Data Warehouse Architecture to Meet Your Modern Analytical Needs”. Henry
Cook, Gartner April 2018
DATA VIRTUALIZATION
− Market Guide for Data Virtualization, Gartner, November, 16, 2018
“Through 2022, 60% of all organizations will
implement data virtualization as one key delivery
style in their data integration architecture.”
16
17
Denodo ‘Use Case’ Categories
Customer Centricity
APIs/Services
✓ Data as a Service
✓ Microservices/containers
✓ API Data Services
✓ Application Migration
Machine Learning
✓ Data Catalog
✓ Metadata management
✓ Universal data access
✓ Governance, Risk, Compliance (GRC)
✓ Data Masking/Data Privacy
✓ Auditing/data lineage
✓ Hybrid/Multicloud Integration
✓ Cloud Data Analytics
✓ Cloud IT Modernization
Product Demonstration
Data Virtualization – An Introduction
18
Sales Engineer, Denodo
Paul Shimabukuro
19
Demo Architecture
What’s the impact of a new
marketing campaign for each
country?
▪ Historical sales data offloaded
to Hadoop cluster for cheaper
storage
▪ Marketing campaigns managed
in an external cloud app
▪ Country is part of the customer
details table, stored in the DW
Sources
Combine,
Transform
&
Integrate
Consume
Base View
Source
Abstraction
join
group by country
join
Sales Campaign Customer
Demo
20
Key Takeaways
21
Key Takeaways
22
FIRST
Takeaway
Data Virtualization is a key technology when building a
modern data architecture
SECOND
Takeaway
It provides flexibility and agility and reduces the time to
deliver data to the business by up to 10X
THIRD
Takeaway
Data Virtualization hides the complexity of a constantly
changing data infrastructure from the users
FOURTH
Takeaway
In doing so, it allows you to introduce new technologies,
formats, protocols, etc. without causing user disruption
Q&A
24
Next Steps
Access Denodo Platform in the Cloud!
Take a Test Drive today!
www.denodo.com/TestDrive
GET STARTED TODAY
Thank you!
© Copyright Denodo Technologies. All rights reserved
Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and
microfilm, without prior the written authorization from Denodo Technologies.

Data Virtualization: An Introduction

  • 1.
    DATA VIRTUALIZATION PACKEDLUNCH WEBINAR SERIES Sessions Covering Key Data Integration Challenges Solved with Data Virtualization
  • 2.
    Data Virtualization: AnIntroduction Paul Shimabukuro Sales Engineer, Denodo Paul Moxon SVP Data Architectures & Chief Evangelist, Denodo
  • 3.
    Agenda 1. Data Virtualization:An Introduction 2. Data Virtualization – How it works 3. Product Demo 4. Q&A 5. Next Steps
  • 4.
  • 5.
    Data Integration –A Classic Approach 5 Operational Data Stores Staging Area Data Warehouse Data Marts Analytics and Reporting ETLETLETL
  • 6.
    Data Integration –A Modern Data Ecosystem 6
  • 7.
    The Data IntegrationChallenge 7 Manually access different systems IT responds with point-to- point data integration Takes too long to get answers to business users MarketingSales ExecutiveSupport Database Apps Warehouse Cloud Big Data Documents AppsNo SQL Businesses are reporting that integrating data from silos to support real-time insights has become a nightmare, especially when supporting large and complex data sets Big Data Fabric 2.0 Drives Data Democratization, May 9, 2019
  • 8.
    8 The Data IntegrationChallenge It is difficult to integrate numerous on-premises and cloud data sources. Traditional tools cannot integrate streaming data and data-at-rest in real time. It is difficult to maintain consistent data access and governance policies across data siloes. Traditional data integration is extremely resource intensive.
  • 9.
    The Solution –A Data Abstraction Layer 9 Abstracts access to disparate data sources Acts as a single repository (virtual) Makes data available in real-time to consumers DATA ABSTRACTION LAYER “Enterprise architects are finding that traditional data architectures are failing to meet new business requirements, especially around data integration for streaming analytics and real-time analytics.” The Forrester Wave: Enterprise Data Virtualization, Jan 12, 2018 DATA VIRTUALIZATION PLATFORM
  • 10.
    Source: “Gartner MarketGuide for Data Virtualization, November 16, 2018” Data virtualization can be used to create virtualized and integrated views of data in-memory rather than executing data movement and physically storing integrated views in a target data structure. It provides a layer of abstraction above the physical implementation of data, to simplify query logic.
  • 11.
    Data Virtualization 11 Consume in businessapplications Combine related data into views Connect to disparate data sources 2 3 1 DATA CONSUMERS DISPARATE DATA SOURCES Enterprise Applications, Reporting, BI, Portals, ESB, Mobile, Web, Users Databases & Warehouses, Cloud/Saas Applications, Big Data, NoSQL, Web, XML, Excel, PDF, Word... Analytical Operational Less StructuredMore Structured CONNECT COMBINE PUBLISH Multiple Protocols, Formats Query, Search, Browse Request/Reply, Event Driven Secure Delivery SQL, MDX Web Services Big Data APIs Web Automation and Indexing CONNECT COMBINE CONSUME Share, Deliver, Publish, Govern, Collaborate Discover, Transform, Prepare, Improve Quality, Integrate Normalized views of disparate data “Data virtualization integrates disparate data sources in real time or near-real time to meet demands for analytics and transactional data.” – Create a Road Map For A Real-time, Agile, Self- Service Data Platform, Forrester Research, Dec 16, 2015
  • 12.
    12 How Does ItWork? Development Lifecycle Mgmt Monitoring & Audit Governance Security Development Tools and SDK Scheduled Tasks Data Caching Query Optimizer JDBC/ODBC/ADO.Net SOAP / REST WS U Business View Data Mart View J Application Layer Business Layer Unified View Unified View Unified View Unified View A J J Derived View Derived View J JS Transformation & Cleansing Data Source Layer Base View Base View Base View Base View Base View Base View Base View Abstraction
  • 13.
    Data Virtualization Connectsthe Users to the Data That They Need 1. Data Virtualization allows you to connect to (almost) any data source 2. You can combine and transform that data into the format needed by the consumer 3. The data can be exposed to the consumers in a format and interface that is usable by them • Typically consumers use the tools that they already use – they don’t have to learn new tools and skills to access the data 4. All of this can be done without copying or moving the data • The data stays in the original sources (databases, applications, files, etc.) and is retrieved, in real-time, on demand 13 Cliffs Notes version (TL;DR)
  • 14.
    Gartner – TheEvolution of Analytical Environments 14 Operational Application Operational Application Operational Application IoT Data Other NewData Operational Application Operational Application Cube Operational Application Cube ? Operational Application Operational Application Operational Application IoT Data Other NewData 1980s Pre EDW 1990s EDW 2010s2000s Post EDW Time LDW Operational Application Operational Application Operational Application Data Warehouse Data Warehouse Data Lake ? LDW Data Warehouse Data Lake Marts ODS Staging/Ingest Unified analysis › Consolidated data › "Collect the data" › Single server, multiple nodes › More analysis than any one server can provide ©2018 Gartner, Inc. Unified analysis › Logically consolidated view of all data › "Connect and collect" › Multiple servers, of multiple nodes › More analysis than any one system can provide ID: 342254 Fragmented/ nonexistent analysis › Multiple sources › Multiple structured sources Fragmented analysis › "Collect the data" (Into › different repositories) › New data types, › processing, requirements › Uncoordinated views
  • 15.
    15 Gartner – LogicalData Warehouse “Adopt the Logical Data Warehouse Architecture to Meet Your Modern Analytical Needs”. Henry Cook, Gartner April 2018 DATA VIRTUALIZATION
  • 16.
    − Market Guidefor Data Virtualization, Gartner, November, 16, 2018 “Through 2022, 60% of all organizations will implement data virtualization as one key delivery style in their data integration architecture.” 16
  • 17.
    17 Denodo ‘Use Case’Categories Customer Centricity APIs/Services ✓ Data as a Service ✓ Microservices/containers ✓ API Data Services ✓ Application Migration Machine Learning ✓ Data Catalog ✓ Metadata management ✓ Universal data access ✓ Governance, Risk, Compliance (GRC) ✓ Data Masking/Data Privacy ✓ Auditing/data lineage ✓ Hybrid/Multicloud Integration ✓ Cloud Data Analytics ✓ Cloud IT Modernization
  • 18.
    Product Demonstration Data Virtualization– An Introduction 18 Sales Engineer, Denodo Paul Shimabukuro
  • 19.
    19 Demo Architecture What’s theimpact of a new marketing campaign for each country? ▪ Historical sales data offloaded to Hadoop cluster for cheaper storage ▪ Marketing campaigns managed in an external cloud app ▪ Country is part of the customer details table, stored in the DW Sources Combine, Transform & Integrate Consume Base View Source Abstraction join group by country join Sales Campaign Customer
  • 20.
  • 21.
  • 22.
    Key Takeaways 22 FIRST Takeaway Data Virtualizationis a key technology when building a modern data architecture SECOND Takeaway It provides flexibility and agility and reduces the time to deliver data to the business by up to 10X THIRD Takeaway Data Virtualization hides the complexity of a constantly changing data infrastructure from the users FOURTH Takeaway In doing so, it allows you to introduce new technologies, formats, protocols, etc. without causing user disruption
  • 23.
  • 24.
    24 Next Steps Access DenodoPlatform in the Cloud! Take a Test Drive today! www.denodo.com/TestDrive GET STARTED TODAY
  • 25.
    Thank you! © CopyrightDenodo Technologies. All rights reserved Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies.