Denodo
Data Virtualization
Take your Data Management Practice to
the Next Level with Denodo 7
Alberto Pan, CTO
May 2018
Agenda1. Denodo Vision and Architecture
2. Denodo in the Market
3. What’s New in Denodo 7 ?
• Performance
• Cloud Integration
• Data Discovery
4. Q & A
Denodo 7: Vision and Architecture
4
Unified Data Integration and Delivery Platform for the Business
4. Access with any tool / protocol –
Data service / API Layer
5. Centralized metadata, security &
governance
6. 90% time to market reductions and
cost savings
1. Single access point to data – Digital
Marketplace
2. Data in business friendly form –
Semantic Layer
3. Data adapted for the needs of each
LOB, type of user and application
5
Denodo Data Virtualization: Virtual Views
C_Code_Text C_OpenOppAmount Code_Cust_CHAR Total_Orders_Dollars
C_Code_Text C_OpenOppAmount Total_Orders_DollarsCustomer Code Open Opportunities
Amount ($)
Past Orders Amount ($)Customer Code Open Opportunities
Amount (€)
Past Orders
Amount(€)
Total Amount (€)
Total amount of open
opportunities and orders (in
euros)
Total amount of open
opportunities (in euros) as
documented in SaaS CRM
6
A Modern Data Virtualization Architecture
DATA CONSUMERS
DISPARATE DATA SOURCES
SQL Queries
(JDBC, ODBC, ADO.NET)
Web Services
(SOAP, REST, OData)
Web-based catalog
& search
Secure delivery
(SSL/TLS)
DATA CONSUMERS
MPP Processing
Relational Cache
Corporate Security
Monitoring & Auditing
Metadata
Repository
Execution Engine
& Optimizer
DATA VIRTUALIZATION
7
The Enterprise Data Layer for Big Data and Analytics
Reference Architecture
8
The Enterprise Data Layer for Operational and Transactional
Reference Architecture
Denodo: Market Positioning
10
Gartner Gives DV its Highest Maturity Rating
10
“Data
Virtualization
can be deployed
with low risk
and effort to
achieve
maximum
value.”
11
Source: “Predicts 2017: Data Distribution and Complexity Drive Information
Infrastructure Modernization” Gartner
By 2018, organizations with Data virtualization capabilities will
spend 40% less on building and managing (ETL & ESB) data
integration processes for connecting distributed data assets.”
12
Faster Time to Market – Biggest Value
Value Driver Metric Goal Actual
Time to Develop Time to develop web service in days 50% 90%
Time to Deploy Time to Deploy web service in days 50% 90%
TTM
Overall time it takes to make web service
available for use
60% 90%
Time to Engage
Time it takes for business to engage with
IT
75% 75%
Performance Performance of web services 50% 60%
Impact Analysis How fast can we perform impact analysis 50% 90%
Enterprise Architectural
Alignment
Ease at which data from disparate
sources can be integrated
Security, data
classification
High
13
Denodo High Customer Ratings
14
DV Fast Enterprise Deployments < 3 Months
Denodo 7: What’s New
16
New Release of Denodo Platform Redefines Data Management for Mission-critical Enterprise Use Cases.
Reimagine Data Management with Denodo Platform 7.0
Unparalleled Performance
Combined power of Massive Parallel
Processing & Dynamic Query Optimization
for immediate data delivery in big data,
cloud, and real-time scenarios.
Cloud Integration
Multi-location architecture for multi-
cloud, hybrid, and edge scenarios
without sacrificing performance and
governance
Integrated Data Discovery
The Only Data Virtualization offering
that tightly integrates Data Catalog and
Data Delivery for contextual information
delivery and collective intelligence.
17
Denodo Data Virtualization: Advanced Optimization
Minimizes data movement and maximizes
local processing at the origin systems
Leverages in-memory MPP Processing at
the DV layer
Multiple caching options: partial, full,
incremental, …
Optimize distributed queries across large repositories
The Denodo Platform ... incorporates dynamic query
optimization as a key value point…specifically for high data
volume and complexity;”
Gartnet Magic Quadrant for Data Integration, 2017
18
Query Optimization: Classical Federation
Naive Strategy (BI Tools, BDI Tools, Simple
federation engines):
join
union
group by
Customers (3M)
Sales previous years
(3B)Sales this year (290M)
290M rows
300M rows
(sales previous
year)
3M rows 593M rows through the
network
Obtain Total Sales By Customer Country in the Last Two Years
19
Query Optimization: Dynamic Optimizer
Denodo Strategy
join
union
group by
Customers (3M)
Sales previous years
(3B)Sales this year (290M)
3M rows (sales by
customer this year)
3M rows (sales by
customer previous
year)
3M rows 9 M rows through the
network
Obtain Total Sales By Customer Country in the Last Two Years
group by
customer
group by
customer
20
Dynamic Optimizer + MPP Processing (Denodo 7)
union
group by
3M rows
(sales by customer
this year)
3M rows
(sales by
customer
previous year)
3M rows
(customers)
Aggregation
pushdowngroup by
customer
group by
customer
join
Integrated
MPP processing
System Execution Time Optimization Technique
No Rewriting >10 min None
Denodo 6 51 sec Aggregation push-down
Denodo 7 13 sec
Aggregation push-down + MPP
integration
21
merge
Caching
Denodo can persist partial or total copies of datasets from
the data sources and combine them with real-time access:
• Denodo can use all major databases as caching systems,
including major parallel databases and BigData systems (New
in Denodo 7: Hive, Impala, Spark, Presto)
• Optimize performance, reuse complex transformations and
protect data sources from expensive queries
Full flexibility for data replication:
• Cache only some datasets or part of some datasets
• E.g. most recent queries, most frequent data
• Multiple automatic refresh options (e.g. incremental)
join
group by ID
Group by
state
SELECT c.state, AVG(s.amount)
FROM
customer c JOIN sales s
ON c.id = s.customer_id
GROUP BY c.state
Sales Customer
Customer changed /
added since 1:00AM
CACHE
Customers
updated at
1:00AM
22
New Release of Denodo Platform Redefines Data Management for Mission-critical Enterprise Use Cases.
Reimagine Data Management with Denodo Platform 7.0
Unparalleled Performance
Combined power of Massive Parallel
Processing & Dynamic Query Optimization
for immediate data delivery in big data,
cloud, and real-time scenarios.
Cloud Integration
Multi-location architecture for multi-
cloud, hybrid, and edge scenarios
without sacrificing performance and
governance
Integrated Data Discovery
The Only Data Virtualization offering
that tightly integrates Data Catalog and
Data Delivery for contextual information
delivery and collective intelligence.
23
Hybrid Information Architectures: The Integration Problem
Cloud Integration: The Problem
 Data sources at multiple locations: on-prem local, hierarchically distributed (e.g. Edge), in
one or several cloud providers,…
 Integration Problem is even more difficult:
 Performance
 Centralized Management of Security and Governance
 Impact of transition to the Cloud in existing applications
 …
 Data replication problems are aggravated in these scenarios
 Data Virtualization is specifically designed to provide location transparency, minimizing
data replication
23
24
Denodo 7: Multi-Location Architecture
 A Denodo deployment at each location (e.g.
on.prem, cloud providers).
 Every Denodo Server can use as “data
sources” the Denodo Servers at other
locations
 Denodo Servers are close to the data sources:
 Performance: minimize network trafficking,
maximize local processing
 Avoids opening firewall ports for data sources
 Deploy to any location with Denodo 7
Solution Manager
25
Denodo Solution Manager: Manage Multiple Locations
Administration and
Development Client
On-Prem 1
Load Balancer
Cloud Provider 1 Cloud Provider 2
Integrated
Version
Control
MS TFSSubversion git
Solution Manager
Centralized deployment from a web console to
multiple environments / locations:
 Definitions of data sources, views, exposed APIs,…
 Security Policies
 Workload Management
 Catalog, Data Lineage, Dependencies,…
 Version Control
 …
VCS for
Migrations
history
DB for Environment
Dependent properties
26
Cloud and On-Prem Integration
27
New Release of Denodo Platform Redefines Data Management for Mission-critical Enterprise Use Cases.
Reimagine Data Management with Denodo Platform 7.0
Unparalleled Performance
Combined power of Massive Parallel
Processing & Dynamic Query Optimization
for immediate data delivery in big data,
cloud, and real-time scenarios.
Cloud Integration
Multi-location architecture for multi-
cloud, hybrid, and edge scenarios
without sacrificing performance and
governance
Integrated Data Discovery
The Only Data Virtualization offering
that tightly integrates Data Catalog and
Data Delivery for contextual information
delivery and collective intelligence.
28
A “Data Marketplace” for the Business
Denodo provides a unified data integration and delivery
layer for the business
Multiple virtual datasets that can be useful for multiple
users and purposes
Data Catalog: help business users and developers find and
understand the datasets available in Denodo
Secure “Data Marketplace” of virtual datasets available to
solve a particular information need
Data Discovery Infrastructure on Top of the Denodo Layer
29
The Role of the Data Catalog
Catalog of views and web services
 See descriptions, relationships and data lineage
 Classify existing views and services using business categories, tags,..
 Browse and search capabilities using multiple criteria
 See information about the dataset usage
30
Denodo Data Catalog 7.0
31
Denodo Data Catalog: Tagging
32
Denodo Data Catalog: Data Lineage
33
The Role of the Data Catalog
Catalog of views and web services
 Classify existing views and services using business categories, tags,..
 Browse and search capabilities using multiple criteria
 See descriptions, relationships and data lineage
 See information about the dataset usage
Preview and find data
 Quick look at data
 Search based on content
34
Denodo Data Catalog: Content Search
35
The Role of the Data Catalog
Catalog of views and web services
 Classify existing views and services using business categories, tags,..
 Browse and search capabilities using multiple criteria
 See descriptions, relationships and data lineage
 See information about the dataset usage
Preview and find data
 Quick look at data
 Search based on content
Consume
 Obtain connection details to consume data with your favourite tool
 Customize existing views with visual transformation wizards
 “My queries” for personal use & share with other users
 Propose new standard business / canonical views
36
Denodo Data Catalog: Get Access
37
Key Takeaways
Denodo is the leading DV vendor, bridging the gap between the complex and changing
IT data infrastructure and Business Users and Applications
 Integrates and exposes data to the business much faster and at a fraction of the
cost of other approaches
Denodo 7 takes Denodo to new levels:
 Best performance by combining advanced techniques to minimize network traffic,
MPP processing and caching
 Enables seamless hybryd cloud-onprem integration, and facilitates cloud migrations
 Advanced data discovery and catalog functionalities
38
DOWNLOAD DENODO
EXPRESS
DENODO FOR AWS DENODO FOR AZURE
Download Denodo Express
Next Steps
Access Denodo Platform in the cloud!
30 day free trial available!
Q&A
Thanks!
www.denodo.com info@denodo.com
© Copyright Denodo Technologies. All rights reserved
Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm,
without prior the written authorization from Denodo Technologies.

Take your Data Management Practice to the Next Level with Denodo 7

  • 1.
    Denodo Data Virtualization Take yourData Management Practice to the Next Level with Denodo 7 Alberto Pan, CTO May 2018
  • 2.
    Agenda1. Denodo Visionand Architecture 2. Denodo in the Market 3. What’s New in Denodo 7 ? • Performance • Cloud Integration • Data Discovery 4. Q & A
  • 3.
    Denodo 7: Visionand Architecture
  • 4.
    4 Unified Data Integrationand Delivery Platform for the Business 4. Access with any tool / protocol – Data service / API Layer 5. Centralized metadata, security & governance 6. 90% time to market reductions and cost savings 1. Single access point to data – Digital Marketplace 2. Data in business friendly form – Semantic Layer 3. Data adapted for the needs of each LOB, type of user and application
  • 5.
    5 Denodo Data Virtualization:Virtual Views C_Code_Text C_OpenOppAmount Code_Cust_CHAR Total_Orders_Dollars C_Code_Text C_OpenOppAmount Total_Orders_DollarsCustomer Code Open Opportunities Amount ($) Past Orders Amount ($)Customer Code Open Opportunities Amount (€) Past Orders Amount(€) Total Amount (€) Total amount of open opportunities and orders (in euros) Total amount of open opportunities (in euros) as documented in SaaS CRM
  • 6.
    6 A Modern DataVirtualization Architecture DATA CONSUMERS DISPARATE DATA SOURCES SQL Queries (JDBC, ODBC, ADO.NET) Web Services (SOAP, REST, OData) Web-based catalog & search Secure delivery (SSL/TLS) DATA CONSUMERS MPP Processing Relational Cache Corporate Security Monitoring & Auditing Metadata Repository Execution Engine & Optimizer DATA VIRTUALIZATION
  • 7.
    7 The Enterprise DataLayer for Big Data and Analytics Reference Architecture
  • 8.
    8 The Enterprise DataLayer for Operational and Transactional Reference Architecture
  • 9.
  • 10.
    10 Gartner Gives DVits Highest Maturity Rating 10 “Data Virtualization can be deployed with low risk and effort to achieve maximum value.”
  • 11.
    11 Source: “Predicts 2017:Data Distribution and Complexity Drive Information Infrastructure Modernization” Gartner By 2018, organizations with Data virtualization capabilities will spend 40% less on building and managing (ETL & ESB) data integration processes for connecting distributed data assets.”
  • 12.
    12 Faster Time toMarket – Biggest Value Value Driver Metric Goal Actual Time to Develop Time to develop web service in days 50% 90% Time to Deploy Time to Deploy web service in days 50% 90% TTM Overall time it takes to make web service available for use 60% 90% Time to Engage Time it takes for business to engage with IT 75% 75% Performance Performance of web services 50% 60% Impact Analysis How fast can we perform impact analysis 50% 90% Enterprise Architectural Alignment Ease at which data from disparate sources can be integrated Security, data classification High
  • 13.
  • 14.
    14 DV Fast EnterpriseDeployments < 3 Months
  • 15.
  • 16.
    16 New Release ofDenodo Platform Redefines Data Management for Mission-critical Enterprise Use Cases. Reimagine Data Management with Denodo Platform 7.0 Unparalleled Performance Combined power of Massive Parallel Processing & Dynamic Query Optimization for immediate data delivery in big data, cloud, and real-time scenarios. Cloud Integration Multi-location architecture for multi- cloud, hybrid, and edge scenarios without sacrificing performance and governance Integrated Data Discovery The Only Data Virtualization offering that tightly integrates Data Catalog and Data Delivery for contextual information delivery and collective intelligence.
  • 17.
    17 Denodo Data Virtualization:Advanced Optimization Minimizes data movement and maximizes local processing at the origin systems Leverages in-memory MPP Processing at the DV layer Multiple caching options: partial, full, incremental, … Optimize distributed queries across large repositories The Denodo Platform ... incorporates dynamic query optimization as a key value point…specifically for high data volume and complexity;” Gartnet Magic Quadrant for Data Integration, 2017
  • 18.
    18 Query Optimization: ClassicalFederation Naive Strategy (BI Tools, BDI Tools, Simple federation engines): join union group by Customers (3M) Sales previous years (3B)Sales this year (290M) 290M rows 300M rows (sales previous year) 3M rows 593M rows through the network Obtain Total Sales By Customer Country in the Last Two Years
  • 19.
    19 Query Optimization: DynamicOptimizer Denodo Strategy join union group by Customers (3M) Sales previous years (3B)Sales this year (290M) 3M rows (sales by customer this year) 3M rows (sales by customer previous year) 3M rows 9 M rows through the network Obtain Total Sales By Customer Country in the Last Two Years group by customer group by customer
  • 20.
    20 Dynamic Optimizer +MPP Processing (Denodo 7) union group by 3M rows (sales by customer this year) 3M rows (sales by customer previous year) 3M rows (customers) Aggregation pushdowngroup by customer group by customer join Integrated MPP processing System Execution Time Optimization Technique No Rewriting >10 min None Denodo 6 51 sec Aggregation push-down Denodo 7 13 sec Aggregation push-down + MPP integration
  • 21.
    21 merge Caching Denodo can persistpartial or total copies of datasets from the data sources and combine them with real-time access: • Denodo can use all major databases as caching systems, including major parallel databases and BigData systems (New in Denodo 7: Hive, Impala, Spark, Presto) • Optimize performance, reuse complex transformations and protect data sources from expensive queries Full flexibility for data replication: • Cache only some datasets or part of some datasets • E.g. most recent queries, most frequent data • Multiple automatic refresh options (e.g. incremental) join group by ID Group by state SELECT c.state, AVG(s.amount) FROM customer c JOIN sales s ON c.id = s.customer_id GROUP BY c.state Sales Customer Customer changed / added since 1:00AM CACHE Customers updated at 1:00AM
  • 22.
    22 New Release ofDenodo Platform Redefines Data Management for Mission-critical Enterprise Use Cases. Reimagine Data Management with Denodo Platform 7.0 Unparalleled Performance Combined power of Massive Parallel Processing & Dynamic Query Optimization for immediate data delivery in big data, cloud, and real-time scenarios. Cloud Integration Multi-location architecture for multi- cloud, hybrid, and edge scenarios without sacrificing performance and governance Integrated Data Discovery The Only Data Virtualization offering that tightly integrates Data Catalog and Data Delivery for contextual information delivery and collective intelligence.
  • 23.
    23 Hybrid Information Architectures:The Integration Problem Cloud Integration: The Problem  Data sources at multiple locations: on-prem local, hierarchically distributed (e.g. Edge), in one or several cloud providers,…  Integration Problem is even more difficult:  Performance  Centralized Management of Security and Governance  Impact of transition to the Cloud in existing applications  …  Data replication problems are aggravated in these scenarios  Data Virtualization is specifically designed to provide location transparency, minimizing data replication 23
  • 24.
    24 Denodo 7: Multi-LocationArchitecture  A Denodo deployment at each location (e.g. on.prem, cloud providers).  Every Denodo Server can use as “data sources” the Denodo Servers at other locations  Denodo Servers are close to the data sources:  Performance: minimize network trafficking, maximize local processing  Avoids opening firewall ports for data sources  Deploy to any location with Denodo 7 Solution Manager
  • 25.
    25 Denodo Solution Manager:Manage Multiple Locations Administration and Development Client On-Prem 1 Load Balancer Cloud Provider 1 Cloud Provider 2 Integrated Version Control MS TFSSubversion git Solution Manager Centralized deployment from a web console to multiple environments / locations:  Definitions of data sources, views, exposed APIs,…  Security Policies  Workload Management  Catalog, Data Lineage, Dependencies,…  Version Control  … VCS for Migrations history DB for Environment Dependent properties
  • 26.
  • 27.
    27 New Release ofDenodo Platform Redefines Data Management for Mission-critical Enterprise Use Cases. Reimagine Data Management with Denodo Platform 7.0 Unparalleled Performance Combined power of Massive Parallel Processing & Dynamic Query Optimization for immediate data delivery in big data, cloud, and real-time scenarios. Cloud Integration Multi-location architecture for multi- cloud, hybrid, and edge scenarios without sacrificing performance and governance Integrated Data Discovery The Only Data Virtualization offering that tightly integrates Data Catalog and Data Delivery for contextual information delivery and collective intelligence.
  • 28.
    28 A “Data Marketplace”for the Business Denodo provides a unified data integration and delivery layer for the business Multiple virtual datasets that can be useful for multiple users and purposes Data Catalog: help business users and developers find and understand the datasets available in Denodo Secure “Data Marketplace” of virtual datasets available to solve a particular information need Data Discovery Infrastructure on Top of the Denodo Layer
  • 29.
    29 The Role ofthe Data Catalog Catalog of views and web services  See descriptions, relationships and data lineage  Classify existing views and services using business categories, tags,..  Browse and search capabilities using multiple criteria  See information about the dataset usage
  • 30.
  • 31.
  • 32.
  • 33.
    33 The Role ofthe Data Catalog Catalog of views and web services  Classify existing views and services using business categories, tags,..  Browse and search capabilities using multiple criteria  See descriptions, relationships and data lineage  See information about the dataset usage Preview and find data  Quick look at data  Search based on content
  • 34.
  • 35.
    35 The Role ofthe Data Catalog Catalog of views and web services  Classify existing views and services using business categories, tags,..  Browse and search capabilities using multiple criteria  See descriptions, relationships and data lineage  See information about the dataset usage Preview and find data  Quick look at data  Search based on content Consume  Obtain connection details to consume data with your favourite tool  Customize existing views with visual transformation wizards  “My queries” for personal use & share with other users  Propose new standard business / canonical views
  • 36.
  • 37.
    37 Key Takeaways Denodo isthe leading DV vendor, bridging the gap between the complex and changing IT data infrastructure and Business Users and Applications  Integrates and exposes data to the business much faster and at a fraction of the cost of other approaches Denodo 7 takes Denodo to new levels:  Best performance by combining advanced techniques to minimize network traffic, MPP processing and caching  Enables seamless hybryd cloud-onprem integration, and facilitates cloud migrations  Advanced data discovery and catalog functionalities
  • 38.
    38 DOWNLOAD DENODO EXPRESS DENODO FORAWS DENODO FOR AZURE Download Denodo Express Next Steps Access Denodo Platform in the cloud! 30 day free trial available!
  • 39.
  • 40.
    Thanks! www.denodo.com info@denodo.com © CopyrightDenodo Technologies. All rights reserved Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies.