This document provides an overview, agenda, and demonstration of Denodo 7.0. It discusses Denodo's data virtualization architecture, how to set up Denodo including connectivity and modeling, how execution works by optimizing query plans, and how to access the Denodo data model through SQL, web services, and a data catalog. The demonstration shows how Denodo minimizes network traffic and processing loads through techniques like aggregation pushdown and leveraging massive parallel processing in external systems.

1. Technical
Demonstration
Denodo 7.0
Aly Wane Diene
Senior Solution Architect
May 2018

2. Agenda1. Overview
2. How to set up Denodo
• Connectivity
• Modeling
• Execution
3. Access and consumption

3. 3
DATA CONSUMERS
DISPARATE DATA SOURCES
SQL Queries
(JDBC, ODBC, ADO.NET)
Web Services
(SOAP, REST, OData)
Web-based catalog
& search
Secure delivery
(SSL/TLS)
DATA CONSUMERS
MPP Processing
Relational Cache
Corporate Security
Monitoring & Auditing
Metadata
Repository
Execution Engine
& Optimizer
A Modern Data Virtualization Architecture
DATA VIRTUALIZATION

4. 4
The Architecture
Sources
2. Data
Model
Combine,
Transform
&
Semantics
3. Publish
1. Source
Abstraction
Consuming Applications
4.Dev/Ops

5. How to set up Denodo
Connectivity and Modeling

6. 6
Example
What’s the impact of a new
marketing campaign for each
country?
 Historical sales data offloaded to
Hadoop cluster for cheaper storage
 Marketing campaigns managed in an
external cloud app
 Country is part of the customer
details table, stored in the DW
Sources
Combine,
Transform
&
Integrate
Consume
Base View
Source
Abstraction
join
group by state
join
Sales Campaign Customer

7. How does execution work

8. 8
What is the scenario?
The DV system only stores Metadata
Data is external
• Needs to travel through the Network
• To address: minimize network traffic
Data is distributed in multiple systems
• Needs to be integrated in the virtual layer
• Some sources have processing capabilities
• To address: maximize processing at sources to reduce load in virtualization layer

9. 9
What information do we have?
1. The incoming query (SQL)
2. Table metadata
 Source, PK, FK, indexes, “virtual” partitions, etc.
3. Data statistics
 Used by the Cost Based Optimizer to estimating data volumes
4. Source capabilities
 Can the source process data? (eg. RDBMS vs. CSV file)
 “Read-Only” vs. “Can create temp tables”
 In an MPP, size of the cluster

10. 10
What is the optimizer doing?
SELECT c.state, AVG(s.amount)
FROM customer c JOIN sales s
ON c.id = s.customer_id
GROUP BY c.state
Sales Customer
join
group by
Sales Customer
join
group by ID
Group by
state
Sales Customer
Create temp
table
join
group by
Option 1?
Option 2? Option 3?
Temp_Customer
Customer and Sales are in different sources.
What is the best execution plan?
Partial Aggregation PushdownNaïve Strategy Temporary Data Movement
300 M 2 M
2 M
2 M
2 M
50

11. 11
Why is this so important?
SELECT c.name, AVG(s.amount)
FROM customer c JOIN sales s
ON c.id = s.customer_id
GROUP BY c.state
How Denodo works compared with other federation engines
System Execution Time Data Transferred Optimization Technique
Denodo 9 sec. 4 M Aggregation push-down
Others 125 sec. 302 M None: full scan
300 M 2 M
Sales Customer
join
group by
2 M
2 M
Sales Customer
join
group by ID
Group by
state
To maximize push
down to the EDW
the aggregation is
split in 2 steps:
• 1st by customerID
• 2nd by state
This significantly
reduces network
Traffic and processing
In Denodo

12. 12
Massive Parallel Processing
Denodo provides native integration with MPP systems to provide two extra key
capabilities:
 Query Acceleration: move processing to the MPP during query execution
• Parallel processing of operations in Denodo
• Combines the advantages of advanced pushdown to minimize network traffic with massive
parallel processing
 MPP caching
 Fast access to recently used data
Both techniques can be combined to successfully approach very complex scenarios
with big data volumes in a very efficient way, leveraging inexpensive MPP solutions
(Spark, Presto, Impala, etc.)

13. Access & Consumption

14. 14
How to access the Denodo data model?
SQL Based access
 JDBC, ODBC and ADO.NET
• Integration with reporting tools: Tableau, MicroStrategy, PowerBI, BO,
Cognos, Looker, OBIEE, etc.
• Custom built applications
Web Services
 Multiple formats
• RESTful
• OData 4.0
• SOAP
 Compliance with modern standards: OAuth, JWT, SAML, OpenAPI
Denodo’s Data Catalog
 Web-based tool for exploration and discovery by business users

15. Q&A

16. Thanks!
www.denodo.com info@denodo.com
© Copyright Denodo Technologies. All rights reserved
Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm,
without prior the written authorization from Denodo Technologies.