Accelerating Analytics Architectures with Parallel In-
memory Processing and Data Virtualization
Alberto Pan
CTO, Denodo
Agenda1. Logical Architectures for Analytics
2. Performance: Accelerating Data Delivery
3. Demo
3
Logical Architectures for Analytics
4
Logical Architectures: Data Virtualization
5
The Role of Denodo
Combine and publish data from any type of data source with a few clicks
Abstracts application from changes in IT infrastructure
Exposes different logical views adapted to each type of user / LOB
Single entry point to apply security and governance policies
Advanced distributed query optimization
…zero-code development of complex data combinations and transformations
The Denodo Platform enables us to build and deliver data
services, to our internal and external consumers, within a
day instead of the 1 – 2 weeks it would take with ETL.”
– Manager, DrillingInfo
6
Query Optimization: Example (1)
Naive Strategy (BI Tools, BDI Tools, Simple
federation engines):
join
union
group by
Customers (3M)
Sales previous years (3B)
Sales this year (290M)
290M rows
300M rows (sales
previous year)
3M rows
593M rows through the
network
Obtain Total Sales By Customer Country in the Last Two Years
7
Query Optimization: Example (2)
Denodo Strategy
join
union
group by
Customers (3M)
Sales previous years (3B)
Sales this year (290M)
3M rows (sales by
customer this year)
3M rows (sales by
customer previous
year)
3M rows
9 M rows through the
network
Obtain Total Sales By Customer Country in the Last Two Years
group by
customer
group by
customer
8
Query Optimization: Example (and 3)
union
group by
3M rows
(sales by customer this
year)
3M rows
(sales by customer
previous year)
3M rows
(customers)
Aggregation
pushdowngroup by
customer
group by
customer
join
Integrated
MPP processing
System Execution Time Optimization Technique
No Rewriting 20 min None
Denodo 6 51 sec Aggregation push-down
Denodo 7 13 sec
Aggregation push-down + MPP
integration
Demo
Q&A
Thank you!
© Copyright Denodo Technologies. All rights reserved
Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written
authorization from Denodo Technologies.
#DenodoDataFest

Denodo DataFest 2017: Accelerating Analytics Architectures with Parallel In-Memory Processing and Data Virtualization

  • 1.
    Accelerating Analytics Architectureswith Parallel In- memory Processing and Data Virtualization Alberto Pan CTO, Denodo
  • 2.
    Agenda1. Logical Architecturesfor Analytics 2. Performance: Accelerating Data Delivery 3. Demo
  • 3.
  • 4.
  • 5.
    5 The Role ofDenodo Combine and publish data from any type of data source with a few clicks Abstracts application from changes in IT infrastructure Exposes different logical views adapted to each type of user / LOB Single entry point to apply security and governance policies Advanced distributed query optimization …zero-code development of complex data combinations and transformations The Denodo Platform enables us to build and deliver data services, to our internal and external consumers, within a day instead of the 1 – 2 weeks it would take with ETL.” – Manager, DrillingInfo
  • 6.
    6 Query Optimization: Example(1) Naive Strategy (BI Tools, BDI Tools, Simple federation engines): join union group by Customers (3M) Sales previous years (3B) Sales this year (290M) 290M rows 300M rows (sales previous year) 3M rows 593M rows through the network Obtain Total Sales By Customer Country in the Last Two Years
  • 7.
    7 Query Optimization: Example(2) Denodo Strategy join union group by Customers (3M) Sales previous years (3B) Sales this year (290M) 3M rows (sales by customer this year) 3M rows (sales by customer previous year) 3M rows 9 M rows through the network Obtain Total Sales By Customer Country in the Last Two Years group by customer group by customer
  • 8.
    8 Query Optimization: Example(and 3) union group by 3M rows (sales by customer this year) 3M rows (sales by customer previous year) 3M rows (customers) Aggregation pushdowngroup by customer group by customer join Integrated MPP processing System Execution Time Optimization Technique No Rewriting 20 min None Denodo 6 51 sec Aggregation push-down Denodo 7 13 sec Aggregation push-down + MPP integration
  • 9.
  • 10.
  • 11.
    Thank you! © CopyrightDenodo Technologies. All rights reserved Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies. #DenodoDataFest