In this presentation, you will see the new functionalities of the Denodo 6.0 detailing dynamic query optimization engine, managing enterprise deployments, and using information self-service for discovery and search.
This presentation is part of the Fast Data Strategy Conference, and you can watch the video here goo.gl/DzRtkg.
Denodo Platform 6.0 Enhances Big Data Performance and Self-Service Analytics
1. What’s New in Denodo
Platform 6.0
Dr. Alberto Pan, Chief Technical Officer
2. Agenda1.Main Drivers of Denodo 6.0
2.Performance in BigData Scenarios
3.Managing Enterprise Wide Deployments
4.Agile Data Development
5.Information Self-Service
3. Main Drivers for Denodo 6.0
Dynamic Query Optimizer for Big Data
Support for new analytical data sources
Support for new analytic functions
New Information Self-Service Tool
New ODBC Driver
Enhancements in Tableau integration
Workload Management: Denodo Resource
Manager
Monitoring and Diagnosing Tool
New VDP Admin Tool
GIT Support
Performance in BigData Scenarios
Information Self-service
Enterprise Wide Deployments
Agile Development
4. Main Drivers for Denodo 6.0
Dynamic Query Optimizer for Big Data
Support for new analytical data sources
Support for new analytic functions
New Information Self-Service Tool
New ODBC Driver
Enhancements in Tableau integration
Workload Management: Denodo Resource
Manager
Monitoring and Diagnosing Tool
New VDP Admin Tool
GIT Support
Performance in BigData Scenarios
Information Self-service
Enterprise Wide Deployments
Agile Development
5. 5
Dynamic Query Optimizer for BigData
Best Performance Even When Processing Billions of Rows
Move processing to the
data paradigm
Fully automated
optimization decisions
Considers characteristics of
BigData Sources
Find more details at: datavirtualization.blog
http://www.datavirtualizationblog.com/myths-in-data-
virtualization-performance/
6. 6
Query Optimizer for BigData: Example
Best Performance Even When Processing Billions of Rows
Performance Comparison of
Physical vs Logical
Scenario
Big Data volumes
TPC-DS benchmark
Find more details at: datavirtualization.blog
http://www.datavirtualizationblog.com/physical-logical-data-
warehouse-performance-numbers/
Sales
(Netezza)
Customers
(Oracle) Items
(SQLServer)
290M
2M 400K
7. 7
Query Optimizer for BigData: Example
Physical vs Logical DW Performance
Query Description Rows Returned
AVG Time Physical (all
data in Netezza)
AVG Time Logical
Optimization
Technique
(automatically
chosen by Denodo
6.0)
Total sales by customer 1,99 M 20975 ms 21457 ms
Full group by
pushdown
Total sales by customer and year
between 2000 and 2004 5,51 M 52313 ms 59060 ms
Full group by
pushdown
Total sales by item brand 31,35 K 4697 ms 5330 ms
Partial group by
pushdown
Total sales by item where sale
price less than current list price 17,05 K 3509 ms 5229 ms
On the fly data
movement
8. 8
New adapter for Apache
Spark
New adapter for Redshift
Enhanced Integration with
Vertica, Teradata and
Impala
Enhanced Connectivity for BigData Sources
Enhanced Adapters for the BigData Ecosystem
MarketingSales ExecutiveSupport
Database
Apps
Warehouse Cloud
Big Data
Documents AppsNo SQL
9. Main Drivers for Denodo 6.0
Dynamic Query Optimizer for Big Data
Support for new analytical data sources
Support for new analytic functions
New Information Self-Service Tool
New ODBC Driver
Enhancements in Tableau integration
Workload Management: Denodo Resource
Manager
Monitoring and Diagnosing Tool
New VDP Admin Tool
GIT Support
Performance in BigData Scenarios
Information Self-service
Enterprise Wide Deployments
Agile Development
10. Denodo Resource Manager (1)
■ Rules Classify Sessions into Groups (e.g. by user, application,…)
■ E.g. Sessions from application ‘single customer view’ are assigned to group
called ‘high priority_transactional
Allocate Resources according to Business Priorities
11. Denodo Resource Manager (2)
■ Apply Restrictions for Each Group.
Allocate Resources according to Business Priorities
12. Denodo Resource Manager (and 3)
■ Restrictions can apply to each individual query in a group or to the set of all
queries in the group
Allocate Resources according to Business Priorities
13. 13
Monitor current state of
servers and clusters
Inspect sessions, queries
(with real-time trace),
connections,...
Inspect data sources
activity, cache load
processes and content,...
Monitoring and Diagnosing Tool (1)
Graphical Monitoring and Diagnosing of Servers and Clusters
Go back in time to the
moment where a problem
happened
Diagnose root cause of the
problem
14. 14
Monitoring and Diagnosing Tool (2)
Graphical Monitoring and Diagnosing of Servers and Clusters
Start monitoring a server/environment by double clicking
in an item
Diagnostic information about an specific server can be
loaded by an option of the right context menu
Environments can be edited or synchronized
15. 15
Monitoring and Diagnosing Tool (and 3)
Graphical Monitoring and Diagnosing of Servers and Clusters
State: Summary of the state of the server/environment
Resources: physical resources (memory, cpu,…)
Requests: including real-time execution trace
Session: Currently opened sessions, including client application
Cache: cache load processes, cache contents,...
Datasources: pools state, active requests,...
Threads: priorities, CPU usage,...
Errors: Inspect logged errors and warnings
… and many others
Filter and sort information by any criteria
16. Main Drivers for Denodo 6.0
Dynamic Query Optimizer for Big Data
Support for new analytical data sources
Support for new analytic functions
New Information Self-Service Tool
New ODBC Driver
Enhancements in Tableau integration
Workload Management: Denodo Resource
Manager
Monitoring and Diagnosing Tool
New VDP Admin Tool
GIT Support
Performance in BigData Scenarios
Information Self-service
Enterprise Wide Deployments
Agile Development
20. New VDP Admin Tool (and 4)
20
Syntax
Highlighting
and Code
Completion
21. Main Drivers for Denodo 6.0
Dynamic Query Optimizer for Big Data
Support for new analytical data sources
Support for new analytic functions
New Information Self-Service Tool
New ODBC Driver
Enhancements in integration with BI tools
Workload Management: Denodo Resource
Manager
Monitoring and Diagnosing Tool
New VDP Admin Tool
GIT Support
Performance in BigData Scenarios
Information Self-service
Enterprise Wide Deployments
Agile Development
22. 22
Information Discovery and Self-Service (1)
Graphically Expose Data Views to Business Users
Search and Query Data and
Metadata
Browse data associations
Transform and combine views
Publish results to Denodo or
your favourite reporting tool
Find more details at: datavirtualization.blog
http://www.datavirtualizationblog.com/data-exploration-and-
self-service-bi-welcome-to-the-dataweb/