Building a Logical Data Fabric using Data Virtualization (ASEAN)

DENODO LUNCH & LEARN
23 NOVEMBER
BUILDING A LOGICAL DATA FABRIC
USING DATA VIRTUALIZATION

Presenters for this Session
Chris Day
Director, APAC Sales Engineering, Denodo
Regional Vice President, Sales, ASEAN & Korea, Denodo
Elaine Chan

Agenda
1. Data Lakes and Data Warehouses
2. Observations From the Recent TDWI Report
3. What is a Logical Data Fabric ? (Deep Dive)
4. How Does This Apply to a Data Warehouse/Data Lake?
5. Customer Case Study - Autodesk
6. Conclusions
7. Product Demo
8. Q & A and Next Steps

Building A Logical Data Fabric Using
Data Virtualization
Regional Vice President, Sales, ASEAN & Korea, Denodo
Elaine Chan

6
What is a Data Warehouse ?
In computing, a data warehouse (DW or DWH), also known as an enterprise
data warehouse (EDW), is a system used for reporting and data analysis and
is considered a core component of business intelligence.[1] DWs are central
repositories of integrated data from one or more disparate sources. They
store current and historical data in one single place[2] that are used for
creating analytical reports for workers throughout the enterprise.[3]
The data stored in the warehouse is uploaded from the operational
systems (such as marketing or sales). The data may pass through
an operational data store and may require data cleansing[2] for additional
operations to ensure data quality before it is used in the DW for reporting.
Source: https://en.wikipedia.org/wiki/Data_warehouse

7
Etymology of “Data Lake”
Pentaho’s CTO James Dixon is credited with coining the term "data lake".
He described it in his blog in 2010:
“If you think of a data mart as a store of bottled water –
cleansed and packaged and structured for easy consumption
– the data lake is a large body of water in a more natural
state. The contents of the data lake stream in from a source
to fill the lake, and various users of the lake can come to
examine, dive in, or take samples.”
Source: https://jamesdixon.wordpress.com/2010/10/14/pentaho-hadoop-and-data-lakes/

8
Can We Place All Data Into a Single Cloud System?
Taking advantage of the current shift to the cloud, couldn't we
simply consolidate all data in a single system, like a data lake or a
“lakehouse”?
§ They are (relatively) cheap and scale out well for large data
volumes
§ Is that realistic?
§ Is that possible?

9
Do I Need Both a Data Lake and a Data Warehouse?
Data Warehouses:
Typically contain structured data
and more often than not are on
premise
• Top use case BI and Analytics
Data Lakes:
Often contain unstructured data
and typically cloud based.
• Top use case Data Science
2021 TDWI report Building the Unified Data Warehouse and Data Lake BEST PRACTICES REPORT By Fern Halper, Ph.D., and James Kobielus

10
Do I Really Need Both a Data Lake and a Data Warehouse?

11
Why?

12
Data lakes were born to efficiently address
the challenge of cost reduction:
Data lakes allow for cheap, efficient storage
of very large amounts of data.
Cloud implementation simplified the
complexity of managing a large data lake.

13
…Data lakes lack semantic consistency and governed
metadata. Meeting the needs of wider audiences require
curated repositories with governance, semantic
consistency and access controls.”

14
So How Are Organizations Going About It?

15
Final Observations From the Report
Architects own the overall design. It is no
surprise that architects are the top
contributors to the design of the data
warehousing environment. This includes data
warehouse architects (49%), enterprise
architects (43%), and IT architects (21%).
Data scientists (56%) are the top
contributor of various
components to the unified environment.

What is a Logical Data Fabric?

17
A data fabric is an architecture pattern that informs and automates the design, integration
and deployment of data objects regardless of deployment platforms and architectural
approaches.
It utilizes continuous analytics and AI/ML over all metadata assets to provide actionable
insights and recommendations on data management and integration design and
deployment patterns.
This results in faster, informed and, in some cases, completely automated data access and
sharing.
Data Fabric Definition

18
Data Fabric
A data fabric is an architecture pattern that informs and automates the design, integration and deployment
of data objects regardless of deployment platforms and architectural approaches
§ It utilizes continuous analytics and AI/ML over all metadata assets to provide actionable insights
and recommendations on data management and integration design and deployment patterns.
§ This results in faster, informed and, in some cases, completely automated data access and sharing
§ Strongly supported by both Gartner and Forrester
Data Fabric Net
Compounds Customers Products Claims
RDBMS/OLTP Traditional Analytics/BI Data Lakes Cloud Data Stores Apps and Document
Repositories
Flat Files
Third Party
Legacy
Mart
Data Warehouse
Mart
ETL ETL
XML • JSON • PDF
DOC • WEB

19
Logical Data Fabric
Demystifying the Data Fabric,
September 2020
The core of the matter is being
able to consolidate many diverse
data sources in an efficient
manner by allowing trusted data
to be delivered from all relevant
data sources to all relevant data
consumers through one
common layer.

20
Logical Data Fabric
• Data Abstraction: decoupling
applications/data usage from data
sources
• Data Integration without replication
or relocation of physical data
• Easy Access to Any Data, high
performant and real-time/ right-
time
• Data Catalog for self-service data
services and easy discovery
• Unified metadata, security &
governance across all data assets
• Data Delivery in any format with
intelligent query optimization that
leverages new and existing
physical data platforms
A logical data layer – a “logical data fabric” – that provides high-performant, real-time, and secure
access to integrated business views of disparate data across the enterprise

How Does This Apply to a
Data Warehouse/Data Lake ?

22
Logical Data Fabric Reference Architecture
LOGICAL DATA FABRIC

23
Cloud
Applications
Salesforce.com
AWS Ecosystem
AWS Data Sources
Amazon Aurora Amazon Dynamo DB
Amazon Redshift
Amazon Athena Amazon S3
Data Consumers
Amazon QuickSight
OnPrem Data
Sources
MS Excel
Other Applications
SAP Data Sources
Example: Denodo in Multiple Locations
Azure Ecosystem
Denodo Data
Virtualization
Data Catalog
SQL
API
Denodo Data
Virtualization
SQL
API
Azure Data Sources
Azure Synapse Azure Delta Lake

25
v
Autodesk Overview
• Founded 1982 (NASDAQ: ASDK)
• Annual revenues (FY 2018) $2.06B
§ Over 8,800 employees
• 3D modeling and animation software
§ Flagship product is AutoCAD
• Market sectors:
§ Architecture, Engineering, and Construction
§ Manufacturing
§ Media and Entertainment
§ Recently started 3D Printing offerings

26
v
Business Drivers for Change
• Software consumption model is changing
§ Perpetual licenses to subscriptions
§ User want more flexibility in how they use software
• Autodesk needed to transition to subscription
pricing
§ 2016 – some products will be subscription only
• Lifetime revenue higher with subscriptions
§ Over 3-5 years, subscriptions = more revenue
• Changing a licensing model is disruptive

27
v
Technology Challenges
• Current ‘traditional’ BI/EDW architecture not
designed for data streams from online apps
§ Weblogs, Clickstreams, Cloud/Desktop apps, etc.
• Existing infrastructure can’t simply ‘go away’
§ Regulatory reporting (e.g. SEC)
§ Existing ‘perpetual’ customers
• ‘Subscription’ infrastructure work in parallel
§ Extend and enhance existing systems
§ With single access point to all data
• Solution – ‘Logical Data Warehouse/Fabric’

32
v
Autodesk Successfully Changes Their Revenue Model and
Transforms Business
§ Autodesk was changing their business
revenue model from a conventional
perpetual license model to subscription-
based license model.
§ Inability to deliver high quality data in a
timely manner to business stakeholders.
§ Evolution from traditional operational
data warehouse to contemporary logical
data warehouse deemed necessary for
faster speed.
§ Successfully transitioned to subscription-
based licensing.
§ For the first time, Autodesk can do single
point security enforcement and have
uniform data environment for access.
§ General purpose platform to deliver data
through logical data warehouse.
§ Denodo Abstraction Layer helps live
invoicing with SAP.
§ Data virtualization enabled a culture of
“see before you build”.
32
Autodesk, Inc. is an American multinational software corporation that makes software for the
architecture, engineering, construction, manufacturing, media, and entertainment industries.
Case Study

34
A Logical Data Fabric
§ Pillar 1 — Integrates data across multi-cloud environments
§ Pillar 2 - Automates manual tasks using augmented intelligence
§ Pillar 3 - Boosts performance of analytics with rapid data delivery
§ Pillar 4 - Supports data discovery and data science initiatives
§ Pillar 5 - Analyzes across data at rest and data in motion
§ Pillar 6 - Catalogs all data for discovery, lineage, and associations
TDWI Checklist Report: Six Critical Capabilities of a Logical Data Fabric (May 2020) By Fern Halper and David Loshin - denodo.link/tdwi5

Product Demonstration
Director, APAC Sales Engineering, Denodo
Chris Day

Featuring Leading Industry Experts
Angel Vina
Founder & CEO
Alberto Pan
Executive VP & CTO
Ravi Shankar
Senior VP & CMO
David Loshin
President of Knowledge Integrity
Terry Moon
Enterprise Information Architect
Logical Data Fabric: The Future of
Data Management and Analytics
Michele Goetz
VP & Principal Analyst
denodo.link/DF2111
AVAILABLE ON DEMAND

Thanks!
www.denodo.com info@denodo.com
© Copyright Denodo Technologies. All rights reserved
Unless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical,
including photocopying and microfilm, without prior the written authorization from Denodo Technologies.

Building a Logical Data Fabric using Data Virtualization (ASEAN)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Building a Logical Data Fabric using Data Virtualization (ASEAN)

Similar to Building a Logical Data Fabric using Data Virtualization (ASEAN) (20)

More from Denodo

More from Denodo (20)

Recently uploaded

Recently uploaded (20)

Building a Logical Data Fabric using Data Virtualization (ASEAN)