Watch full webinar here: https://buff.ly/3Cgntpf
Data that is increasingly distributed across on-premises and multiple cloud platforms offers both opportunities and challenges for insights-driven business decision making. This is true for any industry and any organization, small or large. While modern data architectures such as data fabric or data mesh offer innovative solutions to extract maximum value from organizational data, in many instances physical replication or consolidation of data makes those solutions ineffective.
In this session Paul Moxon, SVP of Data Architecture and Chief Evangelist at Denodo will present a number of customer examples illustrating how organizations have successfully implemented a logical data architecture.
4. 4
$2.7 TRILLION
is the economic value of goods flowing through
our distribution centers each year, representing:
4.
0 %
of GDP for the 19 countries where
we do business
%
2.8
of the World’s GDP
1983 100
GLOBA
L 1,200MSF
Founded Most sustainable corporations
$196B
Assets under management on four continents
MILLION
employees under Prologis’ roofs
1.1
Prologis – Global Industrial Real Estate Company
7. 7
Seamless Migration to Snowflake
• Large or critical Cloud migrations are
risky
• Big Bang approach is not advised
• Phased approach is recommended
• Select data set to migrate, copy to Cloud
• Test and tune data access, then go live
• Repeat for next data set and so on
• Use Denodo as abstraction layer during
migration process
• Isolate users from shift of data
9. 9
DATA FLOW
• Create a virtual representation of the physical tables
from legacy DWH in Denodo Cloud Platform.
• Connect other on-prem data sources to Denodo. Build
the foundation for a logical data warehouse or logical
data lake.
• Start moving physical objects from on-prem Data
Warehouse to Snowflake in bite-sized chunks to avoid
any downtime and ensure proper testing procedures.
• Switch the connection from legacy DWH to Snowflake
Cloud Platform inside Denodo.
• Maintain one consistent business data model across all
consumers and reporting tools. Reuse analytical objects
across multiple tools and consuming applications.
1
2
3
4
5
Example – Zero-Downtime Migration to Snowflake
11. 11
Landsbankinn
• Leading financial institution in Iceland
• 40% Market share Individual Banking
• 33% Market share Corporate Banking
• Best ESG risk ratings amongst European
banks (Sustainalytics 2021)
• Best bank at the Icelandic consumer
satisfaction ratings (Ánægjuvogin / Stjórnvísi 2021)
12. 12
SAS environment
Year Zero - Before Data Virtualization
▪ Too many query points
▪ Heterogenous technologies
▪ Complex source systems
▪ Scattered business rules
▪ Semantic layers in BI
▪ Business logics in DB views
▪ Many points of access control
▪ Audit points all over the place
▪ Each system has its own access control
KPI DB Source DBs New DWH Old DWH Markets DB
Views
BO
reporting
Self-service
BI
PDF
statements
MS Office
Integration
Views
Views
Views
General Reporting
KPI
Self-Service
data
Analytics
Reports
Analytics
Server
Risk Reporting
Monitoring / Audit Business security
Business rules
Board
Other DBs
SAP BO Semantic Layer
Data
Sources
Semantic
Layer
13. 13
Year 1 - The Logical Data Warehouse
▪ Unique point of query
▪ “Need data? LDW has the answer!”
▪ For reporting, analytics, APIs, …
▪ Unique point of truth
▪ Business logic repository
▪ Lineage available
▪ Unique point of access control
▪ Unified access to the data
▪ Unique point of auditing
KPI DB Source DBs New DWH Old DWH Markets DB
BO
reporting
Self-service
BI
MS Office
Integration
General Reporting
KPI Self-Service
data
Analytics
Reports
Analytics
Server
Risk Reporting
Board
Other DBs
Data
Sources
Logical Data Warehouse w/ Denodo
Monitoring / Audit Business security
Business rules
PDF
statements
14. 14
Years 2 and 3 - Expansion and Modernization
▪ Addition of data consumers
▪ Tableau
▪ REST / Restful APIs
▪ Addition of more data sources
▪ Where ETL is not required
▪ When history is provided in source
▪ Logical data pipelines
▪ Reduces the number of ETL jobs
▪ EDW gets data from LDW
BO
Reporting Tableau
RestWS to
Excel
General Reporting
KPI
Self-Service
data
Analytics
Reports
Analytics
Server
Risk Reporting
Board
Data
Sources
Logical Data Warehouse w/ Denodo
KPI DB
Source
DBs
New
DWH
Old
DWH
Markets
DB
Other
DBs
Flat files
Excel
SaaS
REST
SOAP
WWW
Customers
Domains
Operational
systems
Monitoring / Audit Business security
Business rules
Customer
statements
16. 16
Leading Global Bank – Pain Points
Supporting
Multiple Data
Access Tools
Changing
Technologies
Data
Lifecycle
Management
Data
Discovery
17. 17
Leading Global Bank – Data Marketplace Objectives
DATA DISCOVERY
SECURITY
CONSISTENT
ACCESS
INTERFACE
REDUCE BARRIERS TO ADOPTION
DATA ACCESS
AGILITY
02
03
04
05
06
01
18. 18
Leading Global Bank – Data Marketplace
Data Virtualization Platform
Data Marketplace
Client & Account
Active Clients Client
Accounts
Party Summary
Positions & Holdings Securities & Pricing Market Data Hub Index & Benchmark
Systems of Record Data Lake Data Warehouse
with Business Semantic Layer
Virtual Data Lake
20. 20
About BHP
Company Profile and Background
• Anglo-Australian multinational mining, metals and petroleum dual-listed public company headquartered in Melbourne, Victoria,
Australia.
• BHP ranked as the world's largest mining company, based on market capitalization, and as Melbourne's third-largest
company by revenue,
• BHP has mining operations in Australia, North America, and South America, and petroleum operations in the U.S., Australia,
Trinidad and Tobago, UK, and Algeria.
• The company has four primary operational units
• Coal
• Copper
• Iron ore
• Petroleum
• No of Employees : 80,000
• Revenue : US$65.098 billion (2022)
21. 21
BHP – Globally Distributed Data and Users
Houston DC
Santiago DC
Perth DC Brisbane
AWS US East
Escondida
Jansen London
Singapore
Kuala Lumpor
Shanghai
AWS
APAC
22. 22
BHP – Global Data Fabric
Houston DC
Santiago DC
Perth DC Brisbane
AWS US East
AWS
APAC
Escondida
Jansen London
Singapore
Kuala Lumpor
Shanghai
Every Data Virtualization cluster is connected to local
data sources, and is the access point for local
consumer apps such as BI and analytics tools. Each
Data Virtualization cluster has visibility of the datasets
available from all other clusters, and requests this data
from it's peer cluster as required by end users
24. 24
Denodo Platform: The Foundation of a Logical Data Architecture
Agile Data
Integration
Logical Data
Abstraction
Smart Query
Acceleration
Advanced
Semantics
Automation &
Recommendation
Unified Security
& Governance
Data Catalog
AI/ML
6 Key Capabilities of Logical Data Management Differentiated Use Cases
Hybrid/Multi-Cloud
Data Integration
Data Marketplace/
Self-Service Analytics
Governance &
Compliance
3600
View of Entities
(e.g., Customer)
Accelerated Integration
for M&A Activities
Data Democratization
Enterprise Data Services
Data Fabric/ Data Mesh
25. 25
Benefits of a Logical Data Architecture
“Now, we can do weekly releases.
We’re able to add new data sources
within 2 to 3 hours. We’re about 60%
faster than we were in the old world.”
VP of data and analytics, real estate
“To me, it all boils down to speed to
insights. Not having to wait to get the
question that you have top-of-mind
answered with data is huge.”
VP of data and analytics, real estate