Watch full webinar here: https://bit.ly/3joZa0a
The current data landscape is fragmented, not just in location but also in terms of processing paradigms: data lakes, IoT architectures, NoSQL, and graph data stores, SaaS applications, etc. are found coexisting with relational databases to fuel the needs of modern analytics, ML, and AI. The physical consolidation of enterprise data into a central repository, although possible, is both expensive and time-consuming. A logical data warehouse is a modern data architecture that allows organizations to leverage all of their data irrespective of where the data is stored, what format it is stored in, and what technologies or protocols are used to store and access the data.
Watch this session to understand:
- What is a logical data warehouse and how to architect one
- The benefits of logical data warehouse – speed with agility
- Customer use case depicting logical architecture implementation
A Logical Architecture is Always a Flexible Architecture (ASEAN)
1. A Logical Architecture is
Always a Flexible
Architecture
Chris Day
Director, APAC Sales Engineering
cday@denodo.com
4th Chief Digital Officer Asia Summit
3 June 2021
2. 2
www.cio.com
The goal of data architecture is to translate business
needs into data and system requirements and to
manage data and its flow through the enterprise.
3. 3
Characteristics of an Effective Data Architecture
• User-driven
• Decision makers or data consumers should easily be able to access the
data they need to meet business objectives
• Secure
• Security should built into modern data architecture, ensuring that data
is available on a need-to-know basis as defined by the business. Also
ensure regulatory compliance with legislation like GDPR
• Minimal Disruption
• Designed to accommodate and meet the changes required by either
technology or business requirements
• Elastic and adaptive
• Support multiple types of operations (e.g., batch, stream), query
operations, and deployments (e.g., on premises, public cloud, private
cloud, hybrid)
4. 4
Common Reality of Enterprise Architecture
Becomes Unmanageable & Brittle because:
IT responds by
loosely stitching
together
disparate data
sources
Any changes
break the flow
and affect
business
continuity
Business Wants All of the Data, Now
– So IT creates 100s to 1000s of brittle direct connections and
replicates large volumes of data
Inventory System
(MS SQL Server)
Product Catalog
(Web Service -SOAP)
BI / Reporting
JDBC, ODBC,
ADO .NET
Web / Mobile
WS – REST JSON,
XML, HTML, RSS
Log files
(.txt/.log files)
CRM
(MySQL)
Billing System
(Web Service -
Rest)
ETL
Portals
JSR168 / 286,
Ms Web Parts
SOA,
Middleware,
Enterprise Apps
WS – SOAP
Java API
Customer Voice
(Internet,
Unstruc)
8. 8
Data Virtualization: Unified Data Integration and Delivery
• Data Abstraction: decoupling
applications/data usage from data
sources
• Data Integration without replication
or relocation of physical data
• Easy Access to Any Data, high
performant and real-time/ right-
time
• Data Catalog for self-service data
services and easy discovery
• Unified metadata, security &
governance across all data assets
• Data Delivery in any format with
intelligent query optimization that
leverages new and existing
physical data platforms
Six Essential Capabilities of Data Virtualization
9. 9
1. Data abstraction
Abstracts access to disparate data
sources.
Acts as a single virtual repository.
Abstracts data complexities like location,
format, protocols
…hides data complexity for ease of data access by business
Enterprise architects must revise their data architecture to meet
the demand for fast data.”
– Create a Road Map For A Real-time, Agile, Self-Service Data
Platform, Forrester Research
10. 10
2. Zero replication, zero relocation
…reduces development time and overall TCO
The Denodo Platform enables us to build and deliver data
services, to our internal and external consumers, within a
day instead of the 1 – 2 weeks it would take with ETL.”
– Manager, Enervus
Leaves the data at its source; extracts only what is
needed, on demand.
Diminishes the need for effort-intensive ETL
processes.
Eliminates unnecessary data redundancy.
11. 11
3. Real-time information
Provisions data in real-time to consumers
Creates real-time logical views of data across many
data sources.
Supports transformations and quality functions
without the latency, redundancy, and rigidity of
legacy approaches
…enables timely decision-making
Denodo’s data fabric design relies on data virtualization to provide
integrated data quickly to business users to effect faster outcomes..”
– Gartner Magic Quadrant for Data Integration Tools, 18 August’ 2020
12. 12
4. Self-service data services
Facilitates access to all data, both internal and external
Enables creation of universal semantic models reflecting
business taxonomy
Connects data silos to provide best available information to
drive business decisions
…enables information discovery and self-service
Impressively quick turn around time to "unlock“ data from
additional siloes and from legacy systems - Few vendors (if any) can
compete with Denodo's support of the Restful/Odata standard -
both to provide data (northbound) and to access data from the
sources (southbound).”
– Business Analyst, Swiss Re
13. 13
5. Centralized metadata, security & governance
Abstracts data source security models and enables single-point
security and governance.
Extends single-point control across cloud and on-premises
architectures
Provides multiple forms of metadata (technical, business,
operational) to facilitate understanding of data.
…simplifies data security, privacy, audit
Our Denodo rollout was one of the easiest and most successful rollouts of critical
enterprise software I have seen. It was successful in handling our initial, security,
use case immediately, and has since shown a strong ability to cover additional
use cases, in particular acting as a Data Abstraction Layer via it's web service
functionality.”
– Enterprise Architect, Asurion
14. 14
6. Location-agnostic architecture for multi-cloud, hybrid
acceleration
Optimizes costs by migrating data, applications, and analytics
workloads to cloud without impacting the business
Enables creation of hub architecture to support integration of
data across mixed workloads.
End-to-end management of migrations/promotions and
continuous delivery processes.
…enables cloud adoption
Impressively quick turn around time to "unlock“ data from
additional siloes and from legacy systems - Few vendors (if any) can
compete with Denodo's support of the Restful/Odata standard -
both to provide data (northbound) and to access data from the
sources (southbound).”
– Business Analyst, Swiss Re
16. 16
138,000
employees
50
countries
10th
World’s Largest Bank
Is part of the Crédit Agricole Group.
World leader
in Green, Social and Sustainable Banking
Credit Agricole Story: Logical Data Lake
Today
CACIB’s architecture had
• 1000+ data flows, dozens of domain specific data warehouses, terabytes of curated data processed yearly
• 100s of business processes and 1000s of users worldwide
Credit Agricole wanted a platform
• To perform unified analytics spans its entire data infrastructure & eliminate data trapped in business silos
• Create a universal data delivery layer without having to replicate data
• Enable users to search and access the right data set via existing tools
• Data governance framework allowing definition, management and security of data
17. 17
Siloed Data Ecosystem Enterprise Data Lake?
Costly, Complex , Difficult to maintain
Is Another Data Lake Really the Answer?
18. Target Unified Analytics Using a Logical Data Fabric
Querying with user’s favorite tool
from the semantic layer
3
1
…
Finding the right data
2
Keyword search
Metadata
Logical Layer
API’s
API’s
Physical
Layer
19. 19
The Logical Data Fabric Advantages for CACIB
1. Logical Data Warehouse : Gather data from multiple data stores :
E.g. : Steering Analytics project, Counterparty Risk data Integrity Front to Back, Market Data, etc…
2. Operational analytics : capacity analytics tool to operational data sources (API, Databases, etc) in order to have the real-time data
E.g.: Front Office trading system real-time analytics by logical merging of multiple instances worldwide
3. Data services : capacity to publish an API/ODBC connection for subscriber to consume data
E.g. : IT Financial data APIs
1. Domain data prepared and published as Data Services via API/ODBC/Etc… connectors
2. Seamless connectivity with data management system (including Enterprise data calatog)
3. Data security integrated and governed
Application/process level component - Aimed to serve a specific process or use cases as a middleware
layer for data integration or data push
Domain level component (Hub) – Data Centric Initiative : Aimed to democratize the data
sharing/publication between different business departments
20. 20
Key Takeaways
A logical architecture:
• Allows adoption of newer technologies without
impacting business users.
• Improves decision making and shortens development
cycles.
• Eliminates data silos unified view of company data
from multiple repositories without the need to
replicate.
• Broadens use existing data sources improving their
ROI & value
• Improves governance and metadata management to
avoid “data swamps”