Have you heard the hype that the Data Warehouse is dead?
With technologies like the Data Lake and emerging data visualization tools continuing to evolve in the data space, enthusiasts are questioning whether conventional data layers like the data warehouse are still required to support your enterprise data strategy. While it may seem practical to move away from a data warehouse, it won’t be long before you start realizing the pitfalls of that approach. Like it or not, the data warehouse will continue to play an integral role in your organization’s Enterprise Information Architecture by ensuring actionable insights are being delivered with clean certified data.
In this session, Kunal Sharma, senior enterprise architect at Sense Corp, will:
Highlight the value of establishing a Clean Data Practice through governed data assets
Make a distinction between what “Single Source of Data” and “Best Version of The Truth” mean for an organization
Share uses cases for delivering certified data through a data warehouse
Provide a conceptual viewpoint of Enterprise Data Architecture design
Share an example of a modern analytics infrastructure platform
SQL Database Design For Developers at php[tek] 2024
The Data Warehouse is NOT Dead
1. “THE DATA WAREHOUSE IS NOT DEAD!”
A PRACTICAL GUIDE TO
MODERN ENTERPRISE INFORMATION ARCHITECTURE
2. 2
Specialist, Commercial Division
Austin, TX
ksharma@sensecorp.com
Kunal Sharma
About the Presenter
15+ years leading complex data
transformation projects for Fortune
500 and mid-size companies
Clean Data Practice Leader
3. • Mainframes
• Data Entry
• Basic Reporting
• Primitive Databases
1970s
• Personal Computers
• Business Applications
• Relational Databases
• Business Data
Warehouse
1980s
• Internet
• Centralized Data Storage
• Kimball and Inmon Data
Modeling Theory
• EDW Architecture Model
1990s
• Big Data
• Data Lakes & Hadoop
• Cloud Computing
• AI / ML
• IoT / Telematics
• Data Governance
2010s
• Broadband = More Data
• Business Intelligence
• Data Mining and
Predictive Modeling
• SaaS
• MDM
2000s
A BRIEF HISTORY OF THE
ENTERPRISE DATA WAREHOUSE
5. THE VALUE OF CLEAN DATA
DIRTY DATA CAN LEAD TO COSTLY DECISIONS
6. The Impact of Clean Data
While we know that dirty water can
impact the health of people,
We don’t as easily accept or recognize that
dirty data can impact the health of companies..
10. Use Case Considerations
Compliance Reporting
Governed data produces certified results that ensure no miscues in both internal and external reporting
Impact Analysis
Change management can easily trace and identify any impacts to data consumers
Digital Transformation
Architecture should leverage a hub and spoke model to enable domain based micro service builds
System Replacement
Converting to a new system should leverage clean data as part of any data import activities
Growth By Acquisition
Requires a data strategy that supports a consolidated view of data across multiple data sources
12. Making the Distinction
Single Source of Truth Best Version of Truth
Data storage principle to always source
information from a single source
Multiple sources of similar data across
transactional systems
Enables transparency, traceability, and
clear ownership of the data
Impacts timeliness and completeness of
enterprise data
Data usage principle for a single agreed
upon view of data
Requires a governed Master Data
Management stewardship
Results in certified “trusted” data for all
data consumption needs
Utilize business rules to eliminate data
redundancy and define metrics
18. OLAP Cubes
Defining Characteristics
• Daily data latency at minimum
• Structured by analytical consumer functions
• Semantic Layer with accompanying aggregation(s)
• Data cubes enable consumers to quickly slice, dice,
and summarize data in a presentation tool
Typical Data Consumers
• Production Support
• Presentation Tools
• Reporting Analysts
• Executives / Upper Management
21. Cloud Lake House
Streaming
Mobile
Log Files
IoT
Social
On-Premises
Databases Files
Data
Warehouse
SaaS
Applications ERP
DATA SOURCES
DATA GOVERNANCE
Data Catalog | Master & Reference Data Management | Policies & Procedures
DATA SECURITY
User Provisioning | Protected Information | Network Access
CLOUD DATA LAKE
Raw
Zone
Structured
Zone
Curated
Zone
ANALYTICS SANDBOX
Data Scientists
CLOUD DATA WAREHOUSE
Data
Marts
ODS OLAP
Cubes
CONSUMERS
Data Analysts
Presentation Tools
Business Users
APIs & Extracts
CLOUD STORAGE
STREAM PROCESSING
BATCH PROCESSING
22. Utilize the opportunity to hit the reset button
Planning For Modernization
Data Governance is critical to your success
Avoid the pitfalls of a “lift and shift then fix” migration
Start small with a focus to maximize data enrichment
Take advantage of the ecosystem to avoid vendor lock
23. Thanks For Joining Us
We hope you enjoyed the presentation.
If you’d like to learn more about
The Clean Data Initiative,
we encourage you to download the full eBook.
DOWNLOAD EBOOK
www.sensecorp.com | marketing@sensecorp.com