The document describes the evolution of data warehousing architectures, including the transition to Data Warehousing 2.0 (DW 2.0). DW 2.0 was shaped by the recognition of the life cycle of data, the need to integrate structured and unstructured data, and the need for a formal metadata infrastructure. DW 2.0 is composed of different components or sectors that each perform distinct functions. Metadata acts as the "glue" that coordinates the different components, allowing work to be distributed and ensuring tight communication between sectors, similar to how a conductor directs an orchestra. Different types of metadata are passed between components, including data descriptions, formulas, and processed statistics.
Integrating data across systems has been a perpetual challenge. Unfortunately, the current technology-focused solutions have not helped IT to improve its dismal project success statistics. Data warehouses, BI implementations, and general analytical efforts achieve the same levels of success as other IT projects – approximately 1/3rd are considered successes when measured against price, schedule, or functionality objectives. The first step is determining the appropriate analysis approach to the data system integration challenge. The second step is understanding the strengths and weaknesses of various approaches. Turns out that proper analysis at this stage makes actual technology selection far more accurate. Only when these are accomplished can proper matching between problem and capabilities be achieved as the third step and true business value be delivered. This webinar will illustrate that good systems development more often depends on at least three data management disciplines in order to provide a solid foundation.
Find more Data-Ed webinars here: http://www.datablueprint.com/resource-center/webinar-schedule/
Analytics with Descriptive, Predictive and Prescriptive Techniquesleadershipsoil
How the analytics industry has been affected by descriptive, predictive and prescriptive techniques and how these traditional analytical techniques are going to transform the industry in future
The business dimensional life cycle. Summarized from the second chapter of 'The Data Warehouse Lifecyle Toolkit : Expert Methods for Designing, Developing, and Deploying Data Warehouses' by Ralph Kimball
Big Data Ppt PowerPoint Presentation Slides SlideTeam
Big data has brought about a revolution in the field of information technology. Our content-ready big data PPT PowerPoint presentation slides shed light on the importance and relevance of large volumes of data. The data management presentation covers myriad of topics such as big data sources, market forecast, 3 Vs, technologies, workflow, data analytics process, impact, benefit, future, opportunity and challenges, and many additional slides containing graphs and charts. The biggest benefit that this big data analytics presentation template offers is that it enables you to unearth the information that can be used to shape the future of your business. Moreover, these designs can also be utilized to craft your own presentation on predictive analytics, data processing application, database, cloud computing, business intelligence, and user behavior analytics. Download big data PPT visuals which will help you make accurate business decisions. Enlighten folks on fraud with our Big Data PPt PowerPoint Presentation Slides. Convince them to be highly alert.
Integrating data across systems has been a perpetual challenge. Unfortunately, the current technology-focused solutions have not helped IT to improve its dismal project success statistics. Data warehouses, BI implementations, and general analytical efforts achieve the same levels of success as other IT projects – approximately 1/3rd are considered successes when measured against price, schedule, or functionality objectives. The first step is determining the appropriate analysis approach to the data system integration challenge. The second step is understanding the strengths and weaknesses of various approaches. Turns out that proper analysis at this stage makes actual technology selection far more accurate. Only when these are accomplished can proper matching between problem and capabilities be achieved as the third step and true business value be delivered. This webinar will illustrate that good systems development more often depends on at least three data management disciplines in order to provide a solid foundation.
Find more Data-Ed webinars here: http://www.datablueprint.com/resource-center/webinar-schedule/
Analytics with Descriptive, Predictive and Prescriptive Techniquesleadershipsoil
How the analytics industry has been affected by descriptive, predictive and prescriptive techniques and how these traditional analytical techniques are going to transform the industry in future
The business dimensional life cycle. Summarized from the second chapter of 'The Data Warehouse Lifecyle Toolkit : Expert Methods for Designing, Developing, and Deploying Data Warehouses' by Ralph Kimball
Big Data Ppt PowerPoint Presentation Slides SlideTeam
Big data has brought about a revolution in the field of information technology. Our content-ready big data PPT PowerPoint presentation slides shed light on the importance and relevance of large volumes of data. The data management presentation covers myriad of topics such as big data sources, market forecast, 3 Vs, technologies, workflow, data analytics process, impact, benefit, future, opportunity and challenges, and many additional slides containing graphs and charts. The biggest benefit that this big data analytics presentation template offers is that it enables you to unearth the information that can be used to shape the future of your business. Moreover, these designs can also be utilized to craft your own presentation on predictive analytics, data processing application, database, cloud computing, business intelligence, and user behavior analytics. Download big data PPT visuals which will help you make accurate business decisions. Enlighten folks on fraud with our Big Data PPt PowerPoint Presentation Slides. Convince them to be highly alert.
Introduction to Data Warehouse. Summarized from the first chapter of 'The Data Warehouse Lifecyle Toolkit : Expert Methods for Designing, Developing, and Deploying Data Warehouses' by Ralph Kimball
TekSlate is the leader in Tableau tutorials and other business intelligence tutorials emphasis on delivering complete knowledge through self-paced learning. Tableau Free Tutorials tech to create highly interactive dashboards using actions.
To Learn More Click On Below Link:
http://bit.ly/1zKKnPm
Data Warehouse Tutorial For Beginners | Data Warehouse Concepts | Data Wareho...Edureka!
This Data Warehouse Tutorial For Beginners will give you an introduction to data warehousing and business intelligence. You will be able to understand basic data warehouse concepts with examples. The following topics have been covered in this tutorial:
1. What Is The Need For BI?
2. What Is Data Warehousing?
3. Key Terminologies Related To Data Warehouse Architecture:
a. OLTP Vs OLAP
b. ETL
c. Data Mart
d. Metadata
4. Data Warehouse Architecture
5. Demo: Creating A Data Warehouse
An overview of the different sets of functionality of Tableau solution suite, and how it can address the many facets of a comprehensive data mining solution.
Introduction to Data Warehouse. Summarized from the first chapter of 'The Data Warehouse Lifecyle Toolkit : Expert Methods for Designing, Developing, and Deploying Data Warehouses' by Ralph Kimball
TekSlate is the leader in Tableau tutorials and other business intelligence tutorials emphasis on delivering complete knowledge through self-paced learning. Tableau Free Tutorials tech to create highly interactive dashboards using actions.
To Learn More Click On Below Link:
http://bit.ly/1zKKnPm
Data Warehouse Tutorial For Beginners | Data Warehouse Concepts | Data Wareho...Edureka!
This Data Warehouse Tutorial For Beginners will give you an introduction to data warehousing and business intelligence. You will be able to understand basic data warehouse concepts with examples. The following topics have been covered in this tutorial:
1. What Is The Need For BI?
2. What Is Data Warehousing?
3. Key Terminologies Related To Data Warehouse Architecture:
a. OLTP Vs OLAP
b. ETL
c. Data Mart
d. Metadata
4. Data Warehouse Architecture
5. Demo: Creating A Data Warehouse
An overview of the different sets of functionality of Tableau solution suite, and how it can address the many facets of a comprehensive data mining solution.
An basic ideas about needs and concepts of business intelligence.
Presented on DotNetters Tech Summit - 2015 RUET
Presenter: Maksud Saifullah Pulak
Event Url: https://www.facebook.com/events/512834685530439/
Download at http://DavidHubbard.net/powerpoint - This Introduction to Business Intelligence gives an overview of how Business Intelligence fits into business strategy in general. It does not go into the specific technologies of Business Intelligence. It is meant to be used to explain Business Intelligence to those not already familiar with Business Intelligence.
A Brief History of Information Technology
Databases for Decision Support
OLTP vs. OLAP
Why OLAP & OLTP don’t mix (1)
Organizational Data Flow and Data Storage Components
Loading the Data Warehouse
Characteristics of a Data Warehouse
A Data Warehouse is Subject Oriented
For more visit : http://jsbi.blogspot.com
This presentation will help you understand the basic building blocks of Business Intelligence. Learn how decisions are triggered, the complete decision process and who makes decisions in the corporate world.
More importantly, understand core components of a Business Intelligence architecture such as a data warehouse, data mining, OLAP (Online analytical procession) , OLTP (Online Transaction Processing) and data reporting. Each component plays an integral part which enables today's managers and decision makers collect, analyze and interpret data to make it actionable for decision making.
Business intelligence has become an integral part that needs to be incorporated to ensure business survival. It is a tool that helps analyze historical data and forecast future so that your are always one step ahead in your business.
Please feel free to like, share and comment as you please!
Replay and more: https://blogs.embarcadero.com/pytorch-for-delphi-with-the-python-data-sciences-libraries/
The next installment of the Embarcadero Open Source Live Stream takes a look at the Delphi side of the Python Ecosystem with the new Python Data Sciences Libraries and related projects that make it super easy write Delphi code against Python libraries and easily deploy on Windows, Linux, MacOS, and Android. Specific examples with the Python Natural Language Toolkit and PyTorch, the library that powers projects like Tesla Autopilot, Uber's Pyro, Hugging Face's Transformers.
This is part of a series of regular live streams discussing the latest in Embarcadero open source projects. Hosted by Jim McKeeth and joined by members of the community and developers involved in these open source projects, as well as members of Embarcadero and Idera’s Product Management. A great opportunity to see behind the scenes and help shape the future of Embarcadero’s Open Source projects.
Android on Windows 11 - A Developer's Perspective (Windows Subsystem For Andr...Embarcadero Technologies
The Windows Subsystem for Android (WSA) brings native Android applications to the Windows 11 desktop. Learn how to set up and configure Windows Subsystem for Android for use in software development. See what is required to run WSA as well as what is required to target it from your Android development. Windows Subsystem for Android is available for public preview on Windows 11.
Webinar replay and more: https://blogs.embarcadero.com/?p=134192
for Linux (WSL2) with full GUI and X windows support. Join this webinar to better understand WSL2, how it works, proper setup, configuration options, and learn to target it in your application development. Test your Linux applications on your Windows desktop without the need for a second computer or the overhead of a virtual machine. Learn to leverage additional Linux features and APIs from your applications.
Examples with Delphi 11 Alexandria and FMXLinux
Learn how Embarcadero's newly released free Python modules bring the power and flexibility of Delphi's GUI frameworks to Python. VCL and FireMonkey (FMX) are mature GUI libraries. VCL is focused on native Windows development, while FireMonkey brings a powerful flexible GUI framework to Windows, Linux, macOS, and even Android. This webinar will introduce you to these new free Python modules and how you can use them to build graphical users interfaces with Python. Part 2 will show you how to target Android GUI applications with Python!
Introduction to Python GUI development with Delphi for Python - Part 1: Del...Embarcadero Technologies
Learn how Embarcadero’s newly released free Python modules bring the power and flexibility of Delphi’s GUI frameworks to Python. VCL and FireMonkey (FMX) are mature GUI libraries. VCL is focused on native Windows development, while FireMonkey brings a powerful flexible GUI framework to Windows, Linux, macOS, and even Android. This webinar will introduce you to these new free Python modules and how you can use them to build graphical users interfaces with Python. Part 2 will show you how to target Android GUI applications with Python!
Join Jim McKeeth as he introduces you to FMXLinux, and shows how you can bring the power of FireMonkey to Linux.
Outline:
Installation via GetIt Package Manager
Linux, PAServer, SDK, & Package Installation
FMXLinux usage and Samples
FireDAC Database Access on Linux
Migrating from Windows VCL to FMXLinux
3rd Party FMXLinux Support
Deploying rich web apps via Broadway
https://embt.co/FMXLinuxIntro
Combining the Strenghts of Python and Delphi
Links replay and more
https://blogs.embarcadero.com/combining-the-strengths-of-delphi-and-python/
Python4Delphi repository
https://github.com/pyscripter/python4delphi
Part 1
https://blogs.embarcadero.com/webinar-replay-python-for-delphi-developers-part-1-introduction/
Webinar by Kiriakos Vlahos (aka PyScripter)
and Jim McKeeth (Embarcadero)
Replay https://youtu.be/aCz5h96ObUM
Find out more, and register for part 2
https://embt.co/3hSAKrg
Check out the library
https://github.com/pyscripter/python4delphi
Agenda
Motivation and Synergies
Introduction to Python
Introduction to Python for Delphi
Simple Demo
TPythonModule
TPyDelphiWrapper
Embeddable Databases for Mobile Apps: Stress-Free Solutions with InterBaseEmbarcadero Technologies
When it comes to developing mobile applications, keeping data on your device is a must-have feature, but can still be risky. With embedded InterBase, you can deploy high-performance multi-device applications that maintain 256-bit encryption, have a small footprint and need little, if any, administration.
What can participants expect to learn: Using InterBase in your mobile apps is easier than you may expect. Learn to develop mobile applications using InterBase, and how to take advantage of some of the convenient features about InterBase like Change Views and 256-bit security.
Join Mary Kelly, InterBase Engineer & RAD Software Consultant, and Jim McKeeth, Chief Developer Advocate & Engineer, for this webinar replay.
Replay: https://embt.co/2qUPwWY
TMS Software's Map Packs make it easy to integrate mapping into your applications. Based on the Google Maps and OpenStreet Maps sources. Join us for this webinar to learn how to take your mapping to the next level.
Works on VCL, FireMonkey (FMX), Windows, Android, iOS, macOS, Delphi and C++Builder.
Applications built with Delphi and C++ Builder for the Windows platform have proven to be indispensable instruments for businesses, but rewriting them for the cloud is often cost-prohibiting. rollApp offers a cloud platform that can run existing desktop applications in the cloud without any need to modify them. At this webinar you will learn how to move your application to the cloud and offer the benefits of a cloud solution to your users in a matter of a few weeks.
Learn about the latest features of C++11 that you can take advantage of today in C++Builder 10.1 Berlin.
David Millington, Embarcadero's new C++Builder Product Manager, shows cool C++11 code in the IDE that can be compiled for Windows, macOS, iOS and Android using the Embarcadero C++Builder Clang-enhanced compiler.
C++11 language features covered include:
Auto typed variables
Variadic templates
Lambda expressions
Atomic operations
Unrestricted unions
and more
Slide deck for the June 2, 2016 Embarcadero Webinar
This webinar will show you how to build mobile applications for iOS and Android using Delphi and C++Builder 10.1 Berlin. We will cover getting started, best practices for mobile UI/UX, building your first app, using FireUI Live Preview, creating custom design views and Live Previews, a real world example of creating, submitting and getting store acceptance for an iOS and Android app, working with databases, what’s new for mobile development and more.
This webinar will also give advice to Windows VCL desktop application developers who want to migrate their as much of their existing code to the iOS and Android mobile platforms
In this webinar we take a deeper dive into:
• How to get started building Mobile Apps if you are a Windows VCL desktop developer
• Building Mobile Apps using the different target platforms configurations
• Best practices and Apple/Google UI/UX guidelines for mobile applications – you’ll need to follow these to get your apps accepted.
• Creating FireUI Designer Custom IDE Views for other Mobile Devices
• FireUI Live Preview – extending the App to support custom component viewing
• Accessing Local and Remote Databases from your mobile apps
• Submitting apps to the Apple App Store, Google Play
Technical demonstrations will be presented by the team. Live Q&A will be done during and at the end of the webinar.
Slide deck used during the May 19, 2016 Embarcadero RAD Server Launch Webinar.
RAD Server is a turn-key application foundation for rapidly building and deploying services based applications. RAD Server provides automated Delphi and C++ REST/JSON API publishing and management, Enterprise database integration middleware, IoT Edgeware and an array of application services such as User Directory and Authentication services, Push Notifications, Indoor/Outdoor Geolocation and JSON data storage. RAD Server enables developers to quickly build new application back-ends or migrate existing Delphi or C++ client/server business logic to a modern services based architecture that is open, stateless, secure and scalable. RAD Server is easy to develop, deploy and operate making it ideally suited for ISVs and OEMs building re-deployable solutions.
ER/Studio is the complete business-driven data architecture solution that combines data modeling, business process, and application modeling and reporting with cross-organizational team collaboration for data architectures and enterprises of all sizes.
“Oh my goodness! What did I do?” Chances are you have heard, or even uttered this expression. This demo-oriented session will show many examples where database professionals were dumbfounded by their own mistakes, and could even bring back memories of your own early DBA days.
Businesses make critical decisions using key data assets, but stakeholders often find it difficult to navigate the complex data landscape to ensure they have the right data and understand it correctly. Companies are dealing with a number of different technologies, multiple data formats, and high data volumes, along with the requirements for data security and governance.
Watch the companion webinar at:
Join John Sterrett, Senior Advisor at Linchpin People and Scott Walz, Director of Software Consultants, to learn how execution plans get invalidated and why data skew could be the root cause to seeing different execution plans for the same query. We will look at options for forcing a query to use a particular execution plan. Finally, you will learn how this complex problem can be identified and resolved simply using a new feature in SQL Server 2016 called Query Store.
Troubleshooting Plan Changes with Query Store in SQL Server 2016
Bill inmon-data-warehousing-2-0-whitepaper
1. White Paper
Data Warehousing 2.0
Modeling and Metadata Strategies for Next Generation
Architectures
By Bill H. Inmon
Forest Rim Technology, LLC
April 2010
Corporate Headquarters EMEA Headquarters Asia-Pacific Headquarters
100 California Street, 12th Floor York House L7. 313 La Trobe Street
San Francisco, California 94111 18 York Road Melbourne VIC 3000
Maidenhead, Berkshire Australia
SL6 1SF, United Kingdom
2. Data Warehousing 2.0 – Bill Inmon
INTRODUCTION
Data warehousing has undergone a constant state of evolution since the beginning. Just when
you think that everything has been discovered and developed, data warehousing evolves once
again, mutating into a new form and structure.
EVOLUTION OF THE DATA WAREHOUSE
From the beginning the evolution of data warehousing has been shaped by powerful forces.
First there was the need for access to data. Then there was the need for integration. Then came
the need for a single version of the truth. Then different departmental needs for looking at the
same foundation of data arose.
The general evolution of the first stages of the data warehouse environment is shown by Fig 1.
edw
First there were applications, then there was a
data warehouse, then there was an infrastructure
surrounding the data warehouse
Figure 1
In the beginning there were applications. Applications grew so quickly that there developed
what was termed the “spider web environment” where the same data was scattered all over the
landscape. The frustration with the spider web environment led to the creation of the first data
warehouse. The simple and singular data warehouse addressed many of the problems of the
spider web environment.
But soon it was discovered that other components of the data warehouse infrastructure were
needed. There was a need for ETL, the process that allows data to be read in an unintegrated
application format and written out in a corporate, integrated format. Then there ware data
marts, where different departments had their own version of the base data found in the data
warehouse environment. Soon the ODS appeared where there was a need for high
performance, transaction processing on integrated data.
An entire infrastructure grew up around the world of the simple data warehouse. An
architecture called the “cif”, or the corporate information factory, grew up around the data
warehouse.
But the evolution of the data warehouse did not stop there. Soon the evolution of the data
warehouse grew to include a broader set of requirements. Soon the notion of a data warehouse
expanded to include a full set of data fulfilling a newly discovered set of requirements that
reached well beyond the original notion of what a data warehouse should be.
Embarcadero Technologies, Inc. Page - 1 -
3. Data Warehousing 2.0 – Bill Inmon
DW 2.0: ARCHITECTURE FOR THE NEXT
GENERATION OF DATA WAREHOUSING
This new architecture was called “DW 2.0”. Fig 2 shows the basic description of DW 2.0.
DW 2.0
Architecture for the next
generation of data warehousing
Transaction
data
Interactive A A A
Very
p p p
current p p p
l l l
Textual Detailed Continuous
subjects snapshot
Internal, external S S S S data
Simple u u u u
pointer b b b b
Captured Profile
Integrated text j j j j
data
S
u
j j
b b
u u
S S
b
Current++ Text id ...... j
Linkage
Text to subj Summary
Textual Detailed Continuous
subjects snapshot
Internal, external S S S S data
Simple u u u u
pointer b b b b
Captured Profile
Near line text
j j j j
data
S
u
j j
b b
u u
S S
b
Less than
current
Text id ...... j
Linkage
Text to subj Summary
Textual Detailed Continuous
subjects snapshot
S S S S data
Internal, external u u u u
Simple
b b b b
pointer Profile
Archival Captured j j j j
data
S
u
j
b
u
S j
b
u
S
text b
j
Older Text id ......
Linkage
Summary
Text to subj
Then there was DW 2.0,
architecture for the next
generation of data
warehousing
Figure 2
Like the earlier renditions of data warehouse architecture that preceded DW 2.0, DW 2.0 was
shaped by powerful evolutionary forces.
THE LIFE CYCLE OF DATA
The first powerful evolutionary force that shaped DW 2.0 was the recognition that the data
within the data warehouse contained its own life cycle. It was not enough to merely place data
Embarcadero Technologies, Inc. Page - 2 -
4. Data Warehousing 2.0 – Bill Inmon
on disk storage and call it a data warehouse. As time passed that data began to exhibit its own
characteristics. The first manifestation of the life cycle of the data within the data warehouse was
that over time, the probability of access to data in the data warehouse dropped. The older data
became, the less that data was accessed. A second manifestation of the lifecycle of data with
the data warehouse was that over time the volumes of data in the data warehouse grew rapidly.
This led to a paradox: the larger a data warehouse became and the older the data in the data
warehouse, the smaller percentage of data was being used.
UNSTRUCTURED DATA
The second manifestation of evolutionary forces in the data warehouse was the realization that
unstructured, textual data belonged in the data warehouse. The original data that was placed in
the data warehouse was transaction based, repetitive data. Many corporate decisions were
based on this kind of data. But there is much important data that is not transaction based that
belongs in the data warehouse.
METADATA
A third realization was that metadata belonged in the data warehouse as a formal and integral
component. In first generation data warehouses, metadata was an afterthought. But there are
powerful reasons why metadata needs to become an integral and formal part of the data
warehouse environment.
Fig 3 shows the evolutionary forces and their effect on the DW 2.0 environment.
DW 2.0
Architecture for the next
generation of data warehousing
Transaction
data
Interactive A A A
Very
p p p
current p p p
l l l
Textual Detailed Continuous
subjects
Internal, external
Simple
pointer
S S S S
u u u u
b b b b
snapshot
data
Recognition
Captured Profile
Integrated
Current++
text
Text id ......
j j j j
data
S S S
u
b
j
j j
b b
u u
of the need
Linkage
for a formal
A recognition Text to subj Summary
metadata
of the life cycle Textual
subjects
Detailed Continuous
snapshot infrastructure
of data within Near line
Internal, external
Captured
Simple
pointer
S S S S
u u u u
b b b b
j j j j
data
Profile S S S
j j
b b
u u
the data
text data u
b
Less than
current
Text id ...... j
Linkage
warehouse Text to subj Summary
Textual Detailed Continuous
subjects snapshot
S S S S data
Internal, external u u u u
Simple
b b b b
pointer Profile
Archival Captured j j j j
data
S S S
u
j j
b b
u u
text b
j
Older Text id ......
Linkage
Summary
Text to subj
Recognition of the need to integrate
both structured and unstructured data
in the data warehouse
Figure 3
DW 2.0 has become the architectural paradigm for modern data warehouses. For a detailed
description of DW 2.0 refer to the book DW 2.0 ARCHITECTURE FOR THE NEXT GENERATION
OF DATA WAREHOUSING, Morgan Kaufman, 2008.
Embarcadero Technologies, Inc. Page - 3 -
5. Data Warehousing 2.0 – Bill Inmon
Like most large architectures, DW 2.0 is not build all at once. Instead an organization builds first
one component of DW 2.0 then another component. Indeed some components of the data
warehouse are optional, such as the near line sector.
COMPARTMENTS OF PROCESSING
In any case, the different components of DW 2.0 consist of sectors that perform one basic
function. These sectors are in a sense their own small operating environments. Each sector has
its own purpose and its own functionality.
Fig 4 shows that each sector is neatly compartmentalized.
Interactive
Very
current
Transaction
data
A A A
p p p
p p p
l l l
Integrated
Current++
Textual Detailed Continuous
subjects snapshot
S S S S data
Internal, external
Simple u u u u
pointer b b b b
Captured Profile S S S
j j j j
text data u u u
b b b
j j j
Text id ......
Linkage
Summary
Text to subj
Near line DW 2.0 is made up of
Less than
current Textual
subjects
Detailed
S S S S
Continuous
snapshot
data
different components
Internal, external u u u u
Simple
pointer b b b b
Captured Profile S S S
j j j j
text data u u u
b b b
j j j
Text id ......
Linkage
Summary
Text to subj
Archival
Detailed Continuous
Textual
Older subjects snapshot
S S S S data
Internal, external u u u u
Simple b b b b
Captured pointer Profile S S S
j j j j
text data u u u
b b b
j j j
Text id ......
Linkage
Summary
Text to subj
Figure 4
In light of the individual compartmentalization of the sectors of DW 2.0, the question then
becomes – how is work distributed and coordinated across the different components? The
answer is that work is distributed and coordinated by means of the passage of metadata from
one component to the next.
METADATA AS THE GLUE
Fig 5 shows that metadata is the “glue” that binds the different operating components of DW
2.0 together.
Embarcadero Technologies, Inc. Page - 4 -
6. Data Warehousing 2.0 – Bill Inmon
Interactive
Transaction
Very
data
current
A A A
p p p
p p p
l l l
Integrated
Current++
Textual Detailed Continuous
subjects snapshot
S S S S data
Internal, external
Simple u u u u
pointer b b b b
Captured Profile S S S
j j j j
text data u u u
b b b
j j j
Text id ......
Linkage
Text to subj Summary
There is a
need for tight
Near line communications
Less than
current Textual
subjects
Internal, external
Detailed
S S S S
Continuous
snapshot
data
and tight
u u u u
coordination
Simple
pointer b b b b
Captured Profile S S S
j j j j
text data u u u
b b b
Text id ......
Linkage
j j j
among the
Text to subj
Summary
different
Archival
components of
Older
Textual
subjects
Detailed
S S S S
Continuous
snapshot
data
DW 2.0
Internal, external u u u u
Simple b b b b
Captured pointer Profile S S S
j j j j
text data u u u
b b b
j j j
Text id ......
Linkage
Summary
Text to subj
Figure 5
In a sense, metadata forms a lattice work that holds the components of DW 2.0 together. It is
the passage of metadata that allows the different components of DW 2.0 to work in cooperation
and in coordination with each other.
Stated differently, without metadata, the different components of DW 2.0 would not be able to
achieve a coordinated and cohesive work flow.
Consider a symphony orchestra. What kind of music would be created if the violins were playing
Beethoven’s Fifth, the cellos were playing “Hotel California”, the drums were playing Aretha
Franklin’s “Respect”, the flutes were playing “Happy Birthday”, and the trumpets were playing
“Jingle Bells”. The result would be a bunch of noise – nothing that anyone would want to listen
to.
But now suppose a conductor steps in and gets everyone to play DeBussy’s Le Mer. Now,
suddenly the very same components that just a few seconds ago sounded awful are making very
beautiful music. What is needed is a conductor, and it is metadata that plays the role of the
conductor. Metadata of course does not direct a symphony orchestra. Metadata instead directs
the DW 2.0 environment, with all of its different components and different technologies.
Fig 6 shows the role of metadata and the role of a conductor.
Embarcadero Technologies, Inc. Page - 5 -
7. Data Warehousing 2.0 – Bill Inmon
The orchestra with no conductor The orchestra with a conductor
Figure 6
So what kind of metadata is used to be passed from one DW 2.0 component to another? In
truth there are all sorts of metadata that are passed from one DW 2.0 component to another.
DIFFERENT TYPES OF METADATA
Typical types of metadata that are passed include:
• data descriptions
• data definitions
• formula and algorithms
• the timing of processing
• description of assorted volumes of data
• operating parameters
• encoded values
• processed/calculated/derived data
• ratios and statistics
• status codes, and the like
Fig 7 shows the different kinds of metadata that can be passed from one component to the
next. (Note: this list is merely a representative sample of the different types of metadata that
can be passed from one component to the next.)
Embarcadero Technologies, Inc. Page - 6 -
8. Data Warehousing 2.0 – Bill Inmon
Interactive
Transaction
Very
data
current
A A A
p p p
p p p
l l l
Detailed Continuous
snapshot
S S S S data
u
b
j
u
b
j
u
b
j
u
b
j
Profile S S S
The data that is passed from one
data u u u
b b b
j j j
component to the next includes -
Summary
- data descriptions
- parameters
Detailed Continuous
- processed data
S
u
b
S
u
b
S
u
b
S
u
b
snapshot
data - ratios and statistics
j j j j
Profile
data
S S S
u u u
b b b
j j j
- status codes
Summary
- and so forth
Detailed Continuous
snapshot
S S S S data
u u u u
b b b b
Profile S S S
j j j j
data u u u
b b b
j j j
Summary
Figure 7
The data descriptions that can be passed from one component of DW 2.0 to another often
reflect on the data model that exists for each component of DW 2.0. Indeed, there is a data
model for each of the different components of the DW 2.0 environment.
Embarcadero Technologies, Inc. Page - 7 -
9. Data Warehousing 2.0 – Bill Inmon
DIFFERENT DATA MODELS
Fig 8 shows the different data models that exist for the different components of DW 2.0.
Interactive
Transaction
Very
data
current
A
p
A
p
A
p Operational,
p p p
l l l
application
data model
Detailed
S S S S
u u u u
Continuous
snapshot
data
Integrated,
b b b b
j j j j
Profile
data
S S S
u u u
b b b
corporate
data model
j j j
Summary
Detailed
S S S S
Continuous
snapshot
data
See integrated,
u u u u
b b b b
j j j j
Profile
data
S S S
u u u
b b b
corporate
j j j
data model
Summary
Detailed
S S S S
Continuous
snapshot
data
Archival data
u u u u
b b b b
j j j j
Profile
data
S S S
u u u
model
b b b
j j j
Summary
There is a data model for each
sector of the DW 2.0 environment
Figure 8
At the operational level (i.e., the interactive sector of DW 2.0) is an application oriented data
model. This data model reflects the transactional nature of the work that is done here. While
integration may occur here, it often doesn’t. In addition, this level of data often includes the
reception of data from applications that lie outside of it.
The second data model that is found is that of the data model for the integrated sector. The
integrated sector is the place where corporate integration occurs. The integrated data model is
a classical subject oriented, non volatile model.
The third data model that (optionally) appears is the data model for the near line sector. If the
near line sector is part of the data warehouse environment, then the near line data model is
IDENTICAL to the integrated data model. Stated differently, if the near line sector appears at
all, then the data model for the near line sector is IDENTICAL, in every way, to the integrated
data model.
The fourth data model found in the DW 2.0 environment is the data model for the archival
environment. The archival data model is the data model that reflects several important aspects
of design:
• The need to look at and measure data over lengthy periods of time
• the need to need to reflect a changing metadata structure over time
• The need to store metadata directly in the same physical volume of data as the actual data
itself,
• The need to store calculation algorithms along with summarized or aggregated data
• The need to be able to free data from its software structuring
Embarcadero Technologies, Inc. Page - 8 -
10. Data Warehousing 2.0 – Bill Inmon
• The need to be able to further normalize data as it is placed in the archival environment,
and so forth.
There are then some distinctive needs for a data model for each of the different layers
processing that occur in the DW 2.0 environment.
A COMMON THEME
It is of interest that the different data models for the different sectors of DW 2.0 are in fact very
different. Having stated that, there never the less is a common theme running through each of
the different data models. For example, the integrated data model is undeniably akin to its
interactive data model. And the data model found at the archival environment is undeniably
related to the integrated data model.
The similarities of the different data models to each other are somewhat akin to looking at a
grandfather, his daughter, and the child of the daughter. There will be certain facial and body
similarities throughout the family. But no one will mistake a grandfather for his daughter, and no
one will mistake a daughter for her child.
Fig 9 shows that there are definite familial similarities running one data model and the next.
Interactive
Transaction
Very
data
current
A A A
p p p
p p p
l l l
Detailed Continuous
snapshot
S S S S data
u u u u
b b b b
Profile S S S
j j j j
data u u u
b b b
j j j
Summary
Detailed Continuous
snapshot
S S S S data
u u u u
b b b b
Profile S S S
j j j j
data u u u
b b b
j j j
Summary
Detailed Continuous
snapshot
S S S S data
u u u u
b b b b
Profile S S S
j j j j
data u u u
b b b
j j j
Summary
There is a great deal of commonality
from one type of data model to the next
Figure 9
Embarcadero Technologies, Inc. Page - 9 -
11. Data Warehousing 2.0 – Bill Inmon
THE TECHNICAL COMMUNITY
So who do the metadata and the data model aspects of the DW 2.0 environment help? The first
set of people that are helped by the data models and the metadata is the development and
maintenance organizations. The technicians of the organization find that data models and
metadata become the Bible of development and maintenance. The data models and the
metadata become the true “intellectual roadmap” for the ongoing development and
maintenance of the data warehouse environment. Stated differently, without metadata and a
data model, the technician is like a mechanic working on a Yugo where the Yugo is no longer
produced and where the manuals describing the Yugo are either all lost or all in Serbian, where
the mechanic only reads and speaks English. Trying to do a repair job on a Yugo under those
circumstances is questionable under the best of circumstances.
Fig 10 shows the use of the metadata and data models by the technical community.
Interactive
Transaction
Very
data
current
A A A
p p p
p p p
l l l
Detailed Continuous
snapshot
S S S S data
u u u u
b b b b
Profile S S S
j j j j
data u u u
b b b
j j j
Summary
Detailed Continuous
snapshot
S S S S data
u u u u
b b b b
Profile S S S
j j j j
data u u u
b b b
j j j
Summary
Detailed Continuous
snapshot
S S S S data
u u u u
b b b b
Profile S S S
j j j j
data u u u
b b b
j j j
Summary
The data model for each component of DW 2.0
becomes the intellectual road map for the
building and the maintenance the data
structures found in the component
Figure 10
THE END USER ANALYTICAL COMMUNITY
But there is another audience that is well served by metadata and data models, and that
audience is the end user analyst.
Consider a newly hired end user analyst that has just received an office right across from you.
The end user analyst has just received an MBA and wants to show his/her stuff. The end user
Embarcadero Technologies, Inc. Page - 10 -
12. Data Warehousing 2.0 – Bill Inmon
analyst starts to build an analysis and becomes perplexed. There are so many types of data, so
much similar data, so much data that is of different ages that the analyst does not know where
to start. Merely biting off a chunk of data may be very misleading if the analyst doesn’t chose
the proper data. And there is a lot to choose from.
So the analyst comes to your office and asks –“how do I find my way around this environment?
There is a lot to choose from and I don’t want to start with the wrong data.”
It is at this point that the metadata and the data models become invaluable. Without the
metadata and the data models the end user analyst may wander around the maze of data for a
long time. But with the metadata and the data models, the end user analyst can determine what
data is where and what data provides the best source for the analysis at hand.
Fig 11 shows that metadata and the data models are extremely useful to the analytical
community as well as the technical community.
Interactive
Transaction
Very
data
current
A A A
p p p
p p p
l l l
Detailed Continuous
snapshot
S S S S data
u u u u
b b b b Profile S S S
j j j j
data u u u
b b b
j j j
Summary
Detailed Continuous
snapshot
S S S S data
u u u u
b b b b Profile
j j S S S
j j
data u u u
b b b
j j j
Summary
Detailed Continuous
snapshot
Another group of people who are greatly helped
S S S S
u u u u
b b b b
data
Profile S S S
by metadata and data models are the end user
j j j j
data u u u
b b b
j j j analysts who have to find what data there is to
Summary use as they do their analysis
Figure 11
SUMMARY
In this paper we have discussed DW 2.0 and the evolution of the architecture surrounding data
warehouse. DW 2.0 has different sectors. There is a need for metadata to control the activities in
each of these sectors. In addition there is a data model for each of the sectors. Each data model
is different and yet there is a common theme running through each data model.
The data models and the metadata serve two communities – the technical community and the
end user analysis community. For the technical community the metadata and the data models
act as an intellectual roadmap. For the end user analytical community, the data models and the
Embarcadero Technologies, Inc. Page - 11 -
13. Data Warehousing 2.0 – Bill Inmon
metadata serve as a guidepost for where the end user analyst should go to in order to find the
proper data for analysis.
REFERENCES
Inmon, W H - DW 2.0 ARCHITECTURE FOR THE NEXT GENERATION OF DATA
WAREHOUSING, Morgan – Kaufman, 2008.
ABOUT THE AUTHOR
Bill Inmon, “the father of data warehousing”, has written 50 books translated into 9 languages.
Bill founded and took public the world’s first ETL software company. Bill has written over 1000
articles and published in most major trade journals.
Bill has conducted seminars and spoken at conferences on every continent except Antarctica.
Bill holds three software patents. Bill’s latest company is Forest Rim Technology, a company
dedicated to the access and integration of unstructured data into the structured world. Bill’s
website – inmoncif.com - has attracted over 1,000,000 visitors a month. Bill’s weekly newsletter
in b-eye-network.com is one of the most widely read in the industry and goes out to 75,000
subscribers each week.
Embarcadero Technologies, Inc. is a leading provider of award-winning tools for application
developers and database professionals so they can design systems right, build them faster and
run them better, regardless of their platform or programming language. Ninety of the Fortune
100 and an active community of more than three million users worldwide rely on Embarcadero
products to increase productivity, reduce costs, simplify change management and compliance
and accelerate innovation. The company’s flagship tools include: Embarcadero® RAD Studio,
DBArtisan®, Delphi®, ER/Studio®, JBuilder® and Rapid SQL®. Founded in 1993, Embarcadero is
headquartered in San Francisco, with offices located around the world. Embarcadero is online
at www.embarcadero.com.
Embarcadero Technologies, Inc. Page - 12 -