Data warehouse

WHAT A DATA WAREHOUSE IS
NOT
• A data warehouse is not a copy of a source
database with the name prefixed with “DW”
• It is not a copy of multiple tables (i.e.
customer) from various sources systems
unioned together in a view
• It is not a dumping ground for tables from
various sources with not much design put into
it

WHAT IS A DATA WAREHOUSE
AND WHY USE ONE?
A data warehouse is where you store data from
multiple data sources to be used for historical
and trend analysis reporting. It acts as a central
repository for many subject areas and contains
the "single version of truth". It is NOT to be
used for OLTP applications.

WHAT IS A DATA WAREHOUSE
AND WHY USE ONE?
Reasons for a data warehouse:
 Reduce stress on production system
 Optimized for read access, sequential disk scans
 Integrate many sources of data
 Keep historical records (no need to save hardcopy
reports)
 Restructure/rename tables and fields, model data
 Protect against source system upgrades
 Use Master Data Management, including hierarchies
 No IT involvement needed for users to create reports
 Improve data quality and plugs holes in source systems
 One version of the truth

WHY USE A DATA WAREHOUSE?
Legacy applications + databases = chaos
Production
Control
MRP
Inventory
Control
Parts
Management
Logistics
Shipping
Raw Goods
Order
Control
Purchasing
Marketing
Finance
Sales
Accounting
Management
Reporting
Engineering
Actuarial
Human
Resources
Continuity
Consolidation
Control
Compliance
Collaboration
Enterprise data warehouse = order
Single version
of the truth
Enterprise Data
Warehouse
Every question = decision
Two purposes of data warehouse: 1) save time building reports; 2) slice in dice in ways you could not do before

WHY DATA WAREHOUSING?
• Data warehousing can be considered as an
important preprocessing step for data mining
• A data warehouse also provides on-line analytical
processing (OLAP) tools for interactive
multidimensional data analysis.
Heterogeneous
Databases
Data Warehouse
data selection
data cleaning
data integration
data summarization

DATA WAREHOUSING
• Data Selection
• Only data which are important for analysis are
selected (e.g., information about employees,
departments, etc. are not stored in the warehouse)
• Therefore the data warehouse is subject-oriented
• Data Integration
• Consistency of attribute names
• Consistency of attribute data types. (e.g., dates are
converted to a consistent format)
• Consistency of values (e.g., product-ids are converted
to correspond to the same products from both
sources)
• Integration of data (e.g, data from both sources are
integrated into the warehouse)

DATA WAREHOUSING
•Data Cleaning
• Tuples which are incomplete or logically
inconsistent are cleaned
•Data Summarization
• Values are summarized according to the
desired level of analysis
• For example, HK database records the
daytime a sales transaction takes place,
but the most detailed time unit we are
interested for analysis is the day.

WHAT IS DATA WAREHOUSE?
• Defined in many different ways, but not rigorously.
• A decision support database that is maintained separately
from the organization’s operational database
• Support information processing by providing a solid
platform of consolidated, historical data for analysis.
• “A data warehouse is a subject-oriented, integrated,
time-variant, and nonvolatile collection of data in
support of management’s decision-making
process.”—W. H. Inmon
• Data warehousing:
• The process of constructing and using data warehouses

DATA WAREHOUSE—SUBJECT-
ORIENTED
• Organized around major subjects, such as customer,
product, sales.
• Focusing on the modeling and analysis of data for
decision makers, not on daily operations or
transaction processing.
• Provide a simple and concise view around particular
subject issues by excluding data that are not useful
in the decision support process.

DATA WAREHOUSE—
INTEGRATED
• Constructed by integrating multiple,
heterogeneous data sources
• relational databases, flat files, on-line transaction records
• Data cleaning and data integration techniques are
applied.
• Ensure consistency in naming conventions, encoding
structures, attribute measures, etc. among different data
sources
• E.g., Hotel price: currency, tax, breakfast covered, etc.
• When data is moved to the warehouse, it is converted.

DATA WAREHOUSE—TIME
VARIANT
• The time horizon for the data warehouse is
significantly longer than that of operational systems.
• Operational database: current value data.
• Data warehouse data: provide information from a historical
perspective (e.g., past 5-10 years)
• Every key structure in the data warehouse
• Contains an element of time, explicitly or implicitly
• But the key of operational data may or may not contain
“time element” (the time elements could be extracted from
log files of transactions)

DATA WAREHOUSE—NON-
VOLATILE
• A physically separate store of data transformed
from the operational environment.
• Operational update of data does not occur in the
data warehouse environment.
• Does not require transaction processing, recovery, and
concurrency control mechanisms
• Requires only two operations in data accessing:
• initial loading of data and access of data.

DATABASE AND DATA WARE
HOUSING….
• The Difference…
• DWH Constitute Entire Information Base For All Time..
• Database Constitute Real Time Information…
• DWH Supports DM And Business Intelligence.
• Database Is Used To Running The Business
• DWH Is How To Run The Business
// DM = DIMENSION MODELLING

TRADITIONAL RDBMS USED FOR
OLTP
• Database Systems have been used
traditionally for OLTP
• clerical data processing tasks
• detailed, up to date data
• structured repetitive tasks
• read/update a few records
• isolation, recovery and integrity are critical
• Will call these operational systems
16

ONE WORD
• OLTP Systems are
used to “run” a
business
• The Data Warehouse
helps to “optimize” the
business

DATA WAREHOUSE VS.
OPERATIONAL DBMS
• OLTP (on-line transaction processing)
• Major task of traditional relational DBMS
• Day-to-day operations: purchasing, inventory, banking,
manufacturing, payroll, registration, accounting, etc.
• OLAP (on-line analytical processing)
• Major task of data warehouse system
• Data analysis and decision making
• Distinct features (OLTP vs. OLAP):
• User and system orientation: customer vs. market
• Data contents: current, detailed vs. historical, consolidated
• Database design: ER + application vs. star + subject
• View: current, local vs. evolutionary, integrated
• Access patterns: update vs. read-only but complex queries

OLTP VS. OLAP
OLTP OLAP
users clerk, IT professional manager
function day to day operations Decision support
DB design application-oriented subject-oriented
data current, up-to-date
detailed, flat relational
isolated
historical,
summarized, multidimensional
integrated, consolidated
usage repetitive ad-hoc
access read/write
index/hash on prim. key
lots of scans
unit of work short, simple transaction complex query
# records accessed tens millions
#users thousands hundreds
DB size 100MB-GB 100GB-TB (even PB)
metric transaction throughput query throughput, response

EVOLUTION IN ORGANIZATIONAL USE
OF DATA WAREHOUSES
• Traditional heterogeneous DB integration:
Organizations generally start off with relatively simple use of data warehousing.
Over time, more sophisticated use of data warehousing evolves. The following
general stages of use of the data warehouse can be distinguished:
•Off line Operational Database
•Data warehouses in this initial stage are developed by simply copying the data
off an operational system to another server where the processing load of
reporting against the copied data does not impact the operational system's
performance.
•Off line Data Warehouse
•Data warehouses at this stage are updated from data in the operational
systems on a regular basis and the data warehouse data is stored in a data
structure designed to facilitate reporting.
•Real Time Data Warehouse
•Data warehouses at this stage are updated every time an operational system
performs a transaction (e.g. an order or a delivery or a booking.)
•Integrated Data Warehouse
•Data warehouses at this stage are updated every time an operational system
performs a transaction. The data warehouses then generate transactions that
are passed back into the operational systems.

DATA WAREHOUSE VS.
HETEROGENEOUS DBMS
• Traditional heterogeneous DB integration:
• Build wrappers/mediators on top of heterogeneous databases
• Query driven approach
• When a query is posed to a client site, a meta-dictionary is
used to translate the query into queries appropriate for
individual heterogeneous sites involved, and the results are
integrated into a global answer set
• Complex information filtering, compete for resources
• Data warehouse: update-driven, high performance
• Information from heterogeneous sources is integrated in advance
and stored in warehouses for direct query and analysis

DATA WAREHOUSE VS.
HETEROGENEOUS DBMS
• Example of a Heterogeneous DBMS
• The results from the various sources are integrated and
returned to the user
Heterogeneous
Databases
meta-
data
mediator/
wrapper
user
query
results
query
transformation
Q1
Q2
Q3
R1
R2
R3

DATA WAREHOUSE VS.
HETEROGENEOUS DBMS
• Advantages of a Data Warehouse:
• The information is integrated in advance, therefore there is
no overhead for (i) querying the sources and (ii) combining
the results
• There is no interference with the processing at local sources
(a local source may go offline)
• Some information is already summarized in the warehouse,
so query effort is reduced.
• When should mediators be used?
• When queries apply on current data and the information is
highly dynamic (changes are very frequent).
• When the local sources are not collaborative.

DISADVANTAGES OF DATA
WAREHOUSE
• Data warehouses are not the optimal environment for
unstructured data.
• Because data must be extracted, transformed and loaded into
the warehouse, there is an element of latency in data warehouse
data.
• Over their life, data warehouses can have high costs.
Maintenance costs are high.
• Data warehouses can get outdated relatively quickly. There is a
cost of delivering suboptimal information to the organization.
• There is often a fine line between data warehouses and
operational systems. Duplicate, expensive functionality may be
developed. Or, functionality may be developed in the data
warehouse that, in retrospect, should have been developed in
the operational systems and vice versa.
• // NoSQL

DATA WAREHOUSING AND OLAP
TECHNOLOGY FOR DATA MINING
• What is a data warehouse?
• A multi-dimensional data model
• Data warehouse architecture
• // (advanced) Data warehouse implementation
• // (advanced) Further development of data cube
technology
• From data warehousing to data mining

FROM TABLES AND SPREADSHEETS TO
DATA CUBES
• A data warehouse is based on a multidimensional
data model which views data in the form of a data
cube
• A data cube, such as sales, allows data to be
modeled and viewed in multiple dimensions
• Dimension tables, such as item (item_name, brand, type),
or time(day, week, month, quarter, year)
• Fact table contains measures (such as dollars_sold) and
keys to each of the related dimension tables

FROM TABLES AND SPREADSHEETS TO
DATA CUBES
• A dimension is a perspective with respect to which
we analyze the data
• A multidimensional data model is usually organized
around a central theme (e.g., sales). Numerical
measures on this theme are called facts, and they
are used to analyze the relationships between the
dimensions
• Example:
• Central theme: sales
• Dimensions: item, customer, time, location, supplier, etc.

WHAT IS A DATA CUBE?
• The data cube summarizes the measure with
respect to a set of n dimensions and provides
summarizations for all subsets of them

WHAT IS A DATA CUBE?
• In data warehousing literature, the most detailed
part of the cube is called a base cuboid. The top
most 0-D cuboid, which holds the highest-level of
summarization, is called the apex cuboid. The
lattice of cuboids forms a data cube.

CUBE: A LATTICE OF
CUBOIDS
all
time item location supplier
time,item time,location
time,supplier
item,location
item,supplier
location,supplier
time,item,location
time,item,supplier
time,location,supplier
item,location,supplier
time, item, location, supplier
0-D(apex) cuboid
1-D cuboids
2-D cuboids
3-D cuboids
4-D(base) cuboid

CONCEPTUAL MODELING OF DATA
WAREHOUSES
• The ER model is used for relational database design. For data
warehouse design we need a concise, subject-oriented schema
that facilitates data analysis.
• Modeling data warehouses: dimensions & measures
• Star schema: A fact table in the middle connected to a set of
dimension tables
• Snowflake schema: A refinement of star schema where some
dimensional hierarchy is normalized into a set of smaller
dimension tables, forming a shape similar to snowflake
• Fact constellations: Multiple fact tables share dimension tables,
viewed as a collection of stars, therefore called galaxy schema or
fact constellation

STAR SCHEMA
• A single fact table and for each dimension
one dimension table
• Does not capture hierarchies directly
32
T
i
m
e
p
r
o
d
c
u
s
t
c
i
t
y
f
a
c
t
date, custno, prodno, cityname, sales

DIMENSION TABLES
• Dimension tables
• Define business in terms already familiar to users
• Wide rows with lots of descriptive text
• Small tables (about a million rows)
• Joined to fact table by a foreign key
• heavily indexed
• typical dimensions
• time periods, geographic region (markets, cities),
products, customers, salesperson, etc.
33

FACT TABLE
• Central table
• Typical example: individual sales records
• mostly raw numeric items
• narrow rows, a few columns at most
• large number of rows (millions to a billion)
• Access via dimensions
34

EXAMPLE OF STAR SCHEMA
time_key
day
day_of_the_week
month
quarter
year
time
location_key
street
city
province_or_street
country
location
Sales Fact Table
time_key
item_key
branch_key
location_key
units_sold
dollars_sold
avg_sales
Measures
item_key
item_name
brand
type
supplier_type
item
branch_key
branch_name
branch_type
branch
foreign keys

SNOWFLAKE SCHEMA
• Represent dimensional hierarchy directly
by normalizing tables.
• Easy to maintain and saves storage
36
T
i
m
e
p
r
o
d
c
u
s
t
c
i
t
y
f
a
c
t
date, custno, prodno, cityname, ...
r
e
g
i
o
n

EXAMPLE OF SNOWFLAKE
SCHEMA
time_key
day
day_of_the_week
month
quarter
year
time
location_key
street
city_key
location
Sales Fact Table
time_key
item_key
branch_key
location_key
units_sold
dollars_sold
avg_sales
Measures
item_key
item_name
brand
type
supplier_key
item
branch_key
branch_name
branch_type
branch
supplier_key
supplier_type
supplier
city_key
city
province_or_street
country
city
normalization

FACT CONSTELLATION
• Fact Constellation
• Multiple fact tables that share many dimension
tables
• Booking and Checkout may share many dimension
tables in the hotel industry
38
Hotels
Travel Agents
Promotion
Room Type
Customer
Booking
Checkout

EXAMPLE OF FACT
CONSTELLATION
time_key
day
day_of_the_week
month
quarter
year
time
location_key
street
city
province_or_street
country
location
Sales Fact Table
time_key
item_key
branch_key
location_key
units_sold
dollars_sold
avg_sales
Measures
item_key
item_name
brand
type
supplier_type
item
branch_key
branch_name
branch_type
branch
Shipping Fact Table
time_key
item_key
shipper_key
from_location
to_location
dollars_cost
units_shipped
shipper_key
shipper_name
location_key
shipper_type
shipper

A DATA MINING QUERY LANGUAGE,
DMQL: LANGUAGE PRIMITIVES
• Cube Definition (Fact Table)
define cube <cube_name> [<dimension_list>]:
<measure_list>
• Dimension Definition ( Dimension Table )
define dimension <dimension_name> as
(<attribute_or_subdimension_list>)
• Special Case (Shared Dimension Tables)
• First time as “cube definition”
• define dimension <dimension_name> as
<dimension_name_first_time> in cube
<cube_name_first_time>
40

DEFINING A STAR SCHEMA IN
DMQL
define cube sales_star [time, item, branch, location]:
dollars_sold = sum(sales_in_dollars), avg_sales =
avg(sales_in_dollars), units_sold = count(*)
define dimension time as (time_key, day, day_of_week,
month, quarter, year)
define dimension item as (item_key, item_name, brand,
type, supplier_type)
define dimension branch as (branch_key, branch_name,
branch_type)
define dimension location as (location_key, street, city,
province_or_state, country)
41

DEFINING A SNOWFLAKE SCHEMA IN
DMQL
define cube sales_snowflake [time, item, branch,
location]:
dollars_sold = sum(sales_in_dollars), avg_sales =
avg(sales_in_dollars), units_sold = count(*)
define dimension time as (time_key, day, day_of_week,
month, quarter, year)
define dimension item as (item_key, item_name, brand,
type, supplier(supplier_key, supplier_type))
define dimension branch as (branch_key, branch_name,
branch_type)
define dimension location as (location_key, street,
city(city_key, province_or_state, country))
42

DEFINING A FACT CONSTELLATION
IN DMQL
define cube sales [time, item, branch, location]:
dollars_sold = sum(sales_in_dollars), avg_sales = avg(sales_in_dollars), units_sold
= count(*)
define dimension time as (time_key, day, day_of_week, month, quarter, year)
define dimension item as (item_key, item_name, brand, type, supplier_type)
define dimension branch as (branch_key, branch_name, branch_type)
define dimension location as (location_key, street, city, province_or_state,
country)
define cube shipping [time, item, shipper, from_location, to_location]:
dollar_cost = sum(cost_in_dollars), unit_shipped = count(*)
define dimension time as time in cube sales
define dimension item as item in cube sales
define dimension shipper as (shipper_key, shipper_name, location as location in
cube sales, shipper_type)
define dimension from_location as location in cube sales
define dimension to_location as location in cube sales

AGGREGATE FUNCTIONS ON
MEASURES: THREE CATEGORIES
• distributive: if the result derived by applying the
function to n aggregate values is the same as that
derived by applying the function on all the data
without partitioning.
• E.g., count(), sum(), min(), max().
• algebraic: if it can be computed by an algebraic
function with M arguments (where M is a bounded
integer), each of which is obtained by applying a
distributive aggregate function.
• E.g., avg(), min_N(), standard_deviation().
• holistic: if there is no constant bound on the storage
size needed to describe a sub-aggregate.
44

AGGREGATE FUNCTIONS ON
MEASURES: THREE CATEGORIES
(EXAMPLES)
• Table: Sales(itemid, timeid, quantity)
• Target: compute an aggregate on quantity
• distributive:
• To compute sum(quantity) we can first compute sum(quantity) for
each item and then add these numbers.
• algebraic:
• To compute avg(quantity) we can first compute sum(quantity) and
count(quantity) and then divide these numbers.
• holistic:
• To compute median(quantity) we can use neither median(quantity)
for each item nor any combination of distributive functions, too.

CONCEPT HIERARCHIES
46
 A concept hierarchy is a hierarchy of conceptual
relationships for a specific dimension, mapping
low-level concepts to high-level concepts
 Typically, a multidimensional view of the
summarized data has one concept from the
hierarchy for each selected dimension
 Example:
 General concept: Analyze the total sales with respect to
item, location, and time
 View 1: <itemid, city, month>
 View 2: <item_type, country, week>
 View 3: <item_color, state, year>
 ....

A CONCEPT HIERARCHY:
DIMENSION (LOCATION)
47
all
Europe North_America
MexicoCanadaSpainGermany
Vancouver
M. WindL. Chan
...
......
... ...
...
all
region
office
country
TorontoFrankfurtcity

MULTIDIMENSIONAL DATA
• Sales volume as a function of product, month, and
region
48
Product
Month
Dimensions: Product, Location, Time
Hierarchical summarization paths
Industry Region Year
Category Country Quarter
Product City Month Week
Office Day
total order
partial order
(lattice)

A SAMPLE DATA CUBE
49
Total annual sales
of TV in U.S.A.Date
Country
sum
sum
TV
VCR
PC
1Qtr 2Qtr 3Qtr 4Qtr
U.S.A
Canada
Mexico
sum

CUBOIDS CORRESPONDING TO
THE CUBE
50
all
product date country
product,date product,country date, country
product, date, country
0-D(apex) cuboid
1-D cuboids
2-D cuboids
3-D(base) cuboid
The cuboids are also called multidimensional views

DATACUBE EXAMPLE
‘color’, ‘size’: DIMENSIONS
‘count’: MEASURE
51
C / S S M L TOT
Red 20 3 5 28
Blue 3 3 8 14
Gray 0 0 5 5
TOT 23 6 18 47
f
colorsize
color; size

DATACUBES
52
C / S S M L TOT
Red 20 3 5 28
Blue 3 3 8 14
Gray 0 0 5 5
TOT 23 6 18 47
f
colorsize
color; size

DATACUBES
53
C / S S M L TOT
Red 20 3 5 28
Blue 3 3 8 14
Gray 0 0 5 5
TOT 23 6 18 47
f
colorsize
color; size

DATACUBES
54
C / S S M L TOT
Red 20 3 5 28
Blue 3 3 8 14
Gray 0 0 5 5
TOT 23 6 18 47
f
colorsize
color; size

DATACUBES
55
C / S S M L TOT
Red 20 3 5 28
Blue 3 3 8 14
Gray 0 0 5 5
TOT 23 6 18 47
f
colorsize
color; size

DATACUBES
56
C / S S M L TOT
Red 20 3 5 28
Blue 3 3 8 14
Gray 0 0 5 5
TOT 23 6 18 47
f
colorsize
color; size
DataCube

BROWSING A DATA
CUBE
• Visualization
• OLAP capabilities
• Interactive manipulation
57

TYPICAL OLAP OPERATIONS
• Browsing between cuboids
• Roll up (drill-up): summarize data
• by climbing up hierarchy or by reducing a dimension
• Drill down (roll down): reverse of roll-up
• from higher level summary to lower level summary or detailed data,
or introducing new dimensions
• Slice and dice:
• project and select
• Pivot (rotate):
• reorient the cube, visualization, 3D to series of 2D planes.
• Other operations
• drill across: involving (across) more than one fact table
• drill through: through the bottom level of the cube to its back-
end relational tables (using SQL) 58

EXAMPLE OF OPERATIONS
ON A DATACUBE
59
C / S S M L TOT
Red 20 3 5 28
Blue 3 3 8 14
Gray 0 0 5 5
TOT 23 6 18 47
f
colorsize
color; size

ON A DATACUBE
Roll-up:
• In this example we reduce one dimension
• It is possible to climb up one hierarchy
• Example (product, city)  (product, country)
60
C / S S M L TOT
Red 20 3 5 28
Blue 3 3 8 14
Gray 0 0 5 5
TOT 23 6 18 47
f
colorsize
color; size

ON A DATACUBE
Drill-down
• In this example we add one dimension
• It is possible to climb down one hierarchy
• Example (product, year)  (product, month)
61
C / S S M L TOT
Red 20 3 5 28
Blue 3 3 8 14
Gray 0 0 5 5
TOT 23 6 18 47
f
colorsize
color; size

ON A DATACUBE
Slice: Perform a selection on one dimension
62
C / S S M L TOT
Red 20 3 5 28
Blue 3 3 8 14
Gray 0 0 5 5
TOT 23 6 18 47
f
colorsize
color; size

ON A DATACUBE
Dice: Perform a selection on two or more
dimensions
63
C / S S M L TOT
Red 20 3 5 28
Blue 3 3 8 14
Gray 0 0 5 5
TOT 23 6 18 47
f
colorsize
color; size

A STAR-NET QUERY MODEL
64
Shipping Method
AIR-EXPRESS
TRUCK
ORDER
Customer Orders
CONTRACTS
Customer
Product
PRODUCT GROUP
PRODUCT LINE
PRODUCT ITEM
SALES PERSON
DISTRICT
DIVISION
OrganizationPromotion
CITY
COUNTRY
REGION
Location
DAILYQTRLYANNUALY
Time
Each circle is
called a footprint
(contracts,group,district,country,qtrly)

DATA WAREHOUSING AND OLAP
TECHNOLOGY FOR DATA MINING
• What is a data warehouse?
• A multi-dimensional data model
• Data warehouse architecture
• // (advanced) Data warehouse implementation
• // (advanced) Further development of data cube
technology
• From data warehousing to data mining
65

DESIGN OF A DATA
WAREHOUSE: A BUSINESS
ANALYSIS FRAMEWORK• Four views regarding the design of a data warehouse
• Top-down view
• allows selection of the relevant information necessary for the
data warehouse
• Data source view
• exposes the information being captured, stored, and managed
by operational systems
• Data warehouse view
• consists of fact tables and dimension tables
• Business query view
• sees the perspectives of data in the warehouse from the view
of end-user
66

DATA WAREHOUSE DESIGN
PROCESS
• Top-down, bottom-up approaches or a combination of both
• Top-down: Starts with overall design and planning
• Bottom-up: Starts with experiments and prototypes (rapid)
• From software engineering point of view
• Waterfall: structured and systematic analysis at each step before
proceeding to the next
• Spiral: rapid generation of increasingly functional systems, short turn
around time, quick turn around
• Typical data warehouse design process
• Choose a business process to model, e.g., orders, invoices, etc.
• Choose the grain (atomic level of data) of the business process
• Choose the dimensions that will apply to each fact table record
• Choose the measure that will populate each fact table record 67

Multi-Tiered Architecture
Data
Warehouse
Extract
Transform
Load
Refresh
OLAP Engine
Analysis
Query
Reports
Data mining
Monitor
&
Integrator
Metadata
Data Sources Front-End Tools
Serve
Data Marts
Operational
DBs
other
sources
Data Storage
OLAP Server

THREE DATA WAREHOUSE
MODELS
• Enterprise warehouse
• collects all of the information about subjects spanning the
entire organization
• Data Mart
• a subset of corporate-wide data that is of value to a specific
groups of users. Its scope is confined to specific, selected
groups, such as marketing data mart
• Independent vs. dependent (directly from warehouse) data mart
• Virtual warehouse
• A set of views over operational databases
• Only some of the possible summary views may be
materialized
69

DATA WAREHOUSE
DEVELOPMENT: A
RECOMMENDED APPROACH
70
Define a high-level corporate data model
Data
Mart
Data
Mart
Distributed
Data Marts
Multi-Tier Data
Warehouse
Enterprise
Data
Warehouse
Model refinementModel refinement

FROM THE DATA WAREHOUSE TO DATA
MARTS
71
Departmentally
Structured
Individually
Structured
Data Warehouse
Organizationally
Structured
Less
More
History
Normalized
Detailed
Data
Information

DATA WAREHOUSE VS DATA
MART Data Warehouse: A single organizational repository
of enterprise wide data across many or all subject
areas
 Holds multiple subject areas
 Holds very detailed information
 Works to integrate all data sources
 Feeds data mart
 Data Mart: Subset of the data warehouse that is
usually oriented to specific subject (finance,
marketing, sales)
• The logical combination of all the data marts is a data
warehouse
In short, a data warehouse as contains many subject areas, and a data mart
contains just one of those subject areas

• A data mart is a scaled down version of a data warehouse that
focuses on a particular subject area.
• A data mart is a subset of an organizational data store, usually
oriented to a specific purpose or major data subject, that may be
distributed to support business needs.
• Data marts are analytical data stores designed to focus on specific
business functions for a specific community within an organization.
• Usually designed to support the unique business requirements of a
specified department or business process
• Implemented as the first step in proving the usefulness of the
technologies to solve business problems
Reasons for creating a data mart
• Easy access to frequently needed data
• Creates collective view by a group of users
• Improves end-user response time
• Ease of creation in less time
• Lower cost than implementing a full Data warehouse
• Potential users are more clearly defined than in a full Data
Data Marts

CHARACTERISTICS OF THE
DEPARTMENTAL DATA MART
• Small
• Flexible
• Customized by Department
• OLAP
• Source is departmentally
structured data warehouse
Data mart
Data warehouse

Data Warehousing provides the Enterprise with
a memory
Data Mining provides the Enterprise
with intelligence
DATA MINING WORKS WITH
DATA
WAREHOUSE

DATA MINING FOR DECISION
SUPPORT
Two capabilities are provided new business
opportunities
• Automated prediction of trends and behavior: for
ex, targeted marketing.
• Automated discovery of previously unknown
patterns: for ex, detecting fraudulent credit card
transactions and identifying anomalous data
representing data entry-keying errors.

DATA MINING TOOLS
IT tools and techniques are used by data miners
• Neural computing: It is a machine learning approach by which
historical data can be examined for patterns.
• Intelligent agents: It is the promising approach to retrieve
information from the internet or from intranet-based databases.
• Association analysis: An approach that uses a specialized set
of algorithms that sort through large data sets and expresses
statistical rules among items.

OLAP
• Online Analytical Processing - coined by
EF Codd in 1994 paper contracted by
Arbor Software
• Generally synonymous with earlier terms such
as Decisions Support, Business Intelligence,
Executive Information System
• OLAP = Multidimensional Database

OLAP
• Online analytical processing refers to such end
user activities as DSS modelling using
spreadsheets and graphics that are done
online.
• OLAP involves many different data items in
complex relationships.
• Objective of OLAP is to analyze complex
relationships and look for patterns, trends and
exceptions.

OLAP IS FASMI
• Fast
• Analysis
• Shared
• Multidimensional
• Information

STRENGTHS OF OLAP
• It is a powerful visualization paradigm
• It provides fast, interactive response times
• It is good for analyzing time series
• It can be useful to find some clusters and outliers
• Many vendors offer OLAP tools such as brio.com,
cognus.com, microstrategy.com etc and it is
possible to access an OLAP database from web.

DATA WAREHOUSING INTEGRATION
DATA
SOURCES
(databases)
End Users:
Decision making and other
tasks:
CRM, DSS, EIS
Information Data
Warehouse
(storage)
Analytical
processing,
Data mining
Data visualization
Generate knowledge
Organizational
Knowledge base
Purchased
knowledge
Direct use
Direct use
Use
Use
STORAGE
storage
Use of
knowledge
Data
organization ;
storage
use

BUSINESS INTELLIGENCE
• One ultimate use of the data gathered and
processed in the data life cycle is for business
intelligence.
• Business intelligence generally involves the
creation or use of a data warehouse and/or data
mart for storage of data, and the use of front-end
analytical tools such as Oracle’s Sales Analyzer and
Financial Analyzer or Micro Strategy’s Web.
• Such tools can be employed by end users to access
data, ask queries, request ad hoc (special) reports,
examine scenarios, create CRM activities, devise
pricing strategies, and much more.

BUSINESS INTELLIGENCE AND DATA
WAREHOUSING

Data warehouse

More Related Content

What's hot

Similar to Data warehouse

Recently uploaded

In this document

Data warehouse