Data-Ed: Data Systems Integration & Business Value Pt. 3: Warehousing

Copyright 2013 by Data Blueprint
Data Systems Integration & Business Value Part 3: Warehousing
Certain systems are more data focused than others. Usually their
primary focus is on accomplishing integration of disparate data. In
these cases, failure is most often attributable to the adoption of a single
pillar (silver bullet). The three webinars in the Data Systems Integration
and Business Value series are designed to illustrate that good systems
development more often depends on at least three DM disciplines (pie
wedges) in order to provide a solid foundation. Integrating data across
systems has been a perpetual challenge. Unfortunately, the current
technology-focused solutions have not helped IT to improve its dismal
project success statistics. Data warehouses, BI implementations, and
general analytical efforts achieve the same levels of success as other
IT projects – approximately 1/3rd are considered successes when
measured against price, schedule, or functionality objectives. The first
step is determining the appropriate analysis approach to the data
system integration challenge. The second step is understanding the
strengths and weaknesses of various approaches. Turns out that
proper analysis at this stage makes actual technology selection far
more accurate. Only when these are accomplished can proper
matching between problem and capabilities be achieved as the third
step and true business value be delivered.
Date: September 10, 2013
Time: 2:00 PM ET/11:00 AM PT
Presenter: Peter Aiken, Ph.D.
1

Commonly Asked Questions
1) Will I get copies of the
slides after the event?
2) Is this being recorded so I
can view it afterwards?
2

Get Social With Us!
Live Twitter Feed
Join the conversation!
Follow us:
@datablueprint
@paiken
Ask questions and submit your
comments: #dataed
Like Us on Facebook
www.facebook.com/datablueprint
Post questions and comments
Find industry news, insightful
content
and event updates.
Join the Group
Data Management & Business
Intelligence
Ask questions, gain insights and
collaborate with fellow data
management professionals
3

4
Peter Aiken, PhD
• 25+ years of experience in data
management
• Multiple international awards &
recognition
• Founder, Data Blueprint (datablueprint.com)
• Associate Professor of IS, VCU (vcu.edu)
• President, DAMA International (dama.org)
• 8 books and dozens of articles
• Experienced w/ 500+ data
management practices in 20 countries
• Multi-year immersions with
organizations as diverse as the
US DoD, Nokia, Deutsche Bank,
Wells Fargo, and the Commonwealth
of Virginia
2

Data Systems Integration & Business
Value Part 3: Warehousing
Presented by Peter Aiken, Ph.D.

1. Data management overview
2. Motivation for warehousing integration
technologies (reporting->BI->Analytics)
3. What are warehousing integration
technologies?
4. Warehousing and architecture focus
5. The use of meta models
6. Guiding principles & best practices
7. Take aways, references and Q&A
Tweeting now:
#dataed
Data Systems Integration & BV Part 3: Warehousing
6
3. What are warehousing integration technologies?

technologies?
Tweeting now:
#dataed
7

Data Program
Coordination
Feedback
Data
Development
Standard
Data
Five Integrated DM Practice Areas
Organizational Strategies
Goals
Business
Data
Business Value
Application
Models &
Designs
Implementation
Direction
Guidance
8
Organizational
Data Integration
Data
Stewardship
Data Support
Operations
Data
Asset Use
Integrated
Models
Leverage data in organizational activities
Data management
processes and
infrastructure
Combining multiple
assets to produce
extra value
Organizational-entity subject
area data
integration
Provide reliable data
access
Achieve sharing of data within a
business area

Five Integrated DM Practice Areas
Manage data coherently.
Share data across boundaries.
Assign responsibilities for data.
Engineer data delivery systems.
Maintain data availability.
Data Program
Coordination
Organizational Data
Integration
Data Stewardship Data Development
Data Support
Operations
9

Hierarchy of Data Management Practices (after Maslow)
• 5 Data management
practices areas /
data management
basics ...
• ... are necessary but
insufficient
prerequisites to
organizational data
leveraging
applications that is
self actualizing data
or advanced data
practices Basic Data Management Practices
– Data Program Management
– Organizational Data Integration
– Data Stewardship
– Data Development
– Data Support Operations
http://3.bp.blogspot.com/-ptl-9mAieuQ/T-idBt1YFmI/AAAAAAAABgw/Ib-nVkMmMEQ/s1600/maslows_hierarchy_of_needs.png
Advanced
Data
Practices
• MDM
• Mining
• Big Data
• Analytics
• Warehousing
• SOA
W
arehousing

• Data Management Body of Knowledge
(DMBOK)
– Published by DAMA International, the professional
association for
Data Managers (40 chapters worldwide)
– Organized around primary data management
functions focused around data delivery to the
organization and several environmental elements
• Certified Data Management Professional
(CDMP)
– Series of 3 exams by DAMA International and
ICCP
– Membership in a distinct group of
fellow professionals
– Recognition for specialized knowledge in a
choice of 17 specialty areas
– For more information, please visit:
• www.dama.org, www.iccp.org
DAMA DM BoK & CDMP
11

Series Context
• Certain systems are more data
focused than others. Usually
their primary focus is on
accomplishing integration of
disparate data. In these cases,
failure is most often attributable
to the adoption of a single
technological pillar (silver bullet).
The three webinars in the Data
Systems Integration and Business Value
series are designed to illustrate that
good systems development more often depends on at least three
DM disciplines (pie wedges) in order to provide a solid foundation.
• Data Systems Integration & Business Value
– Pt. 1: Metadata Practices
– Pt. 2: Cloud-based Integration
– Pt. 3: Warehousing, et al.
12

Uses
Part 1: Metadata Take Aways
• Metadata unlocks the value of data, and therefore
requires management attention [Gartner 2011]
• Metadata is the language of data governance
• Metadata defines the essence of integration challenges
Sources
Metadata Governance
Metadata
Engineering
Metadata
Delivery
Metadata Practices
Metadata
Storage
13
Specialized Team Skills

Part 2: Take Aways
• Data governance, architecture,
quality, development maturity are
necessary but insufficient
prerequisites to successful data
cloud implementation
• A variety of cloud options will
influence cloud and data
architectures in general
– You must understand your architecture
and strategy in order to evaluate the
options
• Data must be reengineered to be
– Less
– Better quality
– More shareable
– for the cloud
• Failure to do these will result in more
business value for the cloud vendors/
service providers and less for your
organization

Summary: Data Warehousing & Business Intelligence Management
15

technologies?
Tweeting now:
#dataed
16

technologies?
Tweeting now:
#dataed
17

• Bank accounts are of varying
value and risk
• Cube by
– Social status
– Geographical location
– Net value, etc.
• Balance return on the loan with
risk of default
18
• How to evaluate the portfolio as a whole?
– Least risk loan may be to the very wealthy, but there are a very
limited number
– Many poor customers, but greater risk
• Solution may combine types of analyses
– When to lend, interest rate charged
Example: Portfolio Analysis

Target Isn't Just Predicting Pregnancies
19
http://rmportal.performedia.com/node/1373

15 years ago, CarMax started as a way to make the car buying experience simple, fair, and fun. Today CarMax is a FORTUNE 500 retailer and one of FORTUNE’s “100 Best Companies to
Work For.” And we are hiring talented individuals who are interested in:
--solving original, wide-ranging, and open-ended business problems
--not only discovering new insights, but successfully implementing them
--making a significant mark on a growing company
--developing the fundamental skills for a rewarding business career
If that sounds like you, the Strategy Analyst position is the unique opportunity you’ve been looking for. The strategy team at CarMax currently consists of over 40 analysts, many of whom
are recent college graduates from top schools with a variety of academic backgrounds (computer science, economics, English, engineering, journalism, math, political science). These
analysts lead advances and decisions in several key business areas:-Inventory and pricing—what is the optimal selection of inventory, how do we acquire it, what should we pay for it, what
should we price it for?
-Expansion planning—which markets should we enter and how do we store those markets? Will each $10-30 million store investment generate a sufficient economic return?
-Credit strategy—how can our bank (CarMax Auto Finance) approve more customers for loans and convert more approvals to sales?
-Marketing and consumer insight—how do we reach our customers, increase traffic to our stores, and best use the internet to drive sales and build our brand
-Industry and competitive research—what middle- and long-term risks are we exposed to, and how best do we prepare to respond?
-Production—how do we increase vehicle reconditioning quality while reducing cost and production time?
-Sales process and workforce—what is the best way to serve customers in our stores, and how do we manage, motivate and compensate our sales team?
Even early in your career at CarMax, you will have the responsibility to own an area of the business and will be expected to improve it. For example, one undergraduate recruit used data
analysis to reformulate our retail pricing strategy, pitched and sold his idea to the senior executive team, and implemented a new system nationwide in his first 6 months with the company.
That is the kind of impact you can make at CarMax. And as you do this, you will work closely with the senior executives and analytical managers to develop the fundamental and advanced
skills that underpin a successful career in business. In fact, most of our managers in the strategy group started at CarMax as analysts, and our VP of Strategy and Analysis started his
career here through our undergraduate recruiting program. While an MBA is not required to advance or contribute at CarMax, analysts who have chosen to pursue a business degree have
enjoyed superior acceptance rates at their first choice schools, including Harvard, Chicago, UVa, Columbia, and Duke.
Your opportunities to develop, contribute, and lead as an analyst at CarMax are as great as the company’s opportunity to grow. While CarMax is already the largest used car retailer in the
country (with over $8 billion in sales and over 90 superstores across the country), we have only 2% of the 1 to 6-year-old used car market, which, at $280 billion annually, is bigger than the
home improvement or consumer electronics industries. CarMax is already growing at 15% a year, and over the next 10 years plans to have 250-300 stores and achieve $25+ billion in
annual sales. As an analyst, you can be an integral part of that growth, all while enjoying a casual and friendly environment, a diverse group of talented associates, a healthy work-life
balance, and excellent compensation and benefits.
An ideal candidate will have
--Demonstrated top caliber analytic and problem solving skills --History of achievement demonstrated by top 15% GPA, with a quantitative major(s), and/or other recognition such as
scholarships, awards, honor societies
-- Passion for business and desire to develop into a strong business leader
We encourage you to apply. For more information, please visit us at the career fair, on our website (www.carmax.com/collegerecruiting), or email us at college_recruiting@carmax.com.
http://www.seas.virginia.edu/careerdevelopment/index.php?option=com_careerfairstudent&task=detailView&employerId=216&eventId=3
- datablueprint.com
CarMax Example Job Posting
24
own an area of the business and will be expected to improve it
--solving original, wide-ranging, and open-ended business problems
--not only discovering new insights, but successfully implementing them
--making a significant mark on a growing company
--developing the fundamental skills for a rewarding business career

DW, Analytics, BI, Meta-Integration Technologies
Definitions
• Beyond the nuts and bolts of data
management
• Analysis of information that had
not been integrated previously
Business Intelligence
• Dates at least to 1958
• Support better business decision
making
• Technologies, applications and
practices for the collection,
integration, analysis, and
presentation of business
information
• Also described as decision
support
21
From The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International
Data Warehousing
• Operational extract, cleansing,
transformation, load, and
associated control processes for
integrating disparate data into a
single conceptual database

22
Definitions, cont’d
• Study of data to discover and
understand historical patterns to
improve future performance
• Use of mathematics in business
• Analytics closely resembles
statistical analysis and data mining
– based on modeling involving
extensive computation.
• Some fields within the area of
analytics are
– enterprise decision management,
marketing analytics, predictive
science, strategy science, credit
risk analysis and fraud analytics.

23
from MicroStrategy, Better Business Decisions Every Day: Integrating Business Reporting & Analysis
Example: Set Analysis

Polling Question #1
Do you have start data
warehouse, data marts
and/or other warehousing
forms of integration?
a) Last year (2012)
b) This year (2013)
c) Next Year (2014)
d) Nope
24

technologies?
Tweeting now:
#dataed
25

technologies?
Tweeting now:
#dataed
26

• Inmon:
–"A subject oriented, integrated, time variant, and non-
volatile collection of summary and detailed historical
data used to support the strategic decision-making
processes of the organization."
• Kimball:
–"A copy of transaction data specifically structured for
query and analysis."
• Key concepts focus on:
–Subjects
–Transactions
–Non-volatility
–Restructuring
Warehousing Definitions
27

Top 10 Data Warehouse Failure Causes
1. The project is over budget
2. Slipped schedule
3. Functions and
capabilities not
implemented
4. Unhappy users
5. Unacceptable performance
6. Poor availability
7. Inability to expand
8. Poor quality data/reports
9. Too complicated for users
10.Project not cost justified
28
from The Data Administration Newsletter, www.tdan.com

29
Basic Data Warehouse Analysis
• Emphasis on the
cube
• Permits different
users to "slice
and dice"
subsets of data
• Viewing from
different
perspectives

30
Warehouse Analysis
• Users can "drill"
anywhere
• Entire collection is
accessible
• Summaries to
transaction-level
detail

Oracle
31
from The DAMA Guide to the Data Management Body of Knowledge © 2009 by DAMA International

R& D Applications
(researcher supported, no documentation)
Finance Application
(3rd GL, batch
system, no source)
Payroll Application
(3rd GL)
Payroll Data
(database)
Finance
Data
(indexed)
Personnel Data
(database)
R & D
Data
(raw)
Mfg. Data
(home grown
database) Mfg. Applications
(contractor supported)
Marketing Application
(4rd GL, query facilities,
no reporting, very large)
Marketing Data
(external database)
Personnel App.
(20 years old,
un-normalized data)
32
Multiple Sources of (for example) Customer Data

Corporate Information Factory Architecture
33

34

35

36

MetaMatrix Integration Example
37
• EII Enterprise Information Integration
– between ETL and EAI -
delivers tailored views of
information to users at the
time that it is required

Linked Data
38
Linked Data is about using the Web to connect related data that wasn't
previously linked, or using the Web to lower the barriers to linking data
currently linked using other methods. More speciﬁcally, Wikipedia deﬁnes
Linked Data as "a term used to describe a recommended best practice for
exposing, sharing, and connecting pieces of data, information, and knowledge
on the Semantic Web using URIs and RDF."
linkeddata.org

technologies?
Tweeting now:
#dataed
39

technologies?
Tweeting now:
#dataed
40

Kimball's DW Chess Pieces
41

3
Courtesy of: http://www.infosys.com/industries/healthcare/industryofferings/Pages/healthcare
-data-warehousing.aspx
Data Warehousing

3
Descriptive
Ask: What happened? What is happening?
Find: Structured data
Show: Profiles, Bar/Pie charts, Narrative
Predictive
Ask: What will happen? Why will it happen?
Find: Structured/unstructured data
Show: Risk Profiles, Pros/Cons, Care Recs
Prescriptive
Ask: What should I do? Why should I do it?
Find: Unstructured/structured data
Show: Strategic Goals, Support Recs
u Organization-wide
u Volume and Noise
u Utility
u Meaningful scoring
u Actionable recs
u Realistic goals
u Support
u Manage & measure
Analytics in Health Care

3
Descriptive
Ask: What happened? What is happening?
Find: Structured data
Show: Profiles, Bar/pie charts, Narrative
Predictive
Ask: What will happen? Why will it happen?
Find: Structured/unstructured data
Show: Risk Profiles, Pros/Cons, Care Recs
Prescriptive
Ask: What should I do? Why should I do it?
Find: Unstructured/structured data
Show: Strategic Goals, Support Recs
BioMarin Licenses Factor VIII
Gene Therapy Program for
Hemophilia
Novel Gene Therapy Approach to
Hemophilia B
Sangamo BioSciences Receives
$6.4 Million
Strategic Partnership Award From
California Institute for
Regenerative Medicine to
Develop ZFP Therapeutic®
Treating Hemophilia in the 2010s
Hemophilia Management

45
Styles of Business Intelligence

Health Care Provider Data Warehouse
• 1.8 million members
• 1.4 million providers
• 800,000 providers no key
• 2.2% prov_number = 9 digits (required)
• 29% prov_ssn ≠ 9 digits
• 1 User
46
"I can take a roomful
of MBAs and
accomplish this
analysis faster!"

Top Causes of Data Warehouse Failure
• Poor Quality Data
–Many more values of
gender code than (M/F)
• Incorrectly Structured Data
–Providing the correct
answer to the wrong
question
• Bad Warehouse Design
–Overly complex
47

Indiana Jones: Raiders Of The Lost Ark
48

49
Business Intelligence Features
Problematic Data Quality

5 Key Business Intelligence Trends
1. There's so much data, but too little
insight. More data translates to a
greater need to manage it and make
it actionable.
2. Market consolidation means fewer
choices for business intelligence users.
3. Business Intelligence expands from the Board Room to the front
lines. Increasingly, business intelligence tools will be available at
all levels of the corporation
4. The convergence of structured and unstructured data Will create
better business intelligence.
5. Applications will provide new views of business intelligence data.
The next generation of business intelligence applications is
moving beyond the pie charts and bar charts into more visual
depictions of data and trends.
50
http://www.cio.com/article/150450/Five_Key_Business_Intelligence_Trends_You_Need_to_Know?page=2&taxonomyId=3002

Polling Question #2
Do you have?
a) A single enterprise
data warehouse
b) Coordinated data
marts
c) Both
d) Uncoordinated
efforts
e) None
51

technologies?
Tweeting now:
#dataed
52

technologies?
Tweeting now:
#dataed
53

Source:http://dmreview.com/article_sub.cfm?articleID=1000941 used with permission
Meta Data Models
54

Metadata Data Model
SCREEN
ELEMENT
screen element id #
data item id #
screen element descr.
INTERFACE
ELEMENT
interface element id #
data item id #
interface element descr.
INPUT
ELEMENT
input element id #
data item id #
input element descr.
OUTPUT
ELEMENT
output element id #
data item id #
output element descr.
MODEL
VIEW
model view element id #
data item id #
model view element des.
DEPENDENCY
dependency elem id #
data item id #
process id #
dependency description
CODE
code id #
data item id #
stored data item #
code location
INFORMATION
information id #
data item id #
information descr.
information request
PROCESS
process id #
data item id #
process description
USER TYPE
user type id #
data item id #
information id #
user type description
LOCATION
location id #
information id #
printout element id #
process id #
stored data items id #
user type id #
location description
PRINTOUT
ELEMENT
printout element id #
data item id #
printout element descr.
STORED DATA ITEM
stored data item id #
data item id #
location id #
stored data description
DATA ITEM
data item id #
data item description
55

Warehouse
Process
Warehouse
Opera-on
Transforma-on
XML
Record-‐
Oriented
Mul-
Dimensional
Rela-onal
Business
Informa-on
So?ware
Deployment
ObjectModel
(Core,
Behavioral,
Rela-onships,
Instance)
Warehouse
Management
Resources
Analysis
Object-‐
Oriented
(ObjectModel)
Foundation
OLAP
Data

Mining
Informa-on
Visualiza-on
Business
Nomenclature
Data
Types
Expressions
Keys
Index
Type
Mapping
Overview of CWM Metamodel
http://www.omg.org/technology/documents/modeling_spec_catalog.htm
56

Marco & Jennings's Complete Meta Data Model
Source:http://dmreview.com/article_sub.cfm?articleID=1000941 used with permission
57

technologies?
Tweeting now:
#dataed
58

technologies?
Tweeting now:
#dataed
59

Goals and Principles
1. To support and enable
effective business analysis
and decision making by
knowledgeable workers
2. To build and maintain the
environment/infrastructure to
support business intelligence
activities, specifically
leveraging all the other data
management functions to cost
effectively deliver consistent
integrated data for all BI
activities
60

• Understand BI information needs
• Define and maintain the DW/BI
architecture
• Process data for BI
• Implement data warehouse/data marts
• Implement BI tools and user interfaces
• Monitor and tune DW processes
• Monitor and tune BI activities and performance
61
Activities

Primary Deliverables
• DW/BI Architecture
• Data warehouses, marts,
cubes etc.
• Dashboards-scorecards
• Analytic applications
• Files extracts (for data mining, etc.)
• BI tools and user environments
• Data quality feedback mechanism/loop
62

Roles and Responsibilities
Suppliers:
• Executives/managers
• Subject Matter Experts
• Data governance council
• Information consumers
• Data producers
• Data architects/analysts
Participants:
• Executives/managers
• Data Stewards
• Subject Matter Experts
• Data Architects
• Data Analysts
• Application Architects
• Data Governance Council
• Data Providers
• Other BI Professionals
63

Technology
• ETL
• Change Management Tools
• Data Modeling Tools
• Data Profiling Tools
• Data Cleansing Tools
• Data Integration Tools
• Reference Data Management Applications
• Master Data Management Applications
• Process Modeling Tools
• Meta-data Repositories
• Business Process and Rule Engines
64

Guiding Principles
1. Obtain executive commitment and
support.
2. Secure business SMEs.
3. Be business focused and driven. Let
the business drive the prioritization.
4. Demonstrate data quality is essential.
5. Provide incremental value.
65
6. Transparency and self service.
7. One size does not fit all: Find the right tools and products for each of
your segments.
8. Think and architect globally, act and build locally.
9. Collaborate with and integrate all other data initiatives, especially
those for data governance, data quality and metadata.
10. Start with the end in mind.
11. Summarize and optimize last, not first.

6 Best Practices for Data Warehousing
66
1.Do some initial architecture
envisioning.
2.Model the details just in time (JIT).
3.Prove the architecture early.
4.Focus on usage.
5.Organize your work by requirements.
6.Active stakeholder participation.
http://www.agiledata.org/essays/dataWarehousingBestPractices.html

Polling Question #3
Do you have a separate
data warehouse
department, sub-
department, or group?
a) Yes
b)No
67

technologies?
Tweeting now:
#dataed
68

technologies?
Tweeting now:
#dataed
69

Summary: Data Warehousing & Business Intelligence Management
70

Series Take Aways
71
• Metadata
– Metadata unlocks the value of data, and therefore requires management
attention [Gartner 2011]
– Metadata is the language of data governance
– Metadata defines the essence of integration challenges
• Cloud
– Data governance, architecture, quality, development maturity are necessary but
insufficient prerequisites to successful data cloud implementation
– A variety of cloud options will influence cloud and data architectures in general
– You must understand your architecture and strategy in order to evaluate the
options
– Data must be reengineered to be: less; better quality; more shareable
– Failure to do these will result in more business value for the cloud vendors/
service providers and less for your organization
• Warehousing
– Business value must precede technical design

References
72

References
73

Additional References
• http://www.information-management.com/infodirect/20050909/1036703-1.html
• http://www.agiledata.org/essays/dataWarehousingBestPractices.html
• http://www.cio.com/article/150450/
Five_Key_Business_Intelligence_Trends_You_Need_to_Know?page=2&taxonomyId=3002
• http://www.computerworld.com/s/article/9228736/
Business_Intelligence_and_analytics_Conquering_Big_Data?taxonomyId=9
• http://www.enterpriseirregulars.com/5706/the-top-10-trends-for-2010-in-analytics-business-
intelligence-and-performance-management/
• http://www.itbusinessedge.com/cm/blogs/vizard/taking-the-analytics-pressure-off-the-data-
warehouse/?cs=50698
• http://www.informationweek.com/news/software/bi/240001922
74

Questions?
It’s your turn!
Use the chat feature or Twitter (#dataed) to submit
your questions to Peter now.
75
+ =

Upcoming Events
76
October Webinar:
SHOW ME THE MONEY: MONETIZING DATA MANAGEMENT
October 8, 2013 @ 2:00 PM – 3:30 PM ET
(11:00 AM-12:30 PM PT)
November Webinar:
UNLOCK BUSINESS VALUE THROUGH
REFERENCE & MDM
November 12, 2013 @ 2:00 PM – 3:30 PM ET
(11:00 AM-12:30 PM PT)
Sign up here:
• www.datablueprint.com/webinar-schedule
• www.Dataversity.net
Brought to you by:

Data-Ed: Data Systems Integration & Business Value Pt. 3: Warehousing

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (11)

Similar to Data-Ed: Data Systems Integration & Business Value Pt. 3: Warehousing

Similar to Data-Ed: Data Systems Integration & Business Value Pt. 3: Warehousing (20)

More from DATAVERSITY

More from DATAVERSITY (20)

Recently uploaded

Recently uploaded (20)

Data-Ed: Data Systems Integration & Business Value Pt. 3: Warehousing