All Together Now: A Recipe for Successful Data Governance

Eric.kavanagh@bloorgroup.com

Twitter Tag: #briefr 7/10/12

!   Reveal the essential characteristics of enterprise
software, good and bad

!   Provide a forum for detailed analysis of today s
innovative technologies

!   Give vendors a chance to explain their product to
savvy analysts

!   Allow audience members to pose serious questions...
and get answers!

Twitter Tag: #briefr

!   July: Disruption
!   August: Analytics
!   September: Integration
!   October: Database
!   November: Cloud
!   December: Innovators


!  Disruptive Innovation produces an unexpected new market
and value network, and is usually geared toward a new set
of customers.

!  The consumer technology market teems with such game-
changers: mp3 players, iPhone/iPads, portable storage
devices, digital media, etc.

!  While disruptive technologies often take a degree of time to
obtain a foothold in the market, they can have a serious
impact on industry incumbents, who can be slow to
innovate.


David Loshin, president of Knowledge Integrity,
Inc, is a recognized thought leader and expert
consultant in the areas of data quality, master
data management and business intelligence.
David is a prolific author regarding business
intelligence best practices and has written
numerous books and papers on data
management, including the just-published
“Practitioner’s Guide to Data Quality
Improvement.” David is a frequent invited
speaker at conferences, web seminars, and
sponsored web sites and channels including
www.b-eye-network.com. His best-selling
book, “Master Data Management,” has been
endorsed by data management industry
leaders, and his valuable MDM insights can be
reviewed at www.mdmbook.com.
David can be reached at:
loshin@knowledge-integrity.com or
(301) 754-6350.


!   Focuses on agility and flexibility for data governance
and standards

!   Offers a core technology suite, DataStar, that
delivers data modeling, integration, aggregation and
automation.

!   Developed a NoSQL alternative for data consolidation


Dr. Geoffrey Malafsky earned a Ph.D. in
Nanotechnology from Pennsylvania State
University. He was a research scientist at the
Naval Research Laboratory before becoming a
technology consultant in advanced system
capabilities for numerous Government
agencies and corporate clients. He has over
thirty years of experience and is an expert in
multiple fields including Nanotechnology,
Knowledge Discovery and Dissemination, and
Information Engineering. He founded and
operated the technology consulting company
TECHi2 prior to founding Phasic Systems Inc.,
where he is the CEO and CTO.


Bringing Agility and Flexibility to
Data Design and Integration
Phasic Systems Inc
Delivering Agile Data

www.phasicsystemsinc.com

10

Introduction to Phasic Systems Inc

•  Bringing Agile capabilities to data lifecycle for business success
•  Methods and tools tested and refined over years of in-depth large-
scale efforts
•  Solve toughest data problems where traditional methods fail
•  Based on extensive consulting lessons learned and real-world
results
•  Began in 2005 to commercialize advanced Agile methods
successfully deployed in competitive development contracts

11

Phasic Systems Inc Management

•  Geoffrey Malafsky, Ph.D, Founder and CEO
▫  Research scientist
▫  Supported many organizations in their quest to access the right
information at the right time
•  Tim Traverso, Sr VP Federal
▫  Technical Director, Navy Deputy CIO
•  Marshall Maglothin, Sr VP HealthCare
▫  Sr. Executive multiple large health care systems
•  Deborah Malafsky Sr VP Business Development

12

Our Agile Methods
•  Why be Agile?
▫  Provide flexibility and adaptability to changing business needs while
maintaining accuracy and commonality
▫  Segmented approach is too slow, rigid, and costly
•  How?
▫  Treat data lifecycle as one continuous operation from governance to
modeling to integration to warehouses to Business Intelligence
▫  Emphasize value produced at each step and overall coordination
▫  Seamlessly fit with existing organization, procedures, tools but add Agility,
commonality, flexibility, and reduced cost and time
•  We are Agile and comprehensive
▫  Typical 60-90 day engagement
▫  Deliver completed products not just plans or partial results

13

Methods and Tools
•  DataStar Discovery: Agile data governance, standards and design
▫  Add business and security context to data
▫  Flexible, common data definitions/ semantics, models

•  DataStar Unifier: Agile warehousing and aggregation
▫  Simplified, common semantics using Corporate NoSQL™
▫  Source to target mapping with flexibility, standardization
▫  Aggregate data using all use case and system variations simply and
easily into standard or NoSQL databases

14

PSI Customer Testimonial
“As a COO of a Wall Street firm and a former Vice Admiral in the United
States Navy in charge of a large integrated organization of thousands of people
and numerous IT systems, I have seen firsthand the critical role that high-quality
enterprise data plays in day-to-day operations of an organization. Without
timely access to reliable and trusted data all of our operations were vulnerable
to poor decision making, weak performance, and a failure to compete. With
Phasic Systems Inc.’s agile methodology and technology, we were finally able to
solve our data challenges at a fraction of the time, cost, and organizational
turmoil that all the previous and more expensive, time-consuming approaches
failed to do. Phasic Systems Inc. offers a new and much-needed approach to
this important area of Business Intelligence.”

VADM (ret) J. “Kevin” Moran

15

The Business Case
Today’s Response Timeline (15 to 27 Months)
3 to 6 Months 6 to 9 Months 3 to 6 Months 3 to 6 Months

Business Groups IT Groups BI Groups Users
•  Requirements •  Develop Systems & Applications •  Capability Problems
•  BI Data Models
•  Conceptual/Logical Models •  Physical Data Models •  New Capabilities
•  Reports
•  Data Quality •  Databases / Data Warehouse •  Missing Data
•  Dashboards
•  Business Rules •  ETL controls
•  Standards •  MDM

Tomorrow’s Initial Response Timeline with PSI (Subsequent Response Timeline – Days)
2 to 6 Months
•  Requirements •  Develop Systems & Applications
•  Conceptual Data Model •  Physical Data Models
•  Logical Data Model •  Databases / Data Warehouse
•  Business Rules •  ETL controls
•  Standards •  MDM
•  BI Data Models
•  Data Quality

16

Agile: Overcome Hurdles
•  Group rivalry
▫  Embrace important business variations; recognize no valid reason
to force everyone to use only one view exclusively.
•  Terminology confusion
▫  Use a guided framework of well-known concepts to rapidly identify,
and implement variations as related entities.
•  Poor knowledge sharing
▫  Use integrated metadata where important products (business
models, data models, glossaries, code lists, and integration rules)
are visible, coordinated, and referenceable
•  Inflexible designs
▫  Use a hybrid approach (Corporate NoSQL™) for Agile
warehousing and integration blending traditional tables and
NoSQL for its immense flexibility and inherent speed

Schema Are Not Enough
Governance Integration CEO/CFO/CIO SAP/IBM/ORACLE
Design ? MDM Sales, ?
Accounting

D. Loshin 2008

Which Value? Whose? My “customer” or your “customer”?

How is data used?

Must be agile in order to adapt quickly to new business needs
▫  Continuous change is norm: requirements, consolidation
▫  We must use all the important business variations of key terms (e.g.
account, client, policy) – No such thing as single version for all!

18

Status Quo: Non-Agile Agile: Visible, Common

19

Unified Business Model™ Intuitive, List-based

20

Real Estate Listing Example

•  Seems simple and well-defined
▫  Each house has a type, id, address, etc..
▫  Industry standards: OSCRE, RETS
•  Yet, data systems are very different
▫  Data model tied tightly to business workflow
▫  Extensions and “make-it-work” changes added over time
•  Similar to customer relationship mgmt, ERP, and many
other fields

21

Semantic Conflict in
Real Estate Models NKY

HOMESEEKERS

NKY attribute ‘basement’
does not have a corollary in
HOMESEEKERS

22
Data Value Semantic
Errors = Inconsistent, Lot_dimensions: implied semantics for size
Difficult to Merge, data. Actually has all sorts of data

Report, Analyze
Semiannual_taxes: implied semantics for
numeric data. Actually has all sorts of data

25

Fully Integrated Metadata for Business, IT, and BI

28

DataStar Corporate NoSQL™
•  Large systems use NoSQL for its flexibility, performance,
and adaptability
▫  But, it is poorly suited for corporate use – lacks connection to
business
•  DataStar Corporate NoSQLTM
▫  Blends traditional techniques and NoSQL Speed
▫  Entities come directly from Unified Business Model &
Agility
▫  Object structure with simple tables
▫  Key-value pairs are basic repeating structure of all tables
▫  Business driven terminology
▫  Easily handles semantic variations & updates w/o changes to
logical or physical models
▫  Can be as ‘dimensional’ or ‘normalized’ as desired

Results
•  Applied to production data:
▫  Fully cleaned & integrated data governance approved
–  Requirement: 500,000 records in 2 hrs on Sun E25K
–  Actual: 50 minutes on 3 year low-cost server
•  Governance documents produced and approved
▫  Legacy data models – first time in ten years
▫  Common data model – directly derived from ontology.
Position-Resume model
•  Standing governance board created with short decision-
making monthly meetings
▫  Position-Resume Governance Board
•  Process approach and technology applied to new IT
systems

Navy HR Data Analysis
•  Groups “share” data and control only if they don’t lose
project control or funds
•  Governance, business process, data engineers create
separate designs and don’t know how to coordinate
•  Try hard to follow industry guidance but stuck
•  Actual data is very different than policy, mgmt awareness
▫  Example 1: Multiple Rate/Rating entries. Person xxxxxx has 5
entries: 4 end on the same date, 2 have start dates after they
their end dates , 2 start and end on the same days but are
different
▫  Example 2: 30 different values used for RACE but only 6
allowed values in the Navy Military Personnel Manual derived
from DoD policy

33

Agile Warehousing and BI
v

35

Key-Value Vocabulary Resume Identifiers

36

Key-Value Vocabulary Competency KSAs

Agility and Collaboration for Data Governance

David Loshin
Knowledge Integrity, Inc.
www.knowledge-integrity.com

© 2012 Knowledge Integrity, Inc. 38
www.knowledge-integrity.com (301)
754-6350

Business Metadata Interdependencies

Context

Concept Process

Business
Policy

754-6350

Objective: Translate Business Policies into Data Rules

Business Business Information Business Data
Metadata
Goals Policy Policy Rules Rules

Operational governance integrates monitoring
conformance to data rules

754-6350

754-6350

Motivation: Complexity in Data Meanings & Semantics

p  What is a customer?
Sales: Support:
Someone Someone who
who pays for has a license for
our products use of our
or services product

Customer
Service
p  These are potentially Finance
Human
Resources
conflicting definitions
“customer”
p  Representations and
underlying meanings Sales Legal
from different business
functions may differ
Marketing ? Compliance

754-6350

Build from the Bottom Up

Data Governance
Information Information Data Quality
Access Control
Usage Quality SLAs

Information Architecture
Domain
Entity Models Relational Tables
Directory
Data Elements
Critical Data Element
Data Formats Aliases/Synonyms
Data Elements Definitions

Reference Metadata
Conceptual Value Reference
Mappings
Domains Domains Tables

Business Definitions
Business
Concepts Definitions Semantics
Terms

754-6350

Business Terms
p  Within different contexts, business terms may be used with a
specific definition to refer to:
n  An action
n  An entity
n  A characteristic
p  A business term may be used multiple times with different
definitions

754-6350

Example – Identifying Business Terms
p  Order Confirmation
If you do not receive a confirmation
number (in the form of a confirmation page
or email) after submitting payment
information, or if you experience an error
message or service interruption after
submitting payment information, it is your
responsibility to confirm with FizzDizzle
Customer Service whether or not your
order has been placed.

754-6350

If you do not receive a confirmation Nouns
number (in the form of a confirmation •  You
page or email) after submitting payment
•  Confirmation number
•  Confirmation page
submitting payment information, it is your •  Confirmation email
responsibility to confirm with FizzDizzle •  Payment information
Customer Service whether or not your •  Error message
order has been placed. •  Service interruption
•  FizzDizzle Customer Service
•  Order

754-6350

754-6350

If you do not receive a confirmation Verbs
number (in the form of a confirmation page •  Receive
or email) after submitting payment
•  Submitting
•  Experience
submitting payment information, it is your •  Confirm
responsibility to confirm with FizzDizzle •  Placed
Customer Service whether or not your
order has been placed.

754-6350

Bring it All Together: The Chain of Definition

754-6350

Harmonization
Data Type
Element p  Use Chain of Definition
Data Type First VARCHAR(25) to determine when:
Element Middle VARCHAR(25) n  Similarly-named data
FirstName VARCHAR(35)
Last VARCHAR(30) elements refer to the
LastName VARCHAR(40) same data element
SocialSec CHAR(9)
SSN CHAR(11)
concept
Telephone VARCHAR(20) n  Same-named data
elements refer to
different data element
concepts
n  Consolidating when
possible and
n  Differentiating when
necessary

754-6350

Impact Assessment
Data Type
p  Use chain of definition Element
model to identify the Data Type First VARCHAR(25)
instances that are Element Middle VARCHAR(25)
impacted as a result of FirstName VARCHAR(35)
Last VARCHAR(30)
harmonization LastName VARCHAR(40)
SocialSec CHAR(9)
SSN CHAR(11)

Telephone VARCHAR(20)

754-6350

Questions and Open Discussion
p  www.knowledge-integrity.com

www.dataqualitybook.com

p  If you have questions, comments,
or suggestions, please contact me
David Loshin
301-754-6350
loshin@knowledge-integrity.com www.mdmbook.com

© 2011 Knowledge Integrity, Inc.
2012 60
754-6350

!   One of the common themes in the material you provided is the
need for collaboration as part of the lifecycle management for
the creation of a unified business model. To what extent is this
collaboration driven by the software and how much requires
processes designed around the software?

!   What is your approach for transferring the knowledge for
identifying semantic conflicts and resolving them within the
organization?

!   A lot of the slides suggest that the intent of the use of the
technology is for developing data warehouse or business
intelligence models. Is the use limited to consuming data from
existing systems, or can it be used for reengineering operational
or transaction systems, and if so how, and if not, why?


!   One of the barriers to value for existing metadata and
governance tools is the need for ongoing maintenance of the
content. How can the product be used to facilitate ongoing
management and assurance of consistency of business
terminology?

!   Presuming that I am now a data consumer (say a business analyst)
within the organization, how would I use this technology to
clarify the definitions and lineage of business terms presented to
me in a BI report?


!   What is your approach for capturing the semantics of implicit
business concepts? In your real estate example, one of the
columns for lot dimensions had implied semantics for size data,
with an implication of measurement systems, units of measure,
and even “topography” of the lot size. This implies the use of
business concepts that are not explicit (acreage vs. square
footage, transformations across frames of reference,
qualification of lot shape, presentation of dimensionality). How
does the tool capture implicit semantic information?

!   Going back to collaboration: What types of interactive
notifications are integrated into your environment to apprise
individuals of changes to business terms, data element concepts,
data elements, value domains, etc.?


All Together Now: A Recipe for Successful Data Governance

More Related Content

What's hot

Similar to All Together Now: A Recipe for Successful Data Governance

More from Inside Analysis

Recently uploaded

All Together Now: A Recipe for Successful Data Governance