Data Architecture for Data Governance

IASA is

a non-profit professional association
 run by architects
 for all IT architects
 centrally governed and locally run
 technology and vendor agnostic

Th

Information Architecture

 Module 0: Course Intro, Architecture  Module 3: Data Integration
Fundamentals Introduction: Data,  Lesson 1: Integrating at the Company Level
Information and Knowledge  Lesson 2: Data Characteristics
 Lesson 3: Data Integration at the System level
 Module 1: Information for Business
 Lesson 1 – Information as Strategy  Module 4: Data Quality and Governance
 Lesson 1 – Data and Information Quality
 Lesson 2 – Relating Information to Value
 Lesson 2: Data Compliance
 Lesson 3 – Information Scope and  Lesson 3 – Data and Information Governance
Governance
 Module 5: Advanced Information Management
 Module 2: Information Usage  Lesson 1: Data Warehousing
 Lesson 1 – Who Uses Your Information  Lesson 2: Business Intelligence
 Lesson 2 – How, When, Where and Why  Lesson 3: Data Security and Privacy
is Information Used  Lesson 4: Metadata and Taxonomy Management
 Lesson 3 – Form Factors  Lesson 5: Knowledge Management
 Lesson 4 – Usage Design 1
 Module 6: Architecture throughout the Lifecycle
 Lesson 5 – Quality Attributes for
Information Architecture
 Lesson 6 – Data Tools and Frameworks

Governance is Not a Popular Topic

Governance is Overcomplicated
 It is not about politics
 Itis not about delivering
verdicts
 Itis not about defining
solutions

Step 4: Create Governance Body

Step 6: Define Charter

•Charter

Iasa Architect Services

 Adopt Iasa Standards – Skills taxonomy, role descriptions,
compensation models
 Set your goals for value
 Assess your current team – gap analysis
 Implement processes
 Develop and educate people
 Certify employees and vendors
 Build communities

•December 6th – 7th, 2012 // Austin, TX
•Training: Dec. 3rd – 5th, 2012
•Certification: Dec. 7th – 9th, 2012
•

•www.iasaworldsummit.org

•Leading Innovation in Architecture. Stay ahead of the technology curve and
shape the future of architecture in your organization.

•Connect and share insights with the largest global network of
technology and enterprise architect practitioners.

•Attend sessions on the most current breakthroughs, case studies and
key topics in architecture - presented by a mix of practicing architects
from top performing businesses and organizations.

•Participate in pre-conference workshops and training. Maximize your time
at the summit by taking part in one of six intensive training courses designed
to provide immediate solutions to benefit your everyday practice.

•Featuring Industry Leading Keynote Speakers:

•Sheila Jeffrey •David Del Giudice •Paul Preiss
Senior Information Architect Principal Architect President, Founder
Bank of America AstraZeneca Iasa Global

•David Manning •Scott Whitmire •Cat Susch
IT Enterprise Architect Enterprise Architect Enterprise Architect
Idaho Power Company Nordstrom Microsoft

Data Architecture for Data
Governance

November 8, 2012

Presented by Malcolm Chisholm Ph.D.
mchisholm@refdataportal.com
Telephone 732-687-9283
www.refdataportal.com
www.bizrulesengine.com

© AskGet.com Inc., 2012. All rights reserved

Agenda
• Introducing Enterprise Architecture (EA)

• Introducing Data Governance

• Data Architecture and Data Governance

• Data Architecture Patterns

• Data Architecture, Governance, and The Business


Introducing Enterprise Architecture
(EA)


Need for Enterprise Architecture (EA)
Myriad of Independent Projects in IT Strategy is Set at High Level
Here is where we
want to be in 5
years…

CxO’s

Operationalization

Enterprise …Others…
Architecture

• EA is responsible for ensuring overall business strategy is implemented
by IT

The IT Mindset
That’s not a clear We have a high-level
requirement… strategy…
Tell me what you want me
to build
Then I will design it
Then I will build it
Then I will turn it over to you
Then I will walk away

IT CxO

• IT people just want to build stuff and hand it off to other people
• They want business sponsors who will pay for this stuff and tell them
exactly what is needed
• Strategy does not match this mindset


Design without Architecture

Stairway to Ceiling

All Design – No Architecture
Rooms: 160; Doorways: 467; Doors: 950; Fireplaces: 47 (gas, wood,
coal); Bedrooms: 40
“Circular” Stairway
Constructed 1884 – 1922 (38 continuous years); Cost: $5.5M
Blueprints: Never made; Individual rooms sketched out by Sarah “…staggering amount of creativity, energy, and
Winchester on paper or other media (e.g. tablecloths) expense poured into each and every detail”
http://www.winchestermysteryhouse.com

• The Winchester Mystery House
• Lots of design – but no architecture
• There is a difference between architecture and design


Design vs. Architecture
Changed
Source Integration Data
Mirror Capture
Data
Flow

DDL

• Architecture deals with many instances of a component type that
must interact
• Design deals with one instance of a component type, without regard
to interaction
• E.g. Perspective of Databases : Data Environment (BI or Integration
Environment in this example)


Basic EA Taxonomy (“BAIT”)

Enterprise Architecture

Business Technology
Architecture Architecture

Application Information
Architecture Architecture

• EA is clearly not a monolithic discipline
• It looks more like a collection of dissimilar components
• Each component is a competency – so this matters
• Yet there is no common agreement

Introducing Data Governance


What is Data Governance? Starter Definitions
From the Data Governance Institute

”The exercise of decision-making and authority for data-related
matters.”

”A system of decision rights and accountabilities for information-
related processes, executed according to agreed-upon models which
describe who can take what actions with what information, and when,
under what circumstances, using what methods.”

www.datagovernance.com


Data Governance is about Processes
Processes for…

Deciding who has what rights regarding data
Making decisions about data
Implementing decisions about data

In other Words…

Identifying the processes needed to manage data
Identifying the actors in these processes
Designing these processes
And designing the processes to design these processes


Data Governance – A Caution
Data Governance is the design, building, and operation of formal
business processes to manage the enterprise data resource.

W R ON G
The Business has….
Business Processes
IT has…
Governance

• The decision rights of actors are defined within the context of these
business processes
• There is no real distinction between the processes of “data governance”
and the “business processes” of the business . [Compare HR].
• A distinction that identifies the set of business processes that manage the
enterprise data resource as “data governance” is valid. [Compare HR].
• Use of the term data governance to further the pretence that IT is
universe
totally outside of the enterprise is invalid.
• Therefore, the processes of data governance are truly business
processes

Data Architecture and Data
Governance


Architectural Levels
Reference
Architecture
Assimilate best Conceptual
architectural Architecture
practices from the
IT industry. All Match/select Logical
relevant component architectural Architecture
classes and component classes
patterns are and patterns to the Detailed design of Physical
understood. business needs, future state Architecture
(including future architecture.
plans). Influencing actual
physical
implementations to
ensure
conformance with
logical architecture

A reference architecture is the highest level of architecture. It
can be undertaken prior to even thinking about the enterprise.
Its advantages are that it can be very high level and can cover
the entire enterprise. Useful for thinking about high level data
layers


Reference Data Architecture - Example

• Example of a simple reference data architecture
• Just data layers – no services or components
BUT – just having a picture can be problematic

Need Definitions, Explanations – Not Just Picture

Layer
Characteristic Transactional
Data Warehouse Data Mart
Application
Data structured and
View of data across the
Data produced via the filtered to support
enterprise. Supports
Purpose automation of business
dissemination, derivation
specific information
processes needs of small groups of
of knowledge and history
users.
Derivations (including
Data from Warehouse is
All base (non-derived) aggregations) produced
Data Life Cycle data originates here here, and history is
transformed to support
specific reporting
inferred
Extract / Transform /
Create / Source / Read / Subscribe / Transform /
Data Operations Update / Delete / Archive
Load / Derive / Publish /
Archive
Archive

Subject Oriented / Information Requirement
Data Model Normalized to 3NF Snowflaked / Conformed Oriented / Snowflaked /
Dimensions Conformed Dimensions

• Much more is needed than the above
• Definitions are a technical reference; explanations help stakeholders to
understand the reference architecture
© AskGet.com Inc., 2011. All rights
reserved

Subject Area Model

• A Subject Area Model is a taxonomy of the major areas in the enterprise
that are relevant to data management
• A necessary starting point for MDM
• 1 – A subject area should have stable definitions of entities
• 2 – Production of master data should not cross subject area boundaries
• Identify the major data concepts in each subject area – may will be
master data entities.
© Askget.com Inc., 2011. All rights reserved

A Subject Area Model is Not Costly

Data Management

• 10-15 subject areas per model is the norm
• Do not need elaborate data modeling tools
• Much more about a standardized view of the enterprise
• Can be done with 1-3 people
• Gets a lot of bang for the buck
• Is high-level and important


Example of Use: Establish Vendor Data Profiles
Map Vendor’s Data Vendor
Data to Subject 1
Areas

Data Vendor
Data Management 2

Create & Data Vendor
Map Vendors
Manage Subject to Uses
3
Area Model(s)

Data Vendor
4

Map Subject
Areas to User Data Vendor
Requirements 5

Subject Area Model(s) Users

• A Subject Area Model provides the basis for a common understanding of
the data vendors provide
• Will need to go to major entity clusters within Subject Area

A Subject Area Model is a Taxonomy
I don’t like your
definition of
this subject
Your model area!
does not tell
me what is
financial data!

You mean to tell me
every one of these
subject areas
includes access
security for data!

• People think a taxonomy is a way of unlocking the secrets of the universe
• Taxonomies are a tools – you can have as many as you need to do your
job
• You can never expect the boundaries of a subject area model to be
exactly precise and never changing


Create a Metadata Subject Area Model

The Fundamental Ontology of Metadata for the Enterprise

Metadata Subject Area Model
Advantages
1. Standardizes the metadata in the enterprise at a high level.

2. Can be used to prioritize projects and programmes.

3. Useful in communications.
4. Shows what metadata is being produced by subject area. This is a high-level
inventory.
5. Can distinguish data-related metadata from other data
6. Within each subject area, definitions should be constant. There may be
variations across subject areas.
7. Subject areas are candidates for conceptual models.

Disadvantages
1. It is a taxonomy and can be argued over. No one taxonomy will satisfy every
perspective. We use data categorization for that (see later). The Subject Area
Model should be the most common and natural perspective.
2. The Subject Area Model often has more things expected of it than it can
deliver. Managing expectations can be tricky.

The Subject Area Model is itself Metadata and needs to be managed

Data Architecture Patterns


What is an Architectural Pattern?
“Pattern in architecture is the idea of • An architectural style is a central,
capturing architectural design ideas organizing concept for a system.
as archetypal and reusable • An architectural pattern describes a
descriptions” coarse-grained solution at the level of
- Christopher Alexander, quoted in Business subsystems or modules and their
Model Generation by Osterwalder and relationships.
Pigneur (2010)
• A system metaphor is more conceptual
and it relates more to a real-world
Antipattern: A pattern that is concept over a software engineering
commonly used, but which is concept.
ineffective or damaging. E.g. - Quoted from A Practical Guide to Enterprise
Analysis Paralysis. Architecture by McGovern, Ambler et al, at
http://www.infoq.com/news/2009/02/Architectural-
- Term created by Andrew Koenig, 1995
Styles-Patterns

• Patterns occur a lot in architecture
• There are also styles, metaphors and antipatterns
• Seem to have emerged from the application architecture
competency (e.g. rare to hear of dimensional modeling as a pattern
or style)


Example: Farm and Market Pattern

Traditional MDM Integration Hub

Farm and Market Pattern for MDM


Six Patterns for Data Hubs: 1 – Publish/Subscribe

• Publisher pushes data to hub
• Subscriber pulls data from hub
• No data integration
• Publisher may not know who the subscribers are
• Weak governance - may not even be SLA’s


Six Patterns for Data Hubs: 2 – ODS for Integrated Reporting

• Supports principle that transaction applications will not do
operational reporting (adverse affect on performance)
• Some form of integration
• Anti-pattern of real-time data warehousing
• History (changed data capture) not considered


Six Patterns for Data Hubs: 3 – ODS for Data Warehouses

• No reporting in this kind of ODS
• Probably evolved from ODS for reporting
• Strong integration
• Just to feed warehouses
• At odds with Data Warehouse that has own staging, integration


Six Patterns for Data Hubs: 4 – Traditional MDM Hub

• Master Data only
• Integration does happen (typically just Trust & Survivorship)
• Some content management needed – typically to correct data
quality problems
• Typically DQ functionality an afterthought
• Distributes “Golden Copy” Master Data to enterprise


Six Patterns for Data Hubs: 5 – Message Hub

• Messages – real-time or near-real-time - not batch data
• Message switch
• Command and control for message orchestration
• Not just “listening” – does routing
• Messages also stored in database


Six Patterns for Data Hubs: 6 – Integration Hub

• Like message hub, but for batch data
• Event data equivalent of MDM hub
• Just one place where integration done
• Both warehouse and transaction applications need integrated data
• Removes need for each application to do integration by itself


Data Architecture, Governance, and
The Business


Another View of EA
Many Years Ago… Today…
…”the business” knew the business …it does so much less

• Long ago, people had to know the business rules concerning what they
did
• These are now in applications, and staff are more concerned with
knowing how to make the applications work
• Data is an asset, but staff tend to know little about the data
• Loss of business knowledge

Who Knows The Business?

• The business seems to be more fragmented than long ago (though that is
difficult to prove)
• Managers who know the business know less of it and are so time-
constrained they cannot help much on aligning projects to enterprise
strategy

Populate EA with Business Experts
A Different EA Vision: IT Tendency:
Architects + Business Experts Populate EA with Architects

• Business experts provide better relations with rest of business
• Keep architects with just being oriented to architecture
• Immediate business credibility for EA
• They get poached


Questions and Answers
Data Architecture for Data
Governance

November 8, 2012

Presented by Malcolm Chisholm Ph.D.
Telephone 732-687-9283
mchisholm@MasterDataMgmt.com


Data Architecture for Data Governance