The ABC of Data Governance: driving Information Excellence

The ABC of Data Governance
and its role in driving Information Management Excellence

Alan D. Duncan February 2014

Uncontrolled when printed

Alan Duncan, Director of Data Governance, UNSW
E: Alan.Duncan@unsw.edu.au Tw: @Alan_D_Duncan LinkedIn: http://www.linkedin.com/in/alandduncan

A bit about me....
•
•

21 years Information Management & Business
Consulting
– EDS, KPMG, CPW, Acuma, Pelion, SMS
– Scottish Power, United Distillers, O2, Astra Zeneca,
Carphone Warehouse, Vodafone, Riyad Bank
– Commonwealth Bank, NSW Roads & Maritime
Services, Centrelink, OATSIH, NSW Family &
Community Services, CASA, AMSA, FaHCSIA, DAFF,
Navy…

•
•

Information-Management.com “Top 12 on Twitter”
Best supporting Actor, 2005 Barnet Drama Festival



…and a bit about UNSW.



Agenda
• Why we need to rethink our Information
Management Strategies
• Techniques and capabilities we need to focus on
• What data should be stored, and how long should
we keep it?
• Information Management Excellence



“The value of an idea
lies in the using of it.”
Why we need to rethink our information
Management Strategies,
Sponsored by Thomas Edison



“A” is for “Action”
Identify measurable and targeted Business Outcomes
Why do we need information? For whom? What will we do
differently?
Confirm the Information Holdings & Gaps
What do we need to provide? (Content + Context)
Establish DG Operating Model
Who is accountable? By what
processes?

Implement DG/IMCC Services
Catalogue:

“When it is obvious that the What core capabilities do we need?
Execute Activities & Tasks
goals cannot be reached,
How do we deliver? Who does the
don't adjust the goals,
work?
adjust the action steps.”


“A” is for “Access”
•
•
•
•
•
•

Stimulus to improve data quality
Consistency of data definitions
Openness and trust
Transparency & accountability
Opportunity value
Proactive publication and Open
Data vs. “Need to know”

“Publish and be damned!”
http://www.ted.com/talks/tim_berners_lee_the_year_open_data_went_worldwide.html

7


“A” is for Analytics
•
•
•
•
•

“Traditional” BI (reporting & ad-hoc analysis)
Data Mining
Statistical modelling
Data visualisation
Textual analytics

• What questions do we want to answer?
• What questions can we answer with the data
we’ve got?
• What other data would we need?
• What does the data tell us we should be
asking?

“The alchemists in their
search for gold discovered
many other things of
greater value.”



8

Summary: Information Use Cases
•
•
•
•

Based on our current understanding of business needs, the following classes of
Information Use Case are identified
Detailed Requirements Analysis should be conducted on a project-by-project basis
to explore any detailed Use Cases within each class
Not all detailed Use Cases need to be defined ahead of time
Solutions should be flexible to accommodate new and changing Use Cases

Structured
data
reporting

Strategic
Intelligence
and Data
Mining

Publish
content to a
community

Executive
briefings

Education,
Training,
Learning

Search for
content
previously
created

Records
Management,
Compliance &
Audit

GIPA &
Privacy
Responses

Ability to publish
Filtering/screening/validation of what gets published
Feedback loop, measure of usefulness & continuous improvement
Shared understanding (IT & Business)



“To succeed in life,
you need two things:
ignorance and
confidence.”

Techniques and capabilities,
sponsored by Mark Twain



“A” is for “Asset”
Governance

Owners
Owners
Owners

Information
Asset

User
Community

Steward

Experts

Admin

Asset
Management
Tools

The “Information Asset Community”
Audit Data

Symbion
Health
System

Individual Flight Data

Example High Level Data
Systems & Flows

Human Resources – Time

AOD Case
Mgmt
System

Timelog

Medical

FTNS

Organisational
Flight Data

Version 1.0

Human Resources – Time

FTTO

Alcohol and Other Drugs

AOD Audit
Contacts – Ind, Org’s

Service Log

Contacts – Ind, Org’s

APEX
Workflow

Dangerous
Goods

Retain
PAWS

HRMS

Work Orders

API Upload
System

Dangerous Goods

DTAR/OTAR

Permissions / Change
of Status

MAAT

AOC
Surveys
ATOG Job
Register

Airports

Web Control
Mgmt
System
(WCMS)

FRLI

ADMS

Surveys/
Certifications

Human Resources - Travel

Events/Occurrences,
Aircraft, Aerodromes

Tracker

ASIR

Alternative
Means of
Compliance
(AMOC)

AMOC / Exemptions
Aircraft

AFD

Human
Resources – Flex
Time

ChangePoint
Testing
System

Aerodromes

Service Delivery

Surveillance/
Audit/
Reporting/
Tracking

Payroll (Salary)

TRIM

Aircraft

Finance

Human Resources

Alternative Means of Compliance
(AMOC)

Surveillance/Audit/
Reporting/Tracking

Service Delivery, HR & Finance, Agreements, Permissions, Aerodromes,
Participants, Aircraft

Events/Occurrences, Aircraft,
Aerodromes

AWS

workflow / service delivery

WMS

AIRS
workflow / service
delivery

STI

Medical Exams

Exemptions
Database
Payments

ESIR
Surveys /
Surveillance

Service Delivery

Directives

Incidents

Surveillance /Audits/
Reporting/ Tracking

eRooms

Superannuation Contributions

“Science is organized
knowledge. Wisdom is
organized life.”

Exemptions

Surveys / Surveillance

SM7

Details of
Operators

Comweb

CASA
Internet

MRS

Financial Actuals

Individuals/ARNS
Compensation Payments

CBMS
(DoFD)

AD

Enterprise Data
Warehouse

ASSP

Workflow and Online Collaboration/
Service Delivery

COMCARE

Content

Permissions / Change of
Status

Surveillance /Audits

Landings/ Take Offs

HRFlex

Surveys

Industry Payments
Cash Payments / Organisation Info

Financial Actuals

Calumo

AUSSAR
AIRS
Interchange

FMIS
Reconcile
Invoice against
Flown Hours
Employee
Expenses /
Adjustment
Journals

Search and Rescue

ATSL
Examinations
Cash Payments

ProMaster

Medical
Examinations

ATO
Business
Portal

Business Process
Medical Examinations

Remittance
Advice

Mandatory

AME

Examinations

Cyber Exams
External
Superannuation
Companies

FCAT

PAYG
payments, Salary
payments, and
Superannuation
payments.

Core
Suppliers
Corporate Support

CLIC

Inventory
Cash Payments

External

Bank Data
File

Data
Salary Payments

Mandatory

Inventory
Mgmt
System (IMS)

Defects/Events/Occurrences, Aircraft,
Aerodromes

Trending

Core

SDR
Physical Inventory

Human Resources

Aircraft

300+ Access
Databases

Contacts

ComBIZ
Online

Cash Receipts

Corporate Support

Publishing
System

HR - Time

External

Airspace

Thomas
Logistics

Applications / Permissions

General

System Interfaces map
EPK

Permissions

Aircraft Equipment Finances

Organisational

GMEL

External System

Baseline/Minimal Equipment

MMEL

Automatic One-way Relationship
Automatic Two-way Relationship
Manual One-way Relationship
Manual Two-way Relationship


Info Asset Register
(inventory)


“B” is for “Benchmark”
STRAT002

STRAT005

STRAT007

STRAT015

CUL002

CUL003

ORG002

ORG006

ORG003

ORG007

ORG008
ORG009
ORG010

POL005

POL001

POL003

POL002

POL004

PROC011

PROC014

PROC003

PROC002

PROC001

PROC004
PROC007

PROC005

PROC008

PROC009

PROC010

“One day Alice came to a fork in the
road and saw a Cheshire cat in a
tree. Which road do I take? she
asked. Where do you want to go?
was his response. I don't know,
Alice answered. Then, said the cat, it
doesn't matter.”


PROC013

SYS004

SYS001
SYS006
SYS007

SYS008

SYS002

SYS003

SYS009

QW001

SYS010

QW007

QW002

QW004
QW003

QW005
QW006


“B” is for “Big Data”
•
•
•

•

Three, four, five, six “Vees”?!
A lot of data (Tb/day)
Streaming data (monitoring, flow-of-control and
alerting analytics)
Inference from semi-structured data (Twitter,
Facebook)
Synthesise insight from millions of pages of text
Programmatic analysis for specific scenarios (hard in
SQL)
A disruptive catalyst to put information at the top of
the organisational agenda
Not just about the data! Business scenarios are key

•

Beware the Vendors!

•
•
•
•



“C” is for “Cloud”
In principle, it’s just another place to store data….

… but security, privacy considerations are heightened.

14


“C” is for compliance
•
•

•
•
•
•
•
•
•

“All I want is compliance with
my wishes, after reasonable
discussion.”

15

•
•

Freedom of Information Act 1982 (Cth)
Freedom of Information Amendment (Reform) Act 2010
(Cth)
Privacy Act 1988 (Cth)
Privacy Amendment (Private Sector) Act 2000
Privacy Amendment Act 2012 (Cth)
Privacy Amendments (Privacy Alerts) Bill 2013 (Cth)
State Records Act 1998 (NSW)
Government Information (Public Access) Act 2009 (NSW)
Privacy & Personal Information Protection Act 1998
(NSW)
Health Records & Information Privacy Act 2002 (NSW)
NSW Government Guide To Labelling Sensitive
Information 2011 (NSW Financial & Services)

But is “compliance” a motivator?


“C” is for Collaboration Culture
• A general willingness to share
information
• Co-operative, communicative &
collegiate OR control, coercion
& criticism?
• The “whose data is this?” cue
• Call-to-action?
• Accountability &
measurement?
16

“Respond intelligently even
to unintelligent treatment.”


“D” is for Data Models
Enterprise Information Model
Related Data
Governance Processes

Conceptual Layer (Business)
Business Glossary

Business Content

Data Domains
Data Elements, Definitions,
Aliases, and Security

Business Rules Data

Business Constraints

Information
Asset
Register

Business Measures
Endorsed Standards
for Content

Conceptual Model:
Groupings & Relationships

Master data models

Information
Concepts
Data Subject Areas

Business Rules
Definitions

Classification Entity
Hierarchies

Business Constraints
Business Measures

Data Entities

Mappings

Data Attributes

Physical Layer

Logical
Logical
Message
Message
Schemas
Schemas

Physical
Physical
Messages
Messages
Formats
Formats

Application
Application
Logical Data
Logical Data
Models
Models

Master Data Repositories

Operational
Operational
DB Models
DB Models
Analytical
Analytical
DB Models
DB Models

UNSW Core Systems
Systems Data
Models

MDM Data
Model

Core
Systems

MDM

Information Model Management Process

Reference models

Metadata Management Process

Logical Layer (Transition)

Information Alliances: Data Ownership & Stewardship Process

Business Data
Element
Domain Values

Metadata
Repository

Cubes
Cubes

Physical Instantiations

DWH
DB

HR
DB

G/L


Student
Admin

etc...

SOA/EP
Messages

MDM Processes

“Do not quench your
inspiration and your
imagination; do not
become the slave of your
model.”


“D” is for Data Quality
Careless data entry
& lack of validation

Data is in different
Formats

Multiple codes exist
for the same thing

Overlapping subsets
in different places
Organisations
change rapidly

“Get your facts first,
then you can distort
them as you please.”


Information
Disconnect

Teams have different ways
of reporting data
Region

Multiple, inconsistent
master data
Teams use different
IT systems

Month

?


Summary: Data Governance increases utility
& value of information
Information Asset Register
(Catalogue of data holdings)

Data Quality Management
(Profiling, root-case analysis,
issues tracking & resolution)

Information System(s)
Information System
Information System
Data Set
Information System

Information Requirements &
Business Analysis
(Identification & traceability of
business definitions & rules)


Data Modelling
(Consistent, inter-operable data
structures & semantic meaning)


“The bigger the better;
in everything.”
What data should be stored, and how
long should we keep it?
Sponsored by Freddie Mercury



“E” is for “Everything”
• All of the data, all of the time?
• Granular, forensic history
• Modern data management & analytics solutions can make “all
of the data, all of the time” a reality
• The bigger challenge is that the business community is not
analytically skilled enough to navigate the data and draw
meaning from it…



“E” is for “Expectations”
• Finding data that makes an impact
• Having data for the problem at hand
• Trusting the data to guide your
decision
• Justifying pre-determined answers
• Setting inappropriate goals
• Not having the right data tools
• Not thinking about value


“Two things are infinite.
The universe and human
stupidity. …and I’m not so
sure about the universe.”


“F” is for “Forethought”
•

Think about both current and future demand
– Cf. Google philosophy to “keep everything”
– Every click, every font change…

•

Tiered storage technology for different scenarios
– Fast-access disk for computational processing (TB)
– Drill-through near-line disk (TB/PB)
– Off-line active store (optical, tape) for archive (PB/EB)

•

Information Lifecycle Management
–
–
–
–

Copy/replication management
Test data management
Auditability of data movements between environments
e.g. IBM Optim, Oracle ILM, Informatica ILM, Hadoop Project
Falcon

“Forethought we may have,
undoubtedly, but not
• Integration/coupling of HDFS & EDW/BI?
foresight.”
•


Data Requirements Management…


Summary: Data Requirements Management two modes of operation
Need both “push” and “pull” modes
for evidence-based decision-making
Data Factory
(“push”)

Source new data;
Collection &
Integration;
Preparation & Quality.
Demand-oriented
Inbound requests for
specific requirements
Product-based
delivery (“pull”)

Plus metadata management, info asset management,
data management, process management practices..



“I try not to think with my
gut. If I‘m serious about
understanding the world,
thinking with anything
besides my brain, as
tempting as that might be,
is likely to get me into
trouble.”
Information Management Excellence
sponsored by Carl Sagan



“G” is for Governance
•
•
•
•
•
•
•
•
•

“I don’t know what you’re going to do with my data once
you have it.”
“If I give you my data, you might then ask me to do some
extra work to meet your additional requirements.”
“You may not interpret the data in the same way that I do.”
“I’m an expert in this area, you’re not. The data is too
complex for you to understand.”
“It’s too difficult to get the data out of the system and I’d
need help from I.T.”
“I don’t have the budget to pay for your requirements.”
“I’d like to help but I’m just far too busy.”
“I know there are flaws in the data, but it’s good enough
for my needs. You might criticize me for the errors.”
“Management may ask additional questions and hold me
to account for the work I’m doing”.

26

“I’m not interested in
preserving the status quo;
I want to overthrow it.”


Data Governance: overcoming “I object!”
• Control, structure, discipline &
compliance? OR Advocacy service &
information broker?
• Intimate understanding of business goals
& processes
• Engagement, diagnosis & facilitation
• Understand & articulate the meaning of
data, in context
• Coach, mentor and advocate
• Highly visible point-of-access
• Self-service Information Portal
• Conduit, communicate & co-ordinate
• Leadership & direction
• “Info as a Product”
27

“The art of government is to
make two-thirds of a nation
pay all it possibly can for the
benefit of the other third.”


Conclusion: Information Excellence
EIM Framework:
Enterprise Information Management Framework describes each aspect of an organisations
information management state, provides a baseline of maturity against best practice and a
framework of business transformation to your aspirational information management state.
Provides linkage and balance between business,/IT, and human/technical aspects of EIM.

Information Culture

Enterprise Information Model

Data Governance

Organisation and People

IM Processes

IM Policies

IM Solutions and Technology

Conceptual Model
Information
Governance

Capture & Formalise
Requirements
& Rules

Business Processes

IM Governance Process
Impact
Assessment
& Implementation

Definitions, Derivations, Decision Rules, Execution Rules

Information and IM
Risk Management

Logical Model

Architecture
Mgt
IM Stewardship Process

DB Models

Semantic Specs

ETL Specs

Report
Definitions

Metadata
Lineage
Impact
Etc.

Information and IM
Quality Mgmt

Information Security

Data Mining

Reporting design

Analytics

Business Intelligence

Data Warehousing

Knowledge Transfer

Information Asset
Mgmt
Metadata

Information Asset Access and Use
Management
Assess and
Accessibility

Intellectual Property
Copyright

Ownership &
Stewardship

Licensing and
Rights
Management
Pricing

Exchange

Registration
Information and IM
Governance
Processes

Privacy

Publishing

Record Keeping

ETL

Staging

ETL

Warehouse

Cubes

Standard Report Library

Operational
System

Archiving

Conservation and
Preservation

Retrieval and Access

Digital Continuity

Record Creation
and Capture

Collection Management

Retention and Disposal

Record Management

Data
Marts

Human Resources
Security
Management
Physical and
Environmental
Management

Search and Discovery

Information Asset
Classification

Asset Alignment/Mgt

Physical Model

Information
Security Policy
and Governance
Asset
Management

Meta Knowledge

Information and IM
Policy, Principles
and Architecture
Information IM
Workforce
Management

Architecture Changes

Logical Model

Information and IM
Strategy and
Planning

Knowledge Management

Technical MetaData Management

Data management

Communications
and Operations
Management
Information
Security Incident
Management
Access
Management
Information
system
acquisition,
development and
maintenance
management

Metadata
Collection

Redress Mechanisms

Data Quality and
Integrity

Data Conversion
& Transformation

Business
Continuity

Data Modelling

Data Cleansing

Data Migration

Compliance
Management

Data Integration

Semantic
Layer

Data Capture

Data De-duplication

A holistic, data-centric approach to Information Management & Data Governance,
addressing both human and technical factors in both Business and IT domains


Intellectual curiosity
Skeptical scrutiny
Critical thinking
http://www.informationaction.blogspot.com.au/
@Alan_D_Duncan
http://www.linkedin.com/in/alandduncan



The ABC of Data Governance: driving Information Excellence

More Related Content

What's hot

Viewers also liked

Similar to The ABC of Data Governance: driving Information Excellence

Recently uploaded

The ABC of Data Governance: driving Information Excellence