Presentation from 2014 International Data Quality Summit (www.idqsummit.org, Twitter hashtag #IDQS14). Techniques for business analysts and data scientists to facilitate better requirements gathering in data and analytic projects.
The one question you must never ask!" (Information Requirements Gathering for Analytic Projects)
1. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Information requirements gathering for data projects, Sponsored by W. Edwards Deming
The one question you must never ask!
“If you do not know how to ask the right question, you discover nothing.”
2. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Agenda
•Challenges of info requirements gathering
•Methods for engaging with data stakeholders
•Techniques for identifying info requirements
•Establishing actionable business outcomes
3. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
A bit about me....
•NEW! Research Director, Gartner
•Advisory board member, QFire Software
•22 years Information Strategy, Data Governance, Analytics & Business Consulting
–Director of Data Governance at UNSW
–EDS, KPMG, CPW, Acuma, Pelion, SMS
–Scottish Power, United Distillers, O2, Astra Zeneca, Carphone Warehouse, Vodafone, Riyad Bank
–Commonwealth Bank, NSW Roads & Maritime Services, Centrelink, OATSIH, NSW Family & Community Services, CASA, AMSA, FaHCSIA, DAFF, Navy…
•Information-Management.com “Top 12 on Twitter”
•Best supporting Actor, 2005 Barnet Drama Festival
4. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Challenges for
information requirements gathering,
Sponsored by Sir Arthur Conan Doyle
“Data! Data! Data! I can’t make bricks without clay!”
(Sherlock Holmes,
The Adventure of the Copper Beeches)
5. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
“What’s the one question that you should never ask?”
6. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
“What information do you want?!”
7. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
8. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Challenges for Information Requirements gathering…
Lack of knowledge about new information-processing technologies
People don’t know what they want!
Unmet & changing demands
Competition between teams
9. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
…plus second-guessing future needs.
10. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Methods for engaging data stakeholders,
Sponsored by Brene Brown
“Maybe stories are just data with a soul.”
11. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Information Workshop Agenda
•Context setting
•Business Functions
•Challenges
•Opportunities
•Desired Actions
12. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Don’t just facilitate - provoke!
•“What would happen if…X?”
•“Have you thought about…Y?”
•“Why do you do/want…Z?”
13. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
“Stupid is as stupid does…”
•Don’t do “Lessons Learned”!!!
•Do do “Long Hard Look”
–Start
–Stop
–Continue
•“Short Hard Look”
14. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
“Now that’s magic!”
•Most people don’t want to know how the tricks work…
•Don’t discuss “what we do, how we do it”
•Stay away from practitioner jargon
•Focus on business outcomes
15. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Techniques for identifying information requirements, Sponsored by Johann Wolfgang Von Goethe
“An intelligent man finds almost everything ridiculous.”
16. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Five Whiskies in a Hotel
•What data/information inputs? What outputs? What tests/measures/criteria will be applied to confirm whether the data is fit for purpose or not?
•Why do you want it? (What outcomes do you hope to achieve? Does the data being requested actually support those questions & outcomes? Consider Efficiency/Effectiveness/Risk Mitigation drivers for benefit.
•Who is involved? For whom is the information? Who has rights to see the data? Who is it being provided by? Who is ultimately accountable for the data - both contents and definitions? Consider multiple stakeholder groups?
•When is the information required? When is it first required? Frequency? Trigger events?
•Where is the data to reside? Where is it originating from? Where is it going to?
•How will it be shared? How will the mechanisms/methods work to collect/collate/integrate/store/disseminate/access/archive the data? How should it be structured & formatted?
17. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Categorise the comments
•Functions
•Stakeholders
•Inhibitors
•Enablers
•Constraints
•Considerations
•Source systems
•Data sets
18. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Example Dialectic
•ADD: “What does your team do?”
•Workshop Victim Participant #1: “Well, we’re responsible for reconciling the customer account balances with the individual transactions.”
•ADD: And why do you want to do that?
•Workshop Victim Participant #2: “We think there’s a discrepancy in the warehouse stock balances, compared with what’s been shipped to customers. The sales guys keep their own database of customer contracts and orders and Jim’s already given us dump of the data, while finance run the accounts receivables process. But Sally the Accounts Clerk doesn’t let the numbers out under any circumstances, so basically we’re screwed.”
“The beginning of wisdom is the definition of terms”
19. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Example Dialectic: Analysis
•Functions: Sales Processing, Contract Management, Order Fulfilment, Stock Management, Accounts Receivable.
•Stakeholders: Warehouse team, Sales team (Jim), Finance team.
•Inhibitors: Finance don’t collaborate.
•Enablers: Jim is helpful.
•Source Systems: Stock System, Customer Database, Order Management, Finance System.
•Data Sets: Orders (Quantity & Price by Customer, by Salesman, by Stock Item), Dispatches (Quantity & Price by Customer, by Salesman, by Warehouse Clerk, by Stock Item), Financial Transactions (Value by Customer, by Order Ref)
“A good decision is based on knowledge and not on numbers.”
20. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Information Asset Management is about mapping
information assets to business outcomes
Owners
Asset
Management
Tools
Governance
Admin
Experts
User
Community
Information
Asset
Steward
OOwwnenresrs
The “Information Asset Community”
Directives
AIRS WMS
Example High Level Data
Systems & Flows
Version 1.0
General
Mandatory
Core
Corporate Support
Automatic One-way Relationship
Automatic Two-way Relationship
Manual One-way Relationship
Manual Two-way Relationship
External
External System
AIRS
Interchange
AUSSAR
AME
Cyber Exams
ATSL
CLIC
SDR
Publishing
System
GMEL
TRIM AD
FRLI
Web Control
Mgmt
System
(WCMS)
FMIS
STI
ESIR
Comweb
ATO
Business
Portal
Inventory
Mgmt
System (IMS)
EPK
ComBIZ
Online
ProMaster
FCAT
Calumo
CBMS
(DoFD)
COMCARE
Thomas
Logistics
SM7
HRMS
HRFlex
DTAR/OTAR
APEX
AOD Audit
AOD Case
Mgmt
System
Timelog
eRooms
Symbion
Health
System
API Upload
System
ChangePoint
Testing
System
MRS
ASSP
AWS
AFD
ASIR
ADMS
Tracker
ATOG Job
Register
AOC
Surveys
Industry Payments
Compensation Payments
Financial Actuals
Financial Actuals
Employee
Expenses /
Adjustment
Journals
Salary Payments
Cash Payments
Payrol (Salary)
Cash Payments / Organisation Info
Human Resources Finance
Surveil ance/Audit/
Reporting/Tracking
Workflow and Online Col aboration/
Service Delivery
Service Delivery
Service Delivery
Service Delivery, HR & Finance, Agreements, Permissions, Aerodromes,
Participants, Aircraft
Medical Examinations
Medical Exams
Surveil ance /Audits/
Reporting/ Tracking
Alcohol and Other Drugs
Surveys /
Surveil ance
Events/Occurrences, Aircraft,
Aerodromes
Surveys/
Certifications
Examinations
Work Orders
Surveil ance /Audits
Aerodromes
Aircraft
Events/Occurrences,
Aircraft, Aerodromes
Defects/Events/Occurrences, Aircraft,
Aerodromes
Exemptions
Database
Alternative
Means of
Compliance
(AMOC)
Exemptions
AMOC / Exemptions
Human
Resources – Flex
Time
Human Resources - Travel
Physical Inventory
Audit Data
workflow / service delivery
workflow / service
delivery
Contacts – Ind, Org’s Contacts – Ind, Org’s
Examinations
Medical
Examinations
Search and Rescue
Surveys
Human Resources – Time
Aircraft Equipment Finances
MMEL
Baseline/Minimal Equipment
Medical
PAWS
Retain
Details of
Operators
Incidents
Applications / Permissions
Trending
Workflow
MAAT
Permissions / Change of
Status
Permissions / Change
of Status
Service Log
Alternative Means of Compliance
(AMOC)
Dangerous
Goods
Dangerous Goods
Content
Inventory
FTTO
FTNS
Individual Flight Data
Organisational
Flight Data
Human Resources – Time
Aircraft
Individuals/ARNS
Payments
Cash Receipts HR - Time
Reconcile
Invoice against
Flown Hours
Surveys / Surveil ance
Enterprise Data
Warehouse
CASA
Internet
Airports
Landings/ Take Offs
Data
Mandatory
Core
Corporate Support
External
Business Process
Surveil ance/
Audit/
Reporting/
Tracking
Bank Data
File
PAYG
payments, Salary
payments, and
Superannuation
payments.
External
Superannuation
Companies
Cash Payments
Superannuation Contributions
Suppliers
Remittance
Advice
300+ Access
Databases
Contacts
Airspace
Organisational
Human Resources
Aircraft
Permissions Info Asset Register
(inventory)
System Interfaces map
“Science is organized
knowledge. Wisdom is
organized life.”
21. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Actionable Business Outcomes, Sponsored by Thomas Edison
“The value of an idea lies in the using of it.”
22. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Actionability = Value-add
•Look for self-assigned actions
e.g.
Workshop Victim Participant #1:
“I could go and speak to the Financial Controller about getting access to the finance data. He’s more amenable to working together than Sally, who just does what she’s told.”
23. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Enabling method: CRUD Matrix
•Create
•Read
•Update/Use
•Delete
24. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Enabling method: Information Lifecycle Map
Time tableHandbookArchibusUniForumCompletion rates for HDA
25. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Enabling Method: DQ profiling
•Data Quality Profiling and root- cause analysis
•Identify issues & patterns
•Initiation activity as part of all data warehouse, master data & application migration projects
“Whoever is careless with the truth in small matters cannot be trusted with important matters.”
Common Data Quality processes and collaboration workspace
Collect
Validate
Protect
Monitor
26. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Pay a visit to Whitefellah Burrows…
•Know what you’re looking for (and why)
•Dig in the right areas
•Sift through the mullock
•Be prepared for some hard digging
•Be careful or you might get hurt!
27. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Conclusions & final thoughts,
Sponsored by Terry Pratchett
“It’s still magic even if you know how it’s done.”
28. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Resources: info requirements templates
•http://informationaction.blogspot.co.uk/p/resources-and-downloads.html
•01. Framework Overview
•02. Information Solution Outline & High Level Requirements Template
•03. Business Information Requirements Template
•04. Logical Data Definition Template
•05. Physical Data Specification Template
•06. Transformation Logic Template (Source to TargetMapping)
•07. Analytics & Reporting Requirements Template
29. Alan D. Duncan http://nformationaction.blogspot.com Tw: @Alan_D_Duncan
Information Strategy | Data Governance | Analytics | Better Business Outcomes
Intellectual curiosity
Skeptical scrutiny
Critical thinking
http://www.informationaction.blogspot.com.au/ http://blogs.gartner.com/alan-duncan/
@Alan_D_Duncan
http://www.linkedin.com/in/alandduncan