[Confoo Montreal 2020] From Grief to Growth: The 7 Stages of Observability - ...
Data Modeling for communication
1. A German, a Frenchman and an Italian
walk into a bar:
A case study of data modeling as communication tool
across different cultures
RICHARD FREGGI
SENIOR SUPPLY CHAIN ARCHITECT
HP
DMZ AUSTRALIA 2015 Page 1
2. About this case study
Experience from many years working in China…
◦ Communicate difficult concepts across cultural and language gaps
◦ Invent new business processes to meet complex and unfamiliar requirements
◦ Develop solutions that support the processes
◦ Train users and gain stakeholder support
… tells me that data is “everywhere”
◦ Great benefits from using data correctly
◦ But little understanding on how to do it
◦ This is a seldom explored “no man’s land” – neither users nor technologists are comfortable with it
I’d like to share and learn
◦ New roles for the Data Modeler
◦ Methodology, approaches, lessons learned
◦ Who else has similar experiences and can share insights?
DMZ AUSTRALIA 2015 Page 2
3. A German, a Frenchman and an Italian walk into a bar…
They have a pet panda bear. The panda orders a hamburger and eats it.
Then the panda pulls out a gun, shoots the table and walks out.
The three friends follow the panda, and the bartender stops them at the door:
“What the hell was that about?”
The German answers: “Just Google it”
So the bartender goes online:
"Panda: a black and white bear from China.
Eats shoots and leaves."
DMZ AUSTRALIA 2015 Page 3
4. Why this matters to Data Modelers
This joke has some resonance to doing business in unfamiliar environments
◦ We can’t assume that our understanding is the same as everyone else’s
◦ Differing points of view and mismatched expectations are common
◦ Assumptions and past experience can lead us in the wrong direction
◦ We can end up here… although our goal is here
DMZ AUSTRALIA 2015 Page 4
The Data Model can help us reach the right goal!
5. Setting up operations in China
Many different players
◦ Prime contractors
◦ Suppliers, sub-suppliers
◦ Service providers
◦ Several local, provincial, central agencies
Language gaps
Each has their own culture, jargon, assumptions and expectations
Many regulations
◦ Local laws, foreign government and international regulations
Time pressure
Lack of precedents… most stakeholders have never done this before
DMZ AUSTRALIA 2015 Page 5
6. We need to:
◦ Agree
◦ Understand before we agree
◦ Invent new solutions, understand them and then agree
◦ Make sure that the solutions are realistic and implementable
◦ Communicate the agreement across organizational, cultural and language barriers
Progress can be slow…
DMZ AUSTRALIA 2015 Page 6
We cannot tell how the individual Processes
fit together
Confirm
Responsibilities, SOPs
Define goals Form Team List high level
Processes
Define detailed
Processes
Define detailed
Processes
Argue
Pull in new Stakeholders
Define detailed
Processes
7. We use a Data Model to communicate complex concepts
across cultural and language gaps
• Agree
• Understand before we agree
• Invent new solutions, understand them and then agree
•Make sure that the solutions are realistic and implementable
• Communicate the agreement across barriers
DMZ AUSTRALIA 2015 Page 7
1 to 2 hours per Stakeholder
Semantic Data
Dictionary
Based on facts, not assumptions or
opinions
Form Team, explain
Data Dictionary
Agree on how to use the
information to reach the Goals
(UML Sequence Diagrams)
We can easily tell
if the Information generated from one Process
is enough to feed the next Process
Remove ambiguity of different
terminologies, assumptions, practices
Confirm
Responsibilities,
SOPs
Approx. a dozen SOPs
agreed among ~20 Stakeholders
Stakeholder
Workshop
Stakeholder
Workshop
Stakeholder
Workshop
Stakeholder
Workshop
Stakeholder
Workshop
Stakeholder
Workshop
Model Fragment
Model Fragment
Model Fragment
Model Fragment
Model Fragment
Model Fragment
Factual
validation
Factual
validation
Factual
validation
Factual
validation
Factual
validation
Factual
validation
Consolidate many different views
into a consistent whole
that explains the subject area in question
8. Great benefits from using data correctly
Long discussions about how to obtain a critical document
◦ Who wants to do it?
◦ Who can do it?
◦ Who should do it?
◦ Very hard to answer without a very clear understanding of what is “it”!
Change discussion from “How” it should be done into “What” this document is
◦ What information it contains (Attributes)
◦ How to identify the information (Primary Key)
◦ Where the information comes from (Foreign Keys)
When we agree what the document is, it’s clear who and how should do it!
Then we can progress quickly through complex discussions
DMZ AUSTRALIA 2015 Page 8
9. AGENCYHEADER
loadno
loaddate
periodfrom
periodto
OPSHEADER
opsheaderid
opsname
generationdate
periodfrom
periodto
AGENCYDOCUMENT
transactnumber
loadno (FK)
agentname
processdate
agdestinationarea
agtransactmode
OPSDOCUMENT
transactrecordid
issuingopsname
opsheaderid (FK)
opsname (FK)
casenumber
origin
opsdestination
opsdestinationarea
issuedate
opsagentname
opstransactmode
TRANSACTIONTITEM
itemid
itemrecordid
transactrecordid (FK)
issuingopsname (FK)
transactnumber (FK)
itemparentid
itemparentrecordid
itemquantity
itemtype
itemproperty
itemrecordsource
TRANSACTIONCONTENT
transactcode
itemid (FK)
itemrecordid (FK)
contenttype
contentdescription
contentquantity
◦ Entities defined through stakeholder workshops
+ validation by reviewing actual documents
and data extracts
◦ Attributes and PK/FK help stakeholders
keep track of how each Entity can/cannot
be used in real life processes
◦ The Semantic Data Model provides
a common language to understand each other
and talk about new ideas
◦ This fragment shows how to reconcile
different interpretations of the same transaction
as seen by different functional silos
◦ This fragment leads to significant simplification in business process and R&R
The Data Model helps us understand, invent, agree
Semantic Data Model fragment
DMZ AUSTRALIA 2015 Page 9
10. The Data Model helps us define processes
and communicate the agreement across barriers - 1
Semantic Data Dictionary
AGENCYHEADER
loadno
loaddate
periodfrom
periodto
OPSHEADER
opsheaderid
opsname
generationdate
periodfrom
periodto
AGENCYDOCUMENT
transactnumber
loadno (FK)
agentname
processdate
agdestinationarea
agtransactmode
OPSDOCUMENT
transactrecordid
issuingopsname
opsheaderid (FK)
opsname (FK)
casenumber
origin
opsdestination
opsdestinationarea
issuedate
opsagentname
opstransactmode
TRANSACTIONTITEM
itemid
itemrecordid
transactrecordid (FK)
issuingopsname (FK)
transactnumber (FK)
itemparentid
itemparentrecordid
itemquantity
itemtype
itemproperty
itemrecordsource
TRANSACTIONCONTENT
transactcode
itemid (FK)
itemrecordid (FK)
contenttype
contentdescription
contentquantity
Semantic Data Model fragment
DMZ AUSTRALIA 2015 Page 10
Responsibilities and Operating Procedures
UML Sequence Diagram (Process and R&R)
Check Recordsseq
: Ops Transact
Team
: Agency B: Agency A: Ops
Document :
OpsDocument
Document :
AgencyDocument
1 [Trigger date 1] : Request
2 : Get()
3
4 : Request
5 : Get()
6
7 : Confirm()
8
9 : Update()
10 [Trigger date 2] : Get()
11. The Data Model helps us define processes
and communicate the agreement across barriers - 2
◦ Entities from the Semantic Data Model relate to
Objects in UML Sequence Diagrams
◦ Since the Entities represent the business view of
information, the transformation is easy
◦ The Data Model helps us keep track of what each
Object can and cannot do
◦ This helps us develop new processes very quickly
◦ Sequence Diagrams are documented as SOPs
(Standard Operating Procedures) and communicated
to all stakeholders
◦ Everyone can clearly see who does what: R&Rs,
process flow and how each document is used
DMZ AUSTRALIA 2015 Page 11
AGENCYHEADER
loadno
loaddate
periodfrom
periodto
OPSHEADER
opsheaderid
opsname
generationdate
periodfrom
periodto
AGENCYDOCUMENT
transactnumber
loadno (FK)
agentname
processdate
agdestinationarea
agtransactmode
OPSDOCUMENT
transactrecordid
issuingopsname
opsheaderid (FK)
opsname (FK)
casenumber
origin
opsdestination
opsdestinationarea
issuedate
opsagentname
opstransactmode
TRANSACTIONTITEM
itemid
itemrecordid
transactrecordid (FK)
issuingopsname (FK)
transactnumber (FK)
itemparentid
itemparentrecordid
itemquantity
itemtype
itemproperty
itemrecordsource
TRANSACTIONCONTENT
transactcode
itemid (FK)
itemrecordid (FK)
contenttype
contentdescription
contentquantity
Check Recordsseq
: Ops Transact
Team
: Agency B: Agency A: Ops
Document :
OpsDocument
Document :
AgencyDocument
1 [Trigger date 1] : Request
2 : Get()
3
4 : Request
5 : Get()
6
7 : Confirm()
12. The Data Model helps us implement technology solutions to
support the agreed processes
Very difficult to manually reconcile
tens of thousands of different documents
from multiple incompatible sources
Use the Data and Process Models to
understand reconciliation issues
Derive Physical Data Model and
implement in analytical database
◦ >20 queries help us reconcile
different interpretations
of the same transaction
◦ Agreement on Data and Process Model
is a great help in obtaining right data feeds
◦ The queries enable us to perform the agreed
processes
DMZ AUSTRALIA 2015 Page 12
AGENCYHEADER
loadno
loaddate
periodfrom
periodto
OPSHEADER
opsheaderid
opsname
generationdate
periodfrom
periodto
AGENCYDOCUMENT
transactnumber
loadno (FK)
agentname
processdate
agdestinationarea
agtransactmode
OPSDOCUMENT
transactrecordid
issuingopsname
opsheaderid (FK)
opsname (FK)
casenumber
origin
opsdestination
opsdestinationarea
issuedate
opsagentname
opstransactmode
TRANSACTIONTITEM
itemid
itemrecordid
transactrecordid (FK)
issuingopsname (FK)
transactnumber (FK)
itemparentid
itemparentrecordid
itemquantity
itemtype
itemproperty
itemrecordsource
TRANSACTIONCONTENT
transactcode
itemid (FK)
itemrecordid (FK)
contenttype
contentdescription
contentquantity
13. Conclusion
Data Modeling helps to communicate across cultural and language gaps, invent
new business processes and solutions
Who has similar experiences?
Different experiences?
Feedback?
DMZ AUSTRALIA 2015 Page 13