The data architecture of solutions is frequently not given the attention it deserves or needs. Frequently, too little attention is paid to designing and specifying the data architecture within individual solutions and their constituent components. This is due to the behaviours of both solution architects ad data architects. Solution architecture tends to concern itself with functional, technology and software components of the solution Data architecture tends not to get involved with the data aspects of technology solutions, leaving a data architecture gap. Combined with the gap where data architecture tends not to get involved with the data aspects of technology solutions, there is also frequently a solution architecture data gap. Solution architecture also frequently omits the detail of data aspects of solutions leading to a solution data architecture gap. These gaps result in a data blind spot for the organisation. Data architecture tends to concern itself with post-individual solutions. Data architecture needs to shift left into the domain of solutions and their data and more actively engage with the data dimensions of individual solutions. Data architecture can provide the lead in sealing these data gaps through a shift-left of its scope and activities as well providing standards and common data tooling for solution data architecture The objective of data design for solutions is the same as that for overall solution design: • To capture sufficient information to enable the solution design to be implemented • To unambiguously define the data requirements of the solution and to confirm and agree those requirements with the target solution consumers • To ensure that the implemented solution meets the requirements of the solution consumers and that no deviations have taken place during the solution implementation journey Solution data architecture avoids problems with solution operation and use: • Poor and inconsistent data quality • Poor performance, throughput, response times and scalability • Poorly designed data structures can lead to long data update times leading to long response times, affecting solution usability, loss of productivity and transaction abandonment • Poor reporting and analysis • Poor data integration • Poor solution serviceability and maintainability • Manual workarounds for data integration, data extract for reporting and analysis Data-design-related solution problems frequently become evident and manifest themselves only after the solution goes live. The benefits of solution data architecture are not always evident initially.

1. Data Architecture For
Solutions
Alan McSweeney
http://ie.linkedin.com/in/alanmcsweeney
https://www.researchgate.net/profile/Alan-Mcsweeney
https://www.amazon.com/dp/1797567616

2. Introduction
• These notes discuss how overall organisation data
architecture can positively impact solution design and how
solution data architecture competence within solution
architecture can contribute to solution design success
January 9, 2023 2
Data
Architecture
Solution
Architecture
Solution Data
Architecture
Can
Contribute to
Common Data
Infrastructure
Tools and
Data
Standards
Can
Contribute to
Overall
Organisation
Data Quality
Can Ensure
that the Data
Aspects of
Solution
Design Are
Covered in
Solution
Designs
Can Ensure
that Solution
Data Concerns
Are Addressed
in Solution
Designs

3. Topics
• Data Architecture For Solutions
• Traditional Scope Of Data Architecture
• Solution Data Architecture
• What Do We Mean By Data Architecture?
• Data Architecture And Common Data Tooling And
Standards
• Data Design And Modelling For Solutions
January 9, 2023 3

4. Data Architecture For Solutions
• Data breathes life into solutions
• Solutions get data, use data, share data, process data and create data
• There will be many different types of data used by a solution
− Master data
− Reference data
− Input data
− Interim data
− Generated data
− Solution activity and usage data
• Any solution will consist of many different components of different types
• Solution components and their data will be deployed and operated across a solution
landscape that can span multiple zones and platforms
• Within the solution, each data type will have a different lifecycle
• The solutions within the organisation solution landscape will have both shared and
private data
− Shared - common data (master or reference) or upstream data from other solutions or data sent
downstream
− Private – data held locally within the solution
January 9, 2023 4

5. Solution And Data
• All IT solutions support, implement and operate business
processes that take data inputs, process data, generate result
and create primary and supporting data output
− Direct data outputs – what the process in intended to create
− Indirect data outputs – logs, audit trails, reports, analyses
• Data outputs are then used in different ways
− Generated results
− As a record that the work was performed
− As inputs into other processes and solutions
− To report on the operation of the process or as an audit log
• Data breathes life into and activates the static components of a
solution
• The data architecture of solutions is frequently not given the
attention it deserves or needs
January 9, 2023 5

6. Data Architecture For Solutions
• Frequently, too little attention is paid to designing and
specifying the data architecture within individual solutions
and their constituent components
• This is due to the behaviours of both solution architects ad
data architects
• Solution architecture tends to concern itself with
functional, technology and software components of the
solution
• Data architecture tends to concern itself with post-
individual solutions
January 9, 2023 6

7. Traditional Scope Of Data Architecture
January 9, 2023 7
Data
Ingestion
and
Integration
Data Validation
and Error
Handling
Data Encryption,
Anonymisation,
Pseudonymisation
Security
and Access
Control
Data
Processing
Workflow
Data Model
and Data
Store
API
Interface
Data
Interrogation
and Analysis
Data
Visualisation
Data
Extract
Management
and
Administration
Data
Publication
and Sharing
Existing and
New
Reports
Data Storage
Platform/
Infrastructure
Usage and
Performance
Monitoring
Semantic
Layer
Data Sources
(Internal, External) Extract, Transform, Load Data Platform Access and Usage
Merge,
Aggregate,
Transform
Data
Sources
(from
Solutions)

8. Traditional Scope Of Data Architecture
January 9, 2023 8
Data
Ingestion
and
Integration
Data Validation
and Error
Handling
Data Encryption,
Anonymisation,
Pseudonymisation
Security
and Access
Control
Data
Processing
Workflow
Data Model
and Data
Store
API
Interface
Data
Interrogation
and Analysis
Data
Visualisation
Data
Extract
Management
and
Administration
Data
Publication
and Sharing
Existing and
New
Reports
Data Storage
Platform/
Infrastructure
Usage and
Performance
Monitoring
Semantic
Layer
Merge,
Aggregate,
Transform
Scope Of Data Architecture
Data
Sources
(from
Solutions)

9. Traditional Scope Of Data Architecture
• Traditional approaches to data architecture effectively
appends or layers newer technologies on top on existing
solutions and data sources and their data structures
• Data architecture largely ignores data architectures within
individual solutions
• Data architecture needs to shift left into the domain of
solutions and their data and more actively engage with the
data dimensions of individual solutions
January 9, 2023 9

10. Traditional Scope Of Data Architecture
January 9, 2023 10
Data
Ingestion
and
Integration
Data Validation
and Error
Handling
Data Encryption,
Anonymisation,
Pseudonymisation
Security
and Access
Control
Data
Processing
Workflow
Data Model
and Data
Store
API
Interface
Data
Interrogation
and Analysis
Data
Visualisation
Data
Extract
Management
and
Administration
Data
Publication
and Sharing
Existing and
New
Reports
Data Storage
Platform/
Infrastructure
Usage and
Performance
Monitoring
Semantic
Layer
Merge,
Aggregate,
Transform
This Is Not A Modern Data Architecture
Data
Sources
(from
Solutions)

11. Not A Modern Data Architecture
• You are fooling yourself if you
believe that enveloping
existing data solutions,
sources and structures with a
skin of modernity comprises a
data architecture
• If you put lipstick on a pig, it is
still a pig
January 9, 2023 11

12. Common and
Shared Data
Processes and
Standards
Data
Architecture
Operation,
Measurement
Data
Architecture
Review,
Improvement,
Update
Common and
Shared Data
Infrastructural
Components
Data
Solutions,
Sources and
Structures
This Is A Data Architecture
January 9, 2023 12
Data Architecture Overview
Data Management, Governance, Supporting Processes
Data Infrastructure, Storage and Operations Software, Hardware and Processes
Data Security, Protection, Compliance, Access Control, Authentication, Authorisation
Data Integration, Access, Flow, Exchange, Transfer, Transformation, Load And Extract
Content, Unstructured Data, Records and Document Management
Master and Reference Data Management
Data Warehouse, Data Marts, Data Lakes
Data Reporting and Analytics, Visualisation Tools and Facilities
Data Discovery, Analysis, Design and Modelling
External Data Sources and Interacting Parties Data Transfer/Exchange/Integration/Publication
Metadata Data Management
Data Quality
Data Solution Design

13. Traditional Scope Of Data Architecture And Data Architecture
Gap
Data
architecture
tends not to
get involved
with the
data aspects
of
technology
solutions,
leaving a
data
architecture
gap
January 9, 2023 13
Data
Ingestion
and
Integration
Data Validation
and Error
Handling
Data
Processing
Workflow
Data
Sources
(from
Solutions)
Merge,
Aggregate,
Transform
New Custom Developed
Applications and Their
Data Models
Acquired and Customised
Software Products and
Their Data Models
System Integrations/
Data Transfers/
Exchanges
Reporting and
Analysis
Facilities
Information
Storage
Facilities
Existing Data
Conversions/
Migrations
New Data
Loads
Changes to Existing
Systems and Their
Data Models
New Custom Developed
Applications and Their
Data Models
Acquired and Customised
Software Products and
Their Data Models
System Integrations/
Data Transfers/
Exchanges
Reporting and
Analysis
Facilities
Information
Storage
Facilities
Existing Data
Conversions/
Migrations
New Data
Loads
Changes to Existing
Systems and Their
Data Models
New Custom Developed
Applications and Their
Data Models
Acquired and Customised
Software Products and
Their Data Models
System Integrations/
Data Transfers/
Exchanges
Reporting and
Analysis
Facilities
Information
Storage
Facilities
Existing Data
Conversions/
Migrations
New Data
Loads
Changes to Existing
Systems and Their
Data Models
…
…
Data Architecture Gap Data Architecture

14. Solution Data Aspects Across The Solution Landscape
January 9, 2023 14
New Custom
Developed
Applications and Their
Data Models
Acquired and
Customised Software
Products and Their Data
Models
System
Integrations/ Data
Transfers/
Exchanges
Reporting and
Analysis
Facilities
Information
Storage
Facilities
Existing Data
Conversions/
Migrations
New Data
Loads
Changes to Existing
Systems and Their
Data Models
New Custom
Developed
Applications and Their
Data Models
Acquired and
Customised Software
Products and Their Data
Models
System
Integrations/ Data
Transfers/
Exchanges
Reporting and
Analysis
Facilities
Information
Storage
Facilities
Existing Data
Conversions/
Migrations
New Data
Loads
Changes to Existing
Systems and Their
Data Models
New Custom
Developed
Applications and Their
Data Models
Acquired and
Customised Software
Products and Their Data
Models
System
Integrations/ Data
Transfers/
Exchanges
Reporting and
Analysis
Facilities
Information
Storage
Facilities
Existing Data
Conversions/
Migrations
New Data
Loads
Changes to Existing
Systems and Their
Data Models
New Custom
Developed
Applications and Their
Data Models
Acquired and
Customised Software
Products and Their Data
Models
System
Integrations/ Data
Transfers/
Exchanges
Reporting and
Analysis
Facilities
Information
Storage
Facilities
Existing Data
Conversions/
Migrations
New Data
Loads
Changes to Existing
Systems and Their
Data Models
New Custom
Developed
Applications and Their
Data Models
Acquired and
Customised Software
Products and Their Data
Models
System
Integrations/ Data
Transfers/
Exchanges
Reporting and
Analysis
Facilities
Information
Storage
Facilities
Existing Data
Conversions/
Migrations
New Data
Loads
Span Of Organisation Solution Landscape
Changes to Existing
Systems and Their
Data Models

15. Solution Components And Their Types
January 9, 2023 15
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Changes to Existing Systems
New Custom Developed Applications
Acquired and Customised Software
Products
System Integrations/ Data Transfers/
Exchanges
Reporting and Analysis Facilities
Sets of Installation and
Implementation Services
Information Storage Facilities
Existing Data Conversions/
Migrations
New Data Loads
Central, Distributed and
Communications Infrastructure
Cutover/ Transfer to Production And
Support
Operational Functions and Processes
Parallel Runs
Enhanced Support/ Hypercare
Sets of Maintenance, Service
Management and Support Services
Application Hosting and
Management Services
Changes to Existing Business
Processes
New Business Processes
Organisational Changes, Knowledge
Management
Training and Documentation
Component Type Solution Components

16. Reduced Scope Of Traditional Solution Architecture
Scope
• The solution is the sum
of the components
needed to deliver and
operate it
• Solution architecture
tends not to concern
itself with some key
aspects of the
complete solution,
including some of
those related to data
• Solution architecture
tends to focus on
technology aspects of
a solution, omitting
business and data
facets
• The data dimensions
of other solution
components also
tends to be omitted
partially or completely
by solution
architecture
January 9, 2023 16
Complete
Solution

17. Data Related Solution Components And Their Types
January 9, 2023 17
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Component
Changes to Existing Systems
New Custom Developed Applications
Acquired and Customised Software
Products
System Integrations/ Data Transfers/
Exchanges
Reporting and Analysis Facilities
Sets of Installation and
Implementation Services
Information Storage Facilities
Existing Data Conversions/
Migrations
New Data Loads
Central, Distributed and
Communications Infrastructure
Cutover/ Transfer to Production And
Support
Operational Functions and Processes
Parallel Runs
Enhanced Support/ Hypercare
Sets of Maintenance, Service
Management and Support Services
Application Hosting and
Management Services
Changes to Existing Business
Processes
New Business Processes
Organisational Changes, Knowledge
Management
Training and Documentation
Component Type Solution Components

18. Data Dimensions Of Solution Component Types
January 9, 2023 18
Changes to
Existing
Systems
Possible new data stores
and their design and data
models
Data tools
Data processes
Changes to existing data
stores and their design
and data models
Performance, capacity
and thoughput
Data security, encryption
and access control
Data management and
governance
Master and reference
data
Metadata
Data quality
New Custom
Developed
Applications
New data stores and their
design and data models
Data tools
Data processes
Performance, capacity
and thoughput
Data security, encryption
and access control
Data management and
governance
Master and reference
data design
Metadata design
Data quality design
Acquired
and
Customised
Software
Products
New data stores and their
design and data models
Data tools
Data processes
Performance, capacity
and thoughput
Data security, encryption
and access control
Data management and
governance
Master and reference
data
Metadata
Data quality
System
Integrations/
Data
Transfers/
Exchanges
Source and target data
stores and their design
and data models
Data transformations and
aggregations
Data tools
Data processes
Changes to existing data
stores and their design
and data models
Performance, capacity
and thoughput
Data security, encryption
and access control
Data management and
governance
Master and reference
data
Metadata
Data quality
Reporting
and Analysis
Facilities
Reporting data stores and
their design and data
models
Reporting tools
Reporting processes
Data security, encryption
and access control
Master and reference
data
Metadata
Data quality
Information
Storage
Facilities
Performance, capacity
and throughput
Data management and
governance
Existing Data
Conversions/
Migrations
Source and target data
stores and their design
and data models
Data transformations and
aggregations
Data tools
Data processes
Changes to existing data
stores and their design
and data models
Performance, capacity
and thoughput
Data security, encryption
and access control
Data management and
governance
Master and reference
data
Metadata
Data quality
New Data
Loads
Target data stores and
their design and data
models
Data tools
Data processes
Changes to existing data
stores and their design
and data models
Performance, capacity
and thoughput
Data security, encryption
and access control
Data management and
governance
Master and reference
data
Metadata
Data quality

19. Data Dimensions Of Solution Component Types
• The following solution component types involve data
architecture and design activities:
− Changes to Existing Systems
− New Custom Developed Applications
− Acquired and Customised Software Products
− System Integrations/ Data Transfers/ Exchanges
− Reporting and Analysis Facilities
− Information Storage Facilities
− Existing Data Conversions/ Migrations
− New Data Loads
• The components of each of these types will potentially involve
data and therefore data design work across a range of areas
that will need to be included in the solution design and
subsequent solution implementation activities
• Solution architecture can fail t include some of the data
dimensions of the solution components of these types
January 9, 2023 19

20. Solution Components And Their Types
• Any technology solution will consist of a potentially large
number of components, each of a give type
• Each solution component type belongs to one of three
classes
1. Time-Bounded Delivery Entity Types
• Time-bounded solution component types required to get the solution fully
operational
2. Enduring Functional and Operational Technology Entity Types
• Operational instrumentation and functional component types required for
the solution to operate and be usable by its target consumers
3. Enduring Organisational, Process, Procedure and Structural
Entity Types
• Organisation and process changes and other supporting activities and sets
of effort required to use the solution optimally
January 9, 2023 20

21. Solution Components Classes And Types
January 9, 2023 21
Solution Component Classes
and Types
Time-Bounded Delivery
Entity Types
Sets of Installation and
Implementation Services
Existing Data Conversions/
Migrations
New Data Loads
Parallel Runs
Enhanced Support/ Hypercare
Enduring Functional and
Operational Technology
Entity Types
Changes to Existing Systems
New Custom Developed
Applications
Acquired and Customised Software
Products
System Integrations/ Data
Transfers/ Exchanges
Reporting and Analysis Facilities
Information Storage Facilities
Central, Distributed and
Communications Infrastructure
Application Hosting and
Management Services
Enduring Organisational,
Process, Procedure and
Structural Entity Types
Cutover/ Transfer to Production
And Support
Operational Functions and
Processes
Sets of Maintenance, Service
Management and Support Services
Changes to Existing Business
Processes
New Business Processes
Organisational Changes,
Knowledge Management
Training and Documentation

22. Solution Architecture Data Gap
• Combined with the gap where data architecture tends not to
get involved with the data aspects of technology solutions,
there is also frequently a solution architecture data gap
• Solution architecture also frequently omits the detail of data
aspects of solutions across the various components of the
types:
− Changes to Existing Systems
− New Custom Developed Applications
− Acquired and Customised Software Products
− System Integrations/ Data Transfers/ Exchanges
− Reporting and Analysis Facilities
− Information Storage Facilities
− Existing Data Conversions/ Migrations
− New Data Loads
January 9, 2023 22

23. Solution Architecture Data Gap
• These gaps result in a data blind spot for the organisation
January 9, 2023 23
Central,
Distributed and
Communications
Infrastructure
Changes to
Existing Systems
New Custom
Developed
Applications
Information
Storage Facilities
Acquired and
Customised
Software Products
System
Integrations/ Data
Transfers/
Exchanges
Changes to
Existing Business
Processes
New Business
Processes
Organisational
Changes,
Knowledge
Management
Reporting and
Analysis Facilities
Existing Data
Conversions/
Migrations
New Data Loads
Training and
Documentation
Sets of
Installation and
Implementation
Services
Operational
Functions and
Processes
Parallel Runs
Cutover/ Transfer
to Production
Sets of
Maintenance,
Service
Management and
Support Services
Application
Hosting and
Management
Services
Enhanced
Support/
Hypercare
Data Architecture
Solution Gap Data Architecture
Data
Ingestion
and
Integration
Data Validation
and Error
Handling
Data
Processing
Workflow
Merge,
Aggregate,
Transform
…
Solution Architecture
Data Gap
Solution Architecture

24. Solution Architecture Data Gap
• Data architecture can provide the lead in sealing these data
gaps through a shift-left of its scope and activities as well
providing standards and common data tooling for solution
data architecture
January 9, 2023 24

25. Shift Left Of The Scope Of Data Architecture
January 9, 2023 25
Source
Systems
Extract,
Transform,
Load
Data
Platform
Access
and
Usage
Changes to
Existing
Systems
and Their
Data
Models
New Custom
Developed
Applications
and Their
Data Models
Acquired and
Customised
Software
Products and
Their Data
Models
System
Integrations
/ Data
Transfers/
Exchanges
Reporting
and
Analysis
Facilities
Information
Storage
Facilities
Existing Data
Conversions
/ Migrations
New Data
Loads
• Shifting data architecture to
the left means getting involved
in the data aspects of solution
design, specification, selection
and implementation at the
earliest opportunity
• This then needs to be
repeated for each solution
within the organisation
solution landscape
• The data aspects of solutions
should be closely integrated
within the organisation’s data
architecture
Shift Left of Scope of Data
Architecture

26. Generalised Data Lifecycle
• Each data type within a
solution will have a lifecycle
from design and creation to
ultimate archival and
possible deletion
January 9, 2023 26
Enter, Create, Acquire,
Derive, Update,
Integrate, Capture
Secure, Store,
Replicate and
Distribute
Preserve, Protect and
Recover
Archive and Recall
Delete/Remove
Implement Underlying
Technology
Architect, Budget,
Plan, Design and
Specify
Present, Report,
Analyse, Model
• A set of data lifecycle view
for solutions can assist in
solution data architecture

27. Generalised Data Lifecycle Stages
• Architect, Budget, Plan, Design and Specify - This relates to the design and specification of the data
storage and management and their supporting processes
− This establishes the data management framework
• Implement Underlying Technology - This is concerned with implementing the data-related hardware and
software technology components
− This relates to database components, data storage hardware, backup and recovery software, monitoring and control software and other
items
• Enter, Create, Acquire, Derive, Update, Integrate, Capture - This stage is where data originated, such as
data entry or data capture and acquired from other systems or sources
• Secure, Store, Replicate and Distribute - In this stage, data is stored with appropriate security and access
controls including data access and update audit
− It may be replicated to other applications and distributed
• Present, Report, Analyse, Model - This stage is concerned with the presentation of information, the
generation of reports and analysis and the created of derived information
• Preserve, Protect and Recover - This stage relates to the management of data in terms of availability,
backup, recovery and retention/preservation
• Archive and Recall - This stage is where information that is no longer active but still required in archived
to secondary data storage platforms and from which the information can be recovered if required
• Delete/Remove - The stage is concerned with the deletion of data that cannot or does not need to be
retained any longer
− Data has to be able to be disposed of in a managed, systematic and auditable way
• Define, Design, Implement, Measure, Manage, Monitor, Control, Staff, Train and Administer, Standards,
Governance, Fund - This is not a single stage but a set of processes and procedures that cross all stages
and is concerned with ensuring that the processes associated with each of the lifestyle stages are
operated correctly and that data assurance, quality and governance procedures exist and are operated
January 9, 2023 27

28. Solution Data Types And Lifecycles
• Every solution will
have one or more
types of data it
reads, processes
or creates
• Each data type will
have a separate
lifecycle that
reflects how it is
processed and
how its attributes
need to be
reflected in its
governance and
management
January 9, 2023 28
Solution
Data Type 1
Data Type 2
…
Data Type N

29. Data Lifecycle Stages And Solution Component Types
January 9, 2023 29
Central, Distributed
and Communications
Infrastructure
Changes to Existing
Systems
New Custom
Developed
Applications
System Integrations/
Data Transfers/
Exchanges
Changes to Existing
Business Processes
Organisational
Changes, Knowledge
Management
Training and
Documentation
Sets of Installation
and Implementation
Services
Parallel Runs
Enhanced Support/
Hypercare
Information Storage
Facilities
Acquired and
Customised Software
Products
New Business
Processes
Reporting and
Analysis Facilities
Existing Data
Conversions/
Migrations
New Data Loads
Operational
Functions and
Processes
Cutover/ Transfer to
Production
Sets of Maintenance,
Service Management
and Support Services
Application Hosting
and Management
Services
Secure, Store,
Replicate and
Distribute
Archive and Recall
Delete/Remove
Implement Underlying
Technology
Architect, Budget,
Plan, Design and
Specify
Present, Report,
Analyse, Model
Enter, Create, Acquire,
Derive, Update,
Integrate, Capture
Preserve, Protect and
Recover

30. Data Lifecycle Stages And Solution Components
• Each stage within the lifecycle of a solution data type will be realised by a solution component
• Mapping the stages within the lifecycle of solution data types and identifying the impact on solution
component types can contribute to effective solution data architecture design
− This provides traceability to ensure the data is being handled correctly
• For example, the Preserve, Protect and Recover data stage involving solution activities such as backup
and recovery, replication, business continuity and disaster recovery may require solution components of
the types:
− Information Storage Facilities
− Sets of Maintenance, Service Management and Support Services
− Operational Functions and Processes
January 9, 2023 30
Operational
Functions and
Processes
Sets of Maintenance,
Service Management
and Support Services
Information Storage
Facilities
Preserve, Protect and
Recover

31. So, What Do We Mean By Data Architecture?
• If data architecture can contribute to solution architecture
then the scope of data architecture should be defined and
agreed to ensure this is possible
January 9, 2023 31

32. Data Architecture And Data Strategy
• Data architecture defines the target data structures,
operations, principles, standards, organisation, tools,
management, governance that the organisation is aiming
to define, implement and operate
− The data architecture is designed to be implemented and
operated
• Data strategy defines how the organisation intends to use
data to deliver on its business strategy
− Data strategy precedes and feeds into the data architecture
January 9, 2023 32

33. Data Strategy And Data Architecture In A Wider
Business And Technology Context
January 9, 2023 33
Business
Objectives
Business
Architecture
Enterprise
Architecture
Solution
Implementation
and
Delivery
Support,
Management
and
Operations
Business
Processes
Required
Operational
Business
Solutions
Business
Strategy
Business
Solution
Analysis and
Design/
Selection
Business IT
Strategy
IT Function
Strategy
Required
Operational
Processes
Required
Supporting and
Enabling
Business
Solutions
Support
Solution
Analysis and
Design/
Selection
Required
Structure,
Capabilities
and
Resources
Digital
Strategy
Digital IT
Architecture
Solution
Portfolio
Design And
Specification
Solution
Portfolio
Management
Solution
Change and
Evolution
Business
Structure and
Operational
Model
Data
Strategy
Data
Architecture

34. Data Strategy And Data Architecture In A Wider
Business And Technology Context
• Data strategy
follows from
business strategy
and business
objectives
• Data architecture
translates the
conceptual nature of
the data strategy
into a more
implementation-
specific and –
oriented view
January 9, 2023 34
Business
Architecture
Enterprise
Architecture
Required
Operational
Business
Solutions
Business
Solution
Analysis and
Design/
Selection
Business IT
Strategy
IT Function
Strategy
Digital
Strategy
Digital IT
Architecture
Data
Strategy
Data
Architecture
Business
Objectives

35. Data Architecture
• A Data Architecture
exists to support the
objectives and the
operations of the
organisation
• This includes enabling
individual functional
solutions to be
designed and
implemented in
accordance with the
wider organisation data
architecture
January 9, 2023 35
Organisation Data
Architecture
Data
Infrastructure
Tools and
Facilities
Functional
Solutions
Data Standards

36. Data Architecture Structure
• For each set of subject arears within the data architecture
design and specification process, create an activity
breakdown based on the phases
− Research, Design, Define, Plan
− Implement, Operate
− Administer, Manage, Monitor, Improve
• Data architecture cannot be separated from its
implementation, operation and subsequent measurement
and improvement
• Architecture without execution and employment is
incomplete
January 9, 2023 36

37. Data Architecture Evolution And Development
• The data architecture is not static – it must be responsive to and accommodating of change
• It needs to evolve and develop in response to:
− Changing organisation needs and direction, driven by internal and/or external demands
− Changing organisation business strategy
− New technologies and capabilities that the organisation can usefully avail of
− Experience from implementation and operation
• The architecture should embed within itself explicitly the ability to assess its implementation
and operation and to grow, change, improve in response to these factors
January 9, 2023 37
Research,
Design,
Define,
Plan
Implement,
Operate
Administer,
Manage,
Monitor,
Improve
Define
Measurement
Framework,
Results and
Performance
Indicators
Review Delivery
and Operation
of Architecture
Experience and
Lessons from
Implementation
and Operation
Changing
Organisation
Needs and
Direction
Changes to
Organisation
Business
Strategy
New Data
Technologies
and
Capabilities
Data
Architecture
Changes

38. Data Architecture Subject Areas
Data
Architecture
Data Architecture Overall data architecture and data technology standards and design and implement data infrastructural
technology solutions
Data Management, Governance,
Supporting Processes
Standards, processes and their enforcement, planning, supervision, control and usage of data resources and
the design and implementation of data management processes, data ownership
Data Infrastructure, Storage and
Operations Software, Hardware and
Processes
Infrastructure hardware and software required to store and provide access to data, either on-premises or
hosted and facilities and processes required to operate and support the infrastructure, approach to analysis,
design, implementation, testing, deployment, maintenance and data storage structures
Data Security, Protection, Compliance,
Access Control, Authentication,
Authorisation
Approach to ensuring data security and protection, designing and implementing data security model covering
data, tools and infrastructure, ensuring compliance with regulatory standards, controlling access to data,
designing and implementing data authorisation model
Data Integration, Access, Flow, Exchange,
Transfer, Transformation, Load And
Extract
Data resource integration, extraction, transformation, movement, delivery, replication, transfer, sharing,
federation, virtualisation and operational support and approach to implementing a common approach and
providing a common set of tools
Content, Unstructured Data, Records and
Document Management
Approach to the implementation and management of acquisition, storage, indexing of and access to
unstructured data resources such as files and digitised paper records and the integration of these resources
with structured data resources
Master and Reference Data Management
Approach to the implementation and management of master versions of shared data resources to reduce
redundancy and maintain data quality through standardised data definitions and use of common data lookup
values including data dictionaries
Data Warehouse, Data Marts, Data Lakes Facilities for storing data extracted from operational systems for long-term storage and to enable access for
reporting and analysis
Data Reporting and Analytics,
Visualisation Tools and Facilities
Approach to providing a common approach and providing a common set of tools, facilities and supporting
technologies and standards for data reporting, decision support, analysis and visualisation
Data Discovery, Analysis, Design and
Modelling
Approach to the implementation and management of data description standards and the collection,
categorisation, maintenance, integration, application, use and management of data descriptions including
data catalogs
External Data Sources and Interacting
Parties Data Transfer/Exchange/
Integration/Publication
Management of data sources and targets outside the organisation and the parties that provide that data or to
whim the data is made available including contracts and agreement, service levels, access approaches
Metadata Data Management
Approach to the implementation and management of data description standards and the collection,
categorisation, maintenance, integration, application, use and management of data descriptions including
data catalogs
Data Quality Designing, implementing and operating approach, processes and standards to ensure and maintain data
quality
Data Solution Design Defining and implementing standards relating to the use of data within solutions
January 9, 2023 38

39. Data Architecture Subject Areas
• This is intended to represent a comprehensive view of data
architecture
January 9, 2023 39

40. Data Architecture Subject Areas
• The proposed subject areas do not exist in isolation
• They are interrelated areas on which to focus analysis,
planning and design effort and attention while maintaining
a higher level and more complete and integrated view
• The individual topics allow each subject area to be
analysed and specified in detail that is appropriate for the
organisation
• The topics are designed to be independent of any specific
hardware, software or platform technology
January 9, 2023 40

41. Relationships Between Data Architecture Topics
January 9, 2023 41
Data Solution Design
Data Quality
External Data Sources and
Interacting Parties Data
Transfer/Exchange/
Integration/Publication
Data Discovery, Analysis,
Design and Modelling
Data Infrastructure, Storage
and Operations Software,
Hardware and Processes
Data Security, Protection,
Compliance, Access Control,
Authentication,
Authorisation
Content, Unstructured Data,
Records and Document
Management
Master and Reference Data
Management
Data Architecture
Data Management,
Governance, Supporting
Processes
Data Reporting and
Analytics, Visualisation
Tools and Facilities
Data Warehouse, Data
Marts, Data Lakes
Metadata Data
Management
Data Integration, Access,
Flow, Exchange, Transfer,
Transformation, Load And
Extract

42. Data Architecture Topic Scope
Data Architecture
Research, Design, Define, Plan
Data Architecture Strategy and Scope
Definition
Data Architecture Capability Establishment
Define Current Data Architecture Baseline,
Inventory, Gaps, Issues, Concerns
Define Architecture Supporting Tools and
Processes Definition
Data Architecture Scope and Activities
Data Architecture Strategy and Scope
Definition
Data Architecture Capability Establishment
Define Current Data Architecture Baseline,
Inventory, Gaps, Issues, Concerns
Define Architecture Supporting Tools and
Processes Definition
Data Architecture Scope and Activities
Data Architecture Implementation Planning
Implement, Operate
Data Architecture Supporting Tools and
Processes Implementation and Operation
Data Architecture Team Formation
Data Architecture Implementation
Data Architecture Performance and Results
Indicators and Measurement Framework
Definition
Administer, Manage, Monitor, Improve
Data Architecture Review and Improvement
Data Architecture Management
Data Architecture Operation Assessment
January 9, 2023 42

43. Data Management, Governance, Supporting
Processes Topic Scope
Data Management, Governance,
Supporting Processes
Research, Design, Define, Plan
Data Governance Capability Establishment
Define Governance Strategy
Define Current Data Governance Baseline
Define Governance Supporting Tools and Processes
Definition
Data Governance Scope and Activities
Define Governance Policies Define Governance Standards
Define Governance Compliance, Monitoring and
Reporting Data Persistence Standards
Data Lifecycle Definition and Management Create Data Asset Inventory
Create Business Glossary Perform Data Value Assessment
Data Governance Implementation Planning
Data Governance Process Definition
Implement, Operate
Data Governance Supporting Tools and Processes
Implementation and Operation
Data Governance Team Formation
Data Governance Implementation
Data Governance Performance and Results
Indicators and Measurement Framework Definition
Administer, Manage, Monitor, Improve
Data Governance Review and Improvement
Data Governance Management
Data Governance Operation Assessment
Data Governance Implementation and Operation
Reporting
January 9, 2023 43

44. Data Infrastructure, Storage and Operations Software,
Hardware and Processes Topic Scope
Data Infrastructure, Storage and Operations Software,
Hardware and Processes
Research, Design, Define, Plan
Data Infrastructure, Storage and Operations Capability
Establishment
Data Infrastructure and Storage Hardware, Software and
Platform Inventory
Data Operations and Process Inventory
Data Infrastructure, Storage and Operations Existing
Processes and Standards Inventory and Review
Data Infrastructure, Storage and Operations Supporting
Tools and Processes Definition
Data Infrastructure, Storage and Operations Software and
Hardware Scope and Activities
Data Storage Hardware Technology Target Definition Data Storage Software Technology Target Definition
Data Storage Platform Technology Target Definition
Data Product, Platform and Vendor Selection and
Management
Data Backup and Recovery Data Infrastructure Performance Monitoring Tools
Data Archival and Purge Tools
Data Infrastructure, Storage and Operations Availability,
Business Continuity, Disaster Recovery and Replication
Definition
Data Performance Testing and Validation Approach
Data Infrastructure, Storage and Operations Standards
Definition
Data Infrastructure, Storage and Operations Performance
and Capacity Planning Standards and Data Collection and
Analysis
Data Infrastructure, Storage and Operations
Implementation Planning
Implement, Operate
Data Infrastructure, Storage and Operations Supporting
Tools and Processes Supporting Tools and Processes
Implementation and Operation
Data Infrastructure, Storage and Operations Supporting
Tools and Processes Team Formation
Data Infrastructure, Storage and Operations Supporting
Tools and Processes Implementation
Data Infrastructure, Storage and Operations Supporting
Tools and Processes Performance and Results Indicators
and Measurement Framework Definition
Administer, Manage, Monitor, Improve
Data Infrastructure, Storage and Operations Review and
Improvement
Data Infrastructure, Storage and Operations Management
Data Infrastructure, Storage and Operations Operation
Assessment
January 9, 2023 44

45. Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Topic Scope
Data Security, Protection, Compliance, Access
Control, Authentication, Authorisation
Research, Design, Define, Plan
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Capability Establishment
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Existing Approach
Inventory and Baseline
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Supporting Tools and
Processes Definition
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Scope and Activities
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Architecture Definition
Compliance, Regulatory and Data Protection
Requirements Across All Data Types
Security Information, Event and Alert Logging and
Auditing
Data Loss Prevention
Data Security Product, Platform and Vendor Selection
and Management
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Standards Definition
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Monitoring, Data
Collection and Analysis
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Implementation Planning
Implement, Operate
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Supporting Tools and
Processes Implementation and Operation
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Team Formation
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Implementation
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Performance and Results
Indicators and Measurement Framework Definition
Administer, Manage, Monitor, Improve
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Review and Improvement
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Management
Data Security, Protection, Compliance, Access Control,
Authentication, Authorisation Operation Assessment
January 9, 2023 45

46. Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Topic Scope
Data Integration, Access, Flow, Exchange,
Transfer, Transformation, Load And Extract
Research, Design, Define, Plan
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Capability
Establishment
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Existing Approach
Inventory and Baseline
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Supporting Tools and
Processes Definition
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Scope and Activities
Data Integration Security, Authentication, Authorisation
Data Integration Product, Platform and Vendor Selection
and Management
Data Integration Scheduler and Rules Engine
Internal and External Data Sources, Targets and Channels
Definition
Data Integration Development, Testing and Deployment
Data Integration Operations Management,
Administration
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Standards Definition
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Monitoring, Data
Collection and Analysis
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Implementation
Planning
Implement, Operate
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Supporting Tools and
Processes Implementation and Operation
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Team Formation
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Implementation
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Performance and
Results Indicators and Measurement Framework
Definition
Administer, Manage, Monitor, Improve
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Review and
Improvement
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Management
Data Integration, Access, Flow, Exchange, Transfer,
Transformation, Load And Extract Operation Assessment
January 9, 2023 46

47. Content, Unstructured Data, Records and Document
Management Topic Scope
Content, Unstructured Data, Records
and Document Management
Research, Design, Define, Plan
Content, Unstructured Data, Records and Document
Management Capability Establishment
Content, Unstructured Data, Records and Document
Management Existing Approach Inventory and Baseline
Content, Unstructured Data, Records and Document
Management Supporting Tools and Processes Definition
Content, Unstructured Data, Records and Document
Management Scope and Activities
Content, Unstructured Data, Records and Document
Management Security, Authentication, Authorisation
Data Integration Product, Platform and Vendor Selection
and Management
Records Management Strategy Metadata Management
Content, Unstructured Data, Records and Document
Lifecycle Management
Content, Unstructured Data, Records and Document
Management Standards Definition
Content, Unstructured Data, Records and Document
Management Monitoring, Data Collection and Analysis
Content, Unstructured Data, Records and Document
Management Implementation Planning
Implement, Operate
Content, Unstructured Data, Records and Document
Management Supporting Tools and Processes
Implementation and Operation
Content, Unstructured Data, Records and Document
Management Team Formation
Content, Unstructured Data, Records and Document
Management Implementation
Content, Unstructured Data, Records and Document
Management Performance and Results Indicators and
Measurement Framework Definition
Administer, Manage, Monitor, Improve
Content, Unstructured Data, Records and Document
Management Review and Improvement
Content, Unstructured Data, Records and Document
Management Management
Content, Unstructured Data, Records and Document
Management Operation Assessment
January 9, 2023 47

48. Master and Reference Data Management Topic Scope
Master and Reference Data
Management
Research, Design, Define, Plan
Master and Reference Data Management Capability
Establishment
Master and Reference Data Management Existing
Approach Inventory and Baseline
Master and Reference Data Management Supporting
Tools and Processes Definition
Master and Reference Data Management Scope and
Activities
Industry Data Standards Data Glossaries and Taxonomies
Business Rules Analysis and Definition
Master and Reference Data Management Product,
Platform and Vendor Selection and Management
Master Data Stores Reference Data Stores
Master and Reference Data Management Standards
Definition
Master and Reference Data Management Monitoring,
Data Collection and Analysis
Master and Reference Data Management
Implementation Planning
Implement, Operate
Master and Reference Data Management Supporting
Tools and Processes Implementation and Operation
Master and Reference Data Management Team
Formation
Master and Reference Data Management
Implementation
Master and Reference Data Management Performance
and Results Indicators and Measurement Framework
Definition
Administer, Manage, Monitor, Improve
Master and Reference Data Management Review and
Improvement
Master and Reference Data Management Management
Master and Reference Data Management Operation
Assessment
January 9, 2023 48

49. Data Warehouse, Data Marts, Data Lakes Topic Scope
Data Warehouse, Data Marts, Data
Lakes
Research, Design, Define, Plan
Data Warehouse, Data Marts, Data Lakes Capability
Establishment
Data Warehouse, Data Marts, Data Lakes Existing
Approach Inventory and Baseline
Data Warehouse, Data Marts, Data Lakes Supporting
Tools and Processes Definition
Data Warehouse, Data Marts, Data Lakes Scope and
Activities
Data Models Creation Long-term Data Storage Architecture
Data Integration and Population
Data Warehouse, Data Marts, Data Lakes Product,
Platform and Vendor Selection and Management
Data Access Metadata Management
Data Virtualisation
Data Warehouse, Data Marts, Data Lakes Standards
Definition
Data Warehouse, Data Marts, Data Lakes
Monitoring, Data Collection and Analysis
Data Warehouse, Data Marts, Data Lakes
Performance and Capacity Planning Standards and
Data Collection and Analysis
Data Warehouse, Data Marts, Data Lakes
Implementation Planning
Implement, Operate
Data Warehouse, Data Marts, Data Lakes Supporting
Tools and Processes Implementation and Operation
Data Warehouse, Data Marts, Data Lakes Team
Formation
Data Warehouse, Data Marts, Data Lakes
Implementation
Data Warehouse, Data Marts, Data Lakes
Performance and Results Indicators and
Measurement Framework Definition
Administer, Manage, Monitor,
Improve
Data Warehouse, Data Marts, Data Lakes Review
and Improvement
Data Warehouse, Data Marts, Data Lakes
Management
Data Warehouse, Data Marts, Data Lakes Operation
Assessment
January 9, 2023 49

50. Data Reporting and Analytics, Visualisation Tools and
Facilities Topic Scope
Data Reporting and Analytics,
Visualisation Tools and Facilities
Research, Design, Define, Plan
Data Reporting and Analytics, Visualisation Tools
and Facilities Capability Establishment
Data Reporting and Analytics, Visualisation Tools
and Facilities Existing Approach Inventory and
Baseline
Data Reporting and Analytics, Visualisation Tools
and Facilities Supporting Tools and Processes
Definition
Data Reporting and Analytics, Visualisation Tools
and Facilities Scope and Activities
Reporting and Visualisation Architecture and
Approach
Analytics Architecture and Approach
Data Integration, Access and Security
Data Reporting and Analytics, Visualisation Facility
Access and Security
Data Reporting and Analytics, Visualisation Product,
Platform and Vendor Selection and Management
Data Reporting and Analytics, Visualisation
Development, Testing and Deployment
Data Reporting and Analytics, Visualisation
Distribution and Security
Data Reporting and Analytics, Visualisation Tools
and Facilities Standards Definition
Data Reporting and Analytics, Visualisation Tools
and Facilities Monitoring, Data Collection and
Analysis
Data Reporting and Analytics, Visualisation Tools
and Facilities Performance and Capacity Planning
Standards and Data Collection and Analysis
Data Reporting and Analytics, Visualisation Tools
and Facilities Implementation Planning
Implement, Operate
Data Reporting and Analytics, Visualisation Tools
and Facilities Supporting Tools and Processes
Implementation and Operation
Data Reporting and Analytics, Visualisation Tools
and Facilities Team Formation
Data Reporting and Analytics, Visualisation Tools
and Facilities Implementation
Data Reporting and Analytics, Visualisation Tools
and Facilities Performance and Results Indicators
and Measurement Framework Definition
Administer, Manage, Monitor,
Improve
Data Reporting and Analytics, Visualisation Tools
and Facilities Review and Improvement
Data Reporting and Analytics, Visualisation Tools
and Facilities Management
Data Reporting and Analytics, Visualisation Tools
and Facilities Operation Assessment
January 9, 2023 50

51. Data Discovery, Analysis, Design and Modelling Topic
Scope
Data Discovery, Analysis, Design and Modelling
Research, Design, Define, Plan
Data Discovery, Analysis, Design and Modelling
Capability Establishment
Data Discovery, Analysis, Design and Modelling Existing
Approach Inventory and Baseline
Data Discovery, Analysis, Design and Modelling
Supporting Tools and Processes Definition
Data Discovery, Analysis, Design and Modelling Scope
and Activities
Data Modelling Definition Data Profiling Approach Definition
Data Discovery and Profiling Tool Selection and
Implementation
Data Lineage Definition Including Tool Selection
Data Catalog Definition Including Tool Selection Data Dictionary Definition Including Tool Selection
Semantic Layer Definition Including Tool/Platform
Selection
Data Discovery, Analysis, Design and Modelling
Standards Definition
Data Discovery, Analysis, Design and Modelling
Monitoring, Data Collection and Analysis
Data Discovery, Analysis, Design and Modelling
Performance and Capacity Planning Standards and Data
Collection and Analysis
Data Discovery, Analysis, Design and
Modelling Implementation Planning
Implement, Operate
Data Discovery, Analysis, Design and Modelling
Supporting Tools and Processes Implementation and
Operation
Data Discovery, Analysis, Design and Modelling Team
Formation
Data Discovery, Analysis, Design and Modelling
Implementation
Data Discovery, Analysis, Design and Modelling
Performance and Results Indicators and Measurement
Framework Definition
Administer, Manage, Monitor, Improve
Data Discovery, Analysis, Design and Modelling Review
and Improvement
Data Discovery, Analysis, Design and Modelling
Management
Data Discovery, Analysis, Design and Modelling
Operation Assessment
January 9, 2023 51

52. External Data Sources and Interacting Parties Data
Transfer/ Exchange/ Integration/ Publication Topic Scope
External Data Sources and Interacting Parties
Data Transfer/Exchange/Integration/Publication
Research, Design, Define, Plan
External Data Sources and Interacting Parties Capability
Establishment
External Data Sources and Interacting Parties Existing
Approach Inventory and Baseline
External Data Sources and Interacting Parties Supporting
Tools and Processes Definition
External Data Sources and Interacting Parties Scope and
Activities
Data Transfer/Exchange/Integration/Publication
Request Review and Approval Process
Data Transfer/Exchange/Integration/Publication
Implementation Process Definition
Data Transfer/Exchange/Integration/Publication Toolset
Definition and Acquisition
Data Transfer/Exchange/Integration/Publication Activity
Monitoring and Review
Data Transfer/Exchange/Integration/Publication Access
Standards
Data Transfer/Exchange/Integration/Publication Open
Data Approach
Data Transfer/Exchange/Integration/Publication
Security Definition
External Data Sources and Interacting Parties Standards
Definition
External Data Sources and Interacting Parties
Monitoring, Data Collection and Analysis
External Data Sources and Interacting Parties
Implementation
Implement, Operate
External Data Sources and Interacting Parties Supporting
Tools and Processes Implementation and Operation
External Data Sources and Interacting Parties Team
Formation
External Data Sources and Interacting Parties
Implementation
External Data Sources and Interacting Parties
Performance and Results Indicators and Measurement
Framework Definition
Administer, Manage, Monitor, Improve
External Data Sources and Interacting Parties Review
and Improvement
External Data Sources and Interacting Parties
Management
External Data Sources and Interacting Parties Operation
Assessment
January 9, 2023 52

53. Metadata Data Management Topic Scope
Metadata Data Management
Research, Design, Define, Plan
Metadata Data Management Capability
Establishment
Metadata Data Management Existing Approach
Scope Definition, Inventory and Baseline
Metadata Data Management Supporting Tools
and Processes Definition
Metadata Data Management Scope and Activities
Define Metadata Architecture and Approach Metadata Standard Review
Metadata Data Management Standards Definition Define Metadata Management Tools
Metadata Creation and Maintenance Approach Metadata Repositories Definition
Metadata Integration and Usage Approach
Metadata Data Management Monitoring, Data
Collection and Analysis
Metadata Data Management Performance and
Capacity Planning Standards and Data Collection
and Analysis
Metadata Data Management Implementation
Planning
Implement, Operate
Metadata Data Management Supporting Tools
and Processes Implementation and Operation
Metadata Data Management Team Formation
Metadata Data Management Implementation
Metadata Data Management Performance and
Results Indicators and Measurement Framework
Definition
Baseline Metadata Creation
Administer, Manage, Monitor,
Improve
Metadata Data Management Review and
Improvement
Metadata Data Management Management
Metadata Data Management Operation
Assessment
January 9, 2023 53

54. Data Quality Topic Scope
Data Quality
Research, Design, Define, Plan
Data Quality Capability Establishment
Data Quality Existing Approach Inventory, Profile and
Baseline
Data Quality Supporting Tools and Processes Definition
Data Quality Scope and Activities
Data Quality Requirements Data Quality Rules Approach
Data Quality Service Level Management Approach Data Quality Analysis and Reporting
Data Quality Standards Definition
Data Quality Monitoring, Data Collection and Analysis
Data Quality Implementation Planning
Implement, Operate
Data Quality Supporting Tools and Processes
Implementation and Operation
Data Quality Team Formation
Data Quality Implementation
Data Quality Performance and Results Indicators and
Measurement Framework Definition
Administer, Manage, Monitor, Improve
Data Quality Review and Improvement
Data Quality Management
Data Quality Operation Assessment
January 9, 2023 54

55. Data Solution Design Topic Scope
Data Solution Design
Research, Design, Define, Plan
Data Solution Design Advisory Capability
Establishment
Data Solution Design Existing Approach Review,
Inventory and Baseline
Data Solution Design Scope and Activities
Data Management and Governance Standards Data Modelling and Design Standards
Operational and Archival Data Data Storage and Persistence Standards
Data Infrastructure Standards Data Transfer/Exchange/Integration Standards
Data Reporting and Analysis Standards Data Performance and Throughput Standards
Data Security Standards
Data Solution Design Monitoring, Data Collection
and Analysis
Data Solution Design Implementation Planning
Implement, Operate
Data Solution Design Supporting Tools and
Processes Implementation and Operation
Data Solution Design Team Formation
Data Solution Design Implementation
Data Solution Design Performance and Results
Indicators and Measurement Framework
Definition
Administer, Manage, Monitor,
Improve
Data Solution Design Review and Improvement
Data Solution Design Management
Data Infrastructure, Storage and Operations
Operation Assessment
January 9, 2023 55

56. Data Architecture Coverage Of Solution Component
Types
January 9, 2023 56
Solution
Component
Types
Data Architecture Subject Areas
Data
Architecture
Data
Management,
Governance,
Supporting
Processes
Data
Infrastructure,
Storage and
Operations
Software,
Hardware and
Processes
Data Security,
Protection,
Compliance,
Access Control,
Authentication,
Authorisation
Data
Integration,
Access, Flow,
Exchange,
Transfer,
Transformation,
Load And
Extract
Content,
Unstructured
Data, Records
and Document
Management
Master and
Reference Data
Management
Data
Warehouse,
Data Marts,
Data Lakes
Data Reporting
and Analytics,
Visualisation
Tools and
Facilities
Data Discovery,
Analysis,
Design and
Modelling
External Data
Sources and
Interacting
Parties Data
Transfer/
Exchange/
Integration/
Publication
Metadata Data
Management
Data Quality Data Solution
Design
Changes to Existing Systems
X X X X X X X X X X X X
New Custom Developed
Applications X X X X X X X X X X X X
Acquired and Customised
Software Products X X X X X X X X X X X X
System Integrations/ Data
Transfers/ Exchanges X X X
Reporting and Analysis
Facilities X X X X X X X X
Sets of Installation and
Implementation Services
Information Storage
Facilities X
Existing Data Conversions/
Migrations X X X X X X
New Data Loads
X X X X X X
Central, Distributed and
Communications
Infrastructure
X X
Cutover/ Transfer to
Production And Support
Operational Functions and
Processes
Parallel Runs
Enhanced Support/
Hypercare
Sets of Maintenance, Service
Management and Support
Services
Application Hosting and
Management Services
Changes to Existing Business
Processes
New Business Processes
Organisational Changes,
Knowledge Management
Training and Documentation

57. Data Architecture Coverage Of Solution Component
Types
• The data architecture subject areas impact the data
aspects of many solution component types within solutions
• An effective data architecture can contribute to effective
solution architecture and solution data architecture
January 9, 2023 57

58. Data Architecture And Common Data Tooling And
Standards
• Data architecture needs to provide common infrastructural data tools and
common data standards for solutions
− Tools
• Data storage infrastructure – hardware, software and platforms
• Data warehouse platform
• Data reporting, visualisation and analysis
• Data transfer/exchange/integration/extract/transform/load
• Data operations – backup/recovery/replication/business continuity/disaster recovery
• Data anonymisation/pseudonymisation/encryption
• Data monitoring/performance/capacity planning
• Master data management platform
• Reference data platform
• Data catalog/semantic layer
• Document management
• Data analysis
− Standards
• Data security
• Data quality
• Metadata
• Data discovery
• Data modelling
• Data management – classification/retention/archive/deletion
January 9, 2023 58

59. Organisation Data Architecture And Solution Data
Architecture
January 9, 2023 59
Organisation Data
Architecture
Common Data
Standards
Common Data
Infrastructure
Tools and
Facilities
Common Data
Operations
Individual
Solutions
Individual
Solutions
Individual
Solutions
Common Data
Design and
Implementation
Approaches
Common Data
Model

60. Organisation Data Architecture And Solution Data
Architecture
• Common data infrastructure tools allows reuse, reduces
decision-making overhead and delays, reduces cost,
accelerates individual deployment and achieves
standardisation
• Toolset will need to change in response to changing
business needs and technology landscape
January 9, 2023 60

61. Common Data Plumbing Infrastructure
• Data architecture should take on the projects required to deliver the
common data plumbing infrastructure
• These are foundational components
• Individual solution delivery activities should not have to be responsible for
their implementation
January 9, 2023 61

62. Common Data Plumbing Infrastructure
January 9, 2023 62
Common ETL
Common Data
Transfer/
Exchange
Common API
Layer
Common API
Layer
Common Data
Infrastructure/
Platform
Common Data
Warehouse
Common Data
Reporting
Common Data
Analytics
Common Data
Backup and
Recovery
Common
Business
Continuity and
Disaster Recovery
Common
Document
Management
Common Data
Analytics
Common
Performance
Monitoring and
Capacity Planning
Common Audit
Logging Data
Management
Common Data
Catalog
Common
Reference and
Master Data
Individual
Solutions

63. Common Data Plumbing Infrastructure
• This ideal is regularly not fully in place
• Individual solutions often have to implement some of
these capabilities that are not available centrally
• The leads to sub-optimal solutions with point resolutions
to specific requirements
January 9, 2023 63

64. Data Design And Modelling For Solutions
• The objective of data design for solutions is the same as
that for overall solution design:
− To capture sufficient information to enable the solution design to
be implemented
− To unambiguously define the data requirements of the solution
and to confirm and agree those requirements with the target
solution consumers
− To ensure that the implemented solution meets the requirements
of the solution consumers and that no deviations have taken place
during the solution implementation journey
January 9, 2023 64

65. Why Pay Attention To Solution Data Architecture?
• Solution data architecture avoids problems with solution
operation and use:
− Poor and inconsistent data quality
− Poor performance, throughput, response times and scalability
• Poorly designed data structures can lead to long data update times leading to
long response times, affecting solution usability, loss of productivity and
transaction abandonment
− Poor reporting and analysis
− Poor data integration
− Poor solution serviceability and maintainability
− Manual workarounds for data integration, data extract for reporting and
analysis
• Data-design-related solution problems frequently become
evident and manifest themselves only after the solution goes
live
• The benefits of solution data architecture are not always
evident initially
January 9, 2023 65

66. New Technology And Impact on Solution Data
Architecture
• New solution deployment and operating models affects
solution data architecture
− New solution design, deployment and operating models
− Greater use of platform-based solution implementation and
deployment
− Wider range of complex data technology options, especially in
terms of data analysis
− Distributed solution components, distributed solution consumer
base, distributed access with many interfaces, integration points
and data flows
− Complexity with multiple data integrations
January 9, 2023 66

67. Solution Design – From …
January 9, 2023 67
Solution
Central Data
Store
Solution
Central
Application
Component
Solution API
Solution
Central
Infrastructure
Solution
Hosted
Infrastructure
Solution
Internal
Consumers
Solution
External
Private
Consumers
Solution
Hosted Data
Store
Solution
Hosted
Application
Component
Solution
Hosted
Analytics
Access and
Security
Infrastructure
Central To
Hosting
Facility
Connectivity
Solution
External Public
Consumers
Solution
Mobile App

68. To …
• Increasing solution landscape complexity and diversity gives rise to greater data
design complexity
January 9, 2023 68
Solution
Central Data
Store
Solution
Central
Application
Component
Solution API
Solution
Central
Infrastructure
Solution
Hosted
Infrastructure
Solution
Internal
Consumers
Solution
External
Private
Consumers
Solution
Hosted Data
Store
Solution
Hosted
Application
Component
Solution
Hosted
Analytics
Access and
Security
Infrastructure
Central To
Hosting
Facility
Connectivity
Solution
External Public
Consumers
Solution
Mobile App

69. Solution Entity Model
January 9, 2023 69
Solution
Component
Types
Solution
Components
Solution
Solution Zones
Solution
Zone Types
Solution
Topology
Solution Consists Of
Multiple Components
Each Solution
Component
Has A Type
Solution Exists
Within A
Topology Of
Many Solutions
Solution Components
Are Located In Solution
Zones
Each Solution
Zone Has A Type
Solution
Operational
Entity
Solution
Operational
Entity Type
Deployed
Solution
Consists Of
Multiple
Operational
Entities
Each Solution
Operational
Entity Has A Type
Solution Operational Entities
Are Located In Solution Zones
Some Solution
Components
Become
Deployed
Operational
Entities

70. Solution Zone Types and Zones
January 9, 2023 70
Solution
Component
Types
Solution
Components
Solution
Solution Zones
Solution
Zone Types
Solution
Topology
Solution Consists Of
Multiple Components
Each Solution
Component
Has A Type
Solution Exists
Within A
Topology Of
Many Solutions
Solution Components
Are Located In Solution
Zones
Each Solution
Zone Has A Type
Solution
Operational
Entity
Solution
Operational
Entity Type
Deployed
Solution
Consists Of
Multiple
Operational
Entities
Each Solution
Operational
Entity Has A Type
Solution Operational Entities
Are Located In Solution Zones
Some Solution
Components
Become
Deployed
Operational
Entities

71. Solution Zones
• Solution zones are locations where groups of closely related solution
components reside
• They represent containers for solution components
• Zones are located within the wider physical solution landscape
• Each zone and the components it holds have different security
requirements
• Not all solutions will have components in all zone and not all
organisations will have all the zone types
• The solution and its constituent components can span multiple
different zones of the same type
• The zone approach is useful way of representing the entirety of a
solution, its constituent components, their connectivity, linkages and
interactions, especially data storage, processing and interactions
• You will have different levels of control over different solution zones
(including no control) – this impacts data design considerations
January 9, 2023 71

72. Sample Solution Zone Types
January 9, 2023 72

73. Sample Solution Zone Types
January 9, 2023 73

74. Sample Solution Zone Types
Zone Description
Insecure External Organisation
Presentation And Access
Where publicly accessible or accessing entities reside. These entities are regarded
as insecure and/or untrusted.
Secure External Organisation
Participation and Collaboration
Outside the physical organisation boundary where entities that are provided by or
to trusted external parties reside
Secure External Organisation Access Contain entities that enable secure access or are securely accessible from outside
the organisation
Organisation Contain the entities within the organisation boundary and contains all the
locations, business units and functions within it
Central Solutions and Access Contains the solution entities and their data
Solution Zone Contains the solution entities
Data Zone Zone within the organisation where data is segregated for security
Remote Business Unit Solutions and
Access
Remotely located organisation business unit or location and the entities it
contains
Workstation Zone Zone within the organisation where users accessing data and solutions are
segregated for security
Outsourced Service Provider Solutions
and Access
Contains solutions provided by and located in facilities provided by outsourced
partners
Cloud Service Provider Solutions and
Access
Contains solutions - platform, infrastructure and service - provided by and located
in cloud service providers
Co-Located Solutions and Access Contains solutions the organisation has located in facilities provided by co-
location providers
January 9, 2023 74

75. Solution Consumers
• Any solution with have different sets of consumers of different
types:
− Controlled Consumers – typically organisation personnel over whom the
solution owner has substantial control
− Partially Controlled Consumers – typically external business partners
and other interacting parties over whom the solution owner has some
control and influence
− Uncontrolled Consumers – typically members of the public at whom the
solution is targeted and whose needs must be inferred through groups
of proxy consumers
• Each consumer type will have different data-related needs and
expectations
• Classifying and understanding the target solution consumers
will contribute to solution data architecture design
January 9, 2023 75

76. Data Design And Modelling For Solutions Activities
Data
Modelling
Data Journeys
Design
Data
Processing
Design
January 9, 2023 76
Logical
Data Model
Physical Data
Model
Conceptual
Data Model

77. Data Design And Modelling For Solutions Activities
• This includes the following activities:
− Data Modelling – define the data entities, structures, attributes
and contents
• Conceptual Data Model (CDM) – create an initial high-level view of solution
• Logical Data Model (LDM) – expand the CDM with detailed data
requirements
• Physical Data Model (PDM) – translate the LDM into implementation-
specific details
− Data Journeys Design – create an inventory of data journeys and
identify the steps within the journeys and the data entities
involved
− Data Processing Design – define the detail of the processing
performed on data entities
January 9, 2023 77

78. Data Design And Modelling For Solutions Activities
• The data analysis and design activities are not linear or
sequential
• As the analysis progresses, earlier work may need to be
revisited to be elaborated and expanded on
January 9, 2023 78
Conceptual Data
Model
Logical Data
Model
Physical Data
Model
Data Journeys
Design
Data Processing
Design
Data Modelling

79. Packaged Solutions And Platforms
• Packaged solution components and platforms on which
solutions components are implemented and deployed will
have pre-defined data models with a greater or lesser
degree of configuration and customisation
• The data design activities should still be performed for
these solution components
• The inherent data limitations and restrictions of the
packages and platforms should be clearly defined and
understood
January 9, 2023 79

80. Solutions And Shared/Private Data
January 9, 2023 80
• Solutions and
their
components
within the
organisation
solution
landscape will
have both local
data and data
that is shared
with or between
other solutions,
either for
upstream/
downstream
processing or as
shared data
repositories

81. Solutions And Shared/Private Data
• Data design activities are different for shared and private
solution data
• Shared solution data includes reference and master data
− Reference data consists of common code, structural and identifier
values
• Their purpose is to ensure consistency across data values
• Reference data is static or slowly changing
− Master data relates to common transaction identifiers such interacting
parties (customer, partner, etc,) details
• Having a single version of master data ensures single view of all interactions with
the party across the organisation can be identified
• Solution data modelling activities should identify the
occurrences of reference and master data to maximise reuse
and contribute to maintaining a single version of the truth
January 9, 2023 81

82. Shared/Common Data Issues
• Because shared data is shared, the main concerns and
issues relate to:
− Ownership – who owns and is responsible
− Maintenance – who maintains it and keeps it current, who is
responsible for allowing updates, what is the process for applying
updates and changes
− Quality – who is responsible for maintaining data quality
• Individual solutions should not have to solve these
problems, but commonly and unfortunately have to
January 9, 2023 82

83. Common Data Formats
• Master data will have a common format
• Individual solutions need to adhere to the common master
data format
January 9, 2023 83

84. Data Modelling – Conceptual Data Model
• The Conceptual Data Model (CDM) represents concepts,
entities and their relationships within the scope of the
solution
• It is used to create a common understanding among all the
solution stakeholders
• The CDM defines the scope of the solution
• The functional requirements of the solution provide an
input to the CDM and its constituent entities
January 9, 2023 84

85. Data Modelling – Logical Data Model
• The Logical Data Model (LDM) expands on the agreed CDM
• Detailed data requirements for the specified data entities
are defined, including solution zones
January 9, 2023 85

86. Data Modelling – Physical Data Model
• The Physical Data Model (PDM) translates the LDM into
technology-specific implementation details and a
technology structure across the solution zones
• The LDM may be updated to reflect and accommodate
technology and platform specific features, limitations,
capabilities and restrictions
January 9, 2023 86

87. Data Journeys Design
• Solutions have journeys as consumers use the solution to
achieve results
• Solution journeys reflect solution consumer experiences
• Data journeys represent the data exchanges and transfers
that occur to support the solution journeys
• Solution data architecture should first create an inventory
of data journeys
• The processing data journeys can then be expanded to
reflect the lifecycle for the data type(s) associated with the
data journeys
January 9, 2023 87

88. Data Processing Design
• Data processing design describes the detailed processing
that is performed on data
• It defines the business rules that are applied to data within
the scope of the solution
January 9, 2023 88

89. Data Processing Design
Identify the
data
processing
performed on
data entities
and objects
by each
solution
component
for each
solution
journeys
January 9, 2023 89
Data
Entity/Object
Solution
Component
Data
Entity/Object
Solution
Component
Solution
Component
Solution
Component
Solution
Component
Solution
Component
Solution
Component
Solution
Component
Data
Entity/Object
Data
Entity/Object
Data
Entity/Object
Data
Entity/Object
Data
Entity/Object
Data
Entity/Object
D1
D2
D3
D4 D5

90. Summary
• The data architecture of solutions is frequently not given the attention it deserves or needs
• Frequently, too little attention is paid to designing and specifying the data architecture within
individual solutions and their constituent components
• This is due to the behaviours of both solution architects ad data architects
• Solution architecture tends to concern itself with functional, technology and software
components of the solution
• Data architecture tends not to get involved with the data aspects of technology solutions,
leaving a data architecture gap
• Combined with the gap where data architecture tends not to get involved with the data aspects
of technology solutions, there is also frequently a solution architecture data gap
• Solution architecture also frequently omits the detail of data aspects of solutions leading to a
solution data architecture gap
• These gaps result in a data blind spot for the organisation
• Data architecture tends to concern itself with post-individual solutions
• Data architecture needs to shift left into the domain of solutions and their data and more
actively engage with the data dimensions of individual solutions
• Data architecture can provide the lead in sealing these data gaps through a shift-left of its scope
and activities as well providing standards and common data tooling for solution data
architecture
January 9, 2023 90

91. More Information
Alan McSweeney
http://ie.linkedin.com/in/alanmcsweeney
https://www.researchgate.net/profile/Alan-Mcsweeney
https://www.amazon.com/dp/1797567616
9 January 2023 91