The document outlines the various data storage and processing systems at the University of Waterloo, including operational, dimensional, non-relational, and graph data stores. It highlights the role of Information and Integration Management (IIM) and Institutional Analysis and Planning (IAP) in managing data for university operations and decision-making. Furthermore, it emphasizes the importance of metadata management, data governance, and the use of cognitive processing to enhance data utility.

1. Information and Integration Management (IIM)Colin Bell <cpbell@uwaterloo.ca> - December 2016
Operational Data Stores
Relational databases created by extracting,
transforming, and loading data from operational
systems into restructured logical data models.
Dimensional Data Stores
Relational databases created by extracting,
transforming, and loading data conformed to a star
schema from operational data stores.
Non-Relational Data Stores
Data storage platforms that allow arbitrary data to
be stored. Data can usually be paired with
metadata.
Graph Data Stores
Data storage platforms to capture sets of nodes
and their relationships.
Distributed Data Processing
Data processing platforms allow data processing
workloads to be spread between a number of
hosts. Allows large scale search and inquiry.
Distributed Transaction Stores
Data storage for transaction records maintained in
distributed chains of cryptographically signed
blocks.
Data Access, Lineage, and
Provenance
Interfaces that allow access to core data elements,
meaning of those data elements, and associated
lifecycle / origin metadata
Cognitive Processing (AI)
Semantic modelling, natural language, machine
learning, and cognitive systems to support data
processing.
DataManagementPlatform
On-Premise
QUEST
QUEST
DB
PeopleSoft
CS
Waterloo Works
Works
DB
Orbis
Comms.
Request Tracker (RT)
RT DB
Best
Practical RT
Library
Library
DB
Product
Names
System Name
Database
Name
Product
Name
Operational Systems
The systems running on-premise or in the cloud
that support University operations.
Cloud
Finance
Aggresso
DB
Business
World ERP
Learn
Learn DB
Brightspace
Student Email
Office
Cloud
Office 365
Research
Pure DB
Pure
System Name
Database
Name
Product
Name
InformationLifecycleManagement
KeyCampusRelationships
Secretariat (SEC)
Information and Integration Management (IIM) will support SEC’s University
Records Manager and Privacy Officer through metadata management around
records, data requests, and data uses on campus.
Institutional Analysis and Planning (IAP)
The Director, Institutional Analysis and Planning, "designs measures and collects data to
assess University-wide goals and objectives and coordinates the design, implementation
and management of a University-wide decision support data warehouse.”
Information and Integration Management (IIM) will support IAP in this effort as a data broker
and technical support partner. IIM will be focused on providing high quality data feeds from
operational systems to support IAP’s designed measures on University-wide goals and
objectives.
IAP will implement and manage data warehousing for designed measures and official
reporting/analysis purposes. Internal and external governance reporting and strategic
planning will be serviced from IAP’s warehousing. IIM will support IAP by providing access to
knowledge, expertise, tooling, information infrastructure, and captured data elements.
IST-IIM
Data Capture, Cleaning, and
Enrichment
Capability to capture, clean, and enrich disparate
large volume static or streaming data sources.
Make unstructured, semi-structured, and
structured data a valuable asset.
De-identification of Personal
Information
Capability to de-identify, redact, pseudonymize,
and anonymize data. Transform Personally
Identifiable Information (PII) so that it can be used
in analysis without presenting a privacy risk.
Information Cataloging,
Documentation, and Metadata
Management
Capability to catalog data elements, document
them through information modelling, and describe
them with metadata.
Institutional Analysis
and Planning
(IAP)
Secretariat
(SEC)
University Community
Mission
This group exists at the University of Waterloo to provide the knowledge, expertise, and centralized
capability required to make information a valuable asset for the University—to create a trusted
information asset base to support the University.
Vision
If useful information exists, make it a competitive advantage for Waterloo; capture, organize, and make
information accessible for future decision making.
Policy 46
Principles and Practices
Accountability and
Accessibility
Privacy and
Confidentiality
Compliance
Information Quality Information Security
Information Lifecycle
Management
Capture Organize Store Use Disposition
Digital Imaging &
Auto-Uploads
Coding &
Indexing
Backups &
Versioning
Business Process
Workflows
Retention Schedules &
Record Destruction
IndexCatalog
SearchDiscover
Access Uses
Privacy Risk
Document Management
Physical
Document
Digital
Document
Digital
Imaging
Captured Document
Document
Metadata
Coding and
Indexing
Document
Document
Metadata
Business
Processing
Workflows
Retention /
Disposition
Schedule
Record of
Disposition
Case
Delivery
Plan
Case
Case Event Case Activity
Case
Communication
Case
Participant
Case
Output
Case
Outcome
Case
Document
includes
Case
Facility
Case Party
Relationship
Case
Location
Case-Party-Document Model
Content Management
Sharepoint
WCMS
Network Drives
Email Archives
Confluence
InformationandIntegrationManagement
Meaning and Metadata
Physical
Logical
Conceptual
Data Models
Information Models
Semantic Models Ontologies
Glossaries and
Taxonomies
Data Element
Rules and Definitions
Metadata
Management
A robust metadata management capability
and platform is crucial to the University’s
ability to leverage its information assets.
Capture Organize
Storage
Use Disposition
Integration Platform
Current State Integration Future State Integration
Most systems follow a point-to-point
integration pattern.
Data is exchanged through batch copies.
Some exchange is through direct database
links increasing complexity of change
efforts.
Pattern of operations is effective but
inefficient.
Duplication Complexity
Inaccuracy Risk
Dept Sys Dept Sys Dept Sys Dept Sys
Faculty Sys Faculty Sys Faculty Sys Faculty Sys
HR
Student Finance
Research
IDM
ReliabilityScalability
EvolvabilityConsistency
Systems flow requests for information
through a common integration platform.
The platform:
- enforces security,
- creates audit logs for lineage/
provenance needs,
- clarifies interpretation/meaning, and;
- captures/creates metadata.
Reduces risks and improves efficiency.
Student
HR
Finance
IAM
Resear
ch
Dept
Sys
Faculty
Open
Data
Enterprise
Integration
Plaform
InnovationPlatform
User-Centric Design (UCD)
Information Architecture User Experience Design (UX) Platform and Agile
Current State
Future State
X Y
Y
Y
X
Y YY X X
Intrapreneurship
As a
user…
http://bit.ly/2h3OFeG
Institute for Government
System Error: Fixing the Flaws
in Government IT.
User Developer
Co-op Co-op
Custom
Web Apps
JIRA
Confluence
CompellingDataEnvironment
System Name
Database
Name
Product
Name
System Name
Database
Name
Product
Name
System Name
Database
Name
Product
Name
Operational Data
Capture
Real-Time
Robust
Reliable
Operational Storage
Dimensional Storage
Non-Relational Storage
Graph Data Storage
Distributed Processing
Transaction Storage
Access, Lineage, and
Provenance
Cognitive (AI) Processing
Identify
De-Identify
Clean
EnrichIndex
Store
Acceptable Uses
Report
Users
Data
Scientists
Data
Access
"DIKW Pyramid" by Longlivetheux - Own work. Licensed under CC BY-SA 4.0 via Commons
https://commons.wikimedia.org/wiki/File:DIKW_Pyramid.svg#/media/File:DIKW_Pyramid.svg
Unstructured
Information
“Documents”
Semi-Structured
Information
“Content”
Structured
Information
“Data”
Tableau
Jupyter Notebooks
(Python, R, F#)
Sharepoint
(MSBI)
Visualization
Users
Power BI
Reporting Services
SQL Server
Dapper.NET APIs
Ad-Hoc
Inquiry
Users
Excel
Meaning and
Metadata
Information and Integration Management
Data Curation
QlikView
InformationAssetBase
Local Unit
Operational,
Tactical, and
Strategic
Documentation
Information
System
Architecture
Documentation
Internal and
External
Governance
Reporting
Project / Initative
Documentation
Web Content
Service
Documentation
Information System
Logs
Data
Documentation
…
…
http://motivationmodel.com/
Information Asset
Base
Information
Portfolio
Process
Portfolio
Service
Portfolio
HR
Portfolio
Financial
Portfolio Strategy
Portfolio
Technology
Portfolio
Strategic
Information
Stream
Application
Portfolio
Project
Portfolio