If you have ever been frustrated with your data dictionaries getting stale, you can adopt an API-driven development approach to obtain data definitions from the true, authoritative source in a fully automated fashion that remains current in real-time. This approach uses a variety of free, open source tools and some novel workforce development approaches to tackle this difficult issue. If you would like me to come speak to your business about this method please contact me directly.

1. Automating
Data Dictionaries
By Jared Nielsen

2. There is never enough time
As organizations acquire other companies, grow in the market, race
to compete, and implement new applications in an ever-faster pace,
the precise knowledge of what data lies where and for what purpose
is rarely current or up-to-date.
Standard practices for how to compile the de-facto master data
dictionary essentially become:
• Slow down
• Do a discovery
• Document the work
• Come to agreement
• Enforce
This structure can run counter to core business growth objectives
which results in a tendency by management to bypass them.

3. Who knows first?
There are two tracks to analyze here. New applications and Legacy
applications.
New Applications
1. Product is concepted
2. Database is created
3. APIs + rules are generated
4. Screens are designed
5. Functionality is implemented
6. Documentation is generated
7. Application is enhanced
Legacy Applications
1. Product is discovered
2. APIs + rules are generated
3. Documentation is updated
4. Application is enhanced
It is my proposal that the first person to know the actual, detailed
rules about the data is the API developer who implements the rules
for that data.

4. Why target the API layer?
Every organization has legacy applications. The core asset in those
applications is the data itself. Where in the past you would have to
modernize the entire app, with API technologies, you can leave a
legacy app in operation while plugging in APIs to the side of the
database for that application.
New applications should natively rely on API-driven development so
it is a given that they will be implemented for new apps.
Because the API developer must understand the data to implement
the rules, they become the first and authoritative source of the data
dictionary information that their APIs use.
The application needs an “endpoint” to hit that API, which Swagger
provides. The data governance team needs documentation of what
that data really does and needs it to be current. Because these
endpoints are used they always are the most current definition.

5. Data definition as a database
Defining data can be as large as a complex MDM initiative or as
small as a pile of Excel spreadsheets that become woefully out of
date. I have seen $10mm MDM initiatives collapse under their own
weight, and I’ve seen piles of Excel documents wither and wilt from
lack of use. The reason is they either impede business goals or are
irrelevant and unused.
Used data is good data
The information in data definitions gets stale. When a programmer
changes a feature in an API, they will document their code, but they
will rarely dig up documentation and update it. By using a
combination of Code Comments, Reflection, and Swagger Auto
Documentation, the data definitions themselves become a “data
dictionary database” automatically. Because they are generated “on
the fly” they are always current. Because they are used “live” they
must remain current and updated. The definition *is* a database.

6. Auto data dictionary generation
This process flow illustrates the model for using APIs to originate and
surface data dictionary definitions in a way that is always current and
automated.
DB POCO DTO Rules Swagger
Endpoint
App
Classes
(nSwag)
Service
Layer
UI
Automatic ETL
Sweep
Master Data
Definition
Automatic Documentation Generator

7. Effective method for automated data governance
While this is an oversimplification, if your organization tackles data
governance using this method, it will be able to automate and
emerge the current data with minimal impact to core business growth
objectives:
• Create APIs for all data so it is surfaced and usable outside the
confines of the user application screens
• Centralize business rules and validations at the API level
• Document the data and the rules in the source code
• Use reflection to harvest those comments into a strongly typed
definition (JSON)
• The documented endpoints should become *the* source of data
for applications, documentation, and MDM definitions
I believe that this will have a transformative effect on your
organization.

8. For More Information
Jared Nielsen
(713) 568-4599
Linkedin.com/in/nielsendata
Swagger: http://swagger.io
nSwag: https://github.com/NSwag/NSwag
DataAnnotations: https://goo.gl/YUWuKP
WebAPI AutoDoc: https://goo.gl/4gKCYp