Cut the time of application delivery by reusing Kafka data structure between projects! Expecting boundaries and data definitions to remain consistent between source and consuming projects can be a constant source of surprise - a Kafka spiderweb. Duplicate datasets with mutations often bring with them monetary, opportunity and reputation costs, as well as errors, inconsistencies, tech debt, reactive approach to data problems, audit issues, data ownership confusion and data that is not fit for purpose. Solving this requires moving towards uniform datasets. And this moves requires an Enterprise Domain Driven Design approach, combined with Event Storming. Doing this architecture work upfront allows for real-time, ubiquitous, distributed data implementations, as opposed to a Kafka spiderweb. This talk and demo will show the design, along with a demo illustrating this approach.

1. northerntrust.com | © 2021 Northern Trust Corporation
ENTERPRISE
Domain Driven Design for Realtime,
Ubiquitous, Distributed Data
PRESENTED BY:
Laura Schornack Maureen Penzenik
Kafka Summit 2021:
https://sessionize.com/kafka-summit-eu-2021

2. northerntrust.com | © 2021 Northern Trust Corporation
Introduction
Laura Schornack
Laura Schornack champions Domain
Driven Design whenever possible. She
holds a degree in computer science
from the University of Illinois at
Urbana-Champaign. She holds four
patents. Laura presents frequently and
recently presented at Kubecon. She is
published through O'Reilly’s Katacoda
Laura volunteers with “Girls who
Code”, “Coder Dojo”, “Think Chicago”,
and “City Scholars”.
Maureen Penzenik
Maureen Penzenik has been in
Information Technology for over
25 years with experience in applying
Data Warehousing and Business
Intelligence methods and technologies.
Maureen holds a B.S. degree in Business
Administration and an MBA in
Management Information Systems.

3. northerntrust.com | © 2021 Northern Trust Corporation
Topics
• Summary of Problem
• Data Industry Past Attempts and Why They Were Flawed
• Proposed Solution
• What Is Domain Driven Design (DDD) and Enterprise DDD
• Leveraging Governance to Take the Leap from DDD to EDDD
• DDD And EDDD Example
• EDDD Example Applied in a Distributed Manner
• CQRS with Operational Systems
• Differences Between Data and Responsibility Segregation for
Applications and for Data/Analytics Environments
Agenda

4. northerntrust.com | © 2021 Northern Trust Corporation
Desired Outcome
• Cut the time of application delivery by not redoing all of the data work in every project.
Problem
• Data cannot be sourced from and created in different applications, with the expectation that the boundaries
are the same.
• Inconsistent sourcing and creation of duplicate datasets has Monetary, Opportunity, and Reputational costs:
— Errors
— Inconsistency
— Exponential technical debt
— Reactive approach to data problems
— Audit issues: unknown authentic source and lack of identified data owners.
— Data Owners are not included in decisions on how data is used, assuming the same data has
consistent definitions.
— Duplicate data being consumed that is not fit for purpose.
• Individual applications determining business definitions and boundaries for datasets – lack of uniformity.
• Governance in managing data sets.
Problem

5. northerntrust.com | © 2021 Northern Trust Corporation
Not Ideal
Problem
Ideal
Inconsistent Data Sourcing
AB
B
BC
BD
A, B, B
A, B, C
A, B, B, D
A, B, B
A
B
C
D
A, B, C, D
A, B, C, D
A, B, C, D
A, B, C, D
A, B, C, D

6. northerntrust.com | © 2021 Northern Trust Corporation
Problem
Inconsistent Data Derivation
A, B
A, B
C=A + B
C
C
C
C = C = C
A, B
A, B
C=A + B
C = A + D
C = A + E
C = A + F
C!= C!= C!= C
Not Ideal Ideal

7. northerntrust.com | © 2021 Northern Trust Corporation
History of Data Segregation within Apps
Order
Microservice
Order
Database
Customer
Microservice
Customer
Database
HTTP
Order DTO Order DTO
Customer
DTO
Microservices with
Dedicated Database
Event
Customer
Event
Publisher
Order
Microservice
Customer
Microservice
Order
Database
Customer
Database
Messaging
Microservices with Dedicated Database
Shared Database
N..1
Customer
Table
Order
Table
Order
Microservice
Customer
Microservice
Microservices with
Shared Database

8. northerntrust.com | © 2021 Northern Trust Corporation
CQRS for Operational Systems
Realtime, Ubiquitous, Distributed Data Architecture Achieved through DDD
and delivered through Kafka
Microservices with CQRS and Event Sourcing
Event Store Event Store
Event
Customer
Event
Customer
Event
Publisher
Order Microservice
Command
Service
Query
Service
Query
Model
Command
Endpoint
Query
Endpoint
Customer Microservice
Query
Service
Query
Model
Command
Endpoint
Query
Endpoint
Messaging System
Command
Model
Event
Publisher
Read
Storage
Read
Storage
Command
Service
Command
Model

9. northerntrust.com | © 2021 Northern Trust Corporation
Use Case Table for a Hotel Booking System
DDD for a Single Application – Generic Example
Bounded Contexts/Data Sets
ACTOR USE CASE INPUT OUTPUT
Customer Search Hotels Location, Date, Range,
Room Rate. Facilities
Hotels, Availability
View Hotel Info Hotel Facilities, Room
Specifications
View Rates Hotel, Room Specifications,
Date Range
Room Rates
Make a Reservation Hotel, Room specification,
Date Range, Customer,
Customer Card Details
Reservation, Room
availability
Search for a Reservation Reservation ID Reservation
Cancel a Reservation Reservation ID Confirmation Number
View Reservation Reservation ID Reservation Details
Edit Reservation Reservation Confirmation
Number
New Confirmation Number
Create Rewards Account Customer ID Rewards Info
Create Account Customer Info Customer Object ID
Hotel Manager Manage Hotel Reservation Reservation ID Reservation
Card Vendor Authorize Card Customer Card Details Card Authorization
Customer Hotel Card / Reservation
Customer
<RO>
Points
<E>
Photographs
<VO>
Hotel
<RO>
Local POI
<VO>
Room Type
<VO>
Rate
<VO>
Location
<VO>
Facilities
<VO>
Hotel
<E>
Reservation
<E><RO>
Date Range
<VO>
Room Rate
<VO>
Room
Specification
<VO>
Customer ID
<E>
Payment
<VO>
Authorization
<E>

10. northerntrust.com | © 2021 Northern Trust Corporation
DDD merged with Event Storming
Use Case Merged with Event Storming Table
ACTOR USE CASE INPUT OUTPUT EVENT COMMAND QUERY
Customer Create Customer Customer Info Customer Customer Created Create Customer
Edit Customer Customer Customer Customer Updated Update Customer
Make a Reservation Hotel, Room specification, Date Range,
Customer, Customer Card Details
Reservation, Room
availability
Room Reservation Requested,
Card Authorization Requested,
Payment Processed, Payment
Processing Failed, Reservation
Succeeded, Reservation Failed
Reserve Hotel
Search for a Reservation Reservation ID Reservation No event Search Reservation
Cancel a Reservation Reservation ID Confirmation Number Room Cancelled Cancel Room
View Reservation Reservation ID Reservation Details No event View Reservation
Edit Reservation Reservation Confirmation Number, Reservation New Confirmation Number Room Reservation Updated Update Reservation
Create Rewards Account Customer ID Rewards Info Rewards Account Created Create Rewards Account
Send Email Customer, email content Status Send Email
Hotel Manager Manage Hotel Hotel Info Hotel Hotel Added Add Hotel
Manage Hotel Hotel Hotel Hotel Updated Update Hotel
Manage Hotel Hotel Hotel Hotel Deleted Delete Hotel
Manage Hotel Hotel ID, Picture Status Add Picture
Manage Hotel Hotel ID, Picture ID Status Delete Picture
Search Hotels Location, Date, Range, Room Rate.
Requirements
Hotels, Availability no event Search Hotel
View Hotel Info Hotel Facilities, Room
Specifications
no event View Hotel
Information
View Rates Hotel, Room Specifications, Date Range Room Rates no event View Hotel Rates
Card Vendor Authorize Card Customer Card Details Card Authorization Card Authorized
Card Authorization Failed
Authorize Credit Card
Issue Refund Customer Card Details Card Authorization Card Refund Authorized
Card Refund Authorization Failed
Refund Credit Card

11. northerntrust.com | © 2021 Northern Trust Corporation
Eventing Diagram
Flow of Events
Reserve Hotel Action Flow
Customer
Domain
Reservation
Domain
Reserve
hotel
command
Card
Vendor
Domain
Card
authorization
requested
event
Process
payment
event
Room
reservation
success event
Room
reservation
requested
event
Hotel
Mgmt.
Domain
1 2
3
4
5
6
7
8 8

12. northerntrust.com | © 2021 Northern Trust Corporation
Enterprise Domain Driven Design:
• Ubiquitous, Distributed Data across many systems working in concert
• Governance: Glossary that logs data set and metadata created by DDD
• Each bounded context should be listed along with its metadata:
— Lineage
— Business Data Owner
— Data Steward
— Technical Owner
— Interoperability
— Distributions
• Operational System needs
• Analytics systems needs
Enterprise Domain Driven Design
How to apply DDD to the Enterprise – Reuse of Datasets

13. northerntrust.com | © 2021 Northern Trust Corporation
Key
Using Data Sets in your Analytics Ecosystem
Operational vs Data/Analytics Ecosystems
Operational
Data Sources
Downstream Consumers
of Data Products
Delivery
Compute
Data Set Data Set Data Set
Data
Functionality
App
Data Mesh

14. northerntrust.com | © 2021 Northern Trust Corporation
Monoliths
• Centralized data ownership within tech-focused,
specialized team.
• Highly-coupled data pipelines; difficult to change.
Monoliths vs Data Mesh
Data Mesh*
• Domain-oriented data
• “Data as a product”
• Self-service data infrastructure
• Standards and governance
*Reference: Zhamak Dehghani “How to Move Beyond a Monolithic Data Lake to a Distributed Data Mesh”;
https://martinfowler.com/articles/data-monolith-to-mesh.html

15. northerntrust.com | © 2021 Northern Trust Corporation
• Advancing Domain Driven Design for one application into Enterprise
Domain Driven Design.
• With Enterprise Domain Driven Design, we can merge with the data side of
the house.
• Domain Driven Design for the data warehouse / analytics environment.
• Unify operational and analytical worlds around business-driven data
domains.
Recap

16. northerntrust.com | © 2021 Northern Trust Corporation
Questions?
16

17. northerntrust.com | © 2021 Northern Trust Corporation