This talk given by Cillian Kieran outlined how you engineer DSR for complex distributed systems as given at IAPP PSR, 2022 in Austin, TX -- A summary of privacy engineers, DSAR and data management for distributed data systems

1. How Do You Engineer …
DSAR For Multiple
Profiles Complexity?
Privacy Engineering
@cillian

2. +
Open source privacy engineering platform
~ Free DSR orchestration platform
~ Standard for privacy metadata
~ Privacy labeling built for developers
fid.es/join

3. # DSRs: the cause of complexity
# DSRs: the impacts of complexity
# Architecture for agile DSR at scale
# Recommendations for Engineering DSR
Contents

4. Why are DSRs so painful, slow and costly?

5. A new user’s data is created across systems in seconds

6. A new user’s data is created across systems in seconds
DATA
SOURCES
INTERNAL
STORAGE
BUSINESS
INTELLIGENCE
3RD PARTY
SYSTEMS
NEW USER

7. Deleting the same user’s data can take weeks. Why?

8. REQUEST
INGESTION
SUPPORT
TEAMS
INTERNAL
WORKFLOW
TEAM
PROCESSES
SUBJECT REQUEST
A subject request is processed manually over weeks
DATA
PROCESSING
PRIVACY
WORKFLOW

9. DSRs cost time, money and create risk

10. Engineering teams have perfected the art of data creation
Not the art of data deletion

11. The causes of DSRs Exponential Complexity
# System design prioritizes creation, not deletion, or consolidated access
# Data sprawl increases over time with new technology adoption
# User data structures vary widely
# There is no consistent data labeling convention
# Request types vary (agent, controller, subject)
# Business constraints on what data to process in a request vary widely

12. The impact of DSRs Exponential Complexity
# No data model = no data automation
# Avg. time per request 4 - 80 hours
# Avg. cost per request $1,400
# Creating a resource tax on all business units
# Valuable resources diverted from core business activities
# Certainty of completeness is low

13. Engineering DSR Orchestration for Complexity

14. Our criteria for DSR orchestration
# Deleting a user should be as seamless as creating a user
# DSRs should be easy and free (for users and businesses)
# DSRs should be scalable and a core feature of systems
# Product and technology innovation should not break DSRs

15. The solutions to DSRs Exponential Complexity
# System design prioritizes creation, not deletion, or consolidated access
# Systems designed for DSR by default
# Data sprawl increases over time with new technology adoption
# A standard interface and protocol for DSR
# User data structures vary widely
# An orchestration tool built for flexibility
# There is no consistent data labeling convention
# A consistent and interoperable labeling standard
# Request types vary (agent, controller, subject)
# A standard interface and protocol for DSR (see point 2)
# Business constraints on what data to process in a request vary widely
# Flexible rule and policy engine

16. GEOGRAPHIC
POLICIES
POLICY ENGINE
AGENT
VERIFICATION
ID VERIFICATION
WAREHOUSES
THIRD PARTY
SYSTEMS
INTERNAL
DATA SYSTEMS
DATA MODEL ORCHESTRATION
DE-IDENTIFY
DATA
UPDATE
DATA
RETRIEVE
DATA
EMAIL
INGESTION
SUPPORT TICKET
PHONE CALL
CONSUMER / USER
API
SUBJECT
ID MFA
CONTROLLER
VERIFICATION
BUSINESS
POLICIES
TECHNICAL
POLICIES
AUTOMATED RESPONSE TO SUBJECT / REQUESTING PARTY
Systems & Processes DSR View
AGENT
CONTROLLER
SUBJECT

17. CONSUMER / USER
AUTOMATED RESPONSE TO SUBJECT / REQUESTING PARTY
Abstract Architecture
AGENT
CONTROLLER
SUBJECT
REQUEST INGESTION
IDENTITY VERIFICATION
AUDIT TRAIL
CONFIGURABLE
POLICIES
CONSISTENT
PRIVACY METADATA
ORCHESTRATION
ENGINE

18. CONSUMER / USER
AUTOMATED RESPONSE TO SUBJECT / REQUESTING PARTY
Abstract Architecture
AGENT
CONTROLLER
SUBJECT
REQUEST INGESTION
IDENTITY VERIFICATION
AUDIT TRAIL
CONFIGURABLE
POLICIES
CONSISTENT
PRIVACY METADATA
ORCHESTRATION
ENGINE

19. An open source privacy
standard for data
labeling and policies
that supports GDPR,
CCPA, LGPD and ISO
19944
Explorer fid.es/taxonomy

20. Using this standard privacy language you can describe…
# What type of data your application processes (data_category)
# How your system uses that data (data_use)
# What policies or rules you want your systems to adhere to

21. # Light-weight declarative language
# Dot notation (mostly)
# YAML in your projects (inline declarations coming soon)
Fides Declarations
# System operations data
# User provided email address
system.operations
user.provided.identifiable.contact.email

22. Fides Primitives
Organizations
1. Represents all or any part of an organization.
2. Establishes the root of the resource hierarchy.
3. Organizations are unique, i.e. you cannot
reference other organization scopes.
# Organizations
# Systems
# Datasets
# Policies

23. # Organizations
# Systems
# Datasets
# Policies
Fides Primitives
Systems
1. Represents the privacy properties of a single
project, services, codebase or application.
2. Describes the categories of data being
processed and use of the data in the system.

24. # Organizations
# Systems
# Datasets
# Policies
Fides Primitives
Datasets
1. Represent any location data is stored;
databases, data warehouses or other stores.
2. You can declare individual fields of data and
describe the types of data they are storing.

25. # Organizations
# Systems
# Datasets
# Policies
Fides Primitives
Policies
1. Represents a set of rules that a system must
adhere to — your privacy policy as code.
2. Fidesctl evaluates these policies against
system/dataset declarations for compliance.

26. Intake API’s
Product connectors
Data Subject Interface
Privacy request Intake
Identity Graph Builder Request Fulfillment Services
Policy execution
of datastore
Policy-generated
Identity graph
Stripe Billing Info
Database & 3rd party adaptors
Data package storage
Response to subject
Privacy request response
S3Bucket
SELECT *
FROM CUSTOMERS
WHERE email =
‘james@gmail.com’
Access
Edit
Erasure
postgres.customers.
stripe_id
Programmatic DSR View
CONSUMER / USER
AGENT
CONTROLLER
SUBJECT

27. Strong criteria for DSR orchestration
# Deleting a user should be as seamless as creating a user
# DSRs should be easy and free (for users and businesses)
# DSRs should be scalable and a core feature of systems
# Product and technology innovation should not break DSRs

28. Takeaways: Engineering DSRs for Complexity
# Data orchestration is easy… if you have a great data model
# A consistent, interoperable labeling taxonomy is vital
# Solve the problem upstream with CI enforced data labeling
# Policy rules should be an abstraction of data orchestration

30. +
Open source privacy engineering platform
~ Free DSR orchestration platform
~ Standard for privacy metadata
~ Privacy labeling built for developers
fid.es/join