SlideShare a Scribd company logo
Domain modelling with
types using mypy
Jean Carlo Machado
Motivation / Idea
● I’m a fan of functional programming and Domain Driven Design
● used Haskell, Elm, and saw something valuable there
● But never managed to convert those ideas to business reality properly
● Hard to find material about it, rather you are a super expert or knows
nothing at all
Motivation / Idea
Until...
Motivation / Idea
● Mypy bring this concepts to something more palpable than haskell
● The concepts are replicable to other languages, open-api, etc
● Fits as a glove with event based distributed systems
● Assumes some DDD and FP knowledge
Goal
Leave you inspired to seek more about the topic
What is a type?
Set of possible value that can be used as inputs or outputs of a function
What is a type system?
Formalize and enforce the otherwise implicit categories (types) the programmer uses.
The main purpose of a type system is to reduce possibilities for bugs in computer
programs.
Type systems have other purposes as well, such as expressing business rules, enabling
certain compiler optimizations, providing a form of documentation, etc.
What is mypy?
Mypy is a static type checker for Python 3 and Python 2.7.
A tool that implements pep 484, types are optional
No runtime validation/overhead
Used by Google and Dropbox
Facebook have their own versions of it
Building blocks of the type-system algebra
Using my startup idea to kill goodreads as example (okreads).
Simple values - New type
People don’t think in term of
integers and string. They talk
OrderId, customer name, etc
Isbn10 = NewType('Isbn10', str)
Isbn5 = NewType('Isbn5', str)
Simple values - New type
def download_book_by_isbn(isbn: Isbn10):
...
isbn_string = "978-3-16-148410-0"
isbn_type = Isbn10(isbn_string)
download_book_by_isbn(isbn_type)
download_book_by_isbn(isbn_string)
mypy test.py
test.py:14: error: Argument 1 to "download_book_by_isbn" has
incompatible type "str"; expected "Isbn10"
Found 1 error in 1 file (checked 1 source file)
Combinations of values AND
IsbnInfo = NamedTuple('IsbnInfo', [('isbn5', Optional[Isbn5]),
('isbn10', Optional[Isbn10])])
Modelling choices OR
Attribute undefined
errors are one of the
most common in
programming
Optional = Union[T, None]
Modelling choices OR
Also good for modeling
different states in the
domain. Types can also
represent certainty.
AIsbn = Union[Isbn5, Isbn10]
Modelling choices OR
Also good for modeling
different states in the
domain
UnvalidatedBookData = NamedTuple('UnvalidatedBookData', [('author_data', Any), ('title', str)])
class ValidatedBookData:
def __init__(self, data: UnvalidatedBookData):
if len(data.title) < 3:
raise Exception('Title should be bigger than 3 chars')
self.title = data.title
self.author_data = data.author_data
PersistedBookData = NewType('PersistedBookData', 'ValidatedBookData')
BookState = Union[UnvalidatedBookData, ValidatedBookData, PersistedBookData]
Type Functions
LoadOpenLibraryBookReference = Callable[[ExistingFile, Optional[LinesToLoadLimit]], BookReferences]
Classes to do what the type-system cannot
class ExistingFile:
def __init__(self, location: str):
if not os.path.exists(location):
raise Exception("File does not exist")
self.location = location
Generics
CommandData = TypeVar('CommandData')
class Command(Generic[CommandData]):
def __init__(self, data: CommandData, issuer: CommandIssuer):
self.data = data
self.issuer = issuer
List is a wrapper of something generic.
Putting it all together
Rethinking the paradigm of modelling software
We are biased towards OOP and RDBSs
Database modeling
OOP modeling
Workflow modeling
Workflows are composed by sub workflows
Workflows talking
Avoid flag types
Status = Literal["unvalidated", "validated"]
Book = NamedTuple('Book', [('author_data', Any), ('title', str), ('status', Status)])
def persist(book: Book):
if book.status != 'validated':
raise Exception("Cannot save unvalidated")
#save here
Every business subworkflow returns a new type
UnvalidatedBookData = NamedTuple('UnvalidatedBookData', [('author_data', Any), ('title', str)])
ValidatedBookData = NewType('ValidatedBookData', UnvalidatedBookData)
PersistedBookData = NewType('PersistedBookData', ValidatedBookData)
def persist(book: ValidatedBookData) -> PersistedBookData:
...
Avoid corrupting the model to keep fewer entities
Review = NamedTuple('Review', [('title', Title) ,('author', Title)])
## add activity approach 1
Review = NamedTuple('Review', [('title', Title) ,('author', Author), ('activity', Optional[Activity]) ])
# add activity approach 2
ReviewWithActivity = NamedTuple('ReviewWithActivity', [('Review', Review), ('activity', Activity) ])
# approach 3
QualifiedReview = NamedTuple(QualifiedReview, [...])
Workflow - dependencies
LoadOpenLibraryBookReference = Callable[['ExistingFile', Optional['LinesToLoadLimit']], BookReferences]
LoadOpenLibraryBookData = Callable[[OpenLibraryBookReference], 'UnvalidatedBookData']
PersistBook = Callable[['ValidatedBookData'], PersistedBookData]
class Workflow:
def __init__(self, book_reference_loader: LoadOpenLibraryBookReference, book_data_loader: LoadOpenLibraryBookData,
persist_book: PersistBook):
self.book_reference_loader = book_reference_loader
self.book_data_loader = book_data_loader
self.persist_book = persist_book
A OOP class constructor is a
higher order function that
injects types and returns an
aggregated type(the object)
with the state injected and
the set of callables it contains
Complete Workflow
LoadOpenLibraryBookReference = Callable[['ExistingFile', Optional['LinesToLoadLimit']], BookReferences]
LoadOpenLibraryBookData = Callable[[OpenLibraryBookReference], 'UnvalidatedBookData']
PersistBook = Callable[['ValidatedBookData'], PersistedBookData]
class Workflow:
def run(self, cmd: SyncOpenLibraryCmd) -> BookPersistedEvents:
limit = None
if cmd.limit:
limit = LinesToLoadLimit(cmd.limit)
for reference in self.book_reference_loader(ExistingFile(cmd.filename), limit):
unvalidated_book_data = self.book_data_loader(reference)
validated_book_data = ValidatedBookData(unvalidated_book_data)
persisted_book = persist_book(validated_book_data)
yield BookPersistedEvent(persisted_book_data=persisted_book)
Errors from experience
Business types
should be valid
all the time
(owning its
instantiation)
def activity_from_dict(data: dict) -> Activity:
image = EntitiesFactories.image_from_dict(data['image']) if 'image' in data else None
labels: List[Label] = []
if data.get('labels'):
labels = list(map(EntitiesFactories.label_from_dict, data['labels']))
return Activity(activityId=data['activityId'],
title=data.get('title', ''),
experienceId=data.get('experienceId', None),
isAvailable=data.get('isAvailable', True),
description=data.get('description', ''),
price=price_from_dict(data['price']),
rating=rating_from_dict(data.get('rating', {})),
duration=duration_from_dict(data.get('duration', {})),
isCertified=data.get('isCertified', False),
isOnline=data.get('isOnline', True),
labels=labels,
isOriginal=is_original(data['activityId']),
adpUrl=data.get('adpUrl', ''),
image=image,
type=data.get('type', ''),
attributes=data.get('attributes', []))
Use classes/objects for:
Workflows
Validation of integrity of value objects and aggregate roots
Equality checks, merging aggregate roots
Custom serialization
Limitations of mypy - no recursive types
Employee = NamedTuple('Employee', [('id', Id), ('manager', 'Employee')])
mypy tree.py
tree.py:4: error: Cannot resolve name "Employee" (possible cyclic definition)
Found 1 error in 1 file (checked 1 source file)
Limitations of mypy
no private constructors without hack
no inheritance of union type or aliasing
Key learnings
When types are cheap you can model all valid states of your business with it.
Every function that change something in the business should return a new type.
Make illegal states impossible by modelling all valid ones in the type-system
Strive to represent the system state by the type-system
Complements and alternatives of mypy
pyre - facebook
pytype - google
pyannotate - automatically annotate your functions (Dropbox)
monkeytype: generate runtime annotations for types (instagram)
Resources
● Podcast: Gradual Typing of Production Applications
● Example of mypy in GYG: Search service
● Our journey to type checking
● DDD at 10 years
● Okreads source applying this ideas
The end
Thanks for listening
Just like the builders of Europe’s great Gothic cathedrals we’ve been honing our craft to the limits of material and structure. There is
an unfinished Gothic cathedral in Beauvais, France, that stands witness to this deeply human struggle with limitations. It was intended
to beat all previous records of height and lightness, but it suffered a series of collapses. Ad hoc measures like iron rods and wooden
supports keep it from disintegrating, but obviously a lot of things went wrong. From a modern perspective, it’s a miracle that so many
Gothic structures had been successfully completed without the help of modern material science, computer modelling, finite element
analysis, and general math and physics. I hope future generations will be as admiring of the programming skills we’ve been displaying
in building complex operating systems, web servers, and the internet infrastructure. And, frankly, they should, because we’ve done all
this based on very flimsy theoretical foundations. We have to fix those foundations if we want to move forward. - Bartosz Milewski’s

More Related Content

What's hot

One sink to rule them all: Introducing the new Async Sink
One sink to rule them all: Introducing the new Async SinkOne sink to rule them all: Introducing the new Async Sink
One sink to rule them all: Introducing the new Async Sink
Flink Forward
 
A Prelude of Purity: Scaling Back ZIO
A Prelude of Purity: Scaling Back ZIOA Prelude of Purity: Scaling Back ZIO
A Prelude of Purity: Scaling Back ZIO
Jorge Vásquez
 
Be Smart, Constrain Your Types to Free Your Brain!
Be Smart, Constrain Your Types to Free Your Brain!Be Smart, Constrain Your Types to Free Your Brain!
Be Smart, Constrain Your Types to Free Your Brain!
Jorge Vásquez
 
Java 8 Stream API. A different way to process collections.
Java 8 Stream API. A different way to process collections.Java 8 Stream API. A different way to process collections.
Java 8 Stream API. A different way to process collections.
David Gómez García
 
Fast as C: How to Write Really Terrible Java
Fast as C: How to Write Really Terrible JavaFast as C: How to Write Really Terrible Java
Fast as C: How to Write Really Terrible Java
Charles Nutter
 
Morel, a Functional Query Language
Morel, a Functional Query LanguageMorel, a Functional Query Language
Morel, a Functional Query Language
Julian Hyde
 
Lazy java
Lazy javaLazy java
Lazy java
Mario Fusco
 
Kotlin Coroutines. Flow is coming
Kotlin Coroutines. Flow is comingKotlin Coroutines. Flow is coming
Kotlin Coroutines. Flow is coming
Kirill Rozov
 
Functional Programming Patterns (BuildStuff '14)
Functional Programming Patterns (BuildStuff '14)Functional Programming Patterns (BuildStuff '14)
Functional Programming Patterns (BuildStuff '14)
Scott Wlaschin
 
Testing Spring Boot application in post-JUnit 4 world
Testing Spring Boot application in post-JUnit 4 worldTesting Spring Boot application in post-JUnit 4 world
Testing Spring Boot application in post-JUnit 4 world
Yura Nosenko
 
Beyond SQL: Speeding up Spark with DataFrames
Beyond SQL: Speeding up Spark with DataFramesBeyond SQL: Speeding up Spark with DataFrames
Beyond SQL: Speeding up Spark with DataFrames
Databricks
 
Type Driven Development with TypeScript
Type Driven Development with TypeScriptType Driven Development with TypeScript
Type Driven Development with TypeScript
Garth Gilmour
 
Drupal のコア要素を知る ~構築を支える道具立て~
Drupal のコア要素を知る ~構築を支える道具立て~Drupal のコア要素を知る ~構築を支える道具立て~
Drupal のコア要素を知る ~構築を支える道具立て~
Kenji Shirane
 
Lambdas and Streams Master Class Part 2
Lambdas and Streams Master Class Part 2Lambdas and Streams Master Class Part 2
Lambdas and Streams Master Class Part 2
José Paumard
 
Lambda and Stream Master class - part 1
Lambda and Stream Master class - part 1Lambda and Stream Master class - part 1
Lambda and Stream Master class - part 1
José Paumard
 
L'API Collector dans tous ses états
L'API Collector dans tous ses étatsL'API Collector dans tous ses états
L'API Collector dans tous ses états
José Paumard
 
Java 8-streams-collectors-patterns
Java 8-streams-collectors-patternsJava 8-streams-collectors-patterns
Java 8-streams-collectors-patterns
José Paumard
 
Luigi presentation NYC Data Science
Luigi presentation NYC Data ScienceLuigi presentation NYC Data Science
Luigi presentation NYC Data Science
Erik Bernhardsson
 
Property Based Testing in PHP
Property Based Testing in PHPProperty Based Testing in PHP
Property Based Testing in PHP
vinaikopp
 
Introduction to Git and Github
Introduction to Git and GithubIntroduction to Git and Github
Introduction to Git and Github
Roland Emmanuel Salunga
 

What's hot (20)

One sink to rule them all: Introducing the new Async Sink
One sink to rule them all: Introducing the new Async SinkOne sink to rule them all: Introducing the new Async Sink
One sink to rule them all: Introducing the new Async Sink
 
A Prelude of Purity: Scaling Back ZIO
A Prelude of Purity: Scaling Back ZIOA Prelude of Purity: Scaling Back ZIO
A Prelude of Purity: Scaling Back ZIO
 
Be Smart, Constrain Your Types to Free Your Brain!
Be Smart, Constrain Your Types to Free Your Brain!Be Smart, Constrain Your Types to Free Your Brain!
Be Smart, Constrain Your Types to Free Your Brain!
 
Java 8 Stream API. A different way to process collections.
Java 8 Stream API. A different way to process collections.Java 8 Stream API. A different way to process collections.
Java 8 Stream API. A different way to process collections.
 
Fast as C: How to Write Really Terrible Java
Fast as C: How to Write Really Terrible JavaFast as C: How to Write Really Terrible Java
Fast as C: How to Write Really Terrible Java
 
Morel, a Functional Query Language
Morel, a Functional Query LanguageMorel, a Functional Query Language
Morel, a Functional Query Language
 
Lazy java
Lazy javaLazy java
Lazy java
 
Kotlin Coroutines. Flow is coming
Kotlin Coroutines. Flow is comingKotlin Coroutines. Flow is coming
Kotlin Coroutines. Flow is coming
 
Functional Programming Patterns (BuildStuff '14)
Functional Programming Patterns (BuildStuff '14)Functional Programming Patterns (BuildStuff '14)
Functional Programming Patterns (BuildStuff '14)
 
Testing Spring Boot application in post-JUnit 4 world
Testing Spring Boot application in post-JUnit 4 worldTesting Spring Boot application in post-JUnit 4 world
Testing Spring Boot application in post-JUnit 4 world
 
Beyond SQL: Speeding up Spark with DataFrames
Beyond SQL: Speeding up Spark with DataFramesBeyond SQL: Speeding up Spark with DataFrames
Beyond SQL: Speeding up Spark with DataFrames
 
Type Driven Development with TypeScript
Type Driven Development with TypeScriptType Driven Development with TypeScript
Type Driven Development with TypeScript
 
Drupal のコア要素を知る ~構築を支える道具立て~
Drupal のコア要素を知る ~構築を支える道具立て~Drupal のコア要素を知る ~構築を支える道具立て~
Drupal のコア要素を知る ~構築を支える道具立て~
 
Lambdas and Streams Master Class Part 2
Lambdas and Streams Master Class Part 2Lambdas and Streams Master Class Part 2
Lambdas and Streams Master Class Part 2
 
Lambda and Stream Master class - part 1
Lambda and Stream Master class - part 1Lambda and Stream Master class - part 1
Lambda and Stream Master class - part 1
 
L'API Collector dans tous ses états
L'API Collector dans tous ses étatsL'API Collector dans tous ses états
L'API Collector dans tous ses états
 
Java 8-streams-collectors-patterns
Java 8-streams-collectors-patternsJava 8-streams-collectors-patterns
Java 8-streams-collectors-patterns
 
Luigi presentation NYC Data Science
Luigi presentation NYC Data ScienceLuigi presentation NYC Data Science
Luigi presentation NYC Data Science
 
Property Based Testing in PHP
Property Based Testing in PHPProperty Based Testing in PHP
Property Based Testing in PHP
 
Introduction to Git and Github
Introduction to Git and GithubIntroduction to Git and Github
Introduction to Git and Github
 

Similar to Domain Driven Design Made Functional with Python

pythontraining-201jn026043638.pptx
pythontraining-201jn026043638.pptxpythontraining-201jn026043638.pptx
pythontraining-201jn026043638.pptx
RohitKumar639388
 
C# Summer course - Lecture 1
C# Summer course - Lecture 1C# Summer course - Lecture 1
C# Summer course - Lecture 1
mohamedsamyali
 
Functions, List and String methods
Functions, List and String methodsFunctions, List and String methods
Functions, List and String methods
PranavSB
 
Python training
Python trainingPython training
Python training
Kunalchauhan76
 
Python for dummies
Python for dummiesPython for dummies
Python for dummies
Roberto Stefanetti
 
Oop java
Oop javaOop java
Oop java
Minal Maniar
 
A Gentle Introduction to Coding ... with Python
A Gentle Introduction to Coding ... with PythonA Gentle Introduction to Coding ... with Python
A Gentle Introduction to Coding ... with Python
Tariq Rashid
 
ClassifyingIssuesFromSRTextAzureML
ClassifyingIssuesFromSRTextAzureMLClassifyingIssuesFromSRTextAzureML
ClassifyingIssuesFromSRTextAzureMLGeorge Simov
 
PRESENTATION ON PYTHON.pptx
PRESENTATION ON PYTHON.pptxPRESENTATION ON PYTHON.pptx
PRESENTATION ON PYTHON.pptx
abhishek364864
 
Discovering User's Topics of Interest in Recommender Systems
Discovering User's Topics of Interest in Recommender SystemsDiscovering User's Topics of Interest in Recommender Systems
Discovering User's Topics of Interest in Recommender Systems
Gabriel Moreira
 
biopython, doctest and makefiles
biopython, doctest and makefilesbiopython, doctest and makefiles
biopython, doctest and makefiles
Giovanni Marco Dall'Olio
 
Drupal 8: A story of growing up and getting off the island
Drupal 8: A story of growing up and getting off the islandDrupal 8: A story of growing up and getting off the island
Drupal 8: A story of growing up and getting off the island
Angela Byron
 
Modules of the twenties
Modules of the twentiesModules of the twenties
Modules of the twenties
Puppet
 
python note.pdf
python note.pdfpython note.pdf
python note.pdf
Nagendra504676
 
Python introduction
Python introductionPython introduction
Python introductionRoger Xia
 
Concepts In Object Oriented Programming Languages
Concepts In Object Oriented Programming LanguagesConcepts In Object Oriented Programming Languages
Concepts In Object Oriented Programming Languages
ppd1961
 
First Steps in Python Programming
First Steps in Python ProgrammingFirst Steps in Python Programming
First Steps in Python Programming
Dozie Agbo
 
conceptsinobjectorientedprogramminglanguages-12659959597745-phpapp02.pdf
conceptsinobjectorientedprogramminglanguages-12659959597745-phpapp02.pdfconceptsinobjectorientedprogramminglanguages-12659959597745-phpapp02.pdf
conceptsinobjectorientedprogramminglanguages-12659959597745-phpapp02.pdf
SahajShrimal1
 
Write codeforhumans
Write codeforhumansWrite codeforhumans
Write codeforhumans
Narendran R
 

Similar to Domain Driven Design Made Functional with Python (20)

pythontraining-201jn026043638.pptx
pythontraining-201jn026043638.pptxpythontraining-201jn026043638.pptx
pythontraining-201jn026043638.pptx
 
C# Summer course - Lecture 1
C# Summer course - Lecture 1C# Summer course - Lecture 1
C# Summer course - Lecture 1
 
Functions, List and String methods
Functions, List and String methodsFunctions, List and String methods
Functions, List and String methods
 
Python training
Python trainingPython training
Python training
 
Dynamic Python
Dynamic PythonDynamic Python
Dynamic Python
 
Python for dummies
Python for dummiesPython for dummies
Python for dummies
 
Oop java
Oop javaOop java
Oop java
 
A Gentle Introduction to Coding ... with Python
A Gentle Introduction to Coding ... with PythonA Gentle Introduction to Coding ... with Python
A Gentle Introduction to Coding ... with Python
 
ClassifyingIssuesFromSRTextAzureML
ClassifyingIssuesFromSRTextAzureMLClassifyingIssuesFromSRTextAzureML
ClassifyingIssuesFromSRTextAzureML
 
PRESENTATION ON PYTHON.pptx
PRESENTATION ON PYTHON.pptxPRESENTATION ON PYTHON.pptx
PRESENTATION ON PYTHON.pptx
 
Discovering User's Topics of Interest in Recommender Systems
Discovering User's Topics of Interest in Recommender SystemsDiscovering User's Topics of Interest in Recommender Systems
Discovering User's Topics of Interest in Recommender Systems
 
biopython, doctest and makefiles
biopython, doctest and makefilesbiopython, doctest and makefiles
biopython, doctest and makefiles
 
Drupal 8: A story of growing up and getting off the island
Drupal 8: A story of growing up and getting off the islandDrupal 8: A story of growing up and getting off the island
Drupal 8: A story of growing up and getting off the island
 
Modules of the twenties
Modules of the twentiesModules of the twenties
Modules of the twenties
 
python note.pdf
python note.pdfpython note.pdf
python note.pdf
 
Python introduction
Python introductionPython introduction
Python introduction
 
Concepts In Object Oriented Programming Languages
Concepts In Object Oriented Programming LanguagesConcepts In Object Oriented Programming Languages
Concepts In Object Oriented Programming Languages
 
First Steps in Python Programming
First Steps in Python ProgrammingFirst Steps in Python Programming
First Steps in Python Programming
 
conceptsinobjectorientedprogramminglanguages-12659959597745-phpapp02.pdf
conceptsinobjectorientedprogramminglanguages-12659959597745-phpapp02.pdfconceptsinobjectorientedprogramminglanguages-12659959597745-phpapp02.pdf
conceptsinobjectorientedprogramminglanguages-12659959597745-phpapp02.pdf
 
Write codeforhumans
Write codeforhumansWrite codeforhumans
Write codeforhumans
 

More from Jean Carlo Machado

Python clean code for data producs
Python clean code for data producsPython clean code for data producs
Python clean code for data producs
Jean Carlo Machado
 
Search microservice
Search microserviceSearch microservice
Search microservice
Jean Carlo Machado
 
Git avançado
Git avançadoGit avançado
Git avançado
Jean Carlo Machado
 
Functional php
Functional phpFunctional php
Functional php
Jean Carlo Machado
 
Why functional programming matters
Why functional programming mattersWhy functional programming matters
Why functional programming matters
Jean Carlo Machado
 
Clean code v3
Clean code v3Clean code v3
Clean code v3
Jean Carlo Machado
 
Clean Code V2
Clean Code V2Clean Code V2
Clean Code V2
Jean Carlo Machado
 
Review articles bio inspired algorithms
Review articles bio inspired algorithmsReview articles bio inspired algorithms
Review articles bio inspired algorithms
Jean Carlo Machado
 
Introduction to Rust
Introduction to RustIntroduction to Rust
Introduction to Rust
Jean Carlo Machado
 
Limitações do HTML no Desenvolvimento de Jogos Multiplataforma
Limitações do HTML no Desenvolvimento de Jogos MultiplataformaLimitações do HTML no Desenvolvimento de Jogos Multiplataforma
Limitações do HTML no Desenvolvimento de Jogos Multiplataforma
Jean Carlo Machado
 
Clean code
Clean codeClean code
Clean code
Jean Carlo Machado
 

More from Jean Carlo Machado (11)

Python clean code for data producs
Python clean code for data producsPython clean code for data producs
Python clean code for data producs
 
Search microservice
Search microserviceSearch microservice
Search microservice
 
Git avançado
Git avançadoGit avançado
Git avançado
 
Functional php
Functional phpFunctional php
Functional php
 
Why functional programming matters
Why functional programming mattersWhy functional programming matters
Why functional programming matters
 
Clean code v3
Clean code v3Clean code v3
Clean code v3
 
Clean Code V2
Clean Code V2Clean Code V2
Clean Code V2
 
Review articles bio inspired algorithms
Review articles bio inspired algorithmsReview articles bio inspired algorithms
Review articles bio inspired algorithms
 
Introduction to Rust
Introduction to RustIntroduction to Rust
Introduction to Rust
 
Limitações do HTML no Desenvolvimento de Jogos Multiplataforma
Limitações do HTML no Desenvolvimento de Jogos MultiplataformaLimitações do HTML no Desenvolvimento de Jogos Multiplataforma
Limitações do HTML no Desenvolvimento de Jogos Multiplataforma
 
Clean code
Clean codeClean code
Clean code
 

Recently uploaded

BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024
Ortus Solutions, Corp
 
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns
 
Understanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSageUnderstanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSage
Globus
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
Globus
 
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Globus
 
May Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdfMay Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdf
Adele Miller
 
Quarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden ExtensionsQuarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden Extensions
Max Andersen
 
Graphic Design Crash Course for beginners
Graphic Design Crash Course for beginnersGraphic Design Crash Course for beginners
Graphic Design Crash Course for beginners
e20449
 
Large Language Models and the End of Programming
Large Language Models and the End of ProgrammingLarge Language Models and the End of Programming
Large Language Models and the End of Programming
Matt Welsh
 
Pro Unity Game Development with C-sharp Book
Pro Unity Game Development with C-sharp BookPro Unity Game Development with C-sharp Book
Pro Unity Game Development with C-sharp Book
abdulrafaychaudhry
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
AMB-Review
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Globus
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
Paco van Beckhoven
 
How Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptxHow Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptx
wottaspaceseo
 
Enterprise Software Development with No Code Solutions.pptx
Enterprise Software Development with No Code Solutions.pptxEnterprise Software Development with No Code Solutions.pptx
Enterprise Software Development with No Code Solutions.pptx
QuickwayInfoSystems3
 
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus
 
Launch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in MinutesLaunch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in Minutes
Roshan Dwivedi
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Globus
 
2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx
Georgi Kodinov
 
Enterprise Resource Planning System in Telangana
Enterprise Resource Planning System in TelanganaEnterprise Resource Planning System in Telangana
Enterprise Resource Planning System in Telangana
NYGGS Automation Suite
 

Recently uploaded (20)

BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024BoxLang: Review our Visionary Licenses of 2024
BoxLang: Review our Visionary Licenses of 2024
 
Prosigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology SolutionsProsigns: Transforming Business with Tailored Technology Solutions
Prosigns: Transforming Business with Tailored Technology Solutions
 
Understanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSageUnderstanding Globus Data Transfers with NetSage
Understanding Globus Data Transfers with NetSage
 
First Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User EndpointsFirst Steps with Globus Compute Multi-User Endpoints
First Steps with Globus Compute Multi-User Endpoints
 
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
Innovating Inference - Remote Triggering of Large Language Models on HPC Clus...
 
May Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdfMay Marketo Masterclass, London MUG May 22 2024.pdf
May Marketo Masterclass, London MUG May 22 2024.pdf
 
Quarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden ExtensionsQuarkus Hidden and Forbidden Extensions
Quarkus Hidden and Forbidden Extensions
 
Graphic Design Crash Course for beginners
Graphic Design Crash Course for beginnersGraphic Design Crash Course for beginners
Graphic Design Crash Course for beginners
 
Large Language Models and the End of Programming
Large Language Models and the End of ProgrammingLarge Language Models and the End of Programming
Large Language Models and the End of Programming
 
Pro Unity Game Development with C-sharp Book
Pro Unity Game Development with C-sharp BookPro Unity Game Development with C-sharp Book
Pro Unity Game Development with C-sharp Book
 
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdfDominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
Dominate Social Media with TubeTrivia AI’s Addictive Quiz Videos.pdf
 
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisProviding Globus Services to Users of JASMIN for Environmental Data Analysis
Providing Globus Services to Users of JASMIN for Environmental Data Analysis
 
Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024Cracking the code review at SpringIO 2024
Cracking the code review at SpringIO 2024
 
How Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptxHow Recreation Management Software Can Streamline Your Operations.pptx
How Recreation Management Software Can Streamline Your Operations.pptx
 
Enterprise Software Development with No Code Solutions.pptx
Enterprise Software Development with No Code Solutions.pptxEnterprise Software Development with No Code Solutions.pptx
Enterprise Software Development with No Code Solutions.pptx
 
Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus Compute wth IRI Workflows - GlobusWorld 2024
Globus Compute wth IRI Workflows - GlobusWorld 2024
 
Launch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in MinutesLaunch Your Streaming Platforms in Minutes
Launch Your Streaming Platforms in Minutes
 
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...
 
2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx2024 RoOUG Security model for the cloud.pptx
2024 RoOUG Security model for the cloud.pptx
 
Enterprise Resource Planning System in Telangana
Enterprise Resource Planning System in TelanganaEnterprise Resource Planning System in Telangana
Enterprise Resource Planning System in Telangana
 

Domain Driven Design Made Functional with Python

  • 1. Domain modelling with types using mypy Jean Carlo Machado
  • 2. Motivation / Idea ● I’m a fan of functional programming and Domain Driven Design ● used Haskell, Elm, and saw something valuable there ● But never managed to convert those ideas to business reality properly ● Hard to find material about it, rather you are a super expert or knows nothing at all
  • 4. Motivation / Idea ● Mypy bring this concepts to something more palpable than haskell ● The concepts are replicable to other languages, open-api, etc ● Fits as a glove with event based distributed systems ● Assumes some DDD and FP knowledge
  • 5. Goal Leave you inspired to seek more about the topic
  • 6. What is a type? Set of possible value that can be used as inputs or outputs of a function
  • 7. What is a type system? Formalize and enforce the otherwise implicit categories (types) the programmer uses. The main purpose of a type system is to reduce possibilities for bugs in computer programs. Type systems have other purposes as well, such as expressing business rules, enabling certain compiler optimizations, providing a form of documentation, etc.
  • 8. What is mypy? Mypy is a static type checker for Python 3 and Python 2.7. A tool that implements pep 484, types are optional No runtime validation/overhead Used by Google and Dropbox Facebook have their own versions of it
  • 9. Building blocks of the type-system algebra Using my startup idea to kill goodreads as example (okreads).
  • 10. Simple values - New type People don’t think in term of integers and string. They talk OrderId, customer name, etc Isbn10 = NewType('Isbn10', str) Isbn5 = NewType('Isbn5', str)
  • 11. Simple values - New type def download_book_by_isbn(isbn: Isbn10): ... isbn_string = "978-3-16-148410-0" isbn_type = Isbn10(isbn_string) download_book_by_isbn(isbn_type) download_book_by_isbn(isbn_string) mypy test.py test.py:14: error: Argument 1 to "download_book_by_isbn" has incompatible type "str"; expected "Isbn10" Found 1 error in 1 file (checked 1 source file)
  • 12. Combinations of values AND IsbnInfo = NamedTuple('IsbnInfo', [('isbn5', Optional[Isbn5]), ('isbn10', Optional[Isbn10])])
  • 13. Modelling choices OR Attribute undefined errors are one of the most common in programming Optional = Union[T, None]
  • 14. Modelling choices OR Also good for modeling different states in the domain. Types can also represent certainty. AIsbn = Union[Isbn5, Isbn10]
  • 15. Modelling choices OR Also good for modeling different states in the domain UnvalidatedBookData = NamedTuple('UnvalidatedBookData', [('author_data', Any), ('title', str)]) class ValidatedBookData: def __init__(self, data: UnvalidatedBookData): if len(data.title) < 3: raise Exception('Title should be bigger than 3 chars') self.title = data.title self.author_data = data.author_data PersistedBookData = NewType('PersistedBookData', 'ValidatedBookData') BookState = Union[UnvalidatedBookData, ValidatedBookData, PersistedBookData]
  • 16. Type Functions LoadOpenLibraryBookReference = Callable[[ExistingFile, Optional[LinesToLoadLimit]], BookReferences]
  • 17. Classes to do what the type-system cannot class ExistingFile: def __init__(self, location: str): if not os.path.exists(location): raise Exception("File does not exist") self.location = location
  • 18. Generics CommandData = TypeVar('CommandData') class Command(Generic[CommandData]): def __init__(self, data: CommandData, issuer: CommandIssuer): self.data = data self.issuer = issuer List is a wrapper of something generic.
  • 19. Putting it all together Rethinking the paradigm of modelling software We are biased towards OOP and RDBSs
  • 23. Workflows are composed by sub workflows
  • 25. Avoid flag types Status = Literal["unvalidated", "validated"] Book = NamedTuple('Book', [('author_data', Any), ('title', str), ('status', Status)]) def persist(book: Book): if book.status != 'validated': raise Exception("Cannot save unvalidated") #save here
  • 26. Every business subworkflow returns a new type UnvalidatedBookData = NamedTuple('UnvalidatedBookData', [('author_data', Any), ('title', str)]) ValidatedBookData = NewType('ValidatedBookData', UnvalidatedBookData) PersistedBookData = NewType('PersistedBookData', ValidatedBookData) def persist(book: ValidatedBookData) -> PersistedBookData: ...
  • 27. Avoid corrupting the model to keep fewer entities Review = NamedTuple('Review', [('title', Title) ,('author', Title)]) ## add activity approach 1 Review = NamedTuple('Review', [('title', Title) ,('author', Author), ('activity', Optional[Activity]) ]) # add activity approach 2 ReviewWithActivity = NamedTuple('ReviewWithActivity', [('Review', Review), ('activity', Activity) ]) # approach 3 QualifiedReview = NamedTuple(QualifiedReview, [...])
  • 28. Workflow - dependencies LoadOpenLibraryBookReference = Callable[['ExistingFile', Optional['LinesToLoadLimit']], BookReferences] LoadOpenLibraryBookData = Callable[[OpenLibraryBookReference], 'UnvalidatedBookData'] PersistBook = Callable[['ValidatedBookData'], PersistedBookData] class Workflow: def __init__(self, book_reference_loader: LoadOpenLibraryBookReference, book_data_loader: LoadOpenLibraryBookData, persist_book: PersistBook): self.book_reference_loader = book_reference_loader self.book_data_loader = book_data_loader self.persist_book = persist_book A OOP class constructor is a higher order function that injects types and returns an aggregated type(the object) with the state injected and the set of callables it contains
  • 29. Complete Workflow LoadOpenLibraryBookReference = Callable[['ExistingFile', Optional['LinesToLoadLimit']], BookReferences] LoadOpenLibraryBookData = Callable[[OpenLibraryBookReference], 'UnvalidatedBookData'] PersistBook = Callable[['ValidatedBookData'], PersistedBookData] class Workflow: def run(self, cmd: SyncOpenLibraryCmd) -> BookPersistedEvents: limit = None if cmd.limit: limit = LinesToLoadLimit(cmd.limit) for reference in self.book_reference_loader(ExistingFile(cmd.filename), limit): unvalidated_book_data = self.book_data_loader(reference) validated_book_data = ValidatedBookData(unvalidated_book_data) persisted_book = persist_book(validated_book_data) yield BookPersistedEvent(persisted_book_data=persisted_book)
  • 30. Errors from experience Business types should be valid all the time (owning its instantiation) def activity_from_dict(data: dict) -> Activity: image = EntitiesFactories.image_from_dict(data['image']) if 'image' in data else None labels: List[Label] = [] if data.get('labels'): labels = list(map(EntitiesFactories.label_from_dict, data['labels'])) return Activity(activityId=data['activityId'], title=data.get('title', ''), experienceId=data.get('experienceId', None), isAvailable=data.get('isAvailable', True), description=data.get('description', ''), price=price_from_dict(data['price']), rating=rating_from_dict(data.get('rating', {})), duration=duration_from_dict(data.get('duration', {})), isCertified=data.get('isCertified', False), isOnline=data.get('isOnline', True), labels=labels, isOriginal=is_original(data['activityId']), adpUrl=data.get('adpUrl', ''), image=image, type=data.get('type', ''), attributes=data.get('attributes', []))
  • 31. Use classes/objects for: Workflows Validation of integrity of value objects and aggregate roots Equality checks, merging aggregate roots Custom serialization
  • 32. Limitations of mypy - no recursive types Employee = NamedTuple('Employee', [('id', Id), ('manager', 'Employee')]) mypy tree.py tree.py:4: error: Cannot resolve name "Employee" (possible cyclic definition) Found 1 error in 1 file (checked 1 source file)
  • 33. Limitations of mypy no private constructors without hack no inheritance of union type or aliasing
  • 34. Key learnings When types are cheap you can model all valid states of your business with it. Every function that change something in the business should return a new type. Make illegal states impossible by modelling all valid ones in the type-system Strive to represent the system state by the type-system
  • 35. Complements and alternatives of mypy pyre - facebook pytype - google pyannotate - automatically annotate your functions (Dropbox) monkeytype: generate runtime annotations for types (instagram)
  • 36. Resources ● Podcast: Gradual Typing of Production Applications ● Example of mypy in GYG: Search service ● Our journey to type checking ● DDD at 10 years ● Okreads source applying this ideas
  • 37. The end Thanks for listening Just like the builders of Europe’s great Gothic cathedrals we’ve been honing our craft to the limits of material and structure. There is an unfinished Gothic cathedral in Beauvais, France, that stands witness to this deeply human struggle with limitations. It was intended to beat all previous records of height and lightness, but it suffered a series of collapses. Ad hoc measures like iron rods and wooden supports keep it from disintegrating, but obviously a lot of things went wrong. From a modern perspective, it’s a miracle that so many Gothic structures had been successfully completed without the help of modern material science, computer modelling, finite element analysis, and general math and physics. I hope future generations will be as admiring of the programming skills we’ve been displaying in building complex operating systems, web servers, and the internet infrastructure. And, frankly, they should, because we’ve done all this based on very flimsy theoretical foundations. We have to fix those foundations if we want to move forward. - Bartosz Milewski’s