RailswayCon 2010 - Command Your Domain

Command your Domain
Lourens Naude, WildfireApp.com

Background
Independent Contractor
Ruby / C / integrations
Well versed full stack
Architecture
WildfireApp.com
Social Marketing platform
Large whitelabel clients
Bursty traffic – Lady Gaga, EA, Gatorade

Audience
Monolithic applications
Scale together, fail together
Often have an anemic domain model, scattered logic
Hockey stick function – fast initial growth, increasingly difficult to extend over time
Hacks always have the longest lifecycle in a codebase
Target platforms
Projects with significant complexity
Existing monolithic applications
Environments with excessive coupling or dependencies

MAINTENANCE COST IS MULTIPLE TIMES MORE THAN DEVELOPMENT COST

Anemic Domain Models
Cost to business
No competitive advantage
Long dev -> production cycle
Identified by ...
Tech oriented interfaces
High coupling to neighboring layers
Coupled persistence
Dependencies on host frameworks

Layered Architecture
Layers
UI
Application
Domain
Infrastructure
Rails && other MVC ?
All of the above, monolithic tree
Modeling is often data driven, not behavior driven
Models / entities are often coupled to conventions and dependencies of the host framework

Problems in practice
Rails as a niche framework
Initial conventions scale for small to medium size applications
Fast time to market
Request / response + rendering
Beyond this, generally not a Rails application anymore
Development and growth
Your application, over time, becomes your framework for future development
Conventions, practices and dependencies should grow with and adapt to that also

APPLICATIONS AS A FRAMEWORK

Custom policies and governance
Cater for both developers + production runtime
Environment support with sanity checks
Configuration management
Dependency management etc.
How ?
Thin framework agnostic layer that provides tooling for developers
Piggyback off known contracts eg. cache['something']
Deprecate framework specific references in code
Prefer SomeApp.cache to Rails.cache etc.

Moving off monilithic apps
How features used to roll out
Slapped onto the existing codebase OR
Spawned a new project, with yet another set of conventions
Topic / feature development branches
Challenges
Team likely not structured to support distributed / decoupled systems
Extreme self-discipline, from everyone
Business Process Model skills is required

DOMAIN MODELLING PRIMER

Digesting a domain
Language
Ban acronyms
Push for a common language – dev, product + biz
Be childlike.Why, why, why ...
Open ended
A domain representation is always in flux during development, especially if it's new to developers
It'll be enhanced with new concepts
Weed out vague / deprecated elements
Refactorings at this level is often overlooked

Refactoring strategies
Continuously throw usage scenarios at your system
Refactor to deeper insights
Extract hidden concepts – look for edge cases, policies etc.
In cooperation with domain experts
Refactor your domain first, code second

Behavior and responsibility
Ignore schema and attributes
Should not be the core of your design process
Assigning data to entities prior to knowing that they do ?
Behavior
Partition objects and entities by behavior
Distribute responsibilities first
Then decide what each entity needs to know

DOMAIN FRAGMENTATION

Domain leaks - People
Developers
Code ownership
Some developers don't communicate well
Snapshot thoughts to wiki != fun
People come, people go
Product and business
Not willing to get into domain details
Concerned with features
Concerned with cash flow

Domain leaks – Process
Reorganization
Migrates knowledge excessively
Fine balance between code ownership and moving developers around too much
Practices and culture
Lack of Documentation culture
Not factoring clued up domain experts into projects
Shielding departments from each other – us VS them
Visual communication.Whiteboards ? Diagrams ?
Domain discussions encouraged ? Shot down ?

Domain leaks - concepts
Implicit concepts
Instrumental in understanding a model
But never referenced in an implementation
Frequently would be hinted at, but may not seem important to pull in
Explicit concepts
Merging implicit concepts back into the domain
Over time, this leads to breakthroughs
Less edge cases
Ability to handle offline process programatically

Notes on errors
Common error types
Runtime errors
Exceptions
Errors in business logic
Explicit domain errors
Don't inline raise
Do include business errors in your domain
Define strategies for offline or programatic handling

STATE OF YOUR DOMAIN

Loosely coupled domain
-------------------------------------- X <-- X <-- X <-- X --------------------------------------

Sparse domain
-------------------------------------- X <-- <-- <-- X --------------------------------------

Highly coupled domain
-------------------------------------- X X <-- X X <-- X X <-- XXXX --------------------------------------

COMMUNICATION

Value of speech
Talking out loud
Embrace senses
Colorful speech
“ A valid tax invoice for taxable sales that totals x”
Fail faster – quicker feedback loop
Translation costs
Statement ...
Refine statement ...
Agree on statement ...
repeat

A Common Language
Participants
Domain experts
Stakeholders
Developers
Referenced in
Code
Test code
Discussions and meetings
Specs / diagrams
Documentation

Common Language Elements
Nouns
Entities / classes if it has identity
Invoice, transaction, business, user, expense etc.
Value object if there's no identity.Reusable and immutable
Address, Money etc. (load Address via User)
Verbs
Services, methods etc. - behavior
“ A transaction between businesses generates an invoice.”
@business.sells(2, product, other_biz)

VERBS AS COMMANDS

EVENT IS THE RESULT OF PROCESSING A COMMAND

Command -> event stream
Sell(:seller => 'Acme', :quantity => 2, :product => 'XYZ', :buyer => 'MegaCorp') @business.sells(2, product, other_biz) [PaymentProcessed, TransactionCreated, InventoryUpdated, InvoiceCreated, ShippingLabelsGenerated ]

EXPLICIT STATE CHANGES

Pushing changes
Rethink persistence
Remove persistance behavior from entities
Focus on the domain, not data structure
Delegate storage to a repository
Easy to swap out repository implementations for testing etc.
Domain events
Instead of saving to the database, generate a domain event with the relevant changes as a payload
Published to your whole stack
Eventual consistency

ActiveRecord implications
Reporting contexts
ActiveRecord::Base.read_only!
Bypass persistence entirely – force ready-only mode
@record.attributes #=> value object
@record.changed #=> value object diff
Repository use cases
A Generic Repository implementation (CRUD)
Invoice.find(1) #=> entity by identity
Invoice.find_by_refr(x) #=> dynamic and explicit query methods

EVENTED ARCHITECTURE

Commands
Drive change in your system
Time driven : batched, cron etc.
Request / Response : remote producedure calls
Definition
A well defined task / behavior with parameters
Not expected to return a result
Should be idempotent
Should expire itself when not processed in a timeframe
Yields an event stream through interaction with the domain when processed instead

Domain Events
Granularity
InvoiceChanged, InvoiceStateChanged, InvoicePaid
Try to signal the intent of a change
Wrap InvoiceChanged in a module
InvoicePaid.include InvoiceChanged
Immutability
Represents something that has happened
Thus properties should never be changed

Event Driven Architecture (EDA)
Nervous system anology – hand on stove
Hand is the event producer
Spinal cord is the ESB
Brain is the event listener
Brain is the event processor
Hand is the event reaction – we pull away
Common event patterns
Generation
Consumption
Processing

EDA attributes
Definition
Loosely coupled / decoupled
Async messaging
Granular events
Notification of state changes
Transport
ESB backbone
No centralized control

COMMAND AND QUERY SEPERATION

EITHER PERFORM AN ACTION OR RETURN A RESULT

* NOT BOTH *

COMMANDS CHANGE STATE, QUERIES EXAMINE THEM

Major differences
Queries
Synchronous and blocking
Idempotent by default
Commands
Asyncronous – we expect no results …
Other than a guarantee it's been received
Should be idempotent / not reprocessed

Read and write seperation
Write service
POST /invoices
PUT /invoice/1
DESTROY /invoice/1
Append to a FIFO command queue
Read service
GET /invoice/1
GET /invoices
Serve from an eventually consistent reporting DB

“ Hot” upgrades
Scenario
Read heavy web application
95% read, 5% write
System upgrade required
Ability to disable features is explicit in domain
Solution
Mirror all data
Disable all write services + related features
Continue serving requests

CRUD FAIL

Why ?
Audits
Can't be easily audited
You can, but pointless if your system isn't dependent on this history – no way to trust it
Accountant workflow
Accountants don't change objects
They keep histories
No updates, no deletes
Work is always auditable
Totals is always based on history

Change oriented updates
CRUD API
Create
UPDATE
Delete
What's changed on UPDATE ?
A better API
#rename_invoice
#update_line_item_quantity

AUDITS AND CONSISTENCY

Audit workflow
Commands
Stored in a sequential FIFO fashion
Allows for compensation on multiple updates
Entity state is driven exclusively from this
Guaranteed – as there's no other persistence
Audit logs
Command -> Domain -> Domain event
Aggregates domain events
Feeds into a reporting database

What's really happening here ?
Isolated contexts
Command store
Audit log store
Reporting database
Agree through communication
Eventual consistency
We don't guarantee consistenty accross these contexts
at all times
We do guarantee that it'll be consistent eventually
Relaxed consistency is often acceptable

Relaxed Consistency
Viability ?
Discuss with domain experts
How long is too long ?
Solutions
Explicit SLA's at the command level
Measure propogation and flag SLA violations

PERFORMANCE AND TESTING

Performance benefits
Throughput
High transactions/sec rate
Total throughput is always governed by the weakest link
Read / write specific optimizations
Latency: data being far from where it's needed
Never block a write on waiting for data – it's given
Can use fast disks for write oriented services
Cache read tier ftw!
Formal state changes == better cache invalidation

Testing
Service / daemon structure
Command -> Result
Result being either an Event or Error
Workflow
test/unit, domain elements + your service
Layered architectured == easy to swap out layers
Arrays to mock queue infrastructure
Behavior driven testing
Assert resulting events or errors

Asserting events
process MyApp::Payments do command :PayInvoice, :id => 1 assert_events :InvoicePaid, :RegenInvoice end

Asserting errors
process MyApp::Payments do command :PayInvoice, :id => 200 assert_errors :InvoiceAlreadyPaid end

Commands in test cases
def command(cmd, attrs) cmd = App::Commands.const_get(cmd).new attrs.each{|k,v| cmd[k] = v } @process << cmd end alias input command

Asserting event streams
def assert_events(*events) event_types = @output.map{|e| e.class } expected = events.map{|e| App::Events.const_get(e) } assert_equal expected, event_types, "Expected events #{expected.inspect}, got #{event_types.inspect}" end

MESSAGING

Messages - Communication
Communication styles
Fat consumer : service call to determine destination, then send, slow
Thin consumer : fire and forget, ESB routes, fast
ESB
Take time to research this
Understand the protocol
Client support
Durability and failover requirements
Publish / subscribe is preferred

Messages - Types
Loose coupling requirements
Weak typing, loose data model - hashes
Data types not always harmonized – accept that
Also account for foreign / API messages
Translation support
Requires mapping between systems
Individual systems can modify their internal structures, providing we update our mapping layer
Useful if we have foreign messages (via API) to push in our stack as well – just coerce to the internal representation

Messages - Versioning
Embracing change
Structures will change
Version backward incompatible changes
Intermediate sensible defaults for backward compatible changes
Strategies
Let queues be a part of version strategy also
Allows for keeping old + new messages – no loss
Thin and temp. translator to pop messages off the old queue, translate / coerce them, push to the new one

Messages - Payload
Minimal
Just enough information and context for consumers to determine if updates is relevant to them
Consumer required to lookup any additional information required
Couples to services + point of failure
Self contained
Includes all information a consumer would require to process
Loosely coupled – no additional service interactions
Watch out for size – Amazon SQS is limited to 8k

Messages - Correlation
Why it's required
Troubleshooting – backtraces is always app local
Enforcing SLAs
Idempotency
Helpful correlation id elements
Source system
Destination system
/rails/user/234/session_dssasdadas/controller/action
Consider a “touch” mecanism – history of systems the
message has passed through

QUESTIONS ?

RailswayCon 2010 - Command Your Domain

by Lourens Naudé on May 31, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 113

Statistics

RailswayCon 2010 - Command Your Domain — Presentation Transcript