RailswayCon 2010 - Command Your Domain
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

RailswayCon 2010 - Command Your Domain

  • 4,519 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
4,519
On Slideshare
4,406
From Embeds
113
Number of Embeds
1

Actions

Shares
Downloads
71
Comments
0
Likes
5

Embeds 113

http://www.slideshare.net 113

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1.
      Command your Domain
      Lourens Naude, WildfireApp.com
  • 2.
      Background
    • Independent Contractor
      • Ruby / C / integrations
      • 3. Well versed full stack
      • 4. Architecture
    • WildfireApp.com
      • Social Marketing platform
      • 5. Large whitelabel clients
      • 6. Bursty traffic – Lady Gaga, EA, Gatorade
  • 7.  
  • 8.
      Audience
    • Monolithic applications
      • Scale together, fail together
      • 9. Often have an anemic domain model, scattered logic
      • 10. Hockey stick function – fast initial growth, increasingly difficult to extend over time
      • 11. Hacks always have the longest lifecycle in a codebase
    • Target platforms
      • Projects with significant complexity
      • 12. Existing monolithic applications
      • 13. Environments with excessive coupling or dependencies
  • 14.
      MAINTENANCE COST IS MULTIPLE TIMES MORE THAN DEVELOPMENT COST
  • 15.
      Anemic Domain Models
    • Cost to business
      • No competitive advantage
      • 16. Long dev -> production cycle
    • Identified by ...
      • Tech oriented interfaces
      • 17. High coupling to neighboring layers
      • 18. Coupled persistence
      • 19. Dependencies on host frameworks
  • 20.
      Layered Architecture
    • Layers
    • Rails && other MVC ?
      • All of the above, monolithic tree
      • 24. Modeling is often data driven, not behavior driven
      • 25. Models / entities are often coupled to conventions and dependencies of the host framework
  • 26.
      Problems in practice
    • Rails as a niche framework
      • Initial conventions scale for small to medium size applications
      • 27. Fast time to market
      • 28. Request / response + rendering
      • 29. Beyond this, generally not a Rails application anymore
    • Development and growth
      • Your application, over time, becomes your framework for future development
      • 30. Conventions, practices and dependencies should grow with and adapt to that also
  • 31.
      APPLICATIONS AS A FRAMEWORK
  • 32.
      Custom policies and governance
    • Cater for both developers + production runtime
      • Environment support with sanity checks
      • 33. Configuration management
      • 34. Dependency management etc.
    • How ?
      • Thin framework agnostic layer that provides tooling for developers
      • 35. Piggyback off known contracts eg. cache['something']
      • 36. Deprecate framework specific references in code
      • 37. Prefer SomeApp.cache to Rails.cache etc.
  • 38.
      Moving off monilithic apps
    • How features used to roll out
      • Slapped onto the existing codebase OR
      • 39. Spawned a new project, with yet another set of conventions
      • 40. Topic / feature development branches
    • Challenges
      • Team likely not structured to support distributed / decoupled systems
      • 41. Extreme self-discipline, from everyone
      • 42. Business Process Model skills is required
  • 43.
      DOMAIN MODELLING PRIMER
  • 44.
      Digesting a domain
    • Language
      • Ban acronyms
      • 45. Push for a common language – dev, product + biz
      • 46. Be childlike.Why, why, why ...
    • Open ended
      • A domain representation is always in flux during development, especially if it's new to developers
      • 47. It'll be enhanced with new concepts
      • 48. Weed out vague / deprecated elements
      • 49. Refactorings at this level is often overlooked
  • 50.
      Refactoring strategies
    • Continuously throw usage scenarios at your system
    • 51. Refactor to deeper insights
    • 52. Extract hidden concepts – look for edge cases, policies etc.
    • 53. In cooperation with domain experts
    • 54. Refactor your domain first, code second
  • 55.
      Behavior and responsibility
    • Ignore schema and attributes
      • Should not be the core of your design process
      • 56. Assigning data to entities prior to knowing that they do ?
    • Behavior
      • Partition objects and entities by behavior
      • 57. Distribute responsibilities first
      • 58. Then decide what each entity needs to know
  • 59.
      DOMAIN FRAGMENTATION
  • 60.
      Domain leaks - People
    • Developers
      • Code ownership
      • 61. Some developers don't communicate well
      • 62. Snapshot thoughts to wiki != fun
      • 63. People come, people go
    • Product and business
      • Not willing to get into domain details
      • 64. Concerned with features
      • 65. Concerned with cash flow
  • 66.
      Domain leaks – Process
    • Reorganization
      • Migrates knowledge excessively
      • 67. Fine balance between code ownership and moving developers around too much
    • Practices and culture
      • Lack of Documentation culture
      • 68. Not factoring clued up domain experts into projects
      • 69. Shielding departments from each other – us VS them
      • 70. Visual communication.Whiteboards ? Diagrams ?
      • 71. Domain discussions encouraged ? Shot down ?
  • 72.
      Domain leaks - concepts
    • Implicit concepts
      • Instrumental in understanding a model
      • 73. But never referenced in an implementation
      • 74. Frequently would be hinted at, but may not seem important to pull in
    • Explicit concepts
      • Merging implicit concepts back into the domain
      • 75. Over time, this leads to breakthroughs
      • 76. Less edge cases
      • 77. Ability to handle offline process programatically
  • 78.
      Notes on errors
    • Common error types
      • Runtime errors
      • 79. Exceptions
      • 80. Errors in business logic
    • Explicit domain errors
      • Don't inline raise
      • 81. Do include business errors in your domain
      • 82. Define strategies for offline or programatic handling
  • 83.
      STATE OF YOUR DOMAIN
  • 84.
      Loosely coupled domain
    -------------------------------------- X <-- X <-- X <-- X --------------------------------------
  • 85.
      Sparse domain
    -------------------------------------- X <-- <-- <-- X --------------------------------------
  • 86.
      Highly coupled domain
    -------------------------------------- X X <-- X X <-- X X <-- XXXX --------------------------------------
  • 87.
      COMMUNICATION
  • 88.
      Value of speech
    • Talking out loud
      • Embrace senses
      • 89. Colorful speech
      • 90. “ A valid tax invoice for taxable sales that totals x”
      • 91. Fail faster – quicker feedback loop
    • Translation costs
      • Statement ...
      • 92. Refine statement ...
      • 93. Agree on statement ...
      • 94. repeat
  • 95.
      A Common Language
    • Participants
    • Referenced in
  • 102.
      Common Language Elements
    • Nouns
      • Entities / classes if it has identity
      • 103. Invoice, transaction, business, user, expense etc.
      • 104. Value object if there's no identity.Reusable and immutable
      • 105. Address, Money etc. (load Address via User)
    • Verbs
      • Services, methods etc. - behavior
      • 106. “ A transaction between businesses generates an invoice.”
      • 107. @business.sells(2, product, other_biz)
  • 108.
    • VERBS AS COMMANDS
  • 109.
    • EVENT IS THE RESULT OF PROCESSING A COMMAND
  • 110.
      Command -> event stream
    Sell(:seller => 'Acme', :quantity => 2, :product => 'XYZ', :buyer => 'MegaCorp') @business.sells(2, product, other_biz) [PaymentProcessed, TransactionCreated, InventoryUpdated, InvoiceCreated, ShippingLabelsGenerated ]
  • 111.
    • EXPLICIT STATE CHANGES
  • 112.
      Pushing changes
    • Rethink persistence
      • Remove persistance behavior from entities
      • 113. Focus on the domain, not data structure
      • 114. Delegate storage to a repository
      • 115. Easy to swap out repository implementations for testing etc.
    • Domain events
      • Instead of saving to the database, generate a domain event with the relevant changes as a payload
      • 116. Published to your whole stack
      • 117. Eventual consistency
  • 118.
      ActiveRecord implications
    • Reporting contexts
      • ActiveRecord::Base.read_only!
      • 119. Bypass persistence entirely – force ready-only mode
      • 120. @record.attributes #=> value object
      • 121. @record.changed #=> value object diff
    • Repository use cases
      • A Generic Repository implementation (CRUD)
      • 122. Invoice.find(1) #=> entity by identity
      • 123. Invoice.find_by_refr(x) #=> dynamic and explicit query methods
  • 124.
      EVENTED ARCHITECTURE
  • 125.
      Commands
    • Drive change in your system
      • Time driven : batched, cron etc.
      • 126. Request / Response : remote producedure calls
    • Definition
      • A well defined task / behavior with parameters
      • 127. Not expected to return a result
      • 128. Should be idempotent
      • 129. Should expire itself when not processed in a timeframe
      • 130. Yields an event stream through interaction with the domain when processed instead
  • 131.
      Domain Events
    • Granularity
      • InvoiceChanged, InvoiceStateChanged, InvoicePaid
      • 132. Try to signal the intent of a change
      • 133. Wrap InvoiceChanged in a module
      • 134. InvoicePaid.include InvoiceChanged
    • Immutability
      • Represents something that has happened
      • 135. Thus properties should never be changed
  • 136.
      Event Driven Architecture (EDA)
    • Nervous system anology – hand on stove
      • Hand is the event producer
      • 137. Spinal cord is the ESB
      • 138. Brain is the event listener
      • 139. Brain is the event processor
      • 140. Hand is the event reaction – we pull away
    • Common event patterns
  • 143.
      EDA attributes
    • Definition
      • Loosely coupled / decoupled
      • 144. Async messaging
      • 145. Granular events
      • 146. Notification of state changes
    • Transport
      • ESB backbone
      • 147. No centralized control
  • 148.
      COMMAND AND QUERY SEPERATION
  • 149.
      EITHER PERFORM AN ACTION OR RETURN A RESULT
  • 150.
      * NOT BOTH *
  • 151.
      COMMANDS CHANGE STATE, QUERIES EXAMINE THEM
  • 152.
      Major differences
    • Queries
      • Synchronous and blocking
      • 153. Idempotent by default
    • Commands
      • Asyncronous – we expect no results …
      • 154. Other than a guarantee it's been received
      • 155. Should be idempotent / not reprocessed
  • 156.
      Read and write seperation
    • Write service
      • POST /invoices
      • 157. PUT /invoice/1
      • 158. DESTROY /invoice/1
      • 159. Append to a FIFO command queue
    • Read service
      • GET /invoice/1
      • 160. GET /invoices
      • 161. Serve from an eventually consistent reporting DB
  • 162.
      “ Hot” upgrades
    • Scenario
      • Read heavy web application
      • 163. 95% read, 5% write
      • 164. System upgrade required
      • 165. Ability to disable features is explicit in domain
    • Solution
      • Mirror all data
      • 166. Disable all write services + related features
      • 167. Continue serving requests
  • 168.
      CRUD FAIL
  • 169.
      Why ?
    • Audits
    • 170. Can't be easily audited
      • You can, but pointless if your system isn't dependent on this history – no way to trust it
    • Accountant workflow
      • Accountants don't change objects
      • 171. They keep histories
      • 172. No updates, no deletes
      • 173. Work is always auditable
      • 174. Totals is always based on history
  • 175.
      Change oriented updates
    • CRUD API
    • A better API
      • #rename_invoice
      • 179. #update_line_item_quantity
  • 180.
      AUDITS AND CONSISTENCY
  • 181.
      Audit workflow
    • Commands
      • Stored in a sequential FIFO fashion
      • 182. Allows for compensation on multiple updates
      • 183. Entity state is driven exclusively from this
      • 184. Guaranteed – as there's no other persistence
    • Audit logs
      • Command -> Domain -> Domain event
      • 185. Aggregates domain events
      • 186. Feeds into a reporting database
  • 187.
      What's really happening here ?
    • Isolated contexts
      • Command store
      • 188. Audit log store
      • 189. Reporting database
      • 190. Agree through communication
    • Eventual consistency
      • We don't guarantee consistenty accross these contexts
      • 191. at all times
      • 192. We do guarantee that it'll be consistent eventually
      • 193. Relaxed consistency is often acceptable
  • 194.
      Relaxed Consistency
    • Viability ?
      • Discuss with domain experts
      • 195. How long is too long ?
    • Solutions
      • Explicit SLA's at the command level
      • 196. Measure propogation and flag SLA violations
  • 197.
      PERFORMANCE AND TESTING
  • 198.
      Performance benefits
    • Throughput
      • High transactions/sec rate
      • 199. Total throughput is always governed by the weakest link
    • Read / write specific optimizations
      • Latency: data being far from where it's needed
      • 200. Never block a write on waiting for data – it's given
      • 201. Can use fast disks for write oriented services
      • 202. Cache read tier ftw!
      • 203. Formal state changes == better cache invalidation
  • 204.
      Testing
    • Service / daemon structure
      • Command -> Result
      • 205. Result being either an Event or Error
    • Workflow
      • test/unit, domain elements + your service
      • 206. Layered architectured == easy to swap out layers
      • 207. Arrays to mock queue infrastructure
      • 208. Behavior driven testing
      • 209. Assert resulting events or errors
  • 210.
      Asserting events
    process MyApp::Payments do command :PayInvoice, :id => 1 assert_events :InvoicePaid, :RegenInvoice end
  • 211.
      Asserting errors
    process MyApp::Payments do command :PayInvoice, :id => 200 assert_errors :InvoiceAlreadyPaid end
  • 212.
      Commands in test cases
    def command(cmd, attrs) cmd = App::Commands.const_get(cmd).new attrs.each{|k,v| cmd[k] = v } @process << cmd end alias input command
  • 213.
      Asserting event streams
    def assert_events(*events) event_types = @output.map{|e| e.class } expected = events.map{|e| App::Events.const_get(e) } assert_equal expected, event_types, &quot;Expected events #{expected.inspect}, got #{event_types.inspect}&quot; end
  • 214.
      MESSAGING
  • 215.
      Messages - Communication
    • Communication styles
      • Fat consumer : service call to determine destination, then send, slow
      • 216. Thin consumer : fire and forget, ESB routes, fast
    • ESB
      • Take time to research this
      • 217. Understand the protocol
      • 218. Client support
      • 219. Durability and failover requirements
      • 220. Publish / subscribe is preferred
  • 221.
      Messages - Types
    • Loose coupling requirements
      • Weak typing, loose data model - hashes
      • 222. Data types not always harmonized – accept that
      • 223. Also account for foreign / API messages
    • Translation support
      • Requires mapping between systems
      • 224. Individual systems can modify their internal structures, providing we update our mapping layer
      • 225. Useful if we have foreign messages (via API) to push in our stack as well – just coerce to the internal representation
  • 226.
      Messages - Versioning
    • Embracing change
      • Structures will change
      • 227. Version backward incompatible changes
      • 228. Intermediate sensible defaults for backward compatible changes
    • Strategies
      • Let queues be a part of version strategy also
      • 229. Allows for keeping old + new messages – no loss
      • 230. Thin and temp. translator to pop messages off the old queue, translate / coerce them, push to the new one
  • 231.
      Messages - Payload
    • Minimal
      • Just enough information and context for consumers to determine if updates is relevant to them
      • 232. Consumer required to lookup any additional information required
      • 233. Couples to services + point of failure
    • Self contained
      • Includes all information a consumer would require to process
      • 234. Loosely coupled – no additional service interactions
      • 235. Watch out for size – Amazon SQS is limited to 8k
  • 236.
      Messages - Correlation
    • Why it's required
      • Troubleshooting – backtraces is always app local
      • 237. Enforcing SLAs
      • 238. Idempotency
    • Helpful correlation id elements
      • Source system
      • 239. Destination system
      • 240. /rails/user/234/session_dssasdadas/controller/action
      • 241. Consider a “touch” mecanism – history of systems the
      • 242. message has passed through
  • 243.
      QUESTIONS ?