• Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,528
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
71
Comments
0
Likes
5

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1.
      Command your Domain
      Lourens Naude, WildfireApp.com
  • 2.
      Background
    • Independent Contractor
      • Ruby / C / integrations
      • 3. Well versed full stack
      • 4. Architecture
    • WildfireApp.com
      • Social Marketing platform
      • 5. Large whitelabel clients
      • 6. Bursty traffic – Lady Gaga, EA, Gatorade
  • 7.  
  • 8.
      Audience
    • Monolithic applications
      • Scale together, fail together
      • 9. Often have an anemic domain model, scattered logic
      • 10. Hockey stick function – fast initial growth, increasingly difficult to extend over time
      • 11. Hacks always have the longest lifecycle in a codebase
    • Target platforms
      • Projects with significant complexity
      • 12. Existing monolithic applications
      • 13. Environments with excessive coupling or dependencies
  • 14.
      MAINTENANCE COST IS MULTIPLE TIMES MORE THAN DEVELOPMENT COST
  • 15.
      Anemic Domain Models
    • Cost to business
      • No competitive advantage
      • 16. Long dev -> production cycle
    • Identified by ...
      • Tech oriented interfaces
      • 17. High coupling to neighboring layers
      • 18. Coupled persistence
      • 19. Dependencies on host frameworks
  • 20.
      Layered Architecture
    • Layers
    • Rails && other MVC ?
      • All of the above, monolithic tree
      • 24. Modeling is often data driven, not behavior driven
      • 25. Models / entities are often coupled to conventions and dependencies of the host framework
  • 26.
      Problems in practice
    • Rails as a niche framework
      • Initial conventions scale for small to medium size applications
      • 27. Fast time to market
      • 28. Request / response + rendering
      • 29. Beyond this, generally not a Rails application anymore
    • Development and growth
      • Your application, over time, becomes your framework for future development
      • 30. Conventions, practices and dependencies should grow with and adapt to that also
  • 31.
      APPLICATIONS AS A FRAMEWORK
  • 32.
      Custom policies and governance
    • Cater for both developers + production runtime
      • Environment support with sanity checks
      • 33. Configuration management
      • 34. Dependency management etc.
    • How ?
      • Thin framework agnostic layer that provides tooling for developers
      • 35. Piggyback off known contracts eg. cache['something']
      • 36. Deprecate framework specific references in code
      • 37. Prefer SomeApp.cache to Rails.cache etc.
  • 38.
      Moving off monilithic apps
    • How features used to roll out
      • Slapped onto the existing codebase OR
      • 39. Spawned a new project, with yet another set of conventions
      • 40. Topic / feature development branches
    • Challenges
      • Team likely not structured to support distributed / decoupled systems
      • 41. Extreme self-discipline, from everyone
      • 42. Business Process Model skills is required
  • 43.
      DOMAIN MODELLING PRIMER
  • 44.
      Digesting a domain
    • Language
      • Ban acronyms
      • 45. Push for a common language – dev, product + biz
      • 46. Be childlike.Why, why, why ...
    • Open ended
      • A domain representation is always in flux during development, especially if it's new to developers
      • 47. It'll be enhanced with new concepts
      • 48. Weed out vague / deprecated elements
      • 49. Refactorings at this level is often overlooked
  • 50.
      Refactoring strategies
    • Continuously throw usage scenarios at your system
    • 51. Refactor to deeper insights
    • 52. Extract hidden concepts – look for edge cases, policies etc.
    • 53. In cooperation with domain experts
    • 54. Refactor your domain first, code second
  • 55.
      Behavior and responsibility
    • Ignore schema and attributes
      • Should not be the core of your design process
      • 56. Assigning data to entities prior to knowing that they do ?
    • Behavior
      • Partition objects and entities by behavior
      • 57. Distribute responsibilities first
      • 58. Then decide what each entity needs to know
  • 59.
      DOMAIN FRAGMENTATION
  • 60.
      Domain leaks - People
    • Developers
      • Code ownership
      • 61. Some developers don't communicate well
      • 62. Snapshot thoughts to wiki != fun
      • 63. People come, people go
    • Product and business
      • Not willing to get into domain details
      • 64. Concerned with features
      • 65. Concerned with cash flow
  • 66.
      Domain leaks – Process
    • Reorganization
      • Migrates knowledge excessively
      • 67. Fine balance between code ownership and moving developers around too much
    • Practices and culture
      • Lack of Documentation culture
      • 68. Not factoring clued up domain experts into projects
      • 69. Shielding departments from each other – us VS them
      • 70. Visual communication.Whiteboards ? Diagrams ?
      • 71. Domain discussions encouraged ? Shot down ?
  • 72.
      Domain leaks - concepts
    • Implicit concepts
      • Instrumental in understanding a model
      • 73. But never referenced in an implementation
      • 74. Frequently would be hinted at, but may not seem important to pull in
    • Explicit concepts
      • Merging implicit concepts back into the domain
      • 75. Over time, this leads to breakthroughs
      • 76. Less edge cases
      • 77. Ability to handle offline process programatically
  • 78.
      Notes on errors
    • Common error types
      • Runtime errors
      • 79. Exceptions
      • 80. Errors in business logic
    • Explicit domain errors
      • Don't inline raise
      • 81. Do include business errors in your domain
      • 82. Define strategies for offline or programatic handling
  • 83.
      STATE OF YOUR DOMAIN
  • 84.
      Loosely coupled domain
    -------------------------------------- X <-- X <-- X <-- X --------------------------------------
  • 85.
      Sparse domain
    -------------------------------------- X <-- <-- <-- X --------------------------------------
  • 86.
      Highly coupled domain
    -------------------------------------- X X <-- X X <-- X X <-- XXXX --------------------------------------
  • 87.
      COMMUNICATION
  • 88.
      Value of speech
    • Talking out loud
      • Embrace senses
      • 89. Colorful speech
      • 90. “ A valid tax invoice for taxable sales that totals x”
      • 91. Fail faster – quicker feedback loop
    • Translation costs
      • Statement ...
      • 92. Refine statement ...
      • 93. Agree on statement ...
      • 94. repeat
  • 95.
      A Common Language
    • Participants
    • Referenced in
  • 102.
      Common Language Elements
    • Nouns
      • Entities / classes if it has identity
      • 103. Invoice, transaction, business, user, expense etc.
      • 104. Value object if there's no identity.Reusable and immutable
      • 105. Address, Money etc. (load Address via User)
    • Verbs
      • Services, methods etc. - behavior
      • 106. “ A transaction between businesses generates an invoice.”
      • 107. @business.sells(2, product, other_biz)
  • 108.
    • VERBS AS COMMANDS
  • 109.
    • EVENT IS THE RESULT OF PROCESSING A COMMAND
  • 110.
      Command -> event stream
    Sell(:seller => 'Acme', :quantity => 2, :product => 'XYZ', :buyer => 'MegaCorp') @business.sells(2, product, other_biz) [PaymentProcessed, TransactionCreated, InventoryUpdated, InvoiceCreated, ShippingLabelsGenerated ]
  • 111.
    • EXPLICIT STATE CHANGES
  • 112.
      Pushing changes
    • Rethink persistence
      • Remove persistance behavior from entities
      • 113. Focus on the domain, not data structure
      • 114. Delegate storage to a repository
      • 115. Easy to swap out repository implementations for testing etc.
    • Domain events
      • Instead of saving to the database, generate a domain event with the relevant changes as a payload
      • 116. Published to your whole stack
      • 117. Eventual consistency
  • 118.
      ActiveRecord implications
    • Reporting contexts
      • ActiveRecord::Base.read_only!
      • 119. Bypass persistence entirely – force ready-only mode
      • 120. @record.attributes #=> value object
      • 121. @record.changed #=> value object diff
    • Repository use cases
      • A Generic Repository implementation (CRUD)
      • 122. Invoice.find(1) #=> entity by identity
      • 123. Invoice.find_by_refr(x) #=> dynamic and explicit query methods
  • 124.
      EVENTED ARCHITECTURE
  • 125.
      Commands
    • Drive change in your system
      • Time driven : batched, cron etc.
      • 126. Request / Response : remote producedure calls
    • Definition
      • A well defined task / behavior with parameters
      • 127. Not expected to return a result
      • 128. Should be idempotent
      • 129. Should expire itself when not processed in a timeframe
      • 130. Yields an event stream through interaction with the domain when processed instead
  • 131.
      Domain Events
    • Granularity
      • InvoiceChanged, InvoiceStateChanged, InvoicePaid
      • 132. Try to signal the intent of a change
      • 133. Wrap InvoiceChanged in a module
      • 134. InvoicePaid.include InvoiceChanged
    • Immutability
      • Represents something that has happened
      • 135. Thus properties should never be changed
  • 136.
      Event Driven Architecture (EDA)
    • Nervous system anology – hand on stove
      • Hand is the event producer
      • 137. Spinal cord is the ESB
      • 138. Brain is the event listener
      • 139. Brain is the event processor
      • 140. Hand is the event reaction – we pull away
    • Common event patterns
  • 143.
      EDA attributes
    • Definition
      • Loosely coupled / decoupled
      • 144. Async messaging
      • 145. Granular events
      • 146. Notification of state changes
    • Transport
      • ESB backbone
      • 147. No centralized control
  • 148.
      COMMAND AND QUERY SEPERATION
  • 149.
      EITHER PERFORM AN ACTION OR RETURN A RESULT
  • 150.
      * NOT BOTH *
  • 151.
      COMMANDS CHANGE STATE, QUERIES EXAMINE THEM
  • 152.
      Major differences
    • Queries
      • Synchronous and blocking
      • 153. Idempotent by default
    • Commands
      • Asyncronous – we expect no results …
      • 154. Other than a guarantee it's been received
      • 155. Should be idempotent / not reprocessed
  • 156.
      Read and write seperation
    • Write service
      • POST /invoices
      • 157. PUT /invoice/1
      • 158. DESTROY /invoice/1
      • 159. Append to a FIFO command queue
    • Read service
      • GET /invoice/1
      • 160. GET /invoices
      • 161. Serve from an eventually consistent reporting DB
  • 162.
      “ Hot” upgrades
    • Scenario
      • Read heavy web application
      • 163. 95% read, 5% write
      • 164. System upgrade required
      • 165. Ability to disable features is explicit in domain
    • Solution
      • Mirror all data
      • 166. Disable all write services + related features
      • 167. Continue serving requests
  • 168.
      CRUD FAIL
  • 169.
      Why ?
    • Audits
    • 170. Can't be easily audited
      • You can, but pointless if your system isn't dependent on this history – no way to trust it
    • Accountant workflow
      • Accountants don't change objects
      • 171. They keep histories
      • 172. No updates, no deletes
      • 173. Work is always auditable
      • 174. Totals is always based on history
  • 175.
      Change oriented updates
    • CRUD API
    • A better API
      • #rename_invoice
      • 179. #update_line_item_quantity
  • 180.
      AUDITS AND CONSISTENCY
  • 181.
      Audit workflow
    • Commands
      • Stored in a sequential FIFO fashion
      • 182. Allows for compensation on multiple updates
      • 183. Entity state is driven exclusively from this
      • 184. Guaranteed – as there's no other persistence
    • Audit logs
      • Command -> Domain -> Domain event
      • 185. Aggregates domain events
      • 186. Feeds into a reporting database
  • 187.
      What's really happening here ?
    • Isolated contexts
      • Command store
      • 188. Audit log store
      • 189. Reporting database
      • 190. Agree through communication
    • Eventual consistency
      • We don't guarantee consistenty accross these contexts
      • 191. at all times
      • 192. We do guarantee that it'll be consistent eventually
      • 193. Relaxed consistency is often acceptable
  • 194.
      Relaxed Consistency
    • Viability ?
      • Discuss with domain experts
      • 195. How long is too long ?
    • Solutions
      • Explicit SLA's at the command level
      • 196. Measure propogation and flag SLA violations
  • 197.
      PERFORMANCE AND TESTING
  • 198.
      Performance benefits
    • Throughput
      • High transactions/sec rate
      • 199. Total throughput is always governed by the weakest link
    • Read / write specific optimizations
      • Latency: data being far from where it's needed
      • 200. Never block a write on waiting for data – it's given
      • 201. Can use fast disks for write oriented services
      • 202. Cache read tier ftw!
      • 203. Formal state changes == better cache invalidation
  • 204.
      Testing
    • Service / daemon structure
      • Command -> Result
      • 205. Result being either an Event or Error
    • Workflow
      • test/unit, domain elements + your service
      • 206. Layered architectured == easy to swap out layers
      • 207. Arrays to mock queue infrastructure
      • 208. Behavior driven testing
      • 209. Assert resulting events or errors
  • 210.
      Asserting events
    process MyApp::Payments do command :PayInvoice, :id => 1 assert_events :InvoicePaid, :RegenInvoice end
  • 211.
      Asserting errors
    process MyApp::Payments do command :PayInvoice, :id => 200 assert_errors :InvoiceAlreadyPaid end
  • 212.
      Commands in test cases
    def command(cmd, attrs) cmd = App::Commands.const_get(cmd).new attrs.each{|k,v| cmd[k] = v } @process << cmd end alias input command
  • 213.
      Asserting event streams
    def assert_events(*events) event_types = @output.map{|e| e.class } expected = events.map{|e| App::Events.const_get(e) } assert_equal expected, event_types, &quot;Expected events #{expected.inspect}, got #{event_types.inspect}&quot; end
  • 214.
      MESSAGING
  • 215.
      Messages - Communication
    • Communication styles
      • Fat consumer : service call to determine destination, then send, slow
      • 216. Thin consumer : fire and forget, ESB routes, fast
    • ESB
      • Take time to research this
      • 217. Understand the protocol
      • 218. Client support
      • 219. Durability and failover requirements
      • 220. Publish / subscribe is preferred
  • 221.
      Messages - Types
    • Loose coupling requirements
      • Weak typing, loose data model - hashes
      • 222. Data types not always harmonized – accept that
      • 223. Also account for foreign / API messages
    • Translation support
      • Requires mapping between systems
      • 224. Individual systems can modify their internal structures, providing we update our mapping layer
      • 225. Useful if we have foreign messages (via API) to push in our stack as well – just coerce to the internal representation
  • 226.
      Messages - Versioning
    • Embracing change
      • Structures will change
      • 227. Version backward incompatible changes
      • 228. Intermediate sensible defaults for backward compatible changes
    • Strategies
      • Let queues be a part of version strategy also
      • 229. Allows for keeping old + new messages – no loss
      • 230. Thin and temp. translator to pop messages off the old queue, translate / coerce them, push to the new one
  • 231.
      Messages - Payload
    • Minimal
      • Just enough information and context for consumers to determine if updates is relevant to them
      • 232. Consumer required to lookup any additional information required
      • 233. Couples to services + point of failure
    • Self contained
      • Includes all information a consumer would require to process
      • 234. Loosely coupled – no additional service interactions
      • 235. Watch out for size – Amazon SQS is limited to 8k
  • 236.
      Messages - Correlation
    • Why it's required
      • Troubleshooting – backtraces is always app local
      • 237. Enforcing SLAs
      • 238. Idempotency
    • Helpful correlation id elements
      • Source system
      • 239. Destination system
      • 240. /rails/user/234/session_dssasdadas/controller/action
      • 241. Consider a “touch” mecanism – history of systems the
      • 242. message has passed through
  • 243.
      QUESTIONS ?