Your SlideShare is downloading. ×
0
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
RailswayCon 2010 - Command Your Domain
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

RailswayCon 2010 - Command Your Domain

3,635

Published on

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,635
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
73
Comments
0
Likes
5
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. <ul>Command your Domain </ul><ul>Lourens Naude, WildfireApp.com </ul>
  • 2. <ul>Background </ul><ul><li>Independent Contractor </li></ul><ul><ul><li>Ruby / C / integrations
  • 3. Well versed full stack
  • 4. Architecture </li></ul></ul><ul><li>WildfireApp.com </li></ul><ul><ul><li>Social Marketing platform
  • 5. Large whitelabel clients
  • 6. Bursty traffic – Lady Gaga, EA, Gatorade </li></ul></ul>
  • 7. &nbsp;
  • 8. <ul>Audience </ul><ul><li>Monolithic applications </li></ul><ul><ul><li>Scale together, fail together
  • 9. Often have an anemic domain model, scattered logic
  • 10. Hockey stick function – fast initial growth, increasingly difficult to extend over time
  • 11. Hacks always have the longest lifecycle in a codebase </li></ul></ul><ul><li>Target platforms </li></ul><ul><ul><li>Projects with significant complexity
  • 12. Existing monolithic applications
  • 13. Environments with excessive coupling or dependencies </li></ul></ul>
  • 14. <ul>MAINTENANCE COST IS MULTIPLE TIMES MORE THAN DEVELOPMENT COST </ul>
  • 15. <ul>Anemic Domain Models </ul><ul><li>Cost to business </li></ul><ul><ul><li>No competitive advantage
  • 16. Long dev -&gt; production cycle </li></ul></ul><ul><li>Identified by ... </li></ul><ul><ul><li>Tech oriented interfaces
  • 17. High coupling to neighboring layers
  • 18. Coupled persistence
  • 19. Dependencies on host frameworks </li></ul></ul>
  • 20. <ul>Layered Architecture </ul><ul><li>Layers </li></ul><ul><ul><li>UI
  • 21. Application
  • 22. Domain
  • 23. Infrastructure </li></ul></ul><ul><li>Rails &amp;&amp; other MVC ? </li></ul><ul><ul><li>All of the above, monolithic tree
  • 24. Modeling is often data driven, not behavior driven
  • 25. Models / entities are often coupled to conventions and dependencies of the host framework </li></ul></ul>
  • 26. <ul>Problems in practice </ul><ul><li>Rails as a niche framework </li></ul><ul><ul><li>Initial conventions scale for small to medium size applications
  • 27. Fast time to market
  • 28. Request / response + rendering
  • 29. Beyond this, generally not a Rails application anymore </li></ul></ul><ul><li>Development and growth </li></ul><ul><ul><li>Your application, over time, becomes your framework for future development
  • 30. Conventions, practices and dependencies should grow with and adapt to that also </li></ul></ul>
  • 31. <ul>APPLICATIONS AS A FRAMEWORK </ul>
  • 32. <ul>Custom policies and governance </ul><ul><li>Cater for both developers + production runtime </li></ul><ul><ul><li>Environment support with sanity checks
  • 33. Configuration management
  • 34. Dependency management etc. </li></ul></ul><ul><li>How ? </li></ul><ul><ul><li>Thin framework agnostic layer that provides tooling for developers
  • 35. Piggyback off known contracts eg. cache[&apos;something&apos;]
  • 36. Deprecate framework specific references in code
  • 37. Prefer SomeApp.cache to Rails.cache etc. </li></ul></ul>
  • 38. <ul>Moving off monilithic apps </ul><ul><li>How features used to roll out </li></ul><ul><ul><li>Slapped onto the existing codebase OR
  • 39. Spawned a new project, with yet another set of conventions
  • 40. Topic / feature development branches </li></ul></ul><ul><li>Challenges </li></ul><ul><ul><li>Team likely not structured to support distributed / decoupled systems
  • 41. Extreme self-discipline, from everyone
  • 42. Business Process Model skills is required </li></ul></ul>
  • 43. <ul>DOMAIN MODELLING PRIMER </ul>
  • 44. <ul>Digesting a domain </ul><ul><li>Language </li></ul><ul><ul><li>Ban acronyms
  • 45. Push for a common language – dev, product + biz
  • 46. Be childlike.Why, why, why ... </li></ul></ul><ul><li>Open ended </li></ul><ul><ul><li>A domain representation is always in flux during development, especially if it&apos;s new to developers
  • 47. It&apos;ll be enhanced with new concepts
  • 48. Weed out vague / deprecated elements
  • 49. Refactorings at this level is often overlooked </li></ul></ul>
  • 50. <ul>Refactoring strategies </ul><ul><li>Continuously throw usage scenarios at your system
  • 51. Refactor to deeper insights
  • 52. Extract hidden concepts – look for edge cases, policies etc.
  • 53. In cooperation with domain experts
  • 54. Refactor your domain first, code second </li></ul>
  • 55. <ul>Behavior and responsibility </ul><ul><li>Ignore schema and attributes </li></ul><ul><ul><li>Should not be the core of your design process
  • 56. Assigning data to entities prior to knowing that they do ? </li></ul></ul><ul><li>Behavior </li></ul><ul><ul><li>Partition objects and entities by behavior
  • 57. Distribute responsibilities first
  • 58. Then decide what each entity needs to know </li></ul></ul>
  • 59. <ul>DOMAIN FRAGMENTATION </ul>
  • 60. <ul>Domain leaks - People </ul><ul><li>Developers </li></ul><ul><ul><li>Code ownership
  • 61. Some developers don&apos;t communicate well
  • 62. Snapshot thoughts to wiki != fun
  • 63. People come, people go </li></ul></ul><ul><li>Product and business </li></ul><ul><ul><li>Not willing to get into domain details
  • 64. Concerned with features
  • 65. Concerned with cash flow </li></ul></ul>
  • 66. <ul>Domain leaks – Process </ul><ul><li>Reorganization </li></ul><ul><ul><li>Migrates knowledge excessively
  • 67. Fine balance between code ownership and moving developers around too much </li></ul></ul><ul><li>Practices and culture </li></ul><ul><ul><li>Lack of Documentation culture
  • 68. Not factoring clued up domain experts into projects
  • 69. Shielding departments from each other – us VS them
  • 70. Visual communication.Whiteboards ? Diagrams ?
  • 71. Domain discussions encouraged ? Shot down ? </li></ul></ul>
  • 72. <ul>Domain leaks - concepts </ul><ul><li>Implicit concepts </li></ul><ul><ul><li>Instrumental in understanding a model
  • 73. But never referenced in an implementation
  • 74. Frequently would be hinted at, but may not seem important to pull in </li></ul></ul><ul><li>Explicit concepts </li></ul><ul><ul><li>Merging implicit concepts back into the domain
  • 75. Over time, this leads to breakthroughs
  • 76. Less edge cases
  • 77. Ability to handle offline process programatically </li></ul></ul>
  • 78. <ul>Notes on errors </ul><ul><li>Common error types </li></ul><ul><ul><li>Runtime errors
  • 79. Exceptions
  • 80. Errors in business logic </li></ul></ul><ul><li>Explicit domain errors </li></ul><ul><ul><li>Don&apos;t inline raise
  • 81. Do include business errors in your domain
  • 82. Define strategies for offline or programatic handling </li></ul></ul>
  • 83. <ul>STATE OF YOUR DOMAIN </ul>
  • 84. <ul>Loosely coupled domain </ul>-------------------------------------- X &lt;-- X &lt;-- X &lt;-- X --------------------------------------
  • 85. <ul>Sparse domain </ul>-------------------------------------- X &lt;-- &lt;-- &lt;-- X --------------------------------------
  • 86. <ul>Highly coupled domain </ul>-------------------------------------- X X &lt;-- X X &lt;-- X X &lt;-- XXXX --------------------------------------
  • 87. <ul>COMMUNICATION </ul>
  • 88. <ul>Value of speech </ul><ul><li>Talking out loud </li></ul><ul><ul><li>Embrace senses
  • 89. Colorful speech
  • 90. “ A valid tax invoice for taxable sales that totals x”
  • 91. Fail faster – quicker feedback loop </li></ul></ul><ul><li>Translation costs </li></ul><ul><ul><li>Statement ...
  • 92. Refine statement ...
  • 93. Agree on statement ...
  • 94. repeat </li></ul></ul>
  • 95. <ul>A Common Language </ul><ul><li>Participants </li></ul><ul><ul><li>Domain experts
  • 96. Stakeholders
  • 97. Developers </li></ul></ul><ul><li>Referenced in </li></ul><ul><ul><li>Code
  • 98. Test code
  • 99. Discussions and meetings
  • 100. Specs / diagrams
  • 101. Documentation </li></ul></ul>
  • 102. <ul>Common Language Elements </ul><ul><li>Nouns </li></ul><ul><ul><li>Entities / classes if it has identity
  • 103. Invoice, transaction, business, user, expense etc.
  • 104. Value object if there&apos;s no identity.Reusable and immutable
  • 105. Address, Money etc. (load Address via User) </li></ul></ul><ul><li>Verbs </li></ul><ul><ul><li>Services, methods etc. - behavior
  • 106. “ A transaction between businesses generates an invoice.”
  • 107. @business.sells(2, product, other_biz) </li></ul></ul>
  • 108. <ul><li>VERBS AS COMMANDS </li></ul>
  • 109. <ul><li>EVENT IS THE RESULT OF PROCESSING A COMMAND </li></ul>
  • 110. <ul>Command -&gt; event stream </ul>Sell(:seller =&gt; &apos;Acme&apos;, :quantity =&gt; 2, :product =&gt; &apos;XYZ&apos;, :buyer =&gt; &apos;MegaCorp&apos;) @business.sells(2, product, other_biz) [PaymentProcessed, TransactionCreated, InventoryUpdated, InvoiceCreated, ShippingLabelsGenerated ]
  • 111. <ul><li>EXPLICIT STATE CHANGES </li></ul>
  • 112. <ul>Pushing changes </ul><ul><li>Rethink persistence </li></ul><ul><ul><li>Remove persistance behavior from entities
  • 113. Focus on the domain, not data structure
  • 114. Delegate storage to a repository
  • 115. Easy to swap out repository implementations for testing etc. </li></ul></ul><ul><li>Domain events </li></ul><ul><ul><li>Instead of saving to the database, generate a domain event with the relevant changes as a payload
  • 116. Published to your whole stack
  • 117. Eventual consistency </li></ul></ul>
  • 118. <ul>ActiveRecord implications </ul><ul><li>Reporting contexts </li></ul><ul><ul><li>ActiveRecord::Base.read_only!
  • 119. Bypass persistence entirely – force ready-only mode
  • 120. @record.attributes #=&gt; value object
  • 121. @record.changed #=&gt; value object diff </li></ul></ul><ul><li>Repository use cases </li></ul><ul><ul><li>A Generic Repository implementation (CRUD)
  • 122. Invoice.find(1) #=&gt; entity by identity
  • 123. Invoice.find_by_refr(x) #=&gt; dynamic and explicit query methods </li></ul></ul>
  • 124. <ul>EVENTED ARCHITECTURE </ul>
  • 125. <ul>Commands </ul><ul><li>Drive change in your system </li></ul><ul><ul><li>Time driven : batched, cron etc.
  • 126. Request / Response : remote producedure calls </li></ul></ul><ul><li>Definition </li></ul><ul><ul><li>A well defined task / behavior with parameters
  • 127. Not expected to return a result
  • 128. Should be idempotent
  • 129. Should expire itself when not processed in a timeframe
  • 130. Yields an event stream through interaction with the domain when processed instead </li></ul></ul>
  • 131. <ul>Domain Events </ul><ul><li>Granularity </li></ul><ul><ul><li>InvoiceChanged, InvoiceStateChanged, InvoicePaid
  • 132. Try to signal the intent of a change
  • 133. Wrap InvoiceChanged in a module
  • 134. InvoicePaid.include InvoiceChanged </li></ul></ul><ul><li>Immutability </li></ul><ul><ul><li>Represents something that has happened
  • 135. Thus properties should never be changed </li></ul></ul>
  • 136. <ul>Event Driven Architecture (EDA) </ul><ul><li>Nervous system anology – hand on stove </li></ul><ul><ul><li>Hand is the event producer
  • 137. Spinal cord is the ESB
  • 138. Brain is the event listener
  • 139. Brain is the event processor
  • 140. Hand is the event reaction – we pull away </li></ul></ul><ul><li>Common event patterns </li></ul><ul><ul><li>Generation
  • 141. Consumption
  • 142. Processing </li></ul></ul>
  • 143. <ul>EDA attributes </ul><ul><li>Definition </li></ul><ul><ul><li>Loosely coupled / decoupled
  • 144. Async messaging
  • 145. Granular events
  • 146. Notification of state changes </li></ul></ul><ul><li>Transport </li></ul><ul><ul><li>ESB backbone
  • 147. No centralized control </li></ul></ul>
  • 148. <ul>COMMAND AND QUERY SEPERATION </ul>
  • 149. <ul>EITHER PERFORM AN ACTION OR RETURN A RESULT </ul>
  • 150. <ul>* NOT BOTH * </ul>
  • 151. <ul>COMMANDS CHANGE STATE, QUERIES EXAMINE THEM </ul>
  • 152. <ul>Major differences </ul><ul><li>Queries </li></ul><ul><ul><li>Synchronous and blocking
  • 153. Idempotent by default </li></ul></ul><ul><li>Commands </li></ul><ul><ul><li>Asyncronous – we expect no results …
  • 154. Other than a guarantee it&apos;s been received
  • 155. Should be idempotent / not reprocessed </li></ul></ul>
  • 156. <ul>Read and write seperation </ul><ul><li>Write service </li></ul><ul><ul><li>POST /invoices
  • 157. PUT /invoice/1
  • 158. DESTROY /invoice/1
  • 159. Append to a FIFO command queue </li></ul></ul><ul><li>Read service </li></ul><ul><ul><li>GET /invoice/1
  • 160. GET /invoices
  • 161. Serve from an eventually consistent reporting DB </li></ul></ul>
  • 162. <ul>“ Hot” upgrades </ul><ul><li>Scenario </li></ul><ul><ul><li>Read heavy web application
  • 163. 95% read, 5% write
  • 164. System upgrade required
  • 165. Ability to disable features is explicit in domain </li></ul></ul><ul><li>Solution </li></ul><ul><ul><li>Mirror all data
  • 166. Disable all write services + related features
  • 167. Continue serving requests </li></ul></ul>
  • 168. <ul>CRUD FAIL </ul>
  • 169. <ul>Why ? </ul><ul><li>Audits
  • 170. Can&apos;t be easily audited </li></ul><ul><ul><li>You can, but pointless if your system isn&apos;t dependent on this history – no way to trust it </li></ul></ul><ul><li>Accountant workflow </li></ul><ul><ul><li>Accountants don&apos;t change objects
  • 171. They keep histories
  • 172. No updates, no deletes
  • 173. Work is always auditable
  • 174. Totals is always based on history </li></ul></ul>
  • 175. <ul>Change oriented updates </ul><ul><li>CRUD API </li></ul><ul><ul><li>Create
  • 176. UPDATE
  • 177. Delete
  • 178. What&apos;s changed on UPDATE ? </li></ul></ul><ul><li>A better API </li></ul><ul><ul><li>#rename_invoice
  • 179. #update_line_item_quantity </li></ul></ul>
  • 180. <ul>AUDITS AND CONSISTENCY </ul>
  • 181. <ul>Audit workflow </ul><ul><li>Commands </li></ul><ul><ul><li>Stored in a sequential FIFO fashion
  • 182. Allows for compensation on multiple updates
  • 183. Entity state is driven exclusively from this
  • 184. Guaranteed – as there&apos;s no other persistence </li></ul></ul><ul><li>Audit logs </li></ul><ul><ul><li>Command -&gt; Domain -&gt; Domain event
  • 185. Aggregates domain events
  • 186. Feeds into a reporting database </li></ul></ul>
  • 187. <ul>What&apos;s really happening here ? </ul><ul><li>Isolated contexts </li></ul><ul><ul><li>Command store
  • 188. Audit log store
  • 189. Reporting database
  • 190. Agree through communication </li></ul></ul><ul><li>Eventual consistency </li></ul><ul><ul><li>We don&apos;t guarantee consistenty accross these contexts
  • 191. at all times
  • 192. We do guarantee that it&apos;ll be consistent eventually
  • 193. Relaxed consistency is often acceptable </li></ul></ul>
  • 194. <ul>Relaxed Consistency </ul><ul><li>Viability ? </li></ul><ul><ul><li>Discuss with domain experts
  • 195. How long is too long ? </li></ul></ul><ul><li>Solutions </li></ul><ul><ul><li>Explicit SLA&apos;s at the command level
  • 196. Measure propogation and flag SLA violations </li></ul></ul>
  • 197. <ul>PERFORMANCE AND TESTING </ul>
  • 198. <ul>Performance benefits </ul><ul><li>Throughput </li></ul><ul><ul><li>High transactions/sec rate
  • 199. Total throughput is always governed by the weakest link </li></ul></ul><ul><li>Read / write specific optimizations </li></ul><ul><ul><li>Latency: data being far from where it&apos;s needed
  • 200. Never block a write on waiting for data – it&apos;s given
  • 201. Can use fast disks for write oriented services
  • 202. Cache read tier ftw!
  • 203. Formal state changes == better cache invalidation </li></ul></ul>
  • 204. <ul>Testing </ul><ul><li>Service / daemon structure </li></ul><ul><ul><li>Command -&gt; Result
  • 205. Result being either an Event or Error </li></ul></ul><ul><li>Workflow </li></ul><ul><ul><li>test/unit, domain elements + your service
  • 206. Layered architectured == easy to swap out layers
  • 207. Arrays to mock queue infrastructure
  • 208. Behavior driven testing
  • 209. Assert resulting events or errors </li></ul></ul>
  • 210. <ul>Asserting events </ul>process MyApp::Payments do command :PayInvoice, :id =&gt; 1 assert_events :InvoicePaid, :RegenInvoice end
  • 211. <ul>Asserting errors </ul>process MyApp::Payments do command :PayInvoice, :id =&gt; 200 assert_errors :InvoiceAlreadyPaid end
  • 212. <ul>Commands in test cases </ul>def command(cmd, attrs) cmd = App::Commands.const_get(cmd).new attrs.each{|k,v| cmd[k] = v } @process &lt;&lt; cmd end alias input command
  • 213. <ul>Asserting event streams </ul>def assert_events(*events) event_types = @output.map{|e| e.class } expected = events.map{|e| App::Events.const_get(e) } assert_equal expected, event_types, &amp;quot;Expected events #{expected.inspect}, got #{event_types.inspect}&amp;quot; end
  • 214. <ul>MESSAGING </ul>
  • 215. <ul>Messages - Communication </ul><ul><li>Communication styles </li></ul><ul><ul><li>Fat consumer : service call to determine destination, then send, slow
  • 216. Thin consumer : fire and forget, ESB routes, fast </li></ul></ul><ul><li>ESB </li></ul><ul><ul><li>Take time to research this
  • 217. Understand the protocol
  • 218. Client support
  • 219. Durability and failover requirements
  • 220. Publish / subscribe is preferred </li></ul></ul>
  • 221. <ul>Messages - Types </ul><ul><li>Loose coupling requirements </li></ul><ul><ul><li>Weak typing, loose data model - hashes
  • 222. Data types not always harmonized – accept that
  • 223. Also account for foreign / API messages </li></ul></ul><ul><li>Translation support </li></ul><ul><ul><li>Requires mapping between systems
  • 224. Individual systems can modify their internal structures, providing we update our mapping layer
  • 225. Useful if we have foreign messages (via API) to push in our stack as well – just coerce to the internal representation </li></ul></ul>
  • 226. <ul>Messages - Versioning </ul><ul><li>Embracing change </li></ul><ul><ul><li>Structures will change
  • 227. Version backward incompatible changes
  • 228. Intermediate sensible defaults for backward compatible changes </li></ul></ul><ul><li>Strategies </li></ul><ul><ul><li>Let queues be a part of version strategy also
  • 229. Allows for keeping old + new messages – no loss
  • 230. Thin and temp. translator to pop messages off the old queue, translate / coerce them, push to the new one </li></ul></ul>
  • 231. <ul>Messages - Payload </ul><ul><li>Minimal </li></ul><ul><ul><li>Just enough information and context for consumers to determine if updates is relevant to them
  • 232. Consumer required to lookup any additional information required
  • 233. Couples to services + point of failure </li></ul></ul><ul><li>Self contained </li></ul><ul><ul><li>Includes all information a consumer would require to process
  • 234. Loosely coupled – no additional service interactions
  • 235. Watch out for size – Amazon SQS is limited to 8k </li></ul></ul>
  • 236. <ul>Messages - Correlation </ul><ul><li>Why it&apos;s required </li></ul><ul><ul><li>Troubleshooting – backtraces is always app local
  • 237. Enforcing SLAs
  • 238. Idempotency </li></ul></ul><ul><li>Helpful correlation id elements </li></ul><ul><ul><li>Source system
  • 239. Destination system
  • 240. /rails/user/234/session_dssasdadas/controller/action
  • 241. Consider a “touch” mecanism – history of systems the
  • 242. message has passed through </li></ul></ul>
  • 243. <ul>QUESTIONS ? </ul>

×