Space Based Programming


Published on

Presentation of space based programming given at Skills Matter 02 Sep 2009

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Space Based Programming

  1. 1. Space based programming [email_address] @gojkoadzic
  2. 2. Why should you care? <ul><li>It helps us build applications that: </li></ul><ul><ul><li>can scale out to lots of machines easily </li></ul></ul><ul><ul><li>can grow and shrink dynamically </li></ul></ul><ul><ul><li>have massive throughput </li></ul></ul><ul><ul><li>handle massive amounts of data </li></ul></ul>
  3. 3. So what are spaces? <ul><li>Data spaces are “network attached memory”, allowing us to read, put or take objects </li></ul><ul><li>Space takes care of redundancy, failover, transactions … </li></ul><ul><li>Alternatively, send tasks to the object and let it execute it. </li></ul>
  4. 4. The idea has been around for a while, but somehow has not caught on… however it’s coming back with a bang!
  5. 5. Another language named after a Lovelace
  6. 6. David Gelertner invents Linda in the 80’s <ul><li>Distributed processing based on tuples </li></ul><ul><li>Orthogonal process coordination </li></ul><ul><li>Data coupling rather than process coupling </li></ul>
  7. 7. Sun Jini in the 90’s <ul><li>Evolvable architectures, autodiscovery and lots of other flux capacitors nobody needed or knew how to use at the time… </li></ul>
  8. 8. Grid computing in 00's
  9. 9. Great for computations, but what about transaction processing?
  10. 10. Space-based systems will be key for cloud scalability
  11. 11. Products <ul><li> </li></ul><ul><li> </li></ul><ul><li> </li></ul><ul><li> </li></ul><ul><li> </li></ul>
  12. 12. Command Pattern (GOF)‏ <ul><li>“ is a design pattern in which an object is used to represent and encapsulate all the information needed to call a method at a later time”... (wikipedia)‏ </li></ul>
  13. 13. You need a recipient, probably an entity by ID
  14. 14. You need a “recipient”, probably an entity by ID A “command” with all the information required to run it
  15. 15. You need a “recipient”, probably an entity by ID A “command” with all the information required to run it And an “invoker” to do the job
  16. 16. And the command gets executed...
  17. 17. So what does it have to do with spaces? <ul><li>“ ...makes it easier to construct general components that need to delegate, sequence or execute method calls” (also wikipedia) </li></ul>
  18. 18. You can use many invokers
  19. 19. And do loads of work in parallel
  20. 20. And you can do something more productive with your time...
  21. 21. So what does that have to do with spaces? <ul><li>Space is where recipients reside and where you send commands </li></ul><ul><li>Lots of different processors run in the space, but from the outside appear as a single “mind” </li></ul><ul><li>This scales really well and it is virtually indestructible.... </li></ul>
  22. 22. Space: all your objects
  23. 23. Processing units (=partitions)‏
  24. 24. GigaSpace data objects [SpaceClass] public class Message { [SpaceID(AutoGenerate=true)] public String MessageId {get; set;} [SpaceRouting] public String MessageType{ get; set;} ... }
  25. 25. Space Data Properties <ul><li>[SpaceID] is unique for the class in Space </li></ul><ul><li>[SpaceRouting] determines the partition (defaults to space ID) </li></ul><ul><li>Indexes speed up queries </li></ul><ul><li>[SpaceProperty(Index=SpaceIndexType.Basic)] </li></ul><ul><li>[SpaceVersion] for optimistic locking </li></ul><ul><li>[SpaceExclude] are not serialized </li></ul>
  26. 26. Recipient (Command context)‏ <ul><li>Space object </li></ul><ul><li>Space ID is the entity ID </li></ul><ul><li>Routing ID is the same field </li></ul>
  27. 27. Commands <ul><li>Space object </li></ul><ul><li>Space ID is a GUID (can be auto-generated)‏ </li></ul><ul><li>Target recipient ID is the Routing ID </li></ul>
  28. 28. Processing Units <ul><li>Worker thread pool </li></ul><ul><li>Template matches the command </li></ul><ul><ul><li>Class matching </li></ul></ul><ul><ul><li>Property matching (if not null)‏ </li></ul></ul><ul><li>Works inside a PU container </li></ul>
  29. 29. Example processor [PollingEventDriven(MinConcurrentConsumers = 1, MaxConcurrentConsumers = 4)] internal class MessageProcessor { [EventTemplate] public Message TemplateForThisProcessor { get{ ... } } [DataEventHandler] public Message ProcessMessage(Message message) {.... } }
  30. 30. Processes <ul><li>Contain one or more processing unit containers </li></ul><ul><li>Own a space partition </li></ul><ul><li>Run on the network, balanced, clustered, backed up </li></ul>
  31. 31. Coherence - distributed HashMaps <ul><li>Works on POCO objects, but you can implement PortableObject for .NET/Java interop </li></ul><ul><li>void IPortableObject.ReadExternal(IPofReader reader) </li></ul><ul><li>{ </li></ul><ul><li>firstName = reader.ReadString(0); </li></ul><ul><li>addrHome = (Address)reader.ReadObject(1); </li></ul><ul><li> .... </li></ul><ul><li>void IPortableObject.WriteExternal(IPofWriter writer) </li></ul><ul><li>{ </li></ul><ul><li>writer.WriteString(0, firstName); </li></ul><ul><li>writer.WriteObject(1, addrHome); </li></ul>
  32. 32. Works as a hashmap <ul><li>INamedCache cache = CacheFactory.GetCache(“my map”); </li></ul><ul><li>cache.Add(key, value) </li></ul><ul><li>cache.Remove(key, value) </li></ul><ul><li>Also supports queries, notifications etc </li></ul>
  33. 33. Entry Processors – push code to objects <ul><li>cache.Insert(&quot;BGD&quot;, new Temperature(25, 'c', 12)); </li></ul><ul><li>IValueUpdater updater = new ReflectionUpdater(&quot;setDegree&quot;); </li></ul><ul><li>IEntryProcessor processor = new UpdaterProcessor(updater, 26); </li></ul><ul><li>object result = cache.Invoke(&quot;BGD&quot;, processor); </li></ul>
  34. 34. Key ideas to do it efficiently <ul><li>Forget about n-tier systems </li></ul><ul><li>Group data together with all processes </li></ul><ul><li>Ensure that invokers have all the information needed to run (so no unnecessary serialization)‏ </li></ul><ul><li>Ensure that the recipients are the correct aggregates for execution (so low contention during execution)‏ </li></ul><ul><li>Use asynchronous persistence </li></ul>
  35. 35. That's it for now... <ul><li> </li></ul><ul><li> </li></ul><ul><li>October 1st, Mike Hadlow on MassTransit </li></ul>