Agents and Agency in the Internet by Greg Meredith

2,154 views

Published on

“Agents and Agency in the Internet,” Greg Meredith talks about SpecialK/KVDB, a scala library for writing distributed applications in an embedded DSL that’s based on the applied Pi-calculus. He focuses on two applications that are using SpecialK, Protunity and the Individual Agent.
Greg will introduce the embedded DSL and its relationship to theory and practice. He will then look at the both of the different applications and the talk on the advantages such framework brings to those efforts. Of particular interest is how the framework allows for a higher level automated reasoning about information flow in the distributed network.

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,154
On SlideShare
0
From Embeds
0
Number of Embeds
1,694
Actions
Shares
0
Downloads
6
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Agents and Agency in the Internet by Greg Meredith

  1. 1. ScalaBay October Meetup Agents and Agency in the Internet: How we get there is where we are going L.G. Meredith Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  2. 2. ScalaBay October Meetup Outline • A few verbs • Their implementation • A b2b application using them • A c2c application using them • Summary • Questions L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  3. 3. ScalaBay October Meetup A few verbs • These verbs are designed to make it easy for distributed applications talk to one another • They are built around the constructive interpretation of a single logical principle: A or ~A L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  4. 4. ScalaBay October Meetup A few verbs Let’s illustrate with a cartoon. key key key key key value value value value value L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  5. 5. ScalaBay October Meetup in 5 mins in 10 mins in 2 mins Let's meet at Ladro in 10 | L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13 8 hr getting coffee at Ladro 4 hr Let’s imagine an app...
  6. 6. ScalaBay October Meetup in 5 mins Looks like we'll all be there in 10 mins in 2 mins Let's meet at Ladro in 10 | L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13 8 hr getting coffee at Ladro 4 hr Let’s imagine an app...
  7. 7. ScalaBay October Meetup A few verbs An application like this could benefit from the ability to • • • • Make a standing query Post data to standing queries Store data conventionally Read data conventionally L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  8. 8. ScalaBay October Meetup A few verbs 8 hr 4 hr in 5 mins Looks like we'll all be there Looks like we'll all be there in 10 mins in 10 mins in 2 mins in 2 mins Let's meet at Ladro in 10 | Let's meet at Ladro in 10 | key key key key key value value value value value L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13 8 hr in 5 mins 4 hr getting coffee at Ladro getting coffee at Ladro
  9. 9. ScalaBay October Meetup A few verbs 8 hr in 5 mins 4 hr getting coffee at Ladro When a query arrives at a key and there is no data a continuation is stored Looks like we'll all be there in 10 mins in 2 mins Let's meet at Ladro in 10 | key key key key key value value value value ? L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  10. 10. ScalaBay October Meetup in 5 mins Looks like we'll all be there in 10 mins When data arrives at a key and there is no continuation, the data is stored key key key key key in 2 mins Let's meet at Ladro in 10 | value value value value value value L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13 8 hr getting coffee at Ladro 4 hr A few verbs
  11. 11. ScalaBay October Meetup A few verbs We can make some decisions about the nature of the interaction with respect to the stored entity (continuation or data) • The action can consume the stored entity • The action can make a copy the stored entity and allow interaction on the copy L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  12. 12. ScalaBay October Meetup A few verbs ephemeral - data persistent - data ephemeral - data persistent - data ephemeral - k ephemeral - k peristent - k persistent - k producer put store publish publish with history consumer get read subscribe subscribe L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  13. 13. ScalaBay October Meetup A few verbs ephemeral - data persistent - data ephemeral - data persistent - data ephemeral - k ephemeral - k peristent - k persistent - k producer put store publish publish with history consumer get read subscribe subscribe Traditional DB operations L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  14. 14. ScalaBay October Meetup A few verbs ephemeral - data persistent - data ephemeral - data persistent - data ephemeral - k ephemeral - k peristent - k persistent - k producer put store publish publish with history consumer get read subscribe subscribe Traditional messaging operations L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  15. 15. ScalaBay October Meetup A few verbs ephemeral - data persistent - data ephemeral - data persistent - data ephemeral - k ephemeral - k peristent - k persistent - k producer put store publish publish with history consumer get read subscribe subscribe We use these for itemlevel locking in a distributed setting L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  16. 16. ScalaBay October Meetup A few verbs How is this related to the constructive interpretation of the logical principle A or ~A L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  17. 17. ScalaBay October Meetup A few verbs Now, how is this related to the π-calculus? L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  18. 18. ScalaBay October Meetup A few verbs getting 8 hr in 5 mins 4 hr coffee at Ladro Looks like we'll all be there in 10 mins in 2 mins Let's meet at Ladro in 10 | key key key key key Let’s design a DSL for these verbs. Following LINQ we’ll have consumption map to forcomprehensions. value value value value ? for( e <- collection.<verb>( key ) if cond ) { handle( e ) } L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  19. 19. ScalaBay October Meetup A few verbs Looks like we'll all be there in 10 mins in 2 mins Let's meet at Ladro in 10 | value collection.<verb>( key, value ) key key key key key value value value value value L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13 8 hr in 5 mins 4 hr And we’ll have production follow messaging style getting coffee at Ladro
  20. 20. ScalaBay October Meetup A few verbs If we squint a little we can just make out the π-calculus. L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  21. 21. Looks like we'll all be there ScalaBay October Meetup g e t t i n g c o f f e e a t L a d r o in 5 min s in 10 min s in 2 min s Let's meet at Ladro in 10 | check cache recurse get JVM cache JVM cache forward check store AMQP key key key key key value value value value value AMQP key key key key key L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13 value value value value value
  22. 22. ScalaBay October Meetup How does this relate to agents in the Internet? key key key key key value value value value value key key key key key value value value value value Each channel has its own store A connection between two agents is treated as a pair of channels L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  23. 23. ScalaBay October Meetup How does this relate to agents in the Internet? key key key key key value value value value value key key key key key value value value value value Channels are subpartitioned by keys L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  24. 24. Putting it all together for( event1 <- kvdb1.get( pattern1 ); event2 <- kvdb2.get( pattern2 ); if ( cond( event1, event2 ) ) ){ Agent handle( event1, event2 ) Behavior } A generic process agent is an instance of an agent behavior. These run on platform agents. An agent in the sense Fred defines is realized as an instance of a specific agent behavior that implements one or more of agent services protocols including introduction, invitation, verification, etc. KVDB1 Platform Agent KVDB1, KDVB2 Platform Agent Scala Runtime Scala Runtime RabbitMQ JVM RabbitMQ RabbitMQ Queue JVM XMLDB KVDB1 RabbitMQ Queue RabbitMQ Queue XMLDB Platform Agent Scala Runtime JVM RabbitMQ Queue KVDB2 Platform Agent Scala Runtime RabbitMQ KVDB2 JVM RabbitMQ Queue Scala Runtime JVM XMLDB RabbitMQ Queue Wednesday, October 30, 13 RabbitMQ XMLDB RabbitMQ Platform Agent XMLDB
  25. 25. What is SpecialK? Example only! Not actual implementation for( introReq <- kvdb1.get( introduce( reqAgentId, trgtAgentId ) ) ){ kvdb1.put( acceptIntro?( reqAgentId, trgtAgentId ) ) for( introRsp <- kvdb.get( introRsp( trgtAgentId, reqAgentId ) ) { introRsp match { case Confirm( sharingLevel ) => ... case Deny( reason ) => ... } } Introduction } Behavior Agent services agent Agent properties Process agent KVDB verbs Agent addressing for( event1 <- kvdb1.get( pattern1 ); event2 <- kvdb2.get( pattern2 ); if ( cond( event1, event2 ) ) ){ handle( event1, event2 ) } @sequence( @act( verb, patternSpec ) @separate( P1, P2 ) ) Property Specification Agent Behavior Scala annotation syntax not finalized! get, put, subscribe, publish, fetch, store, ... over patterns described by pattern ::= groundValue | variable | symbol( pattern* ) Wednesday, October 30, 13 KDVB capability
  26. 26. ScalaBay October Meetup A few verbs in action Here’s a version of the introduction protocol for more realz and less lolz L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  27. 27. ScalaBay October Meetup Backup slides L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  28. 28. ScalaBay October Meetup Compositionality In computing scaling sometimes refers to lots of copies the same thing Lots of chips, lots of boxes, lots of databases, lots of servers, lots of data centers L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  29. 29. ScalaBay October Meetup Compositionality Compositionality turns that upside down. It says that scaling is about making the large reflect the small. L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  30. 30. ScalaBay October Meetup Compositionality This is how nature scales. L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  31. 31. ScalaBay October Meetup Compositionality But, it’s actually a fairly new thing that humans have begun to structuring their thinking and doing this way L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  32. 32. ScalaBay October Meetup Compositionality Here are a few people whose computationally effective ideas were not compositional As a result, the physics of the small doesn’t scale to the physics of the large (or vice versa) L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  33. 33. ScalaBay October Meetup Compositionality Computing is naturally organized compositionally L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  34. 34. ScalaBay October Meetup Compositionality Data structures are organized compositionally • • • Lists Trees ... L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  35. 35. ScalaBay October Meetup Compositionality The syntax of programming languages are organized compositionally P ::= ... val identifier [: typeDecl] = valueExpr ; P L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  36. 36. ScalaBay October Meetup Compositionality The fundamental organization of the mathematics of computation is compositional • • • λ-calculus • M, N ::= x | λ x. M | M N π-calculus • P, Q ::= 0 | x?( y )P | x!( v ) | P|Q | (new x)P | !P Category Theory L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  37. 37. ScalaBay October Meetup Agility Agility is not just about being quick, light and responsive L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  38. 38. ScalaBay October Meetup Agility It’s primarily about being iterative -- moving from one expression of the whole to the next L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  39. 39. ScalaBay October Meetup Agility This is really applying compositionality to scaling processes that occur in time As such, thinking about the software development process as finding a fix-point of an iterated transformation goes at least as far back as the 70’s L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  40. 40. ScalaBay October Meetup Agility This also lines up with the organization of the maths underlying computation Y = λf.(λx.f (x x)) (λx.f (x x)) L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  41. 41. ScalaBay October Meetup Agility def Y[A,B](f: (A=>B)=>(A=>B)) = { case class W(wf: W=>A=>B) { def apply(w: W) = wf(w) } val g: W=>A=>B = w => f(w(w))(_) g(W(g)) } L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  42. 42. ScalaBay October Meetup Agility val fac = Y[Int, Int](f => i => if (i <= 0) 1 else f(i - 1) * i) fac(6) //> res0: Int = 720   val fib = Y[Int, Int](f => i => if (i < 2) i else f(i - 1) + f(i - 2)) fib(6) //> res1: Int = 8 L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  43. 43. ScalaBay October Meetup Agility Iteratively moving from a whole expression of a solution to a more refined version of a whole expression of a solution: • • • makes sense mathematically makes sense organizationally is what happens in nature L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  44. 44. ScalaBay October Meetup Transparency Transparency lets us see the inner workings of a solution trace the lines from form to function L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  45. 45. ScalaBay October Meetup Transparency In computing transparency is a form of compositionality -- a semantics for a model of computation is compositional if we follow the form of a program to determine its semantics, i.e. what it is and/or what it does L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  46. 46. ScalaBay October Meetup Transparency In software development transparency is the ability to trace from requirements all the way down to lines of code L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  47. 47. ScalaBay October Meetup Transparency In organizations transparency is the ability for any one member to trace information and communication all the way through the chain or collection of people in the organization who come together to achieve a goal L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  48. 48. ScalaBay October Meetup Transparency In each case there is an inner structure that supports and guides the process Making that inner structure explicit and visible is a vital part of that support L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  49. 49. ScalaBay October Meetup Accountability We stand by the results of our work We are clear about the conditions under which people may rely on the results of our work L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  50. 50. ScalaBay October Meetup Accountability In computing accountability is closely connected to proof. We stand by a claim about how computation works by being able to produce, on demand, a proof of that claim. Proofs only work when the necessary hypothesis explicit and clear. L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  51. 51. ScalaBay October Meetup Accountability In software development -- like science -accountability is closely connected to testing. We stand by a claim about what our software can and cannot do by producing the tests that demonstrate the software’s behavior. Tests results are only reproduceable if we are clear about the test conditions. L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  52. 52. ScalaBay October Meetup Accountability In organizations accountability is closely connected to responsibility. Standing by the results of our work doesn’t mean we don’t make mistakes. It means we own them and are willing to use them as a means of improving our offering (see agility). L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  53. 53. ScalaBay October Meetup Summary Fostering compositionality agility transparency accountability is like following a hidden score L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  54. 54. ScalaBay October Meetup Summary Code and systems scale because they are composed of reconfigurable components the behavior of which we stand by and support L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  55. 55. ScalaBay October Meetup Summary Teams scale because they are composed of agile and responsible players supported by the inner working of their organization L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  56. 56. ScalaBay October Meetup Summary In short, these are the building blocks of self-organization that recognize selves are made of selves are made of ... that become coherent when they are both supported and relied on L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13
  57. 57. ScalaBay October Meetup That’s how we do it if how we get there is where we are going L.G. Meredith, Managing Partner, Biosimilarity LLC Wednesday, October 30, 13

×