From Field-based Coordination to Aggregate Computing
1. From Field-Based Coordination to Aggregate
Computing
Mirko Viroli 1 Jacob Beal 2 Ferruccio Damiani 3
Giorgio Audrito 3 Roberto Casadei 1 Danilo Pianini 1
1Alma Mater Studiorum—Universit`a di Bologna, Italy
{mirko.viroli, roby.casadei, danilo.pianini}@unibo.it
2RaytheonBBN Technologies, USA
jakebeal@ieee.org
3Universit`a di Torino, Italy
{damiani, audrito}@di.unito.it
COORDINATION’18
Madrid, 2018/06/19
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 1 / 61
2. The IoT is becoming a crowded and complex place..
Future and emerging Internet-of-things are witnessing..
increasing availability of wearable / mobile / embedded / flying devices
increasing availability of heterogeneous wireless connectivity
increasing availability of computational resources (device/edge/cloud)
increasing production/analysis of data, everywhere, anytime
⇒ business / security / privacy concerns will probably be drivers, too
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 2 / 61
3. What to do? The challenge..
The challenge
Just directly consider the worst scenario possible..
zillion devices unpredictably located and moving in the environment
heterogeneous displacement, pervasive sensing/actuation
computational services are mostly contextual (i.e. proximity-based)
underlying platform (edge/fog/cloud) as an orthogonal issue
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 3 / 61
4. What to do? The challenge..
The challenge
Just directly consider the worst scenario possible..
zillion devices unpredictably located and moving in the environment
heterogeneous displacement, pervasive sensing/actuation
computational services are mostly contextual (i.e. proximity-based)
underlying platform (edge/fog/cloud) as an orthogonal issue
Let’s try to program system coordination for that “computational system”!
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 3 / 61
5. What to do? The challenge..
The challenge
Just directly consider the worst scenario possible..
zillion devices unpredictably located and moving in the environment
heterogeneous displacement, pervasive sensing/actuation
computational services are mostly contextual (i.e. proximity-based)
underlying platform (edge/fog/cloud) as an orthogonal issue
Let’s try to program system coordination for that “computational system”!
What is the right abstraction?
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 3 / 61
6. Towards “Aggregate Computing”
Systems of interest: collective adaptive situated systems CASS
(possibly very large scale) collective adaptive systems
deployed in physical space (situated, spatial), i.e., IoT-oriented
complex (open, dynamic, in need of much self-*)
Aggregate Computing
the “good” computing/programming model for CASS coordination
it gives nice abstractions, promoting solid engineering principles
it is supported by a toolchain for design and programming
simple idea, few constructs, rather tractable
somehow “very different”
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 4 / 61
7. Towards “Aggregate Computing”
Systems of interest: collective adaptive situated systems CASS
(possibly very large scale) collective adaptive systems
deployed in physical space (situated, spatial), i.e., IoT-oriented
complex (open, dynamic, in need of much self-*)
Aggregate Computing
the “good” computing/programming model for CASS coordination
it gives nice abstractions, promoting solid engineering principles
it is supported by a toolchain for design and programming
simple idea, few constructs, rather tractable
somehow “very different”
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 4 / 61
9. Three key messagges in this talk
Aggregate computing:
1. has a functional compositional model for distributed algorithms
2. has a practical toolchain (ScaFi): experiment with it!
3. targets abstractions and platforms for future IoT
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 6 / 61
10. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Outline
1 Roots: from Coordination to Self-Organisation
2 Aggregate Computing and Field calculus
3 Aggregate Programming in ScaFi
Aggregate Computing API/DSL
Aggregate Computing Platform
4 Perspectives and Roadmap
5 Conclusion
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 7 / 61
11. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Increasing complexity in system coordination
Generative communication
Single-space for process coordination (decoupling, synchronisation, communication
⇒ Linda as archetypal case, with all the many variations
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 8 / 61
12. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Increasing complexity in system coordination
Generative communication
Single-space for process coordination (decoupling, synchronisation, communication
⇒ Linda as archetypal case, with all the many variations
Coordination rules
Adding to a space the ability of programming the coordination rules
⇒ Shared Prolog, MARS, ReSpecT, . . .
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 8 / 61
13. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Increasing complexity in system coordination
Generative communication
Single-space for process coordination (decoupling, synchronisation, communication
⇒ Linda as archetypal case, with all the many variations
Coordination rules
Adding to a space the ability of programming the coordination rules
⇒ Shared Prolog, MARS, ReSpecT, . . .
Distribution
Multiple spaces in multiple locations, addressing distributed/mobile systems
⇒ JavaSpaces, TSpaces, Lime, Klaim, . . .
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 8 / 61
14. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Increasing complexity in system coordination
Generative communication
Single-space for process coordination (decoupling, synchronisation, communication
⇒ Linda as archetypal case, with all the many variations
Coordination rules
Adding to a space the ability of programming the coordination rules
⇒ Shared Prolog, MARS, ReSpecT, . . .
Distribution
Multiple spaces in multiple locations, addressing distributed/mobile systems
⇒ JavaSpaces, TSpaces, Lime, Klaim, . . .
Nature-inspired, adaptive coordination
Programming distributed behaviour by some natural inspiration
⇒ TOTA (physics), chemTS (chemistry), SwarmLinda (biology), SAPERE (ecology)
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 8 / 61
15. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Other works addressing “spatial aggregation” [Bea+13]
Languages abstracting individual networked devices
Hood, Meld, Gro, MPI, . . .
Spatial patterns and languages
Growing Point Language, Origami Shape, Self-healing Geometries, . . .
Languages for summarising/streaming info over space-time regions
E.g., Regiment, TinyLime, TinyDB, Cougar, . . .
Foundation of group interactions
E.g., 3π, Shape Calculus, bi-graphs, SMuC, . . .
Space-time computing models
E.g., MGS, Proto, . . .
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 9 / 61
16. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
A source of problems: the IoT
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 10 / 61
17. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Gathering local context
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 11 / 61
18. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Crowd Detection
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 12 / 61
19. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Crowd Anticipation
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 13 / 61
20. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Crowd-aware Steering
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 14 / 61
21. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Crowd dispersal
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 15 / 61
22. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Crowd evacuation upon alerts
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 16 / 61
23. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Outline
1 Roots: from Coordination to Self-Organisation
2 Aggregate Computing and Field calculus
3 Aggregate Programming in ScaFi
Aggregate Computing API/DSL
Aggregate Computing Platform
4 Perspectives and Roadmap
5 Conclusion
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 17 / 61
24. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Manifesto of aggregate computing
Motto: program the aggregate, not individual devices!
1. The reference computing machine
⇒ an aggregate of devices as single “body”, fading to the actual space
2. The reference elaboration process
⇒ atomic manipulation of a collective data structure (a field)
3. The actual networked computation
⇒ a proximity-based self-organised system hidden “under-the-hood”
neighborhood
device
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 18 / 61
25. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Computational Fields [MZ09; Bea+13]
Traditionally a map: Space → Values
possibly: evolving over time, dynamically injected, stabilising
smoothly adapting to very heterogeneous domains
more easily “understood” on continuous and flat spatial domains
ranging to: booleans, reals, vectors, functions
boolean channel in 2D numeric partition in 2D gradient in 2D
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 19 / 61
26. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
The “channel” example: computing a route
Atemporal visualisation of the “channel” boolean field
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 20 / 61
27. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
The “channel” example: computing a route
Atemporal visualisation of the “channel” boolean field
the figure shows a stabilised time-snapshot of the field,
in time, there’s a transient for the formation of the field
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 20 / 61
28. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
The “channel” example: computing a route
How would you program it?
how could the channel program be platform-independent,
unaware of global map, resilient to changes, faults,..
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 21 / 61
29. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Aggregate programming as a functional approach
Functionally composing fields
Inputs: sensor fields, Output: actuator field
Computation is a pure function over fields (time embeds state!)
⇒ for this to be practical/expressive we need a good programming language
source destination
gradient distancegradient
<=
+
dilate
width
37
10
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 22 / 61
30. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Preview / requirement
How we want that computation to be expressed?
source, dest and width as inputs
gradient, distance and dilate as reusable functions
⇒ note we are reusing and composing global-level, aggregate specs
source destination
gradient distancegradient
<=
+
dilate
width
37
10
;; Here the ‘‘aggregate’’ nature of our approach gets revealed
def channel(source, dest, width) {
dilate( gradient(source) + gradient(dest) <= distance(source,dest), width )
}
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 23 / 61
31. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Field calculus [Dam+15]
Key idea
a sort of λ-calculus with “everything is a field” philosophy!
Syntax (slightly refactored, semi-formal version of [Dam+15])
e ::= x v e(e1, . . . , en) rep(e0){e} nbr{e} (expr)
v ::= < standard-values > λ (value)
λ ::= f o (x)=>e (functional value)
F ::= def f(x) {e} (function definition)
Few explanations
v includes numbers, booleans, strings,..
..tuples/vectors/maps/any-ADT (of expressions)
f is a user-defined function (the key aggregate computing abstraction)
o is a built-in local operator (pure math, local sensors,..)
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 24 / 61
32. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Intuition of global-level (denotational) semantics
The four main constructs at work
⇒ values, application, evolution, and interaction – in aggregate guise
e ::= . . . v e(e1, . . . , en) rep(e0){e} nbr{e}
0
(x)=>x+1
true t<0,1>
()0
1
+
-
1
-1
ef(0,1)
ef
rep
0
(x)=>x+1
t
v0
t
v1
..
rep(0){(x)=>x+1}
nbr de
nbr{e}
φd=[d1→v1,..,dn→vn]
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 25 / 61
33. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Aggregate programming as a functional approach
Functionally composing fields
...so, is field calculus language practical/expressive?
source destination
gradient distancegradient
<=
+
dilate
width
37
10
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 26 / 61
34. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
A preview: the channel pattern
def gradient(source){ ;; reifying minimum distance from source
rep(Infinity) { ;; distance is infinity initially
(distance) => mux(source, 0, minHood( nbr{distance} + nbrRange ))
} }
def distance(source, dest) { ;; propagates minimum distance between source and dest
snd( ;; returning the second component of the pair
rep(pair(Infinity, Infinity)) { ;; computing a field of pairs (distance,value)
(distanceValue) => mux(source, pair(0, gradient(dest)),
minHood( ;; propagating as a gradient, using for first component of the pair
pair(fst(nbr{distanceValue}) + nbrRange, snd(nbr{distanceValue}))))
} ) }
def dilate(region, width) { ;; a field of booleans
gradient(region) < width
}
;; Here the ‘‘aggregate’’ nature of our approach gets revealed
def channel(source, dest, width) {
dilate( gradient(source) + gradient(dest) <= distance(source,dest), width )
}
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 27 / 61
35. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Channel in action: note inherent self-stabilisation
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 28 / 61
36. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Key idea of the functional semantics
A field as a space-time (event) structure: φ : D → V
event E: a triple δ, t, p – device δ, “firing” at time t in position p
events domain D: a coherent set of events (devices cannot move too fast)
field values V : any data value
Time
Space
Domain
Field computations:
(computable, causal) functions over fields: φ × φ × . . . × φ → φ
input/output fields share the same domain D
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 29 / 61
37. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Key idea of the functional semantics
A field as a space-time (event) structure: φ : D → V
event E: a triple δ, t, p – device δ, “firing” at time t in position p
events domain D: a coherent set of events (devices cannot move too fast)
field values V : any data value
Time
Space
Domain
Time Space
Field
Field computations:
(computable, causal) functions over fields: φ × φ × . . . × φ → φ
input/output fields share the same domain D
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 29 / 61
38. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Key idea of the functional semantics
A field as a space-time (event) structure: φ : D → V
event E: a triple δ, t, p – device δ, “firing” at time t in position p
events domain D: a coherent set of events (devices cannot move too fast)
field values V : any data value
Time
Space
Domain
Time Space
Field
Field computations:
(computable, causal) functions over fields: φ × φ × . . . × φ → φ
input/output fields share the same domain D
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 29 / 61
39. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
On the field calculus
Practically, it can express:
complex spreading / aggregation / decay functions [BPV15]
spatial leader election, partitioning, consensus [DVB16]
distributed spatio-temporal sensing [PBV16]
splitting in parallel independent subprocesses [PBV16]
dynamic deployment/spreading of code [Dam+15]
implicit/explicit device selection of what code execute [Vir+15]
“collective teams” forming based on the selected code [Vir+15]
distributed trust mechanisms [CAV18]
⇒ ..and critically, we can smoothly compose/nest the above features..
Key properties
Self-stabilising fragment [Vir+18]
Eventual consistent fragment (a.k.a. density-independence) [Bea+17]
Universality (see Audrito’s talk)
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 30 / 61
40. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Outline
1 Roots: from Coordination to Self-Organisation
2 Aggregate Computing and Field calculus
3 Aggregate Programming in ScaFi
Aggregate Computing API/DSL
Aggregate Computing Platform
4 Perspectives and Roadmap
5 Conclusion
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 31 / 61
41. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
ScaFi: Scala with Computational Fields
Goal: bring Aggregate Computing to mainstream SW development
What
ScaFi [CV16b] is a library/framework for building aggregate systems
Scala-internal DSL for expressing aggregate computations
Linguistic support + execution support (interpreter/VM)
Correct, complete, efficient impl. of the Higher-Order Field
Calculus [Dam+15]
Distributed platform for execution of aggregate systems
Actor-based implementation (based on Akka)
Support for multiple architectural styles and system configurations
⇒ https://github.com/scafi/scafi
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 32 / 61
42. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Outline
1 Roots: from Coordination to Self-Organisation
2 Aggregate Computing and Field calculus
3 Aggregate Programming in ScaFi
Aggregate Computing API/DSL
Aggregate Computing Platform
4 Perspectives and Roadmap
5 Conclusion
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 33 / 61
43. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Aggregate Programs
So, what is an aggregate program?
The “global” program
“Local” programs obtained via global-to-local mapping
An aggregate program consists of
1) A set of function definitions
2) A body of expressions.
Example: an aggregate program in ScaFi
class MyProgram extends AggregateProgram with MyAPI {
def isSource = sense[Boolean]("source")
// Entry point for execution
override def main() = gradient(isSource)
}
Each device of the aggregate system is given an instance of MyProgram
Each device repeatedly runs main at async rounds of execution
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 34 / 61
44. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Computing with fields
Expressing aggregate/field computations in ScaFi
trait Constructs {
def rep[A](init: A)(fun: (A) => A): A
def nbr[A](expr: => A): A
def foldhood[A](init: => A)(acc: (A,A)=>A)(expr: => A): A
def branch[A](cond: => Boolean)(th: => A)(el: => A): A
// Foundational built-ins
def mid(): ID
def sense[A](name: LSNS): A
def nbrvar[A](name: NSNS): A
}
Mechanisms for context-sensitiveness: nbr, nbrvar, sense
Mechanisms for field evolution: rep
Mechanisms for (structurally aligned) interaction: nbr
Mechanisms for field domain restriction/partitioning: branch
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 35 / 61
45. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Aggregate (computing) systems & Execution model
Structure ⇒ (network/graph)
A set of devices (aka nodes/points/things)
Communication links based on an neighbouring relation
App-specific; usually mirrors space-time locality (cf., broadcast range)
Asynchronous, round-based dynamics
(1) Retrieve context
⇐ Local state at the previous round (cf., reps)
⇐ Messages from neighbours (cf., nbrs)
⇐ Sensor values (cf., senses)
(2) Aggregate program execution
⇒ export (a tree-like descriptor) + output (result of main expr)
(3) Broadcast export to neighbourhood
(4) Execute actuation
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 36 / 61
46. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Tree evaluation: pictorial semantics
Neighbour trees Evaluation tree
rep
if
nbr fun
time
persistence
domain
restriction
neighbour
alignment
tree
expansion
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 37 / 61
47. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
A mini-tutorial: http://github.com/metaphori/coord18-demos
[00] 1 // Local & field value (constant, uniform)
[01] 2 + 3 // Expression
[02] (10, 20) // Tuple
[03] Random.nextInt(1000) // Random int < 1000 (implementation-specific)
[04] sense[Boolean]("sens1")// Read sensor (non-const, non-uniform)
[05] if(sense[Boolean]("sens1") && sense[Boolean]("sens2")) "hot" else "cold"
[06] mid() // Device IDs (const for fixed domain, non-uniform)
[07] def sense1() = sense[Boolean]("sens1") // Function for reuse
foldhood(0)(_+_){ if(nbr{sense1}) 1 else 0 }
// Fold over neighbouring value with given init val and aggregator
[08] (minHood{nbrRange}, minHoodPlus{nbrRange})
[09] rep(0){ x => x + 1 }
[10] rep(Random.nextInt(100)){ x => x }
[11] rep(0){ x => x + rep(Random.nextInt(100))(y => y) }
[12] rep(Double.PositiveInfinity)(distance =>
mux(sense[Boolean]("source")){ 0.0 }{
minHoodPlus(nbr{distance} + nbrRange)
})
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 38 / 61
48. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Exports
Interaction protocol: export broadcasting to neighbours
Exports are descriptors for local computations
Support aligned interaction and hence compositionality
Bigger programs ⇒ bigger exports, still same protocol!
def distanceTo(src: Boolean): Double = rep(Double.PositiveInfinity)(distance =>
mux(src){ 0.0 }{ minHoodPlus(nbr{distance} + nbrRange) })
def main: Double =
branch(sense[Boolean]("withinAreaOfInterest")){ // Splits the domain
distanceTo(sense[Boolean]("source")) // * ’then’ branch
}{ Double.PositiveInfinity } // ’ * ’else’ branch
yields an export like the following (on nodes where sensor is active)
Map(P:/ -> 2.0, // Root value (computation result)
P:/Branch(0,true) -> 2.0 // Node corresp. to branch(c){t}{e}
P:/Branch(0,true)/Rep(0) -> 2.0,
P:/Branch(0,true)/Rep(0)/FoldHood(0) -> 2.0, // Cf., *hoodPlus ops
P:/Branch(0,true)/Rep(0)/FoldHood(0)/Nbr(0) -> 5, // For nbr{distance}
P:/Branch(0,true)/Rep(0)/FoldHood(0)/Nbr(1) -> 2.0))
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 39 / 61
49. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Example: distributed sensing
trait DSensing extends BlockG with BlockC with BlockS { I: AggregateProgram =>
def distributedSense(size:Double, metric: =>Double, v:Double): Double = {
val leaders = S(size, metric)
val potential = distanceTo(leaders, metric)
val collection = sumCollect(potential, v)
val count = sumCollect(potential, 1.0)
val avg = collection / count
broadcast(leaders, avg)
}
def sumCollect(potential: Double, v: Double): Double = C(potential, _+_, v, 0)
def broadcast[V: Bounded](src: Boolean, v: V): V = G(src, v, x => x, nbrRange)
}
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 40 / 61
50. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Example: distributed sensing
trait DSensing extends BlockG with BlockC with BlockS { I: AggregateProgram =>
def distributedSense(size:Double, metric: =>Double, v:Double): Double = {
val leaders = S(size, metric)
val potential = distanceTo(leaders, metric)
val collection = sumCollect(potential, v)
val count = sumCollect(potential, 1.0)
val avg = collection / count
broadcast(leaders, avg)
}
def sumCollect(potential: Double, v: Double): Double = C(potential, _+_, v, 0)
def broadcast[V: Bounded](src: Boolean, v: V): V = G(src, v, x => x, nbrRange)
}
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 40 / 61
51. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Example: distributed sensing
trait DSensing extends BlockG with BlockC with BlockS { I: AggregateProgram =>
def distributedSense(size:Double, metric: =>Double, v:Double): Double = {
val leaders = S(size, metric)
val potential = distanceTo(leaders, metric)
val collection = sumCollect(potential, v)
val count = sumCollect(potential, 1.0)
val avg = collection / count
broadcast(leaders, avg)
}
def sumCollect(potential: Double, v: Double): Double = C(potential, _+_, v, 0)
def broadcast[V: Bounded](src: Boolean, v: V): V = G(src, v, x => x, nbrRange)
}
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 40 / 61
52. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Example: distributed sensing
trait DSensing extends BlockG with BlockC with BlockS { I: AggregateProgram =>
def distributedSense(size:Double, metric: =>Double, v:Double): Double = {
val leaders = S(size, metric)
val potential = distanceTo(leaders, metric)
val collection = sumCollect(potential, v)
val count = sumCollect(potential, 1.0)
val avg = collection / count
broadcast(leaders, avg)
}
def sumCollect(potential: Double, v: Double): Double = C(potential, _+_, v, 0)
def broadcast[V: Bounded](src: Boolean, v: V): V = G(src, v, x => x, nbrRange)
}
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 40 / 61
53. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Outline
1 Roots: from Coordination to Self-Organisation
2 Aggregate Computing and Field calculus
3 Aggregate Programming in ScaFi
Aggregate Computing API/DSL
Aggregate Computing Platform
4 Perspectives and Roadmap
5 Conclusion
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 41 / 61
54. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Aggregate Computing and Actor Systems [CV16a]
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 42 / 61
55. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Akka-based distributed platform
Goal: easy configuration and execution of aggregate systems
Setup of an aggregate application
Setup of individual nodes/subsystems
Support for multiple architectural styles and system configurations
Fully peer-to-peer
Server-based (mediating communications)
Implemented on top of the Akka actor framework
An actor is a (re)active entity that
encapsulates a state, behaviour, and control (autonomy);
has a global UID that allows for location transparency;
interacts with other actors via async message passing
ScaFi components are actors: devices, sensors, schedulers...
OO-facade API & message-based API (exposing underlying actors)
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 43 / 61
56. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
(Fully operational) Device actors
Responsibilities (each assigned to an actor)
(i) Sensing/actuating
(ii) Aggregate program execution
(iii) Broadcasting/receiving the exports of computation to/from neighbours
(iv) Storing state (last computation, exports from neighbours)
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 44 / 61
57. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Setup of an (actor-based) aggregate system
// STEP 1: CHOOSE INCARNATION
import scafi.incarnations.{ BasicActorP2P => Platform }
import Platform.{AggregateProgram,Settings,PlatformConfig}
// STEP 2: DEFINE AGGREGATE PROGRAM SCHEMA
class Program extends AggregateProgram with DSensing {
// Specify "distributed sensing" aggregate computation
override def main(): Any = distributedSense(size = 10.0,
metric = nbrRange,
v = sense[Double]("temperature"))
}
// STEP 3: PLATFORM SETUP
val settings = Settings()
val platform = PlatformConfig.setupPlatform(settings)
// STEP 4: NODE SETUP
val sys = platform.newAggregateApplication()
val dm = sys.newDevice(id = Utils.newId(),
program = Program,
neighbours = Utils.discoverNbrs())
val devActor = dm.actorRef // get underlying actor
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 45 / 61
58. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Outline
1 Roots: from Coordination to Self-Organisation
2 Aggregate Computing and Field calculus
3 Aggregate Programming in ScaFi
Aggregate Computing API/DSL
Aggregate Computing Platform
4 Perspectives and Roadmap
5 Conclusion
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 46 / 61
59. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Ready to jump
So...Aggregate Computing has moved
from a fragmented collection of ideas and tools
to a stable core calculus and layered framework for distributed CAS
engineering
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 47 / 61
60. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
The Aggregate Computing Stack
sensors
local functions
actuators
Application
Code
Developer
APIs
Field Calculus
Constructs
Resilient
Coordination
Operators
Device
Capabilities
functions repnbr
TGCfunctions
communication state
PerceptionPerception
summarize
average
regionMax
…
ActionAction StateState
Collective BehaviorCollective Behavior
distanceTo
broadcast
partition
…
timer
lowpass
recentTrue
…
collectivePerception
collectiveSummary
managementRegions
…
Crowd ManagementCrowd Management
dangerousDensity crowdTracking
crowdWarning safeDispersal
restriction
selfstabilisation
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 48 / 61
61. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Current Toolchain for Aggregate Computing
Field Calculus
Protelis
Xtext parsing
Static analysis Interpreter
Alchemist
Android/Rest/Cloud
Formal
foundation
Programming
Language
Language
tools
Simulation
Execution
Properties
Scala
simGUI
Scafi Core
ScafiAkka
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 49 / 61
62. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
State-of-the-art
Main results
Higher-Order Field Calculus & code mobility [Dam+15]
Self-stabilisation and substitution semantics [Vir+18]
Universality [Aud+18]
Eventual consistency [Bea+17]
Variants of key algorithms
Gradients: Flex, CRF, BIS, SVD, ULT [Aud+17]
Collection: single-path, (weighted) multi-path, via time-replicated
gossip
Toolchain
DSLs: Protelis [PBV15], ScaFi [CV16b]
Platform [VCP16]
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 50 / 61
63. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Future work
Research directions
Elaboration of libraries – modular; type safe; usable; ..
Techniques to control dynamics – feedback; stability; convergence; ..
Security — code mobility, trust fields [CAV18]
Applications
IoT scenarios in dense urban environments (cf., Smart City)
Drones, unmanned vehicles
Coordination service for networked systems
Development of software platforms/middlewares
Management of mobile devices and processes
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 51 / 61
64. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Aggregate Computing Platforms: Roadmap
Solid support for distributed Aggregate Systems
⇓
Flexible deployment and execution
Design-time, manual
⇓
Dynamic deployment and execution
Run-time, manually triggered
⇓
Self-adaptive deployment and execution
Run-time, automatic (reacting to available infrastructure)
⇓
Opportunistic deployment and execution
Run-time, automatic (optimising)
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 52 / 61
65. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Execution strategies and architectures [VCP16]
Adaptivity requirements demand for fluid responsibilities
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 53 / 61
66. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Aggregate Processes I
Motivation
Field calculus is space-time universal, but expressing
a dynamic number of
space-time dynamic computations
is not currently as straightforward as it should be.
The Aggregate Process notion
From a single computation for the entire system
To a set of parallel computations (processes)
- with ad-hoc spatial and temporal extension
- i.e., opportunistically executed by a dynamic coalition of device
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 54 / 61
67. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Aggregate Processes II
New construct: spawn
Generation: activation of new process instances
Evolution: propagation and retraction of processes
Fusion: merging of “identical” process instances
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 55 / 61
68. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Aggregate Processes: time-replicated gossip [PBV16]
Gossip #1Gossip #1
Gossip #3Gossip #3
Gossip #2Gossip #2
…
Time
p
k=3
Gossip #4Gossip #4
def replicated[T,R](proc: T => R)(arg: T, p: Double, k: Int): Map[Long,R] = {
val pid = sharedClock(p, dt())
val newProcs = if(changed(pid)) Set(pid) else Set[Long]()
spawn[Long,T,R]((thisPid) => (arg) => {
(proc(arg), if(thisPid-pid < k){Output} else {External})
}, newProcs, arg)
}
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 56 / 61
69. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Outline
1 Roots: from Coordination to Self-Organisation
2 Aggregate Computing and Field calculus
3 Aggregate Programming in ScaFi
Aggregate Computing API/DSL
Aggregate Computing Platform
4 Perspectives and Roadmap
5 Conclusion
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 57 / 61
70. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
Wrap-up
Key takeaways
Aggregate computing:
1. has a functional compositional model for distributed algorithms
2. has a practical toolchain (ScaFi): experiment with it!
3. targets abstractions and platforms for future IoT
Conclusion
A bunch of results and tools emerged, many to come
We’re always eager to find new collaborations!
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 58 / 61
71. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
References I
[Aud+17] Giorgio Audrito et al. “Compositional Blocks for Optimal Self-Healing
Gradients”. In: Self-Adaptive and Self-Organising Systems (SASO), IEEE
International Conference on. Accepted for publication. IEEE. 2017.
[Aud+18] Giorgio Audrito et al. “Space-Time Universality of Field Calculus”. In:
International Conference on Coordination Languages and Models. Springer.
2018, pp. 1–20.
[Bea+13] Jacob Beal et al. “Organizing the Aggregate: Languages for Spatial
Computing”. In: Formal and Practical Aspects of DSLs: Recent Developments.
A longer version at: http://arxiv.org/abs/1202.5509. IGI Global, 2013.
Chap. 16, pp. 436–501. isbn: 978-1-4666-2092-6.
[Bea+17] Jacob Beal et al. “Self-adaptation to Device Distribution in the Internet of
Things”. In: ACM Transactions on Autonomous and Adaptive Systems (TAAS)
12.3 (2017), p. 12.
[BPV15] Jacob Beal, Danilo Pianini, and Mirko Viroli. “Aggregate Programming for the
Internet of Things”. In: IEEE Computer (2015). issn: 1364-503X.
[CAV18] Roberto Casadei, Alessandro Aldini, and Mirko Viroli. “Combining Trust and
Aggregate Computing”. In: Software Engineering and Formal Methods. Ed. by
Antonio Cerone and Marco Roveri. Cham: Springer International Publishing,
2018, pp. 507–522. isbn: 978-3-319-74781-1.
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 59 / 61
72. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
References II
[CV16a] Roberto Casadei and Mirko Viroli. “Programming Actor-based Collective
Adaptive Systems”. In: AGERE16 workshop. To appear. ACM SIGPLAN. 2016.
[CV16b] Roberto Casadei and Mirko Viroli. “Towards Aggregate Programming in Scala”.
In: First Workshop on Programming Models and Languages for Distributed
Computing. PMLDC ’16. Rome, Italy: ACM, 2016, 5:1–5:7. isbn:
978-1-4503-4775-4. doi: 10.1145/2957319.2957372. url:
http://doi.acm.org/10.1145/2957319.2957372.
[Dam+15] Ferruccio Damiani et al. “Code Mobility Meets Self-organisation: A
Higher-Order Calculus of Computational Fields”. English. In: Formal
Techniques for Distributed Objects, Components, and Systems. Vol. 9039.
Lecture Notes in Computer Science. Extended version submitted to ACM TOCL
journal. Springer International Publishing, 2015, pp. 113–128. isbn:
978-3-319-19194-2. doi: 10.1007/978-3-319-19195-9textunderscore8.
[DVB16] Ferruccio Damiani, Mirko Viroli, and Jacob Beal. “A type-sound calculus of
computational fields”. In: Science of Computer Programming 117 (2016),
pp. 17 –44. issn: 0167-6423. doi:
http://dx.doi.org/10.1016/j.scico.2015.11.005. url:
http://www.sciencedirect.com/science/article/pii/S0167642315003573.
[MZ09] Marco Mamei and Franco Zambonelli. “Programming pervasive and mobile
computing applications: The TOTA approach”. In: ACM Transactions on
Software Engineering and Methodologies 18.4 (2009).
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 60 / 61
73. 1. Roots 2. Aggregate Computing 3. Toolchain 4. Roadmap Conclusion Referen
References III
[PBV15] Danilo Pianini, Jacob Beal, and Mirko Viroli. “Practical Aggregate
Programming with Protelis”. In: ACM Symposium on Applied Computing
(SAC 2015). To appear. 2015.
[PBV16] Danilo Pianini, Jacob Beal, and Mirko Viroli. “Improving gossip dynamics
through overlapping replicates”. In: Lecture Notes in Computer Science
(including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in
Bioinformatics) 9686 (2016), pp. 192–207. issn: 16113349. doi:
10.1007/978-3-319-39519-7_12.
[VCP16] Mirko Viroli, Roberto Casadei, and Danilo Pianini. “On Execution Platforms for
Large-Scale Aggregate Computing”. In: 3rd International Workshop on
Collective Adaptation in Very Large Scale Ubicomp (VLSU): Towards a
Superorganism of Wearables - UbiComp. Heidelberg, Germany, 2016.
[Vir+15] Mirko Viroli et al. “Multi-agent Systems Meet Aggregate Programming:
Towards a Notion of Aggregate Plan”. In: PRIMA 2015: Principles and Practice
of Multi-Agent Systems - 18th International Conference, Bertinoro, Italy,
October 26-30, 2015, Proceedings. 2015, pp. 49–64. doi:
url{10.1007/978-3-319-25524-8_4}. url:
url{http://dx.doi.org/10.1007/978-3-319-25524-8_4}.
[Vir+18] Mirko Viroli et al. “Engineering resilient collective adaptive systems by
self-stabilisation”. In: ACM Transactions on Modeling and Computer Simulation
(TOMACS) 28.2 (2018), p. 16.
M. Viroli, R. Casadei et al (UNIBO) Aggregate Computing 2018/06/19 61 / 61