This presentation, held in the context of the CS & Eng M.D. course "Pervasive Computing" (Unibo, Cesena), drafts some analysis for an Aggregate Computing platform and suggests areas of investigation.

1. Aggregate Computing: Bridging the Gaps
Roberto Casadei
PhD Student in CS&Eng
roby.casadei@unibo.it
Department of Computer Science and Engineering (DISI)
Alma Mater Studiorum – Università of Bologna
Pervasive Computing course
R. Casadei Intro ACP Analysis Technology Wrap-up 1/49

2. Engineering apps AC: recap
Outline
1 Introduction
Engineering pervasive computing applications
Aggregate Computing: recap
2 Analysis for an Aggregate Computing Platform (ACP)
3 Technology: quick survey
4 Wrap-up
R. Casadei Intro ACP Analysis Technology Wrap-up 2/49

3. Engineering apps AC: recap
Outline
1 Introduction
Engineering pervasive computing applications
Aggregate Computing: recap
2 Analysis for an Aggregate Computing Platform (ACP)
Requirements
Analysis
Current framework
3 Technology: quick survey
4 Wrap-up
R. Casadei Intro ACP Analysis Technology Wrap-up 3/49

4. Engineering apps AC: recap
The starting point of any engineering process
A vision that must be made concrete / top-down or bottom-up
R. Casadei Intro ACP Analysis Technology Wrap-up 4/49

5. Engineering apps AC: recap
Applications as the interface to the needs
Abstractly defined until some concrete underlying technology is identified
R. Casadei Intro ACP Analysis Technology Wrap-up 5/49

6. Engineering apps AC: recap
Aggregate Computing: bridging the gap to SASO systems
Supports the development and execution of Aggregate Computing applications
R. Casadei Intro ACP Analysis Technology Wrap-up 6/49

7. Engineering apps AC: recap
Zooming the AC layer
R. Casadei Intro ACP Analysis Technology Wrap-up 7/49

8. Engineering apps AC: recap
Outline
1 Introduction
Engineering pervasive computing applications
Aggregate Computing: recap
2 Analysis for an Aggregate Computing Platform (ACP)
Requirements
Analysis
Current framework
3 Technology: quick survey
4 Wrap-up
R. Casadei Intro ACP Analysis Technology Wrap-up 8/49

9. Engineering apps AC: recap
Aggregate Computing (AC) » what
Goal: programming the collective behaviour of aggregates of devices
Main ideas
Macro-level programming with automatic macro-micro bridging
Declarativity
Support for “engineered emergence”
Composition of aggregate behaviour with predictable dynamics
Layers of reusable abstractions
Abstraction from interaction-level and platform-level issues
Inherent adaptivity
Inherent resiliency
Operational flexibility
R. Casadei Intro ACP Analysis Technology Wrap-up 9/49

10. Engineering apps AC: recap
Compositionality in AC: example
trait DistributedSum extends AggregateProgram {
def distribSum(size:Double, metric: =>Double, v:Double): Double = {
???
}
}
trait Broadcast extends BlockG with ... { s: AggregateProgram =>
def broadcast[V: Bounded](src: Boolean, value: V): V =
G(src, value)(x => x, nbrRange)
}
trait SumCollect extends BlockC with ... { s: AggregateProgram =>
def sumCollect(target: Boolean, value: Double): Double =
C(value)(distanceTo(target,nbrRange))(_+_,0)
}
:
Distributed sum
Split the space into regions with grain size
Calculate the mean of field v in each region
Collect the values and the number of nodes in the region
Compute the average
Propagate the computed value
R. Casadei Intro ACP Analysis Technology Wrap-up 10/49

11. Engineering apps AC: recap
Compositionality in AC: example
trait DistributedSum extends AggregateProgram
with BlockG with BlockC with BlockS
with Broadcast with SumCollect with Utilities
{
def distribSum(size:Double, metric: =>Double, v:Double): Double = {
val leaders = S(size, metric)
???
}
}
:
R. Casadei Intro ACP Analysis Technology Wrap-up 10/49

12. Engineering apps AC: recap
Compositionality in AC: example
trait DistributedSum extends AggregateProgram
with BlockG with BlockC with BlockS
with Broadcast with SumCollect with Utilities
{
def distribSum(size:Double, metric: =>Double, v:Double): Double = {
val leaders = S(size, metric)
val potential = distanceTo(leaders, metric)
???
}
}
:
R. Casadei Intro ACP Analysis Technology Wrap-up 10/49

13. Engineering apps AC: recap
Compositionality in AC: example
trait DistributedSum extends AggregateProgram ... with SumCollect {
def distribSum(size:Double, metric: =>Double, v:Double): Double = {
val leaders = S(size, metric)
val potential = distanceTo(leaders, metric)
val collection = sumCollect(leaders, v)
val count = sumCollect(leaders, 1.0)
val avg = collection / count
}
}
:
R. Casadei Intro ACP Analysis Technology Wrap-up 10/49

14. Engineering apps AC: recap
Compositionality in AC: example
trait DistributedSum extends AggregateProgram ... with Broadcast {
def distribSum(size:Double, metric: =>Double, v:Double): Double = {
val leaders = S(size, metric)
val potential = distanceTo(leaders, metric)
val collection = sumCollect(leaders, v)
val count = sumCollect(leaders, 1.0)
val avg = collection / count
broadcast(leaders, avg)
}
}
:
R. Casadei Intro ACP Analysis Technology Wrap-up 10/49

15. Engineering apps AC: recap
Aggregate Computing (AC) » how
Repeated execution
Provides reactivity and incremental adaptivity
Broadcast + aligned interaction
Allows the construction of a “local” context for computation
R. Casadei Intro ACP Analysis Technology Wrap-up 11/49

16. Requirements Analysis Current framework
Outline
1 Introduction
2 Analysis for an Aggregate Computing Platform (ACP)
Requirements
Analysis
Current framework
3 Technology: quick survey
4 Wrap-up
R. Casadei Intro ACP Analysis Technology Wrap-up 12/49

17. Requirements Analysis Current framework
Outline
1 Introduction
Engineering pervasive computing applications
Aggregate Computing: recap
2 Analysis for an Aggregate Computing Platform (ACP)
Requirements
Analysis
Current framework
3 Technology: quick survey
4 Wrap-up
R. Casadei Intro ACP Analysis Technology Wrap-up 13/49

18. Requirements Analysis Current framework
Requirements for an Aggregate Computing platform
High-level, Functional
FR1 Support the specification and configuration of AGGREGATE-SYSTEMs
FR2 Support the runtime execution of AGGREGATE-APPLICATIONs
High-level, Non-functional
NR1 Scalability to a large number of NODEs
NR2 Reliability
An AGGREGATE-SYSTEM made of unreliable INFRASTRUCTURAL-COMPONENTs and
unreliable NODEs should be able to continue operation even in face of faults.
R. Casadei Intro ACP Analysis Technology Wrap-up 14/49

19. Requirements Analysis Current framework
Outline
1 Introduction
Engineering pervasive computing applications
Aggregate Computing: recap
2 Analysis for an Aggregate Computing Platform (ACP)
Requirements
Analysis
Current framework
3 Technology: quick survey
4 Wrap-up
R. Casadei Intro ACP Analysis Technology Wrap-up 15/49

20. Requirements Analysis Current framework
Requirements analysis
Approach: informal but systematic (cartesian)
Give names to things
Refine meaning and relationships
Explore in (mentally simulated) use, adopt perspectives and visualize
(Then, formalize early, when enough committed)
AGGREGATE-LOGIC: the representation-independent view of some
aggregate-level behaviour in space/time
AGGREGATE-PROGRAM: an AGGREGATE-LOGIC expressed in some executable
representation
AGGREGATE-SYSTEM: a set of networked NODEs supporting the collective
execution of AGGREGATE-PROGRAMs
AGGREGATE-APPLICATION: a particular AGGREGATE-LOGIC running on a certain
AGGREGATE-SYSTEM, aimed at solving some problem in some context
R. Casadei Intro ACP Analysis Technology Wrap-up 16/49

21. Requirements Analysis Current framework
High-level, logical, informal model
Starting point for a systematic characterisation of our (fuzzy) class of systems
We think of no specific application/system, but a range of applications/systems
to support/drive analysis and be concrete
R. Casadei Intro ACP Analysis Technology Wrap-up 17/49

22. Requirements Analysis Current framework
Structure / Interaction / Behaviour decomposition
(Logical) Structure
DEVICEs (aka NODEs, AGENTs)
Have an identity (UID)
Have SENSORs and ACTUATORs
(Logical) Behaviour/Interaction
A DEVICE can only directly communicate with the DEVICEs of its
NEIGHBOURHOOD
A DEVICE computes its part of AGGREGATE-LOGIC on a round-by-round basis
Steps of a round of execution
1) Retrieve the device’s CONTEXT (SENSOR-DATA, NBR-DATA)
2) Execute LOCAL-AGGREGATE-LOGIC to produce OUTPUT + EXPORT
3) Broadcast EXPORT to NEIGHBOURHOOD
4) Feed ACTUATORS with OUTPUT
This characterization is mainly logical and “standard”
Many questions arise: who? what? where? when? how?
R. Casadei Intro ACP Analysis Technology Wrap-up 18/49

23. Requirements Analysis Current framework
Analysis » execution rounds I
SCHEDULING-POLICY: who decides when a round has to be executed?
A SCHEDULER (owned by a device or external)
Construction of the CONTEXT using up-to-date SENSOR-DATA and NBR-DATA
Ideally, we’d like to separate the time/frequency in which these information are
collected from the time in which they are received
Retrieval of SENSOR-DATA: request/response to device’s SENSOR or sampling of
an incoming SENSOR-DATA-STREAM
Note: location data (e.g., GPS coords) may need to be sent to the
TOPOLOGY-MANAGER
Retrieval of NBR-DATA: request/response to a FIELD-MANAGER or sampling of an
incoming NBR-DATA-STREAM
Who actually executes the AGGREGATE-LOGIC for a given DEVICE?
An AGGREGATE-LOGIC-EXECUTOR (owned by a device or external)
R. Casadei Intro ACP Analysis Technology Wrap-up 19/49

24. Requirements Analysis Current framework
Analysis » execution rounds II
Broadcast of EXPORT to NEIGHBOURHOOD
By the DEVICE itself (if it knows how to reach its NEIGHBOURHOOD)
But the DEVICE’s awareness of communication is questionable
“No matter who you are! I know where you are” [ZM04]
The DEVICE may simply send its computation descriptors (EXPORTs) to the
FIELD-MANAGER
The FIELD-MANAGER in turn may delegate the task to an EXPORT-PROPAGATOR
which must then know the NEIGHBOURHOODs and how to reach them
Invocation of ACTUATORs
In general, we may have ACTUATION-DATA-STREAMs directed to ACTUATORs
which may be controlled by (but still decoupled from) rounds.
R. Casadei Intro ACP Analysis Technology Wrap-up 20/49

25. Requirements Analysis Current framework
Analysis » neighbourhood
NBR-POLICY: how is NEIGHBOURHOOD determined?
May be communication-determined (i.e., my neighbours are those I can reach)
May be spatially-determined (i.e., derived from the devices’ position in space)
May be ad-hoc (i.e., application-specific)
Who owns the knowledge of a device’s NEIGHBOURHOOD?
A TOPOLOGY-MANAGER (owned by a device or external)
Who uses the knowledge of a device’s NEIGHBOURHOOD?
The EXPORT-PROPAGATOR
Properties of NEIGHBOURHOOD
It is usually dynamic
Can be asymmetric
R. Casadei Intro ACP Analysis Technology Wrap-up 21/49

26. Requirements Analysis Current framework
Analysis » configuration
How do components locate (the address of) other components?
DEVICEs should be addressable to support e.g. peer-to-peer interaction or the
the delivery of ACTUATION-DATA
The mappings between DEVICEs’ UIDs and their ADDRESS may be kept in a
REGISTRY
Devices should also be given some CONFIGURATION data – so that they know the
ADDRESS of required services
How do AGGREGATE-LOGIC-EXECUTOR obtain the AGGREGATE-LOGIC to execute?
Can be provided at deployment or configuration time
R. Casadei Intro ACP Analysis Technology Wrap-up 22/49

27. Requirements Analysis Current framework
Logical components emerging from analysis (dev-centric)
It comes natural to assign many responsibilities to the DEVICEs (cf., p2p setup)..
R. Casadei Intro ACP Analysis Technology Wrap-up 23/49

28. Requirements Analysis Current framework
Logical components emerging from analysis (sys-centric)
It comes natural to consider DEVICEs only as situated contexts..
R. Casadei Intro ACP Analysis Technology Wrap-up 24/49

29. Requirements Analysis Current framework
Many kinds of deployments
Such flexibility demands for fluid responsibilities
R. Casadei Intro ACP Analysis Technology Wrap-up 25/49

30. Requirements Analysis Current framework
The data perspective
Data payloads
SENSOR-DATA: map from sensor UIDs to sensor values
ACTUATION-DATA: map from actuator UIDs to actuation values
NBR-DATA (i.e., EXPORTS): map from neighbour UIDs to exports
CONTEXT-DATA = SENSOR-DATA + NBR-DATA
Control data
SCHEDULE-DATA (i.e., TICKs)
...
Configuration data
Deployment data, addresses, ...
...
Analysis from a data (flow) perspective
» On an app-specific basis, analysis of Volume-Velocity-Variety-Veracity
» Further analysis: persistent vs. transient data
» Further analysis: replication, partitioning, ...
» Analysis of data flow can help architectural design (cf., reactive architectures)
R. Casadei Intro ACP Analysis Technology Wrap-up 26/49

31. Requirements Analysis Current framework
Outline
1 Introduction
Engineering pervasive computing applications
Aggregate Computing: recap
2 Analysis for an Aggregate Computing Platform (ACP)
Requirements
Analysis
Current framework
3 Technology: quick survey
4 Wrap-up
R. Casadei Intro ACP Analysis Technology Wrap-up 27/49

32. Requirements Analysis Current framework
SCAFI: state-of-art
R. Casadei Intro ACP Analysis Technology Wrap-up 28/49

33. Requirements Analysis Current framework
AC Platform: state-of-art
Key features
Actor-based (Akka)
Two styles
a) fully decentralised (P2P)
b) server-based (mediating interactions)
Simple object-oriented API façade
Easy configuration (file, cmd-line, programmatically)
Limitations (work-in-progress)
Not ready for use across the Internet
Serialization issues and constrained architecture due to nested design
Tightly coupled with SCAFI-core
R. Casadei Intro ACP Analysis Technology Wrap-up 29/49

34. Requirements Analysis Current framework
AC actor platform: overview
R. Casadei Intro ACP Analysis Technology Wrap-up 30/49

35. Requirements Analysis Current framework
AC actor platform: building
R. Casadei Intro ACP Analysis Technology Wrap-up 31/49

36. Requirements Analysis Current framework
AC actor platform: modular actor design
R. Casadei Intro ACP Analysis Technology Wrap-up 32/49

37. Requirements Analysis Current framework
AC actor platform: styles
R. Casadei Intro ACP Analysis Technology Wrap-up 33/49

38. Outline
1 Introduction
2 Analysis for an Aggregate Computing Platform (ACP)
3 Technology: quick survey
4 Wrap-up
R. Casadei Intro ACP Analysis Technology Wrap-up 34/49

39. Message-oriented middlewares
Why: decoupling consumers from producers (in both space and time)
RabbitMQ
General purpose message broker
Supports protocols such as AMQP, MQTT, STOMP etc.
Flexible routing
Concepts: publishers, brokers, consumers, exchanges, bindings, queues, ...
Apache Kafka
Distributed, partitioned, replicated commit-log-based publish-subscribe
messaging system
Persistent messaging
Concepts: producers, consumer instances/groups, topics, cluster of brokers,
per-topic partitioned log, log retention, ...
R. Casadei Intro ACP Analysis Technology Wrap-up 35/49

40. Streaming
Dimensions
Types of datasets: bounded vs. unbounded
Types of processing: batch vs. streaming
Why: need for processing that execs continuously as long as data is being produced
Apache Flink
Stream processing framework for distributed, high-performing, always-available,
and accurate data streaming applications
Concepts: data sources, transformations, data sinks, dataflow, operator
subtasks, stream partitions, flexible windows for stream aggregation...
Deployment modes: cloud / on premise / cluster (managed by YARN or Mesos)
Guarantees exactly-once semantics for stateful computations (checkpointing)
Apache Storm
Concepts: topology as graph of spouts (sources) and bolts; streams as
unbounded sequences of tuples; groupings defining how streams are partitioned
among the bolt’s tasks
Trident: exactly-once, high-level API, enabling micro-batching within a stream
R. Casadei Intro ACP Analysis Technology Wrap-up 36/49

41. Lightbend Fast Data Platform
R. Casadei Intro ACP Analysis Technology Wrap-up 37/49

42. NoSQL data stores and spatial indexes
Why: in Aggregate Computing, data has peculiar characteristics and access patterns
Nodes are often spatially situated and mobile
Locality principle (cf., neighbourhood)
Typical query: “get most recent exports of my neighbours”
Spatial DBs and spatial indexes
Redis supports geospatial indexes
Commands: GEOPOS, GEOADD, ...
Neo4J (graph DB) and Neo4j Spatial plug-in
PostGIS: spatial and geographic objects for PostgreSQL
R. Casadei Intro ACP Analysis Technology Wrap-up 38/49

43. Automated cluster management
Why: abstract over resources and support distributed, fault-tolerant, elastic
deployments
Apache Mesos: a distributed systems kernel for managing clusters
Provides APIs for resource management and scheduling across datacenter and
cloud environments
DC/OS: the datacenter operating system, based on Mesos
Manages multiple machines in the cloud or on-premises from a single interface
Deploys containers, distribute services and legacy apps into machines
Provides networking, service discovery and resource management to keep the
services running and communicating with each other
Marathon: a container orchestration platform for Mesos and DC/OS
Docker: SW containerization platform
Docker Swarm for managing clusters of Docker engines
Kubernetes: platform for automating container operations
Supports automating deployment, scaling, & management of containerized apps
R. Casadei Intro ACP Analysis Technology Wrap-up 39/49

44. Outline
1 Introduction
2 Analysis for an Aggregate Computing Platform (ACP)
3 Technology: quick survey
4 Wrap-up
R. Casadei Intro ACP Analysis Technology Wrap-up 40/49

45. Summary: key ideas
Aggregate programming
• A macro-programming approach to CASs, formally grounded in Field Calculus
• Allows composition of (self-stabilizing) SASO patterns.
The abstract model is decoupled from the execution strategy
• Aggregate computing is suitable for exec across several platforms/infrastructures
• From ad-hoc networks to cloud/fog computing
• Can be extended to dynamically exploit available/heterogeneous infrastructure
SCAFI: a Scala framework for Aggregate Programming
• Provides an internal DSL for field-based computations
• Provides simulation facilities
• Provides an actor-based platform for building aggregate systems
R. Casadei Intro ACP Analysis Technology Wrap-up 41/49

46. Perspectives, projects, future works I
Algorithms and SCAFI library
Flocking and mobility (De Castri/Foschi/Paolucci)
Anticipative Gradient (Berlini)
SW engineering (AC as a case study)
AC Platform analysis/design and impl/refactoring (Scala)
Actor-based platform bootstrap (Casadei [Cas])
GUI interface to simulator: bootstrap (Varini) & refactoring (Aguzzi)
Testing/debugging of AC apps
Load/performance testing of AC apps
Tools and automation support in SCAFI
Continuous Integration (Fantini)
R. Casadei Intro ACP Analysis Technology Wrap-up 42/49

47. Perspectives, projects, future works II
Integration and applications
SCAFI on Android (Radaelli, Collini)
SCAFI-Alchemist integration proof-of-concept (Costanzi [Cos])
Web/RESTful interfaces to AC systems, Web of Things (WoT)
AC and Service/Cluster/Cloud Computing
AC and IoT platforms (Farneti [Far])
Development of a custom PaaS for AC
Prototyping a scalable AC cluster with open-source solutions (Paolucci)
Reactive AC architecture
Fields like Big Data and NoSQL solutions (Riciputi)
R. Casadei Intro ACP Analysis Technology Wrap-up 43/49

48. Perspectives, projects, future works III
Innovative themes and experiments
AC and computer security (Mantani [Man])
Aggregate processes (Foschi)
AC and complex networks
AC and topological methods (math-oriented)
Miscellaneous
Comparing AC to BSP-based graph processing systems (Pregel/Giraph)
R. Casadei Intro ACP Analysis Technology Wrap-up 44/49

49. Application scenarios
Crowd engineering
Distributed sensing
Contextual services
Smart city
Smart home
Industry 4.0
WSNs
Swarm intelligence (robots, drones)
Rescue scenarios
R. Casadei Intro ACP Analysis Technology Wrap-up 45/49

50. Project example #1: exploration
Scenario: exploration of unknown environments (Mars?)
Mobile robots
Must move and stay at the right distance, communicating frequently
May need to come back or send information (e.g., hop-by-hop) to a fixed base of
operations
Assumption: no infrastructure, peer-to-peer communication (ad-hoc network)
Progression: analysis - design - simulation - evaluation
Extension: rescue
R. Casadei Intro ACP Analysis Technology Wrap-up 46/49

51. Project example #2: dynamic platform reconfiguration
Goal: draft a simple solution design for dynamic reconfiguration of system nodes
Cf., MAPE design
Monitoring: monitor local status of infrastructure
Analyze: determine the best option for communication and interaction (wrt to
QoS)
Plan: plan a gentle reconfiguration
Execute: distribute the reconfiguration data and commands
Application example on top: crowd density monitoring and warning services
Progression: analysis, design, prototype
R. Casadei Intro ACP Analysis Technology Wrap-up 47/49

52. Project example #3: AC and WoT
Goal: provide a web interface to system nodes
E.g., for de/activation of sensors
E.g., to command actuators (e.g., for mobility)
E.g., to query device state
Not every node may be able to connect to Internet and support this
Internet-enabled neighbour nodes may still be able to provide such info
In practice: design a set of ReSTful services and a web interface
Application example on top: distributed sensing
R. Casadei Intro ACP Analysis Technology Wrap-up 48/49

53. References
References I
[Cas] Roberto Casadei.
Aggregate programming in scala: a core library and actor-based platform for
distributed computational fields.
Master’s thesis.
[Cos] Simone Costanzi.
Integrazione di piattaforme d’esecuzione e simulazione in una toolchain scala
per aggregate programming.
Master’s thesis.
[Far] Thomas Farneti.
Design and deployment of an execution platform based on microservices for
aggregate computing in the cloud.
Master’s thesis.
[Man] Giacomo Mantani.
Towards security-aware aggregate computing.
Master’s thesis.
R. Casadei Appendix 49/49

54. References
References II
[ZM04] Franco Zambonelli and Marco Mamei.
Spatial computing: An emerging paradigm for autonomic computing and
communication.
In Workshop on Autonomic Communication, pages 44–57. Springer, 2004.
R. Casadei Appendix 50/49