The document discusses engineering pervasive computing applications and aggregate computing. It provides an overview of relevant technologies for pervasive computing like message brokers, streaming platforms, databases and cluster managers. It then discusses aggregate computing as a way to program collective behavior of device aggregates. Key aspects of aggregate computing include compositional programming with reusable abstractions, adaptivity, resilience and flexibility. The document analyzes requirements for an aggregate computing platform to support application specification, execution at scale and reliability even with unreliable devices and infrastructure.

1. Bridging the Pervasive Computing Gap
An Aggregate Perspective
Roberto Casadei
PhD Student in CS&Eng
roby.casadei@unibo.it
Department of Computer Science and Engineering (DISI)
Alma Mater Studiorum – Università of Bologna
Pervasive Computing 2018/19 MEng Course
R. Casadei Intro Technology: quick survey AC Wrap-up 1/52

2. Engineering apps
Outline
1 Intro
Engineering pervasive computing applications
2 Technology: quick survey
3 Aggregate Computing
4 Wrap-up
R. Casadei Intro Technology: quick survey AC Wrap-up 2/52

3. Engineering apps
Outline
1 Intro
Engineering pervasive computing applications
2 Technology: quick survey
3 Aggregate Computing
AC: recap
Analysis for an Aggregate Computing Platform (ACP)
4 Wrap-up
R. Casadei Intro Technology: quick survey AC Wrap-up 3/52

4. Engineering apps
The starting point of any engineering process
A vision that must be made concrete / top-down or bottom-up
R. Casadei Intro Technology: quick survey AC Wrap-up 4/52

5. Engineering apps
Applications as the interface to the needs
Abstractly defined until some concrete underlying technology is identified
R. Casadei Intro Technology: quick survey AC Wrap-up 5/52

6. Engineering apps
End-to-end development of IoT Systems
Three key issues:
1) Applications—i.e., the business logic
2) Middleware—i.e., reducing the abstraction gap
3) Operations—i.e., configuration and deployment
R. Casadei Intro Technology: quick survey AC Wrap-up 6/52

7. Engineering apps
PC Middlewares [RCKZ13] I
Pervasive Computing (PvC) aims to create a smart environment with embedded
and networked computing devices providing users with seamless service access
Key point: seamless service access
Key point: human-centricity
Middleware: SW and tools that can help hide the complexity and heterogeneity
of the underlying HW and network platforms, can ease the management of
system resources, and can increase the predictability of application executions.
Goal of PvC middleware: match application-needs with available resources,
while ensuring quality and efficiency
R. Casadei Intro Technology: quick survey AC Wrap-up 7/52

8. Engineering apps
PC Middlewares [RCKZ13] II
R. Casadei Intro Technology: quick survey AC Wrap-up 8/52

9. Engineering apps
IoT Patterns for Edge [QPR+
16] » Edge Provisioning
Problem: how to ensure all edge devices are started with a reliable baseline
environment, as needed?
R. Casadei Intro Technology: quick survey AC Wrap-up 9/52

10. Engineering apps
IoT Patterns for Edge [QPR+
16] » Edge Code Deployment
Problem: how to deploy code to many IoT devices automatically, quickly, safely, and
configure them without a long process of build/deployment/test/release?
R. Casadei Intro Technology: quick survey AC Wrap-up 10/52

11. Engineering apps
IoT Patterns for Edge [QPR+
16] » Edge Orchestration
Problem: How can we orchestrate IoT devices in accordance with their tightly
scripted configurations as node of a cluster remotely?
R. Casadei Intro Technology: quick survey AC Wrap-up 11/52

12. Outline
1 Intro
2 Technology: quick survey
3 Aggregate Computing
4 Wrap-up
R. Casadei Intro Technology: quick survey AC Wrap-up 12/52

13. Technologies
Message-Oriented Middlewares
Why: decoupling consumers from producers (in both space and time)
Examples: RabbitMQ, Kafka
Streaming
Why: need for continuous processing, as long as data is being produced
Types of datasets: un/bounded — processing: batch vs. streaming
Examples: Kafka, Apache Flink, Apache Storm (+ Trident)
NoSQL and spatial data stores
Why: dealing with situated mobile objects, neighbourhoods, locality..
Examples: Redis (geospatial indexes), Neo4J (graph DB + Spatial plugin),
PostGIS (spatial/geographical objects for PostgreSQL)
Automated cluster management
Why: abstract resources & support distributed/fault-tolerant/elastic deployments
Examples: Apache Mesos (+ Marathon), Docker (+ Swarm), Kubernetes
R. Casadei Intro Technology: quick survey AC Wrap-up 13/52

14. RabbitMQ
https://www.cloudamqp.com/blog/2015-05-18-part1-rabbitmq-for-beginners-what-is-rabbitmq.html
R. Casadei Intro Technology: quick survey AC Wrap-up 14/52

15. Kafka
R. Casadei Intro Technology: quick survey AC Wrap-up 15/52

16. Apache Flink I
R. Casadei Intro Technology: quick survey AC Wrap-up 16/52

17. Apache Flink II
R. Casadei Intro Technology: quick survey AC Wrap-up 17/52

18. Kubernetes
R. Casadei Intro Technology: quick survey AC Wrap-up 18/52

19. Mesos
Mesos architecture
Example: Spark on Mesos & process
R. Casadei Intro Technology: quick survey AC Wrap-up 19/52

20. Lightbend Fast Data Platform
R. Casadei Intro Technology: quick survey AC Wrap-up 20/52

21. Choosing the set of technologies
Recipe
Start from the problem
Structure–Interaction–Behaviour
Distinguish logical vs. physical
What non-functional requirements?
Identify an architecture
Identify the abstractions you need
Use imagination: “how may I express X assuming to already have an API/platform?”
Study the tools—what abstractions do they provide?
Meet in the middle from top-down and bottom-up
R. Casadei Intro Technology: quick survey AC Wrap-up 21/52

22. Recap ACP Analysis
Outline
1 Intro
2 Technology: quick survey
3 Aggregate Computing
AC: recap
Analysis for an Aggregate Computing Platform (ACP)
4 Wrap-up
R. Casadei Intro Technology: quick survey AC Wrap-up 22/52

23. Recap ACP Analysis
Aggregate Computing: bridging the gap to SASO systems
Supports the development and execution of Aggregate Computing applications
R. Casadei Intro Technology: quick survey AC Wrap-up 23/52

24. Recap ACP Analysis
Outline
1 Intro
Engineering pervasive computing applications
2 Technology: quick survey
3 Aggregate Computing
AC: recap
Analysis for an Aggregate Computing Platform (ACP)
4 Wrap-up
R. Casadei Intro Technology: quick survey AC Wrap-up 24/52

25. Recap ACP Analysis
Aggregate Computing (AC) » what
Goal: programming the collective behaviour of aggregates of devices
Main ideas
Macro-level programming with automatic macro-micro bridging
Declarativity
Support for “engineered emergence”
Composition of aggregate behaviour with predictable dynamics
Layers of reusable abstractions
Abstraction from interaction-level and platform-level issues
Inherent adaptivity
Inherent resiliency
Operational flexibility
R. Casadei Intro Technology: quick survey AC Wrap-up 25/52

26. Recap ACP Analysis
Compositionality in AC: example
trait DistributedSum extends AggregateProgram {
def distribMean(size:Double, metric: =>Double, v:Double): Double = {
???
}
}
trait Broadcast extends BlockG with ... { s: AggregateProgram =>
def broadcast[V: Bounded](src: Boolean, value: V): V =
G(src, value)(x => x, nbrRange)
}
trait SumCollect extends BlockC with ... { s: AggregateProgram =>
def sumCollect(target: Boolean, value: Double): Double =
C(value)(distanceTo(target,nbrRange))(_+_,0)
}
:
Distributed sum
Split the space into regions with grain size
Calculate the mean of field v in each region
Collect the values and the number of nodes in the region
Compute the average
Propagate the computed value
R. Casadei Intro Technology: quick survey AC Wrap-up 26/52

27. Recap ACP Analysis
Compositionality in AC: example
trait DistributedSensing extends AggregateProgram
with BlockG with BlockC with BlockS
with Broadcast with SumCollect with Utilities
{
def distribMean(size:Double, metric: =>Double, v:Double): Double = {
val leaders = S(size, metric)
???
}
}
:
R. Casadei Intro Technology: quick survey AC Wrap-up 26/52

28. Recap ACP Analysis
Compositionality in AC: example
trait DistributedSensing extends AggregateProgram
with BlockG with BlockC with BlockS
with Broadcast with SumCollect with Utilities
{
def distribMean(size:Double, metric: =>Double, v:Double): Double = {
val leaders = S(size, metric)
val potential = distanceTo(leaders, metric)
???
}
}
:
R. Casadei Intro Technology: quick survey AC Wrap-up 26/52

29. Recap ACP Analysis
Compositionality in AC: example
trait DistributedSensing extends AggregateProgram ... with SumCollect {
def distribMean(size:Double, metric: =>Double, v:Double): Double = {
val leaders = S(size, metric)
val potential = distanceTo(leaders, metric)
val collection = sumCollect(leaders, v)
val count = sumCollect(leaders, 1.0)
val avg = collection / count
}
}
:
R. Casadei Intro Technology: quick survey AC Wrap-up 26/52

30. Recap ACP Analysis
Compositionality in AC: example
trait DistributedSensing extends AggregateProgram ... with Broadcast {
def distribMean(size:Double, metric: =>Double, v:Double): Double = {
val leaders = S(size, metric)
val potential = distanceTo(leaders, metric)
val collection = sumCollect(leaders, v)
val count = sumCollect(leaders, 1.0)
val avg = collection / count
broadcast(leaders, avg)
}
}
:
R. Casadei Intro Technology: quick survey AC Wrap-up 26/52

31. Recap ACP Analysis
Aggregate Computing (AC) » how
Repeated execution
Provides reactivity and incremental adaptivity
Broadcast + aligned interaction
Allows the construction of a “local” context for computation
R. Casadei Intro Technology: quick survey AC Wrap-up 27/52

32. Recap ACP Analysis
Outline
1 Intro
Engineering pervasive computing applications
2 Technology: quick survey
3 Aggregate Computing
AC: recap
Analysis for an Aggregate Computing Platform (ACP)
4 Wrap-up
R. Casadei Intro Technology: quick survey AC Wrap-up 28/52

33. Recap ACP Analysis
Requirements for an Aggregate Computing platform
High-level, Functional
FR1 Support the specification and configuration of AGGREGATE-SYSTEMs
FR2 Support the runtime execution of AGGREGATE-APPLICATIONs
High-level, Non-functional
NR1 Scalability to a large number of NODEs
NR2 Reliability
An AGGREGATE-SYSTEM made of unreliable INFRASTRUCTURAL-COMPONENTs and
unreliable NODEs should be able to continue operation even in face of faults.
R. Casadei Intro Technology: quick survey AC Wrap-up 29/52

34. Recap ACP Analysis
Requirements analysis
Approach: informal but systematic (Cartesian)
Give names to things
Refine meaning and relationships
Explore in (mentally simulated) use, adopt perspectives and visualize
(Then, formalize early, when enough committed)
AGGREGATE-LOGIC: the representation-independent view of some
aggregate-level behaviour in space/time
AGGREGATE-PROGRAM: an AGGREGATE-LOGIC expressed in some executable
representation
AGGREGATE-APPLICATION: AGGREGATE-LOGIC + APP-CONFIGURATION
AGGREGATE-SYSTEM: a set of networked NODEs supporting the collective
execution of AGGREGATE-APPLICATIONs
R. Casadei Intro Technology: quick survey AC Wrap-up 30/52

35. Recap ACP Analysis
High-level, logical, informal model
Starting point for a systematic characterisation of our (fuzzy) class of systems
We think of no specific application/system, but a range of applications/systems
to support/drive analysis and be concrete
R. Casadei Intro Technology: quick survey AC Wrap-up 31/52

36. Recap ACP Analysis
Structure / Interaction / Behaviour decomposition
(Logical) Structure
DEVICEs (aka NODEs, AGENTs)
Have an identity (UID)
Have SENSORs and ACTUATORs
(Logical) Behaviour/Interaction
A DEVICE can only directly communicate with the DEVICEs of its
NEIGHBOURHOOD
A DEVICE computes its part of AGGREGATE-LOGIC on a round-by-round basis
Steps of a round of execution
1) Retrieve the device’s CONTEXT (SENSOR-DATA, NBR-DATA)
2) Execute LOCAL-AGGREGATE-LOGIC to produce OUTPUT + EXPORT
3) Broadcast EXPORT to NEIGHBOURHOOD
4) Feed ACTUATORS with OUTPUT
This characterization is mainly logical and “standard”
Many questions arise: who? what? where? when? how?
R. Casadei Intro Technology: quick survey AC Wrap-up 32/52

37. Recap ACP Analysis
Analysis » execution rounds I
SCHEDULING-POLICY: who decides when a round has to be executed?
A SCHEDULER (owned by a device or external)
Construction of the CONTEXT using up-to-date SENSOR-DATA and NBR-DATA
Ideally, we’d like to separate the time/frequency in which these information are
collected from the time in which they are received
Retrieval of SENSOR-DATA: request/response to device’s SENSOR or sampling of
an incoming SENSOR-DATA-STREAM
Note: location data (e.g., GPS coords) may need to be sent to the
TOPOLOGY-MANAGER
Retrieval of NBR-DATA: request/response to a FIELD-MANAGER or sampling of an
incoming NBR-DATA-STREAM
Who actually executes the AGGREGATE-LOGIC for a given DEVICE?
An AGGREGATE-LOGIC-EXECUTOR (owned by a device or external)
R. Casadei Intro Technology: quick survey AC Wrap-up 33/52

38. Recap ACP Analysis
Analysis » execution rounds II
Broadcast of EXPORT to NEIGHBOURHOOD
By the DEVICE itself (if it knows how to reach its NEIGHBOURHOOD)
But the DEVICE’s awareness of communication is questionable
“No matter who you are! I know where you are” [ZM04]
The DEVICE may simply send its computation descriptors (EXPORTs) to the
FIELD-MANAGER
The FIELD-MANAGER in turn may delegate the task to an EXPORT-PROPAGATOR
which must then know the NEIGHBOURHOODs and how to reach them
Invocation of ACTUATORs
In general, we may have ACTUATION-DATA-STREAMs directed to ACTUATORs
which may be controlled by (but still decoupled from) rounds.
R. Casadei Intro Technology: quick survey AC Wrap-up 34/52

39. Recap ACP Analysis
Analysis » neighbourhood
NBR-POLICY: how is NEIGHBOURHOOD determined?
May be communication-determined (i.e., my neighbours are those I can reach)
May be spatially-determined (i.e., derived from the devices’ position in space)
May be ad-hoc (i.e., application-specific)
Who owns the knowledge of a device’s NEIGHBOURHOOD?
A TOPOLOGY-MANAGER (owned by a device or external)
Who uses the knowledge of a device’s NEIGHBOURHOOD?
The EXPORT-PROPAGATOR
Properties of NEIGHBOURHOOD
It is usually dynamic
Can be asymmetric
R. Casadei Intro Technology: quick survey AC Wrap-up 35/52

40. Recap ACP Analysis
Analysis » configuration
How do components locate (the address of) other components?
DEVICEs should be addressable to support e.g. peer-to-peer interaction or the
the delivery of ACTUATION-DATA
The mappings between DEVICEs’ UIDs and their ADDRESS may be kept in a
REGISTRY
Devices should also be given some CONFIGURATION data – so that they know the
ADDRESS of required services
How do AGGREGATE-LOGIC-EXECUTOR obtain the AGGREGATE-LOGIC to execute?
Can be provided at deployment or configuration time
R. Casadei Intro Technology: quick survey AC Wrap-up 36/52

41. Recap ACP Analysis
Logical components emerging from analysis (dev-centric)
It comes natural to assign many responsibilities to the DEVICEs (cf., p2p setup)..
R. Casadei Intro Technology: quick survey AC Wrap-up 37/52

42. Recap ACP Analysis
Logical components emerging from analysis (sys-centric)
It comes natural to consider DEVICEs only as situated contexts..
R. Casadei Intro Technology: quick survey AC Wrap-up 38/52

43. Recap ACP Analysis
Many kinds of deployments
Such flexibility demands for fluid responsibilities
R. Casadei Intro Technology: quick survey AC Wrap-up 39/52

44. Recap ACP Analysis
The data perspective
Data payloads
SENSOR-DATA: map from sensor UIDs to sensor values
ACTUATION-DATA: map from actuator UIDs to actuation values
NBR-DATA (i.e., EXPORTS): map from neighbour UIDs to exports
CONTEXT-DATA = SENSOR-DATA + NBR-DATA
Control data
SCHEDULE-DATA (i.e., TICKs)
...
Configuration data
Deployment data, addresses, ...
...
Analysis from a data (flow) perspective
» On an app-specific basis, analysis of Volume-Velocity-Variety-Veracity
» Further analysis: persistent vs. transient data
» Further analysis: replication, partitioning, ...
» Analysis of data flow can help architectural design (cf., reactive architectures)
R. Casadei Intro Technology: quick survey AC Wrap-up 40/52

45. Outline
1 Intro
2 Technology: quick survey
3 Aggregate Computing
4 Wrap-up
R. Casadei Intro Technology: quick survey AC Wrap-up 41/52

46. Summary: key ideas
Pervasive Computing Platforms
PervComp needs platforms/middlewares to reduce the abstraction gap!
Aggregate Computing
• A macro-programming approach to CASs, formally grounded in Field Calculus
• Allows composition of (self-stabilizing) SASO patterns
• Is a fully operational approach that requires proper tooling
=⇒ SCAFI: a Scala framework for Aggregate Programming
• Provides an internal DSL for field-based computations
• Provides simulation facilities
• Provides an actor-based platform for building aggregate systems
• The abstract model is decoupled from the execution strategy
• Developing a flexible platform for AC is not easy
R. Casadei Intro Technology: quick survey AC Wrap-up 42/52

47. Aggregate Computing » Food for thought
1 Spatial Computing interpretation—Aggregate Systems as approximation of
space/environment; programs can be used to analyse or manipulate such space.
2 Programming ecosystems—expressing global laws to regulate the dynamics of
large-scale, situated, heterogeneous systems.
3 Context-oriented perspective—Devices can be seen as “contextual probes”;
programs can be used to reify contexts at different levels (from local to global).
4 Aggregate applications and system details—aggregate programs “assume” a
certain execution model and neighbouring relationship; how much can we stretch
such notions still preserving our properties of interest? how much can/should we
hide them?
5 Human-Robot systems—how may we use aggregate specifications to also
integrate humans in collective activities?
Conciliate autonomy and prescription
Handling unpredictability
6 Non-situated systems—can Aggregate Computing be still useful for
non-situated/spatial systems?
R. Casadei Intro Technology: quick survey AC Wrap-up 43/52

48. Perspectives, projects, future works I
Aggregate Computing is a huge effort that deals with several aspects & challenges!
Note: you have multiple choices
You have multiple choice on the style of the activity and level of engagement:
Course ←→ Thesis
Engineering ←→ Research
General ←→ Specific
Novelty, investigation ←→ Well-known challenges
Your activity may touch one or more of the following fields:
Mathematics
Complex Systems
Distributed algorithms
Programming languages/paradigms
Software engineering
Simulation
Devops
Computer security
Read carefully these slides and contact us for any doubt.
Feel free to ask or propose the kind of activity you prefer.
R. Casadei Intro Technology: quick survey AC Wrap-up 44/52

49. Perspectives, projects, future works II
Past projects
Actor-based SCAFI platform bootstrap
Support for a CI process in SCAFI
Improved support for distribution in SCAFI platform
SCAFI-Alchemist integration
Addition of mobility actuators in the SCAFI simulator to support e.g. flocking
Anticipative Gradient algorithm
SCAFI on Android
R. Casadei Intro Technology: quick survey AC Wrap-up 45/52

50. Perspectives, projects, future works III
Algorithms and SCAFI library
Testing of algorithms through simulations
Design of novel algorithms or implementation of “better” variants
E.g., flocking
SW engineering (AC as a case study)
Analysis/design and impl/refactoring of an AC Platform (Scala)
Scafi Simulator
Extensions to the ScaFi simulator
Extension of simulator GUI to support 3D simulations
Distributed simulations in a cluster
Analysis of a complete, flexible middleware solution for AC systems
Devops: IoT/Edge deployment of aggregate applications (containers)
R. Casadei Intro Technology: quick survey AC Wrap-up 46/52

51. Perspectives, projects, future works IV
Integration and applications
Web/RESTful interfaces to AC systems (à la WoT)
AC and IoT/Edge/Cloud Computing
AC and IoT platforms
Development of a custom PaaS for AC
Comparative study of IoT/PervComp middlewares and platforms
Prototyping a scalable AC cluster with open-source solutions
Reactive AC architecture
Fields like Big Data and NoSQL solutions
R. Casadei Intro Technology: quick survey AC Wrap-up 47/52

52. Perspectives, projects, future works V
Innovative themes and experiments
AC and computer security, trust, privacy
Aggregate processes
AC and context-oriented programming
AC and service choreographies (cf., Jolie, Chor)
AC and social networks
AC and complex networks
AC and topological methods (math-oriented)
Miscellaneous
Comparing AC to BSP-based graph processing systems (Pregel/Giraph)
Missing something? Provide your own ideas!
R. Casadei Intro Technology: quick survey AC Wrap-up 48/52

53. Application scenarios
Crowd engineering
Distributed sensing
Contextual services
Smart city (e.g., intelligent lighting)
Smart building
Industry 4.0
WSNs
Swarm intelligence (robots, drones)
Rescue scenarios
R. Casadei Intro Technology: quick survey AC Wrap-up 49/52

54. Project examples
1 Middleware—Analyse the problem of providing flexible cloud/flog/edge
middleware support for Aggregate Computing applications. Identify the key
general concerns, abstractions, non-functional requirements that might arise in
some application scenarios.
2 3DSim—Develop a prototype of a 3D Simulator for Aggregate Systems on the
JVM.
3 Edge—Study how Aggregate Computing techniques can be used to orchestrate
task scheduling in Edge Computing scenarios.
4 Flocks—Extend the ScaFi standard library with high-level flocking capabilities,
and provide nice demos.
5 Processes—Experiment with the process-oriented API of Scafi, and contribute
with additions and demos.
R. Casadei Intro Technology: quick survey AC Wrap-up 50/52

55. References
References I
[QPR+
16] Soheil Qanbari, Samim Pezeshki, Rozita Raisi, Samira Mahdizadeh,
Rabee Rahimzadeh, Negar Behinaein, Fada Mahmoudi, Shiva
Ayoubzadeh, Parham Fazlali, Keyvan Roshani, et al.
Iot design patterns: computational constructs to design, build and
engineer edge applications.
In Internet-of-Things Design and Implementation (IoTDI), 2016 IEEE First
International Conference on, pages 277–282. IEEE, 2016.
[RCKZ13] Vaskar Raychoudhury, Jiannong Cao, Mohan Kumar, and Daqiang Zhang.
Middleware for pervasive computing: A survey.
Pervasive and Mobile Computing, 9(2):177–200, 2013.
[ZM04] Franco Zambonelli and Marco Mamei.
Spatial computing: An emerging paradigm for autonomic computing and
communication.
In Workshop on Autonomic Communication, pages 44–57. Springer, 2004.
R. Casadei Appendix 51/52