Research work published on the 9th International Conference on Cloud Computing and Services Science (CLOSER 2019) held at Heraklion, Crete. The combination of Edge Computing devices and Cloud Computing resources brings the best of both worlds: Data aggregation closer to the source and scalable resources to grow the network on demand. However, the ability to leverage each time more powerful edge nodes to decentralize data processing and aggregation is still a significant challenge for both industry and academia. In this work, we extend the Garua platform to analyze the impact of a model for data aggregation in a global scale smart grid application dataset. The platform is extended to support global data aggregators that are placed nearly to the Edge nodes where data is being collected. This way, it is possible to aggregate data not only at the edge of the network but also pre-process data at nearby geographic areas, before sending data to be aggregated globally by global centralization nodes. The results of this work show that the implemented testbed application, through the usage of edge node aggregation, data aggregators geographically distributed and messaging windows, can achieve collection rates above 400 million measurements per second.

1. GaruaGeo: Global Scale Data Aggregation
in Hybrid Edge and Cloud Computing
Environments
Ot´avio Carvalho, Eduardo Roloff, Philippe O. A. Navaux
Informatics Institute – Federal University of Rio Grande do Sul
9th International Conference on Cloud Computing and Services Science – CLOSER 2019

2. Table of contents
1. Introduction
2. State of the art
3. Architecture
4. Evaluation
5. Conclusion and Future Works
2

3. Introduction – Motivation
• Internet of Things social and economic impact [1].
• By 2025, researchers estimate that the IoT will have a potential economic
impact of 11 trillion per year – which would be equivalent to about 11% of
the world economy. They also expect that one trillion IoT devices will be
deployed by 2025.
• Technologies created for IoT are driving computing toward
dispersion [4].
• Multi-Access Edge Computing
• Fog Computing
• Cloudlets
3

4. Introduction – Main goals
• Explore the potential performance improvements of moving
computation from cloud to edge.
1. Evaluate to what extent is possible to move the workload from cloud to
edge nodes.
2. Explore the limits of the proposed application architecture in terms of
latency and throughput.
3. Create strategies to reduce the amount of data sent to the cloud.
4

5. State of the art – Distributed Computing
• Since the 60s, computing has alternated between centralization
and decentralization [4]
• 60s & 70s: Centralized approaches like batch processing and
timesharing.
• 80s & 90s: Decentralization through the rise of personal computing.
• Mid-00s: Centralized approaches like cloud computing.
• Nowadays: The rise of fog computing and edge computing.
5

6. State of the art – Smart Grids
• For 100 years, there has been no change in the basic structure of
the electrical power grid. Experiences have shown that the
hierarchical, centrally controlled grid of the 20th Century is
ill-suited to the needs of the 21st Century. To address the
challenges of the existing power grid, the new concept of smart
grid has emerged [2].
• Smart Grids are the use case scenario for the testbed application
developed in this work.
• Low latency
• Needs to communicate energy consumption quickly to the grid operator
machines in order to balance demand and energy generation.
• High scalability
• Requires communication across large geographic regions to connect large
numbers of households.
6

7. State of the art – Related Works
Name Cloud Edge Mobility Large Scale Hardware Agnostic
GaruaGeo (this work) • • • •
ENORM • • •
RT-SANE • • •
Tarneberg et al. • • •
HomeCloud • • •
CloudAware • • •
FemtoClouds • •
REPLISOM • •
Cumulus • • •
ParaDrop • •
EdgeIoT • •
7

8. GaruaGeo – Proposal
• An extension to the Garua architecture.
• Aggregator nodes are placed geographically close to its companion
edge nodes.
• Improve the previous architecture by considering latency issues on
the design.
• Evaluate the platform in a globally distributed scenario.
8

9. GaruaGeo – Architecture
• Four-layered architecture
• Cloud layer
• Aggregates data from aggregator nodes.
• More performance (VMs on Azure).
• High latency.
• Aggregator layer
• Aggregates data from edge nodes.
• Nodes are placed geographically close to groups of edge nodes.
• Medium latency.
• Intermediate performance (Cloud nodes or physical hardware).
• Edge layer
• Aggregates data from sensors.
• Low latency.
• Less performance (Raspberry Pi Zero W).
• Sensor layer
• Provides events to edge nodes.
• Bluetooth, LTE, WiFi, etc.
9

10. GaruaGeo – Architecture
VM VM VM
VM
Cloud
Layer
Aggregator Aggregator
Aggregator
Layer
Edge Node Edge Node Edge Node
Edge
Layer
Sensor Sensor Sensor Sensor Sensor
Sensor
Layer
10

11. GaruaGeo – Aggregators Evaluation – Methodology
• The main goal is to explore the impact on throughput of adding a new
network layer.
• Evaluate the GaruaGeo architecture with a single aggregator in
comparison to Garua architecture.
• Explore distinct sets of edge nodes and messages sizes impact on
throughput.
• Evaluate potential performance gains of using multiple aggregator
nodes in the same geographic region to distribute the load from
multiple edge nodes.
11

12. GaruaGeo – Aggregators Evaluation
• Evaluate the throughput obtained when using an aggregation layer.
• Varying groups of message windows, from 1 to 1000 messages per window.
baseline aggregator
105
106
107
108
Execution type
Throughput(QPS)
1 10 100 1000
12

13. GaruaGeo – Aggregators Evaluation
• The main goal of this experiment is to visualize the aggregated impact of message
windowing and number of nodes on throughput.
• Aggregators are limited by the number of messages they can process from edge nodes in
a period of time.
• Message windows combines a larger set of message into a single message and increases
the overall throughput.
13

14. GaruaGeo – Aggregators Evaluation
• Explore the possibility of adding multiple aggregators in a given geographic region to avoid
overloading of a single aggregator.
• Splitting the load of a given aggregator node appears to produce substantial impact on
throughput.
• In this experiment, the same amount of edge nodes is distributed among distinct sets of
aggregator nodes, from 1 to 8 aggregator nodes.
1 2 4 8
0
50,000
1 ·105
Aggregators (1 to 8)
Throughput(QPS)
14

15. GaruaGeo – Geo-distributed Analysis – Methodology
• The main goal is to understand the behavior of the architecture in a
global scale deployment scenario.
• Evaluate the throughput on multiple regions to understand potential
performance discrepancies between regions (Microsoft Azure
datacenters).
• Evaluate the achievable throughput in a global scale deployment,
using up to 15 regions and 1366 machines across the globe.
15

16. GaruaGeo – Geo-distributed Analysis
• In this experiment, it is evaluated the potential performance discrepancies between distinct
Microsoft Azure regions (datacenters).
• It was not found significant discrepancies in performance (in terms of throughput) on the 5
distinct regions analysed.
16

17. GaruaGeo – Geo-distributed Analysis
• The scale of the global deployment used in the experiment (datacenters).
• 15 geographic regions (datacenters on Microsoft Azure).
• A single global cloud node.
• 15 aggregator nodes (one in each region).
• 90 edge nodes (in each region).
17

18. GaruaGeo – Geo-distributed Analysis
• Evaluation of the maximum achievable throughput of the platform across geographic
regions.
• 15 geographic regions (datacenters on Microsoft Azure).
• Message windows of 1000 messages.
• A total of 1366 machines on the scenario with 15 regions.
• Aggregation rates above 400 million measurements per second on the scenario with 15
regions.
1000
108.2
108.4
108.6
Groups of regions (5 to 15 regions)
Throughput(QPS)
5 10 15
18

19. Conclusions
• Evaluated the platform in a geo-distributed environment with
real-world latencies.
• Aggregation rates above 400 million measurements per second.
• Large scale evaluation using virtual machines on 15 geographic regions
across the globe on Microsoft Azure platform.
• A total of 1366 machines in the largest evaluation scenario.
• Improved performance by placing aggregators geographically close to
edge nodes.
• Reduced communication with the cloud by aggregating data at edge
level.
19

20. Future Works
• Explore distinct techniques for data scheduling, windowing and
aggregation at the edge of the network.
• Evolve the testbed application and its middleware into a generic
framework for distributed data processing.
• Apply other communication protocols to the platform.
20

21. Thanks! Questions?
21

22. References I
R. Buyya and A. V. Dastjerdi.
Internet of Things: Principles and paradigms.
Elsevier, 2016.
V. C. G¨ung¨or, D. Sahin, T. Kocak, S. Erg¨ut, C. Buccella, C. Cecati, and
G. P. Hancke.
Smart Grid Technologies: Communication Technologies and
Standards.
Industrial informatics, IEEE transactions on, 7(4):529–539, 2011.
Reuters.
U.S. Smart Grid to Cost Billions, Save Trillions, 2011.
M. Satyanarayanan.
The Emergence of Edge Computing.
Computer, 2017.
22

23. Appendix: Dataset
1. The dataset used to evaluate the platform originates from the 8th ACM
International Conference on Distributed Event-Based Systems (DEBS 2014)
2. The data file contains over 4055 Millions of measurements for 2125 plugs
distributed across 40 houses, for a total amount of 136 GB
3. Generated measurements cover a period of one month, from Sept. 1st, 2013,
00:00:00, to Sept. 30th, 2013, 23:59:59
23

24. Appendix: GaruaGeo – Cloud layer
Parameter Description
Instance Type Basic A3 (4 cores, 7 GB RAM)
Operating System Ubuntu 16.04 LTS
Golang version 1.8
GRPC version 1.3.0-dev
Protocol Buffers version 3.2.0
24

25. Appendix: GaruaGeo – Aggregator layer
Parameter Description
Instance Type Standard DS2 v2 (2 cores, 7 GB RAM)
Operating System Ubuntu 16.04 LTS
Golang version 1.8
GRPC version 1.3.0-dev
Protocol Buffers version 3.2.0
25

26. Appendix: GaruaGeo – Edge layer
Parameter Description
Instance Type Standard DS1 v2 (1 cores, 3.5 GB RAM)
Operating System Ubuntu 16.04 LTS
Golang version 1.8
GRPC version 1.3.0-dev
Protocol Buffers version 3.2.0
26