The document discusses the Snap4City project, which provides an elastically scalable architecture for smart cities and Industry 4.0 by integrating IoT applications across cloud and edge devices. It outlines various features including user access management, data analytics, and real-time monitoring for urban environments, specifically within the Tuscany region. Additionally, it highlights the importance of both vertical and horizontal scaling of resources, alongside robust security measures to ensure compliance with GDPR.

1. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
Snap4City scalability, i‐Cities 2018, DISIT lab (C)
https://www.disit.org/
Paolo Nesi, paolo.nesi@unifi.it
https://www.Km4City.orghttps://www.snap4City.org
IOT/IOE Elastically Scalable Architecture for Smart City and Industry 4.0
1
Pierfrancesco Bellini, Daniele Cenni, Paolo Nesi

2. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org Issues
• IOT Applications are typically being executed on cloud and IOT edge
devices; each of them may:
– Be defined as applications (e.g., NodeJS) or simple rules, horn clauses, if‐then‐
else rules, etc.
– IOT App may be
• more flexible and capable to respect requirements on protection, privacy, GDPR, user
access and personalization, etc.
• executed on Docker, VM, cloud
– End to end Security, robustness, MicroServices, GDPR, etc.
– Relevant problems regarding scalability: vertical and horizontal
– Resources are: CPU, Memory, Network, Storage,
• But also for each IOT App and User: IOT Devices, MicroServices, ExternalServices,
DataAnalytics, MicroApplications, Dashboards, connections/traffic, etc.
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 2

3. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Smart City Cloud Infrastructure
Km4City Smart City API
Knowledge
Base
ETL Processes, Data Analytic, R; IOT App; etc.
Data Processing Tools
Development and Management Tools
ETL Processes
Resource
Manager
DataGate/
CKAN
Km4City Ontology
Phoenix, Hbase
+ indexing
Big Data Storage Knowledge
IoT/IoE Applications
AMMA
Linked
Open Graph
ServiceMap Data Flow Analysis
DevDash
Elastic Management of Containers
Mobile and Web Apps
Final Users’ Tools
Dashboards
Social Media
IoT/IoE
Open Data
Personal Data
Industry 4.0
GIS + Map Data
IOT / IOE Apps
IOT Directory
Management
Authentication, Authorization, GDPR, Security Assessment
Resources and the ArchitectureSnap4City scalability, i‐Cities 2018, DISIT lab (C) 4

4. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 5
Data processing
Development
Environment
R Studio, Tensor
Flow, ……
Test DISCES
Tools for Developers
ETL Processes
Data Analytics
Processes
Cloud Elastic Management
NodeRED
Snap4City
Developer
NodeRED
Snap4City
Final User
Cluster Manager, DISCES Scheduler
Process
Loader
Knowledge Base
NIFI Indexing and
Search Services
ServiceMap
Sharded Index
Map Reduce
Kettle
Hbase
HDFS
Phoenix
Map Reduce
Kettle
Hbase
HDFS
Phoenix
Map Reduce
Kettle
Hbase
HDFS
Phoenix
NodeRED
Snap4City
Developer
NodeRED
Snap4City
Developer
NodeRED
Snap4City
Final User
NodeRED
Snap4City
Final User
Km4City
Event LoggerBig Data Store
IOT Directory
ServiceMap3D
SigFOX Server
IOT Broker
Proxy Filter Security
OneM2M Server
LoraWAN Gateways
….. Gateways
IOT Application Builder
Dashboardsand Applications
Main.snap4city.org / www.snap4city.org
LOG.disit.org/Flint
Swagger
Resource Manager
DataGate/CKAN
External Services…..
Data Mapper
GDPR Services: User Profiling, Auditing, MyPersonalData, MyAnnotation, My IOT Keys, …..
Authentication and Authorisation Services and Auditing (KeyCloak, LDAP), Certification, HTTPS
Annotations
IOT Registration
Notificator
Dashboard Builder
Wizard & Widgets
Chat Manager
Tools for Administrators
KnowlegeManagement
ResDash
AMMA
DevDash
External Services
City Operators Srv
Legacy Srv
Social Media
……………
Snap4City MicroServices
Smart City API
Tools for Users
Data Inspector
MicroApplications
IOT Broker
Proxy Filter Security
IOT Edge
NodeRED
Snap4City
Developer
IOT Apps
IOT Edge
NodeRED
Snap4City
Developer
IOT Apps
IOT Edge
NodeRED
Snap4City
Developer
IOT Apps

5. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Coverage: examples
• Search all services in the area Snap4City scalability, i‐Cities 2018, DISIT lab (C)
Antwerp
Bologna Toscana
Sardegna
Pisa
6

6. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Km4City in Tuscany Area
What is enabling and providing smart services
• Smart Parking, in Tuscany
• Smart First Aid in Tuscany
• Smart search for POI and public transport srv.
• Public Transportation in Tuscany
• Routing in Tuscany, simple and multimodal
• Social Media Monitoring and acting
• Traffic events and Resilience in Florence
• Bike Sharing in Pisa and Siena
• Recharge stations for e‐vehicles
• Entertainment Events in Florence
• Traffic Sensors in Tuscany
• IOT/IOE sensors and actuators
• Weather forecast/condition in Tuscany
• Pollution and Pollination in Tuscany
• People Monitoring, in Tuscany via App
• ..People Monitoring Assessment in the City, in
Florence via Wi‐Fi
All Point of Interests, cultural activities, IOT, …
Over than 1.2 Million of complex events per day!https://servicemap.km4city.org 7

7. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Level 4 users: dashboard with intelligence App
• Dashboards with IOT Applications for enforcing smart and
intelligence into them.
DashboardsIOT and City data World IOT Applications
My IOT Devices
Dashboard‐IOT App
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 8
Applications

8. Snap4City scalability, i‐Cities 2018, DISIT lab (C) 9
City Dashboard + IOT App
Control Room Operator
Would like to:
‐ Monitor traffic flow,
Environment, Car parking,
Cycling, First aid, temp., ..
‐ Act and monitor Dynamic
Plates
‐ Act and monitor red lights
Driver, Policeman
Would like to:
‐ Monitor traffic,
Parking, env., speed
limit, …
‐ Act and monitor red
lights

9. IOT Application with City Dash
simpler development
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 10

10. Snap4City scalability, i‐Cities 2018, DISIT lab (C) 11

11. Snap4City scalability, i‐Cities 2018, DISIT lab (C) 12

12. Snap4City scalability, i‐Cities 2018, DISIT lab (C) 13

13. Snap4City MicroServices
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 14

14. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Hosts with VM on Cloud HA, DRS, FT
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 15
Event Log
Zeppelin
Phoenix
HDFS, Hbase
debian
Event Log
Zeppelin
Phoenix
HDFS, Hbase
debian
Event Log
Zeppelin
Phoenix
HDFS, Hbase
debian
Event Log
Zeppelin
Phoenix
HDFS, Hbase
debian
Hosts/VM on Cloud in HA, elastic computing, ..
.
.
.
……
……..
……….
.
containersDistribSrv.
slave slave slave slave
Pool of Containers
NodeRed, Java
Penthao Kettle, Java, Karma
……..
R‐Stat, R‐Parallel
Hadoop/Spark/SparkML
Python, C++, C, java
…………..
Master Master Master
Masters in HA: Marathon/Mesos, Zookeeper, ….
VmWare VSphere
Cnt. Scheduling
Cloud Elastic
Management
Cnt. Scheduling
Cloud Elastic
Management
DISCES
+ Planner +
Elastic Management
Containers
Control API
Cloud Management
System &
Resource Monitoring
Cloud Management
System &
Resource Monitoring
Resource Monitoring
&
Cloud Resource
Management
Monitoring API
VM Control API

15. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Elastic Scaling: allocare / deallocare
• Allocazione / deallocazione, Rebalancing vs compacting
– Verticale delle risorse Docker e/o VM: CPU, Mem
• NodeJS multiflusso per ogni Docker, l’utente richiede i flussi e le IOT App, il sistema le
alloca in modo dinamico e ottimizza il carico
• VM: gestione risorse Mem, CPU; trasparente ed automatica in DRS VmWare
– Orizzontale delle risorse Docker e/o VM e/o [Host]:
• Docker: aggiunta, spostamento (per clonazione) di IOT App
• VM: on/off
• Monitoraggio risorse:
– VM via VmWare API, Docker via Marathon and Mesos API
• Algoritmo in Python: azioni via API: VmWare, Marathon
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 16

16. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 17

17. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 18

18. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 19

19. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Note
• Https://www.snap4city.org
• Registratevi se lo volete provare o se volete saperne di piu’
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 20