Snap4 has been created as an open, standardized, data-driven, service-oriented, user-centric platform enabling large-scale co-creation IOT/IOE applications and services for Helsinki, Copenhagen and Antwerp. Snap4 is a fully open source, robust, scalable, easy to use solution, provides tools for co-creation of mixt data driven, stream and batch processing, extending the powerful semantic reasoner of Km4City https://www.km4city.org, with IOT/IOE, GDPR, and city dashboards. Snap4 for Smart city is Snap4City (Https://www.snap4city.org ) is an open platform and solution for setting up Living Labs engaging different all kinds of stakeholders (city operators, researchers, city users, in house, industries) in contributing to the city evolutions, with a platform providing online tools for developing IOT applications, web and mobile Apps, data analytics, micro Applications, external services, KPI, POI, dashboards, IOT edge, etc. Snap4city has been validated in multiple devices (PC, Android, Raspberry, IOT Button, Arduino, ..), and domains: mobility and transport, tourism, health, welfare, social and cities such as Florence, Pisa, Arezzo, and large area of millions on inhabitants as Tuscany and millions of data per day. The innovation is mainly related to semantic reasoning, IOT interoperability, microservices, automated dashboard production, end-2-end encrypted secure communications, GDPR, .. thus setting up in a Snap smart city solutions. The solution is fully complient with NodeJS with nodex published on JS foundation, is powered by Fi-Ware, compliant with LoraWan, SigFox, Mqtt, AMQP, Coap, NGSI, OMAM2M, WSs, Https, powered by Km4City, TensorFlow NVIDIA, Hadoop, etc. etc. slides and demos: the platform includes full stack, any format, any protocol, from IOT Device, IOT Edges, Data Analytics, and Dashboards.

1. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
Snap4City scalability, i‐Cities 2018, DISIT lab (C)
https://www.disit.org/
Paolo Nesi, paolo.nesi@unifi.it
https://www.Km4City.orghttps://www.snap4City.org
IOT/IOE Elastically Scalable Architecture for Smart City and Industry 4.0
1
Pierfrancesco Bellini, Daniele Cenni, Paolo Nesi

2. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org Issues
• IOT Applications are typically being executed on cloud and IOT edge
devices; each of them may:
– Be defined as applications (e.g., NodeJS) or simple rules, horn clauses, if‐then‐
else rules, etc.
– IOT App may be
• more flexible and capable to respect requirements on protection, privacy, GDPR, user
access and personalization, etc.
• executed on Docker, VM, cloud
– End to end Security, robustness, MicroServices, GDPR, etc.
– Relevant problems regarding scalability: vertical and horizontal
– Resources are: CPU, Memory, Network, Storage,
• But also for each IOT App and User: IOT Devices, MicroServices, ExternalServices,
DataAnalytics, MicroApplications, Dashboards, connections/traffic, etc.
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 2

3. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Smart City Cloud Infrastructure
Km4City Smart City API
Knowledge
Base
ETL Processes, Data Analytic, R; IOT App; etc.
Data Processing Tools
Development and Management Tools
ETL Processes
Resource
Manager
DataGate/
CKAN
Km4City Ontology
Phoenix, Hbase
+ indexing
Big Data Storage Knowledge
IoT/IoE Applications
AMMA
Linked
Open Graph
ServiceMap Data Flow Analysis
DevDash
Elastic Management of Containers
Mobile and Web Apps
Final Users’ Tools
Dashboards
Social Media
IoT/IoE
Open Data
Personal Data
Industry 4.0
GIS + Map Data
IOT / IOE Apps
IOT Directory
Management
Authentication, Authorization, GDPR, Security Assessment
Resources and the ArchitectureSnap4City scalability, i‐Cities 2018, DISIT lab (C) 4

4. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 5
Data processing
Development
Environment
R Studio, Tensor
Flow, ……
Test DISCES
Tools for Developers
ETL Processes
Data Analytics
Processes
Cloud Elastic Management
NodeRED
Snap4City
Developer
NodeRED
Snap4City
Final User
Cluster Manager, DISCES Scheduler
Process
Loader
Knowledge Base
NIFI Indexing and
Search Services
ServiceMap
Sharded Index
Map Reduce
Kettle
Hbase
HDFS
Phoenix
Map Reduce
Kettle
Hbase
HDFS
Phoenix
Map Reduce
Kettle
Hbase
HDFS
Phoenix
NodeRED
Snap4City
Developer
NodeRED
Snap4City
Developer
NodeRED
Snap4City
Final User
NodeRED
Snap4City
Final User
Km4City
Event LoggerBig Data Store
IOT Directory
ServiceMap3D
SigFOX Server
IOT Broker
Proxy Filter Security
OneM2M Server
LoraWAN Gateways
….. Gateways
IOT Application Builder
Dashboardsand Applications
Main.snap4city.org / www.snap4city.org
LOG.disit.org/Flint
Swagger
Resource Manager
DataGate/CKAN
External Services…..
Data Mapper
GDPR Services: User Profiling, Auditing, MyPersonalData, MyAnnotation, My IOT Keys, …..
Authentication and Authorisation Services and Auditing (KeyCloak, LDAP), Certification, HTTPS
Annotations
IOT Registration
Notificator
Dashboard Builder
Wizard & Widgets
Chat Manager
Tools for Administrators
KnowlegeManagement
ResDash
AMMA
DevDash
External Services
City Operators Srv
Legacy Srv
Social Media
……………
Snap4City MicroServices
Smart City API
Tools for Users
Data Inspector
MicroApplications
IOT Broker
Proxy Filter Security
IOT Edge
NodeRED
Snap4City
Developer
IOT Apps
IOT Edge
NodeRED
Snap4City
Developer
IOT Apps
IOT Edge
NodeRED
Snap4City
Developer
IOT Apps

5. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Coverage: examples
• Search all services in the area Snap4City scalability, i‐Cities 2018, DISIT lab (C)
Antwerp
Bologna Toscana
Sardegna
Pisa
6

6. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Km4City in Tuscany Area
What is enabling and providing smart services
• Smart Parking, in Tuscany
• Smart First Aid in Tuscany
• Smart search for POI and public transport srv.
• Public Transportation in Tuscany
• Routing in Tuscany, simple and multimodal
• Social Media Monitoring and acting
• Traffic events and Resilience in Florence
• Bike Sharing in Pisa and Siena
• Recharge stations for e‐vehicles
• Entertainment Events in Florence
• Traffic Sensors in Tuscany
• IOT/IOE sensors and actuators
• Weather forecast/condition in Tuscany
• Pollution and Pollination in Tuscany
• People Monitoring, in Tuscany via App
• ..People Monitoring Assessment in the City, in
Florence via Wi‐Fi
All Point of Interests, cultural activities, IOT, …
Over than 1.2 Million of complex events per day!https://servicemap.km4city.org 7

7. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Level 4 users: dashboard with intelligence App
• Dashboards with IOT Applications for enforcing smart and
intelligence into them.
DashboardsIOT and City data World IOT Applications
My IOT Devices
Dashboard‐IOT App
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 8
Applications

8. Snap4City scalability, i‐Cities 2018, DISIT lab (C) 9
City Dashboard + IOT App
Control Room Operator
Would like to:
‐ Monitor traffic flow,
Environment, Car parking,
Cycling, First aid, temp., ..
‐ Act and monitor Dynamic
Plates
‐ Act and monitor red lights
Driver, Policeman
Would like to:
‐ Monitor traffic,
Parking, env., speed
limit, …
‐ Act and monitor red
lights

9. IOT Application with City Dash
simpler development
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 10

10. Snap4City scalability, i‐Cities 2018, DISIT lab (C) 11

11. Snap4City scalability, i‐Cities 2018, DISIT lab (C) 12

12. Snap4City scalability, i‐Cities 2018, DISIT lab (C) 13

13. Snap4City MicroServices
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 14

14. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Hosts with VM on Cloud HA, DRS, FT
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 15
Event Log
Zeppelin
Phoenix
HDFS, Hbase
debian
Event Log
Zeppelin
Phoenix
HDFS, Hbase
debian
Event Log
Zeppelin
Phoenix
HDFS, Hbase
debian
Event Log
Zeppelin
Phoenix
HDFS, Hbase
debian
Hosts/VM on Cloud in HA, elastic computing, ..
.
.
.
……
……..
……….
.
containersDistribSrv.
slave slave slave slave
Pool of Containers
NodeRed, Java
Penthao Kettle, Java, Karma
……..
R‐Stat, R‐Parallel
Hadoop/Spark/SparkML
Python, C++, C, java
…………..
Master Master Master
Masters in HA: Marathon/Mesos, Zookeeper, ….
VmWare VSphere
Cnt. Scheduling
Cloud Elastic
Management
Cnt. Scheduling
Cloud Elastic
Management
DISCES
+ Planner +
Elastic Management
Containers
Control API
Cloud Management
System &
Resource Monitoring
Cloud Management
System &
Resource Monitoring
Resource Monitoring
&
Cloud Resource
Management
Monitoring API
VM Control API

15. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Elastic Scaling: allocare / deallocare
• Allocazione / deallocazione, Rebalancing vs compacting
– Verticale delle risorse Docker e/o VM: CPU, Mem
• NodeJS multiflusso per ogni Docker, l’utente richiede i flussi e le IOT App, il sistema le
alloca in modo dinamico e ottimizza il carico
• VM: gestione risorse Mem, CPU; trasparente ed automatica in DRS VmWare
– Orizzontale delle risorse Docker e/o VM e/o [Host]:
• Docker: aggiunta, spostamento (per clonazione) di IOT App
• VM: on/off
• Monitoraggio risorse:
– VM via VmWare API, Docker via Marathon and Mesos API
• Algoritmo in Python: azioni via API: VmWare, Marathon
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 16

16. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 17

17. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 18

18. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 19

19. DISIT Lab, Distributed Data Intelligence and Technologies
Distributed Systems and Internet Technologies
Department of Information Engineering (DINFO)
http://www.disit.dinfo.unifi.it
http://www.disit.org
Note
• Https://www.snap4city.org
• Registratevi se lo volete provare o se volete saperne di piu’
Snap4City scalability, i‐Cities 2018, DISIT lab (C) 20