From IoT Devices to Cloud

From IoT Devices to Cloud
Computing Infrastructures
When (bi)millions small entities should work with a few giants
F. Desprez, INRIA
Entretiens Jacques Cartier - Montréal October 2017

Introduction
• Exponential improvement of
– Electronics (energy consumption, size, cost)
– Capacity of networks (WAN, wireless)
• Prediction between 28 and 50 billions of connected devices by 2020
(Ericsson, CISCO)
• Exponential growth of applications near users
– Smartphones, tablets, connected devices, sensors, …
• Large number of Cloud facilities to cope with generated data
– Many platforms and infrastructures available around the world
– Several offers for IaaS, PaaS, and SaaS platforms
– Public, private, community, and hybrid clouds
– Going toward distributed Clouds (FOG, Edge)
Entretiens Jacques Cartier - Oct. 2017F. Desprez - From IoT devices to Cloud Computing Infrastructures - 2

Entretiens Jacques Cartier -
Oct. 2017
F. Desprez - From IoT devices to Cloud Computing Infrastructures - 3http://www.beechamresearch.com/article.aspx?id=4

Target Applications: Industrial Internet
• Integration of complex physical machinery with networked sensors and
software
• Application examples
– Self-driving cars, smart’* (health, cities,
transportation, power grid, retail store, …)
• Ingest data from machines, analyze it (often
in real-time), and use it to adjust operations
• Several fields need to collaborate
– Internet of Things, Big Data,
machine-to-machine communications,
machine learning, Cyber-physical systems, …

Industrial Internet, contd
Recent Advances in Industrial Wireless Sensor Networks Toward Efficient Management in IoT, Sheng.Z., Mahapatra, C., Zhu, C., Leung, V.C.M., A., Kansakar, P.,
U.Kahn, S., IEEE, Jun. 2015.

Citylabs project @ Inria
• Privacy-aware Urban-scale Physical and Social Sensing
(FUN, MiMove, SMIS, AGORA)
• Energy-efficient wireless communication, Leveraging the IoT
• Physical &/vs social sensing, Fixed &/vs mobile sensing
• Ultra large scale & heterogeneous urban systems
• Incentives & privacy for citizens
• From Sensing to Modeling Cities (CLIME, DICE, MYRIADS,
OAK, WILLOW)
• Cloud-based management of semantic urban data
• Data assimilation combining simulation models & available data to
overcome uncertainties
• Urban-scale quantitative visual analysis to leverage the visual records of
urban environment
• Next Generation City Services promoting citizen engagement
(CLIME, MiMove, SMIS, WILLOW)
• AppCivist Social App
• City planning
• Democratizing environmental data
• Smart transportation systems
• Overcoming the Smart City Challenge
• Teams involved: AGORA, CLIME, DICE, FUN, MYRIADS, MIMOVE
SMIS, WILLOW
https://citylab.inria.fr/

Target Applications: Tactile Internet
• Ability to deliver physical experiences remotely
• The complete loop from the physical world, to the digital and back to
the physical
http://www.zeitgeistlab.ca/doc/tactile_internet.html

Target Application: Disaster Resilience
• Keep computing and network services running after a natural disaster or
attack
• Geographic redundancy of the components (over “small” devices?)
• Network (re)-configuration, path restoration and protection
• Backup VM for each working VM
• Modeling the risk!
Network design requirements for disaster resilience in IaaS clouds, R. de Souza Couto, S. Secci, M. E. Mitre Campista, and L. H. Maciel Kosmalski Costa, IEEE
Communications Magazine • October 2014

Needs and Performance Constraints
• Performances
– Big latency issues
• Voice: 100 ms (upper latency limit
for humans)
• Video : 10 ms
• Tactile internet : 1 ms
– Bandwidth (upstream traffic mainly)
– Real-time constraints
– Scalability
• Other constraints
– Security
– Privacy
– Availability
– Durability control

Entretiens Jacques Cartier -
Oct. 2017
F. Desprez - From IoT devices to Cloud Computing Infrastructures - 10
John Mc Carthy,
Speaking at the MIT centennial in 1961
If computers of the kind I have advocated
become the computers of the future, then
computing may someday be organized as a
public utility just as the telephone system
is a public utility...

Current Situation
• Large off shore DCs to cope with the increasing UC demand while handling
energy concerns
• But
• Jurisdiction concerns (data locality), PRISM NSA scandal, Patriot Act
• Reliability (disaster recovery), single point of failure
• Network overhead
• Localization is a key element to deliver efficient as well as sustainable Utility
Computing solutions

Cloud Evolution
Not only mega data centres !
Courtesy to Thierry Coupaye (Orange)

Trends for Next Generation Clouds
Centralized public clouds are in fact generally distributed over multiple (mega) data centres for
availability reasons
Verizon (©)
Orange (©)Microsoft (©)
Amazon (©)

Trends for Next Generation Clouds
• Hybrid and community clouds are by nature distributed over multiple data
centres/clouds

Convergence
ENIAC
1946
Transistor
1947
Computation
Communication
1999 - Salesforces
SaaS Concept
micro processor
1971
1838 - Telegraph
1876 - Telephone
1896 - Radio
1957 - satellite
1969 - ARPANET
1973 - Ethernet
1985 - TCP/IP Adoption
1975 -Personal
Computers
SmartPhones
2007
2002- Amazon Initial
Compute/Storage services
2006 - Amazon EC2 (IaaS)
2010 - Cloud
democratisation
2015
Network/Computers
Convergence
Software Defined XXX
1999 - The Grid
1995 - Commodity
clusters
2002 - Virtualised Infrastructure
1950/1990 - Mainframes
1950 - Batchmode
1960 - Interactive
1970 - Terminals (clients/server concepts)
1967 - First virtualisation attempt

Clouds, FOG, and Edge
• From a Cloud model (centralized mega data-centers) to a set of micro/nano
datacenters
• Locality based utility computing infrastructures
– Provide resources closer to the users
• Leverage network backbones
– Extend any point of presence of network backbones (aka PoP) with servers
• Extend to the edge by including wireless backbones
• Where should these micro-DC be deployed ?
• Energy and cost issues
• In the core network (POPs)
P
a
ul
a
B
o
b
Al
ic
e
D
u
k
e
Ch
arle
s
P
a
m
B
o
b
core backbone

Clouds, FOG, and Edge
• Cloud
– (Quite) centralized, big data centers, large resources, WAN
– Location depending on energy/taxes issues
• FOG
– First coined by CISCO
– OpenFog consortium in 2015 (ARM, Cisco, Dell, Intel, Microsoft, and Princetown)
– Geographically distributed computing architecture
– Resource pool of ubiquitously connected heterogeneous devices at the edge of the
network
• Edge
– Mobile Edge Computing (MEC)
– Edge of the cellular network
• Both Fog and Edge platforms push applications, data, and services away from
centralized nodes
IFCIoT: Integrated Fog Cloud IoT Architectural Paradigm for Future Internet of Things Munir, A., Kansakar, P., U.Kahn, S., arXiv, Jan. 2017.

Cloud-IoT Convergence
• IoT is here (and growing)
• Large Datacenters still efficient for large computations/data
management
• Micro/nano DCs to handle some computations closer to the users
• How should they be managed ?
Stability
Availability
Latency
Low latency
Heterogeneity
Low capacity

Research Issues
• Resource management
– Deployment, reconfiguration, location aware scheduling
• Data management
– User data, checkpoints, application images
• Network operation
– Virtualization
• Energy monitoring and consumption optimization
– Measures, resource management, multi-criteria, multiple sources, …
• Resilience
– Coping with failures (CPU, application, network, …) and attacks
• Security
• …
A Survey of Fog Computing: Concepts, Applications, and Issues? Yi, S., Li, C., Li, Q, Mobidata 2015, June. 2015.

Deployment and Reconfiguration
• Provisioning resources where they are needed
– Provisioning comes with a cost
– Limited capacity (≠ mega data-center)
• Zero-touch provisioning and reconfiguration
– Being able to deploy/reconfigure an edge site without human
interventions
– Data and computation
– Real-time elasticity
• Resource discovery
• Application image management
• Heterogeneous (and dynamic
platforms)
• Network issues (SDN, NFV)

Locality Aware Resource Management
• Mechanisms to manage the life cycle of applications (VM, containers,
bare metal) and data (users, applications) taking locality into account
• Several objective functions (multi-criteria scheduling)
– Resource consumption
– Network cost
– Energy
– $
• Classical scheduling/mapping problems revisited
– Many papers using classical ILP solvers (scaling issues there !)
• Placement of application graphs over infrastructure graphs
– Static or dynamic
• What’s about dynamicity ?
– Clients moving from one place to an other
– Failures

Locality Aware Resource Management
• Problem of placing application graphs, which represent application components and the
communication among these components, onto a physical graph, which represents the
computing devices and communication links in the physical system
– Tree topologies
• Baseline algorithm that provides an optimal solution to the placement of a linear
application graph (decomposable into multiple small building blocks)
• Simplification of the problem to make it tractable, NP-harness proof
• Off-line algorithm
Online Placement of Multi-Component Applications in Edge Computing Environments, Wang, S., Zafer, M., Leung, K.K., Mobidata 2015, June. 2015,
doi: 10.1109/ACCESS.2017.2665971.

Energy Monitoring and Consumption
Optimization
• Energy can be considered as the first metric for placement strategies
– i.e. relocate jobs/data according to the energy sources
• Preemptive jobs
– i.e. we can think about batch approaches and schedule them on the right edge DC at the right
moment
• Multi-criteria resource management
• Taking care of new energy sources (solar, wind, …)
• QoS for applications, resource consumption, energy cost
• Several issues
– Instrument realistic infrastructures,
– measure accurately consumption of resources,
– design the right models,
– isolate influential factors,
– combine energy models with performance models,
– propose models integrating inherent variability,
– perform campaign measurements,
– achieve invalidation studies

Renewable Energy and IoT
• Problem: How to decide to compute at the edge or offload at the edge depending on
QoS and energy-efficiency for a given IoT application?
– Performance/energy tradeoff
• Modeling application for its energy consumption and its response time
– Benchmarking (wattmeters, photovoltaic panel production traces) and simulation
– CPU and network
• Offloading the data to process video streams at edge
– Effectively reduces the response time
– Avoids unnecessary data transmission
between edge and core
– Extends for instance the battery lifetime of
end-user equipment
– On-site renewable energy production and
batteries in our scenario can save up to 50% total
consumed energy consumed at the edge
Leveraging Renewable Energy in Edge Clouds for Data Stream Analysis in IoT, Y. Li, A.-C. Orgerie, I. Rodero, M. Parashar, J.-M.
Menaud, CCGrid 2017.
Edge
Core
Edge1
data
aggregation
v-4 720p
v-5 480p
v-6 360p
Core
Edge
Core
Edge0
v-3 360p
v-2 360p
v-1 360p
r0: p=(a,b),
ac = n%
A
B
C
Data stream
analysis from
cameras
embedded on
vehicles

Resilience
• Several Cloud failures in the past
– Dropbox, Netflix, Amazon
– Huge costs involved
• Advantage of Edge computing platforms
– No single point of failure
• At the infrastructure level
– Replication of VMs and data on various geographic locations
– Proactive and reactive strategies taking into account network latency into
account
• At the middleware level
– Rescheduling of failed tasks
• At the application level
– Periodical checkpointing (taking into account locality)
A Survey of Fog Computing: Concepts, Applications, and Issues? Yi, S., Li, C., Li, Q, Mobidata 2015, June. 2015.

Virtualization/Sandboxing Technologies
• SDN/NFV requirements also requires edge DCs
• VMs/Containers/Baremetals
– How to deliver those abstractions at the edge
– Booting a VM may last minutes if the VM image is a remote attached volume
– Containers boot faster but they also require containers images
• where should we put those images?
• What's about Data?
– Where should be the data put?
– Can we envision data storage repository in every edge site?
• Extreme edge (i.e. inside Rasbperry PI, home gateways, ....)
– No sufficient resources to start VM/containers with local images
– Some system mechanisms should be deployed locally whereas other ones
should stay higher in the infrastructure

Other issues
• FOG networking
– Maintaining connectivity with heterogeneous (and dynamic) networks
– Use/adaptation of Software Defined Networking (SDN) and Network
Function Virtualization (NFV) features
– Quality of Service
• Interfacing and programming model
– Right now assembly code level (bunch of low level models for each kind of
platforms)
– Need of a unified model ?
• Accounting, billing and monitoring
• Privacy
• Simulation and experiments
– How to validate algorithms, protocols, and software stacks

Security Issues

The Discovery Initiative
• Leverage network backbones
– Extend any Point of Presence (PoP) of network backbones with
servers (from network hubs up to major DSLAMs that are operated by
telecom companies, network institutions…).
• Extend to the edge by including radio base stations
• Discovery
– how to operate such a massively distributed infrastructure
USA NREN
http://www.renater.fr/raccourci?lang=fr
http://beyondtheclouds.github.io/

Revise OpenStack to Support Fog/Edge Computing
Infrastructures
• Do not reinvent the wheel… it is too late
• Mitigate development efforts
– By favoring a bottom/up approach
– Investigate whether/how OpenStack core services can become
cooperative by default (using P2P and Self-* technics)

Several research issues for Discovery
• Cost of the network(s) ?
• Partial view of the system ?
• Impact on others VMs ?
• Management of VM images ?
• How to take into account locality aspects?
• Which software abstractions to make the development easier and
more reliable (distributed event programming)? …
• OpenStack distribution and deployment
Beyond The Cloud, How Should Next Generation Utility Computing Infrastructures Be Designed? Lèbre, A., J. Pastor, J., Bertier, M., Desprez, F., Rouzaud-
Cornabas, J., Tedeschi, C., Anedda, P., Zanetti, G., Nou, R., Cortes, T., Riviere, E. and Ropars, T., INRIA Research Report 8348, Aug. 2013.

• Pro
• Locality (jurisdiction concerns, latency-aware apps, minimize network overhead)
• Reliability/redundancy (no critical point/location/center)
• The infrastructure is naturally distributed throughout multiple areas
• Lead time to delivery
• Leverage current PoPs and extend them according to UC demands
• Energy footprint (on-going investigations with RENATER)
• Bring back part of the revenue to NRENs/Telcos
• Cons
• Security concerns (in terms of who can access to the PoPs)
• Operate a fully IaaS in a unified but distributed manner at WAN level
• Not suited for all kinds of applications : Large tightly coupled HPC workloads 50 nodes/1000 cores,
200 nodes / 4000 cores (5 racks), so 1000 nodes in one PoP does not look realistic …
• Peering agreement / economic model between network operators
32Labex UCN@Sophia – F. Desprez Feb. 18, 2016
The DISCOVERY Initiative Pros and Cons

“Good experiments”
A good experiment should fulfill the following properties
– Reproducibility: must give the same result with the same input
– Extensibility: must target possible comparisons with other works
and extensions (more/other processors, larger data sets,
different architectures)
– Applicability: must define realistic parameters and must allow
for an easy calibration
– “Revisability”: when an implementation does not perform as
expected, must help to identify the reasons

SILECS: Super Infrastructure for Large-
scale Experimental Computer Science
• Having a large scale infrastructure to experiment IoT/Edge cloud
applications and software stacks
– Scaling factor
– Exascale platforms
– Virtualized, Programmable
– FOG and Mobile Edge Computing
• Features
– Manageability
• Agility (SDN, NFV)
• Self adaptability
• Global orchestration
– Complexity
• Resources
• Energy
– Data Flow Management
• Data deluge processing

SILECS: based upon two infrastructures
• FIT
– Proving Internet players access to a variety of fixed and mobile technologies and
services, thus accelerating the design of advanced technologies for the Future Internet
– 4 key technologies and a single control point: IoT-Lab (connected objects & sensors,
mobility), CorteXlab (Cognitive Radio), wireless (anechoic chamber), Network Operations
Center (including a PLE access), Advanced Cloud technology including OpenStack
– 9 sites (Paris (2), Evry, Rocquencourt, Lille, Strasbourg, Lyon, Grenoble, Sophia
Antipolis)
• Grid’5000
– A scientific instrument for experimental research on large future infrastructures:
Clouds, datacenters, HPC exascale, Big Data infrastructures, networks, etc.
– 10 sites, service nodes, > 8000 cores, with a large variety of network connectivity and
storage access, dedicated interconnection network granted and managed by RENATER
gathered around a GIS (CNRS, CEA, Inria, CPU, RENATER, Institut Mines-Telecom,
CDEFI)
• Software stacks dedicated to experimentation
• Monitoring tools, resource reservation, data collection and storage

Grid’5000
• Testbed for research on distributed systems
• Born from the observation that we need a better and larger testbed
• HPC, Grids, P2P, and nowCloud computing and BigData systems
• A complete access to the nodes’ hardware in an exclusive mode
(from one node to the whole infrastructure)
• Dedicated network (RENATER)
• Reconfigurable: nodes with Kadeploy and network with KaVLAN
• Current status
• 10 sites, 29 clusters, 1060 nodes, 10474 cores
• Diverse technologies/resources
(Intel, AMD, Myrinet, Infiniband, two GPU clusters, energy probes)
• Some Experiments examples
• In Situ analytics
• Big Data Management
• HPC Programming approaches
• Network modeling and simulation
• Energy consumption evaluation
• Batch scheduler optimization
• Large virtual machines deployments

FIT Infrastructure
FIT-CorteXlab: Cognitive Radio Testbed
40 Software Defined Radio Nodes
(SOCRATE)
FIT-Wireless: WiFi mesh testbed
(DIANA)
FIT-IoT-LAB
• 2700 wireless sensor nodes spread across six different sites in France
• Nodes are either fixed or mobile and can be allocated in various topologies throughout all sites.
Sophia
Lyon

SILECS Design Objectives
• Deploy a large set of digital resources from sensors to data centers
– Open, remotely accessible, virtualized infrastructure
– Provide rich, diverse and advanced tools: test, measurement, benchmarking,
reproducibility, data repository, …
– Typically a « mid-scale » infrastructure
• Mobilize the scientific community in the domain of digital sciences
– Articulate the French and European efforts in this domain
– International attractivity and visibility (unique today at the international level)
• Several challenges
– Heterogeneity of the resulting infrastructures
– Different communities and different software stacks
– Keep reproducibility at its highest level
– Keep the infrastructure up-to-date
– Connect the infrastructure to other platforms in Europe and elsewhere

The GRAIL

SILECS
• New infrastructure based on two existing instruments (FIT and
Grid’5000)
• Keep the aim of previous platforms (their core scientific issues
addressed)
– IoT, wireless networks, future Internet for FIT
– HPC, Big Data, Clouds, Virtualization, … for Grid’5000
• Address new challenges
– IoT and Clouds
– New generation Cloud platforms and software stacks (Edge, FOG)
– Data streaming applications
– Locality aware resource management
– …
• Submitted to ESFRI in August

Conclusions
• Epic battle between centralization and distribution
– Batch processing, supercomputers, P2P, Grid, Cloud, Fog, and Edge
• Tons of new applications (with new related issues) coming
• Probably a mix of different approaches to get the best from every
infrastructure
– Regular DC, Edge, Extreme Edge
– Performance, Quality of Service, energy consumption
• Lots of research issues (both theoretical and software design issues)
• Distributed computing/network convergence
• We need new models to handle heterogeneity (CPU, networks,
storage) and dynamicity
• Scale issue
• How to perform significant experiments for these problems ?
• We live in an exciting time !

Thanks. Any questions ?
Thanks to Adrien Lebre (ASCOLA/STACK,
Inria, France), Anne-Cécile Orgerie (Myriads,
Inria, France), Thierry Coupaye (Orange,
France), Omer Rana (UK)

From IoT Devices to Cloud

More Related Content

What's hot

Similar to From IoT Devices to Cloud

More from Frederic Desprez

From IoT Devices to Cloud