1. Orquestração Multi-Domínio
Christian Esteve Rothenberg
Universidade Estadual de Campinas
(UNICAMP)

2. Legacy
Data Plane
Mgm.APIs
Distributed
L2/L3
Control Plane
Managemt
Software
Southbound
Agent
(e.g. OF)
Network Controller / OS
Southbound
Protocol (e.g. OF)
Business / Control Apps
Northbound APIs
Mgm.
HAL APIs / Drivers
APIs
Compiler
Auto-GeneratedTarget Binary
SDNSDN
VNF
HW Resources
Virtualization
DP
CP
M
g
m
GP-CPU
(x86, ARM)
NFVNFV
VNFM
(Manager)
VIM
(Infra-M)
OSS/BSS
APIs
Southbound
APIs/Plugins
Mgm. Apps
Network OS / Bare Metal Switches
Background: Network Softwarization
Orchestrator (SO/RO/LCM)

3. 3
Background: What does “Orchestration” mean?

4. 4
Background: What does “Orchestration” mean?

5. 5
NSO Fundamentals

6. Network Service Orchestration: High-level reference model
Source: Network Service Orchestration: A Survey. https://arxiv.org/abs/1803.06596

7. Domain
• Administrative: different organizations and therefore may exist within a single
service provider or cover a set of service providers [RFC 1136: “...A group of hosts,
routers, and networks operated and managed by a single organization...”]
• Technological : type of technology (e.g., Cloud, SDN, NFV, Legacy)
Multi-Domain Orchestrator (MDO)
• (Exposes the available services to the marketplace)
• Service Orchestrator (SO): high-level service models
• Resource Orchestrator (RO): managing resource and orchestrating workflows
Domain Orchestrator (DO)
• Performs per-domain orchestration acting on the underlying infrastructure resources
7
Definitions

8. 8
Definitions | Approaches | Scope | Standardization

9. 9
Orchestration: automated, programmatic & coordinated control and
management of resources for creating, adapting or removing network services.
Definition: Orchestration

10. 10
Orchestration = F(Management,Automation)
Source: Network Service Orchestration: A Survey. https://arxiv.org/abs/1803.06596

11. 11
NSO::Functions
Source: Network Service Orchestration: A Survey. https://arxiv.org/abs/1803.06596

12. 12
NSO::Practical Definition
Source: Network Service Orchestration: A Survey. https://arxiv.org/abs/1803.06596

13. ETSI approaches for multiple administrative domains
13
Source: ETSI
NSO::Single and Multi-Domain

14. Taxonomy
Source: Network Service Orchestration: A Survey. https://arxiv.org/abs/1803.06596

15. Standardization

16. Research Projects
Source: arxiv.org/abs/1803.06596

17. 17
Open Source Developments

18. 18
Open Source Implementations
Source: arxiv.org/abs/1803.06596

19. Virtualization
Orchestration
Network APP Network APP Network APP… …
Transport, Radio and Cloud resources
Radio CloudCloud Optical
PacketPacket
uW
Optical
Access Aggregation
control
virtualization
control
virtualization
control
virtualization
control
virtualization control
virtualization
control
virtualization
control
virtualization
control
virtualization
Kista 5G Transport Lab
Scenarios: 5G
Source: Netsoft 2017 Tutorial: End-to-End Programmability and Orchestration in 5G Networks.

20. Network App
Orchestration
Service orchestrator
Resource orchestrator
Resource orchestrator Resource orchestrator
Transport Control A Radio Control Cloud Control Transport Control B
SDN / OpenDayLight Distributed controlOpenStack incl internal DC NW
OSS / BSS
Scenarios: 5G
Source: Netsoft 2017 Tutorial: End-to-End Programmability and Orchestration in 5G Networks.

21. Expose just enough information to make optimal resource orchestration.
Provide service
Orchestration
Layer: ~1
Domain
Controllers: 10s
Network
nodes: 1000s
Detailed control
Technology
dependent
Simplified view
Relevant data
Scenarios: 5G
Source: Netsoft 2017 Tutorial: End-to-End Programmability and Orchestration in 5G Networks.

22. Transport
Control
RAN
Control
Orchestration
Programmable Transport
Network App
(Joint Optimization)
Joint Optimization of RAN & Transport
• Elastic Mobile Broadband Service
• Joint RAN-Transport Slicing (Multi-
operator)
• Joint Load-balancing
• Energy saving
• Dynamic clustering
• Pooling
• Shared fronthaul
• Resilience
Scenarios: 5G
Source: Netsoft 2017 Tutorial: End-to-End Programmability and Orchestration in 5G Networks.

23. 23
Source: Network Slicing for 5G with SDN/NFV: Concepts, Architectures, and Challenges.
● Mechanism to provide flexible management of network resources
● Enable operators to create multiple network resources and (virtual) network
● Functions isolated and customized over a shared physical infrastructure
NSO to automate the lifecycle of a slice, providing multi-operator coordination management in
order to create end-to-end network slices across multiple administrative domains
Scenarios: Network Slicing

24. Net App
Net App
NFs
Net App
Net App
L7 Apps
Network
Resources
NIM
Slicing
Application Services
Vertical
Use
Case i
Control & Management plane
Infrastructure
Business (Application & Service) plane
Slicing
Compute
Resources
VIM
Slicing
MonitoringMonitoringMonitoring
VIM-independent Slicing [0]
(“Bare-metal”)
[Infrastructure Slice aaS]
VIM-dependent Slicing [1]
[Resource Slice aaS]
(R) Orchestration
Service-based Slicing [3]
[Service Slice aaS]
Network Service Orchestration
MANO-based Slicing [2]
[NFV aaS]
Slicing
S
Vertical
S
Service
iS
Scenarios: Network Slicing
Source: NECOS (Novel Enablers for Cloud Slicing). http://www.h2020-necos.eu/
co-funded by the European Union (H2020-777067) and the Rede Nacional de Ensino e Pesquisa under the EU-Brazil Joint Call EUB-01-2017

25. ……
PROVIDER TENANTS
Internal
Slices
External /
Provider-
managed
Slices
External /
Tenant-
managed
Slices
Infrastructure
Source: A Network Service Provider Perspective on Network Slicing. Luis M. Contreras and Diego R. López. IEEE Softwarization, January 2018
Orchestration
under diferent types of slices and control responsibilities

26. • Software-defined infrastructures embracing Network Service Orchestration
as strategic elements of the evolving networking landscape.
• NSO aims at converging various technologies by providing a broader and
comprehensible view of network services
• Single and Multi-domain
• Many open challenges and issues....
26
Conclusions

27. http://www.intrig.dca.fee.unicamp.br
Thanks! Gracias! Obrigado!
Acknowledgments:
This work was partially supported by the Innovation Center, Ericsson
S.A., Brazil, grant UNI.62, and by the European Union's Horizon 2020
grant agreement no. 777067 (NECOS - Novel Enablers for Cloud Slicing),
as well as from the Brazilian Ministry of Science, Technology, Innovation,
and Communication (MCTIC) through RNP and CTIC and NECOS
Christian Esteve Rothenberg
chesteve@dca.fee.unicamp.br

28. Christian Esteve Rothenberg
chesteve@dca.fee.unicamp.br
Obrigado

30. But, wait…., what is a Slice?

31. ……
PROVIDER TENANTS
Internal
Slices
External /
Provider-
managed
Slices
External /
Tenant-
managed
Slices
Infrastructure
Tenant
monitoring the
slice and the
services
Provider
monitoring the
slice and
tenants
monitoring the
services
Provider
monitoring the
slice and the
services
Provider
monitoring the
infra
MONITORING
Source: A Network Service Provider Perspective on Network Slicing. Luis M. Contreras and Diego R. López.
IEEE Softwarization, January 2018
Types of slices and control responsibilities

32. Net App
Net App
Net App
Net App
Net App
Net App
Network infrastructure
Southbound Interface
Network Operating System
Northbound Interface
Language-based Virtualization
Programming languages
Network Applications
Debugging,Testing&Simulation
Network Operating
System and
Hypervisors
Network Applications
Routing
Access
Control
Load
balancer
Control plane
Data plane
Management plane
Network Hypervisor
Título da Palestra

33. Net App
Net App
NFs
Net App
Net App
L7 Apps
Network
Resources
NIM
Slicing
Application Services
Vertical
Use
Case i
Control & Management plane
Infrastructure
Business (Application & Service) plane
Slicing
Compute
Resources
VIM
Slicing
MonitoringMonitoringMonitoring
VIM-independent Slicing [0]
(“Bare-metal”/ à la IaaS)
[Infra Slice aaS ?]
VIM-dependent Slicing [1]
[Platform Slice aaS ?]
(R) Orchestration
Service-based Slicing [3]
[Service Slice aaS ?]
Network Service Orchestration
MANO-based Slicing [2]
[NFV aaS ?]
Slicing
S
Vertical
S
Service
iS
Título da Palestra

34. Scenarios::Intelligent Transport System 34
● Network service orchestration can
contribute largely in Intelligent
Transport System.
● All components and network need
to operate for offerrig integrated
services and fine-configurations.
Another issue is the dynamism of
the traffic with big amount of data.
● NSO can handle a big amount of
data, contexts, and interfaces
under an automatic and agile way.

35. Scenarios::Internet of Things
● IoT will lead to scalability and management issues in the
process of transport, processing, and storage of the data in
real time
● NSO along with NFV and SDN can help to process and
manage significant amounts of IoT-generated data with
better network efficiency.
● Besides, they reduce the human intervention in the operation
of the network, feature that is essential to the achievement
of Internet of Things.
35

36. Challenges::Scalability
● 5G network might connect 50 billion devices until 2020.
● Orchestration process requires the ability to handle the
growth of networks and services to support the huge amount
of connected nodes over a heterogeneous infrastructure.
● This environment demands high scalability of the
components involved, including orchestrators, controllers,
and managers.
● A key challenge is therefore to develop an orchestration
process that is massively scalable.
36

37. • NSO features:
• High-level vision of the NS
• Smart services deployment and provisioning
• Single and multi-domain environment support
• Proper interaction with different MANO and non-MANO elements
• New markets opportunities
37
NSO::Practical Definition

38. • Orchestrator: automatic resource coordination and control, as well as
service provision to customers
• ETSI NFV-MANO defines the orchestrator with two main functions:
• resources orchestration across multiple VIMs
• network service orchestration*
• Different orchestrators can exist in each plane, not being limited to a
single orchestrator
• Tackle scalability issues
38
NSO::Functions

39. Challenges::Security and Resiliency
● Security and resiliency must be considered both in design and
operation stages of network services.
● Service instantiation involves automated processes that add
and delete network elements. A critical problem is the
addition of a malicious node that can perform attacks.
● Multi-domain orchestration hide specific details of each
domain that ensures privacy and confidentiality.
● Resilience in main NSO components is also a critical problem
because it can impact directly in overall service operation.
39

40. Challenges::Resource/Service Model
● Network services need to be efficiently modeled towards
deploying resource requirements, configuration parameters,
management policies, and performance metrics.
● It is a challenge to translate higher-level policies into a lower
level configuration.
● There are templates and data modeling languages:
○ TOSCA, YANG, HOT, Unify
● Currently, there is no a standard resource and service
modeling.
40

41. Challenges::Performance/Assurance
● Orchestration technology -> virtualized and software-based
infrastructure.
● Performance and Service Assurance are constant challenges
in a highly dynamic environment (performance monitoring
coupled with network services maintenance)
● To keep NS performance, it is demanded that the system
equally performs in different layers
○ Multi-domain scenarios: exchange of information and resources between
different organizations/domains
● Projects within 5G-PPP are targeting enhanced performance
41

42. Challenges::Interoperability
● Challenge: create and to manage services across unique and
proprietary interfaces, making integration and startup.
● Interoperability is essential to enable the deployment of end-
to-end network services.
● There is no consensus about how would be the exchanging
process in multi-domain environment.
● Standardization is a path to enable interoperability of
network services.
42

43. Challenges::NS Lifecycle Management
● Network Service Lifecycle Management is fundamental to
ensure the correct operation of the service
● Service lifecycle automation can be obtained through
heuristic algorithms and machine learning techniques.
● ONAP:
○ Use closed control loop to provide automation, performance and lifecycle
management
○ BPMN/TOSCA workflow to meet the needs of NSO-based lifecycle
automation
43

44. Network
Service
Orchestratio
n in
Multiple
Administrati
ve Domains

45. What is an administrative domain?
● RFC 1136: “...A group of hosts, routers, and networks operated
and managed by a single organization...”
○ i.e., Amazon EC2, AT&T, Comcast….
● Where are the organization boundaries?
○ There are devices in multiple administrative domains
○ i.e., Your home, an industry, a farm, IoT gadgets, mobile-phone/tablet

46. Who owns the network?
● Cabling and black/white-boxes spread in regions
○ Big content-providers become infrastructure providers (e.g., amazon, google,
facebook)
○ Getting closer to the edge
● There are services in different layers/levels,
○ i.e., Mobile Virtual Network Operators
● Walking into end-to-end programmable paths
○ In business: network slices

47. What if (in SDN/NFV words)…
● … programmable paths have predictable performance?
● … there is a reliable manner to audit such paths?
● … such paths are established from radio to core (end-to-end)?
● … heterogeneous business needs fit into shared network
infrastructures?

48. Is routing enough?
● BGP developed for inter-domain routing
○ Overloaded with many added features/patches
○ Management complexity (i.e., path inflation, route leaks)
○ Limited set of mechanisms
● More and more business models coming to networks
○ Augmented reality
○ IoT
○ Vehicular-to-X

49. In the real world ahead
● ETSI NFV
○ MANO to MANO interactions
● MEF SD-WAN
○ Low cost end-to-end VPN-like on-demand
● 3GPP Network Slicing
○ Mobile Radio and Core sharing among multiple operators

50. At the same time ISPs want service
diversification
● ISPs also want to keep
○ Minimum information revealed: expose only the needed/abstracted internal
info
○ Efficiency: intra-domain traffic engineering
○ Fairness: competition with cooperation

51. Operational Challenges
● From routes to network function forwarding graphs (service
chains)
● Edge evolves enabling close-to-customer network
diversification
● Operators initiated to define and understand network analytics
● Dynamic network behaviors closer to be understood (control-
loop)
● However:
○ Avoid unpredictable shifts in network traffic volumes
○ Shared hot infrastructure vs. diverse tenant isolated traffic
○ From handshake agreements to explicit coordination

52. Goal: to define network services at high-level, relying on multiple
technologies and paradigm implementations.
• comprises the semantics of requested service.
• coordinates specific actions in order to fulfill the service requirements and to
manage end-to-end lifecycles.
• Involves business and operations that go beyond the delivery of network services
(as defined by ETSI).
52
Towards a Practical Definition of NSO