Automation, Agility and NFV
Responding to the OTT/web-scale threat
James Crawshaw
Senior Analyst
Heavy Reading
November 2016 1
November 2016 2
Agenda
1. Introduction
2. Automation
3. Agility
4. NFV – opportunities & challenges
5. NFV - standards vs open source
6. Web giant approach to virtualization
7. Conclusion
November 2016 3
1. Introduction
Automation
Agility
NFV
November 2016 4
AUTOMATION
Section 2
November 2016 5
Automation across the service lifecycle
Stage Automation opportunity
Order & fulfillment Self service for consumers already prevalent. Needs more flexibility to cater to
enterprise customers.
Configuration and
control
Allow the customer to take control of network capacity and QoS, on-demand
Security Filter malicious traffic without requiring human intervention
Policy Adjust network resources (bandwidth, traffic priorities) to provide differentiated
services in the face of ever-changing network conditions
Assurance Error detection and fault reporting; reroute services to limit disruptions
Performance Quality monitoring and capacity analysis
Analytics Real-time picture of end-to-end services, network components, and infrastructure
November 2016 6
Network Automation Maturity
Source: Heavy Reading
1
5
4
3
2
November 2016 7
Operator Automation Comfort Levels
November 2016 8
Machine learning and AI
• Virtualized networks may require AI to sift through billions of
different network scenarios to find an optimal configuration.
• AI is already being experimented within data center
operations.
• Telcos use AI for network planning and optimisation.
• According to a 2007 paper by Qi et al, expert systems and ML
are the AI techniques that have been widely used in telecoms
so far.
https://www.researchgate.net/publication/220658193_Artificial_intelligence_applications_in_the_telecommunications_industry
November 2016 9
AGILITY
Section 3
November 2016 10
Agile
Source: Netcracker
November 2016 11
DevOps approach
Metamodel - enables
services/resources to be
described in a common,
abstract way
Modelling launguages / tools:
TOSCA, YANG, YAML, Chef,
Puppet, etc
Scripts, service models, resource descriptors,
policies) – proprietary, third party, open source
November 2016 12
Cultural challenge of DevOps
Plan RunBuild
From NetOps and SysOps to DevOps
November 2016 13
NFV - OPPORTUNITIES & CHALLENGES
Section 4
November 2016 14
NFV opportunities and challenges
Opportunity Challenge
Offer new services (eg vFW, NaaS, VR) created internally or
outside
Ensure services are not simply provided OTT
Enable customer self-service – empower customer to make
changes in near real time
Integration with existing OSS/BSS
Network automation - adaptive, self-healing networks with
closed loop assurance
• Need for greater standardisation of alarm and
performance management.
• Shift from hardware monitoring to service monitoring.
Integration of network and cloud (storage, compute) Silo mentality
Replace antiquated systems • Tech immaturity – “It's all full of bugs, hard to upgrade
and a nightmare to operate.”
• Operating a hybrid (legacy + NFV) network for years to
come.
November 2016 15
Barriers to NFV
2.73
2.73
2.73
2.74
2.8
2.84
2.86
2.91
3.13
2.5 2.6 2.7 2.8 2.9 3 3.1 3.2
IT/network convergence issues
Lack of evidence for new virtualization-enabled, revenue-
generating services
Length of time required to achieve
RoI from virtualization
Need to manage virtual and
physical networks in parallel
Complexity of virtualizing
high-scale mobile networks
Lack of NFV MANO standards
Performance gap between
COTS and dedicated hardware
Immaturity of VNFs
Integration with existing OSS
Source: Heavy Reading OSS and MANO survey 2015
November 2016 16
ATIS - Operational Challenges of NFV
• Greatest challenge lies in the upper control and management layers of the
network, where new and legacy systems must co-exist
• Fragmented domain specific control and management must be migrated
into an integrated whole to achieve seamless automation
• A comprehensive standardization program for NFV is not required.
• Information models developed byTM Forum will likely be retained
• Specifications drive interoperability for hardware
• For software it is the availability of open source implementations
https://access.atis.org/apps/group_public/download.php/20398/Operational Opportunities.pdf
November 2016 17
NFV: STANDARDSVS OPEN SOURCE
Section 5
November 2016 18
SDOs, Consortia and Open Source
Source: Heavy Reading
Legend – blue = open source, red = SDO or consortium
Interoperability
ATIS
MEF
OPNFV
OSS
OPNFV, TMForum
OpenNMS, Drools
NFVO - Orchestration
Open-O, Open Mano, ECOMP
VNFM - Control
ONOS, OpenDaylight, ONF
VIM - Infrastructure
3GPP
OpenStack, OpenFlow, Cloudstack
Architecture
ETSI
November 2016 19
Open Source advantage
Source: Linux Foundation
November 2016 20
Key SDO activity for NFV
• ETSI NFV ISG – 11 specs including architectural framework, infrastructure, service
quality metrics, management & orchestration, resiliency requirements, and security
guidance.
• IETF/IRTF - NetworkVirtualization OverlaysWG works on signalling for tunnelling
protocol; Service Function Chaining WG; Source Packet Routing in NetworkingWG
is about how specific data packets should be routed. NETCONF is a protocol for
network device configuration and YANG is an associated data modelling language.
• TM Forum – ZOOM (Zero-touch Orchestration, Operations and Management) – to
developVirtualization and NFV & SDN best practices and standards
• MEF - LSO (Lifecycle Service Orchestration) specifies service orchestration based on
lifecycles of connectivity services across network domains. Principle capabilities
cover Fulfillment, Control, Performance, Assurance, Usage and Analytics.
November 2016 21
Key consortia activity for NFV
• Open Networking Foundation (ONF) – inherited the OpenFlow
protocol/API (used asVIM) from Stanford University. Led by
Facebook, DT, AT&T, Microsoft, Google, NTT,Verizon.
• Open Data Center Alliance (ODCA) – dedicated to cloud
computing. Led by CenturyLink, Intel, Capgemini, China
Unicom, SAP, DT, Infosys.
• ATIS - Network FunctionsVirtualization Forum (inter-CSP and
enterprise-to-CSP interconnection, interoperability and
interworking).Very large Board of Directors.
November 2016 22
Key open source NFV projects
• OPNFV (Open Platform for NFV) – a framework for software development
covering NFV Infrastructure,Virtualized Infrastructure Management, and APIs.
• OpenStack – Nova (compute) provisions/manages large networks ofVMs; Neutron
(networking) provides pluggable, scalable, API-driven network and IP management;
Swift provides object storage and Cinder provides block storage.
• OpenDaylight – Around 30 projects including (1) network applications,
orchestration and services; (2) controller platform; (3) southbound interfaces &
protocols such as netconf or OpenFlow plugins; (4) data plane elements such as
OpenFlow-enabled devices or Open vSwitches. Supported by the Linux Foundation.
• ONOS – Open Network Operating System is an open source community hosted by
the Linux Foundation which aims to create a SDN operating system for
communications service providers.
November 2016 23
Open warfare: may the best MANO win
November 2016 24
Open Source MANO ECOMP OPEN-O
Hosted by ETSI NFV AT&T / Linux Foundation Linux Foundation
Operator support Bell Mobility, BT, Portugal
Telecom, Sprint, Telefonica,
Telenor
AT&T, Orange Business
Services
China Mobile, China Telecom
Key vendor members ADVA Optical, Canonical,
Comptel, Intel, Ixia, Mirantis,
Procera, Red Hat, RIFT.io,
Sandvine, Technicolor, Viavi
Ericsson, GigaSpaces, Huawei,
Intel, ZTE
Other vendor participants Brocade, Netcracker,
Netrounds, Radcom, VMWare
Amdocs, Radcom Canonical, Red Hat
Lines of code 0.3m 8.5m 2.1m
Commits 411 n/a 2,674
Contributors 49 n/a 86
Downloads 803 n/a n/a
Source: Heavy Reading.
THE WEB GIANT APPROACH
Section 6
November 2016 25
Busting some FANG myths
• Web-scale giants use custom-built or heavily customized appliances – not
just commodity hardware.
• This allows them to reduce their overall cost and/or meet specific
performance requirements.
• They also build applications that are heavily coupled and specifically
designed for their hardware/infrastructure environment – this is the
opposite of virtualization.
• This allows software designers to squeeze the most performance from the
custom hardware at their disposal.
November 2016 26
Facebook
• "Building 8" is a FB team developing connectivity hardware products.
• By developing technology and releasing it as open source (Open Compute,
Telecom Infra Project) FB aims to help service providers provide better
connectivity for its users.
– Open Cellular - wireless access platform.
– Terragraph - networks of nodes that combine WiFi and small cells
– ARIES Massive MIMO radio antenna for rural areas.
• Network automation: SDN makes more of underlying architecture of
network equipment available so programmers can control the hardware.
Further info: https://code.facebook.com/posts/ https://www.facebook.com/Engineering
November 2016 27
Amazon
• Amazon uses:
– Custom processors
– Many hardware configurations, custom routers, protocol stacks, and
even proprietary power supplies
– Custom switches - “If you build it yourself with minimalist attitudes and
only with the features you need, it is a lot cheaper”
• Further info:
– AWS blog - https://aws.amazon.com/blogs/aws/
– AWS architecture blog - https://www.awsarchitectureblog.com
– CTO blog - http://www.allthingsdistributed.com/
November 2016 28
Netflix
• Netflix innovates on top ofAWS and EC2 infrastructure with
their own management tools:
– Meson project for “workflow orchestration and scheduling framework “
– Titus – Docker management
– Mantis - real-time dash-boarding, alerting, anomaly detection, and
metric generation.
– Chaos Monkey - tool that randomly disables production instances to
make sure can survive this common failure without customer impact.
Further info: http://techblog.netflix.com/
November 2016 29
Google
• Google has been at the forefront of SDN, first implementing Open Flow in 2012. Andromeda is
Google’s SDN architecture with load-balancing, security and firewall services built on top
which it offers to Compute Engine customers on-demand.
• In a paper published last year Google pulled back the curtain on its SDN infrastructure.
• Google doesn’t just use commodity hardware, it constructs its own servers and custom
networking switches. Google’s Spanner is built with a specific requirement on hardware-level
synchronized clocks. Kubernetes assumes that every compute node is given its own subnet.
Resource allocation in Google Borg is tied to Google’s capacity planning, while cluster
definitions in Borg rely on Google’s network topologies, among other factors.
• Google Research is actively involved in exploring data center architecture, wide area network
interconnects, SDN control and management infrastructure, as well as congestion control and
bandwidth allocation.
November 2016 30
CONCLUSION
Section 7
November 2016 31
The only constant is change
Heraclitus
535-475 BC
Digitalisation
Packet switching
Virtualization
?
Intelligent network
November 2016 32

Automation, Agility and NFV

  • 1.
    Automation, Agility andNFV Responding to the OTT/web-scale threat James Crawshaw Senior Analyst Heavy Reading November 2016 1
  • 2.
  • 3.
    Agenda 1. Introduction 2. Automation 3.Agility 4. NFV – opportunities & challenges 5. NFV - standards vs open source 6. Web giant approach to virtualization 7. Conclusion November 2016 3
  • 4.
  • 5.
  • 6.
    Automation across theservice lifecycle Stage Automation opportunity Order & fulfillment Self service for consumers already prevalent. Needs more flexibility to cater to enterprise customers. Configuration and control Allow the customer to take control of network capacity and QoS, on-demand Security Filter malicious traffic without requiring human intervention Policy Adjust network resources (bandwidth, traffic priorities) to provide differentiated services in the face of ever-changing network conditions Assurance Error detection and fault reporting; reroute services to limit disruptions Performance Quality monitoring and capacity analysis Analytics Real-time picture of end-to-end services, network components, and infrastructure November 2016 6
  • 7.
    Network Automation Maturity Source:Heavy Reading 1 5 4 3 2 November 2016 7
  • 8.
    Operator Automation ComfortLevels November 2016 8
  • 9.
    Machine learning andAI • Virtualized networks may require AI to sift through billions of different network scenarios to find an optimal configuration. • AI is already being experimented within data center operations. • Telcos use AI for network planning and optimisation. • According to a 2007 paper by Qi et al, expert systems and ML are the AI techniques that have been widely used in telecoms so far. https://www.researchgate.net/publication/220658193_Artificial_intelligence_applications_in_the_telecommunications_industry November 2016 9
  • 10.
  • 11.
  • 12.
    DevOps approach Metamodel -enables services/resources to be described in a common, abstract way Modelling launguages / tools: TOSCA, YANG, YAML, Chef, Puppet, etc Scripts, service models, resource descriptors, policies) – proprietary, third party, open source November 2016 12
  • 13.
    Cultural challenge ofDevOps Plan RunBuild From NetOps and SysOps to DevOps November 2016 13
  • 14.
    NFV - OPPORTUNITIES& CHALLENGES Section 4 November 2016 14
  • 15.
    NFV opportunities andchallenges Opportunity Challenge Offer new services (eg vFW, NaaS, VR) created internally or outside Ensure services are not simply provided OTT Enable customer self-service – empower customer to make changes in near real time Integration with existing OSS/BSS Network automation - adaptive, self-healing networks with closed loop assurance • Need for greater standardisation of alarm and performance management. • Shift from hardware monitoring to service monitoring. Integration of network and cloud (storage, compute) Silo mentality Replace antiquated systems • Tech immaturity – “It's all full of bugs, hard to upgrade and a nightmare to operate.” • Operating a hybrid (legacy + NFV) network for years to come. November 2016 15
  • 16.
    Barriers to NFV 2.73 2.73 2.73 2.74 2.8 2.84 2.86 2.91 3.13 2.52.6 2.7 2.8 2.9 3 3.1 3.2 IT/network convergence issues Lack of evidence for new virtualization-enabled, revenue- generating services Length of time required to achieve RoI from virtualization Need to manage virtual and physical networks in parallel Complexity of virtualizing high-scale mobile networks Lack of NFV MANO standards Performance gap between COTS and dedicated hardware Immaturity of VNFs Integration with existing OSS Source: Heavy Reading OSS and MANO survey 2015 November 2016 16
  • 17.
    ATIS - OperationalChallenges of NFV • Greatest challenge lies in the upper control and management layers of the network, where new and legacy systems must co-exist • Fragmented domain specific control and management must be migrated into an integrated whole to achieve seamless automation • A comprehensive standardization program for NFV is not required. • Information models developed byTM Forum will likely be retained • Specifications drive interoperability for hardware • For software it is the availability of open source implementations https://access.atis.org/apps/group_public/download.php/20398/Operational Opportunities.pdf November 2016 17
  • 18.
    NFV: STANDARDSVS OPENSOURCE Section 5 November 2016 18
  • 19.
    SDOs, Consortia andOpen Source Source: Heavy Reading Legend – blue = open source, red = SDO or consortium Interoperability ATIS MEF OPNFV OSS OPNFV, TMForum OpenNMS, Drools NFVO - Orchestration Open-O, Open Mano, ECOMP VNFM - Control ONOS, OpenDaylight, ONF VIM - Infrastructure 3GPP OpenStack, OpenFlow, Cloudstack Architecture ETSI November 2016 19
  • 20.
    Open Source advantage Source:Linux Foundation November 2016 20
  • 21.
    Key SDO activityfor NFV • ETSI NFV ISG – 11 specs including architectural framework, infrastructure, service quality metrics, management & orchestration, resiliency requirements, and security guidance. • IETF/IRTF - NetworkVirtualization OverlaysWG works on signalling for tunnelling protocol; Service Function Chaining WG; Source Packet Routing in NetworkingWG is about how specific data packets should be routed. NETCONF is a protocol for network device configuration and YANG is an associated data modelling language. • TM Forum – ZOOM (Zero-touch Orchestration, Operations and Management) – to developVirtualization and NFV & SDN best practices and standards • MEF - LSO (Lifecycle Service Orchestration) specifies service orchestration based on lifecycles of connectivity services across network domains. Principle capabilities cover Fulfillment, Control, Performance, Assurance, Usage and Analytics. November 2016 21
  • 22.
    Key consortia activityfor NFV • Open Networking Foundation (ONF) – inherited the OpenFlow protocol/API (used asVIM) from Stanford University. Led by Facebook, DT, AT&T, Microsoft, Google, NTT,Verizon. • Open Data Center Alliance (ODCA) – dedicated to cloud computing. Led by CenturyLink, Intel, Capgemini, China Unicom, SAP, DT, Infosys. • ATIS - Network FunctionsVirtualization Forum (inter-CSP and enterprise-to-CSP interconnection, interoperability and interworking).Very large Board of Directors. November 2016 22
  • 23.
    Key open sourceNFV projects • OPNFV (Open Platform for NFV) – a framework for software development covering NFV Infrastructure,Virtualized Infrastructure Management, and APIs. • OpenStack – Nova (compute) provisions/manages large networks ofVMs; Neutron (networking) provides pluggable, scalable, API-driven network and IP management; Swift provides object storage and Cinder provides block storage. • OpenDaylight – Around 30 projects including (1) network applications, orchestration and services; (2) controller platform; (3) southbound interfaces & protocols such as netconf or OpenFlow plugins; (4) data plane elements such as OpenFlow-enabled devices or Open vSwitches. Supported by the Linux Foundation. • ONOS – Open Network Operating System is an open source community hosted by the Linux Foundation which aims to create a SDN operating system for communications service providers. November 2016 23
  • 24.
    Open warfare: maythe best MANO win November 2016 24 Open Source MANO ECOMP OPEN-O Hosted by ETSI NFV AT&T / Linux Foundation Linux Foundation Operator support Bell Mobility, BT, Portugal Telecom, Sprint, Telefonica, Telenor AT&T, Orange Business Services China Mobile, China Telecom Key vendor members ADVA Optical, Canonical, Comptel, Intel, Ixia, Mirantis, Procera, Red Hat, RIFT.io, Sandvine, Technicolor, Viavi Ericsson, GigaSpaces, Huawei, Intel, ZTE Other vendor participants Brocade, Netcracker, Netrounds, Radcom, VMWare Amdocs, Radcom Canonical, Red Hat Lines of code 0.3m 8.5m 2.1m Commits 411 n/a 2,674 Contributors 49 n/a 86 Downloads 803 n/a n/a Source: Heavy Reading.
  • 25.
    THE WEB GIANTAPPROACH Section 6 November 2016 25
  • 26.
    Busting some FANGmyths • Web-scale giants use custom-built or heavily customized appliances – not just commodity hardware. • This allows them to reduce their overall cost and/or meet specific performance requirements. • They also build applications that are heavily coupled and specifically designed for their hardware/infrastructure environment – this is the opposite of virtualization. • This allows software designers to squeeze the most performance from the custom hardware at their disposal. November 2016 26
  • 27.
    Facebook • "Building 8"is a FB team developing connectivity hardware products. • By developing technology and releasing it as open source (Open Compute, Telecom Infra Project) FB aims to help service providers provide better connectivity for its users. – Open Cellular - wireless access platform. – Terragraph - networks of nodes that combine WiFi and small cells – ARIES Massive MIMO radio antenna for rural areas. • Network automation: SDN makes more of underlying architecture of network equipment available so programmers can control the hardware. Further info: https://code.facebook.com/posts/ https://www.facebook.com/Engineering November 2016 27
  • 28.
    Amazon • Amazon uses: –Custom processors – Many hardware configurations, custom routers, protocol stacks, and even proprietary power supplies – Custom switches - “If you build it yourself with minimalist attitudes and only with the features you need, it is a lot cheaper” • Further info: – AWS blog - https://aws.amazon.com/blogs/aws/ – AWS architecture blog - https://www.awsarchitectureblog.com – CTO blog - http://www.allthingsdistributed.com/ November 2016 28
  • 29.
    Netflix • Netflix innovateson top ofAWS and EC2 infrastructure with their own management tools: – Meson project for “workflow orchestration and scheduling framework “ – Titus – Docker management – Mantis - real-time dash-boarding, alerting, anomaly detection, and metric generation. – Chaos Monkey - tool that randomly disables production instances to make sure can survive this common failure without customer impact. Further info: http://techblog.netflix.com/ November 2016 29
  • 30.
    Google • Google hasbeen at the forefront of SDN, first implementing Open Flow in 2012. Andromeda is Google’s SDN architecture with load-balancing, security and firewall services built on top which it offers to Compute Engine customers on-demand. • In a paper published last year Google pulled back the curtain on its SDN infrastructure. • Google doesn’t just use commodity hardware, it constructs its own servers and custom networking switches. Google’s Spanner is built with a specific requirement on hardware-level synchronized clocks. Kubernetes assumes that every compute node is given its own subnet. Resource allocation in Google Borg is tied to Google’s capacity planning, while cluster definitions in Borg rely on Google’s network topologies, among other factors. • Google Research is actively involved in exploring data center architecture, wide area network interconnects, SDN control and management infrastructure, as well as congestion control and bandwidth allocation. November 2016 30
  • 31.
  • 32.
    The only constantis change Heraclitus 535-475 BC Digitalisation Packet switching Virtualization ? Intelligent network November 2016 32