Software Defined Network (SDN) using ASR9000 :: BRKSPG-2722 | San Diego 2015

Deploying SDN on ASR
9000
Joel Roberts, Technical Leader, Advanced Services
joerober@cisco.com
Rob Piasecki, Solutions Architect, Advanced Services
rpiasecki@cisco.com
BRKSPG-2722

• SDN Drivers, Concepts, & Definitions
• ASR9000 SDN Enabled Platform
• BGP-LS
• Stateful PCEP
• OpenFlow
• NETCONF/YANG
• Summary
Agenda

Drivers, Concepts, &
Definitions

There Are Many Reasons to Change
Industry Realignment
Significant Traffic Growth
0.0
20.0
40.0
60.0
80.0
100.0
120.0
140.0
2012 2013 2014 2015 2016 2017
ExabytesperMonth
Web/Data (24.2%, 18.9%)
File Sharing (15.7%, 8.1%)
Managed IP Video (21.8%, 21.0%)
Internet Video (38.3%, 52.0%)
50 Billion
Connected Things
by 2020
Connected Things
Growing 5X
Faster than
Mobile Devices
More than 22% of all
networked events will be
Machine Driven by 2017
Emergence of Internet of Everything
Acquires 45%
of Verizon
Wireless from
+
+
+
+
+
+
Evolution & Intersections of Technology
Video
CloudMobility
M2M

There Are Many Key Challenges
Traffic Growth
Complexity
Competition
Total Cost Ownership
Speed of
Innovation

Video
M2M
Cloud
Mobility
EPN
Evolved Programmable Network (EPN)
NCS NCS
APIs
APIs
EDGE
CORE
UA
VM VM
Edge
Core
VM
Orchestration
VM/ Storage
Control
ServiceCatalogService
OrchestrationApps
VM
Application
CDN
AGILITY
OPTIMIZE
MONETIZE ¥£€$
Always“ON”
On-Demand
Services Anywhere
Dynamic
Scale
Application
Interaction
Seamless
Experience
Policy
Real-Time Analytics
Services
Orchestration
Fully
Virtualized
Intelligent
Convergence
Automated
Open and
Programmable
Unified
Access

SDN: Means Many Things to Many People
Practical Attributes:
• Software platform with (abstracted) view
of the network
• Northbound APIs for Application
interaction to/from network
• Southbound methods to collect and
program information from/to network
Optical
IP/MPLS
NB API
Centralized Platform with
Global View
Apps

Today’s Infrastructure: Many Silos, Complex…
Data
Center
Customers
Wide Area Network Cloud
Optical
IP/MPLS
Software Engineering (Applications)
System Engineering
(Compute, Storage, Virtual
Machines)
Network Engineering
(WAN Services, NfV)
Enterprise
Virtual

Evolved Programmable Network
Data
Center
Customers
Optical
IP/MPLS
System Engineering
Machines)
Network Engineering
(WAN Services, NfV)
Enterprise
Virtual
Physical and Virtual
API’s
Orchestration
API’s
Applications

SDN: Place In Network
Data
Center
Access/Customers
Optical
IP/MPLS
DC/Cloud
SDN
Enterprise
SDN
Applications DC-WAN X-Domain OrchEnt-SP WAN X-Domain Orch
SP SDN
(e.g WAN Orchestration)

SDN: Place In Network
Data
Center
Access/Customers Wide Area Network Cloud
Optical
IP/MPLS
DC/Cloud
SDN
Enterprise
SDN
Applications DC-WAN X-Domain OrchEnt-SP WAN X-Domain Orch
SP SDN
(e.g WAN Orchestration)
APIC
EM
APIC
DC

Many SDN Software Platform Options
“Buy” “Build/Develop”
Tail-F
NCS
Orchestration
NETCONFPCEPBGP-LSNETFLOWSNMP CLI
Multiple Methods to/from Physical & Virtual Devices

Scope
Applications
Orchestration
API’s + Protocols

Orchestration – Then and Now
OSS/BSS
WANCPE Data Center
CPE
Prov
WAN
Prov
DC
Prov
OSS/BSS
WANCPE Data Center
Orchestration
Platform
YANG
Service
Models
YANG
Device
Models
REST, Netconf, …
Customized BSS/OSS and
Provisioning Systems Model-driven Network and Service
Orchestration

Services Platform Evolution
Modular Framework for SDN and NFV
Customer-Facing
Services (Broker)
Service Provider OSS
Service Provider Existing Portal and Catalog
Portal/Service Catalog RT-OSS (Service Fulfillment, Inventory, Assurance)
Service
Orchestration
with Tail-f
Applications and
Controllers
DCN Compute and
Storage
Orchestration Engine
Netconf/
YANG
Access WAN
WAE
Network Services Orchestration with Tail-f Technology
OSCPCEP
Infrastructure
DCNCPE SP Network Compute and Storage
MERAKI
APIC
EM VTS APIC VNF-M

SDN Strategy for SPs – High Level View
• Model driven, end-to-end service
lifecycle and customer experience
focus
• Seamless integration with existing
and future OSS/BSS environment
• Loosely-coupled and modular
architecture leveraging open APIs
and standard protocols
• Orchestration across Multi-domain
& multi-layer provides centralized
policy and services across entire
network
Service-Intent API
Orchestration, Service & Policy Implementation
Branch/CPE
Control
EMS/NMS
DC & NFV
Control
Multi-layer WAN
SDN
CLI/
SNMP
SDN / APIs
Openstack /
vCenter
OpenflowPCEP
Segment
Routing
Netconf/
YANG
BGP
BSS
OSS (Fulfillment & Assurance)
Multi Vendor End to End Management & Orchestration
(Physical & Virtual)
Metro and Access WAN Data CentreCPE
WAE

WAN Automation Engine
Network Plan
WAN Automation Software Suite
Design and Network Planning
Network
Planning
Optimization Failure
Analysis
Visualization, Analytics, BI, Inventory
Weather Map
Business
Intelligence
Network
Inventory
Service, Network,
and Analytics
REST APIs
Optimization and Prediction
DeployerCollector
New ModelCurrent Model
CalendaringAnalytics
Collection Drivers
NetFlowCLISNMP BGP-LS ...PCEPOSC
NC/YA
NG

Orchestration
RESTful APIs
DeployerCollection
Use-Case: Bandwidth Scheduling (On-Demand)
WAN
R1
R2
R3
Data Center #1 Data Center #2
5
1
2 3
4
Congested!!
Challenge/Opportunity:
Customer has “on demand”
need for a DC backup
Network conditions reported
to collector (continuous)
1
Customer requests DC1 to
DC2 bandwidth NOW
2
Demand admission request:
<R1-R3, B/W, NOW>
3
Orchestration returns option
and customer confirms
4
R1-R3 LSP tunnel
programmed5

Orchestration
RESTful APIs
DeployerCollection
Use-Case: Bandwidth Calendaring
WAN
R1
R2
R3
5
1
2 3
4
Congested!!
Customer uses self-service
portal to request and schedule
bandwidth between DC’s
to collector (continuous)
1
DC2 bandwidth at future date
2
Demand admission request:
<R1-R3, B/W, Future Date>
3
Orchestration returns
confirmation of date/time
4
R1-R3 LSP tunnel
programmed5

Use Case: Coordinated Maintenance
Select
• Network Element
• ID circuits traversing
node
Evaluate
• Impact to global Network
• Network capacity to
reroute LSPs
Schedule
• Time changes prior to
outage
• Time normalization

ASR 9001 ASR9904 ASR 9006 ASR 9010 ASR9912 ASR 9922
Size 2RU 6RU 10RU 21RU 30RU 44RU
# of I/O Slots 2 MPAs 2 4 8 10 20
Air Flow Side to side Side to side Side to back Front to back Front to back Front to back
Fabric BW/slot N/A 770G/385G 440G/220G 440G/220G 770G/660G 770G/660G
Chassis
scalability/slot
120Gbps 2Tbps 880Gbps 880Gbps 2Tbps 2Tbps
Total Chassis BW 120Gbps 8Gbps 3.5 Tbps 7.0 Tbps 20Tbps 40Tbps
Cisco ASR 9000 Portfolio Evolution
ASR 9001-S
Virtual
XR DP
IOS XRv 9000

ASR9K SDN Enabled Platform & Strategy
Controller
OpenFlow
OpenFlow
Agent
Applications
Policy Servers OSS/BSS User AppAnalytics
Orchestration
Control Plane
Management
Plane
Controller
Controllers3
Program
Policies for
Optimized
Experience
BGP-LS
APIs and Protocols
OpenFlow Agent ,
BGP-LS, PCEP,
NETCONF
2
SDN Enabled Data
Plane
1
E-PBR Data Plane
Policy Fwd Flows
ASR 9000
SDN Platforms
Orchestration,
Analytics, Services, etc
4
VSM
Harvest
Network
Intelligence
Controller
PCEP
Applications5

Various Tools for Testing & Evaluation
OpenDaylight (ODL)
https://wiki.opendaylight.org/view/Getting_started
http://www.opendaylight.org/software/downloads
REST Client:
Chrome: Postman REST Client
Firefox: REST Client
Wireshark: https://www.wireshark.org/
IOS XR 5.2.2 with physical routers and
Virtual Internet Routing Lab (VIRL): virl.cisco.com
For Your
Reference

ASR9000 BGP Link-State
(BGP-LS)

Network Discovery & Data Collection
• Traditional
• Methods: SNMP, SNMPCollect (polling), Netflow, CLI Parsing
• Platforms: Network Management Systems (NMS)
• Are and will continue to be used…
• Some Challenges:
• Delay in topology changes to NMS
• E.g. delay based on polling intervals
• Unreliable transport
• e.g. missed SNMP traps
• Must Augment with (near) real-time information about the state of the network

Overview BGP Link-State (BGP-LS)
• Use BGP to advertise LSDB and TED of a
network
• New link-state address family
• Support for OSPF and ISIS LSDB
• Advantages
• Single upstream topology feed (BGP)
• IGP isolated from external entities
• Leverage well-known BGP security,
transport and policy knobs
• Enables operator control
• Support introduced in IOS XR 5.1.1
• Implementations: WAN Automation Engine
(WAE), OpenDaylight (ODL)
Domain 1 Domain 2
Domain 0
BGP-LS
BGP-LS BGP-LS
RR
PCE
TED
LSP DB

Reference Diagram: BGP-LS
OpenDaylight
192.168.96.0/29
.1
.2
192.168.x.x/31
64.4/31
64.0/31
192.4/31 192.6/31
128.2/31
128.0/31
192.0/31
AG04-4
10.99.8.8/32
AG04-2
10.99.2.2/32
AG06-3
10.99.5.5
AG06-1
10.99.3.3
AG06-2
10.99.4.4
AG06-4
10.99.6.6
AG04-1
10.99.1.1
172.18.152.3/24
For Your
Reference
BGP-LS
REST

router ospf 1
distribute bgp-ls instance-id 1
router-id 10.99.8.8
area 0
router bgp 65000
bgp router-id 10.99.8.8
address-family link-state link-state
!
neighbor 192.168.96.2
remote-as 65000
update-source Loopback0
address-family link-state link-state
!
!
ASR9000 BGP Link State Configuration
Distribute link state
database into BGP-LS
Specify BGP-LS peer
Enable address-family
link-state

BGP Link State Prefixes
• BGP-LS prefix string has the following general format
[NLRI-Type][Area][Protocol-ID][Local node descriptor][Remote node
descriptor][Attributes]/prefix-length
• Node descriptors and attributes consists of potentially multiple TLVs
• Node descriptors and attributes are shown as
[X[TLV1][TLV2]…]
• Where X identifies object (e.g. local node, remote node, link, etc.)
• TLVs are shown in the format
[yVALUE]
• Where y identifies field type (e.g. AS number, interface address, etc.)
For Your
Reference

RP/0/RSP0/CPU0:ASR9K-4#sh bgp link-state link-state
Tue May 6 00:09:45.523 UTC
BGP router identifier 10.99.8.8, local AS number 100
Status codes: s suppressed, d damped, h history, * valid, > best
i - internal, r RIB-failure, S stale, N Nexthop-discard
Origin codes: i - IGP, e - EGP, ? - incomplete
Prefix codes: E link, V node, T IP reacheable route, u/U unknown
I Identifier, N local node, R remote node, L link, P prefix
L1/L2 ISIS level-1/level-2, O OSPF, D direct, S static
a area-ID, l link-ID, t topology-ID, s ISO-ID,
c confed-ID/ASN, b bgp-identifier, r router-ID,
i if-address, n nbr-address, o OSPF Route-type, p IP-prefix
d designated router address
Network Next Hop Metric LocPrf Weight Path
*> [V][O][I0x1][N[c100][b10.99.8.8][a0.0.0.0][r10.99.1.1]]/376
0.0.0.0 0 i
*>
[E][O][I0x1][N[c100][b10.99.8.8][a0.0.0.0][r10.99.1.1]][R[c100][b10.99.8.8][a0.0.0
.0][r10.99.2.2]][L[i192.168.192.2][n192.168.192.3]]/792
0.0.0.0 0 i
ASR9000 BGP Link State Verification
Prefix codes
Node
Link

BGP-LS *OpenDaylight Sample Configuration
<type xmlns:prefix="urn:opendaylight:params:xml:ns:yang:controller:bgp:rib:impl">prefix:rib-impl</type>
<name>example-bgp-rib</name>
<rib-id>example-bgp-rib</rib-id>
<local-as>65000</local-as>
<bgp-id>192.168.96.2</bgp-id>
<bgp>
<type
xmlns:prefix="urn:opendaylight:params:xml:ns:yang:controller:bgp:listener">prefix:listener</type>
<name>example-bgp-peer</name>
<host>10.99.8.8</host>
</bgp>
* Approximately Lines 55 to 67 of <install-path>/etc/opendaylight/karaf/41-bgp-sample.xml
BGP-LS peer (ASR9000)
OpenDaylight Controller

RESTCONF* Sample OpenDaylight BGP LS
Topology
{ "topology": [
{ "link": [
{ "link-id": "bgpls://Ospf:1/type=link&local-as=100&local-
domain=174262280&local-area=0&local-router=174261509&remote-as=100&remote-
domain=174262280&remote-area=0&remote-router=174261766&ipv4-iface=192.168.128.0&ipv4-
neigh=192.168.128.1",
"l3-unicast-igp-topology:igp-link-attributes": {
"ospf-topology:ospf-link-attributes": {
"ospf-topology:ted": {} },
"l3-unicast-igp-topology:metric": 10 },
"destination": {
"dest-node":
"bgpls://Ospf:1/type=node&as=100&domain=174262280&area=0&router=174261766",
"dest-tp": "bgpls://Ospf:1/type=tp&ipv4=192.168.128.1”},
"source": {
"source-tp": "bgpls://Ospf:1/type=tp&ipv4=192.168.128.0",
"source-node":
"bgpls://Ospf:1/type=node&as=100&domain=174262280&area=0&router=174261509”}
* http://localhost:8080/restconf/operational/network-topology:network-topology/topology/example-linkstate-topology

BGP-LS Sample* Topology Visualization
*Visit DevNet for Sample BGP-LS Application

ASR9000 Path
Computation Element
Protocol

Path Computational Element (PCE)
• Path computation in large, multi-domain, and multi-layer networks
• Can reside inside router or on an external server platform
• Setup intra-, inter-area (or domain) and inter-layer paths
• Computed Path might be:
• Explicit route identifying a contiguous set of strict hops between the source and
destination
• Combination of strict/loose hops between the source and destination
Basic function is to compute paths (e.g. MPLS TE LSPs)
in MPLS/GMPLS Networks

PCE Definitions
• Traffic Engineering Database (TED)
• Contains topology and resource
information
• Inputs from IGP LSDB
• PCE Server (PCE)
• Path Computation Client (PCC)
• Agent on router(s) that interact with
PCE Server
• PCE Protocol (PCEP)
• Protocol that runs between PCC on
router and PCE server

• Stateless
• PCE has no knowledge of previously established paths
• Stateful
• Synchronization between PCCs and PCEs
• PCC maintains state synchronization with PCE
• PCC may delegate LSP control to PCE
• Either PCE or PCC can initiate LSP setup
• PCC always owns LSP state
Stateless and Stateful PCE
PCEP
Stateful PCE
TED
LSP DB
PCC
PCE-initiated
LSP

Stateful PCE
• LSP Database
• Contains info/status on active LSPs
communicated by PCCs in LSP state
report messages
• Active Stateful PCE
• References LSP DB for path
computations
• Programs LSP state in network
• PCC reports LSP status to PCE
• PCE may reprogram an existing LSP
• LSP Delegation
• PCC delegates LSP control responsibility
to PCE LSP=Label Switch Path

ASR9000 PCE-Initiated Tunnels
• Treated as dynamically created tunnels
(auto-tunnel)
• Router does NOT verify or compute path
that PCE provides
• treated as verbatim path
• PCE responsible for LSP re-optimization
• PCE sends an PCEP Update when a better
path exists
• Tunnels may be inter-area
• *PCE Initiated tunnel does not appear in
configuration
• “show mpls traffic-eng tunnels”
PCEP
Initiate /
Create
Stateful PCE
TED
LSP DB
PCC

Reference Diagram: PCE Initiated LSP
OpenDaylight
192.168.96.0/29
.1
.2
192.168.x.x/31
64.4/31
64.0/31
192.4/31 192.6/31
128.2/31
128.0/31
192.0/31
AG04-4
10.99.8.8/32
AG04-2
10.99.2.2/32
AG06-3
10.99.5.5
AG06-1
10.99.3.3
AG06-2
10.99.4.4
AG06-4
10.99.6.6
AG04-1
10.99.1.1
172.18.152.3/24
For Your
Reference
PCEP
REST

ASR9000 PCC Configuration
hostname ASR9K-AG04-4
!
ipv4 unnumbered mpls traffic-eng Loopback0
!
mpls traffic-eng
pce
peer ipv4 192.168.96.2
stateful-client
instantiation
delegation
!
auto-tunnel pcc
tunnel-id min 7000 max 9999
!
Allow PCE-initiated LSP
User defined
tunnel number
range.
Required for
auto-tunnel
PCE server
Delegate statically
configured tunnels

ASR9000 Verification PCE Peer
RP/0/RSP0/CPU0:ASR9K-4#show mpls traffic-eng pce peer
Tue May 6 03:21:17.216 UTC
Address State Learned From
--------------- ------------ --------------------
192.168.96.2 Up Static config

PCE Add-LSP (OpenDaylight) Sample
• REST URL: http://localhost:8080/restconf/operations/network-topology-pcep:add-lsp
• Method: POST Content-Type:application/xml
<input>
<node>pcc://10.99.8.8</node>
<name>test-tunnel</name>
<arguments>
<endpoints-obj><ipv4>
<source-ipv4-address>10.99.8.8</source-ipv4-address>
<destination-ipv4-address>10.99.6.6</destination-ipv4-address>
</ipv4></endpoints-obj>
</arguments>
<network-topology-ref xmlns:topo="urn:TBD:params:xml:ns:yang:network-
topology">/topo:network-topology/topo:topology[topo:topology-id="pcep-
topology"]</network-topology-ref>
</input>
PCC
Source
Destination

ASR9000 Verification PCE Tunnel
RP/0/RSP0/CPU0:ASR9K-AG04-4#show mpls traffic-eng pce tunnels
Mon May 19 17:36:20.237 UTC
Tunnel : tunnel-te7004
Destination : 10.99.6.6
State : down

PCE Update LSP (OpenDaylight) Sample
• URL: http://localhost:8080/restconf/operations/network-topology-pcep:update-lsp
<input>
<node>pcc://10.99.8.8</node> <name>test-tunnel</name>
<network-topology-ref xmlns:topo="urn:TBD:params:xml:ns:yang:network-topology">/topo:network-
topology/topo:topology[topo:topology-id="pcep-topology"]</network-topology-ref>
<arguments>
<operational xmlns:stateful02="urn:opendaylight:params:xml:ns:yang:pcep:crabbe:stateful:02">true</operational>
<ero>
<subobject><loose>false</loose><ip-prefix><ip-prefix>192.168.64.4/32</ip-prefix></ip-prefix></subobject>
</ero>
</arguments>
</input> Explicit-Route Object (ERO)

RP/0/RSP0/CPU0:ASR9K-AG04-4#show mpls traffic-eng pce tunnels
Mon May 19 17:39:38.550 UTC
Tunnel : tunnel-te7004
Destination : 10.99.6.6
State : up
Current path option : 10

PCE Remove LSP (OpenDaylight) Sample
• URL: http://localhost:8080/restconf/operations/network-topology-pcep:remove-lsp
<input>
<node>pcc://10.99.8.8</node>
<name>test-tunnel</name>
<network-topology-ref xmlns:topo="urn:TBD:params:xml:ns:yang:network-
topology">/topo:network-topology/topo:topology[topo:topology-id="pcep-
topology"]</network-topology-ref>
</input>

RP/0/RSP0/CPU0:ASR9K-AG04-4#sh mpls traffic-eng pce tunnels
Mon May 19 17:43:35.610 UTC
No PCE tunnels.

ASR9000 Show Commands
• New commands
• show mpls traffic-eng auto-tunnel pcc […]
• Modified Commands
• show mpls traffic-eng pce peer [ all|ipv4 <addr>|node-id <id>|stateful|stateless]
• Other useful commands
• show mpls traffic-eng pce tunnels <id>
• show mpls traffic-eng pce trace […]
• show mpls traffic-eng tunnels tabular

Segment Routing
• Simple routing extensions to implement source routing
• Packet path determined by prepended segment identifiers (one or more)
• Reduced network state
• Increased network scalability and virtualization
• Data-plane agnostic
• MPLS: label as segment id
• IPv6: address as segment id
• The state is no longer in the network but in the packet
• datatracker.ietf.org/wg/spring/documents/
*BRKRST-2124 Introduction to Segment Routing

Traffic Steering into PCE-Initiated Tunnels
Example approaches:
• Autoroute announce
• Policy-based tunnel selection (forwarding class id)

Policy Based Tunnel Selection
• Local mechanism at head-end
• PBR policy sets forwarding class for
incoming traffic
• Traffic switched to tunnel with matching
forwarding class
• Seven forwarding classes supported (1-
7)
• One forwarding class reserved as
default (0)
PE1
PE2
Tunnel-te1000
Forwarding class 1
Tunnel-te2000
Forwarding class 0
(default)
Input policy matches
traffic based on ACL
and sets forwarding
class

Traffic Steering PCE Initiated Tunnel
Tunnel-te1000
Forwarding
class 1
Tunnel-te2000
Forwarding
class 0 (default)
Tunnel-te7000
PCE Initiated LSP
Forwarding class 2
ASR9K-1
CE
TenGigE0/1/0/1
10.201.7.43
ASR9K-2
ASR9K-3
Set Forward ClassID 2

Orchestration
RESTful APIs
PCEPProgramming
Use-Case: PCE Initiated LSP
WAN
R1
R2
R3
4
3
Service Request
1
Analytic to determine LSP
2
PCCreate LSP
3
Traffic steering onto LSP
6
10.201.7.43
2
6
TE LSP Signaling
4
Delegate & LSP State Report
5
5

OpenFlow (OF)
• Open Network Foundation (ONF)
• Layer 2 Communication Protocol:
• Access to forwarding plane
• Switch or Router
• OF Controller: Control Plane
• OF Agent (device): Forwarding Plane

OpenFlow Agent/Plugin
• Implements standard OpenFlow switch
model.
• Speaks ‘standard’ OpenFlow protocol
• Native dedicated CLI for
troubleshooting
Availability:
Universal
User Experience:
Consistent
Deployment:
End-to-end
• Available* IOS-XR, IOS and
IOS-XE, NX-OS
• Common code, base features and CLI
across platforms
• Across SP, Data Center,
Campus…
* Please check roadmap for details on supported platforms & timelines

OpenFlow
Table:
• Consists of a set of flows (each row is a flow)
• Each flow contains a set of matches and actions.
• Applied only in ingress direction
Sends instructions…
OpenFlow Controller
Table
Match Action

OpenFlow Table
Match:
• Ingress port
• Priority: Highest priority flow entry
that matches packet gets selected
• Lowest priority is zero and highest is
32768
Action:
• Packet forwarding actions
• Packet modification actions
• Actions are immediately applied to
the packet

OpenFlow: ASR9000
• Support starting IOS XR 5.1.2
• OpenFlow Switch Version 1.3
• IOS XR Images
• asr9k-mini-px.vm
• asr9k-mpls-px.pie (required for L3VPN, L2VPN)
• asr9k-k9sec-px.pie (required for TLS)
• Default port TCP 6653
• Any physical interface for OF controller
OpenFlow Agent
OpenFlow Controller
Control Plane
Management
Plane
E-PBR Data Plane
Policy Fwd Flows
ASR9K

ASR9000: OpenFlow Hybrid Mode
• Recommended
• OpenFlow and XR forwarding
• Ability to take advantages of both XR features and OpenFlow flexibility
• OpenFlow as a policy based routing (PBR) type of feature.
• Can provide ‘fine-tuning’ on incoming traffic based on OpenFlow rule matching and rule actions

ASR9000 OpenFlow Layer 2 Table Options
OpenFlow Controller
L2 Only Table
Match- L2 Header Action: L2 Only
L2_L3 Table
Match: L2 & L3 Hdr Action: L2 Only
Supported L2 Matches & Actions are detailed in
ASR9000 System Management Configuration Guide, Release 5.3.x

ASR9000 OpenFlow Layer 3 Table Options
OpenFlow Controller
L3_V4 Table
Match: L3 IPv4 Hdr Action: L3 Only
L3_DS Table
Match:L3 v4 & v6 Action: L3 Only
Supported L3 Matches & Actions are detailed in

ASR9000 OpenFlow Tables
Layer Table Type Pipeline Supported
Interfaces
Description
Layer 2
L2 129 Bridge-domain, Gigabit ethernet, Bundle,
Bundle-subinterfaces, PWHE-
subinterfaces
• Supports L2 header matches
• Supports L2 actions.
• Can be applied to ingress L2 interface
L2_L3 130 L2Bridge-domain, Gigabit ethernet,
Bundle, Bundle-subinterfaces, PWHE-
subinterfaces
• Supports L2 & L3 (IPv4/IPv6) header
matches
• Supports L2 actions
• Can be applied ingress L2 interface
Layer 3
L3_V4 131 VRF and global interfaces, BVI (IPv4
only), Bridge-domain, Gigabit ethernet,
Bundle, Bundle-subinterfaces
• Supports L3 (IPv4) header matches
• Supports L3 (IPv4) actions
• Can be applied to ingress L3
interfaces
L3_DS 132 VRF and global interfaces, BVI (IPv4
only), Bridge-domain, Gigabit ethernet,
Bundle, Bundle-subinterfaces
• Supports L2 & L3 (IPv4/IPv6) header
matches
• Supports L3 (IPv4/IPv6) actions
• Can be applied to ingress L3
interfaces
For Your
Reference

OpenFlow Cisco Extension Actions
• Additional OpenFlow features on ASR9000
Cisco Added Actions
OpenFlow Switch Types supported on
ASR9000
Applied L2 Bridge
Domain
Applied L3 or L3 VRF
Interface
Action Description L2
Only
L2_L3 L3_V4 L3_DS
Set IPv4 Next Hop Set IPv4 Nexthop
Address
No No Yes Yes
Set IPv6 Next Hop Set IPv6 Nexthop
Address
No No No Yes
Set Forward Class ID Set Forward Class ID No No Yes Yes
Netflow Enable/Disable Enable/Disable Netflow No No Yes Yes

Openflow for Traffic Steering
• Use Openflow to program
classifiers/actions on WAN Edge
• Flow entries would be something
like:
• MATCH/Set_IPv4_Next_Hop
• MATCH/Set_IPv6_Next_Hop
• MATCH/Set_FCID (tunnel identifier)
• Useful for services and applications
requiring steering of specific flows
into a programmed WAN resource
Head-End
PCEP
Create LSP Tunnel
WAN
Edge Routers
Flow Entries
<MATCH/”Forward to
Tunnel”>
Tunnel
Traffic Steering
App
SDN WAN Orchestration
OF PCEP
OF
Flows

Orchestration
RESTful APIs
ProgrammingOpenFlow
Use-Case: OpenFlow for Traffic Steering
WAN
R1
R2
R3
3
3
Service Request
1 Analytic to determine LSP
2
Create LSP Tunnel with
FCID 2
3
Use OpenFlow to program
match/action on WAN edge
4
L3_DS Table
Match: SRC 10.201.7.43 Action: Set FCID=2
10.201.7.43
2
4

interface TenGigE0/0/0/3
l2transport
!
l2vpn
bridge group SDN-1
bridge-domain OF-1
!
openflow
switch 7 pipeline 129
tls trust-point local openfCA remote openfCA
bridge-group SDN-1 bridge-domain OF-1
controller ipv4 172.16.1.45 port 6653 security tls
!
!
ASR9000 OpenFlow L2 Switch Configuration
Openflow Switch attached to
bridge-domain
OpenFlow Controller
L2 Only Switch

l2transport
!
l2vpn
bridge group SDN-2
bridge-domain OF-2
!
openflow
tls trust-point local openfCA remote openfCA
bridge-group SDN-2 bridge-domain OF-2
!
!
ASR9000 OpenFlow L2 + L3 Switch Configuration
Openflow Switch attached to
bridge-domain
OpenFlow Controller
L2 + L3 Switch

openflow
interface Bundle-Ether2.1
interface GigabitEthernet0/1/0/7
interface GigabitEthernet0/0/0/0/4.1301
openflow
vrf ONE
ASR9000 OpenFlow L3_V4 Switch Configuration
All interfaces in VRF become
part of OpenFlow Switch
Specify L3
Interfaces
L3_V4 Switch
L3_V4 Switch

openflow
switch 100 netflow
flow monitor mmap sample smap
interface Bundle-Ether3
interface GigabitEthernet0/0/0/0/4.1201
ASR9000 OpenFlow Netflow Configuration
• Cisco extension to OpenFlow
• Used to enable/disable NETFLOW on Layer 3 Interfaces

Sample ASR9000 OpenFlow Show Commands
RP/0/RSP0/CPU0:ASR9K-AG06-4#show openflow switch 102 controller
Logical Switch Id: 102
Controller: 1
Address : 192.168.96.3:6653
Protocol : tcp
VRF : default
Local Trustpoint: : Not available
Remote Trustpoint: : Not available
Connected : Yes
Role : Master
last_error : Invalid argument
state : ACTIVE
sec_since_connect : 431579
RP/0/RSP0/CPU0:ASR9K-AG06-4#show tcp brief
PCB VRF-ID Recv-Q Send-Q Local Address Foreign Address State
0x501e3690 0x60000000 0 0 10.99.6.6:58633 192.168.96.3:6653
ESTAB
0x501c8720 0x60000000 0 0 10.99.6.6:38212 10.99.8.8:179
ESTAB

Sample ASR9000 OpenFlow Show Commands
RP/0/RSP0/CPU0:ASR9K-AG06-4#show openflow switch 102 flows
Total flows: 1
Flow: 1
Match: ipv6,ipv6_dst=192:1::2
Actions: output_nh(ipv6=2001:20::2)
Priority: 0
Table: 0
Cookie: 0x1
Duration: 196.353s
Number of packets: 0
Number of bytes: 0
RP/0/RSP0/CPU0:ASR9K-AG06-4#show openflow switch 102 ports
Tue Jan 27 14:07:21.025 UTC
Port Interface Name Config-State Link-State Features
1 Te0/1/0/1 PORT_UP LINK_UP 0

OpenDaylight Controller
Visit DevNet for OpenDaylight and Cisco Open SDN Controller

OpenFlow Show Commands
• show openflow switch <>
• show openflow switch <> controllers | stats
• show openflow switch <> ports
• show openflow switch stats
• show openflow switch flows | brief/summary
• show openflow interface switch <>
• show openflow hardware capabilities pipeline <>
• show table-cap table-type <>
For Your
Reference

OpenFlow Show Commands
Show policy-map commands:
• show policy-map transient list type pbr
• show policy-map transient type pbr pmap-name <>
• show policy-map transient targets summary
PBR platform show commands
• show pbr-pal ipolicy [<policy_name> | all ] location <loc>
• show pbr-pal ipolicy <policy_name> iclass [<iclass_handle> | all ] vmr-info location <loc>
• show pbr-pal ipolicy <policy_name> iclass [<iclass_handle> | all] stats [clear-on-read] location <loc>
• show prm server tcam entries <table> vmr-id <> 100 np0 loc <>
• show prm server tcam summary <table> PBR all loc <>
For Your
Reference

OpenFlow Debug
Debug for OpenFlow Agent:
• debug openflow switch ovs module ofproto level debug
• debug openflow switch ovs module ofproto-plif level debug
• debug openflow switch ovs module plif-onep level debug
• debug openflow switch ovs module plif-onep-util level debug
• debug openflow switch ovs module plif-onep-wt level debug
Debug for Policy Manager
• debug policymgr all
• debug policymgr trace
• debug policymgr lib all
• debug policymgr lib trace
• Debug for PBR: debug pbr-pal all loc
For Your
Reference

ASR9000 OpenFlow References
ASR9000 and OpenFlow Whitepaper:
https://supportforums.cisco.com/document/12402506/asr-9000-sdn-openflow-whitepaper
http://www.cisco.com/c/en/us/support/routers/asr-9000-series-aggregation-services-
routers/products-installation-and-configuration-guides-list.html
For Your
Reference

Traditional: Command Line Interface (CLI)
• Generally a majority of today’s
configuration
• Device Specific
• Human Friendly
• Manual (challenging to automate)
• Developer un-friendly
• Such parsing CLI screen scrape
• No Common Data Model
• No Built-In Error Reporting
[joerober@rtp-odl ~]$ ssh lab@10.99.1.1
lab@10.99.1.1's password:
RP/0/RSP0/CPU0:ASR9K-AG04-1#configure terminal
Tue Jan 6 17:18:58.350 EST
RP/0/RSP0/CPU0:ASR9K-AG04-1(config)#

NETCONF
• Network Configuration Protocol (NETCONF)
• XML based interface between network device and NMS
• Mechanism to manage, configure, and monitor network device
• Published RFC 4741 (Dec 2006)
• Updated RFC 6241 (Jun 2011)
• RFC 6242 (Jun 2011) Using the NETCONF Protocol over Secure Shell
• Overcome SNMP Limitations
• 2003 IAB Network Management Workshop (RFC3535)

YANG
• Modeling language defined in RFC 6020
• Used by NETCONF to define objects and data in requests and replies
• Models configuration, operational, and RPC data
• Provides semantics to better define NETCONF data
• Provides common data model:
• In order for NETCONF to be useful as network-wide protocol
• To consume NETCONF data from any network device
• YANG modules are for NETCONF what MIBs are for SNMP

Comparing SNMP and NETCONF
SNMP NETCONF
Data Models Defined in MIBs Defined in YANG modules (or
XML schema documents)
Data Modeling Language Structure of Management
Information (SMI)
YANG (and XML schema)
Management Operations SNMP NETCONF
RPC Encapsulation Basic Encoding Rules (BER) XML
Transport Protocol UDP TCP (reliable transport)

NETCONF Layers
• Content
• Configuration and Operational Data
• Operations
• Defines base operations
• What clients do to servers
• Messages/Remote Procedure Call (RPC)
• Transport-independent mechanism for encoding messages
• RPC, RPC-Reply
• Transport
• Reliable communication between client and server
Content
Operations
Messages
Transport

ASR9000 NETCONF Transport & Messages
• SSH NETCONF (1.1) session:
• IOS XR 5.3.1
• On client utilize ssh with –s to request invocation of subsystem on server (router)
• ssh lab@172.18.152.20 -p 830 -s netconf
Transport
Messages
Operations
Content
Client Server
<rpc>
<rpc-reply>

NETCONF Operations
Operation Description
<get-config> Retrieve all or part of specified configuration datastore
<edit-config> Loads all or part of a configuration to the specified
configuration datastore
<get-schema> Retrieve YANG-based XML Data
<commit> Copy candidate datastore to running datastore
<get> Retrieve running configuration and device state information
<lock> / <unlock> Lock or unlock the entire configuration datastore system
<close-session> Graceful termination of NETCONF session
<kill-session> Forced termination of NETCONF session
Complete List Protocol Operations: https://tools.ietf.org/html/rfc6241
Transport
Messages
Operations
Content

ASR9000 NETCONF Data Stores
• Target of NETCONF Operations
• Data stores are named containers that may hold an entire copy of the configuration
• IOS XR Supported Datastores:
• <running>
• <candidate>
• *XR two stage commit (no startup datastore)
Running Candidate
Transport
RPC
Operations
Content

YANG XR 5.3.1 Data Models
• YANG data modules are part of the software image.
• Models can be retrieved from router using <get-schema> operation.
• YANG Model file for each configuration module
• For example, to configure CDP, the relevant yang model is Cisco-IOS-XR-cdp-cfg
• 5.3.1 Data Models:
http://www.cisco.com/c/en/us/td/docs/routers/crs/software/crs_r5-
3/security/configuration/guide/b-syssec-cg53x-
crs/Implementing_the_Network_Configuration_Protocol.pdf#unique_8

ASR9000 NETCONF 1.1 Configuration (XR 5.3.1)
• Pre-requisite:
• k9sec pie must be installed
• Crypto keys must be generated
• Enable NETCONF Agent:
ssh server netconf port 830
netconf-yang agent ssh
• Session verification:
show netconf-yang clients
show netconf-yang statistics
! IOS XR 5.3.1
hostname ASR9K-R1
domain name cisco.com
!
ssh server v2
ssh server vrf default
ssh server netconf port 830
netconf-yang agent
ssh
!

XR 5.3.1 Sample NETCONF/YANG Workflow
ASR9000
Orchestration Platform
NETCONF Client
1 XR Image contains YANG
2 Client (application) retrieves
models with <get-schema>
3 Client installs and processes
YANG models
YANG
Data Model
1
YANG
Data Model
3
4 Client sends operation request
Using YANG-based XML data
5 Router understands YANG-based
XML data & is configured
accordingly
2 <get-schema>
<edit-config>4
Candidate Running5 6 Client-router interaction continues
Until network configured as desired

<edit-config>
<PBR>
<ServicePolicy>
<Input>PBTS</Input>
</ServicePolicy>
</PBR>
Orchestration
RESTful APIs
Programming
Use-Case:NETCONF Detailed Policy/Traffic Steering
WAN
R1
R2
R3
3
3
Service Request
1 Analytic to determine LSP
2
Create LSP Tunnel FCID 2
3
NETCONF to program:
-detailed policy*
-apply policy inbound
4
10.201.7.43
2
4
*example policy in reference slides

Starting SSH NETCONF Session
nms$ ssh lab@10.99.8.8 -p 830 –s netconf
lab@10.99.8.8's password:
<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<capabilities>
<capability>urn:ietf:params:netconf:base:1.1</capability>
<capability>urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring</capability>
<capability>urn:ietf:params:netconf:capability:candidate:1.0</capability>
<capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</capability>
<capability>urn:ietf:params:netconf:capability:validate:1.1</capability>
<capability>urn:ietf:params:netconf:capability:confirmed-commit:1.1</capability>
<!– yang capability output omitted -->
</capabilities>
<session-id>2110493906</session-id>
</hello>
]]>]]>

NETCONF Operation <hello>
• Used to exchange capabilities
• Initiated by the NETCONF Server
• Must be acknowledged by client before client can send any other messages
IOS XR (5.3.1):
<hello xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<capabilities>
<capability>urn:ietf:params:netconf:base:1.1</capability>
<capability>urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring</capability>
<capability>urn:ietf:params:netconf:capability:candidate:1.0</capability>
<capability>urn:ietf:params:netconf:capability:rollback-on-error:1.0</capability>
<capability>urn:ietf:params:netconf:capability:validate:1.1</capability>
<capability>urn:ietf:params:netconf:capability:confirmed-commit:1.1</capability>
<!– list of yang capability omitted from output -->
</capabilities>
<session-id>285212672</session-id>
</hello>
Transport
RPC
Operations
Content

NETCONF Operations - <edit-config>
• Modify configuration of particular data store
• Can only be used on writable data store
• Support on IOS XR:
• Candidate Configuration Capability
urn:ietf:params:netconf:capability:candidate:1.0
Transport
RPC
Operations
Content

Example <edit-config>
<?xml version="1.0" encoding="UTF-8" ?>
<rpc message-id="106" xmlns="urn:ietf:params:netconf:capability:candidate:1.0">
<edit-config>
<target><candidate/></target>
<config>
<Configuration><InterfaceConfigurationTable><InterfaceConfiguration><Naming>
<Active>act</Active>
<InterfaceName>TenGigE0/3/0/0</InterfaceName>
</Naming>
<Description>NETCONF CONFIG</Description>
<IPV4Network><Addresses><Primary>
<Address>10.223.1.1</Address>
<Netmask>255.255.255.0</Netmask>
</Primary></Addresses></IPV4Network>
</InterfaceConfiguration></InterfaceConfigurationTable>
</Configuration></config>
</edit-config> </rpc>
##
Datastore
Interface
Add interface
description
IPv4 Address
Netmask
description NETCONF CONFIG
ipv4 address 10.223.1.1 255.255.255.0

NETCONF Operations - <get-config>
• Used to retrieve all or portions of configuration
• Subtree filtering support:
• Attribute Match Expression
• Can only be specified in Table classes (e.g. <InterfaceName Match=“TenGig.*”/> )
• Containment Nodes:
• Filtering is by specifying container classes (e.g. <InterfaceConfigurationTable/> )
• Selection Node:
• Filtering by specific selection (e.g. <InterfaceName>Loopback0</InterfaceName>
Transport
RPC
Operations
Content

Example <get-config>(Attribute match)
<get-config>
<source><running/></source>
<filter>
<Configuration>
<InterfaceConfigurationTable>
<InterfaceConfiguration>
<Naming>
<InterfaceName Match=“TenGigE.*”/>
</Naming>
</InterfaceConfiguration>
</InterfaceConfigurationTable>
</Configuration>
</filter>
</get-config> </rpc>
Attribute match: all TenGigE interfaces
Datastore
For Your
Reference

Example <get-config>(Containment node)
<get-config>
<filter>
<Configuration>
<InterfaceConfigurationTable/>
</Configuration>
</filter>
</get-config>
</rpc>
Containment: all interfaces
Datastore
For Your
Reference

Example <get-config> (selection node)
<get-config>
<filter>
<Configuration>
<InterfaceConfigurationTable>
<InterfaceConfiguration>
<Naming>
<InterfaceName>Loopback0</InterfaceName>
</Naming>
</InterfaceConfiguration>
</InterfaceConfigurationTable>
</Configuration>
</filter>
</get-config>
</rpc>
Datastore
Selection: Loopback0 only…
For Your
Reference

Example <get-config> ASR9000 <rpc-reply>
<?x<?xml version="1.0" encoding="UTF-8"?>
<rpc-reply message-id="106" xmlns="urn:ietf:params:netconf:capability:candidate:1.0">
<data><Configuration>
<InterfaceConfigurationTable MajorVersion="5" MinorVersion="3">
<InterfaceConfiguration> <Naming><Active>act</Active>
<InterfaceName>Loopback0</InterfaceName>
</Naming>
<InterfaceVirtual>true</InterfaceVirtual>
<IPV4Network MajorVersion="6" MinorVersion="3">
<Addresses><Primary>
<Address>10.99.8.8</Address>
<Netmask>255.255.255.255</Netmask>
</Primary></Addresses></IPV4Network>
</InterfaceConfiguration></InterfaceConfigurationTable>
</Configuration></data>
</rpc-reply>
Tagged Configuration
For Your
Reference

Sample: Get YANG Model <get-schema> list
#229
<rpc message-id="101" xmlns="urn:ietf:params:xml:ns:netconf:base:1.0">
<get>
<filter type="subtree">
<netconf-state xmlns= "urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring">
<schemas/>
</netconf-state>
</filter>
</get>
</rpc>
##
For Your
Reference

Example: <get-schema> sample
#286
<?xml version="1.0" encoding="utf-8"?>
<get-schema xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring">
<identifier>Cisco-IOS-XR-ifmgr-cfg</identifier>
<version>2015-01-07</version>
</get-schema>
</rpc>
##
For Your
Reference

Closing SSH NETCONF Session
#140
<close-session>
</close-session>
</rpc>
## Connection to 10.99.8.8 closed by remote host.
odl-1$
RPC close-session
Connection Closed
For Your
Reference

Traditional Infrastructure
Data
Center
Customers
Optical
IP/MPLS
System Engineering
Machines)
Network Engineering
(WAN Services, NfV)
Enterprise
Virtual

Orchestration
RESTful APIs
ProgrammingCollection
WAN
R1
R2
R3
5
1
2
3
4 Congested!!
to collector
1
DC2 bandwidth NOW
2
Analytic and confirmation to
determine LSP
3
R1-R3 LSP Tunnel
programmed
4
Traffic Steering onto
R1-R3 LSP5

Orchestration
RESTful APIs
ProgrammingCollection
WAN
R1
R2
R3
5
1
2
3
4 Congested!!
Augment Collection: BGP-LS
1
Service Request
2
Analytic, Orchestration
e.g. WAE
3
LSP: PCEP, NETCONF, etc
Controller-ODL, Tail-f, etc
4
Traffic Steering:
OpenFlow, NETCONF, etc5

Summary-ASR9000 APIs & Protocols
Key Function Protocol/API Comments
IGP Topology BGP Link-State Wraps up LSDB in BGP transport and pushes to BGP speaker
on SDN WAN Orch Platform
Create, Modify and Delete TE
or SR Tunnels
Stateful Extensions to
PCEP
Introduced as part of Stateful PCE effort
Classification and Action Openflow Extensions Leveraging per-flow MATCH/Action semantics
Read/Write of Persistent
Configuration Data on
Network Devices
Netconf/Yang Finally gaining traction with vendor implementations and now on
OpenDaylight Platform

Call to Action
• Visit the World of Solutions for
• Service Provider Section
• ASR9000
• WAN Automation Engine
• Tail-f
• DevNet Zone
• Related labs and sessions
• Cisco Open SDN Controller
• WAN Automation Engine (WAE)-Hands-On Labs
• DevNet: developer.cisco.com

DevNet: Getting Started with WAE
• DevNet – WAE Tutorials and API
Documentation
https://developer.cisco.com/site/wae/index.gsp
• Learning Labs – Hands on with the
WAE APIs over in the DevNet zone
https://learninglabs.cisco.com/#/home
• dCloud – demos of WAE
http://dcloud.cisco.com/
• WAE home page on cisco.com:
http://www.cisco.com/go/wae
WAE Learning labs
For Your
Reference

World of Solutions: Tail-f NCS Overview
• Logically centralized network
services
• Data models for data structures
• Structure representations of:
• Service instances
• Network configuration and state
• Mapping service operations to
network configuration changes
• Transactional integrity
• Multiprotocol support
• Multivendor support
Device
ModelsNetwork Element Drivers
Device Manager
Service Manager
Tail-f NCS Service
Models
Networkwide CLI and Web UIREST, NETCONF, Java, etc.
Network
Engineer
Management
Applications
NETCONF, CLI, SNMP, REST, etc.
• Execution
management
systems (EMSs)
• Applications
• Controllers
For Your
Reference

DevNet: Open SDN Controller (OSC)
• Cisco Distribution of
OpenDaylight platform
• Enhancements
• Simplified install
• Admin/Monitoring tools
• Embedded Apps
• Plug-in Clustering
• Technical Support
• developer.cisco.com/site/op
enSDN/
Community Support Cisco Supported
“HELIUM”
Open SDN Controller
Common Content
Incremental
Cisco
Value
Precluded
OpenDaylight
Content
SDNi
Defense4all
PacketCable
PCMM
VTN Project
OpenContrail
Plugin
SNMP4SDN
AAA
BGP-LS
Controller
DLUX
Group Policy
L2 Switch
LISP Flow
Mapping
Openflow
Plugin
OVSDB
Secure Network
Bootstrap Infra
Service Function
Chaining
Yang Tools
MD-SAL
AD-SAL
Log
Aggregation
Metrics
Aggregation
Monitoring
Basic
Clustering
Central Admin
OVA Distribution
Sample Apps
One Click InstallPCEP
Plug-in Clustering
To be contributed back
to the “open community”
For Your
Reference

Cisco Developer Resources
• Devloper Network: http://developer.cisco.com
• OpenDaylight: http://www.opendaylight.org/
• SDLC - https://developer.cisco.com/site/networking/one/sdlc/overview/
• EEM - https://supportforums.cisco.com/community/netpro/network-infrastructure/eem
• ONE Forums -
https://developer.cisco.com/site/devnet/forums/index.gsp#L2CiscoONE
• APIC-EM - https://developer.cisco.com/site/networking/one/apic/enterprise-module/
• APIC-DC
• APIs https://developer.cisco.com/site/networking/routers-
switches/nexus9000/documents/
• GitHub - https://github.com/datacenter/nexus9000
For Your
Reference

Complete Your Online Session Evaluation
Don’t forget: Cisco Live sessions will be available
for viewing on-demand after the event at
CiscoLive.com/Online
• Give us your feedback to be
entered into a Daily Survey
Drawing. A daily winner
will receive a $750 Amazon
gift card.
• Complete your session surveys
though the Cisco Live mobile
app or your computer on
Cisco Live Connect.

Continue Your Education
• Demos in the Cisco campus
• Walk-in Self-Paced Labs
• Table Topics
• Meet the Engineer 1:1 meetings
• Related sessions

Software Defined Network (SDN) using ASR9000 :: BRKSPG-2722 | San Diego 2015

Software Defined Network (SDN) using ASR9000 :: BRKSPG-2722 | San Diego 2015

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Software Defined Network (SDN) using ASR9000 :: BRKSPG-2722 | San Diego 2015

Similar to Software Defined Network (SDN) using ASR9000 :: BRKSPG-2722 | San Diego 2015 (20)

More from Bruno Teixeira

More from Bruno Teixeira (10)

Recently uploaded

Recently uploaded (16)

Software Defined Network (SDN) using ASR9000 :: BRKSPG-2722 | San Diego 2015