Large Scale Data center Solution Guide: eBGP based design

ipinfusion™
Solution Guide
Open Compute Network
Operating System Version 1.2.1
OcNOS™
Validated Solution Guide
EBGP-based Data Center with OcNOS

2
Solution Guide
Contents
CHAPTER 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Data Center Interconnect Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Large-Scale Data Center Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Large-Scale Data Center Topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Large-Scale Data Center Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
EBGP-Routed Clos Topology-Based Data Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
EBGP Data Center Design using OcNOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
CHAPTER 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
ToR (leaf node) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Tier-2 (spine node) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Tier-3 (core node) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Tier-2 (border router) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Tier-3 (WAN router) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Other Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
Appendix A: Configuring the Data Center through NetConf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Appendix B: NetConf User Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3
Solution Guide
BGP Border Gateway Routing Protocol
EBGP External BGP
STP Spanning Tree Protocol
TRILL Transparent Interconnection of Lots of Links
SPB Shortest Path Bridging
ECMP Equal Cost Multipath
Glossary

4
Solution Guide
Chapter 1
Data Center Overview
Network Automation provides IT administrators and network operators significant benefits. This solution guide
describes an approach to build data centers using Layer3 BGP routing protocol.
It also summarizes on some design philosophies for data center and why E-BGP is better suited.
■■ Large-scale data center requirements
■■ Large-scale data center topologies
■■ Large-scale data center routing
■■ EBGP-routed large-scale Clos topology-based data center
Large-Scale Data Center Requirements
The design of large-scale data centers is driven by operational simplicity and network stability. Operational
simplicity and network stability ensures easier manageability and therefore reduced operational expenses.
From the network design aspect, the requirements are:
■■ Ability to accommodate the variable-application bandwidth and strict latency requirements.
■■ Ability to handle the increased east-west (server-to-server) traffic within the data center due to massive
data replication between clusters and virtual machine migrations.
■■ Traffic-Engineering with application load balancing. The network infrastructure should itself perform
controlled per-hop traffic engineering.
■■ Minimize CAPEX and incorporate vendor diversity by using a simple, interoperable routing protocol with a
minimal set of features.
■■ A design to minimize OPEX by keeping the failure domain at the lowest level in the network hierarchy.
Large-Scale Data Center Topologies
A traditional tree-based (upside down) topology with a three-layer hierarchy of core, aggregation and access
layer can be used in a data center design. This approach is suited if the majority of the traffic is entering
and leaving (north-south) the data center. An increase in bandwidth requirements then can be addressed by
upgrading the device line cards or port density. However with the current trend of increasing server-to-server
(east-west) traffic, scaling these networks horizontally is expensive or impossible at times.
A Clos network (leaf and spine) is a horizontally scalable topology where every leaf node is connected to
every other spine. The topology can be extended to different stages for scaling. Clos networks are fully non-
blocking and load balancing is inherent in the topology itself as all available paths are ECMP. Clos networks
are ideal for the current requirements of a large-scale data center.
Large-Scale Data Center Routing
Layer 2-only routing was used in a traditional tree-based data center topology. Traditional layer-2 protocols
such as STP do not give bi-sectional bandwidth, whereas recent developments such as TRILL, SPB have
selected vendor support.
However, a hybrid of layer 2 /layer 3 can be used to limit the size of failure domain and scale up the data
center. Layer 3 routing can be used in tier 1 (core) and layer 2 in tier 3 (access). Tier 2 can be based on either
layer 2 or layer 3. A hybrid model has the advantage of seamless Virtual Machine mobility and requires less
IP subnets for the data center. Although this design can scale-up, it is difficult and complex to manage the
different protocols.

5
Solution Guide
A layer 3 only design simplifies the network and improves network stability and scalability, as well as
localizing the failure domain (confined to the L2 broadcast domain). Seamless virtual mobility can be achieved
in a L3 only based data center by using L2 overlay networks. From experiment and analysis, External BGP
(EBGP) is considered ideal compared to IGPs due to the following [See Reference]:
■■ Less complex protocol, simple state machine
■■ Information flooding overhead is less, no frequent updates unlike IGPs
■■ Network failure recovery is very fast. Although BGP convergence is slower than IGP, in a Clos topology
with ECMP links, the failure is masked as soon as an alternate path is found.
■■ Failure domain is minimized in a Clos topology with EBGP. Some of the failures are local/hidden/not
propagated if the failed link was not selected/advertised as the best path among the ECMP paths by the
BGP speaker. The failures, where all devices have to withdraw some prefixes or update the ECMP groups
in the FIB, are very limited and in those failures the failed link/node does not impact the re-convergence
process.
■■ Administrator can define the application traffic path. BGP provides services like prefix distribution, prefix
filtering, traffic engineering, traffic tagging, and multi-vendor stability better than other IGPs.
■■ Easier to troubleshoot.
EBGP-Routed CLOS Topology-Based Data Center
EBGP-routed CLOS topology is considered the best choice for laying the IP fabric in a data center because
of the horizontal scalability feature of Clos topology and the ease of use and services provided by EBGP
especially prefix-filtering, prefix distribution, and traffic engineering which are required extensively in a
data center.
Configuration Guidelines
Configuration guidelines for laying IP fabric using EBGP efficiently are as follows:
■■ Run all EBGP sessions over single-hop point-to-point links.
■■ Use private Autonomous System Numbers (ASNs) (64512-65534) to avoid ASN conflicts.
■■ Give all tier 1 (core) devices a single ASN.
■■ Give all tier 2 devices in the same cluster the same unique ASN. A cluster or pod is a group of tier 2 (spine
switches) + tier 3 switches (ToR/leaf) + servers.
■■ Give every tier 3 (ToR) device in a cluster a unique ASN.
■■ Reuse tier-3 ASNs across clusters. Configure tier-3 devices with the BGP “allowas-in” feature to allow route
learning of prefixes from the same ASNs in other clusters.
■■ Announce server subnets on tier-3 devices via BGP without using route summarization on tier-2 and tier-1
devices.
■■ Use edge clusters (pods) for external connectivity. Each edge cluster consists of border routers (tier-2) and WAN
routers (tier-3). Give each WAN router a unique public ASN to connect the data center to the external world.
■■ For border routers, remove private ASNs before sending the information to WAN routers by configuring
border routers with the “remove-private-AS” BGP feature.
■■ To relax the BGP ECMP criteria for AS paths, configure BGP “as-path multipath-relax” on all routers/
switches. This way, an equal cost path with a different AS PATH, but the same AS PATH length is also
considered an equal cost path (ECMP).

6
Solution Guide
■■ For faster failure detection, configure the BGP session with BFD.
■■ To avoid recurring BPG update/selection for a single failure through all peers or BGP update message
dispersion on a particular speaker, use BGP update groups. The BGP update group feature processes an
update once and sends it to a group of neighbors that share a common outbound policy. The BGP RIB is
scanned every time for each peer to apply the outbound filter.
■■ To avoid micro routing loops, configure tier-2 and tier-1 with static discard or null routes rather than a default
route. Routing loops can happen when a tier-2 device has lost all its learned prefixes, but has a default
route to a tier-1 device and that tier-1 device still has a route back to the tier-2 device.
EBGP Data Center Design using OcNOS
Figure-1 shows a minimal representation that encompasses all the elements in a layer 3 data center.
The number of ECMPs in the data center is equal to the number of cores (tier-1 switches).
Figure 1. IP fabric using EBGP
TOR 1 TOR 2
Spine Tier 2
Cluster/Pod
TOR 3 TOR 1 TOR 2 TOR 3
Internet
WAN Routers
External Cluster/Pod
Border Router
Core Tier 1
Tier 3

7
Solution Guide
Figure 2 shows the Autonomous System Number (ASN) allocation scheme used in the data center. The WAN
routers are assigned a public ASN, which connects the data center to external world. The tier-3 ASNs per
ToR are reused across the clusters.
Figure 2: ASN allocation in an EBGP-based data center
TOR 1 TOR 2
ASN 64601 ASN 64602
TOR 3 TOR 1 TOR 2 TOR 3
Internet
ASN 101ASN 100
ASV 64603
External Cluster
ASN 65534
R11 R12 R13 R14
Cluster 1 Cluster 2
ASN65500
ASN65501
ASN65502
ASN65500
ASN65501
ASN65502

8
Solution Guide
Chapter 2
Configuration
ToR (Leaf node)
Command Purpose
Step 1 (config)#interface xe1 Enter interface mode.
Step 2 (config-if)#ip address 32.1.0.3/24 Configure ip address on the Interface
Step 3 (config-if)#exit Exit interface mode.
Step 6 (config)#router bgp 65500 Configure the EBGP routing process with private ASN
Step 7 (config-router)#max-paths ebgp 8 Exit interface mode.
Step 8 (config-router)#neighbor 32.1.0.2
remote-as 64601
Configure maximum EBGP ECMP that can be installed in
BGP.
fall-over bfd
Configure the EBGP neighbor over the connected
interface using the neighbor IP and remote private ASN
allowas-in
Configure BFD for the BGP session for faster failure
detection.
remote-as 64601
Configure “allowas-in” for the neighbor to accept routes
with same ASN learned over this neighbor
fall-over bfd
interface using the neighbor IP and remote private ASN
allowas-in
detection.
Step 14 (config-router)#exit Configure “allowas-in” for the neighbor to accept routes
with same ASN learned over this neighbor
Step 15 (config-router)#exit Exit Router mode.
Tier-2 (Spine node)
Command Purpose
Configure the interfaces
Step 2 (config-if)#ip address 32.1.0.2/24 Configure ip address on the interface
Step 5 (config-if)#ip address 32.3.0.2/24 Configure an IP address on the interface
Step 12 (config)#router bgp 64601 Configure the eBGP routing process with private ASN

9
Solution Guide
Command Purpose
Configure BGP on the router
Step 13 (config-router)#bgp bestpath as-path
multipath-relax
Configure “as-path multipath-relax” to relax the AS-PATH
exact match (if AS-PATH length are same) criteria for BGP
ECMP
Step 14 (config-router)#max-paths ebgp 8 Configure maximum EBGP ECMP that can be installed
in BGP.
remote-as 65000
interface using the neighbor IP and remote ASN
fall-over bfd
detection.
remote-as 65001
fall-over bfd
detection.
remote-as 65002
fall-over bfd
detection.
remote-as 65534
fall-over bfd
detection.
remote-as 65534
fall-over bfd
detection.
Step 25 (config-router)#exit Exit Router mode.
Tier-2 (Spine node) cont.

10
Solution Guide
Tier-3 (Core node)
Command Purpose
Step 2 (config-if)#ip address 21.1.0.1/24 Configure ip address on the interface
Step 11 (config-router)#bgp bestpath
as-path multipath-relax
exact match (if AS-PATH length are same) criteria for
BGP ECMP
Step 12 (config-router)#max-paths ebgp 8 Configure maximum EBGP ECMP that can be installed
in BGP.
remote-as 64601
fall-over bfd
detection.
remote-as 64602
fall-over bfd
detection.
remote-as 64603
fall-over bfd
detection.
remote-as 64603
fall-over bfd
detection.
Step 21 (config-router)#exit Exit BGP mode

11
Solution Guide
Command Purpose
Step 15 (config-router)#bgp bestpath
as-path multipath-relax
exact match (if AS-PATH length are same) criteria for
BGP ECMP
Step 16 (config-router)#max-paths ebgp 8 Configure maximum EBGP ECMP that can be installed in BGP.
remote-as 65534
Configure the EBGP neighbor over the connected interface
using the neighbor IP and remote ASN
fall-over bfd
detection.
remote-as 65534
fall-over bfd
detection.
remote-as 65534
fall-over bfd
detection.
remote-as 65534
fall-over bfd
detection.
remote-as 100
fall-over bfd
detection.
remove-private-AS
Configure “remove –private-AS” to remove the private ASNs
for the routes advertised to this neighbor.
remote-as 101
Tier 2 (Border router)

12
Solution Guide
Command Purpose
fall-over bfd
detection.
remove-private-AS
Configure “remove –private-AS” to remove the private ASNs
for the routes advertised to this neighbor.
Tier-3 (WAN router)
This is a partial list and does not contain the Internet configuration.
Command Purpose
Step 6 (config)#router bgp 100 Configure the eBGP routing process with public ASN
Step 7 (config-router)#max-paths ebgp 8 Configure maximum EBGP ECMP that can be installed in
BGP.
remote-as 64603
fall-over bfd
detection.
remote-as 64603
fall-over bfd
detection.
Other Configurations
You must repeat similar configurations for all ToR, spine, core, border, and WAN devices as well.
Tier 2 (Border router) Cont.

13
Solution Guide
Validation
Use the show ip bgp command to validate the output at each node.
Consider the following case: for application load balancing and high availability/reliability, two similar
application servers can be placed at two clusters. For users accessing the application server through the
Internet, the access to the server is load balanced and failure of one of the application servers does not
impact the accessibility. The following is the output at various nodes for a subnet, such as:
■■ 70.70.70.1 (application server) at ToR1 in cluster 1 and cluster 2
■■ 80.80.80.1 at ToR 1 cluster 1
■■ 90.90.90.1 at ToR 1 cluster 2
ToR 1 Cluster 1
ToR 1 Cluster 2

14
Solution Guide
R1 Cluster 1
R11 Tier 1
Border Router B1 Tier 2 External Cluster

15
Solution Guide
WAN Router WR1 External Cluster
Conclusion
OcNOS with EBGP routing is a highly scalable, simple and flexible way of laying IP fabric in a data center.
The data center can be easily scaled for:
■■ Higher computing needs by adding more clusters.
■■ Higher performance and redundancy by adding more cores
■■ Higher uplink speeds by adding more external/edge clusters.
References
Use of BGP for routing in large scale data centers:
https://tools.ietf.org/html/draft-ietf-rtgwg-bgp-routing-large-dc-09

16
Solution Guide
Appendix A: Configuring the Data
Center through NetConf
This appendix shows how to configure a data center through NetConf. This section contains XML payloads
which can be used by any NetConf client to send configurations to OcNOS devices.
The configurations below in XML are communicated via RPC messages from the NetConf client to the
NetConf server.
This document uses the yang-cli which is part of open source OpenYumaMaster which and is preinstalled on
a ZebM-enabled OcNOS ONIE package as a NetConf client.
Pre-requisite
Refer to Yangcli Operations section of the NetConf User Guide and perform these operations first:
1. Establish Connection
2. Load Modules
Once the above operations are complete, edit-config operation can be performed with XML payloads using
the command:
yangcli ocnos@Hostname> edit-config config=@/<path-to-xml-file>/<xml-file>.xml
Save the below xml payloads as an .xml file in any desired location. Make sure that each XML payload starts
with <vr xmlns=”<namespace>”> and ends with </vr>.
TOR (leaf node)
TOR.xml
This table shows the XML to configure an IP address:
XML Payload Description
<vr xmlns=”http://www.ipinfusion.com/CMLSchema/
ZebOS”>
namespace of the ZebOS module
<vrId>0</vrId> Vrid
<interface>
<ifName>xe49</ifName>
<ipAddr>50.52.1.1/24</ipAddr>
</interface>
<interface>
<ifName>ge2</ifName>
</interface>
Configure the interfaces which are connected to the
SPINE routers.
This payload configures the IP address of the
interfaces xe49 and ge2.
</vr> Close namespace for the ZebOS module

17
Solution Guide
This is an example of the XML payload to be sourced in the yang CLI to configure an interface IP addresses:
<vr xmlns=”http://www.ipinfusion.com/CMLSchema/ZebOS”>
<vrId>0</vrId>
<interface>
</interface>
<interface>
</interface>
</vr>
This table shows the XML to configure BGP attributes:
<vr xmlns=”http://www.ipinfusion.com/CMLSchema/
ZebOS”>
Namespace of the ZebOS module
<vrId>0</vrId> Enter Vrid
<bgp>
<bgpAs>64602</bgpAs>
Open BGP tag.
Configure the BGP AS number.
<multipathRelax>1</multipathRelax>
<vrfName>default</vrfName>
Configure “as-path multipath-relax” to
relax the AS-PATH criteria for BGP ECMP
<multipath>
<bgpType>ebgp</bgpType>
<multipathType>
<multipathsNum>8</ multipathsNum>
</multipathType>
</multipath>
Configure maximum EBGP ECMP that can be
installed in BGP.
<bgpPeer>
<peerAddr>50.52.1.2</peerAddr>
<allowAsNum>3</allowAsNum>
<bgpPeerBfd>1</bgpPeerBfd>
<peerAs>64902</peerAs>
</bgpPeer>
Configure BGP peer with AS 64902 and peer
address 50.52.1.2
Enable BFD for link failure detection
Configure “allowas” in for the neighbor to
accept routes with same ASN learned over this
neighbor
<bgpPeer>
<allowAsNum>3</allowAsNum>
</bgpPeer>
address 50.54.1.2
Configure “allowas” in for the neighbor to
accept routes with same ASN learned over this
neighbor
Enable BFD for link failure detection.
<bgpRedistList>
<redistType>connected</redistType>
</bgpRedistList>
<bgpRedistList>
<redistType>ospf</redistType>
</bgpRedistList>
Redistribute the connected and OSPF routes
</bgp> Close BGP tag.

18
Solution Guide
Tier-2 (spine node)
SPINE.xml
This table shows the XML to configure IP addresses:
<vr xmlns=”http://www.ipinfusion.com/CMLSchema/ZebOS”> Enter namespace of the ZebOS module
<interface>
<ifName>xe49/1</ifName>
</interface>
<interface>
<ifName>xe49/2</ifName>
</interface>
Configure the interfaces which are connected
to the SPINE routers.
interfaces xe49/1 and xe49/2.
<bgp>
Open BGP tag.
Configure “as-path multipath-relax” to relax the
AS-PATH criteria for BGP ECMP
<multipath>
<multipathType>
<multipathsNum>8</multipathsNum>
</multipathType>
</multipath>
installed in BGP.
<bgpRedistList>
</bgpRedistList>
Redistribute the connected routes
<bgpPeer>
</bgpPeer>
address 50.52.1.1
<bgpPeer>
</bgpPeer>
address 52.31.1.2

19
Solution Guide
Tier-3 (core node)
CORE.xml
<interface>
<linkStatus>1</linkStatus>
<ipLabel>NULL</ipLabel>
</interface>
<interface>
<linkStatus>0</linkStatus>
<ipLabel>NULL</ipLabel>
</interface>
Configure the interfaces that are connected to
the spine routers.
interfaces xe1 and ge8.
<bgp>
Open BGP tag.
Configure “as-path multipath-relax” to relax
the AS-PATH criteria for BGP ECMP
<multipath>
<multipathType>
</multipathType>
</multipath>
Configure maximum EBGP ECMP that can
be installed in BGP.
<bgpRedistList>
</bgpRedistList>
<bgpPeer>
</bgpPeer>
address 52.31.1.1
<bgpPeer>
</bgpPeer>
address 31.22.1.2

20
Solution Guide
Tier-2 (Border Router)
<interface>
</interface>
<interface>
</interface>
<interface>
</interface>
Configure the interfaces which are connected
to the SPINE routers.
interface ge7, ge6 and ge2.
<bgp>
Open BGP tag
Configure “as-path multipath-relax” to relax
the AS-PATH criteria for BGP ECMP
<multipath>
<multipathType>
</multipathType>
</multipath>
installed in BGP.
<bgpRedistList>
</bgpRedistList>
<bgpPeer>
</bgpPeer>
address 52.31.1.1
<bgpPeer>
</bgpPeer>
address 31.22.1.2
<bgpPeer>
<peerRemovePvtAs>1</peerRemovePvtAs>
</bgpPeer>
address 23.22.1.2
Configure remove-private-as for 23.22.1.2 peer

21
Solution Guide
Tier-3 (WAN Router) –Partial -Does Not Contain the Internet Configuration
This table shows the XML to configure an IP address:
<interface>
<ifName>ge1/1/1</ifName>
</interface>
Configure the interfaces that are connected to
the SPINE routers.
interface ge1/1/1.
<bgp>
Open BGP tag
<bgpRedistList>
</bgpRedistList>
<bgpPeer>
</bgpPeer>
address 52.31.1.1
Similar configurations must be repeated for all TORs, spines, cores, and border routers.
Appendix B: NetConf User Guide
This document describes managing OcNOS devices using NetConf.
This document is intended for network administrators and other engineering professionals who configure and
manage devices running OcNOS.
There are three different northbound applications in OcNOS (CLI , NetConf, and SNMP). All the northbound
applications are text-based, with each command usually associated to a specific task.
OcNOS NetConf supports transactions as described in Transactions.

22
Solution Guide
NetConf Quick Start
The NetConf protocol defines a simple mechanism through which a network device can be managed,
configuration information can be retrieved, and new configuration data can be uploaded.
NetConf uses a simple RPC-based mechanism to communicate between a client and a server. The client can be
a script or application typically running as part of a network manager. The server is usually a network device.
A NetConf session is the logical connection between a network administrator or network configuration application
and a network device. A device must support at least one NetConf session and can support multiple sessions.
Global configuration attributes can be changed during any session and the effects are visible in all sessions. The
candidate and running and startup configuration are shared across all the sessions.
Note: A limited number of protocol modules are supported through NetConf. A detailed list is in the NetConf command reference.
NetConf Clients
You can use any NetConf client application to manage the device using Yang modules. These client
applications require a Yang module which you downlaod from:
https://github.com/IPInfusion/OcNOS/tree/1.2
This application establishes a secure connection with the daemon running on the device to perform
commands and send system responses.
There are different NetConf client applications. In this document, OpenYuma’s yangcli application is used to
show NetConf operations.
Refer to the standard Yangcli operations at:
https://github.com/OpenClovis/OpenYuma/blob/master/netconf/doc/yuma_docs/openyuma-yangcli-manual.odt
The NetConf operations get-config and get return large amounts of data. To improve the readability of the
output, the subtree filter based sget-config and sget operations are used in this document.
Install Yangcli
Check out the git repository, compile and install OpenYuma components. Refer to the README for more
details.
Download Yang files
Download the Yang files from the website:
https://github.com/IPInfusion/OcNOS/tree/1.2
Copy the files to /usr/share/yuma/modules/netconfcentral on the host machine where the client application
runs. This system path only works with OpenYuma’s Yangcli client application. If you are running a different
client application, follow the respective reference document to copy the Yang files to the appropriate location.
Creating User Accounts
User level access control applies to all NetConf operations. The table below describes access levels for
different user types and commands to create different type of user account
Account type Access Command
User Read username <user name> role network-user password
<password>
Operator All, except full configuration store
level change.
username <user name> role network-operator
password <password>
Admin All username <user name> role network-admin password
<password>

23
Solution Guide
You must login into the device using network administrator account to create new user accounts. These are
the steps:
[root@localhost ~]# ssh ocnos@10.12.45.253
ocnos@10.12.45.253’s password:
Last login: Wed Jun 15 21:27:39 2016
OcNOS version 1.2.0.179-OCNOS-DC-IPBASE-ZEBM IPIRouter 06/14/16 21:30:36
OcNOS>en
OcNOS#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
OcNOS(config)#username netuser role network-user password Abc@6789
OcNOS(config)#username netoperator role network-operator password Abc@6789
OcNOS(config)#username netadmin role network-admin password Abc@6789
After creating the account, use the write command to write the configured data into persistent storage as
described in Copy Running Configuration to Startup
OcNOS(config)#write
Building configuration...
[OK]
The NetConf operations for different user account types are shown in Supported Operations.
Yangcli Operations
Establish Connection
These are the steps to establish a connection between the NetConf client and the server that is running on
the device.
# yangcli server=<ip _ address> user=<user _ name> password=<password>
ip _ address: Address of the device to be managed
user _ name & password: User account detail for authentication
The interactive version of this command is shown below:
# yangcli
yangcli> connect
Enter string value for leaf <user>
yangcli:connect> <user _ name>
Enter string value for leaf <server>
yangcli:connect> <ip _ address>
Enter string value for leaf <password>
yangcli:connect> <password>
Load Modules
The load command loads a YANG module into the server. This command is not part of the RFC and is only
supported by the Yang client.
The ZebOS module is the parent for all protocol modules supported in OcNOS. This module includes all the
sub modules. Therefore, loading this module is mandatory to start managing the device. Here is the portion of
ZebOS.yang file that includes other sub modules.
module ZebOS {
namespace “http://www.ipinfusion.com/CMLSchema/ZebOS”;
prefix “ZebOS”;
include nsmLACP;
include oamBfd;

24
Solution Guide
include bridge;
include ospf;
include vlan;
include mstp;
include layer2LACP;
include bgp;
include vr;
include vrf;
include interface;
include lldpv2;
include ospf6;
include rib;
include vlaninterface;
...
}
> load ZebOS
RPC Data Reply 1 for session 1:
rpc-reply {
mod-revision 2015-10-08
}
After using the load command, you can also use the mgrload command to keep the current session
synchronized with the server.
> mgrload ZebOS
Load module ‘ZebOS’ OK
Configure the Device
Configuration details are placed in an XML file and sent to the netconfd server. You must refer to the Yang file
to prepare the XML based configuration file with the correct hierarchy. If the hierarchy is not correct, yangcli
throws an error.
One portion of the BGP protocol module Yang model is presented below. This module is a sub-module for the
parent ZebOS yang module.
submodule bgp {
belongs-to ZebOS { prefix ZebOS; }
include interface;
include vrf;
import cml _ data _ types {
prefix cml _ data _ types;
}
revision “2015-04-25” {
description “Revised on 2015-04-25.”; }
grouping bgp-grouping {
list bgp {
description “bgp”;
config true;
key “bgpAs”;
leaf vrId {
mandatory false;
type leafref {
path “/vr/vrId”;
}
} // END of vrId definition.
...
leaf keepAlive {
mandatory false;

25
Solution Guide
type cml _ data _ types:CML _ UINT16 _ T {
range “0..65535”;
}
default “30”;
config true;
} // END of keepAlive definition.
leaf holdTime {
mandatory false;
type cml _ data _ types:CML _ UINT16 _ T {
range “0..65535”;
}
default “90”;
config true;
} // END of holdTime definition.
...
} // END of bgpDebug-grouping definition.
Based on the hierarchy in the Yang module. the following XML file named bgp.xml is created with the
configuration data. The bgp.xml file is referenced in the edit-config operation specified below.
<vrId>0</vrId>
<bgp>
<bgpAs>100</bgpAs>
<keepAlive>60</keepAlive>
<holdTime>180</holdTime>
</bgp>
</vr>
Use this command to globally set or reset the keepalive and holdtime values for all the neighbors.
yangcli ocnos@10.12.45.253> edit-config config=@/root/bgp.xml
Filling container /edit-config/input/target:
RPC OK Reply 15 for session 1:
Retrieve Candidate Configuration
Candidate configuration datastore is used to hold configuration data that can be manipulated without
impacting the device’s current configuration. The candidate configuration is a full configuration data set that
serves as a work place for creating and manipulating configuration data. Additions, deletions, and changes
can be made to this data to construct the desired configuration data.
>sget-config /vr/bgp source=candidate
Filling list /vr/bgp:
rpc-reply {
data {
vr {
bgp 100 {
bgpAs 100
holdTime 180
keepAlive 60
vrfName default
}
}
}
}

26
Solution Guide
Commit Candidate Configuration
A <commit> operation MAY be performed at any time that causes the device’s running configuration to be set
to the value of the candidate configuration.
yangcli ocnos@10.12.45.253> commit
Retrieve Running Configuration
Configuration data is the set of writable data that is required to transform a system from its initial default state
into its current state. You can use the get-config operation to fetch the running configuration data.
>sget-config /vr/bgp source=running
rpc-reply {
data {
vr {
bgp 100 {
bgpAs 100
holdTime 180
keepAlive 60
vrfName default
}
}
}
}
Retrieve Running Configuration and State Data
State data is the additional data on a system that is not configuration data such as read-only status
information and collected statistics. You can use the get operation to fetch a protocol module’s running
configuration and state data.
>sget /vr/bgp
rpc-reply {
data {
vr {
bgp 100 {
bgpAs 100
holdTime 180
keepAlive 60
vrfName default
bgpTableVersion 0
ntwkPrefixCount 0
ibgpMetric 0
pathAttrBest 0
clusterList fill _ value
routerRunIpAddr fill _ value
bgpShowTypeStr fill _ value
bgpShowType 0
rfdMaxPenaltyCeil 0
rfdMinPenaltyFloor 0
dampeningStr 0
rfdCbStr 0

27
Solution Guide
ibgpMetric 0
}
}
}
}
Copy Running Configuration to Startup
NetConf supports startup options, so if you have configured the device and want to retain the configuration
after a device reboot, copy the running configuration into startup configuration. The yangcli command is:
> copy-config source=running target=startup
The equivalent OcNOS command is
# write
Error Messages
NetConf operations return protocol, management layer, and protocol module errors. The example below
depicts an error returned by a protocol module.
Copy the content below into bgp_err.xml.
<vrId>0</vrId>
<bgp>
<bgpAs>1</bgpAs>
<keepAlive>300</keepAlive>
<holdTime>200</holdTime>
</bgp>
</vr>
Execute the following command and commit the changes
yangcli tbyran@10.12.45.253> edit-config config=@/root/bgp _ err.xml
yangcli tbyran@10.12.45.253> commit
mgr _ rpc: got invalid reply on session 1 (invalid XPath expression syntax)
RPC Error Reply 13 for session 1:
rpc-reply {
rpc-error {
error-type protocol
error-tag
error-severity warning
error-app-tag general-warning
error-path ‘RPC operation failed’
error-message ‘%% Hold time should be greater than the keepalive time’
error-info {
error-number 4294962411
}
}
}
Transactions
OcNOS supports transaction-oriented configuration management. Transactions are created implicitly by edit-
config operations; commit and discard-changes operations close or terminate the transactions. Successive
edit-config operations are placed in the same transaction.

28
Solution Guide
Discard Changes
Discard the transaction or candidate configuration changes.
yangcli ocnos@10.12.45.253> sget-config /vr/ospf source=candidate
Filling list /vr/ospf:
mgr _ rpc: got invalid reply on session 1 (missing index component)
rpc-reply {
data {
vr {
ospf 0 {
ospfProcessId 0
ospfShutDown true
}
ospf 200 {
ospfProcessId 200
}
}
}
}
yangcli ocnos@10.12.45.253> discard-changes
yangcli ocnos@10.12.45.253> sget-config /vr/ospf source=candidate
rpc-reply {
data {
vr {
ospf 0 {
ospfProcessId 0
ospfShutDown true
}
}
}
}
Commit
Commit the transaction or candidate configuration changes.
yangcli ocnos@10.12.45.253> create /vr/ospf
Filling key leaf /vr/ospf/ospfProcessId:
Enter int32 value for leaf <ospfProcessId>
yangcli ocnos@10.12.45.253:create> 500
Filling key leaf /vr/vrId:
Enter uint32 value for leaf <vrId>
yangcli ocnos@10.12.45.253> 0

29
Solution Guide
yangcli ocnos@10.12.45.253> sget-config /vr/ospf source=running
rpc-reply {
data {
vr {
ospf 0 {
ospfProcessId 0
ospfShutDown true
}
ospf 500 {
ospfProcessId 500
}
}
}
}
Save Point
OcNOS supports the savepoint feature. A savepoint is a snapshot of the device current state. You can switch
the device state to any savepoint at any time.
This example creates two savepoints HT180 and HT300
■■ HT180 has holdTime set to 180
■■ HT300 has holdTime set to 300
yangcli ocnos@10.12.45.253> create
create create-savepoint create-subscription
yangcli ocnos@10.12.45.253> create-savepoint
Enter string value for leaf <savepointName>
yangcli ocnos@10.12.45.253:create-savepoint> HT180
yangcli ocnos@10.12.45.253> edit-config config=@/root/bgp _ ht.xml
yangcli ocnos@10.12.45.253> sget-config /vr/bgp source=running
rpc-reply {
data {
vr {
bgp 100 {
bgpAs 100
holdTime 300
keepAlive 60

30
Solution Guide
vrfName default
}
}
}
}
yangcli ocnos@10.12.45.253> create-savepoint HT300
Roll Back
This example shows rolling back the configuration. First, notice the current value of the holdTime attribute:
rpc-reply {
data {
vr {
bgp 100 {
bgpAs 100
holdTime 180
keepAlive 60
vrfName default
}
}
}
}
Now roll back to savepoint HT300 and note the value of holdTime attribute.
yangcli ocnos@10.12.45.253> rollback-transaction HT300
rpc-reply {
data {
vr {
bgp 100 {
bgpAs 100
holdTime 300
keepAlive 60
vrfName default
}
}
}
}
Supported Operations
All the NetConf operations are captured based on the capability. Hence, any operation falling in multiple

31
Solution Guide
capability are documented separately.
Note: Capability “base:1.0” supports candidate and running configuration store.
Capability Operation User Role Supported
(Yes/No)
Comments
:base:1.0 <get> User, Operator,
Admin
Yes
:base:1.0 <get> with subtree
filter
User, Operator,
Admin
Yes
:base:1.0 <get-config> User, Operator,
Admin
Yes
:base:1.0 get-config
source=<target>
User, Operator,
Admin
Yes
:base:1.0 <get-config> with
subtree filter
User, Operator,
Admin
Yes
:base:1.0 <edit-config> <target>
as parameter
Operator, Admin Yes Running configuration as
target is not supported
because writtable-
running capability is not
supported
:base:1.0 <edit-config> <config>
as parameter
Operator, Admin Yes
:base:1.0 <edit-config>:
<merge>
Operator, Admin Yes
<replace>
Operator, Admin Yes
<create>
Operator, Admin Yes
<delete>
Operator, Admin Yes
<remove>
Operator, Admin Yes
(Yes/No)
Comments
:base:1.0 <edit-config>: <none> Operator, Admin Yes
:base:1.0 <edit-config>: <error-
option > as stop-on-
error
Operator, Admin No
:base:1.0 <edit-config>: <error-
option > as continue-
on-error
Operator, Admin No

32
Solution Guide
:rollback-on-
error:1.0
<edit-config>: <error-
option > as rollback-
on-error
Operator, Admin Partial By default, this is the
behavior. So there is no
need to pass this error-
option.
:validate: 1.1 <edit-config>: <test-
option >
Operator, Admin No By default configuration
entries are validated
and stored, hence
this operation and its
parameters are not
handled. External
configuration store
validation is not
supported (i.e URL).
:base:1.0 <copy-config> : <
target> <source>
Operator, Admin Yes
:base:1.0 <lock> : < target> Operator, Admin Yes Candidate configuration
lock is not supported.
:base:1.0 <unlock> : < target> Operator, Admin Yes Candidate configuration
unlock is not supported
:base:1.0 <close-session> close
current session
User, Operator,
Admin
Yes
:base:1.0 <kill-session> : Close
other session
User, Operator,
Admin
Yes
:base:1.0 subtree filtering User, Operator,
Admin
Yes
:Startup:1.0 get-config
<source=startup>
User, Operator,
Admin
Yes
:Startup:1.0 copy-config
<source=startup>
Admin No
:Startup:1.0 copy-config
<target=startup>
Admin Partial Running to startup copy
is supported.
:Startup:1.0 lock <startup> Admin Yes
:Startup:1.0 unlock <startup> Admin Yes
(Yes/No)
Comments
:Startup:1.0 validate
<source=startup>
Admin No Always configuration
entries are validated
and stored, but
external configuration
store validation is not
supported.
:Startup:1.0 delete-config Admin Yes

33
Solution Guide
:url:1.0 URL capability User, Operator,
Admin
No Not supported
:with-defaults <get> User, Operator,
Admin
Partial By default, “report-all”
functionality is supported
when with-default
option is passed. Other
options (explicit, tagged,
explicit-tagged) are not
supported
:with-defaults <get-config> User, Operator,
Admin
Partial By default, “report-all”
functionality is supported
when with-default
option is passed. Other
options (explicit, tagged,
explicit-tagged) are not
supported
:confirmed-
commit
<commit> <confirmed> Operator, Admin Yes
:confirmed-
commit
<commit> <confirmed>
<persist>
Operator, Admin Yes

Solution Guide
About IP Infusion
IP Infusion, the leader in disaggregated networking solutions, delivers the best network OS for white box and network
virtualization. IP Infusion offers network operating systems for both physical and virtual networks to carriers, service
providers and enterprises to achieve the disaggregated networking model. With the OcNOS™
and VirNOS™
network
operating systems, IP Infusion offers a single, unified physical and virtual software solution to deploy new services
quickly at reduced cost and with greater flexibility. Over 300 customers worldwide, including major networking equipment
manufacturers, use IP Infusion’s respected ZebOS platform to build networks to address the evolving needs of cloud,
carrier and mobile networking. IP Infusion is headquartered in Santa Clara, Calif., and is a wholly owned and independently
operated subsidiary of ACCESS CO., LTD. Additional information can be found at http://www.ipinfusion.com.
© 2016 IP Infusion, Inc. All rights reserved. ZebOS and IP Infusion are registered trademarks and the ipinfusion logo,
OcNOS and VirNOS are trademarks of IP Infusion, Inc. All other trademarks and logos are the property of their respective
owners. IP Infusion assumes no responsibility for any inaccuracies in this document. IP Infusion reserves the right to
change, modify, transfer, or otherwise revise this publication without notice.
Phone: +1 877-MYZEBOS
Email: sales@ipinfusion.com
Web: www.ipinfusion.com
U.S. (Santa Clara), +1 408-400-1912
Japan (Tokyo), +81 03-5259-3771
Korea (Seoul) +82 (2) 3153-5224
India (Bangalore), +91 (80) 6728 7000
China (Shanghai), +86 186 1658-6466
EMEA (Stockholm), +46 8-566 300 4
IP Infusion
An ACCESS Company
(408) 400-3000
www.ipinfusion.com
3965 Freedom Circle, Suite 200
Santa Clara, CA 95054
ipinfusion™
HM-001

Large Scale Data center Solution Guide: eBGP based design

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Large Scale Data center Solution Guide: eBGP based design

Similar to Large Scale Data center Solution Guide: eBGP based design (20)

More from Dhiman Chowdhury

More from Dhiman Chowdhury (20)

Recently uploaded

Recently uploaded (20)

Large Scale Data center Solution Guide: eBGP based design