This session focuses on a customer case study
in which Network Virtualization has been deployed.

The focus of this session is to cover the actual business requirements of
the customer involved, how Network Virtualization met those requirements,
the network design that was employed, and the benefits that were derived.

Introducing the session will be a brief outline of Cisco's approach to
Network Virtualization design methodology. The customer case study itself
will focus on a Campus Network Virtualization deployment.

Presenting this case study will be Dave Zacks, a Technical Solution Architect
with Cisco Systems. Attendees at this session will learn about virtualized
network deployments, and how these can be used to provide unique and
compelling architectural solutions, addressing both business and technical
requirements.
CNSF - May, 2011 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public 3

#CNSF2011

  Groups and services
Resources Internet are logically separated …
Guest / partner access
Departmental separation
Regulatory compliance (HIPPA, PCI …)
Non-Virtualized Network Building controls, video surveillance
Mergers, acquisitions … and many others

  Closed User Groups …
Virtualized
Network Private
Secure
Independent policies

  Service differentiation
is configured per group / service …
Dept A Dept B Partner Guest The same application
can be unique per group / service

  One physical network
supports many virtual networks
Department Function Guests /
A B Partners

Virtual Virtual Virtual

Actual Physical Network Infrastructure


Service Access Control Path Isolation Services Edge
Branch – Campus WAN – MAN – Campus Data Center – Internet Edge – Campus

GRE MPLS

VRFs

Internet

Functions   Authenticate   Maintain traffic partitions   Provide access
client (user, device, application) over Layer 3 infrastructure to services –
attempting to gain network
  Transport traffic over Shared
access
isolated Layer 3 partitions Dedicated
  Authorize client
into a partition (VLAN)   Map Layer 3 isolated path   Apply policy per partition
to VLANs in access and services edge
  Deny access to   Isolate application
unauthenticated clients environments
if necessary

Per VRF:
Virtual Routing Table
Virtual Forwarding Table
  Device virtualization –
Control plane virtualization VRF
Data plane virtualization VRF
Global
Services virtualization

  Data path virtualization –
Hop-by-Hop – 802.1q
(VRF-Lite End-to-End)
IP
Multi-Hop –
(VRF-Lite+GRE, MPLS VPN)

VRF – Virtual Routing and Forwarding

1.  Create L2 VLANs
and trunk them to the first L3 device
2.  Define VRFs at the first L3 devices (PE) PE
and map the L2 VLANs to the proper VRF
Enable MPLS
3.  Enable MPLS on all
Layer 3 interfaces in the network
Label Switch
Router (LSR)
P P
4.  Enable MP-BGP on the
PE devices to exchange VPN routes
Enable MPLS
PEs become iBGP neighbors
PE
5.  VPN traffic is now carried end-to-end
across the network, maintaining logical
isolation between the defined groups
Each frame is double-tagged
(IGP label + VPN label)


iBGP requires a full mesh of neighbors
172.16.5.0/24 172.16.8.0/24
N * (N-1) / 2 = 8 * 7 / 2 = 28
S1 S2 S3 S4
R1 R4

172.17.6.0/24 172.17.9.0/24

172.18.7.0/24 172.18.10.0/24

For Your
Reference

Ro ute Reflector Route Reflector

172.16.5.0/24 172.16.8.0/24

S1 S2 S3 S4
R1 R4

172.17.6.0/24 172.17.9.0/24

172.18.7.0/24 172.18.10.0/24

For Your
Reference

  Firewall contexts in transparent mode Shared Services
act as L2 bridges E-mail
Storage
  Fusion router establishes routing Web
peering with the various VRFs …

The fusion router has complete knowledge
of all the routes existing in the VRFs EIGRP, OSPF,
eBGP, Static
  The peering protocol may vary (no ISIS)
depending on the path isolation strategy
Use IGP (EIGRP or OSPF)
L2 L2 L2
for VRF-Lite deployments
Use eBGP for MPLS VPN scenarios
  The fusion router could typically
advertise only a default route
into the various VRFs Red VPN Blue GreenVP
VPN N
  A dedicated “Fusion” VRF may be
used in place of an external fusion Campus Core
router device


  Firewall contexts in routed mode Shared Services
act as an L3 hop routing traffic E-mail
between interfaces Storage
Web
No routing protocol support on …
firewall deployed in multi-context mode

  The only recommended peering protocol
eBGP
is eBGP, independently from the
Path Isolation technique adopted
in the Campus
L3 L3 L3
Configuring static routing
is also possible
  The fusion router could typically
advertise only a default route
into the various VRFs Red VPN Blue GreenVP
VPN N
  A dedicated “Fusion” VRF may be used
in place of an external fusion router device Campus Core


Shared Services Global Table

Red VPN Blue Global Table Red VPN Blue GreenVP
VPN VPN N

The global table is considered as another The global table is treated as a shared
VPN (in fact can be usually considered service – access to the global table
the “default VPN”) and it is front-ended from each VPN is subject to the policy
by its own security device enforcement provided by the Services Edge

#CNSF2011

  UBC is one of the largest
Universities in Canada.

  The main UBC Campus is
located in Vancouver, BC
(UBCV).

  UBCV Campus is 1.55 square miles
in extent, covers several hundred
buildings in total.

  Three additional (smaller)
UBC Campuses located around BC
is here
You are here

On a beautiful summer day …
June 25th, 2010

Clock Tower,
I.K Barber
adjacent to the I.K Barber
Learning Centre
Learning Centre,
University of BC
University of BC

  50,332 students – 4,669 faculty – 8,953 staff
Source: www.ubc.ca (as of November, 2008)

  Number of Ethernet switches installed = 2,709
  Number of wired Ethernet ports = approx. 60,000
  Number of VLANs allocated = 3,394
  All core network links are 10 Gigabit Ethernet
  Core network links are Jumbo Frame-enabled
  Number of Wireless APs deployed = approx. 2,000
  Total Internet bandwidth used = approx. 1750Mbps


  UBC has adopted a standardized
Campus network architecture.

  Due to the size of UBC’s Vancouver Campus,
a four-layer network architecture has been used –
Access Layer – Stackable Layer 2 switches
Distribution Layer – Mixture of stackable and chassis
switches, mostly Layer 2 but with some Layer 3 termination
Outer Core Layer – All remaining Layer 3 terminations
Inner Core Layer – Layer 3 routing between
Outer Core platforms


Typical
Large
No deliberate Building
Fully resilient core
Access
Layer 2 loops network, based on
in the network OSPF routing
Distribution

Outer Core Data Center

Inner Core
Additional
Buildings

No VLAN bridging Layer 3 FHRP
across the core – provided by HSRP
routed links only Internet Edge
at Outer Core layer

  UBC departments and faculties use VLANs
to segregate functions within buildings –
servers, students, faculty, staff, admin, labs, etc.

  Any port within a building at UBC can be on
any VLAN within that building (or building complex).

  Some large UBC buildings, holding multiple
departments, have over 100 VLANs.

  Don’t forget – VLANs are a form of virtualization
(at Layer 2)!


  Statement of the Problem –
Nineteen faculties – and over 200 departments.
Several thousand subnets, Campus-wide.

Faculties / departments have space in many buildings.
Example – Faculty of Arts has space in 31 buildings.

Many UBC faculties / departments require secure
connections within, and between, their areas.
To this end, many have implemented
their own firewall solutions.


Proliferation of
different firewall
types

Performance issues
as the Campus
scales to 10GE

Difficult to support
and troubleshoot

Many UBC faculties / departments
want a single, fully redundant
firewall system for their subnets
and VLANs, Campus-wide.
How to accommodate this?


  Technically, it would be possible to
bridge VLANs across the core UBC network –
This has often been requested historically.

  However, in practice this creates many difficulties…
It does not scale well.
The proliferation of Layer 2 loops in a fully-resilient
network design leads to many blocking / forwarding ports –
this is considered to be undesired in UBC’s network design.

  As a consequence, bridging of VLANs
across the core is not supported by UBC.


  To meet their needs, UBC looked towards
a Network Virtualization solution.

  VRFs (Virtual Routing and Forwarding instances) –
provide for a separate IP routing table per
department or faculty – even when
distributed over multiple buildings.

  VRFs provide the required
segmentation – to provide
a “virtual network” for the
department or faculty
involved.

  Within a VRF
High-performance routing for the department / faculty.

  Between VRFs
Traffic flows across Virtual Firewalls, hosted on UBC’s
Catalyst 6500-based Firewall Services Modules (FWSMs).
Each Virtual Firewall can be managed separately
by the department / faculty involved
(aids in scaling manageability).

  No need for VLAN bridging
The UBC core network remains entirely
routed – promotes stability, maintains
core network policies.

  UBC needed a solution that could scale to handle
the number of departments / faculties on Campus.

  The options for a Network Virtualization
deployment include …
- VRF-Lite with GRE Tunnels,
- VRF-Lite End-to-End, or
- MPLS VPNs

  For the scale requirements
of UBC, an MPLS VPN
deployment model was chosen.

  When designing, planning, and
building their Network Virtualization
deployment, UBC used Cisco’s
Network Virtualization
Design Guides extensively …

NV Access Control Design Guide
NV Path Isolation Design Guide
NV Services Edge Design Guide

www.cisco.com/go/designzone


  MPLS VPN uses Multi-Protocol BGP (MP-BGP)
for exchanging MPLS tags (for creating isolated domains).

The first step was
implementing the
Route Reflectors
for MP-BGP.
UBC used a pair
of Cisco 7200
routers for
this purpose.


  The next step was determining where to place the
P (Provider) and PE (Provider Edge) routers.

UBC uses
Catalyst 6509
switches, which
support MPLS.
The Inner Core
6509s are the
P routers.
The Outer Core
6509s are the
PE routers.

  After determining the P and PE router locations,
it was necessary to set up the appropriate routing.

All of the Outer
Core 6509s peer
(MP-BGP) with the
two Route Reflectors –
for MPLS VPN
(VRF) route exchange.

The Inner Core
routers are not
aware of VRFs –
(and do not run MP-BGP –
OSPF only, with MPLS
tagging).

  Providing for Layer 3 VLAN termination,
mapping into VRFs, and First Hop redundancy were the next tasks.

VLANs terminate
at the Outer Core
layer, and are
mapped into
VRFs using SVIs.

HSRP is used
across the PE
router pair for
first-hop redundancy.


  VLAN and VRF definitions at the PE routers …
Note – VRFs are only defined on the PE routers where the VRFs need
to terminate … not all VRFs are defined on all PE routers …

Outer Core (PE), VLANs and VRFs

vlan 627
name VLAN-627-MED-ADMIN VLAN definition – Layer 2
ip vrf MED-ADMIN
rd 65111:521
route-target export 65111:521
route-target import 65111:521
VRF definition – Layer 3

The RD is a 64-bit value Routes are imported
(unique per VRF). When added and exported within the VRF,
to the 32-bit IP address, this forms to populate the VRF’s
a unique 96-bit VPNv4 address. IP routing table.

  SVI definition on the PE router,
showing SVIs with VRF mapping,
and HSRP and DHCP Relay functionality enabled …

Outer Core (PE), VLANs and VRFs Outer Core (PE), SVI configuration
OUTER-2# show run int Vlan 627
vlan 627
name VLAN-627-MED-ADMIN !
interface Vlan627
description VLAN-627-MED-ADMIN
ip vrf MED-ADMIN mtu 8500
rd 65111:521 ip vrf forwarding MED-ADMIN
route-target export 65111:521 ip address 172.16.10.253 255.255.255.0
route-target import 65111:521 ip helper-address 10.10.5.1
ip helper-address 10.10.6.1
standby ip 172.16.10.254
standby priority 105
standby preempt
For DHCP forwarding standby authentication md5 key-string 7 <secret>
(within the VRF)
For HSRP
(within the VRF)

  VLANs are mapped into VRFs via definitions on SVIs.
“show ip vrf” shows all VLAN-to-VRF mappings …

Outer Core (PE), VLANs and VRFs Outer Core (PE), SVI configuration
OUTER-2# show run int Vlan 627
vlan 627
name VLAN-627-MED-ADMIN !
interface Vlan627
description VLAN-627-MED-ADMIN
ip vrf MED-ADMIN mtu 8500
rd 65111:521 ip vrf forwarding MED-ADMIN
route-target export 65111:521 ip address 172.16.10.253 255.255.255.0
route-target import 65111:521 ip helper-address 10.10.5.1
ip helper-address 10.10.6.1
standby ip 172.16.10.254
standby priority 105
OUTER-2# show ip vrf standby preempt
Name Default RD Interfaces standby authentication md5 key-string 7 <secret>

MED-ADMIN 65111:521 Vl185
Vl431 VLANs mapped into the VRF
Vl627 (in this example, 4 VLANs total
Vl2199 in this VRF, on this PE)

  Connected routes and static routes within the VRF
are redistributed into MP-BGP …
making these routes available on other PEs
that also host this VRF …

Outer Core (PE), Connected / Static
interface Vlan185 Connected routes within the VRF
ip vrf forwarding MED-ADMIN (local SVIs on this PE router)
ip address 172.16.2.253 255.255.255.248

interface Vlan431
ip vrf forwarding MED-ADMIN
ip address 172.16.5.253 255.255.255.128

interface Vlan627
ip vrf forwarding MED-ADMIN Static route to virtual firewall
ip address 172.16.10.253 255.255.255.0 (in this example, co-located
interface Vlan2199 on this PE router, via VLAN 185)
ip address 172.16.33.253 255.255.255.192
Named
ip route vrf MED-ADMIN 0.0.0.0 0.0.0.0 172.16.2.249 name DEFAULT-ROUTE=MED-ADMIN Static
Routes

  Connected routes and static routes within the VRF
are redistributed into MP-BGP …
making these routes available on other PEs
that also host this VRF …

Outer Core (PE), Connected / Static Outer Core (PE), Redistribution
interface Vlan185 router bgp 65111
ip vrf forwarding MED-ADMIN !
ip address 172.16.2.253 255.255.255.248 address-family ipv4 vrf MED-ADMIN
redistribute connected
interface Vlan431 redistribute static
ip vrf forwarding MED-ADMIN no synchronization
ip address 172.16.5.253 255.255.255.128 network 0.0.0.0
exit-address-family
interface Vlan627
ip address 172.16.10.253 255.255.255.0
Redistribution of routes
to other PEs (via MP-BGP)
interface Vlan2199
ip address 172.16.33.253 255.255.255.192

ip route vrf MED-ADMIN 0.0.0.0 0.0.0.0 172.16.2.249 name DEFAULT-ROUTE=MED-ADMIN


  Now that we have done all of our VLAN and VRF definitions,
SVI configuration, and route redistribution …
… let’s see the results …

Outer Core (PE), IP Routing Table
OUTER-2# show ip route vrf MED-ADMIN

Routing Table: MED-ADMIN
Codes: C - connected, S - static, R - RIP, M - mobile, B – BGP
. . .
Route via PE –
Gateway of last resort is 172.16.2.249 to network 0.0.0.0
Outer-3
172.16.0.0/16 is variably subnetted, 3 subnets, 3 masks
C 172.16.2.248/29 is directly connected, Vlan185
B
C
172.16.139.192/27 [200/0] via 192.168.1.3, 5w3d
172.16.5.128/25 is directly connected, Vlan431
Route via PE –
C
B
172.16.155.128/26 [200/0] via 192.168.1.5, 5w3d
Outer-5
B 172.16.211.0/26 [200/0] via 192.168.1.5, 5w3d
C 172.16.10.0/24 is directly connected, Vlan627
S* 0.0.0.0/0 [1/0] via 172.16.2.249


  Now we have Campus-wide “virtual networks”
for departments and faculties, on demand!

Outer Core (PE), IP Routing Table
OUTER-2# show ip route vrf MED-ADMIN

Routing Table: MED-ADMIN   Simplified
. . . departmental
Gateway of last resort is 172.16.2.249 to network 0.0.0.0
routing table

C
172.16.0.0/16 is variably subnetted, 3 subnets, 3 masks
172.16.2.248/29 is directly connected, Vlan185   Secure
B
C
172.16.139.192/27 [200/0] via 192.168.1.3, 5w3d
network
C
B
172.16.155.128/26 [200/0] via 192.168.1.5, 5w3d
segmentation
B 172.16.211.0/26 [200/0] via 192.168.1.5, 5w3d
C
S*
0.0.0.0/0 [1/0] via 172.16.2.249   Scalable

Faculty
Virtual Firewall
From this …

To this …


  Network maintenance and troubleshooting tasks
are readily adapted into a multi-VRF environment …

“ping” within a VRF “traceroute” within a VRF

OUTER-2# ping vrf MED-ADMIN 172.16.155.188 OUTER-2# traceroute vrf MED-ADMIN
172.16.155.188
Type escape sequence to abort.
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.155.188, Tracing the route to 172.16.155.188
timeout is 2 seconds:
1 10.1.1.1 [MPLS: Labels 165/27 Exp 0]
!!!!! 4 msec 0 msec 0 msec

Success rate is 100 percent (5/5), 2 172.16.155.188
round-trip min/avg/max = 1/1/1 ms 0 msec 0 msec 0 msec

OUTER-2# OUTER-2#


  Network maintenance and troubleshooting tasks
are readily adapted into a multi-VRF environment …

“show ip arp” within a VRF

OUTER-2# show ip arp vrf MED-ADMIN

Protocol Address Age (min) Hardware Addr Type Interface

Internet 172.16.2.250 204 0034.a984.4c80 ARPA Vlan185
Internet 172.16.2.249 169 0086.c873.d200 ARPA Vlan185
Internet 172.16.2.254 - 0000.0c07.ac00 ARPA Vlan185
Internet 172.16.2.253 - 0018.a4e4.d700 ARPA Vlan185
Internet 172.16.5.253 - 0018.a4e4.d700 ARPA Vlan431
Internet 172.16.33.202 10 0019.e8db.7da3 ARPA Vlan2199
Internet 172.16.5.177 3 001c.f257.4581 ARPA Vlan431
Internet 172.16.10.14 28 001e.a599.b9a7 ARPA Vlan627
Internet 172.16.10.25 1 001a.02ec.3af2 ARPA Vlan627

. . .


Logical Network Flow

PE

P
RED = traffic within the Red VRF
GOLD = MPLS-encapsulated traffic
PE

OUTER-2# show mls cef vrf
MED-ADMIN 172.16.139.92 Tag push IP Header IP payload

IGP Tag VPN Tag IP Header IP payload
Codes: + - Push Label
Prefix Adjacency VPN Tag IP Header IP payload
Tag pop (IGP)
172.16.139.92/27 Te1/1 19(+), 1193(+) Tag pop (VPN), IP Header IP payload
CNSF - May, 2011 © 2011 Cisco and/or its affiliates. All rights reserved. Cisco Public
IP lookup 43

  Traffic between VRFs will traverse a Virtual Firewall –
… these are hosted on UBC’s FWSMs …

UBC operates
multiple FWSMs,
distributed around
Campus.
The default route in
each departmental VRF
points to that VRF’s
redundant Virtual OUTER-2# show ip route vrf MED-ADMIN

Firewall instance … Routing Table: MED-ADMIN
… hosted on an . . .

FWSM pair (on the PEs). Gateway of last resort is 172.16.2.249 to network 0.0.0.0
...
S* 0.0.0.0/0 [1/0] via 172.16.2.249


PE

Virtual FW

Policy
PURPLE = IP routed traffic (via OSPF)
PE
GREEN = traffic within the Green VRF

P

PE

Virtual FW

Policy

PE



Border

P
PURPLE = IP routed traffic (via OSPF)

PE

Virtual FW

Policy

PE


  Previously, UBC did not provide Campus-wide multicast –
… inability to securely limit multicast use.

  Now with Network Virtualization, UBC can
provision multicast support in some VRFs, and not in others.

  UBC employs both intra-VRF (MVPN) and inter-VRF
(Extranet MVPN) in their Campus multicast deployment.

  Benefit – UBC can now securely enable
multicast on-demand, within and between
VRFs, Campus-wide.
This opens up many new possibilities
for educational data delivery.

  UBC is virtualizing their Data Center –
using industry-leading virtualized load-balancing,
Virtual Machine, and virtualized storage technologies.

  Virtual Load Balancing –
UBC is leveraging the capabilities of their ACE
platforms to create per-department / faculty virtual
load balancer instances, associated with the
department / faculty VRFs and Virtual Firewalls.

  Benefit – allows for individualized,
high-performance load-balancing,
with separated management
(aids in scaling, manageability,
and solution customization).

  Virtual Machines – can be mapped into VLANs …
which are mapped into VRFs …
Direct VM access, at high speed, from anywhere on Campus
VMs within the DC can be protected behind
departmental / faculty Virtual Firewalls as needed.

  New virtualized services can be spun up on demand
in the DC, and mapped out to departments and faculties –
Fully secured by the UBC virtualized network infrastructure.

  Benefit – allows rapid adoption of VM services –
Hosted in the UBC Data Center, securely integrated Campus-wide.


  Virtual Storage – UBC is using Nexus 5000 switches
and IP-based NAS storage to front-end their SAN.
This NAS storage can be virtualized, mapped into VLANs / VRFs,
provided to Virtual Machines, and located behind virtual firewalls.

  This allows UBC to deploy virtualized, IP-based storage –
Mapped into their virtualized Campus and
Data Center network topology, for secure,
flexible access.

  Benefit – allows high-performance,
virtualized, IP-accessible storage
Securely integrated into, and made available
seamlessly across, the UBC virtualized
network architecture.

  UBC is examining the use of IPv6 on Campus –
Small deployments underway, with a larger Campus deployment under
consideration / in planning.

Catalyst 6500
core switches
offer support
for 6VPE
capability.

Cisco Catalyst 6500:
Building IPv6-Ready Campus Networks -

http://www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/solution_overview_c22-531339.html

  Response from UBC Faculties and Departments –

Demand for UBC’s virtualized network deployment
has been strong since inception in January, 2009.

Departments and faculties have seen the advantages
of virtualized networking, and are embracing the
technical and business benefits which UBC’s
virtualized network infrastructure provides.

UBC is also employing virtualization for
additional Campus services and functions.


  UBC has derived many technical benefits
from their Network Virtualization deployment …
Scalable deployment for departmental / faculty segmentation.

End-to-end high performance –
High-performance IP / MPLS routing within a VRF.
Ability to consolidate and scale virtualized
firewalls, for traffic moving between VRFs.

Ability to securely deploy multicast
(MVPN / Extranet MVPN).

Ability to securely roll out
a suite of virtualized DC services,
for high-performance, secure data access.

  UBC has derived many business benefits
from their Network Virtualization deployment …
The network design reflects UBC organizational boundaries.
For the first time, security is
an integral part of network deployment.

The network design can scale to 10Gbps+,
low-performance firewalls are no longer a constraint.

UBC departments and faculties can locate services anywhere
on Campus (including within the UBC DC), and enjoy the
same secure, high-performance, seamless access.

Economies of scale, “greener” –
using centralized services.

  Published –
Summer
2010

www.cisco.com/en/US/prod/collateral/switches/ps5718/ps708/case_study_c36-609342.html

www.cisco.com/go/networkvirtualization

#CNSF2011

Thank you.

#CNSF2011

Network Infrastructure Virtualization Case Study

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (14)

Similar to Network Infrastructure Virtualization Case Study

Similar to Network Infrastructure Virtualization Case Study (20)

More from Cisco Canada

More from Cisco Canada (20)

Recently uploaded

Recently uploaded (20)

Network Infrastructure Virtualization Case Study