MetaFabric Architecture Virtualized Data Center Design Guide

MetaFabric™ Architecture Virtualized
Data Center
Design and Implementation Guide
Release
1.0
Published: 2014-03-18
Copyright © 2014, Juniper Networks, Inc.

Juniper Networks, Inc.
1194 North Mathilda Avenue
Sunnyvale, California 94089
USA
408-745-2000
www.juniper.net
Copyright © 2014, Juniper Networks, Inc. All rights reserved.
Juniper Networks, Junos, Steel-Belted Radius, NetScreen, and ScreenOS are registered trademarks of Juniper Networks, Inc. in the United
States and other countries. The Juniper Networks Logo, the Junos logo, and JunosE are trademarks of Juniper Networks, Inc. All other
trademarks, service marks, registered trademarks, or registered service marks are the property of their respective owners.
Juniper Networks assumes no responsibility for any inaccuracies in this document. Juniper Networks reserves the right to change, modify,
transfer, or otherwise revise this publication without notice.
[Insert Series Title] [Insert Book Title]
Copyright © 2014, Juniper Networks, Inc.
All rights reserved.
The information in this document is current as of the date on the title page.
YEAR 2000 NOTICE
Juniper Networks hardware and software products are Year 2000 compliant. Junos OS has no known time-related limitations through the
year 2038. However, the NTP application is known to have some difficulty in the year 2036.
END USER LICENSE AGREEMENT
The Juniper Networks product that is the subject of this technical documentation consists of (or is intended for use with) Juniper Networks
software. Use of such software is subject to the terms and conditions of the End User License Agreement (“EULA”) posted at
http://www.juniper.net/support/eula.html. By downloading, installing or using such software, you agree to the terms and conditions of
that EULA.
ii Copyright © 2014, Juniper Networks, Inc.

Table of Contents
Part 1 MetaFabric™ Architecture Virtualized IT Data Center Design and
Implementation Guide
Chapter 1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
MetaFabric Architecture Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Validated Solution Design and Implementation Guide Overview . . . . . . . . . . . . . . 8
MetaFabric 1.0 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Solution Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Compute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Class of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Network Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Chapter 2 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Design Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Design Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Design Topology Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Design Highlights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
Solution Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Compute . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Virtual Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Hypervisor Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Blade Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Access and Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Core Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Edge Routing and WAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Compute Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Network Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Network Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Business-Critical Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Copyright © 2014, Juniper Networks, Inc. iii

MetaFabric™ Architecture Virtualized Data Center
High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Hardware Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
Software Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Class of Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
Application Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
Perimeter Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Secure Remote Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Network Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Out-of-Band Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Network Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Security Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Performance and Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Summary of Key Design Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Chapter 3 MetaFabric 1.0 High Level Testing and Validation Overview . . . . . . . . . . . . . 61
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
Key Characteristics of Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
POD1 (QFX3000-M QFabric) Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 63
POD2 (QFX3000-M QFabric) Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Core Switch (EX9214) Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Edge Firewall (SRX3600) Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Edge routers (MX240) Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
Compute (IBM Flex chassis) Implementation . . . . . . . . . . . . . . . . . . . . . . . . . 65
OOB-Mgmt (EX4300-VC) Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 65
Hardware and Software Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Chapter 4 Transport (Routing and Switching) Configuration . . . . . . . . . . . . . . . . . . . . . 73
Network Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Configuring the Network Between the Data Center Edge and the Data Center
Core . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Implementing MC-LAG Active/Active with VRRP . . . . . . . . . . . . . . . . . . . . . . . . . . 85
Summary of Implementation Details for MC-LAG Active/Active . . . . . . . . . . 85
MC-LAG Configuration for Better Convergence . . . . . . . . . . . . . . . . . . . . . . . . 86
Configuring the Network Between the Data Center Core and the Data Center
PODs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
Routing Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
Configuring BGP Between the EDGE and Service Provider . . . . . . . . . . . . . . 106
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Configuring OSPF in the Data Center . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
iv Copyright © 2014, Juniper Networks, Inc.

Table of Contents
Chapter 5 High Availability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
High Availability Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Hardware Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Software Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
QFabric-M Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
Configurint the Core and Edge Router . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
Configuring the Perimeter Firewall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
Chapter 6 Class-of-Service Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Class-of-Service Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Configuring Class-of-Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Configuring Class-of-Service (POD Level) . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Configuring Data Center Bridging and Lossless Ethernet . . . . . . . . . . . . . . . . . . . 130
Configuring Class-of-Service (POD Level) . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
Chapter 7 Security Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
Perimeter Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
Configuring Chassis Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
Configure Chassis Clustering Data Fabric . . . . . . . . . . . . . . . . . . . . . . . . 138
Configuring Chassis Clustering Groups . . . . . . . . . . . . . . . . . . . . . . . . . . 139
Configuring Chassis Clustering Redundancy Groups . . . . . . . . . . . . . . . 139
Configuring Chassis Clustering Data Interfaces . . . . . . . . . . . . . . . . . . . 140
Configuring Chassis Clustering – Security Zones and Security Policy . . . 141
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Configuring Network Address Translation . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Configure Source NAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
Configure Destination NAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
Configuring Intrusion Detection and Prevention . . . . . . . . . . . . . . . . . . . . . . 148
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
Host Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
Configuring the Firefly Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
Chapter 8 Data Center Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Data Center Services Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Configuring Compute Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
Compute Hardware Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Configuring Compute Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
Configuring Compute Nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
Configuring POD to Pass-thru Chassis Compute Nodes . . . . . . . . . . . . . 172
Configuring the CNA Fabric Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
Copyright © 2014, Juniper Networks, Inc. v

Configuring the 10Gb CNA Module Connections . . . . . . . . . . . . . . . . . . . 181
Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186
Virtualization Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
Configuring LACP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
Configuring VMware Clusters, High Availability, and Dynamic Resource
Scheduler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Configuring VMware Enhanced vMotion Compatibility . . . . . . . . . . . . . . . . . 197
Mounting Storage Using the iSCSI Protocol . . . . . . . . . . . . . . . . . . . . . . . . . 200
Configuring Fault Tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
Configuring VMware vMotion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
EMC Storage Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
Configuring EMC Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
Configuring EMC FAST Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Configuring FAST Cache . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Configuring Storage Pools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
Configuring Logical Unit Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
Enabling Storage Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
Configuring the Network File System . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
Configuring VNX Snapshot Replicas . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
Configuring Redundancy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
Configuring the Link and Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Configuring VIP and Server Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
Load-Balanced Traffic Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Microsoft Exchange Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
Installation Checklist and Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Deploying Network for Exchange VM . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
Configuring Storage for Exchange VM . . . . . . . . . . . . . . . . . . . . . . . . . . 242
Enabling Storage Groups with Unisphere . . . . . . . . . . . . . . . . . . . . . . . . 245
Provisioning LUNs to ESXi Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
Configuring vMotion Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
Chapter 9 Network Management and Orchestration . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
Configuring Junos Space with Network Director . . . . . . . . . . . . . . . . . . . . . . 268
Configuring VM Orchestration in the Network Director 1.5 Virtual View . . . . 269
Network Director Provisioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
Configuring Class of Service Using Network Director . . . . . . . . . . . . . . . . . . . 272
Creating VLANs Using Network Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
Setting Up QFabric Using Network Director . . . . . . . . . . . . . . . . . . . . . . . . . . 275
Setting Up QFabric Using Network Director – Ports and VLAN . . . . . . . . . . . 277
Setting Up a QFabric System Using Network Director – Create Link
Aggregation Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
vi Copyright © 2014, Juniper Networks, Inc.

Table of Contents
Network Director – Downloading and Upgrading Software Images . . . . . . . 283
Network Director – Monitoring the QFabric System . . . . . . . . . . . . . . . . . . . 285
Configuring Security Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
Discovery and Basic Configuration Using Security Director . . . . . . . . . . 288
Resolving DMI Mismatch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
Object Builder (Using Security Director) . . . . . . . . . . . . . . . . . . . . . . . . . 291
Creating Firewall Policy Using Security Director . . . . . . . . . . . . . . . . . . . 292
Creating NAT Policy Using Security Director . . . . . . . . . . . . . . . . . . . . . . 294
Jobs Workspace in Security Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
Audit Logs in Security Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
Chapter 10 Solution Scale and Known Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
Overview of Solution Scale Known Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
Known Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
Copyright © 2014, Juniper Networks, Inc. vii

List of Figures
Figure 1: Applications Drive IT Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Figure 2: Data Center Before MetaFabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Figure 3: Data Center After MetaFabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Figure 4: MetaFabric – Putting It All Together . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Figure 5: MetaFabric Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Figure 6: Juniper Networks Virtualized IT Data Center - Sizing Options . . . . . . . . . . 9
Figure 7: Juniper Networks Virtualized IT Data Center – Solution
Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Figure 8: Network Management Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
Chapter 2 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Figure 9: Virtualized IT Data Center Ecosystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
Figure 10: Virtualized IT Data Center Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
Figure 11: Virtual Machine Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Figure 12: Server Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Figure 13: VMware Distributed Virtual Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Figure 14: VMware Network I/O Control Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Figure 15: Sample Blade Switch, Rear View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Figure 16: Juniper Networks QFabric Systems Enable a Flat Data Center
Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Figure 17: Core Switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Figure 18: Core Switching Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
Figure 19: Edge Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
Figure 20: Edge Routing Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
Figure 21: Storage Lossless Ethernet Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Figure 22: Storage Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Figure 23: Virtualized IT Data Center Solution Software Stack . . . . . . . . . . . . . . . 38
Figure 24: MC-LAG – ICCP and ICL Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Figure 25: VRRP and MC-LAG – Active/Active Option . . . . . . . . . . . . . . . . . . . . . . 43
Figure 26: MC-LAG – MAC Address Synchronization Option . . . . . . . . . . . . . . . . . 44
Figure 27: MC-LAG – Traffic Forwarding Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
Figure 28: MC-LAG – ICCP Down . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
Figure 29: MC-LAG – ICL Down . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Figure 30: MC-LAG – Peer Down . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Figure 31: Class of Service – Classification and Queuing . . . . . . . . . . . . . . . . . . . . 47
Figure 32: Class of Service – Buffer and Transmit Design . . . . . . . . . . . . . . . . . . . 48
Figure 33: Physical Security Compared to Virtual Network Security . . . . . . . . . . . 49
Copyright © 2014, Juniper Networks, Inc. ix

Figure 34: Application Security Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Figure 35: Physical Security Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
Figure 36: Remote Access Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
Figure 37: Seven Tier Model of Network Management . . . . . . . . . . . . . . . . . . . . . . 54
Figure 38: Out of Band Management Network Design . . . . . . . . . . . . . . . . . . . . . . 56
Figure 39: Out of Band Management – Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
Figure 40: The End to End Lab Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
Figure 41: MC-LAG Active/Active Logical Topology . . . . . . . . . . . . . . . . . . . . . . . . 69
Figure 42: Topology of Core-to-POD Roles in the Data Center . . . . . . . . . . . . . . . . 71
Figure 43: Configuration of RETH Interfaces and MC-LAG Between Core and
Perimeter (Right) Compared to Configuration of RETH Interfaces and AE
(Left) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
Figure 44: Interface Configuration Between Edge, Perimeter, and Core . . . . . . . . . 75
Figure 45: MetaFabric 1.0 Routing Configuration and Topology . . . . . . . . . . . . . . 105
Figure 46: OSPF Area Configuration Between Edge and Core (Including
Out-of-Band Management) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Figure 47: OSPF Area Configuration Between Core and PODs . . . . . . . . . . . . . . . 112
Figure 48: Loop-Free Alternate Convergence Example . . . . . . . . . . . . . . . . . . . . . 114
Figure 49: The VDC POD and Compute/Storage Topology . . . . . . . . . . . . . . . . . . 131
Chapter 7 Security Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
Figure 50: Logical View of Juniper Networks Firefly Host Installation . . . . . . . . . . 155
Figure 51: An Example dvPort Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
Figure 52: Configure an Application Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
Figure 53: The Annotation Allows Firefly Host to Detect Related VMs . . . . . . . . . 158
Figure 54: Define Security Policies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
Chapter 8 Data Center Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
Figure 55: Compute and Virtualization as Featured in the MetaFabric 1.0
Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Figure 56: IBM x3750 M4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
Figure 57: IBM Flex System Enterprise Chassis (Front View) . . . . . . . . . . . . . . . . 166
Figure 58: IBM Flex System (Rear View) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
Figure 59: IBM Flex System Fabric CN4093 10Gb/40Gb Converged Scalable
Switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
Figure 60: IBM Flex System EN4091 10Gb Ethernet Pass-thru Module . . . . . . . . 169
Figure 61: IBM Flex System x220 Compute Node . . . . . . . . . . . . . . . . . . . . . . . . . 170
Figure 62: IBM Pure Flex Pass-thru Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
Figure 63: POD1 Topology with the IBM Pure Flex Chassis + 40Gbps CNA
Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
Figure 64: POD 2 Topology Using the IBM Pure Flex System Chassis with the
10-Gbps CNA I/O Module . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
Figure 65: VMware vSphere Client Manages vCenter Server Which in Turn
Manages Virtual Machines in the Data Center . . . . . . . . . . . . . . . . . . . . . . . . 187
Figure 66: VMWare vSphere Distributed Switch Topology . . . . . . . . . . . . . . . . . . 188
x Copyright © 2014, Juniper Networks, Inc.

List of Figures
Figure 67: VMware vSphere Distributed Switch Topology . . . . . . . . . . . . . . . . . . 189
Figure 68: Log In to vCenter Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
Figure 69: vCenter Web Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Figure 70: Click Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
Figure 71: Click Related Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
Figure 72: Click Uplink Ports and Select a Port . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
Figure 73: Enable LACP Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
Figure 74: Infra Cluster Hosts Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Figure 75: POD1 Cluster Hosts Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Figure 76: POD2 Cluster Hosts Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
Figure 77: INFRA Cluster VMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
Figure 78: POD1 Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
Figure 79: POD2 Cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
Figure 80: Port Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
Figure 81: Port Group and NIC Teaming Example . . . . . . . . . . . . . . . . . . . . . . . . . 199
Figure 82: Configure Teaming and Failover . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
Figure 83: POD1 PG-STORAGE-108 Created for iSCSI . . . . . . . . . . . . . . . . . . . . . 201
Figure 84: VMware Fault Tolerance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
Figure 85: VMware Fault Tolerance on POD1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
Figure 86: VMware vMotion Enables Virtual Machine Mobility . . . . . . . . . . . . . . 203
Figure 87: VMware vMotion Configured in the Test Lab . . . . . . . . . . . . . . . . . . . . 204
Figure 88: EMC FAST Cache Configuration (Select System, then Properties in
the Drop-Down) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
Figure 89: EMC FAST Cache Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
Figure 90: Pool 1 - Exchange-DB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
Figure 91: Selected Storage Pool Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
Figure 92: Storage Pool Disks Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
Figure 93: Storage Pool Properties, Advanced Tab . . . . . . . . . . . . . . . . . . . . . . . 208
Figure 94: VM-Pool Selected . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
Figure 95: VM-Pool Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
Figure 96: VM-Pool Disk Membership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
Figure 97: Exchange-DB-LUN Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
Figure 98: LUN Created for All ESX Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
Figure 99: The Selected Pool Was Created for MS Exchange Logs . . . . . . . . . . . . 213
Figure 100: Exchange Logs the LUN Created . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
Figure 101: Example Storage Group Properties Window . . . . . . . . . . . . . . . . . . . . 215
Figure 102: LUN Added to Storage Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
Figure 103: ESXi Hosts Added to Storage Group . . . . . . . . . . . . . . . . . . . . . . . . . . 217
Figure 104: Add LUNs to Storage Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
Figure 105: NFS Pool Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
Figure 106: LUN Created on the New Storage Pool . . . . . . . . . . . . . . . . . . . . . . . . 219
Figure 107: NFS Pool Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
Figure 108: NFS Export Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
Figure 109: Snapshot Configuration Wizard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
Figure 110: Select Source Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
Figure 111: Select Snapshot Target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
Figure 112: Select Source LUNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
Figure 113: Select Snapshot Storage Overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
Figure 114: Choose When to Create LUN Snapshot . . . . . . . . . . . . . . . . . . . . . . . . 225
Copyright © 2014, Juniper Networks, Inc. xi

Figure 115: Assign Snapshot to a Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
Figure 116: Summary of Snapshot Wizard Configuration . . . . . . . . . . . . . . . . . . . 227
Figure 117: Load Balancing Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
Figure 118: Configure nPath . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
Figure 119: Verify Objects during nPath Configuration . . . . . . . . . . . . . . . . . . . . . . 233
Figure 120: Configure and Verify VIP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Figure 121: Load-Balancing Traffic Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
Figure 122: Home > Inventory > Networking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
Figure 123: Create New Port Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
Figure 124: Modify Teaming Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
Figure 125: PG-STORAGE-108 Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
Figure 126: PG-STORAGE-208 Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242
Figure 127: EMC Unisphere Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
Figure 128: Create Storage Pool Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
Figure 129: FAST Cache enabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
Figure 130: Exchange-DB LUN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
Figure 131: Storage Group Created . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
Figure 132: Storage Group Properties - LUNs Tab . . . . . . . . . . . . . . . . . . . . . . . . . 247
Figure 133: Hosts Allowed to Access the Storage Group . . . . . . . . . . . . . . . . . . . 248
Figure 134: Add LUN to Storage Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 248
Figure 135: Manage Virtual Adapters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
Figure 136: Add New VMkernel Port . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
Figure 137: Select VMkernel as Adapter Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
Figure 138: Select Port Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
Figure 139: VMkernel IP Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
Figure 140: Install iSCSI Software Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
Figure 141: iSCSI Initiator Is Enabled . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
Figure 142: iSCSI Initiator Network Configuration . . . . . . . . . . . . . . . . . . . . . . . . . 254
Figure 143: Add iSCSI Server Location in Dynamic Discovery . . . . . . . . . . . . . . . . 255
Figure 144: LUN Present on the Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
Figure 145: Add Storage from vSphere Client . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
Figure 146: Select Disk/LUNfor Storage Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
Figure 147: Select LUN to Mount . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
Figure 148: Select VMFS-5 as a File System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
Figure 149: Name the Datastore . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
Figure 150: Datastore Creation Complete . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
Figure 151: Create New VM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
Figure 152: VM Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
Figure 153: Give the VM a Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
Figure 154: Select Storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
Figure 155: Select Operating System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
Figure 156: Configure Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
Figure 157: Select Virtual Disk Size . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
Figure 158: Virtual Machine with Additional Disks and Network Adapters . . . . . 266
Chapter 9 Network Management and Orchestration . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
Figure 159: The OOB Management Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
Figure 160: Select IP address, IP Range, IP-Subnet, or HostName . . . . . . . . . . . 269
Figure 161: Configure Virtual Network target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
xii Copyright © 2014, Juniper Networks, Inc.

List of Figures
Figure 162: Enable Orchestration Mode in Network Director . . . . . . . . . . . . . . . . 270
Figure 163: Configure Device Common Settings . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Figure 164: Change in Pending Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271
Figure 165: Change in Pending Deployment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
Figure 166: Select the Data Center Switching Device Family . . . . . . . . . . . . . . . . 272
Figure 167: Select the Profile "Hierarchal Port Switching (ELS)" . . . . . . . . . . . . . 273
Figure 168: Enable PFC Code-point and Queue for NO-LOSS Behavior . . . . . . . 273
Figure 169: COS Profile Deployed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
Figure 170: Create VLAN-ID and VLAN Name . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
Figure 171: Configure Layer 2 Filters and MAC Move Limit . . . . . . . . . . . . . . . . . . . 275
Figure 172: VLAN Profile ND-Test1Created . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
Figure 173: Select Setup QFabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
Figure 174: Configure Device Aliases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
Figure 175: Configure Node Group Type RNSG . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
Figure 176: Configure Center Switching Non ELS . . . . . . . . . . . . . . . . . . . . . . . . . . 277
Figure 177: Configure VLAN Service, Port, CoS, and so on . . . . . . . . . . . . . . . . . . . 278
Figure 178: Port Profile Created (NDTestport) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
Figure 179: Assign Port Profile to Available Port . . . . . . . . . . . . . . . . . . . . . . . . . . 279
Figure 180: Assign Port Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
Figure 181: Click Assign . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
Figure 182: New Physical Port Added to Port Profile List . . . . . . . . . . . . . . . . . . . 280
Figure 183: Port Profile Created Successfully . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
Figure 184: Check to Confirm Port Profile Is Pending . . . . . . . . . . . . . . . . . . . . . . . 281
Figure 185: Select Deploy Now . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
Figure 186: Add New Port Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
Figure 187: Select Devices to Add as LAG Member Links . . . . . . . . . . . . . . . . . . . 283
Figure 188: Links Selected to Be LAG Member Links . . . . . . . . . . . . . . . . . . . . . . 283
Figure 189: Network Director Image Repository . . . . . . . . . . . . . . . . . . . . . . . . . . 284
Figure 190: Image Staging on Network Director . . . . . . . . . . . . . . . . . . . . . . . . . . 284
Figure 191: Stage Image to Device for Install or for Later Installation . . . . . . . . . . 285
Figure 192: Select Image to Stage to Remote Device . . . . . . . . . . . . . . . . . . . . . . 285
Figure 193: Device Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
Figure 194: QFabric Traffic Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
Figure 195: Hardware Dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
Figure 196: Confirmation of Run Fabric Analyzer Operation . . . . . . . . . . . . . . . . . 287
Figure 197: DMI Mismatch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 289
Figure 198: DMI Schema Repository Requires Authentication . . . . . . . . . . . . . . . 290
Figure 199: Security Zone Creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
Figure 200: Address Object Creation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
Figure 201: New Rule Created (Test-1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
Figure 202: Add New Source Address to Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
Figure 203: Example NAT Policies in Security Director . . . . . . . . . . . . . . . . . . . . . 296
Copyright © 2014, Juniper Networks, Inc. xiii

List of Tables
Table 1: Juniper Networks Virtualized IT Data Center – Details of Sizing
Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Chapter 2 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Table 2: MetaFabric 1.0 Solution Design Highlights . . . . . . . . . . . . . . . . . . . . . . . . 20
Table 3: Comparison of Pass-Through Blade Servers and Oversubscribed Blade
Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
Table 4: Core Switch Hardware - Comparison of the EX9200 and EX8200
Switches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Table 5: Core Switch Forwarding - Comparison of MC-LAG and Virtual
Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Table 6: Comparison of Storage Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
Table 7: Application Security Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
Table 8: Data Center Remote Access Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
Table 9: Summary of Key Design Elements – Virtualized IT Data Center
Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Table 10: Hardware and Software deployed in solution testing . . . . . . . . . . . . . . . 66
Table 11: Software deployed in MetaFabric 1.0 test bed . . . . . . . . . . . . . . . . . . . . . 67
Table 12: Networks and VLANs Deployed in the Test Lab . . . . . . . . . . . . . . . . . . . . 67
Table 13: Applications Tested in the MetaFabric 1.0 Solution . . . . . . . . . . . . . . . . . 68
Table 14: MC-LAG Configuration Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
Table 15: IRB, IP Address Mapping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
Table 16: MC-LAG Settings Between Core 1 and Edge 1 . . . . . . . . . . . . . . . . . . . . . 76
Table 17: MC-LAG Between Core 1 and Edge 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
Table 18: MetaFabric 1.0 Class-of-Service Queues . . . . . . . . . . . . . . . . . . . . . . . . 130
Chapter 10 Solution Scale and Known Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
Table 19: Application Scale Targets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
Copyright © 2014, Juniper Networks, Inc. xv

PART 1
MetaFabric™ Architecture Virtualized IT
Data Center Design and Implementation
Guide
• Overview on page 3
• Design on page 17
• MetaFabric 1.0 High Level Testing and Validation Overview on page 61
• Transport (Routing and Switching) Configuration on page 73
• High Availability on page 123
• Class-of-Service Configuration on page 129
• Security Configuration on page 137
• Data Center Services on page 161
• Network Management and Orchestration on page 267
• Solution Scale and Known Issues on page 299
Copyright © 2014, Juniper Networks, Inc. 1

CHAPTER 1
Overview
The benefits of virtualization are driving data center operators to rethink their legacy data
center networks and look for new ways to reduce costs and improve efficiency in the
data center. Moving from a legacy network to a state-of-the-art solution allows you to
deploy new applications in seconds rather than days, weeks, or months. If you want to
harness the power of virtualization in your data center network, this guide will help you
to achieve your goal.
• MetaFabric Architecture Overview on page 3
• Domain on page 5
• Goals on page 6
• Audience on page 7
• Validated Solution Design and Implementation Guide Overview on page 8
• MetaFabric 1.0 Overview on page 8
• Solution Overview on page 10
MetaFabric Architecture Overview
Cloud, mobility, and big data are driving business change and IT transformation. Enterprise
businesses and service providers across all industries are constantly looking for a
competitive advantage, and reliance on applications and the data center have never
been greater (Figure 1 on page 3).
Figure 1: Applications Drive IT Transformation
Traditional networks are physically complex, difficult to manage, and not suited for the
dynamic application environments prevalent in today’s data centers. Because of mergers,

acquisitions, and industry consolidation, most businesses are dealing with data centers
that are distributed across multiple sites and clouds, which adds even more complexity.
Additionally, the data center is so dynamic because the network is constantly asked to
do more, become more agile, and support new applications while ensuring integration
with legacy applications. Consequently, this dynamic environment requires more frequent
refresh cycles.
The network poses two specific problems in the data center:
1. Impedes time to value—Network complexity gets in the way of delivering data center
agility.
2. Low value over time—Every time a new application, technology, or protocol is
introduced, the network needs to be ripped out and replaced.
The growing popularity and adoption of switching fabrics, new protocols, automation,
orchestration, security technologies, and software-defined networks (SDNs) are strong
indicators of the need for a more agile network in the data center. Juniper Networks has
applied its networking expertise to the problems of today’s data centers to develop and
deliver the MetaFabric™ architecture—a combination of switching, routing, security,
software, orchestration, and SDN—all working in conjunction with an open technology
ecosystem to accelerate the deployment and delivery of applications for enterprises and
service providers.
With legacy data center networks, you needed to create separate physical and virtual
resources at your on-premises data center, your managed service provider, your hosted
service provider, and your cloud provider. All of these resources required separate
provisioning and management (Figure 2 on page 4).
Figure 2: Data Center Before MetaFabric
Now, implementing a MetaFabric architecture allows you to combine physical and virtual
resources across boundaries to provision and manage your data center efficiently and
holistically (Figure 3 on page 5).

Figure 3: Data Center After MetaFabric
Chapter 1: Overview
The goal of the MetaFabric architecture is to allow you to connect any physical network,
with any combination of storage, servers, or hypervisors, to any virtual network, and with
any orchestration software (Figure 4 on page 5). Such an open ecosystem ensures that
you can add new equipment, features, and technologies over time to take advantage of
the latest trends as they emerge.
Figure 4: MetaFabric – Putting It All Together
The MetaFabric architecture addresses the problems common in today’s data center by
delivering a network and security architecture that accelerates time to value, while
simultaneously increasing value over time. The MetaFabric 1.0 virtualized IT data center
solution described in this guide is the first implementation of the MetaFabric architecture.
Future solutions and guides are planned, including a larger scale virtualized IT data center,
IT as a service (ITaaS), and a massively scalable cloud data center.
Domain
This guide addresses the needs that enterprise companies have for an efficient and
integrated data center. It discusses the design and implementation aspects for a complete
suite of compute resources, network infrastructure, and storage components that you
need to implement and support a virtualized environment within your data center. This
guide also discusses the key customer requirements provided by the solution, such as
business-critical applications (such as Microsoft Exchange and SharePoint), high
availability, class of service, security, and network management.

Goals
The primary goal of this solution is to enable data center operators to design and
implement an IT data center that supports a virtualized environment for large Enterprise
customers. The data center scales up to 2,000 servers and 20,000 virtual machines
(VMs) that run business-critical applications.
The MetaFabric 1.0 solution provides a simple, open, and smart architecture and solves
several challenges experienced in traditional data centers:
• Complexity—Typically, legacy data centers have been implemented in an incremental
fashion with whatever vendor gave them the best deal. The result is that the architecture
provides no end-to-end services or management. The solution is to reduce the
complexity and make the data center simple to operate and manage.
• Cost—The cost of managing a complex data center can be high. The solution is to
create an open data center to drive operational efficiencies and reduce cost.
• Rigidity—Building a data center based on incremental demands ultimately results in
an architecture that is too rigid and not able to adapt to new workloads or provide the
agility that an evolving business demands. The solution is to create a smart architecture
from the beginning that can adapt and be agile to new demands.
Figure 5: MetaFabric Architecture
Examples of how the MetaFabric architecture solves real-world problems include:
• Simple—This solution uses two QFabric systems. Each QFabric system acts like a
single, very large switch and only requires one management IP address for 16 racks of
equipment. In effect, management tasks are reduced by over 90%.
• Open—Juniper Networks devices use standards-based Layer 3 protocols and interact
with VMware vCenter APIs. In addition, this solution includes interoperability with
ecosystem partners such as VMware, EMC, IBM, and F5 Networks.
• Smart—In this solution, smart workload mobility with automated orchestration and
template-based provisioning is provided by using Network Director.
The features in a simple, open, and smart architecture in your data center include:

Chapter 1: Overview
• Integrated solution—By designing a data center with integration in mind, you can blend
heterogeneous equipment and software from multiple vendors into a comprehensive
system. This enables your network to interact efficiently with compute and storage
components that work well together.
• Seamless VM Mobility—By designing an architecture that supports the movement of
VMs from one location in the data center to another, VMs can be stopped, moved, and
restarted in a different location in the data center with flexibility and ease.
• Network visibility—By designing a data center to provide VM visibility, you can connect
the dots between the virtual and physical components in your system. You will know
how your VMs are connected to switches and understand the vMotion history of a VM.
• Scale and virtualization—The solution scales to 20,000 VMs and can support either a
100 percent virtualized compute environment or a mixed physical and virtual
environment.
Benefits of the solution include:
• Peace of mind—Knowing that a solution has been tested and validated reduces the
anxiety of implementing a new IT project. This solution provides peace of mind because
it has been thoroughly tested by the Juniper Networks Solutions Validation team.
• Reduce deployment rime—Integrating products from multiple vendors takes time and
effort, resulting in lost productivity caused by interoperability issues. This solution
eliminates such issues because the interoperability and integration has already been
verified by the Juniper Networks Solutions Validation team.
• Reduce CAPEX—Capital expenditures go up when different equipment is added in a
piecemeal fashion and needs to be replaced or upgraded to achieve new business
goals. This solution factors in the goals and scalability ahead of time, resulting in lower
cost of ownership.
• Best of breed—Another pitfall of buying equipment in an incremental fashion is that
legacy equipment often cannot scale to the same levels as newer equipment. This
solution selects cutting-edge equipment that is designed to work together seamlessly
and in harmony.
• Pre-packaged solution—Having to design, evaluate, and test a data center
implementation from a variety of vendors is a lot of work. This solution takes the
guesswork out of such an effort and provides a cohesive set of products designed to
meet your business needs for your data center.
Audience
This MetaFabric 1.0 solution is designed for enterprise IT departments that wish to build
a complete end-to-end data center that contains compute, storage, and network
components optimized for a virtualized environment. The enterprise IT data center
segment represents the majority of Fortune 500 companies.
The primary audience for this guide includes the following technical staff members:

• Network/data center/cloud architects—Responsible for creating the overall design of
the network architecture that supports their company’s business objectives.
• Data center engineers—Responsible for working with architects, planners, and operation
engineers to design and implement the solution.
Validated Solution Design and Implementation Guide Overview
Juniper Networks creates end-to-end solutions in conjunction with select third-party
partners, such as VMware and IBM. These integrated solutions enable our customers to
implement comprehensive IT projects to accomplish business goals. Our reference
architectures are designed by subject matter experts and verified through in-house
solution testing, which uses a detailed framework to validate the solution from both a
network and an application perspective. Testing and measuring applications at scale
verify the integration of the network, compute, storage, and related components.
Juniper Networks validated solutions are complete, purpose-built, domain architectures
that:
• Solve specific problems
• Have undergone end-to-end validation testing
• Are thoroughly documented to provide clear deployment guidance
Juniper Networks solution validation labs subject all solutions to extensive testing using
both simulation and live network elements to ensure comprehensive validation. Customer
use cases, common domain examples, and field experience are combined to generate
prescriptive configurations and architectures to inform customer and partner
implementations of Juniper Networks solutions. A solution-based approach enables
partners and customers to reduce time to certify and verify new designs by providing
tested, prescriptive configurations to use as a baseline. Juniper Networks solution
validation provides the peace of mind and confidence that the solution behaves as
described in a real-world production environment.
This guide is intended to be the first in a series of guides that enable our customers to
build effective data centers to meet specific business goals.
MetaFabric 1.0 Overview
To provide flexibility to your implementation of the virtualized IT data center, there are
several sizes of the MetaFabric 1.0 solution. As seen in Figure 6 on page 9, you can start
with a small implementation and grow your data center network into a large one over
time. The reference architecture tested and documented in this guide uses the large
topology option with two QFX3000-M QFabric points of delivery (PODs) instead of six.

Chapter 1: Overview
Figure 6: Juniper Networks Virtualized IT Data Center - Sizing Options
The small option shown in Figure 6 on page 9 uses two QFX3600 switches for
aggregation and six QFX3500 switches for access. Two 40-Gigabit Ethernet ports on
the QFX3500 switch are used as uplinks, while the other two are split into four 40-Gigabit
Ethernet server ports. As a result, each QFX3500 switch has 56 network ports and
implements 7:1 oversubscription. The medium option is a single QFX3000-M QFabric
system with 64 network ports and 768 server ports, resulting in 3:1 oversubscription. The
large option uses 7:1 oversubscription and consists of 6 QFX3000-M QFabric systems.
NOTE: A fourth option not shown in the diagram would be to replace the 6
QFX3000-M QFabric systems with one QFX3000-G QFabric system to build
a data center containing 6144 ports.
The different sizing options solution offer different port densities to meet the growing
needs of the data center. The predefined configuration and provisioning options that
cover the small, medium, and large deployment scenarios are shown in Table 1 on page 9.
Table 1: Juniper Networks Virtualized IT Data Center – Details of Sizing Options
Small Medium Large
Network Ports 12 64 144
Server Ports 336 768 4032
Switches 8 1 (QFX3000-M QFabric) 6 (QFX3000-M QFabric)
Rack Units 8 20 128

Solution Overview
This MetaFabric 1.0 solution identifies the key components necessary to accomplish the
specified goals. These components include compute, network, and storage requirements,
as well as considerations for business-critical applications, high availability, class of
service, security, and network management (Figure 7 on page 10). As a result of these
requirements and considerations, it is critical that all components are configured,
integrated, and tested end-to-end to guarantee service-level agreements (SLAs) to
support the business.
Figure 7: Juniper Networks Virtualized IT Data Center – Solution
Components
The following sections describe the general requirements you need to include in a
virtualized IT data center.
• Compute on page 10
• Network on page 11
• Storage on page 12
• Applications on page 12
• Class of Service on page 13
• Security on page 13
• Network Management on page 14
Compute
Because this solution is focused on a virtualized IT environment, naturally many of the
requirements are driven by virtualization itself. Compute resource management involves
the provisioning and maintenance of virtual servers and resources that must be centrally
managed. The requirements for compute resources within a virtualized IT data center
include:

Chapter 1: Overview
• Workload mobility and migration for VMs—Applications must be able to be migrated
to other virtual machines when resource contention thresholds are reached.
• Location independence for VMs—An administrator must be able to place the VMs on
any available compute resource and move them to any other server as needed, even
between PODs.
• VM visibility—An administrator must be able to view where the virtual machines are
located in the data center and generate reports on VM movement.
• High availability—Compute resources must be ready and operational to meet user
demands.
• Fault tolerance—If VMs fail, there should be ways for the administrator to recover the
VMs or move them to another compute resource.
• Centralized virtual switch management—Keeping the management for VMs and virtual
switches in one place alleviates the hassle of logging into multiple devices to manage
dispersed virtual equipment.
Network
The network acts as the glue that binds together the data center services, compute, and
storage resources. To support application and storage traffic, you need to consider what
is required at the access and aggregation switching levels, core switching, and edge router
tiers of your data center. These are the areas that Juniper Networks understands best,
so we can help you in selecting the correct networking equipment to support your
implementation of the virtualized IT data center.
The requirements for a virtualized IT data center network include:
• 1-Gigabit, 10-Gigabit, and 40-Gigabit Ethernet Ports—This requirement covers the most
common interface types in the data center.
• Converged data and storage—By sending data and storage traffic over a single network,
this reduces the cost required to build, operate, and maintain separate networks for
data and storage.
• Load balancing—By distributing and alternating the traffic over multiple paths, this
ensures an efficient use of bandwidth and resources to prevent unnecessary
bottlenecks.
• Application quality of experience—By designing class of service requirements for different
traffic queues, this ensures prioritization for mission-critical traffic (such as storage
and business-critical applications) and best effort handling for routine traffic (such as
e-mail).
• Network segmentation—Breaking the network into different portions lowers the amount
of traffic congestion, and improves security, reliability, and performance.
• Traffic isolation and separation—By carefully planning traffic flows, you can keep
East-to-West and North-to-South data center traffic separate from each other and
prevent traffic from traveling across unnecessary hops to reach its destination. This

allows most traffic to flow locally, which reduces latency and improves application
performance.
• Time synchronization—This requirement ensures that a consistent time stamp is
standardized across the data center for management and monitoring purposes.
Generally speaking, you need to determine which Layer 2 and Layer 3 hardware and
software protocols meet your needs to provide a solid foundation for the traffic that
flows through your data center.
Storage
There are two primary types of storage: local storage and shared storage. Local storage
is generally directly attached to a server or endpoint. Shared storage is a shared resource
in the data center that provides storage services to a set of endpoints. The MetaFabric
1.0 solution focuses primarily on shared storage as it is the foundation for all of the
endpoint storage within a data center. Shared storage can be broken down into six primary
roles: controller, front end, back end, disk shelves, RAID groups, and storage pools.
Although there are many different types of shared storage that vary per vendor, the
architectural building blocks remain the same. Each storage role has a very specific role
and function in order to deliver shared storage to a set of endpoints.
The requirements for storage within a virtualized IT data center include:
• Scale—The storage component must be able to handle sufficient input/output
operations per second (IOPS) to support business-critical applications.
• Lossless Ethernet—This is a requirement for converged storage.
• Boot from shared storage—The advantages of this requirement include easier server
maintenance, more robust storage (such as more disks, more capacity, and faster
storage processors), and easier upgrade options.
• Multiple protocol storage—The storage device must be able to support multiple types
of storage protocols, such as Internet Small Computer System Interface (iSCSI),
Network File System (NFS), and Fiber Channel over Ethernet (FCoE). This provides
flexibility to the administrator to integrate different types of storage as needed.
Applications
For your applications, you need to consider the user experience and plan your
implementation accordingly. Business-critical applications provide the main reason for
the existence of the data center. The other data center components (such as compute,
network, and storage) serve to ensure that these applications are hosted securely in a
manner that can provide a high-quality user experience. Web services, e-mail, database,
and collaboration tools are housed in the data center – these tools form the basis for
business efficiency and must deliver application performance at scale. As such, the data
center architecture should focus on delivering a high-quality user experience through
coordinated operation across all tiers of the data center.
For example, can the Web, application, and database tiers communicate properly with
each other? If you plan to allow VM motion to occur only within an access and aggregation

Chapter 1: Overview
POD, you can include Layer 3 integrated routing and bridging (IRB) within the access and
aggregation layer. However, if you choose to move VMs from one POD to another, you
need to configure the IRB interface at the core layer to allow the VM to reach the Web,
application, and database servers that are still located in the original POD. Factoring in
such design aspects ahead of time prevents headaches to the data center administrator
in the months and years to come.
The requirements for applications within a virtualized IT data center include:
• Business-critical applications—The solution must address common data center
applications.
• High performance—Applications must be delivered to users in a timely fashion to ensure
smooth operations.
High Availability
Keeping your equipment up and running so that traffic can continue to flow through the
data center is a must to ensure that applications run smoothly for your customers. You
should strive to build a robust infrastructure that can withstand outages, failover, and
software upgrades without impacting your end users. High availability should include
both hardware and software components, along with verification. Key considerations
for high availability in an virtualized IT data center include:
• Hardware redundancy—At least two redundant devices should be placed at each layer
of the data center to ensure resiliency for traffic. If one device fails, the other device
should still be able to forward data and storage packets to their destinations. The data
center requires redundant network connectivity and the ability for traffic to use all
available bandwidth.
• Software redundancy—Features such as nonstop software upgrade, Virtual Router
Redundancy Protocol (VRRP), graceful restart, MC-LAG, and graceful Routing Engine
switchover (GRES) are needed to maintain device uptime, provide multiple forwarding
paths, and ensure stability in the data center.
Class of Service
Because of the storage requirements in the virtualized IT data center, you must include
lossless Ethernet transport in your design to meet the needs for converged storage in the
solution. Also, you must consider the varying levels of class of service necessary to support
end-to-end business-critical applications, virtualization control, network control, and
best-effort traffic.
Security
Another important task is to secure your data center environment from both external
and internal threats. Because this solution contains both physical and virtual components,
you must secure both the applications and traffic that flow through the heart of the data
center (often across VMs) as well as the perimeter of the data center (consisting primarily
of physical hardware, such as an edge firewall). You must also provide secure remote
access to the administrators who are managing the data center.

Security requirements for this solution include:
• Perimeter security—Using hardware-based security provides services such as Network
Address Translation (NAT), encrypted tunnels, and intrusion detection to prevent
attacks and prohibit unauthorized access.
• Application security—Use of a software solution for application security provides
network segmentation, robust policies, and intrusion detection.
• Remote access—Implementing a secure access method provides two-factor
authentication and Role-Based Access Control (RBAC) to allow access to authorized
data center administrators.
Network Management
The final challenge is connecting the dots between physical and virtual networking;
bridging this gap enables the data center engineer to quickly troubleshoot and resolve
issues. For network management in a virtualized IT data center, you need to consider
management of fault, configuration, accounting, performance and security (FCAPS) in
your network (Figure 8 on page 14).
Figure 8: Network Management Requirements
For more information about FCAPS (the ISO model for network management), see
ISO/IEC 10040.
Network management requirements for the solution include:
• Virtual and physical—You must be able to manage all types of components in the data
center network, regardless if they are hardware-based or virtualized.
• Fault—Errors in the network must be isolated and managed in the most efficient way
possible. You should be able to recognize, isolate, correct, and log faults that occur in
your network.
• Configuration—You should be able to provision your network flexibly from a central
location and manage configurations for the devices in your data center.
• Accounting—You must be able to gather network usage statistics, and establish users,
passwords, and permissions.

Chapter 1: Overview
• Performance—You should be able to the monitor throughput, network response times,
packet loss rates, link utilization, percentage utilization, and error rates to ensure the
network continues to perform at acceptable levels.
• Security—You must be able to control access to network components through use of
authorization, encryption, and authentication protocols.

CHAPTER 2
Design
• Design Considerations on page 17
• Design Scope on page 18
• Design Topology Diagram on page 19
• Design Highlights on page 20
• Solution Design on page 21
• Summary of Key Design Elements on page 58
• Benefits on page 58
Design Considerations
As seen in the Overview, designing a virtualized IT data center requires careful
consideration of the three key segments of compute, network, and storage, along with
their related subareas:
• Compute
• Virtual machines
• Servers
• Hypervisor switch
• Blade switch
• Network
• Access
• Aggregation
• Core switching
• Edge routing
• WAN
• Storage

The design must also include careful planning of other architectural considerations:
• Applications
• High availability
• • Class of service
• Security
• Network management
In general, the design for the solution must satisfy the following high-level requirements:
• The entire data center must have end-to-end convergence for application traffic of
under one second from the point of view of the application.
• Compute nodes must be able to use all available network links for forwarding.
• Traffic must be able to travel between the points of delivery PODs.
• Virtual resources must be able to be moved within a POD.
• The out-of-band (OOB) management network must be able to survive the failure of
the data plane within a POD.
Design Scope
This MetaFabric 1.0 solution covers the areas shown in Figure 9 on page 18. Juniper
Networks supplies products that appear in the blue portions of the diagram, while open
ecosystem partner products appear in the black portion. The ecosystem partners for this
solution include IBM (Compute), EMC (Storage), F5 Networks (Services), and VMware
(Virtualization).
Figure 9: Virtualized IT Data Center Ecosystem

Design Topology Diagram
Chapter 2: Design
Figure 10 on page 19 shows the general layout of the hardware components included in
the MetaFabric 1.0 solution architecture.
Figure 10: Virtualized IT Data Center Topology

Design Highlights
Table 2 on page 20 shows the key features of the MetaFabric 1.0 solution and how they
are implemented with hardware and software from Juniper Networks and our third-party
ecosystem partners.
Table 2: MetaFabric 1.0 Solution Design Highlights
Feature Implementation
Compute and virtualization IBM Flex Systemservers, VMware vSphere 5.1, vCenter
Core and edge network MX240 routers, EX9214 switches
Access and aggregation QFX3000-M QFabric system
Layer 2 and Layer 3 protocols OSPF, BGP, IRB, and VLANs
Storage EMC VNX5500 unified storage
Applications Microsoft SharePoint, Microsoft Exchange, andWikiMedia run at scale
Nonstop software upgrade, in-service software upgrade, SRX JSRP cluster,
MC-LAG Active/Active with VRRP
High availability
Class of service Lossless Ethernet, end-to-end application class of service
Security SRX3600, Firefly Host
Remote access Junos Pulse Gateway SA
Networkmanagement Junos Space Network Director 1.5, Security Director
Out-of-bandmanagement network EX4300 Virtual Chassis
Application load balancer F5 LTM Load Balancer

Solution Design
Chapter 2: Design
This section explains the compute resources, network infrastructure, and storage
components required to implement the MetaFabric 1.0 solution. It also discusses the
software applications, high availability, class of service, security, and network management
components of this solution.
The purpose of the data center is to host business-critical applications for the enterprise.
Each role in the data center is designed and configured to ensure the highest quality user
experience possible. All of the functional roles within the data center exist to support the
applications in the data center.
• Compute on page 21
• Network on page 26
• Storage on page 35
• Applications on page 37
• Class of Service on page 47
• Security on page 48
• Network Management on page 53
• Performance and Scale on page 57
Compute
In the compute area, you need to select the physical and virtual components that will
host your business-critical applications, network management, and security services.
This includes careful selection of VMs, servers, hypervisor switches, and blade switches.
Virtual Machines
A virtual machine (VM) is a virtual computer that is made up of a host operating system
and applications. A hypervisor is software that runs on a physical server, emulating
physical hardware for VMs. The VM operates on the emulated hardware of the hypervisor.
The VM believes that it is running on dedicated, physical hardware. This layer of
abstraction enables the benefit of presentation to the operating system; regardless of
changes to the hardware, the operating system sees the same set of logical hardware.
This enables operators to make changes to the physical environment without causing
issues on the servers hosted in the virtual environment, as seen in Figure 11 on page 22.

Figure 11: Virtual Machine Design
Virtualization also enables flexibility that is not possible on physical servers. Operating
systems can be migrated from one set of physical hardware to another with very little
effort. Complete environments, to include the operating system and installed applications,
can be cloned in a virtual environment, enabling complete backups of the environment
or, in some cases, you can clone or recreate identical servers on different physical hardware
for redundancy or mobility purposes. These clones can be activated upon primary VM
failure and enable an easy level of redundancy to exist at the data center application
layer. An extension to the benefit of cloning is that new operating systems can be created
from these clones very quickly, enabling faster service rollouts and faster time to revenue
for new services.
Servers
The server in the virtualized IT data center is simply the physical compute resource that
hosts the VMs. The server offers processing power, storage, memory, and I/O services
to the VMs. The hypervisor is installed directly on top of the servers without any sort of
host operating system, becoming a bare-metal operating system that provides a
framework for virtualization in the data center.
Because the server hosts the revenue generating portion of the data center (the VMs
and resident applications), redundancy is essential at this layer. A virtualized IT data
center server must support full hardware redundancy, management redundancy, the
ability to upgrade software while the server is in service, hot swapping of power supplies,
cooling, and other components, and the ability to combine multiple server or blade chassis
into a single, logical management plane.
The server chassis must be able to provide transport between the physical hardware and
virtual components, connect to hosts through 10-Gigabit Ethernet ports, use 10-Gigabit
Ethernet or 40-Gigabit Ethernet interfaces to access the POD, consolidate storage, data,
and management functions, provide class of service, reduce the need for physical cables,
and provide active/active forwarding.

Figure 12: Server Design
Chapter 2: Design
As seen in Figure 12 on page 23, this solution includes 40-Gigabit Ethernet connections
between QFabric system redundant server Node groups and IBM Flex servers that host
up to 14 blade servers. Other supported connection types include 10-Gigabit Ethernet
oversubscribed ports and 10-Gigabit Ethernet pass-through ports. The solution also has
two built-in switches per Flex server and uses MC-LAG to keep traffic flowing through
the data center.
Hypervisor Switching
The hypervisor switch is the first hop from the application servers in the MetaFabric 1.0
architecture. Virtual machines connect to a distributed virtual switch (dvSwitch) which
is responsible for mapping a set of physical network cards (pNICs) across a set of physical
hosts into a single logical switch that can be centrally managed by a virtualization
orchestration tool such as VMware vCenter (Figure 13 on page 23). The dvSwitch enables
intra-VM traffic on the same switching domain to pass between the VMs locally without
leaving the blade server or virtual environment. The dvSwitch also acts like a Virtual
Chassis, connects multiple ESXi hosts simultaneously, and offers port group functionality
(similar to a VLAN) to provide access between VMs.
Figure 13: VMware Distributed Virtual Switch
This poses an interesting security challenge on the hypervisor switch, as traditional,
appliance-based firewalls do not have visibility into the hypervisor switching environment.

In cases where restrictions must be placed on VM-to-VM traffic, security software can
be installed on the hypervisor to perform firewall functions between VMs.
The hypervisor switch is a critical piece of the MetaFabric 1.0 architecture. As such, it
should support functions that enable class of service and SLA attainment. Support for
IEEE 802.1p is required to support class of service. Support for link aggregation of parallel
links (IEEE 802.3ad) is also required to ensure redundant connection of VMs. As in the
other switching roles, support for SLA attainment is also a necessity at this layer. The
hypervisor switch should support SNMPv3, flow accounting and statistics, remote port
mirroring, and centralized management and reporting to ensure that SLAs can be
measured and verified.
To complete the configuration for the hypervisor switch, provide class of service on flows
for IP storage, vMotion, management, fault tolerance, and VM traffic. As shown in
Figure 14 on page 24, this solution implements the following allocations for network
input/output (I/O) control shares: IP storage (33.3 percent), vMotion (33.3 percent),
management (8.3 percent), fault tolerance (8.3 percent), and VM traffic (16.6 percent).
These categories have been maximized for server-level traffic.
Figure 14: VMware Network I/O Control Design
Blade Switching
The virtualized IT data center features virtual appliances that are often hosted on blade
servers, or servers that support multiple interchangeable processing blades that give the
blade server the ability to host large numbers of VMs. The blade server includes power
and cooling modules as well as input/output (I/O) modules that enable Ethernet
connection into the blade server (Figure 15 on page 25). Blade switching is performed
between the physical Ethernet port on the I/O module and the internal Ethernet port on
the blade. In some blade servers, a 1:1 subscription model (one physical port connects to
one blade) is used (this is called pass-thru switching), with one external Ethernet port
connecting directly to a specific blade via an internal Ethernet port. The pass-through
model offers the benefit of allowing full line bandwidth to each blade server without
oversubscription. The downside to this approach is often a lack of flexibility in VM mobility
and provisioning as VLAN interfaces need to be moved on the physical switch and the
blade switch when a move is required.

Figure 15: Sample Blade Switch, Rear View
Chapter 2: Design
Another mode of blade switch operation is where the blade switch enables
oversubscription to the blade servers. In this type of blade server, there may be only 4
external ports that connect internally to 12 separate blade servers. This would result in
3:1 oversubscription (three internal ports to every one external port). The benefit to this
mode of operation is that it minimizes the number of connected interfaces and access
switch cabling per blade server, even though the performance of oversubscribed links
and their connected VMs can degrade as a result. While this architecture is designed for
data centers that utilize blade servers, the design works just as well in data centers that
do not utilize blade servers to host VMs.
Table 3 on page 26 shows that both pass-through blade servers and oversubscribed
blade servers are acceptable choices for this solution in your data center network. In
some cases, you might need the faster speed provided by the 40-Gigabit Ethernet
connections to support newer equipment, while in others you would prefer the line-rate
performance offered by a pass-through switch. As a result, all three blade server types
are supported in this design.

Table 3: Comparison of Pass-Through Blade Servers and Oversubscribed Blade Servers
Attribute Pass-Through SW 10G Chassis SW 40G Chassis SW
Transport Yes Yes Yes
10-Gigabit Ethernet host interface Yes Yes Yes
40-Gigabit Ethernet uplink interface No No Yes
Consolidate storage, data, andmanagement Yes Yes Yes
Class of service Yes Yes Yes
Cable reduction No Yes (12:14) Yes (2:14)
Oversubscription 1:1 1.2:1 3.5:1
Active/Active Yes Yes Yes
To provide support for compute and virtualization in the virtualized IT data center, this
solution uses:
• Virtual machines—VMs running Windows and applications, such as Microsoft SharePoint,
Microsoft Exchange, and WikiMedia
• Servers—IBM x3750 and IBM Flex System chassis
• Configure an IBM Flex System server with multiple ESXi hosts supporting all the VMs
running business-critical applications (SharePoint, Exchange, and MediaWiki).
• Configure a distributed vSwitch between multiple physical ESXi hosts configured
on the IBM servers.
• Hypervisor—VMware vSphere 5.1 and vCenter
• Blade switches—IBM EN4091 and CN4093
This design for the compute and virtualization segment of the data center meets the
requirements of this solution for workload mobility and migration for VMs, location
independence for VMs, VM visibility, high availability, fault tolerance, and centralized
virtual switch management.
Network
The network is often the main focus of the data center as it is built to pass traffic to, from,
and between application servers hosted in the data center. Given the criticality of this
architectural role, and the various tiers within the data center switching block, it is further
broken up into access switching, aggregation switching, core switching, edge routing,
and WAN connectivity. Each segment within the data center switching role has unique
design considerations that relate back to business criticality, SLA requirements,
redundancy, and performance. It is within the data center switching architectural roles

Chapter 2: Design
that the network must be carefully designed to ensure that your data center equipment
purchases maximize network scale and performance while minimizing costs.
Access and Aggregation
The access layer consists of physical switches that connect to servers and end hosts.
Access switching typically focuses on implementing Layer 2 switches, but can include
Layer 3 components (such as IRB) to support more robust VM mobility. Access switching
should also support high availability. In a multi-chassis or virtual chassis environment,
where multiple physical switches can be combined to form a single, logical switch,
redundancy can be achieved at the access layer. This type of switch architecture is built
with control plane redundancy, MC-LAG, and the ability to upgrade individual switches
while they are in service. Additionally, the access switching role should support storage
traffic, or the ability to pass data traffic over Ethernet via iSCSI and Fiber Channel over
Ethernet (FCoE). Data Center Bridging (DCB) should also be supported by the access
switching role to enable full support of storage traffic. Within DCB, support for
priority-based flow control (PFC), enhanced transmission selection (ETS), and data
center bridging exchange (DCBX) should also be supported as these features enable
storage traffic to pass properly between all servers and storage devices within a data
center segment.
The aggregation switch acts as a multiplexing point between the access and the core of
the data center. The aggregation architectural role serves to combine a large number of
smaller interfaces from the access into high bandwidth trunk ports that can be more
easily consumed by the core switch. Redundancy should be a priority in the design of the
aggregation role as all Layer 2 flows between the data center and the core switch are
combined and forwarded by the data center aggregation switch role. At this layer, a
switching architecture that supports the combination of multiple switches into a single,
logical system with control and forwarding plane redundancy is recommended. This
switching architecture enables redundancy features such as MC-LAG, loop-free redundant
paths, and in-service software upgrades to enable data center administrators to
consistently meet and exceed SLAs.
One recommendation is to combine the access and aggregation layers of your network
by using a QFabric system. Not only does a QFabric system offer a single point of
provisioning, management, and troubleshooting for the network operator, it also collapses
switching tiers for any-to-any connectivity, provides lower latency, and enables all access
devices to be only one hop away from one another, as shown in Figure 16 on page 28.

Figure 16: Juniper Networks QFabric Systems Enable a Flat Data Center
Network
To implement the access and aggregation switching portions of the virtualized IT data
center, this solution uses the QFX3000-M QFabric system. There are two QFabric systems
(POD1 and POD2) in this solution to provide performance and scale. The QFabric PODs
support 768 ports per POD and feature low port-to-port latency, a single point of
management per POD, and lossless Ethernet to support storage traffic. The use of
predefined POD configurations enables the enterprise to more effectively plan data
center rollouts by offering predictable growth and scale in the solution architecture. Key
configuration steps include:
• Configure the QFX3000-M QFabric systems with 3 redundant server Node groups
(RSNGs) connected to 2 IBM Flex System blade servers to deliver application traffic.
• The first IBM Flex System server uses a 40-Gigabit Ethernet converged network
adapter (CNA) connected to a QFabric system RSNG containing QFX3600 Node
devices (RSNG4).
• The second IBM Flex System server has 10-Gigabit Ethernet pass through modules
connected to RSNG2 and RSNG3 on the second QFabric system.
• Connect the EMC VNX storage platform to the QFabric systems for storage access
using iSCSI and NFS.
• Connect the QFabric systems with the EX9214 core switch by way of a network Node
group containing 2 Node devices which use four 24-port LAGs configured as trunk
ports.
• Configure OSPF in the PODs (within the QFabric system network Node group) towards
the EX9214 core switch and place these connections in Area10 as a totally stubby area.
Core Switching
The core switch is often configured as a Layer 3 device that handles routing between
various Layer 2 domains in the data center. A robust implementation of the core switch
in the virtualized IT data center will support both Layer 2 and Layer 3 to enable a full
range of interoperability and service provisioning in a multitenant environment. Much like
in the edge role, the redundancy of core switching is critical as it too is a traffic congestion

Chapter 2: Design
point between the customer and the application. A properly designed data center includes
a fully redundant core switch layer that supports a wide range of interfaces (1-Gigabit,
10-Gigabit, 40-Gigabit, and 100-Gigabit Ethernet) with high density. The port density in
the core switching role is a critical factor as the data center core should be designed to
support future expansion without requiring new hardware (beyond line cards and interface
adapters). The core switch role should also support a wide array of SLA statistics
collection, and should be service-aware to support collection of service-chaining statistics.
The general location of the core switching function in this solution is shown in
Figure 17 on page 29.
Figure 17: Core Switching
Table 4 on page 30 shows some of the reasons for choosing an EX9200 switch over an
EX8200 switch to provide core switching capabilities in this solution. The EX9200 switch
provides a significantly larger number of 10-Gigabit Ethernet ports, support for 40-Gigabit
Ethernet ports, ability to host more analyzer sessions, firewall filters, and BFD connections,
and critical support for in-service software upgrade (ISSU) and MC-LAG. These reasons
make the EX9200 switch the superior choice in this solution.

MetaFabric Architecture Virtualized Data Center Design Guide

MetaFabric Architecture Virtualized Data Center Design Guide

Recommended

Recommended

More Related Content

What's hot

What's hot (18)

Viewers also liked

Viewers also liked (20)

Similar to MetaFabric Architecture Virtualized Data Center Design Guide

Similar to MetaFabric Architecture Virtualized Data Center Design Guide (20)

More from Juniper Networks

More from Juniper Networks (20)

Recently uploaded

Recently uploaded (20)

MetaFabric Architecture Virtualized Data Center Design Guide