SlideShare a Scribd company logo
MetroCluster Design and Implementation Guide
Jim Lanson, Network Appliance, Inc.

May, 2007 | TR-3548




Abstract
This document is intended to serve as a guide for architecting and deploying MetroCluster in a customer
environment. This design and implementation guide describes the basic MetroCluster architecture,
considerations for deployment, and best practices. As always, please refer to the latest technical
publications on the NOW™ (NetApp on the Web) site for specific updates on processes, Data ONTAP®
command syntax, and the latest requirements, issues, and limitations. This document is intended for field
personnel who require assistance in architecting and deploying a MetroCluster solution.
Table of Contents
    1.          Introduction ..................................................................................................................................................4

         1.1.            Intended Audience .............................................................................................................................4
         1.2.            Scope .................................................................................................................................................4
         1.3.            Requirements and Assumptions ........................................................................................................4
    2.          Overview ......................................................................................................................................................5

         2.1.            The Basics..........................................................................................................................................5
         2.2.            Fibre Channel Switch Overview.........................................................................................................9
    3.          Deployment Planning.................................................................................................................................11

         3.1.            Data Gathering.................................................................................................................................11
         3.2.            Distance Considerations ..................................................................................................................12
         3.3.            Cluster Configuration Checker.........................................................................................................14
         3.4.            Best Practices and Recommendations ............................................................................................14
    4.          Installation and Configuration ....................................................................................................................15

         4.1.            Primary Site......................................................................................................................................15
         4.2.            Remote Site......................................................................................................................................22
         4.3.            Testing..............................................................................................................................................24
         4.4.            Adding More Shelves .......................................................................................................................25
         4.5.            Best Practices and Recommendations ............................................................................................26
    5.          Site Failover and Recovery........................................................................................................................28

         5.1.            Site Failover .....................................................................................................................................28
         5.2.            Split Brain Scenario..........................................................................................................................28
         5.3.            Recovery Process ............................................................................................................................28
         5.4.            Giveback Process ............................................................................................................................29
    6.          Relevant Documentation ...........................................................................................................................30

         6.1.            MetroCluster Documentation ...........................................................................................................30
         6.2.            Brocade Switch Documentation.......................................................................................................30
    7.          Appendixes ................................................................................................................................................31

         7.1.            Appendix A: Switch Ports ................................................................................................................31
         7.2.            Appendix B: Switch Software Upgrade Procedure ..........................................................................32
         7.3.            Appendix C: Fabric MetroCluster Worksheet .................................................................................33
         7.4.            Appendix D: Fabric MetroCluster (16-Port Switches) ......................................................................35
         7.5.            Appendix E: Fabric MetroCluster (8-Port Switches) ........................................................................37




    MetroCluster Design & Implementation Guide – version 1.0                                                                                                                        2




2
Figures and Tables
    Figure 2-1) Stretch MetroCluster. ...........................................................................................................................5
    Figure 2-2) Fabric MetroCluster. ............................................................................................................................6
    Figure 2-3) SyncMirror............................................................................................................................................7
    Table 2-1) Hardware disk ownership......................................................................................................................8
    Figure 2-4) Dual Fabric MetroCluster. ..................................................................................................................10
    Figure 2-5) 16-port switch.....................................................................................................................................10
    Figure 2-6) 8-port switch.......................................................................................................................................11
    Figure 4- 1) Primary node cabling – appliance to switches..................................................................................19
    Figure 4-2) Primary node cabling – disk shelf to switch.......................................................................................20
    Figure 4-3) Remote node – appliance to switch...................................................................................................23
    Figure 4-4) Remote node cabling – disk shelf to switch.......................................................................................24
    Figure 4-5) 16-port switch port assignments. .......................................................................................................25
    Figure 4-6) 8-port switch port assignments. .........................................................................................................25
    Figure 4-7a) Adding disk shelves to primary node. ..............................................................................................26
    Figure 4-7b) Adding disk shelves to remote node................................................................................................26




    MetroCluster Design & Implementation Guide – version 1.0                                                                                                              3




3
1.     Introduction
    1.1.   Intended Audience
           The information in this document is intended for field personnel and administrators who are responsible
           for architecting and deploying successful MetroCluster high-availability and disaster-recovery
           configurations. A brief overview of MetroCluster is presented in order to establish baseline knowledge
           before discussing implementation planning, installation, configuration, and operation.

    1.2.   Scope
           This document covers the following MetroCluster configurations:

             •    Stretched or non-switched MetroCluster (NetApp storage)

             •    Fabric or switched MetroCluster (NetApp Storage)
           Topics that apply only to non-switched MetroCluster refer to Stretch MetroCluster.

           Topics that are specific to NetApp storage-based fabric MetroCluster refer to Fabric MetroCluster.

           Topics that apply to all configurations refer simply to MetroCluster.

           Other than a short description, V-Series MetroCluster is not covered in this document.

           This document refers mostly to Fabric MetroCluster, because Fabric MetroCluster configurations
           introduce an increased level of complexity in design and implementation. To many, Stretch MetroCluster
           is simply an active-active configuration with longer cables and mirroring. The introduction of Fibre
           Channel switches and longer distances requires much more consideration and discussion. The
           operational sections (creating mirrors, forced takeover, etc.) apply to both.

    1.3.   Requirements and Assumptions
           For the methods and procedures described in this document to be useful to the reader, the following
           assumptions are made:

             • The reader has at least basic Network Appliance administration skills and has administrative
               access to the storage system via the command-line interface.

             • The reader has a full understanding of clustering as it applies to the NetApp storage controller
               environment.

             • The reader has at least a basic understanding of Fibre Channel switch technology and operation,
               along with access to the switches via command line.

             • In the examples in this report, all administrative commands are performed at the storage system or
               Fibre Channel switch command line.




    MetroCluster Design & Implementation Guide – version 1.0                                                         4




4
2.     Overview
    2.1.   The Basics
           FEATURES
           MetroCluster configurations consist of a pair of active-active storage controllers configured with mirrored
           aggregates and extended distance capabilities to create a high-availability disaster recovery solution.
           The primary benefits include:

             •    Faster high availability and geographic disaster recovery protection

             •    Minimal risk of lost data, easier disaster recovery, and reduced system downtime

             •    Quicker recovery when disaster occurs

             •    Minimal disruption to users and client applications

           METROCLUSTER TYPES
           Stretch MetroCluster (sometimes referred to as non-switched) is simply an active-active configuration
           that can extend up to 500m depending on speed and cable type. It also includes synchronous mirroring
           (SyncMirror®) and the ability to do a site failover with a single command. See Figure 2-1. Additional
           resiliency can be provided through the use of multipathing.
                                                                                                                                                                                                                                                                                                                                                       Total Distance
                                                                                                                                                                                                                                                                                                                                                 500m @2Gbps (270m @4Gbps )


                                                                                                                                                  Primary                                                                                                                                                                                                                                                                                                                                                                                                                      Secondary

                                                                                                                                                                                                                NetApp                                                                                                                                                                                                                                                                                                                                                                                                                                  NetApp



                                                                                                                                                                                                                                                                                                                                                                                                                              CI
                                                                                                                                                                                                           FAS 3020                                                                                                                                                                                                                                                                                                                                                                                                                                FAS 3020




                                                                                                                                                                                                     activity        status             power                                                                                                                                                                                                                                                                                                                                                                                                activity         status          power




                                                                                                                                                                                                                                                                                                                                                                                                                   DS 14
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  DS 14
                                                                                                                                                                                                                                                                                                                                                                                                                   MK2
                                                                                                Network Appl ianc e     NetworkAppliance      Net work Applia nce     Network Appli anc e   NetworkAppliance      NetworkApplia nce       Network Applia nc e     NetworkAppliance      NetworkApplia nce     NetworkAppliance     NetworkAppliance       NetworkAppliance   NetworkAppliance   NetworkAppliance                                                                                                                                                                                                                                                                                                                  MK2
                                                                                                                                                                                                                                                                                                                                                                                                                     AT            NetworkApplia nce   NetworkAppliance   NetworkAppliance   NetworkAppliance   NetworkAppliance   NetworkAppliance   NetworkAppliance      NetworkAppliance     NetworkAppliance     Network Appl ianc e   NetworkAppliance      NetworkAppliance    Network Appli anc e   Network Appl ianc e
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    AT
                                                                                                                                                                                                                                                                                                                                                                                                                    Pow er

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   Pow er

                                                                                                                                                                                                                                                                                                                                                                                                                    F ault

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   F ault
                                                                                                                                                                                                                                                                                                                                                                                                                   Module A

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Module A
                                                                                                                                                                                                                                                                                                                                                                                                                   Module B

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Module B
                                                                                                                                                                                                                                                                                                                                                                                                                   Sys tem

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Sys tem
                                                                                                                                                                                                                                                                                                                                                                                                                   Shelf ID

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Shelf ID

                                                                                                      212A                    212A                 212A                     212A                 212A                  212A                     212A                    212A                  212A                 212A                  212A                     212A            212A               212A

                                                                                                      13                       12                    11                     10                     09                    08                       07                   06                     05                   04                    03                       02              01                 00                                 212A                212A               212A               212A               212A               212A               212A                   212A                212A                  212A                  212A                 212A                 212A                  212A

                                                                                                                                                                                                                                                                                                                                                                                                                                         13                 12                 11                  10                 09                 08                 07                   06                   05                   04                     03                   02                   01                    00




                                                                                                                                       Primary FC Storage                                                                                                                                                                                                                                                                                              Secondary FC Storage

                                                                                                                                                                                                                                                                                                                                                       DS 14
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 DS 14
                                                                                                                                                                                                                                                                                                                                                       MK2
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 MK2
                                   NetworkAppliance   Network Appl ianc e   NetworkAppliance   NetworkAppliance       Network Appl ianc e   NetworkAppliance        NetworkAppliance        Network Appli anc e   Network Appl ianc e     Net work Applia nce   Network Applia nc e   Network Appl ianc e   NetworkAppliance     Net work Applia nce
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             NetworkAppliance   NetworkAppliance   NetworkAppliance   NetworkAppliance   Network Appl ianc e   NetworkAppliance     NetworkAppliance        Network Appl ianc e   NetworkAppliance    Net work Applia nce   Network Appli anc e   NetworkAppliance   Net work Applia nce   Network Applia nc e
                                                                                                                                                                                                                                                                                                                                                         AT
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  AT
                                                                                                                                                                                                                                                                                                                                                        P ower
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  P ower


                                                                                                                                                                                                                                                                                                                                                        Fau lt
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Fau lt

                                                                                                                                                                                                                                                                                                                                                       Module A
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 Module A

                                                                                                                                                                                                                                                                                                                                                       Module B
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 Module B

                                                                                                                                                                                                                                                                                                                                                       S ystem
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 S ystem


                                                                                                                                                                                                                                                                                                                                                       Shelf ID
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 Shelf ID



                                        212A                212A                 212A               212A                    212A                 212A                    212A                     212A                  212A                    212A                  212A                  212A                 212A                  212A
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  212A               212A               212A               212A                212A                 212A                 212A                     212A                 212A                 212A                  212A                      212A           212A                 212A
                                        13                  12                   11                  10                      09                    08                      07                    06                    05                       04                   03                     02                   01                    00
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  13                 12                 11                  10                  09                   08                   07                     06                    05                  04                    03                         02             01                    00




                                                                       Secondary FC Mirror                                                                                                                                                                                                                                                                                                                                                                                                                                               Primary FC Mirror



           Figure 2-1) Stretch MetroCluster.
           Fabric MetroCluster (also referred to as switched) uses four Fibre Channel switches in a dual fabric
           configuration and a separate cluster interconnect card to achieve an even greater distance (up to
           100km) between primary and disaster recovery locations. See Figure 2-2.




    MetroCluster Design & Implementation Guide – version 1.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                5




5
Fabric (switched)
                                                                                                                                                                                                                                                                                                                                           MetroCluster
                                                                                                                                                                                                                                                                                                                                                                                        Total Distance
                                                                                                                                                                                                                                                                                                                                                                                           100Km


                                                                                                                                      Primary Node                                                                                                                                                                                                                                                                                                                                                                     Secondary Node

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            NetApp
                                                                                                                                                                                                                                  NetApp

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        FAS 3020
                                                                                                                                                                                                                             FAS 3020



                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 activity        status            power
                                                                                                                                                                                                                       activity        status        po r
                                                                                                                                                                                                                                                       we




                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              CI
                                                                                                                                                                                                                                                                                                                                         CI
                                                                                                                                                                                                                                                                                                                                                                                                                                           CI
                                              CI
                                                                                                                                                                                                                                                                                                                                                                                                  ISL

                                                                                                                                                        Brocade 200E                                                                                                                                                                                                                                                                                                                                                                                                        Brocade 200E

                                                                                                                                                                                                                                                                                                                                                                                         ISL




                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          Brocade 200E
                                                                                                                                                                                                                                                                                                                                                                             DS14                                                                                                                                                                                                                                                                                                                                                                                 DS 14
                                                                                                                                                                                                                                                                                                                                                                             M K2
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  MK2
                                                                       NetworkA pplia nce   Net workAppliance   Net workAppliance   Net workAppliance   NetworkA ppliance   NetworkA ppliance      NetworkA ppliance         Network Appliance    Network Appliance   Ne tworkApplianc e   Ne tworkApplianc e   Ne tworkApplianc e   NetworkA ppliance   NetworkA ppliance
                                                                                                                                                                                                                                                                                                                                                                                                                                                          Netw orkAppliance   N etworkAppliance    N etworkAppliance    N etworkAppliance   Netw orkAppliance     Netw orkAppliance     Netw orkAppliance       Ne tworkAppliance    Ne tworkAppliance     NetworkA ppliance   NetworkA ppliance   NetworkA ppliance      Net workAppliance   Net workAppliance
                                                                                                                                                                                                                                                                                                                                                                               AT
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   AT

                                                                                                                                                                                                                                                                                                                                                                              P ow er
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   Power



                                                                                                                                                                                                                                                                                                                                                                              Fault
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   Fault

                                                                                                                                                                                                                                                                                                                                                                             Module A
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  Module A


                                                                                                                                                                                                                                                                                                                                                                             Module B                                                                                                                                                                                                                                                                                                                                                                             Module B


                                                                                                                                                                                                                                                                                                                                                                             S ystem                                                                                                                                                                                                                                                                                                                                                                              Sys tem



                                                                                                                                                                                                                                                                                                                                                                             Shelf ID                                                                                                                                                                                                                                                                                                                                                                             Shelf ID




                                                                            212A                 212A                212A                212A                212A                212A                   212A                       212A                212A                212A                 212A                 212A                  212A               212A                                                                                               212A                212A                 212A                 212A                212A                 212A                  212A                    212A                  212A                 212A               212A                 212A                   212A               212A

                                                                             13                  12                   11                  10                  09                   08                     07                      06                   05                  04                   03                   02                    01                 00                                                                                                 13                  12                   11                   10                  09                     08                    07                   06                    05                   04                  03                   02                     01                  00




                                                                                    Primary FC Storage                                                                                                                                                                                                                                                                                                                                                           Secondary FC Storage

                                                                                                                                                                                                                                                                                                                                     DS14                                                                                                                                                                                                                                                                                                                                                                            DS14
                                                                                                                                                                                                                                                                                                                                     MK2
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     MK2
                              Netw ork Appliance   Ne tworkAppliance    Ne tworkAppliance   Ne tworkAppliance   Network Appliance   Network Appliance   Network Appliance      Netw orkAppliance      Netw orkAppliance       N etworkAppliance   N etworkAppliance   N etworkAppliance     Netw orkAppliance    Netw orkAppliance
                                                                                                                                                                                                                                                                                                                                                                                                         N etworkA ppl iance   N etworkA ppl iance   Netw orkAppliance   N etworkA ppliance   N etworkA ppliance   Net workAppliance   N etworkA ppliance       Ne tworkAppliance     Netw orkApplianc e    Netw orkApplianc e   N etworkA pplia nce   Netw orkAppliance   Netw orkAppliance   N etworkA ppl iance
                                                                                                                                                                                                                                                                                                                                      AT
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       AT
                                                                                                                                                                                                                                                                                                                                      P ow er
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      P ower


                                                                                                                                                                                                                                                                                                                                      Fault
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       Fault

                                                                                                                                                                                                                                                                                                                                     Module A
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     Module A

                                                                                                                                                                                                                                                                                                                                     Module B
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     Module B

                                                                                                                                                                                                                                                                                                                                     System
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     System


                                                                                                                                                                                                                                                                                                                                     Shelf ID
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     Shelf ID



                                   212A                 212A                 212A                 212A                212A                212A                212A                   212A                   212A                   212A                 212A                212A                  212A                 212A
                                                                                                                                                                                                                                                                                                                                                                                                               212A                 212A                  212A                212A                 212A                 212A                212A                     212A                  212A                  212A                  212A                 212A                212A                 212A
                                    13                  12                    11                   10                  09                  08                   07                  06                     05                      04                   03                   02                   01                   00
                                                                                                                                                                                                                                                                                                                                                                                                               13                    12                    11                  10                    09                  08                   07                     06                    05                    04                    03                   02                  01                   00




                                                                                    Secondary FC Mirror                                                                                                                                                                                                                                                                                                                                                           Primary FC Mirror




          Figure 2-2) Fabric MetroCluster.
          V-Series MetroCluster is simply either of the above configurations with a NetApp V-Series system.
          Because of the architectural differences between V-Series and a standard active-active configuration,
          V-Series MetroCluster has additional flexibility when it comes to the maximum number of spindles and
          the Fibre Channel switches. See the V-Series Compatibility Guide on the NOW site for additional
          information. The implementation of V-Series MetroCluster is outside the scope of this document. See
          the V-Series documentation on the NOW site.

          For more information about MetroCluster, see the Data ONTAP Cluster Installation and Administration
          Guide (for Data ONTAP 7.1) or the Data ONTAP Active-Active Configuration Guide (for Data ONTAP
          7.2 or later). (For Data ONTAP 7.0 and earlier, see the Data Protection Online Backup and Recovery
          Guide and the NetApp Hardware Cluster Guide.)

          Note: When referring to the documentation just listed, use the wiring diagrams in this technical report.
          The diagrams in this document facilitate easier installation and expansion of MetroCluster.

          MIRRORING
          NetApp SyncMirror, an integral part of MetroCluster, combines the disk-mirroring protection of RAID1
          with NetApp industry-leading RAID4 and RAID-DP™ technology. In the event of an outage—whether it's
          due to a disk problem, cable break, or host bus adapter (HBA) failure—SyncMirror can instantly access
          the mirrored data without any operator intervention or disruption to client applications. SyncMirror
          maintains a strict physical separation between the two copies of your mirrored data. Each of these
          copies is referred to as a plex. Each controller’s data has its “mirror” at the other site.




    MetroCluster Design & Implementation Guide – version 1.0                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 6




6
Aggregate

                                                                               Located at
                                                                               Partner Site



                                                            Mirror



                                                 Plex0                 Plex1




                                                   Pool0                         Pool1


          Figure 2-3) SyncMirror Pool s and Plexes.
          When SyncMirror is licensed and hardware ownership is used (discussed in quot;Disk Ownershipquot;), spare
          disks are split into two pools—Pool0 and Pool1. Each plex of a mirror uses disks from separate pools.
          Disks assigned to Pool0 must be on separate loops from disks assigned to Pool1. When software
          ownership is used, disks are explicitly assigned by the administrator, as described in “Disk Ownership.”

          To maximize availability, Pool0 and Pool1 disks need to be on separate loops and use separate HBAs,
          cables, and shelves.

          Before enabling the SyncMirror license, ensure that disks for each pool are located on the appropriate
          loops that are fault isolated from each other.

          DISK OWNERSHIP
          In a MetroCluster where disk shelves at either side are mirrored and thus accessible by either controller,
          disk ownership comes into play. There are two methods of establishing disk ownership: hardware and
          software. Hardware-based ownership is the default for all platforms except V-Series, FAS 6000 series,
          and FAS 3040/3070. Software disk ownership capability became available in Data ONTAP 6.3.1. A brief
          description of each method follows. For more detail, see quot;Installation and Configuration,quot; later in this
          document or the Data ONTAP documentation.
          Hardware disk ownership establishes which controller owns which disks by how the shelves are
          connected. For more information, see the System Configuration Guide:
          http://now.netapp.com/NOW/knowledge/docs/hardware/NetApp/syscfg/




    MetroCluster Design & Implementation Guide – version 1.0                                                           7




7
Model               Pool Ownership
            FAS9xx              Slots 2, 3, 4, 5, and 7 are Pool0
                                Slots 8, 9, and 10 are Pool1
                                Optional software-based ownership and pool selection (as of Data ONTAP 7.1.1)
            FAS3020/ 3050       0a, 0b, and slots 1 and 2 are Pool0 (slot 1 is usually NVRAM)
                                0c, 0d and slots 3 and 4 are Pool1
                                Optional software-based ownership and pool selection (as of Data ONTAP 7.1.1)
            FAS3040//3070       Software-based ownership and pool selection
            FAS60xx             Software-based ownership and pool selection
            V-Series            Software-based ownership and pool selection
          Table 2-1) Hardware disk ownership.
          Note: Physical connectivity is extremely important to ensure adequate distribution of disks between the
          two pools.

          Software disk ownership allows greater flexibility in cabling by allowing disk ownership to be
          assigned through explicit commands. Rather than determine which controller owns the disk by hardware
          connection, the ownership information is written on each disk. Because disk ownership can be
          implemented by a system administrator, it is important to perform the configuration to maximize
          availability. For more information, see quot;Installation and Configuration,quot; later in this document.

          FIBRE CHANNEL SAN IN A METROCLUSTER WORLD
          For those who are experienced in Fibre Channel technology, and storage area networks (SANs) in
          particular, there are some differences and restrictions worth discussing relative to how Fabric
          MetroCluster utilizes this technology.

          Fabric MetroCluster configurations use Fibre Channel switches as the means to separate the controllers
          by a greater distance. The switches are connected between the controller heads and the disk shelves,
          and to each other. Each disk spindle or LUN individually logs into a Fibre Channel fabric. Except for the
          V-Series, the nature of this architecture requires, for performance reasons, that the two fabrics be
          completely dedicated to Fabric MetroCluster. Extensive testing was performed to ensure adequate
          performance with switches included in a Fabric MetroCluster configuration. For this reason, Fabric
          MetroCluster requirements prohibit the use of any other model or vendor of Fibre Channel switch than
          the Brocade 200E included with the Fabric MetroCluster.

          Also for performance reasons there is a current maximum of 336 spindles. Higher spindle count
          solutions will be available in the future. Keep in mind that this is for Fabric MetroCluster only. The
          maximum number of disks in a Stretch MetroCluster depends solely on the NetApp model.

          In a traditional SAN, there is great flexibility in connecting devices to ports as long as the ports are
          configured correctly and any zoning requirements are met. In a MetroCluster, when using hardware
          ownership, Data ONTAP expects certain devices to be connected to specific ports or ranges of ports. It
          is therefore critical that cabling be exactly as described in the installation procedures. Also, no switch-
          specific functions such as trunking or zoning are currently used in a NetApp Fabric MetroCluster (non-V-
          Series), making switch management minimal.

          RESTRICTIONS
          For further restrictions in using a Fabric MetroCluster configuration, see the Data ONTAP Cluster
          Installation and Administration Guide (for Data ONTAP version 7.1) or the Data ONTAP Active-Active



    MetroCluster Design & Implementation Guide – version 1.0                                                            8




8
Configuration Guide (for Data ONTAP version 7.2 or later). (For Data ONTAP 7.0 and earlier, see the
           Data Protection Online Backup and Recovery Guide and the NetApp Hardware Cluster Guide.)

           COMPONENTS
           A NetApp fabric includes the following components:

             •    A clustered pair of FAS900, 3000, or 6000 series controllers running Data ONTAP 6.4.1 or later
                  (See the MetroCluster Compatibility Matrix on the NOW site for supported models.)

             •    Four Brocade Fibre Channel switches with supported firmware
                  (See the MetroCluster Compatibility Matrix on the NOW site for supported models.)
                  There is a pair at each location. Supported models may differ between locations but must be the
                  same at each given location.

             •    Brocade Extended Distance license (if over 10km)

             •    Brocade Full-Fabric License

             •    Brocade Ports-On-Demand (POD) licenses for second 8 ports (if using a 16-port Brocade 200E
                  switch)

             •    A VI-MC cluster adapter

             •    A syncmirror_local license

             •    A cluster_remote license

             •    A cluster license

           METROCLUSTER VERSUS STANDARD SYNCHRONOUS REPLICATION
           As a disaster recovery solution, MetroCluster is often compared with other synchronous replication
           products. Even though it includes SyncMirror, MetroCluster can be differentiated by the following
           features:

             •    Low-level RAID mirroring (less performance impact)

             •    Site failover with a single command

             •    No extensive scripting required to make data available after failover

    2.2.   Fibre Channel Switch Overview
           OPERATION OVERVIEW
           The NetApp Fabric or switched MetroCluster configuration uses four Brocade 200E switches in a dual
           fabric configuration to connect two active-active controllers. These switches cannot be combined with
           any other switch model.

           SWITCH FABRIC CONFIGURATION
           A Fabric MetroCluster contains two switch fabrics. A switch fabric consists of a switch on the primary
           controller connected to a switch on the remote controller (see Figure 2-1). The two switches are
           connected to each other through ISL (Inter-Switch Link) cables.




    MetroCluster Design & Implementation Guide – version 1.0                                                        9




9
Fabric 1
                                               SW1                         SW3



                                                          Fabric 2
                                         SW2                                     SW4




            Figure 2-4) Dual Fabric MetroCluster.
            Figure 2-4 shows a Fabric MetroCluster configuration in which the first fabric begins at “SWITCH 1” on
            the primary side and is completed by connecting the ISL cable to the first switch, “SWITCH 3“, on the
            remote side. The second fabric is created by using “SWITCH 2” on the primary side, connected through
            a second ISL cable to the second switch “SWITCH 4” on the remote side. The reason for two fabrics is
            redundancy. The loss of a switch in a fabric or the loss of a fabric will not affect the availability of the
            Fabric MetroCluster.

            Note: Only one ISL is supported per fabric.

            SWITCH PORT ALLOCATION
            For purposes of this discussion, hardware disk ownership is assumed. The Fibre Channel switch ports
            are divided into banks and pools. Each switch has two banks, dividing the switch into two equal parts.
            (See Figures 2-5 and 2-6.) A storage controller connects to a bank through its Fibre Channel ports (HBA
            or embedded). It then owns the disks that are connected to ports of the opposite bank. If the storage
            system connects to bank 1, only shelves connected to bank 2 belong to it. Additionally, Pool0 disks are
            considered “local” – all reads will be satisfied from those disks only. Pool1 disks are “remote”; they will
            be written to, but cannot be read from under normal circumstances. Together, these two rules limit the
            usable ports for a certain disk pool to 4 on a 16-port switch, and to 2 on an 8-port switch.

            16-PORT SWITCHES
            As shown in Figure 2-5, ports 0-3 and 8-11 belong to Pool0, while ports 4-7 and 12-15 belong to Pool1.

            The Brocade 200E 16-port switch has ports 0-15 licensed and operational, as shown in Figure 2-5.

            Note: Pools are identified only as either Pool0 or Pool1. However, Data ONTAP can assign arbitrary
            numbers to plexes.

                                                           16 Port Switch
                                                                            Pool
                                                                             0
                                                                             1

     Figure 2-5) 16-port switch.

            8-PORT SWITCHES
            The 8-port Brocade 200E switch physically has 16 ports and looks like the 16-port version. However,
            only ports 0-7 are licensed, have small form-factor Pluggables (SFPs), and are operational, as shown in
            Figure 2-6.


     MetroCluster Design & Implementation Guide – version 1.0                                                              10




10
Note: Do not install SFPs in unused ports 8-15. This might cause Data ONTAP to operate as if it's a 16-
            port switch and erroneously assign banks, quadrants, and pools to unused ports.

            As shown in Figure 2-6, ports 0-1 and 4-5 belong to Pool0, while ports 2-3 and 6-7 belong to Pool1.

                                                          8 Port Switch
                                                            Pool 0 Pool 1



     Figure 2-6) 8-port switch.

            LICENSES
            A number of licenses are required for the Fibre Channel switches. When ordered with a Fabric
            MetroCluster configuration, the switches should include all necessary licenses. For reference, they are:

              •     Full Fabric License

              •     Extended Distance License (if over 10km)

              •     Brocade Ports-On-Demand (POD) licenses for second 8 ports (if using a 16-port Brocade 200E
                    switch)

     3.     Deployment Planning
     3.1.   Data Gathering
            CUSTOMER REQUIREMENTS
            To facilitate a successful MetroCluster installation, the following information should be gathered early
            on.

            ENVIRONMENTAL
            Distance between the primary and remote sites. This information is necessary to determine which type
            of MetroCluster is appropriate, or even whether either version is appropriate. In calculating the effective
            distance, factors such as cable type, speed, and number of patch panels must be considered. (See
            Section 3.2, quot;Distance Considerations.quot;) Although NetApp recommends that dedicated dark fiber be
            used for a MetroCluster, WDM devices are supported. Refer to the Brocade Compatibility Guide at
            www.brocade.com for supported devices.

            SETUP
            Gather the following items before you begin any MetroCluster installation:

                      NetApp and Fibre Channel switch licenses

                      Hostnames and IP addresses for each of the nodes and the Fibre Channel switches

                      Brocade switch licenses

                      Cables




     MetroCluster Design & Implementation Guide – version 1.0                                                             11




11
3.2.   Distance Considerations
            OVERVIEW
            Stretch MetroCluster can support a maximum of 500 meters between nodes at a speed of 2Gbps.
            Fabric MetroCluster, through the use of Fibre Channel switches, extends this distance to 100km at the
            same speed. At 4Gbps speeds, these distances are roughly cut in half. This extended distance
            capability gives customer greater flexibility in the physical location of the active-active controllers while
            maintaining the high-availability benefits of clustered failover.

            This section describes a number of factors that affect the overall effective distance permissible between
            the MetroCluster nodes.

              •    Physical distance

              •    Number of connections

              •    Desired speed

              •    Cable type

            PHYSICAL DISTANCE
            As stated earlier, the Stretch MetroCluster configuration can extend to a maximum of 500m (2Gbps).
            This distance is reduced by speed, cable type, and number of connections. A Fabric MetroCluster can
            extend out to 100km. This distance is affected by the same factors. At a distance of 100km, latency
            would be around 1ms. Greater distances would obviously result in greater latencies (500km = 5ms),
            which may be unacceptable to an application.

            CABLE TYPE
            As shown in Table 3.1, the cable type affects both distance and speed. Single-mode cable is supported
            only for the Inter-Switch Links.

            Example 1: A customer has 250 meters between sites and wants to run at 4Gbps. The OM-3 cable type
            is required.

            Example 2: A customer currently has a MetroCluster configuration running at 2Gbps with a distance of
            300 meters over OM2 cabling and wants to upgrade to 4Gbps speeds. Upgrading the cabling will not
            help, because OM3 has a maximum of 270 meters. In this case the choices would be:

              •    Remain at 2Gbps speeds. Customers with the new ESH4 disk shelves could still use them at this
                   distance, as long as the shelf speed is set to 2Gbps.

              •    Test current optical network infrastructure to make sure that attenuation and latency are
                   acceptable.
                                   Fibre Type               Data Rate           Max Distance (M)
                                                               1 Gb/s                 500
                                OM-2 (50/125UM)                2 Gb/s                 300
                                                               4Gb/s                  150
                                                               1 Gb/s                 860
                                OM-3 (50/125UM)                2 Gb/s                 500
                                                               4Gb/s                  270
                                OS1 Single Mode                2 Gb/s               10,000*
                                (9/125UM)                     4Gb/s                 10,000*
            Table 3-1) Cables, speed, and distance.

     MetroCluster Design & Implementation Guide – version 1.0                                                               12




12
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design
Brocade200 E Metro Cluster Design

More Related Content

What's hot

Internship report_Georgios Katsouris
Internship report_Georgios KatsourisInternship report_Georgios Katsouris
Internship report_Georgios KatsourisGeorgios Katsouris
 
ANSI_EVSP_Roadmap_May_2013.pdf
ANSI_EVSP_Roadmap_May_2013.pdfANSI_EVSP_Roadmap_May_2013.pdf
ANSI_EVSP_Roadmap_May_2013.pdf
Instansi
 
EVS-06-33e.pdf
EVS-06-33e.pdfEVS-06-33e.pdf
EVS-06-33e.pdf
Instansi
 
Report on Integrated Modular Avionics (DO-297/ED-124) for Requirement Enginee...
Report on Integrated Modular Avionics (DO-297/ED-124) for Requirement Enginee...Report on Integrated Modular Avionics (DO-297/ED-124) for Requirement Enginee...
Report on Integrated Modular Avionics (DO-297/ED-124) for Requirement Enginee...
Nikhil Dantkale
 
Castor Reference Guide 1 3 1
Castor Reference Guide 1 3 1Castor Reference Guide 1 3 1
Castor Reference Guide 1 3 1paripelly
 
Physical exercise web application documentation
Physical exercise web application documentationPhysical exercise web application documentation
Physical exercise web application documentation
TuononenP
 
How to manage future grid dynamics: system value of Smart Power Generation in...
How to manage future grid dynamics: system value of Smart Power Generation in...How to manage future grid dynamics: system value of Smart Power Generation in...
How to manage future grid dynamics: system value of Smart Power Generation in...
Smart Power Generation
 
Billion BiPAC 7300NX Router Manual
Billion BiPAC 7300NX Router ManualBillion BiPAC 7300NX Router Manual
Billion BiPAC 7300NX Router Manual
Fanus van Straten
 
Electronics en engineering-basic-vocational-knowledge
Electronics en engineering-basic-vocational-knowledgeElectronics en engineering-basic-vocational-knowledge
Electronics en engineering-basic-vocational-knowledgesandeep patil
 
VMware Networking 5.0
VMware Networking 5.0VMware Networking 5.0
VMware Networking 5.0
rashedmasood
 
Solar blinds
Solar blindsSolar blinds
Solar blinds
SharathKumar528
 
Neural Network Toolbox MATLAB
Neural Network Toolbox MATLABNeural Network Toolbox MATLAB
Neural Network Toolbox MATLABESCOM
 
CSP Report Group E (1)
CSP Report Group E (1)CSP Report Group E (1)
CSP Report Group E (1)Rachit Kansal
 
Solar tracking system
Solar tracking systemSolar tracking system
Solar tracking systemParviz Bora
 
Essmaxl
EssmaxlEssmaxl
Matconvnet manual
Matconvnet manualMatconvnet manual
Matconvnet manual
RohitKeshari
 
Emf2192 ib _ethercat aif module__v3-1__en
Emf2192 ib _ethercat aif module__v3-1__enEmf2192 ib _ethercat aif module__v3-1__en
Emf2192 ib _ethercat aif module__v3-1__en
Charles Santos
 

What's hot (20)

Internship report_Georgios Katsouris
Internship report_Georgios KatsourisInternship report_Georgios Katsouris
Internship report_Georgios Katsouris
 
ANSI_EVSP_Roadmap_May_2013.pdf
ANSI_EVSP_Roadmap_May_2013.pdfANSI_EVSP_Roadmap_May_2013.pdf
ANSI_EVSP_Roadmap_May_2013.pdf
 
EVS-06-33e.pdf
EVS-06-33e.pdfEVS-06-33e.pdf
EVS-06-33e.pdf
 
Report on Integrated Modular Avionics (DO-297/ED-124) for Requirement Enginee...
Report on Integrated Modular Avionics (DO-297/ED-124) for Requirement Enginee...Report on Integrated Modular Avionics (DO-297/ED-124) for Requirement Enginee...
Report on Integrated Modular Avionics (DO-297/ED-124) for Requirement Enginee...
 
Castor Reference Guide 1 3 1
Castor Reference Guide 1 3 1Castor Reference Guide 1 3 1
Castor Reference Guide 1 3 1
 
MSC-2013-12
MSC-2013-12MSC-2013-12
MSC-2013-12
 
Physical exercise web application documentation
Physical exercise web application documentationPhysical exercise web application documentation
Physical exercise web application documentation
 
How to manage future grid dynamics: system value of Smart Power Generation in...
How to manage future grid dynamics: system value of Smart Power Generation in...How to manage future grid dynamics: system value of Smart Power Generation in...
How to manage future grid dynamics: system value of Smart Power Generation in...
 
Billion BiPAC 7300NX Router Manual
Billion BiPAC 7300NX Router ManualBillion BiPAC 7300NX Router Manual
Billion BiPAC 7300NX Router Manual
 
Electronics en engineering-basic-vocational-knowledge
Electronics en engineering-basic-vocational-knowledgeElectronics en engineering-basic-vocational-knowledge
Electronics en engineering-basic-vocational-knowledge
 
VMware Networking 5.0
VMware Networking 5.0VMware Networking 5.0
VMware Networking 5.0
 
Solar blinds
Solar blindsSolar blinds
Solar blinds
 
Neural Network Toolbox MATLAB
Neural Network Toolbox MATLABNeural Network Toolbox MATLAB
Neural Network Toolbox MATLAB
 
thesis-2005-029
thesis-2005-029thesis-2005-029
thesis-2005-029
 
FinalReport-1
FinalReport-1FinalReport-1
FinalReport-1
 
CSP Report Group E (1)
CSP Report Group E (1)CSP Report Group E (1)
CSP Report Group E (1)
 
Solar tracking system
Solar tracking systemSolar tracking system
Solar tracking system
 
Essmaxl
EssmaxlEssmaxl
Essmaxl
 
Matconvnet manual
Matconvnet manualMatconvnet manual
Matconvnet manual
 
Emf2192 ib _ethercat aif module__v3-1__en
Emf2192 ib _ethercat aif module__v3-1__enEmf2192 ib _ethercat aif module__v3-1__en
Emf2192 ib _ethercat aif module__v3-1__en
 

Viewers also liked

The Frozen World
The Frozen WorldThe Frozen World
The Frozen World
alastairlee
 
i. Guru Agama Pro
i. Guru Agama Proi. Guru Agama Pro
i. Guru Agama ProDr22s
 
El Rostro De JesúS
El Rostro De JesúSEl Rostro De JesúS
El Rostro De JesúS
Marcela Patricelli
 
ii.Integrasi Tekno dlm Kelas
ii.Integrasi Tekno dlm Kelasii.Integrasi Tekno dlm Kelas
ii.Integrasi Tekno dlm KelasDr22s
 
Teknologi Pendidikan dlm P&P di NBD
Teknologi Pendidikan dlm P&P di NBDTeknologi Pendidikan dlm P&P di NBD
Teknologi Pendidikan dlm P&P di NBDDr22s
 
Install Note Juniper Netscreen Avaya Vsu Gateway And IP Softphones
Install Note Juniper Netscreen Avaya Vsu Gateway And IP SoftphonesInstall Note Juniper Netscreen Avaya Vsu Gateway And IP Softphones
Install Note Juniper Netscreen Avaya Vsu Gateway And IP Softphonesbsdux
 
Teaching Approach
Teaching ApproachTeaching Approach
Teaching ApproachDr22s
 
T1G3
T1G3T1G3
T1G3Dr22s
 
The world of Japan
The world of JapanThe world of Japan
The world of Japan
charmed
 
G1: Pencapaian Perisian Komputer P&P Islam
G1: Pencapaian Perisian Komputer P&P IslamG1: Pencapaian Perisian Komputer P&P Islam
G1: Pencapaian Perisian Komputer P&P IslamDr22s
 
Perisian TMK Islam
Perisian TMK IslamPerisian TMK Islam
Perisian TMK Islam
Dr22s
 
Viii Kurikulum Pi Nbd Edt
Viii Kurikulum Pi Nbd EdtViii Kurikulum Pi Nbd Edt
Viii Kurikulum Pi Nbd EdtDr22s
 
ABM dlm P&P
ABM dlm P&PABM dlm P&P
ABM dlm P&P
Dr22s
 
Penubuhan 2 IPT
Penubuhan 2 IPTPenubuhan 2 IPT
Penubuhan 2 IPTDr22s
 
Panduan WordPress
Panduan WordPressPanduan WordPress
Panduan WordPressDr22s
 
Perisian PUI_MEd
Perisian PUI_MEdPerisian PUI_MEd
Perisian PUI_MEdDr22s
 
Intro PA1401_I_09/10
Intro PA1401_I_09/10Intro PA1401_I_09/10
Intro PA1401_I_09/10Dr22s
 
Teste
TesteTeste
Teste
kkm_kaori
 

Viewers also liked (20)

The Frozen World
The Frozen WorldThe Frozen World
The Frozen World
 
i. Guru Agama Pro
i. Guru Agama Proi. Guru Agama Pro
i. Guru Agama Pro
 
El Rostro De JesúS
El Rostro De JesúSEl Rostro De JesúS
El Rostro De JesúS
 
Demo Vs Pitch
Demo Vs PitchDemo Vs Pitch
Demo Vs Pitch
 
ii.Integrasi Tekno dlm Kelas
ii.Integrasi Tekno dlm Kelasii.Integrasi Tekno dlm Kelas
ii.Integrasi Tekno dlm Kelas
 
Teknologi Pendidikan dlm P&P di NBD
Teknologi Pendidikan dlm P&P di NBDTeknologi Pendidikan dlm P&P di NBD
Teknologi Pendidikan dlm P&P di NBD
 
Install Note Juniper Netscreen Avaya Vsu Gateway And IP Softphones
Install Note Juniper Netscreen Avaya Vsu Gateway And IP SoftphonesInstall Note Juniper Netscreen Avaya Vsu Gateway And IP Softphones
Install Note Juniper Netscreen Avaya Vsu Gateway And IP Softphones
 
Teaching Approach
Teaching ApproachTeaching Approach
Teaching Approach
 
T1G3
T1G3T1G3
T1G3
 
The world of Japan
The world of JapanThe world of Japan
The world of Japan
 
Am Grupo Eto
Am Grupo EtoAm Grupo Eto
Am Grupo Eto
 
G1: Pencapaian Perisian Komputer P&P Islam
G1: Pencapaian Perisian Komputer P&P IslamG1: Pencapaian Perisian Komputer P&P Islam
G1: Pencapaian Perisian Komputer P&P Islam
 
Perisian TMK Islam
Perisian TMK IslamPerisian TMK Islam
Perisian TMK Islam
 
Viii Kurikulum Pi Nbd Edt
Viii Kurikulum Pi Nbd EdtViii Kurikulum Pi Nbd Edt
Viii Kurikulum Pi Nbd Edt
 
ABM dlm P&P
ABM dlm P&PABM dlm P&P
ABM dlm P&P
 
Penubuhan 2 IPT
Penubuhan 2 IPTPenubuhan 2 IPT
Penubuhan 2 IPT
 
Panduan WordPress
Panduan WordPressPanduan WordPress
Panduan WordPress
 
Perisian PUI_MEd
Perisian PUI_MEdPerisian PUI_MEd
Perisian PUI_MEd
 
Intro PA1401_I_09/10
Intro PA1401_I_09/10Intro PA1401_I_09/10
Intro PA1401_I_09/10
 
Teste
TesteTeste
Teste
 

Similar to Brocade200 E Metro Cluster Design

B7.2 a1353-ra platform commissioning solaris 2.6
B7.2 a1353-ra platform commissioning solaris 2.6B7.2 a1353-ra platform commissioning solaris 2.6
B7.2 a1353-ra platform commissioning solaris 2.6
chungminh1108
 
Presentation data center deployment guide
Presentation   data center deployment guidePresentation   data center deployment guide
Presentation data center deployment guide
xKinAnx
 
Tap watch 22 user manual-03253d
Tap watch 22 user manual-03253dTap watch 22 user manual-03253d
Tap watch 22 user manual-03253d
czimet
 
Win plc engine-en
Win plc engine-enWin plc engine-en
Win plc engine-en
dreamtech2
 
CloudAnalyst: A CloudSim-based Tool for Modelling and Analysis of Large Scale...
CloudAnalyst: A CloudSim-based Tool for Modelling and Analysis of Large Scale...CloudAnalyst: A CloudSim-based Tool for Modelling and Analysis of Large Scale...
CloudAnalyst: A CloudSim-based Tool for Modelling and Analysis of Large Scale...
ambitlick
 
Motorola ws2000 wireless switch system reference guide
Motorola ws2000 wireless switch system reference guideMotorola ws2000 wireless switch system reference guide
Motorola ws2000 wireless switch system reference guideAdvantec Distribution
 
Motorola ws2000 wireless switch system reference guide
Motorola ws2000 wireless switch system reference guideMotorola ws2000 wireless switch system reference guide
Motorola ws2000 wireless switch system reference guideAdvantec Distribution
 
S Pii Plus+C+Library+Programmer+Guide
S Pii Plus+C+Library+Programmer+GuideS Pii Plus+C+Library+Programmer+Guide
S Pii Plus+C+Library+Programmer+Guideguestd2fe1e
 
S Pii Plus+C+Library+Programmer+Guide
S Pii Plus+C+Library+Programmer+GuideS Pii Plus+C+Library+Programmer+Guide
S Pii Plus+C+Library+Programmer+Guideguestd2fe1e
 
350209_NGU-2000 BB (799020).pdf
350209_NGU-2000 BB (799020).pdf350209_NGU-2000 BB (799020).pdf
350209_NGU-2000 BB (799020).pdf
Gerson37561
 
2000402 en juniper good
2000402 en juniper good2000402 en juniper good
2000402 en juniper good
Achint Saraf
 
software-eng.pdf
software-eng.pdfsoftware-eng.pdf
software-eng.pdf
fellahi1
 
Ibm total storage productivity center for replication on windows 2003 sg247250
Ibm total storage productivity center for replication on windows 2003 sg247250Ibm total storage productivity center for replication on windows 2003 sg247250
Ibm total storage productivity center for replication on windows 2003 sg247250Banking at Ho Chi Minh city
 
Ibm total storage productivity center for replication on windows 2003 sg247250
Ibm total storage productivity center for replication on windows 2003 sg247250Ibm total storage productivity center for replication on windows 2003 sg247250
Ibm total storage productivity center for replication on windows 2003 sg247250Banking at Ho Chi Minh city
 
41713559 radio-access-network-protocols-and-signalling-analysis
41713559 radio-access-network-protocols-and-signalling-analysis41713559 radio-access-network-protocols-and-signalling-analysis
41713559 radio-access-network-protocols-and-signalling-analysisKevin Kang
 
MetaFabric™ Architecture Virtualized Data Center: Design and Implementation G...
MetaFabric™ Architecture Virtualized Data Center: Design and Implementation G...MetaFabric™ Architecture Virtualized Data Center: Design and Implementation G...
MetaFabric™ Architecture Virtualized Data Center: Design and Implementation G...
Juniper Networks
 
O&m manual(mux 2200 e)v1.1
O&m manual(mux 2200 e)v1.1O&m manual(mux 2200 e)v1.1
O&m manual(mux 2200 e)v1.1
Van Anh Lizaris
 
An introduction to tivoli net view for os 390 v1r2 sg245224
An introduction to tivoli net view for os 390 v1r2 sg245224An introduction to tivoli net view for os 390 v1r2 sg245224
An introduction to tivoli net view for os 390 v1r2 sg245224Banking at Ho Chi Minh city
 
Automated provisioning using ibm tivoli intelligent orchestrator and enterpri...
Automated provisioning using ibm tivoli intelligent orchestrator and enterpri...Automated provisioning using ibm tivoli intelligent orchestrator and enterpri...
Automated provisioning using ibm tivoli intelligent orchestrator and enterpri...Banking at Ho Chi Minh city
 

Similar to Brocade200 E Metro Cluster Design (20)

B7.2 a1353-ra platform commissioning solaris 2.6
B7.2 a1353-ra platform commissioning solaris 2.6B7.2 a1353-ra platform commissioning solaris 2.6
B7.2 a1353-ra platform commissioning solaris 2.6
 
Presentation data center deployment guide
Presentation   data center deployment guidePresentation   data center deployment guide
Presentation data center deployment guide
 
Tap watch 22 user manual-03253d
Tap watch 22 user manual-03253dTap watch 22 user manual-03253d
Tap watch 22 user manual-03253d
 
Win plc engine-en
Win plc engine-enWin plc engine-en
Win plc engine-en
 
CloudAnalyst: A CloudSim-based Tool for Modelling and Analysis of Large Scale...
CloudAnalyst: A CloudSim-based Tool for Modelling and Analysis of Large Scale...CloudAnalyst: A CloudSim-based Tool for Modelling and Analysis of Large Scale...
CloudAnalyst: A CloudSim-based Tool for Modelling and Analysis of Large Scale...
 
Motorola ws2000 wireless switch system reference guide
Motorola ws2000 wireless switch system reference guideMotorola ws2000 wireless switch system reference guide
Motorola ws2000 wireless switch system reference guide
 
Motorola ws2000 wireless switch system reference guide
Motorola ws2000 wireless switch system reference guideMotorola ws2000 wireless switch system reference guide
Motorola ws2000 wireless switch system reference guide
 
S Pii Plus+C+Library+Programmer+Guide
S Pii Plus+C+Library+Programmer+GuideS Pii Plus+C+Library+Programmer+Guide
S Pii Plus+C+Library+Programmer+Guide
 
S Pii Plus+C+Library+Programmer+Guide
S Pii Plus+C+Library+Programmer+GuideS Pii Plus+C+Library+Programmer+Guide
S Pii Plus+C+Library+Programmer+Guide
 
350209_NGU-2000 BB (799020).pdf
350209_NGU-2000 BB (799020).pdf350209_NGU-2000 BB (799020).pdf
350209_NGU-2000 BB (799020).pdf
 
2000402 en juniper good
2000402 en juniper good2000402 en juniper good
2000402 en juniper good
 
HRpM_UG_731_HDS_M2
HRpM_UG_731_HDS_M2HRpM_UG_731_HDS_M2
HRpM_UG_731_HDS_M2
 
software-eng.pdf
software-eng.pdfsoftware-eng.pdf
software-eng.pdf
 
Ibm total storage productivity center for replication on windows 2003 sg247250
Ibm total storage productivity center for replication on windows 2003 sg247250Ibm total storage productivity center for replication on windows 2003 sg247250
Ibm total storage productivity center for replication on windows 2003 sg247250
 
Ibm total storage productivity center for replication on windows 2003 sg247250
Ibm total storage productivity center for replication on windows 2003 sg247250Ibm total storage productivity center for replication on windows 2003 sg247250
Ibm total storage productivity center for replication on windows 2003 sg247250
 
41713559 radio-access-network-protocols-and-signalling-analysis
41713559 radio-access-network-protocols-and-signalling-analysis41713559 radio-access-network-protocols-and-signalling-analysis
41713559 radio-access-network-protocols-and-signalling-analysis
 
MetaFabric™ Architecture Virtualized Data Center: Design and Implementation G...
MetaFabric™ Architecture Virtualized Data Center: Design and Implementation G...MetaFabric™ Architecture Virtualized Data Center: Design and Implementation G...
MetaFabric™ Architecture Virtualized Data Center: Design and Implementation G...
 
O&m manual(mux 2200 e)v1.1
O&m manual(mux 2200 e)v1.1O&m manual(mux 2200 e)v1.1
O&m manual(mux 2200 e)v1.1
 
An introduction to tivoli net view for os 390 v1r2 sg245224
An introduction to tivoli net view for os 390 v1r2 sg245224An introduction to tivoli net view for os 390 v1r2 sg245224
An introduction to tivoli net view for os 390 v1r2 sg245224
 
Automated provisioning using ibm tivoli intelligent orchestrator and enterpri...
Automated provisioning using ibm tivoli intelligent orchestrator and enterpri...Automated provisioning using ibm tivoli intelligent orchestrator and enterpri...
Automated provisioning using ibm tivoli intelligent orchestrator and enterpri...
 

Recently uploaded

A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
sonjaschweigert1
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
Jen Stirrup
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofszkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
Alex Pruden
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfSAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
Peter Spielvogel
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
Thijs Feryn
 
Free Complete Python - A step towards Data Science
Free Complete Python - A step towards Data ScienceFree Complete Python - A step towards Data Science
Free Complete Python - A step towards Data Science
RinaMondal9
 
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Nexer Digital
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
Aftab Hussain
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Albert Hoitingh
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
UiPathCommunity
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
SOFTTECHHUB
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
Laura Byrne
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
Alpen-Adria-Universität
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
Safe Software
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
Kari Kakkonen
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Paige Cruz
 

Recently uploaded (20)

A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...The Metaverse and AI: how can decision-makers harness the Metaverse for their...
The Metaverse and AI: how can decision-makers harness the Metaverse for their...
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofszkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
 
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfSAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
 
Accelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish CachingAccelerate your Kubernetes clusters with Varnish Caching
Accelerate your Kubernetes clusters with Varnish Caching
 
Free Complete Python - A step towards Data Science
Free Complete Python - A step towards Data ScienceFree Complete Python - A step towards Data Science
Free Complete Python - A step towards Data Science
 
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
 
Removing Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software FuzzingRemoving Uninteresting Bytes in Software Fuzzing
Removing Uninteresting Bytes in Software Fuzzing
 
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
 
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...
 
The Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and SalesThe Art of the Pitch: WordPress Relationships and Sales
The Art of the Pitch: WordPress Relationships and Sales
 
Video Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the FutureVideo Streaming: Then, Now, and in the Future
Video Streaming: Then, Now, and in the Future
 
Essentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FMEEssentials of Automations: The Art of Triggers and Actions in FME
Essentials of Automations: The Art of Triggers and Actions in FME
 
Climate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing DaysClimate Impact of Software Testing at Nordic Testing Days
Climate Impact of Software Testing at Nordic Testing Days
 
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfObservability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
 

Brocade200 E Metro Cluster Design

  • 1. MetroCluster Design and Implementation Guide Jim Lanson, Network Appliance, Inc. May, 2007 | TR-3548 Abstract This document is intended to serve as a guide for architecting and deploying MetroCluster in a customer environment. This design and implementation guide describes the basic MetroCluster architecture, considerations for deployment, and best practices. As always, please refer to the latest technical publications on the NOW™ (NetApp on the Web) site for specific updates on processes, Data ONTAP® command syntax, and the latest requirements, issues, and limitations. This document is intended for field personnel who require assistance in architecting and deploying a MetroCluster solution.
  • 2. Table of Contents 1. Introduction ..................................................................................................................................................4 1.1. Intended Audience .............................................................................................................................4 1.2. Scope .................................................................................................................................................4 1.3. Requirements and Assumptions ........................................................................................................4 2. Overview ......................................................................................................................................................5 2.1. The Basics..........................................................................................................................................5 2.2. Fibre Channel Switch Overview.........................................................................................................9 3. Deployment Planning.................................................................................................................................11 3.1. Data Gathering.................................................................................................................................11 3.2. Distance Considerations ..................................................................................................................12 3.3. Cluster Configuration Checker.........................................................................................................14 3.4. Best Practices and Recommendations ............................................................................................14 4. Installation and Configuration ....................................................................................................................15 4.1. Primary Site......................................................................................................................................15 4.2. Remote Site......................................................................................................................................22 4.3. Testing..............................................................................................................................................24 4.4. Adding More Shelves .......................................................................................................................25 4.5. Best Practices and Recommendations ............................................................................................26 5. Site Failover and Recovery........................................................................................................................28 5.1. Site Failover .....................................................................................................................................28 5.2. Split Brain Scenario..........................................................................................................................28 5.3. Recovery Process ............................................................................................................................28 5.4. Giveback Process ............................................................................................................................29 6. Relevant Documentation ...........................................................................................................................30 6.1. MetroCluster Documentation ...........................................................................................................30 6.2. Brocade Switch Documentation.......................................................................................................30 7. Appendixes ................................................................................................................................................31 7.1. Appendix A: Switch Ports ................................................................................................................31 7.2. Appendix B: Switch Software Upgrade Procedure ..........................................................................32 7.3. Appendix C: Fabric MetroCluster Worksheet .................................................................................33 7.4. Appendix D: Fabric MetroCluster (16-Port Switches) ......................................................................35 7.5. Appendix E: Fabric MetroCluster (8-Port Switches) ........................................................................37 MetroCluster Design & Implementation Guide – version 1.0 2 2
  • 3. Figures and Tables Figure 2-1) Stretch MetroCluster. ...........................................................................................................................5 Figure 2-2) Fabric MetroCluster. ............................................................................................................................6 Figure 2-3) SyncMirror............................................................................................................................................7 Table 2-1) Hardware disk ownership......................................................................................................................8 Figure 2-4) Dual Fabric MetroCluster. ..................................................................................................................10 Figure 2-5) 16-port switch.....................................................................................................................................10 Figure 2-6) 8-port switch.......................................................................................................................................11 Figure 4- 1) Primary node cabling – appliance to switches..................................................................................19 Figure 4-2) Primary node cabling – disk shelf to switch.......................................................................................20 Figure 4-3) Remote node – appliance to switch...................................................................................................23 Figure 4-4) Remote node cabling – disk shelf to switch.......................................................................................24 Figure 4-5) 16-port switch port assignments. .......................................................................................................25 Figure 4-6) 8-port switch port assignments. .........................................................................................................25 Figure 4-7a) Adding disk shelves to primary node. ..............................................................................................26 Figure 4-7b) Adding disk shelves to remote node................................................................................................26 MetroCluster Design & Implementation Guide – version 1.0 3 3
  • 4. 1. Introduction 1.1. Intended Audience The information in this document is intended for field personnel and administrators who are responsible for architecting and deploying successful MetroCluster high-availability and disaster-recovery configurations. A brief overview of MetroCluster is presented in order to establish baseline knowledge before discussing implementation planning, installation, configuration, and operation. 1.2. Scope This document covers the following MetroCluster configurations: • Stretched or non-switched MetroCluster (NetApp storage) • Fabric or switched MetroCluster (NetApp Storage) Topics that apply only to non-switched MetroCluster refer to Stretch MetroCluster. Topics that are specific to NetApp storage-based fabric MetroCluster refer to Fabric MetroCluster. Topics that apply to all configurations refer simply to MetroCluster. Other than a short description, V-Series MetroCluster is not covered in this document. This document refers mostly to Fabric MetroCluster, because Fabric MetroCluster configurations introduce an increased level of complexity in design and implementation. To many, Stretch MetroCluster is simply an active-active configuration with longer cables and mirroring. The introduction of Fibre Channel switches and longer distances requires much more consideration and discussion. The operational sections (creating mirrors, forced takeover, etc.) apply to both. 1.3. Requirements and Assumptions For the methods and procedures described in this document to be useful to the reader, the following assumptions are made: • The reader has at least basic Network Appliance administration skills and has administrative access to the storage system via the command-line interface. • The reader has a full understanding of clustering as it applies to the NetApp storage controller environment. • The reader has at least a basic understanding of Fibre Channel switch technology and operation, along with access to the switches via command line. • In the examples in this report, all administrative commands are performed at the storage system or Fibre Channel switch command line. MetroCluster Design & Implementation Guide – version 1.0 4 4
  • 5. 2. Overview 2.1. The Basics FEATURES MetroCluster configurations consist of a pair of active-active storage controllers configured with mirrored aggregates and extended distance capabilities to create a high-availability disaster recovery solution. The primary benefits include: • Faster high availability and geographic disaster recovery protection • Minimal risk of lost data, easier disaster recovery, and reduced system downtime • Quicker recovery when disaster occurs • Minimal disruption to users and client applications METROCLUSTER TYPES Stretch MetroCluster (sometimes referred to as non-switched) is simply an active-active configuration that can extend up to 500m depending on speed and cable type. It also includes synchronous mirroring (SyncMirror®) and the ability to do a site failover with a single command. See Figure 2-1. Additional resiliency can be provided through the use of multipathing. Total Distance 500m @2Gbps (270m @4Gbps ) Primary Secondary NetApp NetApp CI FAS 3020 FAS 3020 activity status power activity status power DS 14 DS 14 MK2 Network Appl ianc e NetworkAppliance Net work Applia nce Network Appli anc e NetworkAppliance NetworkApplia nce Network Applia nc e NetworkAppliance NetworkApplia nce NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance MK2 AT NetworkApplia nce NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance Network Appl ianc e NetworkAppliance NetworkAppliance Network Appli anc e Network Appl ianc e AT Pow er Pow er F ault F ault Module A Module A Module B Module B Sys tem Sys tem Shelf ID Shelf ID 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 13 12 11 10 09 08 07 06 05 04 03 02 01 00 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 13 12 11 10 09 08 07 06 05 04 03 02 01 00 Primary FC Storage Secondary FC Storage DS 14 DS 14 MK2 MK2 NetworkAppliance Network Appl ianc e NetworkAppliance NetworkAppliance Network Appl ianc e NetworkAppliance NetworkAppliance Network Appli anc e Network Appl ianc e Net work Applia nce Network Applia nc e Network Appl ianc e NetworkAppliance Net work Applia nce NetworkAppliance NetworkAppliance NetworkAppliance NetworkAppliance Network Appl ianc e NetworkAppliance NetworkAppliance Network Appl ianc e NetworkAppliance Net work Applia nce Network Appli anc e NetworkAppliance Net work Applia nce Network Applia nc e AT AT P ower P ower Fau lt Fau lt Module A Module A Module B Module B S ystem S ystem Shelf ID Shelf ID 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 13 12 11 10 09 08 07 06 05 04 03 02 01 00 13 12 11 10 09 08 07 06 05 04 03 02 01 00 Secondary FC Mirror Primary FC Mirror Figure 2-1) Stretch MetroCluster. Fabric MetroCluster (also referred to as switched) uses four Fibre Channel switches in a dual fabric configuration and a separate cluster interconnect card to achieve an even greater distance (up to 100km) between primary and disaster recovery locations. See Figure 2-2. MetroCluster Design & Implementation Guide – version 1.0 5 5
  • 6. Fabric (switched) MetroCluster Total Distance 100Km Primary Node Secondary Node NetApp NetApp FAS 3020 FAS 3020 activity status power activity status po r we CI CI CI CI ISL Brocade 200E Brocade 200E ISL Brocade 200E DS14 DS 14 M K2 MK2 NetworkA pplia nce Net workAppliance Net workAppliance Net workAppliance NetworkA ppliance NetworkA ppliance NetworkA ppliance Network Appliance Network Appliance Ne tworkApplianc e Ne tworkApplianc e Ne tworkApplianc e NetworkA ppliance NetworkA ppliance Netw orkAppliance N etworkAppliance N etworkAppliance N etworkAppliance Netw orkAppliance Netw orkAppliance Netw orkAppliance Ne tworkAppliance Ne tworkAppliance NetworkA ppliance NetworkA ppliance NetworkA ppliance Net workAppliance Net workAppliance AT AT P ow er Power Fault Fault Module A Module A Module B Module B S ystem Sys tem Shelf ID Shelf ID 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 13 12 11 10 09 08 07 06 05 04 03 02 01 00 13 12 11 10 09 08 07 06 05 04 03 02 01 00 Primary FC Storage Secondary FC Storage DS14 DS14 MK2 MK2 Netw ork Appliance Ne tworkAppliance Ne tworkAppliance Ne tworkAppliance Network Appliance Network Appliance Network Appliance Netw orkAppliance Netw orkAppliance N etworkAppliance N etworkAppliance N etworkAppliance Netw orkAppliance Netw orkAppliance N etworkA ppl iance N etworkA ppl iance Netw orkAppliance N etworkA ppliance N etworkA ppliance Net workAppliance N etworkA ppliance Ne tworkAppliance Netw orkApplianc e Netw orkApplianc e N etworkA pplia nce Netw orkAppliance Netw orkAppliance N etworkA ppl iance AT AT P ow er P ower Fault Fault Module A Module A Module B Module B System System Shelf ID Shelf ID 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 212A 13 12 11 10 09 08 07 06 05 04 03 02 01 00 13 12 11 10 09 08 07 06 05 04 03 02 01 00 Secondary FC Mirror Primary FC Mirror Figure 2-2) Fabric MetroCluster. V-Series MetroCluster is simply either of the above configurations with a NetApp V-Series system. Because of the architectural differences between V-Series and a standard active-active configuration, V-Series MetroCluster has additional flexibility when it comes to the maximum number of spindles and the Fibre Channel switches. See the V-Series Compatibility Guide on the NOW site for additional information. The implementation of V-Series MetroCluster is outside the scope of this document. See the V-Series documentation on the NOW site. For more information about MetroCluster, see the Data ONTAP Cluster Installation and Administration Guide (for Data ONTAP 7.1) or the Data ONTAP Active-Active Configuration Guide (for Data ONTAP 7.2 or later). (For Data ONTAP 7.0 and earlier, see the Data Protection Online Backup and Recovery Guide and the NetApp Hardware Cluster Guide.) Note: When referring to the documentation just listed, use the wiring diagrams in this technical report. The diagrams in this document facilitate easier installation and expansion of MetroCluster. MIRRORING NetApp SyncMirror, an integral part of MetroCluster, combines the disk-mirroring protection of RAID1 with NetApp industry-leading RAID4 and RAID-DP™ technology. In the event of an outage—whether it's due to a disk problem, cable break, or host bus adapter (HBA) failure—SyncMirror can instantly access the mirrored data without any operator intervention or disruption to client applications. SyncMirror maintains a strict physical separation between the two copies of your mirrored data. Each of these copies is referred to as a plex. Each controller’s data has its “mirror” at the other site. MetroCluster Design & Implementation Guide – version 1.0 6 6
  • 7. Aggregate Located at Partner Site Mirror Plex0 Plex1 Pool0 Pool1 Figure 2-3) SyncMirror Pool s and Plexes. When SyncMirror is licensed and hardware ownership is used (discussed in quot;Disk Ownershipquot;), spare disks are split into two pools—Pool0 and Pool1. Each plex of a mirror uses disks from separate pools. Disks assigned to Pool0 must be on separate loops from disks assigned to Pool1. When software ownership is used, disks are explicitly assigned by the administrator, as described in “Disk Ownership.” To maximize availability, Pool0 and Pool1 disks need to be on separate loops and use separate HBAs, cables, and shelves. Before enabling the SyncMirror license, ensure that disks for each pool are located on the appropriate loops that are fault isolated from each other. DISK OWNERSHIP In a MetroCluster where disk shelves at either side are mirrored and thus accessible by either controller, disk ownership comes into play. There are two methods of establishing disk ownership: hardware and software. Hardware-based ownership is the default for all platforms except V-Series, FAS 6000 series, and FAS 3040/3070. Software disk ownership capability became available in Data ONTAP 6.3.1. A brief description of each method follows. For more detail, see quot;Installation and Configuration,quot; later in this document or the Data ONTAP documentation. Hardware disk ownership establishes which controller owns which disks by how the shelves are connected. For more information, see the System Configuration Guide: http://now.netapp.com/NOW/knowledge/docs/hardware/NetApp/syscfg/ MetroCluster Design & Implementation Guide – version 1.0 7 7
  • 8. Model Pool Ownership FAS9xx Slots 2, 3, 4, 5, and 7 are Pool0 Slots 8, 9, and 10 are Pool1 Optional software-based ownership and pool selection (as of Data ONTAP 7.1.1) FAS3020/ 3050 0a, 0b, and slots 1 and 2 are Pool0 (slot 1 is usually NVRAM) 0c, 0d and slots 3 and 4 are Pool1 Optional software-based ownership and pool selection (as of Data ONTAP 7.1.1) FAS3040//3070 Software-based ownership and pool selection FAS60xx Software-based ownership and pool selection V-Series Software-based ownership and pool selection Table 2-1) Hardware disk ownership. Note: Physical connectivity is extremely important to ensure adequate distribution of disks between the two pools. Software disk ownership allows greater flexibility in cabling by allowing disk ownership to be assigned through explicit commands. Rather than determine which controller owns the disk by hardware connection, the ownership information is written on each disk. Because disk ownership can be implemented by a system administrator, it is important to perform the configuration to maximize availability. For more information, see quot;Installation and Configuration,quot; later in this document. FIBRE CHANNEL SAN IN A METROCLUSTER WORLD For those who are experienced in Fibre Channel technology, and storage area networks (SANs) in particular, there are some differences and restrictions worth discussing relative to how Fabric MetroCluster utilizes this technology. Fabric MetroCluster configurations use Fibre Channel switches as the means to separate the controllers by a greater distance. The switches are connected between the controller heads and the disk shelves, and to each other. Each disk spindle or LUN individually logs into a Fibre Channel fabric. Except for the V-Series, the nature of this architecture requires, for performance reasons, that the two fabrics be completely dedicated to Fabric MetroCluster. Extensive testing was performed to ensure adequate performance with switches included in a Fabric MetroCluster configuration. For this reason, Fabric MetroCluster requirements prohibit the use of any other model or vendor of Fibre Channel switch than the Brocade 200E included with the Fabric MetroCluster. Also for performance reasons there is a current maximum of 336 spindles. Higher spindle count solutions will be available in the future. Keep in mind that this is for Fabric MetroCluster only. The maximum number of disks in a Stretch MetroCluster depends solely on the NetApp model. In a traditional SAN, there is great flexibility in connecting devices to ports as long as the ports are configured correctly and any zoning requirements are met. In a MetroCluster, when using hardware ownership, Data ONTAP expects certain devices to be connected to specific ports or ranges of ports. It is therefore critical that cabling be exactly as described in the installation procedures. Also, no switch- specific functions such as trunking or zoning are currently used in a NetApp Fabric MetroCluster (non-V- Series), making switch management minimal. RESTRICTIONS For further restrictions in using a Fabric MetroCluster configuration, see the Data ONTAP Cluster Installation and Administration Guide (for Data ONTAP version 7.1) or the Data ONTAP Active-Active MetroCluster Design & Implementation Guide – version 1.0 8 8
  • 9. Configuration Guide (for Data ONTAP version 7.2 or later). (For Data ONTAP 7.0 and earlier, see the Data Protection Online Backup and Recovery Guide and the NetApp Hardware Cluster Guide.) COMPONENTS A NetApp fabric includes the following components: • A clustered pair of FAS900, 3000, or 6000 series controllers running Data ONTAP 6.4.1 or later (See the MetroCluster Compatibility Matrix on the NOW site for supported models.) • Four Brocade Fibre Channel switches with supported firmware (See the MetroCluster Compatibility Matrix on the NOW site for supported models.) There is a pair at each location. Supported models may differ between locations but must be the same at each given location. • Brocade Extended Distance license (if over 10km) • Brocade Full-Fabric License • Brocade Ports-On-Demand (POD) licenses for second 8 ports (if using a 16-port Brocade 200E switch) • A VI-MC cluster adapter • A syncmirror_local license • A cluster_remote license • A cluster license METROCLUSTER VERSUS STANDARD SYNCHRONOUS REPLICATION As a disaster recovery solution, MetroCluster is often compared with other synchronous replication products. Even though it includes SyncMirror, MetroCluster can be differentiated by the following features: • Low-level RAID mirroring (less performance impact) • Site failover with a single command • No extensive scripting required to make data available after failover 2.2. Fibre Channel Switch Overview OPERATION OVERVIEW The NetApp Fabric or switched MetroCluster configuration uses four Brocade 200E switches in a dual fabric configuration to connect two active-active controllers. These switches cannot be combined with any other switch model. SWITCH FABRIC CONFIGURATION A Fabric MetroCluster contains two switch fabrics. A switch fabric consists of a switch on the primary controller connected to a switch on the remote controller (see Figure 2-1). The two switches are connected to each other through ISL (Inter-Switch Link) cables. MetroCluster Design & Implementation Guide – version 1.0 9 9
  • 10. Fabric 1 SW1 SW3 Fabric 2 SW2 SW4 Figure 2-4) Dual Fabric MetroCluster. Figure 2-4 shows a Fabric MetroCluster configuration in which the first fabric begins at “SWITCH 1” on the primary side and is completed by connecting the ISL cable to the first switch, “SWITCH 3“, on the remote side. The second fabric is created by using “SWITCH 2” on the primary side, connected through a second ISL cable to the second switch “SWITCH 4” on the remote side. The reason for two fabrics is redundancy. The loss of a switch in a fabric or the loss of a fabric will not affect the availability of the Fabric MetroCluster. Note: Only one ISL is supported per fabric. SWITCH PORT ALLOCATION For purposes of this discussion, hardware disk ownership is assumed. The Fibre Channel switch ports are divided into banks and pools. Each switch has two banks, dividing the switch into two equal parts. (See Figures 2-5 and 2-6.) A storage controller connects to a bank through its Fibre Channel ports (HBA or embedded). It then owns the disks that are connected to ports of the opposite bank. If the storage system connects to bank 1, only shelves connected to bank 2 belong to it. Additionally, Pool0 disks are considered “local” – all reads will be satisfied from those disks only. Pool1 disks are “remote”; they will be written to, but cannot be read from under normal circumstances. Together, these two rules limit the usable ports for a certain disk pool to 4 on a 16-port switch, and to 2 on an 8-port switch. 16-PORT SWITCHES As shown in Figure 2-5, ports 0-3 and 8-11 belong to Pool0, while ports 4-7 and 12-15 belong to Pool1. The Brocade 200E 16-port switch has ports 0-15 licensed and operational, as shown in Figure 2-5. Note: Pools are identified only as either Pool0 or Pool1. However, Data ONTAP can assign arbitrary numbers to plexes. 16 Port Switch Pool 0 1 Figure 2-5) 16-port switch. 8-PORT SWITCHES The 8-port Brocade 200E switch physically has 16 ports and looks like the 16-port version. However, only ports 0-7 are licensed, have small form-factor Pluggables (SFPs), and are operational, as shown in Figure 2-6. MetroCluster Design & Implementation Guide – version 1.0 10 10
  • 11. Note: Do not install SFPs in unused ports 8-15. This might cause Data ONTAP to operate as if it's a 16- port switch and erroneously assign banks, quadrants, and pools to unused ports. As shown in Figure 2-6, ports 0-1 and 4-5 belong to Pool0, while ports 2-3 and 6-7 belong to Pool1. 8 Port Switch Pool 0 Pool 1 Figure 2-6) 8-port switch. LICENSES A number of licenses are required for the Fibre Channel switches. When ordered with a Fabric MetroCluster configuration, the switches should include all necessary licenses. For reference, they are: • Full Fabric License • Extended Distance License (if over 10km) • Brocade Ports-On-Demand (POD) licenses for second 8 ports (if using a 16-port Brocade 200E switch) 3. Deployment Planning 3.1. Data Gathering CUSTOMER REQUIREMENTS To facilitate a successful MetroCluster installation, the following information should be gathered early on. ENVIRONMENTAL Distance between the primary and remote sites. This information is necessary to determine which type of MetroCluster is appropriate, or even whether either version is appropriate. In calculating the effective distance, factors such as cable type, speed, and number of patch panels must be considered. (See Section 3.2, quot;Distance Considerations.quot;) Although NetApp recommends that dedicated dark fiber be used for a MetroCluster, WDM devices are supported. Refer to the Brocade Compatibility Guide at www.brocade.com for supported devices. SETUP Gather the following items before you begin any MetroCluster installation: NetApp and Fibre Channel switch licenses Hostnames and IP addresses for each of the nodes and the Fibre Channel switches Brocade switch licenses Cables MetroCluster Design & Implementation Guide – version 1.0 11 11
  • 12. 3.2. Distance Considerations OVERVIEW Stretch MetroCluster can support a maximum of 500 meters between nodes at a speed of 2Gbps. Fabric MetroCluster, through the use of Fibre Channel switches, extends this distance to 100km at the same speed. At 4Gbps speeds, these distances are roughly cut in half. This extended distance capability gives customer greater flexibility in the physical location of the active-active controllers while maintaining the high-availability benefits of clustered failover. This section describes a number of factors that affect the overall effective distance permissible between the MetroCluster nodes. • Physical distance • Number of connections • Desired speed • Cable type PHYSICAL DISTANCE As stated earlier, the Stretch MetroCluster configuration can extend to a maximum of 500m (2Gbps). This distance is reduced by speed, cable type, and number of connections. A Fabric MetroCluster can extend out to 100km. This distance is affected by the same factors. At a distance of 100km, latency would be around 1ms. Greater distances would obviously result in greater latencies (500km = 5ms), which may be unacceptable to an application. CABLE TYPE As shown in Table 3.1, the cable type affects both distance and speed. Single-mode cable is supported only for the Inter-Switch Links. Example 1: A customer has 250 meters between sites and wants to run at 4Gbps. The OM-3 cable type is required. Example 2: A customer currently has a MetroCluster configuration running at 2Gbps with a distance of 300 meters over OM2 cabling and wants to upgrade to 4Gbps speeds. Upgrading the cabling will not help, because OM3 has a maximum of 270 meters. In this case the choices would be: • Remain at 2Gbps speeds. Customers with the new ESH4 disk shelves could still use them at this distance, as long as the shelf speed is set to 2Gbps. • Test current optical network infrastructure to make sure that attenuation and latency are acceptable. Fibre Type Data Rate Max Distance (M) 1 Gb/s 500 OM-2 (50/125UM) 2 Gb/s 300 4Gb/s 150 1 Gb/s 860 OM-3 (50/125UM) 2 Gb/s 500 4Gb/s 270 OS1 Single Mode 2 Gb/s 10,000* (9/125UM) 4Gb/s 10,000* Table 3-1) Cables, speed, and distance. MetroCluster Design & Implementation Guide – version 1.0 12 12