havcs-410-101 a-2-10-srt-pg_4

783 views

Published on

Published in: Technology, Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
783
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
81
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

havcs-410-101 a-2-10-srt-pg_4

  1. 1. VERITAS Cluster Server for UNIX, Implementing Local Clusters HA-VCS-410-101A-2-10-SRT (100-002148)
  2. 2. COURSE DEVELOPERS Bilge Gerrits Siobhan Seeger Dawn Walker LEAD SUBJECT MATTER EXPERTS Geoff Bergren Connie Economou Paul Johnston Dave Rogers Pete Toemmes Jim Senicka TECHNICAL CONTRIBUTORS AND REVIEWERS Billie Bachra Barbara Ceran Gene Henriksen Bob Lucas Disclaimer The information contained in this publication is subject to change without notice. VERITAS Software Corporation makes no warranty of any kind with regard to this guide, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. VERITAS Software Corporation shall not be liable for errors contained herein or for incidental or consequential damages in connection with the furnishing, performance, or use of this manual. Copyright Copyright © 2005 VERITAS Software Corporation. All rights reserved. No part of the contents of this training material may be reproduced in any form or by any means or be used for the purposes of training or education without the written permission of VERITAS Software Corporation. Trademark Notice VERITAS, the VERITAS logo, and VERITAS FirstWatch, VERITAS Cluster Server, VERITAS File System, VERITAS Volume Manager, VERITAS NetBackup, and VERITAS HSM are registered trademarks of VERITAS Software Corporation. Other product names mentioned herein may be trademarks and/or registered trademarks of their respective companies. VERITAS Cluster Server for UNIX, Implementing Local Clusters Participant Guide April 2005 Release VERITAS Software Corporation 350 Ellis Street Mountain View, CA 94043 Phone 650–527–8000 www.veritas.com
  3. 3. Table of Contents i Copyright © 2005 VERITAS Software Corporation. All rights reserved. Course Introduction VERITAS Cluster Server Curriculum ................................................................ Intro-2 Course Prerequisites......................................................................................... Intro-3 Course Objectives............................................................................................. Intro-4 Lesson 1: Workshop: Reconfiguring Cluster Membership Introduction ............................................................................................................. 1-2 Workshop Overview................................................................................................ 1-4 Task 1: Removing a System from a Running VCS Cluster..................................... 1-5 Objective................................................................................................................... 1-5 Assumptions.............................................................................................................. 1-5 Procedure for Removing a System from a Running VCS Cluster............................ 1-6 Solution to Class Discussion 1: Removing a System ............................................... 1-9 Commands Required to Complete Task 1 .............................................................. 1-11 Solution to Class Discussion 1: Commands for Removing a System .................... 1-14 Lab Exercise: Task 1—Removing a System from a Running Cluster.................... 1-18 Task 2: Adding a New System to a Running VCS Cluster.................................... 1-19 Objective................................................................................................................. 1-19 Assumptions............................................................................................................ 1-19 Procedure to Add a New System to a Running VCS Cluster ................................. 1-20 Solution to Class Discussion 2: Adding a System.................................................. 1-23 Commands Required to Complete Task 2 .............................................................. 1-25 Solution to Class Discussion 2: Commands for Adding a System......................... 1-28 Lab Exercise: Task 2—Adding a New System to a Running Cluster .................... 1-32 Task 3: Merging Two Running VCS Clusters........................................................ 1-33 Objective................................................................................................................. 1-33 Assumptions............................................................................................................ 1-33 Procedure to Merge Two VCS Clusters.................................................................. 1-34 Solution to Class Discussion 3: Merging Two Running Clusters .......................... 1-37 Commands Required to Complete Task 3 .............................................................. 1-39 Solution to Class Discussion 3: Commands to Merge Clusters.............................. 1-42 Lab Exercise: Task 3—Merging Two Running VCS Clusters............................... 1-46 Lab 1: Reconfiguring Cluster Membership............................................................ 1-48 Lesson 2: Service Group Interactions Introduction ............................................................................................................. 2-2 Common Application Relationships ........................................................................ 2-4 Online on the Same System...................................................................................... 2-4 Online Anywhere in the Cluster ............................................................................... 2-5 Online on Different Systems..................................................................................... 2-6 Offline on the Same System ..................................................................................... 2-7 Service Group Dependency Definition .................................................................... 2-8 Startup Behavior Summary....................................................................................... 2-8 Failover Behavior Summary..................................................................................... 2-9 Table of Contents
  4. 4. ii VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Service Group Dependency Examples ................................................................. 2-10 Online Local Dependency...................................................................................... 2-10 Online Global Dependency.................................................................................... 2-14 Online Remote Dependency .................................................................................. 2-16 Offline Local Dependency ..................................................................................... 2-18 Configuring Service Group Dependencies............................................................ 2-19 Service Group Dependency Rules ......................................................................... 2-19 Creating Service Group Dependencies .................................................................. 2-20 Removing Service Group Dependencies ............................................................... 2-20 Alternative Methods of Controlling Interactions..................................................... 2-21 Limitations of Service Group Dependencies ......................................................... 2-21 Using Resources to Control Service Group Interactions ....................................... 2-22 Using Triggers to Control Service Group Interactions .......................................... 2-24 Lab 2: Service Group Dependencies .................................................................... 2-26 Lesson 3: Workload Management Introduction ............................................................................................................. 3-2 Startup Rules and Policies...................................................................................... 3-4 Rules for Automatic Service Group Startup ............................................................. 3-4 Automatic Startup Policies........................................................................................ 3-5 Failover Rules and Policies................................................................................... 3-10 Rules for Automatic Service Group Failover......................................................... 3-10 Failover Policies...................................................................................................... 3-11 Integrating Dynamic Load Calculations ................................................................ 3-15 Controlling Overloaded Systems........................................................................... 3-16 The LoadWarning Trigger ..................................................................................... 3-16 Example Script....................................................................................................... 3-17 Additional Startup and Failover Controls............................................................... 3-18 Limits and Prerequisites......................................................................................... 3-18 Selecting a Target System...................................................................................... 3-19 Combining Capacity and Limits ............................................................................ 3-20 Configuring Startup and Failover Policies............................................................. 3-21 Setting Load and Capacity ..................................................................................... 3-21 Setting Limits and Prerequisites............................................................................. 3-22 Using the Simulator............................................................................................... 3-24 Modeling Workload Management ......................................................................... 3-24 Lab 3: Testing Workload Management ................................................................. 3-26 Lesson 4: Alternate Storage and Network Configurations Introduction ............................................................................................................. 4-2 Alternative Storage and Network Configurations .................................................... 4-4 The Disk Resource and Agent on Solaris ................................................................. 4-5 The DiskReservation Resource and Agent on Solaris .............................................. 4-5 The LVMVolumeGroup Agent on AIX.................................................................... 4-6 LVM Setup on HP-UX.............................................................................................. 4-7 The LVMVolumeGroup Resource and Agent on HP-UX........................................ 4-8 LVMLogicalVolume Resource and Agent on HP-UX ............................................. 4-9
  5. 5. Table of Contents iii Copyright © 2005 VERITAS Software Corporation. All rights reserved. LVMCombo Resource and Agent on HP-UX .......................................................... 4-9 The DiskReservation Resource and Agent on Linux.............................................. 4-10 Alternative Network Configurations....................................................................... 4-11 Network Resources Overview ................................................................................ 4-13 Additional Network Resources.............................................................................. 4-14 The MultiNICA Resource and Agent ..................................................................... 4-14 MultiNICA Resource Configuration....................................................................... 4-17 MultiNICA Failover................................................................................................ 4-20 The IPMultiNIC Resource and Agent..................................................................... 4-21 IPMultiNIC Failover............................................................................................... 4-25 Additional Network Design Requirements............................................................. 4-26 MultiNICB and IPMultiNICB ................................................................................ 4-26 How the MultiNICB Agent Operates ..................................................................... 4-27 The MultiNICB Resource and Agent ..................................................................... 4-29 The IPMultiNICB Resource and Agent.................................................................. 4-36 Configuring IPMultiNICB...................................................................................... 4-37 The MultiNICB Trigger.......................................................................................... 4-39 Example MultiNIC Setup....................................................................................... 4-40 Comparing MultiNICA and MultiNICB................................................................. 4-41 Testing Local Interface Failover............................................................................. 4-42 Lab 4: Configuring Multiple Network Interfaces .................................................... 4-44 Lesson 5: Maintaining VCS Introduction ............................................................................................................. 5-2 Making Changes in a Cluster Environment............................................................. 5-4 Replacing a System................................................................................................... 5-4 Preparing for Software and Hardware Upgrades...................................................... 5-5 Operating System Upgrade Example........................................................................ 5-6 Performing a Rolling Upgrade in a Running Cluster................................................ 5-7 Upgrading VERITAS Cluster Server ....................................................................... 5-8 Preparing for a VCS Upgrade................................................................................... 5-8 Upgrading to VCS 4.x from VCS 1.3—3.5.............................................................. 5-9 Upgrading from VCS QuickStart to VCS 4.x......................................................... 5-10 Other Upgrade Considerations................................................................................ 5-11 Alternative VCS Installation Methods.................................................................... 5-12 Options to the installvcs Utility .............................................................................. 5-12 Options and Features of the installvcs Utility......................................................... 5-12 Manual Installation Procedure................................................................................ 5-14 Licensing VCS........................................................................................................ 5-16 Creating a Single-Node Cluster .............................................................................. 5-17 Staying Informed................................................................................................... 5-18 Obtaining Information from VERITAS Support.................................................... 5-18 Lesson 6: Validating VCS Implementation Introduction ............................................................................................................. 6-2 VCS Best Practices Review.................................................................................... 6-4 Cluster Interconnect.................................................................................................. 6-4
  6. 6. iv VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Shared Storage .......................................................................................................... 6-5 Public Network.......................................................................................................... 6-6 Failover Configuration.............................................................................................. 6-7 External Dependencies.............................................................................................. 6-8 Testing....................................................................................................................... 6-9 Other Considerations.............................................................................................. 6-10 Solution Acceptance Testing ................................................................................ 6-11 Examples of Solution Acceptance Testing ............................................................ 6-12 Knowledge Transfer.............................................................................................. 6-13 System and Network Administration..................................................................... 6-13 Application Administration.................................................................................... 6-14 The Implementation Report ................................................................................... 6-15 High Availability Solutions..................................................................................... 6-16 Local Cluster with Shared Storage......................................................................... 6-16 Campus or Metropolitan Shared Storage Cluster................................................... 6-17 Replicated Data Cluster (RDC).............................................................................. 6-18 Wide Area Network (WAN) Cluster for Disaster Recovery ................................. 6-19 High Availability References................................................................................. 6-20 VERITAS High Availability Curriculum .............................................................. 6-22 Appendix A: Lab Synopses Lab 1 Synopsis: Reconfiguring Cluster Membership .............................................. A-2 Lab 2 Synopsis: Service Group Dependencies....................................................... A-7 Lab 3 Synopsis: Testing Workload Management.................................................. A-14 Lab 4 Synopsis: Configuring Multiple Network Interfaces..................................... A-20 Appendix B: Lab Details Lab 1 Details: Reconfiguring Cluster Membership.................................................. B-3 Lab 2 Details: Service Group Dependencies ........................................................ B-17 Lab 3 Details: Testing Workload Management ..................................................... B-29 Lab 4 Details: Configuring Multiple Network Interfaces ........................................ B-37 Appendix C: Lab Solutions Lab Solution 1: Reconfiguring Cluster Membership................................................ C-3 Lab 2 Solution: Service Group Dependencies ...................................................... C-25 Lab 3 Solution: Testing Workload Management ................................................... C-45 Lab 4 Solution: Configuring Multiple Network Interfaces ...................................... C-63 Appendix D: Job Aids Service Group Dependencies—Definitions............................................................. D-2 Service Group Dependencies—Failover Process................................................... D-6 Appendix E: Design Worksheet: Template Index
  7. 7. Course Introduction
  8. 8. Intro–2 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. VERITAS Cluster Server Curriculum The VERITAS Cluster Server curriculum is a series of courses that are designed to provide a full range of expertise with VERITAS Cluster Server (VCS) high availability solutions—from design through disaster recovery. VERITAS Cluster Server for UNIX, Fundamentals This course covers installation and configuration of common VCS configurations, focusing on two-node clusters running application and database services. VERITAS Cluster Server for UNIX, Implementing Local Clusters This course focuses on multinode VCS clusters and advanced topics related to more complex cluster configurations. VERITAS Cluster Server Agent Development This course enables students to create and customize VCS agents. High Availability Design Using VERITAS Cluster Server This course enables participants to translate high availability requirements into a VCS design that can be deployed using VERITAS Cluster Server. Disaster Recovery Using VVR and Global Cluster Option This course covers cluster configurations across remote sites, including Replicated Data Clusters (RDCs) and the Global Cluster Option for wide-area clusters. Learning Path VERITAS Cluster Server, Implementing Local Clusters Disaster Recovery Using VVR and Global Cluster Option High Availability Design Using VERITAS Cluster Server VERITAS Cluster Server, Fundamentals VERITAS Cluster Server Curriculum VERITAS Cluster Server Agent Development
  9. 9. Course Introduction Intro–3 Copyright © 2005 VERITAS Software Corporation. All rights reserved. Course Prerequisites This course assumes that you have complete understanding of the fundamentals of the VERITAS Cluster Server (VCS) product. You should understand the basic components and functions of VCS before you begin to implement a high availability environment using VCS. You are also expected to have expertise in system, storage, and network administration of UNIX systems. Course Prerequisites To successfully complete this course, you are expected to have: The level of experience gained in the VERITAS Cluster Server Fundamentals course: – Understanding VCS terms and concepts – Using the graphical and command-line interfaces – Creating and managing service groups – Responding to resource, system, and communication faults System, storage, and network administration expertise with one or more UNIX-based operating systems
  10. 10. Intro–4 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Course Objectives In the VERITAS Cluster Server Implementing Local Clusters course, you are given a high availability design to implement in the classroom environment using VERITAS Cluster Server. The course simulates the job tasks that you perform to configure advanced cluster features. Lessons build upon each other, exhibiting the processes and recommended best practices that you can apply to implementing any design cluster. The core material focuses on the most common cluster implementations. Other cluster configurations emphasizing additional VCS capabilities are provided to illustrate the power and flexibility of VERITAS Cluster Server. Course Objectives After completing the VERITAS Cluster Server Implementing Local Clusters course, you will be able to: Reconfigure cluster membership to add and remove systems from a cluster. Configure dependencies between service groups. Manage workload among cluster systems. Implement alternative storage and network configurations. Perform common maintenance tasks. Validate your cluster implementation.
  11. 11. Course Introduction Intro–5 Copyright © 2005 VERITAS Software Corporation. All rights reserved. Lab Design for the Course The diagram shows a conceptual view of the cluster design used as an example throughout this course and implemented in hands-on lab exercises. Each aspect of the cluster configuration is described in greater detail where applicable in course lessons. The cluster consists of: • Four nodes • Three to five high availability services, including Oracle • Fibre connections to SAN shared storage from each node through a switch • Two Ethernet interfaces for the private cluster heartbeat network • Ethernet connections to the public network Additional complexity is added to the design to illustrate certain aspects of cluster configuration in later lessons. The design diagram shows a conceptual view of the cluster design described in the worksheet. Lab Design for the Course vcs1 name1SG1, name1SG2 name2SG1, name2SG2 NetworkSG
  12. 12. Intro–6 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Course Overview This training provides comprehensive instruction on the deployment of advanced features of VERITAS Cluster Server (VCS). The course focuses on multinode VCS clusters and advanced topics related to more complex cluster configurations, such as service group dependencies and workload management. Course Overview Lesson 1: Reconfiguring Cluster Membership Lesson 2: Service Group Interactions Lesson 3: Workload Management Lesson 4: Storage and Network Alternatives Lesson 5: Maintaining VCS Lesson 6: Validating VCS Implementation
  13. 13. Lesson 1 Workshop: Reconfiguring Cluster Membership
  14. 14. 1–2 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Introduction Overview This lesson is a workshop to teach you to think through impacts of changing the cluster configuration while maximizing the application services availability and plan accordingly. The workshop also provides the means of reviewing everything you have learned so far about VCS clusters. Importance To maintain existing VCS clusters and clustered application services, you may be required to add or remove systems to and from existing VCS clusters or merge clusters to consolidate servers. You need to have a very good understanding of how VCS works and how the configuration changes impact the application services availability before you can plan and execute these changes in a cluster. Lesson Introduction Lesson 1: Reconfiguring Cluster Membership Lesson 2: Service Group Interactions Lesson 3: Workload Management Lesson 4: Storage and Network Alternatives Lesson 5: Maintaining VCS Lesson 6: Validating VCS Implementation
  15. 15. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–3 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Outline of Topics • Task 1: Removing a System • Task 2: Adding a System • Task 3: Merging Two Running VCS Clusters Labs and solutions are located on the following pages. “Lab 1 Synopsis: Reconfiguring Cluster Membership,” page A-2 “Lab 1 Details: Reconfiguring Cluster Membership,” page B-3 “Lab Solution 1: Reconfiguring Cluster Membership,” page C-3 Merge two running VCS clusters.Task 3: Merging Two Running Clusters Add a new system to a running VCS cluster. Task 2: Adding a System Remove a system from a running cluster. Task 1: Removing a System After completing this lesson, you will be able to: Topic Lesson Topics and Objectives
  16. 16. 1–4 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Workshop Overview During this workshop, you will change two 2-node VCS clusters into a 4-node VCS cluster with the same application services. The workshop is carried out in three parts: • Task 1: Removing a system from a running VCS cluster • Task 2: Adding a new system to a running VCS cluster • Task 3: Merging two running VCS clusters Note: During this workshop students working on two clusters need to team up to carry out the discussions and the lab exercises. Each task has three parts: 1 Your instructor will first describe the objective and the assumptions related to the task. Then you will be asked as a team to provide a procedure to accomplish the task while maximizing application services availability. You will then review the procedure in the class discussing the reasons behind each step. 2 After you have identified the best procedure for the task, you will be asked as a team to provide the VCS commands to carry out each step in the procedure. This will again be followed up by a classroom discussion to identify the possible solutions to the problem. 3 After the task is planned in detail, you carry out the task as a team on the lab systems in the classroom. You need to complete one task before proceeding to the next. Reconfiguring Cluster Membership B A A B B A D C C D C C D C D B B C DD 1 2 3 4 3 4 4 2 2 2 1 1 3 DC B B C D AA Task 1 Task 2 Task 3 D A C A
  17. 17. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–5 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Task 1: Removing a System from a Running VCS Cluster Objective The objective of this task is to take a system out of a running VCS cluster and to remove the VCS software on the system with minimal or no impact on application services. Assumptions Following is a list of assumptions that you need to take into account while planning a procedure for this task: • The VCS cluster consists of two or more systems, all of which are up and running. • There are multiple service groups configured in the cluster. All of the service groups are online somewhere in the cluster. Note that there may also be online service groups on the system that need to be removed from the cluster. • The application services that are online on the system to be removed from the cluster can be switched over to other systems in the cluster. – Although there are multiple service groups in the cluster, this assumption implies that there are no dependencies that need to be taken into account. – There are also no service groups that are configured to run only on the system to be removed from the cluster. • All the VCS software should be removed from the system because it is no longer part of a cluster. However, there is no need to remove any application software from the system. Task 1: Removing a System from a Running VCS Cluster Objective To remove a system from a running VCS cluster while minimizing application and VCS downtime Assumptions – The cluster has two or more systems. – There are multiple service groups, some of which may be running on the system to be removed. – All application services should be kept under the cluster control. – There is nothing to restrict switching over application services to the remaining systems in the cluster. – VCS software should be removed from the system taken out of the cluster. X
  18. 18. 1–6 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Procedure for Removing a System from a Running VCS Cluster Discuss with your class or team the steps required to carry out Task 1. For each step, decide how the application services availability would be impacted. Note that there may not be a single answer to this question. Therefore, state your reasons for choosing a step in a specific order using the Notes area of your worksheet. Also, in the Notes area, document any assumptions that you are making that have not been explained as part of the task. Use the worksheet on the following page to provide the steps required for Task 1. Classroom Discussion for Task 1 Your instructor either groups students into teams or leads a class discussion for this task. For team-based exercises: Each group of four students, working on two clusters, forms a team to discuss the steps required to carry out task 1 as outlined on the previous slide. After all the teams are ready with their proposed procedures, have a classroom discussion to identify the best way of removing a system from a running VCS cluster, providing the reasons for each step. Note: At this point, you do not need to provide the commands to carry out each step. Note: At this point, you do not need to provide the commands to carry out each step. X
  19. 19. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–7 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Procedure for Task 1 proposed by your team or class: Steps Description Impact on application availability Notes
  20. 20. 1–8 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Use the following worksheet to document the procedure agreed upon in the classroom. Final procedure for Task 1 agreed upon as a result of classroom discussions: Steps Description Impact on application availability Notes
  21. 21. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–9 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Solution to Class Discussion 1: Removing a System 1 Open the configuration and prevent application failover to the system to be removed. 2 Switch any application services that are running on the system to be removed to any other system in the cluster. Note: This step can be combined with either step 1 as an option to a single command line. 3 Close the configuration and stop VCS on the system to be removed. 4 Remove any disk heartbeat configuration on the system to be removed. Notes: – You need to remove both the GAB disk heartbeats and service group heartbeats. – After you remove the GAB disk heartbeats, you may also remove the corresponding lines in the /etc/gabtab file that starts the disk heartbeat so that the disk heartbeats are not started again in case the system crashes and is rebooted before you remove the VCS software. 5 Stop VCS communication modules (GAB, LLT) and I/O fencing on the system to be removed. Note: On the Solaris platform, you also need to unload the kernel modules. 6 Physically remove cluster interconnect links from the system to be removed. 7 Remove VCS software from the system taken out of the cluster. Notes: – You can either use the uninstallvcs script to automate the removal of the VCS software or use the command specific to the operating platform, such as pkgadd for Solaris, swinstall for HP-UX, installp -a for AIX, or rpm for Linux, to remove the VCS software packages individually. – If you have remote shell access (rsh or ssh) for root between the cluster systems, you can run uninstallvcs on any system in the cluster. Otherwise, you have to run the script on the system to be removed. – You may need to manually remove configuration files and VCS directories that include customized scripts. 8 Update service group and resource configurations that refer to the system that is removed. Note: Service group attributes, such as AutoStartList, SystemList, SystemZones, and localized resource attributes, such as Device for NIC or IP resource types, may need to be modified. 9 Remove the system from the cluster configuration.
  22. 22. 1–10 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. 10 Modify the VCS communication configuration files on the remaining systems in the cluster to reflect the change. Note: You do not need to stop and restart LLT and GAB on the remaining systems when you make changes to the configuration files unless the /etc/llttab file contains the following directives that need to be changed: – include system_id_range – exclude system_id_range – set-addr systemid tag address For more information on these directives, check the VCS manual pages on llttab.
  23. 23. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–11 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Commands Required to Complete Task 1 After you have agreed on the steps required to accomplish Task 1, determine which VCS commands are used to carry out each step in the procedure. You will first work as a team to propose a solution, and then discuss each step in the classroom. Note that there may be multiple methods to carry out each step. You can use the Participant Guide, VCS manual pages, the VERITAS Cluster Server User’s Guide, and the VERITAS Cluster Server Installation Guide as sources of information. If there are topics that you do not feel comfortable with, ask your instructor to discuss them in detail during the classroom discussion. Use the worksheet on the following page to provide the commands required for Task 1. VCS Commands Required for Task 1 Provide the commands to carry out each step in the recommended procedure for removing a system from a running VCS cluster. You may need to refer to previous lessons, VCS manuals, or manual pages to decide on the specific commands and their options. For each step, complete the worksheet provided in the Participant Guide and include the command, the system to run it on, and any specific notes. X Note: When you are ready, your instructor will discuss each step in detail. Note: When you are ready, your instructor will discuss each step in detail.
  24. 24. 1–12 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Commands for Task 1 proposed by your team: Order of Execution VCS Command to Use System on which to run the command Notes
  25. 25. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–13 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Use the following worksheet to document any differences to your proposal. Commands for Task 1 agreed upon in the classroom: Order of Execution VCS Command to Use System on which to run the command Notes
  26. 26. 1–14 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Solution to Class Discussion 1: Commands for Removing a System 1 Open the configuration and prevent application failover to the system to be removed, persisting through VCS restarts. haconf -makerw hasys -freeze -persistent -evacuate train2 2 Switch any application services that are running on the system to be removed to any other system in the cluster. Note: You can combine this step with step 1 as an option to a single command line. This step has been combined with step 1. 3 Close the configuration and stop VCS on the system to be removed. haconf -dump -makero hastop -sys train2 Note: You can accomplish steps 1-3 using the following commands: haconf -makerw hasys -freeze train2 haconf -dump -makero hastop -sys train2 -evacuate 4 Remove any disk heartbeat configuration on the system to be removed. Notes: – Remove both the GAB disk heartbeats and service group heartbeats. – After you remove the GAB disk heartbeats, also remove the corresponding lines in the /etc/gabtab file that starts the disk heartbeat so that the disk heartbeats are not started again in case the system crashes and is rebooted before you remove the VCS software. gabdiskhb -l gabdiskhb –d devicename -s start gabdiskx -l gabdiskx -d devicename -s start Also, remove the lines starting with gabdiskhb -a in the /etc/gabtab file.
  27. 27. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–15 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 5 Stop VCS communication modules (GAB, LLT) and fencing on the system to be removed. Note: On the Solaris platform, unload the kernel modules. On the system to be removed, train2 in this example: /etc/init.d/vxfen stop (if fencing is configured) gabconfig -U lltconfig -U Solaris Only modinfo | grep gab modunload -i gab_id modinfo | grep llt modunload -i llt_id modunload | grep vxfen modinfo -i fen_ID 6 Physically remove cluster interconnect links from the system to be removed. 7 Remove VCS software from the system taken out of the cluster. For purposes of this lab, you do not need to remove the software because this system is put back in the cluster later. Notes: – You can either use the uninstallvcs script to automate the removal of the VCS software or use the command specific to the operating platform, such as pkgadd for Solaris, swinstall for HP-UX, installp -a for AIX, or rpm for Linux, to remove the VCS software packages individually. – If you have remote shell access (rsh or ssh) for root between the cluster systems, you can run uninstallvcs on any system in the cluster. Otherwise, you have to run the script on the system to be removed. – You may need to manually remove configuration files and VCS directories that include customized scripts. WARNING: When using the uninstallvcs script, you are prompted to remove software from all cluster systems. Do not accept the default of Y or you will inadvertently remove VCS from all cluster systems. cd /opt/VRTSvcs/install ./uninstallvcs
  28. 28. 1–16 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. After the script completes, remove any remaining files related to VCS on train2: rm /etc/vxfendg rm /etc/vxfentab rm /etc/llttab rm /etc/llthosts rm /etc/gabtab rm -r /opt/VRTSvcs rm -r /etc/VRTSvcs ... 8 Update service group and resource configurations that refer to the system that is removed. Note: Service group attributes, such as AutoStartList, SystemList, SystemZones, and localized resource attributes, such as Device for NIC or IP resource types, may need to be modified. On the system remaining in the cluster, train1 in this example: haconf -makerw For all service groups that have train2 in their AutoStartList or SystemList: hagrp -modify groupname AutoStartList –delete train2 hagrp -modify groupname SystemList –delete train2 9 Remove the system from the cluster configuration. hasys -delete train2 When you have completed the modifications: haconf -dump -makero 10 Modify the VCS communication configuration files on the remaining systems in the cluster to reflect the change. – Edit /etc/llthosts on all the systems remaining in the cluster (train1 in this example) to remove the line corresponding to the removed system (train2 in this example). – Edit /etc/gabtab on all the systems remaining in the cluster (train1 in this example) to reduce the –n option to gabconfig by 1.
  29. 29. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–17 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Note: You do not need to stop and restart LLT and GAB on the remaining systems when you make changes to the configuration files unless the /etc/llttab file contains the following directives that need to be changed: – include system_id_range – exclude system_id_range – set-addr systemid tag address For more information on these directives, check the VCS manual pages on llttab.
  30. 30. 1–18 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Lab Exercise: Task 1—Removing a System from a Running Cluster Complete this exercise now, or at the end of the lesson, as directed by your instructor. One person from each team carries out the commands discussed in the classroom to accomplish Task 1. For detailed lab steps and solutions for the classroom lab environment, see the following sections of Appendix A, B or C. “Task 1: Removing a System from a Running VCS Cluster,” page A-3 “Task 1: Removing a System from a Running VCS Cluster,” page B-6 “Task 1: Removing a System from a Running VCS Cluster,” page C-6 At the end of this lab exercise, you should end up with: • One system without any VCS software on it Note: For purposes of the lab exercises, do not remove the VCS software. • A one-node cluster that is up and running with three service groups online • A two-node cluster that is up and running with three service groups online This cluster should not be affected while performing Task 1 on the other cluster. Lab Exercise: Task 1—Removing a System from a Running Cluster Complete this exercise now or at the end of the lesson, as directed by your instructor. One person from each team executes the commands discussed in the classroom to accomplish Task 1. See Appendix A, B, or C for detailed steps and classroom-specific information. XUse the lab appendix best suited to your experience level: Use the lab appendix best suited to your experience level: Appendix A: Lab Synopses Appendix B: Lab Details Appendix C: Lab Solutions Appendix A: Lab Synopses Appendix B: Lab Details Appendix C: Lab Solutions
  31. 31. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–19 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Task 2: Adding a New System to a Running VCS Cluster Objective The objective of this task is to add a new system to a running VCS cluster with no or minimal impact on application services. Ensure that the cluster configuration is modified so that the application services can make use of the new system in the cluster. Assumptions Take these assumptions into account while planning a procedure for this task: • The VCS cluster consists of two or more systems, all of which are up and running. • There are multiple service groups configured in the cluster. All of the service groups are online somewhere in the cluster. • The new system to be added to the cluster does not have any VCS software. • The new system has the same version of operating system and VERITAS Storage Foundation as the systems in the cluster. • The new system may not have all the required application software. • The storage devices can be connected to all systems. Task 2: Adding a New System to a Running VCS Cluster Objective Add a new system to a running VCS cluster while keeping the application services and VCS available and enabling the new system to run all of the application services. Assumptions – The cluster has two or more systems. – The new system does not have any VCS software. – The storage devices can be connected to all systems. +
  32. 32. 1–20 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Procedure to Add a New System to a Running VCS Cluster Discuss with your team or class the steps required to carry out Task 2. For each step, decide how the application services availability would be impacted. Note that there may not be a single answer to this question. Therefore, state your reasons for choosing a step in a specific order using the Notes area of your worksheet. Also, in the Notes area, document any assumptions that you are making that have not been explained as part of the task. Use the worksheet on the following page to provide the steps required for Task 2. Classroom Discussion for Task 2 Your instructor either groups students into teams or leads a class discussion for this task. For team-based exercises: Each group of four students, working on two clusters, forms a team to discuss the steps required to carry out task 2 as outlined on the previous slide. After all the teams are ready with their proposed procedures, have a classroom discussion to identify the best way of removing a system from a running VCS cluster, providing the reasons for each step. Note: At this point, you do not need to provide the commands to carry out each step. Note: At this point, you do not need to provide the commands to carry out each step. +
  33. 33. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–21 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Procedure for Task 2 proposed by your team: Steps Description Impact on application availability Notes
  34. 34. 1–22 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Use the following worksheet to document the procedure agreed upon by the class. Final procedure for Task 2 agreed upon as a result of classroom discussions: Steps Description Impact on application availability Notes
  35. 35. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–23 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Solution to Class Discussion 2: Adding a System 1 Install any necessary application software on the new system. 2 Configure any application resources necessary to support clustered applications on the new system. Note: The new system should be capable of running the application services in the cluster it is about to join. Preparing application resources may include: – Creating user accounts – Copying application configuration files – Creating mount points – Verifying shared storage access – Checking NFS major and minor numbers 3 Physically cable cluster interconnect links. Note: If the original cluster is a two-node cluster with crossover cables for the cluster interconnect, change to hubs or switches before you can add another node. Ensure that the cluster interconnect is not completely disconnected while you are carrying out the changes. 4 Install VCS. Notes: – You can either use the installvcs script with the -installonly option to automate the installation of the VCS software or use the command specific to the operating platform, such as pkgadd for Solaris, swinstall for HP-UX, installp -a for AIX, or rpm for Linux, to install the VCS software packages individually. – If you are installing packages manually: › Follow the package dependencies. For the correct order, refer to the VERITAS Cluster Server Installation Guide. › After the packages are installed, license VCS on the new system using the /opt/VRTSvcs/install/licensevcs command. a Start the installation. b Specify the name of the new system to the script (train2 in this example). c After the script has completed, create the communication configuration files on the new system. 5 Configure VCS communication modules (GAB, LLT) on the new system. 6 Configure fencing on the new system, if used in the cluster.
  36. 36. 1–24 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. 7 Update VCS communication configuration (GAB, LLT) on the existing systems. Note: You do not need to stop and restart LLT and GAB on the existing systems in the cluster when you make changes to the configuration files unless the /etc/llttab file contains the following directives that need to be changed: – include system_id_range – exclude system_id_range – set-addr systemid tag address For more information on these directives, check the VCS manual pages on llttab. 8 Install any VCS Enterprise agents required on the new system. 9 Copy any triggers, custom agents, scripts, and so on from existing cluster systems to the new cluster system. 10 Start cluster services on the new system and verify cluster membership. 11 Update service group and resource configuration to use the new system. Note: Service group attributes, such as SystemList, AutoStartList, SystemZones, and localized resource attributes, such as Device for NIC or IP resource types, may need to be modified. 12 Verify updates to the configuration by switching the application services to the new system.
  37. 37. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–25 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Commands Required to Complete Task 2 After you have agreed on the steps required to accomplish Task 2, you need to determine which VCS commands are required to perform each step in the procedure. You will first work as a team to propose a solution, and then discuss each step in the classroom. Note that there may be multiple methods to carry out each step. You can use the participants guide, VCS manual pages, the VERITAS Cluster Server User’s Guide, and the VERITAS Cluster Server Installation Guide as sources of information. If there are topics that you do not understand well, ask your instructor to discuss them in detail during the classroom discussion. Use the worksheet on the following page to provide the commands required for Task 2. VCS Commands Required for Task 2 Provide the commands to perform each step in the recommended procedure for adding a system to a running VCS cluster. You may need to refer to previous lessons, VCS manuals, or manual pages to decide on the specific commands and their options. For each step, complete the worksheet provided in the participants guide by providing the command, the system to run it on, and any specific notes. + Note: When you are ready, your instructor will discuss each step in detail. Note: When you are ready, your instructor will discuss each step in detail.
  38. 38. 1–26 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Commands for Task 2 proposed by your team: Order of Execution VCS Command to Use System on which to run the command Notes
  39. 39. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–27 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Use the following worksheet to document any differences to your proposal. Commands for Task 2 agreed upon in the classroom: Order of Execution VCS Command to Use System on which to run the command Notes
  40. 40. 1–28 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Solution to Class Discussion 2: Commands for Adding a System 1 Install any necessary application software on the new system. 2 Configure any application resources necessary to support clustered applications on the new system. Note: The new system should be capable of running the application services in the cluster it is about to join. Preparing application resources may include: – Creating user accounts – Copying application configuration files – Creating mount points – Verifying shared storage access – Checking NFS major and minor numbers 3 Physically cable cluster interconnect links. Note: If the original cluster is a two-node cluster with crossover cables for the cluster interconnect, change to hubs or switches before you can add another node. Ensure that the cluster interconnect is not completely disconnected while you are carrying out the changes. 4 Install VCS and configure VCS communication modules (GAB, LLT) on the new system. If you skipped the removal step in the previous section, you do not need to install VCS on this system. Notes: – You can either use the installvcs script with the -installonly option to automate the installation of the VCS software or use the command specific to the operating platform, such as pkgadd for Solaris, swinstall for HP-UX, installp -a for AIX, or rpm for Linux, to install the VCS software packages individually. – If you are installing packages manually: › Follow the package dependencies. For the correct order, refer to the VERITAS Cluster Server Installation Guide. › After the packages are installed, license VCS on the new system using the /opt/VRTSvcs/install/licensevcs command. a Start the installation. cd /install_location ./installvcs -installonly b Specify the name of the new system to the script (train2 in this example).
  41. 41. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–29 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 5 After the script completes, create the communication configuration files on the new system. › /etc/llttab This file should have the same cluster ID as the other systems in the cluster. This is the /etc/llttab file used in this example configuration: set-cluster 2 set-node train2 link tag1 /dev/interface1:x - ether - - link tag2 /dev/interface2:x - ether - - link-lowpri tag3 /dev/interface3:x - ether - - › /etc/llthosts This file should contain a unique node number for each system in the cluster, and it should be the same on all systems in the cluster. This is the /etc/llthosts file used in this example configuration: 0 train3 1 train4 2 train2 › /etc/gabtab This file should contain the command to start GAB and any configured disk heartbeats. This is the /etc/gabtab file used in this example configuration: › /sbin/gabconfig -c -n 3 Note: The seed number used after the -n option shown previously should be equal to the total number of systems in the cluster. 6 Configure fencing on the new system, if used in the cluster. Create /etc/vxfendg and enter the coordinator disk group name. 7 Update VCS communication configuration (GAB, LLT) on the existing systems. Note: You do not need to stop and restart LLT and GAB on the existing systems in the cluster when you make changes to the configuration files unless the /etc/llttab file contains the following directives that need to be changed: – include system_id_range – exclude system_id_range – set-addr systemid tag address For more information on these directives, check the VCS manual pages on llttab.
  42. 42. 1–30 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. a Edit /etc/llthosts on all the systems in the cluster (train3 and train4 in this example) to add an entry corresponding to the new system (train2 in this example). On train3 and train4: # vi /etc/llthosts 0 train3 1 train4 2 train2 b Edit /etc/gabtab on all the systems in the cluster (train3 and train4 in this example) to increase the –n option to gabconfig by 1. On train3 and train4: # vi /etc/gabtab /sbin/gabconfig -c -n 3 8 Install any VCS Enterprise agents required on the new system. This example shows installing the Enterprise agent for Oracle. On train2: cd /install_dir Solaris pkgadd -d /install_dir VRTSvcsor AIX installp -ac -d /install_dir/VRTSvcsor.rte.bff VRTSvcsor.rte HP-UX swinstall -s /install_dir/pkgs VRTSvcsor Linux rpm -ihv VRTSvcsor-2.0-Linux.i386.rpm 9 Copy any triggers, custom agents, scripts, and so on from existing cluster systems to the new cluster system. Because this is a new system to be added to the cluster, you need to copy these trigger scripts to the new system. On the new system, train2 in this example: cd /opt/VRTSvcs/bin/triggers rcp train3:/opt/VRTSvcs/bin/triggers/* .
  43. 43. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–31 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 10 Start cluster services on the new system and verify cluster membership. On train2: lltconfig -c gabconfig -c -n 3 gabconfig -a Port a membership should include the node ID for train2. /etc/init.d/vxfen start hastart gabconfig -a Both port a and port h memberships should include the node ID for train2. Note: You can also use LLT, GAB, and VCS startup files installed by the VCS packages to start cluster services. 11 Update service group and resource configuration to use the new system. Note: Service group attributes, such as SystemList, AutoStartList, SystemZones, and localized resource attributes, such as Device for NIC or IP resource types, may need to be modified. haconf -makerw For all service groups in the vcs2 cluster, modify the SystemList and AutoStartList attributes: hagrp -modify groupname SystemList –add train2 hagrp -modify groupname AutoStartList –add train2 priority When you have completed modifications: haconf -dump -makero 12 Verify updates to the configuration by switching the application services to the new system. For all service groups in the vcs2 cluster: hagrp -switch groupname -to train2
  44. 44. 1–32 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Lab Exercise: Task 2—Adding a New System to a Running Cluster Before starting the discussion about Task 3, one person from each team executes the commands discussed in the classroom to accomplish Task 2. For detailed lab steps and solutions for the classroom lab environment, see the following sections of Appendix A, B, or C. “Task 2: Adding a System to a Running VCS Cluster,” page A-4 “Task 2: Adding a System to a Running VCS Cluster,” page B-9 “Task 2: Adding a System to a Running VCS Cluster,” page C-10 At the end of this lab exercise, you should end up with: • A one-node cluster that is up and running with three service groups online There should be no changes in this cluster after Task 2. • A three-node cluster that is up and running with three service groups online All the systems should be capable of running all the service groups after Task 2. Lab Exercise: Task 2—Adding a New System to a Running Cluster Complete this exercise now or at the end of the lesson, as directed by your instructor. One person from each team executes the commands discussed in the classroom to accomplish Task 2. See Appendix A, B, or C for detailed steps and classroom-specific information. +
  45. 45. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–33 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Task 3: Merging Two Running VCS Clusters Objective The objective of this task is to merge two running VCS clusters with no or minimal impact on application services. Also, ensure that the cluster configuration is modified so that the application services can make use of the systems from both clusters. Assumptions Following is a list of assumptions that you need to take into account while planning a procedure for this task: • All the systems in both clusters are up and running. • There are multiple service groups configured in both clusters. All of the service groups are online somewhere in the cluster. • All the systems have the same version of operating system and VERITAS Storage Foundation. • The clusters do not necessarily have the same application services software. • New application software can be installed on the systems to support application services of the other cluster. • The storage devices can be connected to all systems. • The cluster interconnects of both clusters are isolated before the merge. For this example, you can assume that a one-node cluster is merged with a three- node cluster as in this lab environment. Task 3: Merging Two Running VCS Clusters Objective Merge two running VCS clusters while maximizing application services and VCS availability. Assumptions – The storage devices can be connected to all systems. – You should enable all the application services to run on all the systems in the cluster. – The private networks of both clusters are isolated before the merge. – All systems have the same version of OS and Storage Foundation. +
  46. 46. 1–34 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Procedure to Merge Two VCS Clusters Discuss with your team the steps required to carry out Task 3. For each step, decide how the application services availability would be impacted. Note that there may not be a single answer to this question. Therefore, state your reasons for choosing a step in a specific order using the Notes area of your worksheet. Also, in the Notes area, document any assumptions that you are making that have not been explained as part of the task. Use the worksheet on the following page to provide the steps required for Task 3. Classroom Discussion for Task 3 Note: At this point, you do not need to provide the commands to carry out each step. Note: At this point, you do not need to provide the commands to carry out each step. Your instructor either groups students into teams or leads a class discussion for this task. For team-based exercises: Each group of four students, working on two clusters, forms a team to discuss the steps required to carry out task 3 as outlined on the previous slide. After all the teams are ready with their proposed procedures, have a classroom discussion to identify the best way of removing a system from a running VCS cluster, providing the reasons for each step. +
  47. 47. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–35 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Procedure for Task 3 proposed by your team: Steps Description Impact on application availability Notes
  48. 48. 1–36 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Use the following worksheet to document the procedure agreed upon by the class. Final procedure for Task 3 agreed upon as a result of classroom discussions: Steps Description Impact on application availability Notes
  49. 49. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–37 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Solution to Class Discussion 3: Merging Two Running Clusters In the following steps, it is assumed that the small (first) cluster is merged to the larger (second) cluster. That is, the merged cluster keeps the name and ID of the second cluster, and the second cluster is not brought down during the whole process. 1 Modify VCS communication files on the second cluster to recognize the systems to be added from the first cluster. Note: You do not need to stop and restart LLT and GAB on the existing systems in the second cluster when you make changes to the configuration files unless the /etc/llttab file contains the following directives that need to be changed: – include system_id_range – exclude system_id_range – set-addr systemid tag address For more information on these directives, check the VCS manual pages on llttab. 2 Add the names of the systems in the first cluster to the second cluster. 3 Install and configure any additional application software required to support the merged configuration on all systems. Notes: – Installing applications in a VCS cluster would require freezing systems. This step may also involve switching application services and rebooting systems depending on the application installed. – All the systems should be capable of running the application services when the clusters are merged. Preparing application resources may include: › Creating user accounts › Copying application configuration files › Creating mount points › Verifying shared storage access 4 Install any additional VCS Enterprise agents on each system. Note: Enterprise agents should only be installed, not configured. 5 Copy any additional custom agents to all systems. Note: Custom agents should only be installed, not configured. 6 Extract service group configuration from the small cluster, so you can add it to the larger cluster configuration without stopping VCS. 7 Copy or merge any existing trigger scripts on all systems. Notes: – The extent of this step depends on the contents of the trigger scripts. Because the trigger scripts are in use on the existing cluster systems, it is recommended to merge the scripts on a temporary directory. – Depending on the changes required, it may be necessary to stop cluster services on the systems before copying the merged trigger scripts.
  50. 50. 1–38 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. 8 Stop cluster services (VCS, fencing, GAB, and LLT) on the systems in the first cluster. Note: Leave application services running on the systems. 9 Reconfigure VCS communication modules on the systems in the first cluster and physically connect cluster interconnects. 10 Start cluster services (LLT, GAB, fencing, and VCS) on the systems in the first cluster and verify cluster memberships. 11 Update service group and resource configuration to use all the systems. Note: Service group attributes, such as AutoStartList, SystemList, SystemZones, and localized resource attributes, such as Device for NIC or IP resource types, may need to be modified. 12 Verify updates to the configuration by switching application services between the systems in the merged cluster.
  51. 51. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–39 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Commands Required to Complete Task 3 After you have agreed on the steps required to accomplish Task 3, determine the VCS commands required to perform each step in the procedure. You will first work as a team to propose a solution, and then discuss each step in the classroom. Note that there may be multiple methods to carry out each step. You can use the participants guide, VCS manual pages, the VERITAS Cluster Server User’s Guide, and the VERITAS Cluster Server Installation Guide as sources of information. If there are topics that you do not understand, ask your instructor to discuss them in detail during the classroom discussion. Use the worksheet on the following page to provide the commands required for Task 3. VCS Commands Required for Task 3 Provide the commands to perform each step in the recommended procedure for merging two VCS clusters. You may need to refer to previous lessons, VCS manuals, or manual pages to decide on the specific commands and their options. For each step, complete the worksheet provided in the participants guide, providing the command, the system to run it on, and any specific notes. + Note: When you are ready, your instructor will discuss each step in detail. Note: When you are ready, your instructor will discuss each step in detail.
  52. 52. 1–40 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Commands for Task 3 proposed by your team: Order of Execution VCS Command to Use System on which to run the command Notes
  53. 53. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–41 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Use the following worksheet to document any differences to your proposal. Commands for Task 3 agreed upon in the classroom: Order of Execution VCS Command to Use System on which to run the command Notes
  54. 54. 1–42 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Solution to Class Discussion 3: Commands to Merge Clusters In the following steps, it is assumed that the first cluster is merged to the second; that is, the merged cluster keeps the name and ID of the second cluster, and the second cluster is not brought down during the whole process. 1 Modify VCS communication files on the second cluster to recognize the systems to be added from the first cluster. Note: You do not need to stop and restart LLT and GAB on the existing systems in the second cluster when you make changes to the configuration files unless the /etc/llttab file contains the following directives that need to be changed: – include system_id_range – exclude system_id_range – set-addr systemid tag address For more information on these directives, check the VCS manual pages on llttab. – Edit /etc/llthosts on all the systems in the second cluster to add entries corresponding to the new systems from the first cluster. On train2, train3, and train4: vi /etc/llthosts 0 train4 1 train3 2 train2 3 train1 – Edit /etc/gabtab on all the systems in the second cluster to increase the –n option to gabconfig by the number of systems in the first cluster. On train2, train3, and train4: vi /etc/gabtab /sbin/gabconfig -c -n 4 2 Add the names of the systems in the first cluster to the second cluster. haconf -makerw hasys -add train1 hasys -add train2 haconf -dump -makero
  55. 55. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–43 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 3 Install and configure any additional application software required to support the merged configuration on all systems. Notes: – Installing applications in a VCS cluster would require freezing systems. This step may also involve switching application services and rebooting systems depending on the application installed. – All the systems should be capable of running the application services when the clusters are merged. Preparing application resources may include: › Creating user accounts › Copying application configuration files › Creating mount points › Verifying shared storage access 4 Install any additional VCS Enterprise agents on each system. Note: Enterprise agents should only be installed, not configured. 5 Copy any additional custom agents to all systems. Note: Custom agents should only be installed, not configured. 6 Extract service group configuration from the first cluster and add it to the second cluster configuration. a On the first cluster, vcs1 in this example, create a main.cmd file. hacf -cftocmd /etc/VRTSvcs/conf/config b Edit the main.cmd file and filter the commands related with service group configuration. Note that you do not need to have the commands related to the ClusterService and NetworkSG service groups because these already exist in the second cluster. c Copy the filtered main.cmd file to a running system in the second cluster, for example, to train3. d On the system in the second cluster where you copied the main.cmd file, train3 in vcs2 in this example, open the configuration. haconf -makerw e Execute the filtered main.cmd file. sh main.cmd Note: Any customized resource type attributes in the first cluster are not included in this procedure and may require special consideration before adding them to the second cluster configuration. 7 Copy or merge any existing trigger scripts on all systems. Notes: – The extent of this step depends on the contents of the trigger scripts. Because the trigger scripts are in use on the existing cluster systems, it is recommended to merge the scripts on a temporary directory. – Depending on the changes required, it may be necessary to stop cluster services on the systems before copying the merged trigger scripts.
  56. 56. 1–44 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. 8 Stop cluster services (VCS, fencing, GAB, and LLT) on the systems in the first cluster. Note: Leave application services running on the systems. a On one system in the first cluster (train1 in vcs1 in this example), stop VCS. hastop -all -force b On all the systems in the first cluster (train1 in vcs1 in this example), stop fencing, and then stop GAB and LLT. /etc/init.d/vxfen stop gabconfig -U lltconfig -U 9 Reconfigure VCS communication modules on the systems in the first cluster and physically connect cluster interconnects. On all the systems in the first cluster (train1 in vcs1 in this example): a Edit /etc/llttab and modify the cluster ID to be the same as the second cluster. # vi /etc/llttab set-cluster 2 set-node train1 link interface1 /dev/interface1:0 - ether - - link interface2 /dev/interface2:0 - ether - - link-lowpri interface2 /dev/interface2:0 - ether - - b Edit /etc/llthosts and ensure that there is a unique entry for all systems in the combined cluster. # vi /etc/llthosts 0 train4 1 train3 2 train2 3 train1 c Edit /etc/gabtab and modify the –n option to gabconfig to reflect the total number of systems in combined clusters. vi /etc/gabtab /sbin/gabconfig -c -n 4
  57. 57. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–45 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 10 Start cluster services (LLT, GAB, fencing, and VCS) on the systems in the first cluster and verify cluster memberships. On train1: lltconfig -c gabconfig -c -n 4 gabconfig -a The port a membership should include the node ID for train1, in addition to the node IDs for train2, train3, and train4. /etc/init.d/vxfen start hastart gabconfig -a Both port a and port h memberships should include the node ID for train1 in addition to the node IDs for train2, train3, and train4. Note: You can also use LLT, GAB, and VCS startup files installed by the VCS packages to start cluster services. 11 Update service group and resource configuration to use all the systems. Note: Service group attributes, such as AutoStartList, SystemList, SystemZones, and localized resource attributes, such as Device for NIC or IP resource types, may need to be modified. a Open the cluster configuration. haconf -makerw b For the service groups copied from the first cluster, add train2, train3, and train4 to the SystemList and AutoStartList attributes: hagrp -modify groupname SystemList -add train2 priority2 train3 priority3 train4 priority4 hagrp -modify groupname AutoStartList add train2 train3 train4 c For the service groups that existed in the second cluster before the merging, add train1 to the SystemList and AutoStartList attributes: hagrp -modify groupname SystemList -add train1 priority1 hagrp -modify groupname AutoStartList add train1 d Close and save the cluster configuration. haconf -dump -makero 12 Verify updates to the configuration by switching application services between the systems in the merged cluster. For all the systems and service groups in the merged cluster, verify operation: hagrp –switch groupname –to systemname
  58. 58. 1–46 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Lab Exercise: Task 3—Merging Two Running VCS Clusters To complete the workshop, one person from each team executes the commands discussed in the classroom to accomplish Task 3. For detailed lab steps and solutions for the classroom lab environment, see the following sections of Appendix A, B, or C. “Task 3: Merging Two Running VCS Clusters,” page A-5 “Task 3: Merging Two Running VCS Clusters,” page B-13 “Task 3: Merging Two Running VCS Clusters,” page C-16 At the end of this lab exercise, you should have a four-node cluster that is up and running with six application service groups online. All the systems should be capable of running all the application services after Task 3 is completed. Lab Exercise: Task 3—Merging Two Running VCS Clusters Complete this exercise now or at the end of the lesson, as directed by your instructor. One person from each team executes the commands discussed in the classroom to accomplish Task 3. See Appendix A, B, or C for detailed steps and classroom-specific information. +
  59. 59. Lesson 1 Workshop: Reconfiguring Cluster Membership 1–47 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 1 Summary This workshop introduced procedures to add and remove systems to and from a running VCS cluster and to merge two VCS clusters. In doing so, this workshop reviewed the concepts related to how VCS operates, how the configuration changes in VCS communications, and how the cluster configuration impacts the application services’ availability. Next Steps The next lesson describes how the relationships between application services can be controlled under VCS in a multinode and multiple application services environment. This lesson also shows the impact of these controls during service group failovers. Additional Resources • VERITAS Cluster Server Installation Guide This guide provides information on how to install VERITAS Cluster Server (VCS) on the specified platform. • VERITAS Cluster Server User’s Guide This document provides information about all aspects of VCS configuration. Lesson Summary Key Points – You can minimize downtime when reconfiguring cluster members. – Use the procedures in this lesson as guidelines for adding or removing cluster systems. Reference Materials – VERITAS Cluster Server Installation Guide – VERITAS Cluster Server User's Guide
  60. 60. 1–48 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Lab 1: Reconfiguring Cluster Membership You instructor may choose to have you complete the exercises as a single lab. Labs and solutions for this lesson are located on the following pages. Appendix A provides brief lab instructions for experienced students. • “Lab 1 Synopsis: Reconfiguring Cluster Membership,” page A-2 Appendix B provides step-by-step lab instructions. • “Lab 1 Details: Reconfiguring Cluster Membership,” page B-3 Appendix C provides complete lab instructions and solutions. • “Lab Solution 1: Reconfiguring Cluster Membership,” page C-3 Lab 1: Reconfiguring Cluster Membership B A A B B A D C C D C C D C D B B C DD 1 2 3 4 3 4 4 2 2 2 1 1 3 DC B B C D AA Task 1 Task 2 Task 3 D A C AUse the lab appendix best suited to your experience level: Use the lab appendix best suited to your experience level: Appendix A: Lab Synopses Appendix B: Lab Details Appendix C: Lab Solutions Appendix A: Lab Synopses Appendix B: Lab Details Appendix C: Lab Solutions
  61. 61. Lesson 2 Service Group Interactions
  62. 62. 2–2 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Introduction Overview This lesson describes how to configure VCS to control the interactions between application services. In this lesson, you learn how to implement service group dependencies and use resources and triggers to control the startup and failover behavior of service groups. Importance In order to effectively implement dependencies between applications in your cluster, you need to use a methodology for translating application requirements to VCS service group dependency rules. By analyzing and implementing service group dependencies, you can factor performance, security, and organizational requirements into your cluster environment. Lesson Introduction Lesson 1: Reconfiguring Cluster Membership Lesson 2: Service Group Interactions Lesson 3: Workload Management Lesson 4: Storage and Network Alternatives Lesson 5: Maintaining VCS Lesson 6: Validating VCS Implementation
  63. 63. Lesson 2 Service Group Interactions 2–3 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 2 Outline of Topics • Common Application Relationships • Service Group Dependency Definition • Service Group Dependency Examples • Configuring Service Group Dependencies • Alternative Methods of Controlling Interactions Configure alternative methods for controlling service group interactions. Alternative Methods of Controlling Interactions Configure service group dependencies.Configuring Service Group Dependencies Describe example uses of service group dependencies. Service Group Dependency Examples Define service group dependencies.Service Group Dependency Definition Describe common example application relationships. Common Application Relationships After completing this lesson, you will be able to: Topic Lesson Topics and Objectives
  64. 64. 2–4 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Common Application Relationships Several examples of application relationships are shown to illustrate common scenarios where service group dependencies are useful for managing services. Online on the Same System In this type of relationship, services must run on the same system due to some set of constraints. In the example in the slide, App1 and DB1 communicate using shared memory and therefore must run on the same system. If a fault occurs, they must both be moved to the same system. Online on the Same System Example criteria: App1 uses shared memory to communicate with DB1. Both must be online on the same system to provide the service. DB1 must come online first. If either faults (or the system), they must fail over to the same system. App1App1 DB1DB1
  65. 65. Lesson 2 Service Group Interactions 2–5 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 2 Online Anywhere in the Cluster This example shows an application and database that must be running somewhere in the cluster in order to provide a service. They do not need to run on the same system, but they can, if necessary. For example, if multiple servers were down, DB2 and App2 could run on the remaining server. Online Anywhere in the Cluster Example criteria: App2 communicates with DB2 using TCP/IP. Both must be online to provide the service. They do not have to be online on the same system. DB2 must be running before App2 starts. App2App2 DB2DB2
  66. 66. 2–6 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Online on Different Systems In this example, both the database and the Web server must be online, but they cannot run on the same system. For example, the combined resource requirements of each application may exceed the capacity of the systems, and you want to ensure that they run on separate systems. WebWeb DB3DB3 Online on Different Systems Example criteria: The Web server requires DB3 to be online first. Both must be online to provide the service. The Web and DB3 cannot run on the same system, due to system usage constraints. If Web faults, DB3 should continue to run.
  67. 67. Lesson 2 Service Group Interactions 2–7 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 2 Offline on the Same System One example relationship is where you have a test version of an application and want to ensure that it does not interfere with the production version. You want to give the production application precedence over the test version for all operations, including manual offline, online, switch, and failover. Offline on the Same System Example criteria: One node is used for a test version of the service. Test and Prod cannot be online on the same system. Prod always has priority. Test should be shut down if Prod faults and needs to fail over to that system. TestTest ProdProd
  68. 68. 2–8 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Service Group Dependency Definition You can set up dependencies between service groups to enforce rules for how VCS manages relationships between application services. There are four basic criteria for defining how services interact when using service group dependencies. • A service group can require another group to be online or offline in order to start and run. • You can specify where the groups must be online or offline. • You can determine the startup order for service groups by designating one group the child (comes online first) and another a parent. In VCS, parent groups depend on child groups. If service group B requires service group A to be online in order to start then B is the parent and A is the child. • Failover behavior of linked service groups is specified by designating the relationship soft, firm, or hard. These types determine what happens when a fault occurs in the parent or child group. Startup Behavior Summary For all online dependencies, the child group must be online in order for the parent to start. A location of local, global, or remote determines where the parent can come online relative to where the child is online. For offline local, the child group must be offline on the local system for the parent to come online. Service Group Dependencies You can use service group dependencies to specify most application relationships according to these four criteria: – Category: Online or offline – Location: Local, remote, or global – Startup behavior: Parent or child – Failover behavior: Soft, firm, or hard You can specify combinations of these characteristics to determine how dependencies affect service group behavior, as shown in a series of examples in this lesson.
  69. 69. Lesson 2 Service Group Interactions 2–9 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 2 Failover Behavior Summary These general properties apply to failover behavior for linked service groups: • Target systems are determined by the system list of the service group and the failover policy in a way that should not conflict with the existing service group dependencies. • If a target system exists, but there is a dependency violation between the service group and a parent service group, the parent service group is migrated to another system to accommodate the child service group that is failing over. • If conflicts between a child service group and a parent service group arise, the child service group is given priority. • If there is no system available for failover, the service group remains offline, and no further attempt is made to bring it online. • If the parent service group faults and fails over, the child service group is not taken offline or failed over except for online local hard dependencies. Examples are provided in the next section. A complete description of both failover behavior and manual operations for each type of dependency is provided in the job aid. Failover Behavior Summary Types apply to online dependencies and define online, offline, and failover operations: Soft: The parent can stay online when the child faults. Firm: – The parent must be taken offline when the child faults. – When the child is brought online on another system, the parent is brought online. Hard: – The child and parent fail over together to the same system when either the child or the parent faults. – Hard applies only to an online local dependency. – This is allowed only between a single parent and a single child.
  70. 70. 2–10 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Service Group Dependency Examples A set of animations are used to show how service group dependencies affect failover when different kinds of faults occur. The following sections provide illustrations and summaries of these examples. A complete description of startup and failover behavior for each type of dependency is provided as a job aid in Appendix D. Online Local Dependency In an online local dependency, a child service group must be online on a system before a parent service group can come online on the same system. Online Local Soft A link configured as online local soft designates that the parent group stays online while the child group fails over, and then migrates to follow the child. • Online Local Soft: The child faults. Failover behavior examples: Firm: – Child faults: Parent follows child – Parent faults: Child continues to run Hard: Same as Firm except when parent faults: – Child is failed over – Parent then started on the same system Online Local Dependency App1App1 DB1DB1 Startup behavior: Child must be online Parent can come online on only on the same system AnimationSlides
  71. 71. Lesson 2 Service Group Interactions 2–11 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 2 If a child group in an online local soft dependency faults, the parent service group is migrated to another system only after the child group successfully fails over to that system. If the child group cannot fail over, the parent group is left online. • Online Local Soft: The parent faults. If the parent group in an online local soft dependency faults, it stays offline, and the child group remains online. Online Local Firm A link configured as online local firm designates that the parent group is taken offline when the child group faults. After the child group fails over, the parent is migrated to that system. • Online Local Firm: The child faults. If a child group in an online local firm dependency faults, the parent service group is taken offline on that system. The child group fails over and comes online on another system. The parent group is then started on the system where the child group is now running. If the child group cannot fail over, the parent group is taken offline and stays offline.
  72. 72. 2–12 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. • Online Local Firm: The parent faults. If a parent group in an online local firm dependency faults, the parent service group is taken offline and stays offline. • Online Local Firm: The system faults. If a system faults, the child group in an online local firm dependency fails over to another system, and the parent is brought online on the same system.
  73. 73. Lesson 2 Service Group Interactions 2–13 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 2 Online Local Hard Starting with VCS 4.0, online local dependencies can also be formed as hard dependencies. A hard dependency indicates that the child and the parent service groups fail over together to the same system when either the child or the parent faults. Prior to VCS 4.0, trigger scripts had to be used to cause a fault in the parent service group to initiate a failover of the child service group. With the introduction of hard dependencies, there is no longer a need to use triggers for this purpose. Hard dependencies are allowed only between a single parent and a single child. • Online Local Hard: The child faults. If the child group in an online local hard dependency faults, the parent group is taken offline. The child is failed over to an available system. The parent group is then started on the system where the child group is running. The parent service group remains offline if the parent service group cannot fail over. • Online Local Hard: The parent faults. If the parent service group in an online local hard dependency faults, the child group is failed over to another system. The parent group is then started on the system where the child group is running. The child service group remains online if the parent service group cannot fail over.
  74. 74. 2–14 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Online Global Dependency In an online global dependency, a child service group must be online on a system before the parent service group can come online on any system in the cluster, including the system where the child is running. Online Global Soft A link configured as online global soft designates that the parent service group remains online when the child service group faults. The issue of whether the child service group can fail over to another system or not does not impact the parent service group. • Online Global Soft: The child faults. If the child group in an online global soft dependency faults, the parent continues to run on the original system, and the child fails over to an available system. • Online Global Soft: The parent faults. If the parent group in an online global soft dependency faults, the child continues to run on the original system, and the parent fails over to an available system. App2App2 DB3DB3 Online Global Dependency Failover behavior example for online global firm: Child faults and is taken offline Parent group is taken offline Child fails over to an available system Parent restarts on an available system Startup behavior: Child must be online Parent can come online on any system AnimationSlides
  75. 75. Lesson 2 Service Group Interactions 2–15 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 2 Online Global Firm A link configured as online global firm designates that the parent service group is taken offline when the child service group faults. When the child service group fails over to another system, the parent is migrated to an available system. The child and parent can be running on the same or different systems after the failover. • Online Global Firm: The child faults. The child faults and is taken offline. The parent group is taken offline. The child fails over to an available system, and the parent fails over to an available system. • Online Global Firm: The parent faults. If the parent group in an online global firm dependency faults, the child continues to run on the original system, and the parent fails over to an available system.
  76. 76. 2–16 VERITAS Cluster Server for UNIX, Implementing Local Clusters Copyright © 2005 VERITAS Software Corporation. All rights reserved. Online Remote Dependency In an online remote dependency, a child service group must be online on a remote system before the parent service group can come online on the local system. Online Remote Soft An online remote soft dependency designates that the parent service group remains online when the child service group faults, as long as the child service group chooses another system to fail over to. If the child service group chooses to fail over to the system where the parent was online, the parent service group is migrated to any other available system. WebWeb DB3DB3 Online Remote Dependency Startup behavior: Child must be online Parent can come online only on a remote system Failover behavior example for online remote soft: The child faults and fails over to an available system. If the only available system is where the parent is online, the parent is taken offline before the child is brought online. The parent then restarts on a system different than the child. Otherwise, the parent continues to run on the original system. AnimationSlides
  77. 77. Lesson 2 Service Group Interactions 2–17 Copyright © 2005 VERITAS Software Corporation. All rights reserved. 2 • Online Remote Soft: The child faults. The child group faults and fails over to an available system. If the only available system has the parent running, the parent is taken offline before the child is brought online. The parent then restarts on a different system. If the parent is online on a system that is not selected for child group failover, the parent continues to run on the original system. • Online Remote Soft: The parent faults. The parent group faults and is taken offline. The child group continues to run on the original system. The parent group fails over to an available system. If the only available system is running the child group, the parent stays offline. Online Remote Firm A link configured as online remote firm is similar to online global firm, with the exception that the parent service group is brought online on any system other than the system on which the child service group was brought online. • Online Remote Firm: The child faults. The child group faults and is taken offline. The parent group is taken offline. The child fails over to an available system. If the child fails over to the system where the parent was online, the parent restarts on a different system; otherwise, the parent restarts on the system where it was online. • Online Remote Firm: The parent faults. The parent group faults and is taken offline. The child group continues to run on the original system. The parent fails over to an available system. If the only available system is where the child is online, the parent stays offline.

×