Bheritaas5 foundation

1,554 views

Published on

Published in: Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,554
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
53
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Bheritaas5 foundation

  1. 1. 'symantec.. VERITAS Storage Foundation 5.0 for UNIX: Maintenance 100-002353-8
  2. 2. COURSE DEVELOPERS (;ail Adey Bill4t.'Gerrits TECHNICAL CONTRIBUTORS AND REVIEWERS Jade Arrtngton Marg) Cassid) l{fI) Freeman Jue (;allagher Bruce(;arncr Tomer (~urantz Bill Havey (;l'm,' Henriksen (;l'rald Jackson Ravmond Karns Hill Lehman IIflh Lucus Dartvoue 'Ianikhong Chrlstian Rabanus naw I{flgers Kleber Saldanha Albrecht Seriha ,lieh.1 Simoni Ananda Sil'iscnu Pete TtH..-m mes Cupyright ( 20()() Sym.uuec Corporurion. All riglns reserved. Symantcc. the Symantcc Logo. and "'FRn AS arc trademarks or registered trademarks of Symantcc Corporation or its alfili,Hes in the U.S. and other countries. Other names may be trademarks ortheir rcspccuve 0 ners. IIIIS PUBLICAIIO'J IS PKUVIDEi) "'S IS" AND ALL EXPKFSS OR IMPLIED CONDITIUNS. IU·.PRESENIAtIO'JS A'JD '-"ARRAN ru.s. INCIL'OINe; ANY I~IPI.ILD WAKKANTY or MERCIIAN rABII.IIY-ITI NESS [,UK A PAKfiCULAR PURPOSE OR N( ):-.I-INIRIN(jF~IENT. AKE DISCLAIMED. EXCEPT TO IIII' EX lENT Tilt!' SUCII DISCLAIM ERSARF HELD 10 BE LEGALLY 1:-.1VALID. SY~IAN1EC U>RI'O[{ATION SIIAI.L NO r BE LIAIlLE FOR INClIJE:-.I·1:1. OR CONSEQL'EN riAL DAMA(;[S IN CONNEC liON ,ITII Till' H'R:-.IISIII:-.IG PERF()R~IANCE. OR USE OF TillS I'UIlLICAtIO:-.l.IIIl: l:-.IrORMAtION CO:-.lTAINLD HI,REIN IS SUBJECT TO CII/'Jljl' WITHOUT NOTICE. No part uf the COIlIL'lIts of this honk Illay be reproduced or transmitted in any lonu or by ilny means without the written permission of the publisher. / ERn:·IS .1oragc FOlllldclfioI15.() torL 'SI.": .!tlillh'lItIl1Ce August 2006 Printing Symantcc <.. orpor.uion 203311 SICI ells Creek Ilh d. Cupertino. CA 45014 I1l1p: '.syIlWlltL'L'.COIll
  3. 3. Table of Contents Course Introduction VERITAS Volume Manager Maintenance Tasks VERITAS Storage Foundation Curriculum . Lesson 1: Maintaining Data Consistency Resynchronization Operations . Interpreting State Information for VxVM Objects Modifying VxVM Object States .. Lesson 2: Managing Devices Within the VxVM Architecture Managing Components in the VxVM Architecture. Discovering Disk Devices... . . Managing Multiple Paths to Disk Devices . Lesson 3: Encapsulation and Rootability Placing the Boot Disk Under VxVM Control Creating an Alternate Boot Disk .. Removing the Boot Disk from VxVM Control. Lesson 4: Troubleshooting the Boot Process Operating System Boot Processes . Troubleshooting the Boot Process . Recovering the Boot Disk Group .. Lesson 5: Volume Maintenance Changing the Volume Layout..... . Managing Volume Tasks... . Analyzing Volume Configurations with Storage Expert . Lesson 6: Performance Monitoring Storage Performance Analysis Process .. VxVM Performance Monitoring Tools and Techniques Lesson 7: Point-ln-Tlme Copies What Is a Point-In-Time Copy? ... Types of PITC Solutions in Storage Foundation .. Creating and Managing Volume Snapshots . Using Volume Snapshots for Off-Host Processing Creating and Managing Storage Checkpoints . Lesson 8: Other Enterprise Features Overview What Is Dynamic Storage Tiering? . What Is Intelligent Storage Provisioning? . What Is the Storage Foundation Management Server? . tntro-z . Intro-S 1·3 1·9 1·18 . .... 2·3 2·13 . 2·16 3·3 . 3·18 3·22 . 4·3 ... 4-4 .... 4·15 5·3 5·12 5·21 6·3 6·7 7·3 . 7·8 7·17 7·27 7·31 . 8·3 . 8·10 8·18 Copvnqtu ?' 2006 Syr'lanltof Corporatrcn All nqnts reserved Table of Contents
  4. 4. Appendix A: Lab Exercises Lab 1: Maintaining Data Consistency ... Lab 2: Managing Devices Within the VxVM Architecture Lab 3: Encapsulation and Rootability... . Lab 4: Troubleshooting the Boot Process .. Lab 5 Volume Maintenance. . Lab 6: Performance Monitoring Lab 7 Point-in-Time Copies. Appendix B: Lab Solutions Lab 1 Solutions: Maintaining Data Consistency .. Lab 2 Solutions: Managing Devices Within the VxVM Architecture. Lab 3 Solutions: Encapsulation and Rootability .... Lab 4 Solutions: Troubleshooting the Boot Process Lab 5 Solutions: Volume Maintenance ... Lab 6 Solutions: Performance Monitoring . Lab 7 Solutions Point-in- Time Copies .. Appendix C: Boot Processes and VxVM Start-Up Scripts VxVM and the Solaris Boot Process. VxVM and the HP-UX Boot Process Index . A-3 · A-13 .... A-25 · A-35 . A-43 .... A-49 · A-63 . B-3 ... B-21 . B-37 .. B-51 . B-65 . B-75 .. B-97 ..... C-2 ....... C-15 VERITAS Storage Founttetion 5.0 for UNIX: Maintenance
  5. 5. Course Introduction
  6. 6. How does VxVM integrate into my system architecture? · How do I discover new devices? · How do I manage dynamic multlpathing? How can I recover critical data? ·How do I resolve disk failure? · How do I recover a plex? ·How do I recover the boot disk? How can I accelerate access to critical data? ·Where are the performance problems? · How is my hardware affecting performance? · How do I tune VxVM and optimize I/O? symantcc VxVM Maintenance Device Management Recovery Management Performance Management VERITAS Volume Manager Maintenance Tasks Before you perform any maintenance tasks, you should understand the VxVM architecture and how to manage devices connected to your system. When you encounter a problem on a system running VERITAS Volume Manager, you must know how to accurately identify the problem and select the appropriate solution. By learning how to use Vx VM recovery tools and apply recovery techniques in appropriate ways, you can troubleshoot problems that may occur in your environment and minimize the loss of critical data. A variety or factors. such as hardware. location or data on drives. and the application I!O profile. can impact the performance of VlRITAS Volume Manager (VxVM). The pcrtonnancc management techniques discussed in this training enable you to idcmily and remove performance bottlenecks without disrupting users.and to accelerate accessto critical infornuuion. tutro-Z Copyrl~tll ~- 2006 Symantec Corpor auon fill nqtots reserved VERITAS Storage Foundation 5.0 for UNIX' Maintenance
  7. 7. Storage Foundation Curriculum Path VERITAS Storage Foundation for UNIX: Fundamentals •••••• VERITAS Storage Foundation for UNIX: Maintenance ~------------ ------------~-----v-- VERITAS Storage Foundation for UNIX symantec. VERITAS Storage Foundation Curriculum VERITAS Storag« Foundationfor UNIX: Maintenance training is designed to provide you with comprehensive instruction on making the most of VERIT AS Storage Foundation. C(jp~flqht ~ :!006 Symaruec Coeoorenoo 111 "'lht<; reserveo Inlro-3Course Introduction
  8. 8. • Lesson 1: Maintaining Data Consistency • Lesson 2: Managing Devices Within the VxVM Architecture • Lesson 3: Encapsulation and Rootability • Lesson 4: Troubleshooting the Boot Process • Lesson 5: Volume Maintenance • Lesson 6: Performance Monitoring • Lesson 7: Point-in-Time Copies • Lesson 8: Other Enterprise Features Overview syrnantec VERITAS Storage Foundation for UNIX: Maintenance Overview This training provides instruction on device management, troubleshooting, recovery, and performance monitoring lorusers ofVERITAS Storage Foundation. Objectives After completing this course. you will be able tu: Interpret plcx. volume, and kernel states, fix plcx and volume failures by using VxVM tools. and resolve data consistency problems by analyzing plex states. Describe the Vx VM architecture and manage the device discovery layer and dynamic multipathing feature. Place the rout disk under Vx Vlvl control and mirror the root disk. Identify boot processes. debug VxVM during system start-up, and resolve boot disk problems. Rccoufigurc volumes online and use the Storage Expert utility to analyze volume configurations, Monitor VxVM performance and identify how volume configuratious contribute tu performance optimization. Create and manage Instant volume snapshots and storage checkpoints, Describe Dynamic Storage Tiering (DST). Intelligent Storage Provisioning (ISP). and the Storage Foundation Management Server (SFMS). Intro-4 Copynqnt :~ 200n Svma-uec COfpc.JrdIIOIl 111 nqhts reserved VERITAS Storage Fountietiot) 5.0 for UNIX: Meintenence
  9. 9. Course Resources • Lab Exercises (Appendix A) • Lab Solutions (Appendix B) • Boot Processes and VxVM Start-Up Scripts (Appendix C) Additional Course Resources symantec Appendix A: Lab Exercises This section contains hands-on exercises that enable you to practice the concepts and procedures presented in the lessons. Appendix B: Lab Solutions This section contains detailed solutions to the lab exercises for each lesson. Appendix C: Boot Processes and VxVM Start-Up Scripts This section contains a summary of the scripts involved in VxVM startup. Course Introduction Copvnqtu '~ 20(]6 Symantec Corroranoo 111nqtus ft'!ser',RC1 Intro-5
  10. 10. Typographic Conventions Used in This Course The following tables describe the typographic conventions used in this course. Typographic Conventions in Text and Commands Cuuveution Courier New. bold Element Command input. both syntax and examples Examples To display the robot and drive configuration: tpconfig -d 1"0 display disk information: vxdisk -0 alldgs list Courier New. plain Courier New, Italic. bold or plain Command output Command names. directory names. tile names. path name'S. user names. passwords. L RLs when used within 1"1'glll'll'tcxt paragraphs, Varia hies in command syntax. and examples: Variables in command input arc Italic. plain. Variable, in command (Hit put arc Italic. bold. In the output: protocol minimum: 40 protocol_maximum: 60 protocol current: 0 Locate the al tnames directory. Cio to http://www.symantec.com. l.nrer the value 300. Log011 asuser 1. To install the media server: / cdrom_ directory/ install To ~I•.cessa manual page: man command name To display detailed information tor a disk: vxdisk -g disk_group list disk name Convention Typographic Conventions in Graphicailiser Interface Descrlptlons Arrow Element Examples ~--------------~ Menu navigarion paths Select l-ile- ->Save, Initial capitalization Hunons. menus. window». Select tire Next button. options. and other interface Open the Task Status clements window. Remove the checkmark from the Print File check box. Quotation marks Interface clements with long names Select the "Include subvolumes in object view window' check box. lntro-B VERITAS Storage Foundation 5.0 for UNIX: Maintenance Copvnqht . 200f, Symaruec Corporation All rights reserved
  11. 11. Lesson 1 Maintaining Data Consistency
  12. 12. Lesson Introduction • Lesson 1; Maintaining Data ••• .. _Q~n~s~st~'!.£l'______.~ --i • Lesson 2: Managing Devices Within the VxVM Architecture • Lesson 3: Encapsulation and Rootability • Lesson 4: Troubleshooting the Boot Process • Lesson 5: Volume Maintenance • Lesson 6: Performance Monitoring • Lesson 7: Point-in-Time Copies • Lesson 8: Other Enterprise Features Overview svmantcc Lesson Topics and Objectives Topic After completing this lesson, you will be able to: -- Topic 1: Resynchronization Describe mirror resynchronization Operations processes. Topic 2: Interpreting State Interpret plex state and condition Information for VxVM flags, volume states, and kernel Objects states. Topic 3: Modifying VxVM Fix plex and volume failures by using Object States VxVM tools. 1-2 VERITAS Storage Foundation 5.0 for UNIX. Maintenance
  13. 13. svmantec IResynchronization is the process of ensuring that after a system crash: • All mirrors in a volume contain exactly the same data. • Data and parity in RAID-5 volumes agree. Ig~~~:::==~1Did all writes '; complete? Do all mirrors contain the same data? Types of mirror resynchronization: • Atomic-copy resynchronization • Read-writeback resynchronization Resynchronization Operations What Is Resynchronization? Resynchronization is the process of ensuring that. after a system crash: All mirrors in mirrored volumes contain exactly the same data. Data and parity in RAID-5 volumes agree. Data is written to the mirrors of a volume in parallel. If a system crash occurs before all the individual writes complete. some writes may complete while other writes do not. This system crash can cause 111'0 reads from the same region of the volume to return different results ifdifferent mirrors are used to satisfy the read request. In the case of RA ID-5 volumes. two reads returning different results can lead to rarity corruption and incorrect data rcconstrucuon. VxVM uses volume resynchronization processes to ensure that all copies of the data match exactly. VxVM records when a volume is first written to and marks it as dirty. When a volume is closed by all processes or stopped cleanly by the administrator. all writes have been completed. and Volume Manager removes the dirty flag for the volume. Only volumes that are marked dirty when the system reboots require resynchronization. Not all volumes require resynchronizarion after a system failure. Volumes that were never written or that had no active 1/0 when the system failure occurred do not require resynchronization. The volume is completely accessible during the two modes of resynchronizaiion. Lesson 1 Maintaining Data Consistency 1-3 Copvnqnt L 2006 Syntaruec Corpo-anon All fights reserve.t
  14. 14. Atomic-Copy Resynchronization Atomic-copy rcsynchronizauon refers to the sequential writing of ~II blocks of the volume to a plcx. This operation is used anytime a new mirror is added to a volume, or all existing mirror is in stale mode and has to be rcsynchronizcd. Atomic-Copy Resynchronization Atomic-copy resynchronization involves the sequential writing of all blocks of a volume to a plex. This type of resynchronization is used in: • Adding a new plex (mirror) • Reattaching a detached plex (mirror) to a volume • Online reconfiguration operations: - Moving a plex - Copying a plex - Creating a snapshot - Moving a subdisk Atomic-Copy Resvnchrunizatlon Process 1 The plcx being copied to is set tu a write-only state. 2 A read thread is started on the whole volume. (Every block is read intcrnally.) 3 Blocks arc written from the "good' plcx tu the stale or new plcx. 1-4 Copvnqht ~', 200£ Sym<ll"'!p.c Corpmaltcn "II fights reserved VERITAS Storage Foundation 5.0 for UNIX' Maintenance
  15. 15. svrnantec. Read-Writeback Resynchronization Read-writeback resynchronization makes all plexes identical by alternately copying regions between plexes. This type of resynchronization is used in: • Recovery of mirrors after a system crash • Growing a volume In this type of resynchronization: • Mirrors marked ACTIVE remain ACTIVE, and the volume is placed in the SYNC state. • An internal read thread is started. Blocks are read from the plex specified in the read policy, and the data is written to the other plexes. • Upon completion, the SYNC flag is turned off. Read-Writeback Resynchronization Read-writeback rcsynchronization is a process of ensuring that two plcxes have the same content. Because the application must ensure that all writes arc completed. the application must fix any writes that are not completed. The responsibility ofYxYM is to guarantee that the mirrors have the same data. A database (as an application) usually docs this by writing the original data back to the disk. A tile system checks to ensure that all of its structures are intact. The applications using the tile system must do their own checking. Read-Writeback Resynchronlzatlon Process All plexes that were ACTIVE at the time of the crash are set to the ACTIVE state again. and the volume is placed in the SYNC state (or the NEEDSYNC state if the disk group has more than one volume). 2 An internal read thread is started to read the entire volume. and blocks are read from whatever plex is in the read policy and are written hack to the other plexcs, Because the default read policy is Select and this chooses Round Robin over Preferred. blocks are read from one plcx and written to another. alternately. 3 When the resynchronization process is complete. the SYNC flag is turned off (set to ACTIVE). User-initiated reads are also written to the other plexcs in the volume but otherwise have 110 effect 011 the internal read thread. Lesson 1 Maintaining Data Consistency Copyright if 2006 Sy!l181l1eC Corporanon. /III "gtll~ reserv-e 1-5
  16. 16. syrnantec Impact of Resynchronization Resynchronization takes time and impacts performance. To minimize this performance impact, VxVM provides the following solutions: • Dirty region logging for mirrored volumes • RAID-5 logging for RAID-5 volumes • FastResync for mirrored and snapshot volumes Minimizing the Impact of Resynchronization The process of rcsynchronizauon call impact system performance and can take time. Tu minimize the performance impact otrcsynchronizmion. VxVM provides: Dirty region lugging lor mirrored volumes RAID-5 logging lor RAID-5 volumes FastResync lor mirrored and snapshut volumes Till; FastResync option requires the lluslrSnup license. 1-6 VERITAS Storage Foundation 5.0 for UNIX: Maintenance Copvnqnt 2()1)6 Svmanrec Corporation 1111I<.111lsreserved
  17. 17. symamec. Dirty Region Logging • For mirrored volumes with logging enabled, DRL speeds plex resynchronization. Only regions that are dirty need to be resynchronized after a crash. • If you resize a volume, the log size does not change. To resize the log, you must delete the log and add it back after resizing the volume. Dirty Region Logging You were introduced to dirty region logging (DRL) when you created a volume with a log. This section describes how dirty region logging works. How Does DRL Work? DRL logically divides a volume into a set of consecutive regions and keeps track of the regions to which writes occur. A log is maintained that contains a status bit representing each region of the volume. For any write operation to the volume. the regions being written are marked dirty in the log before the data is written. If a write causesa log region to become dirty when it was previously clean. the log is synchronously written to disk before the write operation can occur. On system restart, VxVM recovers only those regions of the volume that are marked as dirty in the dirty region log. Log subdisks store the dirty region log of a volume that has DRL enabled. Only one log subdisk can exist per plex. Multiple log subdisks can be used to mirror the dirty region log. If a plcx contains a log subdisk and no data subdisks. it is called a log pier. Only a limited number of bits can be marked dirty in the log at any time. The dirty bit for a region is not cleared immediately after writing the data to the region. Instead. it remains marked asdirty until the corresponding volume region becomes the least recently used. 1-7Lesson 1 Maintaining Data Consistency Cr;pyngf):f. 2(J06 Syrnantec Corporanon All n!jfl'<; reserved
  18. 18. ji·}_I· Dirty Region Logging: Example 0123 ... [0010000.-::-0 0 100 .. 000 ... 0 0~iiJ 01 23 ... [O!OOOOO... 00000 .. 000 ... O~ Active Bitmap Volume DRL Before a Crash J=;;:;::::I 0 1 2 3 Recovery Bitrnap II DRL After a Crash ~ 0123 ... ): : LL_~_~_:_1_~_:_':_:_,,_. _:_:_:_0_0_:_·._:·._:_:_:_·.·.·_.:_O_~_:_:_:_:_I_R_~_~~_~_~_;y---J Dirty Region Log Size In the dirty region log: A small number of bytes of the DRL arc reserved for internal use. The remaining bytes arc used forthe DRL biunap, The bytes arc divided into two bitmaps: an active biunap and a recovery biunap. Each bit in the active biunap maps to a single region of the volume. A maximum of 204S dirty regions per system is allowed by default. How the Bitmaps Are Used in Dirty Region Logging Both bitmaps arc zeroed when the volume is started initially, after a clean shutdown. As regions transition to dirty, the corresponding bits arc set before the wntcs to the volume occur. If the system crashes, the activ c map is OR' d with the recovery map. Mirror rcsyuchronizatiun is now limited to the dirty bits in the recovery map. The active map is simultaneously reset, and normal volume ''0 is permitted. Usageof two biunaps in this way allows VxVM to handle multiple system crashes. 1-8 VERITAS Storage Foundation 5.0 for UNIX: Maintenance Copyright 2006 Svmaotec Corporanon All flgills reserved
  19. 19. symantcc Why Is It Important to Understand Object States • VxVM uses plex and volume states or condition flags to decide: - If a volume can be started - If a mirrored volume needs any synchronization - Which copy of data (plex) is to be used as the source if synchronization is required - If I/O is allowed to different copies of data (plexes) • You may need to manipulate plex or volume states under certain conditions: - To recover volumes if one or more failures prevent VxVM from taking automatic recovery actions - To recover from situations where interrupted configuration tasks leave volumes in unusable states - To take specific copies of data out of the I/O path for maintenance purposes - Use disk group configuration backups to create specific configurations Interpreting State Information for VxVM Objects VxVM uses plcx and volume states or condition Ilags to decide which operations can be performed. You may need to manipulate plex or volume states under certain conditions. 1-9Lesson 1 Maintaining Data Consistency Copy-rqtn ? 2UO(; Symanter coroorauoo. All "gills reserveo
  20. 20. syrnanter How Volumes Are Created vxassist is a top-down utility-that is, you only specify the properties of the volume you want to create-that creates volumes bottom-up: 1. Create subdisks. 2. Associate subdisks to plexes. 3. Associate plexes to a volume. 4. Initialize the volume's plexes. 5. Start the volume. How Volumes Are Created In order to troubleshoot and solve problems associated with mirrors, you must understand how volumes arc created. The vxassi st utility is a top-down utility. which means that you specify only the properties of the volume that you want to create. However, vxassist actually creates the volumes using a bottom-up approach. which means that subdisks an: created first and used to build volumes. To create a volume, vxassi st follows this process: Determine where you 1 ill place the data and create subdisks on the appropriate disk drives. 2 Create mirrors and associate each of the subdisks to the mirrors that will be used in the volume. 3 Create the volume and associate the mirrors to the volume. The result is a volume with one or more plcxcs. 4 Initialize the volume's plcxcs by selecting the plcx that represents the data for the volume. You perform this action by using the vxvol ini t command. Initializing a volume is similar to using a low-lcvcl lormat command on a disk drive: it states how to nuvig.uc to the data. (13ydefault. vxassist creates both plcxcs as hay ing the data and copies them together-using rcad-writcback synchronization. ) 5 Start the volume. Starting a volume involves enabling the area that the volume represents on disk, and enabling its object in the disk group configuration database. to accept user and system 1,'0. 1-10 VERITAS Storage Foundation 5.0 for UNIX' Maintenance Copyrl(.jht~' 2006 Svrnaruec Corpuranon All rights reserved
  21. 21. symantec. Identifying Plex Problems To identify and solve pie x problems, use the following information: • Plex states • Volume states • Plex kernel states • Volume kernel states • Object condition flags Commands to display plex, volume, and kernel states: vxprint -g diskgroup -ht [volume_name] vxinfo -p -g diskgroup [volume] You can use STATE fields in the output of the vxprint and vxinfo commands to determine that a problem has occurred, and to assist in determining how to fix the problem, VxVM displays state information for: Plcx states Volume states Plcx kernel states Volume kernel states Identifying Plex Problems The plex and volume state fields are not always accurate, because administrators can change them. However. kernel state flags are absolute; that is, only VxVM can change them. Therefore, kernel state flags arc always accurate. A particular plex state docs not necessarily mean that the data is good or bad. The plex state represents VxVM's perception of the data in a plex. VxVM is usually conservative; that is, ifVxVM detects that data is not synchronized. then the plcx states are set accordingly. Lesson 1 MaintainingData Consistency COOY(1g111 'f) 2006 Symantec Corporation All nqt-ts reserved 1-11
  22. 22. symantec. Displaying Object States vo I volOl plex voIDl-Q! plex volOl-02 Esgen ACTIn ACTIVE Sta.rted vxinfo -p -g datadg volOl vxprint -g datadg -ht volOl v NAME PVG!VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NeQL/WID MODE so NAME PLEX DISK DISKOFFS LENGTH [COLI) OFF DEVICE MODE sv NAME PLEX VOLNAME NVOLLA YR LENGTH tcor./: aFF !>.MINH MODE v voiOl ENABLED ACTIW 204800 SELECT fsgen pi 10101-01 volO! ENABLED ACTIV!: 205200 CONCAT RW ad datadgOl 01 voIOl-O! datadgOl 0 205200 0 diskO 1 ENA p I volOl-02 voiOl ENABLED ACTIVE 205200 CONCAT RW sd datadg02-01 volOl-02 datadg02 0 205200 0 diskO 2 ENA Example of vxinfo and vxprint If you do not specify the volume name on the command line for the vxprint or vxinfo commands, information on all the volume, within the specified disk group is displayed. 1-12 VERITAS Storage Foundation 5.0 for UNIX: Maintenance
  23. 23. syruantec Plex States and Condition Flags EMPTY • CLEAN (SNAPDONE) ACTIVE (SNAPDONE) Pl:ENABLED/ACTIVE P2:ENABLED/ACTIVE [V: ENABLED/ACTIVE] I vxvol start ~J Eval stopvi Key ~ Pl:DISABLED/CLEAN P2:DISABLED/CLEAN [V: DISABLED/CLEAN] i P1: 1st Plex States j P2: 2nd Plex States 1 [v: Volume States} i Interpreting Plex States Plex States EMPTY: When you creme a volume, all of the plexes and the volume are set to the EMPTY state.This state indicates that you have not yet defined which plex has the good data (CLEAN). and which plcx does not have the good data (STALE). You can only achieve the EMPTY state by creating a new volume using vxmake. or by using related administrative commands. CLEAN: The CLEAN state is normal and indicates that the plex has a copy of the data that represents the volume. CLEAN also means that the volume is not started and is not currently able to handle 110 (by the administrator's control). ACTIVE: The ACTIVE state is the same as CLEAN. but the volume is or was currently started, and the volume is or was able to perform 110. SNAPDONE: The SNAPDONE state is the same as ACTIVE or CLEAN, but SNAPDONE is a plcx that has been synchronized with the volume as a result of a vxassist snapstart operation. After a reboot or a manual start of the volume. a plcx in the SNAPDONE state still exists. It is persistent. 1-13Lesson 1 MaintainingDataConsistency Copyrlght;&' 20D6 Svrnantec Corporauon. 111fights reserved
  24. 24. syrnantec. Plex States and Condition Flags STALE (SNAPATT) OFFLINE TEMP Key ~ P1: tst Plex States P2: 2nd Plex States [V: Volume States) STALE: The STALE state indicates that VxVM detects that the data in the plcx is not synchronized with the data in the CLEAN plcxcs. This state is usually causedby taking the plcx offline (110 can still be going to the other plcxcs, making them unvynchronizcd) or by a disk tailurc-c-which means that the plcx was not updated when new writes came into the volume. SNAPATT: The SNAPATT state indicates that the object is a snapshot that is currently being synchronized but docs not yet have a complete copy of tile data. OFFLINE: The OFFLINE stare indicates that the administrator has issued the vxmend of f command on the plcx. The plcx does not participate in any I/O when it is offline, so actively writing to the volume causesthe contents to become outdated. When the administrator brings the plcx back online using the vxmend on command, the plcx changes to the STALE state. TEMP: The TEMP state flags (TEMP, TEMPRM, TEMPRMSD) usually indicate that the data was never a copy olthc volume's data, and it is recommended that you not use these plcxcs. These temporary states indicate that the plcx is currently involved in a synchronization operation with the volume, 1-14 VERITAS Storage Foundation 5,0 for UNIX: Maintenance COlJynqhl 7. 2006 Svmantet, Corpornnou All nqhts reservec
  25. 25. symantcc Note: If the volume is nonredundant at the time that you reattach the drive, the plex state changes from NODEVICE to FE COVER instead of IOFAIL. Condition Flags If a plcx is not synchronized with the volume and VxVM has information about why it is not synchronized. then a condition flag is displayed. Multiple condition flags can be set on the same plex at the same time. Only the most informative flags are displayed in the state field of the vxprint output. For example, if a disk fails during an I/O operation, the NODEVICE, IOFAIL, and RECOVER flags arc all set for the plex, but only the NODEVICE flag is displayed in the state field. NODEVICE: NODEVICE indicates that the disk drive below the plcx has failed. REMOVED: REMOVED has the same meaning as NODEVICE. but the system administrator has requested that the device appear to have failed (for example. by using the vxdiskadm option. "Remove a disk for replacement"). IOFAIL: IOFAIL is similar to NODEVICE, but it indicates that an unrecoverable failure occurred on the device, and VxVM has not yet verified whether the disk is actually bad. (110 to both the public and the private regions must fail to change the state from IOFAIL to NODEVICE.) RECOVER: The RECOVER flag is set on a plcx when two conditions arc met: A failed disk has been fixed (by using vxreattach or the vxdiskadm option, "Replace a failed or removed disk"). The plex was in the ACTIVE state prior to the failure, This flag indicates that even after fixing the volume, additional action tlIay be required. The data may be lost and must be recovered from backup, or the administrator must verify that the data on the disk is current by using utilities provided by the application that uses that volume. Lesson 1 Maintaining Data Consistency 1-15 Copvrrqhf " 2(106 Symantec Corporation All fI'1hl<; reserved
  26. 26. syrnantec Volume States • EMPTY } These volume states have the same • CLEAN meanings as they do for plexes. ACTIVE • NEED SYNC This state is the same as SYNC, but the internal read thread has not been started. • SYNC Plexes are involved in read- writeback or RAIO-5 parity synchronization. • NODEVICE None of the plexes have currently accessible disk devices underneath the volume. Interpreting Volume States Volume States EMPTY, CLEAN. and ACTIVE: The EMPTY. CLEAN. and ACTIVE volume states have the same meanings as they do for plcxcs. NEEDSYNC: The NEEDSYNC volume state is the same as SYNC, but the internal read thread has not been started. This state exists so that volumes that use the same disk arc not synchronized at the same time, and head thrashing is avoided. SYNC: The SYNC volume state indicates that the plcxcs arc involved in read- writcback or RAID-S parity synchronization: Each time that a read occurs from a plcx, it is written back to all the other plcxcs that arc in the ACTIVE state. An internal read thread is started to read the entire volume (or, alter a system crash. only the dirty regions ifdirty region logging (DIU.) is being used). forcing the data to be synchronized completely. On a RAID-S volume, the presence ofa RAID-S log decreases the time ofa SYNC operation. Starting an empty mirrored volume by using the vxvol start command places the volume in SYNC mode. NODEVICE: The NODEVICE volume state indicates that none of the plcxcs have currently accessible disk devices underneath the volume. 1-16 VERITAS Storage Foundation 5,0 for UNIX: Maintenance COtlyfigtil'i'· 2006 Syrnantpc Couorauoo All nqnts reserved
  27. 27. symantec. Kernel States Kernel states represent VxVM's ability to transfer 110 to the volume or plex. • ENABLED The object can transfer both system 110and user 110. The object can transfer system 110, but not user 110(maintenance mode). No 110can be transferred. Kernel states represent VxVM ability to transfer 1/0 to the object. Volume kernel stat 1': VxVM's ability to transfer I/O to the volume Plex kernel state: VxVM's ability to transfer I/O to the plex ENABLED: The ENABLED kernel state indicates that the object is currently able to transfer system I/O to the private region and user I/O to the public region. DETACHED: The DETACHEDkernel state indicates that the object can currently transfer system I/O. but 110t user [/0. This state is also considered the maintenance mode where internal plex operations and ioct 1 functions arc accepted. DISABLED: The DISABLED state is the offline state for the volume or the plex. When all object is in this state. no I/O is transferred. • DETACHED • DISABLED Interpreting Kernel States Kernel States Lesson 1 Maintaining Data Consistency Copvnqht f 2006 Syrnanter Corporation, 111nqtus reserved 1-17
  28. 28. symantec Example Scenarios • SCENARIO 1: - You are planning to go through an upgrade procedure that may corrupt data. You want to keep one safe copy in case things go wrong. - You do not have enough space to add snapshot volumes, so you take one plex out of the 110 path during the upgrade . • SCENARIO 2: - You are mirroring a critical application data across disk arrays in multiple sites. - A disaster first causes your remote site to be temporarily disconnected and then causes the primary copy to be lost permanently before you can recover from the temporary failure. - You now have the option of using the remote site data, which is several minutes old, or recovering from last night's backup. Modifying VxVM Object States Example Scenarios Determine the best action 1'01' the: scenarios that arc described on the slide. 1-18 Cupynght t- 2006 Sy-u.uuec COrpura11011 Ail nqnts resarvec VERITAS Storage Foundation 5.0 for UNIX Maintenance
  29. 29. symantcc. Solving Plex Problems Commands used to fix plex problems include: • vxrecover • vxvol -f start • vxmend fix • vxmend off Ion Resolving Plex Problems When resolving disk and plcx problems, after you fix the underlying disk drives by using the disk commands, you must fix plcx problems by using the following commands: • vxrecover • vxvol -f start • vxmend fix • vxmend off Ion Copvnuht (' 2006 Syraantec Corporation. All nqhts reserved 1-19Lesson 1 Maintaining Data Consistency
  30. 30. When vxrecover is executed. VxVM notes the state of the plcxcs in the volume. lfboth ACTIVE and STALE plcxcs exist. the ACTIVE plcxcs issue unconditional block writes over the STALE plcxcs, If there arc only ACTIVE plcxcs. the rcad-wriicback copy procedure is performed. Recovery is performed only on volumes that require recovery (such as volumes marked as dirty before a sudden system failure). During the recovery process. the volume remains online and started. When the synchronization process is complete, the volume and all of its plcxcs arc ACTIVE and ENABLED. Running vxrecover without specifying a volume name can cause a synchronizauon operation to be started in parallel on all volumes that need recovery. One synchronization operation runs on each drive (if necessary), and volumes on different drives are synchronized in parallel. Synchronization can affect system performance. If you have many volumes that need to be recovered. you may prefer to: Start the volumes without recovery by using vxrecover - sri. Note: The - s option is only used when the volume is stopped. 2 Recover individual volumes or recover all of the volumes when I/O traffic is lo« by u,ing vxr e cove r. Note: As lung as one CLEAN or ACTIVE. non-volatile plcx (a plcx with no Ilags set) is available inside a volume. you can start the volume using that plcx. The administrator can recover any other plcxcs in the volume immediately. or defer recovery to a later time. Recovering Volumes vxrecover -g diskgroup -8 [volume] • Recovers and resynchronizes all plexes in a started volume • Runs the vxvol start and vxplexatt commands (and sometimes vxvol resync) • Works in normal situations • Resynchronizes all volumes that need recovery if a volume name is not included • Examples: vxrecover -s vxrecover -s vol0l The vxrecover Command 1-20 C()p'lfl,j~lI yi 2006 Svn.en.ec Cmpl)f.JtHlI1 All pghts reserved VERITAS Storage Foundation 5.0 for UNIX Maintenance
  31. 31. symantcc vxvol -g diskgroup -f start volume name • This command ignores problems with the volume and starts the volume. • Only use this command on nonredundant volumes. If this command is used on redundant volumes, data can be corrupted unless all mirrors have the same data. • Example: vxvol -g datadg -f start volOl vxvol -f start volume name When you force a volume to start: Ifall plexes have the same state. then read-writcback synchronization is performed. If the pie xes do not have the same state. then atomic-copy rcsynchronization is performed. The vxvol start Command If a volume does not start with this command. it usually indicates that there is a problem with the underlying plcxes. Forcing a Volume to Start If you add the - f flag. YxYM ignores the underlying problem and forces the volume to start: Caution: Force-starting a volume can have catastrophic resuits. Use extreme caution when force-starting a mirrored volume after a disk failure and replacement. Forcing a mirrored volume to start can unconditionally synchronize the volume using a rcad-writeback method of alternating between plex blocks. NULL plex blocks may overwrite good data in the volume. corrupting the data. Only perform a forced start 011 a nonredundant volume. Lesson 1 Maintaining Data Consistency Copynqtn f' 2006 Svruame- Corpoeanon All nqhls reserveo 1-21
  32. 32. Modifying Plex and Volume States Manually syrnaniec • The volume that the plex is associated with must be in DISABLED mode to modify the plex state. • You may need to move a plex to STALE state as an intermediate step before changing to CLEAN or ACTIVE state. • Use this command as a last resort if none of the other recovery options help. vxmend -g diskgroup fix option object • stale • clean • active • empty (only used on a volume) Examples: vxmend -g datadg fix stale volOl-Ol vxmend -g datadg fix clean volOl-Ol The vxmend Command To manually reset or change the stare ofa plcx or volume, you can use the vxme nd fix command. Use this command if you know more about a plcxs data than YxYM docs. You can only set plcx states with vxrne nd fix when the host volume or the plcx is stopped. Caution: Use caution and discretion when issuing the vxmend fix command and its options. The vxmend fix command changes states set and cleared automatically by the vxconfigd daemon. Ifused incorrectly. this command can make the plcx, its volume, and its data inaccessible, and you may have to restore the data Irom backup. 1-22 Copvnqnt -, 2006 Syrnantec Corcoralton All nqnts reserved VERITAS Storage Foundation 5.0 for UNIX: Maintenance
  33. 33. symantec vxmend -g diskgrollp on plex_name Taking Plexes Out of the 1/0 Path When analyzing plexes, you can temporarily take plexes offline while validating the data in another plex. • To take a plex offline, use the command: vxmend -g diskgrollp off plex_name • To take the plex out of the offline state, use: vXIIlend off Ion When analyzing plex problems to determine which plex has the correct data. you may need to take some plcxes offline temporarily while you are testing a particular plex by using this command. Lesson 1 Maintaining Data Consistency 1-23 Cooynqtu f. 2006 Symanter Corporation All nums reserved
  34. 34. syrnantec Example: If the Good Plex Is Not Known The volume is disabled and not startable, and you do not know what happened. There are no CLEAN plexes. To resolve: 1. Take all but one plex offline and set that plex to CLEAN. 2. Run vxrecover -so 3. Verify data on the volume. 4. Run vxvol stop. 5. Repeat for each plex until you identify the plex with the good data. P1 RvolOl-Ol: volOl-02: DISABLED/STALE :DISABLED/STALE Analyzing Plex Problems Example: If the Good l'Icx Is :'IIot Known What if both plcxcs are in the STALE state? Regardless of what happened to the plcxcs or the disks underneath. it is not sale to guess which plcx has the more recent (or good) data and start the volume. I I' you arc not sure which plcx has good data, then the salest solution is to test each plcx one by one. 1 Take all but one plcx online and set that plcx to CLEAN. 2 Run vxrecover - s. 3 Verify data on the volume. Mount the file system as read-only so you do not have to run a file system check. 4 Run vxvol stop. S Repeat for each plcx until you identity the plcx with the good data. This process requires step-by-step attention 10 all volume and plcx object details. Use vxprint -ht to monitor any volume and plcx state changes that occur as a result of your vxmend commands. Without a method to test the validity of the datu, you must restore the data from backup. For example. if your application is staning. call you guarantee that the data it contains is the 1110strecent With a Iilc system. is f sck enough to guarantee that the data in a file is there'! Even if you can mount the file system. you can lose the data in some files in the process. 1-24 VERITAS Storage Foundation 5.0 for UNIX Maintenance Copvnqnt ,2006 Svm<ln!ec Corporanon llil rights reserved
  35. 35. If the Good Plex Is Not Known: Example In the example, you can resolve the problem by using the following commands. Set the good plex to CLEAN. vxmend -g diskgroup off volOl-02 vxmend -g diskgroup fix clean volOI-OI Verity that data is on the plcx by using the volume: vxrecover -s volDI vxvol -g diskgroupstop volOI vxmend -g diskgroup -0 force off voIOI-Ol(lastcleanplexin the volume) vxmend -g diskgroupon volOl-02 vxmend -g diskgroupfix clean volOl-02 Verity that data is on the plcx by using the volume: vxrecover -s volOI If the current plex (vol01-02) has the correct data: vxmend -g diskgroupon volOl-Ol vxrecover volOI If vol 01- 01 had the correct data: vxvol -g diskgroup stop volOl vxmend -g diskgroup fix stale volOl-02 vxmend -g diskgroup on volOl-Ol vxmend -g diskgroup fix clean volOI-OI vxrecover -s volDI Lesson 1 Maintaining Data Consistency Conyflg~lt 1':20UOSyruantec Corporation All nqhts reserved 1-25
  36. 36. symantec. symanrec. Appendix l3 provides complete lab instructions and solutions. "1 ab 1S"luiiull''': lJiIIL}ljJl[l~ !)~!u{ I)li,j'li,.'n<..'l P~!~'..'H··~ Lesson Summary • Key Points This lesson described mirror resynchronization processes. This lesson also introduced the various states in which Volume Manager objects. such as volumes and plexes. can exist and described the tools that you can use to solve problems related to data consistency by analyzing and changing these states. • Reference Materials - VERITAS Volume Manager Administrator's Guide - VERITAS Volume Manager Troubleshooting Guide Lab 1 Lab 1: Maintaining Data Consistency In this lab, you practice recovering from a variety of plex problem scenarios, and optionally, observe the benefits of a dirty region log during a system crash. To investigate and practice recovery techniques, you will use a set of interactive lab scripts. For Lab Exercises, see Appendix A. For Lab Solutions, see Appendix B. Labs and solutions for this lesson JI'C located on the:Iollo« ing pages: Appendix A provides complete lab instructions. "1 dh i Llililciiillnc:: ):i1:1 {·on"i'kHt.·~,'· pa~i,l'/"'~ 1-26 VERITAS Storage Foundation 5.0 for UNIX: Maintenance Cr)llvflgt11 2006 Sy-nar-tec CJrr)()(iliIOf1 All fightS reserved
  37. 37. Lesson 2 Managing Devices Within the VxVM Architecture
  38. 38. syrnantec. Lesson Introduction Lesson 1: Maintaining Data Consistency • ~:~Y;Y~;~~r&;!~~~~ices ~ithin ..- Lesson 3: Encapsulation and Rootability Lesson 4: Troubleshooting the Boot Process Lesson 5: Volume Maintenance Lesson 6: Performance Monitoring Lesson 7: Point-in-Time Copies Lesson 8: Other Enterprise Features Overview svrnantec Lesson Topics and Objectives Topic After completing this lesson. you will be able to: Topic 1: Managing Components in the VxVM Architecture Manage components of the VxVM architecture, including the VxVM configuration database, the configuration daemon, and volboot. Topic 2: Discovering Disk Devices Describe the VxVM device discovery function. Topic 3: Managing Multiple Paths to Disk Devices • Describe how dynamic multipathing works with active/active and active/passive disk arrays. • Prevent multipathing for a specific device. • Disable a specific 110 path. • Control the DMP restore daemon. 2-2 VER/TAS Storage Foundation 5.0 for UN/X: Maintenance
  39. 39. VxVM Architecture User Applications File Operating System Block Device Switch [dak] Character Device Switch (rdsk) VxVM Config Databases Managing Components in the VxVM Architecture VxVM Architecture VxVM is a device driver that is placed between the UNIX operating system and the SCSI device drivers. When VxVI'v1 is running. UNIX invokes the VxVM device drivers instead of the SCSI device drivers. YxVM determines which SCSI drives are involved ill the requested I/O and delivers the I/O request to the drives. VxVM Daemons YxYM relies on the following constantly running daemons for its operation: vxconfigd- The VxVM configuration daemon maintains disk and group configurations. communicates configuration changes to the kernel. and modifies contigurution information stored on disks. When a system is booted. the vxdctl enable command is automatically executed to start vxconfigd. YxVM reads the /etc/vx/volboot file to determine disk ownership and automatically imports disk groups owned by the host. vxiod The VxYM 1/0 daemon provides extended 1/0 operations without blocking calling processes.Several vxiod daemons arc usually started at boot time. and they continue to run at all times. vxrelocd~~vxrelocd is the hot-relocation daemon that monitors events that affect data redundancy. If redundancy failures are detected. vxrelocd automatically relocates affected data from mirrored or R!ID-5 subdisks to spare disks or other free space within the disk group. ~~~~~- --~--~~---- Lesson 2 ManagingDevicesWithinthe VxVM Architecture 2~3 Copyrlght,f 20(,6 Syrnantec Corporation. All nqhts res erveo
  40. 40. symaruec VxVM Configuration Database • Contains all disk, volume, plex, and subdisk configuration records • Is stored in the private region of a VxVM disk • Is replicated to maintain a copy on multiple disks in a disk group - VxVM maintains an appropriate number of active copies per disk group. - Copies are stored across enclosures to maximize redundancy. • Is updated by the vxconfigd process VxVM Configuration Database The Vx VM configuration database stores all disk. volume. plcx. and subdisk configuration records. The vxconf ig device (/ dev/vx/ conf ig) is the interface through which all changes to the volume driver state are performed. This device can only be opened by one process at a time. and the initial volume configuration is downloaded into the kernel through this device. The coufigurution database is stored in the private region ofa VxVM disk. Each disk that has a private region holds an entire copy of the configuration database for the disk group. The size of the configuration database for a disk group is limited by the size of the smallest copy of the configuration database on any of its member disks. Tile VxVM configuration is replicated within the disk group to protect against loss of the configuration in case of physical disk failure. vxconf igd actively monitors live or more copies of the configuration database for each disk group. VxYM balances their locations based on the number of controllers. targets and disks in the disk group. With VxVM 3.2 and later. VxVM configuration copies are placed across the enclosures spanned by a disk group to ensure maximum redundancy across enclosures, The vxconf igd configuration daemon. is the process that updates the configuration through the vxconf ig device. The vxconf igd daemon was designed to be the sole and exclusive owner ofthis device. 2-4 CUiJYrly~ll ~,2006 Svmanlec Corpor<lllol1 All fights reserved VERITAS Storage Foundation 5.0 for UNIX: Maintenance
  41. 41. symantcc Displaying VxVM Configuration Database Information ivxdg list acctdg Group: acctdg _ .. _ ~gid: . 1023996467.1130. trainsrCOnfi9Uration Impo r t= Ld . 0.1129 database size config copy 1 len=48144 state=clean online config disk c1t1dOs2 copy 1 len=48144 state=clean online config disk c1t2dOs2 copy online config disk c1t3dOs2 copy 1 config disk c1t9dOs2 copy config disk c1t10dOs2 copy 1 online config disk c1t11dOs2 copy 1 log disk c1tOdOs2 copy 1 1en=7296 log disk c1t1dOs2 copy 1 len=7296 iNot active "I copies: config: Displaying Disk Group Configuration Data To display the status of the configuration database for a disk group: vxdg list diskgroup Ifno disk group is specified, information from all disk groups is displayed in an abbreviated format. When you specify a disk group. a longer format is used to display the status of the disk group and its configuration. In the example, live disks have active configuration databases (online). and two disks do not have an active copy of the data (disabled). The configuration database for a disk group is the size of the smallest private region in the disk group. Log entries are on all disks that have databases. The log is used by the VxVM kernel to keep the state of the drives accurate if the database cannot be kept accurate (for example, if the configuration daemon is stopped). By default. for each disk group, VxVM maintains a minimum of live active database copies on the same controller. In most cases. VxVM also attempts to alternate active copies with inactive copies. In the example. clt3dO and clt9dO are disabled. If different controllers are represented on the disks in the same disk group, VxVM maintains a minimum of two active copies per controller. In the output on the slide, the Configuration database size (pe rml en«) is next to a field named f r e e«. The free= field can be used to check how fast the configuration database is filling lip so that action can be taken before the disk group runs out of database space. lesson 2 Managing Devices Within the VxVM Architecture 2-5 Copyright '," 2006 Symanter Corporancn. ,.11nallis reserved
  42. 42. Displaying Disk Header Information The terms displayed in the output oi'vxdisk list include: Term Description f----------+- Full LNIX de ice nameotdiskDevice Device name used by VxVM to refer to the physical disk type Method of placing the disk under VxVM control ~------------4------------------------------------------------ devicetag hostid Name of system that manages the disk group (If blank, no host is currently controlling this group.) Disk group name and internal ID disk VM disk media name and internallD info group flags Disk fonnat. private region offset, and partition numbers for public and private regions ------------------------ Settings that describe status and options till' the disk Paths for block and character device tiles of the public region "I' the disk pubpaths iosize The iosize range that the disk accepts version Version number of header format public, private Partition (slice) number. offset trorn beginning of the partition. length of the partition. and disk offset 2--6 C:"PYflynl <, 21l!)6 Svntantec C')'1}(,rd[,on All nqtus reserved VERITAS Storage Foundation 5.0 for UNIK Maintenance
  43. 43. symantcc config priv 000256-048207 [047952] :copy=Ol offset=000192 enabled log priv 048208-055503 [007296] :copy=01 offset=OOOOOOenabled lockrgn priv 055504-055647[000144] :part=O offset=OOOOOO configuration database copies, logs, and lock ;-i="7'===~=:;;;;;:7':~:::::7.~;;;;;;;;;:;;;;;;;;;;~~Last update to the private region and location of header (sector 0) and offset to header copies ~~~~"":~~=-r.~7'Z''1''Trr----1 (sector 240) Defined regions: config priv 000048-000239 [000192] :copy=01 Location of offset=OOOOOOenabled Multipathing information: numpaths: 2 cltOdOs2 state=enabled c2tOdOs2 state=disab The following is a continuation of vxdi sk 1 i st output descriptions: Term Description update Date, time, and sequence number ofthe last update to the private ssb region headers Offset to two copies of the private region header configs Number of configuration database copies kept in the private region logs Number ofkerncllogs kept in the private region Defined Location ofconfiguration databases, kernel logs, and lock regions regions in the private region Because the database or logs can be split, there can be multiple pieces. Therefore, the otlset is the starting location within the private region where this piece of the database begins. Copy represents the copy of the database to which this piece belongs. Multipathing ltdynamic multipathing is enabled and there are multiple paths to Lnf orrna t i on the disk, this item shows information about the paths and their status. Lesson 2 Managing Devices Within the VxVM Architecture 2-7 COi1yrlght;!':, 21)06 Symanter. Corporaton All (lallS reserved
  44. 44. symantec. Disk Types and Disk Formats Disk types and formats include: auto indicates that when the vxconf igd daemon has been started. YxYM automatically COli figures a disk accessrecord for the disk basedon a list of known disk device addressesobtained trom the operating system. Auto- configured disks arc displayed with their type aud qualified by their format. For example, auto: cdsdisk indicates an auto-configured disk that is formatted as a cross-platform data sharing (CDS) disk that is suitable lor moving between different operating systems. This is the default format lor most disks on a system, hut not lor hoot. root, or swap disks. If a disk is automatically configured by YxYM as a simple or sliced disk, you will seedisk types and formats, such as auto: simple and auto: sliced. auto:none indicates that the disk is not Iormaucd lor YxYM. cdsdisk indicates that the public and private regions are contiguous on the same partition and suitable lor moving between different operating systems. sliced indicates that the public and private regions are separatepartitions. none indicates that there an: no public or private regions on the disk. VxVM Disk Types and Formats • auto: Automaticaffy configured by VxVM auto: cdsdisk (Default for 4.x and higher) • auto:simple • auto: hpdisk (Only on HP-UX) • auto:none • cdsdisk: Public and private regions are contiguous on the same partition and suitable for moving between different operating systems. • sliced: Public and private regions are on separate partitions. • hpdisk: HP-UX specific disk format is used for system disk and before version 4.x. • none: There are no public and private regions. Notes: You can change the default format by using the vxdiskadmoption. "Change/Display the default disk layouts" or in /etc/vx/disk. Non-boot sliced disks can be converted to CDS disks by using the vxcdsconvert command. 2-8 VER/TAS Storage Foundation 5.0 for UN/X: Maintenance COPyright 2nD€> SvrnantscCorrorauon All r<gtlls reserved
  45. 45. ,a", VxVM Configuration Daemon vxconfigd: Maintains the configuration database Synchronizes changes between multiple requests, based on a database transaction model: symantec. All utilities make changes through vxconfigd. Utilities identify resources needed at the start of the transaction. Transactions are serialized, as needed. Changes are immediately reflected in all copies. Does not interfere with access to data on disk Must be running for changes to be made to the configuration database If vxconfigd is not running. VxVM operates, but configuration changes are not allowed and queries of the database are not possible. The VxVM configuration daemon must be running in order for configuration changes to be made to the VxVM configuration database. Ifvxconfigd is not running, VxVM operates properly. but configuration changes are not allowed and queries of the database arc not possible. The vxconf igd daemon synchronizes multiple requests and incorporates configuration changes based on a database transaction model: All utilities make changes through vxconf igd. Utilities must identify all resources needed at the start of a transaction. Transactions arc serialized. as needed. Changes are immediately reflected in all copies of the configuration database. The vxconf igd daemon does nut interfere with user or operating system access to data on disk. Controlling the VxVM Configuration Daemon Copynght@,2006 Symaf1ll'!i; Cocooranoo. All flqms reserv"d 2-9Lesson 2 Managing Devices Within the VxVM Architecture
  46. 46. VxVM Configuration Daemon • vxconfigd reads the kernel log to determine current states of VxVM components and updates the configuration database. • Kernel logs are updated even if vxconfigd is not running. For example, upon startup, vxconfigd reads the kernel log and determines that a volume needs to be resynchronized. • vxconfigd modes: - Enabled Normal operating state - Disabled Most operations not allowed Booted Part of normal system startup while acquiring the boot disk group vxconfigd Modes vxconfigd reads the kernel log to determine current states ofVxVM components ami updates the configuration database. Kernel logs arc updated even ifvxconfigd is not running. For example, upon startup, vxconfigd reads the kernel log and determines that a volume needs to be rcsynchronizcd. vxconf igd operates in one of three modes: Enabled Enabled is the normal operating mode in which most configuration operations arc allowed. Disk groups arc imported. and Vx"M begins to manage device nodes stored in / dev /vx/ dsk and / dev /vx/ rdsk. Disabled In the disabled mode, most operations arc not allowed. vxconf igd docs not retain configuration information lor the imported disk groups and docs not maintain the volume and plcx device directories. Certain failures, most commonly the loss of all disks or configuration copies in the boot disk group, cause vxconf igd to enter the disabled state automatically. Buotcd The booted mode is part of normal system startup. prior to checking the root file system. The booted mode imports the boot disk group and waits tor a request to enter the enabled mode. Volume device node directories arc not rnauuaincd, because it may not be possible to I rite to the root file system. Copyngtll "::. 2006 Swnantec Corporation All rights reservec 2-10 VERITAS Storage Foundation 5.0 for UNIX: Maintenance
  47. 47. ~m Managing the VxVM Configuration Daemon Use vxdctl to control vxconfigd. vxdctl mode Displays vxconf igd status vxdctl enable Enables vxconfigd vxdctl disable Disables vxcon f igd vxdctl stop Stops vxconfigd vxdctl -k stop Sends a kill - 9 vxconfigd Starts vxconfigd vxdctl license Checks licensing vxdctl support Displays version information The vxdctl Utility symantec. vxconf igd is invoked by startup scripts during the boot procedure. To manage some aspects ofvxconf igd, you can use the vxdct 1 utility. Displaying vxconfigdStatus To determine whether the configuration daemon is enabled. you use the vxdct 1 mode command. This command displays the status of the configuration daemon, If the configurauon daemon is not running. it must be started in order to make configuration changes. Disk failures arc also configuration changes, but there is another way of tracking them if the daemon is down (kernel logs). Enabling vxconfigd If vxc on f igd is running, but not enabled, you use vxdct 1 enable to enable the configuration daemon. This command forces the configuration daemon to read all the disk drives in the system and to set up its tables to reflect each known drive. When a drive fails and the administrator fixes the drive. this command enables YxYM to recognize the drive. Disabling vxconfigd To prevent configuration changes from occurring, you can disable the daemon by using vxdctl disable. vxconf igd records all commands executed. whether through the YEA or the CLI. These commands are stored in /var/adm/vx. The veacmdlog tile records YEA commands and the cmdlog tile records ell commands. Lesson 2 Managing Devices Within the VxVM Architecture Ccpyrrqnt ~) 2006 Svmantar- cororcuoo All fights reserve,1 2-11
  48. 48. syrnantec This host ID is used to ensure that two or more hosts that can access disks on a shared SCSI bus do not interfere with each other in their use of those disks. This host ID is important in the generation of unique ID strings that arc used internally lor stamping disks and disk groups. The volboot liic also contains the name of the system-wide default disk group if this has been configured. If the boot disk is under V-xVM control, the vol boot file also contains the name of the boot disk group to which the boot disk belongs. Caution: Never edit the volboot file manually. If you do so. its checksum is im alidated. The volboot File /etc/vx/volboot contains: • The host 10that is used by VxVMto establish ownership of physical disks . • The values of defaultdg and bootdgif these values were set by the user Caution: Do not edit volboot, or its checksum is invalidated. To display the contents of volboot: vxdctl list To change the host 10in volboot: vxdctl hostid newhostid vxdctl enable To re-create volboot: vxdctl init llostid Note: The hostid field in /etc/vx/valboot is not returned by the UNIX hostid command, but rather by the hostnamecommand. Managing the volboot File Viewing the Contents of volboot To view the decoded contents of the vol boot file: vxdctl list volboot file ve rs i on : 3/1 seqno: 0.1 cluster protocol version: 70 hostid: trainl 2-12 VERITAS Storage Foundation 5.0 for UNIX: Maintenance Curyroqlll lO[)6 Svm.sruec Curp'H,11I011 All nor.ts reserveo
  49. 49. symaniec Device Discovery Layer (DOL) vxdiskconfig (Solaris only) vxdisk scandisks vxdctl enable User process level Device discovery is the process of locating and identifying disks attached to a host. Kernel process level VxVM Kernel Device discovery occurs automatically whenever you add a new disk array. Discovering Disk Devices What Is Device Discovery? Device discovery is the process of locating and identifying the disks that are accessible to a host. YxYM 3.2 and later features. such asdynamic multipathing (DMP). depend on device discovery. Device discovery enables you to dynamically add support for disk arrays from a variety of vendors without rebooting the system. Discovering and Configuring Disk Devices To dynamically discover new devices, use the vxdiskconf ig utility. This utility scans for disks that were added since YxYM's configuration daemon was last started and dynamically configures the disks to be recognized by YxYM. The vxdiskconf ig utility invokes OS utilities, such as devf sadmon Solaris, to ensure that the OS recognizes the disks. vxdi skconf ig then invokes vxdct 1 enable, which rebuilds volume node directories and the DMP internal database tu reflect the new state of the system. DOL enables YxYM to use more descriptive names when using enclosure-based naming, for example. emcO_1 rather than Di sk_l. Note: The vxdi skconf ig utility docs nut exist on IIP-UX. Lesson 2 Managing Devices Within the VxVM Architecture 2-13 Copynqbt ~~ 2006 Syrnantec Corporal Ion II! nqhts reserven
  50. 50. Adding Support for a New Disk Array With V x VM version 3.1 and later. to add support for a new type of disk array that is developed by a third-party vendor, you must add vendor-supplied libraries by using plauorm-spccific package installation commands. The new disk array docs not need to be connected to the system hen the package is installed. You may need to scan tor new devices by issuing plauonn-spccilic commands. Theil run vxdctl enable to ensure that VxVM updates the device list. Adding Disk Array Support • To add support for a new type of disk array, add vendor- supplied libraries. For example: pkgadd -d /edrom/pkgdir SEAGTda (Solaris) swinstall -5 /edrom/depotdir SEAGTda (HP-UX) installp -ae /edrom/pkgfile SEAGTda (AIX) rpm -ihv /edrom/pkgdir SEAGTda,rpm (Linux) Scan for new devices: vxdetl enable - This command invokes vxconfigd to scan lor all disk devices, updates the device list, and reconligures DMP. - You do not need to reboot the host. Note: VxVM supports many arrays "out-of-the-box." See vxddladm listsupport lor a complete list. To remove support for a disk aITaY. you remove the vendor-supplied library package by using the OS-spcci lie command. For example, to remove support lor the SEAGTda disk array: Sularis Removing Support for a Disk Array pkgrm SEAGTda HP-UX swremove SEAGTda AIX I.iIlUX installp -u SEAGTda rpm -ev SEAGTda Ir the arrays remain physically connected to thc host alter support has been removed. they arc listed in the OTHER_DISKS category. and the VOIUllh:S remain available. 2-14 VERITAS Storage Foundation 5. afor UNIX: Maintenance Cooynght 2006 Symantec CorPOfil!lQf1 All rights reserved
  51. 51. svmantcc You can use the vxdisk scandiskscommand to scan part of the OS device tree, as follows: • Discover newly added devices previously unknown to VxVM: vxdisk scandisks new • Discover fabric devices: vxdisk scandisks fabric • Scan for the specific devices: vxdisk scandisks device=cltldO,c2t2dO • Scan for all devices except those that are listed: vxdisk scandisks !device=cltldO,c2t2dO • Scan for devices that are connected to logical or physical controllers: vxdisk scandisks ctlr=cl,c2 • Discover devices that are connected to the specified physical controller: vxdisk scandisks pctlr=/pci@lf,4000/scsi@3/ Partial Device Discovery VxVM supports partial device discovery where you can include or exclude sets of disks or disks attached to controllers from the discovery process. Partial device discovery reduces redundant discovery operations by scanning only a part of the OS device tree. The vxdisk scandisks command rescansthe devices in the OS device tree and triggers a DMP rcconfiguration. You can specify parameters to vxdi sk scandisks to implement partial device discovery. Lesson 2 Managing Devices Within the VxVM Architecture 2-15 Ccpynqht r 2006 Svruantec Corporauon All nqtus reserved
  52. 52. Managing Multiple Paths to Disk Devices The dynamic multipathing (OMP) feature or Vx VM provides greater reliability and performance for your system by enabling path Iailovcr and load balancing. Dynamic Multipathing (DMP) DMP: A method VxVM uses to manage two or more hardware paths to a single drive (?!Il.......,ru••• !!H:], ( Fibre Channelswitche~/ j Dynamic muhipathing is the method that VxVM usesto manage two or more hardware paths directing 1:0 to a single drive. VxVM arbitrarily selects one ofthe two namesand creates a single del ice entry. and then transfers data across both paths to spread the 1:0. V,VM detects multipath systems by using the universal world-wide device identifiers (WWD IDs) and manages mulupaih targcts. such as disk arrays. which define policies for using 1I10rcIIJ:.lnone p.uh. 1= Host I • Mapped ==>by DMP Single OMP Metanode Benefits or DM P include: High availabitity DMP provides greater reliability using a path Iailovcr mechanism. When one connection to a disk is 1051.the system coniinucs to accessthe critical data over the other sound connections 10 the disk until you replace the railed path. Improved performance DMP provides greater I/O throughput by balancing the I/O load uniformly across multiple I/O paths to the disk device. 2-16 IfBcsiOILfBCBil~ rnn1IIIIIIb~~~o;ure LOiskis disk15 or disk27, dependingon the path. What Is Dynamic Multipathing? Benefits of DMP Copvnqnt ;:; 2006 Syrnamec Corpcranon All rights reSf'rved VERITAS Storage Foundation 5.0 for UNIX: Maintenance
  53. 53. 'symanlCC. Types of Multiported Arrays Active/Active I • Active/Passive If I Path2 I Passive I Path +- (Secondary) Pathl Path2 Pathl Active Path (Primary) Active Path Active Path Used for: • Load balancing • Path failover Used for path failover only What Is a Multiported Disk Array? A multiported disk array is an array that can be connected to host systems through multiple paths. The two basic types of multiported disk arrays are: Active/active disk arrays Active/passive disk arrays For each supported array type, V.xVM usesa multipathing policy that is based on the characteristics of the disk array. Active/Active Disk Arrays Active/active disk an'ays permit several paths to be used concurrently for 110.With these arrays, DMP provides greater I/O throughput by balancing the I/O load uniformly across the multiple paths to the disk devices. If one connection to an array is lost, DMP automatically routes I/O over the other available connections to the array. Active/Passive Disk Arrays Active/passive disk arrays permit only one path at a time to be used for 1/0. The path that is used for I/O is called the active path. or primary path. An alternate path, or secondary path. is configured for use in the event that the primary path fails. If the primary path to the array is lost. DMP automatically routes I/O over the secondary path or other available primary paths. Lesson 2 Managing Devices Within the VxVM Architecture 2-17 Copyright ." 2006 Symantec Ccrocranoo 111 nqt-ts reservec
  54. 54. syrnantec Setting 1/0 Policies and Path Attributes To change the 1/0 policy for balancing the 1/0 load across multiple paths to a disk array or enclosure: vxdmpadm setattr enclosure enc name iopolicy=policy • adaptive • balanced • minimumq • priority • round· robin • singleactive To set path attributes for a disk array or enclosure: vxdmpadm setattr path path_name pathtype=type • active • primary • nomanual secondary • nopreferred • standby • preferred Setting the 1/0 Policy for an Enclosure Alter analyzing statistics. you can use the vxdmpadm setat tr command with the iopolicy option to change the 1/0 policy for balancing the 110load across multiple paths to a disk array or enclosure. You can set policies loran enclosure (Iorcxumplc. HDSOI). for all enclosures ola particular type (for example. HDS). or for all enclosures ora particular array type (AlA lor active.active. or Ail' for active/passive). adapti ve automatically determines the paths that have the least delay and schedules I/O Oil paths that arc expected to curry a higher load. balanced takes the track cache into consideration when balancing I/Oacross paths. minimumq sends 110on paths that have the minimum number of 1/0 requests in the queue. This is suitable lor low-end disks or .IBODs where a signi Iicant track cache does not cxist. priori ty assigns the path with the highest load carrying capacity as the priority path. round-robin sets a simple round-robin policy tor 1/0. singleac ti ve channels liO through the single active path. To display the current I/Opolicy: vxdmpadmgetattr enclosure enclosure name iopolicy 2-18 VERITAS Storage Foundation 5.0 for UNIX: Maintenance
  55. 55. Setting Path Attributes You can set the following attributes of the paths to an enclosure or disk array by using the command: vxdmpadmsetattr path path_name pathtype=type active changes a standby path to active. nomanual restores the original primary or secondary attributes of a path. nopreferred restores the normal priority of the path. preferred [priori ty=N] specifies a preferred path and optionally assigns a priority value to it. This indicates a path that is able to carry a higher I/O load. The priority value must be an integer greater than or equal to I. Larger priority values indicate a greater load carrying capacity. ,,"ote: Marking a path as a preferred path does not change its I/O load balancing pol icy, primary assigns a primary path for an Active/Passive disk array. secondary assigns a secondary path for an Active/Passive disk array. standby marks a path as not available for normal I/Oscheduling. This path is only invoked if there are no active paths available for 110. See the 1'ERITAS 1'olume MOl1ager Administrator 's Guide and the vxdmpadm (1m) manual page for more information. Lesson 2 Managing Devices Within the VxVM Architecture Copyr,ght s: 2006 Symantec Corporation All nqhts resr,rved 2-19
  56. 56. Displaying 1/0 Statistics for Paths syrnantec 1. Enable the gathering of statistics: vxdmpadm iostat start [memory=sizeJ 2. Reset the I/O counters to zero: vxdmpadm iostat reset 3. Display the accumulated statistics for all paths: vxdrnpadrn iostat show all cpu usage = 7952us OPERATIONS per cpu memory = 8192b BYTES AVG TIME(ms) PATHNAME cOtOdO c2t1l8dO c3 t1l8dO READSWRITES READSWRITES 1088 557056 0 READS WRITES 0.009542 0.000000 0.001194 0.000000 0.000000 0.000000 87 o 44544 o • The displayed statistics can be filtered by path name, DMP node name, and enclosure name. • You can also specify the number of times to display the statistics and the time interval. Displaying 1/0 Statistics for Paths You can use the vxdmpadmiostat command to gather and display JiO statistics 1'01' a specified DMP node, enclosure, or path. The statistics that arc displayed arc the CPU usage and amount 01' memory pCI' CPU used to accumulate statistics. the number 01' read and write operations, the number ofblocks read and written. and the average time in milliseconds pc:r read and write opcrat ion. The interval and count attributes may be used to specify the interval in seconds between displaying the I/O statistics. and the number of lines to be displayed. The actual interval may be smaller than the value specified if insufficient memory is available to record the statistics. 2-20 VER/TAS Storage Foundation 5.0 for UNIX. Maintenance Cunynqlu '<:U06 Swuarnec Corpcrauon All nqnts reserved
  57. 57. IPreventing DMP for a Device m;-------- If an array cannot support DMP, you can prevent multipathing for the device by using vxdiskadm: ~~~en~ multipathing/s~press d~~ces ~rom VX~~S Vie~ Allow multipathing/Unsuppress devices from VxVM's Vie~ List currently suppressed/non-multipathed devices IWarning: I If you do not prevent DMP for unsupported arrays: Commands like vxdisk list show duplicate sets of disks as ONLINE, even though only one path is used for 1/0. Disk failures can be represented incorrectly. Preventing Multipathing for a Device If you have an array that cannot support the use ofDMP, or if you want to use Sun's Alternate Pathing driver with Vx VM. you can suppress DI'vIP for some or all devices by using the vxdiskadm menu. Suppressing DMP for a device prevents multipathing without removing the DMP layer. It is important for you to suppress DMP for devices that do not support OM!'. If you do not prevent DMP for unsupported arrays: A VxVM command, such as vxdisk list, shows duplicated sets of disks as ONLINE for each path, even though the command is only using one path for 110. Disk failures can be represented or displayed incorrectly by VxVM if DM!' is running with an unsupported, unsuppressed array. To manage the devices that participate ill DMP. you call use vxdiskadm. Lesson 2 Managing Devices Within the VxVM Architecture Copyright '$ 2()06 Syrnantec Corporation 111nonts reserved 2-21
  58. 58. syrnanrec Preventing DMP for a Device When you select the option to prevent multipathing in the vxdiskadm main menu, you have these choices: , I,i. suppre-;;'';ii' paths th;ough a controller from ] . VxVM's view Suppress a path from VxVM's view Suppress disks from VxVM's view by specifying a VID:PID combination Su resa all but one aths to a di.~s~k ~ Prevent multipathing of all disks on a controller l __ .:::by£-V:;X::.VM:.::.. •• _ Prevent multipathing of a disk by VxVM Prevent multipathing of disks by specifying a VID:PID combination List currently suppressed/non-multipathed devices Similar choices exist when you reinclude devices for DMP. Excluding Devices from Multipathing When you select the option to prevent mulupathing in the vxdi skadmmain menu, the Exclude Devices submenu is displayed. Both of the following options send the command vxdmpadmdi sable to the kernel: The option "Suppress all paths through a controller from VxVM's view" coutinucs to allow the I/O to use both paths internally. Aller a reboot, vxdisk 1i s t dues not show the suppressed disks. "Prcvcnt multipathing of all disks on a controller by VxVM" docs not allow the l.O to use intcrnal muliipathiug. The vxdisk list command shows all disks as ONLINE. This option has no effect on arrays that arc not performing dynamic multipathing or that do not support VxVM DM!'. Including Devices for Multipathing Fur previously excluded devices. if you later decide that you want to rcincludc the device in muluparhing. then you select the vxdiskadm option "Allow mulripathing.Unsupprcss dcv iccs lrom VxVM's view." A similar set of options is available in the Include Devices submenu. 2-22 Copvnqht " 20!)6 Syrnantec Corpo-auon All r'lj~lls reserved VERITAS Storage Foundation 5.0 for UNIX.' Maintenance
  59. 59. You can disable 1/0 to a controller to perform maintenance, for example: • To replace a system board • To test path failover Use the following commands: • To disable 1/0 to a particular controller: vxdmpadm disable ctlr=ctlI~name • To disable 1/0 to a particular enclosure: vxdmpadm disable enclosure=enc_name • To reenable 1/0 to a particular controller: vxdmpadm enable ctlr=ctlr_name In VEA: Select Actions->Disable (or Actions->Enable) and completethe associateddialog box. symantcc. Enabling or Disabling 1/0 to a Controller By disabling 110 to a host disk controller. you can prevent DMP from issuing 110 through a specified controller. You can disable I/O to a controller to perform maintenance on disk arrays or controllers attached to the host. For example. when replacing a system board. you can stop all 110 to the disk controllers connected to the board before you detach the board. For active/active disk arrays. when you disable [/0 to one active path, all 1'0 shifts to other active paths. For active/passive disk arrays. when you disable I/O to one active path. all 110 shifts to a secondary path or to an active primary path on another controller. You cannot disable the last enabled path to the root disk or any other disk. On IIP-UX. you can disable the last enabled path to any other disk (without using the -f (force) option). When you disable 1/0 to a controller. disk. or path. you override the DMP restore daemon's ability to reset the path to ENABLED. When you enable I/O to a controller: For active/active disk arrays. the controller is used again tor load balancing. For active/passive disk arrays. the operation results in tailback of I/O to the primary path. Enablingor Disablinga Controller: VEA To disable or enable a controller in YEA. select the controller. select Actions >Disablc or Actions >Enable. and complete the associated dialog box. Copvnont 'f 20D6 Svmamer- Corporation All nqhts reservec 2-23Lesson 2 Managing Devices Within the VxVM Architecture
  60. 60. The [)MP restore daemon is an internal process that monitors DMP paths and automatically enables paths that were previously disabled due to hardware failures after the paths arc back online. Controlling the Restore Daemon The DMP restore daemon is an internal process that monitors DMP paths. To check its status: vxdmpadm stat restored The number of daemons running: 1 The interval of daemon: 300 The policy of daemon: check_disabled To start the [)MP restore daemon, you use the start restore option in the vxdmpadm command. vxdmpadm start restore [interval=intervall [policy=check_disabledlcheck_alll The restore daemon unalyzcs the health olpaths every interval seconds. The: default interval is 300 seconds. Decreasing the interval can adversely affect performance. You can specify one oftwo types olpolicics: If the policy is check_disabled, the restore daemon checks the health of paths that were previously disabled due to hardware failures and revives them if they arc back online. If the policy is check_all, the restore daemon analyzes all paths in the system, revives the paths that arc back online, and disables the paths that arc inaccessible. interval: Frequency of analysis (default: 300seconds) check disabled: Only checks disabled paths (default) To change daemon properties: • Stop the DMP restore daemon: vxdmpadm stop restore • Restart the daemon with new attributes: vxdmpadm start restore interval=400 policy=check_all I check all: All paths are checked. Controlllnq Automatic Restore Processes Starting the DMP Restore Daemon The default policy is check_disabled. 2-24 VERITAS Storage Foundation 5.0 for UNIX. Maintenance Copyr.qnt 2006 Svmantec Corpoeanon All nqhts reservec
  61. 61. symantcc. Lesson Summary • Key Points This lesson described components in the VxVM architecture and the device discovery process and described how to administer dynamic multipathing. • Reference Materials - VERITAS Volume Manager Administrator's Guide - VERITAS Storage Foundation Release Notes - VERITAS Volume Manager Hardware Notes svmantcc Labs and solutions for this lesson are located on the following pages: Appendix A provides complete lab instructions. 'Tab 2: r..LlIu~ing Dc ices 11I:il1 the Y ' I :rL'ililccrllll'," 1',1ic' ,_1,1 Appendix B provides complete lab instructions and solutions, "I ah :: '.;,'!u!iU!h: M:llll1:2i!l)1 Dcvic •.> Withl11 the ' '1 .vnhirccturc.' pagl 11·: I Lab 2 Lab 2: Managing Devices Within the VxVM Architecture In this lab, you explore the VxVM tools used to manage the device discovery layer (DOL) and dynamic multipathing (DMP). The objective of this exercise is to make you familiar with the commands used to administer multipathed disks. For Lab Exercises, see Appendix A. For Lab Solutions, see Appendix B. Lesson 2 Managing Devices Within the VxVM Architecture 2-25
  62. 62. 2-26 VERITAS Storage Foundation 5.0 for UNIX: Maintenance CllJlYfI!-lot!' 2006 Symamec Cor{lor;;llul"' All fights reserved
  63. 63. Lesson 3 Encapsulation and Rootability
  64. 64. 'symankc. Lesson Introduction • Lesson 1;Maintaining Data Consistency • Lesson 2: Managing Devices Within the VxVM Architecture • ~~~~~~i~~yEnCapsulati::_a.:~~ ._~ • Lesson 4: Troubleshooting the Boot Process • Lesson 5: Volume Maintenance • Lesson 6: Performance Monitoring • Lesson 7: Point-in-Time Copies • Lesson 8: Other Enterprise Features Overview symantec. Lesson Topics and Objectives Topic After completing this lesson, you will be able to: Topic 1: Placing the Boot Place the boot disk under VxVM Disk Under VxVM Control control. .. Topic 2: Creating an Create an alternate boot disk by Alternate Boot Disk mirroring the boot disk that is under VxVM control. Topic 3: Removing the Remove the boot disk from VxVM Boot Disk from VxVM control. Control 3-2 VERITAS Storage Foundation 5.0 for UNIX Maintenance
  65. 65. Placing Disks with Data Under VxVM Control • On Solaris, encapsulation is the process of converting partitions into volumes to bring those partitions under VxVM control. • On HP-UX, conversion is the process of enabling LVM physical volumes to be used byVxVM. SOlarisl HP-UXI Placing the Boot Disk Under VxVM Control What Is Encapsulation or Conversion? symantcc. On Solaris. encapsulationis the process of converting partitions into volumes to bring those partitions under VxVM control. On IIP-UX, conversionis the process of enabling LVM physical volumes to be used by Vx VM, A Her a disk has been encapsulated or converted. the disk is handled like an initialized disk, Placing Disks with Data Under VxVM Control: Solaris Encapsulation is the process of converting partitions into volumes to bring those partitions under Vx Vlvl control. For example. if a system has three partitions on the disk drive and you encapsulate the disk to bring it under VxVM control. there will be three volumes ill the disk group, Solaris Encapsulation Requirements Disk encapsulation cannot occur unless these requirements arc met: Partition table entries must be available on the disk for the public and private regions, During encapsulation. you are prompted to select the disk layout. If you choose a CDS disk layout. then only one partition is needed. However, if encapsulation as a CDS disk fails. you can specify that a sliced layout be used instead, in which case you will need two free partitions, The disk must contain an s2 slice that represents the full disk (The s2 slice cannot contain a tile system), 2048 sectors of unpartitioned free space. rounded up to the nearest cylinder boundary. must be available. either at the beginning or at the end of the disk, Lesson 3 Encapsulation and Rootability COPYright ~~ 2006 Syrnanter- Corporaucn All nqms reserved 3-3
  66. 66. Encapsulated data disk ,---._,_ Private region -......--~1~-·-;1----+I homev~ engvol eng --- '----- acct -----_'[~ jBW~--.---~~dist ~~ Solaris: Encapsulating a Data Disk vxdiskadm: "Encapsulate one or more disks" Follow the prompts by specifying: Name of the device to add Name of the disk group to which the disk will be added vxencap: /etc/vx/bin/vxencap -g diskgroup access name Run the script: /etc/init.d/vxvm-reconfig access name Solaris: Reversing the Encapsulation Process Limitations: There arc no tools to help with uncncapsulation. (The system disk is an exception to this.) Encapsulation should not be attempted if: Volume layouts have been altered in any way (for example, hot rclocauon ). Volumes have mirrors. The disk has been used for parts of other volumes. The partition table before encapsulation is stored in: /etc/vx/reconfig.d/disk.d/device/vtoc Follow this procedure to encapsulate: a Stop applications. b Remove volumes on the disk and take the disk out of VxVM control. c Re-create thc partition table as provided ill the stored vtoc tile. d Manually modify /etc/vfstab (itnecessary). e Reboot or manually mount the partitions and start applications. 3-4 VERITAS Storage Foundation 5,0 for UNIX: Maintenance CIJPynghl;: 2006 Svmautec Corrorauou All nqtus reservecr
  67. 67. Placing Disks with Data Under VxVM Control: HP-UX Conversion is the process of enabling LVM physical volumes to be used by VxVM. You can convert: Unused physical volumes Physical volumes in volume groups Unused Physical Volume ~.•..... - ...'..•... '. U[VXVM Disk Group Ii,y .-k_·4JWJi8L'!£&;;.'*~#Jiik" ~. ' LVM Physical Volumes VxVM Disks HP-UX: Limitations of LVM Conversion LVM configurations that you cannot convert to VxVM include: A volume group with insufficient space for mctadata A volume group containing the root volume A volume group containing the lusr file system A volume group with any dump or primary swap volumes A volume group disk used in ScrviceGuard clusters A volume group with any disks that have bad blocks HP-UX: Converting Unused Physical Volumes 1 View group membership information to ensure that then: is no data on the LVM disk: pvdisplay disk_name pvdisplay /dev/dsk/c4tldO 2 Remove LVM disk information: pvremove disk_name pvremove /dev/dsk/c4tldO 3 Initialize the disk for VxVM use, Lesson 3 Encapsulationand Rootability 3-5 Copyright? 2006 Svrnantar Corporation All nqtus reserved
  68. 68. UP-LJX: Cunverting Volume Groups vxvmconvert Volulne Manager SUPPOI"( Operations Menu: Volume Manager/LvM_Conversion list listvg Allalyze LVM Volume Groups for Conversion convert LVM Voillme G~oups to VxVM Roll back tram VxVM to LVM List disk informatioIl List LVM Volume Group inforntation ?? Display help about menu Display help about the menuing system Ex i t f iorn menusq Select an operation to perfOrl": UP-LJX: Conversion PrOC('SS(LV'I to VxV'I) 1 Identify volume groups. 2 Analyze volume groups. 3 l3ack lip LV M data. 4 Plan for new names. 5 Stup applications. 6 Unrnount the file system. 7 Convert volume groups. 8 Make name changes. 9 Restart applications. 10 Customize the configuration. IIP-LJX: Restoring LVi! Volume Group Cunfiguration Roll Back Using vxvmconvert Full LVM Restoration fur the Volume Group LVM Volume Group VxVM Disk Group LVM Physical Volumes VxVM Disks 3-6 VERITAS Storage Foundation 5.0 for UNIX: Maintenance CIlPVflljlllL 2006 Syruantec Corpnratrcu All r.ghts reserved
  69. 69. HP-UX: Roll Back Using vxvmconvert vxvmconvert Option 3, roll hack from ',',1 to 1.',1 Select Volume Group(s) to rollback: [<pattern-list>,all,list,listvg,q,?) vg08 Roll back this Volume Group? [y,n,q,?) (default: y) Rolling back LVM configuration records for Volume Group vg08 Selected Volume Groups have been restored. Hit RETURN to continue. Rollback other LVM Volume Groups? [y,n,q,?) (default: n) UP-lIX: Full L'll Restoration for the Volume Group To restore LVM internal data: mkdir /dev/vol group_name mknod /dev/vol_group_name/group c 64 oxoeoooo vxdg destroy diskgroup For each disk in the volume group: vxdiskunsetup disk_name vgcfgrestore -F -f pathname/filename raw_device_name vgimport -s -m pathname/mapfilename vol grp_name raw device name vgchange -a y vol group_name To restore user or application data: mount -F fstype /dev/vol group_name/lvname /mount po~nt frecover -r -f /dev/rmt/cOtOdOBEST ------------------------------------------------------ Lesson 3 Encapsulation and Rootability Copyrlgt1~, 2()06 Syrn"lnl€r Corporation 111nor.ts reserved 3-7
  70. 70. What Is Rootability? Rootability is the process of placing the root file system, swap device, and other file systems on the boot disk under VxVM control. On Solaris, you encapsulate the system disk. Encapsulated boot disk II Partitions are mapped to subdisks that are used to create the volumes ¥ that overlay the original partitions. W:---""Sii1l Private region 1Ir---"':!III / ------< 1:? -?i1/usr----------~====~-'------ ~-----~;::;~ ~ Note: Only sliced layout is allowed. What Is Rootability? Rooiability is the process ofplncing the root file system. swap device, and other file systems on the boot disk under VxVM control. Solaris On Solaris, VxVM converts existing partitions ofthe boot disk into VxVM volumes. The system can then mount the standard boot disk file systems (that is. /, /usr. ami so 011) from volumes instead of disk partitions. Boot disk encapsulation has the same requirements as data disk encapsulation, but requires two Ircc partitions (for the public and private regions). When encapsulating the boot disk. you can create the private region from tile swap area, which reduces the swap area by the size otthc private region. The private region is created at the beginning of the swap area. and the swap partition begins one cylinder from its original location. When creating new boot disks, you should start the partitions on the new hoot disks on the next cylinder beyond the 2048 default used tor the private region. ur-ux 011 HP-UX. rootability is carried out by creating a coJlY of the system disk on another VxVM disk. Copvnqhr 'C' 20U6 Svmantec Co-poianon 111nqnts reserved VERITAS Storage Foundation 5.0 for UNIX: Maintenance3-8
  71. 71. symantec. Why Put the Boot Disk Under VxVM Control? On Solaris: • You should encapsulate the boot disk only if you plan to mirror the boot disk. • Benefits of mirroring the boot disk: -Enables high availability -Fixes bad blocks automatically (for reads) -Improves performance • There is no benefit to boot disk encapsulation for its own sake. You should not encapsulate the boot disk if you do not plan to mirror the boot disk. On HP-UX: Single storage virtualization tool for system disk and data disks. It is highly recommended that you encapsulate and mirror the boot disk. Some of the benefits of encapsulating and mirroring root include: High availability Encapsulating and mirroring root sets up a high availability environment for the boot disk. Ifthe boot disk is lost. the system continues to operate on the mirror disk. Bad block revecturing If the boot disk has bad blocks. then YxYM reads the block from the other disk and copies it back to the bad block to fix it. SCSI drives automatically fix bad blocks on writes, which is called bad Mock rvvcctoring. Improved performance By adding additional mirrors with different volume layouts. you can achieve better performance. Mirroring alone can also improve performance if the root volumes arc performing more reads than writes, which is the case on many systems. Why Put the Boot Disk Under VxVM Control? When Not to Encapsulate Root If you do not plan to mirror root. then you should not encapsulate it. Encapsulation adds a level of complexity to system administration, which increases the complexity of upgrading the operating system. Lesson 3 Encapsulation and Rootability Copyngm t; :'001'; Syruamec Corpoeauon 111r.qhrs reserved 3-9
  72. 72. Limitations of the VxVM Boot Disk • Placing the boot disk under VxVM control adds steps to OS upgrades on Solaris platform. • A system cannot boot from a boot disk that spans multiple devices. • Never grow or change the layout of boot disk volumes. These volumes map to a physical underlying partition on disk and must be contiguous. HP-UX Note: HP-UX supports OS installations on a VxVM disk. However, the version of VxVM that is installed from the HP-UX installation media for 11iv2 is 3.5. Limitations of the VxVM Boot Disk A system cannot boot from a boot disk that spans multiple devices. You should never expand or change the layout of boot volumes, No volume associated with an encapsulated boot disk (r oot.vo l , usr. var. opt. swapvoL, and so on) should be expanded or shrunk. because these volumes map to a physical underlying partition on the disk and must be conuguous, If you attempt to expand these volumes. the system can become unbootublc ifit becomes necessary to reven back to slices in order to boot the system. Expanding these volumes can also prevent a successful OS upgrade. and a fresh install can be required. Solaris Additionally. the upgrade_start script (USedin upgrading VxVM to a new version) may Jail. Note: You can add a mirror ofa dillcrcut layout, but the mirror is not bootablc. 3-10 VERITAS Storage Foundation 5.0 for UNIX' Maintenance Copyright! 2006 Symantec Corporation All uqhts reserved
  73. 73. symantec.Th -File System Requirements: Solaris Only For root, usr, var, and opt volumes: • Use UFS file systems. (VxFS is not available until later in the boot process.) • Use contiguous disk space. (Volumes cannot use striped, RAID-5, concatenated mirrored, or striped mirrored layouts.) • Do not use dirty region logging on the system volumes. (You can use DRl for the opt and var volumes.) For swap volumes: • The first swap volume must be contiguous, and, therefore, cannot use striped or layered layouts. • Other swap volumes can be noncontiguous and can use any layout. However, there is an implied 2-GB limit of usable swap space per device for 32-bit operating systems. HP.UXI File System Requirements for Root Volumes: Solaris Only To boot from volumes. follow these requirements and recommendations for the file systems on root volumes: For the root. usr. var. and opt volumes: Use lJFS file systems: You must use UFS file systems for these volumes. because the VERITAS File System (VxFS) package is not available until later in the boot process when the scripts run in /etc/rc2.d (multiuser mode). Use contiguous disk space: These volumes must be located in a contiguous area on disk. as required by the OS. For this reason. these volumes cannot use striped. RAID-S. concatenated mirrored. or striped mirrored layouts. Do not use dirty region logging for root or uar: You cannot use dirty region logging (DRL) on the root and usr volumes. If you attempt to add a dirty region log to the root and usr volumes. you receive an error. Note: The opt and var volumes call use dirty region logging. Swap Space Considerations: If you have swap defined, then it needs to be contiguous disk space. The first swap volume (as listed in the /etc/vEstab tile) must be contiguous and. therefore. cannot use striped or layered layouts. Additional swap volumes can be noncontiguous and can use any layout. Note: You can add noncontiguous swap space through VxVM. However, Solaris automatically uses swap devices in a round-robin method. which may reduce expected performance benefits of adding striped swap volumes. For 32-bit operating systems. usable space per swap device is limited to 2 GB. For M-bit operating systems. this limit is much higher (up to :zi,,l - I bytes). lesson 3 Encapsulation and Rootability Copynqtu T: 2(J06 Symaruec Cornoranon. 1(1 ••ghlf> reserved 3-11
  74. 74. Volume Requirements: HP·UX All volumes on the root disk must be in thc disk group that you choose to be the bootdg disk group. The names otthc volumes with entries in the LIF LABEL record must be standvol, rootvol, swapvol, and dumpvol (ifpresent). The names ofthe volumes for other file systems on the root disk arc generated by appending vol to the name of their mount point under /. Any volume with an entry in the LI F LABEL record must be contiguous. Thc volume can have only one subdisk. and it cannot span to another disk. The root vol and swapvol volumes must have the special volume usage types, root and swap respectively. Only the disk access types auto with hpdisk and simple formats arc suitable 1(lJ" use as Vx VM root disks, root disk mirrors, or as hot-relocation spares for such disks. The volumes on the root disk cannot use dirty region logging (DRL). 3-12 VERITAS Storage Foundation 5.0 for UNIX: Mamtenance Copyright 2006 Svmautec Corroranon 111nylliS reserve-s
  75. 75. Before Placing the Boot Disk Under VxVM Control Plan your rootability configuration: Plan your rootabjlltv conflguration. bootdg is a system-wide reserved disk group name that is an alias for the disk group which contains the volumes that arc used to boot the system. When you place the boot disk tinder VxVM control. VxVM sets bootdg to the appropriate disk group. You should never attempt to change the assigned value nfbootdg: doing so may render your system unbooiable. An example configuration is to place the boot disk into a disk group named sysdg. and add at least two more disks to the disk group: one for a boot disk mirror and one as a spare disk. VxVM then sets bootdg to sysdg. Solaris Encapsulated Boot disk boot disk mirror Spare disks Enable boot disk aliases. On Solaris. before encapsulating your boot disk. set the EEPROM variable use -nvramrc? to true. This enables VxVM to take advantage of boot disk aliases to identify the mirror of the boot disk if a replacement is needed. If this variable is set to false. you must determine which disks are bootable yourself On Solaris, set this variable to true as follows: On Solaris: Enable boot disk aliases: eeprom "use-nvramrc?=true" • Record the layout of the partitions on the unencapsulated boot disk to save for future use. On HP-UX: Set the primary or alternate boot path from the boot menu or from the command line: Main Menu: Enter command or menu: co pa alt path setboot -p primary_path -a alternace __path Save the LVM volume group configuration for vgOO. Before Placing the Boot Disk Under VxVM Control eeprom "use-nvramrc?=true" Save the layout of partitions before you encapsulate the boot disk. For example. on Solaris. you can use the prtvtoc command to record the layout of the partitions on the unencapsulated boot disk (/ dev / rdsk/ cO t OdOs2 in this example): prtvtoc /dev/rdsk/cOtOdOs2 Record the output from this command fur future reference. Lesson 3 Encapsulation and Rootability Copynqru e. 2006 Syrnantac. Corporanon All nqhts reserved 3-13

×