• Save
ASM
Upcoming SlideShare
Loading in...5
×
 

ASM

on

  • 2,001 views

Automatic Storage Management

Automatic Storage Management

Statistics

Views

Total Views
2,001
Views on SlideShare
2,000
Embed Views
1

Actions

Likes
2
Downloads
0
Comments
1

1 Embed 1

http://www.docshut.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Could you send me this presentation to my email id pls : diptimayeebhadra@gmail.com
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

ASM ASM Presentation Transcript

  • Automatic Storage Management
  • Objectives
    • Understand how Oracle database files are stored in ASM
    • Calculate how long ASM rebalance operations will take
  • Agenda
    • ASM Instances
    • ASM Disk Groups
      • Metadata
      • Extent Distribution
      • Rebalancing
      • Redundancy
  • ASM Instances
  • ASM Single Instance Architecture OCSSD Daemon Only Dedicated Storage Oracle Clusterware ASM Instance RDBMS Instance Server
  • ASM Single Instance Background Processes Fixed Area Variable Area ASM Cache Oracle 11.1 PMON SMON CKPT LGWR DBW0 VKTM PSP0 MMAN RBAL GMON X000 DIAG DIA0
  • ASM RAC Architecture Storage Network Private Network Public Network Shared Storage Oracle Clusterware ASM Instance RDBMS Instance Node 1 Oracle Clusterware ASM Instance RDBMS Instance Node 2 Oracle Clusterware ASM Instance RDBMS Instance Node 3 Oracle Clusterware ASM Instance RDBMS Instance Node 4
  • ASM RAC Architecture PROD1 TEST1 CLUSTERWARE +ASM1 PROD2 TEST2 CLUSTERWARE +ASM2 PROD TEST Clusterware ASM Instances RDBMS Instances Database Files
  • ASM RAC Instance Background Processes Fixed Area Variable Area ASM Cache Oracle 11.1 PMON SMON CKPT LGWR DBW0 VKTM PSP0 MMAN RBAL GMON X000 DIAG DIA0 LMON LMS0 LMD0 LCK0 DIAG MARK KATE
  • ASM Disk Groups
  • ASM Disk Groups and Disks Disk Group 1 Disk Group 2 Disk Group 3 Disk 1 Disk 4 Disk 2 Disk 6 Disk 5 Disk 7 Disk 3
  • ASM Disk Groups and Disks Disk Group 1 Disk 1 Disk 4 Disk Group 2 Disk 2 Disk 5 Disk 6 Disk 7 Disk 3 Disk Group 3
  • ASM Disk Groups, Disks and Database Files Disk Group 1 Disk 1 Disk 4 Disk Group 2 Disk 2 Disk 5 Disk 6 Disk 7 Disk 3 Disk Group 3 File 1 File 2 File 3 File 4 File 5 File 5 File 6
  • File Extents versus Allocation Units
    • File Extent
      • Logical unit of ASM file
      • Map to allocation units
        • One to many mapping
    • Allocation Unit
      • Physical unit of ASM disk
      • Oracle 10.2 and below
        • Always 1MB
        • Can be increased using _asm_ausize
      • Oracle 11.1 and above
        • Variable size
        • 1MB, 2MB ,4MB, 8MB, 16MB, 32MB, 64MB
  • X$KFFXP
    • Maps file extents to allocation units
    • Only populated in ASM instance
    • Columns include
    Column Name Description GROUP_KFFXP Disk Group Number NUMBER_KFFXP File Number COMPOUND_KFFXP Disk Group Number || File Number INCARN_KFFXP Incarnation Number PXN_KFFXP Physical Extent Number (within file) XNUM_KFFXP Logical Extent Number (within file) LXN_KFFXP 0=primary, 1=first mirror, 2=second mirror DISK_KFFXP Disk Number AU_KFFXP Allocation Unit Number (within disk) SIZE_KFFXP Size (# allocation units)
  • ASM Metadata
    • Metadata is stored in first 256 files in ASM disk group
      • Space is initially allocated when disk group is created
      • Can be subsequently extended
    • Metadata allocation units are divided into blocks
      • Each block is 4096 bytes
      • Block size specifed using _asm_blksize
    • Metadata files include
    File# Description 0 Metadata Header 1 File Directory 2 Disk Directory 3 Active Change Directory 4 Continuing Operations Directory 5 Template Directory 6 Alias Directory 9 Attribute directory (optional) 12 Staleness registry (optional)
  • ASM Metadata Active Change Directory Continuing Operations Directory Template Directory Alias Directory Disk Directory File Directory Partner Status Table Metadata Header Free Space Table Allocation Table Disk Header Metadata Header
  • ASM Metadata
    • Initial Allocation (Single Instance)
    • Active Change Directory
      • Records changes to metadata
      • Used during recovery of instance or operation failures
    • Continuing Operations Directory
      • Maintains state of active operations
    File# AU Description # AUs 0 0 Disk Header, Free Space Table, Allocation Table 1 1 Partner Status Table 1 1 File Directory 1 2 Disk Directory 1 3 Active Change Directory 42 4 Continuing Operations Directory 2 5 Template Directory 1 6 Alias Directory 1
  • ASM Metadata Block Types Type Description 1 KFBTYP_DISKHEAD 2 KFBTYP_FREESPC 3 KFBTYP_ALLOCTBL 4 KFBTYP_FILEDIR 5 KFBTYP_LISTHEAD 6 KFBTYP_DISKDIR 7 KFBTYP_ACDC 8 KFBTYP_CHNGDIR 9 KFBTYP_CODBGO 10 KFBTYP_TMPLTDIR 11 KFBTYP_ALIASDIR 12 KFBTYP_INDIRECT Type Description 13 KFBTYP_PST_NONE 14 KFBTYP_HASHNODE 15 KFBTYP_COD_RBO 16 KFBTYP_COD_DATA 17 KFBTYP_PST_META 18 KFBTYP_PST_DTA 19 KFBTYP_HBEAT 20 KFBTYP_SR 21 KFBTYP_STALEDIR 22 KFBTYP_VOLUMEDIR 23 KFBTYP_ATTRDIR
  • KFED Utility
    • In Oracle 10.2 and above the kfed utility can be used to inspect and edit the contents of ASM blocks
    [ oracle@server3 ~]$ $ORACLE_HOME/bin/kfed -h as/mlib ASM Library [asmlib='lib'] aun/um AU number to examine or update [AUNUM=number] aus/z Allocation Unit size in bytes [AUSZ=number] blkn/um Block number to examine or update [BLKNUM=number] blks/z Metadata block size in bytes [BLKSZ=number] ch/ksum Update checksum before each write [CHKSUM=YES/NO] cn/t Count of AUs to process [CNT=number] d/ev ASM device to examine or update [DEV=string] o/p KFED operation type [OP=READ/WRITE/MERGE/NEW/FORM/FIND/STRUCT] p/rovnm Name for provisioning purposes [PROVNM=string] s/eek AU number to seek to [SEEK=number] te/xt File name for translated block text [TEXT=string] ty/pe ASM metadata block type number [TYPE=number]
    • This utility should only be used under the guidance of Oracle Support
  • KFED Utility
    • For example to dump blocks in aliases directory in DISKGROUP1
    • Find group number
    SELECT disk_kffxp, au_kffxp FROM x$kffxp WHERE group_kffxp = 1 AND number_kffxp = 6 AND lxn_kffxp = 0; Disk Allocation Unit 0 49
    • Find disk name
    SELECT group_number FROM v$asm_diskgroup WHERE name = 'DISKGROUP1';
    • Alias directory is stored in file number 6
    SELECT path FROM v$asm_disk WHERE group_number = 1 AND disk_number = 0; Path /dev/oracleasm/disks/VOL1
  • KFED Utility
    • Example (continued)
      • Allocation unit is 1MB
      • Block size is 4096
      • Therefore there are 256 blocks per allocation unit
      • Starting block offset = 256 * 49 = 12544
    for (( f = 12544 ; f < 12544 + 256 ; f++ )) do kfed op=read blkn=$f dev='/dev/oracleasm/disks/VOL1' > blk${f} done
  • Extent Distribution
  • Extent Distribution
    • Creating a disk group:
    CREATE DISKGROUP diskgroup1 EXTERNAL REDUNDANCY DISK '/dev/oracleasm/disks/VOL1';
    • Dropping a disk group:
    DROP DISKGROUP diskgroup1 INCLUDING CONTENTS
  • Extent Distribution 1 disk Disk 0 Metadata Metadata Data 0 1 2 3 4 5 6 7
  • Extent Distribution 2 disks Disk 0 Disk 1 Metadata Metadata Data 0 2 4 6 8 10 12 14 1 3 5 7 9 11 13 15
  • Extent Distribution 4 disks Disk 0 Disk 1 Disk 2 Disk 3 Metadata Metadata Data 0 4 8 12 16 20 24 28 2 6 10 14 18 22 26 30 1 5 9 13 17 21 25 29 3 7 11 15 19 23 27 31
  • Extent Distribution 1 large disk - 1 small disk Disk 0 Disk 1 Metadata Metadata Data 0 1 3 4 6 7 9 10 2 5 8 11
  • Extent Distribution 1 large disk - 3 small disks Disk 1 Disk 2 Metadata Disk 0 Disk 4 Metadata Data 1 6 11 16 0 3 5 8 10 13 15 18 2 7 12 17 4 9 14 19
  • Extent Distribution 2 large disks - 2 small disks Disk 1 Disk 2 Metadata Disk 0 Disk 4 Metadata Data 3 9 15 21 0 2 6 8 12 14 18 20 1 4 7 10 5 11 17 23 13 16 19 22
  • Rebalancing
  • Extent Distribution
    • Adding a disk:
    ALTER DISKGROUP diskgroup1 ADD DISK '/dev/oracleasm/disks/VOL2' REBALANCE POWER 0;
    • Dropping a disk:
    ALTER DISKGROUP diskgroup1 DROP DISK 'DISKGROUP1_0002' REBALANCE POWER 0;
    • Rebalancing a disk group:
    ALTER DISKGROUP diskgroup1 REBALANCE POWER 1;
  • Rebalancing Adding disks - 1 disks to 2 disks Disk 0 Metadata Data Disk 1 0 1 2 3 4 5 6 7 5 7 0 1 2 3 4 5 6 7 5 7 1 3 1 3 STOP
  • Rebalancing Adding disks - 1 disks to 4 disks Disk 0 Disk 1 Disk 2 Disk 3 Metadata Data 0 1 2 3 4 5 6 7 2 6 1 5 0 4 0 1 2 3 4 5 6 7 1 0 4 5 2 6 STOP
  • Rebalancing Adding disks - 2 disks to 3 disks Disk 0 Disk 1 Disk 2 Metadata Data 1 3 5 7 9 11 13 15 0 2 4 6 8 10 12 14 2 5 8 11 0 1 2 3 4 5 0 3 1 4 14 17 16 17
  • Rebalancing Adding disks - 2 disks to 4 disks Disk 0 Disk 1 Disk 2 Disk 3 Metadata Data 1 3 5 7 9 11 13 15 0 2 4 6 8 10 12 14 1 5 9 13 0 4 8 12 0 1 2 3 4 5 6 7 1 0 4 5 2 6 STOP
  • Rebalancing Dropping disks - 3 disks to 1 disk Disk 0 Disk 1 Disk 2 Metadata Data 0 3 6 1 2 4 5 7 4 5 0 3 1 2 4 5 2 5 1 4 8 7 8 1 2 STOP
  • Rebalancing Moving disks - 2 disks to 2 disks Disk 0 Disk 1 Disk 2 Metadata Data 1 3 5 7 9 11 13 15 4 6 8 10 12 14 10 6 2 0 4 8 0 1 2 3 4 5 6 7 0 4 2 6 STOP 2 0 12 14
  • Rebalancing V$ASM_OPERATION
    • Contains details of ongoing rebalance operations
    Estimate of remaining time Column Name Data Type GROUP_NUMBER NUMBER OPERATION CHAR(5) STATE VARCHAR2(4) POWER NUMBER ACTUAL NUMBER SOFAR NUMBER EST_WORK NUMBER EST_RATE NUMBER EST_MINUTE NUMBER ERROR_CODE VARCHAR2(44)
  • Rebalancing Power Limit
    • Power limit can be 0 to 11
      • 0 disables rebalance operation
      • 1 to 11 specifies number of ARBn background processes used for rebalance
    • In Oracle 10.2
      • RBAL manages rebalance operation
      • Each ARBn background process is allocated a range of 128 allocation units to rebalance
        • When complete another range is requested
      • AD lock is taken while an allocation unit is being rebalanced
      • Rebalance operations take much longer than theoretically necessary. Possible reasons include:
        • Locking
        • GES updates with other ASM instances
        • Updates to RDBMS instance
  • Rebalancing Summary
    • EST_MINUTES column of V$ASM_OPERATION is reasonably accurate
      • Allow a few minutes for SAN cache to stabilize
      • Check regularly for changes to estimate
    • ASM rebalance operations do not affect workload
      • Locks are only taken briefly
        • Lock mechanism has changed in Oracle 11.1
      • SAN cache and I/O performance will be affected
    • In Oracle 10.2 rebalancing is fastest if
      • Other ASM instances are shutdown
      • RDBMS instance is shutdown
    • Estimated completion time will be affected by:
      • Use of SAN cache and I/O by rest of workload
      • Rate of change by applications to blocks in ASM files being rebalanced
  • Redundancy
  • Redundancy
    • ASM Supports three levels of redundancy
      • External Redundancy
        • Implemented externally using storage layer
        • Most common configuration in production
      • Normal Redundancy
        • Two copies of each extent maintained in separate failure groups
        • Used with extended clusters
        • Used occasionally in production e.g. CERN
        • Increases CPU overhead on servers
      • High Redundancy
        • Three copies of each extent maintained in separate failure groups
        • Very rare in production
  • ASM Failure Groups - External Redundancy Disk Group Disk 1 Disk 2 Disk 3
  • ASM Failure Groups - Normal Redundancy Failure Group 1 FailureGroup 2 Disk Group Disk 1 Disk 2 Disk 3 Disk 4 Disk 5 Disk 6
  • ASM Failure Groups - High Redundancy Failure Group 1 Disk Group Failure Group 2 Failure Group 3 Disk 1 Disk 2 Disk 1 Disk 2 Disk 1 Disk 2
  • Normal Redundancy 1 Disk Per Failure Group Disk 0 Disk 1 Primary Secondary Metadata Data Primary Secondary 0 1 0 1 2 3 2 3 10 11 Failure Group 1 8 9 6 7 0 1 2 3 4 5 9 10 7 8 5 6 0 1 2 3 4 11 Failure Group 2
  • Normal Redundancy 2 Disks per Failure Group Disk 0 Disk 1 Disk 2 Disk 3 Failure Group 1 Failure Group 2 Primary Secondary Metadata Data Primary Secondary 0 3 0 1 2 3 1 2 4 7 5 6 4 7 5 6 20 23 16 19 12 15 0 3 4 7 8 11 18 21 14 17 10 13 1 2 5 6 9 22 19 20 15 16 11 12 0 3 4 7 8 23 21 22 17 18 13 14 1 2 5 6 9 10
  • High Redundancy 1 Disk per Failure Group Disk 0 Disk 1 Disk 2 Failure Group 1 Primary Secondary Tertiary Metadata Data Primary Secondary Tertiary 0 0 0 0 Failure Group 2 Failure Group 3 0 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 0 1 1 1 2 2 2 3 3 3 4 4 4 5 5 5 6 6 6 7 7 7 8 8 8 9 9 9
  • References
    • Oracle Automatic Storage Management (Oracle Press)
      • Nitin Vengurlekar
      • Murali Vallath
      • Rich Long
    • What ASM and ZFS Can Do For You
      • Jason Arneil - Nominet
    • A Closer Look Inside Oracle ASM
      • Luca Canali - CERN
    • Implementing ASM Without HW Raid
      • Luca Canali - CERN
  • Thanks !