Oracle ASM 11g
The Evolution
Alex Gorbachev
RMOUG Training Days 2010
Denver, 18-Feb-10
Alex Gorbachev

    • CTO, The Pythian Group
    • Blogger

    • OakTable Network member
    • Oracle ACE Director

    •...
Why Companies Trust Pythian
    • Recognized Leader:
    •   Global industry-leader in remote database administration serv...
ASM adoption poll

    • How    many use ASM?
     •   Plan to use?
     •   Plan NOT to use?


    • Adoption     rates a...
Pre-ASM - simple file system on JBOD

                       • Resilience?
                               •   multiplexing
...
Pre-ASM - raw devices




    Server
    /dev/sdb   /dev/sdc          • One file => disk or partition
                     ...
Pre-ASM - raw devices (cluster)



Server 1                          • One file => disk or partition
                      ...
Pre-ASM - volume manager

      • FS   or “raw device”

      • Logical   volume

      • Volume     groups

      • Physi...
Pre-ASM - volume manager

        • FS   or “raw device”

        • Logical   volume

        • Volume     groups

       ...
Pre-ASM - cluster VM and FS

    • Cluster       Volume Manager (CVM)
     •   Synchronizes volume metadata
     •   Still...
Pre-ASM storage issues




10           © 2009/2010 Pythian
Pre-ASM storage issues



     • Too complex
     • 3rd party vendors
      •   Additional licensing cost
     • Many     ...
Pre-ASM storage issues



     • Too complex                                      • Too simple
     • 3rd party vendors   ...
ASM introduction

     • Simplify
      •   Fewer vendors involved
      •   Fewer layers
      •   Move many operations u...
ASM principles

     • Organizing     Oracle files on physical disks

     • Mirroring
      •   reliability & failure tol...
ASM striping 10g




13                      © 2009/2010 Pythian
ASM striping 10g




13                      © 2009/2010 Pythian
ASM striping 10g




13                      © 2009/2010 Pythian
ASM striping 10g




13                      © 2009/2010 Pythian
ASM striping 10g




13                      © 2009/2010 Pythian
ASM striping 10g




13                      © 2009/2010 Pythian
ASM striping 10g




          file1




13                      © 2009/2010 Pythian
ASM striping 10g




          file1    file2




13                       © 2009/2010 Pythian
ASM striping 10g

     • Allocation    unit (AU) 1MB
      •   Coarse striping - whole AU
      •   Fine striping - 128K i...
ASM striping 10g

     • Allocation    unit (AU) 1MB
      •   Coarse striping - whole AU
      •   Fine striping - 128K i...
ASM striping 10g

     • Allocation    unit (AU) 1MB
      •   Coarse striping - whole AU
      •   Fine striping - 128K i...
ASM rebalancing




14                     © 2009/2010 Pythian
ASM rebalancing




14                     © 2009/2010 Pythian
ASM rebalancing




14                     © 2009/2010 Pythian
ASM rebalancing




14                     © 2009/2010 Pythian
ASM rebalancing




14                     © 2009/2010 Pythian
ASM rebalancing




14                     © 2009/2010 Pythian
ASM rebalancing

     •   Add / remove disks
     •   Migrate to another storage
     •   Extend / shrink
     •   asm_pow...
ASM mirroring




15                   © 2009/2010 Pythian
ASM mirroring




15                   © 2009/2010 Pythian
ASM mirroring




15                   © 2009/2010 Pythian
ASM mirroring




15                   © 2009/2010 Pythian
ASM mirroring




15                   © 2009/2010 Pythian
ASM mirroring




15                   © 2009/2010 Pythian
ASM mirroring




15                   © 2009/2010 Pythian
ASM mirroring




15                   © 2009/2010 Pythian
ASM mirroring




15                   © 2009/2010 Pythian
ASM mirroring

     •   Primary extent & mirror extent


     •   Failure tolerance
         •   two or three way mirrorin...
ASM failure groups

     •   Group of disks can fail at once
     •   Mirror extents between failure groups




          ...
ASM failure groups

     •   Group of disks can fail at once
     •   Mirror extents between failure groups




          ...
ASM failure groups

     •   Group of disks can fail at once
     •   Mirror extents between failure groups




          ...
ASM instance

     • Special   kind of Oracle instance
      •   Clustered in RAC
     • ASM disks and diskgroups
     • A...
ASM 10g issues

     • ASM upgrade requires full RAC outage
     • Fixed 1MB allocation unit

     • Overhead with multi-T...
ASM 11gR1 - diskgroup compatibility

     • Default
        SQL> select name, compatibility, database_compatibility
      ...
ASM 11g - rolling updates for RAC

 ALTER SYSTEM START ROLLING MIGRATION TO
 11.2.0.0.0;

• Limited   services from ASM
  ...
ASM 11gR1 - variable AU size

• Allocation    Unit - 1/2/4/.../64 MB
 •   10g - 1MB (_asm_ausize)
• Data   sits closer tog...
ASM 11gR1 - variable extent size

     • 10g
      •   1 extent = 1 AU                   •   1 TB = 1+ mil. extents


    ...
ASM 11gR1 - recovery from failures

• Fast   Mirror Resync
 •   OFFLINE disks
 •   disk_repair_time attribute
 •   ASM ext...
ASM 11gR1 - Preferred Mirror Read

• For   extended clusters
 •   Storage mirrored across 2 or 3 datacenters


• 10g
 •   ...
ASM Dynamic Volume Manager (ADVM)

     • Clustervolume manager
     • Can place any filesystem on top of an ASM Volume
  ...
ADVM on Linux

      Loadable kernel modules

     [root@epa01 init.d]# ./acfsload start -s
     [root@epa01 init.d]# lsmo...
ADVM: Creating a volume

     ASMCMD & SQL Interfaces (+GUI Tools: ASMCA & OEM)

     ASMCMD> volcreate -G DG_A1 -s 50M DV...
ADVM in the OS

     OS devices
     [oradb@epa01 ~]$ ls -l /dev/asm/
     total 0
     brwxrwx--- 1 root dba 252, 12289 N...
ADVM: creating filesystems

     # /sbin/mkfs -t ext4 /dev/asm/dv_a3-461
     # /sbin/mkfs -t acfs -b 4k /dev/asm/dv_a3-461...
Oracle ASM Cluster Filesystem (ACFS)

                                                       • General   purpose cluster
 ...
ACFS Snapshots

     • Filesystem     technology
      •   No snapshots for ASM diskgroups and ADVM volumes
     • Read-on...
ASM 11gR2 - “small” enhancements

     • ASM  supports placement of OCR and voting disks
     • Intelligent Data Placement...
ASM Overhead?

     • ASM is not in the IO code path for Oracle Database
     • ASM only operates extent maps and disk hea...
What’s still missing?




34                      © 2009/2010 Pythian
What’s still missing?


     •   Everything ZFS has




34                      © 2009/2010 Pythian
What’s still missing?

     • Read-write  ACFS snapshots / clones
     • ASM diskgroup level snapshots for database files
...
What’s next?

     • Book   draw
        •   Leave me your cards or email+phone
      Q&A




      Email me - gorbachev@p...
Upcoming SlideShare
Loading in...5
×

Oracle ASM 11g - The Evolution

13,717

Published on

Oracle Automatic Storage Management has proven to be one of the most widely adopted new features in Oracle Database 10g and it has been dramatically improved in the later 11g releases. This presentation will explain what changes are solved by ASM, how these challenges are solved, what barriers there are to ASM adoptions, and how 11g Release 2 addresses these barriers.

Published in: Technology
4 Comments
38 Likes
Statistics
Notes
  • very nice post
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Ed, ASM is *not* LVM. It wasn't designed to replace LVM in the first case and it's been quite good at it.
    Lots of customers demanded new features in ASM to replace universal volume managers and that's the reason behind introduction of ADVM and ACFS - customer demand. When it comes to ADVM and ACFS, I believe there is a long way to go so if you consider ADVM as replacement for LVM - then it's still a baby.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • God, ASM sucks as an LVM. It's begging, borrowing, and stealing from Veritas as fast as possible, and still so far behind the curve as to be pitiful.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • -By Alex Gorbachev, From Pythian Group
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
13,717
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
31
Comments
4
Likes
38
Embeds 0
No embeds

No notes for slide


  • - Successful growing business for more than 10 years
    - Served many customers with complex requirements/infrastructure just like yours.
    - Operate globally for 24 x 7 “always awake” services

  • - simplifying
    - could also partition

  • Huge management overhead
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication
  • Advanced features
    - rebalancing and online moves
    - clones / snapshots / checkpoints
    - storage replication





  • ASM is VM + CFS = specialized FS
    minimal you get is RAW device performance with manageability of FS
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Introduce diskgroup
  • Easy to...
    Same happens when adding new disks - ASM re-distributes equally
  • Easy to...
    Same happens when adding new disks - ASM re-distributes equally
  • Easy to...
    Same happens when adding new disks - ASM re-distributes equally
  • Easy to...
    Same happens when adding new disks - ASM re-distributes equally
  • Easy to...
    Same happens when adding new disks - ASM re-distributes equally
  • Easy to...
    Same happens when adding new disks - ASM re-distributes equally
  • Easy to...
    Same happens when adding new disks - ASM re-distributes equally
  • Easy to...
    Same happens when adding new disks - ASM re-distributes equally
  • Easy to...
    Same happens when adding new disks - ASM re-distributes equally


















  • Previous slide - no failure groups = each disk in its own failure group
    What about recovery?
    Rebalancing as result of failure - enough space + enough failure groups.
    What about temporary failures?
    Extended clusters - even worse!
  • Previous slide - no failure groups = each disk in its own failure group
    What about recovery?
    Rebalancing as result of failure - enough space + enough failure groups.
    What about temporary failures?
    Extended clusters - even worse!
  • Previous slide - no failure groups = each disk in its own failure group
    What about recovery?
    Rebalancing as result of failure - enough space + enough failure groups.
    What about temporary failures?
    Extended clusters - even worse!
  • Previous slide - no failure groups = each disk in its own failure group
    What about recovery?
    Rebalancing as result of failure - enough space + enough failure groups.
    What about temporary failures?
    Extended clusters - even worse!
  • Previous slide - no failure groups = each disk in its own failure group
    What about recovery?
    Rebalancing as result of failure - enough space + enough failure groups.
    What about temporary failures?
    Extended clusters - even worse!


  • ASM compat - metadata format. RDBMS compat - files content format.
    11gR2 - COMPATIBLE.ADVM

    Default 10.1 - no new features. Defined per disk group.

    Case study - upgrade to 11g & downgrade, smooth upgrade path, transportable tablespaces.

    select i.ksppinm, v.ksppstvl from x$ksppi i, x$ksppcv v where i.ksppinm in ('_rdbms_compatibility','_asm_compatibility') and i.indx=v.indx;
  • 1. start rolling migration
    2. upgrade all instances one by one
    ASM provides limited services
    3. stop rolling migration
  • AU - smallest chunk of space that Oracle can allocate; coarse stripe size
    problem - inflexible physical layout + high overhead for VLDB
    10g - underscore parameter - instance-wide + undocumented/unsupported
    11g - official way to change AU size per diskgroup. AU 1-8 MB - compat 10.1 but 16-64 MB - compat 11.1.

    Fine striping is not changed!
  • Fully automatic. Explain how it works.

    Main purpose - reduce overhead on disk mount and file open for huge files, sga overhead for extent maps.
  • 10g - disk failure -> BROKEN & rebalancing
    11g FMR - disk failure -> OFFLINE -> wait for disk_repair_time (3.6h default) -> rebalancing if too log or ALTER DISKGROUP DISK ... ONLINE;
    * can do manual ALTER DISKGROUP DISK ... OFFLINE;

    FMR - case study SAN firmware update, FMR - case study extended clusters
    FB - case study - rebalancing takes days and impacts performance but night can be offline
  • Case study - extended clusters - in addition to Fast Mirror Resync
  • Works on single node as well.


  • Size - 256M increments
    Redundancy - unprotected
    Stripe Columns - 1 to 8
    Stripe Width - 4K - 1MB in powers of two
    Primary/Secondary extents - hot/cold (Intelligent Data Placement)
    Usage - comment text


  • ACFS - general purpose POSIX/X-OPEN standard FS
    All ASM features are leveraged like mirroring and striping


  • new ASMCMD commands
    - find and cp in R1
    - disk/group/volume management in R2, instance stop/start and etc.


  • Install Oracle home on ASM took hours!

  • Oracle ASM 11g - The Evolution

    1. 1. Oracle ASM 11g The Evolution Alex Gorbachev RMOUG Training Days 2010 Denver, 18-Feb-10
    2. 2. Alex Gorbachev • CTO, The Pythian Group • Blogger • OakTable Network member • Oracle ACE Director • BattleAgainstAnyGuess.com • Vice-president, Oracle RAC SIG 2 © 2009/2010 Pythian
    3. 3. Why Companies Trust Pythian • Recognized Leader: • Global industry-leader in remote database administration services and consulting for Oracle, Oracle Applications, MySQL and SQL Server • Work with over 150 multinational companies such as Forbes.com, Fox Interactive media, and MDS Inc. to help manage their complex IT deployments • Expertise: • One of the world’s largest concentrations of dedicated, full-time DBA expertise. • Global Reach & Scalability: • 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response 3 © 2009/2010 Pythian
    4. 4. ASM adoption poll • How many use ASM? • Plan to use? • Plan NOT to use? • Adoption rates amongst Pythian customers - very high 4 © 2009/2010 Pythian
    5. 5. Pre-ASM - simple file system on JBOD • Resilience? • multiplexing • redo + controlfiles Server • hardware mirroring /u01 /u02 • Scaleability? • Manual datafile layout • separate indexes and tables? • Hardware striping • Manageability? Disk Disk • poor • resize? move? 5 © 2009/2010 Pythian
    6. 6. Pre-ASM - raw devices Server /dev/sdb /dev/sdc • One file => disk or partition • Why raw devices? • remove overhead of FS • Later CIO, direct IO, etc Disk Disk 6 © 2009/2010 Pythian
    7. 7. Pre-ASM - raw devices (cluster) Server 1 • One file => disk or partition • Why raw devices? /dev/sdb /dev/sdc • remove overhead of FS • Later CIO, direct IO, etc Disk Disk • Clusters - shared storage /dev/sdb /dev/sdc Server 2 7 © 2009/2010 Pythian
    8. 8. Pre-ASM - volume manager • FS or “raw device” • Logical volume • Volume groups • Physical disks 8 © 2009/2010 Pythian
    9. 9. Pre-ASM - volume manager • FS or “raw device” • Logical volume • Volume groups • Physical disks Storage box 8 © 2009/2010 Pythian
    10. 10. Pre-ASM - cluster VM and FS • Cluster Volume Manager (CVM) • Synchronizes volume metadata • Still lots of manageability overhead • 1 volume = one file • Cluster File System (CFS) • On top of CVM • Synchronize file metadata & concurrent IO • Network Attached Storage (NAS) • FS and VM on storage box • ZFS opens the whole new dimension in manageability 9 © 2009/2010 Pythian
    11. 11. Pre-ASM storage issues 10 © 2009/2010 Pythian
    12. 12. Pre-ASM storage issues • Too complex • 3rd party vendors • Additional licensing cost • Many groups involved • Storage engineers • SA’s • DBA’s • Loosing layout visibility 10 © 2009/2010 Pythian
    13. 13. Pre-ASM storage issues • Too complex • Too simple • 3rd party vendors • Doesn’t scale • Not reliable • Additional licensing cost • Not flexible • Many groups involved • Storage engineers • SA’s • DBA’s • Loosing layout visibility 10 © 2009/2010 Pythian
    14. 14. ASM introduction • Simplify • Fewer vendors involved • Fewer layers • Move many operations under DBA umbrella • oh well, Storage Administrator role • Cost efficient • No need to license volume manager • Cheap storage boxes are good for ASM • Manageable 11 © 2009/2010 Pythian
    15. 15. ASM principles • Organizing Oracle files on physical disks • Mirroring • reliability & failure tolerance • Striping • performance & scaleability • Rebalancing • manageability 12 © 2009/2010 Pythian
    16. 16. ASM striping 10g 13 © 2009/2010 Pythian
    17. 17. ASM striping 10g 13 © 2009/2010 Pythian
    18. 18. ASM striping 10g 13 © 2009/2010 Pythian
    19. 19. ASM striping 10g 13 © 2009/2010 Pythian
    20. 20. ASM striping 10g 13 © 2009/2010 Pythian
    21. 21. ASM striping 10g 13 © 2009/2010 Pythian
    22. 22. ASM striping 10g file1 13 © 2009/2010 Pythian
    23. 23. ASM striping 10g file1 file2 13 © 2009/2010 Pythian
    24. 24. ASM striping 10g • Allocation unit (AU) 1MB • Coarse striping - whole AU • Fine striping - 128K inside AU file1 file2 13 © 2009/2010 Pythian
    25. 25. ASM striping 10g • Allocation unit (AU) 1MB • Coarse striping - whole AU • Fine striping - 128K inside AU • One extent = 1 AU file1 file2 13 © 2009/2010 Pythian
    26. 26. ASM striping 10g • Allocation unit (AU) 1MB • Coarse striping - whole AU • Fine striping - 128K inside AU • One extent = 1 AU • Random striping -> equal distribution • No auto-magic re-layout based on performance file1 file2 13 © 2009/2010 Pythian
    27. 27. ASM rebalancing 14 © 2009/2010 Pythian
    28. 28. ASM rebalancing 14 © 2009/2010 Pythian
    29. 29. ASM rebalancing 14 © 2009/2010 Pythian
    30. 30. ASM rebalancing 14 © 2009/2010 Pythian
    31. 31. ASM rebalancing 14 © 2009/2010 Pythian
    32. 32. ASM rebalancing 14 © 2009/2010 Pythian
    33. 33. ASM rebalancing • Add / remove disks • Migrate to another storage • Extend / shrink • asm_power_limit - rebalancing power/overhead • No mirroring (external redundancy) • planned maintenance 14 © 2009/2010 Pythian
    34. 34. ASM mirroring 15 © 2009/2010 Pythian
    35. 35. ASM mirroring 15 © 2009/2010 Pythian
    36. 36. ASM mirroring 15 © 2009/2010 Pythian
    37. 37. ASM mirroring 15 © 2009/2010 Pythian
    38. 38. ASM mirroring 15 © 2009/2010 Pythian
    39. 39. ASM mirroring 15 © 2009/2010 Pythian
    40. 40. ASM mirroring 15 © 2009/2010 Pythian
    41. 41. ASM mirroring 15 © 2009/2010 Pythian
    42. 42. ASM mirroring 15 © 2009/2010 Pythian
    43. 43. ASM mirroring • Primary extent & mirror extent • Failure tolerance • two or three way mirroring 15 © 2009/2010 Pythian
    44. 44. ASM failure groups • Group of disks can fail at once • Mirror extents between failure groups SAN 1 SAN 2 16 © 2009/2010 Pythian
    45. 45. ASM failure groups • Group of disks can fail at once • Mirror extents between failure groups SAN 1 16 © 2009/2010 Pythian
    46. 46. ASM failure groups • Group of disks can fail at once • Mirror extents between failure groups SAN 1 16 © 2009/2010 Pythian
    47. 47. ASM instance • Special kind of Oracle instance • Clustered in RAC • ASM disks and diskgroups • ASM extent map • Rebalancing operations • Must be started before database instance 17 © 2009/2010 Pythian
    48. 48. ASM 10g issues • ASM upgrade requires full RAC outage • Fixed 1MB allocation unit • Overhead with multi-TB databases • Recovery from failure is inefficient • Extended clusters read IO is inefficient • Limited FS functionality and tools • No snapshot-like functionality 18 © 2009/2010 Pythian
    49. 49. ASM 11gR1 - diskgroup compatibility • Default SQL> select name, compatibility, database_compatibility from v$asm_diskgroup; NAME COMPATIBIL DATABASE_C ---- ---------- ---------- DG1 10.1.0.0.0 10.1.0.0.0 • New features SQL> ALTER DISKGROUP dg1 SET ATTRIBUTE 'compatible.asm'='11.1'; Diskgroup altered. SQL> ALTER DISKGROUP dg1 SET ATTRIBUTE 'compatible.rdbms'='11.1'; Diskgroup altered. SQL> select name, compatibility, database_compatibility from v$asm_diskgroup; NAME COMPATIBIL DATABASE_C ---- ---------- ---------- DG1 11.1.0.0.0 11.1.0.0.0 19 © 2009/2010 Pythian
    50. 50. ASM 11g - rolling updates for RAC ALTER SYSTEM START ROLLING MIGRATION TO 11.2.0.0.0; • Limited services from ASM • Normal database operation • No diskgroup configuration changes • Mount / unmount diskgroups ALTER SYSTEM STOP ROLLING MIGRATION; © 2009/2010 Pythian
    51. 51. ASM 11gR1 - variable AU size • Allocation Unit - 1/2/4/.../64 MB • 10g - 1MB (_asm_ausize) • Data sits closer together and can be read in bigger chunks • Can improve sequential IO • Reduces striping over SAN striped LUN’s • Configuring Oracle ASM hidden parameters for EVA8000, HP Knowledge Brief • Reduces SGA for metadata for large files • 11g - can be done per diskgroup © 2009/2010 Pythian
    52. 52. ASM 11gR1 - variable extent size • 10g • 1 extent = 1 AU • 1 TB = 1+ mil. extents • 11g • 1-20,000 extents • 1MB AU: 1 extent = 1 AU • 1 TB = 53,572 extents • 20,001 - 40,000 • 64MB AU: 1 extent = 8 AU’s • 1 TB = 16,384 extents • 40,001 - ... • 100 TB = 66,788 extents 1 extent = 64 AU’s 22 © 2009/2010 Pythian
    53. 53. ASM 11gR1 - recovery from failures • Fast Mirror Resync • OFFLINE disks • disk_repair_time attribute • ASM extent change tracking • Suitable for transient failures and maintenance • Fast Rebalancing in restricted mount • Rebalancing => many lock/unlock extent map • Restrict mode rebalancing => no locks • Restricted mounted DG => service outage © 2009/2010 Pythian
    54. 54. ASM 11gR1 - Preferred Mirror Read • For extended clusters • Storage mirrored across 2 or 3 datacenters • 10g • Read is first done on primary extent • 11g • Preferred read failure groups (to local disks) • asm_preferred_read_failure_groups in init.ora © 2009/2010 Pythian
    55. 55. ASM Dynamic Volume Manager (ADVM) • Clustervolume manager • Can place any filesystem on top of an ASM Volume • Oracle ACFS or third-party like ext3 • Unix/Linux & Windows • Dynamically resizable volumes • Leveraging all standard ASM features (like mirroring) • Extent size 64MB - allocation unit • Number of columns - 1 to 8 configurable • Striping across columns • Dirty Region Logging (DRL) 25 © 2009/2010 Pythian
    56. 56. ADVM on Linux Loadable kernel modules [root@epa01 init.d]# ./acfsload start -s [root@epa01 init.d]# lsmod | grep ora oracleacfs 787460 0 oracleadvm 177792 0 oracleoks 226656 2 oracleacfs,oracleadvm [root@epa01 init.d]# 26 © 2009/2010 Pythian
    57. 57. ADVM: Creating a volume ASMCMD & SQL Interfaces (+GUI Tools: ASMCA & OEM) ASMCMD> volcreate -G DG_A1 -s 50M DV_A2 Volume Name: DV_A2 Volume Device: /dev/asm/dv_a2-354 State: ENABLED Size (MB): 256 Resize Unit (MB): 256 Redundancy: UNPROT Stripe Columns: 4 Stripe Width (K): 128 Usage: Mountpath: SQL> ALTER DISKGROUP DG_A1 ADD VOLUME DV_A2 SIZE 50M; 27 © 2009/2010 Pythian
    58. 58. ADVM in the OS OS devices [oradb@epa01 ~]$ ls -l /dev/asm/ total 0 brwxrwx--- 1 root dba 252, 12289 Nov 16 02:15 dv_a1-24 brwxrwx--- 1 root dba 252, 111617 Nov 16 02:15 dv_main_db-218 [oradb@epa01 ~]$ Dynamically resizable ASMCMD> volresize -G DG_A1 -s 400M DV_A2 28 © 2009/2010 Pythian
    59. 59. ADVM: creating filesystems # /sbin/mkfs -t ext4 /dev/asm/dv_a3-461 # /sbin/mkfs -t acfs -b 4k /dev/asm/dv_a3-461 mkfs.acfs: version = 11.2.0.1.0.0 mkfs.acfs: on-disk version = 39.0 mkfs.acfs: volume = /dev/asm/dv_a3-461 mkfs.acfs: volume size = 268435456 mkfs.acfs: Format complete. # mount -t acfs /dev/asm/dv_a3-461 /a03 # df –h /a03 Filesystem Size Used Avail Use% Mounted on /dev/asm/dv_a3-461 256M 37M 220M 15% /a03 29 © 2009/2010 Pythian
    60. 60. Oracle ASM Cluster Filesystem (ACFS) • General purpose cluster filesystem • Journaling, variable extent-based • Windows/Linux/Unix • Oracle homes, dumps, traces, export files and etc - *anything* • Native ASM files (i.e. no datafiles, Diagram from Oracle ASM 10gR2 Documentation controlfiles and etc) unsupported • Direct path IO or cached • ASM tools • Dynamic resize SQL/ASMCMD/ASMCA/OEM • Mount registry (like /etc/fstab) • Standard OS commands mkfs / mount / fsck / fuser / cp / chmod / chown / ... 30 © 2009/2010 Pythian
    61. 61. ACFS Snapshots • Filesystem technology • No snapshots for ASM diskgroups and ADVM volumes • Read-only • Space-efficient • Initially sparse with copy-on-write technology • Up to 63 snapshots per filesystem • Access via .ACFS/snaps hidden directory • CLI acfsutil snap and OEM support 31 © 2009/2010 Pythian
    62. 62. ASM 11gR2 - “small” enhancements • ASM supports placement of OCR and voting disks • Intelligent Data Placement (hot/cold -> outer/inner) • Disk Group rename • renamedg utility • ASMCMD enhancements • New ASMCA tool, OEM support • Access control for ASM files (owner/group/others) • 4K sector-size for diskgroups 32 © 2009/2010 Pythian
    63. 63. ASM Overhead? • ASM is not in the IO code path for Oracle Database • ASM only operates extent maps and disk headers • Database -> thousands IO’s per second vs ASM -> few IO’s per minute • Additional ASM instance - memory and processes but not much • Database instance keeps ASM extent maps in SGA • Mirroring generates additional write IO • ADVM - another layer • ACFS - yet another layer 33 © 2009/2010 Pythian
    64. 64. What’s still missing? 34 © 2009/2010 Pythian
    65. 65. What’s still missing? • Everything ZFS has 34 © 2009/2010 Pythian
    66. 66. What’s still missing? • Read-write ACFS snapshots / clones • ASM diskgroup level snapshots for database files • ADVM snapshots (do we need them?) • ACFS or ADVM compression • Diskgroup mirroring conversion • Split mirror functionality • De-duplication capabilities • Still need multipathing • Proper throttling of rebalancing operations • Is ADVM and ACFS are production ready? 35 © 2009/2010 Pythian
    67. 67. What’s next? • Book draw • Leave me your cards or email+phone Q&A Email me - gorbachev@pythian.com Read my blog - http://www.pythian.com Follow me on Twitter - @AlexGorbachev Join Pythian fan club on Facebook & LinkedIn 36 © 2009/2010 Pythian
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×