Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Oracle ASM 11g - The Evolution

15,622 views

Published on

Oracle Automatic Storage Management has proven to be one of the most widely adopted new features in Oracle Database 10g and it has been dramatically improved in the later 11g releases. This presentation will explain what changes are solved by ASM, how these challenges are solved, what barriers there are to ASM adoptions, and how 11g Release 2 addresses these barriers.

Published in: Technology

Oracle ASM 11g - The Evolution

  1. 1. Oracle ASM 11g The Evolution Alex Gorbachev RMOUG Training Days 2010 Denver, 18-Feb-10
  2. 2. Alex Gorbachev • CTO, The Pythian Group • Blogger • OakTable Network member • Oracle ACE Director • BattleAgainstAnyGuess.com • Vice-president, Oracle RAC SIG 2 © 2009/2010 Pythian
  3. 3. Why Companies Trust Pythian • Recognized Leader: • Global industry-leader in remote database administration services and consulting for Oracle, Oracle Applications, MySQL and SQL Server • Work with over 150 multinational companies such as Forbes.com, Fox Interactive media, and MDS Inc. to help manage their complex IT deployments • Expertise: • One of the world’s largest concentrations of dedicated, full-time DBA expertise. • Global Reach & Scalability: • 24/7/365 global remote support for DBA and consulting, systems administration, special projects or emergency response 3 © 2009/2010 Pythian
  4. 4. ASM adoption poll • How many use ASM? • Plan to use? • Plan NOT to use? • Adoption rates amongst Pythian customers - very high 4 © 2009/2010 Pythian
  5. 5. Pre-ASM - simple file system on JBOD • Resilience? • multiplexing • redo + controlfiles Server • hardware mirroring /u01 /u02 • Scaleability? • Manual datafile layout • separate indexes and tables? • Hardware striping • Manageability? Disk Disk • poor • resize? move? 5 © 2009/2010 Pythian
  6. 6. Pre-ASM - raw devices Server /dev/sdb /dev/sdc • One file => disk or partition • Why raw devices? • remove overhead of FS • Later CIO, direct IO, etc Disk Disk 6 © 2009/2010 Pythian
  7. 7. Pre-ASM - raw devices (cluster) Server 1 • One file => disk or partition • Why raw devices? /dev/sdb /dev/sdc • remove overhead of FS • Later CIO, direct IO, etc Disk Disk • Clusters - shared storage /dev/sdb /dev/sdc Server 2 7 © 2009/2010 Pythian
  8. 8. Pre-ASM - volume manager • FS or “raw device” • Logical volume • Volume groups • Physical disks 8 © 2009/2010 Pythian
  9. 9. Pre-ASM - volume manager • FS or “raw device” • Logical volume • Volume groups • Physical disks Storage box 8 © 2009/2010 Pythian
  10. 10. Pre-ASM - cluster VM and FS • Cluster Volume Manager (CVM) • Synchronizes volume metadata • Still lots of manageability overhead • 1 volume = one file • Cluster File System (CFS) • On top of CVM • Synchronize file metadata & concurrent IO • Network Attached Storage (NAS) • FS and VM on storage box • ZFS opens the whole new dimension in manageability 9 © 2009/2010 Pythian
  11. 11. Pre-ASM storage issues 10 © 2009/2010 Pythian
  12. 12. Pre-ASM storage issues • Too complex • 3rd party vendors • Additional licensing cost • Many groups involved • Storage engineers • SA’s • DBA’s • Loosing layout visibility 10 © 2009/2010 Pythian
  13. 13. Pre-ASM storage issues • Too complex • Too simple • 3rd party vendors • Doesn’t scale • Not reliable • Additional licensing cost • Not flexible • Many groups involved • Storage engineers • SA’s • DBA’s • Loosing layout visibility 10 © 2009/2010 Pythian
  14. 14. ASM introduction • Simplify • Fewer vendors involved • Fewer layers • Move many operations under DBA umbrella • oh well, Storage Administrator role • Cost efficient • No need to license volume manager • Cheap storage boxes are good for ASM • Manageable 11 © 2009/2010 Pythian
  15. 15. ASM principles • Organizing Oracle files on physical disks • Mirroring • reliability & failure tolerance • Striping • performance & scaleability • Rebalancing • manageability 12 © 2009/2010 Pythian
  16. 16. ASM striping 10g 13 © 2009/2010 Pythian
  17. 17. ASM striping 10g 13 © 2009/2010 Pythian
  18. 18. ASM striping 10g 13 © 2009/2010 Pythian
  19. 19. ASM striping 10g 13 © 2009/2010 Pythian
  20. 20. ASM striping 10g 13 © 2009/2010 Pythian
  21. 21. ASM striping 10g 13 © 2009/2010 Pythian
  22. 22. ASM striping 10g file1 13 © 2009/2010 Pythian
  23. 23. ASM striping 10g file1 file2 13 © 2009/2010 Pythian
  24. 24. ASM striping 10g • Allocation unit (AU) 1MB • Coarse striping - whole AU • Fine striping - 128K inside AU file1 file2 13 © 2009/2010 Pythian
  25. 25. ASM striping 10g • Allocation unit (AU) 1MB • Coarse striping - whole AU • Fine striping - 128K inside AU • One extent = 1 AU file1 file2 13 © 2009/2010 Pythian
  26. 26. ASM striping 10g • Allocation unit (AU) 1MB • Coarse striping - whole AU • Fine striping - 128K inside AU • One extent = 1 AU • Random striping -> equal distribution • No auto-magic re-layout based on performance file1 file2 13 © 2009/2010 Pythian
  27. 27. ASM rebalancing 14 © 2009/2010 Pythian
  28. 28. ASM rebalancing 14 © 2009/2010 Pythian
  29. 29. ASM rebalancing 14 © 2009/2010 Pythian
  30. 30. ASM rebalancing 14 © 2009/2010 Pythian
  31. 31. ASM rebalancing 14 © 2009/2010 Pythian
  32. 32. ASM rebalancing 14 © 2009/2010 Pythian
  33. 33. ASM rebalancing • Add / remove disks • Migrate to another storage • Extend / shrink • asm_power_limit - rebalancing power/overhead • No mirroring (external redundancy) • planned maintenance 14 © 2009/2010 Pythian
  34. 34. ASM mirroring 15 © 2009/2010 Pythian
  35. 35. ASM mirroring 15 © 2009/2010 Pythian
  36. 36. ASM mirroring 15 © 2009/2010 Pythian
  37. 37. ASM mirroring 15 © 2009/2010 Pythian
  38. 38. ASM mirroring 15 © 2009/2010 Pythian
  39. 39. ASM mirroring 15 © 2009/2010 Pythian
  40. 40. ASM mirroring 15 © 2009/2010 Pythian
  41. 41. ASM mirroring 15 © 2009/2010 Pythian
  42. 42. ASM mirroring 15 © 2009/2010 Pythian
  43. 43. ASM mirroring • Primary extent & mirror extent • Failure tolerance • two or three way mirroring 15 © 2009/2010 Pythian
  44. 44. ASM failure groups • Group of disks can fail at once • Mirror extents between failure groups SAN 1 SAN 2 16 © 2009/2010 Pythian
  45. 45. ASM failure groups • Group of disks can fail at once • Mirror extents between failure groups SAN 1 16 © 2009/2010 Pythian
  46. 46. ASM failure groups • Group of disks can fail at once • Mirror extents between failure groups SAN 1 16 © 2009/2010 Pythian
  47. 47. ASM instance • Special kind of Oracle instance • Clustered in RAC • ASM disks and diskgroups • ASM extent map • Rebalancing operations • Must be started before database instance 17 © 2009/2010 Pythian
  48. 48. ASM 10g issues • ASM upgrade requires full RAC outage • Fixed 1MB allocation unit • Overhead with multi-TB databases • Recovery from failure is inefficient • Extended clusters read IO is inefficient • Limited FS functionality and tools • No snapshot-like functionality 18 © 2009/2010 Pythian
  49. 49. ASM 11gR1 - diskgroup compatibility • Default SQL> select name, compatibility, database_compatibility from v$asm_diskgroup; NAME COMPATIBIL DATABASE_C ---- ---------- ---------- DG1 10.1.0.0.0 10.1.0.0.0 • New features SQL> ALTER DISKGROUP dg1 SET ATTRIBUTE 'compatible.asm'='11.1'; Diskgroup altered. SQL> ALTER DISKGROUP dg1 SET ATTRIBUTE 'compatible.rdbms'='11.1'; Diskgroup altered. SQL> select name, compatibility, database_compatibility from v$asm_diskgroup; NAME COMPATIBIL DATABASE_C ---- ---------- ---------- DG1 11.1.0.0.0 11.1.0.0.0 19 © 2009/2010 Pythian
  50. 50. ASM 11g - rolling updates for RAC ALTER SYSTEM START ROLLING MIGRATION TO 11.2.0.0.0; • Limited services from ASM • Normal database operation • No diskgroup configuration changes • Mount / unmount diskgroups ALTER SYSTEM STOP ROLLING MIGRATION; © 2009/2010 Pythian
  51. 51. ASM 11gR1 - variable AU size • Allocation Unit - 1/2/4/.../64 MB • 10g - 1MB (_asm_ausize) • Data sits closer together and can be read in bigger chunks • Can improve sequential IO • Reduces striping over SAN striped LUN’s • Configuring Oracle ASM hidden parameters for EVA8000, HP Knowledge Brief • Reduces SGA for metadata for large files • 11g - can be done per diskgroup © 2009/2010 Pythian
  52. 52. ASM 11gR1 - variable extent size • 10g • 1 extent = 1 AU • 1 TB = 1+ mil. extents • 11g • 1-20,000 extents • 1MB AU: 1 extent = 1 AU • 1 TB = 53,572 extents • 20,001 - 40,000 • 64MB AU: 1 extent = 8 AU’s • 1 TB = 16,384 extents • 40,001 - ... • 100 TB = 66,788 extents 1 extent = 64 AU’s 22 © 2009/2010 Pythian
  53. 53. ASM 11gR1 - recovery from failures • Fast Mirror Resync • OFFLINE disks • disk_repair_time attribute • ASM extent change tracking • Suitable for transient failures and maintenance • Fast Rebalancing in restricted mount • Rebalancing => many lock/unlock extent map • Restrict mode rebalancing => no locks • Restricted mounted DG => service outage © 2009/2010 Pythian
  54. 54. ASM 11gR1 - Preferred Mirror Read • For extended clusters • Storage mirrored across 2 or 3 datacenters • 10g • Read is first done on primary extent • 11g • Preferred read failure groups (to local disks) • asm_preferred_read_failure_groups in init.ora © 2009/2010 Pythian
  55. 55. ASM Dynamic Volume Manager (ADVM) • Clustervolume manager • Can place any filesystem on top of an ASM Volume • Oracle ACFS or third-party like ext3 • Unix/Linux & Windows • Dynamically resizable volumes • Leveraging all standard ASM features (like mirroring) • Extent size 64MB - allocation unit • Number of columns - 1 to 8 configurable • Striping across columns • Dirty Region Logging (DRL) 25 © 2009/2010 Pythian
  56. 56. ADVM on Linux Loadable kernel modules [root@epa01 init.d]# ./acfsload start -s [root@epa01 init.d]# lsmod | grep ora oracleacfs 787460 0 oracleadvm 177792 0 oracleoks 226656 2 oracleacfs,oracleadvm [root@epa01 init.d]# 26 © 2009/2010 Pythian
  57. 57. ADVM: Creating a volume ASMCMD & SQL Interfaces (+GUI Tools: ASMCA & OEM) ASMCMD> volcreate -G DG_A1 -s 50M DV_A2 Volume Name: DV_A2 Volume Device: /dev/asm/dv_a2-354 State: ENABLED Size (MB): 256 Resize Unit (MB): 256 Redundancy: UNPROT Stripe Columns: 4 Stripe Width (K): 128 Usage: Mountpath: SQL> ALTER DISKGROUP DG_A1 ADD VOLUME DV_A2 SIZE 50M; 27 © 2009/2010 Pythian
  58. 58. ADVM in the OS OS devices [oradb@epa01 ~]$ ls -l /dev/asm/ total 0 brwxrwx--- 1 root dba 252, 12289 Nov 16 02:15 dv_a1-24 brwxrwx--- 1 root dba 252, 111617 Nov 16 02:15 dv_main_db-218 [oradb@epa01 ~]$ Dynamically resizable ASMCMD> volresize -G DG_A1 -s 400M DV_A2 28 © 2009/2010 Pythian
  59. 59. ADVM: creating filesystems # /sbin/mkfs -t ext4 /dev/asm/dv_a3-461 # /sbin/mkfs -t acfs -b 4k /dev/asm/dv_a3-461 mkfs.acfs: version = 11.2.0.1.0.0 mkfs.acfs: on-disk version = 39.0 mkfs.acfs: volume = /dev/asm/dv_a3-461 mkfs.acfs: volume size = 268435456 mkfs.acfs: Format complete. # mount -t acfs /dev/asm/dv_a3-461 /a03 # df –h /a03 Filesystem Size Used Avail Use% Mounted on /dev/asm/dv_a3-461 256M 37M 220M 15% /a03 29 © 2009/2010 Pythian
  60. 60. Oracle ASM Cluster Filesystem (ACFS) • General purpose cluster filesystem • Journaling, variable extent-based • Windows/Linux/Unix • Oracle homes, dumps, traces, export files and etc - *anything* • Native ASM files (i.e. no datafiles, Diagram from Oracle ASM 10gR2 Documentation controlfiles and etc) unsupported • Direct path IO or cached • ASM tools • Dynamic resize SQL/ASMCMD/ASMCA/OEM • Mount registry (like /etc/fstab) • Standard OS commands mkfs / mount / fsck / fuser / cp / chmod / chown / ... 30 © 2009/2010 Pythian
  61. 61. ACFS Snapshots • Filesystem technology • No snapshots for ASM diskgroups and ADVM volumes • Read-only • Space-efficient • Initially sparse with copy-on-write technology • Up to 63 snapshots per filesystem • Access via .ACFS/snaps hidden directory • CLI acfsutil snap and OEM support 31 © 2009/2010 Pythian
  62. 62. ASM 11gR2 - “small” enhancements • ASM supports placement of OCR and voting disks • Intelligent Data Placement (hot/cold -> outer/inner) • Disk Group rename • renamedg utility • ASMCMD enhancements • New ASMCA tool, OEM support • Access control for ASM files (owner/group/others) • 4K sector-size for diskgroups 32 © 2009/2010 Pythian
  63. 63. ASM Overhead? • ASM is not in the IO code path for Oracle Database • ASM only operates extent maps and disk headers • Database -> thousands IO’s per second vs ASM -> few IO’s per minute • Additional ASM instance - memory and processes but not much • Database instance keeps ASM extent maps in SGA • Mirroring generates additional write IO • ADVM - another layer • ACFS - yet another layer 33 © 2009/2010 Pythian
  64. 64. What’s still missing? 34 © 2009/2010 Pythian
  65. 65. What’s still missing? • Everything ZFS has 34 © 2009/2010 Pythian
  66. 66. What’s still missing? • Read-write ACFS snapshots / clones • ASM diskgroup level snapshots for database files • ADVM snapshots (do we need them?) • ACFS or ADVM compression • Diskgroup mirroring conversion • Split mirror functionality • De-duplication capabilities • Still need multipathing • Proper throttling of rebalancing operations • Is ADVM and ACFS are production ready? 35 © 2009/2010 Pythian
  67. 67. What’s next? • Book draw • Leave me your cards or email+phone Q&A Email me - gorbachev@pythian.com Read my blog - http://www.pythian.com Follow me on Twitter - @AlexGorbachev Join Pythian fan club on Facebook & LinkedIn 36 © 2009/2010 Pythian

×