Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Do I actually need to care about upgrading my       existing environment to race with              Oracle latest release? ...
Does it really matter, which version of Oracle                   do we use?Presented by : Syed Jaffer Hussain   red-gate/A...
Upgrading to Oracle 11gR2                                      A Successful case studyPresented by : Syed Jaffer Hussain  ...
We really had a roller coaster journey…..                A successful upgrade               A successful downgrade    De-i...
Disclaimer            This views/content in this slides are those of the author and            do not necessarily reflect ...
Foreword            The objective of this presentation is to explain you the best            practices that we had adopted...
What will be covered            o    Why upgrade?            o    Upgrade compatibility matrix.            o    Upgrade pa...
Know your presenter                                     Syed Jaffer Hussain                                     Database S...
Know your presenter                                                                Technologist of the Year, DBA 2011     ...
What will be covered            o    WHY upgrade?            o    Upgrade compatibility matrix.            o    Upgrade pa...
Why upgrade? – A burning question           Do we really need to care which           version are we using?           Is i...
Why upgrade?Presented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 12
Why upgrade?        To best of my knowledge the        following situations may        contributes to oppose an        upg...
Why upgrade?  Situations that are likely to  invite you to embrace for an  upgrade:          Organization standards.     ...
What will be covered            o    Why upgrade?            o    Upgrade compatibility MATRIX.            o    Upgrade pa...
Upgrade compatibility matrix                                                                     Courtesy from Oracle Corp...
Upgrade compatibility matrix                                                                     Courtesy from Oracle Corp...
Upgrade compatibility matrix                   Minimum Version of the database that can be directly upgraded to Oracle 11g...
What will be covered            o    Why upgrade?            o    Upgrade compatibility matrix.            o    Upgrade pa...
Upgrade pattern [BEST PRACTICE]              UPGRADE             Clusterware                + ASMPresented by : Syed Jaffe...
Upgrade pattern [best practice]              UPGRADE                 INSTALL             Clusterware              RDBMS   ...
Upgrade pattern [best practice]              UPGRADE                 INSTALL                                              ...
Upgrade pattern [best practice]            UPGRADE                   INSTALL                                              ...
What will be covered            o    Why upgrade?            o    Upgrade compatibility matrix.            o    Upgrade pa...
Overview about our environment                                                                               CISCO Switch ...
Overview about our environment                            Production 1                                    Production 2    ...
Overview about our environment                            Production 1                                             Product...
Overview about our environment       The framework that we have followed for the upgrade                                  ...
Overview about our environment       The framework that we have followed for the upgrade                                  ...
Overview of our environment       The framework that we have followed for the upgrade                                     ...
What will be covered            o    Why upgrade?            o    Upgrade compatibility matrix.            o    Upgrade pa...
What will be covered        Preparation is the headstone to        every successful installation or        upgrade project...
Prepare for upgrade – prerequisites checks   Verify:                 o OS certification for Oracle 11gR2.                ...
Prepare for upgrade – prerequisites checks     Space:                   o Additional space for Oracle software           ...
Prepare for upgrade – prerequisites checks    Verify nodes readiness for upgrade:    While upgrading from pre 11gR2, use t...
What will be covered            o    Why upgrade?            o    Upgrade compatibility matrix.            o    Upgrade pa...
Clusterware, ASM upgrade – What’s new?                                                                Grid Infrastructure ...
Clusterware, ASM upgrade        Is it necessary to upgrade        Clusterware AND ASM        together?Presented by : Syed ...
Clusterware, ASM upgrade                                                   Clusteware +                                   ...
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 40
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 41
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 42
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 43
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 44
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 45
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 46
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 47
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 48
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 49
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 50
Clusterware and ASM upgrade                                                                ORACLE BASE must be            ...
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 52
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 53
Clusterware and ASM upgrade                                     Cause :                                     False warning ...
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 55
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 56
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 57
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 58
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 59
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 60
Clusterware and ASM upgrade         Can I run the         rootupgrade.sh script in         PARALLEL across nodes?Presented...
Clusterware and ASM upgrade         Can I run the         rootupgrade.sh script in         PARALLEL across nodes?         ...
Clusterware and ASM upgrade                  Always run the script on the LOCAL node first.Presented by : Syed Jaffer Huss...
Clusterware and ASM upgrade                  Always run the script on the LOCAL node first.                               ...
Clusterware and ASM upgrade                  Always run the script on the LOCAL node first.                               ...
Clusterware and ASM upgrade         rootupgrade.sh         Shuts down the existing clusterware stack.         Brings up th...
Clusterware and ASM upgrade            Average rootupgrade.sh execution time on each node was just about 15 min.          ...
Clusterware and ASM upgrade            Average rootupgrade.sh execution time on each node was just about 15 min.          ...
Clusterware and ASM upgrade      rootupgrade.sh      execution failed?Presented by : Syed Jaffer Hussain   red-gate/AllThi...
Clusterware and ASM upgrade              How to Proceed from Failed Upgrade to 11gR2 Grid Infrastructure on Linux/Unix    ...
Clusterware and ASM upgrade          Use the –force flag to force the upgrade to complete in case some nodes becomes      ...
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 72
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 73
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 74
Clusterware and ASM upgrade                                                                Cried for HELP!Presented by : S...
Clusterware and ASM upgrade      Internet                                                                Cried for HELP!Pr...
Clusterware and ASM upgrade      Internet                       MOS                                                       ...
Clusterware and ASM upgrade      Internet                       MOS                                                       ...
Clusterware and ASM upgrade                                     Life Savior                                     MOS [ID 39...
Clusterware and ASM upgrade       cd $ORA_ASM_HOME/log/hostname/racg/ora.<hostname>.ASM.asm.log   SQL> ORACLE instance shu...
Clusterware and ASM upgrade      What might have gone wrong???        ASM instance name registered as ASMn          inste...
Clusterware and ASM upgrade      Remedy         Stop ASM and dependent instances.         Remove dependency on ASM insta...
Clusterware and ASM upgrade       In action      srvctl stop asm –n <hostname>      srvctl modify instance –d DBNAME –i ...
Clusterware and ASM upgrade       In action      srvctl stop asm –n <hostname>      srvctl modify instance –d DBNAME –i ...
Clusterware and ASM upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 85
Clusterware and ASM upgrade        Post upgrade checks – active/software version        #crsctl query crs activeversion   ...
Clusterware and ASM upgrade        Post upgrade checks – active/software version        #crsctl query crs activeversion   ...
Clusterware and ASM upgrade        Post upgrade checks – cluster stats across nodes        # crsctl check cluster -all    ...
Clusterware and ASM upgrade       Post upgrade checks – query voting disk       # ocrcheck       Status of Oracle Cluster ...
Clusterware and ASM upgrade       Post upgrade checks – query voting disk       # crsctl query css votedisk       ##    ST...
Clusterware and ASM upgrade       Post upgrade checks – b/g and daemon processes       #ps -ef |grep d.bin         oracle ...
Clusterware and ASM upgrade      Remember       You can easily resume/downgrade from a failed or successful upgrade.     ...
Remove OLD homes      Removing OLD cluster home       It’s a best practice to keep the       OLD_HOME for a while after a ...
What will be covered            o    Why upgrade?            o    Upgrade compatibility matrix.            o    Upgrade pa...
11gR2 RDBMS software installation       Verify the READINESS of the nodes for        DB software installation   cluvfy sta...
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 96
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 97
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 98
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 99
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 100
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 101
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 102
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 103
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 104
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 105
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 106
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 107
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 108
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 109
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 110
11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 111
11gR2 RDBMS software installation            How to manage the pre            11gR2 databases under            11gR2 GIH?P...
11gR2 RDBMS software installation        PINNING NODES        You need to PIN the nodes in order to run your pre 11gR2 dat...
What will be covered            o    Why upgrade?            o    Upgrade compatibility matrix.            o    Upgrade pa...
Database upgrade – Caution                                      Ensure a valid database backup                            ...
Database upgrade – upgrade paths                                            UPGRADE                    SQL> @catupgrade.sq...
Database upgrade            For a smooth rise, follow the best practice, run            @OH_11G/rdbms/admin/utlu112i.sql  ...
Database upgrade            As a best practice, run @OH_11G/rdbms/admin/utlu112i.sql            **************************...
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 119
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 120
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 121
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 122
Database upgrade                                                                Listener will                             ...
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 124
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 125
Database upgrade               Note 453903.1 - Enabling and Disabling Oracle Database Vault in UNIX               Note 453...
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 127
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 128
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 129
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 130
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 131
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 132
Database upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 133
What will be covered            o    Why upgrade?            o    Upgrade compatibility matrix.            o    Upgrade pa...
Database post upgrade – list     Run @utlrp.sql to compile ANY invalid objects.     Run @utlu112s.sql to check post upgr...
Database upgrade – measuring upgradePresented by : Syed Jaffer Hussain   red-gate/AllThingsOracle   Slide # 136
Database upgrade – trivia   Upgraded over 100 databases successfully using     MANUAL and DBUA methods.   Avg. database ...
Database upgrade – CLI upgrade stepsCourtesy from Oracle CorporationThanks to Roy Swonger & Mike DietrichPresented by : Sy...
What will be covered            o    Why upgrade?            o    Upgrade compatibility matrix.            o    Upgrade pa...
Upgrade aftermath stories          Snapshot Controlfile must be placed on a shared storage for a RAC database.          ...
Very important, apply the latest PSU patch, right after        your upgrade.Presented by : Syed Jaffer Hussain   red-gate/...
What will be covered            o    Why upgrade?            o    Upgrade compatibility matrix.            o    Upgrade pa...
References       Oracle Database Upgrade Path Reference List [ID 730365.1]       Complete Checklist for Manual Upgrades ...
A big thank to red-gate                                           and you all                                         for ...
Upcoming SlideShare
Loading in …5
×

Upgrading to Oracle 11gR2

4,391 views

Published on

A step-by-step procedure to upgrade clusterware followed by the databases from 10gR2 to 11gR2

Published in: Technology, Business
  • Assalam alaikum Syed
    Very helpful document for upgrading the RAC systems,
    I am planning to upgrade our 10gR2 RAC to 11gR2 RAC where we don't have ASM for our 10gR2 RAC,Kindly share me the document for upgrading 10gR2 Non-ASM RAC to 11gR2 ASM RAC.
    Jazakallah Khair
    Ali
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Upgrading to Oracle 11gR2

  1. 1. Do I actually need to care about upgrading my existing environment to race with Oracle latest release? Is it mandatory?Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 1
  2. 2. Does it really matter, which version of Oracle do we use?Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 2
  3. 3. Upgrading to Oracle 11gR2 A Successful case studyPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 3
  4. 4. We really had a roller coaster journey….. A successful upgrade A successful downgrade De-installed failed GI home before start over Manual upgrade Tried out of the box options….Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 4
  5. 5. Disclaimer This views/content in this slides are those of the author and do not necessarily reflect that of Oracle Corporation and/or its affiliates/subsidiaries. The material in this document is for informational purposes only and is published with no guarantee or warranty, express or implied.. This material should not be reproduced or used without the authors written permission.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 5
  6. 6. Foreword The objective of this presentation is to explain you the best practices that we had adopted/followed/practiced to upgrade successfully our FOUR Oracle 10.2.0.4 clusterware environments with over 100 databases to Oracle 11g R2 on HPUX Servers.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 6
  7. 7. What will be covered o Why upgrade? o Upgrade compatibility matrix. o Upgrade pattern [best practices]. o Overview about our environment. o Preparing to upgrade. o Clusterware and ASM upgrade. o 11gR2 RDBMS software installation. o Database upgrade. o Post upgrade steps. o Upgrade aftermath stories. o References.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 7
  8. 8. Know your presenter Syed Jaffer Hussain Database Support Manager Over 20 years IT experience 12 years as an Oracle DBA Oracle ACE Director Oracle 10g Certified Master(OCM) Oracle 10g RAC Certified Expert OCP v8i,9i,10g & 11g ITIL v3 Foundation Certified Authored Oracle 11g R1/R2 Real Application Clusters Essentials Twitter: @sjaffarhussain http://jaffardba.blogspot.comPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 8
  9. 9. Know your presenter Technologist of the Year, DBA 2011 http://www.oracle.com/technetwork/issue-archive/2012/12- jan/o12awards-tech-1403083.htmlPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 9
  10. 10. What will be covered o WHY upgrade? o Upgrade compatibility matrix. o Upgrade pattern [best practices]. o Overview of our setup. o Preparing to upgrade. o Clusterware and ASM upgrade. o 11gR2 RDBMS software installation. o Database upgrade. o Post upgrade steps. o Upgrade aftermath stories. o References.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 10
  11. 11. Why upgrade? – A burning question Do we really need to care which version are we using? Is it mandatory to upgrade an existing environment with every Oracle new release?Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 11
  12. 12. Why upgrade?Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 12
  13. 13. Why upgrade? To best of my knowledge the following situations may contributes to oppose an upgrade:  Legacy application.  Doesn’t want to take the risk.  Lack of skills and support.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 13
  14. 14. Why upgrade? Situations that are likely to invite you to embrace for an upgrade:  Organization standards.  Policy standards by Oracle for the old releases.  Tantalizing new features.  Bug fixes.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 14
  15. 15. What will be covered o Why upgrade? o Upgrade compatibility MATRIX. o Upgrade pattern [best practices]. o Overview of our setup. o Preparing to upgrade. o Clusterware and ASM upgrade. o 11gR2 RDBMS software installation. o Database upgrade. o Post upgrade steps. o Upgrade aftermath stories. o References.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 15
  16. 16. Upgrade compatibility matrix Courtesy from Oracle Corporation Thanks to Roy Swonger & Mike DietrichPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 16
  17. 17. Upgrade compatibility matrix Courtesy from Oracle Corporation Thanks to Roy Swonger & Mike DietrichPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 17
  18. 18. Upgrade compatibility matrix Minimum Version of the database that can be directly upgraded to Oracle 11g Release 2 Source Database Target Database 9.2.0.8 or higher 11.2.x 10.1.0.5 or higher 11.2.x 10.2.0.2 or higher 11.2.x 11.1.0.6 or higher 11.2.x The following database version will require an indirect upgrade path. Source Database ---> Upgrade Path for Target Database--->Target Database 7.3.3 (or lower)-----> 7.3.4 ---> 9.2.0.8 ---->11.2.x 8.0.5 (or lower)----> 8.0.6 ---> 9.2.0.8 ---->11.2.x 8.1.7 (or lower)----> 8.1.7.4---> 10.2.0.4---->11.2.x 9.0.1.3 (or lower)----> 9.0.1.4-- ->10.2.0.4---->11.2.x 9.2.0.7(or lower)---->9.2.0.8---->11.2.xPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 18
  19. 19. What will be covered o Why upgrade? o Upgrade compatibility matrix. o Upgrade pattern [BEST PRACTICE]. o Overview of our setup. o Preparing to upgrade. o Clusterware and ASM upgrade. o 11gR2 RDBMS software installation. o Database upgrade. o Post upgrade steps. o Upgrade aftermath stories. o References.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 19
  20. 20. Upgrade pattern [BEST PRACTICE] UPGRADE Clusterware + ASMPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 20
  21. 21. Upgrade pattern [best practice] UPGRADE INSTALL Clusterware RDBMS + ASM SoftwarePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 21
  22. 22. Upgrade pattern [best practice] UPGRADE INSTALL UPGRADE Clusterware RDBMS databases + ASM SoftwarePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 22
  23. 23. Upgrade pattern [best practice] UPGRADE INSTALL UPGRADE Post Clusterware RDBMS databases upgrade + ASM SoftwarePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 23
  24. 24. What will be covered o Why upgrade? o Upgrade compatibility matrix. o Upgrade pattern [best practices]. o Overview about our environment. o Preparing to upgrade. o Clusterware and ASM upgrade. o 11gR2 RDBMS software installation. o Database upgrade. o Post upgrade steps. o Upgrade aftermath stories. o References.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 24
  25. 25. Overview about our environment CISCO Switch HP Superdome NetBackup 6.5.6 Intel(R) Itanium 2 9000 series EMC Symmetrix DMX-4Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 25
  26. 26. Overview about our environment Production 1 Production 2 8 nodes 6 nodes Non-Production 1 Non-Production 2 7 nodes 3 nodesPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 26
  27. 27. Overview about our environment Production 1 Production 2 8 nodes 6 nodes •31 databases •4 logical processors •11 databases •3 logical processors •51 instances •22GB Memory •20 instances •12GB Memory Non-Production 1 Non-Production 2 7 nodes 3 nodes •60 databases •3 logical processors •12 databases •3 logical processors •80 instances •16GB Memory •20 instances •12GB MemoryPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 27
  28. 28. Overview about our environment The framework that we have followed for the upgrade V11.31 V11.23 /var/opt /var/opt HP UX upgrade from V11.23 TO v11.31Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 28
  29. 29. Overview about our environment The framework that we have followed for the upgrade V11.31 V11.23 /var/opt /var/opt HP UX upgrade from EMC Storage V11.23 TO v11.31 Configuration updatePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 29
  30. 30. Overview of our environment The framework that we have followed for the upgrade 10.2.0.4 11.2.0.2 V11.31 V11.23 /var/opt /var/opt HP UX upgrade from EMC Storage Clusterware V11.23 TO v11.31 Configuration Databases upgrade upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 30
  31. 31. What will be covered o Why upgrade? o Upgrade compatibility matrix. o Upgrade pattern [best practices]. o Overview of our setup. o Preparing to upgrade. o Clusterware and ASM upgrade. o 11gR2 RDBMS software installation. o Database upgrade. o Post upgrade steps. o Upgrade aftermath stories. o References.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 31
  32. 32. What will be covered Preparation is the headstone to every successful installation or upgrade projects.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 32
  33. 33. Prepare for upgrade – prerequisites checks  Verify: o OS certification for Oracle 11gR2. o OS kernel, packages. o Additional OS groups. o Oracle base location. o GIH location.  Must set GI home different from OB.  Configure Single Client Access Name(SCAN) o Maximum of 3 IP’s (same across nodes). o DNS round robin resolution. o Must be on the subnet as Public and VIP IP’s. o Not recommended to configure in HOSTS file.  root user access  UNSET the following o ORACLE_BASE o ORACLE_HOME o ORACLE_CRS_HOME o TNS_ADMIN o ORA_NLS10Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 33
  34. 34. Prepare for upgrade – prerequisites checks  Space: o Additional space for Oracle software home, typically /u00 o Enough /tmp space  Back up: o Back up the Cluster and Oracle homes. o Back up Voting disk. o Back up OCR.  CRS version: o Remember, the crs active and software version must be the same. crsctl query crs activeversion crsctl query crs softwareversion  Nodes readiness: runcluvfy.sh stage –pre crsinst –n nodes_list –fixup –fixupdir <dirname> -verbosePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 34
  35. 35. Prepare for upgrade – prerequisites checks Verify nodes readiness for upgrade: While upgrading from pre 11gR2, use the following syntax runcluvfy.sh stage –pre crsinst –n nodes_list –fixup –fixupdir <dirname> -verbose While upgrading 11.2.0.1 to 11.2.0.2 or higher, use the following syntax runcluvfy.sh stage –pre crsinst -upgrade –n nodes_list –rolling -scr_crshome existing_GI_HOME –dest_crshome new_GI_HOME -dest_version 11.2.0.3.0 –fixup –fixupdirpath <dirname> -verbosePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 35
  36. 36. What will be covered o Why upgrade? o Upgrade compatibility matrix. o Upgrade pattern [best practices]. o Overview of our setup. o Preparing to upgrade. o Clusterware and ASM upgrade. o 11gR2 RDBMS software installation. o Database upgrade. o Post upgrade steps. o Upgrade aftermath stories. o References.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 36
  37. 37. Clusterware, ASM upgrade – What’s new? Grid Infrastructure Clusterware Automatic Storage Management (ASM) Out-of-Place and In-place upgrades New GI homePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 37
  38. 38. Clusterware, ASM upgrade Is it necessary to upgrade Clusterware AND ASM together?Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 38
  39. 39. Clusterware, ASM upgrade Clusteware + ASM = no rolling upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 39
  40. 40. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 40
  41. 41. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 41
  42. 42. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 42
  43. 43. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 43
  44. 44. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 44
  45. 45. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 45
  46. 46. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 46
  47. 47. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 47
  48. 48. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 48
  49. 49. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 49
  50. 50. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 50
  51. 51. Clusterware and ASM upgrade ORACLE BASE must be different from the SW location……Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 51
  52. 52. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 52
  53. 53. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 53
  54. 54. Clusterware and ASM upgrade Cause : False warning generated by unpublished CVU bug 10024549 Solution: The warning can be ignoredPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 54
  55. 55. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 55
  56. 56. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 56
  57. 57. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 57
  58. 58. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 58
  59. 59. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 59
  60. 60. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 60
  61. 61. Clusterware and ASM upgrade Can I run the rootupgrade.sh script in PARALLEL across nodes?Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 61
  62. 62. Clusterware and ASM upgrade Can I run the rootupgrade.sh script in PARALLEL across nodes? The answer is YES, however….Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 62
  63. 63. Clusterware and ASM upgrade Always run the script on the LOCAL node first.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 63
  64. 64. Clusterware and ASM upgrade Always run the script on the LOCAL node first. Run the script in parallel on all nodes, except the LAST node.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 64
  65. 65. Clusterware and ASM upgrade Always run the script on the LOCAL node first. Run the script in parallel on all nodes, except the LAST node. Run the script on the LAST node.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 65
  66. 66. Clusterware and ASM upgrade rootupgrade.sh Shuts down the existing clusterware stack. Brings up the new version clusterware stack. Up on successfully running the script on the LAST node, the CRS active/release version will set to the new version.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 66
  67. 67. Clusterware and ASM upgrade Average rootupgrade.sh execution time on each node was just about 15 min. Took more time on couple nodes, mostly while stopping the existing cluster stack. Look closely for the logs under /u00/app/11.2.0/grid/cfgtoollogs.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 67
  68. 68. Clusterware and ASM upgrade Average rootupgrade.sh execution time on each node was just about 15 min. Took more time on couple nodes, mostly while stopping the existing cluster stack. Look closely for the logs under /u00/app/11.2.0/grid/cfgtoollogs.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 68
  69. 69. Clusterware and ASM upgrade rootupgrade.sh execution failed?Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 69
  70. 70. Clusterware and ASM upgrade How to Proceed from Failed Upgrade to 11gR2 Grid Infrastructure on Linux/Unix MOS [ID 969254.1] How to Proceed from Failed Upgrade to 11gR2 Grid Infrastructure on Linux/Unix OUI Failure Grid Infrastructure Cluster rootupgrade.sh Failure Grid Infrastructure Standalone rootupgrade.sh Failure OUI Plug-in Failure Old Clusterware Home Removal After Upgrade Patching Pre-upgrade Clusterware Home Scalability RAC Community ReferencesPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 70
  71. 71. Clusterware and ASM upgrade Use the –force flag to force the upgrade to complete in case some nodes becomes unreachable whilst performing an upgrade. /u01/app/11.2.0/grid/rootupgrade –force (>=11.2.0.2 only)Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 71
  72. 72. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 72
  73. 73. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 73
  74. 74. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 74
  75. 75. Clusterware and ASM upgrade Cried for HELP!Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 75
  76. 76. Clusterware and ASM upgrade Internet Cried for HELP!Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 76
  77. 77. Clusterware and ASM upgrade Internet MOS Cried for HELP!Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 77
  78. 78. Clusterware and ASM upgrade Internet MOS Cried for HELP! ASM alert.log, trace files & log files Doesn’t help much!Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 78
  79. 79. Clusterware and ASM upgrade Life Savior MOS [ID 398374.1]Presented by : Syed Jaffer Hussain Slide # 79
  80. 80. Clusterware and ASM upgrade cd $ORA_ASM_HOME/log/hostname/racg/ora.<hostname>.ASM.asm.log SQL> ORACLE instance shut down. SQL> Disconnected [PRSR][1]prsr_get_enabled_key: OCR api procr_open_key failed for key DATABASE.ASM.node01.+asm1.ENABLED with 4 [ PRSR][1]OCR error code : 4 [ PRSR][1]OCR error msg : PROC-4: The cluster registry key to be operated on does not exist. 2009-06-23 16:12:26.332: [ RACG][1] [28830][1][ora.node01.ASM1.asm]: prsr_fetch_asm_config() error, rc = 8 2009-06-23 16:12:27.443: [ RACG][1] [28830][1][ora.node01.ASM1.asm]: SQL*Plus: Release 10.2.0.4.0 - Production on Tue Jun 23 16:12:26 2009 Copyright (c) 1982, 2007, Oracle. All Rights Reserved. Enter user-name: Connected to an idle instance. SQL> ORA-01506: missing or illegal database name SQL> DisconnectedPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 80
  81. 81. Clusterware and ASM upgrade What might have gone wrong???  ASM instance name registered as ASMn instead of +ASMn in the OCR.  Could be due to a manual entry.  GI is very particular about + in contrast to previous cluster version.  Bug 5644904: WRONG ASM KEY PROC-4: THE CLUSTER REGISTRY KEY TO BE OPERATED ON DOES NOT EXISTPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 81
  82. 82. Clusterware and ASM upgrade Remedy  Stop ASM and dependent instances.  Remove dependency on ASM instance.  Remove ASM from OCR.  Add ASM back in with ‘+’ correct name  Add dependency to ASM.  Startup ASM.  Resume the failed step.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 82
  83. 83. Clusterware and ASM upgrade In action  srvctl stop asm –n <hostname>  srvctl modify instance –d DBNAME –i <instance_name> -f [do it for all active instances on the node]  srvctl remove asm –n <hostname> -i <ASM_INSTANCE1>  srvctl add asm –n <hostname> -i <+ASM_INSTANCE1> -o <ORA_ASM_HOME>  srvctl modify instance –d <DBNAME> -i <instance_name> -s +ASM_INSTANCE1 [do it for all active instances on the node]  srvctl start asm –n <hostname>Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 83
  84. 84. Clusterware and ASM upgrade In action  srvctl stop asm –n <hostname>  srvctl modify instance –d DBNAME –i <instance_name> -f [do it for all active instances on the node]  srvctl remove asm –n <hostname> -i <ASM_INSTANCE1>  srvctl add asm –n <hostname> -i <+ASM_INSTANCE1> -o <ORA_ASM_HOME>  srvctl modify instance –d <DBNAME> -i <instance_name> -s +ASM_INSTANCE1 [do it for all active instances on the node]  srvctl start asm –n <hostname> Repeat on all nodes!!!Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 84
  85. 85. Clusterware and ASM upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 85
  86. 86. Clusterware and ASM upgrade Post upgrade checks – active/software version #crsctl query crs activeversion Oracle Clusterware active version on the cluster is [11.2.0.2.0] # crsctl query crs releaseversion Oracle High Availability Services release version on the local node is [11.2.0.2.0]Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 86
  87. 87. Clusterware and ASM upgrade Post upgrade checks – active/software version #crsctl query crs activeversion Oracle Clusterware active version on the cluster is [11.2.0.2.0] # crsctl query crs releaseversion Oracle High Availability Services release version on the local node is [11.2.0.2.0] Ensure active/release version reflect to 11.2.0.2 for a successful upgrade.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 87
  88. 88. Clusterware and ASM upgrade Post upgrade checks – cluster stats across nodes # crsctl check cluster -all ************************************************************** node01: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** node02: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online ************************************************************** node03: CRS-4537: Cluster Ready Services is online CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online **************************************************************Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 88
  89. 89. Clusterware and ASM upgrade Post upgrade checks – query voting disk # ocrcheck Status of Oracle Cluster Registry is as follows : Version : 3 Total space (kbytes) : 1051708 Used space (kbytes) : 27048 Available space (kbytes) : 1024660 ID : 1837247566 Device/File Name : /dev/rdsk/oracle/ocr/ln1/ora_ocr_001 Device/File integrity check succeeded Device/File Name : /dev/rdsk/oracle/ocr/ln1/ora_ocr_002 Device/File integrity check succeeded Up to FIVE Device/File not configured OCR files Device/File not configured Device/File not configured Cluster registry integrity check succeeded Logical corruption check bypassed due to non-privileged userPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 89
  90. 90. Clusterware and ASM upgrade Post upgrade checks – query voting disk # crsctl query css votedisk ## STATE File Universal Id File Name Disk group -- ----- ----------------- --------- --------- 1. ONLINE 3ee7c55614f24fdcfff130e1abf94384 (/dev/rdsk/oracle/vote/ln1/ora_vote_001) [] 2. ONLINE a2c4e40a31f97f4bff6d65d558d03be8 (/dev/rdsk/oracle/vote/ln1/ora_vote_002) [] 3. ONLINE 1c9ed71914a54f9bbf55dd1c86048628 (/dev/rdsk/oracle/vote/ln1/ora_vote_003) []Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 90
  91. 91. Clusterware and ASM upgrade Post upgrade checks – b/g and daemon processes #ps -ef |grep d.bin oracle 14458 1 0 15:14:12 ? 0:41 /u00/app/11.2.0/grid/bin/oraagent.bin oracle 14462 1 0 15:14:13 ? 0:00 /u00/app/11.2.0/grid/bin/mdnsd.bin root 14539 1 0 15:14:28 ? 0:24 /u00/app/11.2.0/grid/bin/cssdagent oracle 14549 1 0 15:14:29 ? 0:01 /u00/app/11.2.0/grid/bin/diskmon.bin -d -f root 14541 1 0 15:14:28 ? 4:17 /u00/app/11.2.0/grid/bin/orarootagent.bin root 14732 1 0 15:15:48 ? 7:35 /u00/app/11.2.0/grid/bin/crsd.bin reboot oracle 14469 1 0 15:14:15 ? 0:11 /u00/app/11.2.0/grid/bin/gpnpd.bin oracle 14739 1 0 15:15:49 ? 3:21 /u00/app/11.2.0/grid/bin/evmd.bin oracle 14535 1 0 15:14:25 ? 2:51 /u00/app/11.2.0/grid/bin/gipcd.bin oracle 14545 1 0 15:14:29 ? 4:42 /u00/app/11.2.0/grid/bin/ocssd.bin root 20675 1 0 15:36:03 ? 4:28 /u00/app/11.2.0/grid/bin/orarootagent.bin root 14595 1 0 15:14:50 ? 3:06 /u00/app/11.2.0/grid/bin/octssd.bin root 13999 1 0 15:13:20 ? 2:47 /u00/app/11.2.0/grid/bin/ohasd.bin reboot root 14533 1 0 15:14:24 ? 0:26 /u00/app/11.2.0/grid/bin/cssdmonitor oracle 20671 1 0 15:36:03 ? 0:08 /u00/app/11.2.0/grid/bin/appagent.bin oracle 20685 1 0 15:36:03 ? 0:26 /u00/app/11.2.0/grid/bin/oraagent.bin #ps -ef |grep init.d root 66 0 0 Feb 9 ? 0:00 pagetable_init_daemon oracle 8645 15234 0 19:31:02 pts/0 0:00 grep init.d root 13977 1 0 15:13:19 ? 0:00 /bin/sh /sbin/init.d/init.ohasd runPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 91
  92. 92. Clusterware and ASM upgrade Remember  You can easily resume/downgrade from a failed or successful upgrade.  Can fall back to the pre 11gR2 version.  You hardly need a 3-step procedure to do that.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 92
  93. 93. Remove OLD homes Removing OLD cluster home It’s a best practice to keep the OLD_HOME for a while after a successful Upgrade. The following procedure can be used to remove the OLD_HOMES. export ORACLE_HOME=$OLD_HOME ## detach OLD_HOME $OLD_HOME/oui/bin/runInstaller -detachHome -silent -local ORACLE_HOME=$OLD_HOME #confirm $OLD_HOME is removed from central inventory: $OLD_HOME/OPatch/opatch lsinventory -all ## remove files in OLD_HOME manually: /bin/rm -rf $OLD_HOME unset ORACLE_HOMEPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 93
  94. 94. What will be covered o Why upgrade? o Upgrade compatibility matrix. o Upgrade pattern [best practices]. o Overview of our setup. o Prepare for upgrade. o Clusterware and ASM upgrade. o 11gR2 RDBMS software installation. o Database upgrade. o Post upgrade steps. o Upgrade aftermath stories. o References.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 94
  95. 95. 11gR2 RDBMS software installation Verify the READINESS of the nodes for DB software installation cluvfy stage -pre dbinst -n node01,node02,node03 -d $ORACLE_HOME –verbosePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 95
  96. 96. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 96
  97. 97. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 97
  98. 98. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 98
  99. 99. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 99
  100. 100. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 100
  101. 101. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 101
  102. 102. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 102
  103. 103. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 103
  104. 104. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 104
  105. 105. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 105
  106. 106. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 106
  107. 107. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 107
  108. 108. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 108
  109. 109. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 109
  110. 110. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 110
  111. 111. 11gR2 RDBMS software installationPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 111
  112. 112. 11gR2 RDBMS software installation How to manage the pre 11gR2 databases under 11gR2 GIH?Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 112
  113. 113. 11gR2 RDBMS software installation PINNING NODES You need to PIN the nodes in order to run your pre 11gR2 databases UNDER The Grid Infrastructure umbrella. root@node01: /u00/app/11.2.0/grid/bin # ./crsctl pin css -n node01 node02 node03 CRS-4664: Node node01 successfully pinned. CRS-4664: Node node02 successfully pinned. CRS-4664: Node node03 successfully pinned.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 113
  114. 114. What will be covered o Why upgrade? o Upgrade compatibility matrix. o Upgrade pattern [best practices]. o Overview of our setup. o Prepare for upgrade. o Clusterware and ASM upgrade. o 11gR2 RDBMS software installation. o Database upgrade. o Post DB upgrade steps. o Upgrade aftermath stories. o References.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 114
  115. 115. Database upgrade – Caution Ensure a valid database backup BEFORE you kick off the db upgrade procedure.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 115
  116. 116. Database upgrade – upgrade paths UPGRADE SQL> @catupgrade.sql • RAC aware and most recommended for RAC dbs. • Also can be run in –silent mode.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 116
  117. 117. Database upgrade For a smooth rise, follow the best practice, run @OH_11G/rdbms/admin/utlu112i.sql ********************************************************************** Tablespaces: [make adjustments in the current environment] ********************************************************************** --> SYSTEM tablespace is adequate for the upgrade. .... minimum required size: 788 MB --> UNDOTBS1 tablespace is adequate for the upgrade. .... minimum required size: 43 MB --> SYSAUX tablespace is adequate for the upgrade. .... minimum required size: 566 MB --> TEMP tablespace is adequate for the upgrade. .... minimum required size: 61 MB PURGE DBA_RECYCLEBIN .... must be executed immediately prior to executing your upgrade. . ********************************************************************** Recommendations ********************************************************************** Oracle recommends gathering dictionary statistics prior to upgrading the database. To gather dictionary statistics execute the following command while connected as SYSDBA: EXECUTE dbms_stats.gather_dictionary_stats;Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 117
  118. 118. Database upgrade As a best practice, run @OH_11G/rdbms/admin/utlu112i.sql ********************************************************************** Components: [The following database components will be upgraded or installed] ********************************************************************** --> Oracle Catalog Views [upgrade] VALID --> Oracle Packages and Types [upgrade] VALID --> JServer JAVA Virtual Machine [upgrade] VALID --> Oracle XDK for Java [upgrade] VALID --> Real Application Clusters [upgrade] VALID --> Oracle Workspace Manager [upgrade] VALID --> OLAP Analytic Workspace [upgrade] VALID --> OLAP Catalog [upgrade] VALID --> EM Repository [upgrade] VALID --> Oracle Text [upgrade] VALID --> Oracle XML Database [upgrade] VALID --> Oracle Java Packages [upgrade] VALID --> Oracle interMedia [upgrade] VALID --> Spatial [upgrade] VALID --> Data Mining [upgrade] VALID --> Expression Filter [upgrade] VALID --> Rule Manager [upgrade] VALID --> Oracle OLAP API [upgrade] VALIDPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 118
  119. 119. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 119
  120. 120. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 120
  121. 121. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 121
  122. 122. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 122
  123. 123. Database upgrade Listener will be created on all nodesPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 123
  124. 124. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 124
  125. 125. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 125
  126. 126. Database upgrade Note 453903.1 - Enabling and Disabling Oracle Database Vault in UNIX Note 453902.1 - Enabling and Disabling Oracle Database Vault in WINDOWS $ cd $ORACLE_HOME/rdbms/lib $ make -f ins_rdbms.mk dv_off $ cd $ORACLE_HOME/bin $ relink all (For RAC installations, run these commands on all nodes.) #cd /u00/app/oracle/product/11.2.0/db_1/rdbms/lib # make -f ins_rdbms.mk dv_off /usr/ccs/bin/ar d /u00/app/oracle/product/11.2.0/db_1/rdbms/lib/libknlopt.a kzvidv.o /usr/ccs/bin/ar cr /u00/app/oracle/product/11.2.0/db_1/rdbms/lib/libknlopt.a /u00/app/oracle/product/11.2.0/db_1/rdbms/lib/kzvndv.oPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 126
  127. 127. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 127
  128. 128. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 128
  129. 129. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 129
  130. 130. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 130
  131. 131. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 131
  132. 132. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 132
  133. 133. Database upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 133
  134. 134. What will be covered o Why upgrade? o Upgrade compatibility matrix. o Upgrade pattern [best practices]. o Overview of our setup. o Prepare for upgrade. o Clusterware and ASM upgrade. o 11gR2 RDBMS software installation. o Database upgrade. o Post DB upgrade steps. o Upgrade aftermath stories. o References.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 134
  135. 135. Database post upgrade – list  Run @utlrp.sql to compile ANY invalid objects.  Run @utlu112s.sql to check post upgrade components status.  Adjust time zone data in the database.  Gather statistics for system and fixed objects.  Amend scripts [if any] to affect the new OH settings.  Verify the database upgrade srvctl config database –d <DBNAME>  Back up the database.  Modify the COMPATIBLE init. Parameter to 11gR2.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 135
  136. 136. Database upgrade – measuring upgradePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 136
  137. 137. Database upgrade – trivia  Upgraded over 100 databases successfully using MANUAL and DBUA methods.  Avg. database upgrade time was just about 2 hrs.  Upgrade using DBUA for RAC DB proved a wise decision.  Nearly identical duration between CLI and DBUA upgrade methods.  Duration is typically proportional to the number of installed database components.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 137
  138. 138. Database upgrade – CLI upgrade stepsCourtesy from Oracle CorporationThanks to Roy Swonger & Mike DietrichPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 138
  139. 139. What will be covered o Why upgrade? o Upgrade compatibility matrix. o Upgrade pattern [best practices]. o Overview of our setup. o Prepare for upgrade. o Clusterware and ASM upgrade. o 11gR2 RDBMS software installation. o Database upgrade. o Post DB upgrade steps. o Upgrade aftermath stories. o References.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 139
  140. 140. Upgrade aftermath stories  Snapshot Controlfile must be placed on a shared storage for a RAC database.  ORA-24247: network access denied by access control list (ACL) Solution: UTL_TCP, UTL_HTTP, UTL_SMTP, UTL_MAIL is used grant the privileges explicitly DBMS_NETWORK_ACL_ADMIN.CREATE_ACL ADD_PRIVILEGE ASSIGN_ACL  ASM instance resource status MIGHT turn into INTERMEDIATE with CHECK TIMED OUT state_details [BUG 11807012] Solution: crsctl modify resource “ora.asm” –attr “CHECK_TIMEOUT=132” MOS [ID 1323679.1]  IMPDP fails with ORA-29913, 31011, 19202, LPX-00217 [BUG 11877267] Solution: no solution… workaround exists  ORA-07445: exception core dump [evaopn3()] Solution: _simple_view_merge, _complex_view_merging=falsePresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 140
  141. 141. Very important, apply the latest PSU patch, right after your upgrade.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 141
  142. 142. What will be covered o Why upgrade? o Upgrade compatibility matrix. o Upgrade pattern [best practices]. o Overview of our setup. o Prepare for upgrade. o Clusterware and ASM upgrade. o 11gR2 RDBMS software installation. o Database upgrade. o Post DB upgrade steps. o How to proceed from failed upgrade. o Upgrade aftermath stories. o References.Presented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 142
  143. 143. References  Oracle Database Upgrade Path Reference List [ID 730365.1]  Complete Checklist for Manual Upgrades to 11gR2 [ID 837570.1]  Complete checklist to upgrade the database to 11g R2 using DBUA [ID 870814.1]  Master Note for Real Application Clusters (RAC) Oracle Clusterware and Oracle Grid Infrastructure [ID 1096952.1]  How to Proceed from Failed Upgrade to 11gR2 Grid Infrastructure on Linux/Unix [ID 969254.1]  Oracle Grid Infrastructure 11.2.0.2 Installation or Upgrade may fail due to Multicasting Requirement [ID 1212703.1]  Do You Need to Relink Oracle Clusterware When Upgrading the Operating System? [ID 743649.1]  Oracle 11g R1/R2 Real Application Clusters Essentials  Move to Oracle Database 11g – The whole Story blogs.oracle.com/UPGRADE  http://download.oracle.com/docs/cd/E11882_01/server.112/e17222/toc.htm  http://www.oracle.com/technetwork/database/upgrade/index.html  http://oukc.oracle.com/static09/opn/core09/oracle9i_database/95960/060311_95960/index.htmPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 143
  144. 144. A big thank to red-gate and you all for listening ... You can write me at sjaffarhussain@gmail.comPresented by : Syed Jaffer Hussain red-gate/AllThingsOracle Slide # 144

×