10.2.0.5 Patch Set Apply on                     Oracle Clusterware,                    ASM and RDBMS on               2-No...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012Pre-installation Tasks   RACNODE1 and RACNODE2 have separate homes f...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012      (A) When patching from a lower 10.2 release or upgrading from ...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012   substr(object_type,1,30) type, status from   dba_objects where st...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012      rman target / log /tmp/before_upgrade_full_db.log<<EOF      ru...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012      sql alter system archive log current;      BACKUP AS COMPRESSE...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012      $ srvctl stop nodeapps –n node1      $ srvctl stop nodeapps –n...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012      (B) Select CRS_HOME      (C) Click next by default all the nod...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012      (D) Warning: Backup /etc/sysctl.conf and modify fs.file_max=10...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012      (F) stop crs as ROOT user; As ROOT user run CRS_HOME/install/r...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012   7. Issues faced during upgrade:-      - Prerequisite check failed...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012      (B) Select ASM homeRaheel H Syed                              ...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012      Click Next      Click NextRaheel H Syed                       ...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012      Click OK and      Run the script root.sh on node1 and then on ...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012      Click Exit   3. Error during upgrade:-      - When OUI was try...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012Upgrade Oracle Database   1. Start ASM on both the nodes.      $ srv...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012   10. $ srvctl start instance -d DB -i DB1 -o nomount   11. SQL> AL...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012   5. unzip p13343471_10205_Linux-x86-64_PSU_JAN2012.zip      $ cd 1...
10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012    Oracle® Database Patch Set Notes 10g Release 2 (10.2.0.5) Patch...
Upcoming SlideShare
Loading in …5
×

Upgrade 10204-to-10205 on-2-node_rac_linux_x86_64_detail-steps_v0.1

1,085 views

Published on

Published in: Technology, News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,085
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
40
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Upgrade 10204-to-10205 on-2-node_rac_linux_x86_64_detail-steps_v0.1

  1. 1. 10.2.0.5 Patch Set Apply on Oracle Clusterware, ASM and RDBMS on 2-Node RAC Linux x86_64 Version: V0.1ByRaheel H Syed10g OCP CertifiedBlog: http://raheeldba.wordpress.comTwitter: https://twitter.com/raheelhsyedLinkedIn: http://in.linkedin.com/pub/raheel-hashmi-syed/33/34a/226
  2. 2. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012Pre-installation Tasks RACNODE1 and RACNODE2 have separate homes for ASM and Oracle Home and CRS Home. All homes will be patched on both the nodes Existing CRS_HOME, ASM_HOME & ORACLE_HOME version is 10.2.0.4 +ASM1:/app/asm PROD1:/app/oracle/10.2.0/db_1 CRS home  /app/crs/home_1 1. Backup CRS, ASM, DB Home and inventory for both the nodes - Use tar to backup the homes as ‘ tar –cvf ’ 2. Check opatch lsinventory on CRS, ASM and Oracle Home. Save the output. 3. Check cluster health, crs activeversion and software version on both the nodes and save the output. $ crsctl check crs $ crsctl query css activeversion CRS active version on the cluster is [10.2.0.4.0] $ crsctl query css software version CRS software version on node [racnode1] is [10.2.0.4.0] 4. Upgrading Installations that use Java Virtual Machine or Oracle interMedia - If the database uses JVM or Oracle interMedia, then install the oracle database 10g Products installation type from 10g Companion CD before installing 10.2.0.5 Patchset. MOS ID: 293658.1 5. Update Oracle Time Zone DefinitionsRaheel H Syed 2
  3. 3. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 (A) When patching from a lower 10.2 release or upgrading from 10.1.0.x directly to 10.2.0.5, check the current version of the Oracle time zone definitions with the following command: SELECT version FROM v$timezone_file; >>> If this query reports version 4, no action is required. >>> If this reports a version lower or higher than 4, see My Oracle Support document 1086400.1 Actions for the DSTv4 update in the Release 10.2.0.5 patchset. (No Action required in our case as version is 4) 6. chmod -R +w CRS_HOME/inventory/Templates/* 7. Save all db links - Incase downgrade is required you can drop the db links and recreate after if downgrade is required spool dblinks_pre_upgrade.lst SELECT create ||DECODE(U.NAME,PUBLIC,public )||database link ||CHR(10) ||DECODE(U.NAME,PUBLIC,Null, U.NAME||.)|| L.NAME||chr(10) ||connect to || L.USERID || identified by ||L.PASSWORD|| using || L.host || ||chr(10)||; TEXT FROM sys.link$ L, sys.user$ U WHERE L.OWNER# = U.USER#; spool off 8. Check invalid objects spool invalid_pre_upgrade.lst select substr(owner,1,12) owner, substr(object_name,1,30) object,Raheel H Syed 3
  4. 4. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 substr(object_type,1,30) type, status from dba_objects where status <> VALID; spool off Run utlrp.sql and re-query invalid objects The last queries (invalid objects check) will lists all objects that cannot be compiled before upgrade in file invalid_pre.lst 9. Create pfile from spfile AND backup tnsnames.ora and listener.ora 10. Check AUD$ table, if auditing is enabled then upgrade performance will be affected SELECT count (*) FROM SYS.AUD$; count (*) --------- 0 MOS Id: 979942.1 (No action required in our case as NO auditing is enabled) 11. Verify database components, status and version from dba_registry. Save the output. set pages 500 set lines 120 col COMP_NAME for a40 select comp_name, status, version from dba_registry; 12. Backup the database. Using RMAN, backup the full database and archive log in compressed format to disk. Though we had tape backups configured, we opted for disk backups as backup completion time to tapes varied to 4-6 hours. Disk backup completed in 2 hours. #!/usr/bash export NLS_DATE_FORMAT="DD-MON-YY HH24:MI:SS"Raheel H Syed 4
  5. 5. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 rman target / log /tmp/before_upgrade_full_db.log<<EOF run { allocate channel ch001 device type disk; allocate channel ch002 device type disk; allocate channel ch003 device type disk; allocate channel ch004 device type disk; BACKUP AS COMPRESSED BACKUPSET DATAFILE 1,2,3,4,5,22,23,24 format /app/prod1/before_upgrade/%d_%s_%p_%c_%t.rmn; BACKUP AS COMPRESSED BACKUPSET DATAFILE 25,26,28,29,31,32,33,34 format /app/prod1/before_upgrade/ %d_%s_%p_%c_%t.rmn; BACKUP AS COMPRESSED BACKUPSET DATAFILE 35,36,37,39,40,42,43,44 format /app/prod1/before_upgrade/ %d_%s_%p_%c_%t.rmn; BACKUP AS COMPRESSED BACKUPSET DATAFILE 45,46,47,48,49,50,51,52 format /app/prod1/before_upgrade/ %d_%s_%p_%c_%t.rmn; BACKUP AS COMPRESSED BACKUPSET DATAFILE 53,54,55,56,58,59,60,61 format /app/prod1/before_upgrade/ %d_%s_%p_%c_%t.rmn; BACKUP AS COMPRESSED BACKUPSET DATAFILE 62,63,64,65,66,67,68,69 format /app/prod1/before_upgrade/ %d_%s_%p_%c_%t.rmn; BACKUP AS COMPRESSED BACKUPSET DATAFILE 70,71,72,73,74,75,76,77 format /app/prod1/before_upgrade/ %d_%s_%p_%c_%t.rmn; BACKUP AS COMPRESSED BACKUPSET DATAFILE 78,79,80,81,82,83,84,85 format /app/prod1/before_upgrade/ %d_%s_%p_%c_%t.rmn; BACKUP AS COMPRESSED BACKUPSET DATAFILE 6,7,8,9,10,11,12,13 format /app1/prod1/before_upgrade/%d_%s_%p_%c_%t.rmn; BACKUP AS COMPRESSED BACKUPSET DATAFILE 14,15,16,17,18,19,20,21 format /app1/prod1/before_upgrade/ %d_%s_%p_%c_%t.rmn; BACKUP AS COMPRESSED BACKUPSET DATAFILE 27,30,38 format /app1/prod1/before_upgrade/%d_%s_%p_%c_%t.rmn; backup as backupset format /app1/prod1/before_upgrade/%d_ %s_%p_%c_%t.ctl current controlfile tag=Before_upgrade_CTL; sql alter system archive log current; sql alter system archive log current;Raheel H Syed 5
  6. 6. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 sql alter system archive log current; BACKUP AS COMPRESSED BACKUPSET format /app1/prod1/before_upgrade/%d_%s_%p_%c_%t.arc archivelog all tag=Before_upgrade_ARCH; backup as backupset format /app1/prod1/before_upgrade/ %d_before_upgrade_AFTER_ARCH_%s_%p_%c_%t.ctl current controlfile tag=Before_upgrade_CTL; release channel ch001; release channel ch002; release channel ch003; release channel ch004; } quit EOF Note: I have explicitly specified datafiles for each compressed backupset, this is because there was not enough space at /app mount point. Database size 1230 GB and compressed backup size on disk 139 GB.Upgrade Clusterware 1. Resolve any outstanding unresolved distributed transactions: SQL> select * from dba_2pc_pending; If this returns rows then run the following SQL> select local_tran_id from dba_2pc_pending; SQL> execute dbms_transaction.purge_lost_db_entry (<input_id>); SQL> commit; 2. Disable the jobs that run from scheduler before upgrading the database.  Downtime starts here. Note: In our case all the production jobs were scheduled from DBMS_SCHEDULER hence we have disabled the jobs. 3. Stop database, ASM and nodeapps on both the nodes. $ srvctl stop database –d <> $ srvctl stop asm –n node1 $ srvctl stop asm –n node2Raheel H Syed 6
  7. 7. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 $ srvctl stop nodeapps –n node1 $ srvctl stop nodeapps –n node2 4. Download and unzip patch p8202632_10205_Linux-x86-64.zip 5. We have used VNC viewer to install this patch set, most of the Linux system comes with vnc service running on the system. Please check with your SA if it’s not running. To check if vnc service is running on the system or not, issue the following command $ ps –ef | grep vnc If the output shows …… oracle ….. <your hostname>:<number> then it means you have vnc session running under oracle user, eg: racnode1:1 or racnode1:2 If VNC service is not running then you can do this, it worked for me As ‘Oracle’ user: $ cd /usr/bin $ ls –lrt vnc* $ ./vncserver It will ask for password, provide the password and verify. 6. CD to Disk1 and Invoke runInstaller and select CRS HOME from the path. Follow the instructions as displayed on the screen. OR Execute the following to ignore pre-requisite checks of parameters (If required) $ ./runinstaller –ignoreSysPrereqs (A)Raheel H Syed 7
  8. 8. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 (B) Select CRS_HOME (C) Click next by default all the nodes in cluster are selected as belowRaheel H Syed 8
  9. 9. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 (D) Warning: Backup /etc/sysctl.conf and modify fs.file_max=101365 (see below) (E)Raheel H Syed 9
  10. 10. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 (F) stop crs as ROOT user; As ROOT user run CRS_HOME/install/root102.shRaheel H Syed 10
  11. 11. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 7. Issues faced during upgrade:- - Prerequisite check failed as fs.file-max = 655367, required value 101365. Asked SA Admin to backup the /etc/sysctl.conf and modify the parameter. - Error in writing to CRS_Home:/app/oracle/product/10.2.0/crs/OPatch Change permission as $ chmod +w opatch 8. The script will automatically start crs daemon on the patched node upon completion 9. crs_stat -t (or) crs_status -t on completed node 10. Proceed to node2 and repeat. This completes CRS upgrade to 10.2.0.5 11. Check crs softwareversion and activeversion on both the nodes. On successful upgrade crs softwareversion and activeversion should be at 10.2.0.5 $ crsctl check crs softwareversion $ crsctl check crs activeversionUpgrade ASM and Oracle Home 1. Stop database, ASM and nodeapps on both the nodes. $ srvctl stop database –d <> $ srvctl stop asm –n node1 $ srvctl stop asm –n node2 $ srvctl stop nodeapps –n node1 $ srvctl stop nodeapps –n node2 2. Invoke runInstaller and select ASM Home (ASM is separate home so we will have to upgrade ASM home first and then ORACLE_HOME.Raheel H Syed 11
  12. 12. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 (B) Select ASM homeRaheel H Syed 12
  13. 13. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 Click Next Click NextRaheel H Syed 13
  14. 14. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 Click OK and Run the script root.sh on node1 and then on node 2 as root user click OKRaheel H Syed 14
  15. 15. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 Click Exit 3. Error during upgrade:- - When OUI was trying to copy files in /app/oracle/product/10.2.0/db_1/jdk/bin/java (Text file busy) I have closed and started the vncviwer, that resolved the issue - Oracle Configuration Manager Configuration failed with oracle.sysman.ccr.configCCR.ConfigCCRPlugIn. Skip this step. 4. Execute root.sh as root on first node and then on second node. 5. This completes ASM Home upgrade to 10.2.0.5 6. $ASM_HOME/OPatch/opatch lsinventory. Save the output. 7. Once the ASM Home is upgraded, invoke runInstaller and select ORACLE HOME to upgrade ORACLE HOME. 8. $ORACLE_HOME/OPatch/opatch lsinventory. Save the outputRaheel H Syed 15
  16. 16. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012Upgrade Oracle Database 1. Start ASM on both the nodes. $ srvctl start asm -n node1 $ srvctl start asm -n node2 2. srvctl start instance -d DB -i DB1 -o nomount 3. SQL> ALTER SYSTEM SET CLUSTER_DATABASE=FALSE SCOPE=spfile; 4. Create pfile from spfile. Edit pfile and comment all obsolete parameters. (Eg: sql_trace) 5. srvctl stop instance -d DB -i DB1 My advice – Run the below steps 6 & 7 in vnc, even in worst case if connectivity is lost your vnc session is still running in background 6. As ‘Oracle’ User, check or set $ORACLE_HOME, $ORACLE_SID, $PATH $ sqlplus “/as sysdba” 7. SQL> startup upgrade SQL> spool upgrade_10205.log SQL> @?/rdbms/admin/catupgrd.sql SQL> spool off SQL> shut immediate SQL> startup SQL> @?/rdbms/admin/utlrp.sql 8. SQL> SELECT COMP_NAME, VERSION, STATUS FROM SYS.DBA_REGISTRY; 9. SQL> shut immediate;Raheel H Syed 16
  17. 17. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 10. $ srvctl start instance -d DB -i DB1 -o nomount 11. SQL> ALTER SYSTEM SET CLUSTER_DATABASE=TRUE SCOPE=spfile; 12. $ srvctl stop instance -d DB -i DB1 13. $ srvctl start database -d DB $ srvctl start nodeapps -n node1 $ srvctl start nodeapps -n node2 14. Health check on crs. $ sh crs_status.shApply PSU patch 13343471 - 10.2.0.5.6 to Oracle Home 1. PRODUCTION nodes RACNODE1 and RACNODE2 2. Have 2 separate homes for ASM and Oracle Home. 3. Apply OPatch patch 6880880 to apply 10.2.0.5.6. - Backup the existing OPatch directory and then unzip p6880880_102000_Linux- x86-64_Opatch_4_PSUJan2012.zip in the Oracle Home - $opatch version - Output should be like this: 10.2.0.5.1 4. Stop database, asm, listener and nodeapps on all the nodes $ srvctl stop database –d <> $ srvctl stop asm –n node1 $ srvctl stop asm –n node2 $ srvctl stop nodeapps –n node1 $ srvctl stop nodeapps –n node2 Before applying patch please read “Patch 13343471 - 10.2.0.5.6 Patch Set Update” readme.htmlRaheel H Syed 17
  18. 18. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012 5. unzip p13343471_10205_Linux-x86-64_PSU_JAN2012.zip $ cd 13343471 $ opatch apply 6. start database and listener. 7. cd $ORACLE_HOME/rdbms/admin 8. @catbundle.sql psu apply 9. @utlrp.sql 10. set pages 500 select * from dba_registry_history; Check the following log files in $ORACLE_HOME/cfgtoollogs/catbundle for any errors: catbundle_PSU_<database SID>_APPLY_<TIMESTAMP>.log catbundle_PSU_<database SID>_GENERATE_<TIMESTAMP>.log 11. Quick CRS check with custom scirpt: crs_status.sh 12. Enable all the jobs that were disabled in earlier step. 13. Running changePerm.sh Script on an Oracle Database Server Home 14. $ cd $ORACLE_HOME/install $ sh changePerm.sh and specify the patched server Oracle home locationReferencesRaheel H Syed 18
  19. 19. 10.2.0.5 PATCH APPLY ON_2-NODE_RAC_STEPS July 10, 2012  Oracle® Database Patch Set Notes 10g Release 2 (10.2.0.5) Patch Set 4 for Linux x86-64 Part Number E15229-01 - April 2010  Complete Checklist for Manual Upgrades to 10gR2 [ID 316889.1]  10.2.0.5 Patch Set - List of Bug Fixes by Problem Type [ID 1088172.1]  10.2.0.5 Patch Set - Availability and Known Issues [ID 1087991.1]  How to Log Good Service Requests for Database Upgrade/Migration Issues [ID 732897.1]  How to Check Oracle JVM is Installed in the Database [ID 397770.1]Raheel H Syed 19

×