Presentation 12c grid_upgrade

Basel Bern Lausanne Zurich Düsseldorf Frankfurt/M. Freiburg i. Br. Hamburg Munich Stuttgart Vienna
Upgrade 8 nodes 11GR2 cluster to 12c.
Everything going smoothly ?
Let’s see…

© 2013TechEvent 27 September 2013 2
Agenda
 Target 11GR2 infrastructure
 Pre-upgrade Checks and restrictions
 Run CVU to validate readiness
 Upgrade the GRID
 Upgrade the RDBMS
 Post upgrade task
 Q/A
Trivadis
Tech-Event
27 Sept 2013

Target 11GR2 infrastructure

OS is OEL 6.4 x64
3 server pools
 PMU: ACTIVE_SERVERS=node3 node5 node6 node8
 LUDO: ACTIVE_SERVERS=node1 node7 (rac one node)
 TST: ACTIVE_SERVERS=node2 node4 (rac one node)

NAME TARGET STATE SERVER STATE_DETAILS
-------------------------------------------------------------------
ora.ludo.db
1 ONLINE ONLINE node1 Open
ora.pmu.db
3 ONLINE OFFLINE Instance Shutdown
ora.vico.db

© 2013
TechEvent 27 September 2013 7
The apex application

Pre-upgrade Checks and restrictions

 Unset all environment variables
 $ unset ORACLE_BASE
 $ unset ORACLE_HOME
 $ unset ORACLE_SID
 $ unset TNS_ADMIN
 $ unset ORA_NLS10
 $ unset ORA_CRS_HOME (under oracle support)
 Also, ensure that the $ORACLE_HOME/bin path is
removed from your PATH environment variable.

 To upgrade existing Oracle Grid Infrastructure
installations from Oracle Grid Infrastructure 11g
Release 2 (11.2.0.2) to a later release, you must
apply patch 11.2.0.2.3 (11.2.0.2 PSU 3) or later.
 Back Up the Oracle Software Before Upgrades!

Run CVU to validate readiness

 CVU runs the appropriate system checks
automatically, and either prompts you to fix
problems, or provides a fixup script to be run on all
nodes in the cluster before proceeding with the
upgrade.
./runcluvfy.sh stage -pre crsinst -upgrade -n node1, node2,
node3, node4, node5, node6, node7, node8 -rolling -src_crshome
/oracle/grid/11.2.0.3 -dest_crshome /oracle/grid/12.1.0.0 -
dest_version 12.1.0.1.0 -fixup -verbose
dest_version must be: 12.1.0.1.0

 For my cluster the test failed because of an NTP
issue. Timing between servers was ok but the time
server was not the same.
 Solution for me was to set the same time server for
all nodes.
 An other option would have been to disable NTP
during the upgrade phase and let CTSS run in
active mode.
 I did not report other issues for the check.

NTP common Time Server Check started...
PRVF-5408 : NTP Time Server "94.23.228.130" is common only to the following nodes "node4"
PRVF-5408 : NTP Time Server "149.20.64.28" is common only to the following nodes "node8,node5"
PRVF-5408 : NTP Time Server "145.238.203.14" is common only to the following nodes
"node3,node8,node7,node6"
PRVF-5416 : Query of NTP daemon failed on all nodes

Upgrade the GRID

Upgrade the GRID
Run ./runInstaller

Upgrade the GRID

Upgrade the GRID
Initially I have decide to just upgrade without new functionalities

Upgrade the GRID
But to use some of them, you have to reinstall the grid 

Upgrade the GRID
So I choose YES 

Upgrade the GRID

Upgrade the GRID
Cool feature to group nodes to upgrade 

Upgrade the GRID

Upgrade the GRID
 11gR2 alertNode1.log
[ctssd(3025)]CRS-2405:The Cluster Time Synchronization Service on host node1 is shutdown by user
 12c alertNode1.log
[client(28956)]CRS-1808:Node-specific configuration for node node1 in Oracle Local Registry was
upgraded to version 12.1.0.1.0.
[ohasd(29129)]CRS-2112:The OLR service started on node node1.
[ohasd(29129)]CRS-1301:Oracle High Availability Service started on node node1.
...
[crsd(9720)]CRS-1012:The OCR service started on node node1.
2013-09-06 21:22:19.214:
[crsd(9720)]CRS-1201:CRSD started on node node1.
2013-09-06 21:22:44.713:
[client(9981)]CRS-1804:Node node1 upgraded to version 12.1.0.1.0. 

Upgrade the GRID
 Databases are automaticaly relocated to other nodes during the upgrade
thanks to policy managed
-------------------------------------------------------------------
ora.ludo.db
ora.pmu.db
4 ONLINE OFFLINE Instance Shutdown,STARTING
5 ONLINE OFFLINE Instance Shutdown,STARTING
ora.vico.db

Upgrade the GRID

Upgrade the GRID
Oups, what the hell going on!!  Let’s see the logs

Upgrade the GRID
Log says: +GRID does not have enough space. Required space is 3358 MB ,
available space is 161 MB!
I would have appreciated a message during prerequisite checks
Actions:
1. Added more disks to the GRID diskgroup
2. Retried the configuration
3. Failed again as the available space is not updated
1. So I skipped the configuration and close the installer
2. Checked the log for the command that needs to be run
3. Run the command(s) manually

Upgrade the GRID
-bash-4.1$ /oracle/grid/12.1.0.0/bin/dbca -silent -createDatabase -templateName
MGMTSeed_Database.dbc -sid -MGMTDB -gdbName _mgmtdb -storageType ASM -
diskGroupName GRID -datafileJarLocation /oracle/grid/12.1.0.0/assistants/dbca/templates -
characterset AL32UTF8 -autoGeneratePasswords -oui_internal
Copying database files
1% complete
..
35% complete
Creating and starting Oracle instance
37% complete
..
58% complete
Registering database with Oracle Grid Infrastructure
64% complete
Completing Database Creation
68% complete
..
100% complete
Look at the log file "/oracle/gridbase/cfgtoollogs/dbca/_mgmtdb/_mgmtdb.log" for further details.

Upgrade the GRID
-bash-4.1$ /oracle/grid/12.1.0.0/oui/bin/runConfig.sh ORACLE_HOME=/oracle/grid/12.1.0.0
MODE=perform ACTION=configure RERUN=false $*
Setting the invPtrLoc to /oracle/grid/12.1.0.0/oraInst.loc
perform - mode is starting for action: configure
perform - mode finished for action: configure
You can see the log file: /oracle/grid/12.1.0.0/cfgtoollogs/oui/configActions2013-09-07_01-35-58-PM.log
Warning: if you do not set ORACLE_SID=-MGMTDB the process will create a new database!
This manual installation step should normally be avoided if you make sure you have 5Gb free
in your GRID diskgroup.

Upgrade the GRID
 After the upgrade process some listeners failed to start and the post
upgrade check script failed of 4 nodes on the VIP section.
 If reboot the nodes one by one and everything was ok after that.
Post-check for cluster services setup was successful.
The plug-in Oracle Cluster Verification Utility has successfully been performed
The action configuration has successfully completed

Upgrade the GRID
At the end
You should see something like this in the alertlog!
[crsd(9720)]CRS-1001:The OCR was formatted using version 4.
[cssd(9016)]CRS-1626:A Configuration change request completed successfully
[cssd(9016)]CRS-1601:CSSD Reconfiguration complete. Active nodes are node1 node2
node3 node4 node5 node6 node7 node8 .
[crsd(9720)]CRS-1005:The Oracle Clusterware upgrade was completed. Version has
changed from [186647296, 11.2.0.3.0] to [202375424, 12.1.0.1.0].
Details in /oracle/grid/12.1.0.0/log/node1/crsd/crsd.log.

Upgrade the GRID
-------------------------------------------------------------------
ora.ludo.db
1 ONLINE ONLINE node5 Open,STABLE
ora.mgmtdb
ora.pmu.db
1 ONLINE OFFLINE Instance Shutdown,STABLE
ora.vico.db
Note: I have started to reboot from node8. The first node that is ok get the
mgmtdb management database.

Upgrade the RDBMS

Upgrade the RDBMS
 CVU runs the appropriate system checks automatically in
the RAC before proceeding with the upgrade.
 Before running CVU, make sure the dbsnmp account is
unlocked and the password is set. You can specify and other
account.
Run CVU:/oracle/db/12.1.0.0/bin/cluvfy stage -pre dbinst -
upgrade -src_dbhome /oracle/db/11.2.0.3 -dest_dbhome
/oracle/db/12.1.0.0 -dest_version 12.1.0.0.0 –verbose
…
ASM and CRS versions are compatible
Database version "12.1.0.0.0" is compatible with the
Clusterware version "12.1.0.1.0".
Pre-check for database installation was successful.

Upgrade the RDBMS
Run the installer and install “software only” across all nodes

Upgrade the RDBMS
Would have preferred to be prompt for root credentials…

Upgrade the RDBMS
 To upgrade a database, you must run the upgrade assistant
from a node where the database is running
Is Rolling Upgrade possible?
YES but using:
 Using Data Guard SQL Apply
 Using Physical Standby Databases and Transient Logical
Standby

Upgrade the RDBMS
Run dbua from the node where the target db is running

Upgrade the RDBMS

Upgrade the RDBMS
Script can be generated to fix up upgrade prerequisites 

Upgrade the RDBMS
Adjust some settings, the parallelism one is cool 

Upgrade the RDBMS
Check the summary…

Upgrade the RDBMS
Strange error as the oracle user has the right to do it. 

Upgrade the RDBMS
You can then check the Upgrade Result 

Upgrade the RDBMS
Important
It’s not possible to convert a 11gR2 database to a container
database.
To use this feature you have:
 Create a 12c container database
 Upgrade your 11gR2 databases to 12c
 Plug your upgraded 12c databases in the container database

© 2013
Upgrade the RDBMS
-------------------------------------------------------------------
ora.ludo.db
ora.mgmtdb
ora.pmu.db
ora.vico.db

Post upgrade tasks

Post upgrade tasks
Remove Grid 11gR2 home
 You must first change the permission and ownership of the
previous release Grid home
#chmod -R 755 /oracle/grid/11.2.0.3
#chown -R grid /oracle/grid/11.2.0.3
 Use the Oracle Grid Infrastructure 11g standalone deinstallation
tool to remove the previous release Grid home.
 And not the 12c one as stated in the documentation 
 Download it from here:
http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html

Post upgrade tasks
./deinstall -home /oracle/grid/11.2.0.3
####################### CHECK OPERATION SUMMARY #######################
Oracle Grid Infrastructure Home is: /oracle/grid/12.1.0.0
The cluster node(s) on which the Oracle home deinstallation will be performed
are:node1,node2,node3,node4,node5,node6,node7,node8
Oracle Home selected for deinstall is: /oracle/grid/11.2.0.3
Inventory Location where the Oracle home registered is: /oracle/oraInventory
The home being deconfigured is NOT a configured Grid Infrastructure home (/oracle/grid/12.1.0.0)
ASM was not detected in the Oracle Home
Do you want to continue (y - yes, n - no)? [n]: Y
…
####################### CLEAN OPERATION SUMMARY #######################
The home being deconfigured is NOT a configured Grid Infrastructure home (/oracle/grid/12.1.0.0)
Successfully detached Oracle home '/oracle/grid/11.2.0.3' from the central inventory on the local node.
Successfully deleted directory '/oracle/grid/11.2.0.3' on the local node.
Successfully detached Oracle home '/oracle/grid/11.2.0.3' from the central inventory on the remote nodes
'node5,node2,node3,node7,node6,node4,node8'.
Successfully deleted directory '/oracle/grid/11.2.0.3' on the remote nodes
Oracle Universal Installer cleanup completed Successfully.
Oracle deinstall tool successfully cleaned up temporary directories.
#######################################################################

Post upgrade tasks
Remove RDBMS11gR2 home
 After having completed all 11gR2 database upgrades
 Use the Oracle Grid Infrastructure 11g standalone deinstallation
tool to remove the previous release Grid home.
 And not the 12c one as stated in the documentation 
 Download it from here:
http://www.oracle.com/technetwork/database/enterprise-edition/downloads/index.html

Post upgrade tasks
./deinstall -home /oracle/db/11.2.0.3
####################### CHECK OPERATION SUMMARY #######################
Oracle Grid Infrastructure Home is: /oracle/grid/12.1.0.0
The cluster node(s) on which the Oracle home deinstallation will be performed
are:node1,node2,node3,node4,node5,node6,node7,node8
Oracle Home selected for deinstall is: /oracle/db/11.2.0.3
Inventory Location where the Oracle home registered is: /oracle/oraInventory
No Enterprise Manager configuration to be updated for any database(s)
No Enterprise Manager ASM targets to update
No Enterprise Manager listener targets to migrate
Checking the config status for CCR
node1 : Oracle Home exists with CCR directory, but CCR is not configured
CCR check is finished
Do you want to continue (y - yes, n - no)? [n]: YES

Post upgrade tasks
######################### CLEAN OPERATION END #########################
####################### CLEAN OPERATION SUMMARY #######################
Cleaning the config for CCR
As CCR is not configured, so skipping the cleaning of CCR configuration
CCR clean is finished
Successfully detached Oracle home '/oracle/db/11.2.0.3' from the central inventory on the local
node.
Successfully deleted directory '/oracle/db/11.2.0.3' on the local node.
Successfully detached Oracle home '/oracle/db/11.2.0.3' from the central inventory on the remote
nodes 'node5,node2,node3,node7,node6,node4,node8'.
Successfully deleted directory '/oracle/db/11.2.0.3' on the remote nodes
Oracle Universal Installer cleanup was successful.
Oracle deinstall tool successfully cleaned up temporary directories.
#######################################################################
############# ORACLE DEINSTALL & DECONFIG TOOL END #############

Post upgrade tasks
Upgrade ASM DiskGroup attributes to 12,1,0,0
BUT, think about compatible parameter of the database!
You will not be able to start your database if you do not update
the parameter to “12.1.0.0.0” before changing ASM DiskGroup
attributes.

Post upgrade tasks
After the upgrade, think to update
 Documentation,
 Procedure,
 DRP scenarios,
 …
Welcome to 12c!

Thank you very much for your attention
?

Presentation 12c grid_upgrade

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Presentation 12c grid_upgrade

Similar to Presentation 12c grid_upgrade (20)

More from Jacques Kostic

More from Jacques Kostic (10)

Recently uploaded

Recently uploaded (20)

Presentation 12c grid_upgrade