Patching like a hero - DOAG

Rodrigo Mufalani
Rodrigo MufalaniDatabase Administrator
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING LIKE A HERO
TIPS AND TRICKS FOR DBAS
Rodrigo Mufalani
Principal DB Architect
Oracle Certified Master (OCM)
Oracle ACE Member
eProssed - Luxembourg
Mobile Process /
Integration
Analytics /
Big Data
Database /
Data
Management
Identity &
Access
Management
Oracle Cloud
eProseed Confidential2
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
BOUT ME
Principal DB Architect at eProseed Lux
Oracle Certified Master (OCM)
Oracle ACE Member
Chairman of LuxOUG
Board Member at GUOB
Contacts
mufalani@gmail.com
Twitter.com/mufalani
https://www.linkedin.com/in/mufalani/
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
bit.ly/OracleACEProgram
500+ Technical Experts
Helping Peers Globally
Nominate yourself or someone you know: acenomination.oracle.com
RP4
Slide 4
RP4 Rodrigo Pinto, 12/7/2019
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
Patching like a hero -  DOAG
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
AGENDA
Planning
Applying
Checking
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
DISCLAIMER
“The postings on this document are my own and don’t necessarily represent my
actual employer positions, strategies or opinions. The information here was edited to
be useful for general purpose, specific data and identifications were removed to
allow reach the generic audience and to be useful for the community.”
9
PLANNING
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
WHY PATCH IS SO IMPORTANT?
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
WHY PATCH IS SO IMPORTANT?
• PSU – Patch Set Updates
• BP – Proactive Bundle Patches
>= 12cR2
• RU – Release Update
• RUR – Release Update Revision
• https://mikedietrichde.com/2017/10/24/differences-psu-bp-ru-rur/
• https://mikedietrichde.com/2018/11/08/why-release-update-revisions-rur-are-tricky/
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCH DOWNLOAD ASSISTANT - DOC ID 2118136.2
12
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCH DOWNLOAD ASSISTANT - DOC ID 2118136.2
13
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCH CONFLICT ANALYZER
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCH CONFLICT ANALYZER
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCH READ ME
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCH READ ME
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
IMPORTANT MOS NOTES
Doc ID 756671.1 Master Note for Database Proactive Patch Program
Doc ID 2285040.1 Release Update Introduction and FAQ
Doc ID 2118136.2 Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU,
SPU(CPU), Bundle Patches, Patchsets and Base Releases
Doc ID 2521164.1 Database 19c Proactive Patch Information
Doc ID 2420120.1 18c : Datapatch New Features And Changes from 12.1
Doc ID 1265700.1 Oracle Patch Assurance - Data Guard Standby-First Patch Apply
19
APPLYING
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
BACKUP OF ORACLE HOME
[root@o19c]# cd /u01/app/oracle/product/19.0.0/db_1
[root@o19c]# tar -cf /tmp/backupOH140620.tar .
https://mufalani.wordpress.com/2019/06/13/backing-up-and-restoring-your-oracle-home/
Note id 1493677.1
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
CLONING AN ORACLE HOME
[root@o19c]# cd /u01/app/oracle/product/19.0.0/
[root@o19c]# mkdir db_2 && chown –R oracle.oinstall db_2
[root@o19c]# chmod –R 775 db_2
[root@o19c]# cd /u01/app/oracle/product/19.0.0/db_2
[root@o19c]# tar -xf /tmp/backupOH140620.tar
https://mufalani.wordpress.com/2020/03/03/saving-your-time-cloning-a-patched-oh-oracle-home-to-another-
machine/
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
CLONING AN ORACLE HOME
[oracle@o19c]$ export ORACLE_HOME=
/u01/app/oracle/product/19.0.0/db_2
[oracle@o19c]$ cd $ORACLE_HOME/clone/bin
$ORACLE_HOME/perl/bin/perl clone.pl 
ORACLE_BASE="/u01/app/oracle/" 
ORACLE_HOME="/u01/app/oracle/product/19.0.0/db_2" 
OSDBA_GROUP=oinstall OSOPER_GROUP=dba 
OSBACKUPDBA_GROUP=oinstall OSRACDBA_GROUP=oinstall 
ORACLE_HOME_NAME="Db_home19700"
https://mufalani.wordpress.com/2020/03/03/saving-your-time-cloning-a-patched-oh-oracle-home-to-another-machine/
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
RESTORE OF ORACLE HOME
[root@o19c]# cd /u01/app/oracle/product/19.0.0/
[root@o19c]# mv db_1 db_1_damaged_bad
[root@o19c]# mkdir db_1 && chown –R oracle.install
[root@o19c]# chmod –R 775 db_1
[root@o19c]# cd /u01/app/oracle/product/19.0.0/db_1
[root@o19c]# tar -xf /tmp/backupOH140620.tar
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
RESTORE OF ORACLE HOME
[oracle@o19c]$ cd $ORACLE_HOME/bin
[oracle@o19c]$ relink all
[oracle@o19c]$ cd $ORACLE_HOME/install
[oracle@o19c]$ grep -i "Code 1|Fatal|Error” relink_2020-06-14_*.log
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
TIPS AND TRICKS
Check Inventory
Check opatch required version
Backup OH
Clone OH
Relink OH
Check the OS version of the patch
Read the entire readme file of the patch
Don’t use NFS to store patch or backups
Check free memory because of the unzip step
Check PATH
Check space
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING OPATCH UTILITY - 6880880
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING OPATCH UTILITY - 6880880
[oracle@o19c ~]$ cd $ORACLE_HOME
[oracle@o19c ~]$ mv OPatch OPatch.14062020
[oracle@o19c ~]$ unzip -qa /tmp/p6880880_190000_Linux-x86-64.zip
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
TIPS AND TRICKS
More about opatch prereqs options? (Doc ID 459360.1)
[oracle@o19c]$ ./opatch prereq CheckSystemSpace -phBaseDir
/tmp/30869156
[oracle@o19c]$ ./opatch prereq CheckConflictAgainstOHWithDetail -
phBaseDir /tmp/29301682/29301631
[oracle@o19c]$ ./opatch prereq CheckConflictAgainstOHWithDetail -
phBaseDir /tmp/29301682/29302264
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
TIPS AND TRICKS
[oracle@o19c]$ ORACLE_HOME/.patch_storage/NApply/<TIMESTAMP>
restore.sh
make.txt
Can we delete $ORACLE_HOME/.patch_storage Directory? (Doc ID 403218.1)
opatch util cleanup (Doc ID 550522.1)
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
TIPS AND TRICKS
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
TIPS AND TRICKS
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
TIPS AND TRICKS
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
TIPS AND TRICKS
To solve this:
I restored the damaged lib file, copying it from my backup to $ORACLE_HOME/lib
Then, I issued a relink
$ORACLE_HOME/bin/relink all
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING ORACLE HOME
[oracle@ora19c ~]$ export PATH=$PATH:/usr/ccs/bin
[oracle@ora19c ~]$ export
ORACLE_HOME=/oracle/app/oracle/product/19.0.0/db_2
[oracle@ora19c ~]$ export
PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING ORACLE HOME
[oracle@ora19c ~]$ cd /tmp/patches
[oracle@ora19c ~]$ unzip -qa p30869156_190000_Linux-x86-64.zip
[oracle@ora19c ~]$ cd 30869156/
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING ORACLE HOME
[oracle@o19c]$ opatch prereq CheckConflictAgainstOHWithDetail -ph ./
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING ORACLE HOME
[oracle@o19c]$ cd /tmp/patches/30869156/
[oracle@o19c]$ opatch apply
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING ORACLE HOME
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING ORACLE HOME
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
DATA DICTIONARY PATCHES (SQL)
datapatch is online
Datapatch is smart (automatically rollback)
Old days
@catbundle.sql PSU|CPU|EXA apply
$ORACLE_HOME/cfgtools/catbundle/
Note id 605795.1
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
DATA DICTIONARY PATCHES (SQL)
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
SOLVING CONFLICTS
[oracle@o19c]$ opatch rollback –id 8702869
Sometimes you need to request a merge to Oracle support manually open a SR or
automatically by Conflict Analyze Tool online.
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING GRID INFRA BEFORE TO INSTALL
cd /u01/app/grid/product/12201
unzip /tmp/linuxx64_12201_grid_home.zip
mkdir /tmp/patches
cd /tmp/patches
unzip /tmp/p26133434_122010_Linux-x86-64-GI.zip -d /tmp/patches/
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING GRID INFRA BEFORE TO INSTALL
cd /u01/app/grid/product/12201
./gridSetup.sh -silent -applyOneOffs /tmp/patches/26133434/25586399/
./gridSetup.sh -silent -applyOneOffs /tmp/patches/26133434/26002778/
./gridSetup.sh -silent -applyOneOffs /tmp/patches/26133434/26123830/
./gridSetup.sh
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING OEMCC
export ORACLE_HOME=/u01/oem/middleware
cd /u01/
tar cf - oem | gzip >
/nfsbak/backup_system/OEMCC13c_bkp_19062019.tar.gz
Obs.: You can ignore the errors during backup of… $OH/network/log/cgisock.22741192
is not a valid tar file type.
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING OEMCC
export ORACLE_HOME=/u01/oem/middleware
export omspatcher=/u01/oem/middleware/OMSPatcher/omspatcher
cd /u01/oracle/patch/OMS
unzip p6880880_139000_Generic.zip
$ORACLE_HOME/bin/emctl stop oms
-- You can find java under agent home
/u01/oem/agent/agent_13.2.0.0.0/jdk/bin/java -jar
/u01/oracle/patch/OMS/6880880/opatch_generic.jar -silent
oracle_home=/u01/oem/middleware
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING OEMCC
export ORACLE_HOME=/u01/oem/middleware
export omspatcher=/u01/oem/middleware/OMSPatcher/omspatcher
cd /u01/oracle/patch/OMS
mv $ORACLE_HOME/OMSPatcher $ORACLE_HOME/OMSPatcher.bkp190619
unzip -qa -d $ORACLE_HOME/
/u01/oracle/patch/OMS/p19999993_132000_Generic.zip
$ORACLE_HOME/OMSPatcher/omspatcher version
$ORACLE_HOME/bin/emctl start oms
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING OEMCC
export ORACLE_HOME=/u01/oem/middleware
export omspatcher=/u01/oem/middleware/OMSPatcher/omspatcher
$ORACLE_HOME/OMSPatcher/wlskeys/createkeys.sh -oh $ORACLE_HOME -
location /u01/oracle/patch/OMS/
username: weblogic
password: MYorclTST12c
vi /u01/oracle/patch/OMS/property.txt
AdminServerURL=t3s://mysrvoem:7101
AdminConfigFile=/u01/oracle/patch/OMS/config
AdminKeyFile=/u01/oracle/patch/OMS/key
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING OEMCC
cd /u01/oracle/patch/OMS/
unzip -qa p29762588_132000_Generic.zip
cd /u01/oracle/patch/OMS/29762588/
$omspatcher apply /u01/oracle/patch/OMS/29762588/ -analyze -
invPtrLoc /u01/oem/middleware/oraInst.loc
OMSPatcher.OMS_DISABLE_HOST_CHECK=true -property_file
/u01/oracle/patch/OMS/property.txt
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING OEMCC
export ORACLE_HOME=/u01/oem/middleware
export omspatcher=/u01/oem/middleware/OMSPatcher/omspatcher
$ORACLE_HOME/bin/emctl stop oms
$omspatcher apply /u01/oracle/patch/OMS/29762588/ -invPtrLoc
/u01/oem/middleware/oraInst.loc
OMSPatcher.OMS_DISABLE_HOST_CHECK=true -property_file
/u01/oracle/patch/OMS/property.txt
$ORACLE_HOME/bin/emctl start oms
$ORACLE_HOME/OPatch/opatch lspatches
51
CHECKING
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
PATCHING ORACLE HOME
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
CHECKING INVENTORY
Expanded
$ORACLE_HOME/OPatch/opatch lsinvetory
Sumarized
$ORACLE_HOME/OPatch/opatch lspatches
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
CHECKING APPLIED PATCHES
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
CHECKING APPLIED PATCHES
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
CHECKING COMPONENTS
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
1 of 59

Recommended

Percona Live Europe 2018 MySQL Group Replication... the magic explained by
Percona Live Europe 2018  MySQL Group Replication... the magic explainedPercona Live Europe 2018  MySQL Group Replication... the magic explained
Percona Live Europe 2018 MySQL Group Replication... the magic explainedFrederic Descamps
563 views168 slides
DataOpsbarcelona 2019: Deep dive into MySQL Group Replication... the magic e... by
DataOpsbarcelona 2019:  Deep dive into MySQL Group Replication... the magic e...DataOpsbarcelona 2019:  Deep dive into MySQL Group Replication... the magic e...
DataOpsbarcelona 2019: Deep dive into MySQL Group Replication... the magic e...Frederic Descamps
880 views191 slides
MySQL Group Replication: the magic explained v.2 by
MySQL Group Replication: the magic explained v.2MySQL Group Replication: the magic explained v.2
MySQL Group Replication: the magic explained v.2Frederic Descamps
385 views188 slides
MySQL Shell: the best DBA tool ? by
MySQL Shell: the best DBA tool ?MySQL Shell: the best DBA tool ?
MySQL Shell: the best DBA tool ?Frederic Descamps
531 views55 slides
Informatica transformation guide by
Informatica transformation guideInformatica transformation guide
Informatica transformation guidesonu_pal
4.8K views434 slides
ILOUG 2019 - 18c/19c features by
ILOUG 2019 - 18c/19c featuresILOUG 2019 - 18c/19c features
ILOUG 2019 - 18c/19c featuresConnor McDonald
557 views116 slides

More Related Content

Similar to Patching like a hero - DOAG

Shell Script 4 DBAs by
Shell Script 4 DBAsShell Script 4 DBAs
Shell Script 4 DBAsRodrigo Mufalani
412 views41 slides
AIOUG - Groundbreakers - Jul 2019 - 19 Troubleshooting Tips and Tricks for Da... by
AIOUG - Groundbreakers - Jul 2019 - 19 Troubleshooting Tips and Tricks for Da...AIOUG - Groundbreakers - Jul 2019 - 19 Troubleshooting Tips and Tricks for Da...
AIOUG - Groundbreakers - Jul 2019 - 19 Troubleshooting Tips and Tricks for Da...Sandesh Rao
419 views108 slides
AUSOUG - NZOUG - Groundbreakers - Jun 2019 - 19 Troubleshooting Tips and Tric... by
AUSOUG - NZOUG - Groundbreakers - Jun 2019 - 19 Troubleshooting Tips and Tric...AUSOUG - NZOUG - Groundbreakers - Jun 2019 - 19 Troubleshooting Tips and Tric...
AUSOUG - NZOUG - Groundbreakers - Jun 2019 - 19 Troubleshooting Tips and Tric...Sandesh Rao
373 views111 slides
MySQL Group Replication - HandsOn Tutorial by
MySQL Group Replication - HandsOn TutorialMySQL Group Replication - HandsOn Tutorial
MySQL Group Replication - HandsOn TutorialKenny Gryp
2.2K views191 slides
GraalVM Native Images by Oleg Selajev @shelajev by
GraalVM Native Images by Oleg Selajev @shelajevGraalVM Native Images by Oleg Selajev @shelajev
GraalVM Native Images by Oleg Selajev @shelajevOracle Developers
278 views49 slides
OpenChain AutomotiveWG(OSS license tools() by
OpenChain AutomotiveWG(OSS license tools()OpenChain AutomotiveWG(OSS license tools()
OpenChain AutomotiveWG(OSS license tools()Yuichi Kusakabe
113 views14 slides

Similar to Patching like a hero - DOAG(20)

AIOUG - Groundbreakers - Jul 2019 - 19 Troubleshooting Tips and Tricks for Da... by Sandesh Rao
AIOUG - Groundbreakers - Jul 2019 - 19 Troubleshooting Tips and Tricks for Da...AIOUG - Groundbreakers - Jul 2019 - 19 Troubleshooting Tips and Tricks for Da...
AIOUG - Groundbreakers - Jul 2019 - 19 Troubleshooting Tips and Tricks for Da...
Sandesh Rao419 views
AUSOUG - NZOUG - Groundbreakers - Jun 2019 - 19 Troubleshooting Tips and Tric... by Sandesh Rao
AUSOUG - NZOUG - Groundbreakers - Jun 2019 - 19 Troubleshooting Tips and Tric...AUSOUG - NZOUG - Groundbreakers - Jun 2019 - 19 Troubleshooting Tips and Tric...
AUSOUG - NZOUG - Groundbreakers - Jun 2019 - 19 Troubleshooting Tips and Tric...
Sandesh Rao373 views
MySQL Group Replication - HandsOn Tutorial by Kenny Gryp
MySQL Group Replication - HandsOn TutorialMySQL Group Replication - HandsOn Tutorial
MySQL Group Replication - HandsOn Tutorial
Kenny Gryp2.2K views
GraalVM Native Images by Oleg Selajev @shelajev by Oracle Developers
GraalVM Native Images by Oleg Selajev @shelajevGraalVM Native Images by Oleg Selajev @shelajev
GraalVM Native Images by Oleg Selajev @shelajev
Oracle Developers278 views
OpenChain AutomotiveWG(OSS license tools() by Yuichi Kusakabe
OpenChain AutomotiveWG(OSS license tools()OpenChain AutomotiveWG(OSS license tools()
OpenChain AutomotiveWG(OSS license tools()
Yuichi Kusakabe113 views
API Design Principles Essential  by Oracle Korea
API Design Principles Essential API Design Principles Essential 
API Design Principles Essential 
Oracle Korea245 views
10thMeetup-20190420-REST API Design Principles 되새기기 by DongHee Lee
10thMeetup-20190420-REST API Design Principles 되새기기10thMeetup-20190420-REST API Design Principles 되새기기
10thMeetup-20190420-REST API Design Principles 되새기기
DongHee Lee565 views
Api design and prototype by DonghuKIM2
Api design and prototypeApi design and prototype
Api design and prototype
DonghuKIM2218 views
OpenWorld 2018 - 20 years of hints and tips by Connor McDonald
OpenWorld 2018 - 20 years of hints and tipsOpenWorld 2018 - 20 years of hints and tips
OpenWorld 2018 - 20 years of hints and tips
Connor McDonald1.3K views
MySQL InnoDB Cluster and Group Replication in a nutshell hands-on tutorial by Frederic Descamps
MySQL InnoDB Cluster and Group Replication in a nutshell  hands-on tutorialMySQL InnoDB Cluster and Group Replication in a nutshell  hands-on tutorial
MySQL InnoDB Cluster and Group Replication in a nutshell hands-on tutorial
Frederic Descamps1.4K views
ITOUG 2019 - 25 years of hints and tips by Connor McDonald
ITOUG 2019 - 25 years of hints and tipsITOUG 2019 - 25 years of hints and tips
ITOUG 2019 - 25 years of hints and tips
Connor McDonald122 views
Serverless Java: JJUG CCC 2019 by Shaun Smith
Serverless Java: JJUG CCC 2019Serverless Java: JJUG CCC 2019
Serverless Java: JJUG CCC 2019
Shaun Smith296 views
Dockerize a Django app elegantly by frentrup
Dockerize a Django app elegantlyDockerize a Django app elegantly
Dockerize a Django app elegantly
frentrup263 views
20180921_DOAG_BigDataDays_OracleSpatialandPython_kpatenge by Karin Patenge
20180921_DOAG_BigDataDays_OracleSpatialandPython_kpatenge20180921_DOAG_BigDataDays_OracleSpatialandPython_kpatenge
20180921_DOAG_BigDataDays_OracleSpatialandPython_kpatenge
Karin Patenge158 views
2019 indit blackhat_honeypot your database server by Georgi Kodinov
2019 indit blackhat_honeypot your database server2019 indit blackhat_honeypot your database server
2019 indit blackhat_honeypot your database server
Georgi Kodinov134 views
Is 12 Factor App Right About Logging by Phil Wilkins
Is 12 Factor App Right About LoggingIs 12 Factor App Right About Logging
Is 12 Factor App Right About Logging
Phil Wilkins347 views
Serverless survival kit by Steve Houël
Serverless survival kitServerless survival kit
Serverless survival kit
Steve Houël56 views

Recently uploaded

Helko van den Brom - VSL by
Helko van den Brom - VSLHelko van den Brom - VSL
Helko van den Brom - VSLDutch Power
87 views18 slides
Christan van Dorst - Hyteps by
Christan van Dorst - HytepsChristan van Dorst - Hyteps
Christan van Dorst - HytepsDutch Power
89 views24 slides
Gym Members Community.pptx by
Gym Members Community.pptxGym Members Community.pptx
Gym Members Community.pptxnasserbf1987
7 views5 slides
PB CV v0.3 by
PB CV v0.3PB CV v0.3
PB CV v0.3Pedro Borracha
12 views16 slides
The Throne of Your Heart 11-26-23 PPT.pptx by
The Throne of Your Heart 11-26-23 PPT.pptxThe Throne of Your Heart 11-26-23 PPT.pptx
The Throne of Your Heart 11-26-23 PPT.pptxFamilyWorshipCenterD
5 views24 slides
Pdffromtextfile_1.pdf by
Pdffromtextfile_1.pdfPdffromtextfile_1.pdf
Pdffromtextfile_1.pdfTRIEU QUANG NGO
6 views2 slides

Recently uploaded(20)

Helko van den Brom - VSL by Dutch Power
Helko van den Brom - VSLHelko van den Brom - VSL
Helko van den Brom - VSL
Dutch Power87 views
Christan van Dorst - Hyteps by Dutch Power
Christan van Dorst - HytepsChristan van Dorst - Hyteps
Christan van Dorst - Hyteps
Dutch Power89 views
Gym Members Community.pptx by nasserbf1987
Gym Members Community.pptxGym Members Community.pptx
Gym Members Community.pptx
nasserbf19877 views
OSMC 2023 | Will ChatGPT Take Over My Job? by Philipp Krenn by NETWAYS
OSMC 2023 | Will ChatGPT Take Over My Job? by Philipp KrennOSMC 2023 | Will ChatGPT Take Over My Job? by Philipp Krenn
OSMC 2023 | Will ChatGPT Take Over My Job? by Philipp Krenn
NETWAYS22 views
Synthetic Biology.pptx by ShubNoor4
Synthetic Biology.pptxSynthetic Biology.pptx
Synthetic Biology.pptx
ShubNoor47 views
Post-event report intro session-1.docx by RohitRathi59
Post-event report intro session-1.docxPost-event report intro session-1.docx
Post-event report intro session-1.docx
RohitRathi5912 views
Roozbeh Torkzadeh - TU Eindhoven by Dutch Power
Roozbeh Torkzadeh - TU EindhovenRoozbeh Torkzadeh - TU Eindhoven
Roozbeh Torkzadeh - TU Eindhoven
Dutch Power85 views
Managing Github via Terrafom.pdf by micharaeck
Managing Github via Terrafom.pdfManaging Github via Terrafom.pdf
Managing Github via Terrafom.pdf
micharaeck5 views
231121 SP slides - PAS workshop November 2023.pdf by PAS_Team
231121 SP slides - PAS workshop November 2023.pdf231121 SP slides - PAS workshop November 2023.pdf
231121 SP slides - PAS workshop November 2023.pdf
PAS_Team158 views

Patching like a hero - DOAG

  • 1. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING LIKE A HERO TIPS AND TRICKS FOR DBAS Rodrigo Mufalani Principal DB Architect Oracle Certified Master (OCM) Oracle ACE Member eProssed - Luxembourg
  • 2. Mobile Process / Integration Analytics / Big Data Database / Data Management Identity & Access Management Oracle Cloud eProseed Confidential2
  • 3. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential BOUT ME Principal DB Architect at eProseed Lux Oracle Certified Master (OCM) Oracle ACE Member Chairman of LuxOUG Board Member at GUOB Contacts mufalani@gmail.com Twitter.com/mufalani https://www.linkedin.com/in/mufalani/
  • 4. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential bit.ly/OracleACEProgram 500+ Technical Experts Helping Peers Globally Nominate yourself or someone you know: acenomination.oracle.com RP4
  • 5. Slide 4 RP4 Rodrigo Pinto, 12/7/2019
  • 6. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
  • 8. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential AGENDA Planning Applying Checking
  • 9. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential DISCLAIMER “The postings on this document are my own and don’t necessarily represent my actual employer positions, strategies or opinions. The information here was edited to be useful for general purpose, specific data and identifications were removed to allow reach the generic audience and to be useful for the community.”
  • 11. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential WHY PATCH IS SO IMPORTANT?
  • 12. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential WHY PATCH IS SO IMPORTANT? • PSU – Patch Set Updates • BP – Proactive Bundle Patches >= 12cR2 • RU – Release Update • RUR – Release Update Revision • https://mikedietrichde.com/2017/10/24/differences-psu-bp-ru-rur/ • https://mikedietrichde.com/2018/11/08/why-release-update-revisions-rur-are-tricky/
  • 13. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCH DOWNLOAD ASSISTANT - DOC ID 2118136.2 12
  • 14. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCH DOWNLOAD ASSISTANT - DOC ID 2118136.2 13
  • 15. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCH CONFLICT ANALYZER
  • 16. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCH CONFLICT ANALYZER
  • 17. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCH READ ME
  • 18. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCH READ ME
  • 19. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential IMPORTANT MOS NOTES Doc ID 756671.1 Master Note for Database Proactive Patch Program Doc ID 2285040.1 Release Update Introduction and FAQ Doc ID 2118136.2 Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases Doc ID 2521164.1 Database 19c Proactive Patch Information Doc ID 2420120.1 18c : Datapatch New Features And Changes from 12.1 Doc ID 1265700.1 Oracle Patch Assurance - Data Guard Standby-First Patch Apply
  • 21. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential BACKUP OF ORACLE HOME [root@o19c]# cd /u01/app/oracle/product/19.0.0/db_1 [root@o19c]# tar -cf /tmp/backupOH140620.tar . https://mufalani.wordpress.com/2019/06/13/backing-up-and-restoring-your-oracle-home/ Note id 1493677.1
  • 22. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential CLONING AN ORACLE HOME [root@o19c]# cd /u01/app/oracle/product/19.0.0/ [root@o19c]# mkdir db_2 && chown –R oracle.oinstall db_2 [root@o19c]# chmod –R 775 db_2 [root@o19c]# cd /u01/app/oracle/product/19.0.0/db_2 [root@o19c]# tar -xf /tmp/backupOH140620.tar https://mufalani.wordpress.com/2020/03/03/saving-your-time-cloning-a-patched-oh-oracle-home-to-another- machine/
  • 23. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential CLONING AN ORACLE HOME [oracle@o19c]$ export ORACLE_HOME= /u01/app/oracle/product/19.0.0/db_2 [oracle@o19c]$ cd $ORACLE_HOME/clone/bin $ORACLE_HOME/perl/bin/perl clone.pl ORACLE_BASE="/u01/app/oracle/" ORACLE_HOME="/u01/app/oracle/product/19.0.0/db_2" OSDBA_GROUP=oinstall OSOPER_GROUP=dba OSBACKUPDBA_GROUP=oinstall OSRACDBA_GROUP=oinstall ORACLE_HOME_NAME="Db_home19700" https://mufalani.wordpress.com/2020/03/03/saving-your-time-cloning-a-patched-oh-oracle-home-to-another-machine/
  • 24. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential RESTORE OF ORACLE HOME [root@o19c]# cd /u01/app/oracle/product/19.0.0/ [root@o19c]# mv db_1 db_1_damaged_bad [root@o19c]# mkdir db_1 && chown –R oracle.install [root@o19c]# chmod –R 775 db_1 [root@o19c]# cd /u01/app/oracle/product/19.0.0/db_1 [root@o19c]# tar -xf /tmp/backupOH140620.tar
  • 25. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential RESTORE OF ORACLE HOME [oracle@o19c]$ cd $ORACLE_HOME/bin [oracle@o19c]$ relink all [oracle@o19c]$ cd $ORACLE_HOME/install [oracle@o19c]$ grep -i "Code 1|Fatal|Error” relink_2020-06-14_*.log
  • 26. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential TIPS AND TRICKS Check Inventory Check opatch required version Backup OH Clone OH Relink OH Check the OS version of the patch Read the entire readme file of the patch Don’t use NFS to store patch or backups Check free memory because of the unzip step Check PATH Check space
  • 27. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING OPATCH UTILITY - 6880880
  • 28. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING OPATCH UTILITY - 6880880 [oracle@o19c ~]$ cd $ORACLE_HOME [oracle@o19c ~]$ mv OPatch OPatch.14062020 [oracle@o19c ~]$ unzip -qa /tmp/p6880880_190000_Linux-x86-64.zip
  • 29. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential TIPS AND TRICKS More about opatch prereqs options? (Doc ID 459360.1) [oracle@o19c]$ ./opatch prereq CheckSystemSpace -phBaseDir /tmp/30869156 [oracle@o19c]$ ./opatch prereq CheckConflictAgainstOHWithDetail - phBaseDir /tmp/29301682/29301631 [oracle@o19c]$ ./opatch prereq CheckConflictAgainstOHWithDetail - phBaseDir /tmp/29301682/29302264
  • 30. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential TIPS AND TRICKS [oracle@o19c]$ ORACLE_HOME/.patch_storage/NApply/<TIMESTAMP> restore.sh make.txt Can we delete $ORACLE_HOME/.patch_storage Directory? (Doc ID 403218.1) opatch util cleanup (Doc ID 550522.1)
  • 31. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential TIPS AND TRICKS
  • 32. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential TIPS AND TRICKS
  • 33. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential TIPS AND TRICKS
  • 34. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential TIPS AND TRICKS To solve this: I restored the damaged lib file, copying it from my backup to $ORACLE_HOME/lib Then, I issued a relink $ORACLE_HOME/bin/relink all
  • 35. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING ORACLE HOME [oracle@ora19c ~]$ export PATH=$PATH:/usr/ccs/bin [oracle@ora19c ~]$ export ORACLE_HOME=/oracle/app/oracle/product/19.0.0/db_2 [oracle@ora19c ~]$ export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/OPatch:$PATH
  • 36. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING ORACLE HOME [oracle@ora19c ~]$ cd /tmp/patches [oracle@ora19c ~]$ unzip -qa p30869156_190000_Linux-x86-64.zip [oracle@ora19c ~]$ cd 30869156/
  • 37. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING ORACLE HOME [oracle@o19c]$ opatch prereq CheckConflictAgainstOHWithDetail -ph ./
  • 38. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING ORACLE HOME [oracle@o19c]$ cd /tmp/patches/30869156/ [oracle@o19c]$ opatch apply
  • 39. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING ORACLE HOME
  • 40. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING ORACLE HOME
  • 41. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential DATA DICTIONARY PATCHES (SQL) datapatch is online Datapatch is smart (automatically rollback) Old days @catbundle.sql PSU|CPU|EXA apply $ORACLE_HOME/cfgtools/catbundle/ Note id 605795.1
  • 42. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential DATA DICTIONARY PATCHES (SQL)
  • 43. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential SOLVING CONFLICTS [oracle@o19c]$ opatch rollback –id 8702869 Sometimes you need to request a merge to Oracle support manually open a SR or automatically by Conflict Analyze Tool online.
  • 44. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING GRID INFRA BEFORE TO INSTALL cd /u01/app/grid/product/12201 unzip /tmp/linuxx64_12201_grid_home.zip mkdir /tmp/patches cd /tmp/patches unzip /tmp/p26133434_122010_Linux-x86-64-GI.zip -d /tmp/patches/
  • 45. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING GRID INFRA BEFORE TO INSTALL cd /u01/app/grid/product/12201 ./gridSetup.sh -silent -applyOneOffs /tmp/patches/26133434/25586399/ ./gridSetup.sh -silent -applyOneOffs /tmp/patches/26133434/26002778/ ./gridSetup.sh -silent -applyOneOffs /tmp/patches/26133434/26123830/ ./gridSetup.sh
  • 46. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING OEMCC export ORACLE_HOME=/u01/oem/middleware cd /u01/ tar cf - oem | gzip > /nfsbak/backup_system/OEMCC13c_bkp_19062019.tar.gz Obs.: You can ignore the errors during backup of… $OH/network/log/cgisock.22741192 is not a valid tar file type.
  • 47. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING OEMCC export ORACLE_HOME=/u01/oem/middleware export omspatcher=/u01/oem/middleware/OMSPatcher/omspatcher cd /u01/oracle/patch/OMS unzip p6880880_139000_Generic.zip $ORACLE_HOME/bin/emctl stop oms -- You can find java under agent home /u01/oem/agent/agent_13.2.0.0.0/jdk/bin/java -jar /u01/oracle/patch/OMS/6880880/opatch_generic.jar -silent oracle_home=/u01/oem/middleware
  • 48. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING OEMCC export ORACLE_HOME=/u01/oem/middleware export omspatcher=/u01/oem/middleware/OMSPatcher/omspatcher cd /u01/oracle/patch/OMS mv $ORACLE_HOME/OMSPatcher $ORACLE_HOME/OMSPatcher.bkp190619 unzip -qa -d $ORACLE_HOME/ /u01/oracle/patch/OMS/p19999993_132000_Generic.zip $ORACLE_HOME/OMSPatcher/omspatcher version $ORACLE_HOME/bin/emctl start oms
  • 49. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING OEMCC export ORACLE_HOME=/u01/oem/middleware export omspatcher=/u01/oem/middleware/OMSPatcher/omspatcher $ORACLE_HOME/OMSPatcher/wlskeys/createkeys.sh -oh $ORACLE_HOME - location /u01/oracle/patch/OMS/ username: weblogic password: MYorclTST12c vi /u01/oracle/patch/OMS/property.txt AdminServerURL=t3s://mysrvoem:7101 AdminConfigFile=/u01/oracle/patch/OMS/config AdminKeyFile=/u01/oracle/patch/OMS/key
  • 50. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING OEMCC cd /u01/oracle/patch/OMS/ unzip -qa p29762588_132000_Generic.zip cd /u01/oracle/patch/OMS/29762588/ $omspatcher apply /u01/oracle/patch/OMS/29762588/ -analyze - invPtrLoc /u01/oem/middleware/oraInst.loc OMSPatcher.OMS_DISABLE_HOST_CHECK=true -property_file /u01/oracle/patch/OMS/property.txt
  • 51. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING OEMCC export ORACLE_HOME=/u01/oem/middleware export omspatcher=/u01/oem/middleware/OMSPatcher/omspatcher $ORACLE_HOME/bin/emctl stop oms $omspatcher apply /u01/oracle/patch/OMS/29762588/ -invPtrLoc /u01/oem/middleware/oraInst.loc OMSPatcher.OMS_DISABLE_HOST_CHECK=true -property_file /u01/oracle/patch/OMS/property.txt $ORACLE_HOME/bin/emctl start oms $ORACLE_HOME/OPatch/opatch lspatches
  • 53. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential PATCHING ORACLE HOME
  • 54. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential CHECKING INVENTORY Expanded $ORACLE_HOME/OPatch/opatch lsinvetory Sumarized $ORACLE_HOME/OPatch/opatch lspatches
  • 55. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential CHECKING APPLIED PATCHES
  • 56. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential CHECKING APPLIED PATCHES
  • 57. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential CHECKING COMPONENTS
  • 58. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential
  • 59. Copyright © 2019, eProseed and/or its affiliates. All rights reserved. | Confidential