Exadata Patching Demystified

5,024 views

Published on

Published in: Technology

Exadata Patching Demystified

  1. 1. Exadata Patching Demystified Presented by: Andy Colvin August 14, 2012
  2. 2. About Me  Working around Oracle since 1999  Background in systems, network, database  6 years at Enkitec  Working on Exadata for 2+ years
  3. 3. Exadata Patching Survey  How many people patch themselves?  How many have had problems patching?  How many never patch?
  4. 4. A Quick Disclaimer Patching was very rough in the beginning It’s much better now!
  5. 5.  Do I have to patch every month? Oracle will patch my Exadata for me* If I don’t patch my Exadata, Oracle won’t support me Exadata patches are scary
  6. 6. Why Patch? Bug fixes New features Security updates
  7. 7. Where to Start?  MOS Note #888828.1  Latest patch updates  New patching methods  Links to patch notes for other products
  8. 8. Oracle-Provided Patches Compute Nodes Exadata Storage  Oracle Stack Servers  Linux/Solaris  Intelligent storage  Always run Linux  ProprietaryInfiniband Switches application - cellsrv  Custom CentOS
  9. 9. Exadata Storage Server Patches Exadata Storage Server  Single patch  Operating system•  Firmware Single patch download • SameDriversapplies to all hardware  patch • Exadata software, OS, ILOM, firmware  Updates storage and compute nodes • README and Support Note  Released quarterly (usually) # imageinfo Active image version: 11.2.2.3.2.110520
  10. 10. Exadata Storage Server Versions 11.2.3.1.1DB Major Release Maintenance Release DB Minor Release Feature Release DB Patchset
  11. 11. Exadata Storage Server Versions What version am I running?[root@enkcel03 ~]# imageinfoKernel version: 2.6.18-274.18.1.0.1.el5 #1 SMP Thu Feb 9 19:07:16 EST 2012 x86_64Cell version: OSS_11.2.3.1.1_LINUX.X64_120607Cell rpm version: cell-11.2.3.1.1_LINUX.X64_120607-1Active image version: 11.2.3.1.1.120607Active image activated: 2012-06-28 21:45:30 -0500Active image status: successActive system partition on device: /dev/md6Active software partition on device: /dev/md8
  12. 12. Exadata Storage Server Patches Apply with patchmgr – included with the patch release Run from compute node Uses dcli and SSH keys to deploy patch across storage grid
  13. 13. What Happens During Patching? New OS image pushed to inactive partitions on each cell Filesystem labels are switched to newly patched partitions The cell reboots and firmware is patched – multiple reboots are common Post-install validation checks are run USB recovery media is updated to match the new software version
  14. 14. Rolling Storage Server Patching offline disks No outage required online cel01 disks Single command ASM resync Patch Staging end-to-end offline disks online cel02 Consider your disks workload ASM resync  6 hours – ¼ rack offline disks online celNN disks ASM resync
  15. 15. Non-Rolling Storage Server Patching Full outage cel01  1.5 – 3 hours Patch Staging All cells in parallel cel02 Shorter maintenance window celNN
  16. 16. Patching Database Servers Database servers updated with storage servers Utilize yum for updates Each version has a yum channel  exadata_dbserver_11.2_x86_64_latest  exadata_dbserver_11.2.3.1.0_x86_64_base  exadata_dbserver_11.2.3.1.0_x86_64_patch
  17. 17. Patching Database Servers Database servers updated along with storage servers Utilize yum for updates  Replaces old “minimal pack” Each version has a yum channel  exadata_dbserver_11.2_x86_64_latest  exadata_dbserver_11.2.3.1.0_x86_64_base  exadata_dbserver_11.2.3.1.0_x86_64_patch
  18. 18. Patching Database Servers Compute nodes must be running OEL 5.5  V2 Exadata owners – see MOS note #1284070.1 Several options for installation  Update directly from ULN  Configure local yum repository  Mount ISO file containing RPMs
  19. 19. Quarterly Database Patch for Exadata  Patches for RDBMS & Grid Infrastructure  Applied using OPatch  Each BP/QDPE contains 3 patches, applied together  RDBMS  CRS  Diskmon  Patches are cumulative
  20. 20. Quarterly Database Patch for Exadata  Originally called bundle patches  Release frequency  Current patchset (11.2.0.3) – quarterly recommended patches, with monthly bundles  Previous patchset (11.2.0.2) - quarterly  Oracle recommends quarterly patching according to CPU/PSU cycle
  21. 21. QDPE / BP Versions  Check patch version with “opatch lsinventory”[oracle@enkdb01 ~]$ opatch lsinventoryInterim patches (3) :Patch 13688022 : applied on Fri Apr 27 14:52:31 CDT 2012Unique Patch ID: 14639565Patch description: "Diskmon Patch for Exadata (APR 2012 - 11.2.0.3.5) : (13688022)”Patch 13696251 : applied on Fri Apr 27 14:51:36 CDT 2012Unique Patch ID: 14639565Patch description: "CRS Patch for Exadata (APR 2012 - 11.2.0.3.5) : (13696251)”Patch 13734832 : applied on Fri Apr 27 14:49:34 CDT 2012Unique Patch ID: 14639565Patch description: "Database Patch for Exadata (APR 2012 - 11.2.0.3.5) : (13734832)”
  22. 22. QDPE / BP Application Always rolling patches Applied one node at a time Use opatch auto to apply Requires root privileges  sudo is acceptable Post-install database scripts
  23. 23. Patchset Upgrades Check MOS #888828.1 for certification Read upgrade-specific notes before upgrading  11.2.0.2 – MOS #1315926.1  11.2.0.3 – MOS #1373255.1 GI upgrade always rolling Database upgrade requires an outage*
  24. 24. Patchset Upgrades Minimum cell version requirements  11.2.0.2 – cell version 11.2.2.1.1+  11.2.0.3 – cell version 11.2.2.4.0+ Install latest BP on current release before upgrading GI/RDBMS Apply BP/QDPE after installing software, before running upgrade scripts
  25. 25. Infiniband Patches Released yearly Infiniband switches run CentOS Linux Patch method depends on version Rolling patches – no outage required  Spine switch first, followed by leaf switches
  26. 26. What’s a QFSDP? Quarterly Full Stack Download Patch Released January, April, July, October Contains latest patches for all stacks  Infrastructure  Database  OEM Patches still installed individually You may not need to apply all patches (Infiniband, PDU, etc)
  27. 27. Minimizing Patching Risk  Create a concise patching plan  Read all of the notes before starting  Patch a non-production system first  Patch the same way on all systems
  28. 28. Standby First Patch Apply Apply patches to standby system Perform testing with snapshot standby or active data guard Dataguard switchover Patch previous production system Switch back to production or leave as is for next patch cycle
  29. 29. OPlan Used with BP/QDPE deployment Creates patch deployment report customized to your environment Includes steps for in place, out of place patch and rollback MOS Note #1306814.1  Patch #11846294  Available for 11.2.0.2 and 11.2.0.3 bundle patches or QDPEs
  30. 30. A Quick Note on Platinum Services Read the T’s and C’s before you trust sales Oracle will patch for free (to a certain extent) Monitoring is great
  31. 31. Useful MOS Notes Supported versions (11.2) - #888828.1 e-Business Suite patch compatibility - #1392527.1 OPlan – #1306814.1 Exachk – #1070954.1 11.2.0.2 upgrade – #1315926.1 11.2.0.3 upgrade – #1373255.1
  32. 32. Questions?Contact Information: Andy Colvinemail - andy.colvin@enkitec.comweb - http://www.enkitec.comblog – http://blog.oracle-ninja.comtwitter - @acolvin

×