Migrating from Oracle Enterprise Manager 10g to 12c Cloud Control

Migrating from Oracle
Enterprise Manager 10g
to 12c Cloud Control
Leighton L. Nelson

About Me
•  Oracle DBA for 10+ years
•  Oracle RAC SIG US Events Chair/IOUG Liaison
•  Blogs at blogs.griddba.com
•  Started presenting in 2012 (Collaborate 12)

Agenda
•  Oracle Enterprise Manger Cloud Control12c
Architecture
•  Upgrade Path
•  Overview
•  Pre-Requisites
•  Patches
•  Certification Matrix
•  2-System Upgrade
•  Repository Database Upgrade
•  OMS Migration & Upgrade
•  Lessons Learned
•  Managing Databases (Demo)

Current and Proposed States
•  Current Architecture (IBM JS21 Blades)
•  OMS – 10.2.0.5 / AIX 6.1
•  Repository Database – 11.1.0.6 / AIX 6.1
•  Agents – 100 (AIX + Linux)
•  Targets – 500+
•  Proposed Architecture (VMware vSphere)
•  OMS – 12.1.0.2 / RHEL 6.2
•  Repository Database – 11.2.0.3 / RHEL 6.2
•  Agents – 100 (AIX + Linux)
•  Targets – 500+

•  Database + OS + EM Upgrade = Trinity!

Enterprise Manager 12c Upgrade
Paths
•  Upgrade done on same host as 10.2.0.5/11.1 EM
•  All agents, OMS and repository upgraded at once
•  Requires significant downtime
•  New OMS created on separate host (platform migration
possible)
•  Both 10.2.0.5/11.1 and 12.1 EM run concurrently
•  Requires almost zero downtime
•  Migrates existing metrics, jobs, report etc.

Which Path to Choose?
•  Migrate/consolidate onto
VMware
•  EM 10.2.0.5 primary monitoring
tool for Oracle databases.
Minimal downtime required
•  User-defined metrics, scheduled
reports and jobs in EM 10.2.0.5.
Don’t want to recreate

Upgrade Overview
1.  Apply Upgrade Console patch to 10.2.0.5 OMS
2.  Deploy 12.1 agents to monitored targets from
Upgrade Console
•  Only 12c agents monitored by EM 12c
3.  Backup repository database. This will be used as the
12.1 repository.
•  DBCA, RMAN can be used for backups (same platform)
•  Datapump, Cross-Platform tablespaces/databases (different
platforms)
4.  Install and upgrade OMS on new RHEL 6.2 host
5.  Shutdown 10.2.0.5 agents and startup 12c agents
6.  Decommission 10.2.0.x agents/OMS or rollback

Pre-requisites
•  Patches
•  OMS requires PSU 3 or higher
•  opatch lsinventory –bugs_ﬁxed | grep PSU
•  Pre-Upgrade Console
•  Certification Matrix
•  Database, OMS, Agent and OS
•  Plug-ins and agent software
•  Plug-ins required can be identified from the Upgrade
Console Manage Software
•  Plug-ins will be migrated/upgraded to new OMS
•  Agents required for each OS being monitored

Pre-requisites
Certification Matrix

2-System Upgrade
Repository Database Upgrade
•  The EM installer does not
upgrade the database
•  Database can be upgraded
before EM upgrade or after
repository backup

11.1.0.6

11.2.0.2

2-System Upgrade
Apply Pre-Upgrade Console Patch

2-System Upgrade
Identify Host and Port for new EM System

2-System Upgrade
Manage Software
All targets and agents should be upgradeable.

2-System Upgrade
Manage Software
Check for missing agents or plug-ins

2-System Upgrade
Backup Repository
•  DBCA or RMAN recommended
•  Unlock SYSMAN account if using DBCA
•  Duplicate from Active feature in 11g makes cloning DB easy
•  For cross-platform migration take a look at note
388090.1
•  Datapump instructions not specific to EM12c repository
upgrade (Open SR with Oracle Support)
•  Cross platform transportable tablespaces instructions not
specific to EM12c repository upgrade (Open SR with Oracle
Support)
•  RMAN Convert database also possible for supported
platforms

2-System Upgrade
Backup Repository
•  Using an existing database via Datapump
•  Enterprise Manager schema needs to be removed from
target database first
$OMS_HOME/sysman/admin/emdrep/bin/RepManager
omshost port SID -sys_password <password> -action
drop
•  Not all objects dropped in some cases
•  See MOS NOTE [ID 1365820.1]
•  EM 12c R2 has a database template

2-System Upgrade
Backup Repository
•  Copy emkey to repository before taking backup
emctl config emkey –copy_to_repos –sysman_pwd <password>
•  Re-secure emkey after taking the backup
•  Verify that SYSMAN schema doesn’t contain invalid
object
SELECT count(*) from user_objects
WHERE status=‘INVALID’

2-System Upgrade
Provide Repository Backup Details

2-System Upgrade
Upgrade OMS and Repository Manually
•  Check pre-requisites for the intended 12.1 repository
using EM Prereq Kit
•  EM Prereq Kit can validate database parameters and
configuration prior to upgrade
e.g. install/requisites/bin/emprereqkit -executionType
upgrade -prerequisiteXMLLoc install/requisites/list -
connectString
"(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)
(HOST=dbhost1)(PORT=1521)))(CONNECT_DATA=(SID=emrep)))"
-dbUser SYS -dbPassword -dbRole sysdba -showPrereqs
•  Some parameters can be automatically adjusted (e.g.
process)
•  The utility is also run automatically by OUI

2-System Upgrade
Repository Database Pre-requisites
•  Verify your initialization parameters
•  PROCESSES parameter is at least 300. The installer can also make the
changes if desired.
•  Set JOB_QUEUE_PROCESSES to 0 (temporarily)
•  Set SHARED_POOL_SIZE to 700MB or greater
•  PGA_AGGREGATE_TARGET should be at least 1GB
•  SGA_TARGET should be at least twice PGA_AGGREGATE_TARGET
•  Grant execute on DBMS_RANDOM to DBSNMP and SYSMAN
•  Unlock ORACLE_OCM account

•  Increase your redo log size to at least 300MB due to
the high level of transactions during the repository.
•  Check to make sure your emkey is in the repository

2-System Upgrade
•  Check to make sure your emkey is in the repository
SQL> select count(*) from MGMT_REPOS_TIME_COEFFICIENT;

COUNT(*)

----------
1

  TIP
Create guaranteed restore point on both the old and
new repository database so that changes can be
easily flashed back;
create restore point PRE_OMS_UPGRADE guarantee
flashback database;

2-System Upgrade
OMS Server Verification
•  Verify that server meets pre-requisites
•  Check ulimit to make sure it’s at least 4096
•  ulimit –n
•  If using RHEL 6 or Oracle Linux 6 – make sure you
have the following package installed or else your
installation may fail at OMS Configuration Assistant.
•  glibc-2.12-1.47.0.2.el6_2.12.x86_64
•  See MOS Note: EM12c: Installation Fails On Linux 6 at "Start
Oracle Management Service" step (Doc ID 1472216.1) for
workaround

2-System Upgrade
cd /stage/
./runInstaller

2-System Upgrade

2-System Upgrade
rpm -qa|grep glibc-2.12-1.47

glibc-2.12-1.47.el6.x86_64

glibc-2.12-1.47.el6.i686

2-System Upgrade

•  EM Prereq Kit generates false warnings for ORACLE_OCM
account if it has expired and processes parameter if less
than 1000!!
•  Run EM Prereq Kit manually and double check database

2-System Upgrade
•  At the end of the installation you can verify whether if
the OMS is running.
/u01/app/oracle/Middleware/oms/bin/
emctl status oms -details

2-System Upgrade
Post-Upgrade
•  Both old and new OMSs are running

2-System Upgrade
Post-Upgrade
•  Create a DB link between the old and new
repositories
•  Upgrade Console in old EM creates/recreates link
•  The link between the two repositories allows
operations performed on the old repository to be
propagated to the new repository.
•  Link also enables DDMP and ADDMP

2-System Upgrade
•  Create DB Link
•  Connect descriptor found in MW_HOME/gc_inst/em/EMGC_OMS1/
emgc.properties
e.g. /u01/app/oracle/Middleware/gc_inst/em/EMGC_OMS1/emgc.properties
•  Remove escape characters
$ cat emgc.properties |grep EM_REPOS_CONNECTDESCRIPTOR
EM_REPOS_CONNECTDESCRIPTOR=(DESCRIPTION=(ADDRESS_LIST
=(ADDRESS=(PROTOCOL=TCP)(HOST=host)(PORT=1521)))
(CONNECT_DATA=(SID=sid)))

2-System Upgrade
Agent Upgrade Steps
•  EM12c only communicates with 12c agents
•  Deploy and configure 12c agents
•  Generate and sign off on health check reports
•  Switch over the agents to the new EM System.
•  Brief outage incurred while 10g agents are stopped and 12c
agents are started

2-System Upgrade
Deploy and Configure Agents

2-System Upgrade
View Job status

2-System Upgrade
Verify and signoff

2-System Upgrade
Verify and sign off Health Report

2-System Upgrade
Switch Agents

2-System Upgrade
Status of agents before switchover in old EM & new EM

2-System Upgrade
View status of Switch Agents Job

2-System Upgrade
After switchover agents status is up in EM 12c
Very minimal (< 1 minute) monitoring downtime!

2-System Upgrade
Post-Upgrade Steps
•  Stop the OCM Scheduler in the 10.2.0.5 OMS
•  Delete Obsolete targets (via console or EMCLI)
•  EM Website
•  EM Website System
•  Oracle Application Server target of 10g OMS ORACLE
HOME consisting of:
•  Oracle Application Server
•  OC4J
•  Oracle HTTP Server
•  Web Cache
•  Delete default incident Rule sets

2-System Upgrade
Post-Upgrade Console
•  Location changed in 12.1.0.2.0
•  In 12.1.0.1.0 go to Setup -> Post Upgrade Tasks
•  In 12.1.0.2.0 go to Setup->Manage Cloud Control->Post
Upgrade Tasks

12.1.0.1.0

12.1.0.2.0

2-System Upgrade
•  Track the status of Accrued Data Migration (ADMP)
job
•  ADMP is data accrued since the repository backup
and agent switchover

2-System Upgrade
•  Track the status of Deferred Data Migration (DDMP)
job
•  DDMP is historical metrics, configuration data which
is moved over time (lazy migration)

2-System Upgrade
•  Diff Report
•  Shows configuration differences between old and new OMS
•  Sign Off
•  Removes old 10.2.0.5 agents

How far did the rabbit hole go?
•  Host level jobs on 10.2.0.5 EM should work before
starting agent deployment
•  Configure PAM on Linux/UNIX for 3rd party authentication if
in use. E.g. LDAP, Kerberos, SecureID
•  Configure Privilege Delegation Provisioning (PDP). i.e. sudo
or Powerbroker for easier deployment
•  Verify that OMS host server can send emails for
notifications
•  Some servers require authentication for SMTP relay
•  Cross-platform repository migration is your
responsibility!
•  Complex but not scary. Read instructions carefully.
•  Test, Test and Test Again!

Managing Databases with EM12c

•  Demo

•  Website - blogs.griddba.com
•  LinkedIn – Leighton Nelson
•  Twitter - @leight0nn
•  Email: leighton.nelson@mercy.net

References
•  OTN Enterprise Manager Page
•  Announcing Enterprise Manager Cloud Control 12c Release 12.1.0.1
Bundle Patch 1(BP1) and 12.1.0.2 Plug-ins [ID 1395505.1]
•  Upgrade Advisor: Enterprise Manager 2-System Upgrade from 10.2.0.5
or from 11.1.0.1 to 12.1.0.1 [ID 327.1]
•  EM 12c: Upgrading to Enterprise Manager Cloud Control 12.1.0.1 on
Linux x86_64 with the 2-System Method [ID 1367919.1]
•  Grid Control Repository: Steps for Cross Platform Migration of the 10g
Grid Control Repository [ID 388090.1]
•  EM12c R2: How to Upgrade to EM 12c R2 - Two System Method [ID
1363162.1]
•  EM 12c R2: How to Apply Upgrade Console Patch for Upgrading to EM
12.1.0.2 [ID 1362537.1]
•  Grid Control Repository: How to Recompile Invalid SYSMAN Objects?
[ID 372706.1]

SAVE THE DATE!

COLLABORATE 13
April 7-11, 2013
Colorado Convention Center
Denver, Colorado

http://collaborate13.ioug.org

Migrating from Oracle Enterprise Manager 10g to 12c Cloud Control

More Related Content

What's hot

Viewers also liked

Similar to Migrating from Oracle Enterprise Manager 10g to 12c Cloud Control

More from Leighton Nelson

Recently uploaded

Migrating from Oracle Enterprise Manager 10g to 12c Cloud Control