The document presents expert tips and tools for managing Oracle RAC 12c, including features, management tools, support utilities, and enhancements in version 12.1.0.2. It covers the management of services, modification commands, and the use of Oracle Enterprise Manager and diagnostic tools like Orachk and TFA. The document emphasizes the importance of system monitoring and optimization tools to ensure database efficiency and health.

1. EXPERT TIPS AND
TOOLS FOR
MANAGING ORACLE
RAC 12C
LEIGHTON NELSON
LEAD DBA, WORLDWIDE
TECHNOLOGIES
2.25.2015

2. ABOUT ME
• Oracle DBA
• Oracle ACE
• Speaker
• Co-Author – Expert Oracle Enterprise Manager Cloud Control 12c
• RAC Attack Ninja
http://blogs.griddba.com
linkedin.com/pub/leighton-nelson/4/6b8/a7a
@leight0nn

3. AGENDA
• Oracle RAC 12c Features
• Management Tools
• SRVCTL
• CRSCTL
• Oracle Enterprise Manager
• Support Tools
• ORAchk
• TFA
• ADR
• PDBs and Services in RAC
• What’s New in 12.1.0.2

4. ORACLE RAC 12C OVERVIEW
• Flex ASM
• Flex Clusters

5. ORACLE RAC 12C OVERVIEW
Flex ASM
• Pre-12c ASM instances required 1-to-1 mapping to database
instances
• Flex ASM instance not needed on all nodes

6. ORACLE RAC 12C OVERVIEW
Flex Clusters
• Hub and Leaf Nodes
• Leaf nodes require shared storage
• Requires Flex ASM
• Only Hub node host databases

7. MANAGEMENT TOOLS
• SRVCTL enhanced to accept full-word options
• New SRVCTL commands have full-word options only
• Existing SRVCTL commands support single letters for backwards
compatibility
• More intuitive
• Single letter options confusing at times
• Inconsistent
• Allows for more options

8. MANAGEMENT TOOLS
srvctl modify service –s oltp –d orcl –r orcl1 –a orcl2 –P
BASIC –e SELECT –m BASIC –w 60 –j –SHORT –B SERVICE_TIME

9. MANAGEMENT TOOLS
srvctl modify service –s oltp –d orcl –r orcl1 –a orcl2 –P
BASIC –e SELECT –m BASIC –w 60 –j –SHORT –B SERVICE_TIME
srvctl modify service –service oltp –db orcl –preferred
orcl1 –available orcl2 –tafpolicy BASIC –failovermethod
NONE –timeout 60 –clbgoal SHORT –rlbgoal SERVICE_TIME
• Show single letter options using the –compatibile option after -help

10. MANAGEMENT TOOLS
• Specify input file for passing lengthy options using –file option
>cat services.in
-service radwwt_basic -db radwwt -serverpool wwt_sp -
cardinality uniform -clbgoal SHORT -rlbgoal SERVICE_TIME -
role PRIMARY -policy AUTOMATIC –verbose
>srvctl add service -file services.in

11. MANAGEMENT TOOLS
What-If Command Evaluation
• The –eval option allows commands to be simulated without
modifying the system
• Available for both SRVCTL and CRSCTL
• Supported SRVCTL Commands
srvctl add database
srvctl add service
srvctl add srvpool
srvctl modify database
srvctl start database
srvctl start service
srvctl stop database
srvctl stop service

12. MANAGEMENT TOOLS
What-If Command Evaluation

13. MANAGEMENT TOOLS
What-If Command Evaluation

14. MANAGEMENT TOOLS
• Using CRSCTL some attributes can be modified online without a
restart
• Modification of ora.* resources enforced
• crsctl modify ora.* resource fails with CRS-4995 in 12.1.0.2 and above
(Doc ID 1918102.1)
• Only user-defined resources can be modified
• There are exceptions
• CRS_STAT commands deprecated as of 11.2
• Replaced with CRSCTL status resource

15. MANAGEMENT TOOLS
What if?

16. MANAGEMENT TOOLS
Show what would happen if a resource failed and cannot be restarted
on the local node
• SRVCTL predict
• CRSCTL eval fail

17. MANAGEMENT TOOLS
Display status of all databases running running in a particular home
• srvctl status database–DB -thishome
• Display status of all databases running with same version
• srvctl status database–DB -thisversion

18. MANAGEMENT TOOLS
Oracle Enterprise Manager
• Cloud Control 12c
• EM Express 12c

19. MANAGEMENT TOOLS

20. MANAGEMENT TOOLS
Oracle Enterprise Manager

21. SUPPORT TOOLS
• ORAchk aka RACcheck
• Trace File Analyzer (TFA) Collector
• Automatic Diagnostics Repository (ADR)

22. SUPPORT TOOLS
• ORAchk provides health checks for Oracle stack
• Databases (Single Instance and RAC)
• Enterprise Manager Repository
• E-Business Suite
• ORAchk now included in 12c binaries
• $ORACLE_HOME/suptools
• Recommended to download latest version from MOS
• ORAchk - Health Checks for the Oracle Stack (Doc ID 1268927.2)

23. SUPPORT TOOLS
When to run ORAchk
• After Initial Oracle RAC Deployment
• Before maintenance
• After maintenance
• At least once every three months

24. SUPPORT TOOLS
• Setup ORAChk to check start automatically
>./orachk –auto_restart
• Configure expect utility or password-less sudo
• see http://expect.sourceforge.org
• Configure scheduled collection
>./orachk -id DBA -set
"NOTIFICATION_EMAIL=leighton.nelson@wwt.com;AUTORUN_SCHEDU
LE=4,8,12,16,20 * * *;AUTORUN_FLAGS=-profile
dba;COLLECTION_RETENTION=30"

25. SUPPORT TOOLS
• Use profiles to run specific checks
• >./orachk –profile <profilename>

26. SUPPORT TOOLS
Trace File Analyzer
• Installed in $GI_HOME/suptools
• Executed from $GI_HOME/tfa/bin
• Collects diagnostics, traces, log files for DB, Clusterware, OS across
the cluster
• Can be configured to trigger collection automatically
• Starts automatically on reboot

27. SUPPORT TOOLS
Trace File Analyzer
• Ability to analyze individual components
• $ tfactl analyze –node racdb01,racdb02 -comp crs -since 1h
• Integrates with oratop, oswatcher etc
• Some components require root or sudo access
• Repository stored in GI_BASE/tfa/repository
• Default size of 1GB

28. SUPPORT TOOLS
Trace File Analyzer

29. SUPPORT TOOLS
• Automatic Diagnostics Repository for G.I.
• Clusterware log files managed and treated like database related
logs, trace
• $GI_BASE/diag/
• adrci interface

30. SUPPORT TOOLS
• ADRCI for CRS logs

31. SUPPORT TOOLS
• ADRCI for CRS logs

32. SUPPORT TOOLS
Cluster Health Monitor (CHM/OS)
• Introduced since 11.2.0.1
• Collects OS level metrics using OS APIs
• Stores data in management repository (GIMR)
• Integrated with QoS and Oracle Enterprise Manager

33. SUPPORT TOOLS
Cluster Health Monitor (CHM/OS)
Processes
• osysmond
• ologgerd
• oclumon
Clusterware Resource
$ crsctl status resource ora.crf -init
NAME=ora.crf
TYPE=ora.crf.type
TARGET=ONLINE
STATE=ONLINE on ngdcracdb01

34. PDBS AND SERVICES IN RAC
• A default service is created with each PDB
• Default service is the same global name as the PDB
• Default service is NOT registered with Clusterware
• Create additional services for application access
• PDBs are mounted by default
• Starting a service opens the PDB
• SINGLETON service opens PDB on the node it runs on
• Use UNIFORM service to access PDB on more than one node
• Stopping a service doesn’t close the PDB
• PDB remains open but inaccessible
• Remember PDBs exist on all RAC nodes running CDB!
• Specify INSTANCES keyword when modifying PDB state

35. MANAGING PDBS
• SQL Plus
• SQL Developer
• Oracle Enterprise Manager Cloud Control 12c
• Oracle Enterprise Manager Express 12c

36. WHAT’S NEW IN 12.1.0.2
• Ping Targets
• Probe nodes in Virtual Cluster for network failure
• Detects reachability for targets on public network
• Rapid Home Cloning
• Store gold images of database software in RHP server
• Deploy images to dev/test environments
• ASM Filter Driver
• Consistent naming of ASM disks across cluster (replaces ASMLib)
• Filters I/O to ASM disks to prevent accidental overwriting

37. WHAT’S NEW IN 12.1.0.2
Ping Targets
• Configured during initial cluster install
• Set after installation using srvctl
[root@racdb01 ~]# srvctl modify network -k 1 -pingtarget "10.255.224.198"
[root@racdb01 ~]# srvctl config network
Network 1 exists
Subnet IPv4: 10.255.224.0/255.255.255.0/eth0, static
Subnet IPv6:
Ping Targets: 10.255.224.198
Network is enabled
Network is individually enabled on nodes:
Network is individually disabled on nodes:

38. WHAT’S NEW IN 12.1.0.2
Rapid Home Cloning
• Store gold images of database software in RHP server
• rhpctl import image -image DB12102_base -path
/u01/db/oracle/12.1.0.2/db_1/ -imagetype ORACLEDBSOFTWARE
• Deploy images to dev/test clusters
• rhpctl add workingcopy -workingcopy DB12102Home -image
DB12102_base
• Fast and standardized deployment of Oracle binaries

39. WHAT’S NEW IN 12.1.0.2
ASM Filter Driver
• The verdict is still out
• http://flashdba.com/2014/07/25/oracle-12-1-0-2-asm-filter-driver-first-
impressions/

40. REFERENCES
• Oracle RAC 12c
http://www.oracle.com/technetwork/database/options/clustering/rac-
wp-12c-1896129.pdf
• Oracle® Clusterware Administration and Deployment Guide 12c
Release 1 (12.1)
• Oracle RAC 12c Operational Best Practices
http://www.slideshare.net/MarkusMichalewicz
• Oracle RAC, Oracle Data Guard, and Pluggable Databases: When
MAA Meets Oracle Multitenant
http://www.ludovicocaldara.net/dba/maa-multitenant-presentation/

41. FREE TRIAL--RESOLVE PERFORMANCE
ISSUES QUICKLY
• Try Database Performance Analyzer FREE for 14 days
• Insight into overhead caused by delays in RAC coordination and
cache operations
• Improve root cause of slow performance
• Quickly identify root cause of issues that impact end-user response time
• See historical trends over days, months, and years
• Understand impact of VMware® performance
• Agentless architecture, installs in minutes
41© 2015 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
www.solarwinds.com/dpa-download/

42. QUESTIONS?
42© 2015 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.

43. THANK YOU
© 2015 SOLARWINDS WORLDWIDE, LLC. ALL RIGHTS RESERVED.
The SOLARWINDS and SOLARWINDS & Design marks are the exclusive property of SolarWinds Worldwide, LLC, are registered with the U.S.
Patent and Trademark Office, and may be registered or pending registration in other countries. All other SolarWinds trademarks, service marks,
and logos may be common law marks, registered or pending registration in the United States or in other countries. All other trademarks
mentioned herein are used for identification purposes only and may be or are trademarks or registered trademarks of their respective companies.
43