This is my Collaborate 13 presentation. ODA is a pre-configured, simple setup, high performance engineered system running 11gR2 cluster. It is a great choice for small to medium sized DBs and if you wish it can be used for Oracle EBS DB too. This paper will show you how the standardized configuration of ODA can be adjusted to comply with the specific requirements of e-Business Suite without sacrificing ODA’s flexibility and supportability. The paper will also share author’s experience migrating, running and maintaining R12 database tier on ODA.

1. Running E-Business Suite Database
on Oracle Database Appliance
Session ID# 11792
Maris Elsins, Yury Velikanov

2. About Maris
Google: Elsins Oracle
[phone|email]
LinkedIn,Twitter, Blog, Slideshare, Latvia
Oracle [Apps] DBA 8 years PL/SQL Developer 3 years
Oracle Certified Master 9i, 10g, 11g, 11i, R12 OCP
Speaker at Oracle Conferences since 2007, 9 of them international
Pythian since 2011

3. About Yury (co-author)
Google: Yury Oracle
[phone|email]
Twitter, LinkedIn, Blog, Slideshare,YouTube
Oracle ACE Director (RAC SIG international chair)
Oracle Certified Master (9i/10g/11g + 15 OCP certificates)
Oracle DBA with 12+ years Apps experience

4. About Pythian
• Recognized Leader:
- Global industry-leader in remote database administration services and
consulting for Oracle, Oracle Applications, MySQL and Microsoft SQL Server
- Work with over 250 multinational companies such as Forbes.com, Fox Sports,
Nordion and Western Union to help manage their complex IT deployments
• Expertise:
- Pythian’s data experts are the elite in their field. We have the highest
concentration of Oracle ACEs on staff – 10 including 3 ACE Directors – and 2
Microsoft MVPs.
- Pythian holds 7 Specializations under Oracle Platinum Partner program,
including Oracle Exadata, Oracle GoldenGate & Oracle RAC
• Global Reach & Scalability:
- Around the clock global remote support for DBA and consulting, systems
administration, special projects or emergency response

5. Agenda
• Oracle Database Appliance as the DB platform
for e-Business Suite
• How Pythian migrated e-Business Suite DB to
ODA

6. Oracle Database Appliance?
- Engineered by Oracle for Oracle
- Simple deployment / pre-configured
- Reliable / pre-tested
- Highly Available
- Good performance
- Easy management
- One vendor
- Pay as you grow licensing
- 50K$ (60K$ for X3-2)

7. Inside the box…
• 2 DB servers, each configured with:
ODA ODA X3-2
2* Xeon X5675 6-core CPUs 2* Xeon E5-2690 8-core CPUs
96Gb Memory 256Gb Memory
500Gb - mirrored storage for OS / SW 600Gb - mirrored storage for OS / SW
1x 10 GbE and 3x 1 GbE 4x 10 GbE

8. Inside the box…
• Shared storage
ODA ODA X3-2
20x 600Gb 3.5inch 15Krpm SAS 20x 900Gb 2.5-inch 10Krpm SAS-2
+DATA
4TB ASM High redundancy 6TB ASM High Redundancy
+RECO
6TB ASM Normal Redundancy (v2.4) 9TB ASM Normal Redundancy
4x 73Gb SSD drives for +REDO 4x 200Gb SSD drives for +REDO
+REDO
90Gb ASM High Redundancy 260Gb ASM High Redundancy
*Optional 2nd storage shelf doubles the
Non-expandable
capacity

9. Limitations
• Limitations by design
* No Scaling Up
* No Scaling Out
* possibility of using NFS Storage (why would you do that)
* Storage Expansion is available for X3-2
• Supportability limitations
* Use oakcli (OracleApplianceKitClient) for any patching
* Additional software not allowed (except agents)
* Virtualization support option (v2.5 / X3-2) can help!
* One-off patches are not supported [ID 1399055.1]

10. ODA Application Kit Releases
• Every ~3 months
* Bundle patch
* Bare Metal OS ISO Images (Patch 12999313)
• Major new features (ODA v1)
* v2.1 – (Oct 2011) – Single 11.2.0.2 OH
* v2.2 – Upgrade to 11.2.0.3
* v2.3 – Support for multiple OHs 11.2.0.[23]
* v2.4 – Normal redundancy for +DATA and +RECO
* v2.5 – (Feb 2013) Oracle VM /X3-2 starts here/
• Think about the upgrade strategies!

11. Certification and Support
• How certification works?
* Oracle certifies HW platforms and Operational Systems
* Oracle does not certify specific hardware configurations
for software products
* ODA is a cluster of two x86-64 OEL 5 servers
* 11gR2 is certified on Linux x86-64 with OEL5
* eBS is certified with 11gR2
eBS DB is certified on ODA
11gR2 and R12.1 are in premier support

12. Monitoring and Management
• How to operate ODA in your IT infrastructure?
* [ID 1415773.1] 3rd party agents are supported!
* Agents “manage, monitor, backup, replicate,
authenticate, or otherwise act on the database, the
server, or the environment”
* NetBackup, Networker, etc.
• Monitoring is important!
* HW: Use Auto Service Request (ASR)
* Monitors HW Faults
* Collects information and creates SRs automatically
* SW: Use your favorite monitoring framework

13. How Pythian Migrated
R12.1 e-Business Suite DB to ODA v2.2

14. Scope of the project
• The Source
- R12.1.3, 10g RAC, old Hardware
• The Target
- Migrate the DB to ODA
- Upgrade the DB from 10.2.0.5 to 11.2.0.3
- Consolidate 4 DBs into one ODA
- Install 900+ e-Business Suite patches

15. Important MOS notes
- Interoperability Notes EBS 12.0 and 12.1 with
Database 11gR2 [ID 1058763.1]
- Using Oracle 11g Release 2 Real Application
Clusters with Oracle E-Business Suite Release
12 [ID 823587.1]

16. eBS DB tier requirements
• Requirements colliding with ODA supportability
* Install products from the 11g Examples CD
* 8 one-off patches for 11.2.0.3
* Requires dedicated oracle listener
* Customized TNS_ADMIN and ORA_NLS10 variables
* No specific requirements for the grid home
Default ODA Oracle Home is not suitable

17. How we did it…
• /u01/cst for the customized software
• Separate Oracle Inventory
* /u01/cst/oraInventory
* Allows keeping the default inventory clean
• Separate Oracle Home
* /u01/cst/{DB_NAME}/product/11.2.0.3
* Cloned it from the default OH
* Installed the Products from Examples CD
* Installed required one-off patches

18. Challenge: the DB upgrade
• Why it was not simple?
* Source DB is 10.2.0.5 on old/slow hardware
* compatible.rdbms = 11.2.0.2 for all ASM diskgroups on
ODA
• What options did we review?
* In-place upgrade + migration to ODA = SLOW
* Install 10gR2 on ODA and create standby = IMPOSSIBLE
because of Database Compatibility of ASM diskgroups
* Rebuild the ASM diskgroups with lower compatible.rdbms
setting = TOO INTRUSIVE

19. Solution: the DB upgrade
- FAQ about Restoring Or Duplicating Between
Different Versions And Platforms [ID 369644.1]
* Restore 10.2 backup using 11.2 software
* Apply archived logs periodically
* alter database open resetlogs upgrade
* Continue the upgrade process
The DB upgrade completed in 1.5 hours

20. Initialization parameters
• Mandatory settings for eBS in [ID 396009.1]
• From a DB created by “oakcli create database”
* _disable_interface_checking=TRUE * db_block_checksum='FULL'
* _ENABLE_NUMA_SUPPORT=FALSE * db_lost_write_protect='TYPICAL'
* _FILE_SIZE_INCREASE_INCREMENT=2143289344 * filesystemio_options='setall'
* _gc_policy_time=0 * parallel_adaptive_multi_user=FALSE
* _gc_undo_affinity=FALSE * parallel_execution_message_size=16384
* _KGL_CLUSTER_LOCK_READ_MOSTLY=TRUE * parallel_min_servers=0
* _kill_diagnostics_timeout=140 * parallel_threads_per_cpu=2
* _lm_rcvr_hang_allow_time=140 * use_large_pages='ONLY'
* db_block_checking='FULL' * compatible=11.2.0.3

21. Separate listener for eBS
• Don’t use port pool 0 for eBS
* The ODA default listener runs on 1521
* Avoid port conflicts!
• Create the listener
* srvctl add listener -l LISTENER_EBSDB -o
$ORACLE_HOME –p 1522
* srvctl setenv listener -l LISTENER_EBSDB -T
TNS_ADMIN=$ORACLE_HOME/network/admin

22. Clusterware DB resource
- srvctl add database -d EBSDB -o /u01/cst/EBSDB/product/11.2.0.3
-p +DATA/EBSDB/PARAMETERFILE/spfile.704.792145311
-a "DATA,RECO,REDO" -n EBSDB
- srvctl setenv database -d EBSDB
-t "TNS_ADMIN=$ORACLE_HOME/network/admin,ORA_NLS10=
$ORACLE_HOME/nls/data/9idata”
- srvctl add instance -d EBSDB -i EBSDB1 -n oda01a-net1
- srvctl add instance -d EBSDB -i EBSDB2 -n oda01b-net1

23. Net Services configuration
• Cluster services use default TNS_ADMIN
* $ORACLE_HOME/network/admin
* It has to be the same on both nodes
• AutoConfig TNS_ADMIN differs on each node
* $ORACLE_HOME/network/admin/$CONTEXT_NAME
• Set the IFILE settings:
* echo "IFILE=${ORACLE_HOME}/network/admin/$
{CONTEXT_NAME}/sqlnet.ora" > $ORACLE_HOME/sqlnet.ora
* Repeat the same steps for tnsnames.ora and listener.ora

24. Impact on ODA’s patching
• Minimal impact
* Default OH and inventory are not customized.
• eBS configuration points visible to oakcli
* /etc/oraInst.loc – switch between inventories manually
* /etc/oratab
* Patch ODA’s components one by one [ID 1458353.1]: use --
infra, --grid, --database options of “oakcli update”
* Avoid running automated update of all DBs on ODA
* Clusterware resources for eBS DB
* Should not cause issues
* Easy to remove and recreate if needed
* Data on ASM diskgroups

25. Impact on eBS DB patching
• No restrictions to apply one-off patches on the
eBS DB Oracle Home
• ODA’s Patch Bundle is needed for
- Patchsets and PSUs to eBS database
* Can’t be installed separately, dependencies on grid OH
* Grid OH has to be patched using ODA Patch Bundles
- Security Updates to eBS database
* PSU patches include security fixes
* eBS database has a PSU installed (11.2.0.3.2)
* ID 854428.1 – If a PSU was installed, future security fixes have to
be applied using subsequent PSUs

26. Future Perspective
• Transformation from ODA to *ODAA
* v2.5 – Oracle VM on ODA
* *ODAA = Oracle Database and Application Appliance J
* Oracle eBS is certified with Oracle VM 3.1.1 [465915.1]
• ODA X3-2
* It’s unlikely ODA v1 will ever support HW upgrades
* Bigger! - more HDD/RAM/CPU
* Better? – (careful with IO)

27. Conclusions
- ODA is great for small/medium sized DBs
- No official support for e-Business Suite DBs yet
- Low-impact customization needed to run eBS DB
- 5 months in production – 0 crashes

28. Questions?
Comments?

29. THANK YOU
#11792 #C13TCH #C13DEN
elsins@pythian.com
@MarisElsins @yvelik
@Pythian