Name: Christina Zanoni Position: Database Administrator with Attorneys’ Liability Assurance Society located in Chicago Worked with Oracle databases for 5 years Been a DBA for 3 years Involved in planning and implementing upgrades Oracle Applications 10.7NCA, 11.5.3, and 11.5.5 Oracle database upgrades for releases 7.3.4 through 8.1.7 Session discusses Tips on successfully upgrading Applications 11i for the Traditional DBA An experienced DBA who typically does not work with Oracle Applications
Information discussed during presentation pertains to Upgrade of Oracle Applications 10.7NCA on Windows NT to Oracle Applications 11.5.3 on Windows NT System Upgraded had very little customization Who in the audience has Oracle Applications Experience Oracle 9i Application Server experience Oracle 9iAs experience useful since it is used as part of the Applications technology tree
The Agenda will consist of An Oracle Applications definition And a discussion of the upgrade process including hardware used Timing of upgrade General tips and techniques on completing a successful upgrade
Oracle Applications is A set of financial management and accounting applications Called modules or products Integrated around a single database Uses stored procedures, functions, Forms, and Reports to perform applications tasks Relies on a set of external “concurrent managers” that are Execution queues to process reports and complex tasks Oracle Applications provides over 150 modules We use only four products: General Ledger, Accounts Receivables, Accounts Payables, Fixed Assets that are part of the Oracle Financials Suite However, we still have to maintain all products in the Product Family. These products belong to the Oracle Financials Product Family. Examples of other Product Families include: Manufacturing, Human Resources, and Public Sector Products Our installation involves almost no customization Typically a site will customize reports, forms, and code objects to provide functionality for their environment Most upgrades require additional time to convert and re-customize functionality
Oracle Applications Modules Involve data and code stored in individual schemas Dependent on more general schemas such as the APPS schema and Applications Object Library (AOL Schema) These schemas help form the primary core of the Applications database APPS schema has access to the complete Oracle Applications Data Model APPLSYS is the user that owns the AOL Schema As any DBA knows, Sys & System, which are installed in every database, are used to perform database administration tasks, APPLSYS & APPS are installed with Oracle Applications, is used to perform applications system administration tasks Passwords for APPLSYS and APPS must be the same Users access data in modules through Granted responsibilities which act like granted database roles
Diagram an example of dependencies between modules Each line represents code or data dependencies between schemas As you can see a module can be dependent on more than one module Must maintain other modules for proper functionality Modules listed on this slide are General Ledger, Accounts Payable, Accounts Receivable, and Cash Management As I stated in the previous slide APPS schema has access to the complete Oracle Applications Data Model Please note that the diagram does not list all the dependencies between these modules
During the Upgrade Objects are modified to provide new functionality with the Upgrade process divided into categories Each category is broken down into database upgrade, applications technology, and product specific tasks Each task has a sequence of steps that run scripts or require user interaction Many steps are not required for every installation – We use only the Financials Product Family so we did not have to perform steps for other Product Families. Categories 1-3 steps are considered PRE-UPGRADE steps AutoUpgrade is an Applications Utility that runs after the completion of Category 3 steps AutoUpgrade will take anywhere from 8 – 13 hours depending on server configuration Upgrades functionality of data from Applications 10.7NCA to 11.5.0 Uses another Applications Utility (called ADPATCH) internally Acts as a Manager that coordinates workers Assigns workers jobs to convert, compile, translate, and upgrade database objects Workers are dependent and operate in parallel of each other Workers may wait on on a failed worker Workers need to be monitored with the AD Controller utility and fixed if necessary Prior to starting the post-upgrade steps, you will need to apply Database driver through ADPATCH to upgrade functionality of data from 11.5.0 to 11.5.3 Will take 11-15 hours again depending on server configuration Post-upgrade and finishing steps are completed after the database driver
1 st Upgrade Preparation Step is to have a solid understanding of the upgrade process and components of Applications 11i This can be accomplished by Researching & reading material that will assist you in planning & successfully implementing the upgrade Oracle Applications Upgrade Manual Contains category upgrade steps needed to be performed to upgrade installation of Oracle Apps Maintaining Oracle Applications Manual Discusses in detail the Applications utilities such as AutoUpgrade & AD Patch that utilized through out the upgrade Oracle Applications Release 11i Notes Lists version of current Rapid Install patch and Consolidated Upgrade Patches & Contains corrections or modifications to the Upgrade/Installation steps You should download the most recent version of the document from Metalink prior to starting the upgrade and check for updates to the document during the upgrade process Other documents Oracle Applications Installation Manual – discusses the Rapid Install process pertinent to any upgrade or installation Oracle Applications Installation Update Notes – discusses platform specific updates to installing Oracle Applications Applications Issues Document on Metalink – lists pre-upgrade patches & solutions to documented issues that may occur during the upgrade. This should be your first resource when an error occurs during the upgrade process The three documents in yellow are the ones that you should read first.
Tapping into outside resources can assist you with the Upgrade This slide lists some resources that assisted us with planning & implementing our Applications 11i upgrade For example… We were having a performance issue when applying the 11.5.5 maintenance pack on our test system We attended a recent Oracle Applications User Group meeting and in the meeting a presenter discussed our same issue. We solved our performance problem with the presenters suggestion (Gather schema statistics for ALL schemas prior to running the maintenance pack) You made need to be a member to attend these meetings
Important Aspect of Upgrade Understanding Server Configuration of Applications 11i Utilizes a 3 tier Architecture Database Tier Middle Tier –Applications Technology stack consisting of Admin Concurrent Processing Forms Reports Apache Web Servers Desktop Tier – consisting of Jinitiator within a standard web browser Uses split configuration to support multiple Oracle Homes which include Database Home – Oracle 8.1.6 files linked to the Oracle 8.1.6 database server Technology Home – Oracle 8.0.6 files linked to the technology stack HTTP Home – Oracle 8.1.6 files linked to the HTTP server
ALAS Chose a single node Configuration or a 2 tier approach due to the fact we have ~10 concurrent users Where the Database Server & Applications Technology Stack is installed onto one server Advantage Maintenance is more simplistic You only have to apply patches to one server Some of the Disadvantages are that Tuning is more complex When you tune one component you are affecting all components of Applications No fail over option when a component of your server goes down
Understanding Applications Release Maintenance is important during the upgrade because it involves applying applications patches to the installation. The Applications Utility ADPATCH is used to apply drivers supplied with the patch Patches usually consist of three patch drivers that are applied against the Applications environment A copy driver that copies files from the patch to directories, relinks executables, and regenerates jar files Jar files – a collection of Java classes compressed into a file for faster download to the desktop client A database driver that runs SQL statements to modify database objects including creating packages, new error messages, adding a new column, view, or table to the database, and adding new seed data to a table A generate driver to generate messages, forms or reports For Windows NT/2000 The patching process requires additional tools to be installed on the server to provide an environment for re-linking executables under the windows operating system Unix already has this type of environment Note: To avoid re-linking issues Tools should not be installed under the Oracle Applications Directory Structure and in directories with spaces in its name When patching an Applications environment - You are altering operating system files and objects in the database That’s why you should Always back up Apps and database servers prior to applying patch Sometimes a patch can cause more harm than good and this will allow you to revert to the backup Different types of patches Individual, One-off, stand-alone patch– fixes one particular bug Mini Pack – cumulative patch that fixes most or all bugs for the release Family Pack – group of mini packs for related products bundled together Family Consolidated Upgrade Patch –needs to be applied prior to starting an upgrade. Includes bug fixes for processes that run during an upgrade. Listed in the Release Notes. Maintenance Pack – a collection of Mini Packs bundled togther on a set of CDs. 3 rd Digit of release will change with full installation of pack. Slide lists a brief summary of the Applications Patching steps on Windows NT
The next few slides pertain to the actual upgrade performed at my company. ALAS used three servers to upgrade to Applications 11i Applications 10.7NCA/Oracle 7.3.4 database server Our production Apps 10.7NCA Server. Function of server during the upgrade was to export full Oracle 7.3.4 database whenever required Apps 10.7NCA remained in production until the completion of Parallel testing after the upgrade Applications 10.7NCA/Oracle 8.1.6 database server Clone of our Apps 10.7NCA/Oracle 7.3.4 database server with Oracle 8.1.6 installed on it in split configuration mode Applications 11i server server had the 11i system installed “out of the box” for each iteration Server became our production Applications 11.5.3 server Type of configuration allowed us to move rapidly from production Applications 10.7NCA/Oracle 7.3.4 to Applications 11.5.3/Oracle 8.1.6
On Oracle 8.1.6 server we performed the following step one time only Installed Oracle 8.1.6 in server partition mode Copied over needed directories from Apps 11i Server to complete Categories 2 & 3 Overall this process functioned with only one issue -- Had to repeat one of the steps in Category 3 on the Apps 11i Server due to differing directory structures The following steps on Oracle 7.3.4 server were performed for each iteration Created full export of Oracle 7.3.4 database Copied export to Oracle 8.1.6 server On Oracle 8.1.6 server for each upgrade iteration Imported Oracle 7.3.4 database export into newly created Oracle 8.1.6 database Ran adprepdb SQL script that runs some RDBMS scripts against the database that are required to upgrade Applications. Sets up some additional grants and synonyms that are required. Obsolete in Apps 11 – replaced with a addbXXX.sql (XXX – Version of RDBMS database) Applied Interoperability patches for functionality between Apps 10.7NCA and Oracle 8.1.6 Initially, we thought we might upgrade the database first and then applications. So, we included this step in the database upgrade We did not change this step since numerous invalid database objects owned by the APPS schema were fixed by these patches This slide will summarize the steps taken to upgrade to Applications 11i On Apps 11i Server First step was to install the Release Maintenance Tools on a clean server containing only the Windows NT operating system Installed Applications 11.5.0 by running the most recent version of Rapid Install downloaded from METALINK. While running Rapid Install Performed a single node install Also, We discovered that selecting the “Install Oracle Applications” operation through Rapid Install was the only option that gave us all the components needed for Apps 11i to function properly. The option “Create Upgrade File System” did not give us the necessary components to upgrade to Applications 11.5.3. A config.txt file is created during this step that will be needed when Rapid Install is run again during a Category 4 step Applied most current Applications DBA and family consolidated patches to modules using the Applications Utility ADPATCH in pre-install mode These steps should be performed for every iteration of the upgrade process -We did not do this the first time and had issues
Performed Categories 1 through 3 Pre-Upgrade Steps We did not have to perform a majority of Categories 1 and 2 steps – since we use such few products & our installation had minimal customization. We applied every patch listed in the upgrade process since Oracle Applications Modules are dependent on so many other modules We performed all of the Category 3 steps for our Product Family Intermedia & Java are installed during Category 3 – If you do not have sufficient memory in your server this step will not complete successfully. After the completion of Category 3 steps, we copied the upgraded dbf files to Applications 11i Server and replaced the dbf files created by Rapid Install The following steps on the Apps 11i Server were performed for each upgrade iteration Executed Applications Utility AutoUpgrade to Upgrade data from Apps 10.7NCA to Apps 11.5.0 Executed Applications Utility AD Patch and ran the database driver to upgrade data from 11.5.0 to 11.5.3 As part of Category 4 steps, we ran Rapid Install and chose the “Configure Existing Applications Instance” operation to configure and start server processes. Verified that the Concurrent Manager for the database SID had been created If not, create it by running an exe file (ccmsetup.exe) located under the FND directory structure. Verify that Concurrent Manager NT Service is running under the Applications owner account or the NT Service will not function properly. We completed the remaining Categories 4 through 6 steps on Apps 11i Server we executed Final Upgrade steps including setting up the Applications Client Software by installing Jinitiator on client PCs Finally, the software was released to the Applications Users
I started the upgrade at 6:00pm on a Friday Users were using the system at Noon on the following Wednesday Data files grew from ~ 7GB prior to upgrade for an Oracle 7.3.4 database to ~14GB after upgrade for an Oracle 8.1.6 database Oracle Applications Users performed 8 weeks of Acceptance testing prior to Upgrade Most issues were found during Acceptance testing Apps Users performed 6 weeks of parallel testing after the Upgrade Found very few issues due to 8 weeks of Acceptance testing by the actual users of the system
The majority of the upgrade, excluding addressing new issues, was spent completing the database upgrade, category 3 steps, AutoUpgrade and the database driver to upgrade functionality to Apps 11.5.3. An interesting note: The database driver for the upgrade from 11.5.0 to 11.5.3 took longer than the upgrade from 10.7NCA to 11.5.0. Plus the database driver for the 11.5.5 Maintenance Pack took longer to upgrade the data from 11.5.3 to 11.5.5 than it did to upgrade the data from 11.5.0 to 11.5.3 New issues resulted from the fact that we only practice the upgrade steps one time
Most upgrades usually involve issues that prevent you from upgrading your system until the issues are resolved First resource to check for solutions on issues is the current version of the Applications Issues Document as discussed in a previous slide If no solution exists in the document for the issue -- search METALINK for notes, documents, and fixes for published bugs that can assist with solving the issue during an upgrade For instance, we had an ORA-25150 error relating to locally-managed tablespaces to cause all our workers to fail during AUTOUPGRADE Found a note on METALINK that recommends converting locally-managed tablespaces back to dictionary-managed. Applied suggestion and workers for AUTOGRADE continued successfully Resolved issue within 15 minutes without assistance from Oracle Support
Issues may occur in the upgrade when you have to open an iTar with Oracle support due lack of information or solutions on Metalink. You should supply as much info as possible when opening an iTar to expedite the resolution of the issue. The information pertinent to solving any application upgrade issue is listed on this slide. When performing an upgrade, this information is always requested for by the Support Analyst It will save you some time going back & forth with the Analyst if you provide this information upfront A little tidbit on getting an Analyst to update the status of your iTar An iTar’s response status changes every time you update an iTar Updating it 3 times will mark the iTar with an immediate call back status and the analyst will post some kind of status to the iTar to let you know if they are working on it
Extremely important to Document each pre-category, category, and post-category upgrade steps in detail If you do not… May overlook a step that may seem insignificant, but actually affects the success of upgrade Practice documented upgrade steps as many times possible To resolve any additional issues that may arise during the test phase To calculate an accurate time for the upgrade Follow the exact documented steps because varying from your test plan can introduce new issues
In Summary When preparing for the Apps 11i Upgrade Research as much material as possible prior to starting the upgrade to assist you with building a solid foundation of knowledge on the new products and functionality in Apps 11i. For example, Oracle Applications requires Cost Based Optimization and locally managed tablespaces in the new release Also, you should have a solid understanding of the components and configuration of the software Flowchart steps to be performed during the upgrade to assist you with documenting and planning upgrade steps Please see my white paper for a detailed example of a flow chart Finally you should practice, practice, practice the upgrade to test upgrade steps and timing expectations.
Finally, here are some lessons that we learned after the upgrade It is important to understand how Cost Based Optimization and gathering schema statistics affects performance during an Applications upgrade, patch application, or when its in production. Upgrading your database directly to Oracle 8.1.7 will avoid the ORA-3113 end of file on communication channel errors during the Gather Schema statistics process. Applications 11.5.3 works with Oracle 8.1.7 after applying interoperability patches to the environment to provide functionality between Oracle 8.1.7 and Applications 11.5.3. However, the 11.5.5 installation installs Oracle 8.1.7 thus avoiding the install of the interoperability patches. Allow sufficient disk space for growth of data files and new products added during Applications maintenance packs Need to monitor locally managed tablespaces closely because they many have to be manually extended even if they have been set to autoextend
I would like to thank you for attending my presentation. Please make sure that you fill out the survey before you leave. Any questions?
Power point 707 tips on successfully upgrading apps 11i for the non apps dba
707- Tips on SuccessfullyUpgrading Applications 11i forthe Non-Apps DBA Christina Zanoni ALAS, Inc.
Introduction Upgrade of Non-customized Oracle Applications version 10.7NCA to 11.5.3 on Windows NT Useful Experience – Oracle Applications – Oracle 9i Application Server (9iAs)ALAS, Inc.
Agenda What is Oracle Applications? Understanding the Upgrade Process Preparing for the Upgrade Server Configuration for Apps 11i Performing the Upgrade Tips on Solving Upgrade Issues The Importance of DocumentationALAS, Inc.
What is Oracle Applications? Set of financial management and accounting applications Provides over 150 modules Uses stored procedures and functions “Concurrent managers” - execution queues that process reports and complex tasksALAS, Inc.
Oracle Applications Modules Data and code stored in individual schemas Dependent on other more general schemas Users access data in modules through granted “Responsibilities” (e.g. System Administrator, Receivables Manager)ALAS, Inc.
Oracle Applications Modules - Dependencies Note: This diagram does not include all dependencies! APPS AP AR GL CEALAS, Inc.
Understanding the UpgradeProcess Pre-upgrade Steps (Categories 1-3) AutoUpgrade – Acts as a manager that coordinates WORKERS – Assigns workers jobs to convert, compile, translate, or upgrade objects – Workers need to be monitored with the AD Controller utility (ADCTRL) and fixed if necessary Post-upgrade Steps (Categories 4-6) Finishing Steps (After Post-upgrade steps)ALAS, Inc.
Preparing for the Upgrade –Required Reading Oracle Applications Concepts Manual Oracle Applications Upgrade Manual Maintaining Oracle Applications Manual Oracle Applications Release 11i Notes – download most recent version from Metalink Oracle Applications Installation Manual Oracle Applications Installation Update Notes Applications Issues Document on MetalinkALAS, Inc.
Preparing for the Upgrade –Upgrade Resources 11i Upgrade Section of Oracle Appsnet http://www.oracle.com/appsnet/technology/upgrade Attend local Oracle Applications User Group meetings http://www.oaug.org/geo/geoweb.html Join listserv - http://www.oaug.org/public/oaugnet/oaugnet.htmlALAS, Inc.
Applications 11i ServerConfiguration – 3 Tier Database Database Server (Oracle8i Enterprise Edition) Tier Applications Technology Stack (8.0.6 Middle Oracle Home) Tier (Includes the Admin, Concurrent Processing, Forms, Reports, and Apache Web Servers) Desktop JInitiator within a Standard Web Browser TierALAS, Inc.
ALAS’ Server Configuration –Single Node Install Database Server & Applications Technology Stack (dual 500Mhz CPU, 1 GB RAM, 54 GB of hard disk space) JInitiator within a Standard Web Browser (IE 5.5 & 256 MB RAM for desktops) Advantages – Simple Maintenance Disadvantages – Complex Application Tuning – Fail Over Server does not existALAS, Inc.
Applications ReleaseMaintenance Required tools for Release Maintenance on Windows NT/2000 – Microsoft Visual C++ – MKS Toolkit – emulates the Unix environment – gnu make 3.79.1 Applications Patching Steps – Back up Oracle Applications and database servers – Shut down all forms services, web listener services, and concurrent manager services – Verify that the database and database listeners are running – Execute command file under APPL_TOP to set Applications Environment – run ADPATCHALAS, Inc.
Servers Used during Upgrade Applications 10.7NCA/Oracle 7.3.4 database server (dual 350Mhz CPU, 1 GB RAM, ~30 GB of hard disk space) Applications 10.7NCA/Oracle8i database server (dual 500Mhz CPU server, 500 MB RAM, ~30 GB of hard disk space) Applications 11i server (dual 500Mhz CPU server, 1 GB RAM, 54 GB of hard disk space – added an additional 18 GB after upgrade)ALAS, Inc.
Performing the UpgradeApps 10.7NCA/7.3.4 Apps 10.7NCA/8.1.6 Apps 11i1. Create full 1. Install Oracle 8.1.6 1. Install Release export of Maintenance Tools Oracle 7.3.4 1. Import Oracle 7.3.4 2. Run Rapid Install database database export into 3. Apply most current AD2. Copy export Oracle 8.1.6 database and family consolidated to 8.1.6 2. Run adprepdb & patches in pre-install mode server apply Interop Patches 1. Autoupgrade 3. Categories 1-3 steps 2. 11.5.0 to 11.5.3 database 4. Copy dbf files to Apps driver 11i server 3. Categories 4-6 steps 4. Final upgrade stepsALAS, Inc. 5. Production!
Performing the UpgradeApps 10.7NCA/7.3.4 Apps 10.7NCA/8.1.6 Apps 11i Create full Install Oracle 8.1.6 Install Release export of Maintenance Tools Oracle 7.3.4 Import Oracle 7.3.4 Run Rapid Install database database export into Apply most current AD Copy export to Oracle 8.1.6 database and family consolidated 8.1.6 server Run adprepdb & apply patches in pre-install mode Interop Patches Autoupgrade Categories 1-3 steps 11.5.0 to 11.5.3 database Copy dbf files to Apps driver 11i server Categories 4-6 steps Final upgrade stepsALAS, Inc. Production!
Summary of Upgrade Started at 6:00pm on Friday and completed at Noon on Wednesday Data file sizes – ~7 GB prior to upgrade – ~ 14 GB after upgrade 8 weeks of acceptance testing prior to Upgrade found most issues 6 weeks of parallel testing after Upgrade found very few issuesALAS, Inc.
Upgrade Timing by Category(Production Upgrade) Oracle 8.1.6 database upgrade – 8 hrs Categories 1 and 2 steps – 4 hrs Category 3 steps – 8 hrs AutoUpgrade – 13 hrs Database driver for 11.5.3 upgrade– 15 hrs Post Category and Finishing Your Upgrade Steps – 3 hrs Fix Acceptance testing issues – 5 hrs Run Gather Schema Statistics for All Schemas – 4 hrs Address new issues during production upgrade – 36 hrs Total Time : 96 hoursALAS, Inc.
Solving Upgrade Issues Search Metalink notes and bugs for solutions Example of solving an upgrade issue – Received an oracle error: ORA-25150 that caused all the workers to fail during AutoUpgrade – Logged on to Metalink and searched on “ORA-25150”ALAS, Inc.
Solving Upgrade Issues(continued) Creating an iTar – Information to submit Category Step Log files, including adpatch.log & adwork0x.log Script name and header information that caused error List of patches applied Updating an iTar – 3 updates = immediate call back statusALAS, Inc.
The Importance of Documentation Document each pre-category, category, and post-category step of the upgrade in detail Document the amount of time that each step took to complete Document every issue and solution Follow the exact steps in your documentation during the production upgradeALAS, Inc.
Summary Prepare before starting the upgrade Have a solid understanding of the components and configuration for Apps 11i server Document each upgrade step and timing in detail Allow enough time to address any issues that arise during the upgrade Practice, Practice, PracticeALAS, Inc.
Lessons Learned Gather schema statistics on a frequent basis for Cost Based Optimizer (CBO) – Beginning with 11i, all query optimization uses CBO – Tables must be analyzed for CBO to effectively determine the best execution plan for SQL statements Upgrade database to Oracle 8.1.7 to avoid Ora-3113 errors when gathering schema statistics Allow enough disk space for growth of data files and Applications maintenance packs Monitor growth of Locally-managed TablespacesALAS, Inc.