0
From 20 hours to 20 minutesOracle e-business cloningusing Oracle Data GuardORACLE E-Business Suite DBA { APPS DBA }onsite ...
Target Audience•   Oracle APPS DBAs•   Oracle DBAs•   Technical Managers•   Oracle Sysadmins•   Disaster Recovery planners...
Technology Used
Current setupProduction Instance                   Dev Instance                                        To               Ar...
What is Active DataGuard• The ability to use your Physical Standby  for much more than just High Availability or  Disaster...
Backup done to   Sent to    3 Hours to restoretape             Bryansto   from Tape                 n
Implementation of Data Guard and Oracle RMAN             Archives shipped to                 Data Guard                   ...
New Business Requirement / Audit              finding• Ebus support team must not have access to  production environment• ...
Requirement•   Create a exact copy of Production daily•   Scramble Salary information•   Close all open workflows•   Reset...
IDEA was born !!!      • Create a exact copy of        Production daily using        dataGuard as an engine
Mirror image of Production
Requirements for daily copy• Every patch which is applied to Production is applied TEST { Strict change  control}• ORACLE ...
Datafiles            Log and out             {APPLCSF}
Below is a diagram on the components and how they arerefreshed onsite
Step 1 Application Server shutdown
Step 2 Shutdown Dataguard instance#### SHUTDOWN DATAGUARD FOR EBUS CLONING ###########00 1 * * * /home/oradg/refresh/shutd...
Step 3 Database Tier05 1 * * * /home/oratst/refresh/clone.sh > /home/oratst/refresh/log/Dailyclone-$(date+%Y%m%d_%H:%M:%S)...
##### KILLING LOCAL PROCESSES #######echo KILLING LOCAL PROCESSESdate. /home/oratst/.profilekill_local > /home/oratst/refr...
########## COPING DATAFILES ##################echo COPY DATAFILES 1cp /dg001/oradata/*.dbf /tstdb001/oradataecho COPY DATA...
Step 3 Start Dataguard instance#### SHUTDOWN DATAGUARD FOR EBUS CLONING ###########00 1 * * * /home/oradg/refresh/shutdown...
Step 4 Appstier00 4 * * * /home/appltst/refresh/cloneapps.sh > /home/appltst/refresh/log/CLONE-APPS--$(date+%Y%m%d_%H:%M:%...
appltst@ebusapp:/tstapp001/tstapp/inst/apps/tst_ebusapp/admin/scripts> more /home/appltst/refresh/changeappspassword.sh. /...
Additional steps• Close all workflowsUpdate apps.wf_notificationsSet mail_status=SENT,status=CLOSEDwhere mail_status=MAILa...
• Set override addressupdate fnd_user set email_address = rynod@medscheme.co.za;update wf_local_roles set email_address = ...
Scramble Salary informationupdate peR_pay_proposals set proposed_salary_n = ROUND((proposed_salary_n/assignment_id)*4732,2...
We have noticed the following cost           saving to business•   No requirements of tapes•   No requirements for assista...
We have noticed the following cost           saving to business•   No requirements of tapes•   No requirements for assista...
Disaster Recovery•   Shutdown DataGuard     –   Stop the listener {lsnrctl stop dg}     –   Stop DataGuard { recover manag...
Database Cleanup  Files which can be deleted              –
APPS Server Cleanup   Files which can be deleted                –
Summary of Environment
What does functional support team                   say !               Due to audit requirements that none of support sta...
The benefit that we have is that we can now experiment (play around)                  with the latest records when problem...
The old process took extremely long and had a lot of manual                           steps. After the new process was imp...
We use TST on a daily basis for any issues we come across on                           PROD that we first want to test out...
TST is very helpful and has enabled us to assist our end users                         with their system queries – logging...
By applying Oracle’s active Data Guard solution we’ve solved key                        issues we had in the past both fro...
Questions & AnswersFeedback
AHMED JASSAT SOUTH ARICAN ORACLE USER GROUP PRESENTATION
AHMED JASSAT SOUTH ARICAN ORACLE USER GROUP PRESENTATION
AHMED JASSAT SOUTH ARICAN ORACLE USER GROUP PRESENTATION
AHMED JASSAT SOUTH ARICAN ORACLE USER GROUP PRESENTATION
AHMED JASSAT SOUTH ARICAN ORACLE USER GROUP PRESENTATION
AHMED JASSAT SOUTH ARICAN ORACLE USER GROUP PRESENTATION
Upcoming SlideShare
Loading in...5
×

AHMED JASSAT SOUTH ARICAN ORACLE USER GROUP PRESENTATION

412

Published on

How Medscheme saved time , money and resources on cloning Oracle EBUS using dataguard

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
412
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
10
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "AHMED JASSAT SOUTH ARICAN ORACLE USER GROUP PRESENTATION"

  1. 1. From 20 hours to 20 minutesOracle e-business cloningusing Oracle Data GuardORACLE E-Business Suite DBA { APPS DBA }onsite at Medscheme
  2. 2. Target Audience• Oracle APPS DBAs• Oracle DBAs• Technical Managers• Oracle Sysadmins• Disaster Recovery planners• Technical Project Managers• Technology architects
  3. 3. Technology Used
  4. 4. Current setupProduction Instance Dev Instance To Archives Shipped DGuard
  5. 5. What is Active DataGuard• The ability to use your Physical Standby for much more than just High Availability or Disaster Recovery!
  6. 6. Backup done to Sent to 3 Hours to restoretape Bryansto from Tape n
  7. 7. Implementation of Data Guard and Oracle RMAN Archives shipped to Data Guard - Archives backed up every 30 minutes – Database backup every Friday
  8. 8. New Business Requirement / Audit finding• Ebus support team must not have access to production environment• Oracle Developers must not have access to production data• Only directors and managers can have access to sensitive data• Sysadmin login should be controlled
  9. 9. Requirement• Create a exact copy of Production daily• Scramble Salary information• Close all open workflows• Reset all database passwords { system,dbsnmp}• Reset application passwords { apps, applsys}• Reset Sysadmin password• Set workflow over-ride address• Reset functional support team passwords• No budget for this !!!!
  10. 10. IDEA was born !!! • Create a exact copy of Production daily using dataGuard as an engine
  11. 11. Mirror image of Production
  12. 12. Requirements for daily copy• Every patch which is applied to Production is applied TEST { Strict change control}• ORACLE HOMES between the servers are at the same patch levels• We do not copy log and out files { $APPLCSF }• All custom forms and reports are copied both to TST and Production environment• If a Data file is added on the dbTier on the Target then you would have to recreate the xml file on the dbTier on Test environment• Printers which are configured on production must be configured using the same name on development servers• Not tested on Windows• Initial clone is done on the source system to create the xml file using Cloning Oracle Applications Release 12 with Rapid Clone [ID 406982.1]
  13. 13. Datafiles Log and out {APPLCSF}
  14. 14. Below is a diagram on the components and how they arerefreshed onsite
  15. 15. Step 1 Application Server shutdown
  16. 16. Step 2 Shutdown Dataguard instance#### SHUTDOWN DATAGUARD FOR EBUS CLONING ###########00 1 * * * /home/oradg/refresh/shutdownDG.sh > /home/oradg/refresh/log/shutdownDG.log################ STARTUP DATAGUARD FOR EBUS CLONING ################################15 2 * * * /home/oradg/refresh/startDG.sh > /home/oradg/refresh/log/startupDG.logoradg@ebusdb:~> more /home/oradg/refresh/shutdownDG.sh##### Running environment file #####################################. /home/oradg/env_DG.sh########### Shuting down the listener ##############################lsnrctl stop dg######## Shutting down the DB ######################################sqlplus /as sysdba << 0xffselect DB_UNIQUE_NAME from v$database;recover managed standby database cancel;shutdown immediate;exit0xffdatedate##### DATAGUARD ENVIRONMENT IS SHUTDOWN ##########################oradg@ebusdb:~>
  17. 17. Step 3 Database Tier05 1 * * * /home/oratst/refresh/clone.sh > /home/oratst/refresh/log/Dailyclone-$(date+%Y%m%d_%H:%M:%S).log 2>&1oratst@ebusdb:~> more /home/oratst/refresh/clone.sh. /home/oratst/env_tstdb.shexport ORACLE_HOME=/tstdb001/11.2/TSTexport ORACLE_SID=tstexport TNS_ADMIN=/tstdb001/11.2/TST/network/admin/tst_ebusdbexport PATH=$PATH:$ORACLE_HOME/binecho CLEANING OUT THE INVENTORYexport PERL5LIB=/tstdb001/11.2/TST/perl/lib/site_perl/5.10.0:/tstdb001/11.2/TST/appsutil/perlrm -rf /home/oratst/oraInventory/*rm -rf /tstdb001/11.2/TST/appsutil.bakrm -rf /tstdb001/11.2/TST/admin/tst_ebusdb/diag/rdbms/tst/tst/incident/*> /tstdb001/11.2/TST/admin/tst_ebusdb/diag/tnslsnr/ebusdb/tst/trace/tst.logrm /tstdb001/11.2/TST/log/diag/tnslsnr/ebusdb/tst/alert/*.xmlrm /tstdb001/11.2/TST/admin/tst_ebusdb/diag/rdbms/tst/tst/trace/*.trc Log filerm /tstdb001/11.2/TST/admin/tst_ebusdb/diag/rdbms/tst/tst/trace/*.trm cleanuprm /tstdb001/11.2/TST/admin/tst_ebusdb/diag/rdbms/tst/tst/trace/alert_tst.logrm /tstdb001/11.2/TST/admin/tst_ebusdb/diag/rdbms/tst/tst/trace/*.outrm /tstdb001/11.2/TST/appsutil/tst_ebusdb.xml.bak*rm /admin/utl_file_dir/tst/*tmp
  18. 18. ##### KILLING LOCAL PROCESSES #######echo KILLING LOCAL PROCESSESdate. /home/oratst/.profilekill_local > /home/oratst/refresh/log/kill_local.shchmod 777 /home/oratst/refresh/log/kill_local.sh. /home/oratst/refresh/log/kill_local.shdate#### SHUTING DOWN LISTENER #########echo SHUTING DOWN LISTENERcd /tstdb001/11.2/TST/bin./lsnrctl stop tst#### SHUTING DOWN DATABASE ###############echo SHUTING DOWN DATABASEcd /tstdb001/11.2/TST/bin./sqlplus /as sysdba << 0xffselect SYSDATE,NAME from v$database;shutdown immediate;exit0xffdate################## DELETING DATAFILES ########echo DELETING DATAFILES 1rm /tstdb001/oradata/*.dbfecho DELETING DATAFILES 2rm /tstdb002/oradata/*.dbfecho DELETING DATAFILES 3rm /tstdb003/oradata/*.dbfecho DELETING DATAFILES 4rm /tstdb004/oradata/*.dbfecho DATAFILES DELETEDdate
  19. 19. ########## COPING DATAFILES ##################echo COPY DATAFILES 1cp /dg001/oradata/*.dbf /tstdb001/oradataecho COPY DATAFILES 2cp /dg002/oradata/*.dbf /tstdb002/oradataecho COPY DATAFILES 3cp /dg003/oradata/*.dbf /tstdb003/oradataecho COPY DATAFILES 4cp /dg004/oradata/*.dbf /tstdb004/oradataecho DATAFILES COPY DONEdatedate########## DOING CLONE ###################echo STARTING DBTIER CLONEcd /tstdb001/11.2/TST/appsutil/clone/binperl adcfgclone.pl dbTier /home/oratst/refresh/contextfile/tst_ebusdb.xml << EOFappsEOFdate########echo REFRESH OF PRODUCTION DONE
  20. 20. Step 3 Start Dataguard instance#### SHUTDOWN DATAGUARD FOR EBUS CLONING ###########00 1 * * * /home/oradg/refresh/shutdownDG.sh > /home/oradg/refresh/log/shutdownDG.log################ STARTUP DATAGUARD FOR EBUS CLONING ################################15 2 * * * /home/oradg/refresh/startDG.sh > /home/oradg/refresh/log/startupDG.logoradg@ebusdb:~> more /home/oradg/refresh/startDG.sh##### Running environment file #####################################. /home/oradg/env_DG.sh########### Shuting down the listener ##############################lsnrctl start dg######## Shutting down the DB ######################################sqlplus /as sysdba << 0xffstartup nomount;alter database mount standby database; alter database open readonly;recover managed standby database using current logfile disconnect;select DB_UNIQUE_NAME from v$database;exit0xffdate##### DATAGUARD ENVIRONMENT IS UP ##########################oradg@ebusdb:~>
  21. 21. Step 4 Appstier00 4 * * * /home/appltst/refresh/cloneapps.sh > /home/appltst/refresh/log/CLONE-APPS--$(date+%Y%m%d_%H:%M:%S).log 2>&1appltst@ebusapp:/tstapp001/tstapp/inst/apps/tst_ebusapp/admin/scripts> more /home/appltst/refresh/cloneapps.shecho CLEARING THE INVENTORYrm -rf /home/appltst/oraInventory/*echo STARTING APPS CLONEcd /tstapp001/tstapp/apps/apps_st/comn/clone/binperl adcfgclone.pl appsTier /home/appltst/refresh/contextfile/tst_ebusapp.xml << EOFappsnEOFdateecho APPS CLONE DONEecho CHANGING SYSADMIN PASSWORD. /tstapp001/tstapp/apps/apps_st/appl/APPStst_ebusapp.envcd /tstapp001/tstapp/apps/apps_st/appl/fnd/12.0.0/bin./FNDCPASS apps/apps 0 Y system/apps USER SYSADMIN appsecho SYSADMIN PASSWORD CHANGED TO apps./FNDCPASS apps/apps 0 Y system/apps USER MEDSUSER appsecho MEDSUSER PASSWORD CHANGED TO appsecho ALL CLONE OF PRODUCTION DONEecho Clearing the CACHErm /tstapp001/tstapp/inst/apps/tst_ebusapp/logs/ora/10.1.2/reports/cache/*dateecho Cache Doneecho Clearing APPLTMPrm -rf /tstapp001/tstapp/inst/apps/tst_ebusapp/appltmp/*echo Clearing APPLTMP donedateecho ALL CLONE OF PRODUCTION DONE
  22. 22. appltst@ebusapp:/tstapp001/tstapp/inst/apps/tst_ebusapp/admin/scripts> more /home/appltst/refresh/changeappspassword.sh. /tstapp001/tstapp/apps/apps_st/appl/APPStst_ebusapp.envcd /tstapp001/tstapp/apps/apps_st/appl/fnd/12.0.0/bin./FNDCPASS apps/apps0 Y system/apps ORACLE APPLSYSPUB PUB./FNDCPASS apps/apps 0 Y system/apps SYSTEM APPLSYS appsecho CHANGED APPS PASSWORDecho CHANGED FILEecho START APPS SERVER. /tstapp001/tstapp/apps/apps_st/appl/APPStst_ebusapp.envcd /tstapp001/tstapp/inst/apps/tst_ebusapp/admin/scriptscp /home/appltst/refresh/customtst_ebusapp.env /tstapp001/tstapp/apps/apps_st/appl./adautocfg.sh << EOFappsEOFecho AUTOCONFIG COMPLETED SUCCESSFULLYsleep 180./adstrtal.sh apps/appsecho ALL STARTEDdateecho deleting old log filescd /tstapp001/tstapp/inst/apps/tst_ebusapp/logs/appl/conc/logfind /tstapp001/tstapp/inst/apps/tst_ebusapp/logs/appl/conc/log -mtime +1 -exec rm -f {} ;dateecho deleting old out filescd /tstapp001/tstapp/inst/apps/tst_ebusapp/logs/appl/conc/out Log and out filefind /tstapp001/tstapp/inst/apps/tst_ebusapp/logs/appl/conc/out -mtime +1 -exec rm -f {} ;date cleanupecho cleaning out Apache log filesfind /tstapp001/tstapp/inst/apps/tst_ebusapp/logs/ora/10.1.3/Apache -mtime +1 -exec rm -f {} ;dateecho CLONE COMPLETEDappltst@ebusapp:/tstapp001/tstapp/inst/apps/tst_ebusapp/admin/scripts>
  23. 23. Additional steps• Close all workflowsUpdate apps.wf_notificationsSet mail_status=SENT,status=CLOSEDwhere mail_status=MAILand status=OPEN;commit;• Close all alertsupdate apps.alr_alertsset enabled_flag=Nwhere enabled_flag=Yand upper(alert_name) like MEDS%;Update apps. fnd_concurrent_requestsset phase_code = C,status_code = Dwhere concurrent_program_id in (select concurrent_program_idfrom apps.fnd_concurrent_programswhere concurrent_program_name in (ALECDC, ALECTC));
  24. 24. • Set override addressupdate fnd_user set email_address = rynod@medscheme.co.za;update wf_local_roles set email_address = rynod@medscheme.co.za;commit;
  25. 25. Scramble Salary informationupdate peR_pay_proposals set proposed_salary_n = ROUND((proposed_salary_n/assignment_id)*4732,2);Updating Site ParameterDECLAREstat boolean;BEGINdbms_output.disable;dbms_output.enable(100000);stat := FND_PROFILE.SAVE(SITENAME, ****DAILY CLONE OF PRODUCTION****TST******DAILY CLONE OF PRODUCTION***Data from||to_char(trunc(sysdate)-1,DD-MON-YYYY)||(12:00AM)*****TST**, SITE);IF stat THENdbms_output.put_line( Stat = TRUE - profile updated );ELSEdbms_output.put_line( Stat = FALSE - profile NOT updated );END IF;commit;END;/
  26. 26. We have noticed the following cost saving to business• No requirements of tapes• No requirements for assistance of Linux admin• No requirements for restores• No human interaction , only to check logs• No functional support or super users on production• Time saved from 24 working hours to 20 minutes• DR available immediately• DataGuard environment is read only which means it can be used for reporting• Production Servers are up and running 24 X 7 X 365• Clones to DEV, UAT, Training and dba environments takes 1 hour as clones on local servers• Users and support team can test anything at anytime• Copy of production available 7 days a week.• Backup of production can be verified by dataguard.• Rman backups can be run from the DataGuard instance.• No need to purchase expensive software for cloning.• Daily copy of production which functional support team can use• Assistance with OWC with Oracle as scripts and updates can be tested with production data immediately• We noticed a reduction in TARS as support team could test solutions from metalink• Patches can be tested immediately using Production data• No more P1 SRs• Super users can test changes to the system• Demos can be given to Directors using latest data• Training can be given with data from Production• Disaster recovery environment is available immediately• Changes can be made on TEST system and user acceptance testing can take place easy• Emergency changes can be tested• No additional money was spent on this solution
  27. 27. We have noticed the following cost saving to business• No requirements of tapes• No requirements for assistance of Linux admin• No requirements for restores• No human interaction , only to check logs• No functional support or super users on production• Time saved from 24 working hours to 20 minutes• DR available immediately• DataGuard environment is read only which means it can be used for reporting• Production Servers are up and running 24 X 7 X 365• Clones to DEV, UAT, Training and dba environments takes 1 hour as clones on local servers• Users and support team can test anything at anytime• Copy of production available 7 days a week.• Backup of production can be verified by dataguard.• Rman backups can be run from the DataGuard instance.• No need to purchase expensive software for cloning.• Daily copy of production which functional support team can use• Assistance with OWC with Oracle as scripts and updates can be tested with production data immediately• We noticed a reduction in TARS as support team could test solutions from metalink• Patches can be tested immediately using Production data• No more P1 SRs• Super users can test changes to the system• Demos can be given to Directors using latest data• Training can be given with data from Production• Disaster recovery environment is available immediately• Changes can be made on TEST system and user acceptance testing can take place easy• Emergency changes can be tested• No additional money was spent on this solution
  28. 28. Disaster Recovery• Shutdown DataGuard – Stop the listener {lsnrctl stop dg} – Stop DataGuard { recover managed standby database cancel; } – Shutdown the database {shutdown immediate; } – Copy the datafiles to any instance.• On the DB server run post-clone – cd $ORACLE_HOME/appsutil/clone/bin { taking into account you know the locations} – perl adcfgclone.pl dbTier { configure your dbTier}• On your Apps Server { we normally use our TST environment or have a dedicated disk } – cd $COMMON_TOP/clone/bin { taking into account you know the locations} – perl adcfgclone.pl appsTier• Users can login onto your DR/ DataGuard environment• Update your log and out files locations update apps.fnd_concurrent_requests set logfile_node_name = <New APPS SERVER>, outfile_node_name = <New APPS SERVER> where logfile_node_name = <OLD APPS SERVER> and outfile_node_name = <OLD APPS SERVER>;
  29. 29. Database Cleanup Files which can be deleted –
  30. 30. APPS Server Cleanup Files which can be deleted –
  31. 31. Summary of Environment
  32. 32. What does functional support team say ! Due to audit requirements that none of support staff are allowed to have Production access, this solution has been ‘forced’ upon us with actually very positive results: 1) We’re still able to support users timeously 2) We’re able to replicate Production issues with up-to-date data and configuration without any risk to ProductionShounese KhanHR Functional Specialist Oracle •Assistance can be given the next day if people are having trouble with a specific transaction on self-service / professional forms as the data would have been replicated and no need to wait for a scheduled refresh. •You can log on as the user to confirm the error they have and assist then to resolve it (this could be to get print screens to show them where to click ect…)Wayne WrightOracle System Administrator We are also using this environment for post-implementation testing – in cases where testing requires that transactions need to be created, this environment is ideal as no Production data is compromisedShalantha RampersadOracle HR Developer
  33. 33. The benefit that we have is that we can now experiment (play around) with the latest records when problems are experienced by users.Ryno DurrheimFinance Functional Support TST gives me the ability to work with “real” data and cases, and find proper solutions that almost guarantees flawless implementations to PROD.Juan FerreiraOracle Finance Developer The Daily Clone has allowed us to meet our audit requirements has we now have the ability to remove all supports staff’s access to production and they can use the daily clone for most of the support issues.Alwin WeirSenior Manager: Corporate IT Systems and Data Warehouse
  34. 34. The old process took extremely long and had a lot of manual steps. After the new process was implemented we don’t even know when the APPS DBAs are cloning. All is done in the background, we no more work overtime on development environments Saving us time to concentrate on production hardware support made our lifes easierFortunate & FrikkieMedscheme LinuxAdministrators My team { APPS DBAs & linux admins } can now concentrate on Production support and supporting Medscheme users instead of spending time cloning development environments. This new process has freed our resources for business critical tasks Johann Laurens Senior Manager: Infrastructure
  35. 35. We use TST on a daily basis for any issues we come across on PROD that we first want to test out and it is great that TST has the updated information from PROD. We are then easily able to resolve payroll run, elements, formula, leave queries or personal record issues. We have also found benefit using TST where we can run tax year end processes and payroll runs to test out new patch functionalities as well as new element builds and formula results using the data. We find the daily clone of PROD to beMohomed Suliman very valuable for the Payroll department.Senior Manager : Payroll We use TST to replicate the problem which we experience in production and try to fix it on TST first before we do it on production. I use it for Performance Management, I-Recruitment and Learning Management ,Employee Self Server and Manager Self Service Tshivhiya Mulaudzi Systems and Support Specialist HRMS | Human Capital Shared Services
  36. 36. TST is very helpful and has enabled us to assist our end users with their system queries – logging into a system and being able to see what they see. It has also been very useful in testing setup changes and confirming minimal impact on the system. We also make use of it for training purposes, where users can log in and go and test the different performance processes, before working on the live environment.Tabita VenterPerformance Consultant | HC - Centre ofExcellence I used TST to assist with my up skilling on Purchase order training when it was handed over from Yolande Mathews, I have since used TST for training sessions to train PA’s on the purchase order process. We also use it to train new staff on the employee self service component- on TST they are given an opportunity to explore the navigation and practice how to use the applicable update functionality on Oracle employee self service.
  37. 37. By applying Oracle’s active Data Guard solution we’ve solved key issues we had in the past both from a recovery, usability and audit perspective. The automated solution deployed by the IT Technology DBA team has decreased the refresh turnaround time to such and extend that it has really become a non-issue for us. We’ve solved real business issues/requirements with no additional costs apart from the Data Guard license costs.Marius DreyerGeneral Manager : Technology The customized automated clone has saved us many repetitive man hours. With system and human errors likely to occur with the previous manual cloning, this new process guarantees our environment is available and accessible on time every day
  38. 38. Questions & AnswersFeedback
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×