Ab initio best practices and useful tips


Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Ab initio best practices and useful tips

  1. 1. Ab Initio Best Practices and Useful TipsThese are general guidelines ideal for implementing in the development, maintenanceand testing of Ab Initio projects. These have been gathered from various sources on theInternet, as well as from experienced Ab Initio developers.Project Access Control – CheckIn, CheckOut1) Before Checking In a graph, ensure that it has been deployed successfully2) Inform the ETL Admin before Checking In3) To get the latest version of the graph, Check Out from EME Data Store4) Always Check Out from EME Data store to your individual sand box before running a graph5) In case the graph is not present in the EME Data Store, Check In and run it6) Ab Initio Sand Box should be created only by the ETL Admin for all authorized users7) Ensure that User-ID and Password in the EME Settings and the Run Settings are the same, before creating graphs on the server8) Ensure that a graph is locked before modifying a graph, to prevent any sharing conflicts9) Try to see that individual abinitio graphs are handled by separate users – when a user locks a graph, it prevents other users from modifying it while it is locked10) Ask the DBA to create any tables needed in the target database – do not create yourself11) Report all database related activities and issues to the DBA promptly12) Inform the DBA and get his approval before you modify any table in the target database13) Only the ETL Admin has the rights to set or modify environment variables – do not change environment variables – these are global to all graphs and should not be tampered withThis article has been adapted from a part of the Tutorial here.Project Implementation – Recommended Best Practices2201 Walnut Ave, Suite 240, Toll Free: 1.877.USA.DCKAP (1-877-872-3252)Fremont , CA 94538 Fax : 1-415-367-2525Email : infoca@dckap.com
  2. 2. 1) One may encounter errors while running a graph. Maintain error logs for every error you come across.2) Maintain a consolidated, detailed error sheet with error related information and resolution information for all users. This can be used for reference when facing similar errors later on.3) In case you have a database error, contact the DBA promptly4) In all your graphs, ensure you are using the relevant dbc file5) Always validate a graph before executing it – deploy the graph only after successful validation6) Run the ab_project_setup.ksh on a regular basis – Contact ETL Admin if you need further details7) Check whether the test parameters are valid before running an Abinitio graph8) Save and unlock the graph after implementing the modifications desiredHandling Run Time Errors1) When testing a graph, contact the person who created it or modified it recently for assistance2) Contact the ETL Admin if the error encountered relates to Admin settings3) When you face a problem that you have not encountered and resolved before, look in to the consolidated error sheet to see if it has previously been faced and resolved by any other user4) Check with online tech forums for further inputs on the errorDocumentation Practices1) Maintain documents regarding all the modifications performed on existing graphs and scripts2) Maintain ETL design documents for all ab initio graphs created or modified – when changes are made to the existing graphs, the documents should be suitably modified3) Follow testing rules as per the testing template when testing a graph4) Document all testing activities performed2201 Walnut Ave, Suite 240, Toll Free: 1.877.USA.DCKAP (1-877-872-3252)Fremont , CA 94538 Fax : 1-415-367-2525Email : infoca@dckap.com
  3. 3. Underlying Tables1) In all graphs with RDBMS tables as input, ensure that the join condition is on indexed columns or indexes are created on the columns in the join condition – This is very important because if indexes are absent there will be a full table scan affecting performance. Use the Oracle Explain Plan utility before executing a graph to find the execution path of the query.2) If there are indexes on target table, ensure that they are dropped before running the graph and recreated after the graph is run3) Try to shift the sorting or aggregating of data to the source tables (assuming the data source is an RDBMS, not a flat file). SQL OrderBy or GroupBy clause is faster than AbInitio because the database server would invariably be more powerful than Ab Initio server (in any case, SQL OrderBy or GroupBy is more efficient than an ETL tool) because Oracle runs the statement in optimal mode.4) Bitmap indexes may not be created on tables that are updated frequently. Bitmap indexes tend to occupy a large amount of disk space. Create a normal index such as a B-tree instead.2201 Walnut Ave, Suite 240, Toll Free: 1.877.USA.DCKAP (1-877-872-3252)Fremont , CA 94538 Fax : 1-415-367-2525Email : infoca@dckap.com