Your SlideShare is downloading. ×
Informatica Power Center 7.1
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Informatica Power Center 7.1

12,495

Published on

Overview & Components …

Overview & Components
Informatica Server & Data Movement
Repository Server & Repository Manager
Designer
Transformations used in Informatica
Re-usable Transformations & Mapplets
Workflow Manager & Workflow Monitor
Performance Tuning & Troubleshooting
Informatica PowerCenter 7.1 Enhancements

Published in: Technology
3 Comments
25 Likes
Statistics
Notes
  • please share the ppt.. anil.khillar@gmail.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • plzzzzzzzzzzz
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • plz mail me informatica power center 7.1 ppt..
    email id: guntururamya@gmail.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
12,495
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
0
Comments
3
Likes
25
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Global repository. The global repository is the hub of the domain. Use the global repository to store common objects that multiple developers can use through shortcuts. These objects may include operational or application source definitions, reusable transformations, mapplets, and mappings. Local repositories. A local repository is within a domain that is not the global repository. Use local repositories for development. From a local repository, you can create shortcuts to objects in shared folders in the global repository. These objects typically include source definitions, common dimensions and lookups, and enterprise standard transformations. You can also create copies of objects in non-shared folders.
  • Global repository. The global repository is the hub of the domain. Use the global repository to store common objects that multiple developers can use through shortcuts. These objects may include operational or application source definitions, reusable transformations, mapplets, and mappings. Local repositories. A local repository is within a domain that is not the global repository. Use local repositories for development. From a local repository, you can create shortcuts to objects in shared folders in the global repository. These objects typically include source definitions, common dimensions and lookups, and enterprise standard transformations. You can also create copies of objects in non-shared folders.
  • The Figure shows the processing path between the Informatica Server, repository, source, and target
  • The Informatica Server can combine data from different platforms and source types. For example, you can join data from a flat file and an Oracle source, and write the transformed data to a Microsoft SQL Server database. When a session starts, the Informatica Server retrieves mapping and session metadata from the repository to extract data from the source, transform it, and load it into the target. The Informatica Server can combine data from different platforms and source types
  • The Informatica Server can combine data from different platforms and source types. For example, you can join data from a flat file and an Oracle source, and write the transformed data to a Microsoft SQL Server database. When a session starts, the Informatica Server retrieves mapping and session metadata from the repository to extract data from the source, transform it, and load it into the target. The Informatica Server can combine data from different platforms and source types
  • The Load Manager is the primary Informatica Server process
  • The Load Manager holds the connection to the repository for the value set in the Informatica Server configuration, LMStayConnectToRepositDuration When you start the Informatica Server. When you start the Informatica Server, the Load Manager launches and queries the repository for a list of sessions configured to run on the Informatica Server. When you configure a session. When you add, update, or schedule a session in the Server Manager, the repository stores all the session metadata. The Load Manager maintains a list of sessions and session start times. When a session starts. When a session starts, the Load Manager fetches the session information from the repository to perform the validations and verifications prior to starting the DTM process. The execute lock allows the Informatica Server to run the session and prevents you from starting the session again until it completes. If the session is already locked, the Informatica Server cannot start the session. A session may be locked if it is already running, or if an error occurred during the previous run that prevented the repository from releasing the execute lock.
  • Four transformation threads. The DTM creates one transformation thread for each partition. The DTM creates an additional transformation thread for partition for each Aggregator or Rank transformation. So, the DTM creates four transformation threads to process the mapping in Figure above.
  • Standalone repository. A repository that functions individually, unrelated and unconnected to other repositories. Global repository. (PowerCenter only.) The centralized repository in a domain, a group of connected repositories. Each domain can contain one global repository. The global repository can contain common objects to be shared throughout the domain through global shortcuts. Once created, you cannot change a global repository to a local repository. You can promote an existing local repository to a global repository. Local repository. (PowerCenter only.) A repository within a domain that is not the global repository. Each local repository in the domain can connect to the global repository and use objects in its shared folders. A folder in a local repository can be copied to other local repositories while keeping all local and global shortcuts intact.
  • When restoring a repository, you must have a database available for the repository. You can restore the repository in a database that has a different code page from the original database, if the code pages are compatible. If a repository already exists at the location, the Repository Manager asks you to delete the repository before restoring a backup repository. If no repository exists, the Repository Manager creates a repository before restoring the backup repository.
  • Source and target dependencies report (S2t_dep.rpt) - Shows the source and target dependencies as well as the transformations performed in each mapping
  • Write lock - Created when you create or edit a repository object in a folder for which you have write permission Execute lock - Created when you start a session or batch, or when the Informatica Server starts a scheduled session or batch Save lock - Created when you save information to the repository The repository permits multiple read locks, one write lock, and one execute lock simultaneously on each repository object. This means that one user can edit a session while the Informatica Server runs the session, and another user can view the session properties at the same time. You can view existing locks in the repository in the Repository Manager. The Repository Manager provides two ways to view locks: Browse the repository. Use the Navigator and main windows to display the folders, versions, and objects in use. Show locks. Use a menu command to view all locks in the repository. This method provides more detailed information and allows you to sort your view of the locks.
  • You must create a folder in a new repository before you can connect to the repository using the Designer or Workflow Manager. You can copy objects from one folder to another, so if you want to use an object in a non-shared folder, you can copy it into your working folder. If you work with multiple repositories, you can also copy objects across repositories. You can continue working in the new version, while preserving the older version. You might use versions to archive work while continuing with development.
  • In your repository, you might create folders for each data warehouse development project, subject area, user, or type of metadata. If you can divide the data warehouse into different types of information, you might create a single folder for each type of data. For instance, when you set up the accounting data warehouse, you might create one folder for accounts payable and another for accounts receivable. You can create a folder for each repository user, designed to store work for that user only. If users work on separate projects, this technique avoids any problems that might occur if two people attempt to edit the same piece of metadata at the same time. You might create a different folder for each type of metadata (source definitions, target definitions, mappings, schemas, and reusable transformations) that you create through the Designer.
  • When you copy a folder from a global repository to a local repository in the same domain, the Repository Manager verifies whether a folder of the same name exists in the global repository. If it does not, the Repository Manager uses the folder name for the copied folder. If it does, the Repository Manager asks you to rename the folder. The Repository Manager preserves shortcuts to shared folders in the global repository, changing the local shortcuts to global shortcuts. When you copy both a shared folder and a non-shared folder with dependent shortcuts across repositories and then recopy the shared folder from the source repository, the shortcuts in the non-shared folder in the target repository point to the folder in the source repository. The shortcuts in the non-shared folder always point to the folder you select when you copy/replace a shared folder.
  • Re-establish shortcuts. Maintain shortcuts to objects in shared folders. If the Repository Manager cannot re-establish shortcuts, it marks the affected mappings, mapplets, and sessions invalid in the repository and lists them in the Output window. Choose an Informatica Server. Use the Informatica Server to run all sessions and batches in the folder if a matching Server does not exist in the target repository. Copy connections. Copy database, FTP, and external loader connection information if matching connection names do not exist in the target repository. Copy persisted values. Copy the saved persisted values for mapping variables used in a session. Compare folders. Compare folders to determine how they are related with the compare folders functionality. Replace folders. Replace an existing folder, including all objects associated with the folder. The Repository Manager copies and replaces folders as a single transaction. If you cancel the copy before it completes, the Repository Manager rolls back all changes.
  • Versions to compare. The wizard automatically selects pairs of versions with the same version number in each folder for comparison. You can also specify the versions to compare in each folder. Object types to compare. You can specify the object types to compare and display between folders. The Repository Manager compares objects based upon specific object attributes. See Table 6-3 for a list of compared attributes for object types. Direction of comparison. The Repository Manager performs directional comparisons. A directional comparison checks the contents of one folder against the contents of the other. You can specify either one-way or two-way comparisons.
  • Figure shows two folders in the same repository, Orders1 and Orders2. If you compare the folders using a one-way comparison, the source definition ORDER_ITEMS, present in Orders2 but not in Orders1, is not noted as a comparison. If you compare the folders using a two-way comparison, the absence of ORDER_ITEMS in Orders1 is noted as a difference.
  • Because sessions and batches are not associated with version numbers, the version pairs specified in the Versions to compare list do not impact a comparison of sessions or batches. If you want to compare only sessions and batches, you can accept the default version pairs without affecting the outcome of the comparison. The Repository Manager does not compare the field attributes of the objects in the folders when performing the comparison. For example, if two folders that have matching source names and column or port names but differing port or column attributes, such as precision or datatype, the Repository Manager does not note these as different.
  • Can delete a folder version to remove unnecessary versions from the repository By archiving the contents of a folder into a version each time you reach a development landmark, you can access those versions if later edits prove unsuccessful. For example, you can create a folder version after completing a version of a difficult mapping, then continue working on the mapping. If you are unhappy with the results of subsequent work, you can revert to the previous version, then create a new version to continue development. Thus you keep the landmark version intact, but available for regression. When working with multiple versions, make sure you have the appropriate version active. The repository saves version information by workspace, so if someone else uses your machine and changes the active version, that version remains active on your machine until changed
  • Exporting and importing an object is similar to copying an object from one folder or repository to another folder or repository. When you copy objects between folders or repositories, you must be connected to both repositories simultaneously. However, when you export an object from one repository and import the object into another repository, you do not need to connect to both repositories simultaneously. You might want to export an object in any of the following circumstances: You want to copy an object between two repositories, but you cannot connect to both repositories from the same client. Export the object and electronically transfer the XML file to the target machine. Then import the object from the XML file into the target repository. You previously copied a mapping or mapplet that uses a reusable transformation to another repository. Then later you changed the reusable transformation. Instead of copying the entire mapping or mapplet again, you can export and import the reusable transformation. You want to export an object from your development repository and deploy it in the production repository. You have an invalid session that you need to troubleshoot. Export the invalid session and its associated mapping, electronically transfer the XML file to someone else for troubleshooting.
  • To import a source definition: In the Source Analyzer, choose Sources-Import from Database. Select the ODBC data source used to connect to the source database. If you need to create or modify an ODBC data source, click the Browse button to open the ODBC Administrator. Create the appropriate data source and click OK. Select the new ODBC data source. Enter a database username and password to connect to the database. Note: The username must have the appropriate database permissions to view the object. You may need to specify the owner name for database objects you want to use as sources. Click Connect. If no table names appear or if the table you want to import does not appear, click All. Drill down through the list of sources to find the source you want to import. Select the relational object or objects you want to import. You can hold down the Shift key to select blocks of record sources within one folder, or hold down the Ctrl key to make non-consecutive selections within a folder. You can also select all tables within a folder by selecting the folder and clicking the Select All button. Use the Select None button to clear all highlighted selections.
  • When you create a flat file source definition, you must define the properties of the file. The Source Analyzer provides a Flat File Wizard to prompt you for the above mentioned file properties. You can import fixed-width and delimited flat file source definitions that do not contain binary data. When importing the definition, the source file must be in a directory local to the client machine. In addition, the Informatica Server must be able to access all source files during the session.
  • You can create the overall relationship, called a schema , as well as the target definitions, through wizards in the Designer. The Cubes and Dimensions Wizards follow common principles of data warehouse design to simplify the process of designing related targets.
  • Connectors - Connect sources, targets, and transformations so the Informatica Server can move the data as it transforms it A mapplet is a set of transformations that you build in the Mapplet Designer and can use in multiple mappings
  • When you edit and save a mapping, some changes cause the session to be invalid even though the mapping remains valid. The Informatica Server does not run invalid sessions
  • The Designer marks a mapping invalid when it detects errors that will prevent the Informatica Server from executing the mapping The Designer performs connection validation each time you connect ports in a mapping and each time you validate or save a mapping. At least one mapplet input port and output port is connected to the mapping. If the mapplet includes a Source Qualifier that uses a SQL override, the Designer prompts you to connect all mapplet output ports to the mapping. You can validate an expression in a transformation while you are developing a mapping. If you did not correct the errors, the Designer writes the error messages in the Output window when you save or validate the mapping. When you validate or save a mapping, the Designer verifies that the definitions of the independent objects, such as sources or mapplets, match the instance in the mapping. If any of the objects change while you configure the mapping, the mapping might contain errors.
  • Example of an active transformation is a Filter transformation that removes rows that do not meet the configured filter condition. Example of a passive transformation is an an Expression transformation that performs a calculation on data and passes all rows through the transformation An unconnected transformation is not connected to other transformations in the mapping. It is called within another transformation, and returns a value to that transformation.
  • The Informatica Server performs aggregate calculations as it reads, and stores necessary data group and row data in an aggregate cache Aggregate expression - Entered in an output port, can include non-aggregate expressions and conditional clauses Group by port - Indicates how to create groups. can be any input, input/output, output, or variable port Sorted Input option - Use to improve session performance. To use Sorted Input, you must pass data to the Aggregator transformation sorted by group by port, in ascending or descending order Aggregate cache - Aggregator stores data in the aggregate cache until it completes aggregate calculations. It stores group values in an index cache and row data in data cache
  • You can enter multiple expressions in a single Expression transformation. As long as you enter only one expression for each output port, you can create any number of output ports in the transformation. In this way, you can use one Expression transformation rather than creating separate transformations for each calculation that requires the same set of data.
  • As an active transformation, the Filter transformation may change the number of rows passed through it. A filter condition returns TRUE or FALSE for each row that passes through the transformation, depending on whether a row meets the specified condition. Only rows that return TRUE pass through this transformation. Discarded rows do not appear in the session log or reject files. To maximize session performance, include the Filter transformation as close to the sources in the mapping as possible. Rather than passing rows you plan to discard through the mapping, you then filter out unwanted data early in the flow of data from sources to targets. You cannot concatenate ports from more than one transformation into the Filter transformation. The input ports for the filter must come from a single transformation. The Filter transformation does not allow setting output default values.
  • Allows to join sources that contain binary data To join more than two sources in a mapping, add additional Joiner transformations An input transformation is any transformation connected to the input ports of the current transformation. Specify one of the sources as the master source, and the other as the detail source. This is specified on the Properties tab in the transformation by clicking the M column. When you add the ports of a transformation to a Joiner transformation, the ports from the first source are automatically set as detail sources. Adding the ports from the second transformation automatically sets them as master sources. The master/detail relation determines how the join treats data from those sources based on the type of join. For example, you might want to join a flat file with in-house customer IDs and a relational database table that contains user-defined customer IDs. You could import the flat file into a temporary database table, then perform the join in the database. However, if you use the Joiner transformation, there is no need to import or create temporary tables.
  • Can configure the lookup transformation to be connected or unconnected, cached or uncached
  • Connected and unconnected transformations receive input and send output in different ways. Sometimes you can improve session performance by caching the lookup table. If you cache the lookup table, you can choose to use a dynamic or static cache. By default, the lookup cache remains static and does not change during the session. With a dynamic cache, the Informatica Server inserts rows into the cache during the session. Informatica recommends that you cache the target table as the lookup. This enables you to look up values in the target and insert them if they do not exist.
  • NewLookupRow. The Designer automatically adds this port to a Lookup transformation configured to use a dynamic cache. Indicates whether or not the row is in lookup cache. To keep the lookup cache and the target table synchronized, you want to pass rows to the target when the NewLookupRow value is equal to 1. Associated Port. Associate lookup ports with either an input/output port or a sequence ID. The Informatica Server uses the data specified in the associated ports to insert into the lookup cache when it does not find a row in the lookup cache. If you associate a sequence ID, the Informatica Server generates a primary key for the inserted row in the lookup cache.
  • The Informatica Server builds the cache when it processes the first request lookup request. It queries the cache based on the lookup condition for each row that passes into the transformation. When the Informatica Server receives a new row (a row that is not in the cache), it inserts the row into the cache. You can configure the transformation to insert rows into the cache based on input ports or generated sequence IDs. The Informatica Server flags the row as new. When the Informatica Server receives an existing row (a row that is in the cache), it flags the row as existing. The Informatica Server does not insert the row into the cache. Use a Router or Filter transformation with the dynamic Lookup transformation to route new rows to the cached target table. You can route existing rows to another target table, or you can drop them. When you partition a source that uses a dynamic lookup cache, the Informatica Server creates one memory cache and one disk cache for each transformation.
  • The Router transformation is more efficient when you design a mapping and when you run a session For example, to test data based on three conditions, you only need one Router transformation instead of three filter transformations to perform this task. Likewise, when you use a Router transformation in a mapping, the Informatica Server processes the incoming data only once. When you use multiple Filter transformations in a mapping, the Informatica Server processes the incoming data for each transformation
  • The Informatica Server generates a value each time a row enters a connected transformation, even if that value is not used. When NEXTVAL is connected to the input port of another transformation, the Informatica Server generates a sequence of numbers. When CURRVAL is connected to the input port of another transformation, the Informatica Server generates the NEXTVAL value plus one.
  • Connect NEXTVAL to multiple transformations to generate unique values for each row in each transformation. For example, you might connect NEXTVAL to two target tables in a mapping to generate unique primary key values. The Informatica Server creates a column of unique primary key values for each target table. If you want the same generated value to go to more than one target that receives data from a single preceding transformation, you can connect a Sequence Generator to that preceding transformation. This allows the Informatica Server to pass unique values to the transformation, then route rows from the transformation to targets.
  • The Source Qualifier displays the transformation datatypes. The transformation datatypes in the Source Qualifier determine how the source database binds data when you import it. Do not alter the datatypes in the Source Qualifier. If the datatypes in the source definition and Source Qualifier do not match, the Designer marks the mapping invalid when you save it.
  • In the mapping shown above, although there are many columns in the source definition, only three columns are connected to another transformation. In this case, the Informatica Server generates a default query that selects only those three columns: SELECT CUSTOMERS.CUSTOMER_ID, CUSTOMERS.COMPANY, CUSTOMERS.FIRST_NAME FROM CUSTOMERS When generating the default query, the Designer delimits table and field names containing the slash character (/) with double quotes.
  • In the mapping shown above, although there are many columns in the source definition, only three columns are connected to another transformation. In this case, the Informatica Server generates a default query that selects only those three columns: SELECT CUSTOMERS.CUSTOMER_ID, CUSTOMERS.COMPANY, CUSTOMERS.FIRST_NAME FROM CUSTOMERS When generating the default query, the Designer delimits table and field names containing the slash character (/) with double quotes.
  • In the mapping shown above, although there are many columns in the source definition, only three columns are connected to another transformation. In this case, the Informatica Server generates a default query that selects only those three columns: SELECT CUSTOMERS.CUSTOMER_ID, CUSTOMERS.COMPANY, CUSTOMERS.FIRST_NAME FROM CUSTOMERS When generating the default query, the Designer delimits table and field names containing the slash character (/) with double quotes.
  • In the mapping shown above, although there are many columns in the source definition, only three columns are connected to another transformation. In this case, the Informatica Server generates a default query that selects only those three columns: SELECT CUSTOMERS.CUSTOMER_ID, CUSTOMERS.COMPANY, CUSTOMERS.FIRST_NAME FROM CUSTOMERS When generating the default query, the Designer delimits table and field names containing the slash character (/) with double quotes.
  • It determines how to handle changes to existing records When you design your data warehouse, you need to decide what type of information to store in targets. As part of your target table design, you need to determine whether to maintain all the historic data or just the most recent changes. For example, you might have a target table, T_CUSTOMERS, that contains customer data. When a customer address changes, you may want to save the original address in the table, instead of updating that portion of the customer record. In this case, you would create a new record containing the updated address, and preserve the original record with the old customer address. This illustrates how you might store historical information in a target table. However, if you want the T_CUSTOMERS table to be a snapshot of current customer data, you would update the existing customer record and lose the original address.
  • It determines how to handle changes to existing records When you design your data warehouse, you need to decide what type of information to store in targets. As part of your target table design, you need to determine whether to maintain all the historic data or just the most recent changes. For example, you might have a target table, T_CUSTOMERS, that contains customer data. When a customer address changes, you may want to save the original address in the table, instead of updating that portion of the customer record. In this case, you would create a new record containing the updated address, and preserve the original record with the old customer address. This illustrates how you might store historical information in a target table. However, if you want the T_CUSTOMERS table to be a snapshot of current customer data, you would update the existing customer record and lose the original address.
  • The Rank transformation differs from the transformation functions MAX and MIN, in that it allows you to select a group of top or bottom values, not just one value. For example, you can use Rank to select the top 10 salespersons in a given territory. Or, to generate a financial report, you might also use a Rank transformation to identify the three departments with the lowest expenses in salaries and overhead. While the SQL language provides many functions designed to handle groups of data, identifying top or bottom strata within a set of rows is not possible using standard SQL functions. Allows to create local variables and write non-aggregate expressions
  • During a session, the Informatica Server compares an input row with rows in the data cache. If the input row out-ranks a stored row, the Informatica Server replaces the stored row with the input row. If the Rank transformation is configured to rank across multiple groups, the Informatica Server ranks incrementally for each group it finds.
  • During a session, the Informatica Server compares an input row with rows in the data cache. If the input row out-ranks a stored row, the Informatica Server replaces the stored row with the input row. If the Rank transformation is configured to rank across multiple groups, the Informatica Server ranks incrementally for each group it finds.
  • Limitations exist on passing data, depending on the database implementation Stored procedures are stored and run within the database. Not all databases support stored procedures, and database implementations vary widely on their syntax. You might use stored procedures to: Drop and recreate indexes. Check the status of a target database before moving records into it. Determine if enough space exists in a database. Perform a specialized calculation. Database developers and programmers use stored procedures for various tasks within databases, since stored procedures allow greater flexibility than SQL statements. Stored procedures also provide error handling and logging necessary for mission critical tasks. Developers create stored procedures in the database using the client tools provided with the database.
  • You can run several Stored Procedure transformations in different modes in the same mapping. For example, a pre-load source stored procedure can check table integrity, a normal stored procedure can populate the table, and a post-load stored procedure can rebuild indexes in the database. However, you cannot run the same instance of a Stored Procedure transformation in both connected and unconnected mode in a mapping. You must create different instances of the transformation. If the mapping calls more than one source or target pre- or post-load stored procedure in a mapping, the Informatica Server executes the stored procedures in the execution order that you specify in the mapping.
  • The stored procedure issues a status code that notifies whether or not the stored procedure completed successfully
  • You can pass a value from a port, literal string or number, variable, Lookup transformation, Stored Procedure transformation, External Procedure transformation, or the results of another expression. Separate each argument in a function with a comma. Except for literals, the transformation language is not case-sensitive. Except for literals, the Designer and Informatica Server ignore spaces. The colon (:), comma (,), and period (.) have special meaning and should be used only to specify syntax. The Informatica Server treats a dash (-) as a minus operator. If you pass a literal value to a function, enclose literal strings within single quotation marks. Do not use quotation marks for literal numbers. The Informatica Server treats any string value enclosed in single quotation marks as a character string. When you pass a mapping parameter or variable to a function within an expression, do not use quotation marks to designate mapping parameters or variables. Do not use quotation marks to designate ports. You can nest multiple functions within an expression (except aggregate functions, which allow only one nested aggregate function). The Informatica Server evaluates the expression starting with the innermost function.
  • After you save a mapplet, you can use it in a mapping to represent the transformations within the mapplet. When you use a mapplet in a mapping, you use an instance of the mapplet. Like a reusable transformation, any changes made to the mapplet are automatically inherited by all instances of the mapplet. Can use it in a mapping to represent the transformations within the mapplet
  • After you save a mapplet, you can use it in a mapping to represent the transformations within the mapplet. When you use a mapplet in a mapping, you use an instance of the mapplet. Like a reusable transformation, any changes made to the mapplet are automatically inherited by all instances of the mapplet. Can use it in a mapping to represent the transformations within the mapplet
  • Apply the following rules while designing mapplets: Use only reusable Sequence Generators Do not use pre- or post-session stored procedures in a mapplet Use exactly one of the following in a mapplet: Source Qualifier transformation ERP Source Qualifier transformation Input transformation Use at least one Output transformation in a mapplet
  • This does not replace the Server Manager, since there are many tasks that you can perform only with the Server Manager
  • Repository username. This can be configured optionally as an environment variable. Repository password. This can be configured optionally as an environment variable. Connection type. The type of connection from the client machine to the Informatica Server (TCP/IP or IPX/SPX). Port or connection. The TCP/IP port number or IPX/SPX connection (Windows NT/2000 only) to the Informatica Server. Host name. The machine hosting the Informatica Server (if running pmcmd from a remote machine through a TCP/IP connection). Session or batch name. The names of any sessions or batches you want to start or stop. Folder name. The folder names for those sessions or batches (if their names are not unique in the repository). Parameter file . The directory and name of the parameter file you want the Informatica Server to use with the session or batch.
  • Target-based commit. The Informatica Server commits data based on the number of target rows and the key constraints on the target table. The commit point also depends on the buffer block size and the commit interval. Source-based commit. The Informatica Server commits data based on the number of source rows. The commit point is the commit interval you configure in the session properties.
  • For example, a session is configured with target-based commit interval of 10,000. The writer buffers fill every 7,500 rows. When the Informatica Server reaches the commit interval of 10,000, it continues processing data until the writer buffer is filled. The second buffer fills at 15,000 rows, and the Informatica Server issues a commit to the target. If the session completes successfully, the Informatica Server issues commits after 15,000, 22,500, 30,000, and 40,000 rows.
  • Although the Filter, Router, and Update Strategy transformations are active transformations, the Informatica Server does not use them as active sources in a source-based commit session.
  • The Informatica Server might commit less rows to the target than the number of rows produced by the active source. For example, you have a source-based commit session that passes 10,000 rows through an active source, and 3,000 rows are dropped due to transformation logic. The Informatica Server issues a commit to the target when the 7,000 remaining rows reach the target. The number of rows held in the writer buffers does not affect the commit point for a source-based commit session. For example, you have a source-based commit session that passes 10,000 rows through an active source. When those 10,000 rows reach the targets, the Informatica Server issues a commit. If the session completes successfully, the Informatica Server issues commits after 10,000, 20,000, 30,000, and 40,000 source rows.
  • If a session fails or if you receive unexpected results in your target, you can run the Debugger against the session You might also want to run the Debugger against a session if you want the Informatica Server to process the configured session properties
  • Can create data or error breakpoints for transformations or for global conditions Cannot create breakpoints for mapplet Input and Output transformations Create breakpoints. You create breakpoints in a mapping where you want the Informatica Server to evaluate data and error conditions. Configure the Debugger. Use the Debugger Wizard to configure the Debugger for the mapping. You can choose to run the Debugger against an existing session or you can create a debug session. When you run the Debugger against an existing session, the Informatica Server runs the session in debug mode. When you create a debug session, you configure a subset of session properties within the Debugger Wizard, such as source and target location. You can also choose to load or discard target data. Run the Debugger. Run the Debugger from within the Mapping Designer. When you run the Debugger the Designer connects to the Informatica Server. The Informatica Server initializes the Debugger and runs session. The Informatica Server reads the breakpoints and pauses the Debugger when the breakpoints evaluate to true. Monitor the Debugger. While you run the Debugger, you can monitor the target data, transformation and mapplet output data, the debug log, and the session log. When you run the Debugger, the Designer displays the following windows: Debug log. View messages from the Debugger. Session log. View session log. Target window. View target data. Instance window. View transformation data. Modify data and breakpoints. When the Debugger pauses, you can modify data and see the effect on transformations, mapplets, and targets as the data moves through the pipeline. You can also modify breakpoint information.
  • Transcript

    • 1. INFORMATICA POWER CENTER 7.1
    • 2. AGENDA
        • Overview & Components
        • Informatica Server & Data Movement
        • Repository Server & Repository Manager
        • Designer
        • Transformations used in Informatica
        • Re-usable Transformations & Mapplets
        • Workflow Manager & Workflow Monitor
        • Performance Tuning & Troubleshooting
        • Informatica PowerCenter 7.1 Enhancements
    • 3. OVERVIEW & COMPONENTS
    • 4. PRODUCT OVERVIEW
      • Repository
      • Repository Server
      • Informatica Server
      • Informatica Client
        • Repository Manager
        • Designer
        • Workflow Manager
        • Workflow Monitor
        • Repository Administration Console
      Informatica components
    • 5. OVERVIEW .. INFORMATICA REPOSITORY
      • Stores the metadata created using the Informatica Client tools
      • Repository Manager creates the metadata tables in the database
      • Tasks in the Informatica Client application such as creating users, analyzing sources, developing mappings or Mapplets, or creating sessions creates metadata
      • Informatica Server reads metadata created in the Client application when a session runs
      • Global and local repositories can be created to share metadata
      • Added Features in PowerCenter 7.1
      • Exchange of Metadata with other BI Tools ,Metadata can be Exported to and Imported from BO, Cognos ..The objects exported or Imported can be compared in their XML formats itself.
      • MX Views ,This Feature enables the User to view the Information on the Server Grids and the Object History in the Repository Server. Can be done using the REP_SERVER_NET, REP_SERVER_NET_REF ..tables.
    • 6. OVERVIEW .. INFORMATICA CLIENT TOOLS
      • Repository Manager
        • To create and administrator the Metadata Repository using the OPB tables and the MX views.
        • To create Repository Users and Groups, assign Privileges and Permissions.
        • Manage Folders and Locks.
        • Print Reports containing Repository Data.
      • Designer
        • To add source and target definitions to the repository
        • To create mappings that contain data transformation instructions
      • Workflow Manager & Workflow Monitor
        • To create, schedule, execute and monitor sessions
    • 7. OVERVIEW .. INFORMATICA SERVER
      • The Informatica Server reads mapping and session information from the Repository.
      • It extracts data from the mapping Sources and stores the data in memory while it applies the transformation rules in the mapping.
      • The Informatica Server loads the transformed data into the mapping Targets.
      • Platforms
        • Windows NT/2000
        • UNIX / Linux
        • Solaris
    • 8. OVERVIEW .. SOURCES
      • Relational - Oracle, Sybase, Informix, IBM DB2, Microsoft SQL Server, and Teradata
      • File - Fixed and delimited flat file, COBOL file, and XML
      • Extended – PowerConnect products for PeopleSoft, SAP R/3, Siebel, and IBM MQSeries
      • Mainframe – PowerConnect for IBM DB2 on MVS
      • Other - MS Excel and Access
    • 9. OVERVIEW .. TARGETS
        • Relational - Oracle, Sybase, Sybase IQ, Informix, IBM DB2, Microsoft SQL Server, and Teradata.
        • File - Fixed and delimited flat files and XML
        • Extended –
          • Integration server to load data into SAP BW.
          • PowerConnect for IBM MQSeries to load data into IBM MQSeries message queues.
        • Other - Microsoft Access.
        • ODBC or native drivers, FTP, or external loaders.
    • 10. Questions???
    • 11. INFORMATICA SERVER & DATA MOVEMENT
    • 12. INFORMATICA SERVER AND DATA MOVEMENT
      • The Informatica Server moves data from sources to targets based on mapping and session metadata stored in a repository database
      • A session is a set of instructions that describes how and when to move data from sources to targets
      • Workflow Manager creates and manages and executes sessions, worklets and workflows.
      • Workflow Monitor is used to monitor session for debugging in case of any error
    • 13. INFORMATICA SERVER
      • When a session starts, the Informatica Server retrieves mapping and session metadata from the repository database through Repository Server initiating a Repository Agent
      • The Informatica Server runs as a daemon on UNIX and as a service on Windows NT/2000
      • The Informatica Server uses the following processes to run a session:
        • The Load Manager process - Starts the session, creates the DTM process, and sends post-session email when the session completes
        • The DTM process - Creates threads to initialize the session, read, write, and transform data, and handle pre- and post-session operations
    • 14. INFORMATICA SERVER
      • Added Features in PowerCenter 7.1
      • 64-bit support. 64-bit PowerCenter Servers on AIX and HP-UX (Itanium).
      • Partitioning enhancements. If Partitioning option is used, up to 64 partitions at any partition point in a pipeline that supports multiple partitions can be used.
      • PowerCenter Server processing enhancements. The PowerCenter Server now reads a block of rows at a time. This improves processing performance for most sessions.
      • CLOB/BLOB datatype support. You can now read and write CLOB/BLOB datatypes.
        • BLOB: BLOB (binary large object) is a variable-length binary string that can be up to 2 gigabytes (2 147 483 647 bytes) long.
        • CLOB: CLOB (character large object) is a variable-length character string that can be up to 2 gigabytes (2 147 483 647 bytes) long. If the length is zero, the value is called the empty string. This value should not be confused with the null value. Mainly Used with XML and Messaging data.
    • 15. THE LOAD MANAGER PROCESS
      • The Load Manager performs the following tasks:
        • Manages session, worklet and workflow scheduling
        • Locks the session and reads session properties
        • Reads the parameter file
        • Expands the server and session variables and parameters
        • Verifies permissions and privileges
        • Validates source and target code pages
        • Creates the session log file
        • Creates the Data Transformation Manager (DTM) process, which executes the session
    • 16. THE LOAD MANAGER PROCESS
      • The Load Manager and repository communicate with each other using Unicode
      • To prevent loss of information during data transfer, the Informatica Server and repository require compatible code pages
      • It communicates with the repository in the following situations:
        • When you start the Informatica Server
        • When you configure a session
        • When a session starts
    • 17. DATA TRANSFORMATION MANAGER PROCESS
      • DTM process is the second process associated with a session run
      • The primary purpose of the DTM process is to create and manage threads that carry out the session tasks
      • The DTM allocates process memory for the session and divides it into buffers. This is also known as buffer memory
      • It creates the main thread, which is called the master thread
      • The master thread creates and manages all other threads
      • If you partition a session, the DTM creates a set of threads for each partition to allow concurrent processing
      • When the Informatica Server writes messages to the session log, it includes the thread type and thread ID
    • 18. DTM THREADS
    • 19. DTM THREADS
      • For example, a pipeline contains one source, one target. You configure two partitions in the session properties. The DTM creates the following threads to process the pipeline:
        • Two reader threads - One for each partition.
        • Two writer threads - One for each partition
      • When the pipeline contains an Aggregator or Rank transformation, the DTM creates one additional set of threads for each Aggregator or Rank transformation
    • 20. DTM THREADS
      • When the Informatica Server processes a mapping with a Joiner transformation, it first reads the master source and builds caches based on the master rows
      • The Informatica Server then reads the detail source and processes the transformation based on the detail source data and the cache data
      • The pipeline for the master source ends at the Joiner transformation and may not have any targets
      • You cannot partition the master source for a Joiner transformation
    • 21. Questions???
    • 22. REPOSITORY SERVER
    • 23. REPOSITORY SERVER
      • Informatica client applications and Informatica server access the repository database tables through the Repository Server.
      • Informatica client connects to the repository server through the host name/IP Address and its port number
      • The Repository Server can manage multiple repositories on different machines on the network.
      • For each repository database registered with the Repository Server, it configures and manages a Repository Agent process.
      • The Repository Agent is a multi-threaded process that performs the action needed to retrieve, insert, and update metadata in the repository database tables.
    • 24. REPOSITORY SERVER (CONTD..)
      • Added Features in PowerCenter 7.1
      • Updating Repository Statistics, PowerCenter, identifies and updates statistics for all repository tables and indexes when you copy, upgrade, and restore repositories. This improves performance when PowerCenter accesses the repository
      • Pmrep, Can use pmrep to back up, disable, or enable a repository, delete a relational connection from a repository, delete repository details, truncate log files, and run multiple pmrep commands sequentially. Can also use pmrep to create, modify, and delete folder.
    • 25. REPOSITORY ADMINISTRATION CONSOLE
      • Use the Administration Console to connect to the Repository Server and perform repository administration tasks, such as creating, starting, and backing up repositories.
      • The Administration Console uses Microsoft Management Console (MMC) technology. MMC is a console framework for server and network management applications called snap-ins. The Administration Console is a snap-in for MMC.
      • The Administration Console allows you to navigate multiple Repository Servers and repositories and perform repository management tasks.
    • 26. REPOSITORY ADMINISTRATION CONSOLE
      • Repository Name Node Details
        • When you select a repository in the Console Tree, the Main window displays details about the repository in HTML view. The Main window also displays HTML links that allow you perform some repository tasks.
    • 27. Questions???
    • 28. REPOSITORY MANAGER
    • 29. REPOSITORY MANAGER WINDOW
    • 30. REPOSITORY
      • The Informatica repository tables have an open architecture
      • Metadata can include information such as
        • mappings describing how to transform source data
        • sessions indicating when you want the Informatica Server to perform the transformations
        • connect strings for sources and targets
      • The repository also stores administrative information such as
        • usernames and passwords
        • permissions and privileges
      • There are three different types of repositories:
        • Standalone repository
        • Global repository
        • Local repository
    • 31. REPOSITORY
      • Can create and store the following types of metadata in the repository:
        • Database connections
        • Global objects
        • Mappings
        • Mapplets
        • Multi-dimensional metadata
        • Reusable transformations
        • Sessions and batches
        • Shortcuts
        • Source definitions
        • Target definitions
        • Transformations
    • 32. REPOSITORY
      • Added Features in PowerCenter 7.1
      • Exchange of Metadata with other BI Tools ,Metadata can be Exported to and Imported from BO, Cognos ..The objects exported or Imported can be compared in their XML formats itself.
      • MX Views ,This Feature enables the User to view the Information on the Server Grids and the Object History in the Repository Server. Can be done using the REP_SERVER_NET, REP_SERVER_NET_REF ..tables.
    • 33. REPOSITORY MANAGER TASKS
      • Perform repository functions:
        • Create, backup, copy, restore, upgrade, and delete repositories
        • Make a repository as global repository
        • Register and unregister local repositories with a global repository
      • Implement repository security:
        • Create, edit, and delete repository users and user groups
        • Assign and revoke repository privileges and folder permissions
        • View locks and unlock objects
      • Perform folder functions:
        • Create, edit, and delete folders
        • Copy a folder within the repository or to another repository
        • Compare folders within a repository or in different repositories
        • Add and remove repository reports
      • Import and export repository connection information in the registry
      • Analyze source/target, mapping, and shortcut dependencies
    • 34. DEPENDENCY WINDOW
      • The Dependency window can display the following types of dependencies:
      • Source-target dependencies - lists all sources or targets related to the selected object and relevant information
      • Mapping dependencies - lists all mappings containing the selected object as well as relevant information
      • Shortcut dependencies - lists all shortcuts to the selected object and relevant details
    • 35. COPYING AND BACKING UP A REPOSITORY
      • Can copy a repository from one database to another
      • If the database into which the repository has to be copied contains an existing repository, the Repository Manager deletes the existing repository
      • Backing up a repository, saves the entire repository in a file
      • Can save this file in any local directory
      • Can recover data from a repository backup file
    • 36. CRYSTAL REPORTS
      • The Repository Manager includes four Crystal Reports that provide views of your metadata:
      • Mapping report (map.rpt) - Lists source column and transformation details for each mapping
      • Source and target dependencies report (S2t_dep.rpt)
      • Target table report (Trg_tbl.rpt) - Provides target field transformation expressions, descriptions, comments for each target table
      • Executed session report (sessions.rpt) - Provides information about executed sessions
    • 37. REPOSITORY SECURITY
      • Can plan and implement security using the following features:
        • User groups
        • Repository users
        • Repository privileges
        • Folder permissions
        • Locking
      • Can assign users to multiple groups
      • Privileges are assigned to groups
      • Can assign privileges to individual usernames and must assign each user to at least one user group
    • 38. REPOSITORY SECURITY – VIEWING LOCKS
      • Can view existing locks in the repository in the Repository Manager
    • 39. TYPES OF LOCKS
      • There are five kinds of locks on repository objects:
      • Read lock - Created when you open a repository object in a folder for which you do not have write permission
      • Write lock - Created when you create or edit a repository object
      • Execute lock - Created when you start a session or batch
      • Fetch lock - Created when the repository reads information about repository objects from the database
      • Save lock - Created when the repository is being Saved.
    • 40. FOLDERS
      • Folders provide a way to organize and store all metadata in the repository, including mappings and sessions
      • They are used to store sources, transformations, cubes, dimensions, Mapplets, business components, targets, mappings, sessions and batches
      • Can copy objects from one folder to another
      • Can copy objects across repositories
      • The Designer allows you to create multiple versions within a folder
      • When a new version is created, the Designer creates a copy of all existing mapping metadata in the folder and places it into the new version
      • Can copy a session within a folder, but you cannot copy an individual session to a different folder
    • 41. FOLDERS
      • To copy all sessions within a folder to a different location, you can copy the entire folder
      • Any mapping in a folder can use only those source and target definitions or reusable transformations that are stored:
        • in the same folder
        • in a shared folder and accessed through a shortcut
      • The configurable folder properties are:
        • Folder permissions
        • Folder owner
        • Owner’s group
        • Shared or not shared
    • 42. FOLDERS
      • Folders have the following permission types:
        • Read permission
        • Write permission
        • Execute permission
      • Shared folders allow users to create shortcuts to objects in the folder
      • Shortcuts inherit changes to their shared object
      • Once you make a folder shared, you cannot reverse it
    • 43. COPYING FOLDERS
      • Each time you copy a folder, the Repository Manager copies the following:
        • Sources, transformations, Mapplets, targets, mappings, and business components
        • Sessions and batches
        • Folder versions
    • 44. COPYING FOLDERS
      • When you copy a folder, the Repository Manager allows to:
        • Re-establish shortcuts
        • Choose an Informatica Server
        • Copy connections
        • Copy persisted values
        • Compare folders
        • Replace folders
    • 45. COMPARING FOLDERS
      • The Compare Folders Wizard allows to perform the following comparisons:
        • Compare objects between two folders in the same repository
        • Compare objects between two folders in different repositories
        • Compare objects between two folder versions in the same folder
      • Each comparison also allows to specify the following comparison criteria:
        • Versions to compare
        • Object types to compare
        • Direction of comparison
      • Whether or not the Repository Manger notes a similarity or difference between two folders depends on the direction of the comparison
      • One-way comparisons check the selected objects of Folder1 against the objects in Folder2
      • Two-way comparisons check objects in Folder1 against those in Folder2 and also check objects in Folder2 against those in Folder1
    • 46. COMPARING FOLDERS
    • 47. COMPARING FOLDERS
      • The comparison wizard displays the following user-customized information:
        • Similarities between objects
        • Differences between objects
        • Outdated objects
      • Can edit and save the result of the comparison
      • The Repository Manager does not compare the field attributes of the objects in the folders when performing the comparison
      • A two-way comparison can sometimes reveal information a one-way comparison cannot
      • A one-way comparison does not note a difference if an object is present in the target folder but not in the source folder
    • 48. FOLDER VERSIONS
      • Maintaining different versions lets you revert to earlier work when needed
      • When you save a version, you save all metadata at a particular point in development
      • Later versions contain new or modified metadata, reflecting work that you have completed since the last version
      • Maintaining different versions lets you revert to earlier work when needed
      • Added Features in PowerCenter 7.1
      • Can run object queries that return shortcut objects. Can also run object queries based on the latest status of an object. The query can return local objects that are checked out, the latest version of checked in objects, or a collection of all older versions of objects.
    • 49. EXPORTING AND IMPORTING OBJECTS
      • In the Designer and Workflow Manager, you can export repository objects to an XML file and then import repository objects from the XML file
      • Can export the following repository objects:
        • Sources
        • Targets
        • Transformations
        • Mapplets
        • Mappings
        • Sessions
      • Can share objects by exporting and importing objects between repositories with the same version
    • 50. Questions???
    • 51. DESIGNER
    • 52. SCREEN SHOT OF DESIGNER
    • 53. DESIGNER APPENDIX
      • Informatica’s Designer is the client application used to create and manage sources, targets, and the associated mappings between them.
      • The Informatica Server uses the instructions configured in the mapping and its associated session to move data from sources to targets.
      • The Designer allows you to work with multiple tools, in multiple folders, and in multiple repositories at a time.
      • The client application provides five tools with which to create mappings:
        • Source Analyzer. Use to import or create source definitions for flat file, ERP, and relational sources.
        • Warehouse Designer. Use to import or create target definitions.
        • Transformation Developer. Used to create reusable object that generates or modifies data.
        • Mapplet Designer. Used to create a reusable object that represents a set of transformations.
        • Mapping Designer. Used to create mappings.
    • 54. DESIGNER APPENDIX
      • The Designer consists of the following windows:
      • Navigator. Use to connect to and work in multiple repositories and folders. You can also copy objects and create shortcuts using the Navigator
      • Workspace. Use to view or edit sources, targets, Mapplets, transformations, and mappings. You can work with a single tool at a time in the workspace
      • Output. Provides details when you perform certain tasks, such as saving your work or validating a mapping. Right-click the Output window to access window options, such as printing output text, saving text to file, and changing the font size
      • Overview. An optional window to simplify viewing workbooks containing large mappings or a large number of objects
    • 55. SOURCE ANALYZER
      • The following types of source definitions can be imported or created or modified in the Source Analyzer:
        • Relational Sources – Tables, Views, Synonyms
        • Files – Fixed-Width or Delimited Flat Files, COBOL Files
        • Microsoft Excel Sources
        • XML Sources – XML Files, DTD Files, XML Schema Files
        • Data models using MX Data Model PowerPlug
        • SAP R/3, SAP BW, Siebel, IBM MQ Series by using PowerConnect
    • 56. SOURCE ANALYZER – IMPORTING RELATIONAL SOURCE DEFINITIONS
      • After importing a relational source definition, Business names for the table and columns can be entered
    • 57. SOURCE ANALYZER – IMPORTING RELATIONAL SOURCE DEFINITIONS
      • The source definition appears in the Source Analyzer. In the Navigator, the new source definition appears in the Sources node of the active repository folder, under the source database name
    • 58. SOURCE ANALYZER – FLAT FILE SOURCES
      • Supports Delimited & Fixed length files
      • Flat File Wizard prompts for the following file properties
        • File name and location
        • File code page
        • File type
        • Column names and data types
        • Number of header rows in the file
        • Column size and null characters for fixed-width files
        • Delimiter type, quote character, and escape character for delimited files
    • 59. SOURCE ANALYZER – FLAT FILE SOURCES
      • Flat file properties in the Source Analyzer :
        • Table name, business purpose, owner, and description
        • File type
        • Null characters for fixed-width files
        • Delimiter type, quote character, and escape character for delimited files
        • Column names and datatypes
        • Comments
        • HTML links to business documentation
    • 60. WAREHOUSE DESIGNER
      • To create target definitions for file and relational sources
        • Import the definition for an existing target - Import the target definition from a relational target
        • Create a target definition based on a source definition, Relational source definition, Flat file source definition
        • Manually create a target definition or design several related targets at the same time
      • Tasks of Warehouse Designer:
        • Edit target definitions
          • Change in the target definitions gets propagated to the mappings using that target
        • Create relational tables in the target database
          • If the target tables do not exist in the target database, generate and execute the necessary SQL code to create the target tables
        • Preview relational target data
    • 61. WAREHOUSE DESIGNER – CREATE/EDIT TARGET DEFINITIONS
      • Can edit Business Names, Constraints, Creation Options, Description, Keywords on the Table tab of the target definition
      • Can edit Column Name, Datatype, Precision and Scale, Not Null, Key Type, Business Name on the Columns tab of the target definition
    • 62. MAPPING
      • Mappings represent the data flow between sources and targets
      • When the Informatica Server runs a session, it uses the instructions configured in the mapping to read, transform, and write data
      • Every mapping must contain the following components:
        • Source / Target definitions.
        • Transformation / Transformations.
        • Connectors Or Links.
      • A mapping can also contain one or more Mapplets
    • 63. MAPPING
      • Sample Mapping
      Source Source Qualifier Links or Connectors Target Transformation
    • 64. MAPPING - INVALIDATION
      • On editing a mapping, the Designer invalidates sessions under the following circumstances:
        • Add or remove sources or targets
        • Remove Mapplets or transformations
        • Replace a source, target, Mapplet, or transformation while importing or copying objects
        • Add or remove Source Qualifiers or COBOL Normalizers, or change the list of associated sources for these transformations
        • Add or remove a Joiner or Update Strategy transformation.
        • Add or remove transformations from a Mapplet in the mapping
        • Change the database type for a source
    • 65. MAPPING - COMPONENTS
      • Every mapping requires at least one transformation object that determines how the Informatica Server reads the source data:
        • Source Qualifier transformation
        • Normalizer transformation
        • ERP Source Qualifier transformation
        • XML Source Qualifier transformation
      • Transformations can be created to use once in a mapping, or reusable transformations to use in multiple mappings
    • 66. MAPPING - UPDATES
      • By default, the Informatica Server updates targets based on key values
      • The default UPDATE statement for each target in a mapping can be overrode
      • For a mapping without an Update Strategy transformation, configure the session to mark source records as update
    • 67. MAPPING - VALIDATION
      • The Designer marks a mapping valid for the following reasons:
        • Connection validation - Required ports are connected and that all connections are valid
        • Expression validation - All expressions are valid
        • Object validation - The independent object definition matches the instance in the mapping
      • The Designer performs connection validation each time you connect ports in a mapping and each time you validate or save a mapping
      • You can validate an expression in a transformation while you are developing a mapping
    • 68. MAPPING - VALIDATION
    • 69. Questions???
    • 70. TRANSFORMATIONS USED IN INFORMATICA
    • 71. TRANSFORMATIONS
      • A transformation is a repository object that generates, modifies, or passes data
      • The Designer provides a set of transformations that perform specific functions
      • Transformations in a mapping represent the operations the Informatica Server performs on data
      • Data passes into and out of transformations through ports that you connect in a mapping or mapplet
      • Transformations can be active or passive
    • 72. TRANSFORMATION - TYPES
      • An active transformation can change the number of rows that pass through it
      • A passive transformation does not change the number of rows that pass through it
      • Transformations can be connected to the data flow, or they can be unconnected
      • An unconnected transformation is not connected to other transformations in the mapping. It is called within another transformation, and returns a value to that transformation
    • 73. TRANSFORMATIONS - TYPES
      • Source Qualifier Transformation Object
        • This object is used to define the reader process, or the selection for relational sources.
        • Can automatically generate the SQL, User can override it and write own SQL.
        • In the source qualifier you can also specify filters on the source data or distinct selection.
      • Update Strategy Transformation Object
        • Use the update strategy to finely control on a row by row basis, whether we want the row to be inserted, updated, deleted or a reject based on some logical condition we have derived in the mapping process.
        • The update strategy transformation is used frequently for handling Slowly Changing Dimensions.
    • 74. TRANSFORMATIONS - TYPES
      • Expression Transformation Object
        • The Expression Transformation is used for data cleansing and scrubbing.
        • There are over 80 functions within PowerCenter, such as concatenate, instring, rpad, ltrim and we use many of them in the Expression Transformation.
        • We can also create derived columns and variables in the Expression Transformation.
      • Filter Transformation
        • The Filter Transformation is used to do just that – filter data.
        • We can filter at the source using the Source Qualifier, but maybe I need to filter the data somewhere within the pipeline, perhaps on an aggregate that I’ve calculated, and we use the Filter Transformation for that. We also use the Filter to branch and load a portion of the data into one target based on some condition and the rest of the data into another target.
      • NOTE – Router transformation is used for branching, it’s MUCH more efficient than the filter transformation.
    • 75. TRANSFORMATIONS - TYPES
      • Aggregator Transformation
        • The Aggregator Transformation is used when doing those types of functions that require sorting or a group by. Functions like SUM, AVG.
        • Aggregates are done in memory and we can accept pre-sorted input to reduce the amount of memory necessary.
      • Lookup Transformation
        • Use the Lookup Transformation to lookup tables in the source database, the target database or any other database as long as we have connectivity.
        • Lookups are by default cached in memory, however you can turn off the caching by checking the option.
        • The lookup condition can be any type of Boolean expression.
      • Like the Update Strategy, the Lookup Transformation is used frequently for handling Slowly Changing Dimensions.
    • 76. TRANSFORMATIONS - TYPES
      • Sequence Generator Transformation
        • We provide a Sequence Generator Transformation for those target databases that do not have these capabilities (such as MS-SQL 6.5) and ours works just like Oracle’s sequence generator.
        • You can specify the starting value, the increment value, the upper limit and whether or not you want to reinitialize.
      • (Extended discussion: the Sequence Generator can also be used to stamp a “batch load id” into every new row loaded in a given update/refresh load. This is very useful for backing out loads if that becomes necessary.)
      • Stored Procedure Transformation
        • One of the main benefits of using PowerCenter is that we eliminate the need for coding, however if you have already written a stored procedure that you cannot duplicate within PowerCenter you can call them from within a mapping using the Stored Procedure Transformation. (We import the stored procedure arguments the same way we import the source catalog definition.)
    • 77. TRANSFORMATIONS - TYPES
      • External Procedure Transformation
        • The same holds true for External Procedures written in C, C++, or Visual Basic – they can be called from within the mapping process using the External Procedure transformation.
        • External procedure transformations are reusable transformations and we define those in the Transformation Developer (the fourth component tool in the Designer).
      • Joiner Transformation
        • The Joiner Transformation is used for heterogeneous joins within a mapping – perhaps we need to join a flat file with an Oracle table or Oracle and a Sybase table or 2 flat files together. (Joins are done in memory – and the memory profile is configurable!).
      • Sorter Transformation
        • The Sorter is used to sort the data in the Ascending or the Descending order, generally used before aggregation of data.
    • 78. TRANSFORMATIONS - TYPES
      • Normalizer Transformation
        • The Normalizer Transformation is used when you have occurs or arrays in your source data and you want to flatten the data into a normalized table structure – PowerCenter will do this automatically for you. The Normalizer is also used to pivot data.
      • Ranking Transformation
        • This is used when you are doing a very specific data mart, such as Top Performers, and you want to load the top 5 products and the top 100 customers – or the bottom 5 products.
    • 79. ACTIVE TRANSFORMATION NODES
      • Advanced External Procedure - Calls a procedure in a shared library or in the COM layer of Windows NT
      • Aggregator - Performs aggregate calculations
      • ERP Source Qualifier - Represents the rows that the Informatica Server reads from an ERP source when it runs a session
      • Filter - Filters records
      • Joiner - Joins records from different databases or flat file systems
      • Rank - Limits records to a top or bottom range
      • Router - Routes data into multiple transformations based on a group expression
      • Source Qualifier - Represents the rows that the Informatica Server reads from a relational or flat file source when it runs a session
      • Update Strategy - Determines whether to insert, delete, update, or reject records
    • 80. PASSIVE TRANSFORMATION NODES
      • Expression - Calculates a value
      • External Procedure - Calls a procedure in a shared library or in the COM layer of Windows NT
      • Input - Defines Mapplet input rows. Available only in the Mapplet Designer
      • Lookup - Looks up values
      • Output - Defines Mapplet output rows. Available only in the Mapplet Designer
      • Sequence Generator - Generates primary keys
      • Stored Procedure - Calls a stored procedure
      • XML Source Qualifier - Represents the rows that the Informatica Server reads from an XML source when it runs a session
    • 81. TRANSFORMATION
      • Added Features in PowerCenter 7.1
      • Flat file lookup: Can now perform lookups on flat files. To create a Lookup transformation using a flat file as a lookup source, the Designer invokes the Flat File Wizard. To change the name or location of a lookup between session runs, the lookup file parameter can be used .
        • Dynamic lookup cache enhancements: When you use a dynamic lookup cache, the PowerCenter Server can ignore some ports when it compares values in lookup and input ports before it updates a row in the cache. Also, you can choose whether the PowerCenter Server outputs old or new values from the lookup/output ports when it updates a row. You might want to output old values from lookup/output ports when you use the Lookup transformation in a mapping that updates slowly changing dimension tables.
      • Union transformation: Can use the Union transformation to merge multiple sources into a single pipeline. The Union transformation is similar to using the UNION ALL SQL statement to combine the results from two or more SQL statements.
    • 82. TRANSFORMATION
      • Added Features in PowerCenter 7.1
      • Custom transformation API enhancements: The Custom transformation API includes new array-based functions that allow you to create procedure code that receives and outputs a block of rows at a time. Use these functions to take advantage of the PowerCenter Server processing enhancements.
      • Midstream XML transformations: Can now create an XML Parser transformation or an XML Generator transformation to parse or generate XML inside a pipeline. The XML transformations enable you to extract XML data stored in relational tables, such as data stored in a CLOB column. You can also extract data from messaging systems, such as TIBCO or IBM MQSeries.
    • 83. TRANSFORMATIONS - PROPERTIES
      • Port Name
        • Copied ports will inherit the name of contributing port
        • Copied ports with the same name will be appended with a number
        • Types Of Ports:
          • Input: Data Input from previous stage.
          • Output: Data Output to the next stage.
          • Lookup: Port to be used to compare Data.
          • Return: The Port (Value) returned from Looking up.
          • Variable: The port that stores value temporarily.
      • Data types
        • Transformations use internal data types
        • Data types of input ports must be compatible with data types of the feeding output port
      • Port Default values - can be set to handle nulls and errors
      • Description - can enter port comments
    • 84. AGGREGATOR TRANSFORMATION
      • Performs aggregate calculations
      • Components of the Aggregator Transformation
        • Aggregate expression
        • Group by port
        • Sorted Input option
        • Aggregate cache
      • The Aggregator is an active and connected transformation
    • 85. AGGREGATOR TRANSFORMATION
      • The following aggregate functions can be used within an Aggregator transformation:
        • AVG, COUNT, FIRST, LAST, MAX , MEDIAN
        • MIN, PERCENTILE, STDDEV, SUM, VARIANCE
    • 86. EXPRESSION TRANSFORMATION
      • Can use the Expression transformation to
        • perform any non-aggregate calculations
        • Calculate values in a single row
        • test conditional statements before you output the results to target tables or other transformations
      • Ports that must be included in an Expression Transformation:
        • Input or input/output ports for each value used in the calculation
        • Output port for the expression
        • Variable Ports are used to temporarily store values.
    • 87. EXPRESSION TRANSFORMATION
      • Can enter multiple expressions in a single expression transformation
      • Can enter only one expression for each output port
      • Can create any number of output ports in the transformation
      • Can create Variable ports to store Data temporarily.
    • 88. FILTER TRANSFORMATION
      • It provides the means for filtering rows in a mapping
      • All ports in a Filter transformation are input/output
      • Only rows that meet the condition pass through it
      • Cannot concatenate ports from more than one transformation into the Filter transformation
      • To maximize session performance, include the Filter transformation as close to the sources in the mapping as possible
      • Does not allow setting output default values
      !! Filter should always be used as close to the Source, so that the Load of data carried ahead is decreased at / or near to the Source Itself.
    • 89. JOINER TRANSFORMATION
      • Joins two related heterogeneous sources residing in different locations or file systems
      • Can be used to join
        • Two relational tables existing in separate databases
        • Two flat files in potentially different file systems
        • Two different ODBC sources
        • Two instances of the same XML source
        • A relational table and a flat file source
        • A relational table and an XML source
    • 90. JOINER TRANSFORMATION
      • Use the Joiner transformation to join two sources with at least one matching port
      • It uses a condition that matches one or more pairs of ports between the two sources
      • Requires two input transformations from two separate data flows
      • It supports the following join types
        • Normal (Default)
        • Master Outer
        • Detail Outer
        • Full Outer
      • Added Features in PowerCenter 7.1
      • Can be used to join two data streams that originate from the same source.
    • 91. JOINER TRANSFORMATION
      • Joiner on Same Sources, Used when a Calculation has to be done on One part of the data and join the Transformed data to the Original Data set.
      Joiner on Multiple Sources.
    • 92. LOOKUP TRANSFORMATION
      • Used to look up data in a relational table, view, synonym or Flat File.
      • The Informatica Server queries the lookup table based on the lookup ports in the transformation.
      • It compares Lookup transformation port values to lookup table column values based on the lookup condition.
      • Can use the Lookup transformation to perform many tasks, including:
        • Get a related value
        • Perform a calculation
        • Update slowly changing dimension tables
      • Types Of Lookup’s
        • Connected
        • Unconnected
    • 93. CONNECTED LOOKUP
      • Connected Lookup Transformation
      • Receives input values directly from another transformation in the pipeline
      • For each input row, the Informatica Server queries the lookup table or cache based on the lookup ports and the condition in the transformation
      • Passes return values from the query to the next transformation
    • 94. UNCONNECTED LOOKUP
      • Unconnected Lookup Transformation
        • Receives input values from an expression using the
        • :LKP (:LKP.lookup_transformation_name(argument, argument, ...)) reference qualifier to call the lookup and returns one value.
        • Some common uses for unconnected lookups include, Testing the results of a lookup in an expression, Filtering records based on the lookup results, Marking records for update based on the result of a lookup (for example, updating slowly changing dimension tables), Calling the same lookup multiple times in one mapping.
        • With unconnected Lookups, you can pass multiple input values into the transformation, but only one column of data out of the transformation
        • Use the return port to specify the return value in an unconnected lookup transformation
    • 95. LOOKUP CACHING
      • Session performance can be improved by caching the lookup table
      • Caching can be static or dynamic
      • By default, the lookup cache remains static and does not change during the session
      • Types of Caching
        • Persistent – Cache used across sessions.
        • ReCache from Source – To Synchronize persistent cache.
        • Static – Read only cache for single Lookup.
        • Dynamic – To reflect the Change Data Directly, Used when Target Table is looked up.
        • Shared – Cache shared among multiple transformations.
      • Added Features in PowerCenter 7.1
      • Dynamic lookup cache enhancements: When you use a dynamic lookup cache, the PowerCenter Server can ignore some ports when it compares values in lookup and input ports before it updates a row in the cache. Also, you can choose whether the PowerCenter Server outputs old or new values from the lookup/output ports when it updates a row. You might want to output old values from lookup/output ports when you use the Lookup transformation in a mapping that updates slowly changing dimension tables.
    • 96. DYNAMIC LOOKUP TRANSFORMATION
      • A Lookup transformation using a dynamic cache has the following properties that a Lookup transformation using a static cache does not have:
        • NewLookupRow
        • Associated Port
    • 97. DYNAMIC LOOKUP TRANSFORMATION
      • You might want to configure the transformation to use a dynamic cache when the target table is also the lookup table. When you use a dynamic cache, the Informatica Server inserts rows into the cache as it passes rows to the target.
    • 98. ROUTER TRANSFORMATION
      • A Router transformation tests data for one or more conditions and gives the option to route rows of data that do not meet any of the conditions to a default output group
      • It has the following types of groups:
        • Input
        • Output
      • There are two types of output groups:
        • User-defined groups
        • Default group
      • Create one user-defined group for each condition that you want to specify
    • 99. COMPARING ROUTER & FILTER TRANSFORMATIONS
    • 100. SEQUENCE GENERATOR TRANSFORMATION
      • Generates numeric values
      • It can be used to
        • create unique primary key values
        • replace missing primary keys
        • cycle through a sequential range of numbers
      • It provides two output ports: NEXTVAL and CURRVAL
      • These ports can not be edited or deleted
      • Can not add ports to the sequence generator transformation
      • When NEXTVAL is connected to the input port of another transformation, the Informatica Server generates a sequence of numbers
    • 101. SEQUENCE GENERATOR TRANSFORMATION
      • Connect the NEXTVAL port to a downstream transformation to generate the sequence based on the Current Value and Increment By properties
      • The CURRVAL port is connected, only when the NEXTVAL port is already connected to a downstream transformation
    • 102. SOURCE QUALIFIER TRANSFORMATION
      • The SQ represents the records that the Informatica Server reads when it runs a session
      • Can use the SQ to perform the following tasks:
        • Join data originating from the same source database
        • Filter records when the Informatica Server reads source data
        • Specify an outer join rather than the default inner join
        • Specify sorted ports
        • Select only distinct values from the source
        • Create a custom query to issue a special SELECT statement for the Informatica Server to read source data
        • Parameter Variables can be used in the SQ.
      • For relational sources, the Informatica Server generates a query for each SQ when it runs a session
      • The default query is a SELECT statement for each source column used in the mapping
      • The Informatica Server reads only those columns in SQ that are connected to another transformation
      • The Target Load Order can be specified depending on the SQ
    • 103. SOURCE QUALIFIER TRANSFORMATION
      • The SQ Override can be used to Select only the Required Ports from the Source.
      • The SQ Override can contain Parameter Variables like $$$Sessstarttime, $$IPAddress.
      • Clicking the Generate SQL tab would result in the generation of Default Select query. This can be altered by the User.
      For an SQL to Generate a Default Query, Its should have Linked Output port !!
    • 104. SOURCE QUALIFIER TRANSFORMATION
      • The SQ Override can be used to Join two different Sources based on a key (In the WHERE Clause) and Select only the Required Ports from either of the Sources.
      For an SQL to Generate a Default Query, Its should have Linked Output port !!
    • 105. SOURCE QUALIFIER TRANSFORMATION
      • The SQ can be used to define a Source Filter which would filter off the Unwanted records at the Source itself, thus reducing the cache time.
      • The SQ can be used to Define a User-Defined Join condition also. This Join condition would be added to the Default Query generated by Informatica as a WHERE clause.
    • 106. SOURCE QUALIFIER TRANSFORMATION
      • The SQ can be used to Sort ports if it is being succeeded by an Aggregator.
      • Select Distinct Check Box when checked performs a Distinct Select on the Input Data.
      • Pre SQL can be used to run a pre Caching command, either a Delete Target or Join two source systems. Similarly Post SQL command can also be used.
    • 107. UPDATE STRATEGY TRANSFORMATION
      • The Is used to Control how the Rows are flagged for Insert, Update, Reject or Delete in a Mapping.
      • To define the flagging of rows in a session, It can be Insert, Update, Delete or Data Driven. In Data Driven Option the Session follows the Flag specified in the Mapping's Update strategy.
      • It can be configured to either Drop the Rejected rows or to forward it to next transformation, In the Later case a separate Reject Table can be used.
    • 108. UPDATE STRATEGY TRANSFORMATION
      • The Control can be separately done for Each of the Targets in the mapping.
      • Update Strategy Options:
        • Insert - Inserts row into the table.
        • Delete - Deletes row from the table.
        • Update - Updates row existing in the table.
      • Target Table Update strategy Options:
        • Update as Update - Updates each row flagged for Update if it exists in the table.
        • Update as Insert - Inserts a new row for each Update.
        • Update else Insert - Updates if row exists, else Inserts.
    • 109. RANK TRANSFORMATION
      • Allows to select only the top or bottom rank of data, not just one value
      • Can use it to return
        • the largest or smallest numeric value in a port or group
        • the strings at the top or the bottom of a session sort order
      • During the session, the Informatica Server caches input data until it can perform the rank calculations
      • Can select only one port to define a rank
    • 110. RANK TRANSFORMATION
      • When you create a Rank transformation, you can configure the following properties:
        • Enter a cache directory
        • Select the top or bottom rank
        • Select the input/output port that contains values used to determine the rank. You can select only one port to define a rank
        • Select the number of rows falling within a rank
        • Define groups for ranks
      • Rank Transformation Ports:
        • Variable port - Can use to store values or calculations to use in an expression
        • Rank port - Use to designate the column for which you want to rank values
    • 111. RANK TRANSFORMATION
      • Properties
        • Top/Bottom – This is used to specify if the top or the bottom rank needs to be output.
        • Number of Ranks – Specifies the number of Rows to be Ranked.
        • Transformation Scope –
          • Transaction – Applies the transformation logic to only the rows of current transaction
          • All-Input - Applies the transformation logic to all the rows of Input data.
    • 112. STORED PROCEDURE TRANSFORMATION
      • A Stored Procedure transformation is
        • an important tool for populating and maintaining databases
        • a precompiled collection of Transact-SQL statements and optional flow control statements, similar to an executable script
        • used to call a stored procedure
      • The stored procedure must exist in the database before creating a Stored Procedure transformation
      • One of the most useful features of stored procedures is the ability to send data to the stored procedure, and receive data from the stored procedure
      • There are three types of data that pass between the Informatica Server and the stored procedure:
        • Input/Output parameters - For many stored procedures, you provide a value and receive a value in return
        • Return values - Most databases provide a return value after running a stored procedure
        • Status codes - Status codes provide error handling for the Informatica Server during a session
    • 113. STORED PROCEDURE TRANSFORMATION
      • The following list describes the options for running a Stored Procedure transformation:
        • Normal - During a session, the stored procedure runs where the transformation exists in the mapping on a row-by-row basis
        • Pre-load of the Source - Before the session retrieves data from the source, the stored procedure runs
        • Post-load of the Source - After the session retrieves data from the source, the stored procedure runs
        • Pre-load of the Target - Before the session sends data to the target, the stored procedure runs
        • Post-load of the Target - After the session sends data to the target, the stored procedure runs
    • 114. STORED PROCEDURE TRANSFORMATION
      • Can set up the Stored Procedure transformation in one of two modes, either connected or unconnected
      • The flow of data through a mapping in connected mode also passes through the Stored Procedure transformation
      • Cannot run the same instance of a Stored Procedure transformation in both connected and unconnected mode in a mapping. You must create different instances of the transformation
      • The unconnected Stored Procedure transformation is not connected directly to the flow of the mapping
      • It either runs before or after the session, or is called by an expression in another transformation in the mapping
    • 115. STORED PROCEDURE TRANSFORMATION
      • A Connected Stored Procedure transformation is linked with the Input port from the preceding transformation and the Output port can be linked ahead.
      • A Unconnected Stored Procedure transformation can be called by passing the parameters of Input through an expression in the following format,
      • :SP.<Proc_Name>(Input, Output)
    • 116. CUSTOM TRANSFORMATION
      • Custom Transformation Operates in Conjunction with the Procedures that are created outside the Designer. The PowerCenter Server uses generated Functions to Interface with the procedure and the Procedure code should be developed using API functions. The PowerCenter Server can pass a Single row of data or an Array depending on the Custom function.
      • The types of generated functions are -
        • Initialization Function - To initialize processes before data is passed to custom function.
        • Notification Function - To send notifications
        • Deinitialization Function - To deinitialize the processes after data is passed to custom function.
      • The Types of API Functions are -
        • Set Data Access Mode Functions
        • Navigation Functions
        • Property Functions
        • Data Handling Functions
      • Most of these Functions are associated with Handles, These Handles are Internal Informatica Functions which Guide the process flow.
    • 117. SORTER TRANSFORMATION
      • A Sorter Transformation is used to sort Data.
      • Types of Sorting:
        • Ascending/Descending
        • Case Sensitive - Does Case sensitive ascending or descending sort.
        • Distinct - Gives distinctly sorted output rows.
      • A sorter can be used to sort data from a Relational Source or a Flat file. The Sort key has to be specified based on which the sort would be done.
    • 118. SORTER TRANSFORMATION
      • The Fields can be sorted in Either Ascending or Descending orders by specifying the type of sort required in the Direction field. Multiple fields can be marked as Keys and different sorts can be done on them.
      • When Case Sensitive Check box is Checked, the sorting is done on case sensitive basis.
      • Working Directory is the directory in which Informatica Caches the files for sort.
      • Distinct Check box is used to get Distinct sorted output.
    • 119. TRANSFORMATION LANGUAGE
      • The designer provides a transformation language to help you write expressions to transform source data
      • With the transformation language, you can create a transformation expression that takes the data from a port and changes it
      • Can write expressions in the following transformations:
        • Aggregator
        • Expression
        • Filter
        • Rank
        • Router
        • Update Strategy
    • 120. TRANSFORMATION LANGUAGE
      • Expressions can consist of any combination of the following components:
        • Ports (input, input/output, variable)
        • String literals, numeric literals
        • Constants
        • Functions
        • Local and system variables
        • Mapping parameters and mapping variables
        • Operators
        • Return values
    • 121. TRANSFORMATION LANGUAGE
      • The functions available in PowerCenter are
        • Aggregate Functions e.g. AVG, MIN, MAX
        • Character Functions e.g. CONCAT, LENGTH
        • Conversion Functions e.g. TO_CHAR, TO_DATE
        • Date Functions e.g. DATE_DIFF, LAST_DAY
        • Numeric Functions e.g. ABS, CEIL, LOG
        • Scientific Functions e.g. COS, SINH
        • Special Functions e.g. DECODE, IIF, ABORT
        • Test Functions e.g. ISNULL, IS_DATE
        • Variable Functions e.g. SETMAXVARIABLE
    • 122. TRANSFORMATION EXPRESSIONS
      • The pre-compiled and tested transformation expressions help you create simple or complex transformation expressions:
      • Functions. Over 60 SQL-like functions allow you to change data in a mapping.
      • Aggregates.
        • Calculate a single value for all records in a group.
        • Return a single value for each group in an Aggregator transformation.
        • Apply filters to calculate values for specific records in the selected ports.
        • Use operators to perform arithmetic within the function.
        • Calculate two or more aggregate values derived from the same source columns in a single pass.
        • Filter condition can be applied to all aggregate functions. The filter condition must evaluate to TRUE, FALSE, or NULL. If the filter condition evaluates to NULL or FALSE, the Informatica Server does not select the record.
        • For example, the following expression calculates the median salary for all employees who make more than $50,000:
      • MEDIAN( SALARY, SALARY > 50000 )
    • 123. TRANSFORMATION EXPRESSIONS
      • Characters.
        • Character functions assist in the conversion, extraction and identification of sub-strings.
        • For example, the following expression evaluates a string, starting from the end of the string. The expression finds the first space and then deletes characters from first space to end of line.
        • SUBSTR( CUST_NAME,1,INSTR( CUST_NAME,' ' ,-1,1 ))
      • Conversions.
        • Conversion functions assist in the transformation of data from one type to another.
        • For example, the following expression converts the dates in the DATE_PROMISED port to text in the format MON DD YYYY: TO_CHAR( DATE_PROMISED, 'MON DD YYYY' )
      DATE_PROMISED RETURN VALUE Apr 1 1998 12:00:10AM 'Apr 01 1998'
    • 124. TRANSFORMATION EXPRESSIONS
      • Dates.
        • Date functions help you round, truncate, or compare dates; extract one part of a date; or to perform arithmetic on a date.
        • For example, the following expression would return the month portion of the date,
      • GET_DATE_PART(Apr 1 1997 00:00:00, ‘MM’)
      • Numerical and Scientific.
        • Numerical and scientific functions assist with mathematical operations needed in transformation processing.
        • For example, the following expression returns the average order for a Stabilizing Vest, based on the first five records in the Sales port, and thereafter, returns the average for the last five records read: MOVINGAVG( SALES, 5 )
      RECORD_NO SALES RETURN VALUE 1 600 NULL 5 550 358 6 39 245.8
    • 125. TRANSFORMATION EXPRESSIONS
      • Miscellaneous.
        • Informatica also provides functions to assist in:
          • aborting or erroring out records
          • developing if then else structures
          • looking up values from a specified external or static table
          • testing values for validity (such as date or number format)
        • For example, the following expression might cause the server to skip a value.
        • IIF(SALES < 0, ERROR( 'Negative value found'), EMP_SALARY)
      SALES RETURN VALUE 100 100 -500 Server skips record 400.55 400.55 800.10 800.10
    • 126. TRANSFORMATION EXPRESSIONS
      • Operators. Use transformation operators to create transformation expressions to perform mathematical computations, combine data, or compare data.
      • Constants. Use built-in constants to reference values that remain constant, such as TRUE, FALSE, and NULL.
      • Variables. Use built-in variables to write expressions that reference values that vary, such as the system date. You can also create local variables within a transformation.
      • Return values. You can also write expressions that include the return values from Lookup, Stored Procedure, and External Procedure transformations
    • 127. Questions???
    • 128. RE – USABLE TRANSFORMATIONS AND MAPPLETS
    • 129. REUSABLE TRANSFORMATION
      • A Transformation is said to be in reusable mode when multiple instances of the same transformation can be created.
      • Reusable transformations can be used in multiple mappings.
      • Creating Reusable transformations:
        • Design it in the Transformation Developer
        • Promote a standard transformation from the Mapping Designer.
    • 130. MAPPLET
      • A Mapplet is a reusable object that represents a set of transformations
      • It allows to reuse transformation logic and can contain as many transformations as needed
      • Mapplets can:
        • Include source definitions
        • Accept data from sources in a mapping
        • Include multiple transformations
        • Pass data to multiple pipelines
        • Contain unused ports
    • 131. SAMPLE MAPPLET IN A MAPPING
    • 132. EXPANDED MAPPLET
    • 133. MAPPLET - COMPONENTS
      • Each Mapplet must include the following:
        • One Input transformation and/or Source Qualifier transformation
        • At least one Output transformation
      • A Mapplet should contain exactly one of the following:
        • Input transformation with at least one port connected to a transformation in the Mapplet
        • Source Qualifier transformation with at least one port connected to a source definition
    • 134. Questions???
    • 135. WORKFLOW MANAGER & WORKFLOW MONITOR
    • 136. WORKFLOW MANAGER & WORKFLOW MONITOR Server Manager Workflow Manager Workflow Monitor
        • 1. Task Developer 2. Workflow Designer 3. Worklet Designer
      1. Gantt Chart 2. Task View
    • 137. WORKFLOW MANAGER
      • The Workflow Manager replaces the Server Manager in version 5.0. Instead of running sessions, you now create a process called the workflow in the Workflow Manager.
      • A workflow is a set of instructions on how to execute tasks such as sessions, emails, and shell commands.
      • A session is now one of the many tasks you can execute in the Workflow Manager.
      • The Workflow Manager provides other tasks such as Assignment, Decision, and Events. You can also create branches with conditional links. In addition, you can batch workflows by creating worklets in the Workflow Manager.
    • 138. WORKFLOW MANAGER – SCREEN SHOT
    • 139. WORKFLOW MANAGER TOOLS
      • Task Developer
        • Use the Task Developer to create tasks you want to execute in the workflow.
      • Workflow Designer
        • Use the Workflow Designer to create a workflow by connecting tasks with links. You can also create tasks in the Workflow Designer as you develop the workflow.
      • Worklet Designer
        • Use the Worklet Designer to create a worklet.
    • 140. WORKFLOW TASKS
      • Command. Specifies a shell command run during the workflow.
      • Control. Stops or aborts the workflow.
      • Decision. Specifies a condition to evaluate.
      • Email. Sends email during the workflow.
      • Event-Raise. Notifies the Event-Wait task that an event has occurred.
      • Event-Wait. Waits for an event to occur before executing the next task.
      • Session. Runs a mapping you create in the Designer.
      • Assignment. Assigns a value to a workflow variable.
      • Timer. Waits for a timed event to trigger.
    • 141. CREATE TASK
    • 142. WORKFLOW MONITOR
      • PowerCenter 6.0 provides a new tool, the Workflow Monitor, to monitor workflow, worklets, and tasks.
      • The Workflow Monitor displays information about workflows in two views:
      • 1. Gantt Chart view 2. Task view.
      • You can monitor workflows in online and offline mode.
    • 143. WORKFLOW MONITOR – GANTT CHART
    • 144. WORKFLOW MONITOR – TASK VIEW
    • 145. WORKFLOW MONITOR
      • Added Features in PowerCenter 7.1
      • The Workflow Monitor includes the following performance and usability enhancements:
      • When you connect to the PowerCenter Server, you no longer distinguish between online or offline mode.
      • You can open multiple instances of the Workflow Monitor on one machine.
      • You can simultaneously monitor multiple PowerCenter Servers registered to the same repository.
      • The Workflow Monitor includes improved options for filtering tasks by start and end time.
      • The Workflow Monitor displays workflow runs in Task view chronologically with the most recent run at the top. It displays folders alphabetically.
      • You can remove the Navigator and Output window.
    • 146. Questions???
    • 147. PERFORMANCE TUNING
    • 148. PERFORMANCE TUNING
      • First step in performance tuning is to identify the performance bottleneck in the following order:
        • Target
        • Source
        • Mapping
        • Session
        • System
      • The most common performance bottleneck occurs when the Informatica Server writes to a target database.
    • 149. TARGET BOTTLENECKS
      • Identifying
        • A target bottleneck can be identified by configuring the session to write to a flat file target.
      • Optimizing
        • Dropping Indexes and Key Constraints before loading.
        • Increasing commit intervals.
        • Use of Bulk Loading / External Loading.
    • 150. SOURCE BOTTLENECKS
      • Identifying
        • Add a filter condition after Source qualifier to false so that no data is processed past the filter transformation. If the time it takes to run the new session remains about the same, then there is a source bottleneck.
        • In a test mapping remove all the transformations and if the performance is similar, then there is a source bottleneck.
      • Optimizing
        • Optimizing the Query by using hints.
        • Use Informatica Conditional Filters if the source system lacks indexes.
    • 151. MAPPING BOTTLENECKS
      • Identifying
        • If there is no source bottleneck, add a Filter transformation in the mapping before each target definition. Set the filter condition to false so that no data is loaded into the target tables. If the time it takes to run the new session is the same as the original session, there is a mapping bottleneck.
      • Optimizing
        • Configure for Single-Pass reading
        • Avoid unnecessary data type conversions.
        • Avoid database reject errors.
        • Use Shared Cache / Persistent Cache
    • 152. SESSION BOTTLENECKS
      • Identifying
        • If there is no source, Target or Mapping bottleneck, then there may be a session bottleneck.
        • Use Collect Performance Details. Any value other than zero in the readfromdisk and writetodisk counters for Aggregator, Joiner, or Rank transformations indicate a session bottleneck. Low (0-20%) BufferInput_efficiency and BufferOutput_efficiency counter values also indicate a session bottleneck.
      • Optimizing
        • Increase the number of partitions.
        • Tune session parameters.
          • DTM Buffer Size (6M – 128M)
          • Buffer Block Size (4K – 128K)
          • Data (2M – 24 M )/ Index (1M-12M) Cache Size
        • Use incremental Aggregation if possible.
    • 153. SESSION BOTTLENECKS - MEMORY
      • Configure the index and data cache memory for the Aggregator, Rank, and Joiner transformations in the Configuration Parameters dialog box
      • The amount of memory you configure depends on partitioning, the transformation that requires the largest cache, and how much memory cache and disk cache you want to use
    • 154. INCREMENTAL AGGREGATION
      • First Run creates idx and dat files.
      • Second Run performs the following actions:
      • For each i/p record, the Server checks historical information in the index file for a corresponding group, then:
        • If it finds a corresponding group, it performs the aggregate operation incrementally, using the aggregate data for that group, and saves the incremental change
        • If it does not find a corresponding group, it creates a new group and saves the record data
      • When writing to the target Informatica Server
        • Updates modified aggregate groups in the target
        • Inserts new aggregate data
        • Deletes removed aggregate data
        • Ignores unchanged aggregate data
        • Saves modified aggregate data in the index and data files
    • 155. INCREMENTAL AGGREGATION
      • You can find options for incremental aggregation on the Transformations tab in the session properties
      • The Server Manager displays a warning indicating the Informatica Server overwrites the existing cache and a reminder to clear this option after running the session
    • 156. SYSTEM BOTTLENECKS
      • Identifying
        • If there is no source, Target, Mapping or Session bottleneck, then there may be a system bottleneck.
        • Use system tools to monitor CPU usage, memory usage, and paging.
        • On Windows :- Task Manager
        • On Unix – Systems toots like sar, iostat. For E.g. sar –u (%usage on user, idle time, i/o waiting time)
      • Optimizing
        • Improve network speed.
        • Improve CPU performance
        • Check hard disks on related machines
        • Reduce Paging
    • 157. PMCMD
      • Can use the command line program pmcmd to communicate with the Informatica Server
      • Can perform the following actions with pmcmd:
        • Determine if the Informatica Server is running
        • Start sessions and batches
        • Stop sessions and batches
        • Recover sessions
        • Stop the Informatica Server
      • Can configure repository usernames and passwords as environmental variables with pmcmd
      • can also customize the way pmcmd displays the date and time on the machine running the Informatica Server
      • pmcmd returns zero on success and non-zero on failure
      • You can use pmcmd with operating system scheduling tools like cron to schedule sessions, and you can embed pmcmd into shell scripts or Perl programs to run or schedule sessions
    • 158. PMCMD
      • Need the following information to use pmcmd:
        • Repository username
        • Repository password
        • Connection type - The type of connection from the client machine to the Informatica Server
        • Port or connection - The TCP/IP port number or IPX/SPX connection (Windows NT/2000 only) to the Informatica Server
        • Host name - The machine hosting the Informatica Server
        • Session or batch name - The names of any sessions or batches you want to start or stop
        • Folder name - The folder names for those sessions or batches
        • Parameter file
    • 159. COMMIT POINTS
      • A commit interval is the interval at which the Informatica Server commits data to relational targets during a session
      • The commit point can be a factor of the commit interval, the commit interval type, and the size of the buffer blocks
      • The commit interval is the number of rows you want to use as a basis for the commit point
      • The commit interval type is the type of rows that you want to use as a basis for the commit point
      • Can choose between the following types of commit interval
        • Target-based commit
        • Source-based commit
      • During a source-based commit session, the Informatica Server commits data to the target based on the number of rows from an active source in a single pipeline
    • 160. COMMIT POINTS
      • During a target-based commit session, the Informatica Server continues to fill the writer buffer after it reaches the commit interval
      • When the buffer block is filled, the Informatica Server issues a commit command
      • As a result, the amount of data committed at the commit point generally exceeds the commit interval
    • 161. COMMIT POINTS
      • During a source-based commit session, the Informatica Server commits data to the target based on the number of rows from an active source in a single pipeline
      • These rows are referred to as source rows
      • A pipeline consists of a source qualifier and all the transformations and targets that receive data from the source qualifier
      • An active source can be any of the following active transformations:
        • Advanced External Procedure
        • Source Qualifier
        • Normalizer
        • Aggregator
        • Joiner
        • Rank
        • Mapplet, if it contains one of the above transformations
    • 162. COMMIT POINTS
      • When the Informatica Server runs a source-based commit session, it identifies the active source for each pipeline in the mapping
      • The Informatica Server generates a commit row from the active source at every commit interval
      • When each target in the pipeline receives the commit row, the Informatica Server performs the commit
    • 163. COMMIT POINTS
    • 164. MULTIPLE SERVERS
      • You can register multiple PowerCenter Servers with a PowerCenter repository
      • Can run these servers at the same time
      • Can distribute the repository session load across available servers to improve overall performance
      • Can use the Server Manager to administer and monitor multiple servers
      • With multiple Informatica Servers, you need to decide which server you want to run each session and batch
      • You can register and run only one PowerMart Server in a local repository
      • Cannot start a PowerMart Server if it is registered in a local repository that has multiple servers registered to it
    • 165. MULTIPLE SERVERS
      • When attached to multiple servers, you can only view, or monitor, one Informatica Server at a time, but you have access to all the servers in the repository
    • 166. Questions???
    • 167. DEBUGGER
    • 168. DEBUGGER
      • Can debug a valid mapping to gain troubleshooting information about data and error conditions
      • To debug a mapping, you configure and run the Debugger from within the Mapping Designer
      • When you run the Debugger, it pauses at breakpoints and allows you to view and edit transformation output data
      • After you save a mapping, you can run some initial tests with a debug session before you configure and run a session in the Server Manager
    • 169. DEBUGGER
    • 170. DEBUGGER
      • Can Use the following process to debug a mapping:
        • Create breakpoints
        • Configure the Debugger
        • Run the Debugger
        • Monitor the Debugger
          • Debug log
          • Session log
          • Target window
          • Instance window
        • Modify data and breakpoints
      • A breakpoint can consist of an instance name, a breakpoint type, and a condition
    • 171. DEBUGGER
      • After you set the instance name, breakpoint type, and optional data condition, you can view each parameter in the Breakpoints section of the Breakpoint Editor
    • 172. Questions???
    • 173. INFORMATICA POWERCENTER 7.1 ENHANCEMENTS This Presentation part describes new features and enhancements to Informatica PowerCenter version 7.0. forming Informatica PowerCenter version 7.1
    • 174. INFORMATICA POWERCENTER 7.1
        • Configuration Management Deployment Groups
        • Repository & Repository Server
        • PowerCenter Server
        • PowerCenter Webservices
        • XML Enhancements
        • Security
        • Client Usability Enhancements
        • Other Enhancements
        • Data Profiling Built into PowerCenter
        • Data Profiling
        • PowerCenter Metadata Reporter & Server
        • Transformations
        • Usability
    • 175. CONFIGURATION MANAGEMENT
      • Full Object Level Versioning – Run Object Queries that return shortcut Objects and also the Latest used / Checked out object.
      • Labels
      • Compare/Difference of Objects
      • Query/Reporting
      • Deployment Groups
      • Command Line Facilities
      • XML Object Export/Import Enhancements
    • 176. DEPLOYMENT GROUPS
      • Define collections of objects to deploy (move from one repository to another)
      • Enables automation of deployment
      • Can contain objects within any folder in the repository
      • Dynamic Deployment Groups execute a query to determine the objects to move
    • 177.
      • Repository
        • Exchange metadata with business intelligence tools. You can export metadata to and import metadata from other business intelligence tools, such as Cognos Report Net and Business Objects.
        • Object import and export enhancements. You can compare objects in an XML file to objects in the target repository when you import objects.
        • MX views. MX views have been added to help you analyze metadata stored in the repository. REP_SERVER_NET and REP_SERVER_NET_REF views allow you to see information about server grids. REP_VERSION_PROPS allows you to see the version history of all objects in a PowerCenter repository.
      • Repository Server
        • Updating repository statistics. PowerCenter now identifies and updates statistics for all repository tables and indexes when you copy, upgrade, and restore repositories. This improves performance when PowerCenter accesses the repository.
      Repository & Repository Server
    • 178.
      • Increased repository performance. You can increase repository performance by skipping information when you copy, back up, or restore a repository. You can choose to skip MX data, workflow and session log history, and deploy group history.
      • pmrep . You can use pmrep to back up, disable, or enable a repository, delete a relational connection from a repository, delete repository details, truncate log files, and run multiple pmrep commands sequentially. You can also use pmrep to create, modify, and delete a folder.
      Repository & Repository Server
    • 179.
      • DB2 bulk loading. You can enable bulk loading when you load to IBM DB2 8.1.
      • Distributed processing. If you purchase the Server Grid option, you can group PowerCenter Servers registered to the same repository into a server grid. In a server grid, PowerCenter Servers balance the workload among all the servers in the grid.
      • Row error logging. The session configuration object has new properties that allow you to define error logging. You can choose to log row errors in a central location to help understand the cause and source of errors.
      • External loading enhancements. When using external loaders on Windows, you can now choose to load from a named pipe. When using external loaders on UNIX, you can now choose to load from staged files.
      • External loading using Teradata Warehouse Builder. You can use Teradata Warehouse Builder to load to Teradata. You can choose to insert, update, upsert, or delete data. Additionally, Teradata Warehouse Builder can simultaneously read from multiple sources and load data into one or more tables.
      PowerCenter Server
    • 180.
      • Mixed mode processing for Teradata external loaders. You can now use data driven load mode with Teradata external loaders. When you select data driven loading, the PowerCenter Server flags rows for insert, delete, or update. It writes a column in the target file or named pipe to indicate the update strategy. The control file uses these values to determine how to load data to the target.
      • Concurrent processing. The PowerCenter Server now reads data concurrently from sources within a target load order group. This enables more efficient joins with minimal usage of memory and disk cache.
      • Real time processing enhancements. You can now use real time processing in sessions that also process active transformations, such as the Aggregator transformation. You can apply the transformation logic to rows defined by transaction boundaries.
      PowerCenter Server
    • 181. POWERCENTER WEBSERVICES
      • Web services client
        • − Pulling/pushing data from a Web service data source
        • − Call an external Web service like any other Transform
        • − Access and manage PowerCenter metadata
      • Web services server
        • − Receive, process and respond to Web services requests from external applications
        • − Integrate seamlessly into enterprise Web services processes
        • − Web Services interface to read/write repository metadata
      • Web Services Provider
      • Real-time Web Services. Real-time Web Services allows you to create services using the Workflow Manager and make them available to web service clients through the Web Services Hub. The PowerCenter Server can perform parallel processing of both request-response and one-way services.
      • Web Services Hub. The Web Services Hub now hosts Real-time Web Services in addition to Metadata Web Services and Batch Web Services. You can install the Web Services Hub on a JBoss application server.
    • 182.
      • Enhanced support for XML Processing
        • Large XML files
        • XML transformations
        • Schema, namespace, elements and attributes
      • Full XML import/export of repository objects
        • Workflows, worklets, sessions, mappings, transformations
        • Multiple objects in a single XML file
        • Automatic handling of dependent objects
        • Objects can span multiple folders across a repository
      XML Enhancements
    • 183.
      • LDAP Support
      • Login security through LDAP
      • Audit trail using log file
      • SDK for custom authentication schemes
      • Support corporate security policies
      • Automatically and without manual intervention
        • Changes in employee status
        • Changes in project responsibility
      Security
    • 184.
      • Port Change Propagation
      • Mass validation of objects
      • Partitioning UI improvements
      • Session Editor enhancements
      • Copy objects in Repository Manager
      Client Usability Enhancements
    • 185. Other Enhancements
        • Session Level Error Logging
        • Custom Transformations (MGEP)
        • PowerCenter Metadata Reporter
        • Connectivity Enhancements
      • − PowerChannel, PowerConnect for SAP
        • Sort/Merge Joiner Enhancements
        • New Metadata Exchanges for Data Models - Embarcadero Studio and ERWin AllFusion Model Manager
    • 186. Applications / Database / DataMarts / Legacy Systems / Real Time Profiling Access Via Designer / Wizard Profile Rules Mapping INFORMATICA POWERCENTER Profiling Warehouse In PowerCenter In PowerAnalyser In 3 rd Party reporting tool Reports Data Profiling Built into PowerCenter
    • 187. Data Profiling
      • Data Profiling for VSAM sources. You can now create a data profile for VSAM sources.
      • Support for verbose mode for source-level functions. You can now create data profiles with source-level functions and write data to the Data Profiling warehouse in verbose mode.
      • Aggregator function in auto profiles. Auto profiles now include the Aggregator function.
      • Creating auto profile enhancements. You can now select the columns or groups you want to include in an auto profile and enable verbose mode for the Distinct Value Count function.
      • Purging data from the Data Profiling warehouse. You can now purge data from the Data Profiling warehouse.
    • 188.
      • Source View in the Profile Manager. You can now view data profiles by source definition in the Profile Manager.
      • PowerCenter Data Profiling report enhancements. You can now view PowerCenter Data Profiling reports in a separate browser window, resize columns in a report, and view verbose data for Distinct Value Count functions.
      • Prepackaged domains. Informatica provides a set of prepackaged domains that you can include in a Domain Validation function in a data profile.
      Data Profiling
    • 189.
      • PowerCenter Server
      • 64-bit support. You can now run 64-bit PowerCenter Servers on AIX and HP-UX (Itanium).
      • Partitioning enhancements. If you have the Partitioning option, you can define up to 64 partitions at any partition point in a pipeline that supports multiple partitions.
      • PowerCenter Server processing enhancements. The PowerCenter Server now reads a block of rows at a time. This improves processing performance for most sessions.
      • CLOB/BLOB datatype support. You can now read and write CLOB/BLOB datatypes.
      •  
      • PowerCenter Metadata Reporter
      • PowerCenter Metadata Reporter modified some report names and uses the PowerCenter 7.1 MX views in its schema.
      PowerCenter Metadata Reporter & Server
    • 190.
      • Flat file lookup. You can now perform lookups on flat files. When you create a Lookup transformation using a flat file as a lookup source, the Designer invokes the Flat File Wizard. You can also use a lookup file parameter if you want to change the name or location of a lookup between session runs.
      • Dynamic lookup cache enhancements. When you use a dynamic lookup cache, the PowerCenter Server can ignore some ports when it compares values in lookup and input ports before it updates a row in the cache. Also, you can choose whether the PowerCenter Server outputs old or new values from the lookup/output ports when it updates a row. You might want to output old values from lookup/output ports when you use the Lookup transformation in a mapping that updates slowly changing dimension tables.
      • Union transformation. You can use the Union transformation to merge multiple sources into a single pipeline. The Union transformation is similar to using the UNION ALL SQL statement to combine the results from two or more SQL statements.
      Transformations
    • 191.
      • Custom transformation API enhancements. The Custom transformation API includes new array-based functions that allow you to create procedure code that receives and outputs a block of rows at a time. Use these functions to take advantage of the PowerCenter Server processing enhancements.
      • Midstream XML transformations. You can now create an XML Parser transformation or an XML Generator transformation to parse or generate XML inside a pipeline. The XML transformations enable you to extract XML data stored in relational tables, such as data stored in a CLOB column. You can also extract data from messaging systems, such as TIBCO or IBM MQSeries.
      Transformations
    • 192. Usability
      • Copying objects. You can now copy objects from all the PowerCenter Client tools using the copy wizard to resolve conflicts. You can copy objects within folders, to other folders, and to different repositories. Within the Designer, you can also copy segments of mappings to a workspace in a new folder or repository.
      • Comparing objects. You can compare workflows and tasks from the Workflow Manager. You can also compare all objects from within the Repository Manager.
      • Change propagation. When you edit a port in a mapping, you can choose to propagate changed attributes throughout the mapping. The Designer propagates ports, expressions, and conditions based on the direction that you propagate and the attributes you choose to propagate.
      • Enhanced partitioning interface. The Session Wizard is enhanced to provide a graphical depiction of a mapping when you configure partitioning.
    • 193. Usability
      • Revert to saved. You can now revert to the last saved version of an object in the Workflow Manager. When you do this, the Workflow Manager accesses the repository to retrieve the last-saved version of the object.
      • Enhanced validation messages. The PowerCenter Client writes messages in the Output window that describe why it invalidated a mapping or workflow when you modify a dependent object.
      • Validate multiple objects. You can validate multiple objects in the repository without fetching them into the workspace. You can save and optionally check in objects that change from invalid to valid status as a result of the validation. You can validate sessions, mappings, mapplets, workflows, and worklets.
      • View dependencies. Before you edit or delete versioned objects, such as sources, targets, mappings, or workflows, you can view dependencies to see the impact on other objects. You can view parent and child dependencies, and global shortcuts across repositories. Viewing dependencies assists you in modifying objects and composite objects without breaking dependencies
    • 194.  

    ×