Informatica Training

ETL Tool – Informatica Training

Course Objectives
At the end of this course you will:
Understand how to use all major PowerCenter 7 components
Be able to perform basic Repository administration tasks
Be able to build basic ETL Mappings and Mapplets
Be able to create, run and monitor Workflows
Be able to troubleshoot most problems

PowerCenter 7 Architecture Not Shown: Client ODBC Connections for Source and Target metadata Targets Sources native native TCP/IP Heterogeneous Targets Repository Repository Server Repository Agent TCP/IP native Server Heterogeneous Sources Repository Designer Workflow Workflow Rep Server Manager Manager Monitor Administrative Console

Repository Server
Each Repository has an independent architecture for the management of the physical Repository tables
Components: one Repository Server, and a Repository Agent for each Repository
Client overhead for Repository management is greatly reduced by the Repository Server Repository Repository Server Repository Agent Server Repository Manager Repository Server Administration Console

Design Process
Create Source definition(s)
Create Target definition(s)
Create a Mapping
Create a Session Task
Create a Workflow from Task components
Run the Workflow and verify the results

Methods of Analyzing Sources
Import from Database
Import from File
Import from Cobol File
Import from XML file
Create manually
Repository Relational Flat file COBOL file XML file Source Analyzer

Creating Target Definitions
Methods of creating Target Definitions
Import from Database
Import from an XML file
Manual Creation
Automatic Creation

Import Definition from Database
Can “ Reverse engineer” existing object definitions from a database system catalog or data dictionary
Table View Synonym Warehouse Designer Database DEF Repository Repository Server Repository Agent TCP/IP native DEF ODBC

Transformation Types
Source Qualifier: reads data from Flat File and Relational Sources
XML Source Qualifier: reads XML data
Normalizer: reorganizes records from VSAM, Relational and Flat File
Expression: performs row-level calculations
Aggregator: performs aggregate calculations
Filter: drops rows conditionally
Router: splits rows conditionally
Sorter: sorts data
Informatica PowerCenter 6 provides 17 objects for data transformation:

Transformation Types
Update Strategy: tags rows for insert, update, delete, reject
Lookup: looks up values and passes them to other objects
Joiner: joins heterogeneous sources
Stored Procedure: calls a database stored procedure
Sequence Generator: generates unique ID values
Rank: limits records to the top or bottom of a range
Transformations objects (continued)

Active Transformation
Connected
Ports
All input / output
Specify a Filter condition
Usage
Filter rows from flat file sources
Single pass source(s) into multiple targets
Drops rows conditionally Filter Transformation

Aggregator Transformation
Connected
Ports
Mixed
Variables allowed
Group By allowed
Create expressions in output or variable ports
Usage
Standard aggregations
Performs aggregate calculations

Incremental Aggregation Trigger in Session Properties, Performance Tab
Cache is saved into $PMCacheDir: aggregatorname.DAT
aggregatorname.IDX
Upon next run, files are overwritten with new cache information
Best Practice is to copy these files in case a rerun of data is ever required. Reinitialize when no longer needed, e.g. – at the beginning new month processing Example: When triggered, PowerCenter Server will save new MTD totals. Upon next run (new totals), Server will subtract old totals; difference will be passed forward MTD calculation

Joiner Transformation
Connected
Ports
All input or input / output
“ M” denotes port comes from master source
Specify the Join condition
Usage
Join two flat files
Join two tables from different databases
Join a flat file with a relational table
Performs heterogeneous joins on records from different databases or flat file sources

Mid-Mapping Join
The Joiner does not accept input in the following situations:
Both input pipelines begin with the same Source Qualifier
Both input pipelines begin with the same Normalizer
Both input pipelines begin with the same Joiner
Either input pipeline contains an Update Strategy

Sorter Transformation Sorts data from any source, at any point in a data flow
Connected
Ports
Input/Output
Define one or more sort keys
Define sort order for each key
Usage
Sort data from Flat Files
Re-sort data in a mapping
Sort Keys Sort Order

Sequence Generator Transformation Generates unique keys for any port on a row
Passive Transformation
Connected
Ports
Two predefined output ports,
NEXTVAL and
CURRVAL
No input ports allowed
Usage
Generate sequence numbers
Shareable across mappings

Lookup Transformation Looks up values in a database table and provides data to other components in a Mapping
Passive Transformation
Connected / Unconnected
Ports
Mixed
“L” denotes Lookup port
“R” denotes port used as a return value (unconnected Lookup only)
Specify the Lookup Condition
Usage
Get related values
Verify if records exists or if data has changed

To Cache or not to Cache?
Caching can significantly impact performance
Cached
Lookup table data is cached locally on the Server
Mapping rows are looked up against the cache
Only one SQL SELECT is needed
Uncached
Each Mapping row needs one SQL SELECT
Rule Of Thumb : Cache if the number (and size) of records in the Lookup table is small relative to the number of mapping rows requiring lookup

Update Strategy Transformation Used to specify how each individual row will be used to update target tables (insert, update, delete, reject)
Connected
Ports
All input / output
Specify the Update Strategy Expression
Usage
Updating Slowly Changing Dimensions
IIF or DECODE logic determines how to handle the record

Router Transformation Rows sent to multiple filter conditions
Connected
Ports
All input/output
Specify filter conditions for each Group
Usage
Link source data in one pass to multiple filter conditions

Router Groups
Input group (always one)
User-defined groups
Each group has one condition
ALL group conditions are evaluated for EACH row
One row can pass multiple conditions
Unlinked Group outputs are ignored
Default group (always one) can capture rows that fail all Group conditions

Mappings
By the end of this section you will be familiar with:
Mapping components
Source Qualifier transformation
Mapplets
Mapping validation
Data flow rules
System Variables
Mapping Parameters and Variables

Mapplet Advantages
Useful for repetitive tasks / logic
Represents a set of transformations
Mapplets are reusable
Use an ‘instance’ of a Mapplet in a Mapping
Changes to a Mapplet are inherited by all instances
Server expands the Mapplet at runtime

Unsupported Transformations
Use any transformation in a Mapplet except:
XML Source definitions
COBOL Source definitions
Normalizer
Pre- and Post-Session stored procedures
Target definitions
Other Mapplets

Object Sharing
Reuse existing objects
Enforces consistency
Decreases development time
Share objects by using copies and shortcuts
Required security settings for sharing objects:
Repository Privilege: Use Designer
Originating Folder Permission: Read
Destination Folder Permissions: Read/Write
Created from a shared folder Copy from shared or unshared folder Preserves space Duplicates space Dynamically reflects changes to original object Changes to original object not captured Link to an object in another folder Copy object to another folder SHORTCUT COPY

Data Flow Rules
Each Source Qualifier starts a single data stream (a dataflow)
Transformations can send rows to more than one transformation (split one data flow into multiple pipelines)
Two or more data flows can meet together -- if (and only if) they originate from a common active transformation
Cannot add an active transformation into the mix
Example holds true with Normalizer in lieu of Source Qualifier. Exceptions are: Mapplet Input and Joiner transformations DISALLOWED T T Active ALLOWED T Passive T

Mapping Validation
Mappings must:
Be valid for a Session to run
Be end-to-end complete and contain valid expressions
Pass all data flow rules
Mappings are always validated when saved; can be validated without being saved
Output Window will always display reason for invalidity

Workflows
By the end of this section, you will be familiar with:
The Workflow Manager GUI interface
Workflow Schedules
Setting up Server Connections
Relational, FTP and External Loader
Creating and configuring Workflows
Workflow properties
Workflow components
Workflow Tasks

Workflow Manager Tools
Workflow Designer
Maps the execution order and dependencies of Sessions, Tasks and Worklets, for the Informatica Server
Task Developer
Create Session, Shell Command and Email tasks
Tasks created in the Task Developer are reusable
Worklet Designer
Creates objects that represent a set of tasks
Worklet objects are reusable

Workflow Scheduler Objects
Setup reusable schedules to associate with multiple Workflows
Used in Workflows and Session Tasks

Server Connections
Configure Server data access connections
Used in Session Tasks
Configure:
Relational
MQ Series
FTP
Custom
External Loader

Task Developer
Create basic Reusable “building blocks” – to use in any Workflow
Reusable Tasks
Session Set of instructions to execute Mapping logic
Command Specify OS shell / script command(s) to run
during the Workflow
Email Send email at any point in the Workflow
Session Command Email

Session Tasks
After this section, you will be familiar with:
How to create and configure Session Tasks
Session Task properties
Transformation property overrides
Reusable vs. non-reusable Sessions
Session partitions

Session Task
Created to execute the logic of a mapping (one mapping only)
Session Tasks can be created in the Task Developer (reusable) or Workflow Developer (Workflow-specific)
Steps to create a Session Task
Select the Session button from the Task Toolbar or
Select menu Tasks | Create
Session Task Bar Icon

Session Task
Steps to create a Session Task (continued)
Double click on the Session object
Valid Mappings are displayed in the dialog box
Session Task tabs
General
Properties
Config Object
Sources
Targets
Components
Transformation
Partitions
Metadata Extensions

Session Task
Server instructions to runs the logic of ONE specific Mapping
e.g. - source and target data location specifications, memory allocation, optional Mapping overrides, scheduling, processing and load instructions
Becomes a component of a Workflow (or Worklet)
If configured in the Task Developer, the Session Task is reusable (optional)

Monitor Workflows
By the end of this section you will be familiar with:
The Workflow Monitor GUI interface
Monitoring views
Server monitoring modes
Filtering displayed items
Actions initiated from the Workflow Monitor
Truncating Monitor Logs

Monitor Workflows
The Workflow Monitor is the tool for monitoring Workflows and Tasks
Review details about a Workflow or Task in two views
Gantt Chart view
Task view
Gantt Chart view Task view

Monitoring Workflows
Perform operations in the Workflow Monitor
Restart -- restart a Task, Workflow or Worklet
Stop -- stop a Task, Workflow, or Worklet
Abort -- abort a Task, Workflow, or Worklet
Resume -- resume a suspended Workflow after a failed Task is corrected
View Session and Workflow logs
Abort has a 60 second timeout
If the Server has not completed processing and committing data during the timeout period, the threads and processes associated with the Session are killed
Stopping a Session Task means the Server stops reading data

Parameters and Variables
By the end of this section you will understand:
System Variables
Creating Parameters and Variables
Features and advantages
Establishing values for Parameters and Variables

System Variables SESSSTARTTIME $$$SessStartTime
Returns the system date value as a string. Uses system clock on machine hosting Informatica Server
format of the string is database type dependent
Used in SQL override
Has a constant value
Returns the system date value on the Informatica Server
Used with any function that accepts transformation date/time data types
Not to be used in a SQL override
Has a constant value
SYSDATE
Provides current datetime on the Informatica Server machine
Not a static value

Apply to all transformations within one Mapping
Represent declared values
Variables can change in value during run-time
Parameters remain constant during run-time
Provide increased development flexibility
Defined in Mapping menu
Format is $$ VariableName or $$ ParameterName

Sample declarations
Declare Variables and Parameters in the Designer Mappings menu Set the appropriate aggregation type Set optional Initial Value User-defined names

Mapping Parameters and Variables Apply Parameter / Variable in formula

Functions to Set Mapping Variables
SetCountVariable -- Counts the number of evaluated rows and increments or decrements a mapping variable for each row
SetMaxVariable -- Evaluates the value of a mapping variable to the higher of two values
SetMinVariable -- Evaluates the value of a mapping variable to the lower of two values
SetVariable -- Sets the value of a mapping variable to a specified value

Error Handling (Row Level)
Reject files on Server store data rejected by the Writer and/or target database
Conditions causing data to be rejected include:
Target database constraint violations, out-of-space errors, log space errors, null values not accepted
Data-driven records, containing value ‘3’ or DD_REJECT
Target table properties ‘reject truncated/overflowed rows’
Data, Overflow, Null or Truncated Column Level Indicators INSERT 0,D,1313,D,Regulator System,D,Air Regulators,D,250.00,D,150.00,D UPDATE 1,D,1314,D,Second Stage Regulator,D,Air Regulators,D,365.00,D,265.00,D DELETE 2,D,1390,D,First Stage Regulator,D,Air Regulators,D,170.00,D,70.00,D REJECT 3,D,2341,D,Depth/Pressure Gauge,D,Small Instruments,D,105.00,D,5.00,D Transformation errors are written to the Session log file, not to the .bad file

Thank you !

http://free-informatica-tutorials.blogspot.com	36795
http://informatica-tutorials.blogspot.com	16632
http://free-informatica-tutorials.blogspot.in	5546
http://informatica-tutorials.blogspot.in	3935
http://www.free-informatica-tutorials.blogspot.in	2693
http://www.free-informatica-tutorials.blogspot.com	2222
http://mimishandshuna.com	1572
http://www.slideshare.net	496
http://manthiramoorthy.blogspot.com	385
http://translate.googleusercontent.com	227
http://free-informatica-tutorials.blogspot.ca	165
http://informatica-tutorials.blogspot.ca	127
http://free-informatica-tutorials.blogspot.co.uk	117
http://manthiramoorthy.blogspot.in	116
http://www.free-informatica-tutorials.blogspot.co.uk	92
http://informatica-tutorials.blogspot.co.uk	86
http://www.free-informatica-tutorials.blogspot.ca	75
http://free-informatica-tutorials.blogspot.com.au	71
http://free-informatica-tutorials.blogspot.fr	49
http://informatica-tutorials.blogspot.com.au	47
http://informatica-tutorials.blogspot.fr	39
http://www.free-informatica-tutorials.blogspot.de	35
http://free-informatica-tutorials.blogspot.de	35
http://free-informatica-tutorials.blogspot.com.es	35
http://www.free-informatica-tutorials.blogspot.com.au	34
http://informaticadocs.blogspot.com	32
http://webcache.googleusercontent.com	31
http://g33kyt.blogspot.com	31
http://free-informatica-tutorials.blogspot.com.br	28
http://free-informatica-tutorials.blogspot.it	28
http://free-informatica-tutorials.blogspot.mx	24
http://informaticatutoring.blogspot.com	23
http://www.free-informatica-tutorials.blogspot.com.es	21
http://informatica-tutorials.blogspot.com.br	20
http://libraries.ge.com	19
http://elblogdewafaa.blogspot.com	19
http://www.free-informatica-tutorials.blogspot.it	18
http://www.free-informatica-tutorials.blogspot.com.br	18
http://www.free-informatica-tutorials.blogspot.fr	18
http://informatica-tutorials.blogspot.it	14
http://feeds.feedburner.com	12
http://informatica-tutorials.blogspot.de	12
http://www.free-informatica-tutorials.blogspot.mx	11
http://dmaguina25.blogspot.com	11
http://manthiramoorthy.blogspot.ca	10
http://jdinformatic.blogspot.com	10
http://free-informatica-tutorials.blogspot.pt	10
http://free-informatica-tutorials.blogspot.se	10
http://informatica-tutorials.blogspot.com.es	9
http://informatica-tutorials.blogspot.mx	8
http://www.google.com	7

Informatica Training

by dwiner on Jan 28, 2008

Accessibility

Categories

Tags

Upload Details

Usage Rights

103 Embeds 72,219

Statistics

Informatica Training — Presentation Transcript