SlideShare a Scribd company logo
1 of 4
Download to read offline
Something about DataStage, DataStage Administration, Job Designing,Developing, DataStage troubleshooting, DataStage Installation & Configuration, ETL, DataWareHousing, DB2,
Teradata, Oracle and Scripting.
Nuts & Bolts of DataStage
Home Interview Questions DataStage Scenarios Series Posts E­Books About Me !!
Thursday, April 24, 2014
DataStage Scenario Problem ­­>  DataStage Scenario ­ Problem5
  
Solution Design :
a) Job Design :
Below is the design which can achieve the output as we needed. Here, we are reading seq file as a input, then data is passing through a Sort and
Transformer stage to achieve the output. 
b) Sort Stage Properties
In Sort stage, we will sort the data based on column “Char” in ascending order.
DataStage Scenario ­ Design5 ­ job1
Total Pageviews
1 4 5 4 6 1 6
Search
Try Me
DataSet in DataStage
Issuing commands to a Queue Manager (runmqsc)
Hash Files in DataStage
XMeta DB : Datastage Repository
InfoSphere DataStage Jobstatus returned Codes from
dsjob
Conductor Node in Datastage
Schema File in Datastage
Sort stage to remove duplicate
14 Good design tips in Datastage
Datastage Coding Checklist
Must Reads
3   More    Next Blog» Create Blog   Sign In
 
c) Transformer Stage Properties
Here, we took 2 stage variable : StageVar, StageVar1, StageVar2
and their derivations are ­
StageVar  = If StageVar1=DSLink6.Char Then StageVar+1 Else1
StageVar1 = DSLink6.Char
Create a new column in output which contains the Occurrence of characters and assigned the StageVar.
Occurrence = StageVar
 
Get daily dose of Tech Food
Email address... Submit
DataStage4You
111 have us in circles View all
Follow
tech foodies
▼  2014 (103)
►  October (7)
►  September (9)
►  August (5)
►  July (12)
►  June (10)
►  May (13)
▼  April (10)
How to Create Custom Stages in
Datastage Parallel ...
Performance Tunings in DataStage
SQL Best Practices ­ Part1
DataStage Scenario ­ Design5 ­ job1
DataStage Scenario ­ Design4 ­ job1
DataStage Scenario ­ Design3 ­ job1
Some more design tips for DataStage Job
Developmen...
Setting up "CRON" Jobs in Nix
DataStage Naming Standard
DataStage error upon login: DSR.ADMIN:
Failed to a...
Blog Archive
e) OutPut File 
In Output file, We will use the in­line sorting to sort the data on "Occurrence" column in ascending order.
no, char, occurrence
1,a,1
3,a,2
5,a,3
6,a,4
8,a,5
2,b,1
4,b,2
7,b,3
For More ­­­> VISIT THIS LINK
By ETL DataStage at 08:05  0 Comments
Labels: Data, DataSet, DataStage, design, develop, duplicate, input, output, problem, sort, stages, transformer
►  March (9)
►  February (16)
►  January (12)
►  2013 (167)
►  2012 (175)
►  2011 (8)
Administration 
application  authorities
client  Code  column
commands  Concept
Configuration 
create  Data  database  DataSet
DataStage  DataWareHouse  DB2  DBMS
debug  delete  design  develop
difference  director
Documentation  dsenv  dsjob  DSRPC 
environment Errors  ETL 
file 
function 
Information input  install  Interview 
Job  keys  Link Linux list 
Logging Logical  logs lookup 
managers  message  queue
Metadata Model  MQ  names
Optimizing  Oracle 
output  Parallel  parameter  partition
performance  Physical 
port  problem  process 
Project  Putty Questions 
remove 
routine  rows 
scenario  Schema  Script 
Seq  File  sequence  Server  Service  Setting
Shell  shell  scripting 
sort source  SQL  stages
Tags Cloud
&PH& 421  advantage Agents aggregator
Answers  architecture ASB attribute 
backup basic binary block books Buffer certification  change
channel  checkpoint  cleanup  clear 
Column  Generator  compiler 
Conceptual  conductor  container  copy
counter  Crontab 
deadlock  deploy 
dimension  Dimensional 
DSparam  dump
duplicate encrypt engine  exception
execution export fact factless  FAQ  FileSet filter free ftp
fun  fundamentals granularity  Guest hadoop handling
hash  head hide horizontal  Host huge  hyperlink  import  increase
index  issue  istool Java
jdbc  join  leaders  listener load
local locks  Login  macro mail
maintenance  memory  merge 
modify Monitor  MQSC multiple 
NLS  node  notes  notification  odbc  odbc.ini  operator
orchadmin  ORLogging  orphan  OS  osh
package  Parallelism 
password peek  Perl phantom  pivot
player  Practices  profile
programming  purge  read registry
reject  release  report  Resource  Restart  Roles
row  generator  RTLogging  run  sample  SCD
scheduler  score  Scratch  section
session 
Share  shortcuts  show  slowly
snowflake solution  space  SSH 
Newer Post Older PostHome
Subscribe to: Post Comments (Atom)
0 Comments DataStage4You  Login
Sort by Best Share ⤤
Start the discussion…
Be the first to comment.
Subscribe✉ Add Disqus to your sited Privacy
Favorite ★
Start  Stop 
surrogate  table  target  teradata
tips  tool  transformer 
Troubleshoot  Tutorial  Unix User
Utility  UV  variables 
warnings  WAS  websphere
windows  XMETA 
Standards  Star  statistics  status  storage
switch system  tail  temporary 
time  trace  transformation  trigger
tuning  type unique 
uvodbc.config  version  videos  view
Vincent  McBurney  Virtual 
write Write Range Map  xml z/OS
The postings on this site are my own and don't necessarily represent IBM's or other companies positions, strategies or opinions. All content provided on this blog is for informational purposes only. The owner of this
blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. The owner will not be liable for any errors or omissions in this information
nor for the availability of this information. The owner will not be liable for any losses, injuries, or damages from the display or use of his information. //­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­­
Disclaimer
Did you find this Blog helpful ?? Let me know @ www.facebook.com/datastage4you. Ethereal template. Powered by Blogger.

More Related Content

Viewers also liked

Oracle PL/SQL online training | PL/SQL online Training
Oracle PL/SQL online training | PL/SQL online TrainingOracle PL/SQL online training | PL/SQL online Training
Oracle PL/SQL online training | PL/SQL online Trainingsuresh
 
Data stage scenario design 2 - job1
Data stage scenario   design 2 - job1Data stage scenario   design 2 - job1
Data stage scenario design 2 - job1Naresh Bala
 
Unix interview questions
Unix interview questionsUnix interview questions
Unix interview questionsKalyan Hadoop
 
Sql interview question part 2
Sql interview question part 2Sql interview question part 2
Sql interview question part 2kaashiv1
 
Sql interview question part 4
Sql interview question part 4Sql interview question part 4
Sql interview question part 4kaashiv1
 
Shell Scripting With Arguments
Shell Scripting With ArgumentsShell Scripting With Arguments
Shell Scripting With ArgumentsTechronology Inc.
 
Data stage scenario design6 - job1
Data stage scenario   design6 - job1Data stage scenario   design6 - job1
Data stage scenario design6 - job1Naresh Bala
 
Data stage interview questions and answers|DataStage FAQS
Data stage interview questions and answers|DataStage FAQSData stage interview questions and answers|DataStage FAQS
Data stage interview questions and answers|DataStage FAQSBigClasses.com
 
Datastage free tutorial
Datastage free tutorialDatastage free tutorial
Datastage free tutorialtekslate1
 
Datastage real time scenario
Datastage real time scenarioDatastage real time scenario
Datastage real time scenarioNaresh Bala
 
data stage-material
data stage-materialdata stage-material
data stage-materialRajesh Kv
 

Viewers also liked (14)

Oracle PL/SQL online training | PL/SQL online Training
Oracle PL/SQL online training | PL/SQL online TrainingOracle PL/SQL online training | PL/SQL online Training
Oracle PL/SQL online training | PL/SQL online Training
 
Data stage scenario design 2 - job1
Data stage scenario   design 2 - job1Data stage scenario   design 2 - job1
Data stage scenario design 2 - job1
 
Unix interview questions
Unix interview questionsUnix interview questions
Unix interview questions
 
Sql interview question part 2
Sql interview question part 2Sql interview question part 2
Sql interview question part 2
 
Sql interview question part 4
Sql interview question part 4Sql interview question part 4
Sql interview question part 4
 
Shell Scripting With Arguments
Shell Scripting With ArgumentsShell Scripting With Arguments
Shell Scripting With Arguments
 
Data stage scenario design6 - job1
Data stage scenario   design6 - job1Data stage scenario   design6 - job1
Data stage scenario design6 - job1
 
SQL Differences SQL Interview Questions
SQL Differences  SQL Interview QuestionsSQL Differences  SQL Interview Questions
SQL Differences SQL Interview Questions
 
Data stage interview questions and answers|DataStage FAQS
Data stage interview questions and answers|DataStage FAQSData stage interview questions and answers|DataStage FAQS
Data stage interview questions and answers|DataStage FAQS
 
Datastage free tutorial
Datastage free tutorialDatastage free tutorial
Datastage free tutorial
 
Datastage real time scenario
Datastage real time scenarioDatastage real time scenario
Datastage real time scenario
 
Ibm info sphere datastage tutorial part 1 architecture examples
Ibm info sphere datastage tutorial part 1  architecture examplesIbm info sphere datastage tutorial part 1  architecture examples
Ibm info sphere datastage tutorial part 1 architecture examples
 
data stage-material
data stage-materialdata stage-material
data stage-material
 
Intorduction to Datapower
Intorduction to DatapowerIntorduction to Datapower
Intorduction to Datapower
 

Similar to Data stage scenario design5 - job1

Testing database content with DBUnit. My experience.
Testing database content with DBUnit. My experience.Testing database content with DBUnit. My experience.
Testing database content with DBUnit. My experience.Serhii Kartashov
 
Frustration-Reduced PySpark: Data engineering with DataFrames
Frustration-Reduced PySpark: Data engineering with DataFramesFrustration-Reduced PySpark: Data engineering with DataFrames
Frustration-Reduced PySpark: Data engineering with DataFramesIlya Ganelin
 
Spark Based Distributed Deep Learning Framework For Big Data Applications
Spark Based Distributed Deep Learning Framework For Big Data Applications Spark Based Distributed Deep Learning Framework For Big Data Applications
Spark Based Distributed Deep Learning Framework For Big Data Applications Humoyun Ahmedov
 
Best Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache SparkBest Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache SparkDatabricks
 
New Developments in Spark
New Developments in SparkNew Developments in Spark
New Developments in SparkDatabricks
 
Plmce2012 scaling pinterest
Plmce2012 scaling pinterestPlmce2012 scaling pinterest
Plmce2012 scaling pinterestMohit Jain
 
High Performance Jdbc
High Performance JdbcHigh Performance Jdbc
High Performance JdbcSam Pattsin
 
Spark SQL In Depth www.syedacademy.com
Spark SQL In Depth www.syedacademy.comSpark SQL In Depth www.syedacademy.com
Spark SQL In Depth www.syedacademy.comSyed Hadoop
 
Rapidly Building Data Driven Web Pages with Dynamic ADO.NET
Rapidly Building Data Driven Web Pages with Dynamic ADO.NETRapidly Building Data Driven Web Pages with Dynamic ADO.NET
Rapidly Building Data Driven Web Pages with Dynamic ADO.NETgoodfriday
 
ETL to ML: Use Apache Spark as an end to end tool for Advanced Analytics
ETL to ML: Use Apache Spark as an end to end tool for Advanced AnalyticsETL to ML: Use Apache Spark as an end to end tool for Advanced Analytics
ETL to ML: Use Apache Spark as an end to end tool for Advanced AnalyticsMiklos Christine
 
SAS Online Training Institute in Hyderabad - C-Point
SAS Online Training Institute in Hyderabad - C-PointSAS Online Training Institute in Hyderabad - C-Point
SAS Online Training Institute in Hyderabad - C-Pointcpointss
 
Jump Start into Apache® Spark™ and Databricks
Jump Start into Apache® Spark™ and DatabricksJump Start into Apache® Spark™ and Databricks
Jump Start into Apache® Spark™ and DatabricksDatabricks
 
ETL and pivoting in spark
ETL and pivoting in sparkETL and pivoting in spark
ETL and pivoting in sparkSubhasish Guha
 
ETL and pivoting in spark
ETL and pivoting in sparkETL and pivoting in spark
ETL and pivoting in sparkSubhasish Guha
 
Karen's Favourite Features of SQL Server 2016
Karen's Favourite Features of  SQL Server 2016Karen's Favourite Features of  SQL Server 2016
Karen's Favourite Features of SQL Server 2016Karen Lopez
 
GIDS 2016 Understanding and Building No SQLs
GIDS 2016 Understanding and Building No SQLsGIDS 2016 Understanding and Building No SQLs
GIDS 2016 Understanding and Building No SQLstechmaddy
 
Spark ETL Techniques - Creating An Optimal Fantasy Baseball Roster
Spark ETL Techniques - Creating An Optimal Fantasy Baseball RosterSpark ETL Techniques - Creating An Optimal Fantasy Baseball Roster
Spark ETL Techniques - Creating An Optimal Fantasy Baseball RosterDon Drake
 

Similar to Data stage scenario design5 - job1 (20)

Testing database content with DBUnit. My experience.
Testing database content with DBUnit. My experience.Testing database content with DBUnit. My experience.
Testing database content with DBUnit. My experience.
 
Frustration-Reduced PySpark: Data engineering with DataFrames
Frustration-Reduced PySpark: Data engineering with DataFramesFrustration-Reduced PySpark: Data engineering with DataFrames
Frustration-Reduced PySpark: Data engineering with DataFrames
 
Spark Based Distributed Deep Learning Framework For Big Data Applications
Spark Based Distributed Deep Learning Framework For Big Data Applications Spark Based Distributed Deep Learning Framework For Big Data Applications
Spark Based Distributed Deep Learning Framework For Big Data Applications
 
Best Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache SparkBest Practices for Building and Deploying Data Pipelines in Apache Spark
Best Practices for Building and Deploying Data Pipelines in Apache Spark
 
My Master's Thesis
My Master's ThesisMy Master's Thesis
My Master's Thesis
 
New Developments in Spark
New Developments in SparkNew Developments in Spark
New Developments in Spark
 
Plmce2012 scaling pinterest
Plmce2012 scaling pinterestPlmce2012 scaling pinterest
Plmce2012 scaling pinterest
 
High Performance Jdbc
High Performance JdbcHigh Performance Jdbc
High Performance Jdbc
 
Spark SQL In Depth www.syedacademy.com
Spark SQL In Depth www.syedacademy.comSpark SQL In Depth www.syedacademy.com
Spark SQL In Depth www.syedacademy.com
 
4 jdbc step1
4 jdbc step14 jdbc step1
4 jdbc step1
 
Rapidly Building Data Driven Web Pages with Dynamic ADO.NET
Rapidly Building Data Driven Web Pages with Dynamic ADO.NETRapidly Building Data Driven Web Pages with Dynamic ADO.NET
Rapidly Building Data Driven Web Pages with Dynamic ADO.NET
 
ETL to ML: Use Apache Spark as an end to end tool for Advanced Analytics
ETL to ML: Use Apache Spark as an end to end tool for Advanced AnalyticsETL to ML: Use Apache Spark as an end to end tool for Advanced Analytics
ETL to ML: Use Apache Spark as an end to end tool for Advanced Analytics
 
SAS Online Training Institute in Hyderabad - C-Point
SAS Online Training Institute in Hyderabad - C-PointSAS Online Training Institute in Hyderabad - C-Point
SAS Online Training Institute in Hyderabad - C-Point
 
Jump Start into Apache® Spark™ and Databricks
Jump Start into Apache® Spark™ and DatabricksJump Start into Apache® Spark™ and Databricks
Jump Start into Apache® Spark™ and Databricks
 
ETL and pivoting in spark
ETL and pivoting in sparkETL and pivoting in spark
ETL and pivoting in spark
 
ETL and pivoting in spark
ETL and pivoting in sparkETL and pivoting in spark
ETL and pivoting in spark
 
Datastage trining
Datastage triningDatastage trining
Datastage trining
 
Karen's Favourite Features of SQL Server 2016
Karen's Favourite Features of  SQL Server 2016Karen's Favourite Features of  SQL Server 2016
Karen's Favourite Features of SQL Server 2016
 
GIDS 2016 Understanding and Building No SQLs
GIDS 2016 Understanding and Building No SQLsGIDS 2016 Understanding and Building No SQLs
GIDS 2016 Understanding and Building No SQLs
 
Spark ETL Techniques - Creating An Optimal Fantasy Baseball Roster
Spark ETL Techniques - Creating An Optimal Fantasy Baseball RosterSpark ETL Techniques - Creating An Optimal Fantasy Baseball Roster
Spark ETL Techniques - Creating An Optimal Fantasy Baseball Roster
 

Data stage scenario design5 - job1