Teradata Utilities: FastExportReprinted for KV Satish Kumar, IBMkvskumar@in.ibm.comReprinted with permission as a subscrip...
Table of ContentsChapter 6: FastExport.......................................................................................
Chapter 6: FastExport"An invasion of armies can be resisted, but not an idea whose time has come."- Victor HugoWhy it is C...
FastExport Fundamentals#1: FastExport EXPORTS data from Teradata. The reason they call it FastExport is because ittakes da...
The Mainframe (Channel Attached) environment supports the following Operating Systems:MVS•VM•Maximum of 15 LoadsThe Terada...
ACCEPT Allows the value of utility variables to be accepted directly from a file or fromenvironmental variables.DATEFORM S...
FastExport Supported SQL CommandsFastExport accepts the following Teradata SQL statements. Each has been placed in alphabe...
Exports the data to the specified file or OUTMOD routine•Logs off of Teradata•/* Created by CoffingDW *//* Setup the Fast ...
/* ---------------------------------------------------*//*@(#) FASTEXPORT SCRIPT *//*@(#) Version 1.1 *//*@(#) Created by ...
Both modes return data in a client internal format with variable-length records. Each individualrecord has a value for all...
Last_name (CHAR(20)),First_name (CHAR(14)),Class_code (CHAR(2)),Grade_Pt (CHAR(8))FROM SQL_CLASS.Student_Table;/* Finish t...
Upcoming SlideShare
Loading in …5
×

Teradata utilities-fast export

8,766 views

Published on

Published in: Technology
  • Be the first to comment

Teradata utilities-fast export

  1. 1. Teradata Utilities: FastExportReprinted for KV Satish Kumar, IBMkvskumar@in.ibm.comReprinted with permission as a subscription benefit of Books24x7,http://www.books24x7.com/
  2. 2. Table of ContentsChapter 6: FastExport.....................................................................................................................1Why it is Called "FAST" Export..............................................................................................1How FastExport Works....................................................................................................1FastExport Fundamentals......................................................................................................2FastExport Supported Operating Systems.......................................................................2Maximum of 15 Loads............................................................................................................3FastExport Support and Task Commands.............................................................................3Support Environment Commands..........................................................................................3Task Commands....................................................................................................................4FastExport Supported SQL Commands................................................................................5SQL Commands...............................................................................................................5A FastExport in its Simplest Form..........................................................................................5Sample FastExport Script......................................................................................................6FastExport Modes and Formats.............................................................................................7Modes..............................................................................................................................7Formats............................................................................................................................8A FastExport Script using Binary Mode.................................................................................8i
  3. 3. Chapter 6: FastExport"An invasion of armies can be resisted, but not an idea whose time has come."- Victor HugoWhy it is Called "FAST" ExportFastExport is known for its lightning speed when it comes to exporting vast amounts of data fromTeradata and transferring the data into flat files on either a mainframe or network-attachedcomputer. In addition, FastExport has the ability to use OUTMOD routines, which provide the userthe capability to write, select, validate, and preprocess the exported data. Part of this speed isachieved because FastExport takes full advantage of Teradatas parallelism.In this book, we have already discovered how BTEQ can be utilized to export data from Teradata ina variety of formats. As the demand increases to store data, the ever-growing requirement for toolsto export massive amounts of data also increases.This is the reason why FastExport (FEXP) is brilliant by design. A good rule of thumb is that if youhave more than half a million rows of data to export to either a flat file format or with NULLindicators, then FastExport is the best choice to accomplish this task.Keep in mind that FastExport is designed as a one-way utility — that is, the sole purpose ofFastExport is to move data out of Teradata. It does this by harnessing the parallelism that Teradataprovides.FastExport is extremely attractive for exporting data because it takes full advantage of multiplesessions, which leverages Teradata parallelism. By default, Fastexport will take up 4 sessions thatwill be running on the local computer. FastExport can also export from multiple tables during asingle operation. In addition, FastExport utilizes the Support Environment, which provides a jobrestart capability from a checkpoint if an error occurs during the process of executing an export job.How FastExport WorksWhen FastExport is invoked, the utility logs onto the Teradata database and retrieves the rows thatare specified in the SELECT statement and puts them into SPOOL. From there, it must build blocksto send back to the client. In comparison, BTEQ starts sending rows immediately for storage into afile.If the output data is sorted, FastExport may be required to redistribute the selected data two timesacross the AMP processors in order to build the blocks in the correct sequence. Remember, a lot ofrows fit into a 64K block and both the rows and the blocks must be sequenced. While all of thisredistribution is occurring, BTEQ continues to send rows. FastExport is getting behind in theprocessing. However, when FastExport starts sending the rows back a block at a time, it quicklyovertakes and passes BTEQs row at time processing.The other advantage is that if BTEQ terminates abnormally, all of your rows (which are in SPOOL)are discarded. You must rerun the BTEQ script from the beginning. However, if FastExportterminates abnormally, all the selected rows are in worktables and it can continue sending themwhere it left off in a very smart and very fast manner!Also, if there is a requirement to manipulate the data before storing it on the computers hard drive,an OUTMOD routine can be written to modify the result set after it is sent back to the client on eitherthe mainframe or LAN. Just like the BASF commercial states, "We dont make the products you buy,we make the products you buy better". FastExport is designed off the same premise, it does notmake the SQL SELECT statement faster, but it does take the SQL SELECT statement andprocesses the request with lighting fast parallel processing!Reprinted for ibmkvskumar@in.ibm.com, IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited
  4. 4. FastExport Fundamentals#1: FastExport EXPORTS data from Teradata. The reason they call it FastExport is because ittakes data off of Teradata (Exports Data). FastExport does not import data into Teradata.Additionally, like BTEQ it can output multiple files in a single run.#2: FastExport only supports the SELECT statement. The only DML statement that FastExportunderstands is SELECT. You SELECT the data you want exported and FastExport will take care ofthe rest.#3: Choose FastExport over BTEQ when Exporting Data of more than half a million+ rows.When a large amount of data is being exported, FastExport is recommended over BTEQ Export.The only drawback is the total number of FastLoads, FastExports, and MultiLoads that can run atthe same time, which is limited to 15. BTEQ Export does not have this restriction. Of course,FastExport will work with less data, but the speed may not be much faster than BTEQ.#4: FastExport supports multiple SELECT statements and multiple tables in a single run. Youcan have multiple SELECT statements with FastExport and each SELECT can join information upto 64 tables.#5: FastExport supports conditional logic, conditional expressions, arithmetic calculations,and data conversions. FastExport is flexible and supports the above conditions, calculations, andconversions.#6: FastExport does NOT support error files or error limits. FastExport does not recordparticular error types in a table. The FastExport utility will terminate after a certain number of errorshave been encountered.#7: FastExport supports user-written routines INMODs and OUTMODs. FastExport allows youwrite INMOD and OUTMOD routines so you can select, validate and preprocess the exported dataFastExport Supported Operating SystemsThe FastExport utility is supported on either the mainframe or on LAN. The information belowillustrates which operating systems are supported for each environment:The LAN environment supports the following Operating Systems:UNIX MP-RAS•Windows 2000•Windows 95/98/XP•Windows NT/2000•UNIX HP-UX•AIX•Solaris SPARC•Solaris Intel•Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 2Reprinted for ibmkvskumar@in.ibm.com, IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited
  5. 5. The Mainframe (Channel Attached) environment supports the following Operating Systems:MVS•VM•Maximum of 15 LoadsThe Teradata RDBMS will only support a maximum of 15 simultaneous FastLoad, MultiLoad, orFastExport utility jobs. This maximum value is determined and configured in the DBS Controlrecord. This value can be set from 0 to 15. When Teradata is initially installed, this value is set at 5.The reason for this limitation is that FastLoad, MultiLoad, and FastExport all use large blocks totransfer data. If more then 15 simultaneous jobs were supported, a saturation point could bereached on the availability of resources. In this case, Teradata does an excellent job of protectingsystem resources by queuing up additional FastLoad, MultiLoad, and FastExport jobs that areattempting to connect.For example, if the maximum number of utilities on the Teradata system is reached and another jobattempts to run, that job will not start. This limitation should be viewed as a safety control feature. Atip for remembering how the load limit applies is this, "If the name of the load utility contains eitherthe word "Fast" or the word "Load", then there can be only a total of fifteen of them running at anyone time".BTEQ does not have this load limitation. FastExport is clearly the better choice when exportingdata. However, if two many load jobs are running. BTEQ is an alternate choice for exporting data.FastExport Support and Task CommandsFastExport accepts both FastExport commands and a subset of SQL statements. The FastExportcommands can be broken down into support and task activities. The table below highlights the keyFastExport commands and their definitions. These commands provide flexibility and control duringthe export process.Support Environment Commands(see Support Environment chapter for details)Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 3Reprinted for ibmkvskumar@in.ibm.com, IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited
  6. 6. ACCEPT Allows the value of utility variables to be accepted directly from a file or fromenvironmental variables.DATEFORM Specifies the style of the DATE data types for FastExport.DISPLAY Writes messages to the specific location.ELSE Used in conjunction with the IF statement. ELSE commands and statements willexecute when a proceeding IF condition is false.ENDIF Used in conjunction with the IF or ELSE statements. Delimits the commands thatwere subject to previous IF or ELSE conditions.IF Introduces a conditional expression. If true then execution of subsequentcommands will happen.LOGOFF Disconnects all FastExport active sessions and terminates FastExport.LOGON LOGON command or string used to connect sessions established through theFastExport utility.LOGTABLE FastExport utilizes this to specify a restart log table. The purpose is for FastExportcheckpoint information.ROUTEMESSAGESWill route FastExport messages to an alternate destination.RUN FILE Used to point to a file that FastExport is to use as standard input. This will Invokethe specified external file as the current source of utility and Teradata SQLcommands.SET Assigns a data type and value to a variable.SYSTEM Suspends the FastExport utility temporarily and executes any valid local operatingsystem command before returning.Figure 3-1Task CommandsBEGIN EXPORT Begins the export task and sets the specifications for the number of sessionswith Teradata.END EXPORT Ends the export task and initiates processing by Teradata.EXPORT Provides two things which are:The client destination and file format specifications for the export dataretrieved from Teradata•A generated MultiLoad script file that can be used later to reload theexport data back into Teradata•FIELD Constitutes a field in the input record section that provides data values for theSELECT statement.FILLER Specifies a field in the input record that will not be sent to Teradata forprocessing. It is part of the input record to provide data values for the SELECTstatement.IMPORT Defines the file that provides the USING data values for the SELECT.LAYOUT Specifies the data layout for a file. It contains a sequence of FIELD andFILLER commands. This is used to describe the import file that can optionallyprovide data values for the SELECT.Figure 3-2Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 4Reprinted for ibmkvskumar@in.ibm.com, IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited
  7. 7. FastExport Supported SQL CommandsFastExport accepts the following Teradata SQL statements. Each has been placed in alphabeticorder for your convenience.SQL CommandsALTER TABLE Change a column or table options of a table.CHECKPOINT Add a checkpoint entry in the journal table.COLLECT STATISTICS Collect statistics for one or more columns or indexes in atable.COMMENT Store or retrieve a comment string for a particular object.CREATE DATABASE Creates a new database.CREATE TABLE Creates a new table.CREATE VIEW Creates a new view.CREATE MACRO Creates a new macro.DATABASE Specify a default database for the session.DELETE Delete rows from a table.DELETE DATABASE Removes all tables, views, macros, and stored proceduresfrom a database.DROP DATABASE Drops a database.GIVE Transfer ownership of a database or user to another user.GRANT Grant access privileges to an object.MODIFY DATABASE Change the options for a database.RENAME Change the name of a table, view, or macro.REPLACE MACRO Change a macro.REPLACE VIEW Change a view.REVOKE Revoke privileges to an object.SET SESSIONCOLLATIONOverride the collation specification during the currentsession.UPDATE Change a column value of an existing row or rows in atable.Figure 3-3A FastExport in its Simplest FormThe hobby of racecar driving can be extremely frustrating, challenging, and rewarding all at thesame time. I always remember my driving instructor coaching me during a practice session in a newcar around a road course racetrack. He said to me, "Before you can learn to run, you need to learnhow to walk." This same philosophy can be applied when working with FastExport. If FastExport isbroken into steps, then several things that appear to be complicated are really very simple. With thisbeing stated, FastExport can be broken into the following steps:Logging onto Teradata•Retrieves the rows you specify in your SELECT statement•Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 5Reprinted for ibmkvskumar@in.ibm.com, IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited
  8. 8. Exports the data to the specified file or OUTMOD routine•Logs off of Teradata•/* Created by CoffingDW *//* Setup the Fast Export Parameters */.LOGTABLE sql01.SWA_Log;Creates the logtable -Required.LOGON CDW/sql01,whynot;Logon to Teradata.BEGIN EXPORT SESSIONS 12;Begin the Export and set thenumber of sessions onTeradata.EXPORT OUTFILE Student.txtMODE RECORD FORMAT TEXT;Defines the output file name.In addition, specifies theoutput mode and format (LAN- ONLY)/* Get Data From the Student Table */SELECTStudent_ID (CHAR(11)),Last_name (CHAR(20)),First_name (CHAR(14)),Class_code (CHAR(2)),Grade_Pt (CHAR(9))FROM SQL_CLASS.Student_Table;The SELECT defines thecolumns used to create theexported file.NOTE: The selected columnsfor the export are beingconverted to character types.This will simplify theimporting process into adifferent database./* Finish the Export Job and Write to File */.END EXPORT;.LOGOFF;End the Export and logoffTeradata.Figure 3-4Sample FastExport ScriptNow that the first steps have been taken to understand FastExport, the next step is to journeyforward and review another example that shows builds upon what we have learned. In the scriptbelow, Teradata comment lines have been placed inside the script [/*. . . . */]. In addition, FastExportand SQL commands are written in upper case in order to highlight them. Another note is that thecolumn names are listed vertically. The recommendation is to place the comma separator in front ofthe following column. Coding this way makes reading or debugging the script easier to accomplish.Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 6Reprinted for ibmkvskumar@in.ibm.com, IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited
  9. 9. /* ---------------------------------------------------*//*@(#) FASTEXPORT SCRIPT *//*@(#) Version 1.1 *//*@(#) Created by CoffingDW *//* ---------------------------------------------------*/ALWAYSGOOD TOIDENTIFY THESCRIPT ANDAUTHOR INCOMMENTS/* Setup the Fast Export Parameters */.LOGTABLE SQL01.CDW_Log;.LOGON CDW/SQL01,whynot;CREATELOGTABLEAND LOGON;.BEGIN EXPORTSESSIONS 12;BEGINEXPORTSTATEMENT.SESSIONS 12;.EXPORT OUTFILE Join_Export.txtMODE RECORD FORMAT TEXT;DEFINES THEOUTPUT FILENAME. INADDITION,SPECIFIESTHE OUTPUTMODE ANDFORMAT(LAN- ONLY) MODERECORDFORMATTEXT;/* Join Data From the Employee and Department Table */SELECT EMP.Employee_No (CHAR(11)),EMP.First_Name (CHAR(14)),EMP.Last_Name (CHAR(20)),DEPT.Dept_No (CHAR(6)),DEPT.Dept_name (CHAR(20))FROM SQL_CLASS.Employee_table AS EMPINNER JOIN SQL_CLASS.Department_Table AS DEPTON EMP.Dept_No = DEPT.Dept_No ;THE SELECTPULLS DATAFROM TWOTABLES. IT ISGOOD TOQUALILYWHEN DOINGA TWO-TABLEJOIN./* Finish the Export Job and Write to File */.END EXPORT;.LOGOFF;END THE JOBAND LOGOFFTERADATA;Figure 3-5FastExport Modes and FormatsModesFastExport has two modes: RECORD or INDICATOR. In the mainframe world, only use RECORDmode. In the UNIX or LAN environment, INDICATOR mode is the default, but you can useRECORD mode if desired. The difference between the two modes is INDICATOR mode will set theindicator bits to 1 for column values containing NULLS.Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 7Reprinted for ibmkvskumar@in.ibm.com, IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited
  10. 10. Both modes return data in a client internal format with variable-length records. Each individualrecord has a value for all of the columns specified by the SELECT statement. All variable-lengthcolumns are preceded by a two-byte control value indicating the length of the column data. NULLcolumns have a value that is appropriate for the column data type. Remember, INDICATOR modewill set bit flags that identify the columns that have a null value.FormatsFastExport has many possible formats in the UNIX or LAN environment. The FORMAT statementspecifies the format for each record being exported which are:FASTLOAD•BINARY•TEXT•UNFORMAT•The default FORMAT is FASTLOAD in a UNIX or LAN environment.FASTLOAD format has a two-byte integer, followed by the data, followed by an end-of-recordmarker. It is called FASTLOAD because the data is exported in a format ready for FASTLOAD.BINARY format is a two-byte integer, followed by data.TEXT format is an arbitrary number of bytes followed by an end-of-record marker.UNFORMAT format is exactly as received from CLIv2 without any client modifications.A FastExport Script using Binary Mode/*-------------------------------------------------- *//*@(#)FASTEXPORT SCRIPT - SWA *//*@(#)Version 1.1 *//*@(#)Created by CoffingDW *//*-------------------------------------------------- */COMMENTS/* Setup the Fast Export Parameters */.LOGTABLE SQL01.SWA_LOG;.LOGON CDW/sql01,whynot;CREATE LOGTABLE ANDLOGON TO TERADATA;.BEGIN EXPORTSESSIONS 12;BEGIN EXPORTSTATEMENT;.EXPORT OUTFILE Cdw_import.txtFORMAT BINARY;NAME THE OUTPUT FILEAND SET THE FORMAT TOBINARY;/* Get Data From the Student Table */SELECTStudent_ID (CHAR(11)),GET THE DATA FROM THESTUDENT TABLE;Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 8Reprinted for ibmkvskumar@in.ibm.com, IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited
  11. 11. Last_name (CHAR(20)),First_name (CHAR(14)),Class_code (CHAR(2)),Grade_Pt (CHAR(8))FROM SQL_CLASS.Student_Table;/* Finish the Export Job and Write to File */.END EXPORT;.LOGOFF;END THE JOBFigure 3-6Teradata Utilities: BTEQ, FastLoad, MultiLoad, TPump, and FastExport, Second Edition 9Reprinted for ibmkvskumar@in.ibm.com, IBM Coffing Data Warehousing, Coffing Publishing (c) 2005, Copying Prohibited

×