Your SlideShare is downloading. ×
Data load utility
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Data load utility

571
views

Published on

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
571
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Data Load utility Contents 1. Customizing the Data Load utility to encrypt data 1.1. Creating a CSV file with user password information 1.2. Configuring the Data Load utility 1.3. Extending the generic table loader to encrypt user passwords 1.4. Verifying your customization 2. Customizing the Data Load utility for XML input 2.1. Creating an XML input file 2.2. Creating a custom data reader 2.3. Configuring the Data Load utility 2.4. Verifying your customization 2.5. Deploying your customized code 3. Loading data into custom tables 3.1. Create the custom tables 3.2. Creating an input file 3.3. Configuring the environment settings 3.4. Configuring the business object configuration file 3.5. Configuring the load order configuration file 3.6. Run the Data Load utility
  • 2. Data Load utility tutorials The Data Load utility tutorials introduce the various tasks that are related to customizing the Data Load utility for WebSphere Commerce applications. Tutorial Learning objectives Description Tutorial: Customizing the Data Load utility to encrypt data After completing this tutorial, you should be able to: Configure the Data Load utility. Extend the generic table loader. Encrypt data by using Java classes. This tutorial demonstrates how to customize the Data Load utility to load user and password information into the WebSphere Commerce database separately. Customizing the Data Load utility for XML input After completing this tutorial, you should be able to complete the following tasks. Create your XML data reader Configure the Data Load utility to use your data reader and mediator This tutorial demonstrates how to customize the data load utility to load data from an XML file. Tutorial: Loading data into custom tables with the Data Load utility After completing this tutorial, you should be able to complete the following tasks: Configure the Data Load utility to use the table object builder and table object mediator Configure the table object builder to load data into custom tables This tutorial demonstrates how to use a table-based business object builder and mediator to load data into extended or custom database tables.
  • 3. 1. Next > Tutorial: Customizing the Data Load utility to encrypt data This tutorial demonstrates how to customize the Data Load utility to load user and password information into the WebSphere Commerce database separately. For security reasons, the Data Load utility does not load user password information from CSV files into the WebSphere Commerce database by default. After user information is loaded into the database from a CSV file, the value for the user passwords is set to NULL. After user data is loaded to the WebSphere Commerce database, each user password is automatically reset the first time that a user attempts to logon to the store. The password is reset regardless of the password that was entered at logon time, since the password in the database is null. This automatic reset of the null password at logon time can be disabled by setting the following flag in the instance configuration file to false: MemberSubSystem/ResetPassword/resetNullPasswordEnabled This flag is enabled by default when the Foundation feature of WebSphere Commerce 7 Feature Pack 3 is enabled. An email is sent to the user with a temporary password that they can use to log on to the store and then change the password. This tutorial shows you how to handle user password information by customizing the Data Load utility to load user passwords separately from the users. Learning objectives After completing this tutorial, you should be able to: Configure the Data Load utility. Extend the generic table loader. Encrypt data by using Java classes. Time required Expect this tutorial to take 1 hour to complete. Skill level Intermediate Audience This tutorial is intended for WebSphere Commerce developers who want to extend the generic table loader to load customized data. System requirements Before you begin this tutorial, ensure that the following tasks are complete: Member data is loaded into the database. Password input (plain message text) is obtained from the external system by using data readers. Passwords are stored in a CSV file. Prerequisites To complete this tutorial, ensure that you have a basic understanding of the following concepts: Data Load utility architecture. Password encryption. Generic table loader with extension table or column handler. Next >
  • 4. < Previous | 1.1. Next > Creating a CSV file with user password information In this lesson, you create a new CSV file that contains the user password information to load into the WebSphere Commerce database. Note: This tutorial demonstrates how to write a custom column handler and encrypt a clear text password by using the WCPasswordEncrypter class. It assumes that customers obtain password input (plain message text) from the external system by using their own encryption algorithm for password encryption and decryption. If you do not want to define clear text password information in a CSV file, you can use an alternative method to define user password information: Export the user logon ID and encrypted password in the CSV file. Modify the PasswordHandler tutorial code to decrypt the encrypted user password before the WCPasswordEncrypter class is called to encrypt the user password information. Create a custom data reader to read the user logon ID and password directly from the external system. By reading the ID and password directly you can ensure that no clear text or encrypted passwords are saved in a CSV file. Use the custom data reader to decrypt the user password by using your own algorithm. By combining the custom data reader with the customized PasswordHandler, you can securely load the user password into the database. Sample data is provided in the WCDE_installdircomponentsfoundationsamplesDataloadMemberPerson directory. In this step, you load the sample customer data into the USERREG database table. Then, load the corresponding password information into the database table separately after the customization is complete. Procedure 1. Ensure that your WebSphere Commerce Server is stopped before you run the Data Load utility. For more information about stopping your server, see Starting and stopping WebSphere Commerce Test Server. 2. To load the sample customer data into the USERREG database table, enter the following command: ./dataload.sh ../components/foundation/samples/DataLoad/Member/Person/wc-dataload.xml dataload ..componentsfoundationsamplesDataLoadMemberPersonwc-dataload.xml 3. Verify the results of the data load: a. Review the Data Load utility summary report. For more information, see Verifying the results of the data load. b. Query the USERREG database table to verify that the data for the registered customers is loaded into the table. Note: The values for the following columns are set to null: LOGONPASSWORD SALT After the customization is complete, the encrypted data for these columns is loaded into the database table. 4. Create the WCDE_installdircomponentsfoundationsamplesDataloadMemberPassword directory. 5. In this directory, create the MemberDataload_Password.csv file. 6. Copy the following content into your MemberDataload_Password.csv file: logonId logonPassword jacobsmith shopper1 ryanjohnson shopper2 olivialee shopper3 This CSV file provides the user passwords for three different customers. The encrypted data for these passwords is loaded into the USERREG database table after the customization is complete. 7. Save and close the file. < Previous | Next >
  • 5. < Previous | 1.2. Next > Configuring the Data Load utility In this lesson, you configure the Data Load utility environment, business object, and load order configuration files. Environment configuration file Use to specify your business context information, database connection property, ID Resolver, and data writer class. For more information about this file, see wc-dataload-env.xsd. Business object configuration file Use to specify your data reader, business object builder, and business object mediator. For more information about this file, see wc-dataload- businessobject.xsd. Load order configuration file Use to specify a pointer to your CSV input file, environment variable settings file, and business object configuration file. You can also specify which data load mode you want to perform. For more information about this file, see wc-dataload.xsd. Procedure 1. Configure the environment variable settings that are used by the Data Load utility within an environment configuration file: a. Create an XML file named wc-dataload-env.xml in the WCDE_installdircomponentsfoundationsamplesDataLoadMemberPassword directory. b. Open the wc-dataload-env.xml file and paste the following content into the file: <?xml version="1.0" encoding="UTF-8"?> <_config:DataLoadEnvConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../../xml/config/xsd/wc-dataload-env.xsd" xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config"> <!-- Madisons Store in Toolkit --> 1 <_config:BusinessContext storeIdentifier="Madisons" languageId="-1" currency="USD"> </_config:BusinessContext> <!-- database setting for derby in Toolkit --> 2 <_config:Database type="derby" name="..dbmall" schema="APP"/> <!— IDResolver class --> <_config:IDResolver className="com.ibm.commerce.foundation.dataload.idresolve.IDResolverImpl" cacheSize="10000000"/> <!— DataWriter class --> 3 <_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" /> </_config:DataLoadEnvConfiguration> c. Update the attribute values in the following elements within the file: Element Required update Example 1 <_config:BusinessContext> Specify your store settings. <_config:BusinessContext storeIdentifier="Madisons" catalogIdentifier="Madisons" /> 2 <_config:Database> Specify your database environment properties. <_config:Database name="..dbmall" type="derby" /> 3 <_config:DataWriter> Specify your data writer class. <_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" /> For more information about data load environment settings, see Configuring the data load environment settings. 2. Specify your data reader, business object builder, and business object mediator that is used by the Data Load utility: a. Create a business object configuration file named wc-Passwordobject-loader.xml in the WCDE_installdircomponentsfoundationsamplesDataLoadMemberPassword directory. b. Open the wc-Passwordobject-loader.xml file and paste the following content into the file: <?xml version="1.0" encoding="UTF-8"?> <_config:DataloadBusinessObjectConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../xsd/wc-dataload-businessobject.xsd" xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config"> <_config:DataLoader className="com.ibm.commerce.foundation.dataload.BusinessObjectLoader"> <!-- Define data reader class to read the input from CSV file for loading --> 1 <_config:DataReader className="com.ibm.commerce.foundation.dataload.datareader.CSVReader" firstLineIsHeader="true" > </_config:DataReader> 2 <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" > <!-- exclude the columns whose values are loaded previously from USERREG table except LOGONPASSWORD & SALT --> <_config:Table name="USERREG" excludeUnListedColumns="true"> <_config:Column name="USERS_ID" value="USERS_ID" valueFrom="IDResolve"> <_config:IDResolve tableName="USERREG" generateNewKey="false" >
  • 6. <_config:UniqueIndexColumn name="LOGONID" value="logonId" /> </_config:IDResolve> </_config:Column> <_config:Column name="LOGONID" value="logonId" /> <!-- The value of the SALT will be generated and added back to the userreg table later --> <_config:Column name="SALT" value="" /> <!-- Plain password read from CSV file will be encrypted by the customized class PasswordEncryptionhandler --> <_config:Column name="LOGONPASSWORD" value="LOGONPASSWORD" valueFrom="ColumnHandler" > <_config:ColumnHandler className="com.mycompany.commerce.dataload.password.PasswordEncryptionHandler"> <!-- The following parameters will be passed to the class PasswordEncryptionhandler --> <_config:Parameter name="logonPassword" value="logonPassword" /> <!-- WC-server.xml is the file to store the merchant key --> <_config:Parameter name="instanceConfigLocation" value="c:IBMWCDE_ENT70workspaceWCxmlconfigwc-server.xml" valueFrom="Fixed" /> <!-- Customer may choose the merchant key in customKeyConfigLocation <_config:Parameter name="customKeyConfigLocation" value="Path to customKeyConfigLocation" valueFrom="Fixed" /> --> </_config:ColumnHandler> </_config:Column> </_config:Table> 3 <_config:BusinessObjectMediator className="com.ibm.commerce.foundation.dataload.businessobjectmediator.TableObjectMediator" componentId="com.ibm.commerce. </_config:BusinessObjectMediator> </_config:BusinessObjectBuilder> </_config:DataLoader> </_config:DataloadBusinessObjectConfiguration> c. Update the attribute values in the following elements within the file: Element Required update Example 1 <_config:DataReader> Specify the data reader class that you implement. <_config:DataReader className="com.ibm.commerce.foundation.dataload.datareader.CSVReader" firstLineIsHeade </_config:DataReader> 2 <_config:BusinessObjectBuilder> Specify the business object builder class that you implement. Note: Specify the TableObjectBuilder class in this tutorial since you extend the generic table loader to load the customized data into the database. The table object builder is an extension of a business object builder for data load. It accepts data that is read by the data reader and constructs value objects to represent physical tables in the database. <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableOb 3 <_config:BusinessObjectMediator> Specify the business object mediator class. Note: Specify the TableObjectMediator class in this tutorial since you extend the generic table loader to load the customized data into the database instead of extending the business object logic. <_config:BusinessObjectMediator className="com.ibm.commerce.foundation.dataload.businessobjectmediator.Tabl </_config:BusinessObjectMediator> d. Set the excludeUnListedColumns parameter to true to exclude database columns that were previously loaded to the USERREG database table during the initial load: <_config:Table name="USERREG" excludeUnListedColumns="true"> When a column is excluded, the value in that column is not modified during the data load. The following code snippet shows the parameter values for the columns from the USERREG database table. Any columns that are defined in the database table (such as STATUS and PLCYACCT_ID) that are not included in this snippet were already loaded into the database during the initial load. <_config:Column name="USERS_ID" 1 value="USERS_ID" valueFrom="IDResolve">
  • 7. <_config:IDResolve tableName="USERREG" generateNewKey="false" > <_config:UniqueIndexColumn name="LOGONID" value="logonId" /> </_config:IDResolve> </_config:Column> <_config:Column name="LOGONID" 2 value="logonId" /> <!-- The value of the SALT will be generated and added back to the userreg table later --> <_config:Column name="SALT" 3 value="" /> <!-- Plain password read from CSV file will be encrypted by the customized class PasswordEncryptionhandler --> <_config:Column name="LOGONPASSWORD" 4 value="LOGONPASSWORD" valueFrom="ColumnHandler" > <_config:ColumnHandler className="com.mycompany.commerce.dataload.password.PasswordEncryptionHandler"> <!-- The following parameters will be passed to the class PasswordEncryptionhandler --> <_config:Parameter name="logonPassword" value="logonPassword" /> <!-- WC-server.xml is the file to store the merchant key --> <_config:Parameter name="instanceConfigLocation" value="c:IBMWCDE_ENT70workspaceWCxmlconfigwc-server.xml" valueFrom="Fixed" /> </_config:ColumnHandler> </_config:Column> Note: Customer also can choose to store their merchant key in their own customKeyConfigLocation, the usage example is provided below as well. <!-- Customer may choose the merchant key in customKeyConfigLocation: <_config:Parameter name="customKeyConfigLocation" value="Path to customKeyConfigLocation" valueFrom="Fixed" /> 1 USERS_ID The value for this parameter is retrieved from the IDRESOLVE column of the USERREG database table, according to the LOGONID input. 2 LOGONID The value for this parameter is read from the LOGONID column that is defined in the CSV input file. 3 SALT The value for this parameter is generated by the generateSalt method and then added to the USERREG database table later. Specify the column for SALT here so that the corresponding salt value is written into the database later. The excludeUnlistedColumns parameter is set to true to specify that all of the columns that are not listed in the preceding code snippet are excluded and are not written to the database. 4 LOGONPASSWORD The value for this parameter is read from the ColumnHandler interface. A new class that is called PasswordEncryptionhandler will be created as an extension class of AbstractColumnHandler in the next step of this tutorial. This class is used to generate the value for the SALT parameter and encrypt the user password. This value is achieved by taking the user password from the CSV input file and the merchant key from the wc-server.xml file. Parameters such as logonPassword and instanceConfigLocation are passed to the new PasswordEncryptionhandler class. 3. Specify the data load order and mode that the Data Load utility uses to load data: a. Create an XML file named wc-dataload.xml in the WCDE_installdircomponentsfoundationsamplesDataLoadMemberPassword directory. b. Open the wc-dataload.xml file and paste the following content into the file: <?xml version="1.0" encoding="UTF-8"?> <_config:DataLoadConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../xsd/wc-dataload.xsd" xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config"> 1 <_config:DataLoadEnvironment configFile="wc-dataload-env.xml" /> 2 <_config:LoadOrder commitCount="100" batchSize="1" dataLoadMode="Replace"> <!-- Dataload password --> <_config:LoadItem name="Password" businessObjectConfigFile="wc-Passwordobject-loader.xml"> <!--Indicates the location of input csv files, can be overrided at runtime.--> <_config:DataSourceLocation location="MemberDataload_Password.csv" /> </_config:LoadItem> </_config:LoadOrder> </_config:DataLoadConfiguration> c. Update the attribute values in the following elements within the file: Element Required update Example 1 <_config:DataLoadEnvironment> Specify your environment variable file. <_config:DataLoadEnvironment configFile="wc-dataload-env.xml" /> 2 <_config:LoadOrder> Specify your CSV input file, business object configuration file, and data load mode. <_config:LoadOrder commitCount="100" batchSize="1" dataLoadMode="Replace"> <!-- Dataload password --> <_config:LoadItem name="Password" businessObjectConfigFile="wc-Passwordobject-loader.xml"> <!--Indicates the location of input csv files, can be overrided at runtime.--> <_config:DataSourceLocation location="MemberDataload_Password.csv" /> </_config:LoadItem> </_config:LoadOrder> For more information about configuring your data load order, see Configuring the data load order. < Previous | Next >
  • 8. < Previous | 1.3. Next > Extending the generic table loader to encrypt user passwords In this lesson, you create a new Java class to implement the ColumnHandler interface to encrypt user password data. Create a custom column handler to resolve data from database columns that are based on input values that cannot be mapped to the column value through the configuration. For example, if your input to populate the password column in the database is a clear text password, but the column in the database table expects an encrypted value, create a custom column handler to encrypt the clear text password and return an encrypted password from the resolveColumnValue() method. Procedure 1. Open WebSphere Commerce Developer. 2. Switch to the Java EE perspective. 3. In the Enterprise Explorer view, navigate to WebSphereCommerceServerExtensionsLogic > src. 4. Right-click src; then click New > Package. 5. In the Name field, type com.mycompany.commerce.dataload.password. Click Finish to create a new package. 6. Right-click com.mycompany.commerce.dataload.password; then click New > Class. 7. In the Name field, type PasswordEncryptionHandler. 8. From the Superclass field, click Browse to browse for the AbstractColumnHandler abstract class; then click OK. 9. Click Finish to create the class. A new file opens for editing. 10. Paste the following code snippet into the new file: package com.mycompany.commerce.dataload.password; import java.util.Map; import com.ibm.commerce.foundation.common.util.logging.LoggingHelper; import com.ibm.commerce.foundation.dataload.config.AbstractColumnHandler; import com.ibm.commerce.foundation.dataload.exception.DataLoadException; import com.ibm.commerce.foundation.dataload.object.ExtendedTableDataObject; import com.ibm.commerce.member.dataload.helper.MemberDataLoadHelper; import com.ibm.commerce.util.WCPasswordEncrypter; /** *The main purpose of the PasswordEncryptionHandler is to resolve the column value based * on the input values which cannot be simply mapped to the column value * through the configuration. For example, to populate the password column in the * database, your input is a clear text password, but the column in the database expects * an encrypted value, you need to write a custom handler to encrypt the clear * password and return an encrypted password from the method resolveColumnValue(). */ public class PasswordEncryptionHandler extends AbstractColumnHandler { private final static String CLASSNAME = PasswordEncryptionHandler.class.getName(); private static final java.util.logging.Logger LOGGER = LoggingHelper.getLogger(PasswordEncryptionHandler.class); private static final String CUSTOM_KEY_CONFIG_LOCATION = "customKeyConfigLocation"; private static final String INSTANCE_CONFIG_LOCATION = "instanceConfigLocation"; /** * Resolve the column value based on the input parameters passed in. * The parameter map will contain the map from LOGONID to the value of logonId. * The extenededTableDataObject represents the row of the table. * @param parameterMap a parameter map which are name-value pairs * defined in the configuration. * @param extenededTableDataObject the current table data object which contains * the table configuration and some column values which have been * resolved so far. * @return resolved new column value for the encrypted password * @throws DataLoadException if there are some errors to resolve the value. */ public String resolveColumnValue(Map<String, String> parameterMap, ExtendedTableDataObject extendedTableDataObject) throws DataLoadException { // TODO Auto-generated method stub final String METHODNAME = "resolveColumnValue"; /* Enable the log trace for the method */ if (LoggingHelper.isEntryExitTraceEnabled(LOGGER)){ LOGGER.entering(CLASSNAME, METHODNAME); } /* Call API to get the salt value */ String salt = MemberDataLoadHelper.generateSalt(); 1 /* Store the salt value back into the USREEG table */ extendedTableDataObject.addColumnData 2 ("SALT", salt); String password = parameterMap.get("logonPassword"); String InstanceconfigLocation = parameterMap.get(INSTANCE_CONFIG_LOCATION); String customKeyConfigLocation = parameterMap.get(CUSTOM_KEY_CONFIG_LOCATION); String encryptedPWD = null;
  • 9. try { WCPasswordEncrypter encrypter = new WCPasswordEncrypter(); /* Provide the file location which the merchant key is stored in so that to initialize the Merchant key registration */ encrypter.initializeKeyRegistry 3 (customKeyConfigLocation, InstanceconfigLocation); /*Encrypt the plain text password */ encryptedPWD = encrypter.encryptPassword 4 (password, salt); } catch (Exception e) { // TODO Auto-generated catch block e.printStackTrace(); } if (LoggingHelper.isEntryExitTraceEnabled(LOGGER)) { LOGGER.exiting(CLASSNAME, METHODNAME); } /* return the encrypted password */ return encryptedPWD ; } } 1 generateSalt() Generates a random salt string with a length of 12 characters. 2 extendedTableDataObject.addColumnData Extends TableDataObject to include more ID resolver information for certain database table columns. This class represents a row in a physical database table such as USERREG. addColumnData takes the column name and value as input for Salt and adds the value back to the SALT column in the USERREG database table. 3 initializeKeyRegistry Initializes the key registry by providing one of the following parameters: customKeyConfigFilename Specifies the full path to the custom key configuration file when the merchant key is stored in an external medium. instanceXMLFilename Specifies the full path to the instance configuration XML file when the merchant key is stored within the instance configuration file. If both parameters are specified, an initial attempt is made to initialize the key registry with the custom key configuration file. If that fails, an attempt is made to initialize the key registry with the instance configuration file. 4 encryptPassword Returns the WebSphere Commerce encrypted password with the given clear text password and salt. The caller of this method must first initialize the key registry by calling the initializeKeyRegistry(String instanceXMLFilename, String customKeyConfigFilename) method. < Previous | Next >
  • 10. < Previous Verifying your customization In this step of the tutorial, you verify your customization by running the Data Load utility in WebSphere Commerce Developer. Procedure 1. Open WebSphere Commerce Developer. 2. Ensure that your WebSphere Commerce Server is stopped before you run the Data Load utility. For more information about stopping your server, see Starting and stopping WebSphere Commerce Test Server. 3. Click Run > Run Configurations. The Run Configurations dialog displays. 4. Create a new Java Application configuration. Double-click the Java Application on the left panel; then on the Main tab, enter the following information: a. In the Name field, type a name for your configuration. b. In the Project field, enter the project WebSphereCommerceServerExtensionsLogic. c. In the Main class field, enter the class name com.ibm.commerce.foundation.dataload.DataLoaderMain. 5. Click the Arguments tab; then enter the following information: a. In the Program arguments field, enter ..samplesDataLoadMemberPasswordwc-dataload.xml. b. In the VM arguments field, set the following parameters -Dj2se Sets the Java platform type to J2SE. Set the value for the parameter to true. -Djava.util.logging.config.file Sets the JVM parameter. Set the value of the parameter to ..workspaceWCxmlconfigdataloadlogging.properties. c. In the Working directory field, enter the WCDE_installdir/bin directory.
  • 11. 6. Click the Classpath tab; then click User Entries and add the following class path information: a. Click Add External JARs, then browse to the location of the JDBC driver JAR file for the specific database you are using. Click Open. WCDE_installdirlibderby.jar db2_installdirjavadb2jcc4.jar oracle_installdirjdbclibojdbc6.jar b. Click Advanced > Add Folder, then click OK. Navigate to WC > xml, then click OK. 7. Click Run. 8. Verify your results by running the following SQL statement: Select * from userreg The USERREG database table displays the content that you specified in your CSV file.
  • 12. 9. Open your store in an internet browser. 10. Click Sign In and log on to the store with the following user credentials: User name Password jacobsmith shopper1 This user information is defined in the MemberDataload_Password.csv file and loaded into the WebSphere Commerce database. < Previous
  • 13. 2. Next > Customizing the Data Load utility for XML input This tutorial demonstrates how to customize the data load utility to load data from an XML file. Note: Support for loading XML formatted files with the Data Load utility is provided by default. You do not have to complete this customization to use XML formatted input files with the Data Load utility. For more information about the supported format for Data Load utility input files, see File format for Data Load input files. If your XML files use a different format than the format that is supported by default, you can use this tutorial to help you create an XML reader that supports the XML format of your files. In the tutorial "Adding new properties to a WebSphere Commerce service by using the data service layer", you customize the catalog service. The customization adds support warranty information and care instructions for catalog entries. Now, you load the warranty and care instruction information into your target database by using the Data Load utility. The following diagram shows the data load process flow. 1. The data reader reads the input file, and transforms the input into a data object. Data objects are typically Map objects. 2. The business object builder instantiates the business object and populates it from the data in the data object. 3. The business object mediator converts the business object into physical object. 4. The data writer writes the physical object into the database. The provided CSV reader reads the input files and creates Map objects. XML files are difficult to represent in a Map object because they have complex structures. In this scenario, you create a reader that builds the logical object directly from the input XML file. Since this reader is creating a logical object, the default business object builder is used. The default business object builder returns the input data object without changing it. Learning objectives After completing this tutorial, you should be able to complete the following tasks. Create your XML data reader Configure the Data Load utility to use your data reader and mediator Time required This tutorial is expected to take 1 hour to complete. Skill level Intermediate Audience This tutorial is intended for WebSphere Commerce developers who want to customize the data load utility to read from XML files. System requirements Before you begin this tutorial, ensure that you complete the following prerequisites: Completed the following steps in:Tutorial: Extending a BOD service to manage UserData with the Data Service layer Customizing the WebSphere Commerce schema Customizing the physical layer to include the new information Prerequisites Saving the sample code Before you begin this tutorial, download the DataLoadSample.zip. Save this file onto your development machine. As an example, you might save it into the WCDE_installdir directory. Knowledge prerequisites Before you begin this tutorial, ensure that you have: A basic understanding of the Data Load utility architectural overview A basic understanding of DOM
  • 14. An understanding of Service Data Objects (SDO) Next >
  • 15. < Previous | 2.1. Next > Creating an XML input file In this lesson, you create an XML file to contain the catalog entry, warranty, and care instruction data to load with the Data Load utility. The XML file is based on the data that is created in the Tutorial: Extending a BOD service to manage UserData with the Data Service layer. This file contains catalog entry base and description information, warranty information, and care instruction information, which extends the catalog entry object. Procedure 1. Create the WCDE_installdirsamplesDataloadwarranty directory. 2. Create a warranty.xml file in the WCDE_installdirsamplesDataloadwarranty directory. 3. Copy the following content into your warranty.xml file. <?xml version="1.0" encoding="utf-8"?> <customInfo> <customProductInfo partNumber="MW-0003" Type="ProductBean"> <warranty warrantyTerm="30" warrantyType="LIMITED"/> <attributes> <attribute name="manufacturerPartNumber" value="" /> <attribute name="manufacturer" value="MapleWear" /> <attribute name="url" value="" /> <attribute name="field1" value="1" /> <attribute name="field2" value="1" /> <attribute name="field3" value="1" /> <attribute name="field4" value="" /> <attribute name="field5" value="" /> <attribute name="onSpecial" value="1" /> <attribute name="onAuction" value="1" /> <attribute name="buyable" value="1" /> </attributes> <description language="-1" Name="Striped turtleneck" ShortDescription="Soft, stretchy rib-knit turtleneck." LongDescription="This soft, stretchy rib-knit turtleneck is perfect either on its own or layered under a jumper." Thumbnail="images/catalog/apparel/apparel_70x70/IMG_0003_e.jpg" FullImage="images/catalog/apparel/apparel_160x160/IMG_0003_e.jpg" Keyword=""> <attribute name="careinstruction" value="Avoid soap and water"/> <attribute name="auxDescription1" value=""/> <attribute name="auxDescription2" value=""/> <attribute name="available" value="1"/> <attribute name="availabilityDate" value="2008-09-03 16:00:00"/> <attribute name="published" value="1"/> </description> </customProductInfo> </customInfo> The customProductInfo element maps to a catalog entry noun, while the description element maps to a catalog entry description noun part. The data load utility supports creating, replacing, or deleting data objects in the database table. To replace data objects in the database, ensure that you supply all attributes. In the preceding example, all catalog entry information is provided. 4. Save and close the file. < Previous | Next >
  • 16. < Previous | 2.2. Next > Creating a custom data reader In this lesson, you create a custom data reader to read your XML input file and then transform the input data into business objects. The Data Load utility already provides you with a CSV data reader, which is implemented by the CSVReader class. It is necessary to create custom data readers in order to support data source of types other than CSV files. This tutorial shows you how to create a custom data reader to support the XML file that you created (warranty.xml). A DOM parser is used to parse the XML file to a DOM tree, and then build a commerce business object. To create a custom data reader, you need to implement the DataReader interface. The AbstractDataReader class implements most methods in the DataReader interface. The custom data reader class can extend this class and provide the additional implementation for the next(), getSourcePosition(), and init() methods. The following Java classes are provided: WarrantyReader.java: This class implements the default behavior of the DataReader interface. It reads the XML input file, and then creates the business object. WarrantyConstants.java: This class defines the constants that are used in parsing the XML file. Procedure 1. Extract the DataLoadSample.zip file to a temporary directory on your machine. 2. Open WebSphere Commerce Developer. 3. In the Enterprise explorer view, right-click the WebSphereCommerceServerExtensionsLogic folder and click New > Package. 4. In the Name field, enter com.mycompany.commerce.dataload.reader and click Finish. 5. Right-click the com.mycompany.commerce.dataload.reader package, and click Import > General > File system. 6. Click Next; then click Browse and navigate to the Dataload_Temp directory, where Dataload_Temp is the location where you extracted the sample Java files. Select the WarrantyReader.java file and WarrantyConstants.java file. Click Finish to import the files. 7. Open the WarrantyReader.java file and examine its content. The following methods are implemented: init(): This method initializes the data reader. It gets the input XML file location from the configuration file, and then reads the XML file into the DOM tree object. next(): This method parses the XML DOM tree object, builds the next logical object, and encapsulates it within a DataLoadBusinessObject. It then returns the DataLoadBusinessObject. The returned data is passed to the business object builder to generate business objects. Each business object is passed to the corresponding business object mediator to process the data. In this tutorial, a business object builder is not needed because the data reader is creating the business objects, which are encapsulated in the DataLoadBusinessObject. For example, the following code snippets generate SDO for the attribute part number: // Get partNumber attrvalue = ((Element) node) .getAttribute(WarrantyConstants.PART_NUMBER); if (attrvalue != null) { attrvalue = attrvalue.trim(); } if (LOGGER.isLoggable(Level.FINER)) { LOGGER.logp(Level.FINER, CLASSNAME, METHODNAME, WarrantyConstants.PART_NUMBER + ":" + attrvalue); } CatalogEntryType catentry = CatalogFactory.eINSTANCE .createCatalogEntryType(); CatalogEntryIdentifierType identifier = CommerceFoundationFactory.eINSTANCE .createCatalogEntryIdentifierType(); PartNumberIdentifierType partNumber = CommerceFoundationFactory.eINSTANCE .createPartNumberIdentifierType(); identifier.setExternalIdentifier(partNumber); catentry.setCatalogEntryIdentifier(identifier); partNumber.setPartNumber(attrvalue); If you want to know more about generating SDOs, see Service Data Objects (SDO). getSourcePosition(): This method returns the source position of the data object returned by the next() method last called. parseWarranty(Node child,CatalogEntryType catentry): This method is called by the next() method. The method parses the warranty element and its subelements. It then populates the appropriate attributes of the logical object. parseDescription(Node child,CatalogEntryType catentry): This method is called by the next() method. The method parses the description element and its subelements. It then populates the appropriate attributes of the logical object. parseCatentryAttribute(Node child,CatalogEntryType catentry): This method is called by the next() method. The method parses the CatentryAttribute element and its subelements. It then populates the appropriate attributes of the logical object. < Previous | Next >
  • 17. < Previous | 2.3. Next > Configuring the Data Load utility In this step of the tutorial, you configure the environment settings, load order, and the business object configuration file. Procedure 1. Create your data load environment configuration file. In the environment configuration file, specify the business context information, database connection property, ID Resolver, and data writer class. For more information about creating this file, see Configuring the data load environment settings. The environment variables that are used by the Data Load utility must be configured in this XML file. a. Copy the sample wc-dataload-env.xml environment configuration file into the WCDE_installdirsamplesDataLoadwarranty directory. The sample environment configuration file is located within the DataLoadSample.zip compressed file. b. Open the wc-dataload-env.xml file for editing. Change the storeIdentifier, and catalogIdentifier attribute values for the <_config:BusinessContext> configuration element to match your store settings. For example, <_config:BusinessContext storeIdentifier="Aurora" catalogIdentifier="Aurora" /> Change the name, and type attribute values for the <_config:Database> configuration element to match your database settings. For example, <_config:Database name="..dbmall" type="derby" /> Change the classname attribute value for the <_config:DataWriter> configuration element to specify the data writer class. By default the Data Load utility uses the com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter data writer class. For example, <_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" /> c. Save and close the file. 2. Create your business object configuration file. In the business object configuration file, you specify your data reader, business object builder, and business object mediator. For more information, see Configuring the business object configuration file. a. Copy the sample wc-warrantyobject-loader.xml business object configuration file into the WCDE_installdirsamplesDataLoadwarranty directory. The sample configuration file is located within the DataLoadSample.zip compressed file. b. Open the wc-warrantyobject-loader.xml for editing. Change the classname attribute value for the <_config:DataReader> element to specify the data reader class that you implemented. For example, <_config:DataReader className="com.mycompany.commerce.dataload.reader.WarrantyReader" /> Change the classname attribute value for the <_config:BusinessObjectBuilder> element to specify the business object builder class that you implemented. For example, <_config:BusinessObjectBuilder className="" > Change the className and componentId attribute values for the <_config:BusinessObjectMediator> element to specify the business object mediator. In this tutorial, the warranty and care instruction data are extension of the catalog entry business object. Specify the CatalogEntryMediator class and componentId. For example, <_config:BusinessObjectMediator className="com.ibm.commerce.catalog.dataload.mediator.CatalogEntryMediator" componentId="com.ibm.commerce.catalog"/> c. Save and close the file. 3. Configure the data load order configuration file. In the load order configuration file, you specify a pointer to your XML input file, environment settings file, and business object configuration file. You also specify which data load mode that the utility is to run. For more information, see Configuring the data load order. a. Copy the sample wc-dataload-warranty.xml load order configuration file into the WCDE_installdirsamplesDataLoadwarranty directory. The sample configuration file is located within the DataLoadSample.zip compressed file. b. Open the wc-dataload-warranty.xml file for editing Change the configFile attribute value for the <_config:DataLoadEnvironment> element to specify the environment configuration file. For example, <_config:DataLoadEnvironment configFile="wc-dataload-env.xml" /> Within the <_config:LoadOrder > element, configure the <_config:LoadItem> and <_config:DataSourceLocation elements. Change the commitCount, batchSize, dataLoadMode, name, and businessObjectConfigFile attribute values. For example, <_config:LoadItem commitCount="1" batchSize="1" dataLoadMode="Replace" name="Warranty" businessObjectConfigFile="wc-warrantyobject-loader.xml"> </_config:LoadItem> Within the <_config:DataSourceLocation element, change the location attribute value to specify the input file. For example, <_config:LoadOrder > <_config:LoadOrder > <_config:LoadItem commitCount="1" batchSize="1" dataLoadMode="Replace" name="Warranty" businessObjectConfigFile="wc-warrantyobject-loader.xml"> <_config:DataSourceLocation location="warranty.xml"/> </_config:LoadItem> </_config:LoadOrder> c. Save and close the file. In the workspace_dirWCxmlconfigdataloadlogging.properties, you can specify the trace level, and enable console and file logging. The following settings are used by default: Logging level is set to both console and file. Trace level is set to INFO. Log file location is set to WCDE_installdirlogswc-dataload.log. You do not need to modify the log file location. However, you might want to change the trace level when you are debugging a data load problem. The Data Load
  • 18. utility provides some command line options to change the trace level. For more information, see Data Load utility. If any errors occur, an error log file is generated per load item. The error file name and location is specified in the summary report on the console. The error log file is generated in the same directory as the wc-dataload.log file. < Previous | Next >
  • 19. < Previous | 2.4. Next > Verifying your customization In this lesson, you verify your customization by running the Data Load utility in the development environment. Procedure 1. Open WebSphere Commerce Developer. 2. Ensure that the WebSphere Commerce Server is stopped before you run the Data Load utility. For more information about stopping the server, see Starting and stopping WebSphere Commerce Test Server. 3. Click Run > Run Configurations. The Run Configurations dialog displays. 4. Create a new Java Application configuration. Double-click the Java Application on the left panel; then on the Main tab, enter the following information: a. In the Name field, type a name for your configuration. b. In the Project field, enter the project WebSphereCommerceServerExtensionsLogic. c. In the Main class field, enter the class name com.ibm.commerce.foundation.dataload.DataLoaderMain. 5. Click the Arguments tab; then enter the following information: a. In the Program arguments field, enter ..samplesDataLoadwarrantywc-dataload-warranty.xml. b. In the VM arguments field, set the following parameter -Dj2se Sets the Java platform type to J2SE. Set the value of the parameter to true. -Djava.util.logging.config.file Sets the JVM parameter. Set the value of the parameter to ..workspaceWCxmlconfigdataloadlogging.properties. c. In the Working directory field, enter the WCDE_installdir/bin directory.
  • 20. 6. Click the Classpath tab; click User Entries, then add the following class path information: a. Click Add External JARs, then browse to the location of the JDBC driver JAR file for the specific database you are using. Click Open. WCDE_installdirlibderby.jar db2_installdirjavadb2jcc4.jar oracle_installdirjdbclibojdbc6.jar b. Click Add JARs, then browse to the location of the Inventory-Server.jar and Price-Server.jar files in the WCDE_installdir/WC directory. c. Click Advanced > Add Folder, then click OK. Navigate to WC > xml, then click OK. 7. Click Run. 8. Verify your results by running the following SQL statements: SELECT * FROM XWARRANTY;
  • 21. SELECT * FROM XCAREINSTRUCTION; You see the warranty and care information content you specified in your XML file. < Previous | Next >
  • 22. < Previous Deploying your customized code In this lesson, you deploy your custom code to a target WebSphere Commerce Server. Deploying to the server involves transferring the code from the WebSphere Commerce development environment to the target WebSphere Commerce Server. Procedure 1. Create the XWARRANTY and XCAREINSTRUCTION tables in the database that is used by your target WebSphere Commerce Server: a. Connect to your DB2 database. b. Run the SQL statement to create the tables. CREATE TABLE XWARRANTY ( CATENTRY_ID BIGINT NOT NULL, WARTERM INTEGER, WARTYPE VARCHAR(32), OPTCOUNTER SMALLINT, CONSTRAINT XWARRANTY_PK PRIMARY KEY(CATENTRY_ID), CONSTRAINT XWARRANTY_FK FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY(CATENTRY_ID) ); CREATE TABLE XCAREINSTRUCTION ( CATENTRY_ID BIGINT NOT NULL, LANGUAGE_ID INTEGER NOT NULL, CAREINSTRUCTION VARCHAR(254), OPTCOUNTER SMALLINT, CONSTRAINT XCAREINST_PK PRIMARY KEY (CATENTRY_ID, LANGUAGE_ID), CONSTRAINT XCAREINST_FK1 FOREIGN KEY (CATENTRY_ID, LANGUAGE_ID) REFERENCES CATENTDESC(CATENTRY_ID, LANGUAGE_ID), CONSTRAINT XCAREINST_FK2 FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY (CATENTRY_ID) ); a. Connect to your Oracle database. b. Run the SQL statement to create the tables. CREATE TABLE XWARRANTY ( CATENTRY_ID NUMBER NOT NULL, WARTERM INTEGER, WARTYPE VARCHAR(32), OPTCOUNTER SMALLINT, CONSTRAINT XWARRANTY_PK PRIMARY KEY(CATENTRY_ID), CONSTRAINT XWARRANTY_FK FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY(CATENTRY_ID) ); CREATE TABLE XCAREINSTRUCTION ( CATENTRY_ID NUMBER NOT NULL, LANGUAGE_ID INTEGER NOT NULL, CAREINSTRUCTION VARCHAR(254), OPTCOUNTER SMALLINT, CONSTRAINT XCAREINSTRUCTION_PK PRIMARY KEY (CATENTRY_ID, LANGUAGE_ID), CONSTRAINT XCAREINSTRUCTION_FK1 FOREIGN KEY (CATENTRY_ID, LANGUAGE_ID) REFERENCES CATENTDESC(CATENTRY_ID, LANGUAGE_ID), CONSTRAINT XCAREINSTRUCTION_FK2 FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY (CATENTRY_ID) ); 2. Create the WC_installdirinstancesinstance_namexmlconfigcom.ibm.commerce.catalog-ext directory if the directory does not exist. 3. Copy all the custom Data Service Layer configuration files from the WebSphere Commerce development environment to the WebSphere Commerce server instance directory. a. Navigate to the workspace_dirWCxmlconfigcom.ibm.commerce.catalog-ext directory. b. Copy all of the custom Data Service Layer configuration files within this directory. c. Navigate to the WC_installdirinstancesinstance_namexmlconfigcom.ibm.commerce.catalog-ext directory. d. Paste all of the files into this directory. 4. Package your custom XML reader class and the custom physical SDO classes. Package these classes in the WebSphereCommerceServerExtensionsLogic project to a JAR file. 5. Create the WC_installdirextlib directory if the directory does not exist. Copy the JAR file to the WC_installdirextlib directory. Your custom JAR files must be placed within the WC_installdirextlib directory. 6. Copy all the custom data load configuration files from the WebSphere Commerce development environment to the WebSphere Commerce server environment. a. Navigate to the WCDE_installdirsamplesDataLoadwarranty directory. b. Copy all of the custom data load configuration files within this directory. c. Navigate to the WC_installdirsamplesDataLoadwarranty directory. d. Paste all of the files into this directory. 7. Modify the data load environment configuration file, wc-dataload-env.xml, to match your WebSphere Commerce Server instance information: Change the attribute values for the <_config:BusinessContext> configuration element to match your store settings. Change the attribute values for the <_config:Database> configuration element to match your database environment properties.
  • 23. For more information about configuring the data load environment file, see Configuring the data load environment settings. 8. Test your customization on the WebSphere Commerce server by running the Data Load utility. Ensure that you include the parameter - Dinstance=instance_name when you run the utility For example, ./dataload.sh ..samples/DataLoad/warranty/wc-dataload-warranty.xml -Dinstance=instance_name dataload ..samplesDataLoadwarrantywc-dataload-warranty.xml -Dinstance=instance_name 9. Verify the results of the data load. You can verify your results by reviewing the summary and by checking that the data exists in the appropriate database tables. To check the database tables for the sample custom tables, run the following SQL statements. SELECT * FROM XWARRANTY; SELECT * FROM XCAREINSTRUCTION; If you see the warranty and care information that is included in the csv file that you loaded, your data load is successful. < Previous
  • 24. 3. Next > Tutorial: Loading data into custom tables with the Data Load utility This tutorial demonstrates how to use a table-based business object builder and mediator to load data into extended or custom database tables. By default the Data Load utility supports a table object builder and table object mediator class for loading data directly into a database table. This table-based builder and mediator can be used to load data when there is no component-based business object mediator available that supports the loaded objects. This table-based builder and mediator can also be used to load custom data into custom database tables when physical SDOs are not defined for the data that is being loaded. The table-based builder and mediator is used by the Data Load utility to map and then load the data to the appropriate database tables. To use the table object builder and table object mediator, you do not have to map the data to a logical noun, create a custom business object mediator or builder, or extend an existing mediator or builder. The table object builder and mediator are specified and configured within the data load business object configuration file. For more information about the table object builder and mediator, see Data load table-based mediator and builder. The following diagram shows the data load process flow. 1. The data reader reads the input file, and transforms the input into a data object. Data objects are typically map objects. In this tutorial, the default CSV data reader is used to read a provided sample CSV input file. 2. The business object builder instantiates the business object and populates it from the data in the data object. The table object builder is used in this tutorial as the business object builder for instantiating and populating the business objects. 3. The business object mediator converts the business object into physical object. The table object mediator is always used with the table object builder. This table-based mediator is used as the business object mediator in this tutorial to convert the business objects. 4. The data writer writes the physical object into the database. This tutorial uses the default data writer to write the generated physical objects into the database. Learning objectives After completing this tutorial, you should be able to complete the following tasks: Configure the Data Load utility to use the table object builder and table object mediator Configure the table object builder to load data into custom tables Time required This tutorial is expected to take 1 hour to complete. Skill level Intermediate Audience This tutorial is intended for WebSphere Commerce administrators who want to configure and run Data Load utility to load data into custom tables. Prerequisites Before you begin this tutorial, ensure that you complete the following tasks. Install WebSphere Commerce Developer version 7. This tutorial uses a WebSphere Commerce development environment to create custom database tables, and run the Data Load utility to populate the tables. You can choose to configure the Data Load utility and run this utility in an authoring or production environment. Download the sample WarrantyTableLoad.csv data load input file to a temporary directory. Ensure that you have an understanding of the following concepts and tasks. Overview of the Data Load utility Configuring and running the Data Load utility Next >
  • 25. < Previous | 3.1. Next > Create the custom tables In this lesson, you create custom extension tables that you can use the Data Load utility table-based builder and mediator to load data into. The following diagram outlines the changes to the WebSphere Commerce schema: The preceding diagram shows the new XWARRANTY and XCAREINSTRUCTION tables, and how they relate to the existing CATENTRY and CATENTDESC database tables. The XWARRANTY table has a foreign key to the CATENTRY table. The XCAREINSTRUCTION table has a foreign key to the CATENTDESC table. The XCAREINSTRUCTION table has a foreign key to the CATENTRY table. Procedure 1. Connect to the development database.. 2. Create the customization tables by running the following SQL statements: CREATE TABLE XWARRANTY ( CATENTRY_ID BIGINT NOT NULL, WARTERM INTEGER, WARTYPE VARCHAR(32), OPTCOUNTER SMALLINT, CONSTRAINT XWARRANTY_PK PRIMARY KEY(CATENTRY_ID), CONSTRAINT XWARRANTY_FK FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY(CATENTRY_ID) ON DELETE CASCADE ); CREATE TABLE XCAREINSTRUCTION ( CATENTRY_ID BIGINT NOT NULL, LANGUAGE_ID INTEGER NOT NULL, CAREINSTRUCTION VARCHAR(254), OPTCOUNTER SMALLINT, CONSTRAINT XCAREINST_PK PRIMARY KEY (CATENTRY_ID, LANGUAGE_ID), CONSTRAINT XCAREINST_FK1 FOREIGN KEY (CATENTRY_ID, LANGUAGE_ID) REFERENCES CATENTDESC(CATENTRY_ID, LANGUAGE_ID) ON DELETE CASCADE, CONSTRAINT XCAREINST_FK2 FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY (CATENTRY_ID) ON DELETE CASCADE ); CREATE TABLE XWARRANTY ( CATENTRY_ID NUMBER NOT NULL, WARTERM INTEGER, WARTYPE VARCHAR(32), OPTCOUNTER SMALLINT, CONSTRAINT XWARRANTY_PK PRIMARY KEY(CATENTRY_ID), CONSTRAINT XWARRANTY_FK FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY(CATENTRY_ID)); CREATE TABLE XCAREINSTRUCTION ( CATENTRY_ID NUMBER NOT NULL, LANGUAGE_ID INTEGER NOT NULL, CAREINSTRUCTION VARCHAR(254),
  • 26. OPTCOUNTER SMALLINT, CONSTRAINT XCAREINSTRUCTION_PK PRIMARY KEY (CATENTRY_ID, LANGUAGE_ID), CONSTRAINT XCAREINSTRUCTION_FK1 FOREIGN KEY (CATENTRY_ID, LANGUAGE_ID) REFERENCES CATENTDESC(CATENTRY_ID, LANGUAGE_ID), CONSTRAINT XCAREINSTRUCTION_FK2 FOREIGN KEY (CATENTRY_ID) REFERENCES CATENTRY (CATENTRY_ID) ); < Previous | Next >
  • 27. < Previous | 3.2. Next > Creating an input file to load data into multiple tables In this lesson, you create an input file that contains data for multiple database tables. The data can be loaded in a single data load operation to populate multiple tables. Procedure 1. Create the WCDE_installdirsamplesDataloadwarranty directory. 2. Copy the following WarrantyTableLoad.csv file into the WCDE_installdirsamplesDataloadwarranty directory. The following sample data is included within this file. This data is loaded into both the XWARRANTY and XCAREINSTRUCTION database tables from the single CSV file. PartNumber Type ParentPartNumber Sequence Name ShortDescription WarrentyTerm WarrantyType CareInstruction Cords Product 1 Men's corduroy pants Men's 8-wale corduroy pants 30 Limited Tumble Dry, Normal, Low Heat Cords - Black - 29 W x 28 L Item Cords 1 Men's corduroy pants - Black 29 W x 28 L Men's corduroy pants - Black 29 Width x 28 Length 30 Limited Tumble Dry, Normal, Low Heat Cords - Brown - 29 W x 28 L Item Cords 2 Men's corduroy pants - Brown 29 W x 28 L Men's corduroy pants - Brown 29 Width x 28 Length 30 Limited Tumble Dry, Normal, Low Heat Classic pleated dress pant Product 2 Classic pleated dress pants Wrinkle-free 5 pocket classic pleated dress pant 60 Limited Tumble Dry, Normal, No Heat Classic pleated dress pant - Black - 29 W x 32 L Item Classic pleated dress pant 1 Classic pleated dress pants - Black - 29 W x 32 L Classic pleated dress pants - Black 29 Width x 32 Length 60 Limited Tumble Dry, Normal, No Heat Dress shirt Product 1 Dress shirt Solid color button- down dress shirt 30 Comprehensive 100% cotton. Dry clean only. Dress shirt - White - 15 Item Dress shirt 1 Dress shirt - White - 15 Dress shirt - White - 15 collar - 32 Sleeve 30 Comprehensive 100% cotton. Dry clean only. Dress shirt - White - 16 Item Dress shirt 2 Dress shirt - White - 16 Dress shirt - White - 16 collar - 35 Sleeve 30 Comprehensive 100% cotton. Dry clean only. Casual shirt Product 2 Casual shirt Solid color button down casual shirt 90 Comprehensive 100% cotton. Machine washable. Casual shirt - White-Small Item Casual shirt 1 Casual shirt - White - Small Casual shirt - White - Small. Wrinkle free 90 Comprehensive 100% cotton. Machine washable. Casual shirt - Blue - Medium Item Casual shirt 2 Casual shirt - Blue - Medium Casual shirt - Blue - Medium. Wrinkle free 90 Comprehensive 100% cotton. Machine washable. < Previous | Next >
  • 28. < Previous | 3.3. Next > Configuring the environment settings In this lesson, you configure the Data Load utility environment settings. In the data load environment configuration file, you specify your business context information, database connection property, ID Resolver, and data writer class. The environment variables that are used by the Data Load utility are configured in this XML file. For more information, see Configuring the data load environment settings. Procedure 1. Go to the directory, WCDE_installdirsamplesDataLoad. 2. Create a folder Warranty. 3. Copy the sample wc-dataload-env.xml file from the WCDE_installdirsamplesDataLoadCatalog directory into the Warranty directory. 4. Open the new wc-dataload-env.xml copy for editing. Change the values in this file to match your system values. 5. Find the <_config:BusinessContext> element. Change the value for the attributes of this element to match the values for your store. For example, <_config:BusinessContext storeIdentifier="Aurora" catalogIdentifier="AuroraCatalog" /> Where Aurora is your store identifier, and AuroraCatalog is the identifier for you store catalog. 6. Find the <_config:Database> element. Change the value for the type, name, and schema attributes of this element to match the values for your database environment. For example, <_config:Database type="derby" name="..dbmydb" schema="mydbschema"/> 7. Find the <_config:DataWriter> element. This element identifies the data writer class that is used during the data load process. Ensure that the default data writer is specified. <_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" /> 8. Save your changes. Your file can resemble the following code: <?xml version="1.0" encoding="UTF-8"?> <_config:DataLoadEnvConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../../xml/config/xsd/wc-dataload-env.xsd" xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config"> <_config:BusinessContext storeIdentifier="Aurora" catalogIdentifier="AuroraCatalog" languageId="-1" currency="USD"> </_config:BusinessContext> <_config:Database type="derby" name="..dbmydb" schema="mydbschema"/> <_config:IDResolver className="com.ibm.commerce.foundation.dataload.idresolve.IDResolverImpl" cacheSize="10000000"/> <_config:DataWriter className="com.ibm.commerce.foundation.dataload.datawriter.JDBCDataWriter" /> </_config:DataLoadEnvConfiguration> < Previous | Next >
  • 29. < Previous | 3.4. Next > Configuring the business object configuration file In this lesson, you configure the data load business object configuration file. In the business object configuration file, you specify your data reader, business object builder, and business object mediator. For more information, see Configuring the business object configuration file. This configuration file contains the information that is required by the Data Load utility to resolve the column and table identifiers for the input data. This file is used by the Data Load utility to populate a business object with the input data and any default or specified fixed values. To load the data included in the sample WarrantyTableLoad.csv input file, this configuration file identifies two builders and two mediators for the Data Load utility to use. To load the input data, the configuration file specifies that the utility is to use the default base business object builder and a component-based business object mediator to load data into the CATENTRY table. This configuration file utility species that the utility is to use the table-based business object builder and mediator to load data into the XWARRANTY AND XCAREINSTRUCTION tables. You can also use the table-based builder and mediator to load data into the CATENTRY table, but you are recommended to use the component-based mediator. Procedure 1. Copy a sample business object configuration file from the WCDE_installdirsamplesDataLoadCatalog directory into the WCDE_installdirsamplesDataLoadwarranty directory. For example, wc-loader-catalog-entry.xml. Rename the file. For example, wc-warrantytableobject- loader.xml. 2. Open the new business object configuration file for editing. 3. Find the <_config:BusinessObjectBuilder> element. If you are creating a business object configuration instead of editing an existing file, create this <_config:BusinessObjectBuilder> element within the element. 4. Set the attributes for the business object builder element. a. Set the className attribute value to be the com.ibm.commerce.foundation.dataload.businessobjectbuilder.BaseBusinessObjectBuilder object builder class. b. Set the value for the packageName attribute to be com.ibm.commerce.catalog.facade.datatypes.CatalogPackage. c. Set the value for the dataObjectType attribute to be CatalogEntryType. For example, the following snippet specifies the component-based object builder class for catalog entries. <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.BaseBusinessObjectBuilder" packageName="com.ibm.commerce.catalog.facade.datatypes.CatalogPackage" dataObjectType="CatalogEntryType" > </_config:BusinessObjectBuilder> 5. Configure the data mappings for the catalog entry business object. For example, the following code snippet includes data mappings for catalog entry objects. <_config:DataMapping> <_config:mapping xpath="CatalogEntryIdentifier/ExternalIdentifier/PartNumber" value="PartNumber" /> <_config:mapping xpath="catalogEntryTypeCode" value="Type" /> <_config:mapping xpath="ParentCatalogEntryIdentifier/ExternalIdentifier/PartNumber" value="ParentPartNumber" /> <_config:mapping xpath="displaySequence" value="Sequence" /> <_config:mapping xpath="Description[0]/Name" value="Name" /> <_config:mapping xpath="Description[0]/ShortDescription" value="ShortDescription" /> </_config:DataMapping> 6. Specify the business object mediator className to be the CatalogEntryMediator class. This class is the mediator for loading catalog entry data. Set the value for the componentId attribute for the mediator to be com.ibm.commerce.catalog. For example, the following snippet specifies the component-based mediator and component identifier. <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.BaseBusinessObjectBuilder" packageName="com.ibm.commerce.catalog.facade.datatypes.CatalogPackage" dataObjectType="CatalogEntryType" > ... <_config:BusinessObjectMediator className="com.ibm.commerce.catalog.dataload.mediator.CatalogEntryMediator" componentId="com.ibm.commerce.catalog" > </_config:BusinessObjectMediator> </_config:BusinessObjectBuilder> 7. Create a second <_config:BusinessObjectBuilder> element. Set the className attribute value to be the TableObjectBuilder table object builder class. For example, the following snippet specifies the table object builder class <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" > </_config:BusinessObjectBuilder> 8. Add elements to include the configurations for the XWARRANTY, and XCAREINSTRUCTION tables that the data is to be loaded into. Remove any other existing table and column configurations within this <_config:BusinessObjectBuilder> element. Specify each of the tables that the Data Load utility is to load data into within a separate <_config:Table> element. You do not need to include the CATENTRY table. The data for this table is loaded by using the component-based builder and mediator that you previously configured. For example, the following code snippet includes separate table configuration elements for the XWARRANTY and XCAREINSTRUCTION tables. <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" > <_config:Table name="XWARRANTY"> </_config:Table> <_config:Table name="XCAREINSTRUCTION"> </_config:Table> </_config:BusinessObjectBuilder> 9. Configure the columns for each of the tables that are to be loaded with the table-based builder and mediator. Specify the configuration for each column separately within the appropriate <_config:Table>. Use the <_config:Column> element to configure a column. Specify the attributes for each column configuration. The following snippet specifies the column configuration for loading data in the two extension tables: <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" >
  • 30. <_config:Table name="XWARRANTY"> <_config:Column name="CATENTRY_ID" value="catentry_Id" valueFrom="IDResolve" <_config:IDResolve tableName="CATENTRY" generateNewKey="false" > <_config:UniqueIndexColumn name="PARTNUMBER" value="PartNumber" /> <_config:UniqueIndexColumn name="MEMBER_ID" value="storeOwnerId" valueFrom="BusinessContext" /> </_config:IDResolve> </_config:Column> <_config:Column name="WARTERM" value="WarrantyTerm" /> <_config:Column name="WARTYPE" value="WarrentyType" /> </_config:Table> <_config:Table name="XCAREINSTRUCTION"> <_config:Column name="CATENTRY_ID" value="catentry_Id" valueFrom="IDResolve" /> <_config:Column name="LANGUAGE_ID" value="langId" valueFrom="BusinessContext" /> <_config:Column name="CAREINSTRUCTION" value="CareInstruction" /> </_config:Table> </_config:BusinessObjectBuilder> 10. Specify the business object mediator to be the TableObjectMediator class. If you are using the TableObjectBuilder class as the builder, always use this table object mediator as the business object mediator. This class is the mediator that the Data Load utility uses for all table-based data load. The following snippet specifies the table object mediator: <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" > ... <_config:BusinessObjectMediator className="com.ibm.commerce.foundation.dataload.businessobjectmediator.TableObjectMediator" > </_config:BusinessObjectMediator> </_config:BusinessObjectBuilder> 11. Save your file. Your file can resemble the following code: <?xml version="1.0" encoding="UTF-8"?> <_config:DataloadBusinessObjectConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../../../xml/config/xsd/wc-dataload-businessobject.xsd" xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config"> <_config:DataLoader className="com.ibm.commerce.foundation.dataload.BusinessObjectLoader" > <_config:DataReader className="com.ibm.commerce.foundation.dataload.datareader.CSVReader" firstLineIsHeader="true" useHeaderAsColumnName="true" > <_config:property name="keyColumns" value="PartNumber" /> </_config:DataReader> <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.BaseBusinessObjectBuilder" packageName="com.ibm.commerce.catalog.facade.datatypes.CatalogPackage" dataObjectType="CatalogEntryType" > <_config:DataMapping> <_config:mapping xpath="CatalogEntryIdentifier/ExternalIdentifier/PartNumber" value="PartNumber" /> <_config:mapping xpath="catalogEntryTypeCode" value="Type" /> <_config:mapping xpath="ParentCatalogEntryIdentifier/ExternalIdentifier/PartNumber" value="ParentPartNumber" /> <_config:mapping xpath="displaySequence" value="Sequence" /> <_config:mapping xpath="Description[0]/Name" value="Name" /> <_config:mapping xpath="Description[0]/ShortDescription" value="ShortDescription" /> </_config:DataMapping> <_config:BusinessObjectMediator className="com.ibm.commerce.catalog.dataload.mediator.CatalogEntryMediator" componentId="com.ibm.commerce.catalog" > </_config:BusinessObjectMediator> </_config:BusinessObjectBuilder> <_config:BusinessObjectBuilder className="com.ibm.commerce.foundation.dataload.businessobjectbuilder.TableObjectBuilder" > <_config:Table name="XWARRANTY"> <_config:Column name="CATENTRY_ID" value="catentry_Id" valueFrom="IDResolve" <_config:IDResolve tableName="CATENTRY" generateNewKey="false" > <_config:UniqueIndexColumn name="PARTNUMBER" value="PartNumber" /> <_config:UniqueIndexColumn name="MEMBER_ID" value="storeOwnerId" valueFrom="BusinessContext" /> </_config:IDResolve> </_config:Column> <_config:Column name="WARTERM" value="WarrantyTerm" /> <_config:Column name="WARTYPE" value="WarrentyType" /> </_config:Table> <_config:Table name="XCAREINSTRUCTION"> <_config:Column name="CATENTRY_ID" value="catentry_Id" valueFrom="IDResolve" /> <_config:Column name="LANGUAGE_ID" value="langId" valueFrom="BusinessContext" /> <_config:Column name="CAREINSTRUCTION" value="CareInstruction" /> </_config:Table> <_config:BusinessObjectMediator className="com.ibm.commerce.foundation.dataload.businessobjectmediator.TableObjectMediator" > </_config:BusinessObjectMediator> </_config:BusinessObjectBuilder> </_config:DataLoader> </_config:DataloadBusinessObjectConfiguration> < Previous | Next >
  • 31. < Previous | 3.5. Next > Configuring the load order configuration file In this lesson, you configure the Data Load utility load order configuration file. In the load order configuration file, you specify a pointer to your XML input file, environment settings file, and business configuration file. The load order configuration file defines the mode in which the Data Load utility uses to load data. You also specify the order in which Data Load utility loads the data. For more information, see Configuring the data load order. Procedure 1. Copy the sample wc-dataload.xml file from the WCDE_installdirsamplesDataLoadCatalog directory into the WCDE_installdirsamplesDataLoadWarranty directory. Rename the file wc-tableobjectdataload.xml. 2. Open the new wc-tableobjectdataload.xmldata load order configuration file for editing. 3. Find the <_config:DataLoadEnvironment> element. Change the value for the configFile attribute to be your data load environment configuration file. For example, <_config:DataLoadEnvironment configFile="wc-dataload-env.xml" /> 4. Find the <_config:LoadOrder > element. Change the value for the businessObjectConfigFile attribute to be your business object configuration file. Change the value for the location element of the <_config:DataSourceLocation> configuration property to be the path for your input file. For example, <_config:LoadOrder > <_config:LoadItem commitCount="1" batchSize="1" dataLoadMode="Replace" name="Warranty" businessObjectConfigFile="wc-warrantytableobject-loader.xml"> <_config:DataSourceLocation location="WarrantyTableLoad.csv"/> </_config:LoadItem> </_config:LoadOrder> 5. Save your changes. Your file can resemble the following code: <?xml version="1.0" encoding="UTF-8"?> <_config:DataLoadConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.ibm.com/xmlns/prod/commerce/foundation/config ../../../../xml/config/xsd/wc-dataload.xsd" xmlns:_config="http://www.ibm.com/xmlns/prod/commerce/foundation/config"> <_config:DataLoadEnvironment configFile="wc-dataload-env.xml" /> <_config:LoadOrder > <_config:LoadItem commitCount="1" batchSize="1" dataLoadMode="Replace" name="Warranty" businessObjectConfigFile="wc-warrantytableobject-loader.xml"> <_config:DataSourceLocation location="WarrantyTableLoad.csv"/> </_config:LoadItem> </_config:LoadOrder> </_config:DataLoadConfiguration> Note: If you want to know more information on configuring your data load order, see Configuring the data load order. < Previous | Next >
  • 32. < Previous Run the Data Load utility In this lesson, you verify the configuration by running the Data Load utility to load data into the custom tables. Procedure 1. Optional: Specify the trace level, and enable console and file logging. a. Navigate to the workspace_dirWCxmlconfigdataload directory b. Open the logging.properties file for editing. c. Change the logging settings for running the Data Load utility. The following settings are used by default: Logging level is set to both console and file. Trace level is set to INFO. Log file location is set to WCDE_installdirlogswc-dataload.log. You do not need to modify the log file location. However, you might want to change the trace level when you are debugging a data load problem. The Data Load utility provides some command-line options to change the trace level. For more information, see the Data Load utility. An error log file is generated per load item when there are any errors. The error file name and location is specified in the summary report on the console. The error log file is in the same directory as the wc-dataload.log file. 2. Open WebSphere Commerce Developer. 3. Ensure that your WebSphere Commerce Server is stopped before you run the Data Load utility. For more information about stopping your server, see Starting and stopping WebSphere Commerce Test Server. 4. Open a command-line interface and navigate to the appropriate directory: WC_installdir/bin WCDE_installdirbin 5. Enter the command to run the Data Load utility to load your data load configuration file: ./dataload.sh ../samples/DataLoad/Warranty/wc-tableobjectdataload.xml dataload ..samplesDataLoadWarrantywc-tableobjectdataload.xml Where wc-tableobjectdataload.xml is the name of your data load configuration file. 6. Verify the results of the data load. You can verify your results by reviewing the summary and by checking that the data exists in the appropriate database tables. To check the database tables for the sample custom tables, run the following SQL statements. SELECT * FROM XWARRANTY; SELECT * FROM XCAREINSTRUCTION; If you see the warranty and care information that is included in the csv file that you loaded, your data load is successful. < Previous