Dynamic Publishing with Arbortext Data Merge


Published on

Explains how Arbortext Data Merge feature lets you publish live database data in XML documents

  • Be the first to comment

Dynamic Publishing with Arbortext Data Merge

  1. 1. Dynamic Publishing with ArbortextData MergeClay HelbergTerraXML
  2. 2. What is Data Merge?
  3. 3. What is Data Merge?
  4. 4. Why Data Merge?• Any time you need to publish data managed in a database• Examples: – Price lists – Employee directories – Research reports – Account statements
  5. 5. Advantages Over Cut-and-Paste• Avoids tedium of cut-and-paste• Less error-prone• Automatic updating—set and forget
  6. 6. How Does It Work?• Admin/Application Developer – Configure database as an ODBC Data Source – Define queries for your application that extract the needed data• Author – Insert queries in the documents and set the update preferences – Publish the document
  7. 7. Configuring an ODBC Data Source• ODBC = Open Data Base Connectivity – A standard system for letting DB’s to talk to each other – Define a Data Source to provide an access point for your database
  8. 8. Data Source Setup• Details depend on Database type• If unsure, check with DB Admin
  9. 9. Setting up Queries• Defining a Query – Written using Structured Query Language (SQL) • Common database language – Specify the table (FROM clause) – Select specific fields (or * for all fields) • Can also use table.field format, useful when doing joins – Select specific rows (WHERE clause) – Calculations (SUM, AVG, etc) • Usually used with GROUP BY
  10. 10. Parameterizing Queries• Hard-coded queries lack flexibility• Using parameters makes them more flexible• Authors use UI to select parameter values
  11. 11. How to Add Queries to the ArbortextCustom Application1. Add Data Merge info to the DCF file2. Create the DMF file3. Set up the ODBC connection string – Tells which data source to use, which user account to use, etc.4. Add the database query code (SQL)5. Set up any UI items for query parameters6. Decide how to handle the data returned from the database
  12. 12. DCF Settings for Data Merge• Add <DataMerge> element – Contains <DataSource> element • config attr gives name of query file • name attr gives a name for the data source – Multiple <DataSource> elements can be used
  13. 13. Create the DMF file• Tip: start with an existing file! – Look in $ARBORTEXT/Editor/samples/datamerge• Put it in $APTCUSTOM/datamerge directory• Make sure name matches DataMerge/DataSource/@config value in DCF
  14. 14. Specifying the Connection String• In the file find the <ParameterGroupRef> where nameref=“connectionProperties”• Set the child <Value> to the connection string• Connection generally string consists of jdbc:odbc:<DSN name> – Syntax varies by database type; check with DBA for details• For some DB’s, you can also include login credentials – E.g. jdbc:oracle:thin:jdoe/passwd@localhost:1521:XE – Stored as cleartext; be careful of security concerns – If you do this, consider using a dummy acct with read-only privileges
  15. 15. Other Connection Properties• connectionProperties parameters let you pass values to the database – For simple JDBC data sources (e.g. Excel), the URL is all you need – For other DB’s you can pass info such as username and password (if not embedded in URL) – See comments in $ARBORTEXT/Editor/datamerge/*.ent corresponding to your database type for details on available and required connection parameters
  16. 16. Adding the SQL Query• SQL Query goes inside ParameterRef/Value where nameref=―sqlStatement‖: <ParameterRef name="p_statement" nameref="sqlStatement"> <Documentation>This is a source filter that retrieves data from the Excel database using jdbc-odbc bridge.</Documentation> <Value>select * from [Employees$] where DEPARTMENT_NAME = ?</Value> </ParameterRef>• Note the question mark—that denotes a query parameter. It will be replaced by the author’s selection when the query is inserted.
  17. 17. Defining Query Parameters• For each ? in the query, define parameter in queryParameters parameterGroupRef (in DMF): <!-- Query Parameters --> <ParameterGroupRef name="queryParameters” nameref="sqlParameters“> <ParameterRef datatype="string" name="pdesc“> <Documentation>This parameter defines the values to be passed to the SQL statement; in this case, the department name to select</Documentation> <QueryFieldRef nameref="p01"></QueryFieldRef> </ParameterRef> </ParameterGroupRef>• You can just give names sequentially, i.e. p01, p02, etc.
  18. 18. Setting UI for Query Parameters• <Interface> section of DMF• Types of controls: Text, Password, Combo, List, File chooser• Where appropriate, list of possible values for the parameter
  19. 19. Query UI Definition Example<Interface> <Documentation>The Interface section contains query parameters to be shown in the insert query dialog box. This query defines one parameter which is shown as a combo box in the query dialog box.</Documentation> <QueryField displayType="combo" name="p01" value="Accounting"> <Label>Department</Label> <ListItem>Accounting</ListItem> <ListItem>Administration</ListItem> <ListItem>Executive</ListItem> … </QueryField></Interface>• The name attribute matches the UI to the query parameter
  20. 20. Handling Resultsets• Two different ways to format resulting data for publishing: – As a straightforward table – Using arbitrary formatting via XSL transformation
  21. 21. Formatting Data as a Table• This is built-in, no additional work to do
  22. 22. Applying Arbitrary Formatting• Using an XSL transformation, you can do anything you want with the resultset data – Advantage: complete control – Downside: requires additional work to develop the XSLT stylesheet that handles the transformation• In DMF, set <Query queryType=“freeform”>• Note: XSLT result tree should have a single container element
  23. 23. Resultset XML Structure<atidm:ResultSet> <atidm:RowHeader> <atidm:Header>#PCDATA</atidm:Header> … </atidm:RowHeader> <atidm:Row> <atidm:Column><atidm:Value>#PCDATA</atidm:Value></atidm:Column> … </atidm:Row></atidm:ResultSet>• See $ARBORTEXT/Editor/doctypes/resultset/resultset.dtd for details
  24. 24. Applying Arbitrary Formatting
  25. 25. Completing the Query with Output Info• Defining resultset formatting info in the query file – For table output: <TransformerRef name="s1" nameref="TableModelTransformer"> <ParameterRef nameref="tableModel"> <Value>OASIS Exchange</Value> </ParameterRef> <ParameterRef nameref="wrapperTag"> <Value>table</Value> </ParameterRef> <ParameterRef nameref="headerBGColor"> <Value>blue</Value> </ParameterRef> </TransformerRef> – Also, set <Query queryType=“table”>
  26. 26. Completing the Query with Output Info• For XSL transformation output:<!-- Transformer Parameters --><TransformerRef name="t1" nameref="XSLTransformer"> <ParameterRef name="p_stylesheet" nameref="stylesheet"> <Value>Z:aptcustomdatamergedatamerge.xsl</Value> </ParameterRef></TransformerRef>• Also, set <Query queryType=“freeform”>
  27. 27. Multiple Queries• You can set up multiple queries in a single query file – Queries hitting different databases – Queries that produce different output formatting – Queries that select different subsets of fields and/or records• Each query has a unique name and description
  28. 28. Using a Query in a Document• In document, position cursor, select Insert->Markup, set Mode to Query• Select the desired query• Set any required parameters for the query• Set update preferences – Manual Update – Auto Update on Open – Auto Update on Compose
  29. 29. Data Preview• After running the query, you should see a preview of the imported data
  30. 30. Populated Document• After inserting the query, you should see the data in the document
  31. 31. Manual Updates• Put cursor inside the query result• From the menus choose Tools Data Merge Update Current (or Update All)• You can also ―flatten‖ the query, i.e. remove the query but leave the data as it currently exists in the document
  32. 32. Walking through an Example
  33. 33. Additional Resources• Intro to Databases – http://www.quackit.com/database/tutorial/• ODBC overview – http://msdn.microsoft.com/en- us/library/ms710220(VS.85).aspx• SQL tutorial – http://www.w3schools.com/sql/default.asp• XSLT resource – http://www.w3schools.com/xsl/• Adepters list (via PTC/USER portal) – http://portal.ptcuser.org, under ―Forums‖
  34. 34. Q&A• Contact info: – Clay Helberg TerraXML chelberg@terraxml.com www.terraxml.com