Your SlideShare is downloading. ×
- 1 -
B07: Working With XML in IDS Applications
Keshava Murthy,
Architect, IBM, rkeshav@us.ibm.com
- 2 -
International Informix Users Group.2009 IIUG Informix Conference 2
XML in Application Environment
•XML is ubiquitous...
- 3 -
International Informix Users Group.2009 IIUG Informix Conference 3
Websphere
App server
XML
Planning
Optimization En...
- 4 -
International Informix Users Group.2009 IIUG Informix Conference 4
•Application design follows SOA
• Separation of b...
- 5 -
International Informix Users Group.2009 IIUG Informix Conference 5
HTTP/SOA
P
WebSphere Application
Server
IDS Web S...
- 6 -
International Informix Users Group.2009 IIUG Informix Conference 6
Functions
invoking the
XSLT4C and
XML4C libraries...
- 7 -
International Informix Users Group.2009 IIUG Informix Conference 7
What’s in 11?
• XML Publishing Functions
 Genxml...
- 8 -
International Informix Users Group.2009 IIUG Informix Conference 8
Genxml – simple example
select genxml(mycustrpt, ...
- 9 -
International Informix Users Group.2009 IIUG Informix Conference 9
Genxml – simple example
select genxmlelem(mycustr...
- 10 -
International Informix Users Group.2009 IIUG Informix Conference 10
Extract() and ExtractValue()
SELECT extract(col...
- 11 -
International Informix Users Group.2009 IIUG Informix Conference 11
ExtractClob() and ExtractValueClob()
SELECT ext...
- 12 -
International Informix Users Group.2009 IIUG Informix Conference 12
ExistsNode()
select docid from books
where exis...
- 13 -
International Informix Users Group.2009 IIUG Informix Conference 13
Transform Functions based on
XSLT: New in 11.5
...
- 14 -
International Informix Users Group.2009 IIUG Informix Conference 14
Functions
invoking the
XSLT4C and
XML4C librari...
- 15 -
International Informix Users Group.2009 IIUG Informix Conference 15
XSLT Processor
Xerces and
Xalan
XML
Document
HT...
- 16 -
International Informix Users Group.2009 IIUG Informix Conference 16
Using XSLT Transformation
> select * from t whe...
- 17 -
International Informix Users Group.2009 IIUG Informix Conference 17
Using XSLT Transformation
-- Transforming XML d...
- 18 -
International Informix Users Group.2009 IIUG Informix Conference 18
XSLT Transform Functions
•Three XSLT functions,...
- 19 -
International Informix Users Group.2009 IIUG Informix Conference 19
XSLT Transform Functions
Function name Paramete...
- 20 -
International Informix Users Group.2009 IIUG Informix Conference 20
Text Search Index for XML
- 21 -
International Informix Users Group.2009 IIUG Informix Conference 21
BTS Secondary Access Method
• To create an BTS ...
- 22 -
International Informix Users Group.2009 IIUG Informix Conference 22
Selecting Rows
• Selecting rows using the BTS i...
- 23 -
International Informix Users Group.2009 IIUG Informix Conference 23
• XML document: lusitania.xml
<?xml version="1....
- 24 -
International Informix Users Group.2009 IIUG Informix Conference 24
xmlpaths
• Xmlpaths can be absolute or relative...
- 25 -
International Informix Users Group.2009 IIUG Informix Conference 25
Relative xmlpaths in the example xml
documents
...
- 26 -
International Informix Users Group.2009 IIUG Informix Conference 26
Absolute xmlpaths in the example xml
documents
...
- 27 -
International Informix Users Group.2009 IIUG Informix Conference 27
Query Parser String (revisited) -
Fields
• You ...
- 28 -
International Informix Users Group.2009 IIUG Informix Conference 28
An example using xmlpath_processing
and xmltags...
- 29 -
International Informix Users Group.2009 IIUG Informix Conference 29
An example using xmlpath_processing
and all_xml...
- 30 -
International Informix Users Group.2009 IIUG Informix Conference 30
include_namespaces parameter
(cont.)• Given the...
- 31 -
International Informix Users Group.2009 IIUG Informix Conference 31
bts_index_fields() function
• Given a path to a...
- 32 -
International Informix Users Group.2009 IIUG Informix Conference 32
bts_index_fields() function (cont.)
• Another e...
- 33 -
International Informix Users Group.2009 IIUG Informix Conference 33
Searching XML Documents as part of
Basic Text S...
- 34 -
International Informix Users Group.2009 IIUG Informix Conference 34
HTTP/SOA
P
WebSphere Application
Server
IDS Web...
- 35 -
International Informix Users Group.2009 IIUG Informix Conference 35
Thank You
rkeshav@us.ibm.com
Upcoming SlideShare
Loading in...5
×

Working With XML in IDS Applications

718

Published on

Working With XML in IDS Applications

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
718
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • 1. Business integration scenarios. 2. Using XML to solve business integration issues. 3. IDS Cheetah Features for XML. 1. Publishing Functions 2. Functions evaluating XPATH. 4. Putting it all together. 5. Further thoughts on upcoming features.
  • Xerces (same as XSLT4C) is an XML Parser.
    Xalan (same as XSLT4C) is the transformer…includes XPath.
  • XSLT: new in Cheetah2
    Extensible Stylesheet Language Transformation
    Transforms XML documents
  • Transcript of "Working With XML in IDS Applications"

    1. 1. - 1 - B07: Working With XML in IDS Applications Keshava Murthy, Architect, IBM, rkeshav@us.ibm.com
    2. 2. - 2 - International Informix Users Group.2009 IIUG Informix Conference 2 XML in Application Environment •XML is ubiquitous and solves many problems • Application Integration • Data Centric Applications • Document Centric Applications • Heterogeneity • XML and RDBMS • Persistence • XML in and XML out • Application requirements • XML support in IDS • What’s planned? <? xml version=“1.0” ?> <purchaseOrder id=“12345” > <customer id=“A6789”> <name>John Smith Co</name> <address> <street>1234 W. Main</street> <city>Toledo</city> <state>OH</state> <zip>95141</zip> </address> </customer> …
    3. 3. - 3 - International Informix Users Group.2009 IIUG Informix Conference 3 Websphere App server XML Planning Optimization Engine Informix Dynamic Server (IDS) JDBC ODBC Carrier Integration Business Logic JDBC based Database Layer Lane Tracking Supplier integration Fetch Data TP: Bid winners TPE: Shipment MQ based Event handling Storage WebsphereMQ Procurement Optimization Engine Typical Transportation Application SOA
    4. 4. - 4 - International Informix Users Group.2009 IIUG Informix Conference 4 •Application design follows SOA • Separation of business optimization services from Integration Services • XML documents exchanged follows tXML (transport XML) • tXML documents are shred to relational tables • efficient storage • query performance • Application queries RDBMS to create and publish tXML documents/reports • tXML rarely changes - schema evolution isn’t a requirement, but performance is. Source: IBM SWG Architecture
    5. 5. - 5 - International Informix Users Group.2009 IIUG Informix Conference 5 HTTP/SOA P WebSphere Application Server IDS Web Service Provider J2EE Applications Direct Applications XML Publishing & Extraction with XPATH support XSLT Transformation XML Schema Repository And Shredding Functions XML aware FullText Index (with XPath Support) Websphere MQ Websphere MQ Integration ESQL/C, ODBC, .NET applications JDBC/JCC EGL Apps WORF/DAD X Apache Tomcat Informix Dynamic Server IDS XML Plans
    6. 6. - 6 - International Informix Users Group.2009 IIUG Informix Conference 6 Functions invoking the XSLT4C and XML4C libraries within IDS IDS Client IDS Client Extract functions extract() extracevalue() existsnode() idsXMLParse() Publishing functions genxml() genxmlelem() genxmlqueryhdr() … XSLT4C Processor XML4C Parser ICU Informix Dynamic Server XSLT4C and XML4C What’s in 11? Transform functions XSLTransform() XSLTransformAsCLOB() XSLTransformAsBLOB() New in 11.5
    7. 7. - 7 - International Informix Users Group.2009 IIUG Informix Conference 7 What’s in 11? • XML Publishing Functions  Genxml() and GenxmlClob()  Genxmlqueryhdr() and GenxmlqueryhdrClob()  Genxmlelem() and genxmlelemclob() • XPath functions for pattern matching and extraction within XML document  Extract() and Extractclob()  ExtractValue()  ExistsNode()
    8. 8. - 8 - International Informix Users Group.2009 IIUG Informix Conference 8 Genxml – simple example select genxml(mycustrpt, 'mycustomers') FROM (select c.customer_num, c.customer_name.last, o.order_num from customer c, orders o WHERE c.customer_num = o.customer_num and order_date = '05/20/1998') as mycustrpt; genxml <mycustomers customer_num="104" expression="Kilmore" order_num="1001"/> 1 row(s) retrieved. select genxml(mycustrpt, 'mycustomers') FROM (select c.customer_num, c.customer_name.last, o.order_num from customer c, orders o WHERE c.customer_num = o.customer_num and order_date = '05/20/1998') as mycustrpt; genxml <mycustomers customer_num="104" expression="Kilmore" order_num="1001"/> 1 row(s) retrieved. select genxml(mycustrpt, 'mycustomers') FROM (select first 5 c.customer_num, c.customer_name.last, o.order_num from customer c, orders o WHERE c.customer_num = o.customer_num and order_date < '01/01/1999') as mycustrpt; genxml <mycustomers customer_num="104" last="Kilmore" order_num="1001"/> <mycustomers customer_num="101" last="Mangal" order_num="1002"/> <mycustomers customer_num="104" last="Kilmore" order_num="1003"/> <mycustomers customer_num="106" last="Puente" order_num="1004"/> <mycustomers customer_num="116" last="Westerberg" order_num="1005"/> 1 row(s) retrieved.
    9. 9. - 9 - International Informix Users Group.2009 IIUG Informix Conference 9 Genxml – simple example select genxmlelem(mycustrpt, 'mycustomers') FROM (select c.customer_num, c.customer_name.last, o.order_num from customer c, orders o WHERE c.customer_num = o.customer_num and order_date = '05/20/1998') as mycustrpt; genxmlelem <mycustomers> <row> <customer_num>104</customer_num> <expression>Kilmore</expression> <order_num>1001</order_num> </row> </mycustomers> 1 row(s) retrieved. select genxmlelem(mycustrpt, 'mycustomers') FROM (select c.customer_num, c.customer_name.last, o.order_num from customer c, orders o WHERE c.customer_num = o.customer_num and order_date = '05/20/1998') as mycustrpt; … same result as above
    10. 10. - 10 - International Informix Users Group.2009 IIUG Informix Conference 10 Extract() and ExtractValue() SELECT extract(col2, '/personnel/person[3]/name/given') FROM tab; SELECT extractvalue(col2, '/personnel/person[3]/name/given') FROM tab; execute function extract(“<person><name><fname>john</fname><lnam e>Kelly</lname></person>”, “/person/name/lname”); select col1 from tab where existsnode(col2, '/personnel/person/*/email') = 1;
    11. 11. - 11 - International Informix Users Group.2009 IIUG Informix Conference 11 ExtractClob() and ExtractValueClob() SELECT extractclob(col2, '/personnel/person[3]/name/given') FROM tab_clob_neg; SELECT extractvalueclob(col2, '/personnel/person[3]/name/given') FROM tab_clob_neg;
    12. 12. - 12 - International Informix Users Group.2009 IIUG Informix Conference 12 ExistsNode() select docid from books where existsnode(booklist, '/bookstore/book[author="Kurt Cagle"]/title') = 1;
    13. 13. - 13 - International Informix Users Group.2009 IIUG Informix Conference 13 Transform Functions based on XSLT: New in 11.5 • eXtensible Stylesheet Language Tranformations • XSL and XSLT is part of W3C standard • XSLT is used to transform XML documents  XML to XML (transform to confirm different schema/standard)  XML to HTML  XML to PDF • XML is device independent representation of data • XSLT helps you to transform to customize display • XSLT helps you to transform from one XML schema into another. • XSLT enables loose integration thru transformation • XSLT enables same data to be publishable to multiple targets with unique requirements.
    14. 14. - 14 - International Informix Users Group.2009 IIUG Informix Conference 14 Functions invoking the XSLT4C and XML4C libraries within IDS IDS Client IDS Client Extract functions extract() extracevalue() existsnode() idsXMLParse() Publishing functions genxml() genxmlelem() genxmlqueryhdr() … XSLT4C Processor XML4C Parser ICU Informix Dynamic Server XSLT4C and XML4C What’s new in 11.5? Transform functions XSLTransform() XSLTransformAsCLOB() XSLTransformAsBLOB() New in 11.5
    15. 15. - 15 - International Informix Users Group.2009 IIUG Informix Conference 15 XSLT Processor Xerces and Xalan XML Document HTML Document Any other document Input XML document XSL Style Sheet Document XSLT: what is it?
    16. 16. - 16 - International Informix Users Group.2009 IIUG Informix Conference 16 Using XSLT Transformation > select * from t where id = 1; id 1 info <?xml version='1.0' encoding='ISO-8859-1' ?><doc>Hello world!</doc> style <?xml version='1.0'?> <xsl:stylesheet xmlns:xsl='http://www.w3.org/1999/ XSL/Transform' version='1.0'> <xsl:output encoding='US-ASCII'/> <xsl:te mplate match='doc'> <out><xsl:value-of select='.'/></out> </xsl:template > </xsl:stylesheet> 1 row(s) retrieved. -- info and style simply lvarchar columns. -- xsltransform() returns lvarchar. Max 32K > select xsltransform(info, style) from t where id = 1; (expression) <?xml version="1.0" encoding="UTF-8"?> <out>Hello world!</out> 1 row(s) retrieved.
    17. 17. - 17 - International Informix Users Group.2009 IIUG Informix Conference 17 Using XSLT Transformation -- Transforming XML documents stored in tables select XSLTransform(s.style_doc, x.xml_doc) FROM style_sheets s, xml_docs x WHERE s.style_title = 'ecommerce_ROOT.xsl' AND x.xml_transaction_id = 12345; -- Transforming dynamically generated XML documents select XSLTransform(s.style_doc, genxml(“transaction”, row(o.trnid, o.custid, o.addr))) FROM style_sheets s, orders o WHERE s.style_title = 'ecommerce_ROOT.xsl' AND o.trnid = 12345;
    18. 18. - 18 - International Informix Users Group.2009 IIUG Informix Conference 18 XSLT Transform Functions •Three XSLT functions, but they’re overloaded • XSLTransform() • XSLTransformAsCLOB() • XSLTransformAsBLOB() • They’re overloaded • The functions differ in parameters and return types
    19. 19. - 19 - International Informix Users Group.2009 IIUG Informix Conference 19 XSLT Transform Functions Function name Parameter1 (input XML doc) Parameter2 (input Stylesheet Returning type XSLTransform lvarchar lvarchar lvarchar XSLTransform CLOB lvarchar lvarchar XSLTransform lvarchar CLOB lvarchar XSLTransform CLOB CLOB lvarchar XSLTransform BLOB lvarchar lvarchar XSLTransform lvarchar BLOB lvarchar XSLTransformAsCLOB lvarchar lvarchar CLOB XSLTransformAsCLOB CLOB lvarchar CLOB XSLTransformAsCLOB lvarchar CLOB CLOB XSLTransformAsBLOB lvarchar Lvarchar BLOB XSLTransformAsBLOB BLOB varchar BLOB XSLTransformAsBLOB lvarchar BLOB BLOB
    20. 20. - 20 - International Informix Users Group.2009 IIUG Informix Conference 20 Text Search Index for XML
    21. 21. - 21 - International Informix Users Group.2009 IIUG Informix Conference 21 BTS Secondary Access Method • To create an BTS index, specify the bts secondary access method with the USING clause on the CREATE INDEX • A BTS index is required to perform text searches using the bts_contains() predicate • Required to specify the opclass that is specific to the data type of the column being indexed • Requires an extent space specified with the IN Clause (no default) • Example: CREATE INDEX books_idx ON books(description bts_lvarchar_ops) USING bts IN bts_extspace; Acknowledgement: Slides on BTS functionality were created by Mark Ashworth, Developer of BTS datablade.
    22. 22. - 22 - International Informix Users Group.2009 IIUG Informix Conference 22 Selecting Rows • Selecting rows using the BTS is done with the bts_contains() UDF bts_contains(column, query-parse-string) • The bts_contains() is a Boolean UDF. The first parameter, column, is the column being search and the second parameter, query-parse-string, is the Lucene query string • A BTS index must be created on the column being searched with bts_contains() UDF
    23. 23. - 23 - International Informix Users Group.2009 IIUG Informix Conference 23 • XML document: lusitania.xml <?xml version="1.0"?> <ship> <Name type="oceanliner">RMS Lusitania</Name> <owners> Cunard Line</owners> <builders> John Brown &amp; Company, Limited</builders> <maiden_voyage> 7 September, 1907</maiden_voyage> <sank> <date> 7 May, 1915 </date> <how> by German submarine U-20. </how> </sank> </ship> • XML document: andrea_doria.xml <?xml version="1.0"?> <ship> <Name type="oceanliner">SS Andrea Doria</Name> <owners> Italia Line</owners> <builders> Ansaldo, Sestri Ponente yards, Italy</builders> <maiden_voyage> 14 January, 1953</maiden_voyage> <sank> <date> 26 July, 1956 </date> <how> after colliding with SS Stockholm </how> </sank> </ship> XML Sample Documents
    24. 24. - 24 - International Informix Users Group.2009 IIUG Informix Conference 24 xmlpaths • Xmlpaths can be absolute or relative  Absolute paths start with a / and relative paths do not  very basic X Path support with location paths  like a directory path in a filesystem
    25. 25. - 25 - International Informix Users Group.2009 IIUG Informix Conference 25 Relative xmlpaths in the example xml documents • Relative Paths ship/Name ship/owners ship/builders ship/maiden_voyage sank/date sank/how ship/sank/date ship/sank/how  And all the tags previously listed (Name, owners, builders, maiden_voyage, date, how) <?xml version="1.0"?> <ship> <Name type="oceanliner">RMS Titanic</Name> <owners> White Star Line, Liverpool, England</owners> <builders> Harland &amp; Wolff, Belfast, Ireland</builders> <maiden_voyage> 10 April, 1912</maiden_voyage> <sank> <date> 15 April, 1912 </date> <how> after hitting an iceberg on April 14, 1912 </how> </sank> </ship> ship Name owner builders maiden_voyage sank date how
    26. 26. - 26 - International Informix Users Group.2009 IIUG Informix Conference 26 Absolute xmlpaths in the example xml documents • Absolute Paths /ship/Name /ship/owners /ship/builders /ship/maiden_voyage /ship/sank/date /ship/sank/how <?xml version="1.0"?> <ship> <Name type="oceanliner">RMS Titanic</Name> <owners> White Star Line, Liverpool, England</owners> <builders> Harland &amp; Wolff, Belfast, Ireland</builders> <maiden_voyage> 10 April, 1912</maiden_voyage> <sank> <date> 15 April, 1912 </date> <how> after hitting an iceberg on April 14, 1912 </how> </sank> </ship> ship Name owner builders maiden_voyage sank date how
    27. 27. - 27 - International Informix Users Group.2009 IIUG Informix Conference 27 Query Parser String (revisited) - Fields • You can search any field by typing the field name followed by a colon ":" and then the term you are looking for. • For example: bts_contains(document, 'Name:rms and owner:"White Star Line"')  Here we search the field "Name" for the word "rms" and the field "owner" for the phrase "White Star Line"
    28. 28. - 28 - International Informix Users Group.2009 IIUG Informix Conference 28 An example using xmlpath_processing and xmltags• Index the xmlpaths: ship/Name and ship/sank/how create index ships_bts on ships(document bts_clob_ops) using bts(xmltags="(ship/Name,ship/sank/how)",xmlpat h_processing="yes") in bts_extspace; • For the titanic.xml document, the following fields would be created: ship/Name:RMS Titanic ship/sank/how:after hitting an iceberg on April 14, 1912 • Query example to find ships that sank because of icebergs select name from ships where bts_contains(document, 'ship/sank/how:iceberg*'); name Titanic 1 row(s) retrieved.
    29. 29. - 29 - International Informix Users Group.2009 IIUG Informix Conference 29 An example using xmlpath_processing and all_xmltags • Index the all absolute xmlpaths: create index ships_bts on ships(document bts_clob_ops) using bts(all_xmltags="yes",xmlpath_processing="yes") in bts_extspace; • For the titanic.xml document, the following fields would be created: /ship/Name:RMS Titanic /ship/owners:White Star Line, Liverpool, England /ship/builders:Harland & Wolff, Belfast, Ireland /ship/maiden_voyage:10 April, 1912 /ship/sank/date:15 April, 1912 /ship/sank/how:after hitting an iceberg on April 14, 1912 • Query example to find ships that sank because of icebergs: select name from ships where bts_contains(document, '/ship/sank/how:iceberg*'); name Titanic 1 row(s) retrieved.
    30. 30. - 30 - International Informix Users Group.2009 IIUG Informix Conference 30 include_namespaces parameter (cont.)• Given the xml fragment: <book> <book:title>Graph Theory</book:title> <author>Stewart</author> <date>January 14, 2006</date> </book> • For the given xml fragment, the following three fields would be created: /book/book:title:graph theory /book/author:stewart /book/date:january 14, 2006 • Query: bts_contains(xml_data, '/book/book:title:theory')  Note, because the colon is used to delineate a field, the colon used to signify the namespace must be escaped with a backslash (‘’)
    31. 31. - 31 - International Informix Users Group.2009 IIUG Informix Conference 31 bts_index_fields() function • Given a path to an index path (like bts_index_compact), it returns a list of fields that currently exist in the index • For example: create index ships_bts on ships(document bts_clob_ops) using bts(all_xmltags="yes",xmlpath_processing="yes") in bts_extspace; Index created. execute function bts_index_fields('/local18/clucene_index/testdb/ashwort h/ships_bts')); (expression) /ship/Name /ship/builders /ship/maiden_voyage /ship/owners /ship/sank/date /ship/sank/how 1 row(s) retrieved.
    32. 32. - 32 - International Informix Users Group.2009 IIUG Informix Conference 32 bts_index_fields() function (cont.) • Another example, with xmlpath_processing disabled and include_contents enabled: create index ships_bts on ships(document bts_clob_ops) using bts(all_xmltags="yes",xmlpath_processing="no", include_contents="yes") in bts_extspace; Index created. execute function bts_index_fields('/local18/clucene_index/testdb/ashworth/ships_bts') ); (expression) Name builders contents maiden_voyage owners date how 1 row(s) retrieved.
    33. 33. - 33 - International Informix Users Group.2009 IIUG Informix Conference 33 Searching XML Documents as part of Basic Text Search• Customer can now generate their own stopword lists • Can indicate which xml tags to index at index create time.  Values get indexed, not tags.  Also options to index all tags as well as treat tags as text. • Can now limit searches to only certain XML tags • “XPath" queries supported in search.  Search string can contain a path to search /books/title/chapter/....  Path can be relative or fixed. • tags can be completely ignored  XML doc treated as a set of values  Looses the ability to search by tags • You can specify whether the namespace should be required in searches • Markup in sentences can be ignored  "foo <bold> bar </bold>" can either be indexed as the single "foo bar" or as "foo" and "bar". This matters for proximity searches for instance. • New function provided which returns all the unique tags in the XML doc.
    34. 34. - 34 - International Informix Users Group.2009 IIUG Informix Conference 34 HTTP/SOA P WebSphere Application Server IDS Web Service Provider J2EE Applications Direct Applications XML Publishing & Extraction with XPATH support XSLT Transformation XML Schema Repository And Shredding Functions XML aware FullText Index (with XPath Support) Websphere MQ Websphere MQ Integration ESQL/C, ODBC, .NET applications JDBC/JCC EGL Apps WORF/DAD X Apache Tomcat Informix Dynamic ServerIDS XML Plans
    35. 35. - 35 - International Informix Users Group.2009 IIUG Informix Conference 35 Thank You rkeshav@us.ibm.com

    ×