The document discusses the evolution of XML tools, focusing on third-generation tools that enhance programming models and performance, particularly through schema to Java generation. It highlights the simplification of the development process by reducing the need for XML experience while improving type checking and reliability. Performance comparisons of various XML processing approaches, including streaming and DOM, are also provided, demonstrating memory efficiency and execution time.

1. Third Generation XML Tools Michael Leventhal [email_address]

2. Generations Ubiquity Infrastructure Programming Standard Correctness Performance Simplification Commodity Cost 3rd 2nd 1st Reliability Delivery Application Repository Standardization Data Description Toolkits Transformation Interfaces Standards Presentation Tiers Tools Parsing Browsing Editing

3. Agenda Characterizing XML Tool Evolution – Semantic Space of Change Key Ideas Underlying the Next Generation XML Programming Models – Universal XML Development High Performance XML, e.g., Streaming XML as Infrastructure – Reliability at the core of the computing platform, merging of hardware and software, integration into the application framework Whither else? Your feedback.

4. Schema to Java Generation Lessens development time and resource needs through removal of required of XML experience needed for generic XML APIs such as DOM and SAX. Superior type checking; most constraint and structure checking is done at compilation time as the Java classes have been generated from the schema. Performance worse than DOM with Validation Generation of Java APIs from XML Schemas Developers program against these Schema-specific APIs

5. Schema to Java - Tools JAXB Implementation http://java.sun.com/xml/ Castor http://www.castor.org/ XML Binder for Java http://www.breezefactor.com Zeus http://zeus.enhydra.org/ Schema2Java http://www.creativescience.com DocSOAP XDK XGen http://www.commerceone.com/developer

6. Schema to Java Generation XML Schema Java Classes Generates XML Instance Java Objects Converts XGen DT Schema Compiler XML Schema Java Classes XGen RT Runtime XML Instance Java Objects

7. Schema to Java Example <xsd:schema targetNamespace=&quot;rrn:com:commerceone:schemas/xdk/samples/Car.xsd&quot; xmlns=&quot;rrn:com:commerceone:schemas/xdk/samples/Car.xsd&quot; xmlns:xsd=&quot;http://www.w3.org/2001/XMLSchema&quot;> <xsd:element name=&quot;Car&quot; type=&quot;CarType&quot;/> <xsd:complexType name=&quot;CarType&quot;> <xsd:sequence> <xsd:element name=&quot;Manufacturer&quot; type=&quot;xsd:string&quot;/> <xsd:element name=&quot;Model&quot; type=&quot;xsd:string&quot;/> <xsd:element name=&quot;Year&quot; type=&quot;xsd:int&quot;/> <xsd:element name=&quot;Color&quot; type=&quot;xsd:string&quot;/> </xsd:sequence> <xsd:attribute name=&quot;ForSale&quot; type=&quot;xsd:boolean&quot;/> </xsd:complexType> </xsd:schema> <?xml version=&quot;1.0&quot;?> <prefix_0:Car xmlns:prefix_0=&quot;rrn:com.commerceone:schemas/xdk/samples/Car.xsd&quot; xmlns:xsi=&quot;http://www.w3.org/2001/XMLSchema-instance&quot; ForSale=&quot;true&quot;><Manufacturer>Mazda</Manufacturer><Model>Miata MX-5</Model><Year>91</Year><Color>Blue</Color></prefix_0:Car>

8. <xsd:element name=&quot;Car&quot; type=&quot;CarType&quot;/> <xsd:complexType name=&quot;CarType&quot;> <xsd:sequence> <xsd:element name=&quot;Manufacturer&quot; type=&quot;xsd:string&quot;/> <xsd:element name=&quot;Model&quot; type=&quot;xsd:string&quot;/> <xsd:element name=&quot;Year&quot; type=&quot;xsd:int&quot;/> <xsd:element name=&quot;Color&quot; type=&quot;xsd:string&quot;/> </xsd:sequence> <xsd:attribute name=&quot;ForSale&quot; type=&quot;xsd:boolean&quot;/> </xsd:complexType> </xsd:schema> <?xml version=&quot;1.0&quot;?> <prefix_0:Car xmlns:prefix_0=&quot;rrn:…/Car.xsd“ ForSale=&quot;true&quot;> <Manufacturer>Mazda</Manufacturer> <Model>Miata MX-5</Model> <Year>91</Year><Color>Blue</Color> </prefix_0:Car>

9. Schema to Java Example package com.commerceone.samples.xdk; import com.commerceone.schemas.xdk.samples.Car_xsd.Car; import com.commerceone.schemas.xdk.samples.Car_xsd.types.CarType; import com.commerceone.xdk.castor.types.*; import com.commerceone.xdk.castor.excp.ConstraintValidationException; import java.io.OutputStreamWriter; import java.io.Writer; public class CarCreator { public static void main(String[] args) { try { CarCreator cc = new CarCreator(); Car car = cc.makeCar(); car.marshal(new OutputStreamWriter(System.out)); } catch(Exception e) { e.printStackTrace(); } } public Car makeCar() throws ConstraintValidationException { Car car = new Car(); CarType ct = car.getCarTypeComplexType(); ct.setColor(new XString(&quot;Blue&quot;)); ct.setManufacturer(new XString(&quot;Mazda&quot;)); ct.setModel(new XString(&quot;Miata MX-5&quot;)); ct.setYear(new XInt(91)); ct.setForSale_Attribute(new XBoolean(true)); return car; } }

10. public class CarCreator { public static void main(String[] args) { try { CarCreator cc = new CarCreator(); Car car = cc.makeCar(); car.marshal(new OutputStreamWriter(System.out)); } catch(Exception e) { e.printStackTrace(); } } public Car makeCar()throws ConstraintValidationException { Car car = new Car(); CarType ct = car.getCarTypeComplexType(); ct.setColor(new XString(&quot;Blue&quot;)); ct.setManufacturer(new XString(&quot;Mazda&quot;)); ct.setModel(new XString(&quot;Miata MX-5&quot;)); ct.setYear(new XInt(91)); ct.setForSale_Attribute(new XBoolean(true)); return car; } }

11. Schema to Java in Web Services DocSOAP<->Java SOAP-RPC<->Java

12. Schema to Java – Schema Feature Coverage

13. Schema to Java – Schema Feature Coverage

14. Schema to Java - Performance DOM with Validation Illustrative Schema to Java Tool CPU Secs Size Marshalling Unmarshalling Round Trip First run Average after first run First run Average after first run First run Average after first run 1 Mb 1.8 1.3 19.9 10.4 21.7 11.7 3 Mb 4.9 3.7 41.0 29.9 46.0 33.6 5 Mb 8.9 7.0 60.1 50.5 69.0 57.5 Size Marshalling Unmarshalling Round Trip First run Average after first run First run Average after first run First run Average after first run 1 Mb 1.0 0.4 5.8 3.5 6.8 3.9 3 Mb 3.0 1.3 13.2 10.6 16.2 11.9 5 Mb 4.2 3.2 20.8 17.5 25.1 20.7

15. Schema to Java - Performance Memory Kbs Illustrative Schema to Java Tool Rule of thumb for DOM: Memory use ~4X size of instance Init Unmarshalling Marshalling Peak Done Peak Done Peak Done xCBL 1 meg 1730 1673 9845 9553 15455 9574 xCBL 3 meg 1734 1673 20764 17657 27765 17677 xCBL 5 meg 1769 1674 31556 25746 39183 25766 xCBL3.0 1 meg 1802 1674 23344 16835 24751 16875 xCBL3.0 3 meg 1790 1673 33142 25830 39134 25870 xCBL3.0 5 meg 1803 1673 45113 34700 52206 34741

16. Programming Models: XOBE XML Schema XML Objects Instances of Generated by XML constructors http://www.research.avayalabs.com/user/wadler/planx/planx-eproceed/papers/E00-699879232.pdf See Kempa, Martin and Linnemann, Volker, “On XML Objects” Car makeCar() { Car myCar; myCar = <Car> <Manufacturer>Mazda</Manufacturer> <Model>Miata MX-5</Model> <Year>91</Year> <Color>Blue</Color> </Car>; Return myCar; } Uses XML Schema directly as definition of classes for XML objects; construct XML objects with parameterized XML expressions Eliminates distinction between string and object representation of XML documents Compiler statically guarantees validity; no need for special runtime checking or checking at marshalling Similar: ECMAScript (BEA XML Scripting), Water (http://www.waterlang.org)

17. Car makeCar() { Car myCar; myCar = <Car> <Manufacturer>Mazda</Manufacturer> <Model>Miata MX-5</Model> <Year>91</Year> <Color>Blue</Color> </Car>; Return myCar;

18. Stream Processing - Tools Builds complex XML processing on SAX events to achieve high-performance and ultra-low memory usage compared to tree-construction approaches XSLT-style Transformation STX http://stx.sourceforge.net/ Stream Index (SIX) XST Smart Streaming XPath XMLTK http://sourceforge.net/projects/ xmltk χαος http://www.research.ibm.com/xaos/applications.html SOAP Processing Axis http://ws.apache.org/axis/ DocSOAP XDK http://www.commerceone.com/developer

19. Stream Processing-Algorithms Stream processing is not inherently faster than DOM tree-based approaches; improperly applied it can be slower than DOM. It should always be more memory-efficient than DOM. Some approaches consist mainly of limiting the application domain to problems that can usually be efficiently solved by a simple streaming algorithm. Example: XPP, simplified XML parser. Stream processing is not necessarily just using SAX events. It is a different approach to solving computational problems which uses a variety of sophisticated algorithms. SIX (XMLTK) uses event marker indices. Smart Streaming (XST) combines DOM and SAX and analyzing when to stream X-dag ( χαος) converts backward constraints into forward constraints

20. Stream Processing – Simplifying the problem Source: IBM Developer Works, XML in Java: Document models, Part 1: Performance, Dennis M. Sosnoski, September 2001 XPP: XML Pull Parser Streaming XML Subset No Validation No entities, PIs, Comments Lazy, on-demand tree Supports namespaces http://www.extreme.indiana.edu/ xgws/xsoap/xpp/ Document Modify

21. Stream Processing - XMLTK XMLTK Stream Processing Tools Stream Function Description xsort Sorts an XML Stream xagg Computes an aggregate xnest Groups elements based on key equality or number xflatten Flattens collections xdelete Removes elements or attributes xpair Replicates an element multiple times, Pairing it with each element in a collection xhead Retains only a prefix of a collection xtail Retains only a suffix of a collection file2xml Generates an XML stream for the dir file directory hierarchy

22. Stream Processing - XMLTK XMLTK XSort Performance vs Xalan Sort Sophisticated streaming algorithm can produce both low memory and high performance S tream I nde X Algorithm Additional binary stream consisting of beginOffset, endOffset pairs for each XML element Allows characters to be skipped Small offsets can be skipped to reduce SIX stream and number of jumps Data Size (KB) Xalan (sec) Xsort (sec) 0.41 0.08 0.00 4.91 0.10 0.00 76.22 0.29 0.03 991.79 2.78 0.35 9671.42 29.42 3.54 100964.43 - 35.52 1009653.71 - 358.47

23. Smart Streaming <ContractReferences> <ListOfContract> <Contract> <ContractID> <Identifier> <Agency> …… </Agency> <Ident>0088-858398</Ident> </Identifier> </ListOfContract> </ContractReferences> <Identifier> <Agency> …… <Ident>0088-858396</Ident> </Identifier> <Agency> …… </Agency> <Ident>0088-858398</Ident> Source XML Document <ContractReferences> <ListOfContract> <Contract> <ContractID> <Identifier> <Agency> …… </Agency> <Ident>0088-858398</Ident> </Identifier> </ListOfContract> </ContractReferences> <Identifier> <Agency> …… </Agency> <Ident>0088-858396</Ident> </Identifier> <Agency> …… </Agency> <Ident>0088858398</Ident> Target XML Document

24. Smart Streaming  Streaming  Streaming  Buffering  Streaming < ContractReferences > <ListOfContract> <Contract> <ContractID> <Identifier> <Agency> …… </ Agency> <Ident>0088-858398</I dent> </ Identifier> </ListOfContract> < /ContractReferences > < Identifier > <Agency> …… <Ident>0088-858396</Ident> < /Identifier > < Agency > …… < /Agency > < Ident >0088-858398 </Ident >

25. Smart Streaming - XST <xst:templates version=&quot;1.0&quot; …… <xst:template name=&quot;Agency&quot; deferred=&quot;true&quot;> <Ident><xst:val-of select=&quot;$Ident“></Ident> <xst:var name=&quot;AgencyDescription&quot;/> <xst:if test=&quot;$AgencyDescription&quot;> <AgencyDescription> <xst:val-of select=&quot;$AgencyDescription&quot;/> </AgencyDescription> </xst:if> </xst:template> …… </xst:templates> <xst:templates version=&quot;1.0&quot; …… <xst:template name=&quot;Ident&quot;> <xst:var name=&quot;Ident&quot;/> </xst:template> …… </xst:templates> <?xml version=&quot;1.0&quot;?> <xsl:stylesheet version=&quot;1.0&quot; xmlns:xsl=&quot;http://www.w3.org/1999/XSL/Transform&quot;> <xsl:template match=&quot;/&quot;> <xsl:apply-templates select = &quot;*&quot;/> </xsl:template> <xsl:template match= &quot;*&quot; > <xsl:copy><xsl:copy-of select=&quot;@*&quot;/> <xsl:apply-templates/> </xsl:copy> </xsl:template> <xsl:template match=&quot;Agency&quot;> <Agency> <Ident><xsl:value-of select=&quot;//Ident&quot;/></Ident> <xsl:variable name=&quot;AgencyDescription&quot; select=&quot;//AgencyDescription&quot;/> <xsl:if test=&quot;$AgencyDescription&quot;> <AgencyDescription><xsl:value-of select=&quot;$AgencyDescription&quot;/></AgencyDescription> </xsl:if> </Agency> </xsl:template> <xsl:template match=&quot;Ident&quot;/> </xsl:stylesheet> XST XSLT

26. Streaming – Transformation Automation Transformation Rules Schema Analysis Template Generator   Streaming Templates  Template Generation Template generation process can automate creation of streaming transformations. Transformation rules can be created in a visual tool like Contivo. Schema is analyzed to augment mapping rules. May be unnecessary to introduce YATL (Yet Another Transformation Language).

27. Streaming – Memory/CPU Performance

28. Streaming – Memory/CPU Performance

29. Web Services Infrastructure By one estimate, less than 2% of all enterprise networks traffic in 2002 was XML. That will increase to about 25% of all LAN traffic by 2006. Source: ZapThink, “Solving The Enterprise XML Processing Problem” Much of that traffic will be due to Web Services. XML-based SOAP should replace most other message-oriented protocols.

30. Web Services Infrastructure Implications for 3 rd Generation Tools XML has to become protocol-level safe and truly interoperable. There is a long way to go but W3C, WS-I and other consortiums are focusing intently on this. High performance and scalability will be at a premium. Market will bear higher prices for high-end tools. Interoperability and performance-related simplifications of XML and related standards will receive a second look.

31. Web Services Infrastructure Major Interoperability Issues Faced Today - Many Support for XML Schema Support for Namespaces Canonicalization of XML Documents XML Signing Links and References between Message Parts Handling of Document Subsets WSDL Binding to SOAP

32. Web Services Infrastructure Case Study: DocSOAP XDK Embedded in the Commerce One Conductor Web Services Platform First implementation of an XML document-centric Web Services toolkit High performance handling of large XML document message parts Highest level of support of XML Schema, namespaces, document fragments, inter-message references Major design emphasis on ensuring integrity of messages

33. Web Service Infrastructure transport Transport headers SOAP Header SOAP Payload Header Block D O C S O A P D O C U M E N T D O C U M E N T FRAMEWORK FRAMEWORK XGen RT UniParser A P P L I C A T I O N Generated Schema API DOM Interface Schema DOM Bean Instance Schema unmarshall marshall validate XGen DT Commerce One Conductor

34. Web Services Infrastructure Images provided by and used with the permission of DataPower Technology Case Study: DataPower Network Devices (XS40 XML Security Gateway and XA3 XML Accelerator) High-speed XML Encryption and Signing and XSLT transformation implemented in hardware and firmware Integrated with Altova and Excelon IDEs – applications leverage a common runtime in the network, software and hardware converge

35. Web Services Infrastructure Common Layer of XML Application Services, Development Tools and Runtimes in Software Frameworks such as .NET and J2EE Sun JavaX – the JAXes JAXM - Messaging JAXP - Parsing JAX-RPC – Distributed Processing JAXB – Schema to Java Generation JAXR – XML Registries

36. Not Hot? (Debatable) Native XML Databases/XML-DBMS Convergence Publishing/XSL-FO Client apps/ECMAScript-JavaScript/ embedded Web Services Browsable XML Semantic Web/RDF/Topic Maps/Ontologies

37. Presentation An HTML version of this presentation is available at: http://www.textscience.com/papers/ThirdGenerationXMLTools.html Powerpoint at: http://www.textscience.com/papers/ThirdGenerationXMLTools.ppt The presenter, Michael Leventhal, can be reached at michael@textscience.com.