5. Disclaimer The following are “Rules of Numb” Every environment has its own unique criteria and needs… Pay attention to: “Maintainability” “Extendibility” “Choice” “Design” “Testing” “Performance”
6. “XML is not a ‘fast’ thing, there is a ton of parsing involved. Sorry, I never saw the point in huge XML files – they are many times larger than they should be and the amount of work involved in parsing them is incredible”. Tom Kyte - Januari 9, 2009, AskTom
7. If you’re a performance nerd, this is actually cool… Lots of ground to cover regarding “XML”… Solving the customer problem… Back to basics… New solutions via XML Deeper understanding of the data handling issues… So why the “….” XML…?
9. Oracle XML Offering Oracle Apps Oracle Grid Control Oracle Fusion Middleware Oracle Database J2EE Integration Wireless RAC WebService WebCache BI & Reports DataPump OracleXML DB Portal OID LDAP C/C++ XML Utilities (oraxml10.lib) XSQL TransX Java XML Utilities (xml.jar) (XSU) JSR170 Class Generator XMLDiff XMLDiff JAXB XML Pipeline Oracle JDeveloper Java XML Processing (xmlparserv2.jar) C XML Processing (oraxml10.lib) XSLT VM XSD SAX DOM DOM SAX JAXP XPath XQuery XSLT XSD Pull XPath Oracle XML Broadband Network— Binary XML Multi-Platform, Multi-OS Runtime in Java C and C++ NLS (oranls.lib) JDK 1.5 CORE (oracore10.lib) NLS (orai18n*.jar) JDBC (classes12.jar)
10. History 11g XDB 10gR2 10g Performance, Strictness to Standards 9iR2 XDK 9i 8i XMLType, XSQL XQuery Robustness
11. XDK or XMLDB ? Oracle XML Developer's Kit (XDK) Is a versatile set of components that enables you to build and deploy C, C++, and Java software programs that process XML Inside and Outside the database Oracle XMLDB Oracle XML DB is the name for a set of Oracle Database technologies related to high-performance XML storage and retrieval Is officially supported since version 9.2.0.3.0
12. XML DB - BasedonStandards World Wide Web Consortium(W3) XML, XML Schema, XSLT, XPath, XQuery, (11gR1) XLink, XInclude International Standard Organisation(ISO) SQL/XML Internet Engineering Task Force (IETF) FTP, HTTP(s), WebDAV Java Community Process Content Repository API for Java, JSR-170 standard
19. Milestones – Oracle 9.2/10.1 Oracle Version 9.2.0.3.0 XML DB Build-In instead of a XDK / XMLType CLOB and Object Relational XMLType Storage Oracle 10gR1 Performance, W3C (etc) strict behavior (namespaces!) XML Schema Evolution (copyEvolve)
20. Milestones – Oracle 10.2 Oracle 10gR2 XQuery and XMLTABLE (XPath V2) Query re-write support for eg. xmltype.extract insertChildXML, appendChildXML, insertXMLbefore, deleteXML XMLType support for Advanced Queueing Oracle 10.2.0.3.0 XQuery (also in XMLTABLE) in Database Kernel (build-in via C instead of Java wrappers)
21. Milestones – Oracle 11.1 Oracle 11gR1 XMLType Binary XML Storage Support Unstructured XMLIndex Support DeprecatedCTXXPATH index Repository Events, NDWS Xlink & Xinclude and standard W3C XQuery 1.0 Recommendation In-Place XML Schema Evolution 64K limit on text nodes and attribute values has been lifted
22. Milestones – Oracle 11.2 Oracle 11.1.0.7.0 Oracle Reference or Equi-Partitioning Support Oracle 11.2.0.1.0 Structured XML Index Ability to support huge XML Schema’s Depricated propriety SQL/XML extract, extractvalue, existsnode, … Oracle 11.2.0.2.0 Binary XML default for XMLType (instead of CLOB) Depricated?: ora:view
23. Oracle XML DB SQL*Net Protocol Server Thin, Thick Clients XMLDB Functionality
24. XML Data Handling XML data exchange SQL, C, C++, Java, PL/SQL, .Net, etc Via Protocol Listener supported methods XML data storage CLOB, Object Relational, Binary XML (11gRx) XML data validation XMLSchema, programmatically XML data creation XML Operators, Packages
25. XML DB Mayor Features Storagebasedon the XMLType datatype XMLType Column, XMLTypeTable Retrievalof data via XML/SQL, XPath, XQuery Securitybasedon ACL, Oracle Roles The Protocol Server HTTP(s), FTP, WebDAV, Native Database WebServices (NDWS) The XMLDB Repository XMLSchema Support and Evolution Versioning, CMS Features
30. In Memory: Common XML Parsers Often handle XML tree traversals only via ONEmethod It is not structured, semi-structured or unstructured XML content aware It is not very “smart” / “content aware” regarding XMLhandling based on its XML tree’s and/or XML data content
31. The “Dimensions” in 1 XML doc. 1 3 4 5 2 X Y 6 Z nx rows Elements with maxoccurs=“unbounded”
32. XMLType XML Container (in memory or via storage) In Memory (document) CLOB (document) Object Relational (data) Binary XML (data)
33. Complexities of a database “Relations” “Redundancy” “Nullology” Design, etc… It can contain a database 10 Mb or bigger nowadays More often than less… Enormous complex XSD’s XMLType – Not just a “Container”
34. Checked on XML Well-Formedness One root element Begin & End tags If XML Schema reference XOB methods will be used if an XML Schema is available (faster, smaller/less data) DOM methods will be used if registered XML Schema information is not available XMLType – Not just a “Datatype”
37. Wow, so what to choose…? Performance C based or Java based solutions Deprecated in the near Future Follow the general XML standard Engine enhancements XPath V2 (XMLTable, XMLQuery) Binary XML support Let Oracle do the Work “Am ”||” i “||” stupid?”,CLOB instead of XMLType
38. Java Based or C Kernel embedded? XQuery (10.2.0.1) 8i related: XMLDOM, XMLPARSER, XMLProcessor DBMS_XMLQUERY DBMS_XMLGEN DBMS_XMLSAVE Pre 11.2.0.0.0 (JVM) ? XQuery (10.2.0.3.0 ) DBMS_XMLDOM DBMS_XMLPROCESSOR XQuery, DBMS_XMLGEN SQL/XML DBMS_XMLSTORE Post 11.2.0.1.0 (no JVM) ?
39. Deprecated in 11.2.0.1.0 EXTRACT EXTRACTVALUE EXISTNODE XMLSEQUENCE ora:instanceof ora:instanceof-only getStringVal(), getClobVal(), getBlobVal() getNamespace() getRootElement() Function-based Indexes (XML) XMLQuery XMLTable, XMLCast, XMLQuery XMLExists XMLTable XQuery instance operator XML Schema attribute xsi:type XMLSerialize XQuery function fn:namespace-uri XQuery function fn:local-name Structured XMLIndex
42. XDB Repository declare res BOOLEAN; begin res := dbms_xdb.createResource ('/public/q1.xqy', <My Xquery>); commit; end; / SELECT xmlquery(xdburitype('/public/q1.xqy').getClob() passing OBJECT_VALUE returning content) FROM MY_XML_TABLE /
43. SQL> SELECT xdata 2 FROM (XMLTABLE( 3 '*' 4 PASSING 5 (XMLTYPE(bfilename('XMLSTORE','data.xml') 6 ,NLS_CHARSET_ID('AL32UTF8'))) 7 COLUMNS xdataXMLTYPE path '/*')); XDATA ------------------------------------ <root><content>Text</content></root> 1 row selected. XML data from your Local Disk
44. SQL> SELECT * 2 FROM XMLTABLE 3 (XMLNAMESPACES('http://purl.org/dc/elements/1.1/' 4 as “MGR") 5 ,'//item' 6PASSING HTTPURITYPE('http://my.blog.com/blog/?feed=rss2').getXML() 7 COLUMNS title varchar2(50) 8 path'/item/title/text()', 9 link varchar2(50) 10 path '/item/link/text()', 11 publication_datevarchar2(50) 12 path '/item/pubDate/text()', 13 creator varchar2(50) 14 path '/item/MGR:creator/text()', 15 description varchar2(250) 16 path '/item/description/text()'); RSS Example (1)
45. TITLE LINK PUBLICATION_DATE CREATOR ------------------------------------------------------------ DESCRIPTION ------------------------------------------------------------ Quering RSS Feeds The XMLDB Way http://feeds.feedburner.com/~r/Bloggralikecom/~3/3 Wed, 25 Jun 2008 16:47:19 +0000 Marco Gralike Actually this IS old stuff (2006), but it got lost IN a… RSS Example (2)
46. SQL> SELECT table_name 2 , to_number( 3 extractvalue(xmltype(dbms_xmlgen.getxml 4 ('select count(*) C 5 from '||table_name)) 6 ,'/ROWSET/ROW/C')) count 7 FROM user_tables; TABLE_NAME COUNT ------------------------------ ------ DEPT 4 EMP 14 BONUS 0 SALGRADE 5 Using XML operators with SQL Source Laurent Schneider:How do i store the counts of all tables …
47. XQuery on PL/SQL variable DECLARE v_x XMLType; NumAcc NUMBER; BEGIN v_x := XMLType(…); /* initialize xmltype variable */ SELECT /*+ NO_XML_QUERY_REWRITE */ XMLCAST(XMLQUERY('declare default element namespace "http://custacc";for $cust in $cadoc/Customer return fn:count($cust/Addresses/Address)' PASSINGv_x AS "cadoc" RETURNING CONTENT) AS NUMBER) INTO NumAcc FROM DUAL; END; Hint allows efficient DOM based evaluation XMLExists() can be used similarly Source OOW 2010 “Managing XML Content with XML DB: Getting the Best Bang for the Buck”
50. When to use XMLType Query against XML data Take advantage of structure when you have an XML schema Provide efficient XPath access Shield applications from storage models Prepare for future optimizations Provide structured storage with DOM fidelity
51. Create XML Relational Content XMLType XMLView XQuery DBMS_XML{Packages} Object Orientated Content XML Content Document Content BFilename, BLOB, CLOB, EXIF…
52. Canonical XMLType Result Set XMLType - Very powerful (…and simple) SQL> select xmltype(cursor(select * fromscott.dept 2whererownum=1)) 3 as "Result" 4fromdual; Result --------------------------------------------------------- <?xmlversion="1.0"?> <ROWSET> <ROW> <DEPTNO>10</DEPTNO> <DNAME>ACCOUNTING</DNAME> <LOC>NEW YORK</LOC> </ROW> </ROWSET>
54. Master – Detail (1) SELECTdepartment_id as "ID", XMLElement("Department", XMLAttributes(d.department_name "name"), (SELECTXMLAgg( XMLElement("emp", XMLAttributes(e.last_name name), (SELECTXMLAgg( XMLElement("jobs", XMLAttributes(j.job_id "job"))) FROMjob_history j WHERE j.employee_id=e.employee_id))) FROM employees e WHEREe.department_id=d.department_id)) AS “MD" FROM departments d WHERE department_id < 40;
56. Native XML Generation Generate XML documents by using : SYS_XMLGEN, to accept a single argument and generate an XML document for each row from a SQL statement SYS_XMLAGG, to process groups of rows, aggregating several XML fragments into a single XML document In PL/SQL, use the DBMS_XMLGEN package to execute any SQL query and map the results into an XML document as a CLOB or XMLType.
57. SYS_XMLGEN() Function Converts its argument into an XML document Accepts two arguments: A scalar value, object type, or XMLType instance An optional XMLFormat() object Returns an XMLType instance Creates and queries XML instances within SQL SELECT SYS_XMLGEN( XMLForest (last_name, salary), XMLFormat('EMPLOYEE')) FROM employees WHERE department_id = 30;
58. SYS_XMLAGG() Function The SYS_XMLAGG() function: Groups all the input documents and produces a single XML document Is used to aggregate (concatenate) fragments SELECT SYS_XMLAGG( SYS_XMLGEN(last_name), XMLFormat('Employees')) result FROM employees WHERE department_id < 30 GROUP BY department_id;
59. DBMS_XMLGEN Creates an XML document from SQL query, XMLType Gets the document as a CLOB Provides a fetch interface Is useful for pagination in Web applications
60. DBMS_XMLGEN (1) CREATE TABLE temp_clob_tab(result CLOB); DECLARE qryCtx DBMS_XMLGEN.ctxHandle; result CLOB; BEGIN qryCtx := DBMS_XMLGEN.newContext('SELECT * FROM hr.employees'); -- Set the row header to be EMPLOYEE DBMS_XMLGEN.setRowTag(qryCtx, 'EMPLOYEE'); -- Get the result result := DBMS_XMLGEN.getXML(qryCtx); INSERT INTO temp_clob_tab VALUES(result); --Close context DBMS_XMLGEN.closeContext(qryCtx); END; /
64. National LanguageSettings CHARACTER SET: AL32UTF8 NATIONAL CHARACTER SET: AL16UTF16 NLS_LENGTH_SEMANTICS ='BYTE'; Before $ORACLE_HOMEDBMSDMINatqm.sql Avoid “Strange” NLS related errors: Registering an XML Schema via WebDAV, Drag & Drop Registering an XML Schema via BFILENAME
65. Database character set is AL32UTF Client character set is WIN1252. SQL> drop table test; SQL> create table test of xmltype; SQL> insert into test values (xmltype( '<?xml version="1.0" encoding="UTF-8"?><FOO/>')); SQL> select object_value from test; OBJECT_VALUE --------------------------------------------------- <?xml version="1.0" encoding="WINDOWS-1252"?><FOO/> NLS FAQ (1)
66. SQL> select x.object_value.getClobVal() from test x; X.OBJECT_VALUE.GETCLOBVAL() -------------------------------------------- <?xml version="1.0" encoding="UTF-8"?><FOO/> SQL> exit C:EMP>set NLS_LANG=AMERICAN_AMERICA.AL32UTF8 C:EMP>sqlplusscott/tiger SQL> select * from test; SYS_NC_ROWINFO$ -------------------------------------------- <?xml version="1.0" encoding="UTF-8"?><FOO/> NLS FAQ (2)
68. Troubleshooting XMLDB Use What You Know: SET AUTOTRACE ON, DBMS_XPLAN.DISPLAY, TKPROF, SQL_TRACE, EVENTS like 10046 etc. Schema Registration: ALTER session SET events = '31098 trace name context forever' Protocol Server: ALTER system SET event = '31098 trace name context forever, level 2' scope=spfile Trace files UDUMP Directory
69. Small tip regarding “alias” usage Using an alias called: “X”, “Y” (and not Z…) SQL> select * from v$versionwhererownum=1; BANNER --------------------------------------------------------- Oracle Database 10g EnterpriseEdition Release 10.2.0.4.0 SQL> describeX FUNCTION X RETURNS NUMBER Argument Name Type In/Out Default? ------------------------------ ----------------------- ------ -------- P ST_POINT IN SQL> describeY FUNCTION Y RETURNS NUMBER Argument Name Type In/Out Default? ------------------------------ ----------------------- ------ -------- P ST_POINT IN
70. Reward True understanding Optimal performance Out performing XML (Java) More options from “within” XMLDB: a “No-Cost” option Opening up propriety environments
71.
72. References (1) Oracle XML DB http://download.oracle.com/docs/cd/E11882_01/appdev.112/e16659/toc.htm XML DB OTN / FAQ Thread http://forums.oracle.com/forums/forum.jspa?forumID=34 http://forums.oracle.com/forums/thread.jspa?threadID=410714
73. References (2) Oracle Whitepapers Oracle XML DB : Choosing the Best XMLType Storage Option for Your Use Case (PDF) Oracle XML DB : Best Practices to Get Optimal Performance out of XML Queries (PDF) Blog http://technology.amis.nl/blog http://blog.gralike.com (Dedicated XMLDB blog)
Editor's Notes
Emp/Dept tables, Foreign/Primary Keys…Showing here ONLY 1 XML document…
