Real World Experience With Oracle Xml Database 11g An Oracle Ace’s Perspective - Oracle Open World 2008 - Marco Gralike

7,188 views

Published on

Broad Spectrum Overview of Lessons Learned, Tips & Tricks, while working with Oracle XML DB

Published in: Technology, Business
1 Comment
9 Likes
Statistics
Notes
  • An excellent presentation now embedded at OracleLink.com - The Oracle Community Network



    http://www.oraclelink.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
7,188
On SlideShare
0
From Embeds
0
Number of Embeds
73
Actions
Shares
0
Downloads
0
Comments
1
Likes
9
Embeds 0
No embeds

No notes for slide

Real World Experience With Oracle Xml Database 11g An Oracle Ace’s Perspective - Oracle Open World 2008 - Marco Gralike

  1. 1. Real World Experience with Oracle XML Database 11g An Oracle ACE’s Perspective Marco Gralike – AMIS – OOW 2008
  2. 2. Who Am I <ul><li>Started as DBA with Oracle 7 on Windows NT 3.1 (1994) </li></ul><ul><li>Experienced with Oracle 7.x / 8.x / 9.x / 10.x and 11.1 </li></ul><ul><li>Started with Oracle XMLDB in 2003 (Oracle 10.1.3) </li></ul><ul><li>Oracle 11g Beta tester for Oracle XMLDB </li></ul><ul><li>Dedicated XMLDB Blog (blog.gralike.com) </li></ul><ul><li>Active Oracle OTN XMLDB Forum Member </li></ul><ul><li>Oracle ACE Award for XMLDB Community Contributions </li></ul>
  3. 3. Disclaimer <ul><li>Pay attention to: </li></ul><ul><li>“ Choice” </li></ul><ul><li>“ Design” </li></ul>The following are “ Rules of Numb ” Every environment has its own unique criteria and needs… “ Testing” “ Performance” “ Maintainability” “ Extendibility”
  4. 4. Overview <ul><li>XML IS NOT RELATIONAL </li></ul><ul><li>What is Oracle XML Database </li></ul><ul><li>Set-up and Configuration </li></ul><ul><li>XMLType Storage </li></ul><ul><li>The Protocol Server </li></ul><ul><li>The Repository </li></ul><ul><li>Data Handling </li></ul><ul><li>Security </li></ul><ul><li>Troubleshooting </li></ul>
  5. 5. Overview <ul><li>Small description / Introduction of the functionality </li></ul><ul><li>Encountered issues </li></ul><ul><ul><li>OTN XMLDB Forum </li></ul></ul><ul><ul><li>FAQ </li></ul></ul><ul><ul><li>Hits on keyword, tags on my personal blog </li></ul></ul><ul><ul><li>Personal view on XMLDB Topics / Tips & Tricks </li></ul></ul>
  6. 6. XML IS NOT RELATIONAL
  7. 7. Let’s Presume it is Relational
  8. 8. Structured Relational Data
  9. 9. Query Optimizer - Access Paths
  10. 10. Structured Data Unstructured (XML) Data
  11. 11. How to Containerize a Tree? <ul><li>Keeping it intact? </li></ul><ul><li>Too much empty space…! </li></ul><ul><li>Access Optimization..? </li></ul><ul><li>Shredding it? </li></ul><ul><li>Rebuilding and shredding off the </li></ul><ul><li>tree structure…? </li></ul><ul><li>There is no solution for this…yet… </li></ul>
  12. 12. Impedance Mismatch <ul><li>Different data models. </li></ul><ul><ul><li>E.g. XPath models an XML document as a tree, while most general purpose programming languages have no native data types for a tree. </li></ul></ul><ul><li>Different programming paradigms. </li></ul><ul><ul><li>XSLT is a functional language, while Java is object-oriented , and Perl is a procedural one. </li></ul></ul>Source: Wikipedia SXML
  13. 13. Issues in General <ul><li>NO PROBLEM </li></ul><ul><ul><li>Data Shipping / Transport XML via XML </li></ul></ul><ul><ul><li>UI’s that “Understand”, are based on XML </li></ul></ul><ul><li>PROBLEM </li></ul><ul><ul><li>Accessing Data in XML structures </li></ul></ul><ul><ul><ul><li>CPU Intensive </li></ul></ul></ul><ul><ul><ul><li>Memory Intensive </li></ul></ul></ul><ul><ul><ul><li> ORA-04030, XML Fragments, DOM </li></ul></ul></ul><ul><ul><li>Transformation: </li></ul></ul><ul><ul><ul><li>XML  Relational </li></ul></ul></ul><ul><ul><ul><li>Risk : Data / Information Loss </li></ul></ul></ul>
  14. 14. So the issues you will encounter with XML <ul><li>The nature of XML: </li></ul><ul><ul><li>It is (most of the time) not Structured </li></ul></ul><ul><ul><li>It can not be stored in a Universal XML “Container” </li></ul></ul><ul><ul><li>It can not be accessed via a standard Access Path Method that “understands” the free format XML structure all the time… </li></ul></ul><ul><li>It is all about free format </li></ul><ul><ul><li>OK and Not OK </li></ul></ul><ul><ul><li>Standards: W3C, Oasis, etc </li></ul></ul><ul><li>Performance problem induced by its Nature… </li></ul><ul><ul><li>Nature is not structured…(as is Human Nature) </li></ul></ul>
  15. 15. WHAT IS ORACLE XML DATABASE
  16. 16. Why Oracle XML Database? <ul><li>File Storage </li></ul><ul><li>XML Database </li></ul><ul><ul><li>Native </li></ul></ul><ul><ul><li>XML Enabled </li></ul></ul><ul><li>Oracle XMLDB </li></ul><ul><li>A World within a World </li></ul><ul><li>More than one Option </li></ul><ul><li>Complies to Standards </li></ul><ul><li>Still License Free </li></ul><ul><li>Core part of (OXJR)DBMS </li></ul><ul><li>The (relational) foundation is there and can also be used dealing with XML </li></ul>
  17. 17. Oracle XML Database SQL*Net Protocol Server Thin, Thick Clients XMLDB Funtionality
  18. 18. XML Handling <ul><li>XML data exchange </li></ul><ul><ul><li>SQL, C, C++, Java, PL/SQL, .Net, etc </li></ul></ul><ul><ul><li>Via Protocol Listener supported methods </li></ul></ul><ul><ul><li>Via Oracle Advanced Queuing methods (Oracle 11gR1) </li></ul></ul><ul><li>XML data storage </li></ul><ul><ul><li>CLOB, Object Relational, Binary XML (11gR1) </li></ul></ul><ul><li>XML data validation </li></ul><ul><ul><li>XMLSchema, Programmatically, via Storage definitions </li></ul></ul><ul><li>XML data creation </li></ul><ul><ul><li>XML Operators, Packages </li></ul></ul>
  19. 19. Based on (XML) Standards <ul><li>World Wide Web Consortium (W3) </li></ul><ul><ul><li>XML, XML Schema, XSL, XSLT, XPath, XQuery, </li></ul></ul><ul><ul><li>(11gR1) XLink, XInclude </li></ul></ul><ul><li>International Standard Organisation (ISO) </li></ul><ul><ul><li>SQL/XML </li></ul></ul><ul><li>Internet Engineering Task Force (IETF) </li></ul><ul><ul><li>FTP, HTTP(s), WebDAV </li></ul></ul><ul><li>They don’t always agree on solutions… </li></ul><ul><li>How are we going to solve those problems…  </li></ul>
  20. 20. Making Choices - Versions 8i 9i 10g 9iR2 10gR2 XDK XMLType XSQL XDB XQuery Robustness 11g Binary XML, Partitioning XMLIndex, NDWS Performance, Strictness to Standards
  21. 21. SET-UP AND CONFIGURATION
  22. 22. Memory Structures and Background Processes <ul><li>JAVA_POOL (Java Classes, Wrappers) </li></ul><ul><li>LARGE_POOL (Shared Server, aka MTS) </li></ul><ul><li>PGA_AGGREGATE_TARGET (DOM) </li></ul><ul><li>LOCAL_LISTENER = {NOT_DEFAULT_LISTENER_NAME} </li></ul><ul><li>DISPATCHERS = &quot;(PROTOCOL=TCP) (SERVICE={SID}XDB)“ </li></ul><ul><li>SHARED_SERVERS = 5 </li></ul><ul><li> Security / Performance Issues: 10.1, 10.2, 11.1 </li></ul>
  23. 23. National Langue Settings <ul><li>CHARACTER SET: AL32UTF8 </li></ul><ul><li>NATIONAL CHARACTER SET: AL16UTF16 </li></ul><ul><li>NLS_LENGTH_SEMANTICS ='BYTE'; </li></ul><ul><ul><li>Before $ORACLE_HOMERDBMSADMINcatqm.sql </li></ul></ul><ul><ul><li>SQL>@?/rdbms/admin/catnoqm </li></ul></ul><ul><li>Avoid “Strange” NLS related errors: </li></ul><ul><ul><li>Registering an XML Schema via WebDAV, Drag & Drop </li></ul></ul><ul><ul><li>Registering an XML Schema via BFILENAME </li></ul></ul>
  24. 24. OTN XMLDB Forum - FAQ Database character set is AL32UTF Client character set is WIN1252 . SQL> drop table test; SQL> create table test of xmltype; SQL> insert into test values (xmltype('<?xml version=&quot;1.0&quot; encoding=&quot; UTF-8 &quot;?><FOO/>')); SQL> select object_value from test; OBJECT_VALUE --------------------------------------------------- <?xml version=&quot;1.0&quot; encoding=&quot; WINDOWS-1252 &quot;?><FOO/>
  25. 25. NLS FAQ Continued SQL> select x.object_value.getClobVal() from test x; X.OBJECT_VALUE.GETCLOBVAL() -------------------------------------------- <?xml version=&quot;1.0&quot; encoding=&quot; UTF-8 &quot;?><FOO/> SQL> exit C:TEMP>set NLS_LANG=AMERICAN_AMERICA. AL32UTF8 C:TEMP>sqlplus scott/tiger SQL> select * from test; SYS_NC_ROWINFO$ -------------------------------------------- <?xml version=&quot;1.0&quot; encoding=&quot; UTF-8 &quot;?><FOO/>
  26. 26. Database <ul><li>COMPATIBLE (e.g. XQuery 10.2.0.1 <> 10.2.0.3) </li></ul><ul><li>RECYCLE_BIN </li></ul><ul><ul><li>copy_evolve (xml evolution) </li></ul></ul><ul><ul><li>dropping schemata </li></ul></ul><ul><li>Control via XMLType Storage (  Containers, Models) </li></ul>
  27. 27. XMLType Storage
  28. 28. Relational Contra XML Data 1 Record = ???
  29. 29. XMLTYPE Storage Logical Design Level Physical Design Level
  30. 30. Choose but choose wisely <ul><li>Think in Terms of… </li></ul><ul><li>Design </li></ul><ul><ul><li>Usage of XML Schemata? </li></ul></ul><ul><li>Future Extendibility </li></ul><ul><ul><li>XML Evolution </li></ul></ul><ul><li>Maintainability </li></ul><ul><ul><li>Backup & Recovery </li></ul></ul><ul><li>Access Paths, The Cost Base Optimizer = Performance </li></ul><ul><ul><li>Selecting Data </li></ul></ul><ul><ul><li>Indexes </li></ul></ul>
  31. 31. Size after Inserts
  32. 32. Load Time during Inserts
  33. 33. CLOB, Object Relational, Binary XML XMLType (Tables) XMLType Views XMLDB Data Storage (Object) Relational Objects Relational Tables Relational World XMLDB World Binary XML Structured Mixed complex[y] BINARY XSD [y/n] XMLIndex Content complex[n] OR XSD [y] BTree, IOT Hybrid Content complex[n] Mixed XSD [y] BTree, IOT Unstructured Document na CLOB XSD [n] XMLIndex
  34. 34. Generate or Manual Creation <ul><li>Generate via DBMS_XMLSCHEMA </li></ul><ul><ul><li>XML Schema Based </li></ul></ul><ul><ul><li>XDB Annotations </li></ul></ul><ul><ul><li>JDeveloper or XMLSpy </li></ul></ul><ul><ul><li>Best Practise </li></ul></ul><ul><li>Manual Creation </li></ul><ul><ul><li>DBMS_METADATA.GET_DDL </li></ul></ul><ul><ul><li>EVENT setting </li></ul></ul><ul><ul><ul><li>alter session set events='31098 trace name context forever'; </li></ul></ul></ul><ul><ul><ul><li>/oracle/diag/rdbms/xmldb/XMLDB/trace </li></ul></ul></ul><ul><ul><li>Use Oracle 11g generation to apply in 10gR2 (IOT <> BTREE) </li></ul></ul>
  35. 35. Storage Factors <ul><li>Design </li></ul><ul><ul><li>Unstructured contra Structured (XML Schema) </li></ul></ul><ul><ul><ul><li>Validation, CPU </li></ul></ul></ul><ul><li>Data Optimization </li></ul><ul><ul><li>Binary XML </li></ul></ul><ul><ul><ul><li>(De)Composition of Structures </li></ul></ul></ul><ul><li>Flexibility </li></ul><ul><ul><li>Binary XML contra Object Relational </li></ul></ul>
  36. 36. Continued… <ul><li>Whitespace </li></ul><ul><ul><li>More storage needed </li></ul></ul><ul><ul><li>Do you remember Nullology? </li></ul></ul><ul><ul><li>“ Air” has to be processed as well </li></ul></ul><ul><li>Amount of Storage </li></ul><ul><ul><li>Binary XML contra Object Relational </li></ul></ul><ul><ul><li>Nested Storage contra Out of Line </li></ul></ul><ul><ul><li>XML Partitioning </li></ul></ul>
  37. 37. Big XML Schemata (Object Relational) <ul><li>Out of Line Storage </li></ul><ul><ul><li>Twice the amount of inserts </li></ul></ul><ul><ul><li>Avoiding the 1000 columns per table limit </li></ul></ul><ul><li>Nested Table Storage </li></ul><ul><ul><li>More memory consumption can lead to  ORA-04030 </li></ul></ul><ul><ul><li>Memory needed for all the number of attributes in all descendants of the object </li></ul></ul>http://forums.oracle.com/forums/message.jspa?messageID=2736485
  38. 38. Equi Partitioning EMP_PROJ_P11 reference_id EMPLOYEES_PROJ_TAB “ employees”.”employee” EMP_PROJ_P12 PROJ_DETAILS_TAB
  39. 39. The Protocol Server
  40. 40. Interfacing <ul><li>Build on Shared Server (aka MTS) </li></ul><ul><li>Config file: xdbconfig.xml </li></ul><ul><li>“ Extension” on the Oracle Listener </li></ul><ul><li>Enables access for / via: </li></ul><ul><ul><li>WebDAV, HTTP(s), FTP </li></ul></ul><ul><ul><li>ASM </li></ul></ul><ul><ul><li>DBUri’s </li></ul></ul><ul><li>Foundation for </li></ul><ul><ul><li>Your “local” APEX implementation </li></ul></ul><ul><ul><li>Native Database Web Services (NDWS) </li></ul></ul><ul><ul><li>Your own Servlet’s, PL/SQL </li></ul></ul>
  41. 41. Possibilities via the Protocol Server WebDAV FTP HTTP(s)
  42. 42. Native Database Web Services WSDL Structure Architecture
  43. 43. NDWS is Simple <ul><li>Enable the Protocol Server for HTTP access </li></ul><ul><li>Enable the orawsv entry point in xdbconfig.xml </li></ul><ul><li>Create a user and grant access </li></ul><ul><ul><li>DBMS_NETWORK_ACL_ADMIN </li></ul></ul><ul><ul><li>XDB_WEBSERVICES </li></ul></ul><ul><ul><li>XDB_WEBSERVICES_OVER_HTTP </li></ul></ul><ul><ul><li>XDB_WEBSERVICES_WITH_PUBLIC </li></ul></ul><ul><li>Create your procedure - for instance a proc. SQUARE </li></ul><ul><li>TEST: http://localhost:8080/orawsv/SCOTT/SQUARE?wsdl </li></ul>
  44. 44. The Repository
  45. 45. The Repository
  46. 46. The Repository <ul><li>A foundation between the inside and outside world </li></ul><ul><ul><li>/sys should not be used. </li></ul></ul><ul><ul><li>XDB$RESOURCE </li></ul></ul><ul><ul><li>Security via ACL, XML, References to objects, etc </li></ul></ul><ul><ul><li>Contains info about registered XML schema, its objects, etc </li></ul></ul><ul><li>Out of the Box, Simple CMS </li></ul><ul><ul><li>Security </li></ul></ul><ul><ul><li>Oracle Search </li></ul></ul><ul><ul><li>Accessible via WebDAV and SQL </li></ul></ul><ul><ul><li>Store for, e.g. scripting, etc </li></ul></ul>
  47. 47. Storing your Scripts in the Repository SELECT xmlquery( xdburitype('/public/q1.xqy').getClob() passing OBJECT_VALUE returning content) FROM MY_XML_TABLE / declare res BOOLEAN; begin res := dbms_xdb.createResource ('/public/q1.xqy',<My Xquery>); commit; end; /
  48. 48. Data Handling
  49. 49. XML Operators & Functions in 11gR1 <ul><li>XMLAGG </li></ul><ul><li>XMLCAST </li></ul><ul><li>XMLCDATA </li></ul><ul><li>XMLCOLATTVAL </li></ul><ul><li>XMLCOMMENT </li></ul><ul><li>XMLCONCAT </li></ul><ul><li>XMLDIFF </li></ul><ul><li>XMLELEMENT </li></ul><ul><li>XMLEXISTS </li></ul><ul><li>XMLFOREST </li></ul>XMLPARSE XMLPATCH XMLPI XMLQUERY XMLROOT XMLSEQUENCE XMLSERIALIZE XMLTABLE XMLTRANSFORM … DELETEXML EXTRACTVALUE EXISTNODE EXTRACT INSERTCHILDXML INSERTXMLBEFORE SYS_XMLAGG SYS_XMLGEN UPDATEXML …
  50. 50. Changes <ul><li>More strict in using Standards from: 9.2  10.x  11.x </li></ul><ul><ul><li>Namespaces required while using functions like EXTRACT </li></ul></ul><ul><li>“ Pretty Print” </li></ul><ul><ul><li>EXTRACT(xmltype,’/*’) </li></ul></ul><ul><ul><li>XMLSERIALIZE </li></ul></ul><ul><li>Moving from packages to operators </li></ul><ul><ul><li>Use SQL/XML Operators instead of DBMS_XMLGEN </li></ul></ul>
  51. 51. Extends the Relational World <ul><li>XML Operators, etc, can be used for dealing with SQL </li></ul><ul><li>Extra Opportunities for data handling / interfacing </li></ul><ul><ul><li>WebDAV, FTP </li></ul></ul><ul><ul><li>Storage and Retrieval of data outside the database </li></ul></ul><ul><ul><li>XQuery, XMLTable </li></ul></ul>
  52. 52. Selecting XML data from your Local Disk SQL> SELECT 2 xdata 3 FROM ( XMLTABLE 4 ('*' 5 passing 6 ( XMLTYPE ( bfilename ('XMLSTORE','data.xml') 7 ,NLS_CHARSET_ID('AL32UTF8'))) 8 columns xdata xmltype PATH '/*')); XDATA ------------------------------------ <root><content>Text</content></root> 1 row selected.
  53. 53. Using XML operators with SQL SQL> SELECT 2 table_name, 3 to_number( 4 extractvalue( 5 xmltype(dbms_xmlgen.getxml 6 ('select count(*) C from '||table_name)) 7 ,' /ROWSET/ROW/C ')) count 8 FROM user_tables; TABLE_NAME COUNT ------------------------------ ------ DEPT 4 EMP 14 BONUS 0 SALGRADE 5 Source Laurent Schneider : How do i store the counts of all tables …
  54. 54. RSS Example SQL> SELECT * 2 FROM XMLTABLE (XMLNAMESPACES('http://purl.org/dc/elements/1.1/' AS &quot;MGR&quot;) 3 ,'//item' 4 passing HTTPURITYPE ('http://my.blog.com/blog/?feed=rss2').getXML() 5 columns title varchar2(50) path '/item/title/text()', 6 link varchar2(50) path '/item/link/text()', 7 publication_date varchar2(50) path '/item/pubDate/text()', 8 creator varchar2(50) path '/item/MGR:creator/text()', 9 description varchar2(250) path '/item/description/text()', 10 category XMLTYPE path '/item/category/text()');
  55. 55. Accessing the outside World TITLE LINK PUBLICATION_DATE CREATOR ----------------------------------------------------------------- DESCRIPTION CATEGORY ----------------------------------------------------------------- Quering RSS Feeds The XMLDB Way http://feeds.feedburner.com/~r/Bloggralikecom/~3/3 Wed, 25 Jun 2008 16:47:19 +0000 Marco Gralike Actually this IS old stuff (2006), but it got lost IN a… < ![CDATA[11g]]>< ![CDATA[Howto]]>
  56. 56. Security
  57. 57. XMLDB Security <ul><li>Foundation </li></ul><ul><ul><li>It’s build in so why not (also) use it for your environment </li></ul></ul><ul><ul><li>XMLDB: Core part of Oracle Database </li></ul></ul><ul><ul><li>Across (SOA) Tiers: </li></ul></ul><ul><ul><ul><li>Oracle Web Service Manager </li></ul></ul></ul><ul><ul><ul><li>OLAP </li></ul></ul></ul><ul><li>Methods </li></ul><ul><ul><li>Roles </li></ul></ul><ul><ul><li>Access Control Lists (ACL) </li></ul></ul>
  58. 58. Statement of Direction? <ul><li>Access Control for Network Utility Packages </li></ul><ul><li>( Oracle 11gR1 Upgrade Guide ) </li></ul><ul><li>The default behavior for access control to network utility packages has been changed to disallow network operations to all nonprivileged users. </li></ul><ul><li>This default behavior is different from, and is incompatible with, previous versions of Oracle Database. </li></ul>
  59. 59. ORA-24247 <ul><li>ORA-24247 </li></ul><ul><ul><li>network access denied by access control list (ACL) </li></ul></ul><ul><li>Fine-Grained Access to External Network Services </li></ul><ul><li>Packages involved (11gR1) </li></ul><ul><ul><li>UTL_TCP, UTL_SMTP, </li></ul></ul><ul><ul><li>UTL_MAIL, UTL_HTTP, UTL_INADDR </li></ul></ul><ul><li>Solution: </li></ul><ul><ul><li>DBMS_NETWORK_ACL_ADMIN </li></ul></ul><ul><ul><li>Check, Add, Remove Privileges </li></ul></ul>
  60. 60. Troubleshooting
  61. 61. Troubleshooting XMLDB <ul><li>Use What You Know: </li></ul><ul><ul><li>SET AUTOTRACE ON </li></ul></ul><ul><ul><li>DBMS_XPLAN.DISPLAY </li></ul></ul><ul><ul><li>TKPROF, SQL_TRACE </li></ul></ul><ul><ul><li>EVENTS like 10046 etc. </li></ul></ul><ul><li>Schema Registration: </li></ul><ul><ul><li>ALTER session SET events = '31098 trace name context forever' </li></ul></ul><ul><li>Protocol Server: </li></ul><ul><ul><li>ALTER system SET event = '31098 trace name context forever, level 2' scope=spfile </li></ul></ul><ul><ul><li>Trace files  UDUMP Directory </li></ul></ul>
  62. 62. RECAP…
  63. 63. Recap… <ul><li>XML IS NOT RELATIONAL </li></ul><ul><li>What is Oracle XML Database </li></ul><ul><li>Set-up and Configuration </li></ul><ul><li>XMLType Storage </li></ul><ul><li>The Protocol Server </li></ul><ul><li>The Repository </li></ul><ul><li>Data Handling </li></ul><ul><li>Security </li></ul><ul><li>Troubleshooting </li></ul>
  64. 64. Oracle Open World 2008 - XMLDB Sessions
  65. 65. References <ul><li>Oracle 11g XMLDB Developers Guide </li></ul><ul><li>The XMLDB Forum </li></ul><ul><ul><li>http:// forums.oracle.com/forums/forum.jspa?forumID =34 </li></ul></ul><ul><li>OTN XML DB FAQ Thread </li></ul><ul><ul><li>http://forums.oracle.com/forums/thread.jspa?threadID=410714 </li></ul></ul><ul><li>Technology Blogs </li></ul><ul><ul><li>http:// technology.amis.nl/blog </li></ul></ul><ul><ul><li>http:// blog.gralike.com </li></ul></ul><ul><ul><li>http:// laurentschneider.com </li></ul></ul>
  66. 66. Questions? <ul><li>[email_address] </li></ul><ul><li>http://www.amis.nl </li></ul>

×