Real World Experience With Oracle Xml Database 11g An Oracle Ace’s Perspective - Oracle Open World 2008 - Marco Gralike

Real World Experience with Oracle XML Database 11g An Oracle ACE’s Perspective Marco Gralike – AMIS – OOW 2008

Who Am I

Started as DBA with Oracle 7 on Windows NT 3.1 (1994)

Experienced with Oracle 7.x / 8.x / 9.x / 10.x and 11.1

Started with Oracle XMLDB in 2003 (Oracle 10.1.3)

Oracle 11g Beta tester for Oracle XMLDB

Dedicated XMLDB Blog (blog.gralike.com)

Active Oracle OTN XMLDB Forum Member

Oracle ACE Award for XMLDB Community Contributions

Disclaimer

Pay attention to:

“ Choice”

“ Design”

The following are “ Rules of Numb ” Every environment has its own unique criteria and needs… “ Testing” “ Performance” “ Maintainability” “ Extendibility”

Overview

XML IS NOT RELATIONAL

What is Oracle XML Database

Set-up and Configuration

XMLType Storage

The Protocol Server

The Repository

Data Handling

Security

Troubleshooting

Overview

Small description / Introduction of the functionality

Encountered issues

OTN XMLDB Forum

FAQ

Hits on keyword, tags on my personal blog

Personal view on XMLDB Topics / Tips & Tricks

Let’s Presume it is Relational

Structured Relational Data

Query Optimizer - Access Paths

Structured Data Unstructured (XML) Data

How to Containerize a Tree?

Keeping it intact?

Too much empty space…!

Access Optimization..?

Shredding it?

Rebuilding and shredding off the

tree structure…?

There is no solution for this…yet…

Impedance Mismatch

Different data models.

E.g. XPath models an XML document as a tree, while most general purpose programming languages have no native data types for a tree.

Different programming paradigms.

XSLT is a functional language, while Java is object-oriented , and Perl is a procedural one.

Source: Wikipedia SXML

Issues in General

NO PROBLEM

Data Shipping / Transport XML via XML

UI’s that “Understand”, are based on XML

PROBLEM

Accessing Data in XML structures

CPU Intensive

Memory Intensive

 ORA-04030, XML Fragments, DOM

Transformation:

XML  Relational

Risk : Data / Information Loss

So the issues you will encounter with XML

The nature of XML:

It is (most of the time) not Structured

It can not be stored in a Universal XML “Container”

It can not be accessed via a standard Access Path Method that “understands” the free format XML structure all the time…

It is all about free format

OK and Not OK

Standards: W3C, Oasis, etc

Performance problem induced by its Nature…

Nature is not structured…(as is Human Nature)

WHAT IS ORACLE XML DATABASE

Why Oracle XML Database?

File Storage

XML Database

Native

XML Enabled

Oracle XMLDB

A World within a World

More than one Option

Complies to Standards

Still License Free

Core part of (OXJR)DBMS

The (relational) foundation is there and can also be used dealing with XML

Oracle XML Database SQL*Net Protocol Server Thin, Thick Clients XMLDB Funtionality

XML Handling

XML data exchange

SQL, C, C++, Java, PL/SQL, .Net, etc

Via Protocol Listener supported methods

Via Oracle Advanced Queuing methods (Oracle 11gR1)

XML data storage

CLOB, Object Relational, Binary XML (11gR1)

XML data validation

XMLSchema, Programmatically, via Storage definitions

XML data creation

XML Operators, Packages

Based on (XML) Standards

World Wide Web Consortium (W3)

XML, XML Schema, XSL, XSLT, XPath, XQuery,

(11gR1) XLink, XInclude

International Standard Organisation (ISO)

SQL/XML

Internet Engineering Task Force (IETF)

FTP, HTTP(s), WebDAV

They don’t always agree on solutions…

How are we going to solve those problems… 

Making Choices - Versions 8i 9i 10g 9iR2 10gR2 XDK XMLType XSQL XDB XQuery Robustness 11g Binary XML, Partitioning XMLIndex, NDWS Performance, Strictness to Standards

SET-UP AND CONFIGURATION

Memory Structures and Background Processes

JAVA_POOL (Java Classes, Wrappers)

LARGE_POOL (Shared Server, aka MTS)

PGA_AGGREGATE_TARGET (DOM)

LOCAL_LISTENER = {NOT_DEFAULT_LISTENER_NAME}

DISPATCHERS = "(PROTOCOL=TCP) (SERVICE={SID}XDB)“

SHARED_SERVERS = 5

 Security / Performance Issues: 10.1, 10.2, 11.1

National Langue Settings

CHARACTER SET: AL32UTF8

NATIONAL CHARACTER SET: AL16UTF16

NLS_LENGTH_SEMANTICS ='BYTE';

Before $ORACLE_HOMERDBMSADMINcatqm.sql

SQL>@?/rdbms/admin/catnoqm

Avoid “Strange” NLS related errors:

Registering an XML Schema via WebDAV, Drag & Drop

Registering an XML Schema via BFILENAME

OTN XMLDB Forum - FAQ Database character set is AL32UTF Client character set is WIN1252 . SQL> drop table test; SQL> create table test of xmltype; SQL> insert into test values (xmltype('<?xml version="1.0" encoding=" UTF-8 "?><FOO/>')); SQL> select object_value from test; OBJECT_VALUE --------------------------------------------------- <?xml version="1.0" encoding=" WINDOWS-1252 "?><FOO/>

NLS FAQ Continued SQL> select x.object_value.getClobVal() from test x; X.OBJECT_VALUE.GETCLOBVAL() -------------------------------------------- <?xml version="1.0" encoding=" UTF-8 "?><FOO/> SQL> exit C:TEMP>set NLS_LANG=AMERICAN_AMERICA. AL32UTF8 C:TEMP>sqlplus scott/tiger SQL> select * from test; SYS_NC_ROWINFO$ -------------------------------------------- <?xml version="1.0" encoding=" UTF-8 "?><FOO/>

Database

COMPATIBLE (e.g. XQuery 10.2.0.1 <> 10.2.0.3)

RECYCLE_BIN

copy_evolve (xml evolution)

dropping schemata

Control via XMLType Storage (  Containers, Models)

XMLType Storage

Relational Contra XML Data 1 Record = ???

XMLTYPE Storage Logical Design Level Physical Design Level

Choose but choose wisely

Think in Terms of…

Design

Usage of XML Schemata?

Future Extendibility

XML Evolution

Maintainability

Backup & Recovery

Access Paths, The Cost Base Optimizer = Performance

Selecting Data

Indexes

Size after Inserts

Load Time during Inserts

CLOB, Object Relational, Binary XML XMLType (Tables) XMLType Views XMLDB Data Storage (Object) Relational Objects Relational Tables Relational World XMLDB World Binary XML Structured Mixed complex[y] BINARY XSD [y/n] XMLIndex Content complex[n] OR XSD [y] BTree, IOT Hybrid Content complex[n] Mixed XSD [y] BTree, IOT Unstructured Document na CLOB XSD [n] XMLIndex

Generate or Manual Creation

Generate via DBMS_XMLSCHEMA

XML Schema Based

XDB Annotations

JDeveloper or XMLSpy

Best Practise

Manual Creation

DBMS_METADATA.GET_DDL

EVENT setting

alter session set events='31098 trace name context forever';

/oracle/diag/rdbms/xmldb/XMLDB/trace

Use Oracle 11g generation to apply in 10gR2 (IOT <> BTREE)

Storage Factors

Design

Unstructured contra Structured (XML Schema)

Validation, CPU

Data Optimization

Binary XML

(De)Composition of Structures

Flexibility

Binary XML contra Object Relational

Continued…

Whitespace

More storage needed

Do you remember Nullology?

“ Air” has to be processed as well

Amount of Storage

Binary XML contra Object Relational

Nested Storage contra Out of Line

XML Partitioning

Big XML Schemata (Object Relational)

Out of Line Storage

Twice the amount of inserts

Avoiding the 1000 columns per table limit

Nested Table Storage

More memory consumption can lead to  ORA-04030

Memory needed for all the number of attributes in all descendants of the object

http://forums.oracle.com/forums/message.jspa?messageID=2736485

Equi Partitioning EMP_PROJ_P11 reference_id EMPLOYEES_PROJ_TAB “ employees”.”employee” EMP_PROJ_P12 PROJ_DETAILS_TAB

The Protocol Server

Interfacing

Build on Shared Server (aka MTS)

Config file: xdbconfig.xml

“ Extension” on the Oracle Listener

Enables access for / via:

WebDAV, HTTP(s), FTP

ASM

DBUri’s

Foundation for

Your “local” APEX implementation

Native Database Web Services (NDWS)

Your own Servlet’s, PL/SQL

Possibilities via the Protocol Server WebDAV FTP HTTP(s)

Native Database Web Services WSDL Structure Architecture

NDWS is Simple

Enable the Protocol Server for HTTP access

Enable the orawsv entry point in xdbconfig.xml

Create a user and grant access

DBMS_NETWORK_ACL_ADMIN

XDB_WEBSERVICES

XDB_WEBSERVICES_OVER_HTTP

XDB_WEBSERVICES_WITH_PUBLIC

Create your procedure - for instance a proc. SQUARE

TEST: http://localhost:8080/orawsv/SCOTT/SQUARE?wsdl

The Repository

A foundation between the inside and outside world

/sys should not be used.

XDB$RESOURCE

Security via ACL, XML, References to objects, etc

Contains info about registered XML schema, its objects, etc

Out of the Box, Simple CMS

Security

Oracle Search

Accessible via WebDAV and SQL

Store for, e.g. scripting, etc

Storing your Scripts in the Repository SELECT xmlquery( xdburitype('/public/q1.xqy').getClob() passing OBJECT_VALUE returning content) FROM MY_XML_TABLE / declare res BOOLEAN; begin res := dbms_xdb.createResource ('/public/q1.xqy',<My Xquery>); commit; end; /

Data Handling

XML Operators & Functions in 11gR1

XMLAGG

XMLCAST

XMLCDATA

XMLCOLATTVAL

XMLCOMMENT

XMLCONCAT

XMLDIFF

XMLELEMENT

XMLEXISTS

XMLFOREST

XMLPARSE XMLPATCH XMLPI XMLQUERY XMLROOT XMLSEQUENCE XMLSERIALIZE XMLTABLE XMLTRANSFORM … DELETEXML EXTRACTVALUE EXISTNODE EXTRACT INSERTCHILDXML INSERTXMLBEFORE SYS_XMLAGG SYS_XMLGEN UPDATEXML …

Changes

More strict in using Standards from: 9.2  10.x  11.x

Namespaces required while using functions like EXTRACT

“ Pretty Print”

EXTRACT(xmltype,’/*’)

XMLSERIALIZE

Moving from packages to operators

Use SQL/XML Operators instead of DBMS_XMLGEN

Extends the Relational World

XML Operators, etc, can be used for dealing with SQL

Extra Opportunities for data handling / interfacing

WebDAV, FTP

Storage and Retrieval of data outside the database

XQuery, XMLTable

Selecting XML data from your Local Disk SQL> SELECT 2 xdata 3 FROM ( XMLTABLE 4 ('*' 5 passing 6 ( XMLTYPE ( bfilename ('XMLSTORE','data.xml') 7 ,NLS_CHARSET_ID('AL32UTF8'))) 8 columns xdata xmltype PATH '/*')); XDATA ------------------------------------ <root><content>Text</content></root> 1 row selected.

Using XML operators with SQL SQL> SELECT 2 table_name, 3 to_number( 4 extractvalue( 5 xmltype(dbms_xmlgen.getxml 6 ('select count(*) C from '||table_name)) 7 ,' /ROWSET/ROW/C ')) count 8 FROM user_tables; TABLE_NAME COUNT ------------------------------ ------ DEPT 4 EMP 14 BONUS 0 SALGRADE 5 Source Laurent Schneider : How do i store the counts of all tables …

RSS Example SQL> SELECT * 2 FROM XMLTABLE (XMLNAMESPACES('http://purl.org/dc/elements/1.1/' AS "MGR") 3 ,'//item' 4 passing HTTPURITYPE ('http://my.blog.com/blog/?feed=rss2').getXML() 5 columns title varchar2(50) path '/item/title/text()', 6 link varchar2(50) path '/item/link/text()', 7 publication_date varchar2(50) path '/item/pubDate/text()', 8 creator varchar2(50) path '/item/MGR:creator/text()', 9 description varchar2(250) path '/item/description/text()', 10 category XMLTYPE path '/item/category/text()');

Accessing the outside World TITLE LINK PUBLICATION_DATE CREATOR ----------------------------------------------------------------- DESCRIPTION CATEGORY ----------------------------------------------------------------- Quering RSS Feeds The XMLDB Way http://feeds.feedburner.com/~r/Bloggralikecom/~3/3 Wed, 25 Jun 2008 16:47:19 +0000 Marco Gralike Actually this IS old stuff (2006), but it got lost IN a… < ![CDATA[11g]]>< ![CDATA[Howto]]>

Security

XMLDB Security

Foundation

It’s build in so why not (also) use it for your environment

XMLDB: Core part of Oracle Database

Across (SOA) Tiers:

Oracle Web Service Manager

OLAP

Methods

Roles

Access Control Lists (ACL)

Statement of Direction?

Access Control for Network Utility Packages

( Oracle 11gR1 Upgrade Guide )

The default behavior for access control to network utility packages has been changed to disallow network operations to all nonprivileged users.

This default behavior is different from, and is incompatible with, previous versions of Oracle Database.

ORA-24247

ORA-24247

network access denied by access control list (ACL)

Fine-Grained Access to External Network Services

Packages involved (11gR1)

UTL_TCP, UTL_SMTP,

UTL_MAIL, UTL_HTTP, UTL_INADDR

Solution:

DBMS_NETWORK_ACL_ADMIN

Check, Add, Remove Privileges

Troubleshooting

Troubleshooting XMLDB

Use What You Know:

SET AUTOTRACE ON

DBMS_XPLAN.DISPLAY

TKPROF, SQL_TRACE

EVENTS like 10046 etc.

Schema Registration:

ALTER session SET events = '31098 trace name context forever'

Protocol Server:

ALTER system SET event = '31098 trace name context forever, level 2' scope=spfile

Trace files  UDUMP Directory

RECAP…

Recap…

What is Oracle XML Database

Set-up and Configuration

XMLType Storage

The Protocol Server

The Repository

Data Handling

Security

Troubleshooting

Oracle Open World 2008 - XMLDB Sessions

References

Oracle 11g XMLDB Developers Guide

The XMLDB Forum

http:// forums.oracle.com/forums/forum.jspa?forumID =34

OTN XML DB FAQ Thread

http://forums.oracle.com/forums/thread.jspa?threadID=410714

Technology Blogs

http:// technology.amis.nl/blog

http:// blog.gralike.com

http:// laurentschneider.com

Questions?

[email_address]

http://www.amis.nl

Real World Experience With Oracle Xml Database 11g An Oracle Ace’s Perspective - Oracle Open World 2008 - Marco Gralike

1 comments

Notes on slide 1

4 Favorites