Hotsos 2010 - The Ultimate Performance Challenge: How To Make Xml Perform?

The Ultimate Performance Challenge:“How to Make XML Perform ?!”
Marco Gralike

Agenda

“XML is not a ‘fast’ thing, there is a ton of parsing involved. Sorry, I never saw the point in huge XML files – they are many times larger than they should be and the amount of work involved in parsing them is incredible”.
Tom Kyte - Januari 9, 2009, AskTom

“The foundation is there; So why not use it?”
…referring to the Relational Model…
Chris Date- Hotsos keynote, 2009

Relational…

XML…?

Evolution…

If you’re a performance nerd,
this is actually cool…
No one figured out XML yet…
Solving the customer problem…
Back to basics…
Deeper understanding of
the data handling issues…
So why the “Hxxx” XML…?

Agenda

Free Format…”XML is cool”…
(aka no design effort)
Have to uphold the “Coding Granny Argument” (among others meaningful names)
Everyone for themselves…
Waiting for “Codd, Date”…
Square wheels…
What’s spoiling the soup…?

Different data models
XPath models an XML document as
a tree while most general purpose
programming languages
have no native data types for a tree.
Different programming paradigms
XSLT is a functional language, while Java
is object-oriented and Perl is a procedural one.
Impedance Mismatch

Effects, Costs
Unnecessary CPU and Memory
Overhead
A lot of expensive type and
encoding conversions
Impedance Mismatch

Agenda

Containerization

The “Dimensions” in 1 XML doc.
1
3
4
5
2
X
Y
6
Z
nx
rows
Elements with maxoccurs=“unbounded”

Multi Dimensional Issues…
Its a database…
Its Row based
Its Column based
Its multiple databases…
More then 1 XML doc
Not uncommon 1 Mb >>

Complexities of a database
“Relations”
“Redundancy”
“Nullology”
Design, etc…
It can contain a database
10 Mb or bigger nowadays
More often than less…
Enormous complex XSD’s
XMLType – Not just a “Container”

Checked on
XML Well-Formedness
One root element
Begin & End tags
If XML Schema reference
XOB methods will be used if an XML Schema is available
DOM methods will be used if registered
XML Schema information is not available
XMLType – Not just a “Container”

What you want in access…
Fast DDL
Selects
Inserts, Deletes, Updates
Specific / Smart
Small XML Fragments
Direct Access

Agenda

Document contra Data Driven

Structured / Semi-Structured
Structured
Semi
Structured

Common XML Parsers
Often DOM or Infoset based
CPU intensive
Memory intensive
Serializing, parsing, tree traversals, happen in memory…

In Memory: Common XML Parsers
Often handle XML tree traversals only via
ONEmethod
It is not structured, semi-structured or unstructured XML content aware
It is not very “smart” / “content aware” regarding XMLhandling based on its XML tree’s and/or XML data content

XMLType Physical Storage
CLOB
LOB
LOB index
Object Relational
Varray, Types, Nested Tables
IOT, B-Tree, XML Schema
Binary XML
LOB, LOB Index
Stored in Post Parse Representation

Choosing a Storage Model

Hybrid
CLOB
Mixed
complex[n]
un/structured
XSD [y]
B-Tree, IOT
Document
na
unstructured
XSD [n]
XMLIndex
Relational World
XMLDB World
XML Data Storage
XMLType
column/tables
XMLType
Views
Obj.Rel.
Binary XML
Content
complex[n]
structured
XSD [y]
B-Tree, IOT
(Object)
Relational
Objects
Mixed
complex[y]
un/structured
XSD [y/n]
XMLIndex
Relational
Tables

Partition XML data
EMPLOYEES_PROJ_TAB
PROJ_DETAILS_TAB
EMP_PROJ_P11
“employees”.”employee”
reference_id
EMP_PROJ_P12

XML Partitioning
Object Relational Partitioning
Equi-Partitioning since version Oracle 11.1.0.7.0
Binary XML Partitioning
Range, List, Hash
Local partitioned XMLIndex
LOCAL keyword in XMLIndex create syntax
XMLIndex is not supported for HASH partitioning
Partition Key on virtual Column (Binary XML)
Partition Key on column (Object Relational)

Agenda

Index Quick Sheet

Unstructured XMLIndex (UXI)
PathTable
UsePath Subsetting
FullBlown XMLIndex canbe BIG
Token Tables (XDB.X$......)
Query re-writeonTokens
Fuzzy Searches, //
Optimizer Statistics
CanbemaintainedManually
Recorded inPending Table
Secondaryindexespossible
Unstructured
XMLIndex
f (x)
Path Table

PathTable
INDEXED COLUMNS
PATH INDEX
(PATHID, RID), BTREE
ORDER INDEX
(RID, ORDER_KEY), BTREE
VALUE INDEX
(SUBSTRB("VALUE",1,1599))
FUNCTION BASED
NotIndexed: LOCATOR column, pointer to XML fragments (XDB.X$...)
SECONDARY INDEXES
Unstructured
XMLIndex
f (x)
Path Table

Structured XMLIndex (SXI)
Content Table(s)
BasedonXMLTABLE syntax
XMLTable construct canbe
nestedbut:
Only 1 extra XMLType allowed
VIRTUAL column is passed
CanbemaintainedManually
Secondaryindexespossible
Structured
XMLIndex
f (x)
Content
Tables

Content Table(s)
INDEXED COLUMNS
KEY INDEX
(KEY), Unique BTREE
RID INDEX
(RID), Non-Unique BTREE
Indexesneededforcombined XMLIndex Types
Mixing Unstructured and StructuredXMLIndexes
Yourdefined columns
Secondaryindexes
Structured
XMLIndex
f (x)
Content
Tables

Driving access on CONTENT
BTree Index
bookstore
Secondary Oracle Text Index
Function based Index (XPath)
book
whitepaper
StructuredXMLIndex
Unstructured
XMLIndex
title
author
author
chapter
title
author
id
paragraph
content
structured
content
Structured
XMLIndex

There can be only one XMLIndex…

Agenda

Design

XML Schema will be parsed only once
If registered in the XDB Repository
XML Schema will be cached in memory (SGA)
No additional parsing
No additional validation
XML Schema Advantages

XML Document structure is known, therefore
No parsing is needed when loaded from disk into memory
XML OBject (XOB) structures can be applied
Memory footprint is much less compared to DOM structure
Needed specific nodes can now be handled efficiently in memory
XML Schema Advantages

XDB Annotations
Hybrid: CLOB withinOR

XDB Annotations (OR/Binary XML)
Levels
Root, Simpletype, Complextype
xmlns:xdb="http://xmlns.oracle.com/xdb"
xdb:storeVarrayAsTable
xdb:defaultTable
xdb:maintainDom
xdb:maintainOrder
xdb:SQLInline
Oracle V.11.1.0.7.0 - Partitioning
xdb:tableprops

Mixing Logical and Physical Design

XML Schema - Query Rewrite
String
CHAR
String
Float
bookstore
CLOB
VARCHAR2
(20)
book
whitepaper
title
author
author
chapter
title
author
id
paragraph
NUMBER
(15)
content
content

XML Design
Avoid Cyclic References in XML Schemata
For ease of Maintenance: xdb:annotations
Is DOM validation, fidelity needed ?
CPU / XML parsing:
XML Schema validation “overhead” ?
Index maintenance overhead,
when using “disk” solutions
Y
X

Be aware of what you are doing !
Avoid unneeded (full) XML Schema validation
During Storage (Inserts), Generating XML
xdb:MaintainDOM=false
Avoid Impedance mismatch
Java  XML  Java  XML  Relational  XML  Java (“All In One Go Objective”)
Avoid XML fragments
// and/or via XMLEXISTS
Use Indexes
Y
X

Agenda

Keep XML small
Do not use / enforce Pretty Print if not needed
Avoid namespace reference “Overkill”
Most used Namespace is Leading
Use short Namespace References (aliases)
Make XML data as “sparse” as possible
<employee><name>Marco</name></employee>
<employee name=“Marco”/>
XML Data Partitioning
Binary XML if needed
Y
X

Keep XML small (OR specific)
Don’t use “meaning full element names”
64Kb DDL “create table” buffer
ORA 01792 maximum number of columns in a table or view is 1000
Break XML up
Out of Line
CLOB (unstructured)
Not Accessed Data
Don’t create objects if you don’t need it
Use xdb:defaultTable=“” for global types

Holistic Approach (Recap)

Customer Use Case
Memory
/ DOM
Memory
/ DOM
CLOB
Oracle
Advanced Queue
XMLType
BLOB
Process
Checks
Validation
XML Schema
(JAVA)
Store in
ETL Tables
Shred Elements
Via XMLDOM

Duration (1000 Cases)

New XML Approach
Rewrite on Disk
/ XOB
(Relational)
CLOB
Oracle
Advanced Queue
BLOB
Store in
ETL Tables
Oracle
Workflow
Validation
Against
XML Schema
Checks
XMLType Table
(O.R)

Using the CBO as an XML Parser…
ORA-31186
ORA-31186
ORA-31186
ORA-31186: Document contains too many nodes
Cause: Unable to load the document because it has exceeded the maximum allocated number of DOM nodes.

Using the (XML) Relational Mindset
Design XSD as you would with E(E)R
Design for proper physical access, performance:
Storage, Index
Content Awareness
Partitioning
Overkill of “meaning full” data parsing
Avoid Redundancy, whitespace, “Pretty Print”
Design with the future in mind

So in short: Balanced Design
Inserts, Updates & Deletes
XML Future Changes
Index Maintenance
Selects
In Memory
Via Indexes
XML Validation
Strict, Lazy
Client Side Possibilities

Reward
Optimal performance
Out performing XML
Proper design will give
you 10, 100 fold
performance increase over
XML handling…
…also known as…ehh…
…standard relational database performance…

References
Oracle XML DB
http://www.oracle.com/pls/db112/homepage
XML DB FAQ Thread
http://forums.oracle.com/forums/thread.jspa?threadID=410714
Blog
http://technology.amis.nl/blog
http://blog.gralike.com

Hotsos 2010 - The Ultimate Performance Challenge: How To Make Xml Perform?

by Marco Gralike

Accessibility

Categories

Upload Details

Usage Rights

2 Embeds 25

Statistics