This document discusses various ways to structure unstructured data like Wikipedia articles in an Oracle database. It describes using the XMLType data type with different storage options like object-relational, binary XML, and CLOB. Indexing techniques for structured and unstructured XML data are also covered, including structured XML indexes, unstructured XML indexes, and full-text indexes. The key is choosing a balanced design that optimizes for both data insertion performance and query selection performance based on the specific data and use cases.

1. Creating Structure in
Unstructured Data
What is possible, today…?
Marco Gralike

4. “Big Data” = XML ?

5. Challenges are!
Ahum, the problems are!

6. WikiPedia
• One string of XML data with
structured and unstructured
data sections
• Language: English
• Size
: 42,15 GB
• Pages
: 12.961.997
• Date
: 21 Dec 2012

7. Adventures into
the unknown…?

8. Setup
• VirtualBox VM
– OEL 5U8 (64)
– 8 GB RAM
• LaCie Little Big Disk
– RAID 0
– Thunderbolt
• Database
– SGA
– PGA
4GB
2GB

9. My new LaCie LBD is really fast - 

10. Defeat?! - 1.000.000 pages only

11. Status of Technology used

12. XML - Where are we…?
Gartner

13. Performance
Oracle & XML
XQuery
XML API’s
1998
XQuery-Update
XQuery Full-Text
Binary Storage XQJ, Big-Data
& Indexing
XML Storage
& Repository
2001
2004
2007
2013

14. Achieved…?

15. Horizon…? (Oracle NoSQL, XMLDB)
•JSoniq
•Zorba
•JSON support
•In-Memory

16. Building (streaming) Bridges

17. Oracle XML DB
• NO cost option
• C (native, embedded in the kernel)
• XML / XQuery Standards
• Code maintained by Oracle
• JSON / In-Memory

18. XQuery
XQuery
XMLType Abstraction
XMLType Abstraction
DB XQuery
XQuery Rewrite
XQuery Rewrite
SQL Execution
SQL Execution
Relational
Relational
Access
Access
Methods
Methods
Procedural XQuery
Pushdown
Streaming
Streaming
XPath
XPath
Evaluation
Evaluation
XVM
XVM
(use “no query rewrite”)
(use “no query rewrite”)
XMLIndex
XMLIndex
Object-Relational
Binary XML
Relational Storage
Secure Files
Source: S317428: Building Really Scalable XML Applications with Oracle XML DB and Oracle Text

19. What are we talking about?

27. WikiPedia
• Structured & Unstructured
bits and pieces
• A lot of “unbounded”
elements
• Not a lot of restrictions
• The bit with value is in
element “tekst”

28. How do we get this Structured?

31. Strings = small & defined (12c?)
Ename  pointer += 100;

32. <string1/><string2/><string3/>

33. Flexible, Humans
No Design Patterns

34. <small/><verybigggr/><bigger/>

35. <verybigggr>
<empno>1</empno><ename>Marco</ename>
<empno>2</empno>
</verybigggr>
<small/><verybigggr/><bigger/>

40. We need options!

41. “XMLType” Container
In Memory
In Memory
(document)
(document)
CLOB
CLOB
(document)
(document)
Object Relational
Object Relational
(data)
(data)
Binary XML
Binary XML
(data)
(data)

42. XMLType
In Memory
In Memory
(document)
(document)
XOB
XOB
XML Schema
XML Schema

43. XMLType
Binary XML Securefile
Binary XML Securefile
(document/content)
(document/content)
Post Parse
Post Parse
LOB Index
LOB Index

44. XMLType
Object Relational
Object Relational
(content)
(content)
Fully Shredded
Fully Shredded
Indexes
Indexes

45. Something else to Realize !

46. “What is the fastest way to get this
stuff in the database…?”

47. “…it depends…”

48. “So what is the fastest way to get XML
in the database…
… and
useful in my case…?”

49. Garbage IN – Garbage OUT

50. WikiPedia
•
•
•
•
SQL*Loader
Parallel or Direct
Securefile LOB Column
2.5 hours
And no (performant) way
to get the details out…
a.k.a “completely useless”

51. WikiPedia
•
•
•
•
SQL*Loader
Parallel or Direct
Securefile Binary XML
…2.5 hours ???

52. XML Parsing
• SAX
• DOM
- Simple API for XML
- Document Object Module

53. fast
insert performance
CLOB
XMLType
CLOB
(domain) indexes
XMLType
Binary XML
XMLType
Object Relational
fast
select performance

55. So let’s pick a XMLType storage
method…

56. XMLType
Binary XML Securefile
Binary XML Securefile
(document/content)
(document/content)
Post Parse
Post Parse
LOB Index
LOB Index

57. Driving access on CONTENT needs
BTree
Index
Function
based Index
(XPath)
Unstructured
XMLIndex
Structured XMLIndex
Oracle XML
Text Index
BTree
Index

58. Structured Data

59. Structured XMLIndex (SXI)
• CONTENT TABLE(s)
• Based on XMLTABLE syntax
• XMLTABLE construct can be
nested:
Structured
XMLIndex
f (x)
– VIRTUAL column alias
• Can be maintained manually
• Secondary indexes possible
Content
Tables

60. Describe CONTENT TABLE
• A “regular” heap table with columns…
• Ideal for secondary indexes, if needed.

61. Semi-Structured Data

62. Unstructured XMLIndex (UXI)
• PATH TABLE
• Use Path Subsetting
– Full Blown XMLIndex can be BIG
• Token Tables (XDB.X$......)
Unstructured
XMLIndex
f (x)
– Query re-write on Tokens
– Fuzzy Searches, //
– Optimizer Statistics
• Can be maintained manually
– Recorded in Pending Table
• Secondary indexes possible
Path Table

63. Describe PATH TABLE

64. What’s hidden…

65. Binary XML – No Index

66. Binary XML + XMLIndex (SXI)

67. Binary XML + XMLIndex + Sec.Ind.

68. Binary XML + XMLIndex + Sec.Ind.

69. Un-Structured Data

70. XML Full Tekst Index
• Based on Oracle Text Index, XQuery Full Text
• XML Namespace Aware
• XML Semantic aware full text search
– Full-Tekst Selection Expression – contains text
– Logical Full Text Operator – ftor, ftand, ftMildNot
– Context Aware full text search

81. Balanced Design
• Inserts, Updates & Deletes
– XML Future Changes
– Index Maintenance
• Selects
– In Memory
– Via Indexes
• XML Validation
– Strict, Lazy
– Client Side Possibilities

82. Reward
• Optimal performance
• Out performing XML
• Proper design will give
performance increase over
XML handling…
…proper design is still key…

84. References
Oracle XML DB
– http://www.oracle.com/pls/db112/homepage
XML DB FAQ Thread
– http://forums.oracle.com/forums/thread.jspa?
threadID=410714
Personal Blog
– http://www.xmldb.nl
– http://technology.amis.nl

85. References
Daniela Florescu, Oracle Corporation
Advances in XML and XQuery
Sam Idicula, Oracle XML DB Development Team
Binary XML Storage and Query Processing in Oracle
Jinyu Wang, Scott Brewton
Making XML Technology Easier to Use
Joel Spolsky - Joel on Software
Back to Basics

86. References
Oracle XML DB Main page material
• Oracle XML DB : Best Practices to Get Optimal
Performance out of XML Queries (PDF)
• Oracle XML DB : Choosing the Best XMLType
Storage Option for Your Use Case (PDF)
• A Request for Comments for the Oracle Binary
XML Format