XQuery
A   t e c h n i c a l          O v e r v i e w

        b y    L o r e n C a h l a n d e r
              A p r i l 2 3 , 2 0 1 1
XML Query
XQuery is a standardized language for
combining documents, databases, Web pages
and almost anything else. It is very widely
implemented. It is powerful and easy to
learn.




                 Loren Cahlander - 2011
Functional Programming

     Functional programming, and in
       particular purely functional
 programming, attempts to minimize or
  eliminate side effects, and is therefore
         considered declarative.



                           Source: Wikipedia
        http://en.wikipedia.org/wiki/Declarative_programming
Two Approaches to
     Computation
                                      1940s…




      John von Neumann                                        Alonzo Church

Manage state with a program counter.              Make functions act like math functions.


       Which is simpler? Which will scale to 10,000 CPUs?
                          Courtesy of Dan McCreary & Associates - 2010
            http://www.danmccreary.com/training/xquery/Functional-Programming.pptx
Imperative vs. Functional
Imperative Programming                               Functional Programming

Focus on a discrete series                           Focus on independent
of sequential steps that                             functions that transform
update the state of a                                data with no regard to
system                                               state management




                          Courtesy of Dan McCreary & Associates - 2010
            http://www.danmccreary.com/training/xquery/Functional-Programming.pptx
Different Approaches
Imperative Programming                               Functional Programming
1.Parameters are data, not                           1.Functions are used as
functions                                            parameters to functions
2.Variables are changeable                           2.Variables are immutable
3.Use of state to manage                             3.Outputs only depend on
calculations                                         the inputs
4.Any object can change                              4.No side effects
the state of any other                               5.Use of recursion
object at any time
5.Use of sequential steps



                          Courtesy of Dan McCreary & Associates - 2010
            http://www.danmccreary.com/training/xquery/Functional-Programming.pptx
Different Thinking
Sequential Processing                                   Parallel Processing




 The output of any                                       Each loop of a
 step can be used in                                   FLWOR statement is
    the next step                                        an independent
                                                              thread

                       Courtesy of Dan McCreary & Associates - 2010
         http://www.danmccreary.com/training/xquery/Functional-Programming.pptx
Data Model
<p>
       <q id="x7">The first q</q>
       <q id="x8">The second q</q>
       <q href="#x7">The third q</q>
</p>




                             Loren Cahlander - 2011
XML Schema


<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="note">
     <xs:complexType>
       <xs:sequence>
          <xs:element name="to" type="xs:string"/>
          <xs:element name="from" type="xs:string"/>
          <xs:element name="heading" type="xs:string"/>
          <xs:element name="body" type="xs:string" minOccurs="0"/>
       </xs:sequence>
     </xs:complexType>
  </xs:element>
</xs:schema>
              Loren Cahlander - 2011
Built In Functions
                             Handling XML Schema datatypes
                           Used by XPath 2.0 and XQuery 1.0
Accessors                                              Functions and Operators on Durations, Dates and Times


The Error Function                                     Functions Related to QNames


The Trace Function                                     Operators on base64Binary and hexBinary


Constructor Functions                                  Operators on NOTATION


Functions and Operators on Numerics                    Functions and Operators on Nodes


Functions on Strings                                   Functions and Operators on Sequences


Functions on anyURI                                    Context Functions


Functions and Operators on Boolean Values




                                            Loren Cahlander - 2011
Path Expressions
  Find the first section of the chapter with the title
  “Starry Night” of the document named “alpha.xml”:

document(“alpha.xml”)//chapter[title = “Starry Night”]/section[1]




                          Loren Cahlander - 2011
Element Constructors

 let $url-params :=
    <params>
      <param name="web-path-to-site">{$web-path-to-site}</param>
      <param name="web-path-to-app">{$web-path-to-app}</param>
      <param name="db-path-to-site">{$db-path-to-site}</param>
      <param name="db-path-to-app">{$db-path-to-app}</param>
      <param name="db-path-to-app-data">{$db-path-to-app-data}</param>
      <param name="web-depth-in-site">{$web-depth-in-site}</param>
    </params>




                          Loren Cahlander - 2011
FLWORPronounced FLOWER


 for var                            in expression
                                                             For and let clauses generate a list of
             at posvar
                                                             variable bindings preserving document
             ,                                               order.
 let   var :=          expression
                                                             Where applies a predicate, eliminating
                                                             some of the bindings. It is more
                                                             efficient to apply a predicate in the
                 where expression                            XPath expression before using a where
                                                             clause.
                                                             Order by changes the document order
                   ,
                                                             based on the expression
                              ascending                      Return is executed for each surviving
order by   expression
                                                             binding, generating an ordered list of
                             descending
                                                             output.

       return expression




                                               Loren Cahlander - 2011
Functions
(:~
    URL Rewriting Module
    Contains functions for all URL Rewriting needs across the entire site
 :)

module namespace url = "http://exist-db.org/ns/url-rewriting/1.0";

declare function url:log-variables($params as element(params)) {
   for $param in $params/param
   let $name := string($param/@name)
   let $value := $param/text()
   return
      util:log("DEBUG", concat("URL Rewriter: ", $name, ":     ", $value))
};




                                   Loren Cahlander - 2011
Conditionals
declare function dates:_processYear($year as xs:string?) as xs:string?
{
! if($year castable as xs:integer)
! then
! ! if(xs:integer($year) < 100)
! ! then
! ! ! if(xs:integer($year) > 50)
! ! ! then concat("19", xs:string(xs:integer($year)))
! ! ! else if(xs:integer($year) = 0)
! ! ! then "1900"
! ! ! else concat("20", string-pad("0", 2 - string-length($year)), $year)
! ! else if(xs:integer($year) >= 100 and xs:integer($year) < 200) (: 102 = 2002 in Java :)
! ! then xs:string(1900 + xs:integer($year))
! ! else if(xs:integer($year) > 9999)
! ! then "9999"
! ! else $year
! else $year
};

                                     Loren Cahlander - 2011
Examples


http://en.wikibooks.org/wiki/XQuery




             Loren Cahlander - 2011
XQuery Wikibook
                         FLWOR Expression Example

Motivation
You have a sequence of items and you want to create a report that contains these items.


Method
We will use a basic XQuery FLWOR expression to iterate through each of the items in a sequence. The five parts of a FLWOR expression are:
 ■    for - specifies what items in the sequence you want to select (optional)
 ■    let - used to create temporary names used in the return (optional)
 ■    where - limit items returned (optional)
 ■    order - change the order of the results (optional)
 ■    return - specify the structure of the data returned (required)
Here is a simple example of a FLWOR expression:
for $book in doc("catalog.xml")/books/book
   let $title := $book/title/text()
   let $price := $book/price/text()
   where xs:decimal($price) gt 50.00
   order by $title
   return
      <book>
          <title>{$title}</title>
          <price>{$price}</price>
      </book>


This XQuery FLWOR expression will return all books that have a price over $50.00. Note that we have not just one but two let statements after the for loop.
We also add a where clauses to restrict the results to books over $50.00. The results are sorted by the title and the result is a new sequence of book items
with both the price and the title in them.


                                                 http://en.wikibooks.org/wiki/XQuery/FLWOR_Expression
XQDoc
                          Documentation Generation


(:~
: The controller for constructing the xqDoc HTML information for
: the specified library module. The following information for
: each library module will be generated.
: <ul>
: <li> Module introductory information</li>
: <li> Global variables declared in this module</li>
: <li> Modules imported by this module</li>
: <li> Summary information for each function defined in the module</li>
: <li> Detailed information for each function defined in the module</li>
: </ul>
:
: @param $uri the URI for the library module
: @param $local indicates whether to build static HTML link for offline
: viewing or dynamic links for real-time viewing.
: @return XHTML.
:)
define function print-module($uri as xs:string, $local as xs:boolean) as element()*

                                 Loren Cahlander - 2011
XML Query Test Suite


  The XML Query Test Suite (XQTS) is to show that the
actual specification can be implemented as written, and
 to demonstrate interoperability. W3C does not test or
               certify implementations.




                     Loren Cahlander - 2011
Implementations

       SAXON

Qexo




               http://www.w3.org/XML/Query/#implementations
EXPath/EXQuery
EXQuery
(:~ Collaboratively Defining Open Standards for Portable XQuery Applications :)




EXQuery will define standards for -
● Extending XQuery for Application Development.
● Portable XQuery Applications.




EXQuery is -
● Open – everything is transparent and visible.
● Collaborative – anyone may join us.

● Community driven – standards from XQuery Application Developers.




                 ...launching TODAY at XML Prague 2009 :-)

                                                       www.exquery.org
XML Prague 2009 Poster Session
Adam Retter
adam.retter@exquery.org
                                       Loren Cahlander - 2011
XQuery 3.0
                               W3C Working Draft


                    group by clause in FLWOR Expressions
          tumbling window and sliding window in FLWOR Expressions
                      count clause in FLWOR Expressions
allowing empty in 3.9.2 For Clause, for functionality similar to outer joins in SQL.
                              try/catch expressions
                          Dynamic function invocation
                                 Inline functions
                                Private functions
                           Nondeterministic functions
                               Switch expressions
                       Computed namespace constructors
                               Output declarations
                                   Annotations
                      Annotation assertions in function tests



                                  Loren Cahlander - 2011
Documentation
                             XQuery 1.0: An XML Query Language (W3C Recommendation)
                             http://www.w3.org/TR/xquery/
                             XML Query (XQuery) 1.0 Requirements (W3C Working Group Note)
                             http://www.w3.org/TR/xquery-requirements/
                             XML Query 1.0 Use Cases (W3C Working Group Note)
                             http://www.w3.org/TR/xquery-use-cases/
                             Building a Tokenizer for XPath or XQuery (Joint Note)
                             http://www.w3.org/TR/xquery-xpath-parsing/
                             XQuery 3.0: An XML Query Language (W3C Working Draft)
                             http://www.w3.org/TR/xquery-30/

XQuery
                                  xqDoc
                                  http://xqdoc.org/index.html
Search Across a Variety of XML Data
By Priscilla Walmsley        XML Query Test Suite
Publisher: O'Reilly Media
Released: March 2007         http://dev.w3.org/2006/xquery-test-suite/PublicPagesStagingArea/
                             EXPath/EXQuery
                             http://www.expath.org/
                             http://www.exquery.org/


                                         Loren Cahlander - 2011
Q&A

XQuery - a technical overview

  • 1.
    XQuery A t e c h n i c a l O v e r v i e w b y L o r e n C a h l a n d e r A p r i l 2 3 , 2 0 1 1
  • 2.
    XML Query XQuery isa standardized language for combining documents, databases, Web pages and almost anything else. It is very widely implemented. It is powerful and easy to learn. Loren Cahlander - 2011
  • 3.
    Functional Programming Functional programming, and in particular purely functional programming, attempts to minimize or eliminate side effects, and is therefore considered declarative. Source: Wikipedia http://en.wikipedia.org/wiki/Declarative_programming
  • 4.
    Two Approaches to Computation 1940s… John von Neumann Alonzo Church Manage state with a program counter. Make functions act like math functions. Which is simpler? Which will scale to 10,000 CPUs? Courtesy of Dan McCreary & Associates - 2010 http://www.danmccreary.com/training/xquery/Functional-Programming.pptx
  • 5.
    Imperative vs. Functional ImperativeProgramming Functional Programming Focus on a discrete series Focus on independent of sequential steps that functions that transform update the state of a data with no regard to system state management Courtesy of Dan McCreary & Associates - 2010 http://www.danmccreary.com/training/xquery/Functional-Programming.pptx
  • 6.
    Different Approaches Imperative Programming Functional Programming 1.Parameters are data, not 1.Functions are used as functions parameters to functions 2.Variables are changeable 2.Variables are immutable 3.Use of state to manage 3.Outputs only depend on calculations the inputs 4.Any object can change 4.No side effects the state of any other 5.Use of recursion object at any time 5.Use of sequential steps Courtesy of Dan McCreary & Associates - 2010 http://www.danmccreary.com/training/xquery/Functional-Programming.pptx
  • 7.
    Different Thinking Sequential Processing Parallel Processing The output of any Each loop of a step can be used in FLWOR statement is the next step an independent thread Courtesy of Dan McCreary & Associates - 2010 http://www.danmccreary.com/training/xquery/Functional-Programming.pptx
  • 8.
    Data Model <p> <q id="x7">The first q</q> <q id="x8">The second q</q> <q href="#x7">The third q</q> </p> Loren Cahlander - 2011
  • 9.
    XML Schema <?xml version="1.0"encoding="UTF-8"?> <xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"> <xs:element name="note"> <xs:complexType> <xs:sequence> <xs:element name="to" type="xs:string"/> <xs:element name="from" type="xs:string"/> <xs:element name="heading" type="xs:string"/> <xs:element name="body" type="xs:string" minOccurs="0"/> </xs:sequence> </xs:complexType> </xs:element> </xs:schema> Loren Cahlander - 2011
  • 10.
    Built In Functions Handling XML Schema datatypes Used by XPath 2.0 and XQuery 1.0 Accessors Functions and Operators on Durations, Dates and Times The Error Function Functions Related to QNames The Trace Function Operators on base64Binary and hexBinary Constructor Functions Operators on NOTATION Functions and Operators on Numerics Functions and Operators on Nodes Functions on Strings Functions and Operators on Sequences Functions on anyURI Context Functions Functions and Operators on Boolean Values Loren Cahlander - 2011
  • 11.
    Path Expressions Find the first section of the chapter with the title “Starry Night” of the document named “alpha.xml”: document(“alpha.xml”)//chapter[title = “Starry Night”]/section[1] Loren Cahlander - 2011
  • 12.
    Element Constructors let$url-params := <params> <param name="web-path-to-site">{$web-path-to-site}</param> <param name="web-path-to-app">{$web-path-to-app}</param> <param name="db-path-to-site">{$db-path-to-site}</param> <param name="db-path-to-app">{$db-path-to-app}</param> <param name="db-path-to-app-data">{$db-path-to-app-data}</param> <param name="web-depth-in-site">{$web-depth-in-site}</param> </params> Loren Cahlander - 2011
  • 13.
    FLWORPronounced FLOWER forvar in expression For and let clauses generate a list of at posvar variable bindings preserving document , order. let var := expression Where applies a predicate, eliminating some of the bindings. It is more efficient to apply a predicate in the where expression XPath expression before using a where clause. Order by changes the document order , based on the expression ascending Return is executed for each surviving order by expression binding, generating an ordered list of descending output. return expression Loren Cahlander - 2011
  • 14.
    Functions (:~ URL Rewriting Module Contains functions for all URL Rewriting needs across the entire site :) module namespace url = "http://exist-db.org/ns/url-rewriting/1.0"; declare function url:log-variables($params as element(params)) { for $param in $params/param let $name := string($param/@name) let $value := $param/text() return util:log("DEBUG", concat("URL Rewriter: ", $name, ": ", $value)) }; Loren Cahlander - 2011
  • 15.
    Conditionals declare function dates:_processYear($yearas xs:string?) as xs:string? { ! if($year castable as xs:integer) ! then ! ! if(xs:integer($year) < 100) ! ! then ! ! ! if(xs:integer($year) > 50) ! ! ! then concat("19", xs:string(xs:integer($year))) ! ! ! else if(xs:integer($year) = 0) ! ! ! then "1900" ! ! ! else concat("20", string-pad("0", 2 - string-length($year)), $year) ! ! else if(xs:integer($year) >= 100 and xs:integer($year) < 200) (: 102 = 2002 in Java :) ! ! then xs:string(1900 + xs:integer($year)) ! ! else if(xs:integer($year) > 9999) ! ! then "9999" ! ! else $year ! else $year }; Loren Cahlander - 2011
  • 16.
  • 17.
    XQuery Wikibook FLWOR Expression Example Motivation You have a sequence of items and you want to create a report that contains these items. Method We will use a basic XQuery FLWOR expression to iterate through each of the items in a sequence. The five parts of a FLWOR expression are: ■ for - specifies what items in the sequence you want to select (optional) ■ let - used to create temporary names used in the return (optional) ■ where - limit items returned (optional) ■ order - change the order of the results (optional) ■ return - specify the structure of the data returned (required) Here is a simple example of a FLWOR expression: for $book in doc("catalog.xml")/books/book let $title := $book/title/text() let $price := $book/price/text() where xs:decimal($price) gt 50.00 order by $title return <book> <title>{$title}</title> <price>{$price}</price> </book> This XQuery FLWOR expression will return all books that have a price over $50.00. Note that we have not just one but two let statements after the for loop. We also add a where clauses to restrict the results to books over $50.00. The results are sorted by the title and the result is a new sequence of book items with both the price and the title in them. http://en.wikibooks.org/wiki/XQuery/FLWOR_Expression
  • 18.
    XQDoc Documentation Generation (:~ : The controller for constructing the xqDoc HTML information for : the specified library module. The following information for : each library module will be generated. : <ul> : <li> Module introductory information</li> : <li> Global variables declared in this module</li> : <li> Modules imported by this module</li> : <li> Summary information for each function defined in the module</li> : <li> Detailed information for each function defined in the module</li> : </ul> : : @param $uri the URI for the library module : @param $local indicates whether to build static HTML link for offline : viewing or dynamic links for real-time viewing. : @return XHTML. :) define function print-module($uri as xs:string, $local as xs:boolean) as element()* Loren Cahlander - 2011
  • 19.
    XML Query TestSuite The XML Query Test Suite (XQTS) is to show that the actual specification can be implemented as written, and to demonstrate interoperability. W3C does not test or certify implementations. Loren Cahlander - 2011
  • 20.
    Implementations SAXON Qexo http://www.w3.org/XML/Query/#implementations
  • 21.
    EXPath/EXQuery EXQuery (:~ Collaboratively DefiningOpen Standards for Portable XQuery Applications :) EXQuery will define standards for - ● Extending XQuery for Application Development. ● Portable XQuery Applications. EXQuery is - ● Open – everything is transparent and visible. ● Collaborative – anyone may join us. ● Community driven – standards from XQuery Application Developers. ...launching TODAY at XML Prague 2009 :-) www.exquery.org XML Prague 2009 Poster Session Adam Retter adam.retter@exquery.org Loren Cahlander - 2011
  • 22.
    XQuery 3.0 W3C Working Draft group by clause in FLWOR Expressions tumbling window and sliding window in FLWOR Expressions count clause in FLWOR Expressions allowing empty in 3.9.2 For Clause, for functionality similar to outer joins in SQL. try/catch expressions Dynamic function invocation Inline functions Private functions Nondeterministic functions Switch expressions Computed namespace constructors Output declarations Annotations Annotation assertions in function tests Loren Cahlander - 2011
  • 23.
    Documentation XQuery 1.0: An XML Query Language (W3C Recommendation) http://www.w3.org/TR/xquery/ XML Query (XQuery) 1.0 Requirements (W3C Working Group Note) http://www.w3.org/TR/xquery-requirements/ XML Query 1.0 Use Cases (W3C Working Group Note) http://www.w3.org/TR/xquery-use-cases/ Building a Tokenizer for XPath or XQuery (Joint Note) http://www.w3.org/TR/xquery-xpath-parsing/ XQuery 3.0: An XML Query Language (W3C Working Draft) http://www.w3.org/TR/xquery-30/ XQuery xqDoc http://xqdoc.org/index.html Search Across a Variety of XML Data By Priscilla Walmsley XML Query Test Suite Publisher: O'Reilly Media Released: March 2007 http://dev.w3.org/2006/xquery-test-suite/PublicPagesStagingArea/ EXPath/EXQuery http://www.expath.org/ http://www.exquery.org/ Loren Cahlander - 2011
  • 24.