Introduction and overview ArangoDB query language AQL

4,699 views
4,398 views

Published on

This presentation covers AQL, ArangoDB's query language.

Published in: Technology
1 Comment
8 Likes
Statistics
Notes
  • why not make this downable ?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
4,699
On SlideShare
0
From Embeds
0
Number of Embeds
45
Actions
Shares
0
Downloads
0
Comments
1
Likes
8
Embeds 0
No embeds

No notes for slide

Introduction and overview ArangoDB query language AQL

  1. 1. © 2013 triAGENS GmbH | 2013-06-06 1ArangoDBquery language(AQL)
  2. 2. © 2013 triAGENS GmbH | 2013-06-06 2Documentation the complete AQL manual for the current versionof ArangoDB can be found online at:https://www.arangodb.org/manuals/current/Aql.html all explanations and examples used in thispresentation have been created with ArangoDBversion 1.3
  3. 3. © 2013 triAGENS GmbH | 2013-06-06 3Motivation – why another query language? initially, we implemented a subset of SQLsSELECT for querying ArangoDB we noticed soon that SQL didnt fit well: ArangoDB is a document database, but SQL is alanguage used in the relational world dealing with multi-valued attributes and creatinghorizontal lists with SQL is quite painful, but weneeded these features
  4. 4. © 2013 triAGENS GmbH | 2013-06-06 4Motivation – why another query language? we looked at UNQL, which addressed some of theproblems, but the project seemed dead and therewere no working UNQL implementations XQuery seemed quite powerful, but a bit toocomplex for simple queries and a firstimplementation JSONiq wasnt there when we started :-)
  5. 5. © 2013 triAGENS GmbH | 2013-06-06 5ArangoDB Query Language (AQL) so we rolled our own query language, theArangoDB Query Language (AQL) it is a declarative language, loosely based on thesyntax of XQuery the language uses other keywords than SQL soits clear that the languages are different AQL is implemented in C and JavaScript first version of AQL was released in mid-2012
  6. 6. © 2013 triAGENS GmbH | 2013-06-06 6Collections and documents ArangoDB is not a relational database, butprimarily a document database documents are organised in "collections" (theequivalent of a "table" in the relational world) documents consist of typed attribute name / valuepairs, attributes can optionally be nested the JSON type system can be used for values
  7. 7. © 2013 triAGENS GmbH | 2013-06-06 7An example document{   "name" : {    "official" : "AQL",    "codename" : "Ahuacatl"  },  "keywords" : [    "FOR", "FILTER", "LIMIT", "SORT",     "RETURN", "COLLECT", "LET"  ],  "released" : 2012}
  8. 8. © 2013 triAGENS GmbH | 2013-06-06 8Schemas collections do not have a fixed schema documents in a collection can be homogenousand have different attributes but: the more homogenous the documentstructures in a collection, the more efficientlyArangoDB can store the documents
  9. 9. © 2013 triAGENS GmbH | 2013-06-06 9AQL data types (primitive types) absence of a value:null boolean truth values:false, true numbers (signed, double precision):1, ­34.24 strings:"John", "goes fishing"
  10. 10. © 2013 triAGENS GmbH | 2013-06-06 10AQL data types (compound types) lists (elements accessible by their position):[ "one", "two", false, ­1 ] documents (elements accessible by their name):{   "user" : {     "name" : "John",     "age" : 25   } }
  11. 11. © 2013 triAGENS GmbH | 2013-06-06 11AQL operators logical: will return a boolean value or an error&&  ||  ! arithmetic: will return a numeric value or an error+  ­  *  /  % comparison: will return a boolean value==  !=  <  <=  >  >=  IN ternary: will return the true or the false part? :
  12. 12. © 2013 triAGENS GmbH | 2013-06-06 12Operators: == vs. = to perform an equality comparison of two values,there is the == operator in AQL in SQL, such comparison is done using the =operator in AQL, the = operator is the assignment operator
  13. 13. © 2013 triAGENS GmbH | 2013-06-06 13Strings the + operator in AQL cannot be used for stringconcatenation string concatenation can be achieved by callingthe CONCAT function string comparison with AQLs comparisonoperators does a binary string comparison special characters in strings (e.g. line breaks andquotes) need to be escaped properly
  14. 14. © 2013 triAGENS GmbH | 2013-06-06 14Type casts type casts can be performed by explicitly callingtype cast functions:TO_BOOL, TO_STRING, TO_NUMBER, ... performing an operation with invalid orinappropriate types will result in an error when performing an operation that does not havea valid or defined result, an error will be thrown:1 / 0         => error1 + "John"    => error
  15. 15. © 2013 triAGENS GmbH | 2013-06-06 15Null when referring to something non-existing (e.g. anon-existing attribute of a document), the resultwill be null:users.nammme  => null using the comparison operators, null can becompared to other values and also null itself the result of such comparison will be a boolean(not null as in SQL)
  16. 16. © 2013 triAGENS GmbH | 2013-06-06 16Type comparisons when two values are compared, the types arecompared first if the types of the compared values are notidentical, the comparison result is determined bythis rule:null < boolean < number < string < list < document
  17. 17. © 2013 triAGENS GmbH | 2013-06-06 17Type comparisons – examplesnull  < false        0     != nullfalse < 0            null  != falsetrue  < 0            false != ""true  < [ 0 ]        ""    != [ ]true  < [ ]          null  != [ ]0     < [ ][ ]   < { }
  18. 18. © 2013 triAGENS GmbH | 2013-06-06 18Type comparisons – primitive types if the types of two compared values are equal, thevalues are compared for boolean values, the order is:false < true for numeric values, the order is determined by thenumeric value for string values, the order is determined bybytewise comparison of the strings characters
  19. 19. © 2013 triAGENS GmbH | 2013-06-06 19Type comparisons – lists for list values, the elements from both lists arecompared at each position for each list element value, a type and valuecomparison will be performed using the describedrules
  20. 20. © 2013 triAGENS GmbH | 2013-06-06 20Type comparisons – lists examples[ 1 ]        >  [ 0 ][ 2, 0 ]     >  [ 1, 2 ][ 23 ]       >  [ true ][ [ 1 ] ]    >  99[ ]          >  1[ null ]     >  [ ][ true, 0 ]  >  [ true ][ 1 ]        == [ 1 ]
  21. 21. © 2013 triAGENS GmbH | 2013-06-06 21Type comparisons – documents for document values, the attribute names fromboth documents are collected and sorted the sorted attribute names are then checkedindividually: if one of the documents does nothave the attribute, it will be considered „smaller“. if both documents have the attribute, a type andvalue comparison will be done using thedescribed rules
  22. 22. © 2013 triAGENS GmbH | 2013-06-06 22Type comparisons – documents examples{ }                 <   { "age": 25 }{ "age": 25 }       <   { "age": 26 }{ "age": 25 }       >   { "name": "John" }{                       {  "name": "John",   ==    "age": 25,  "age": 25               "name": "John" }                       }{ "age": 25 }       <   {                           "age": 25,                          "name": "John"                        }
  23. 23. © 2013 triAGENS GmbH | 2013-06-06 23Base building blocks – lists a good part of the AQL is about list processing there are several types of lists: statically declared lists, e.g.[ 1, 2, 3 ] lists of documents from collections, e.g.userslocations results from filters/subqueries/functions, e.g.NEAR(locations, [ 43, 10 ], 100)
  24. 24. © 2013 triAGENS GmbH | 2013-06-06 24FOR keyword – list iteration the FOR keyword can be used to iterate over allelements of a list the general syntax is:FOR variable IN expression where variable is the name the current listelement can be accessed with, and expression isa valid AQL expression that produces a list
  25. 25. © 2013 triAGENS GmbH | 2013-06-06 25FOR – principle to iterate over all documents in collection users:FOR u IN users a result document (named: u) is produced on eachiteration the example produces the following result list:[ u1, u2, u3, ..., un ]this is equivalent to the following SQL:SELECT u.* FROM users u
  26. 26. © 2013 triAGENS GmbH | 2013-06-06 26FOR – nesting nesting of multiple FOR statements is possible this can be used to perform the equivalent of anSQL inner or cross join there will be as many iterations as the product ofthe number of list elements
  27. 27. © 2013 triAGENS GmbH | 2013-06-06 27FOR – nesting exampleto create the cross product of users andlocations:FOR u IN users  FOR l IN locationsthis is equivalent to the following SQL:SELECT u.*, l.* FROM users u, locations l
  28. 28. © 2013 triAGENS GmbH | 2013-06-06 28FOR – nesting exampleto create the cross product of statically declaredyears and quarters:FOR year IN [ 2011, 2012, 2013 ]  FOR quarter IN [ 1, 2, 3, 4 ]this is equivalent to the following SQL:SELECT * FROM (SELECT 2011 UNION SELECT 2012 UNION   SELECT 2013) year, (SELECT 1 UNION SELECT 2 UNION   SELECT 3 UNION SELECT 4) quarter
  29. 29. © 2013 triAGENS GmbH | 2013-06-06 29FILTER keyword – results filtering the FILTER keyword can be used to restrict theresults to elements that match some definableconditions there can be more than one FILTER statement ina query if multiple FILTER statements are used, theconditions are combined with a logical and
  30. 30. © 2013 triAGENS GmbH | 2013-06-06 30FILTER – exampleto retrieve all users that are active:FOR u IN users  FILTER u.active == truethis is equivalent to the following SQL:SELECT * FROM users uWHERE u.active = true
  31. 31. © 2013 triAGENS GmbH | 2013-06-06 31FILTER – inner join exampleto retrieve all users that have matchinglocations:FOR u IN users FOR l IN locations   FILTER u.id == l.idthis is equivalent to the following SQL:SELECT u.*, l.* FROM users u (INNER) JOIN locations l ON u.id == l.id
  32. 32. © 2013 triAGENS GmbH | 2013-06-06 32Base building blocks – scopes AQL is scoped scopes can be made explicit using parentheses variables in AQL can only be used after they havebeen declared in a scope variables can be used in the scope they havebeen declared in, and also in sub-scopes variables must not be redeclared in a scope or ina sub-scope
  33. 33. © 2013 triAGENS GmbH | 2013-06-06 33Scopes example queryFOR u IN users  /* can use u from here on */  FOR l IN locations    /* can use l from here on */    FILTER u.a == l.b        RETURN u/* the RETURN has ended the scope,    u and l cannot be used after it */
  34. 34. © 2013 triAGENS GmbH | 2013-06-06 34FILTER – results filteringthanks to scopes, the FILTER keyword can beused consistently where SQL needs multiplekeywords: ON WHERE HAVING
  35. 35. © 2013 triAGENS GmbH | 2013-06-06 35FILTER – results filteringin ArangoDB, you would use FILTER...FOR u IN users  FOR l IN locations    FILTER u.id == l.id...whereas in SQL you would use either ON or WHERE:SELECT u.*, l.* FROM users u(INNER) JOIN locations l ON u.id == l.idSELECT u.*, l.* FROM users u, locations lWHERE u.id == l.id
  36. 36. © 2013 triAGENS GmbH | 2013-06-06 36FILTER – results filteringFILTER can be used to model both an SQL ON andWHERE in one go:FOR u IN users  FOR l IN locations    FILTER u.active == 1 &&            u.id == l.idthis is equivalent to the following SQL:SELECT u.*, l.* FROM users u (INNER) JOIN locations lON u.id == l.id WHERE u.active = 1
  37. 37. © 2013 triAGENS GmbH | 2013-06-06 37RETURN keyword – results projection the RETURN keyword produces the result of aquery or subquery it is comparable to SELECT part in an SQL query RETURN is mandatory at the end of a query(and at the end of each subquery)
  38. 38. © 2013 triAGENS GmbH | 2013-06-06 38RETURN – return results without modificationto return all documents as they are in the originallist, there is the following pattern:FOR u IN users  RETURN uthis is equivalent to the following SQL:SELECT u.* FROM users u
  39. 39. © 2013 triAGENS GmbH | 2013-06-06 39RETURN – exampleto return just the names of users, use:FOR u IN users  RETURN u.namenote: this is similar to the following SQL:SELECT u.name FROM users u
  40. 40. © 2013 triAGENS GmbH | 2013-06-06 40RETURN – exampleto return multiple values, create a new list with thedesired attributes:FOR u IN users  RETURN [ u.name, u.age ]
  41. 41. © 2013 triAGENS GmbH | 2013-06-06 41RETURN – exampleRETURN can also produce new documents:FOR u IN users  RETURN {    "name"      : u.name,    "likes"     : u.likes,    "numFriends": LENGTH(u.friends)  }
  42. 42. © 2013 triAGENS GmbH | 2013-06-06 42RETURN – exampleto return data from multiple lists at once, thefollowing query could be used:FOR u IN users  FOR l IN locations    RETURN {       "user": u,       "location" : l     }
  43. 43. © 2013 triAGENS GmbH | 2013-06-06 43RETURN – merging resultsto return a flat result from hierchical data (e.g. datafrom multiple collections), the MERGE function canbe employed:FOR u IN users  FOR l IN locations    RETURN MERGE(u, l)
  44. 44. © 2013 triAGENS GmbH | 2013-06-06 44SORT keyword – sorting the SORT keyword will force a sort of the result inthe current scope according to one or multiplecriteria this is similar to ORDER BY in SQL
  45. 45. © 2013 triAGENS GmbH | 2013-06-06 45SORT – examplesort users by first and last name ascending, then byid descending):FOR u IN users  SORT u.first, u.last, l.id DESC  RETURN uthis is equivalent to the following SQL:SELECT u.* FROM users uORDER BY u.first, u.last, l.id DESC
  46. 46. © 2013 triAGENS GmbH | 2013-06-06 46LIMIT keyword – result set slicing the LIMIT keyword allows slicing the result in thecurrent scope using an offset and a count the general syntax is LIMIT offset, count the offset can be omitted
  47. 47. © 2013 triAGENS GmbH | 2013-06-06 47LIMIT – exampleto return the first 3 users when sorted by name(offset = 0, count = 3):FOR u IN users  SORT u.first, u.last  LIMIT 0, 3  RETURN uthis is equivalent to the following SQL (MySQLdialect):SELECT u.* FROM users uORDER BY u.first, u.last LIMIT 0, 3
  48. 48. © 2013 triAGENS GmbH | 2013-06-06 48LET keyword – variable creation the LET keyword can be used to create a namedvariable using an expression the variable is visible in the scope it is declaredand all sub-scopes, but only after the expressionhas been evaluated let can be used to create scalars and lists(e.g. from a sub-query)
  49. 49. © 2013 triAGENS GmbH | 2013-06-06 49LET – exampleTo return all users with their logins in a horizontal list:FOR u IN users  LET userLogins = (    FOR l IN logins      FILTER l.userId == u.id      RETURN l  )  RETURN {            "user" : u,           "numLogins" : userLogins         }
  50. 50. © 2013 triAGENS GmbH | 2013-06-06 50LET – post-filtering the results created using LET can be filteredregularly using the FILTER keyword this can be used to post-filter values it allows achieving something similar to theHAVING clause in SQL
  51. 51. © 2013 triAGENS GmbH | 2013-06-06 51LET – post-filtering exampleTo return all users with more than 5 logins:FOR u IN users  LET userLogins = (    FOR l IN logins      FILTER l.userId == u.id      RETURN l.id  )  FILTER LENGTH(userLogins) > 5  RETURN u
  52. 52. © 2013 triAGENS GmbH | 2013-06-06 52LET – complex exampleto return all users with more than 5 logins along with the groupmemberships:FOR u IN users  LET userLogins = (FOR l IN logins    FILTER l.userId == u.id    RETURN l  )  FILTER LENGTH(userLogins) > 5  LET userGroups = (FOR g IN groups    FILTER g.id == u.groupId    RETURN g  )  RETURN { "user": u, "logins": userLogins }
  53. 53. © 2013 triAGENS GmbH | 2013-06-06 53COLLECT keyword – grouping the COLLECT keyword can be used to group the data inthe current scope the general syntax is:COLLECT variable = expression ... INTO groups variable is the name of a new variable containing thevalue of the first group criteria. there is one variable pergroup criterion the list of documents in each group can optionally beretrieved into the variable groups using the INTOkeyword
  54. 54. © 2013 triAGENS GmbH | 2013-06-06 54COLLECT keyword – grouping in contrast to SQLs GROUP BY, ArangoDBsCOLLECT performs just grouping, but noaggregation aggregation can be performed later using LET orRETURN the result of COLLECT is a new(grouped/hierarchical) list of documents,containing one document for each group
  55. 55. © 2013 triAGENS GmbH | 2013-06-06 55COLLECT – distinct exampleto retrieve the distinct cities of users:FOR u IN users  COLLECT city = u.city  RETURN {     "city" : city,   }
  56. 56. © 2013 triAGENS GmbH | 2013-06-06 56COLLECT – example without aggregationto retrieve the individual users, grouped by city:FOR u IN users  COLLECT city = u.city INTO g  RETURN {     "city" : city,     "usersInCity" : g   }the usersInCity attribute contains a list of userdocuments per city
  57. 57. © 2013 triAGENS GmbH | 2013-06-06 57COLLECT – example with aggregationto retrieve the cities with the number of users ineach:FOR u IN users  COLLECT city = u.city INTO g  RETURN {     "city" : city,     "numUsersInCity": LENGTH(g)   }
  58. 58. © 2013 triAGENS GmbH | 2013-06-06 58Sub-queries AQL allows sub-queries at any place in the querywhere an expression would be allowed functions that process lists can be run sub-queries sub-queries must always be put into parentheses sometimes this requires using double parentheses
  59. 59. © 2013 triAGENS GmbH | 2013-06-06 59Aggregate functionsAQL provides a few aggregate functions that canbe used with COLLECT, but also with any other listexpression: MIN MAX SUM LENGTH AVERAGE
  60. 60. © 2013 triAGENS GmbH | 2013-06-06 60Aggregate functions with sub-queriesto return the maximum rating per city:FOR u IN users  COLLECT city = u.city INTO cityUsers  RETURN {    "city"      : city,    "maxRating" : MAX((      FOR cityUser IN cityUsers       RETURN cityUser.u.rating    ))}
  61. 61. © 2013 triAGENS GmbH | 2013-06-06 61[*] list expanderthe [*] list expander can be used to access allelements of a list:FOR u IN users  COLLECT city = u.city INTO cityUsers  RETURN {    "city"      : city,    "maxRating" :          MAX(cityUsers[*].u.rating)}
  62. 62. © 2013 triAGENS GmbH | 2013-06-06 62UNION UNION is not a keyword in AQL to create the union of two lists, the UNION function can be used:UNION(list1, list2) the result of UNION is a list again
  63. 63. © 2013 triAGENS GmbH | 2013-06-06 63DISTINCT DISTINCT is not a keyword in AQL to get a list of distinct values from a list, theCOLLECT keyword can be used as show before there is also the UNIQUE function, whichproduces a list of unique elements:UNIQUE(list)
  64. 64. © 2013 triAGENS GmbH | 2013-06-06 64Graphs graphs can be used to model tree structures,networks etc. popular use cases for graph queries: find friends of friends find similarities find recommendations
  65. 65. © 2013 triAGENS GmbH | 2013-06-06 65Graphs – vertices and edges in ArangoDB, a graph is composition of vertices:the nodes in the graph. they are stored asregular documents edges:the relations between the nodes in the graph.they are stored as documents in special "edge"collections
  66. 66. © 2013 triAGENS GmbH | 2013-06-06 66Graphs – edges attributesall edges have the following attributes: _from: id of linked vertex (incoming relation) _to: id of linked vertex (outgoing relation)_from and _to are populated with the _id valuesof the vertices to be connected
  67. 67. © 2013 triAGENS GmbH | 2013-06-06 67Graphs – example datadata for vertex collection "users":[   { "_key": "John", "age": 25 },   { "_key": "Tina", "age": 29 },  { "_key": "Bob",  "age": 15 },  { "_key": "Phil", "age": 12 } ]
  68. 68. © 2013 triAGENS GmbH | 2013-06-06 68Graphs – example datadata for edge collection "relations":[   { "_from": "users/John", "_to": "users/Tina" },  { "_from": "users/John", "_to": "users/Bob" },  { "_from": "users/Bob", "_to": "users/Phil" },  { "_from": "users/Phil", "_to": "users/John" },  { "_from": "users/Phil", "_to": "users/Tina" },  { "_from": "users/Phil", "_to": "users/Bob" } ]
  69. 69. © 2013 triAGENS GmbH | 2013-06-06 69Graph queries – PATHS to find all directly and indirectly connected outgoingrelations for users, the PATHS function can be used PATHS traverses a graphs edges and produces a list of allpaths found each path object returned will have the following attributes: _from: _id of vertex the path started at _to: _id of vertex the path ended with _edges: edges visited along the path _vertices: vertices visited along the path
  70. 70. © 2013 triAGENS GmbH | 2013-06-06 70Graph queries – PATHS exampleFOR u IN users  LET userRelations = (    FOR p IN PATHS(users,                    relations,                    "OUTBOUND")      FILTER p._from == u._id      RETURN p )  RETURN {    "user" : u,    "relations" : userRelations  } 
  71. 71. © 2013 triAGENS GmbH | 2013-06-06 71Comments comments can be embedded at any place in anAQL query comments start with /* and end with */ comments can span multiple lines nesting of comments is not allowed
  72. 72. © 2013 triAGENS GmbH | 2013-06-06 72Attribute names attribute names can be used quoted or non-quoted when used unquoted, the attribute names mustonly consist of letters and digits to maximise compatibility with JSON, attributenames should be quoted in AQL queries, too own attribute names should not start with anunderscore to avoid conflicts with ArangoDBssystem attributes
  73. 73. © 2013 triAGENS GmbH | 2013-06-06 73Bind parameters AQL queries can be parametrised using bindparameters this allows separation of query text and actualquery values (prevent injection) any literal values, including lists and documentscan be bound bind parameters can be accessed in the queryusing the @ prefix to bind a collection name, use the @@ prefix
  74. 74. © 2013 triAGENS GmbH | 2013-06-06 74Bind parameters – example query and valuesFOR c IN @@collection  FILTER c.age > @age &&         c.state IN @states  RETURN { "name" : u.name }@@collection : "users"@age: 30@states: [ "CA", "FL" ]
  75. 75. © 2013 triAGENS GmbH | 2013-06-06 75Overview – main keywordsFOR ... IN FILTERRETURNSORTLIMITLETCOLLECT ... INTOlist iterationresults filteringresults projectionsortingresult set slicingvariable creationgroupingKeyword Use case
  76. 76. © 2013 triAGENS GmbH | 2013-06-05Thank you!Stay in touch:Fork me on githubGoogle group: ArangoDBTwitter: @steemann @arangodbwww.arangodb.orgFoxx – Javascript applicationframework for ArangoDB

×