Webinar: ArangoDB 3.8 Preview - Analytics at Scale

Graph and Beyond
One Engine, One Query Language.
All Data Models.
Copyright © ArangoDB Inc., 2021 - Conﬁdential - Do not redistribute
3.8 - Analytics at Scale
1

2
Scalable
Graph
Technology
Kube-Arango
Kubernetes
Integration
Managed
Service
Oasis
Document
Data Model
Key-Value
Data Model
Graph
Data Model
Iterative
Graph
Processing
Pregel
Graph ML
and Analytics
ArangoML
Full-Text
ArangoSearch
AQL
Uniﬁed Engine &
Queries
▸ Single Source of
Truth
▸ Developer Knowledge
▸ Less Maintenance
▸ Faster Performance

Jörg Schad, PhD
● Previous
○ Suki.ai
○ Mesosphere
○ PhD Distributed
DB Systems
● @joerg_schad
@joerg_schad

Chris Woodward
Developer Relations
Engineer
@ArangoDB
● Training
● Development
● Community
● ArangoML
● Twitter: @cw00dw0rd
● Slack: Chris.ArangoDB

Copyright © ArangoDB Inc., 2021 - Conﬁdential - Do not redistribute 5
ArangoDB 3.8 - Analytics at Scale
AQL window operations
Analyze (Timeseries) Data
https://www.arangodb.com/docs/3.8/release-notes-new-features38.html
Weighted Traversal
Advanced Graph Traversals
Pipeline Analyzer
Composing Analyzer Pipelines
Search GEO Support
Leverage Geo functionality
from Views

ArangoDB 3.8 - Analytics at Scale
AQL window operations
Analyze (Timeseries) Data
Weighted Traversal
Advanced Graph Traversals
Pipeline Analyzer
Composing Analyzer Pipelines
Search GEO Support
Leverage Geo functionality
from Views
Disclaimer
We decided to to push back GA to
ﬁrst week of July, but please
download and test RC2!

7
Analytics

AQL Window Operations
Aggregations over related rows
Returning one result per entry (!= Collect)
FOR t IN observations
SORT t.time
WINDOW { preceding: 1, following: 1 }
AGGREGATE rollingAverage = AVG(t.val), rollingSum = SUM(t.val)
WINDOW { preceding: "unbounded", following: 0}
AGGREGATE cumulativeSum = SUM(t.val)
RETURN {
time: t.time,
rollingAverage, // average of the window's values
rollingSum, // sum of the window's values
cumulativeSum // running total
}
https://www.arangodb.com/docs/3.8/aql/operations-window.html

AQL Window Operations
Aggregations over related rows
Returning one result per entry (!= Collect)
SORT t.time
AGGREGATE rollingAverage = AVG(t.val), rollingSum = SUM(t.val)
RETURN {
time: t.time,
val: t.val,
}
https://www.arangodb.com/docs/3.8/aql/operations-window.html
[{
"time": "2021-05-25 07:00:00",
"val": 10,
"rollingAverage": 5,
"rollingSum": 10,
"cumulativeSum": 10
}, …...

10
Window Operations
Row based Aggregation
(adjacent documents)
Range-based Aggregation
(value or duration range)
SORT t.time
AGGREGATE rollingAverage = AVG(t.val),
rollingSum = SUM(t.val)
RETURN {
time: t.time,
val: t.val,
}
WINDOW t.val WITH { preceding: 10, following: 5 }
AGGREGATE rollingAverage = AVG(t.val),
rollingSum = SUM(t.val)
RETURN {
time: t.time,
subject: t.subject,
val: t.val,
rollingAverage,
rollingSum}

11
Graph

Weighted Traversal
Traverse graphs in order of increasing weights
- Order options:
- Weighted, DFS, BFS
FOR x, v, p IN 0..10 OUTBOUND "places/York" GRAPH "kShortestPathsGraph"
OPTIONS {
order: "weighted",
weightAttribute: "travelTime",
uniqueVertices: "path"
}

Weighted Traversal
Traverse graphs in order of increasing weights
- Order options:
- Weighted, DFS, BFS

14
ArangoSearch

FOR d IN v_imdb
SEARCH
ANALYZER(d.description
IN TOKENS('amazing action world alien sci-fi science documental', 'text_en') ||
BOOST(d.description IN TOKENS('galaxy', 'text_en'), 5), 'text_en')
SORT BM25(d) DESC
LIMIT 10
FOR vertex, edge, path IN 1..1 INBOUND d imdb_edges
FILTER path.edges[0].$label == "DIRECTED"
RETURN DISTINCT {
"director" : vertex.name,
"movie" : d.title
}
Full-Text Search
ArangoSearch is a powerful search and similarity ranking
engine natively integrated into ArangoDB. Combine search
with any other data model.
15
ArangoSearch
Graph part
Search part
https://www.arangodb.com/community-server/technical-details/

ArangoSearch Flow
https://www.arangodb.com/community-server/technical-details/

ArangoSearch
Pipeline Analyzer
Chain effects of multiple analyzers into one.
Potential use cases:
▸ Apply ngram tokenization to normalized text
▸ Applying stemming after processing delimited text
AQL Analyzer Ngram Analyzer
Pipeline
Output
Text Input
Pipeline Analyzer

ArangoSearch
GeoJSON Support
▸ GeoJSON Analyzers for indexing coordinate information
▸ GeoJSON Analyzer: Indexes GeoJSON objects
▸ GeoPoint Analyzer: Indexes JSON or array of coordinates
▸ ArangoSearch Geo Functions
▸ Geo_Contains()
▸ Geo_Distance()
▸ Geo_In_Range()
▸ Geo_Intersects()

19
Performance
&
Misc

Merkle Tree Replication
- Faster sync especially for large shards

Monitoring
▸ (Updated) Prometheus compatible endpoint
- GET /_admin/metrics/v2
https://www.arangodb.com/docs/3.8/http/administration-and-monitoring-metrics.html#metrics-api

Monitoring
▸ (Updated) Prometheus compatible endpoint
- GET /_admin/metrics/v2
▸ Default Grafana Dashboards for different personas
- https://github.com/arangodb/dashboards
https://www.arangodb.com/docs/3.8/http/administration-and-monitoring-metrics.html#metrics-api

23
More...
▸ UI and Visualizer Improvements
▸ Default per-query and global mem limit
▸ Metrics 2.0 + dashboards
▸ Arangodump Improvements
▸ Javascript security
▸ AQL bit functions
▸ Sort performance improvements
▸ Agency Memory consumption
▸ Hardware acceleration for Encryption

Get started

Interactive Tutorial
25
https://github.com/arangodb/interactive_tutorials#arangodb-interactive-tutorials
PRs are welcome!

26
Demo Time!

27
Roadmap

28

Roadmap
3.9 4.0 Beyond
DX/UX User Interface, Documentation, Metrics continuous improvements
Data Model Hybrid Smart Graph
Query Capabilities ● ● Custom Pregel GA
● Trigger
● Changefeed
GQL
Cluster ● Improved Alerting Rules ● Optimized replication
● Distributed Transactions
● Cluster 4.0
V8 Engine
Replacement
Ecosystem Bi-Directional Kafka Connector
ArangoML Model Inference (experimental) Graph Library 2.0

Webinar: ArangoDB 3.8 Preview - Analytics at Scale

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Webinar: ArangoDB 3.8 Preview - Analytics at Scale

Similar to Webinar: ArangoDB 3.8 Preview - Analytics at Scale (20)

More from ArangoDB Database

More from ArangoDB Database (20)

Recently uploaded

Recently uploaded (20)

Webinar: ArangoDB 3.8 Preview - Analytics at Scale