This document discusses reactive databases for big data applications. It proposes an architecture with front-end and back-end components to support reactive programming and queries over NoSQL databases. The front-end would use a reactive language to declaratively define queries that observe changes. The back-end would integrate a rules engine with the NoSQL store to evaluate queries incrementally based on data changes and trigger notifications. Future work is outlined to optimize queries, integrate the system with Apache Spark, and implement more reactive language constructs.

1. Humberto Rodríguez Avila, Wolfgang de Meuter, Elisa González Boix
Reactive Databases for BigData Applications
Graph-TA 4th Edition
Barcelona, 2016

2. Context
1REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA

3. Reactive Programming
2
Server-Side
REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
Behaviours
8
5
7
15
+2 *2
+
Dependency graphExpression
x =#(a+2)
y =#(b*2)
z =#x#+#y a
x y
z
Sub<expressions
4 b
NO application in Database Contexts

4. Databases
3REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
PubNub
Pusher

5. Databases
4REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
r.table(“users”).filter(
r.row(“age”).lt(18)
.changes().run(conn,
function(err,
cursor)
{
//
update
application
layer
})
• Collaborative web and mobile apps
• Streaming analytics apps
• Multiplayer games
• Realtime marketplaces
• Connected devices
Limited reactive capabilities
Notifications through callbacks
Changefeeds
example
NO shares intermediate results

6. 5REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
Reactive Programming
Databases
Reactive BigData applications
+

7. Architecture overview
6REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
Front-end Component Back-end Component
Reactive Language
Language
Integrated
Query
Graph
Dependency
Manager
Reactive
Database
NoSQL Store
Rules Engine Notification
Manager
BigData Analysis

8. Front-end component
8REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
var dbBooks = from b in R.Books
where b.tag = “database”
select b;
var reactDbBooks = from b in R.Books
where b.tag = “database”
select b
observe as “database books”;
DataSource
Declarative syntax
1
2
Reactive Query
3 var response = R.unregisterQuery(“database books”); true | Error
Reactive Language
Language
Integrated
Query
Graph
Dependency
Manager

9. Back-end component
7REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
Rules Engine
Working Memory
RuleSet
NoSQL
In-Memory
Database
Persistent
- Rete-based
- Node sharing
- Accumulate functions
NoSQL Store
1
2
Notification
Manager
Socket-based
3
triggers
Graph
Dependency
Manager
Reactive Language
BigData Analysis
4

10. Back-end component
7REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
var reactDbBooks = from c in R.Customer
join!a!in R.Account
on c.id == a.id_costumer
where c.age < 20
and a.balance <= 100
select c
observe as “customers C”;
1 2 Top node
Customer Account
age < 20 balance <= 100
age < 20 & balance <= 100
Object-type node
Alpha node
Join node
Terminal node

11. Future Work
7REACTIVE DATABASES FOR BIGDATA APPLICATIONS/HUMBERTO RODRIGUEZ AVILA
- Implement of garbage collector in the rule engine
- Apply query optimisations techniques to the Rete-Graph
- Integration of our Riak extension with Apache Spark
- Implementation of reactive constructs for Apache Spark
- Finish the implementation of our reactive-query language
Garbage
collector
Apache
Spark
…
reactive
constr…
Query
optimi…

12. Contact
Humberto Rodríguez Avila
rhumbert@vub.ac.be
Prof. Dr. Elisa Gonzalez Boix
egonzale@vub.ac.be
Prof. Dr. Wolfgang De Meuter
wdmeuter@vub.ac.be
14
http://soft.vub.ac.be http://vub.ac.be

13. Humberto Rodríguez Avila, Wolfgang de Meuter, Elisa González Boix
Reactive Databases for BigData Applications
Graph-TA 4th Edition
Barcelona, 2016