durable_rules is a framework for real-time event processing and inference using a Rete algorithm. It was created based on the author's personal research and can analyze streaming data to infer new information using rules. The document discusses the history of rule-based systems and expert systems, describes how the Rete algorithm works to efficiently match patterns in data, benchmarks durable_rules' performance against alternative approaches, and provides an example of using it to solve a seating arrangement problem based on guest attributes. Future plans include improving performance, supporting richer queries, and potentially developing it as a web service.

1. durable_rules

2. Who am I?
Why durable_rules?
durable_rules is the result of my personal research, it does not reflect the positions of my
employer.

3. Agenda
• History
• Rete
• Performance Challenge
• Benchmark
• Demo
• Futures
• Q & A

4. History

5. 2400 years ago
Modus Ponens: Theophrastus on Hypothetical Syllogisms1
"P implies Q and P is asserted to be true, so therefore Q must be true.”2
1. x eats flies and x lives in water -> x is a frog
2. Kermit eats flies
3. Kermit lives in water
4. Kermit is a frog (MP 1, 2, 3)
1. https://plato.stanford.edu/entries/logic-ancient/#ForModPonModTol
2. https://en.wikipedia.org/wiki/Modus_ponens

6. An Expert System emulates the decision-making ability of a human. “starts with
the available data and uses inference rules to extract more data (from an end user,
for example) until a goal is reached”1
(deftemplate animal
(slot name))
(defrule rule-1 ”frogs”
(animal (name ?name) (eats flies))
(animal (name ?name) (lives water))
=>
(printout t ?name ” is frog")
(assert (?name is frog)))
> assert (Kermit eats flies)
> assert (Kermit lives water)
> Kermit is frog
1. https://en.wikipedia.org/wiki/Forward_chaining
40 years ago

7. durable_rules is a framework for real-time coordination of events. To analyze
information about things that happen to infer more complicated circumstances.
var d = require('durable');
d.ruleset('animal', function() {
whenAll: {
first = m.predicate == 'eats' && m.object == 'flies'
m.predicate == 'lives' && m.object == 'water' && m.subject == first.subject
}
run: {
console.log(m.subject + ' is frog');
assert({ subject: first.subject, predicate: 'is', object: 'frog' });
}
});
runAll();
>node animals.js
>curl -H "content-type: application/json" -X POST -d '{"subject": ”Kermit", "predicate": "eats", "object": ”flies"}'
>curl -H "content-type: application/json" -X POST -d '{"subject": ”Kermit", "predicate": ”lives", "object": ”land"}'
>Kermit is frog
Today

8. Rete
: a network especially of blood vessels or nerves : 1
The Rete algorithm was designed by Charles L. Forgy of Carnegie Mellon
University, first published in a working paper in 1974. 2
1. https://www.merriam-webster.com/dictionary/rete
2. https://en.wikipedia.org/wiki/Rete_algorithm

9. Kermit eats flies
Classic Rete
predicate == ‘eats’
object == ‘flies’
predicate == ‘lives’
object == ‘water’
subject == subject
assert(frog)
Beta node
Action node
Alpha nodes
• Alpha nodes filter and push forward
• Beta nodes remember and join
• Action nodes schedule execution
> assert(Kermit eats flies)
> assert(Kermit lives water)
> Kermit is frog
Kermit eats flies
Kermit eats flies Kermit lives water
Kermit lives water
Kermit lives water
Kermit eats flies Kermit lives water

10. Rete D
predicate == ‘eats’
object == ‘flies’
predicate == ‘lives’
object == ‘water’
subject == subject
Animal Queue
• Alpha nodes filter in scripting host
• Beta nodes store and join in Redis
• Actions queued in Redis
assert(frog)
• Actions executed in scripting host
RedisScriptScript
• Work Distribution (massive streams)
• Scale out
• Fault Tolerance
• Multi Language (node.js, Python Ruby)
Why?
Up to 10M/ Sec
Up to 100K/Sec
Up to 30K/Sec

11. Performance Challenge
1. We have 100000 objects stored in Redis
2. We have 100 objects in a Node.js client
3. We want to generate object pairs that match: first.subject == second.subject

12. First solution var redis = require('redis'), client = redis.createClient();
// more details here…
var callback = function (err, res) {
var first = JSON.parse(res);
for (var ii = 0; ii < localObjects.length; ++ii) {
if (first.subject == localObjects[ii].subject) {
results.push({first: first, second: localObjects[ii]});
}
}
if (i != 99999) {
++i;
client.lindex('test', i, callback);
}
}
client.lindex('test', i, callback);
“{‘subject’:1, ’object’:’flies’}”
“{‘subject’:2, ’object’:’flies’}”
“{‘subject’:3, ’object’:’flies’}”
“{‘subject’:4, ’object’:’flies’}”
“{‘subject’:5, ’object’:’flies’}”
“{‘subject’:6, ’object’:’flies’}”
“{‘subject’:7, ’object’:’flies’}”
“{‘subject’:8, ’object’:’flies’}”
“{‘subject’:9, ’object’:’flies’}”
“{‘subject’:2, ’object’:’water’}”
“{‘subject’:3, ’object’:’water’}”
, 100000
“{‘subject’:4, ’object’:’flies’}”
“{‘subject’:5, ’object’:’flies’}”
, 100
“{‘subject’:1, ’object’:’flies’}”
“{‘subject’:1, ’object’:’flies’}”
“{‘subject’:1, ’object’:’flies’}”
“{‘subject’:1, ’object’:’flies’}”
“{‘subject’:1, ’object’:’flies’}”
RedisNode.js
Total execution time -> 4067 ms

13. Second solution
var redis = require('redis'), client = redis.createClient();
// more details here…
var callback = function (err, res) {
var first = JSON.parse(res);
results.push({first: first, second: objects[i]});
if (i != localObjects.length - 1) {
++i;
client.hget('test', localObjects[i].subject, callback);
}
}
client.hget('test', localObjects[i].subject, callback);
“{‘subject’:1, ’object’:’flies’}”
“{‘subject’:2, ’object’:’flies’}”
“{‘subject’:3, ’object’:’flies’}”
“{‘subject’:4, ’object’:’flies’}”
“{‘subject’:5, ’object’:’flies’}”
“{‘subject’:6, ’object’:’flies’}”
“{‘subject’:7, ’object’:’flies’}”
“{‘subject’:8, ’object’:’flies’}”
“{‘subject’:9, ’object’:’flies’}”
“{‘subject’:2, ’object’:’water’}”
“{‘subject’:3, ’object’:’water’}”
, 100000
“{‘subject’:4, ’object’:’flies’}”
“{‘subject’:5, ’object’:’flies’}”
, 100
1
2
3
4
5
6
7
8
9
RedisNode.js
“{‘subject’:3, ’object’:’flies’}”
“{‘subject’:2, ’object’:’flies’}”
Total execution time -> 7 ms

14. Third solution
var redis = require('redis'), client = redis.createClient();
// more details here…
client.script('load',
'local results = {}n' +
'for i = 1, #ARGV, 1 don' +
' results[i] = redis.call("hget", "test", ARGV[i])n' +
'endn' +
'return resultsn',
function (err, res) {
var hash = res;
var callback = function (err, res) {
for (var ii = 0; ii < localObjects.length; ++ii) {
results.push({first: JSON.parse(res[ii]), second: localObjects[ii]});
}
};
var args = [hash, 0];
for (ii = 0; ii < localObjects.length; ++ii) {
args.push(localObjects[ii].subject);
}
args.push(callback);
client.evalsha.apply(client, args);
}
);
“{‘subject’:1, ’object’:’flies’}”
“{‘subject’:2, ’object’:’flies’}”
“{‘subject’:3, ’object’:’flies’}”
“{‘subject’:4, ’object’:’flies’}”
“{‘subject’:5, ’object’:’flies’}”
“{‘subject’:6, ’object’:’flies’}”
“{‘subject’:7, ’object’:’flies’}”
“{‘subject’:8, ’object’:’flies’}”
“{‘subject’:9, ’object’:’flies’}”
“{‘subject’:2, ’object’:’water’}”
“{‘subject’:3, ’object’:’water’}”
, 100000
“{‘subject’:4, ’object’:’flies’}”
“{‘subject’:5, ’object’:’flies’}”
, 100
1
2
3
4
5
6
7
8
9
RedisNode.js
Total execution time -> 600 micros
“{‘subject’:2, ’object’:’flies’}”
“{‘subject’:3, ’object’:’flies’}”
“{‘subject’:4, ’object’:’flies’}”
“{‘subject’:5, ’object’:’flies’}”
“{‘subject’:2, ’object’:’flies’}”
“{‘subject’:3, ’object’:’flies’}”
“{‘subject’:4, ’object’:’flies’}”
“{‘subject’:5, ’object’:’flies’}”

15. Results

16. Benchmark
Miss Manners has decided to throw a party. She wants to seat her
guests such that adjacent people are of opposite sex and share at least
one hobby. The problem has to be solved for 8, 16, 32 and 128 guests.

17. The Challenge
assign: {
to: 'make’
whenAll: {
seating = m.t == 'seating' && m.path == true
rightGuest = m.t == 'guest' && m.name == seating.rightGuestName
leftGuest = m.t == 'guest' && m.sex != rightGuest.sex && m.hobby == rightGuest.hobby
none(m.t == 'path' && m.pid == seating.tid && m.guestName ==leftGuest.name)
none(m.t == 'chosen' && m.cid == seating.tid && m.guestName == leftGuest.name && m.hobby == rightGuest.hobby)
}
run: {
assert({ t: 'seating',
tid: s.count,
pid: seating.tid,
path: false,
leftSeat: seating.rightSeat,
leftGuestName: seating.rightGuestName,
rightSeat: seating.rightSeat + 1,
rightGuestName: leftGuest.name });
assert({ t: 'path',
pid: s.count,
seat: seating.rightSeat + 1,
guestName: leftGuest.name });
assert({ t: 'chosen',
cid: seating.tid,
guestName: leftGuest.name,
hobby: rightGuest.hobby });
s.count += 1;
}
}

18. Results

19. Futures
• Performance
• Richer Queries (array support)
• Facts: Queries (facts, antecedents)
• Events: Bloom Filters
• WebSite
• Service

20. Q & A
To learn more:
https://github.com/jruizgit/rules
@jruiztwi