Rete, Rete II, Rete III & Rete in TIBCO BusinessEvents
Rete, Rete II, Rete III & Rete in TIBCO BusinessEvents™ 4th Draft – November 28, 2006 Tim Bass, CISSP Principal Global Architect, Director TIBCO Software Inc.
Our Agenda <ul><li>Rete </li></ul><ul><li>Rete II </li></ul><ul><li>Rete III </li></ul><ul><li>Rete in TIBCO BusinessEvents </li></ul><ul><li>Appendix: Additional Reading </li></ul>
Rete (1/3) <ul><li>The Rete algorithm was designed by Dr. Charles L. Forgy of Carnegie Mellon University, first published in a working paper in 1974, and later elaborated in his 1979 Ph.D. thesis and a 1982 paper. The Rete algorithm creates a decision tree that combines the patterns in all the rules of the knowledge base. </li></ul><ul><li>Rete was intended to improve the speed of forward-chained rule-based systems by limiting the effort required to recompute the conflict set after a rule is fired. </li></ul><ul><li>Rete uses a rooted acyclic directed graph, where the nodes, with the exception of the root, represent patterns, and paths from the root to the leaves represent left-hand sides of rules. At each node is stored information about the facts satisfied by the patterns of the nodes in the paths from the root up to and including this node. This information is a relation representing the possible values of the variables occurring in the patterns in the path. </li></ul><ul><li>A Rete-based rules-engine builds a network of nodes, where each node (except the root) corresponds to a pattern occurring in the left-hand-side of a rule. The path from the root node to a leaf node defines a complete rule left-hand-side. Each node has a memory of facts which satisfy that pattern. </li></ul>
Rete (2/3) <ul><li>As new facts are asserted or modified in, they propagate along the Rete network, causing nodes to be annotated when that fact matches that pattern. When a fact or combination of facts causes all of the patterns for a given rule to be satisfied, a leaf node is reached and the corresponding rule is triggered. </li></ul><ul><li>The Rete algorithm is designed to sacrifice memory for increased speed. In most cases, the speed increase over naïve implementations is several orders of magnitude (because Rete performance is theoretically independent of the number of rules in the system). </li></ul><ul><li>The Rete algorithm keeps up to date the information associated with the nodes in the graph. When a fact is added or removed from working memory, a token representing that fact and operation is entered at the root of the graph and propagated to its leaves modifying as appropriate the information associated with the nodes. </li></ul>
Rete (3/3) <ul><li>Many implementations of Rete use doubly linked lists to maintain bindings. </li></ul><ul><li>In very large Rete implementations, the original Rete algorithm tends to run into memory consumption problems. </li></ul><ul><li>A Rete implementation using the doubly linked list approach would run into performance limitations once the rule count is extremely large or the dataset is extremely large. </li></ul><ul><li>This means that rules with many bindings increased the memory cost and those familiar with the Rete literature observe that many papers on claim the doubly linked list approach to bindings is a limitation of the algorithm. </li></ul><ul><li>However, In Forgy's original paper, he gave a detailed description of how to implement Rete in C using linked lists. </li></ul>More Rete Info available on Wikipedia
Rete II <ul><li>In the 1980s Charles L. Forgy developed a successor to the Rete algorithm named Rete II. Unlike the original Rete algorithm, which is public domain, the Rete II algorithm was not disclosed. </li></ul><ul><li>Rete II claims orders-of-magnitude better performance for more complex problems, using which over comes the memory limitations in Rete of maintaining bindings </li></ul><ul><li>The primary improvement over Rete, based from discussions on the web, is that Rete II over came the issue of rulesets with numerous bindings. </li></ul><ul><li>The new algorithm is as good as the original Rete at handling large numbers of rules, but Rete II is dramatically faster than the original Rete algorithm when dealing with large amounts of data or rapidly changing data. </li></ul><ul><li>Rete II was created (copyright) by Forgy at Production Systems Technologies Inc (PST) </li></ul>More Rete II Info available at PST Inc
Rete III <ul><li>Rete III, successor to Rete II. was developed and copyright by Charles L. Forgy at RulesPower, acquired by Fair Isaac in September 2005. </li></ul><ul><li>Rete III claims even better performance than Rete III for problems that involve both complex rules and large collections of facts. </li></ul>More Rete III Info available at Fair Issac
Notes: Rete II & III <ul><li>Some claim that Rete II and Rete III are just “marketing hype” because Rete optimization is dependent on many factors and implementation dependent. </li></ul><ul><li>Because Rete II and Rete III are proprietary, it has not been feasible to independently evaluate performance Rete algorithmic optimization claims. </li></ul>
Rete in TIBCO BusinessEvents™ (First Draft) <ul><li>TIBCO has modified/optimized Rete in BusinessEvents for higher performance. </li></ul><ul><li>TIBCO’s BE implementation perform very fast pattern matching. For example, if a new fact (i.e. event / object) is asserted, not all rules need to be evaluated (only the rules connected to this fact </li></ul><ul><li>If a fact is modified, only the dependent conditions need to be re-evaluated. </li></ul><ul><li>The efficiency of Rete in TIBCO BE is independent of the number of facts, in most cases. </li></ul>More information on BE available in the BE users manual.
Appendix: Additional Reading <ul><li>Rete-UL </li></ul><ul><ul><li>Production Matching for Large Learning Systems, Robert B. Doorenbos , January 31, 1995 </li></ul></ul>
Tim Bass, CISSP Principal Global Architect, Director [email_address] Event Processing at TIBCO