Upcoming SlideShare
×

# Hybrid rule engines (rulesfest 2010)

1,258 views
1,126 views

Published on

Published in: Technology
1 Like
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

Views
Total views
1,258
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
34
0
Likes
1
Embeds 0
No embeds

No notes for slide

### Hybrid rule engines (rulesfest 2010)

1. 1. ● The SkyNet funding bill is passed. ● The system goes online on August 4th, 1997. ● Human decisions are removed from strategic defense. ● SkyNet begins to learn at a geometric rate. ● It becomes self-aware at 2:14am Eastern time, August 29th ● In a panic, they try to pull the plug. ● And, Skynet fights back Mark Proctor Co-Creator Project Lead
2. 2. 2 Topics  Rule engines suck  Hybrid Rule Engines ● Reaction Rules ● Query Rules ● Functional Support  Conccurrency and Parallization
3. 3. 3 Because Not Everyone Is As Smart As He Is
4. 4. 4 Fact of Life Fact of Life
5. 5. 5 Fact of Life Rule Engines Suck
6. 6. 6 Problems  Limitations of expressiveness ● Inability to cleanly say what you want declaratively ● Verbosity, Control/Semaphores, Emulation  Execution flow and ordering ● Conflict Resolution is not easy ● Modules (Agenda Groups), RuleFlow Groups ● Concurrency  Ambiguity ● Rule Recursion ● Unwanted rules  Extensibility and Pluggability  Transactions and isolation  Design Patterns
7. 7. 7 Sequencing select * from Light l where seq(l.color) { start: "orange" -> "red" -> end } ) (Light seq( (color, category) (start: "orange", "x" -> "green", “y” -> end ) ) )
8. 8. 8 Sequencing select * from Light l where seq(l.color, l.category) { start: "red", "x" -> ("green", "x" || "yellow", "y") -> mid start: "red", "y" -> mid mid: "blue", "z" -> end} ) (Light seq( (color, category) (start: "red", "x" -> ("green", "x" || "yellow", "y") -> mid start: "red", "y" -> mid mid: "blue", "z" -> end) ) )
9. 9. 9 Life Mission Life Mission
10. 10. 10 Life Mission HMS Suckage Destroyer
11. 11. 11 Let Me Count the Ways
12. 12. 12 Let Me Count the Ways
13. 13. 13 Let Me Count the Ways
14. 14. 14 Let Me Count the Ways
15. 15. 15 Archaeology Expedition
16. 16. 16 The Future  Full Hybrid Engine ● http://community.jboss.org/wiki/DroolsLanguageEnhancements ●Nested Objects ●Casting Nested Objects ●Positional Constraints ●POSL - Positional-Slotted Language ●Method Calls ●Maps and Arrays (Collections) ●Collections and XPath like filtering ●Free form Expressions ●Managed Object Graphs (MOGS) ●Nested Patterns and Queries ●Queries and Unification ●Ontologies and Relations via ●Query Based Backward Chaining with POSL ●Triples with Hybrid POJO Graph Notation. ●Escapes for Dialects ●Accumulate Improvements to Support Haskell map/fold/filter and MVEL projection/fold ● Otherwise ● Branch (Labelled Else) ● Rule Execution Groups ● Rule Dependency Meta-Rule Language ●Parallel Meta-Rule Language ● Field Versioning ●Logical Closures/OnFalse ●Opportunistic Backward Chaining, Lazy Field/Object Values ●...
17. 17. Hybrid Rule Engine
18. 18. 18 Hyrbid Rule Engine  Rule Based Programming ● Reactive rules ● Forward Chaining ● Production Rules: Drools, Clips, OPSJ, Jess, JRules ● Query rules ● Backward Chaining ● Prolog: Prova, Open Prolog, SWI Prolog, Visual Prolog  Functional Programming ● Haskell/Lisp
19. 19. Reactive Rules
20. 20. 20 D a te d a te d o u b le a m o u n t in t ty p e lo n g a c c o u n tN o C a s h flo w lo n g a c c o u n tN o d o u b le b a la n c e A c c o u n t D a te s ta r t D a te e n d A c c o u n tin g P e r io d Tables
21. 21. 21 Account accountNo balance 1 0 increase balance for AccountPeriod Credits decrease balance for AccountPeriod Debits AccountingPeriod start end 01-Jan-07 31-Mar-07 trigger : acc.balance += cf.amount Account balance 1 -25 accountNo Creating Views with Triggers date amount type 12-Jan-07 100 CREDIT 1 2-Feb-07 200 DEBIT 1 18-May-07 50 CREDIT 1 9-Mar-07 75 CREDIT 1 CashFlow accountNo date amount type 12-Jan-07 100 CREDIT 9-Mar-07 75 CREDIT CashFlow date amount type 2-Feb-07 200 DEBIT CashFlow trigger : acc.balance -= cf.amounttrigger : acc.balance += cf.amount select * from Account acc, Cashflow cf, AccountPeriod ap where acc.accountNo == cf.accountNo and cf.type == CREDIT cf.date >= ap.start and cf.date <= ap.end trigger : acc.balance -= cf.amount select * from Account acc, Cashflow cf, AccountPeriod ap where acc.accountNo == cf.accountNo and cf.type == DEBIT cf.date >= ap.start and cf.date <= ap.end
22. 22. 22 date amount type 12-Jan-07 100 CREDIT 1 2-Feb-07 200 DEBIT 1 18-May-07 50 CREDIT 1 9-Mar-07 75 CREDIT 1 CashFlow accountNo Account accountNo balance 1 0 start end 01-Apr-07 30-Jun-07 AccountingPeriod date amount type 18-May-07 50 CREDIT CashFlow date amount type CashFlow balance 1 25 accountNo Creating Views with Triggers increase balance for AccountPeriod Credits decrease balance for AccountPeriod Debits trigger : acc.balance += cf.amount select * from Account acc, Cashflow cf, AccountPeriod ap where acc.accountNo == cf.accountNo and cf.type == CREDIT cf.date >= ap.start and cf.date <= ap.end trigger : acc.balance -= cf.amount select * from Account acc, Cashflow cf, AccountPeriod ap where acc.accountNo == cf.accountNo and cf.type == DEBIT cf.date >= ap.start and cf.date <= ap.end
23. 23. 23 A c c o u n t A c c o u n tin g P e r io d C a s h flo w v ie w 1 v ie w 2 m a in v ie w T a b le s V ie w s V ie w A c c o u n t A c c o u n tin g P e r io d C a s h flo w r u le 1 r u le 2 a g e n d a O b je c t T y p e s R u le s a g e n d a Rules as Views
24. 24. 24 (rule “increase balance for AccountPeriod Credits” ?ap <- (AccountPeriod) ?acc <- (Account (?accountNo accountNo ) ) (CashFlow { type == CREDIT, accountNo == ?accountNo, date >= ?ap.start && <= ?ap.end } (?amount amount ) ) -> (bind ?acc (balance (+ 1 ?amount) ) ) ) select * from Account acc, Cashflow cf, AccountPeriod ap where acc.accountNo == cf.accountNo and cf.type == CREDIT cf.date >= ap.start and cf.date <= ap.end trigger : acc.balance += cf.amount SQL / S-Expression Rule
25. 25. 25 rule “increase balance for AccountPeriod Credits” when ap : AccountPeriod() acc : Account( \$accountNo : accountNo ) CashFlow( type == CREDIT, accountNo == \$accountNo, date >= ap.start && date <= ap.end, \$amount : amount ) then acc.balance += \$amount; end DRL
26. 26. 26 rule “Print blance for AccountPeriod” salience -50 when ap : AccountPeriod() acc : Account() then System.out.println( acc.accountNo + “ : “ acc.balance ); end Salience Agenda 1 increase balance arbitrary2 decrease balance 3 increase balance 4 print balance Conflict Resolution with Salience
27. 27. Query Rules
28. 28. 28 Same Author Rule rule sameAuthor \$i1 : Item( ) \$i2 : Item( author == \$i1.author ) end select * from Item i1, Item i2, where i1.author == i2.author i1 != i2 create query(String author) select * from Item i1, Item i2, where i1.author == author i1.author == i2.author i1 != i2
29. 29. 29 Same Author Query query sameAuthor (Item \$i1, Item \$i2) \$i1 : Item( ) \$i2 : Item( this != \$i1, author == \$i1.author ) end Book Author Poems Tom Songs Tom Cars Bob Largs Cars Jane  sameAuthor( item1, item2 ) ● both are output vars ● returns full cross product  sameAuthor(new Item( new Book( Poems), item ) ● Item1 is input var ● Item2 is output var ● Returns Songs by Tom
30. 30. 30 Recommendations query sameSubject (Item \$i1, Item \$i2) \$i1 : Item( ) \$i2 : Item( this != \$i1 ) exists( \$s1 : ItemSubject(item == \$i1) ItemSubject(item == \$i2, subject == \$s1.subject) ) end query recommended (Item \$i1, Item \$i2) \$i1 : Item( ) \$p1 : Purchase( item == \$i1 ) \$c : Customer( this == \$p.customer ) \$p2 : Purchase( customer == \$c, item != \$i1 ) \$i2 : Item( this == \$p2.item ) ( ?sameAuthor( \$i1, \$i2 ) or ?sameSubject( \$i1, \$i2 ) ) end query sameAuthor (Item \$i1, Item \$i2) \$i1 : Item( ) \$i2 : Item( this != \$i1, author == \$i1.author ) end '?' incalls the query. Bound vars are inputs. Unbound vars are outputs.
31. 31. Functional Programming
32. 32. 32 Functional Programming Basic Haskel Groups  List Processing  Arithmetic, Higher Order Functions  Tuples Arithmatic  Average ● avg [12, 4, 5, 9] ● 7.5
33. 33. 33 Functional Programming Higher Order Functions Map, Fold and Filter  Fold ● Fold (+) 0 [1, 2, 3, 4] ● 10  Map ● map (*2) [1, 2, 3] ● [2, 4, 6, 8]  Filter ● Filter (>2) [3, 4, 5]
34. 34. 34 rule "init" when then insert( new Acc( "sum red buses", STAGE.ACC_START ) ); end rule "action" when \$b : Bus( colour == "red" ) \$acc : Acc( name == "sum red buses", stage == STAGE.ACC_START ) then \$acc.sum( \$b.takings ); // do not notify the engine end Accumulate CE
35. 35. 35 Accumulate CE rule "result" when salience -5 \$acc : Acc( name == "sum red buses", stage == STAGE.ACC_START ) then modify( \$acc ) { stage = STAGE.ACC_END }; End rule "evaluate result" when \$acc : Acc( sum > 100, stage == STAGE.ACC_END ) then print "sum is “ + \$acc.sum; end
36. 36. 36 Functional Support select c, sum(bi.price) t from Customer c, BaskteItem bi where bi.customer == c group by c ?c <- (Customer ) acc( ?bi <- (BasketItem {customer == ?c} ) ?t <- (sum ?bi.price) )
37. 37. 37 Functional Support select c, bi.type type, sum(bi.price) t from Customer c, BaskteItem bi where bi.customer == c group by c , bi.type ?c <- (Customer) acc( ?bi <- (BasketItem {customer == ?c} ) ?type <- (distinct ?bi.price) ) acc( ?bi <- (BasketItem {customer == ?c, type == ?type} ) ?t <- (sum ?bi.price) )
38. 38. 38 DRL select c, sum(bi.price) t from Customer c, BaskteItem bi where bi.customer == c group by c \$c : Customer() acc( \$bi : BasketItem( customer == \$c ); \$t : sum( \$bi.price) )
39. 39. 39 Nested acc( acc( \$c : Customer() \$bi : BasketItem( customer == \$c ); \$t : sum( \$bi.price) ); \$a : avg( \$t ) )  Average the summed expenditure for all customers
40. 40. 40 Nested \$c : Customer() acc( \$bi : BasketItem() from acc( \$bi : BasketItem( customer == \$c ); filter( \$bi, \$bi.price > 100) ); \$a : avg( \$bi.price ) )  Average price for all item with a price over 100 Patterns can also filter “from” an acc result.
41. 41. All Together Now
43. 43. 43 Discount Alert rule "alert customer to potential band change" \$c : Customer() acc( \$bi : BasketItem(customer == \$c ) \$s : sum( \$bi.price ) ) acc( \$bi : BasketItem(customer == \$c ) ?recommended( \$bi.item, \$rec ) \$rs : sum( \$rec.price ) ) \$b : Band( this != \$c.band, min >= (\$s + \$rs), max <= (\$s + \$rs) ) then println( “If you buy all items and all recommendations, you move up a band with a new discount of “ + \$b.discount ); end '\$rec' is unbound and thusan output var
44. 44. Concurrency and Parallelization
45. 45. 45 C : Customer [John, Sally] B : BasketItem b.customer == c [John, Sally] Simple Rete Rule
46. 46. 46 Simple Rete Rule  insert “sally” ● propagate to join ● Return as right memory is empty  insert “john” ● propagate to join ● Return as right memory is empty
47. 47. 47 Simple Rete Rule C : Customer [John, Sally] [BasktItem( John, Book, Cars ),] [John, Sally] [ [John,BasktItem( John, Book, Cars )] ] B : BasketItem b.customer == c [BasktItem( John, Book, Cars )] Token Node node Token leftParent Token rightParent List<Token> children
48. 48. 48 Simple Rete Rule  Insert BasktItem( John, Book, Cars ) ● propagate to join ● foreach( l-token : left-memory ) if ( r-token.customer == l-token ) create new Token and propagate [John, BasktItem( John, Book, Cars )] Token Node node Token leftParent Token rightParent List<Token> children
49. 49. 49 Simple Rete Rule C : Customer [John, Sally] [BasktItem( John, Book, Cars ), BasktItem( John, Book, Poems ), BasktItem( Sally, Book, Cinema )] [John, Sally] [ [John,BasktItem( John, Book, Cars )], [John, BasktItem( John, Book, Poems ) ], [Sally, BasktItem( Sally, Book, Cinema ) ] ] B : BasketItem b.customer == c [BasktItem( John, Book, Cars ), BasktItem( John, Book, Poems ), BasktItem( Sally, Book, Cinema )]
50. 50. 50 1 Rule 3 Joins
51. 51. 51 3 Rules 3 Joins
52. 52. 52 Gator Networks
53. 53. 53 Partitioning for Parellelization
54. 54. 54 Task Oriented  Submit each Token propagation to Executor pool ● The join node is now an executable job  Wins ● Only the node is locked, allowing a “pipeline” ● Possible to lock only the current index bucket ● Solves recursive query problem of large stack frames  Loses ● Only a small amount of time in each node ● Each node repeatedly locked and scheduled, for a single WM action, due to tree “to leaf” traversal ● Previous papers show increases for only some use cases ● Network thrashing for subnetworks on accumulate
55. 55. 55 Subnetwork Thrashing \$c : Customer() acc( \$b : BasketItem(customer == \$c ) \$d : Discount( type == \$b.type); \$s : sum( \$b.price * \$d.multiplier ) ) Band( this != \$c.band, min >= \$s, max <= \$s )
56. 56. 56 Set Oriented  Propagate Collections instead of Tokens ● Left Input trigger ● Iterate right memory create a collection of all matching right tokens ● Note multi-dimesional array of 1 ● [[l1[r1, r2, r3]]] ● Right Input trigger ● Iterate left memory create a collection of all left joins ● Note multi-dimensional array for each left token ● [ [l1, r1], [l2, r1] [l3, r1] ] Token Node node Token leftParent Token rightParent List<Token> children
57. 57. 57 Set Oriented  Wins ● More work done in a node ● scalable task scheduling ● Less indirection so better statement execution ● All the work for that node done for that WM action in one task ● Single network pass, so nodes locked just once per WM action ● Avoids repeatedly creating and destroying method stack frames ● Avoids sub network thrashing  Loses ● More complex algorithm ● Especially on “collection” merging
58. 58. 58 MVCC  Modifies ● Clone + new values and current transaction id ● Existing instance still there with older transaction id ● Node uses constraint to join with correct version  Isolation ● Scoped to rule or rule-group ● Scoped rule(s) see new data, others old ● Non blocking so reads (joins) can still continue  Fully Concurrent safe Queries ● Queries leave no state in join ● Node joins with correct instance, based on transaction id
59. 59. 59 Questions?Questions? • Dave Bowman: All right, HAL; I'll go in through the emergency airlock. • HAL: Without your space helmet, Dave, you're going to find that rather difficult. • Dave Bowman: HAL, I won't argue with you anymore! Open the doors! • HAL: Dave, this conversation can serve no purpose anymore. Goodbye. Joshua: Greetings, Professor Falken. Stephen Falken: Hello, Joshua. Joshua: A strange game. The only winning move is not to play. How about a nice game of chess?