© 2016 IBM Corporation
ODM Rules Compiler
IBM Streams Version 4.2
Chris Recoskie, Streams Developer
recoskie@ca.ibm.com
Ankit Pasricha, Streams Developer
ankitp@ca.ibm.com
© 2016 IBM Corporation2
Important Disclaimer
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL
PURPOSES ONLY.
WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE
INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY
OF ANY KIND, EXPRESS OR IMPLIED.
IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY,
WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.
IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR
OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:
• CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS OR
THEIR SUPPLIERS AND/OR LICENSORS); OR
• ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENT
GOVERNING THE USE OF IBM SOFTWARE.
IBM’s statements regarding its plans, directions, and intent are subject to change or
withdrawal without notice at IBM’s sole discretion. Information regarding potential
future products is intended to outline our general product direction and it should not
be relied on in making a purchasing decision. The information mentioned regarding
potential future products is not a commitment, promise, or legal obligation to deliver
any material, code or functionality. Information about potential future products may
not be incorporated into any contract. The development, release, and timing of any
future features or functionality described for our products remains at our sole
discretion.
THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.
IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
© 2016 IBM Corporation3
Background
 In Streams 4.1: com.ibm.streams.rules toolkit
 ODMRulesetExecutor Java operator
 User workflow
 Develop rules in the Rule Designer IDE
 Export rules to the filesystem or to a DB
 Develop your SPL application and configure an instance of
ODMRulesetExecutor
 Run your SPL application
© 2016 IBM Corporation4
Background
Input Tuple Input Tuple
Ruleset
Execution
Server
ODM Rule Input
Parameters
ODM Rule Output
Parameters
Output Tuple
© 2016 IBM Corporation5
Background
 Issues
 Performance issues
 Sending data to/from ODM rule execution server is inefficient
 Dependency on ODM
 ODM must be installed and ODM_HOME must be set
 Rule Designer IDE for developing rule projects and rules
© 2016 IBM Corporation6
ODM Rules Compiler
Rules
Compiler
Rules Project
Generated SPL Composite
(one operator per package)
© 2016 IBM Corporation7
ODM Rules Compiler
Streams
Compiler
SPL
Application
Application
Bundle
© 2016 IBM Corporation8
Demo
© 2016 IBM Corporation9
Dynamic Rule updates
 Objective: Allow users to update rules used in SPL application WITHOUT
restarting their application, but still retain the performance gain of SPL
compiled rules
 With dynamic updates turned on, the generated composite uses the
ODMCompiledRuleset operator, which compiles the rules logic and places the object
code in a shared object that can be dynamically hot swapped at runtime
 Current ODMRulesetExecutor supports this through the ODM management console
Streams
Compiler
SPL
Application
Application
Bundle
Rules logic
shared
object
Rules logic
shared
object
© 2016 IBM Corporation10
Dynamic Rule updates
 ODMCompiledRuleset operator contains 1 control port
 2 ways to pass new shared object
 Path of the new shared object
 Blob representing the shared object
© 2016 IBM Corporation11
Limitations
 Stateless rule support only
 Custom Java functions have to be converted manually to SPL
 But, the compiler can generate stubs for you to fill out
 Dynamic updates
 Cannot change ruleset parameters
 Cannot add/remove rule packages
 But, can add/remove rules within a package
© 2016 IBM Corporation12
Performance
 Performance gains depend upon the nature of your rules.
 Rules with a lot of common code can be optimized more than those that don’t
 Contrived “best case” scenarios have ranged from the 39x to 600x speedup
range
 A more realistic set of telecom fraud detection rules yielded approximately a
6x speedup
© 2016 IBM Corporation13
Questions?

IBM ODM Rules Compiler support in IBM Streams V4.2.

  • 1.
    © 2016 IBMCorporation ODM Rules Compiler IBM Streams Version 4.2 Chris Recoskie, Streams Developer recoskie@ca.ibm.com Ankit Pasricha, Streams Developer ankitp@ca.ibm.com
  • 2.
    © 2016 IBMCorporation2 Important Disclaimer THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY. WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION. NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF: • CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS); OR • ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENT GOVERNING THE USE OF IBM SOFTWARE. IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion. THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE. IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
  • 3.
    © 2016 IBMCorporation3 Background  In Streams 4.1: com.ibm.streams.rules toolkit  ODMRulesetExecutor Java operator  User workflow  Develop rules in the Rule Designer IDE  Export rules to the filesystem or to a DB  Develop your SPL application and configure an instance of ODMRulesetExecutor  Run your SPL application
  • 4.
    © 2016 IBMCorporation4 Background Input Tuple Input Tuple Ruleset Execution Server ODM Rule Input Parameters ODM Rule Output Parameters Output Tuple
  • 5.
    © 2016 IBMCorporation5 Background  Issues  Performance issues  Sending data to/from ODM rule execution server is inefficient  Dependency on ODM  ODM must be installed and ODM_HOME must be set  Rule Designer IDE for developing rule projects and rules
  • 6.
    © 2016 IBMCorporation6 ODM Rules Compiler Rules Compiler Rules Project Generated SPL Composite (one operator per package)
  • 7.
    © 2016 IBMCorporation7 ODM Rules Compiler Streams Compiler SPL Application Application Bundle
  • 8.
    © 2016 IBMCorporation8 Demo
  • 9.
    © 2016 IBMCorporation9 Dynamic Rule updates  Objective: Allow users to update rules used in SPL application WITHOUT restarting their application, but still retain the performance gain of SPL compiled rules  With dynamic updates turned on, the generated composite uses the ODMCompiledRuleset operator, which compiles the rules logic and places the object code in a shared object that can be dynamically hot swapped at runtime  Current ODMRulesetExecutor supports this through the ODM management console Streams Compiler SPL Application Application Bundle Rules logic shared object Rules logic shared object
  • 10.
    © 2016 IBMCorporation10 Dynamic Rule updates  ODMCompiledRuleset operator contains 1 control port  2 ways to pass new shared object  Path of the new shared object  Blob representing the shared object
  • 11.
    © 2016 IBMCorporation11 Limitations  Stateless rule support only  Custom Java functions have to be converted manually to SPL  But, the compiler can generate stubs for you to fill out  Dynamic updates  Cannot change ruleset parameters  Cannot add/remove rule packages  But, can add/remove rules within a package
  • 12.
    © 2016 IBMCorporation12 Performance  Performance gains depend upon the nature of your rules.  Rules with a lot of common code can be optimized more than those that don’t  Contrived “best case” scenarios have ranged from the 39x to 600x speedup range  A more realistic set of telecom fraud detection rules yielded approximately a 6x speedup
  • 13.
    © 2016 IBMCorporation13 Questions?