Downloaded 13 times
Uploaded byPaul Vincent
Testing begins with requirements - Presentation to BCS SIGiST jun15
The document discusses best practices in testing business logic during the requirements phase, highlighting methodologies like agile development, test-driven development, and decision modeling. It emphasizes the importance of formal models, such as BPMN and decision tables, for verifying and validating business logic to ensure completeness and correctness. Additionally, it presents various decision modeling methodologies and tools, illustrating their application through case studies.
More Related Content
Similar to Testing begins with requirements - Presentation to BCS SIGiST jun15
Testing begins with requirements - Presentation to BCS SIGiST jun15
- 1. Testing Business Logic beginsin Requirements Paul Vincent BCS SIGiST – Summer 2015 Conference - Friday 5 June 2015
- 2. Some Known BestPractices … • Agile development (fail early, deliver often) • Test-driven development (test specification = system specification) • Model-driven development (formal models are more easily transformed into software)
- 3. Ref: BCS SIGiSTSpring 2015
- 4. Ref: BCS SIGiSTWinter 2014
- 5.
- 6. 1. Why Testin Requirements? Ref: http://loadstorm.com/2011/10/cost-fixing-software-defects/
- 7. 2. But howcan Requirements be Formal? • Models can be formal • Example = BPMN process • Can be validated, simulated, loops detected etc Decision Gateway Task Ref OMG DMN
- 8. 3. Decision Modelsfor Requirements • Decision “models” are not new! Babylon 1800BC - Mathematical look-up table
- 9. 3. Decision Modelsfor Requirements • Decision “models” are not new!
- 10. 4. Decision Modelsfor Testing 1977
- 11. 5. Decision Modelsas Requirements • How can we define a formal model for business logic that Business Analysts can use? With appropriate notation, methodology and standards? • Can modelling tools embed appropriate testing to automate tests?
- 12. Decision Methodologies • TDM- The Decision Model (von Halle, Goldberg) • Published, commercially supported, specific notation • Tooling (BiZZDesign, Sapiens, SAP,…) • DTM - Decision Table Methodology (Vanthienen) • Published across papers and books, commercially supported • Tooling (Various) for decision tables • Decision Management Systems Methodology (Taylor) • Published, commercially supported • Own tooling (Decision Management Solutions)
- 13. Decision Notations • TDM- The Decision Model (von Halle, Goldberg) • Published, commercially supported, specific notation • Tooling (BiZZDesign, Sapiens, SAP,…) • OMG DMN - Decision Model and Notation • Published, commercial support starting, specific notation • Tooling (Signavio, DMS, FICO, …)
- 14. Case Study: Exampleusing TDM Determine Policy Renewal Method Manual Renewal Automatic Renewal Manual or Automatic
- 15. A. What isDecision? • Find Manual or Automated status… Determine Policy Renewal Method BPMN TDM
- 16. Per OMG DMN •ADecision is the act of applying decision logic to one or more inputs to produce one output •A Decision Logic is a formal expression that defines the output of a decision with respect to its inputs • A decision logic can be a Computation • Or a Fixed Alternatives Ruleset, represented as a Decision Table or a Decision Tree • Or it can be an Imported Formulation (e.g. a predictive model in PMML)
- 17. B. What isthe Decision Logic
- 18. Decision Table /Rule Family describing logic AND OR FAMILY OF RELATED BUSINESS RULES CONCLUSION
- 19. C. Is thiscomplete, no duplications etc? ALL VALUES TESTED? ALL VALUE COMBINATIONS TESTED? ALL CONCLUSIONS CONSISTENT?
- 20. Verify Rule Family/ Decision Table (static test)
- 21. D. Does thisdo what is intended? ARE THE RESULTS AS EXPECTED AGAINST TEST CASES?
- 22. Validate Rule Family/ Decision Table (test cases)
- 23. Validate Rule Family/ Decision Table (test cases)
- 24. Validate Rule Family/ Decision Table (test cases)
- 25. Validate Rule Family/ Decision Table (test cases)
- 26. Validate Rule Family/ Decision Table (test cases) TEST CASES CREATED, CUSTOMISED, ETC
- 27. Validate Rule Family/ Decision Table (test cases) TEST RESULTS SHOWN
- 28. Validate Rule Family/ Decision Table (test cases) TEST CASE SHOWN IN DECISION TABLE / RULE FAMILY
- 29. E. Unit TestingDone REPEAT EXHAUSTIVE TESTS FOR THESE RULE FAMILIES
- 30. F. Test Connectionsto Rule Families DEFINE TESTS FOR ALL BRANCHES, TESTING EACH LINK
- 31. Assumptions for decisionmodelling for testing 1. I know the decision I need to make (may be re-used across processes) 2. I define a formal / rigorous data model too (name, type, value domain, single/list) • See Data Modelling for Requirements Testing presentation at some future SIGiST!
- 32. For Business Logicrequirements… 1. We can model business logic using Decision Models
- 33. For Business Logicrequirements… 1. We can model business logic using Decision Models 2. We can verify the completeness + uniqueness of this business logic
- 34. For Business Logicrequirements… 1. We can model business logic using Decision Models 2. We can verify the completeness + uniqueness of this business logic 3. We can validate the business logic against all possible use cases for each rule family + all links in a decision model of multiple rule families
- 35. How useful isthis? • Example 1: a product selection requirement • Original analysis = 15,000 rows of business logic in Excel • Decision Modelled = ~220 rows across ~18 decisions by 6 Business Analysts over 2-3 weeks
- 36. How useful isthis? • Example 2: business loan compliance with policy • Original analysis = spreadsheet of X tabs of metrics vs business sector vs threshold levels • Decision Modelled = Validate-able thresholds vs policies, Side by side comparison of thresholds, Champion challenger etc
- 37. Standards to supportthis pattern? • Standardisation effort by OMG DMN • Notation and metamodel • Can be used with multiple methodologies eg DTM, TDM • Tooling to support DMN underway
- 38. DMN Decision RequirementsDiagram DECISION BUSINESS KNOWLEDGE MODEL INPUT
- 39.
- 40.
- 41.
- 42. Decision Table (aBusiness Knowledge Model)
- 43. Decision Table HitPolicies Indicator Indicator Term Description Single Hit U Unique Only 1 rule matched = DEFAULT POLICY A Any Many rules can match but matched rules must have same output P Priority Many rules can match and matched rules can have different outputs; output selected per a prioritised list of possible outputs F First Many rules can match and matched rules can have different outputs; output selected per ordering of rules Multi Hit O Output Matched rule outputs are returned in order per a prioritised list of possible outputs R Rule Matched rule outputs are returned in order per ordering of rules C Collect No operator Matched rule outputs are returned in arbitrary order C+ Collect operator (+, <, >, #) Operator (sum, min, max, count) applied to the outputs and the result returned C> C< C#
- 44.
- 45.
- 46.
- 48. THANK YOU • Credits: •OMG DMN standards team standardising notations for best practice decision analysis and design • Larry Goldberg and Barbara Vonhalle defining The Decision Model as a best practice methodology • Sapiens DECISION product team permitting the use of their tool for this presentation