Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Where did I go wrong? Explaining errors in process models

904 views

Published on

This presentation shows how to reduce diagnostic information returned by general purpose model checkers (counter example paths) to essential parts that help understanding the error. The presentation has been given at the 12th International Conference on Business Process Management (BPM'14), September 2014 in Eindhoven.

Published in: Data & Analytics
  • Be the first to comment

Where did I go wrong? Explaining errors in process models

  1. 1. Where did I go wrong? Explaining errors in process models Niels Lohmann @nlohmann Dirk Fahland @dfahland
  2. 2. Verification of processes and services 2 process model property verification technique diagnostic information
  3. 3. Verification of processes and services 3 verification technique diagnostic information BPMN Soundness domain-specific high-quality
  4. 4. Verification of processes and services 4 CMMN Declare WS-BPEL WS-Policy EPC BPMN YAWL Object Life-Cycles GSM Rules Soundness Security Compliance Conformance to … diagnostic information verification technique domain-specific high-quality -moving target -domain-specific approaches too specific to follow
  5. 5. Verification of processes and services 5 CMMN Declare WS-BPEL WS-Policy EPC BPMN YAWL Object Life-Cycles GSM Rules Soundness Security Compliance Conformance to … diagnostic information verification technique high-quality general purpose
  6. 6. Model checking general purpose verification approach: 1. formalize model and specification* 2. push a button 6 *can be hidden from the user
  7. 7. Effectiveness and efficiency -model checking works in reality -successful applications in many domains -very fast: “verify while you model” 7
  8. 8. Diagnosis -in case of error: outputs target state and produce a witness path -describes how target state can be reached -operational semantics: can be simulated 8 witness path target state
  9. 9. Diagnosis: the bad PATH process.s00000823##s00006200.inputCriterion.s00001053 fork.s00001071.activate.s00001072 fork.s00001071.fire.s00001078 merge.s00001061.activate.s00001065 merge.s00001061.fire.s00001069 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 merge.s00001160.activate.s00001064 merge.s00001160.fire.s00001069 callToTask.s00006203.inputCriterion.s00001053 callToTask.s00006203.outputCriterion.s00001055 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001075 merge.s00001161.activate.s00001064 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 fork.s00001071.fire.s00001073 merge.s00001061.activate.s00001064 join.s00001163.activate.s00001062 merge.s00001061.fire.s00001069 join.s00001163.activate.s00001064 merge.s00001162.activate.s00001062 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 fork.s00001071.fire.s00001075 merge.s00001160.activate.s00001065 callToTask.s00006207.inputCriterion.s00001053 merge.s00001160.fire.s00001069 callToTask.s00006207.outputCriterion.s00001055 callToTask.s00006203.inputCriterion.s00001053 decision.s00001126.activate.s00001072 callToTask.s00006203.outputCriterion.s00001055 decision.s00001126.fire.s00001073 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001073 callToTask.s00006204.inputCriterion.s00001053 callToTask.s00006204.outputCriterion.s00001055 callToTask.s00003714.inputCriterion.s00001053 callToTask.s00003714.outputCriterion.s00001055 callToTask.s00006215.inputCriterion.s00001053 callToTask.s00006215.outputCriterion.s00001055 callToTask.s00006206.inputCriterion.s00001053 join.s00001163.activate.s00001064 callToTask.s00006206.outputCriterion.s00001055 callToTask.s00006205.inputCriterion.s00001053 callToTask.s00006205.outputCriterion.s00001055 merge.s00001161.activate.s00001062 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 join.s00001163.activate.s00001062 callToTask.s00006212.inputCriterion.s00001053 callToTask.s00006212.outputCriterion.s00001055 merge.s00001162.activate.s00001064 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 callToTask.s00006207.inputCriterion.s00001053 callToTask.s00006207.outputCriterion.s00001055 decision.s00001126.activate.s00001072 decision.s00001126.fire.s00001073 STATE decision.s00001126.output.s00001054 : 2 -paths can become very long -length correlates with size of the model -reports all events equally: disregarding importance
  10. 10. This talk: better diagnosis PATH process.s00000823##s00006200.inputCriterion.s00001053 fork.s00001071.activate.s00001072 fork.s00001071.fire.s00001078 merge.s00001061.activate.s00001065 merge.s00001061.fire.s00001069 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 merge.s00001160.activate.s00001064 merge.s00001160.fire.s00001069 callToTask.s00006203.inputCriterion.s00001053 callToTask.s00006203.outputCriterion.s00001055 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001075 merge.s00001161.activate.s00001064 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 fork.s00001071.fire.s00001073 merge.s00001061.activate.s00001064 join.s00001163.activate.s00001062 merge.s00001061.fire.s00001069 join.s00001163.activate.s00001064 merge.s00001162.activate.s00001062 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 fork.s00001071.fire.s00001075 merge.s00001160.activate.s00001065 callToTask.s00006207.inputCriterion.s00001053 merge.s00001160.fire.s00001069 callToTask.s00006207.outputCriterion.s00001055 callToTask.s00006203.inputCriterion.s00001053 decision.s00001126.activate.s00001072 callToTask.s00006203.outputCriterion.s00001055 decision.s00001126.fire.s00001073 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001073 callToTask.s00006204.inputCriterion.s00001053 callToTask.s00006204.outputCriterion.s00001055 callToTask.s00003714.inputCriterion.s00001053 callToTask.s00003714.outputCriterion.s00001055 callToTask.s00006215.inputCriterion.s00001053 callToTask.s00006215.outputCriterion.s00001055 callToTask.s00006206.inputCriterion.s00001053 join.s00001163.activate.s00001064 callToTask.s00006206.outputCriterion.s00001055 callToTask.s00006205.inputCriterion.s00001053 callToTask.s00006205.outputCriterion.s00001055 merge.s00001161.activate.s00001062 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 join.s00001163.activate.s00001062 callToTask.s00006212.inputCriterion.s00001053 callToTask.s00006212.outputCriterion.s00001055 merge.s00001162.activate.s00001064 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 callToTask.s00006207.inputCriterion.s00001053 callToTask.s00006207.outputCriterion.s00001055 decision.s00001126.activate.s00001072 decision.s00001126.fire.s00001073 STATE decision.s00001126.output.s00001054 : 2
  11. 11. This talk: better diagnosis path essential path distill Why useless?
  12. 12. Reasons for useless paths 12 detours depth-first search indisputable parts bootstrapping interleavings concurrency
  13. 13. Running example 13 lack of synchronization
  14. 14. Reduction: obvious parts -classify transitions -only report points of alternative continuations* 14 * XOR-gateways, events, exceptions, … assume progress of flow
  15. 15. Reduction: obvious parts 15 t1 t2 t9 t10 t11 t12 t14 t8 t2 t3 t4 t5 “down” “down” “up”
  16. 16. Non-obvious “core” of a path ≈ 10-25% 16
  17. 17. Reduction: spurious decisions -can be found by model checking -results: 50%-80% spurious, occasionally no reduction (timeout) 17 p1 p3 p2 p4 p5 p6 p1 p3 p5 p6 genuine decision spurious decision = irrelevant for outcome
  18. 18. Reasons for useless paths 18 detours depth-first search indisputable parts bootstrapping interleavings concurrency
  19. 19. Reduction: unorder steps -idea: show independence of steps ( partially ordered runs) -makes synchronization points (milestones) explicit 19 independent steps many paths to same goal state order of steps irrelevant
  20. 20. Reduction: unorder steps 20 t1 t2 t9 t10 t11 t12 t14 t8 t2 t3 t4 t5 t9 t14 t3
  21. 21. More aid: preserve reference points 21 t1 t2 t9 t10 t11 t12 t14 t8 t2 t3 t4 t5 t9 t14 t3 p1 p6 p6 t5 t11 t10 t1
  22. 22. Final: remove obvious/spurious parts 22 t1 t2 t9 t10 t11 t12 t14 t8 t2 t3 t4 t5 t9 t14 t3 p1 p6 p6 t5 t11 t10 t1
  23. 23. Essential path: find source of error PATH process.s00000823##s00006200.inputCriterion.s00001053 fork.s00001071.activate.s00001072 fork.s00001071.fire.s00001078 merge.s00001061.activate.s00001065 merge.s00001061.fire.s00001069 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 merge.s00001160.activate.s00001064 merge.s00001160.fire.s00001069 callToTask.s00006203.inputCriterion.s00001053 callToTask.s00006203.outputCriterion.s00001055 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001075 merge.s00001161.activate.s00001064 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 fork.s00001071.fire.s00001073 merge.s00001061.activate.s00001064 join.s00001163.activate.s00001062 merge.s00001061.fire.s00001069 join.s00001163.activate.s00001064 merge.s00001162.activate.s00001062 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 fork.s00001071.fire.s00001075 merge.s00001160.activate.s00001065 callToTask.s00006207.inputCriterion.s00001053 merge.s00001160.fire.s00001069 callToTask.s00006207.outputCriterion.s00001055 callToTask.s00006203.inputCriterion.s00001053 decision.s00001126.activate.s00001072 callToTask.s00006203.outputCriterion.s00001055 decision.s00001126.fire.s00001073 callToTask.s00006214.inputCriterion.s00001053 callToTask.s00006202.inputCriterion.s00001053 callToTask.s00006202.outputCriterion.s00001055 callToTask.s00006211.inputCriterion.s00001053 callToTask.s00006211.outputCriterion.s00001055 callToTask.s00006209.inputCriterion.s00001053 callToTask.s00006209.outputCriterion.s00001055 decision.s00001158.activate.s00001072 decision.s00001158.fire.s00001075 callToTask.s00006214.outputCriterion.s00001055 callToTask.s00006213.inputCriterion.s00001053 callToTask.s00006213.outputCriterion.s00001055 decision.s00001840.activate.s00001072 decision.s00001840.fire.s00001075 callToTask.s00006201.inputCriterion.s00001053 callToTask.s00006201.outputCriterion.s00001055 decision.s00001123.activate.s00001072 decision.s00001123.fire.s00001073 callToTask.s00006204.inputCriterion.s00001053 callToTask.s00006204.outputCriterion.s00001055 callToTask.s00003714.inputCriterion.s00001053 callToTask.s00003714.outputCriterion.s00001055 callToTask.s00006215.inputCriterion.s00001053 callToTask.s00006215.outputCriterion.s00001055 callToTask.s00006206.inputCriterion.s00001053 join.s00001163.activate.s00001064 callToTask.s00006206.outputCriterion.s00001055 callToTask.s00006205.inputCriterion.s00001053 callToTask.s00006205.outputCriterion.s00001055 merge.s00001161.activate.s00001062 merge.s00001161.fire.s00001069 callToTask.s00006208.inputCriterion.s00001053 callToTask.s00006208.outputCriterion.s00001055 decision.s00001157.activate.s00001072 decision.s00001157.fire.s00001073 join.s00001163.activate.s00001062 callToTask.s00006212.inputCriterion.s00001053 callToTask.s00006212.outputCriterion.s00001055 merge.s00001162.activate.s00001064 merge.s00001162.fire.s00001069 callToTask.s00006210.inputCriterion.s00001053 callToTask.s00006210.outputCriterion.s00001055 decision.s00001159.activate.s00001072 decision.s00001159.fire.s00001073 join.s00001163.activate.s00001065 join.s00001163.fire.s00001069 callToTask.s00006207.inputCriterion.s00001053 callToTask.s00006207.outputCriterion.s00001055 decision.s00001126.activate.s00001072 decision.s00001126.fire.s00001073
  24. 24. Results: typical reduced paths 24 2x lack of synchronization improper completion deadlock
  25. 25. Summary -general purpose verification more user friendly -paths  partial order of important decisions -applicable to any verification goal -keep reference points to aid diagnosis Next steps -error localization vs. explanation -detect useless cycles -How should a good diagnosis for $problem look like? 25
  26. 26. Where did I go wrong? Explaining errors in process models Niels Lohmann

×