Suriadi caise2013 slides
Upcoming SlideShare
Loading in...5
×
 

Suriadi caise2013 slides

on

  • 258 views

 

Statistics

Views

Total Views
258
Views on SlideShare
215
Embed Views
43

Actions

Likes
0
Downloads
2
Comments
0

1 Embed 43

http://www.pros.upv.es 43

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Suriadi caise2013 slides Suriadi caise2013 slides Presentation Transcript

  • Introduction Approach Case Study Summary Future Work Understanding Process Behaviours in a Large Insurance Company in Australia: A Case Study1 S. Suriadi, M. Wynn, C. Ouyang, A.H.M. ter Hofstede, and N. van Dijk Information Systems School Queensland University of Technology Brisbane, Australia s.suriadi@qut.edu.au June 21, 2013 1 This work was supported by the Australian Research Council Discovery Project grant DP110100091. S.Suriadi et al. Process Mining Case Study at Suncorp 1/ 23
  • Introduction Approach Case Study Summary Future Work Outline 1 Introduction 2 Approach 3 Case Study 4 Summary 5 Future Work S.Suriadi et al. Process Mining Case Study at Suncorp 2/ 23
  • Introduction Approach Case Study Summary Future Work Motivation Introduction Case Study A 6-month case study with Suncorp, one of the largest insurance organizations in Australia Goal: to identify reasons for under-performing claims, leading to process improvement Approach: Process Mining and L∗-methodology Why Process Mining? Explosion of data for analysis Extract evidence-based insights from data (>100 orgs.) limited application in Australia S.Suriadi et al. Process Mining Case Study at Suncorp 3/ 23
  • Introduction Approach Case Study Summary Future Work Contributions Introduction Contributions A report on the experiences gained from this case study: challenges, lessons learned, and recommendations What’s new? Validation of existing lessons learned Detailed (new?) insights for every stage of the case study Review of process mining-related tools Novice/beginner’s point of view S.Suriadi et al. Process Mining Case Study at Suncorp 4/ 23
  • Introduction Approach Case Study Summary Future Work Methodology for ‘Spaghetti Process’ L∗ Methodology Initial interview and preliminary data analysis suggest: unstructured process Adopt the L∗ methodology by van der Aalst (2011) for “spaghetti” process - business understanding - data understanding - historic data - handmade models - objectives - questions - explore - discover - check - compare - promote - diagnose - verification - validation - accreditation - redesign - adjust - intervene - support - feedback to objectives S.Suriadi et al. Process Mining Case Study at Suncorp 5/ 23
  • Introduction Approach Case Study Summary Future Work Methodology for ‘Spaghetti Process’ L∗ Methodology Initial interview and preliminary data analysis suggest: unstructured process Adopt the L∗ methodology by van der Aalst (2011) for “spaghetti” process - business understanding - data understanding - historic data - handmade models - objectives - questions - explore - discover - check - compare - promote - diagnose - verification - validation - accreditation - redesign - adjust - intervene - support - feedback to objectives Stage 0: Plan/Justify S.Suriadi et al. Process Mining Case Study at Suncorp 5/ 23
  • Introduction Approach Case Study Summary Future Work Methodology for ‘Spaghetti Process’ L∗ Methodology Initial interview and preliminary data analysis suggest: unstructured process Adopt the L∗ methodology by van der Aalst (2011) for “spaghetti” process - business understanding - data understanding - historic data - handmade models - objectives - questions - explore - discover - check - compare - promote - diagnose - verification - validation - accreditation - redesign - adjust - intervene - support - feedback to objectives Stage 0: Plan/Justify Stage 1: Extract S.Suriadi et al. Process Mining Case Study at Suncorp 5/ 23
  • Introduction Approach Case Study Summary Future Work Methodology for ‘Spaghetti Process’ L∗ Methodology Initial interview and preliminary data analysis suggest: unstructured process Adopt the L∗ methodology by van der Aalst (2011) for “spaghetti” process - business understanding - data understanding - historic data - handmade models - objectives - questions - explore - discover - check - compare - promote - diagnose - verification - validation - accreditation - redesign - adjust - intervene - support - feedback to objectives Stage 0: Plan/Justify Stage 1: Extract Stage 2: Control Flow and Connect S.Suriadi et al. Process Mining Case Study at Suncorp 5/ 23
  • Introduction Approach Case Study Summary Future Work Stage 1 Stage 1: Planning Activities: Presentation and discussions Extract question(s) and assess data availability Mutually-beneficial engagement model Close engagement Main Question Why did the processing of certain ‘simple’ claim take such a long time to complete? Case study was conducted in two phases: First phase: “data-driven” (quite ‘explorative’) Second phase: “question-driven” S.Suriadi et al. Process Mining Case Study at Suncorp 6/ 23
  • Introduction Approach Case Study Summary Future Work Stage 1 Stage 1: Planning - Challenges and Lessons Learned Challenges Defining the concept of a ‘simple’ claim No corresponding business rules ‘commonly-held’ belief, differing opinions Took up to 5 interview sessions spanning up to 6 weeks in total Lessons Learned Composition of team is crucial for effective communication Use “question-driven” approach S.Suriadi et al. Process Mining Case Study at Suncorp 7/ 23
  • Introduction Approach Case Study Summary Future Work Stage 1 Stage 1: Planning - Results Results A clear definition of ‘simple’ claim A claim with less than x-amount of claim value and should be completed in no later than y-number of days Derivation of process mining questions: Q1: What is the performance distribution of ‘simple’ vs ‘non-simple’ claims? Q2: What do the corresponding process models look like? Q3: What are the key differences in the processing of “simple quick” vs. “simple slow” claims that lead to performance differences? S.Suriadi et al. Process Mining Case Study at Suncorp 8/ 23
  • Introduction Approach Case Study Summary Future Work Stage 2 Stage 2: Extract Two phases: First round: data quality issue, omission of important information, poorly-populated fields Second round: cleaner data with more accurate information All finalized claims (no incomplete cases) >32,000 claims over 1 million unique events Data cleaning, filtering, and conversion to XES/MXML One of the most challenging stages in the case study S.Suriadi et al. Process Mining Case Study at Suncorp 9/ 23
  • Introduction Approach Case Study Summary Future Work Stage 2 Stage 2: Extract - Challenges Challenges Data interpretation Complex process (over 20 top-level activities, > 100 second-level activities) ‘High process variant’ is the norm! flexibility and/or ‘sloppy’ logging practice? Inconsistent terminology (nat. hazard vs. storm vs. flood) Data Filtering Necessary to ensure scalability of analysis and interesting comparative analysis Would like to apply hierarchical filtering, but which criteria to use? Related to poor definition of ‘simple’ claims S.Suriadi et al. Process Mining Case Study at Suncorp 10/ 23
  • Introduction Approach Case Study Summary Future Work Stage 2 Stage 2: Extract - Challenges Challenges Data manipulation Many tools involved, each with its own strengths and weaknesses, and ‘quirkiness’ w.r.t data format (input and output) Need to use them all Highly tedious! Spreadsheet Disco Database (e.g. MySQL) Text/XML Editor ProM Tool CSV XES XESAME S.Suriadi et al. Process Mining Case Study at Suncorp 11/ 23
  • Introduction Approach Case Study Summary Future Work Stage 2 Stage 2: Extract - Lessons Learned Lessons Learned Naive interpretation of data –> meaningless findings, e.g. ‘complete’ timestamp interpretation? activity duration (from [scheduled, assign, start] to complete) - big differences, often. schedule (recorded) assign (recorded) actual completion time (not recorded) recorded as complete (recorded) start (not recorded) {actual duration activity duration estimate_1 activity duration estimate 2 S.Suriadi et al. Process Mining Case Study at Suncorp 12/ 23
  • Introduction Approach Case Study Summary Future Work Stage 2 Stage 2: Extract - Lessons Learned Lessons Learned Order of data filter matters! Case-level vs. event-level filtering, e.g. Rule A: remove all events not done by R Rule B: remove all cases longer than 7 days DISCO vs. XESAME Rule A event resource activityA (start) resourceR activityB resourceC activityC resourceC activityD resourceR activityF (end) resourceE {10 days {7 days The case does not satisfy Rule B - thus whole case is removed. Rule A Rule B Rule B 1st Filter 2nd Filter .......... Result caseID:123ABC event resource <event removed> activityB resourceC activityC resourceC <event removed> activityF resourceE caseID:123ABC event resource activityB resourceC activityC resourceC activityF resource E caseID:123ABC S.Suriadi et al. Process Mining Case Study at Suncorp 13/ 23
  • Introduction Approach Case Study Summary Future Work Stage 2 Stage 2: Extract - Result Result Clear identification of data slices to be used for analysis and comparison to address the 3 process mining questions defined earlier. CaseDuration(days) $x Payout amount Simple Slow (SS): <=$x payout > y days Simple Quick (SQ): <=$x payout <= y days Complex Slow (CS): >$x payout > y days Complex Quick (CQ): >$x payout <=y days "in-between" cases y+1 y y-1 S.Suriadi et al. Process Mining Case Study at Suncorp 14/ 23
  • Introduction Approach Case Study Summary Future Work Stage 3 Stage 3: Analysis - Challenges Control-flow analysis, using Disco and ProM Tool. Challenges Discovering meaningful process models Dealing with complex log ‘Inconsistency in behaviour’ is the norm! Practicality: time consuming process (especially if data had to be further filtered/cleaned) genetic miner, ILP miner (sometimes) S.Suriadi et al. Process Mining Case Study at Suncorp 15/ 23
  • Introduction Approach Case Study Summary Future Work Stage 3 Stage 3: Analysis - Lessons Learned Lessons Learned No ‘clean’ structure or we have not done proper data pre-processing? Clues for the non-existence of structured process: the existence of high process variants, low fitness value of Heuristic nets, and flower-like model in simplified Petri Nets, despite the application of hierarchical filtering and other cleaning activities. Useful algorithms: Heuristics Miner, ILP, Uma, as well as Fuzzy miner (in Disco). Not so useful: Genetic miner S.Suriadi et al. Process Mining Case Study at Suncorp 16/ 23
  • Introduction Approach Case Study Summary Future Work Stage 3 Stage 3: Analysis - Results Results Q1: a significant number of under-performing cases Q2: unstructured process indeed! S.Suriadi et al. Process Mining Case Study at Suncorp 17/ 23
  • Introduction Approach Case Study Summary Future Work Stage 4 Stage 4: Interpretation Proper identification of differences via classical data mining techniques Compare Simple Quick vs. Simple Slow claims Challenges Finding a good set of predictor variables (too many attributes) S.Suriadi et al. Process Mining Case Study at Suncorp 18/ 23
  • Introduction Approach Case Study Summary Future Work Stage 4 Stage 4: Interpretation - Lessons Learned and Results Lessons Learned Two useful process-related metrics: average execution of an activity-X per case distribution of an activity-X over all cases Results Q3 is addressed Key differences between ‘Simple Quick’ and ‘Simple Slow’ classes were identified. S.Suriadi et al. Process Mining Case Study at Suncorp 19/ 23
  • Introduction Approach Case Study Summary Future Work Stage 4 Stage 4: Interpretation - Result Activity Simple Quick Simple Slow actFreq actDist actFreq actDist Follow Up Requested 1.86 74.4% 5.79 92.3% Incoming Correspondence 1.75 81.6% 4.27 90.1% Contact Customer 0.66 46.8% 1.29 63.3% Contact Assessor 0.11 4.9% 1.36 21.5% Conduct File Review 2.03 89.8% 6.11 96.9% S.Suriadi et al. Process Mining Case Study at Suncorp 20/ 23
  • Introduction Approach Case Study Summary Future Work Conclusion Conclusion of the Case Study - Stage 5 Useful findings: Surprising number of under-performing claims Highlighted non-uniformity and the need for standardization in the claims processing Triggered some changes in their claims processing system “...by mining and analysing our claims ... our business has been able to make cost saving adjustments to the existing process.” a new system was trialled Another subsequent impact (not involved, but informed verbally) improved the proportion of claims finalized on-time S.Suriadi et al. Process Mining Case Study at Suncorp 21/ 23
  • Introduction Approach Case Study Summary Future Work Future Work Future Work Our case study: too afraid to manipulate data too much (limited data removal, conservative filtering) But, fine-line between abstraction and over-simplification When do we stop simplifying/cleaning data? Data quality - event log quality Claim: objective insights Pitfall: low data quality (incomplete/incorrect/noisy) and/or improper cleaning, abstraction and manipulation Verification of results is needed! Establish links between original data and results Assert the correctness of results Improved accountability in making important decisions S.Suriadi et al. Process Mining Case Study at Suncorp 22/ 23
  • Introduction Approach Case Study Summary Future Work Future Work Understanding Process Behaviours in a Large Insurance Company in Australia: A Case Study2 S. Suriadi, M. Wynn, C. Ouyang, A.H.M. ter Hofstede, and N. van Dijk Information Systems School Queensland University of Technology Brisbane, Australia s.suriadi@qut.edu.au June 21, 2013 2 This work was supported by the Australian Research Council Discovery Project grant DP110100091. S.Suriadi et al. Process Mining Case Study at Suncorp 23/ 23