This document discusses using machine learning to predict the cost of workers' compensation claims early in the claim process. It notes that data preparation and feature engineering are key to building an accurate predictive model. The modeling process faces challenges like incorporating different data sources, identifying predictive features, measuring performance across different metrics, and optimizing for specific business goals. After building a model, it is important to ensure users understand the outputs and can take appropriate actions. Ongoing measurement, reporting, and improvement are also needed to maximize the model's impact.

1. WAYPOINT – FINANCIAL GUIDANCE ON WORKERS COMP CLAIMS

2. What are we trying to do?
• Use machine learning to predict the cost of a Workers Comp Claim early on in
the life of the claim.
Why?
What makes it challenging?
2
ARE WE SOLVING A BUSINESS PROBLEM?
Ability to
predict the cost
accurately
Better
understanding of
the claim
Design an
optimum care
program
Help the injured worker
get well and get back
to work early

3. 64% 63% 66% 63%
74%
61%
36% 37% 34% 37%
26%
39%
0%
20%
40%
60%
80%
2010 2011 2012 2013 2014 2015
WC - Spend by Accident Year
Average of AvgMedicalIncurred Average of AvgIndemnityIncurred
7%
6%
8% 7% 7%
4%
12%
14% 13%
9%
16%
12%
0%
5%
10%
15%
20%
2010 2011 2012 2013 2014 2015
Medical Spend on Pharmacy and Diagnostics
Rx Paid Diagnosis Paid
• The workers' compensation system is a form of social insurance that provides
injured workers with Medical Care, Lost Wages (% of Income), and survivor
benefits in cases of fatalities.
• 133 Million* US workers are covered by Workers Comp Insurance
• $80+ Billion annual direct written premium
• 100%**+ average loss ratio
• 60% of the cost is driven by medical spend that is expected to go up to 70%+ by
2018**
3
I HEARD WORKERS COMP – TELL ME MORE…
*Source https://www.nasi.org
** Insurance Information Institute - http://www.iii.org and https://www.ncci.com

4. 4
OK – FIRST LET US INTRODUCE OURSELVES
ADVISOR
PRODUCT
LEADER
PRODUCT
OWNER
GALLAGHER BASSETT
GRADIENT AI
(Milliman)
SANDIP
CHATTERJEE
Dr. GARY
ANDERBERG
Dr. ABHI
BUTCHIBABU

5. JUST DO IT…..OR IS THERE A METHOD TO THE MADNESS?
Problems &
Outcomes
Data & Prep
Models
Forecasts
Actions
Workflow-
Integration
Process-
Updates
Training
Communication
Pilot
Operationalize
Track & Improve
1
2
3
4
5
Feedback
5

6. LET US TAKE A CLOSER LOOK AT STEPS 2 AND 3
Un-structured
Data
Featurize
Normalize
Filtering
Imputation
Featurize
Data Fusion
Training/
Testing
Analysis/
Insights
Use /
Implement
Input pre-processing Training, Evaluation
& Optimization
Post Analysis
Structured
Dimension
Reduction
6

7. • Used 10 years worth of claims history (multiple data channels) to get started.
DATA – THAT WAS EASY – JUST A BREEZE…
ACCIDENT &
INJURY
CLAIMANT
MEDICAL
DIAGNOSIS &
TREATMENT
NOTESFINANCIALS
Used multiple methodologies to clean, prep
data for training our model
DATA SET for
training
models
FEATURE ENGINEERING
• DATA prep is key to the success of the project – Do not underestimate the effort
required.
• Feature engineering provides valuable insights that shouldn’t be ignored
7

8. THE ACTUAL PROCESS OF MODELING IS EQUALLY CHALLENGING
Split data for
each slice in
time
Feature
Engineering
Train/Test
Performance
Validation
Model
Optimization
Output
Challenge 1: Using the entire history of the claim up to the scoring date
• We need to accommodate for order and sequence of data
Challenge 2: Combining different data channels intelligently
• Structured vs. unstructured data
• High cardinality in categorical data (e.g., Diagnosis codes – 68,000 codes)
8

9. THE ACTUAL PROCESS OF MODELING IS EQUALLY CHALLENGING
Split data for
each slice in
time
Feature
Engineering
Train/Test
Performance
Validation
Model
Optimization
Output
Challenge 3: Identifying features that might be snooping
• Refine feature engineering by carefully looking at changes in performance.
• May not have perfect time-stamping of all information, especially on older
claims.
Challenge 4: Measuring performance is difficult. It can be defined in many ways.
• Precision vs. recall/relative error/mean square errors
• Some metrics may look better than other but might not fit the business needs.
9

10. THE ACTUAL PROCESS OF MODELING IS EQUALLY CHALLENGING
Split data for
each slice in
time
Feature
Engineering
Train/Test
Performance
Validation
Model
Optimization
Output
Challenge 5: Modeled variable (e.g., ultimate claim cost) can vary by several
orders of magnitude
• Modify model topology – multi-stage model/stacked models.
Challenge 6: Optimize for what we care about
• Errors on small claims vs. big claims.
• Moving the accuracy towards expensive claims that happen less frequently vs.
cheaper claims that happen more often.
10

11. THE ACTUAL PROCESS OF MODELING IS EQUALLY CHALLENGING
Split data for
each slice in
time
Feature
Engineering
Train/Test
Performance
Validation
Model
Optimization
Output
Challenge 7: Stability of the predictions we provide back to the resolution
managers over time is important
• Tradeoff between accuracy and volatility
Challenge 8: Interpretability
• We need to provide actionable explanation for the reserve guidance
• E.g., total paid to date is likely one of the most predictive variable, but it does
not provide resolution managers the actionable explanatory information
• Need to be creative
11

12. SO WHAT’S LEFT – AFTER WE BUILD THE MODEL – JUST DUMP AND RUN??
• Take time to know your users as they are going to use the model
outputs not you…
• Resolution Managers couldn’t care less about the model. They are
mostly interested in knowing how is it going to help them do their job…
• So make the model outputs
– Easy to Understand
– Provide information that support the findings/outputs
– Define clear interventions – Define actions they need to perform
• EXTRA CAUTION if you are replacing existing processes, make sure you
are not overwhelming your user base with changes (however cool you
think they are)
• Pay attention to the Machine and Human interface
• Orchestrate the process if you want to be successful 12

13. METRICS – REALLY WHO CARES?
Adequacy
Accuracy & Speed
to Accuracy
Consistency
Compliance
UTILIZATION IMPACT
• METRICS ARE KEYS TO SUCCESS – MEASURE, REPORT,
IMPROVE, REPEAT 13

14. THANK YOU