Combining Linear and Non Linear Modeling Techniques


Published on

Salford Systems demonstrates how to combine linear and non-linear predictive modeling.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Combining Linear and Non Linear Modeling Techniques

  1. 1. EMB America + Salford SystemsGetting the best of Two Worlds
  2. 2.  Who is EMB? Insurance industry predictive modeling applications EMBLEM- our GLM tool How we have used CART with EMBLEM Case studies Other areas of expected synergies
  3. 3.  Global network of p&c insurance consultants servicing clients throughout the world (insert globe)
  4. 4.  Predictive Modeling Ratemaking & Profitability Analysis Underwriting & Credit Scoring Enterprise Risk Management, Pro Forma, Business Planning Retention & Conversion Modeling New Program Development Competitive Analysis Reinsurance Program Analysis Reserve Analysis & Opinion Letters Software Development & Software Support Expert Witness Testimony Regulatory Support & Law Analysis
  5. 5.  EMB’s suite of software products cover all aspects of personal and commercial lines of insurance ◦ EMBLEM ◦ Rate Assessor ◦ Classifier ◦ Igloo Professional ◦ ExtrEMB ◦ ResQ Professional ◦ PrisEMB ◦ RePro
  6. 6.  We use EMBLEM, a GLM tool, for our predictive modeling needs Why?
  7. 7.  Primary application: ◦ Estimating the cost of the product they sell (insurance) two steps:  Reserving= estimating the cost of outstanding insurance claims  Pricing= estimating the cost of future insurance coverage Secondary applications ◦ Retention Modeling= probability that a policyholder will renew ◦ Conversion Modeling= probability that a prospective policyholder will purchase a policy ◦ Price Optimization ◦ Claim fraud detection ◦ Marketing
  8. 8.  Goal is to develop a unique rate for every risk ◦ Don’t think in terms of good/bad risks ◦ State Farm/Allstate vs GEICO/Progressive ◦ Quickly exhausts the data  Credibility/ variability/ stability Risks are described by the predictor variables, not the target. ◦ Need to have a mapping of the predictor variable levels to a target value- not the other way around  Other way around makes it difficult to derive impact of individual predictor variables  Important because actual data often does not describe all possible combinations of potential customers
  9. 9.  Highly regulated marketplace ◦ Restrictions  Predictors can and cannot use  Credit scores  Rules on values for the predictors  Ages 65+ relativities cannot be >110% of ages 40-60  Maximum rate change between adjacent territories  Rules on predictor order and magnitude of importance  CA Sequential Analysis (driving record>annual mileage>years held license) ◦ Regulatory Approval  Rates need to be supported  Black box methodologies will not be accepted
  10. 10.  Response variable is continuous/discrete function (insert graph) ◦ Gamma consistent with severity modeling, or even Inverse Gaussian (insert graph) ◦ Poisson consistent with frequency modeling No single trial/outcome ◦ Trial is measured in terms of time ◦ Actual policy length varies tremendously because of changes  Marital status  New car  moved
  11. 11.  In 1996, EMB designed EMBLEM to provide access to GLM for statisticians and non-statisticians pricing personal and commercial insurance EMBLEM revolutionized the use of GLM’s, enabling analysis that was previously either impossible or too time-consuming to be worth attempting EMBLEM is now used by over 100 insurance companies globally: ◦ 18 of the top 20 personal auto writers in the UK ◦ 50 companies in the US including 8 of the top 10 personal auto writers Fastest GLM tool with the capability to model millions of observations in seconds with a host of diagnostic tools: ◦ Graphical, practical, statistical, automated. ◦ Stand-alone software package that can be integrated with a variety of external software including SAS® ◦ Microsoft® Visual Basic® for Applications provides ultimate flexibility
  12. 12.  GLM characteristics work to our advantage ◦ Exponential family does an excellent job of describing the underlying components of insurance losses ◦ Output of the model is in the form of Beta parameters which can easily be converted to rate relativities ◦ EMBLEM is not automated  User has complete control over the model structure  Complete diagnostic tools to assist the modeler with decisions
  13. 13.  In terms of estimating the cost of insurance: ◦ UK has embraced predictive modeling  Experienced with its techniques  Knowledgeable with the factors that tend to be predictive ◦ US is learning about predictive modeling  Saturation with big players in personal lines marketplace  Companies not using predictive modeling techniques are being adversely selected against  Now expanding dimensionality of databases  Still fairly new concept in commercial lines marketplace  Big players are using techniques but historical rating structures are hindering the rapid expansion
  14. 14.  Result? ◦ UK is expanding into secondary applications  Retention modeling  Conversion modeling  Price optimization  Claim fraud detection ◦ Because Predictive Modeling has been around for some time in the UK, the datasets are getting larger in terms of the number of predictors to evaluate ◦ Experienced US companies are beginning to evaluate the secondary applications ◦ Marketing is used in a manner similar to other industries
  15. 15.  How does CART fit into this? ◦ As we transition into the secondary applications we move from modeling a continuous function to a binary function  Tree-based techniques can add value to the analysis Retention and Conversion modeling ◦ Accept/ Reject target variable ◦ Desirable smooth surface ◦ Price optimization integrates these with premium models Marketing and Fraud detection ◦ Classic tree applications
  16. 16.  Using CART and EMBLEM ◦ Goal is to play off of the strengths of each tool CART strengths ◦ Automatic separation of relevant from irrelevant predictors ◦ Easily rank-orders variable importance ◦ Automatic interaction detection (requires additional work) ◦ Captures multiple structures within a dataset rather than a single dominant structure ◦ Can handle missing values and is impervious to outliers
  17. 17.  EMBLEM Strengths ◦ User has control over the model structure ◦ Ease of communication/conceptualization- effects of each explanatory variable is transparent ◦ Provides predicted response values for new data points
  18. 18.  CART ◦ Factor selection ◦ Interaction detection ◦ Model validation EMBLEM ◦ Model structure ◦ Incorporating time/seasonality trend effects ◦ Implementation of results
  19. 19.  Both CART and EMBLEM are excellent tools both of which produce consistent results in similar situations ◦ This is not an exercise of seeing which is better The purpose of this discussion is to show how efficiencies can be gained in the modeling process ◦ As datasets get larger in terms of the number of predictors time becomes a crucial element
  20. 20.  Retention modeling assignment ◦ 97,227 observations  Each observation represents one trial/outcome  Split 50/50 between training/test datasets ◦ 11 predictors  Grand total number of levels:147
  21. 21.  Modeling Process ◦ Started with Forward Entry Regression  Automated process  Used Chi-Squared statistic for testing significance  Took about 30 minutes to run ◦ Significant factors (8)  Rating Area  Vehicle Category  Age  NCD  Driver Restriction  Vehicle Age  Change Over Last Year’s Premium  Market Competitiveness
  22. 22.  Build a model with no factors and add based on prespecified criteria regarding improvement in model fit: (insert table) Add the factor that performed the best on the Chi Square test. (Policyholder Age) Iterate process with the new base model until no further factors indicated removal
  23. 23.  Compared results with CART/ TreeNet ◦ Significant factors were essentially the same ◦ Model predictiveness was the same (ROC=0.7) Interactions ◦ No significant interactions were found by EMBLEM or CART Test Dataset ◦ ROC=0.7
  24. 24.  Retention modeling assignment ◦ 198,386 observations  Each observation represented one trial/outcome  Split 50/50 between training/test datasets ◦ 135 predictors  Grand total number of levels: approx 3,752
  25. 25.  Forward Entry Regression ◦ Found 57 predictors to be significant ◦ Took a weekend to run Comparison to CART/ TreeNet ◦ Found 24 significant predictors ◦ Top 15 based on variable importance were also found by EMBLEM ◦ Correlations with the rest of the predictors Through the modeling process we reduced the number of predictors to 26
  26. 26.  Interactions ◦ We relied on indications from CART/ TreeNet ◦ 6 interactions were identified and included in the model EMBLEM Results ◦ Training ROC= .862 ◦ Test ROC= .85
  27. 27.  Variable importance Segmentation Super-Profiling
  28. 28.  CART excels at identifying different segments in data CART may also help determine where to segment data Segmentation is a useful alternative to fitting many interactions ◦ Example: In a automobile insurance renewal problem, a CART analysis showed several occurrences of a split between those policyholders with just one years duration and those with a greater duration This suggests segmenting the data into two parts: ◦ Policies renewing with one year duration ◦ Policies renewing with more than one year
  29. 29.  After a GLM model is constructed use CART to model the residuals to see if any patterns exists ◦ If a pattern is discovered, go back to the model structure and incorporate the findings ◦ Test to see if model structure was inadvertently over-simplified