Advanced cart 2007

Data Mining with Decision Trees: An Introduction to CART ® Dan Steinberg Mikhail Golovnya [email_address] http://www.salford-systems.com

In The Beginning… ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

The Years of Struggle ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

The Final Triumph ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

So What is CART? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Typical CART Solution ,[object Object],[object Object],[object Object],PATIENTS = 215 SURVIVE 178 82.8% DEAD 37 17.2% Is BP<=91? Terminal Node A SURVIVE 6 30.0% DEAD 14 70.0% NODE = DEAD Terminal Node B SURVIVE 102 98.1% DEAD 2 1.9% NODE = SURVIVE PATIENTS = 195 SURVIVE 172 88.2% DEAD 23 11.8% Is AGE<=62.5? Terminal Node C SURVIVE 14 50.0% DEAD 14 50.0% NODE = DEAD PATIENTS = 91 SURVIVE 70 76.9% DEAD 21 23.1% Is SINUS<=.5? Terminal Node D SURVIVE 56 88.9% DEAD 7 11.1% NODE = SURVIVE <= 91 > 91 <= 62.5 > 62.5 >.5 <=.5

How to Read It ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Decision Questions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

The Tree is a Classifier ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

The Importance of Binary Splits ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Accuracy of a Tree ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Prediction Success Table ,[object Object],[object Object],Classified As TRUE Class 1 Class 2 "Survivors" "Early Deaths" Total % Correct Class 1 "Survivors" 158 20 178 88% Class 2 "Early Deaths" 9 28 37 75% Total 167 48 215 86.51%

Tree Interpretation and Use ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Binary Recursive Partitioning ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Three Major Stages ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

General Workflow Stage 1 Stage 2 Stage 3 Historical Data Learn Test Validate Build a Sequence of Nested Trees Monitor Performance Best Confirm Findings

Large Trees and Nearest Neighbor Classifier ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Derivation ,[object Object],[object Object],[object Object],[object Object],[object Object]

Impact on the Relative Error ,[object Object],[object Object]

Searching All Possible Splits ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Split Tables ,[object Object],[object Object],Sorted by Age Sorted by Blood Pressure

Categorical Predictors ,[object Object],[object Object],[object Object],[object Object],Left Right 1 A B, C, D 2 B A, C, B 3 C A, B, D 4 D A, B, C 5 A, B C, D 6 A, C B, D 7 A, D B, C

Binary Target Shortcut ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

GYM Model Example ,[object Object],[object Object],[object Object],[object Object],[object Object]

Variable Definitions ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],predictors not used target

Initial Runs ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Pruning Multiple Nodes ,[object Object],[object Object],[object Object]

Pruning Weaker Split First ,[object Object],[object Object]

Understanding Variable Importance ,[object Object],[object Object],[object Object],[object Object]

Competitors and Surrogates ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

The Utility of Surrogates ,[object Object],[object Object],[object Object],[object Object],[object Object]

Competitors and Surrogates Are Different ,[object Object],[object Object],[object Object],[object Object],A B C A B C Split X A B C A C B Split Y ,[object Object],[object Object],[object Object]

Calculating Association ,[object Object],[object Object],[object Object],[object Object],          Split X Split Y           Default           ,[object Object]

Notes on Association ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Calculating Variable Importance ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Mystery Solved ,[object Object],[object Object]

Penalizing Individual Variables ,[object Object],[object Object],[object Object]

Penalizing Missing and Categorical Predictors ,[object Object],[object Object],[object Object],[object Object]

Splitting Rules - Introduction ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Priors Adjusted Probabilities ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

GINI Splitting Rule ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

GINI Properties ,[object Object],[object Object],[object Object]

Example ,[object Object],[object Object]

ENTROPY Splitting Rule ,[object Object],[object Object],[object Object],[object Object],[object Object]

TWOING Splitting Rule ,[object Object],[object Object],[object Object],[object Object],[object Object]

Best Possible Split ,[object Object],[object Object],[object Object],[object Object],[object Object]

Example ,[object Object],A 40 B 30 C 20 D 10 A 40 B 30 C 20 D 10 GINI Best Split A 40 B 30 C 20 D 10 A 40 D 10 B 30 C 20 TWOING Best Split

Symmetric GINI ,[object Object],[object Object],[object Object],[object Object]

Ordered TWOING ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Class Probability Rule ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Favor Even Splits Control ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Fundamental Tree Building Mechanisms ,[object Object],[object Object],[object Object]

Key Players ,[object Object],DM Engine Analyst Model Dataset Priors  i Costs C i  j Population

Fraud Example ,[object Object],[object Object],[object Object],[object Object]

Fraud Example – PRIORS EQUAL (0.5, 0.5) ,[object Object],[object Object],[object Object],[object Object]

Fraud Example – PRIORS SPECIFY (0.9, 0.1) ,[object Object],[object Object],[object Object],[object Object]

Fraud Example – PRIORS DATA (0.98, 0.02) ,[object Object],[object Object],[object Object],[object Object]

Internal Class Assignment Rule ,[object Object],[object Object],[object Object],[object Object]

Putting It All Together ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

The Impact of Priors ,[object Object],[object Object],[object Object], r =0.1 ,  b =0.9  r =0.9 ,  b =0.1  r =0.5 ,  b =0.5

Model Evaluation ,[object Object],Evaluator Analyst Expected Cost Dataset Priors  i Costs C i  j Model PS Table Cells n i  j Population

Estimating Expected Cost ,[object Object],[object Object],[object Object],[object Object]

PRIORS DATA – Relative Cost ,[object Object],[object Object],[object Object],[object Object],[object Object]

PRIORS EQUAL – Relative Cost ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Using Equivalent Costs ,[object Object],[object Object]

Automated CART Runs ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

More Batteries ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Battery MCT ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Battery MVI ,[object Object],[object Object],[object Object],[object Object],[object Object]

Battery PRIOR ,[object Object],[object Object],[object Object],[object Object],[object Object]

Battery SHAVING ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Battery SAMPLE ,[object Object],[object Object],[object Object]

Battery TARGET ,[object Object],[object Object],[object Object],[object Object]

Recommended Reading ,[object Object],[object Object],[object Object],[object Object]

Advanced cart 2007

Recommended

Recommended

More Related Content

What's hot

What's hot (17)

Viewers also liked

Viewers also liked (20)

Similar to Advanced cart 2007

Similar to Advanced cart 2007 (20)

Recently uploaded

Recently uploaded (20)

Advanced cart 2007