Edward Shihadeh

1. REVOLUTIONIZE YOUR MODELS
CASE STUDY: Crime Reduction & Customer Retention
Edward S. Shihadeh, PhD
Chief Data Scientist
Auspice Analytics

2. Current $1.75 million federal grant for risk algorithms for Louisiana DOC
Mathematical demographer, criminologist, 30 years - data scientist
Co-author, Statistical Models for Ordinal Variables
(Advanced SAGE Series)
Advisor to District Attorney on Baton Rouge crime reduction initiative
Chief Data Scientist for AUSPICE ANALYTICS
Dr. Edward S. Shihadeh

3. 32% Murder rate decrease in Baton Rouge since
2012
Pushed customer retention to record highs
Rewriting offender management system in
Louisiana
Our track Record : Breakthroughs

4. Two examples
An unusual context: Predictive
analytics to reduce the murder rate
How one industry’s leading retention
product failed and how this perspective
improved outcomes dramatically
How you can leverage these successes
Proving that Social Science Improves Outcomes

5. Baton Rouge
Gangs
Gangs
Murder

6. Incomplete
Sociological
Dynamic of Gangs
Brokers

7. 66 Lives Saved
66 Spared Prison
100’s of Lives Not Shattered
Homicide Down 32%

8. Blaming the Data
Social Science
Solution
Individual Scoring
vs.
Variable EffectsX

9. Mass Incarceration

10. Customer Retention -- University

12. Loss of Tuition Legislative Penalty
Market Leader
Product “off the shelf”
X

13. Market Leader Probability Distribution
DENSITY
05
04
03
02
01
0
Probability of Returning to School
Average Retention : 72.32
20 40 60 80 100
“Maybe Curve”

14. Our Results
Training Data = 41,000 students over 6 years

15. Comparing Accuracy
BASE ERROR RATE
29 X LOWER
Off-the –shelf system
Predicted 72.3 %
Actual 84.6% (- 14.5% error)
Our Approach
Predicted 85.0%
Actual 84.6% (0.5% error)
Misclassification
Error
Off-the-shelf
34%
Our
Approach
4%

17. % Range # of Predicted % # predicted # actually Total # % of category
Customers (weighted) to return returned misclassified misclassified
0-10% 20 6.9% 1.4 1.0 0.4 1.9%
11-20% 53 15.4% 8.1 11.8 3.6 6.9%
21-30% 52 25.6% 13.3 17.0 3.7 7.1%
31-40% 59 35.3% 20.8 12.0 8.8 14.9%
41-50% 82 45.7% 37.5 30.0 7.5 9.1%
51-60% 115 55.3% 63.6 69.0 5.4 4.7%
61-70% 218 65.5% 142.8 133.0 9.8 4.5%
71-80% 434 75.8% 328.8 334.0 5.3 1.2%
81-90% 1157 85.8% 992.5 993.1 0.6 0.0%
91-100% 3155 95.1% 3000.7 2984.9 15.8 0.5%
Predicted Total 5345 60.7 <=misclassified
Actual Total 5490 205.7
<= misclassified
( missing cases)
Percent
misclassified => 4 %34%

18. Social Science
Solution
Major - College
GPA
Differential
Distance
HOW WE IMPROVED THE OUTCOME

19. Optimization
ROC Correctly Classified True Negatives
Logistic Regression .78 76.7% 2575
J48 Decision Tree .72 76.8% 2514
Multilayer Perceptron .77 75.8% 2632
Max Difference Between = 118
ROC Correctly Classified True Negatives
Logistic Regression .88 81.2% 3442
J48 Decision Tree .81 82.6% 3216
Multilayer Perceptron .87 80.2% 3404
Least Difference Across= 588
Estimation Optimization
Social Science Optimization

20. Efficiency of Predictive
Modelling
This is the distribution of all 250,000+
students from which we recruit each
year.
Recruitment staff often wastes energy
on this large group of people who have
absolutely no chance of coming to LSU.
0
204060
Density
0 .2 .4 .6 .8
Pr(enrolled)

21. Skewed Distribution Works in our Favor
•The distribution of predictions is highly skewed
•This lets you limit resource expenditure on a large number of individuals.
•This also allows you to find your targets more effectively.
•Narrow down 250,000 recruits to a more manageable 29,000. (removed 88.4%)

23. Practitioners People & Places

24. Questions or Comments
Ed@AuspiceAnalytics.com