Noriega

1. © 2017 Chevron Corporation
Open Sourced Workforce Analytics
A Live Demonstration of Tools and Techniques for Predictive Modeling
Jason Noriega
Predictive Analytics World for Workforce
5/16/2017

2. 2© 2017 Chevron Corporation
Agenda
Introduction
Background and Experience
Visualizing the impact of important patterns
Technique, Case Study, Demonstration
Interactive model building
Case Study, Demonstration
Making predictions as accurately as possible
Technique, Case Study

3. 3© 2017 Chevron Corporation
Agenda
Introduction
Background and Experience
Visualizing the impact of important patterns
Technique, Case Study, Demonstration
Interactive model building
Case Study, Demonstration
Making predictions as accurately as possible
Technique, Case Study

4. 4© 2017 Chevron Corporation
Introduction
Jason Noriega
Diversity Analytics Team Lead
Optimizing Recruiting Resources
3rd of 100 Competitors
Prior Experience in People Analytics
- SanDisk
- eBay
- Lawrence Livermore National Lab
- NASA
Predicting Employee TurnoverPredicting Driver Alertness
5th of 176 Competitors 1st of 330 Competitors

5. 5© 2017 Chevron Corporation
Agenda
Introduction
Background and Experience
Visualizing the impact of important patterns
Technique, Case Study, Demonstration
Interactive model building
Case Study, Demonstration
Making predictions as accurately as possible
Technique, Case Study

6. 6© 2017 Chevron Corporation
How can we visualize the impact of
important patterns to drive action?
Improving the Retention of Employees

7. 7© 2017 Chevron Corporation
How can we visualize the impact of
important variables to drive action?
Recursive Partitioning Decision Rules - Example
Rule 1: 55% Likelihood of Leaving
Performance = Does Not Meet
Rule 2: 44% Likelihood of Leaving
Performance = Meets, Exceeds, Exceeds
Most, and Region=Americas, and
Mgr. Effectiveness < 75% Favorable
Rule 3: 4% Likelihood of Leaving
Performance = Meets, Exceeds, Exceeds
Most, And Region=Americas, and
Mgr. Effectiveness >= 75% Favorable
# %
Stayed: 1210 83%
Left: 252 17%
# %
Stayed: 595 68%
Left: 130 32%
Region
Americas
# %
Stayed: 283 89%
Left: 10 11%
Asia Pacific
# %
Stayed: 248 96%
Left: 10 4%
EMEA
# %
Stayed: 145 56%
Left: 115 44%
# %
Stayed: 450 96%
Left: 15 4%
Mgr. Effectiveness
< 75% >= 75%
Performance Rating
# %
Stayed: 84 45%
Left: 102 55%
# %
Stayed: 1126 88%
Left: 150 12%
Does Not
Meet
Meets, Exceeds,
Exceeds Most
Predictors
1. Performance
2. Tenure
3. Age
4. ……
Predictors
1. Region
2. Commute
3. Compa Ratio
4. ……
Predictors
1. Mgr. Effect.
2. Tenure
3. Job Function
4. ……
All data is notional

8. 8© 2017 Chevron Corporation
How can we visualize the impact of
important variables to drive action?
Where does the
Problem Exist?
Why do these
Patterns Exist?
Where should we
prioritize areas of
improvement?
What can do to
reduce high rates of
voluntary turnover?
Improving the Retention of Employees
Decision Trees Survey Analysis Matrix Action Planning

9. 9© 2017 Chevron Corporation
How can we visualize the impact of
important variables to drive action?
2014 2016
Demographics
Age
Gender
Tenure
…
Variables for Attrition Modeling
Job Related
Internal Movement
Upward Movement
Job Function
Pay Grade
Performance Rating
Change in Rating
…
Location
Location Change
Commute
Region
Country
…
Compensation
Comp Ratio
Promotion Flag
Spot Award Flag
Tuition Reimbursement
…
2015
Model Timeframe for Finding Patterns
Attrition Analysis Timeframe
May
Manager
Mgr. Effectiveness
Mgr. Performance
Mgr. Tenure
…
June

10. 10© 2017 Chevron Corporation
How can we visualize the impact of
important variables to drive action?
# %
Stayed: 11000 86%
Left: 1800 14%
# %
Stayed: 4000 78%
Left: 1000 20%
# %
Stayed: 3000 86%
Left: 500 14%
# %
Stayed: 3000 94%
Left: 250 8%
# %
Stayed: 1000 95%
Left: 50 5%
Employee Tenure
Less than 1.1 yrs 1.1-2.6 2.6-7.3 > 7.3 Yrs
# %
Stayed: 2000 87%
Left: 300 13%
# %
Stayed: 500 83%
Left: 100 17%
# %
Stayed: 1500 71%
Left: 600 29%
Pay Grade
<= 16 17-22 > 22
# %
Stayed: 1100 77%
Left: 320 23%
Business Unit
Organization #2Organization #1
# %
Stayed: 400 59%
Left: 280 41%
All data is notional
# %
Stayed: 200 48%
Left: 220 52%
Comp Ratio
> 85<= 85
# %
Stayed: 200 77%
Left: 60 23%

11. 11© 2017 Chevron Corporation
How can we visualize the impact of
important variables to drive action?
Exiting Employees
Where are the greatest areas of improvement?
Job matched expectations
Opportunity
for growth
High Importance/
Low Performance
Low Importance/
Low Performance
Low Importance/
High Performance
High Importance/
High Performance
# %
Stayed: 11000 86%
Left: 1800 14%
# %
Stayed: 4000 80%
Left: 1000 20%
# %
Stayed: 3000 86%
Left: 500 14%
# %
Stayed: 3000 92%
Left: 250 8%
# %
Stayed: 1000 95%
Left: 50 5%
Employee Tenure
Less than 1.1 yrs 1.1-2.6 2.6-7.3 > 7.3 Yrs
# %
Stayed: 2000 87%
Left: 300 13%
# %
Stayed: 500 83%
Left: 100 17%
# %
Stayed: 1500 71%
Left: 600 29%
Pay Grade
<= 16 17-22 > 22
All data is notional
Job made good
use of my skills

12. 12© 2017 Chevron Corporation
Agenda
Introduction
Background and Experience
Visualizing the impact of important patterns
Technique, Case Study, Demonstration
Interactive model building
Case Study, Demonstration
Making predictions as accurately as possible
Technique, Case Study

13. 13© 2017 Chevron Corporation
How to make predictions as accurately
as possible?
Random Forest

14. 14© 2017 Chevron Corporation
How to make predictions as accurately as possible?
Permanent
10%Leavers
90%Stayers
15%Leavers
85%Stayers
35%
65%
Age <
30 yrs.
Tenure
< 2 yrs.
10%Leavers
90%Stayers
25%Leavers
75%Stayers
40%Leavers
60%Stayers
Female
Non-
Exempt
10%Leavers
90%Stayers
20%Leavers
80%Stayers
31%Leavers
69%Stayers
401k
Plan
Can you guess the weight of the Ox?
Wisdom of Crowds Random Subspace
Appointment Age Tenure Gender BenPlan FLSA Target
Permanent 22 5 Male 401k Exempt 0: Stay
Permanent 24 6 Male 401k Exempt 0: Stay
Permanent 60 9 Female Pension Non-Exempt 1: Leave
Flexible 63 5 Female 401k Non-Exempt 1: Leave
Permanent 51 9 Male Pension Exempt 1: Leave
Permanent 26 7 Male 401k Exempt 0: Stay
Permanent 27 8 Male Pension Exempt 0: Stay
Permanent 25 9 Female Pension Non-Exempt 0: Stay
Flexible 27 5 Female 401k Non-Exempt 0: Stay
Permanent 28 6 Male 401k Non-Exempt 0: StayVisualization
Crowd Average: 1,197
Actual Weight: 1,198
https://public.tableau.com/profile/jaysha101#!/vizhome/SanDiskAttrition_0/EmployeeAttritionRiskDashboard
Leavers
Stayers

15. 15© 2017 Chevron Corporation
Agenda
Introduction
Background and Experience
Visualizing the impact of important patterns
Technique, Case Study, Demonstration
Interactive model building
Case Study, Demonstration
Making predictions as accurately as possible
Technique, Case Study

16. 16© 2017 Chevron Corporation
Manager
Compensation
Job Hopper
Location
Skill Demand
Performance
Multiplicity of Good Models
q
Rashomon Principle:
Same Event, Different Stories
Multiplicity of Good Models
(Rashomon Principle)

17. 17© 2017 Chevron Corporation
Where does the
problem exist?
What impact does the
manager have?
How can we target those
most likely to stay?
Compensation
Location/Job/Org
- Region
- Country
- Work Location
- Pay Grade
- Job Function
- Job Level
- Business Unit
- Performance Rating
…
Recruitment
- Recruitment Source
- Avg. Tenure in Prior
Companies
- Total Yrs. Work
Experience
- Education Level
- School
…
Manager Impact
- Mgr. Effectiveness Score
(Survey)
- Expectations
- Development
- Attitude
- Coaching
- Mgr. Performance
- Mgr. Tenure
…
???
???
…
…
…
…
…
…
…
…
…
Multiplicity of Good Models
(Rashomon Principle)

18. 18© 2017 Chevron Corporation
Where does the problem exist?
# %
Stayed: 1210 83%
Left: 252 17%
# %
Stayed: 795 80%
Left: 194 20%
Region
North America
# %
Stayed: 83 87%
Left: 12 13%
Asia Pacific
# %
Stayed: 248 93%
Left: 20 7%
Europe
Performance Rating
# %
Stayed: 84 76%
Left: 26 24%
# %
Stayed: 1126 83%
Left: 226 17%
Does Not
Meet Expectations
Location/Job/Org
- Region
- Country
- Work Location
- Pay Grade
- Job Function
- Job Level
- Business Unit
- Performance Rating
# %
Stayed: 638 77%
Left: 188 23%
A, B, and C
# %
Stayed: 157 96%
Left: 6 4%
All Other
Job Function
Multiplicity of Good Models
(Rashomon Principle)
Exceptional,
Excellent,
Meets Expectations

19. 19© 2017 Chevron Corporation
What impact does the manager have?
# %
Stayed: 459 72%
Left: 182 28%
# %
Stayed: 179 97%
Left: 6 3%
Mgr. Effectiveness
< 75% >= 75%
Manager Impact
- Mgr. Effectiveness
Score (Survey)
- Expectations
- Development
- Attitude
- Coaching
- Mgr. Performance
- Mgr. Tenure
# %
Stayed: 1210 83%
Left: 252 17%
# %
Stayed: 795 80%
Left: 194 20%
Region
North America
# %
Stayed: 83 87%
Left: 12 13%
Asia Pacific
# %
Stayed: 248 93%
Left: 20 7%
Europe
Performance Rating
# %
Stayed: 84 76%
Left: 26 24%
# %
Stayed: 1126 83%
Left: 226 17%
Does Not
Meet Expectations
Exceptional,
Excellent,
Meets Expectations
# %
Stayed: 638 77%
Left: 188 23%
A, B, and C
# %
Stayed: 157 96%
Left: 6 4%
All Other
Job Function
Multiplicity of Good Models
(Rashomon Principle)

20. 20© 2017 Chevron Corporation
Multiplicity of Good Models
(Rashomon Principle)
How can we target those most likely to stay?
Recruitment
- Recruitment Source
- Avg. Tenure in Prior
Companies
- Total Yrs. Work
Experience
- Education Level
- School
# %
Stayed: 1210 83%
Left: 252 17%
# %
Stayed: 795 80%
Left: 194 20%
Region
North America
# %
Stayed: 83 87%
Left: 12 13%
Asia Pacific
# %
Stayed: 248 93%
Left: 20 7%
Europe
Performance Rating
# %
Stayed: 84 76%
Left: 26 24%
# %
Stayed: 1126 83%
Left: 226 17%
Does Not
Meet
Meets, Exceeds,
Exceeds Most
# %
Stayed: 638 77%
Left: 188 23%
A, B, and C
# %
Stayed: 157 96%
Left: 6 4%
All Other
Job Function
# %
Stayed: 325 69%
Left: 145 31%
# %
Stayed: 313 88%
Left: 43 12%
Recruitment Source
A, B C, D, E

21. 21© 2017 Chevron Corporation