HR Analytics – Demystified!
ARUN KRISHNAN, PH.D,
FOUNDER & CEO,
nFactorial Analytical Sciences
What is Analytics?
information resulting from the
systematic analysis of data or statistics
Since when have we
been doing analytics?
Daniel Kahnemann & Amos Tversky
System 1 System 2
 Solve: -5 +2 x 2 + 9 /3 - 8
What words would you choose
to describe her?
So why the buzz around analytics
now?
Technological advances - price / performance
Pervasive digitization
Artificial Intelligence, Machine Learning
Big Data and Analytics
What is Big
Data?
" One bit more data
than your system can
hold"
Source: www.cloudlendinginc.com
Analytics is a continuum …
Complexity
Perspective
Low
Past
High
Future
BusinessValue
Reporting
What happened?
Analysis
Why did it happen?
Monitoring
What is happening now?
Prediction
What will happen?
...and Analytics is a journey!
Source: Applied Insurance Analytics, by Patricia Soparito
Analytics Domains
Retail Sales
Marketing
Collections
Telecom
Financial
Services
Risk & Credit
Consumer
Behavior
Fraud
Supply Chain
Talent / HR
Pricing
Web
Football Analytics
AN EXAMPLE
What happened?
Why did it happen?
What’s happening now?
What could happen in future?
HR Analytics
Analytics is coming to HR!
Source: www.bersin.com
Why HR Analytics?
Measure &
Manage
"What gets measured,
gets managed; what
gets managed, gets
executed"
- Peter Drucker
Linkage of
Business
Objectives to
People
Strategies
HR Dashboards - SAP
"To clearly
demonstrate the
interaction of business
objectives and
workforce strategies"
Return on
Investment
- David Foster
"The business demands
on HR are increasingly
going to be on
analysis just because
people are SO
expensive"
Performance
Improvement
- CedarCrestone
"Global organizations
with workforce
analytics and
workforce planning
outperform all other
organizations by 30%
more sales/employee"
HR capability gaps are increasing
Source: Deloitte Human Capital Report, 2015
HR Analytics - Much promise -
wanting in rewards?
Source: Deloitte Human Capital Report, 2015
The HR Analytics Continuum
Complexity
Perspective
Low
Past
High
Future
BusinessValue
Head Count
Attrition
Training
Payroll reports Performance
Tracking
Requisition
Tracking
Turnover Ratio
Accession Ratio
Low performer
management
Promotion Ratio
Hiring Fit
Hiring No-shows
Prediction
Attrition
Prediction
Attrition
Segmentation
Employee
Segmentation
Candidate
Stickability
Prediction
High Performer
Segmentation
Workforce
Planning
Informal Network
Analysis
Voice of Employee
Analysis
Recruitment
Engagement
Retention
What metrics do we typically
track?
Source: Bersin & Associates 2012 – US research
What metrics should we track?
Recruitment Retention
Performance
Management
Career
Management
Training
Workforce
Planning
Comp &
Benefits
Org.
Effectiveness
Measuring Human Resources
Management
 Over 100 different metrics
across
Hiring and Staffing
Compensation and Benefits
Training and Development
Employee Relations and
Retention
So how about some recruitment-
related metrics to start with?
Cost
•Cost per hire
•Source cost per
hire
•Advertising cost
per hire
•Agency cost per
hire
•Referral bonus
per hire
•Unsolicited no-
cost per hire
•Special costs per
hire
•Interview costs
•Source cost per
hire per interview
•Sign-on bonus
factor
Time
•Response time
•Average
response time
per hire
•Time to fill
•Time to start
•Referral factor
Career
Development
•Job posting
response rate
•Job posting
response factor
•Job posting hire
rate
•Internal hire rate
•Career path ratio
- promotions
•Career path
ration - transfers
Efficiency Metrics
•Average
interview length
•Hire rate
•Hit rate
Quality
•Quality of Hire
•Recruiter
Effectiveness
Detailed Case Study
GOOGLE
The early days - Finding the right
people
 Spent hours screening resumes
from job portals like Monster.com
 Built an applicant tracking system
that checked candidate resumes
against a database of Googler
resumes
 Idea was to get more realistic
"backdoor" references
 Also looked at innovative ways to
identify smart people
The solution to the first riddle will land you at http://7427466391.com/. On this
page you’ll find the following:
“Congratulations. You’ve made it to level 2. Go to www.Linux.org and enter
Bobsyouruncle as the login and the answer to this equation as the password.”
f(1)= 7182818284
f(2)= 8182845904
f(3)= 8747135266
f(4)= 7427466391
f(5)= __________
Initial data analysis & insights
•Academic grades did not correlate well with performance except
for the first 2-3 years.
Analysis
•Stopped asking for academic transcripts except for fresh
graduates.
Actions
•Did not see any discernible drop in performance because of this.
Results
Initial data analysis & insights - 2
• Google's hiring was focused on minimizing "false positives", that is,
candidates who looked good at first glance but turned out to be poor
performers later.
• Their hiring took a long time - 250,000 hours to hire 1000 people/year
Analysis
• Looked at referrals as a way of hiring great candidates.
Actions
• In the initial years - >50% of hires were through referrals
Results
Employee referrals
•The rate started to fall after 2009
Challenge
•Could be because rewards weren’t high enough.
Hypothesis
•Google increased the reward for successful referrals thinking that it would help to bring up
the referral rates
Actions
•They found however, that this brought NO change in the decline.
Results
•Rewards are extrinsic motivators
•People were more motivated by intrinsic factors like pride in their place of work
Analysis
Employee referrals
• Exhausted known networks
Challenge
• Started using aided recalls
Action
• Volume of referrals increased by 33%!
Results
Cultivating the best people
• Requirement of ~300,000 referrals/year vs <100,000 they were getting
Challenge
• Realized that the very best people are not looking for work. They are happy
Analysis
• Rejigged their staffing team and equipped them with a home-grown tool
called gHire to cultivate people across different organizations.
Action
• >50% of Google's hires are found by this in-house team!
Results
Hiring the best people
•People during an interview make up their mind in the first 10 seconds
•Rest of the interview is spent finding corroborative evidence
•CONFIRMATION BIAS!
Challenge
•Most interviews are unstructured.
•Unstructured interviews can predict only ~14% of an employee's performance
•Work sample test predicts ~29% of performance
•General cognitive tests predict ~26% of performance
•biased towards white males (at least in the US) !
•Structured interviews were found to be as good at predicting performance as cognitive tests
Analysis [paper by Frank Schmidt & John Hunter]
•Use a combination of behavioral and situational structured interviews with assessments of cognitive ability,
conscientiousness and leadership
•Identified key attributes essential for "Googleeyness"
Actions
•Consistent scoring mechanism that allows people to compare across interviewers. .
Results
Hiring the best people - 2
•Hiring was taking too much time – median of 90-180 days
Challenge
•What should be the number of rounds of interviews?
• Found that 4 interviews were enough
Analysis
•Brought down the number of interviews from 25 to 4
Action
•Changed median time to hire from 90-180 days to 47 days!
Results
Revisit assumptions - Then test!
 Looked for people with high scores who were rejected
 2010 - ran 300,000 rejected candidates through the system
 Filtered 10,000 and chose 150
 Hit rate of 1.5% > 0.25% - Google's hit rate
 Tested False Negatives as well!
Revisit
program
•Feed
resumes of
all past
candidates
through
algorithm
Common
Keywords
•Assess
common
keywords
found
Score resumes
•Score
keywords
based on
their
occurrences
in rejected
vs successful
resumes
Test
•Score
resumes
over next 6
months
against
weighted
keywords
Predictive Analytics for Recruiting
Some Examples
Best Buy
Could precisely identify a 0.1%
increase in employee
engagement among
employees at a particular store.
This value was identified at
more than $100,000 in the
store's operating income.
Oracle / Sprint
Oracle was able to predict
which top performers would
leave and why.
This information is now driving
key global policy changes for
retaining key performers.
Sprint has identified the factors
that best foretell which
employees will leave after a
relatively short time.
Dow Chemicals
Has evolved its workforce
planning over the past
decade, mining historical data
on its 40,000 employees to
forecast promotion rates,
internal transfers, and overall
labor availability.
Dow uses a custom model to
segment its workforce and
calculates future headcount by
segment and level for each
business unit.
Dow can engage in "what if"
scenario planning altering
assumptions on internal
variables.
State-of-the art for Predictive
Analytics in Recruitment
Hiring Fit
Models
Candidate
Stickability
Predictions
Hiring No-
Shows
Predictions
Workforce
Planning
Personality
Matches
Predictive Modeling - Watchouts!
 All models are wrong! Some are less wrong than others
 Predictive Models cannot be used to predict rare, black-swan events
 Models can’t predict what is not already present in the training data.
 Building the right model depends on the question that needs to be
answered.
 This in turn determines the data that needs to be gathered.
 Even with enough data, we might not have the “right” data to build a
good predictive model.
 Exploratory data analysis and Feature Selection is an extremely critical
part of the model building workflow.
 Always check model performance using any of confusion matrix, p-
values, ROC curve etc.
 Keep updating your model as and when new data comes in.
Keys to success in HR Analytics
Start with
the business
problem in
mind
Develop
culture of
data-driven
decision
making
Empower
line leaders
Be
transparent
Analytics is
a journey,
not an end
Don't wait
for the
perfect
data
You don't
HAVE to
automate
everything -
at least at
first
Deliver
Actionable
Business
Information
Thank you for your patience!

Hr analytics – demystified!

  • 1.
    HR Analytics –Demystified! ARUN KRISHNAN, PH.D, FOUNDER & CEO, nFactorial Analytical Sciences
  • 2.
    What is Analytics? informationresulting from the systematic analysis of data or statistics
  • 3.
    Since when havewe been doing analytics?
  • 4.
    Daniel Kahnemann &Amos Tversky System 1 System 2  Solve: -5 +2 x 2 + 9 /3 - 8 What words would you choose to describe her?
  • 5.
    So why thebuzz around analytics now? Technological advances - price / performance Pervasive digitization Artificial Intelligence, Machine Learning
  • 6.
    Big Data andAnalytics What is Big Data? " One bit more data than your system can hold" Source: www.cloudlendinginc.com
  • 7.
    Analytics is acontinuum … Complexity Perspective Low Past High Future BusinessValue Reporting What happened? Analysis Why did it happen? Monitoring What is happening now? Prediction What will happen?
  • 8.
    ...and Analytics isa journey! Source: Applied Insurance Analytics, by Patricia Soparito
  • 9.
    Analytics Domains Retail Sales Marketing Collections Telecom Financial Services Risk& Credit Consumer Behavior Fraud Supply Chain Talent / HR Pricing Web
  • 10.
  • 11.
  • 12.
    Why did ithappen?
  • 13.
  • 14.
  • 15.
  • 16.
    Analytics is comingto HR! Source: www.bersin.com
  • 17.
    Why HR Analytics? Measure& Manage "What gets measured, gets managed; what gets managed, gets executed" - Peter Drucker Linkage of Business Objectives to People Strategies HR Dashboards - SAP "To clearly demonstrate the interaction of business objectives and workforce strategies" Return on Investment - David Foster "The business demands on HR are increasingly going to be on analysis just because people are SO expensive" Performance Improvement - CedarCrestone "Global organizations with workforce analytics and workforce planning outperform all other organizations by 30% more sales/employee"
  • 18.
    HR capability gapsare increasing Source: Deloitte Human Capital Report, 2015
  • 19.
    HR Analytics -Much promise - wanting in rewards? Source: Deloitte Human Capital Report, 2015
  • 20.
    The HR AnalyticsContinuum Complexity Perspective Low Past High Future BusinessValue Head Count Attrition Training Payroll reports Performance Tracking Requisition Tracking Turnover Ratio Accession Ratio Low performer management Promotion Ratio Hiring Fit Hiring No-shows Prediction Attrition Prediction Attrition Segmentation Employee Segmentation Candidate Stickability Prediction High Performer Segmentation Workforce Planning Informal Network Analysis Voice of Employee Analysis Recruitment Engagement Retention
  • 21.
    What metrics dowe typically track? Source: Bersin & Associates 2012 – US research
  • 22.
    What metrics shouldwe track? Recruitment Retention Performance Management Career Management Training Workforce Planning Comp & Benefits Org. Effectiveness
  • 23.
    Measuring Human Resources Management Over 100 different metrics across Hiring and Staffing Compensation and Benefits Training and Development Employee Relations and Retention
  • 24.
    So how aboutsome recruitment- related metrics to start with? Cost •Cost per hire •Source cost per hire •Advertising cost per hire •Agency cost per hire •Referral bonus per hire •Unsolicited no- cost per hire •Special costs per hire •Interview costs •Source cost per hire per interview •Sign-on bonus factor Time •Response time •Average response time per hire •Time to fill •Time to start •Referral factor Career Development •Job posting response rate •Job posting response factor •Job posting hire rate •Internal hire rate •Career path ratio - promotions •Career path ration - transfers Efficiency Metrics •Average interview length •Hire rate •Hit rate Quality •Quality of Hire •Recruiter Effectiveness
  • 25.
  • 26.
    The early days- Finding the right people  Spent hours screening resumes from job portals like Monster.com  Built an applicant tracking system that checked candidate resumes against a database of Googler resumes  Idea was to get more realistic "backdoor" references  Also looked at innovative ways to identify smart people The solution to the first riddle will land you at http://7427466391.com/. On this page you’ll find the following: “Congratulations. You’ve made it to level 2. Go to www.Linux.org and enter Bobsyouruncle as the login and the answer to this equation as the password.” f(1)= 7182818284 f(2)= 8182845904 f(3)= 8747135266 f(4)= 7427466391 f(5)= __________
  • 27.
    Initial data analysis& insights •Academic grades did not correlate well with performance except for the first 2-3 years. Analysis •Stopped asking for academic transcripts except for fresh graduates. Actions •Did not see any discernible drop in performance because of this. Results
  • 28.
    Initial data analysis& insights - 2 • Google's hiring was focused on minimizing "false positives", that is, candidates who looked good at first glance but turned out to be poor performers later. • Their hiring took a long time - 250,000 hours to hire 1000 people/year Analysis • Looked at referrals as a way of hiring great candidates. Actions • In the initial years - >50% of hires were through referrals Results
  • 29.
    Employee referrals •The ratestarted to fall after 2009 Challenge •Could be because rewards weren’t high enough. Hypothesis •Google increased the reward for successful referrals thinking that it would help to bring up the referral rates Actions •They found however, that this brought NO change in the decline. Results •Rewards are extrinsic motivators •People were more motivated by intrinsic factors like pride in their place of work Analysis
  • 30.
    Employee referrals • Exhaustedknown networks Challenge • Started using aided recalls Action • Volume of referrals increased by 33%! Results
  • 31.
    Cultivating the bestpeople • Requirement of ~300,000 referrals/year vs <100,000 they were getting Challenge • Realized that the very best people are not looking for work. They are happy Analysis • Rejigged their staffing team and equipped them with a home-grown tool called gHire to cultivate people across different organizations. Action • >50% of Google's hires are found by this in-house team! Results
  • 32.
    Hiring the bestpeople •People during an interview make up their mind in the first 10 seconds •Rest of the interview is spent finding corroborative evidence •CONFIRMATION BIAS! Challenge •Most interviews are unstructured. •Unstructured interviews can predict only ~14% of an employee's performance •Work sample test predicts ~29% of performance •General cognitive tests predict ~26% of performance •biased towards white males (at least in the US) ! •Structured interviews were found to be as good at predicting performance as cognitive tests Analysis [paper by Frank Schmidt & John Hunter] •Use a combination of behavioral and situational structured interviews with assessments of cognitive ability, conscientiousness and leadership •Identified key attributes essential for "Googleeyness" Actions •Consistent scoring mechanism that allows people to compare across interviewers. . Results
  • 33.
    Hiring the bestpeople - 2 •Hiring was taking too much time – median of 90-180 days Challenge •What should be the number of rounds of interviews? • Found that 4 interviews were enough Analysis •Brought down the number of interviews from 25 to 4 Action •Changed median time to hire from 90-180 days to 47 days! Results
  • 34.
    Revisit assumptions -Then test!  Looked for people with high scores who were rejected  2010 - ran 300,000 rejected candidates through the system  Filtered 10,000 and chose 150  Hit rate of 1.5% > 0.25% - Google's hit rate  Tested False Negatives as well! Revisit program •Feed resumes of all past candidates through algorithm Common Keywords •Assess common keywords found Score resumes •Score keywords based on their occurrences in rejected vs successful resumes Test •Score resumes over next 6 months against weighted keywords
  • 35.
  • 36.
    Some Examples Best Buy Couldprecisely identify a 0.1% increase in employee engagement among employees at a particular store. This value was identified at more than $100,000 in the store's operating income. Oracle / Sprint Oracle was able to predict which top performers would leave and why. This information is now driving key global policy changes for retaining key performers. Sprint has identified the factors that best foretell which employees will leave after a relatively short time. Dow Chemicals Has evolved its workforce planning over the past decade, mining historical data on its 40,000 employees to forecast promotion rates, internal transfers, and overall labor availability. Dow uses a custom model to segment its workforce and calculates future headcount by segment and level for each business unit. Dow can engage in "what if" scenario planning altering assumptions on internal variables.
  • 37.
    State-of-the art forPredictive Analytics in Recruitment Hiring Fit Models Candidate Stickability Predictions Hiring No- Shows Predictions Workforce Planning Personality Matches
  • 38.
    Predictive Modeling -Watchouts!  All models are wrong! Some are less wrong than others  Predictive Models cannot be used to predict rare, black-swan events  Models can’t predict what is not already present in the training data.  Building the right model depends on the question that needs to be answered.  This in turn determines the data that needs to be gathered.  Even with enough data, we might not have the “right” data to build a good predictive model.  Exploratory data analysis and Feature Selection is an extremely critical part of the model building workflow.  Always check model performance using any of confusion matrix, p- values, ROC curve etc.  Keep updating your model as and when new data comes in.
  • 39.
    Keys to successin HR Analytics Start with the business problem in mind Develop culture of data-driven decision making Empower line leaders Be transparent Analytics is a journey, not an end Don't wait for the perfect data You don't HAVE to automate everything - at least at first Deliver Actionable Business Information
  • 40.
    Thank you foryour patience!