Pulse Surveys //
What, When, How
and Why?
SARAH MARRS
PRINCIPAL CONSULTANT, EMPLOYEE INSIGHTS,
QUALTRICS
2
©2015QUALTRICSLLC.
The recording for today’s presentation
will be available at talentweek.com.
It’s not too late! Make sure you
register for Talentweek.
Housekeeping
@qualtrics
#Talentweek
#HumanCapital
JOIN THE CONVERSATION
Sarah Marrs
PRINCIPAL CONSULTANT, EMPLOYEE INSIGHTS
Sarah has dedicated her career to designing and running employee feedback programs for over 7
years. She joined Qualtrics in April 2015, after working in-house at Tesco overseeing their global
colleague insights survey. At Qualtrics, Sarah works with clients to design survey solutions that
drive real business value, through questionnaire design, sampling methodologies, survey
communication programs, or interpreting survey results. She has particular experience helping
organisations to take the first step from annual surveys to more regular feedback such as Pulse
surveys or Lifecycle feedback.
Prior to working at Tesco, Sarah spent the bulk of her career at the Hay Group, leading survey
delivery for the Pacific region. With the Hay Group Sarah designed and delivered surveys to many
different types of organisations – banks, logistics companies, energy companies and more. She’s a
big believer in the power of listening to and acting on employee feedback, and always energised to
talk about new and innovative ways to collect and act on that information.
ALWAYS ON FEEDBACK
ADHOC SURVEYS
EMPLOYEE LIFECYCLE
SURVEYS
CENSUS ENGAGEMENT SURVEY
Employee
Insights are no
longer just about
Engagement
surveys
PULSE SURVEYS
Slow-moving
Long-term
Robust
Broad
Immediate-feedback
Shorter-term
Specific
5
©2015QUALTRICSLLC.
Today’s session WHAT IS A PULSE SURVEY
WHY RUN PULSE SURVEYS
CONSIDERATIONS FOR PULSE SURVEYS
LENGTH
CADENCE
SAMPLING
HOW TO CONSTRUCT A PULSE SURVEY
TWO COMMON EXAMPLES
Ask yourself these four questions. If the answer to all four is ‘yes,’ then it’s a pulse survey.
Does it track the same construct over
time, measuring it at least twice?
Is it a shorter survey?
1
2
3
4
Is it more frequent? (relative to the
annual census survey)
Does it happen at regular time
intervals?
What is a
Pulse Survey?
A Pulse
Survey is not
Bound to measuring a specific set of content / construct
A “one off” survey that can be sent by anyone at anytime
A survey that changes its content completely over time
SURVEY
OUTCOME
(ENGAGEMENT)
SURVEY
OUTCOME
(ENGAGEMENT)
SURVEY
OUTCOME
(ENGAGEMENT)
DRIVERS
(ORG PRACTICES/
BEHAVIOURS)
DRIVERS
(ORG PRACTICES/
BEHAVIOURS)
DRIVERS
(ORG PRACTICES/
BEHAVIOURS)
SURVEY EVENT 1 SURVEY EVENT 2 SURVEY EVENT 3
The ROI of Pulse Surveys
(NPS, $$)
BUSINESS
OUTCOME
(NPS, $$)
BUSINESS
OUTCOME
(NPS, $$)
BUSINESS
OUTCOME
Pulse Survey
Considerations
Cadence, Length & Sampling
WEEKLY
5-10 qu’s
MONTHLY
10-15 qu’s
QUARTERLY
15-20 qu’s
BI-ANNUAL
20-30 qu’s
WHAT WE’LL TALK ABOUT TODAY
Sampling may be needed Sampling may not be needed
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
%ofpopulationneededtoget
representativesample(±3confidence)
Population size
The bigger population, the smaller % of people you need to ask
2 common sampling
techniques
REPRESENTATIVE SAMPLING
2 common sampling
techniques
EQUAL-SPLIT SAMPLING
J
F
M
A
M
JJ
A
S
O
N
D
Monthly Pulse
Q1
Q2Q3
Q4
Quarterly Pulse
5 steps to
construct a pulse
survey program
5 steps to construct a
pulse survey program
What business outcomes do we want to drive?
What constructs influence that?
STEP 1 // DETERMINE THE GOAL OF THE PULSE SURVEY
• Build predictive model of how org practices / behaviors predict future engagement and business
outcomes.
• Track changes (improvements based on action) over time
5 steps to construct a
pulse survey program
Monthly
Quarterly
Bi-annual
STEP 2 // DECIDE ON THE RIGHT CADENCE FOR YOUR PULSE SURVEY
Note: You should be ready to take action at the
same cadence that you are surveying
Understand the cadence at which important
business metrics / outcomes are measured / tracked
5 steps to construct a
pulse survey program
Monthly // 10-15 items
Quarterly // 15-20 items
Bi-annual // 20-30 items
STEP 3 // DECIDE ON THE LENGTH OF YOUR SURVEY
5 steps to construct a
pulse survey program
OUTCOMES // Engagement, eNPS, customer barriers
DRIVERS // Company practices & behaviours
STEP 4 // IDENTIFY YOUR SURVEY ITEMS
5 steps to construct a
pulse survey program
STEP 5 // IDENTIFY A SAMPLING STRATEGY
Monthly // Sampling could be needed
Quarterly // Potentially no sampling needed
Bi-annual // Likely no sampling needed
ACTION PLANNING FOLLOW-UP
Linked to an engagement survey (i.e. track
actions or key drivers)
1 2 EMPLOYEE ENGAGEMENT PULSE
Track employee attitudes / engagement
(e.g. eNPS) and potential drivers
Two common
monthly / quarterly
pulse survey
examples
Example 1 //
Action Planning
Follow-up
Action Planning
follow-up Pulse
STEP 1 // Determine the goal of the survey:
STEP 2 // Decide your survey cadence:
STEP 3 // Define your survey length:
STEP 4 // Define survey content:
STEP 5 // Define sampling strategy:
To track the success taken of actions on an annual survey
Bi-annual
Up to 20 questions
OUTCOMES: Use the same Engagement measure as annual survey given the two events are linked
DRIVERS: Measure top critical areas for action planning as uncovered in your annual survey. This
might be a combination of lower-scoring key drivers, or items with lower absolute scores.
Ask all participants as cadence is infrequent and want to check in with all employees mid-year.
Action Planning
follow-up Pulse
No
.
ITEM CATEGORY
1 I feel proud to tell people where I work Engagement Outcome
2 This company motivates me to go above and beyond in my role Engagement Outcome
3 I would recommend this company to family and friends as a great place to work Engagement Outcome
4 I see myself working at this company in two years time Engagement Outcome
5 Overall, I am satisfied with my job at this company Engagement Outcome
6 There are good opportunities for career progression at this company Career Progression
7
--- Follow-up for disagree - What would you want to see from this company to give you the career
opportunities you're looking for?
Career Progression
Action Planning
follow-up Pulse
No
.
ITEM CATEGORY
8 My manager encourages collaboration on my team Collaboration
9 It is easy to collaborate with other functions or teams within this company Collaboration
10 I am happy with my current work-life balance Work-life Balance
11 I hear enough communication from the senior leadership team about what is happening at the company Company Leadership
12 --- Follow-up for disagree - What types of communication methods do you prefer? (select up to 3) Company Leadership
13 I have seen action as a result of the last employee survey Post-survey action
14 What do you think we could have done better to take action on the survey? Open Text
Example 2 //
Engagement Pulse
Surveys
Engagement Pulse
STEP 1 // Determine the goal of the survey:
STEP 2 // Decide your survey cadence:
STEP 3 // Define your survey length:
STEP 4 // Define survey content:
STEP 5 // Define sampling strategy:
To measure engagement and it’s drivers more frequently than just once a year and align with
customer data
Monthly
Up to 10 questions
OUTCOMES: Look for a shortened version of your engagement measure, that measures the
components more inclined to fluctuate at your desired cadence
DRIVERS: Look for a broader range of drivers which reflect the driver categories in an annual survey.
Aim for one driver item per category.
As customer data is reported at Divisional level, ask an equal number of employees every month
so that each employee is asked to contribute once a year.
Engagement Pulse
No. ITEM CATEGORY
1 I would recommend this company to family or friends as a great place to work Engagement – Org Commitment
2 Generally speaking, when I get up in the morning I look forward to going to work Engagement - Job Involvement
3 It's up to me how I get things done in my role Autonomy & Empowerment
4 We work well together on my team Collaboration
5 There is open and honest two-way communication on my team Communication
6 I am happy with the recognition and promotion I've received in this organisation to date Recognition
7 I have the tools / equipment / technology I need to be most effective in my role Resources
8 My manager helps me resolve any work-related issues I have Supportive Management
9 I have the training I need to be successful in my role Training & Development
Thank
You

Pulse Surveys // What, When, How and Why?

  • 1.
    Pulse Surveys // What,When, How and Why? SARAH MARRS PRINCIPAL CONSULTANT, EMPLOYEE INSIGHTS, QUALTRICS
  • 2.
    2 ©2015QUALTRICSLLC. The recording fortoday’s presentation will be available at talentweek.com. It’s not too late! Make sure you register for Talentweek. Housekeeping @qualtrics #Talentweek #HumanCapital JOIN THE CONVERSATION
  • 3.
    Sarah Marrs PRINCIPAL CONSULTANT,EMPLOYEE INSIGHTS Sarah has dedicated her career to designing and running employee feedback programs for over 7 years. She joined Qualtrics in April 2015, after working in-house at Tesco overseeing their global colleague insights survey. At Qualtrics, Sarah works with clients to design survey solutions that drive real business value, through questionnaire design, sampling methodologies, survey communication programs, or interpreting survey results. She has particular experience helping organisations to take the first step from annual surveys to more regular feedback such as Pulse surveys or Lifecycle feedback. Prior to working at Tesco, Sarah spent the bulk of her career at the Hay Group, leading survey delivery for the Pacific region. With the Hay Group Sarah designed and delivered surveys to many different types of organisations – banks, logistics companies, energy companies and more. She’s a big believer in the power of listening to and acting on employee feedback, and always energised to talk about new and innovative ways to collect and act on that information.
  • 4.
    ALWAYS ON FEEDBACK ADHOCSURVEYS EMPLOYEE LIFECYCLE SURVEYS CENSUS ENGAGEMENT SURVEY Employee Insights are no longer just about Engagement surveys PULSE SURVEYS Slow-moving Long-term Robust Broad Immediate-feedback Shorter-term Specific
  • 5.
    5 ©2015QUALTRICSLLC. Today’s session WHATIS A PULSE SURVEY WHY RUN PULSE SURVEYS CONSIDERATIONS FOR PULSE SURVEYS LENGTH CADENCE SAMPLING HOW TO CONSTRUCT A PULSE SURVEY TWO COMMON EXAMPLES
  • 6.
    Ask yourself thesefour questions. If the answer to all four is ‘yes,’ then it’s a pulse survey. Does it track the same construct over time, measuring it at least twice? Is it a shorter survey? 1 2 3 4 Is it more frequent? (relative to the annual census survey) Does it happen at regular time intervals? What is a Pulse Survey?
  • 7.
    A Pulse Survey isnot Bound to measuring a specific set of content / construct A “one off” survey that can be sent by anyone at anytime A survey that changes its content completely over time
  • 8.
    SURVEY OUTCOME (ENGAGEMENT) SURVEY OUTCOME (ENGAGEMENT) SURVEY OUTCOME (ENGAGEMENT) DRIVERS (ORG PRACTICES/ BEHAVIOURS) DRIVERS (ORG PRACTICES/ BEHAVIOURS) DRIVERS (ORGPRACTICES/ BEHAVIOURS) SURVEY EVENT 1 SURVEY EVENT 2 SURVEY EVENT 3 The ROI of Pulse Surveys (NPS, $$) BUSINESS OUTCOME (NPS, $$) BUSINESS OUTCOME (NPS, $$) BUSINESS OUTCOME
  • 9.
    Pulse Survey Considerations Cadence, Length& Sampling WEEKLY 5-10 qu’s MONTHLY 10-15 qu’s QUARTERLY 15-20 qu’s BI-ANNUAL 20-30 qu’s WHAT WE’LL TALK ABOUT TODAY Sampling may be needed Sampling may not be needed
  • 10.
    0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% %ofpopulationneededtoget representativesample(±3confidence) Population size The biggerpopulation, the smaller % of people you need to ask 2 common sampling techniques REPRESENTATIVE SAMPLING
  • 11.
    2 common sampling techniques EQUAL-SPLITSAMPLING J F M A M JJ A S O N D Monthly Pulse Q1 Q2Q3 Q4 Quarterly Pulse
  • 12.
    5 steps to constructa pulse survey program
  • 13.
    5 steps toconstruct a pulse survey program What business outcomes do we want to drive? What constructs influence that? STEP 1 // DETERMINE THE GOAL OF THE PULSE SURVEY • Build predictive model of how org practices / behaviors predict future engagement and business outcomes. • Track changes (improvements based on action) over time
  • 14.
    5 steps toconstruct a pulse survey program Monthly Quarterly Bi-annual STEP 2 // DECIDE ON THE RIGHT CADENCE FOR YOUR PULSE SURVEY Note: You should be ready to take action at the same cadence that you are surveying Understand the cadence at which important business metrics / outcomes are measured / tracked
  • 15.
    5 steps toconstruct a pulse survey program Monthly // 10-15 items Quarterly // 15-20 items Bi-annual // 20-30 items STEP 3 // DECIDE ON THE LENGTH OF YOUR SURVEY
  • 16.
    5 steps toconstruct a pulse survey program OUTCOMES // Engagement, eNPS, customer barriers DRIVERS // Company practices & behaviours STEP 4 // IDENTIFY YOUR SURVEY ITEMS
  • 17.
    5 steps toconstruct a pulse survey program STEP 5 // IDENTIFY A SAMPLING STRATEGY Monthly // Sampling could be needed Quarterly // Potentially no sampling needed Bi-annual // Likely no sampling needed
  • 18.
    ACTION PLANNING FOLLOW-UP Linkedto an engagement survey (i.e. track actions or key drivers) 1 2 EMPLOYEE ENGAGEMENT PULSE Track employee attitudes / engagement (e.g. eNPS) and potential drivers Two common monthly / quarterly pulse survey examples
  • 19.
    Example 1 // ActionPlanning Follow-up
  • 20.
    Action Planning follow-up Pulse STEP1 // Determine the goal of the survey: STEP 2 // Decide your survey cadence: STEP 3 // Define your survey length: STEP 4 // Define survey content: STEP 5 // Define sampling strategy: To track the success taken of actions on an annual survey Bi-annual Up to 20 questions OUTCOMES: Use the same Engagement measure as annual survey given the two events are linked DRIVERS: Measure top critical areas for action planning as uncovered in your annual survey. This might be a combination of lower-scoring key drivers, or items with lower absolute scores. Ask all participants as cadence is infrequent and want to check in with all employees mid-year.
  • 21.
    Action Planning follow-up Pulse No . ITEMCATEGORY 1 I feel proud to tell people where I work Engagement Outcome 2 This company motivates me to go above and beyond in my role Engagement Outcome 3 I would recommend this company to family and friends as a great place to work Engagement Outcome 4 I see myself working at this company in two years time Engagement Outcome 5 Overall, I am satisfied with my job at this company Engagement Outcome 6 There are good opportunities for career progression at this company Career Progression 7 --- Follow-up for disagree - What would you want to see from this company to give you the career opportunities you're looking for? Career Progression
  • 22.
    Action Planning follow-up Pulse No . ITEMCATEGORY 8 My manager encourages collaboration on my team Collaboration 9 It is easy to collaborate with other functions or teams within this company Collaboration 10 I am happy with my current work-life balance Work-life Balance 11 I hear enough communication from the senior leadership team about what is happening at the company Company Leadership 12 --- Follow-up for disagree - What types of communication methods do you prefer? (select up to 3) Company Leadership 13 I have seen action as a result of the last employee survey Post-survey action 14 What do you think we could have done better to take action on the survey? Open Text
  • 23.
  • 24.
    Engagement Pulse STEP 1// Determine the goal of the survey: STEP 2 // Decide your survey cadence: STEP 3 // Define your survey length: STEP 4 // Define survey content: STEP 5 // Define sampling strategy: To measure engagement and it’s drivers more frequently than just once a year and align with customer data Monthly Up to 10 questions OUTCOMES: Look for a shortened version of your engagement measure, that measures the components more inclined to fluctuate at your desired cadence DRIVERS: Look for a broader range of drivers which reflect the driver categories in an annual survey. Aim for one driver item per category. As customer data is reported at Divisional level, ask an equal number of employees every month so that each employee is asked to contribute once a year.
  • 25.
    Engagement Pulse No. ITEMCATEGORY 1 I would recommend this company to family or friends as a great place to work Engagement – Org Commitment 2 Generally speaking, when I get up in the morning I look forward to going to work Engagement - Job Involvement 3 It's up to me how I get things done in my role Autonomy & Empowerment 4 We work well together on my team Collaboration 5 There is open and honest two-way communication on my team Communication 6 I am happy with the recognition and promotion I've received in this organisation to date Recognition 7 I have the tools / equipment / technology I need to be most effective in my role Resources 8 My manager helps me resolve any work-related issues I have Supportive Management 9 I have the training I need to be successful in my role Training & Development
  • 26.

Editor's Notes

  • #9 Having outcomes and drivers is general good practice for surveys and pulse surveys are no exception