Designing a Survey Instrument

Developing
A
Survey
Instrument

-‐
An
Overview

Sam
Klaidman

Principal
Adviser

Middlesex
Consul?ng

Sam@Middlesexconsul?ng.com

@Sklaidman

508.877.1924

© 2009 Middlesex Consulting
. All Rights Reserved

How
Much
is
Enough?

Responses Needed for 95% Confidence with ±5% Margin of Error


First
Steps

•  Iden?fy
the
execu?ve
sponsor

–  Plan
her
involvement

•  Determine
objec?ve
of
the
survey

–  Who
is
the
audience?

•  Iden?fy
the
intended
survey
community

–  Size

–  Roles

–  Level

•  Think
about
how
the
results
will
be
used

–  Ac?on

–  Informa?on

–  Baseline


First
Steps
(con?nued)

•  Get
input
from
your
customers

–  What s
on
their
mind?

–  How
much
?me
can
they
spend
on
survey

–  Preferred
media

•  Plan
the
mix
of
qualita?ve
vs.
quan?ta?ve
ques?ons

•  One
?me
or
ongoing
survey?

•  Medium
-‐
web,
phone,
mail

•  Incen?ves?

•  Process

Now
we
can
start
planning
the
survey
instrument


Survey
Types
-‐
Transac?on
(event
driven)

•  A_er
installa?on

•  A_er
a
call
to
Tech
Support,
Inside
Sales,
AR,
Service

•  A_er
an
interview
(HR
process
evalua?on)

•  A_er
customer
uses
your
product
or
service

–  Intruder
detected
and
police
no?ﬁed

–  Accident
claim
submiaed

–  First
delivery
received
with
no
other
company
interac?on
(ship
to
end

user)

• Typically
web

• Typically
2
to
5
ques?ons

• 1
to
3
days
a_er
the
event


Survey
Types
-‐
Rela?onship
(mid-‐level)

•  Used
to
monitor
how
customers
feel
about
your
business

•  Very
useful
to
determine:

–  Sta?s?cal
impact
of
each
touch
point
on
Sa?sfac?on
and
Loyalty

–  Gap
between
Importance
and
Sa?sfac?on

•  Broad
focus
as
compared
to
the
narrower
Transac?on
survey

•  Used
to
plan
changes
and
monitor
their
impact
on
a
long
term
basis

• Typically
15
to
30
ques?ons

• Scheduled
(monthly,
quarterly,
etc.)


Survey
Types
-‐
Key
Accounts

•  Like
the
rela?onship
survey
but
for
decision
makers/recommenders/end-‐
users

•  Find
strengths
and
weaknesses

•  Use
with
similar
employee
surveys
-‐
ensures
alignment

•  Must
be
face-‐to-‐face
with
telephone
as
fallback

•  These
are
special
people
and
deserve
special
treatment

• No
more
than
35
to
40
ques?ons

• Should
take
<30
minutes
to
complete

• As
o_en
as
possible
without
annoying
par?cipants


Survey
Types
-‐
Special
Purpose
Surveys

•  Market
research

•  Lost
business

•  Employees

•  Suppliers

•  Partners


Typical
Areas
to
Rate

•  Reliability
-‐
Deliver
on
your
promise?

•  Responsiveness
-‐
Helpful?

•  Assurance
-‐
Trust
and
conﬁdence

•  Empathy
-‐
Treat
customers
as
individuals

•  Tangibles
-‐
Making
the
intangible
real


Qualita've/Quan'ta've
Research

• Random
samples
•  Convenience
samples

• Closed
ques?ons
•  Polls
with
comments

• Limited
response
op?ons
•  Online
Communi?es

• Can t
get
the
story
behind
Quan?ta?ve
Research
•  Focus
Groups

the
story
•  Provide
the
Story
behind

• Provides
hard
data
that
the
story

can
be
extrapolated
to
a
•  Can t
extrapolate
to
a

Qualita?ve
Research

larger
audience
larger
audience


Quan?ta?ve
Ques?ons

•  Should
have
granularity

–  Yes/no
is
vague

–  1-‐5,
0-‐10
are
typical
and
more
specific

•  Should
have
balanced
anchors

–  Extremely
sa?sfied
/
extremely
dissa?sfied

–  High
value
/
low
value

•  Must
be
self
explanatory
and
unambiguous

•  Must
rate
only
1
item
per
ques?on

–  Professionalism
and
courtesy
are
2
items
usually
lumped
together

•  Should
be
impersonal

–  Rate
processes
and
policies
not
individuals

•  Should
have
an
escape
selec?on

–  No
opinion

–  No
comment


Qualita?ve
Ques?ons

•  Must
not
suggest
answers

•  Must
be
easy
to
understand

•  Must
provide
enough
room
to
answer
completely

•  Ideal
for
text
mining

–  Depending
on
number
of
surveys
could
have
diﬀerent
people
with

individual
biases,
summarizing
results

–  Lose
emo?ons

–  Very
?me
consuming

•  Help
explain
quan?ta?ve
answers

•  Add
credibility
to
survey
results


Choosing
Media

•  Web
survey

–  Lowest
comple?on
rate
but
easiest
to
automate

–  Least
intrusive
to
customer

–  Reports
exactly
what
the
customer
wants
to
say

•  Telephone
survey

–  Higher
comple?on
rate

–  Can
be
pre-‐scheduled

–  Poten?al
for
edi?ng

–  High
tech/high
touch

•  Mail/Fax

–  So
5
minutes
ago
but
may
be
necessary
where
English
is
not
the
primary

language.


Some
Examples
From
a
Market

Research
Telephone
Survey


Example
#1

Please rate the following service delivery attributes in terms of their importance to you, using a scale from 10 to 1, where 10 is
the most important and 1 is least important :

Question 03
Please rate how important it is to easily get your inspection and/or service visits scheduled.

01) 10 04
02) 9
03) 8
04) 7
05) 6
06) 5
07) 4
08) 3
09) 2
10) 1
11) No Opinion

This
is
part
of
a
group
of
ques?ons
asking
customers
to
rate
the

importance
of
various
service
aaributes.


Example
#2

Question 13
What features of your current service agreement(s) are of most value to
you?

01) Comments / What? 14 GL3
02) No Comment

Question 14
What features that you don t currently have in your service agreement(s)
would you like to have?

01) Comment / What? 15 GL3
02) None
03) No Comment Notice the escape selections
and the invitation to comment


Example
#3

Question 17
Assuming you were allowed, how likely would you be to recommend COMPANY to colleagues or others? Would you
be…

01) Extremely Likely 18

02) Likely
03) Neutral / Why? GL3
04) Unlikely / Why? GL3
05) Extremely Unlikely / Why?
DK) No Comment

No?ce
the
symmetry
of
the
choices


Any
Ques?ons?

Thank
You


Designing a Survey Instrument

Recommended

Recommended

More Related Content

What's hot

What's hot (8)

Similar to Designing a Survey Instrument

Similar to Designing a Survey Instrument (20)

Recently uploaded

Recently uploaded (20)

Designing a Survey Instrument