61. Got Phone?
MRA Insights & Strategies Conference | June 2015
M A R Y M C D O U G A L L
M M C D O U G A L L @ C F M C . C O M
Editor's Notes
Good morning
Who is using phone today?
Phone-based research, like Mark Twain said, “Reports of my death have been greatly exaggerated.”
Clearly much of the research focus has shifted to online surveys for good reason.
Today, we will explore scenarios where it may be advantageous, if not required, to leverage phone methodologies to deliver quality projects.
I’m Mary McDougall, CEO of CFMC, which is a software developer of the Survox phone survey automation platform.
Survox powers much of the phone data collection that goes on in the US.
Particularly strong in public opinion research and political polling, which is heating up now for the 2016 elections.
Both sectors turn to phone because there demographics matter greatly.
My goal for today is to have you consider the strategic imperative for phone in your Customer Experience programs and some of your healthcare, financial services and other research.
And that, as a result of this session, you….
Why you should consider phone data collection
The advantages of taking a multi-mode approach.
Impact on research budgets
Changes in technology that allow more flexible field design.
But first a quick tutorial on what I mean when referring to Phone Data Collection…..
Two types of Phone-based collection methods
LIVE PHONE INTERVIEWS
Using trained agents who conduct the interview
Physically housed in a call center or remotely distributed.
Primarily used for outbound surveys.
The other phone method is IVR where a computer, not a person, conducts the interview.
IVR is ideal for both outbound calling and inbound receiving, since the computer is ‘always on’.
Traditionally IVR surveys have been standalone, but we’ll explore how they can be mainstreamed more into your research.
Whenever either method is used for recruitment and outreach, there is a great need to optimize the process in order to:
Reach respondents at the appropriate time of day.
Keep interviewer productivity high and idle time low.
Minimize telco costs by minimizing amount of sample dialed.
Manage the sample flow to maximize quota attainment.
Unlike online surveys where you need to push and prod people to take the survey…
With good sample management, call centers can reach exactly who you need to fill quotas and nurture them through the whole survey to get the results you want.
So that’s what’s meant by ‘phone’.
Now let’s explore why it’s still relevant today.
MISSING
Nearly half of all adults over 65 have not accessible via the web.
Yet they have disposable income, consume services, buy cars, get sick, pay taxes and vote.
1 in 4 of less educated, lower income, and rural customers who may be you primary customer depending on where your store or service is focused.
But they all have a phone…98% of the US population has phone service
Start in online survey
Show existing survey; show existing panel
Switch to Survox Console
Register the Online Project (mention that it only displays surveys that are active and that the user has access to)
Import the sample from a online panel
Activate the survey
Take the survey as an interviewer (talk through the dialer interaction but we won't use an actual dialer)
After running through a few respondents - go back into the online solution and show a report (report should include responses from both the web and phone interviews)
Other reasons to consider phone are
** Interviewers can manage a long, in-depth survey where there may otherwise be a high risk of respondent fatigue and drop-out – good for medical studies where answers may trigger follow-up questions.
** Call centers can manage their sample to produce representative results early ….process call replicates where respondent demographics are managed throughout the period to mirror the overall desired profile for the study.
Other
Customer problems can be immediately handled with call transfer to an agent or supervisor
Survey responses can be examined across modes to determine if the mode is affecting responses.
Which is what the recently published Pew Research Center study shows from a 2014 public opinion survey fielded in both web and phone modes.
Findings showed significant differences in responses between modes.
Sometimes Phone produced the higher response, other times Web.
Challenge is to harmonize the differences, but the key point is that it is important to recognize that there are differences .
Using a single mode would not uncover.
So, now you might consider it! That’s movement.
But…
Still have lots of choices…but let’s compare phone to web.
The costs go up across these modes.
Sample, if you need to purchase it, can differ in price with phone sample actually cheaper than online panels.
Telco is the cost of the phone call, and depending on your service provider, this cost will be incurred whether connected or not.
Interviewer’s labor and the cost of supervisors and other overhead.
If we ignore the sample cost and only consider the collection cost
Online have virtually no variable cost to collecting 1 vs. 1001 responses.
Whereas
IVR has telco costs
Phone interview has telco and labor
If you consider lists where one has to manually dial cell phone numbers
IVR may be eliminated because it can’t be used for cell
Phone cost go way up as interviewer time is used to dial until you connect to a cooperative person
Finally – when you add sample, because panel sample is always higher than listed or RDD phone sample, the total costs may shift depending on what type of sample is needed.
So how do make the trade-off?
Traditionally you may use what you have in-house.
The point of this session is to assure you that you can and that technology should not be the barrier.
Those barriers are coming down and what we are bringing to the market is the ability to create a single survey, employ multiple data collection strategies, and have the data all available for analysis in a single data set.
Now researchers can choose the right methods that fit EACH job and not be constrained by the technology.
And allow you field your study however best for the study,
And analyze the data efficiently and in a way that captures mode.
So let’s look at some use cases.
Once upon a time there was a Franchise Manager named Molly who had the tricky challenge of maintaining a consistent, high quality customer experience from services delivered by organizations that only indirectly reported to her.
Every day Molly made decisions about which locations to renew and which ones to drop, based solely on sales data. This approach created pretty high turnover and meant that at any given time there were a fair number of new franchises coming online, needing training, and depressing sales.
One day, Molly realized that her pipeline of new applicants was running low and she’d have to find a different way to deal with the problem locations.
Because of that, Molly worked with her team to come up with the key metrics that may yield an early warning of problems to come. She also developed a set of questions to ask customers to provide feedback on various aspects of the service. Now she needed to collect the data which is a problem since the franchisees ‘owned’ the customer relationship and customer contact information was not available. Molly was blocked.
Until finally, she realized that her organization supplied the POS system used to print receipts. Now she could appeal directly to the customers themselves with an incentive to tell her what she needed to know….and to maximize the feedback she decided to give her customers a choice of ways to respond.
Now, each receipt is now printed with both an 800# and a URL to a quick survey…giving Molly and her team the insight needed to identify potential problems and how to help them. Franchise turnover has been dramatically reduced.
Since your stores span the country from major city to rural townships….
And your clientele come from all walks of life….
Then why not provide a choice of ways to hear back from customers?
~100% have phone – so offer an 800#
15% increase in potential respondents
75% increase in reach to Seniors
~25% increase in lower socio/economic segments
Now you can give people a choice of ways to engage to maximize your results.
Invite them to a web survey AND provide a phone number.
When they call in you can have a live person greet them but more likely you will use an automated IVR system conduct the survey
Great for short survey like C-Sat or NPS
But if you don’t want to have self-selection bias where you only hear from the very happy or the very unhappy.
Rather you want to make sure that you hear from every region, every store, or every service agent
Ensure you are getting a standardized view of the customer experience.
Then you need to do set up an outbound system to solicit their response, and quota requirements for each reporting category.
This can be done by using transaction data…from a POS for store loyalty customers or a CRM or ticketing system for a sales or services call as the Sample.
Set up quotas per store or agent.
Follow-up each transaction with a call to ask how their experience was.
The call can be connected to a live agent or an IVR system to conduct the interview.
Voice of the Customer Examples
Retailers – provide incentive to collect feedback and now on receipt offer choice – URL or Phone # (which usually goes to an IVR system)
Service Quality checks
Major national home improvement chain that uses 3rd party installers uses phone interviewers to check on customer experience.
Sample is provided by a feed from other systems such as an ERP or POS
Too important to them to let respondent self-select who provides feedback.
Customer Support departments transfer caller to IVR survey post transaction
One large consumer products company uses national sampling and phone recruitment to get the right mix of consumers to take mail and web surveys for new product input.
Another way to think about phone vs other survey methods it the 80:20 rule….where one survey method may not apply to all.
We all understand the concept of how usually
20% of customers ----- 80% revenue or profit
20% donors ……provide 80% of the funding for your non-profit
20% employees …. Are really critical to the success of your business
For example, if you’re a casino you’re going to treat your high rollers differently from your tourists
Or if you’re an airline you’ll treat your Platinum and 1K members differently than your Bronze level members.
If you’re a CEO, you may want more feedback from your staff than from the general workforce.
So you’ll be willing to spend more for richer insights.
One survey, one data set produced….different modes of data collection employed for different audience segments
Online
Phone
In person using a tablet device
For example, if you’re a casino you’re going to treat your high rollers differently from your tourists
Or if you’re an airline you’ll treat your Platinum and 1K members differently than your Bronze level members.
Now you can create one survey,
Use different modes of data collection for different audience segments
- Online
Phone
In person using a tablet device
Analyze it all from the one data set produced
And if you’re a large company…
Differentiated data collection strategy for 360 reviews or new policy/product roll-outs.
Executives – In-person (web survey on tablet)
High Potential & Management – recorded Phone interviews
Staff – Web survey
One survey – possibly different lengths
Central analysis
Use multiple modes to stay within budget AND still reach quota.
Maximize responses from most cost effective method first – online.
Finish with outbound IVR or phone interviewing to complete quota.
- IVR has the Telco expense
- Phone interviewing has Telco and labor expense
When you look at a study with 1000 responses, and compare across differ mode choices
When quota matters
Daily public opinion poll – EVERY day 1000 responses are collected using RDD sampling; Not 999 or 1001 but exactly 1000 or not a valid sample.
- Phone provides reliable response rates so that they get all the responses needed
- Quota management ensures that they don’t oversample
Campaign strategists
Call with the candidate every night at 10 pm
At 9 pm the strategist calls the polling center to get the days results
Accurate insights depend on tight sample control that produces responses from a replicate of the target population.
i.e. The polling center had to have collected input from exactly the demographic mix that is expected to vote
In 2014, the American Institute of Research conducted a field test of survey Health Insurance Marketplace survey
Using Mail, Phone, and Web surveys, placing into each sample only those with known access to the survey method. They tested standalone and combined modes.
Phone was the best standalone mode.
Mail + Phone provided the highest overall response rate
Phone produced better match with the sample frame, reaching more from groups that were underrepresented by mail or web.
Interestingly, and not intuitive to me, the phone research yielded the best response rate from those under 34, the ‘always on’ generations….perhaps because it is harder to ignore than a mail or web survey.
At the end of 2013, the Professional Data Analysts conducted a mode effectiveness test using a Tobacco Health Line survey that had been traditionally done by phone.
Testing the effectiveness of first web with phone follow-up.
Multi-mode approach yielded better results than just phone.
-- Higher response rate
-- Lower cost.
-- Costs could go lower still with choice of technologies.
Back-end processing: dispositions, surveys, merging from two different systems
Infrastructure – software licensing, phone systems, web hosting, backups, office space, PC’s and phones for interviewers
Set-Up: planning, email design, survey design, testing/QA, hiring/staffing interviewers, sampling, training interviewers
Monitoring:
Health survey that had been traditionally done by phone.
Testing the effectiveness of first web with phone follow-up.
Multi-mode approach yielded better results than just phone.
-- Higher response rate
-- Lower cost.
-- Costs could go lower still with choice of technologies.
Back-end processing: dispositions, surveys, merging from two different systems
Infrastructure – software licensing, phone systems, web hosting, backups, office space, PC’s and phones for interviewers
Set-Up: planning, email design, survey design, testing/QA, hiring/staffing interviewers, sampling, training interviewers
Monitoring:
But those barriers are coming down and what we are bringing to the market is the ability to create a single survey, employ multiple data collection strategies, and have the data all available for analysis in a single data set.
Now researchers can choose the right methods that fit EACH job and not be constrained by the technology.
Adding Phone is a decision that can be made from the beginning of the design or after data collection has started.
While you have the option to modify the survey based on mode you do not have to make any changes at all. You can use the same survey as the web
but with phone system and interviewer management options… the choice is yours.
However, you do have the option of mode based logic to manage entire screens such as in this example of introductions.
What the interviewer experiences is the Qualtrics survey with phone operational controls provided and managed by Survox.
What your respondents get is choices in how they engage with you, with your research… web or phone.
You get a broader reach.
you might have your own call center or engage with a partner like Zylun Insights which you can find on the Innovation Exchange.
This allows you to full manage the Qualtrics survey and data and flexibly decide to engage phone using your own call center, a partner or some combination.
The entire work flow then becomes…
So you can set up an online survey and flip it to phone at any stage of the study.
A one and done client setup
Import or Register the project
Import or Add sample
Engage your call center and manage the project… interviewers, sample
Report on your data from rich Qualtrics reporting solutions.
Survox provides the technology and tools to integrate the phone mode and manage the interviewers, project sample and the call center.
While the survey and response data is housed in Qualtrics, Survox manages the call center operations and reporting.
The basic flow is that after a project is registered with Survox and the panel imported, the numbers are sent to the dialer to be called.
Answered calls are connected to interviewers.
The interviewer integrated interface to conduct the the survey (Qualtrics) and manage the call (Survox)
The interviewer is provide an interface the seamlessly integrates the Qualtrics respondent survey with the Survox phone operation controls.
-The interviewer can log in and connect to the dialer
-Is provided with workforce option such as going on break, lunch or meetings
-Has controls to handle call connection and dispositioning
-During the survey, the interviewer can manage the call flow
-Having a live interviewer allows you to collect richer verbatim content and let’s the interviewer probe and clarify responses
while the dialer allows you to collect sound records of the call to deliver the ultimate voice of the customer inflection and all
You get to benefit by leveraging the best of both… Qualtrics survey design and reporting and Survox call center automation and management.
Qualtrics lets you collect and report on your response data.
Survox lets you watch and manage your phone production.
All phone operation data resides within the Survox platform.
A variety of reports and dashboards allow you to manage the full scope of your phone operation from
Interviewer to Project to the Entire Call Center
Ultimately how you implement phone is your choice to meet your research needs.
Whether you have a single phone mode study or a multimode design.
If multimode you might start with web and phone with fixed daily targets.
Or
You might start on web then bring on phone to round out your data collection and fill in the quotas after giving
respondents some time to complete the survey via web and a subsequent reminder blast as illustrated in the
Production report on the right.
Regardless of the point at which phone is brought in to the flow,
all survey response data resides within the Qualtrics database.
No importing, no merging
Existing reports and integrations continue to work
Because the modality of data collection is stored in the data, you can create mode specific reports and analysis.
But those barriers are coming down and what we are bringing to the market is the ability to create a single survey, employ multiple data collection strategies, and have the data all available for analysis in a single data set.
Now researchers can choose the right methods that fit EACH job and not be constrained by the technology.