May 27-30, 2014
Congress Plaza Hotel & Convention Center
#MRMW
Organized by
Title Sponsor Gold Sponsors
Silver Sponsors
MRMW Chicago May 27-30, 2014 #MRMW
Workshop Sponsors
Association & Education Partners
Networking Reception
Sponsor
Event App
Partner
Media Partners
MRMW Chic...
© TNS
© TNS May, 2014© TNS May, 2014
The Data They are A Changin’
Examining the impact of:
Data Sources
Data Collection De...
© TNS
© TNS May, 2014
2
We are facing a crossroads in data collection.
• The dramatic increase in the use of smartphones
•...
© TNS
© TNS May, 2014
1. Does data quality
differ by sample
source?
Research on Research – Questions to be answered
3
2. D...
© TNS
© TNS May, 2014
Methodology
4
Data Collection Platform
• PC
• Tablet
• Smartphone
Questionnaire
• Long – 10 minute t...
© TNS
© TNS May, 2014
Questionnaire Design
5
• Various Category Usage and Ownership
questions
• Mobile phone
• Brand own
•...
© TNS
© TNS May, 2014
Topline Findings
6
Sample Source:
• River Samples produced significantly overstated answers to many ...
© TNS
© TNS May, 2014
Topline Findings
7
Questionnaire – Optimized vs Non-Optimized:
• The Optimized Questionnaire - featu...
© TNS
© TNS May, 2014
Examples from the data
8
TNS Mobile RoR
© TNS May, 2014
PC Tablet
Smart
Phone PC Tablet
Smart
Phone
9
0
10
20
30
40
50
60
70
80
90
100
%
Panel A Pa...
TNS Mobile RoR
© TNS May, 2014
PC Tablet
Smart
Phone PC Tablet
Smart
Phone
10
0
10
20
30
40
50
60
70
80
90
100
%
Panel A P...
TNS Mobile RoR
© TNS May, 2014
PC Tablet
Smart
Phone PC Tablet
Smart
Phone
11
0
10
20
30
40
50
60
70
80
90
100
%
Panel A P...
TNS Mobile RoR
© TNS May, 2014
PC Tablet
Smart
Phone PC Tablet
Smart
Phone
12
0
10
20
30
40
50
60
70
80
90
100 %
Panel A P...
TNS Mobile RoR
© TNS May, 2014
PC Tablet
Smart
Phone PC Tablet
Smart
Phone
Standard Optimized Lumi – LSR Lumi – GMI GCS
PC...
TNS Mobile RoR
© TNS May, 2014
PC Tablet
Smart
Phone PC Tablet
Smart
Phone
14
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
Mean...
TNS Mobile RoR
© TNS May, 2014
PC Tablet
Smart
Phone PC Tablet
Smart
Phone
15
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
Mean...
TNS Mobile RoR
© TNS May, 2014
PC Tablet
Smart
Phone PC Tablet
Smart
Phone
16
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
Mean...
TNS Mobile RoR
© TNS May, 2014
PC Tablet
Smart
Phone PC Tablet
Smart
Phone
17
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
18.0...
TNS Mobile RoR
© TNS May, 2014
PC Tablet
Smart
Phone PC Tablet
Smart
Phone
18
0.0
10.0
20.0
30.0
%
Panel A Panel B River R...
© TNS
© TNS May, 2014
Implications
19
• River sampling requires some caution
- We need to investigate which sources of Riv...
Title Sponsor Gold Sponsors
Silver Sponsors
MRMW Chicago May 27-30, 2014 #MRMW
Workshop Sponsors
Association & Education Partners
Networking Reception
Sponsor
Event App
Partner
Media Partners
MRMW Chic...
May 27-30, 2014
Congress Plaza Hotel & Convention Center
#MRMW
Organized by
Upcoming SlideShare
Loading in …5
×

The Data They are A Changin' - TNS

1,172 views

Published on

Published in: Marketing, Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,172
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The Data They are A Changin' - TNS

  1. 1. May 27-30, 2014 Congress Plaza Hotel & Convention Center #MRMW Organized by
  2. 2. Title Sponsor Gold Sponsors Silver Sponsors MRMW Chicago May 27-30, 2014 #MRMW
  3. 3. Workshop Sponsors Association & Education Partners Networking Reception Sponsor Event App Partner Media Partners MRMW Chicago May 27-30, 2014 #MRMW
  4. 4. © TNS © TNS May, 2014© TNS May, 2014 The Data They are A Changin’ Examining the impact of: Data Sources Data Collection Devices Questionnaire Lengths Kirk Ward EVP, Research Methods and Offer Innovation MRMW May 29, 2014
  5. 5. © TNS © TNS May, 2014 2 We are facing a crossroads in data collection. • The dramatic increase in the use of smartphones • Difficulty in recruiting and retaining respondents for online panels We are going mobile But are we delivering high quality data? To help guide our clients through this transition, we need to gain an understanding of the impact of differing sample sources and data collection methodologies The Business Issue
  6. 6. © TNS © TNS May, 2014 1. Does data quality differ by sample source? Research on Research – Questions to be answered 3 2. Does data quality differ by data collection device? 3. What is the impact of questionnaire length on data an on termination rates? 4. What is the impact of optimizing the programming for smartphones? What impact do sample sources and data collection methods have on survey results?
  7. 7. © TNS © TNS May, 2014 Methodology 4 Data Collection Platform • PC • Tablet • Smartphone Questionnaire • Long – 10 minute target • Short – 5 minute target • Optimized vs. Non- optimized Interviews were conducted in April, 2014 Sample Source • Panel A • Panel B • Router • River sample • Google Consumer Survey Sample Composition • General population, ages 18-64 • Balanced by age, income, ethnicity, and geography • Differential sampling using methods currently used for each sample source
  8. 8. © TNS © TNS May, 2014 Questionnaire Design 5 • Various Category Usage and Ownership questions • Mobile phone • Brand own • Brand attributes • Concept test • Purchase interest • Uniqueness • Value • Liking • Relevance • Excitement • Understanding • Believability • Carbonated soft drinks (Coca-Cola, Pepsi, and Sprite) • Consumed in past week • Attributes for each brand • Demographics: • Income • Education • Marital status • Household size • Postal code The “Short” questionnaire featured a subset of these questions
  9. 9. © TNS © TNS May, 2014 Topline Findings 6 Sample Source: • River Samples produced significantly overstated answers to many questions – sometimes dramatically so. • Panel A and Panel B produced similar answers and answer patterns to each question, with Panel B answers being consistently a bit higher in terms of scale usage (although these differences were not typically meaningful). • The Router samples had results that were similar to the answers and answer patterns seen in the Panel A results • The Google Consumer Survey samples produced patterns similar to those produced by Panel A but answers tended to be a bit lower in terms of scale usage than the other sample sources Device: • Results were generally similar across device type in the pattern of the answers and the answers themselves.
  10. 10. © TNS © TNS May, 2014 Topline Findings 7 Questionnaire – Optimized vs Non-Optimized: • The Optimized Questionnaire - featuring simpler, conversational language and “gamified” response scripts - produced essentially the same answers and answer pattern as did the Non Optimized Questionnaire. • However: Incompletion rates were much higher for the Non-Optimized questionnaire Questionnaire Length: • The longer questionnaire produced dramatically higher drop out rates. • Both the Optimized and Non-Optimized questionnaires produced similar interview lengths with the Optimized Questionnaire taking slightly longer for respondents to complete.
  11. 11. © TNS © TNS May, 2014 Examples from the data 8
  12. 12. TNS Mobile RoR © TNS May, 2014 PC Tablet Smart Phone PC Tablet Smart Phone 9 0 10 20 30 40 50 60 70 80 90 100 % Panel A Panel B River Router GCS Category Usage – Smoked a cigarette Standard Optimized Lumi – LSR Lumi – GMI GCS PC Tablet Smartphone PC Tablet Smartphone Smartphone Smartphone PC Tablet Smart phone Long Short Long Short Long Short Long Short Long Short Long Short Long Short Long Short Short Short Short Standard & Optimized: Which have you done in the past month?
  13. 13. TNS Mobile RoR © TNS May, 2014 PC Tablet Smart Phone PC Tablet Smart Phone 10 0 10 20 30 40 50 60 70 80 90 100 % Panel A Panel B River Router GCS Category Usage – Bought motor oil for your car Standard Optimized Lumi – LSR Lumi – GMI GCS PC Tablet Smartphone PC Tablet Smartphone Smartphone Smartphone PC Tablet Smart phone Long Short Long Short Long Short Long Short Long Short Long Short Long Short Long Short Short Short Short Standard & Optimized: Which have you done in the past month?
  14. 14. TNS Mobile RoR © TNS May, 2014 PC Tablet Smart Phone PC Tablet Smart Phone 11 0 10 20 30 40 50 60 70 80 90 100 % Panel A Panel B River Router Category Usage – Traveled on a commercial airline Standard Optimized Lumi – LSR Lumi – GMI PC Tablet Smartphone PC Tablet Smartphone Smartphone Smartphone Long Short Long Short Long Short Long Short Long Short Long Short Long Short Long Short Standard & Optimized: Which have you done in the past month?
  15. 15. TNS Mobile RoR © TNS May, 2014 PC Tablet Smart Phone PC Tablet Smart Phone 12 0 10 20 30 40 50 60 70 80 90 100 % Panel A Panel B River Router Category Ownership – Cat Standard Optimized Lumi – LSR Lumi – GMI PC Tablet Smartphone PC Tablet Smartphone Smartphone Smartphone Long Short Long Short Long Short Long Short Long Short Long Short Long Short Long Short Standard & Optimized: Which items do you currently own?
  16. 16. TNS Mobile RoR © TNS May, 2014 PC Tablet Smart Phone PC Tablet Smart Phone Standard Optimized Lumi – LSR Lumi – GMI GCS PC Tablet Smartphone PC Tablet Smartphone Smartphone Smartphone PC Tablet Smart phone Long Short Long Short Long Short Long Short Long Short Long Short Long Short Long Short Short Short Short 13 0 10 20 30 40 50 60 70 80 90 100 % Panel A Panel B River Router GCS Brand of Mobile Phone Own - Blackberry Standard: Which brand(s) of mobile phones do you personally use? Optimized: Which brand(s) of mobile phones do you use?
  17. 17. TNS Mobile RoR © TNS May, 2014 PC Tablet Smart Phone PC Tablet Smart Phone 14 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 Mean Panel A Panel B River Router GCS Concept Test Purchase Intent Standard Optimized Lumi – LSR Lumi – GMI GCS PC Tablet Smartphone PC Tablet Smartphone Smartphone Smartphone PC Tablet Smart phone Long Short Long Short Long Short Long Short Long Short Long Short Long Short Long Short Short Short Short Standard: Which statement best describes how likely you would be to buy this product if it were available in stores where you shop? Optimized: How likely would you be to buy this product?
  18. 18. TNS Mobile RoR © TNS May, 2014 PC Tablet Smart Phone PC Tablet Smart Phone 15 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 Mean Panel A Panel B River Router Past Week Consumption – Coca-Cola Standard : How many times have you consumed each of these carbonated soft drinks in the past week? Optimized: In the past week, how many times have you consumed each…? Standard Optimized Lumi – LSR Lumi – GMI PC Tablet Smartphone PC Tablet Smartphone Smartphone Smartphone Long Short Long Short Long Short Long Short Long Short Long Short Long Short Long Short TimesPerweek
  19. 19. TNS Mobile RoR © TNS May, 2014 PC Tablet Smart Phone PC Tablet Smart Phone 16 0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0 8.0 9.0 Mean Panel A Panel B River Router Past Week Consumption – Pepsi Standard : How many times have you consumed each of these carbonated soft drinks in the past week? Optimized: In the past week, how many times have you consumed each…? Standard Optimized Lumi – LSR Lumi – GMI PC Tablet Smartphone PC Tablet Smartphone Smartphone Smartphone Long Short Long Short Long Short Long Short Long Short Long Short Long Short Long Short
  20. 20. TNS Mobile RoR © TNS May, 2014 PC Tablet Smart Phone PC Tablet Smart Phone 17 0.0 2.0 4.0 6.0 8.0 10.0 12.0 14.0 16.0 18.0 20.0 Minutes Panel A Panel B River Router #REF! #REF! Interview Length Standard Optimized Lumi – LSR Lumi – GMI GCS PC Tablet Smartphone PC Tablet Smartphone Smartphone Smartphone PC Tablet Smart phone Long Short Long Short Long Short Long Short Long Short Long Short Long Short Long Short Short Short Short Average Length Short: 5 min Long: 12 min
  21. 21. TNS Mobile RoR © TNS May, 2014 PC Tablet Smart Phone PC Tablet Smart Phone 18 0.0 10.0 20.0 30.0 % Panel A Panel B River Router GCS Term Rates Standard Optimized Lumi – LSR Lumi – GMI GCS PC Tablet Smartphone PC Tablet Smartphone Smartphone Smartphone PC Tablet Smart phone Long Short Long Short Long Short Long Short Long Short Long Short Long Short Long Short Short Short Short Average Length Short: 5 min Long: 12 min
  22. 22. © TNS © TNS May, 2014 Implications 19 • River sampling requires some caution - We need to investigate which sources of River Sampling produce the most reliable results • Sample blending needs to be done with care - Panel A and Panel B are similar but we need to ensure consistent levels of blending if we are comparing results (waves) over time • Device agnostic surveys appear to be very feasible - Great news – helps response rate - Comparing results from mobile surveys to databases is possible • Keep Mobile questionnaires short – under 6 minutes - We should strive to reduce questionnaire length regardless of data collection method as it impacts completion rates • Optimize your questionnaires - It’s good research practice and it reduces early termination
  23. 23. Title Sponsor Gold Sponsors Silver Sponsors MRMW Chicago May 27-30, 2014 #MRMW
  24. 24. Workshop Sponsors Association & Education Partners Networking Reception Sponsor Event App Partner Media Partners MRMW Chicago May 27-30, 2014 #MRMW
  25. 25. May 27-30, 2014 Congress Plaza Hotel & Convention Center #MRMW Organized by

×