JULY 16 - 18, 2013, Minneapolis, USA
WWW.MRMW.NET
The original, premier event
for the Mobile Marketing
Research Industry
WWW.MRMW.NET
TITLE SPONSOR DIAMOND SPONSOR PLATINUM SPONSOR
GOLD SPONSORS
WORKSHOP HOST
SILVER SPONSORS
PREMIERE SPONSOR
N...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Assembling	
  Modular	
  Mobile	
  Surveys	
...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Edward	
  Paul	
  Johnson	
  
Director	
  of...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Current	
  Environment	
  
Researcher	
  to	...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Problem	
  
New	
  Mobile	
  World	
  
Consu...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Ques6ons	
  
Can	
  we	
  	
  
modularize?	
...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Key	
  Hypotheses	
  
Willingness	
  to	
  p...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Methods	
  	
  
Ques1onnaire	
  Design	
  
F...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Methods	
  
Ques1onnaire	
  Design	
  
Full	...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Methods	
  
Ques1onnaire	
  Design	
  
Full	...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Methods	
  
Ques1onnaire	
  Design	
  
Respo...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Need	
  to	
  Assemble	
  the	
  Pieces	
  
...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Assembly	
  Method	
  
Hot	
  Decking	
  Imp...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Results	
  
H1	
  –	
  enjoyable	
  experien...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
11%	
   9%	
  
25%	
  
8%	
  
9%	
   8%	
  
...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Results	
  
H2	
  –	
  willingness	
  to	
  ...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Results	
  
H3	
  -­‐	
  minimal	
  sta1s1ca...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Results	
  
H3	
  -­‐	
  minimal	
  prac1cal...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Results	
  
H4	
  –	
  segmen1ng	
  the	
  d...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Results	
  
H4	
  -­‐	
  checking	
  segment...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Results	
  
H4	
  -­‐	
  segment	
  solu1on	...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Results	
  
H4	
  –	
  accuracy	
  of	
  seg...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Results	
  
H5	
  –	
  bivariate	
  rela1ons...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Results	
  
H6	
  -­‐	
  aStudinal	
  vs.	
 ...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Results	
  
H6:	
  aStudinal	
  vs.	
  demog...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
Conclusions	
  
Allow	
  respondents	
  to	
...
SSI	
  Confiden*al	
  
©	
  2013	
  	
  Survey	
  Sampling	
  Interna6onal	
  
cwalters@gongos.com	
  
Edward.Johnson@surve...
WWW.MRMW.NET
TITLE SPONSOR DIAMOND SPONSOR PLATINUM SPONSOR
GOLD SPONSORS
WORKSHOP HOST
SILVER SPONSORS
PREMIERE SPONSOR
N...
JULY 16 - 18, 2013, Minneapolis, USA
WWW.MRMW.NET
The original, premier event
for the Mobile Marketing
Research Industry
Upcoming SlideShare
Loading in …5
×

Imagine the Possibilities - Assembling Modular Mobile Surveys to Create Complete Datasets - SSI & Gongos

287 views

Published on

Presented by Edward Paul Johnson, Director of Analytics ,SSI
Christi Walters, Principal, Gongos
at Market Research in the Mobile World North America
17 - 18 July 2013, Minneapolis, USA

This event is proudly organised by Merlien Institute
Check out our upcoming events by visiting http://www.mrmw.net

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
287
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Imagine the Possibilities - Assembling Modular Mobile Surveys to Create Complete Datasets - SSI & Gongos

  1. 1. JULY 16 - 18, 2013, Minneapolis, USA WWW.MRMW.NET The original, premier event for the Mobile Marketing Research Industry
  2. 2. WWW.MRMW.NET TITLE SPONSOR DIAMOND SPONSOR PLATINUM SPONSOR GOLD SPONSORS WORKSHOP HOST SILVER SPONSORS PREMIERE SPONSOR NETWORKING EVENING SPONSOR BAG SPONSORPREMIERE SPONSOR
  3. 3. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Assembling  Modular  Mobile  Surveys  to   Create  Complete  Datasets   Market  Research  in  the  Mobile  World   July  17,  2013            
  4. 4. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Edward  Paul  Johnson   Director  of  Analy6cs   SSI     Chris1  Walters   Principal   Gongos  Research       |  2  |  
  5. 5. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Current  Environment   Researcher  to  Respondent  Rela1onship   VS.   respondents  researchers   I  want  to   know  what   you  think   about  soH   drinks   Great!   Here’s  a  30-­‐ minute   survey…     Sure  I     have  some     6me  to  help     you  out     Wait,  what  does     my  hair  color  have     to  do  with     beverages?     |  3  |  
  6. 6. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Problem   New  Mobile  World   Consumers     want  control   Where.    When.    How  long?   How  can  we  give   them  what  they   want?     |  4  |  
  7. 7. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Ques6ons   Can  we     modularize?   →    Within-­‐respondent     →    Between-­‐respondent   Can  we  fuse  the  data?   |  5  |  
  8. 8. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Key  Hypotheses   Willingness  to  par1cipate  in  “long”  surveys  on  mobile  devices  will  be   increased  by  offering  incremental  incen1ves  2   There  are  minimal  data  effects  on  modularizing   surveys  on  smartphones   3   Data  fusion  techniques  allow  advanced  analysis   despite  missing  data   4   Bivariate  analysis  is  possible  with  data  fusion   techniques     5   AStude  and  behavioral  hooks  will     prove  superior  to  demographic  hooks     6   Modularizing  surveys  will  create  a  beVer,  more  enjoyable   experience  for  the  respondent   1   |  6  |  
  9. 9. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Methods     Ques1onnaire  Design   Full  Ques1onnaire   Sec1on  A   Sec1on  B   Sec1on  C       We  started  with  a     full  ques1onnaire   with  three  sec1ons   |  7  |  
  10. 10. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Methods   Ques1onnaire  Design   Full  Ques1onnaire   Hook   Ques6ons   Sec1on  C   Sec1on  B   Sec1on  A     Pieces  from     each  sec1on  were  removed  as   hook  ques1ons   |  8  |  
  11. 11. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Methods   Ques1onnaire  Design   Full  Ques1onnaire   Hook   Ques6ons     The  rest  of  the  ques1onnaire  was     split  into  three  modules   Module   Module   Module   Sec1on  C   Sec1on  B   Sec1on  A   |  9  |  
  12. 12. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Methods   Ques1onnaire  Design   Respondents  were   randomly   assigned  to  a   star1ng  module.       Respondents  always   saw  a  specific  set  of   ‘hook’  ques1ons   within  their  first   module.   Respondent   Survey   Hook  Ques1ons   1st  Module   2nd  Module   3rd  Module     Choice  to  con1nue   Choice  to  con1nue   |  10  |  
  13. 13. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Need  to  Assemble  the  Pieces   |  11  |  
  14. 14. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Assembly  Method   Hot  Decking  Imputa1on   •  Finds  similar  respondents   •  Based  on  similari6es  to  link   respondents  together   •  Drops  unlinked  data   —  Only  actual  responses   Respondent  Matching   •  Es6mates  missing  data   •  Based  on  the  similari6es  to   the  remaining  sample     •  Includes  all  respondents   —  actual  responses  and     —  es6mated  responses  
  15. 15. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Results   H1  –  enjoyable  experience   Average  Difference  in  Sa6sfac6on  Ra6ngs   Sa6sfac6on   -­‐0.06   -­‐0.03   -­‐0.02   -­‐0.14   Bad  Data  Checks     No  Straightlining   84.0%   81.7%   82.3%   87.0%   Failed  Grid  1  Instruc6ons   17.6%   13.9%   12.8%   16.9%   Failed  Grid  2  Instruc6ons   17.3%   18.1%   9.7%   12.7%   Failed  Zip  Code  Match   2.2%   1.6%   3.4%   1.4%   Control     [Online  Complete]   Control     [Mobile  Complete]   Test     [Mobile  Modular]    Test     [Online  Modular]   |  13  |  
  16. 16. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   11%   9%   25%   8%   9%   8%   4%   8%   80%   83%   71%   84%   Abandoned   Removed     Completes   Results   H2  –  willingness  to  par1cipate     Control     [Online  Complete]   Control     [Mobile  Complete]   Test     [Mobile  Modular]    Test     [Online  Modular]   |  14  |  
  17. 17. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Results   H2  –  willingness  to  add  modules   9%   19%   72%   1  Module   2  Modules   3  Modules   1  Module   2  Modules     3  Modules     |  15  |  
  18. 18. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Results   H3  -­‐  minimal  sta1s1cal  effects   Compared   to  Online   Control   Online  Modular   Sta1s1cal:    No  discernible   differences     Mobile  Modular   Sta1s1cal:   25%  of  the  ques6ons   showed  sta6s6cal   differences     |  16  |  
  19. 19. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Results   H3  -­‐  minimal  prac1cal  effects   Compared   to  Online   Control   Online  Modular   Prac1cal:   Findings  and  resul6ng   insights  are  the  same     Mobile  Modular   Prac1cal:   Resul6ng  insights  were   similar,  however,  some   differences  do  exist     |  17  |  
  20. 20. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Results   H4  –  segmen1ng  the  data   Hot  Deck  Data   Imputa1on     Data  Matching   Based  on   AStudes  &   Behaviors   Online     Control   Segmenta1on  analysis  resulted  in  a     three-­‐segment  solu1on.     |  18  |  
  21. 21. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Results   H4  -­‐  checking  segment  algorithms   When  applying  the  online  control   algorithm  to  each  data  set…     When  applying  algorithms  from  each   data  set  to  the  online  control  data…     |  19  |  
  22. 22. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Results   H4  -­‐  segment  solu1on  results   20%   26%   26%   53%   51%   50%   28%   23%   24%   Segment  Sizing  Across  Techniques     Online     Control   Mobile  Hot  Deck   Data  Imputa1on     Segment  1   Segment  2   Segment  3   Data  Matching   Based  on  AStudes   &  Behaviors   |  20  |  
  23. 23. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Results   H4  –  accuracy  of  segment  assignment   83%   86%  86%   86%   0%   10%   20%   30%   40%   50%   60%   70%   80%   90%   100%   Imputed   Matching   Missing   Full     Data  Matching  Based   on  AStudes  &   Behaviors   Mobile  Hot  Deck  Data   Imputa1on     |  21  |  
  24. 24. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Results   H5  –  bivariate  rela1onships   Hot  Deck   Imputa1on   Respondent   Matching   501   424   17%   20%   0.29   0.22   .24   0.29   .19   0.19       Control   Modular  No   Imputa1on   Sample  Size   333   416*   %  Significant  Differences   0.0%   15%   Average  Reasons  r   0.39   0.31   Average  AVribute  r   0.30   0.24   Average  Reason  x  AVribute  r   0.22   0.20   Distance   Correla1on   |  22  |  
  25. 25. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Results   H6  -­‐  aStudinal  vs.  demographic  s1tching   Online     Control   Segment  1   Segment  2   Segment  3   Data  Matching   Based  on  AStudes   &  Behaviors   20%   19%   26%   53%   55%   50%   28%   25%   24%   Data  Matching   Based  on   Demographics   |  23  |  
  26. 26. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Results   H6:  aStudinal  vs.  demographic  s1tching   92%   86%  87%   86%   0%   10%   20%   30%   40%   50%   60%   70%   80%   90%   100%   Demos     Aetudes     Missing   Full     Data  Matching  Based   on  AStudes  &   Behaviors   Data  Matching  Based   on  Demographics   |  24  |  
  27. 27. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   Conclusions   Allow  respondents  to  choose  mode   (online  vs.  mobile  vs.  mul1modal)   1   Within-­‐respondent  modulariza1on     key  to  reducing  holes  in  data   2   Advanced  analy1cs  feasible     (i.e.  segmenta1on)     3   Both  fusion  techniques  work     with  unique  advantages     4   |  25  |  
  28. 28. SSI  Confiden*al   ©  2013    Survey  Sampling  Interna6onal   cwalters@gongos.com   Edward.Johnson@surveysampling.com   |  26  |  
  29. 29. WWW.MRMW.NET TITLE SPONSOR DIAMOND SPONSOR PLATINUM SPONSOR GOLD SPONSORS WORKSHOP HOST SILVER SPONSORS PREMIERE SPONSOR NETWORKING EVENING SPONSOR BAG SPONSORPREMIERE SPONSOR
  30. 30. JULY 16 - 18, 2013, Minneapolis, USA WWW.MRMW.NET The original, premier event for the Mobile Marketing Research Industry

×