CASRO PANEL CONFERENCE 2008 Miami Maximizing respondent engagement through survey design Presented By:  Su Ning Strube, Vi...
Presentation Outline <ul><li>User Interface Design </li></ul><ul><li>Hypothesis </li></ul><ul><li>Methodology </li></ul><u...
Survey Terminology <ul><li>Flat </li></ul><ul><ul><li>Traditional online survey approach with check boxes and radio button...
Hypothesis <ul><li>Fusion relative to flat: </li></ul><ul><ul><li>Higher respondent engagement </li></ul></ul><ul><ul><li>...
Methodology and Sample Design <ul><li>Online data collection </li></ul><ul><li>Sample sourced from Angus Reid Forum  </li>...
Design and Presentation of Controls
Combined Questions
Follow-on and Pop-up Questions
Scale/Sort Questions
Respondent Engagement Fusion is clearly more fun to complete, enjoyable and engaging.
Future Participation Fusion is a more powerful vehicle for promoting participation in future research.
Perceived Length Fusion and flat were perceived as equal in length.  In reality, Fusion took longer to complete.
Response and Completion Rates Completion rates high for both surveys but higher for flat.
Impact on Survey Data  <ul><li>No difference between Fusion and Flat for the majority of questions: </li></ul><ul><ul><li>...
Impact on Survey Data :: Open Ends Fusion encourages more detail.
Open Ended Questions
Impact of Survey Data :: Knowledge Questions with “Don’t Know” Fusion results in more “don’t know” responses.
Knowledge Questions with “Don’t Knows”
Impact of Survey Design :: Scale/Sort Exercise Greater scale use with Fusion, towards stronger disagreement
Impact of Survey Data: Scale/Sort Exercise
Conclusions <ul><li>Fusion relative to flat: </li></ul><ul><ul><li>Higher respondent engagement </li></ul></ul><ul><ul><li...
Conclusions <ul><li>Promise for Fusion bodes well for the future of panels </li></ul><ul><ul><li>Helps to keep panelists i...
Q&A
Upcoming SlideShare
Loading in …5
×

Maximizing Respondent Engagement Through Survey Design

2,324 views

Published on

Explore the next frontier of online surveys using visual and interactive design techniques. Learn how advanced interactive visual questions can:

1) Maximize respondent engagement,
2) Encourage more thorough, detailed responses to questions, and
3) Generate higher panel retention rates.

Presenters:
Su Ning Strube - Vision Critical
Yola Zdanowicz - Vision Critical

Published in: Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,324
On SlideShare
0
From Embeds
0
Number of Embeds
66
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Maximizing Respondent Engagement Through Survey Design

  1. 1. CASRO PANEL CONFERENCE 2008 Miami Maximizing respondent engagement through survey design Presented By: Su Ning Strube, Vision Critical Yola Zdanowicz, Angus Reid Strategies
  2. 2. Presentation Outline <ul><li>User Interface Design </li></ul><ul><li>Hypothesis </li></ul><ul><li>Methodology </li></ul><ul><li>Respondent Engagement </li></ul><ul><li>Impact on Survey Data </li></ul><ul><li>Conclusions </li></ul>
  3. 3. Survey Terminology <ul><li>Flat </li></ul><ul><ul><li>Traditional online survey approach with check boxes and radio buttons from the host operating system </li></ul></ul><ul><ul><li>Familiar “flash card” navigational metaphor </li></ul></ul><ul><li>Fusion </li></ul><ul><ul><li>Advanced design utilizing rich media techniques </li></ul></ul><ul><ul><li>Developed in Adobe Flash </li></ul></ul>
  4. 4. Hypothesis <ul><li>Fusion relative to flat: </li></ul><ul><ul><li>Higher respondent engagement </li></ul></ul><ul><ul><li>Survey perceived as shorter </li></ul></ul><ul><ul><li>Higher completion rates </li></ul></ul>
  5. 5. Methodology and Sample Design <ul><li>Online data collection </li></ul><ul><li>Sample sourced from Angus Reid Forum </li></ul><ul><li>Respondent qualification </li></ul><ul><ul><li>Flash Player version 8 </li></ul></ul><ul><li>Split sample design (1246 Fusion/1227 flat) </li></ul><ul><li>Two languages (English/French) </li></ul><ul><li>Data weighted to Canadian population demographics </li></ul>
  6. 6. Design and Presentation of Controls
  7. 7. Combined Questions
  8. 8. Follow-on and Pop-up Questions
  9. 9. Scale/Sort Questions
  10. 10. Respondent Engagement Fusion is clearly more fun to complete, enjoyable and engaging.
  11. 11. Future Participation Fusion is a more powerful vehicle for promoting participation in future research.
  12. 12. Perceived Length Fusion and flat were perceived as equal in length. In reality, Fusion took longer to complete.
  13. 13. Response and Completion Rates Completion rates high for both surveys but higher for flat.
  14. 14. Impact on Survey Data <ul><li>No difference between Fusion and Flat for the majority of questions: </li></ul><ul><ul><li>Food consumption (behavior) </li></ul></ul><ul><ul><li>Purchasing of food (behavior and knowledge) </li></ul></ul><ul><ul><li>Demographics </li></ul></ul><ul><ul><li>Computer technical profile </li></ul></ul><ul><li>No difference between Fusion and Flat across different types of questions: </li></ul><ul><ul><li>Combined questions </li></ul></ul><ul><ul><li>Follow-on & Pop-Up Questions </li></ul></ul><ul><li>Exceptions were open-ended questions, knowledge questions with “don’t know” option, scale/sort exercise. </li></ul>
  15. 15. Impact on Survey Data :: Open Ends Fusion encourages more detail.
  16. 16. Open Ended Questions
  17. 17. Impact of Survey Data :: Knowledge Questions with “Don’t Know” Fusion results in more “don’t know” responses.
  18. 18. Knowledge Questions with “Don’t Knows”
  19. 19. Impact of Survey Design :: Scale/Sort Exercise Greater scale use with Fusion, towards stronger disagreement
  20. 20. Impact of Survey Data: Scale/Sort Exercise
  21. 21. Conclusions <ul><li>Fusion relative to flat: </li></ul><ul><ul><li>Higher respondent engagement </li></ul></ul><ul><ul><li>Perceived as equal in length – in reality Fusion takes longer </li></ul></ul><ul><ul><li>High completion rates for both but lower for Fusion </li></ul></ul><ul><ul><li>Greater detail for open ended questions </li></ul></ul><ul><ul><li>Greater use of “don’t know” for select knowledge questions </li></ul></ul><ul><ul><li>Greater scale use – particularly toward stronger disagreement </li></ul></ul>
  22. 22. Conclusions <ul><li>Promise for Fusion bodes well for the future of panels </li></ul><ul><ul><li>Helps to keep panelists interested and engaged </li></ul></ul><ul><ul><li>Could improve retention rates </li></ul></ul><ul><ul><li>Encourages panelists to provide thoughtful, detailed responses to research questions </li></ul></ul><ul><ul><li>Challenges the mantra of “shorter is better” </li></ul></ul><ul><li>More research is needed (i.e. usability testing) to expand our knowledge of: </li></ul><ul><ul><li>Respondent engagement </li></ul></ul><ul><ul><li>Strengths & weakness of specific question design </li></ul></ul><ul><ul><li>Overall completion rates </li></ul></ul>
  23. 23. Q&A

×