Mobile Research Conference 2010 Globalpark Surveys

1,245 views

Published on

AGENDA
Usability and (non)participation in self-administered mobile surveys
1. Research questions
2. Method
3. Findings
4. Summary
5. Outlook: Practical Implications
and future research questions

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,245
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
14
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Mobile Research Conference 2010 Globalpark Surveys

  1. 1. Usability and (non)participation in self-administered mobile surveys TANJA PFERDEKÄMPER. MICHAEL BOSNJAK MRC 2010 // London (UK) // March 8. 2010
  2. 2. AGENDA 2 Usability and (non)participation in self-administered mobile surveys 1. Research questions 2. Method 3. Findings 4. Summary 5. Outlook: Practical Implications and future research questions Prof. Dr. Michael Bosnjak http://www.bosnjak.eu
  3. 3. RESEARCH QUESTIONS 3 Replication: Stability of findings presented at MRC 2009? Usability of different mobile question types? Reaction times following an SMS-invitation? Newly introduced sub-questions: Usability-related, focusing on touchscreen-based devices (e.g., iPhone and Android (Non)participation-related: Acceptance of Geo-Location? Reaction times across a working day? Acceptable duration of mobile surveys? Acceptance of Mixed-Mode-Surveys?
  4. 4. METHOD – SUBJECTS & PROCEDURE 4 Subjects 3270 Panelists (Respondi) Response Rate: 540 Participants (16.5%) Agegroups Demographic Characteristics: 30,0 24,4% 58% male. 42% female 25,0 23,3% majority in the age of 25-34 years and 20,0 15,7% Percentage with higher level of education 15,0 14,3% 11,5% 10,0 6,3% 4,3% 5,0 0,2% 0,0 14-19 20-24 25-29 30-34 35-39 40-44 45-49 50-54 Procedure Study made up of three parts during 29.09. - 18.10.2009
  5. 5. METHOD - INSTRUMENTS 5 INSTRUMENTS & EXPERIMENTAL DESIGN Part 1: Mobile Survey “post-visit evaluation of last vacation” Invitation via SMS in 13 randomly selected waves Identification of iPhone and non iPhone users. Identification of Geo-Location if accepted Mobile Questionnaire with six different question types Part 2: Mixed Mode Survey Asking whether participants like to continue mobile or online (desktop) and sending Email-Invitation to participants prefering desktop version Part 3: Online Post Survey Email Invitation of starting sample for Online Post Survey Web-based (desktop) Usability Questionnaire
  6. 6. FINDINGS 1 – REPLICATION USABILITY 6 MOBILE QUESTION TYPES USED Single response vertical Multi response vertical Closed response list
  7. 7. FINDINGS 1 – REPLICATION USABILITY 7 MOBILE QUESTION TYPES USED Single line text input Matrix Question Semantic Differential
  8. 8. FINDINGS 1 – REPLICATION USABILITY 8 KEY RESULTS QUESTION TYPES All question types are positively evaluated. Results from 2008 could be validated (including evaluation tendencies text input versus single response) Single Multiple Closed Single line Matrix Semantic response response response list text input question differential Scrolling 5.16 4.27 5.10 5.23 4.30 3.76 Ease of answering 5.90 5.73 5.66 5.55 5.65 4.93 Unintentional input 5.97 5.81 5.87 5.82 5.79 5.22 Fluent answering 5.78 5.61 5.58 5.57 5.60 4.87 Ease of use 5.76 5.63 5.62 5.53 5.63 4.84 Mean-Values on 7 point scale from [1] 'Extrem often' / ' Totally disagree' to [7] 'Never' / Totally agree' Participants (min. number of answers): n= 318
  9. 9. FINDINGS 2 – REPLICATION REACTION TIME 9 KEY RESULTS Response Time in hours 100% 90% 2009: 35% of 80% responses within first hour after invitation Accumulated Ppercentages 70% 60% 50% 2008: 40% within first 40% hour when invitations 30% were send out at 5 pm 20% 10% 0% 0 1 2 3 4 5 6 7 8 9 10 11 12 hours since invitation
  10. 10. FINDINGS 3 – NEW: iPhone and Geo-Tracking 10 KEY RESULTS iPhone: Best usability (objective and subjective measured) results for mobile survey on iPhone Geo-Location: Acceptance of GPS-Location among participants with iPhone (n=45) yes 91% no 9%
  11. 11. FINDINGS 3 – NEW: Duration & Mixed Mode 11 KEY RESULTS – Preference online (desktop) or mobile „Do you want to continue answering the survey mobile or online (in this case you will get a link via email)?“ 100% 10,3% 90% Participants: n= 540 80% 70% 68,9% 60% Online 50% mobile 89,7% 85% 40% online 30% 20% 31,1% Mobile 10% 15% 0% non iPhone iPhone
  12. 12. FINDINGS 3 – NEW: Duration & Mixed Mode 12 KEY RESULTS – Duration of mobile survey 25 19,8 20,1 19,3 20 15 Part 1: Mobile- Initial Minnutes Survey Part 1 & 2: Mobile 10 Survey 5,2 5,1 5,2 5 0 Total iPhone Non iPhone N= 540 / 82 45 / 31 495/ 51
  13. 13. SUMMARY 13 TAKE HOME MESSAGES Replication: Stability of findings presented at MRC 2009? Robustness of good usability results over time and survey topics Fast Response times independent from particular time of SMS-invitation Newly introduced sub-questions: Usability-related: iPhone user evaluate mobile survey usability better than non iPhone user (Non)participation-related: High Acceptance of Geo-Location Fast Reaction times Acceptance of 5-7 minutes length. iPhone: more than15 minutes All in all participants prefer online surveys after 5 minutes mobile survey. among iPhone users clear preference of the mobile mode
  14. 14. OUTLOOK 14 MANAGERIAL IMPLICATIONS & FUTURE RESEARCH QUESTIONS Managerial Implications Fast Results / Quick Polls Combination Mobile (actual emotions) plus later Desktop (cognition) Growing penetration of smartphones promote mobile surveys New avenues for event evaluation. customer satisfaction survey. ad testing Future Research Usability / Measurement and (non)participation Influence of participation context on mobile survey error dimensions largely unexplored Technical changes in how we interact with mobile devices, usage patterns and usage contexts evolve
  15. 15. THANK YOU!
  16. 16. ABOUT US Globalpark provides panel. community and survey software that enables organizations to manage what matters across the enterprise. By capturing feedback and tracking behavior of customers. employees and partners. they gain insights that drive better business decisions. By identifying and empowering influential advocates. they build reputation and extend reach. Founded in 1999. Globalpark software is German-engineered and globally-tested by leading brands and top market research institutes. including: Continental. Daimler. General Mills. GfK. IDC. Nintendo. SonyBMG. TNS. Warner Music and Wrigley. Globalpark is staffed by renowned research pioneers. with offices across the US. UK. Germany and Austria. Globalpark AG Globalpark Inc. LOCATIONS Kalscheurener Str. 19a 299 Broadway. 19th Floor 50354 Cologne/Huerth New York. NY 10007 Germany (Headquarters) United States of America Tel.: +49 2233 7933 6 Tel.: +1 888 299 9422 info@globalpark.de info@globalpark.com www.globalpark.de www.globalpark.com twitter/Globalpark twitter/Globalpark_News Globalpark UK Ltd. Globalpark Oesterreich GmbH 5 Archie Street Wassergasse 25 London SE1 3JT 1030 Vienna Great Britain Austria Tel.: +44 207 403 3900 Tel.: +43 1715 0289 11 info@globalpark.co.uk info@globalpark.at www.globalpark.co.uk www.globalpark.at
  17. 17. © 2010 – Globalpark AG. Globalpark UK Ltd.. Globalpark Inc.. Globalpark Oesterreich GmbH The information contained in this publication is the property of Globalpark AG and its subsidiaries. Passing on and reproduction of this publication or any parts of it is only permitted with the express written approval of Globalpark AG or its subsidiaries. This publication is issued without any guarantee whatsoever. Information contained herein can be changed without prior notification. In addition Globalpark accepts no guarantee as to the exactness or completeness of the information. text. graphics. links and other items in this publication. This publication is a preliminary version. which is not subject to any valid licence agreement or other agreements with Globalpark. The software products offered by Globalpark AG. its subsidiaries and/or its distributors may contain software components of other software manufacturers. All products may show country-specific differences. Globalpark. Enterprise Feedback Suite. EFS. EFS Survey. EFS Panel. EFS Employee and EFS Leadership. other Globalpark products and services mentioned in the text and their corresponding logos are trademarks and can be registered trademarks of Globalpark AG and/or its subsidiaries in Germany and other countries worldwide. All other names of products and services are trademarks of their respective companies.

×