Making it fit: How survey technology proviers are responding to the challenges of handling web surveys on mobile devices

  • 814 views
Uploaded on

Presentation at the ASC International conference, Bristol, 2011. A full written paper is also available from the meaning website

Presentation at the ASC International conference, Bristol, 2011. A full written paper is also available from the meaning website

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
814
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
10
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Making it fitHow survey technology providers are respondingto the challenges of handling web surveys onmobile devicesTim Macer, meaning ltd, London, UKASC Sixth International Conference, Bristol, 22-23 September 2011
  • 2. Agenda1. Emerging best practices2. Requirements identified3. How are the vendors responding4. Conclusions and recommendations 2
  • 3. Making it fit: web surveys on mobile devices. ASC 20111. Emerging bestpracticesSpecific considerationsOperating systemsApp or browser? 3
  • 4. Making it fit: web surveys on mobile devices. ASC 2011Multi-mode parallel surveys on web/mobile Design the survey around single-choice, multiple-choice and open-ended questions alone. Adopt more concise wording Limit the survey length to fifteen questions. Zahariev et al (2010) 4
  • 5. Making it fit: web surveys on mobile devices. ASC 2011Specific technical requirements identified… Identifying which specific smartphones are supported, as only specific makes or models may be supported in some packages. Verifying that error messages and other components are not dependent on the use of JavaScript. Having a specific „skin‟ or page template to format the survey for delivery on a mobile device, which maximised the limited space available for questions. Zahariev et al (2010) 5
  • 6. Making it fit: web surveys on mobile devices. ASC 2011Recommended optimiziations for mobile surveys Use a concise, well-worded SMS invitation with an integrated link to survey. To ensure any costs a participant may incur are reimbursed Avoid complex matrix questions: replace with questions presented one by one. Encourage active engagement, e.g. taking photographs. Pick time to send invites in order to optimize response (e.g. between 17.00 and 18.00). Pferdekaemper (2010) 6
  • 7. Making it fit: web surveys on mobile devices. ASC 2011Further recommendations for mobile Reduce and simplify –  avoid horizontal scrolling,  reduce clutter from logos, branding,  consider using drop-down lists rather than radio buttons Luck (2011) “How to make mobile surveys work for everyone” Adjust the survey to fit limited the screen of each device  Understand the dimension of the device  Capture the browser agent string Callegaro (2010) “Do You Know Which Device Your Respondent Has Used to Take Your Online Survey?” 7
  • 8. Making it fit: web surveys on mobile devices. ASC 2011Current trends in Mobile operating systems in useOS 2010 2014 trendAndroid (Google) 17.7 29.6 iOS (Apple) 15.4 14.9 Research In Motion (BlackBerry) 17.5 11.7 Symbian 40.1 30.2 Windows Phone 4.7 3.9 Other Operating Systems 4.7 9.6 Mobile Communications devices: sales to end users by OSGartner (2010) 8
  • 9. Making it fit: web surveys on mobile devices. ASC 2011What defines an app or a browser survey?browser appA web survey which has been A survey delivered through aformatted so that it will work dedicated application or appeffectively in the browser of a which the participant must firstrange of mobile devices within download onto his/her devicethe constraints of these and then accesses via an icondevices. on the device‟s main menu. 9
  • 10. Making it fit: web surveys on mobile devices. ASC 2011Browser pros and cons UsesHTML: achieve high ✘ Lowest common denominator coverage; not restricted to a approach to question types subset of devices ✘ Not all devices support flash Participant can engage ✘ Integrating media, e.g. immediately without having to uploading photos, can be too download anything difficult for participants to do Mix formats within one survey ✘ Needs a stable internet instrument: laptop/desktop, connection each time “next” is tablets, handhelds pressed 10
  • 11. Making it fit: web surveys on mobile devices. ASC 2011App pros and cons Does not need a permanent ✘ Download requires high level stable internet connection at of trust and engagement from all times participant Can exploit all the ✘ Apps rarely support the entire programmable capabilities of range of devices out there – the device, e.g. diaries with reach less of the population reminders, integrated capture and upload of pictures and video 11
  • 12. Making it fit: web surveys on mobile devices. ASC 20112. Identifying the keyrequirementsEssential needs for browser toolsEssential needs for appsThe survey 12
  • 13. Making it fit: web surveys on mobile devices. ASC 2011Essential requirements examined: browser-basedtools1. Automatically detect the device 6. Allow respondent to save and resolution and reformat survey resume later in „desktop‟ or to fit mobile mode2. Optimize the presentation of the 7. No dependency on mouse survey for small-format devices controls (shift+click etc)3. Gracefully unwind grids into a 8. No dependency on using Flash series of individual questions controls4. Record device type used in the 9. Can create specific routing and data validation dependent on device type5. Integrated survey invitations and reminders by email and SMS 13
  • 14. Making it fit: web surveys on mobile devices. ASC 2011Essential requirements: app-based tools1. Simple, easy download for 5. Diary surveys: repeated respondent capture of same questions2. Web server-based 6. Diary reminder alerts with authoring and deployment do not disturb option environment 7. Online/offline capability3. Capture location data, with 8. Bar code scanning with respondent permission respondent confirmation4. Capture and upload photos and video 14
  • 15. Making it fit: web surveys on mobile devices. ASC 2011…plus, requirements common to both modes10. Wide range of devices supported11. Device-specific preview in authoring tool or via simulators 15
  • 16. Making it fit: web surveys on mobile devices. ASC 2011The survey Contacted 10 different Contacts made in technology producers April 2011 with a simple active in the field electronic questionnaire 7 firms Eight companieswith browser-based solutions participated 6 firms with apps One refused 5 with both One failed to respond
  • 17. Making it fit: web surveys on mobile devices. ASC 20113. How are the vendorsrespondingPlatforms supportedFeatures provided 17
  • 18. Making it fit: web surveys on mobile devices. ASC 2011Who is providing what?Supplier Browser AppAskia ✓ ✓CfMC WebSurvent ✓Confirmit ✓Globalpark EFS ✓ ✓iPinion Surveys ✓ ✓Kinesis ✓ ✓Surveyswipe & Surveypocket ✓ ✓Techneos ✓ 18
  • 19. Making it fit: web surveys on mobile devices. ASC 2011Browser tools – devices supported Survey- Con- GlobalDevice iPinion Kinesis CfMC Askia swipe firmit -parkAndroid ✓ ✓ ✓ ✓ ✓ ✓ ✓BlackBerry ✓ ✓ ✓ ✓ ✓ ✓ ✓Symbian ✓ ✓ ✓ ✓ ✓iOS (Apple) ✓ ✓ ✓ ✓ ✓ ✓ ✓Windows mobile ✓ ✓ ✓ ✓ ✓ ✓ ✓WML  ✓ ✓XHTML  ✓ ✓ ✓ ✓ ✓ ✓ Capability currently offered  Planned for a Future release 19
  • 20. Making it fit: web surveys on mobile devices. ASC 2011Mobile Apps – devices supported Survey- Techneos Askia Global- iPinion Kinesis pocket SODA mobile parkAndroid ✓  ✓ ✓ ✓ BlackBerry ✓  ✓ ✓ Symbian  ✓ iOS (Apple) ✓ ✓ ✓ ✓ ✓ ✓Windowsmobile   ✓ ✓ ✓ Capability currently offered  Planned for a Future release 20
  • 21. Making it fit: web surveys on mobile devices. ASC 2011Web browser solutionsAnalyzing the capabilities offered inthe market 21
  • 22. Making it fit: web surveys on mobile devices. ASC 2011Browser tools – capabilities offered by all vendorsCapabilityMixed mode – full screen and mobile ✓Detect and reformat to fit device ✓Optimize for mobile ✓No dependency on mouse ✓No dependency on Flash ✓ 22
  • 23. Making it fit: web surveys on mobile devices. ASC 2011Browser tools – capabilities where support varies Survey- Con- Global-Capability iPinion Kinesis CfMC Askia swipe firmit parkUnwind grids ✓ ✓  ✓ ✓ ✓ ✓Store device data• At survey level ✓ ✓ ✓ ✓ ✓ ✓• At question level ✓ ✓ ✓ ✓Invitations, reminders byEmail and SMS ✓ ✓  ✓ ✓ ✓ ✓Preview via simulator  ✓   ✓Save and resume ✓  ✓ ✓ ✓ ✓Routing by device  ✓ ✓ ✓ ✓ 23
  • 24. Making it fit: web surveys on mobile devices. ASC 2011Essential requirements fulfilled: browser tools All offered device auto-detection and optimization for mobile, plus unwinding of grids Only 2/7 products met all the key capabilities 6/7 supported email and SMS invitations/reminders 5/7 supported save and resume Capturing of the agent string  Most will record device data at a survey level;  Only 4/7 record the device at question level or allow conditional routing on the device type 24
  • 25. Making it fit: web surveys on mobile devices. ASC 2011Mobile app solutionsAnalyzing the capabilities offered inthe market 25
  • 26. Making it fit: web surveys on mobile devices. ASC 2011Mobile apps – capabilities offered by all vendorsCapabilityCapture location data, with respondent ✓permissionCapture and upload photos and video ✓Support for Android, BlackBerry and ✓Apple iOS 26
  • 27. Making it fit: web surveys on mobile devices. ASC 2011Mobile Apps – download method supported Survey- Techneos Askia Global- iPinion Kinesis pocket SODA mobile parkSimple 1-click from a ✓ ✓ ✓ ✓ ✓web linkFrom an integrated ✓ ✓ ✓  ✓ ✓panel portalFrom app site or ✓ ✓ ✓ ✓ ✓App Store 27
  • 28. Making it fit: web surveys on mobile devices. ASC 2011Mobile Apps – capabilities where support varies Survey- Techneos Askia Global- iPinion Kinesis pocket SODA mobile parkWeb browser authoringand deployment 1 ✓ ✓ ✓  ✓Device-specific preview   ✓2 ✓Diary survey mode ✓ ✓  ✓  ✓Diary alerts (regulated)  ✓ ✓  ✓Online/offline ✓ ✓ ✓ ✓ QR/Bar code scanning  ✓  ✓  1Authoring via either Confirmit or iBM SPSS Dimensions design module 28 2via 3rd party simulator
  • 29. Making it fit: web surveys on mobile devices. ASC 2011Essential requirements fulfilled: Apps 6/6 will capture location data, photos or video, all under respondent control 6/6 offered a range of one-click download options 5/6 would operate without a constant internet connection 4/6 offer diary support and the other 2 are developing it  Diary reminders are only fully supported (with the option for participants to disable them) by 4 of the 5. QR/bar codes supported by 4 but with 2 more developing 29
  • 30. Making it fit: web surveys on mobile devices. ASC 2011DiscussionConclusionsRecommendations 30
  • 31. Making it fit: web surveys on mobile devices. ASC 2011Observations Some vendors have specialized in serving the browser or the app-based solution, but many cater for both All the browser solutions we examined will integrate seamlessly with standard online surveys (auto-detecting the device) Most but not all vendors support a wide range of devices  iOS, Android and Blackberry universally supported  Symbian only offered by 5/7 browser and 5/6 app products  XHTML supported by just 5/7 browser tools and the old (and very restrictive) WML/WAP by only 2. 31
  • 32. Making it fit: web surveys on mobile devices. ASC 2011Observations Models of best practice are still developing Previewing and simulating how the survey will appear on different small-format devices is really important  Only 2/7 browser tools and 2/6 apps did this,  However, two more in each category plan to support this soon Software developers are not necessarily taking advantage of the capabilities that mobile devices offer 32
  • 33. Making it fit: web surveys on mobile devices. ASC 2011Conclusions 1 of 2Mobile research is still at an Support still rudimentaryearly stage of development Only two of the solutions met all of the key capabilitiesWide variation in what is But developing fastoffered by different vendors Most of the vendors platforms supported, key recorded key capabilities features etc that they were planning to introduce soon
  • 34. Making it fit: web surveys on mobile devices. ASC 2011Conclusions 2 of 2Browser-based tools may All browser-based solutionsoffer better coverage but lack examined are competent atthe more advanced mobile- rending surveys on full-specific capabilities screen and mobile devicesDevelopers seem not to be The more advanced andaware of the importance of distinctive features of Mobiletesting for survey designers Research are not well supported in mobile apps yet
  • 35. Making it fit: web surveys on mobile devices. ASC 2011Further work Future work needs to focus on the effect of mobile devices on conventional online research Survey takers increasingly using mobile devices to take online  Typically around 4% and growing  For them, the survey experience frequently degenerates into chaos Web survey providers in general not addressing mobile surveys – desktop/laptop participation is assumed 35
  • 36. Questions and comments welcomeTim Macer tim@meaning.uk.com 36