Voice recognition in self-administered mobile surveys' - Free University of Bozen-Bolzano and Globalpark (Mobile Research Conference 2011)
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Voice recognition in self-administered mobile surveys' - Free University of Bozen-Bolzano and Globalpark (Mobile Research Conference 2011)

on

  • 1,322 views

Text versus voice? A number of approaches have been suggested to optimise mobile survey response rates. Whilst addressing survey design can help for closed questions, what about open-ended ones? The ...

Text versus voice? A number of approaches have been suggested to optimise mobile survey response rates. Whilst addressing survey design can help for closed questions, what about open-ended ones? The presenters will share the results of an experimental study comparing classical text-input formats in self-administered mobile surveys with an alternative, voice-recognition-based procedure in mobile survey projects. They will report on the relative differences of these two formats on usability and non-response levels, and will provide practical recommendations for mobile survey projects.

Statistics

Views

Total Views
1,322
Views on SlideShare
1,258
Embed Views
64

Actions

Likes
2
Downloads
41
Comments
0

4 Embeds 64

http://michlegger.wikispaces.com 38
http://www.questback.de 16
http://www.globalpark.de 6
http://blog.questback.de 4

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Voice recognition in self-administered mobile surveys' - Free University of Bozen-Bolzano and Globalpark (Mobile Research Conference 2011) Presentation Transcript

  • 1. VOICE RECOGNITION IN SELF- ADMINISTERED MOBILE SURVEYS LORENZ GRÄF, SVEN SCHERRER, MARC HAUPT, MICHAEL BOSNJAK, MARIO CALLEGARO April 19, 2011 Mobile Study III1 Globalpark. Manage what matters.
  • 2. AGENDA 1.Background 2.Research Questions 3.Method 4.Results 5.Discussion and Outlook April 19, 2011 Mobile Study III2 Globalpark. Manage what matters.
  • 3. AGENDA 1.Background 2.Research Questions 3.Method 4.Results 5.Discussion and Outlook April 19, 2011 Mobile Study III3 Globalpark. Manage what matters.
  • 4. USABILITY OF MOBILE SURVEYS: PAST FINDINGS Subjective Usability Assessment Observed Post-hoc survey (Web) one week after mobile survey completion Item- Drop- Indicators for usability score: fluency, simplicity, ease of use NR Out Single choice Einfachauswahl untereinander 89,2 Multiple choice Mehrfachauswahl untereinander 87,3 9%Fragetyp Drop-Down menu Geschlossene Auswahlliste 82,7 Textfield Textfeld einzeilig 74,7 45% 9% Image map Fragetyp mit Bild 87,9 23% 65,00 73,75 82,50 91,25 100,00 Usability score (Range: 0-100 Punkte) April 19, 2011 Mobile Study III 4 Globalpark. Manage what matters.
  • 5. AGENDA 1.Background 2.Research Questions 3.Method 4.Results 5.Discussion and Outlook April 19, 2011 Mobile Study III5 Globalpark. Manage what matters.
  • 6. OVERALL GOAL Improving mobile input formats requiring rich, non- standardized information, which is usually captured using text input in other self-administered survey modes. Alternative to (typed) text-input formats: Embedding voice input into self-administered mobile surveys, similar to CATI or esp. IVR surveys Today: Subjective usability assessments as indicators of a potential improvement text versus voice Next step: Analyzing objective / unobtrusive data on nonresponse (item-nonresponse, drop-out) April 19, 2011 Mobile Study III6 Globalpark. Manage what matters.
  • 7. TECHNICAL IMPLEMENTATION: IPHONE APP April 19, 2011 Mobile Study III7 Globalpark. Manage what matters.
  • 8. TECHNICAL IMPLEMENTATION: ANDROID April 19, 2011 Mobile Study III8 Globalpark. Manage what matters.
  • 9. RESEARCH QUESTION(S) TESTED Differences between iPhone-App-based voice capturing versus Android-based voice recognition versus Classical (touchscreen-based) text input on the following dependent variables: usability assessment immediately following a specific voice/text input format, (objective indicators of nonresponse, esp. item-nonresponse and drop-out,) retrospective usability assessement Ø 5 days following participation? April 19, 2011 Mobile Study III9 Globalpark. Manage what matters.
  • 10. AGENDA 1.Background 2.Research Questions 3.Method 4.Results 5.Discussion and Outlook April 19, 2011 Mobile Study III10 Globalpark. Manage what matters.
  • 11. MINGLE PANEL – MOBILE USERS N=1051811 04/11/11 Mobile Study III Globalpark. Manage what matters.
  • 12. MINGLE PANEL – SMARTPHONE OS N=313612 04/11/11 Mobile Study III Globalpark. Manage what matters.
  • 13. PARTICIPANTS13 April 19, 2011 Mobile Study III Globalpark. Manage what matters.
  • 14. PARTICIPANTS 6,6 % Sales 14,2 % Employment Employed Status Student Unemployed 79,2 %14 April 19, 2011 Mobile Study III Globalpark. Manage what matters.
  • 15. PROCEDURE Part 1: Pre-Survey (23.03. – 28.03.2011, 6 Days, time of invitation: 5 PM GMT) • Device Screening • Android: Technical requirements check • iPhone: Instructions App installation • Pre-commitment Part 2: Main survey (28.03. – 03.04.2011, 7 Days, time of invitation: 5 PM GMT) • Test drive: Voice recording / voice recognition • Three text/voice input items (short: single number, medium: list three items, long answers: elaborated text/answer) • Usability assessment following each text/voice input item Part 3: Post survey (05.04. – 09.04.2011, 5 days, time of invitation: 8 AM GMT) • Retrospective usability assessment • Evaluation of Main-Survey Drop-Out • Preferred input method short / long entries15 April 19, 2011 Mobile Study III Globalpark. Manage what matters.
  • 16. PROCEDURE (PARTS 2 AND 3) Respondi panel members (N= 235) iPhone App Voice Android Voice Android Text Input Capturing Group Recognition Group Group (n3.1= 49) (n1.1= 126) (n2.1= 60) Immediate usability assessment for three corresponding questions. (n1.2= 87) (n2.2= 36) (n3.2= 38) Retrospective usability assessment (after Ø 5 days, Web-based). (n1.3= 79) (n2.3= 36) (n3.3= 34) April 19, 2011 Mobile Study III16 Globalpark. Manage what matters.
  • 17. DEPENDENT MEASURES Immediate usability assessment: Mean score (transformed to 0 -100 range) of three identically worded, closed- ended single items evaluating the extent to which (iPhone) voice recording / (Android VR) voice recognition / (Android text) text input was appropriate to capture the intended response („How do you evaluate the performance of the voice- recognition/-capturing /touchscreen keyboard feature?“, scaled from ´very good´ thru ´very poor´). Retrospective usability assessment (reminder screenshots were displayed): Mean score (transformed to 0 - 100 range) of the following four indicators: ease of use, fluency, usability, unintended entries . Non-response (work in progress): Item-nonresponse: Aggregated item-nonresponse percentage for all three questions with experimentally manipulated input formats (per experimental condition) Drop-Out: Overall drop-out rates (per experimental condition) April 19, 2011 Mobile Study III17 Globalpark. Manage what matters.
  • 18. AGENDA 1.Background 2.Research Questions 3.Method 4.Results 5.Discussion and Outlook April 19, 2011 Mobile Study III18 Globalpark. Manage what matters.
  • 19. USABILITY ASSESSMENT; IMMEDIATE Immediate usability assessment best and most homogeneous for Android-based text input (!) April 19, 2011 Mobile Study III19 Globalpark. Manage what matters.
  • 20. USABILITY ASSESSMENT: RETROSPECT Retrospective usability assessment best forGlobalpark´s iPhone-App- based voice capturing technology. April 19, 2011 Mobile Study III20 Globalpark. Manage what matters.
  • 21. MISC: INPUT FORMAT PREFERENCES iPhone App Android 100 % 57 % 76 % 50 % 9% 9% 33 % 13 % 0% Short Entry Long Entry Short Entry Long Entry Dont Know Touchscreen Voice Capt/Rec Physical Keyboard April 19, 2011 Mobile Study III21 Globalpark. Manage what matters.
  • 22. SUMMARY OF RESULTS Immediate usability assessment: Android touchscreen > better than > iPhone App VC > better than > Android VR Android touchscreen ratings most homogeneous (largest agreements) Retrospective usability assessment: iPhone App VC > better than > Android touchscreen > better than > Android VR Android VR shows largest dispersion among ratings (largest disagreement) Additional result: The larger the amount of text information to be entered, the more preferred are voice input formats on the expense of the touchscreen-based version, but only for iPhone participants (not for Android participants). April 19, 2011 Mobile Study III22 Globalpark. Manage what matters.
  • 23. AGENDA 1.Background 2.Research Questions 3.Method 4.Results 5.Discussion and Outlook April 19, 2011 Mobile Study III23 Globalpark. Manage what matters.
  • 24. DISCUSSION AND OUTLOOK Further analyses using data from the current study: Item-Nonresponse analysis Drop-Out analysis (partly done, see Appendix) Quality of content between groups? Technology and/or selection effects? For instance, iPhone users are more experienced with Apps and more ´attached´ to their Smartphone, ´causing´ more favourable post-hoc evaluations of App-based input formats; men versus women systematically overrepresented in certain groups, other systematic differences? Next steps: Results consistent over time and in the context of ´real life´ applications? Please feel free to use VC/VR in your own studies! April 19, 2011 Mobile Study III24 Globalpark. Manage what matters.
  • 25. THANK YOU! April 19, 2011 Mobile Study III25 Globalpark. Manage what matters.
  • 26. Appendix April 19, 2011 Mobile Study III26 Globalpark. Manage what matters.
  • 27. DROP-OUT (WORK IN PROGRESS) Voice Voice Regular Capturing Recognition Test Drive 9 (9,3%) 5 (10,4%) - Instructions Test Drive 1 (1,0%) 0 (0%) 0 (0%) Single figure 0 (0%) 0 (0%) 0 (0%) List 0 (0%) 0 (0%) 0 (0%) Free text 0 (0%) 0 (0%) 0 (0%) Drop Out on other 0 (0%) 7 (14,6%) 2 (5%) pages Drop Out Overall 10 (10,3%) 12 (25%) 2 (5%) N= 97 (100%) 48 (100%) 40 (100%) April 19, 2011 Mobile Study III27 Globalpark. Manage what matters.
  • 28. Comments and Data Samples Comments Voicecapturing „Da ich nicht sprechen konnte, konnte ich den Fragebogen nicht zu Ende durchgehen.“ „Ich war krank, böse Erkältung und konnte mich nicht motivieren, zumal mir das Sprechen superschwer fiel.“ Comments Voicerecognition „Spracherkennung ist bei meinem Handy für solche Felder nicht verfügbar, nur für die Google-Suche“ „Ich hatte nur die englische Spracheingabe. Habe im Internet nach Hilfestellungen gesucht um auf deutsche Spracheingabe umzustellen, allerdings nichts gefunden.“ „Spracheingabe funktionierte nicht richtig. Spracheingabe ging nur als englischer Text. Alles Einstellungen meines HTC waren auf Deutsch und ich habe keine Möglichkeit gefunden die Spracheingabe auf Deutsch zu berichtigen.“ Comments Touchscreen Input „Gute Erhebungstechnik, gerne wieder über smartphone...“ „Würde mich sehr freuen weitere Umfragen mit dem Handy beantworten zu dürfen.“ Data Samples Voicerecognition – Testphrase „Diese Befragung ist sehr interessant“ „Die herbe frau geküsst interessant“ “These are The 5 sense sons” “disappear.com sia into the sun” „Die befragung sea of sand“ „wiese befragung ist sehr interessant“28 04/11/11 Mobile Study III Globalpark. Manage what matters.
  • 29. Demo - Audio Samples iPhone Apple Quicktime required 0: Testtrial mingle61 (optimal) 0: Single Word mingle 99 (Optimal) mingle 166 (Background noise) mingle 173 (Additional info) 2: List mingle31 / 19 (optimal) mingle16 (Background noise) mingle8 (Real List) 3: Open answer mingle198 (Slight saxon dialect) mingle187 (Whole sentence) mingle192 (slightly annoyed)29 04/11/11 Mobile Study III Globalpark. Manage what matters.