• Like
Usable Government Forms and Surveys: Best Practices for Design (from MoDevGov)
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Usable Government Forms and Surveys: Best Practices for Design (from MoDevGov)

  • 890 views
Published

talk from MoDevGov about usable forms and surveys

talk from MoDevGov about usable forms and surveys

Published in Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
890
On SlideShare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
8
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Usable Government Forms and Surveys: Best Practices for Design Jennifer Romano Bergstrom February 26, 2014 MoDevGov| Rosslyn, VA @romanocog @forsmarshgroup
  • 2. Usability vs. User Experience (UX) Whitney’s 5 Es of Usability •  •  Peter’s User Experience Honeycomb The  5  Es  to  Understanding  Users  (W.  Quesenbery):  h;p://www.wqusability.com/arCcles/geDng-­‐started.html   User  Experience  Design  (P.  Morville):  h;p://semanCcstudios.com/publicaCons/semanCcs/000029.php   2  
  • 3. What People do on the Web Krug, S. Don’t Make Me Think 3  
  • 4. UX Design Failures •  Poor planning •  “It’s all about me.” (Redish: filing cabinets) •  Human cognitive limitations •  Memory and Perception •  •  •  •  Primacy Recency Chunking Patterns 4  
  • 5. Patterns 5  
  • 6. Patterns 6  
  • 7. Patterns 7  
  • 8. Mental Models & Repeating Behavior 8  
  • 9. Mental Models & Repeating Behavior 9  
  • 10. Activity 1.  Today’s date 2.  How long did it took you to get here today? 10  
  • 11. Measuring the UX “the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.” ISO 9241-11 + emotions •  How does it work for the end user? •  What does the user expect? •  How does it make the user feel?   11  
  • 12. Where to Test LABORATORY •  Controlled environment •  All participants have the same experience •  Record and communicate from control room •  Observers watch from control room and provide additional probes (via moderator) in real time •  Incorporate physiological measures (e.g., eye tracking, EDA) REMOTE IN THE FIELD •  Participants in their natural environments (e.g., home, work) •  Participants tend to be more comfortable in their natural environments •  Use video chat (moderated sessions) or online programs (unmoderated) •  Recruit hard-to-reach populations (e.g., children, doctors) •  Conduct many sessions quickly •  Moderator travels to various locations •  Recruit participants in many locations (e.g., states, countries) •  Bring equipment (e.g., eye tracker) •  Natural observations •  No travel costs 12  
  • 13. How to Test ONE-ON-ONE SESSIONS •  In-depth feedback from each participant •  No group think •  Can allow participants to take their own route and explore freely FOCUS GROUPS SURVEYS •  Participants may be more comfortable with others •  Representative •  Interview many people quickly •  Opinions collide •  No interference •  Peer review •  Remote in participant’s environment •  Qualitative •  Large sample sizes •  Collect a lot of data quickly •  No interviewer bias •  No scheduling sessions •  Quantitative analysis •  Flexible scheduling •  Qualitative and Quantitative 13  
  • 14. How to Test ONE-ON-ONE SESSIONS •  In-depth feedback from each participant •  No group think •  Can allow participants to take their own route and explore freely FOCUS GROUPS SURVEYS •  Participants may be more comfortable with others •  Representative •  Interview many people quickly •  Opinions collide •  No interference •  Peer review •  Remote in participant’s environment •  Qualitative •  Large sample sizes •  Collect a lot of data quickly •  No interviewer bias •  No scheduling sessions •  Quantitative analysis •  Flexible scheduling •  Qualitative and Quantitative 14  
  • 15. How to Test ONE-ON-ONE SESSIONS •  In-depth feedback from each participant •  No group think •  Can allow participants to take their own route and explore freely •  No interference •  Remote in participant’s environment •  Flexible scheduling •  Qualitative and Quantitative FOCUS GROUPS SURVEYS •  Participants may be more comfortable with others •  Representative •  Large sample sizes •  Collect a lot of data quickly •  Interview many people quickly Scale Overall Experience Did not like it at all (1) – Liked it a lot (5) 3.9 •  No scheduling sessions Likelihood to Use Site in the Future Not likely at all (1) – Extremely likely (5) 3.1 •  Quantitative analysis General Organization of Website Not clear at all (1) – Extremely Clear (5) 3.6 Helpfulness of Search Functionality Not helpful at all (1) – Extremely helpful (5) 3.9 Ease of Navigation Very Easy (1) – Extremely Difficult (5) 2.0 Usefulness of tool Not useful at all (1) – Extremely useful (5) 3.7 •  Opinions collide •  Peer review •  Qualitative Mean •  No interviewer bias Question 15  
  • 16. When to Test 16  
  • 17. What to Measure EXPLICIT OBSERVATIONAL +  Post-task satisfaction questionnaires +  In-session difficulty ratings +  Verbal responses +  Moderator follow up +  Real-time +/- dial +  Ethnography +  Time to complete task +  Reaction time +  Selection/click behavior +  Ability to complete tasks +  Accuracy IMPLICIT +  Facial expression analysis +  Eye tracking +  Electrodermal activity (EDA) +  Behavioral analysis +  Linguistic analysis of verbalizations +  Implicit associations +  Pupil dilation 17  
  • 18. Why is Design Important in Web Surveys and Forms? •  No interviewer present to correct/advise •  Visual presentation affects responses •  While the Internet provides many ways to enhance surveys, design tools may be misused 18  
  • 19. Why is Design Important? •  Respondents extract meaning from how question and response options are displayed •  Design may distract from or interfere with responses •  Design may affect data quality 19  
  • 20. Why is Design Important? Note: We don’t have much confidence in the totals for ages 5-7 because it appears that some respondents chose these responses rather than scroll through the list to their correct age. http://www.cc.gatech.edu/gvu/user_surveys/ 20  
  • 21. Why is Design Important? •  Respondents are more tech savvy today and use multiple technologies •  It is not just about reducing respondent burden and nonresponse •  We must increase engagement •  High-quality design = trust in the designer Adams & Darwin, 1982; Dillman et al., 1993; Haberlein & Baumgartner, 1978 21  
  • 22. http://www.pewinternet.org/Static-Pages/TrendData-(Adults)/Device-Ownership.aspx 22  
  • 23. Considerations •  Navigation •  Edits and Input Fields •  Checkboxes and Radio Buttons •  Instructions and Help 23  
  • 24. Navigation •  In a paging survey, after entering a response –  Proceed to next page –  Return to previous page (sometimes) –  Quit or stop –  Launch separate page with Help, definitions, etc. 24
  • 25. Navigation: NP •  Next should be on the left –  Reduces the amount of time to move cursor to primary navigation button –  Frequency of use Couper, 2008; Dillman et al., 2009; Faulkner, 1998; Koyani et al., 2004; Wroblewski, 2008 25
  • 26. Navigation NP Example Peytchev    &  Peytcheva,  2011   26  
  • 27. Navigation: PN •  Previous should be on the left –  Web application order –  Everyday devices –  Logical reading order 27
  • 28. Navigation PN Example 28  
  • 29. Navigation PN Example 29  
  • 30. Navigation PN Example 30  
  • 31. Navigation PN Example 31  
  • 32. Comparing the Two Romano & Chen, 2011 •  Participants looked at Previous and Next in PN conditions •  Many participants looked at Previous in the N_P conditions –  Couper et al. (2011): Previous gets used more when it is on the right. 32
  • 33. Navigation Alternative •  Previous below Next –  Buttons can be closer –  But what about older adults? –  What about on mobile? Couper et al., 2011; Wroblewski, 2008 33
  • 34. Navigation Alternative: Large primary navigation button; secondary smaller 34
  • 35. Navigation Alternative: No back/ previous option 35
  • 36. Confusing Navigation 36
  • 37. Considerations •  Navigation •  Edits and Input Fields •  Checkboxes and Radio Buttons •  Instructions and Help 37  
  • 38. Input Fields Example 38  
  • 39. Open-Ended Responses: Narrative •  Avoid vertical scrolling when possible •  Always avoid horizontal scrolling 39
  • 40. Open-Ended Responses: Narrative •  Avoid vertical scrolling when possible •  Always avoid horizontal scrolling 32.8 characters 38.4 characters ~700 Rs Wells et al., 2012 40
  • 41. Open-Ended Responses: Numeric •  Is there a better way? 41
  • 42. Open-Ended Responses: Numeric •  Is there a better way? 42
  • 43. Open-Ended Responses: Numeric •  Use of templates reduces ill-formed responses –  E.g., $_________.00 Couper et al., 2009; Fuchs, 2007 43
  • 44. Open-Ended Responses: Date •  Not a good use: intended response will always be the same format •  Same for state, zip code, etc. •  Note –  “Month” = text –  “mm/yyyy” = #s 44
  • 45. Considerations •  Navigation •  Edits and Input Fields •  Checkboxes and Radio Buttons •  Instructions and Help 45  
  • 46. Check Boxes and Radio Buttons •  Perceived Affordances •  Design according to existing conventions and expectations •  What are the conventions? 46
  • 47. Check Boxes: Select all that apply 47
  • 48. Radio Buttons: Select only one 48
  • 49. Radio Buttons: In grids 49
  • 50. Radio Buttons on mobile •  Would something else be better? 50
  • 51. Reducing Options •  What is necessary? 51
  • 52. Reducing Options •  What is necessary? 52
  • 53. Considerations •  Navigation •  Edits and Input Fields •  Checkboxes and Radio Buttons •  Instructions and Help 53  
  • 54. Placement of Instructions •  Place them near the item •  “Don’t make me think” •  Are they necessary? 54
  • 55. Placement of Instructions •  Place them near the item •  “Don’t make me think” •  Are they necessary? 55
  • 56. Placement of Instructions •  Place them near the item •  “Don’t make me think” •  Are they necessary? 56
  • 57. Placement of Instructions •  Place them near the item •  “Don’t make me think” •  Are they necessary? 57
  • 58. Instructions 58
  • 59. Instructions 59
  • 60. Instructions 60
  • 61. Instructions 61
  • 62. Instructions 62
  • 63. Instructions 63
  • 64. Instructions 64
  • 65. Placement of Clarifying Instructions •  Help respondents have the same interpretation •  Definitions, instructions, examples •  Before item is better than after Conrad & Schober, 2000; Conrad et al., 2006; Conrad et al., 2007; Martin, 2002; Redline, 2013; Schober & Conrad, 1997; Tourangeau et al., 2010 65
  • 66. Placement of Help •  People are less likely to use help when they have to click than when it is near item •  “Don’t make me think” 66
  • 67. Placement of Error Message •  Should be near the item •  Should be positive and helpful, suggesting HOW to help •  Bad error message: 67
  • 68. Placement of Error Message •  Should be near the item •  Should be positive and helpful, suggesting HOW to help •  Bad error message: 68
  • 69. Placement of Error Message •  Should be near the item •  Should be positive and helpful, suggesting HOW to help •  Bad error message: 69
  • 70. Placement of Error Message •  Should be near the item •  Should be positive and helpful, suggesting HOW to help •  Bad error message: 70
  • 71. Error Message Across Devices 71
  • 72. Error Message Across Devices 72
  • 73. Better UX means… •  Higher user satisfaction –  Increased efficiency and accuracy –  Repeat visits and recommendations •  Decreased costs for the organization –  Reduce call center phone calls and staffing •  Data you can trust –  Empirically tested products •  From the end users’ perspective 73
  • 74. Thank you! •  Twitter: @forsmarshgroup •  LinkedIn: http://www.linkedin.com/company/fors-marsh-group •  Blog: www.forsmarshgroup.com/index.php/blog Jennifer Romano Bergstrom @romanocog jbergstrom@forsmarshgroup.com MoDevGov| Rosslyn, VA