More Related Content


More from DigitalGov from General Services Administration(20)

Recently uploaded(20)


ICGI Content Standards: Usability Test Results, Content Managers’ Forum

  1. ICGI Content Standards Usability Test Results Content Managers’ Forum July 15, 2004
  2. The Tests:  Tested eleven content standards  Terminology  Provide description—”What would you call this?”  Rank list of options by preference  Placement on the web page  Identify placement on generic web page  Categorization  Card sort exercise  Reaction to “typical” federal page
  3. Generic Federal Homepage
  4. The Test Participants:  Eleven participants  Mix of races/ethnicities  Variety of web experience  Wide age spectrum  Both genders  Range of formal education
  5. Sponsorship of Gov Web Sites  Participants shown a homepage (HHS/HUD) What tells you it's an official govt. site? .gov in URL 54.5% "U.S. Dept .of…" visible 45.5% "Dept. of/agency name" visible 18.2% graphic logo/seal 18.2% other graphics 9.1%
  6. Comments  “ “Mainly checks for the .gov URL such as and trusts it more than other sites like .com.”  “Definitely goes to .gov for government information and checks for the .gov URL.”
  7. Link to Gov’t-wide Portal Which label most accurately depicts:  An official gateway to all government information in the United States  A portal containing links to federal and state government sites
  8. Ranking: Unadjusted Adjusted 1.90 2.05 1.91 1.91 2.10 2.23 3.89 3.82  Participants thought that “U.S.” and “fed” meant federal, and therefore, were looking for a more encompassing term that did not seem to exclude state governments.
  9. Location Users expected the link to be at the top of the page:  3 participants- top left menu bar  2 participants- top right menu bar  2 participants- top nav bar  2 participants- bottom of page/ footer
  10. Comments  “Regarding the site name – doesn’t know what this means at all.”  “Regarding the site name – said this doesn’t apply to states because of the fed term.”  “Suggested”  “Said is ridiculous and looks like a typo”  “Said to be consistent and put all links on the left. I don’t want to look all over the page for it. There should be one spot where I can expect to see everything.”
  11. About Us Which label most accurately depicts a place where you can:  Learn who operates this web site  Learn what this organization does  Find the organizational structure of the group that runs this site
  12. Ranking Unadjusted Adjusted About Us 2.25 2.50 About [Organization] 2.31 2.56 Learn About Us 2.50 2.50 Who We Are 2.94 2.94 For More Information 4.50 4.50
  13. Location Users were split as to expecting the About Us link to be at the top or the bottom of the page.  3 participants- top left menu bar  1 participant- bottom of page  1 participant- bottom/ middle of page  1 participant- bottom/ right
  14. Comments  “Expects basic stuff near the top of the page and lower links to be more explanatory.”  “Said the test page looked as expected- it has links to specific organizations where you can find more. Expects to find jobs and then sees employment.”  “Said the test page looked as expected. Expects to find email not the mailing address.”  “A guide to the organization structure is great”  “Would like to see contact us closer to the top. At the bottom it makes me think they don’t want me to contact them.”
  15. Laws and Regulations Card Labels Guidance  Policy  Current Legislation in Congress  Major Laws & Statutes  Executive Orders  Proposed Rules  Regulatory Actions  U.S. Code  Federal Register Documents  Preambles to Final Rules  Dockets  Testimony Tester Recommendations  Legislative Rulings  Legal/ Legislation  Building a Law from Start to Finish  Steps to Creating a Law  Information Reference/ Resources  Rules and Regulations  Governmental Procedures
  16. Ranking Laws and Regulations 1.71 Policies and Regulations 2.14 Policy 3.00 Laws and Regulatory Assistance 3.14 The majority of the participants (90%) thought “passed and signed laws are different than proposals, so they would not be grouped together” or “these terms should be arranged in a developmental way.”
  17. Location The majority though the Laws and Regs link belonged at the top of the page  4 participants- top left nav bar  2 participants- top right nav bar  2 participants- middle left menu bar  2 participants- bottom of page/ footer
  18. Comments  “Divided cards into two groups. From the two groups in the card sort would call them Preamble (for future laws) and Statutes (for current laws).”  “Said they should be arranged in the order of the process of becoming a law.”  “Thought these looked like the steps to creating a law.”  “Sees the list in a developmental sense such that proposed rules come before laws. Would arrange the list into three categories and didn’t think “Guidance” fit in the group.”  “Said terms were related but didn’t all fit together. Saw a difference between the concrete laws and other terms.”  “Said the information did not fit together and should be broken into categories: 1. Agency Legislation in Process, 2. Historical Source of Information for Legislation, 3. Executive Orders, 4. Policy & Guidance.”
  19. Search Which label most accurately depicts a place where you can:  Enter terms in order to find content most relevant to your needs  Scan a website using keywords to find matching information  Use single words or phrases to quickly locate information within an agency's web site 6 out of 10 (60%) of the participants came up with at least one label suggestion that included the word “Search” (search, keyword search, quick search, etc.).
  20. Ranking Users were provided the following options:  Find It  Ask  Search  Where do I…  Select a Topic  Clear preference here: 80% of the participants tested ranked the label “Search” #1.  For most participants "Ask" ranked very low, and some commented it sounded like Ask Jeeves or like one could actually ask questions online. Many participants would like a "Select a Topic" feature and would expect to see a drop-down menu, but said this was obviously different than Search.
  21. Location  90% expected Search to be somewhere in the top menu bar  Right hand side of the bar 4  Left hand side of the bar 2  Center of the bar 0  No real preference within bar 3
  22. Search Preferences Participants were given color printouts of three home pages:  example with search box empty (, box in left hand menu)  example with search box filled in (, filled box in header graphic)  example with no search box, just a link to "Search" (DOD, link in menu bar) During this test, 60% of the participants expressed a definite liking for a “search box” (2 with text inside, 2 empty, 2 no preference or didn’t matter).
  23. Contact Us Which label most accurately depicts a place where you can:  Email the owners of the site  Get the phone number of the site owners  Get the address of the site owners  Report a technical issue or a question about the site Suggestions for labels included Public Information (3), Request… (3), For More Info (3), FOIA (2 – one user had worked for govt. and recognized acronym).
  24. Ranking Assistance Comments Help Contact Us For More Information There was a strong preference for Contact Us- eight of the nine participants listed it as #1.
  25. Location 70% expected to find the Contact Us link at the bottom of the page  3 participants- top right menu bar  3 participants- bottom left menu bar  2 participants- bottom of page  1 participant- bottom right menu bar  1 participant- bottom center of page
  26. Comments  “Said this should be a link to a phone directory for agencies, divisions, and bureaus . . . you can never find anybody anymore!”  “Thinks agencies should send at least a boiler plate response in an email.”  “Contact Us should be a way to email an agency.”
  27. FOIA Which label most accurately depicts a place where you can:  Learn about your rights to request from a Government Agency public information that in the past had not been available to the public.  Request information – such as books, charts, emails, and records - on a specific Government Agency’s activities.
  28. Ranking Unadjusted Adjusted Public Rights to Gov Info 2.43 3.18 Public Access to Gov Info 2.50 2.82 Freedom to Request 2.86 3.59 Request Gov Info 3.44 3.55 Freedom of Information Act 3.67 3.77 FOIA 4.07 4.23
  29. Location Participants were mixed on expected location:  5 participants left menu bar  3 participants page center  4 participants bottom of page  2 participants right menu bar
  30. Comments  “Recognized "Freedom of Info" Thinks media/big wigs have this, not people like her… Would not recognize FOIA, Definitely prefers the term "public" Tells her she, a citizen, has access to it.”  “[says] If she didn't know FOIA, she would pick public rights/access.”  “"Rights" suggests these may be problems or roadblocks in getting the info.”  “… people want to go right to submitting a request. The example page [USDA FOIA page] is what's expected but info about law is not useful…. Links on the page should be moved up.”  “Example page: does not like to have to go to another page to find out more about filing a request.”
  31. Current Information Test participants were shown two pages:  Today’s Date at top of page (DOL)  Last Modified Date at bottom of page (BLS)  Is this page up to date?  What does the date indicate?  Is this date useful?
  32. Ranking Participants were then asked to rank, in terms of usefulness, the following types of dates:  Today’s Date  Posted Date  Last Modified Date  Last Reviewed Date  Last Certified Date  Last Updated Date  Creation Date
  33. Results Participants either loved or hated Today’s Date:  “Today's date doesn't tell whether info is current.”  “Likes both Today's Date and Last Modified.. Depends on info he's looking for.”  “Date up on top left corner is not updated date, not useful.”  “Date by itself could mean today. Many people may not know what "modified" means - but there has to be a label with a date.”  “Today's date is important. Indicates that page is kept current all day. Stats should be current!”
  34. Results Participants seemed to like either “Last Modified” and “Last Updated” to indicate the currency of content on a particular page. Several ranked them equally or close to equal.  “Stats and Statements of fact- wants a last modified date. If no date no way of knowing how accurate or up to date is the info.”  “Last modified-content updated. TP likes the last modified date info.”
  35. Location Most participants expect the date to be at the top of the page:  5 participants- top left of page  2 participants gave an expected location vs. actual (desired) location--“prefer top of page, expect upper right”
  36. FAQs or Common Questions Which label most accurately depicts a place where you can find information requested most often from the agency?  4 out of 11 (36%)- Frequently Asked Questions  3 out of 11 (27%)- FAQs  Other suggestions: Frequently Requested…, Most Often Requested…. Strong preference for “standard” labels of Frequently Asked Questions or FAQs. Note: Throughout testing, users often expressed dislike for, or confusion from, acronyms such as FAQ, FOIA, etc.
  37. Location Participants were mixed on expected location:  3 participants- top left of page  2 participants- bottom of page  3 participants- right menu bar (1 bottom, 2 unspecified)  2 participants- middle of left menu bar
  38. Comments  “Doesn't like to hunt. Uses it as a last resort. Looking for questions/answers. On test page [EPA FAQ on Radon]…a lot on one page.”  “Want to reach a human being. Not machine.”  “Usually get the answers I want”  “Would like contact info or help if you can't find what you need.  “Does use FAQs depending on what he is looking for, if he doesn't see info on first page and he thinks it is common information.”
  39. No Fear Act Which label most accurately depicts a place where you can:  See a summary of any grievances related to unfair treatment of a person or group on the basis of prejudice filed against an agency  Find statistical information about any acts of reprisal against government workers who reported unfair treatment at an agency Suggestions for labels included: Grievances, Government Workers Disputes, Report on Grievances, Discrimination Against Government Employees, Employee Actions & Grievances, Reprisal, Discriminatory Site, Unfair Employment…
  40. Ranking  Discrimination and Retaliation Complaints 2.20 2.41  Equal Employment Opportunity Complaints 2.44 2.82  Equal Employment Opportunity Data 3.25 4.45  Employee Complaint Data 3.56 3.73  Federal Employee Antidiscrimination 4.29 4.64 and Retaliation Act  EEO Data 4.75 5.05  Antidiscrimination & Whistleblower 4.88 4.91 Protection Reporting
  41. Location Top locations selected on generic page layout:  5 of 11 (45%) of participants- look for it at bottom of the page, (no predominance for left/right/center).  4 of 11 (36%) of participants- look for it in top nav bar, (2 left side, 2 right side)
  42. Comments  “Didn't like Federal Employee Antidiscrimination & Retaliation Act and Antidiscrimination & Whistleblower Protection Reporting because they were "very wordy"”  “Employee Complaint Data is too generic and vague, not very familiar with EEO or EEOC”
  43. Web Site Policies and Important Notices Card Sort Items  Privacy & Security Statement  Freedom of Information Act (FOIA)  Information Quality Guidelines  Portable Document Format (PDF)  Accessibility Policy  Linking to Outside Websites  Disclaimer of Endorsement  Copyright Information  No Fear Act  Photo Credits  Schedule for Publishing Information  Nondiscrimination Statement  Legal Notices  Free Commercial Downloads Participant Suggestions  Policies  Legal  Overall Info  Site Info  Additional Info  Information Guidelines  Information Statement
  44. Card Sort Results  Users had a lot of difficulty creating one group that covered all topics. Some users suggested two groupings of “legal” and “information”.  The following content items were singled out by multiple (3+) users as either irrelevant or confusing with relationship to grouping the other items: PDF, Linking To Outside Websites, Free Commercial Downloads, and Photo Credits.  Participant comment: “stuff you don’t even know what it is. PDF? Photo Credits?”
  45. Ranking Unadjusted Adjusted Notices and Disclaimers 1.50 1.55 Site Notices and Links 2.44 2.50 Site Policies and Links 2.71 2.85 Important Notices 3.00 3.09 Note: 54% of the participants ranked “Important Notices” at or near the bottom.
  46. Location Whatever we call it, users expect to find it at the bottom of the page:  7 of 11 (63%) of participants- look for it at bottom of the page (0 left side, 2 right side, 1 center, 4 not specified)  No other location got more than 1 suggestion.
  47. Comments  “Important Notices (this doesn't say anything)”  “Does not know what "Accessibility Policy" is. Accessibility does not fit.”  “Keep policies/legal info separate from free stuff .”  “Card Sort: PDF, Linking to Outside Websites, Free Commercial Downloads -these three don't fit with others”  “Free Commercial Downloads - What does that mean! Nothing to do with govt.”  “"I like policies more than notices. " "They are legal policies not all are site policies." Why is "Photo Credits" here.”  “No to [suggested label] Important Notices (Not important)”
  48. What testing could be done next  Two tests were already designed by the testing subcommittee but were dropped from this round of testing because of time constraints  Label, location and expected content for “Jobs”, “Privacy”, “Site Index” and “Site Map”.  Best practices for Linking to Outside Websites.
  49. Possible additional tests  Label, location and content for “Information Quality Guidelines”  Appropriate content and best practices for cross-government portal implementation  Plain language (writing content for the web)  Limited English Proficiency (translation services / cross-language content delivery)