Rich Search Results for Low Literacy Users - UXPA 2013

1,154 views
1,110 views

Published on

Rich Search Results for Low Literacy Users. UXPA 2013 Conference Presentation on Friday, July 12 at 1:30 p.m with speakers Nicole Kerber and Malcolm Kemeny. Research completed at the University of Baltimore as a part of the Advanced Eye Tracking course with Dr. Kathryn Summers.

For the more than 40 percent of the U.S. population reading at or below an eighth grade level, searching for information online can be a surprisingly difficult task. Viewing a search page full of text-heavy results can make finding the right information nearly impossible for low literacy users. With the advent of Rich Search Results displaying plain text answers, online queries could become an easier task for this at-risk population.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,154
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Rich Search Results for Low Literacy Users - UXPA 2013

  1. 1. Malcolm Kemeny Nicole Kerber Kathryn Summers Noel Alton Megan McKeever UXPA International - July 2013 Rich Search Results for Low Literacy Users Thursday, July 11, 13
  2. 2. What is Low Literacy? What are Rich Results? Findings Agenda Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Research Overview Thursday, July 11, 13
  3. 3. What is a Low Literacy User? Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  4. 4. Corporate What is a Low Literacy User? • 43% of Americans are considered low literacy and read at or below an 8th grade level. • Literacy is defined as “using printed and written information to function in society, to achieve one’s goals, and to develop one’s knowledge and potential.” • Measured using a REALM (Rapid Estimate of Adult Literacy in Medicine) test. REALM Test - Side A Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  5. 5. Corporate Recruiting Low Literacy Users • Contacted various nonprofits and literacy groups in Baltimore with little success before settling on ad-hoc recruiting. • Target group is unlikely to have consistent contact method and are difficult to schedule. • Network recruit - participants bring their family/friends to participate. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  6. 6. Working With Low Literacy Users • Tread carefully. • Ask, “Do you think someone else might have problems using this website? What do you think they would find confusing?” • Be respectful and friendly. • Build a relationship. • Be prepared with all materials to get session completed quickly. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  7. 7. Web Use by Low Literacy Populations Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  8. 8. Corporate Low Literacy Web Use • Read every word over scanning the page. • Become disoriented on the page easily - scrolling is difficult. • Accept what they see online as truth without verifying or reading any additional content. • Less likely to access information online. • Easily confused on pages. • Make decisions without the benefit of context. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  9. 9. What are Rich Results? Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  10. 10. Corporate • Information seeking aid incorporated into search engines. • Built to provide answers to queries on the primary results page. • Presented above the rest of the search results in large, visually distinct, unambiguous text. • Increases user satisfaction by presenting relevant content where it can easily be found. • Relatively new. Google started presenting rich results in early 2010.https://support.google.com/webmasters/answer/99170?hl=en Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Rich Search Results Thursday, July 11, 13
  11. 11. Corporate • Style of results vary by topic. • Usually come from a “do” search or a simple question such as, “What day is thanksgiving?” • 20-30% increase in click through rates for websites that support rich results. http://www.wpromote.com/blog/seo/smx-advanced-2012-day-1-part-1/ Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Rich Search Results Thursday, July 11, 13
  12. 12. Corporate Rich Search Results • Rich results are still in their infancy and not fully supported. • Search engines must continue to improve support for misspellings, phrasing, and formatting. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  13. 13. Search Behavior Study Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  14. 14. Purpose: • Understand the presentation of rich results in a search engine results page (SERP) as it relates to information retrieval by low literacy users. Test Plan: • 30 low literacy participants, who read at/below an 8th grade reading level, were recruited. • Literacy levels were determined by administering the Rapid Estimate of Adult Literacy in Medicine (REALM) test. • Participants were asked to complete 4 tasks using one of three search engines; Google, Bing, and Yahoo!. Each participant used one of the three engines exclusively throughout the test. All participants were tested with a Tobii t60 (remote) eye tracker. • Each task was designed to increase the likelihood of seeing rich results. • $40 cash incentive for one hour of participation. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Methodology Thursday, July 11, 13
  15. 15. Methodology Test Scenarios: 1.You and a friend want to go see the movie Looper tonight at the Landmark theatre. What are the showtimes for tonight? 2.Your daughter has a question on her homework about the word fortuitous. You want to make sure you give her the right answer. Look up what the word means on the web. 3.You are planning your families’ thanksgiving dinner. Look up what day Thanksgiving is this year. 4.Your son has been invited to a birthday party at Port Discovery. You want to call and ask about parking. Find the phone number for Port Discovery. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  16. 16. Participant Demographics College Graduate 3% Some High School 33% No High School 3% High School Graduate 37% Some College 23% Participant Education Levels 3rd Grade and Below Grade 4-6 Grade 7-8 18 9 3 Participant Reading Levels 60+ 3% 50-59 27% 40-49 10%30-39 37% 18-29 23% Participant Ages Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever • 30 Total Participants. • 56% access the Internet primarily through a mobile device. • 60% rated themselves as very comfortable using the Internet. Thursday, July 11, 13
  17. 17. Results Overview • When some users reached a results page with rich results, they were able to understand, interact with, and learn from rich results • Identified THREE key behaviors among participants which limited their ability to learn from rich results - 1. Participants used natural language queries, which search engines had trouble providing rich results for. 2. Participants relied heavily on autocomplete, which often led them down the wrong search path due to misspellings. 3. Participants often overlooked rich results when they were displayed. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  18. 18. Success with Rich Results Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  19. 19. Corporate Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever • Users who saw, understood and interacted with the rich results were generally pleased with the information. • Users were visibly surprised that they were able to find the information they were looking for easily. Users interacting with rich results did not click further to verify the information, and overall task time was reduced. Success with Rich Results Top: P01 - Alvin Bottom: P16 - Earlene Thursday, July 11, 13
  20. 20. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Success with Rich Results P17 - Kennis Thursday, July 11, 13
  21. 21. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Success with Rich Results P07 - Danielle Thursday, July 11, 13
  22. 22. P20 - David (offset) Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Success with Rich Results Thursday, July 11, 13
  23. 23. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Success with Rich Results Top: P26 - Victoria Thursday, July 11, 13
  24. 24. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Success with Rich Results Bottom: P06 - James Thursday, July 11, 13
  25. 25. Thursday, July 11, 13
  26. 26. Thursday, July 11, 13
  27. 27. Issues with Rich Results Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  28. 28. Issues with Rich Results Task Found and Reported Overlooked Movie Showtime 1 time 7 times Define: Fortuitous 4 times 5 times What day is Thanksgiving? 8 times 7 times Phone number lookup 6 times 5 times Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Rich Results were displayed 45 times out of a possible 120. Rich Results were discovered and used 19 times out of 45. Thursday, July 11, 13
  29. 29. Difficulty with Computers • The participants during testing that tested as very low literacy had a great deal of difficulty with the study tasks - • 3 participants tested as reading below a 3rd grade level • 9 participants tested as reading between a 4th and 6th grade level. • One participant entered ‘Pour Disrovr’ while trying to find the phone number for Port Discovery’. He later ended up looking at the Pottery Barn site. “It’s not really hard, I just gotta be more specific to actually get to where I want to go. ” Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  30. 30. Difficulty with Computers •Some test participants were not able to understand the goals of each task. These users had trouble during the test and many became visibly frustrated. •Other users were familiar with web use but not comfortable using search tools. Once participant began every search with WWW. in the search bar. •Existing search tools do not adequately support the needs of users new to the web. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  31. 31. Natural Language Queries Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  32. 32. Corporate • User queries frequently matched the language used by the test moderator in the prompt. • These natural language queries often resulted in unsuccessful searches. Participant Search for Task Prompt of “...find the phone number for Port Discovery” ` Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Natural Language Queries Thursday, July 11, 13
  33. 33. Thursday, July 11, 13
  34. 34. Thursday, July 11, 13
  35. 35. Corporate RECOMMENDATIONS: Search engines should work on providing rich results for natural language queries. This will simplify information seeking for users of all literacy levels. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Natural Language Queries Thursday, July 11, 13
  36. 36. Autocomplete Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  37. 37. Autocomplete Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever • Users relied heavily on search engine autocomplete to help guide search. • Autocomplete was used as a way to help users avoid typing and spelling, often a difficult activity. Users would type one letter, look to the autocomplete, type another letter and check again. • Autocomplete failed as user’s queries moved further from the correct spelling. • Some users would look at the keyboard as they typed, missing the autocomplete suggestions altogether. Thursday, July 11, 13
  38. 38. Thursday, July 11, 13
  39. 39. Thursday, July 11, 13
  40. 40. Corporate RECOMMENDATION: Autocorrect should support error correction as the user types. A search for “Thanksgaving” should still return an autocomplete suggestion and rich results for Thanksgiving. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Autocomplete Thursday, July 11, 13
  41. 41. Corporate RECOMMENDATION: Participant queries that receive autocorrect results should support rich results. Search engines currently support autocorrection, showing rich results for these queries can help with information retrieval. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Autocomplete Thursday, July 11, 13
  42. 42. Overlooking Rich Results Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  43. 43. Issues with Rich Results Task Found and Reported Overlooked Movie Showtime 1 time 7 times Define: Fortuitous 4 times 5 times What day is Thanksgiving? 8 times 7 times Phone number lookup 6 times 5 times Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Rich Results were displayed 45 times out of a possible 120. Rich Results were discovered and used 19 times out of 45. Thursday, July 11, 13
  44. 44. Overlooking Rich Results Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever P13 - Joanne Thursday, July 11, 13
  45. 45. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever P05 - John Overlooking Rich Results Thursday, July 11, 13
  46. 46. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever P02 - Kevon Overlooking Rich Results Thursday, July 11, 13
  47. 47. Overlooking Rich Results Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever P06 - James Thursday, July 11, 13
  48. 48. Thursday, July 11, 13
  49. 49. Thursday, July 11, 13
  50. 50. Overall Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Thursday, July 11, 13
  51. 51. • Rich results can be particularly beneficial to low literacy users, who tend to settle on the first answer they find in a search. • There were THREE behaviors from participants which impacted their ability to use rich results - 1. Participants used natural language queries, which search engines had trouble providing rich results for. 2. Participants relied heavily on autocomplete, which often led them down the wrong search path due to misspellings. 3. Participants often overlooked rich results when they were displayed. Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever Overall Thursday, July 11, 13
  52. 52. Overall Rich Search Results for Low Literacy Users Kemeny, Kerber, Summers, Alton, McKeever 1. Search engines should move to provide rich results for natural language queries. 2. Search engines should unobtrusively support live autocorrect inside of the autocomplete options. 4. As rich results become more common, users will start to expect rich results. 2. Search results displayed with autocorrect should support rich results. Thursday, July 11, 13
  53. 53. MALCOLM KEMENY NICOLE KERBER KATHRYN SUMMERS KSUMMERS@UBALT.EDU NOEL ALTON NTALTON@GMAIL.COM MEGAN MCKEEVER MCKEEVER.MEGAN@GMAIL.COM THANK YOU MKEMENY@PAYPAL.COM NKERBER1@GMAIL.COM Thursday, July 11, 13

×