GETTING STARTED Read two books available on our shelves: Web Site Design with the Patron in Mind, by Susanna Davidsen & Everyl Yankee Design talk : understanding the roles of usability practitioners, web designers, and web developers in user-centered web design, by Brenda Reeb Reviewed mission statement of PSU Special Collections (“…to provide primary source materials for researchers from Penn State and around the world”)
GETTING STARTED, CONTINUED • Brainstormed user groups • Brainstormed user tasks • Got input/approval from colleagues • Met with Usability Team • Became certified for IRB • Secured funding
STUMBLING BLOCKS IRB (Institutional Review Board) process Funding/reimbursement process RecruitmentOTHER NOTESo Not testing finding aids or digital projectso Specific eye towards administrative organizationo Having deadline was good to force IT cooperation
STUDY PROTOCOL Introductions Explanation of the study: We are testing the usability of the website, NOT your skills as a searcher. Please think aloud while performing the tasks, about how and why you make choices, and what confuses or frustrates you. We can clarify the tasks, but cannot help you perform them. The session will be recorded, and notes will be taken, but your involvement will be anonymous. Recordings will only ever be shown within the department for internal purposes. Your name will never be associated with the recordings or the notes. After you have done the tasks we will ask you to complete a brief survey. The entire session should last about 1 hour. Participation in this study is voluntary, and you can choose to end this session any time. Do you have any questions? Consent forms Set up computer and give participant list of tasks Take notes Questionnaire Ask again if any questions Thank them, provide compensation, get receipt
TASK LIST Warm-up: Use our website to answer the following questions: What hours is the Special Collections Library open to the public? Where is the Special Collections library located? PSU students (undergrad and grad): You have been assigned to write a paper using historical documents from the Penn State Special Collections library. Your professor has told you that there are special procedures for using Special Collections. Using our website, find two ways using a special collections library is different from using a general academic library. You have found sources within Special Collections and written your paper. Use our website to find the correct format for citing those materials. PSU staff: Your filing cabinets are overflowing, and your supervisor has asked that you transfer old “stuff” to the University Archives to make room for current “stuff”. Locate information on what records we collect on our website. General: Can you locate the following on our website: (Each subject asked 2) Digital images of Pennsylvania railroads? Information about special collections materials related to the 8th air force? Information about special collections materials related to Amish settlements in Pennsylvania? Information about special collections materials related to African American history? Can you use our website to: Figure out who you would ask a question about the United Steelworkers archives? Order a Fred Waring Christmas CD? Find a yearbook photo of David Arthur Valentine, who graduated in 1961? Find out if we have video of the October 1940 football game with Temple?
QUESTIONNAIRE1) How much time do you spend using the Internet for homework or work, per week? ___ Never ___ 1-5 hours ___ 5-10 hours ___More than 10 hours2) Do you consider yourself an expert computer/Internet user? Yes/No3) Have you ever been oriented to special collections (generally or at Penn State)through a class, tour, online tutorial, or other outreach activity? Yes/No4) In the last year, how many times have you used special collections materials? At Penn State: Never, Once or twice, Between 2 and 5 times, More than 5 times Anywhere Else: Never, Once or twice, Between 2 and 5 times, More than 5 times5) The Special Collections Library Web Site is easy to use: __ Strongly Agree __Agree __Disagree __Strongly Disagree6) If you selected Disagree or Strongly Disagree please tell us what causes difficulties:7) What changes would you make to improve the Special Collections Library’s website?
OBSERVATIONS – TASK CONSTRUCTION Terms matter When phrase “historical materials” in question, they go to “Historical Collections” unit – same for “archival materials” and University Archives “Reference” confused for citation Along these lines, subject knowledge matters – “Amish” in question, “Pennsylvania German” in collection title Order of questions can influence approach May need to change order from study to study
OBSERVATIONS – SEARCH (GENERAL) About half didn’t even look at the side bar navigation Everyone likes to use the site search But it is unclear that this searches all the University Libraries. They also want to search on format (and can’t)
OBSERVATIONS – SEARCHING SUBJECTS None of those looking for Amish materials found our largest Amish collection Only one person looking for African-American materials found our largest A/A collection Most commented they wanted collections listed by topic
OBSERVATIONS – THE USELESSNESS OFLONG EXPLANATIONS ON WEBSITES Half of the subjects commented there was too much text on any given page Several could be on the right page but not see a link right in front of them, if there were too many “ other things” to look at Yadda yadda yadda yadda yadda Yadda yadda yadda yadda yadda Yadda yadda yadda yadda yadda Yadda yadda yadda yadda yadda Yadda yadda yadda yadda yadda Yadda yadda yadda yadda yadda Yadda yadda yadda yadda yadda Yadda yadda yadda yadda yadda Yadda yadda yadda yadda yadda Yadda yadda yadda yadda yadda
IMMEDIATE CHANGES Add curator contact info to all collection pages Add “(yearbook)” to LaVie link Add citation info to tutorial & online reference pages Add link to digital collections from search collections page Hopefully, organize by subject rather than unit Hopefully, further edit long textual descriptions on top- level pages
NEXT STEPS First round not perfect, but we’ve learned a lot, not just about the web site but about the user study process. For the next round, we may use a more ethnographic approach. Google Analytics. Site search just for Special Collections? Better finding aids & FA search (then test). I will be asking to join the Usability Team.
AREAS FOR FUTURERESEARCH/COLLABORATION/BRAINSTORMING Apply Kulthau? Standardized list of questions for archival website user studies? Explore ideas for “Research Help”: Diagrams/visual dictionary of common terms Possible to do mouseover definitions in CMS? FAQ since looks simpler/quicker than tutorial? "Search Tips“ Thinking about how to illustrate concepts/collections with just one image
KULTHAU’S INFORMATION SEARCH PROCESShttp://comminfo.rutgers.edu/~kuhlthau/information_search_process.htm