Peeling Back the Discovery Layer: Web-Scale Searching, User Expectations, and Information Literacy
•Download as PPTX, PDF•
0 likes•122 views
1) The document discusses usage statistics and known issues with the OneSearch discovery platform across several CUNY colleges with varying enrollments. 2) It notes how students typically conduct searches on OneSearch using natural language and simple keywords without Boolean operators or limiting facets. They expect results even for misspelled terms. 3) Challenges include authentication issues, inconsistent display of consortial materials, relevance of results, and students relying too heavily on default rankings without evaluating sources critically. Opportunities include streamlining the user experience and instruction.
Recommended
More Related Content
Viewers also liked
Viewers also liked (14)
Similar to Peeling Back the Discovery Layer: Web-Scale Searching, User Expectations, and Information Literacy
Similar to Peeling Back the Discovery Layer: Web-Scale Searching, User Expectations, and Information Literacy (20)
Recently uploaded
Recently uploaded (20)
Peeling Back the Discovery Layer: Web-Scale Searching, User Expectations, and Information Literacy
- 1. Peeling back the Discovery Layer Web-scale searching, user expectations, and information literacy Nora Almeida Helen Georgas Alexandra Hamlett
- 3. One Platform
- 4. Brooklyn College Senior college w/ undergraduate & graduate degrees Enrollment ~17,000 September 2015 OneSearch usage: 3092 searches
- 5. Baruch College Senior College w/ undergraduate & graduate degrees Enrollment ~18,000 September 2015 OneSearch usage: 8,186
- 6. New York City Tech Senior College w/ undergraduate & professional degrees Enrollment ~17,000 September 2015 OneSearch usage: 4917 searches
- 7. Guttman Community college w/ progressive interdisciplinary education model Enrollment ~800 September 2015 OneSearch usage: 58
- 8. OneSearch Everything Welcome to the library, What are you looking for? ^ not quite
- 9. Known Issues ● Authentication issues and inconsistency across campuses ● Consortial lending materials are not accurately displayed (thus limiting access) ● Relevancy of results On the Other Hand... ● OneSearch works with natural language searches ● (Almost) all searches yield results ● Electronic resource usage is up
- 10. Students…. Use natural language searches and simple keyword phrase queries ■ children of incarcerated parents ■ our town thornton wilder ■ egg consumption risk ■ states that allow citizens to carry firearms have lower crime rates ■ patients most likely to try to commit suicide Expect results for mistyped, abbreviated, or misspelled words ■ Average of 50% of searches yielding no results contained a mis-spelled keyword (Oct 2015, NYCCT) ■ ACA medicaid expansionthr urbsn populations ■ William Foukner ■ nypd survelliance technics
- 11. Students…. Don’t use Boolean but if they do, do not capitalize ■ sherlock holmes and dr watson ■ violent video games and cognition ■ deaf culture and language ■ social media and scams ■ stereotypes and race
- 12. Students…. Include source type or publisher in query itself ■ journal of east asia korean war ■ biotinidase case study ■ Journal of health psychology ■ lack of exercise journal ■ grassroots susan fawcet cengage
- 13. Students…. Do not use facets to limit searches ■ 777 facet clicks for 3092 total searches (Sept 2015, BC) ● 331 top level availability (Peer-reviewed journals, full text online, items currently on shelf) ● 162 resource type ■ 3232 facet clicks for 4,917 total searches (Sept. 2015, NYCCT) ● 713 top level availability ● 1477 resource type ■ 4,760 facet clicks for 8,186 total searches (Sept. 2015, Baruch) ● 1,694 top level availability ● 792 resource type ■ 66 facet clicks for 58 total searches (Sept. 2015, Guttman) ● 18 top level availability ● 29 resource type
- 14. Students…. Rely on default relevancy ranking of search tool, do not look beyond first page of results, and use fewer criteria to evaluate results even as they move down first page “If the search engine is good, you can find the most relevant items in the first 1-3 pages within your search.” (BC Undergraduate) “By structuring and ordering the way information is seen and found, any search interface exerts a form of epistemological power by virtue of their relevancy ranking algorithms.” (Asher, Duke, and Wilson)
- 15. Have authentication issues from off-campus ■ “I tried my library barcode and password but got the message, ‘Invalid UserID and/or password. Please re-enter’” ■ “So to clarify, do I have to log in with BOTH sets of credentials at different points of the search process?” ■ “I have a user name and password but I am asked for a barcode?...I am an online student only” Students….
- 16. Consortial Challenges ● Balancing customization with consistency across campuses ● Branding and display impact usage ● Institutional and individual resistance to OneSearch
- 17. Instructional Challenges ● Unpredictable and irrelevant results ● Metadata display ● Designed to use facets (vs. how students conduct research) ● Students don’t understand the results (resource types)
- 18. Consortial Opportunities ● Potentially universal interface, access point, and authentication across CUNY (user experience similar across campuses) ● Rethink user experience and website design ● Streamline troubleshooting / bug fixes w/ CUNY-wide platform ● Certain features improve the more the platform is used
- 19. Instructional Opportunities ● Students exposed to different resource types with one search ● Facilitate open dialogues about instructional strategies, reference interactions, etc. across campuses ● Critical analysis of OneSearch and student search behaviors (leading to more thoughtful teaching)
- 20. Questions? Nora Almeida nalmeida@citytech.cuny.edu Alexandra Hamlett alexandra.hamlett@guttman.cuny.edu Helen Georgas hgeorgas@brooklyn.cuny.edu
Editor's Notes
- Title Introductions: name + campus affiliation
- CUNY: 24 campuses; 31 libraries Variety of disciplinary and pedagogical models Many students transfer & take courses at more than one campus / Many faculty have multiple campus affiliations Office of Library Services Consortial (interCUNY lending / some CUNY-wide licenses)
- OneSearch aka Ex Libris’ PRIMO implemented in August 2014 for all campuses Balance: implementing tool in way that make sense for local campus populations vs. ensuring consistency to enhance student experience
- Some campuses were already using federated or web scale search tools (Summon / EBSCO)
- Variations in how different campuses have implemented OneSearch (default search or not) & branded it
- Or not. Where access points exist.
- How OneSearch results displays are customized.
- Lots of campuses brand / teach OneSearch as comprehensive tool however... Several vendors (eg. Proquest) are not indexed in PCI Results from omitted vendors will only show up with double authentication Issues with article level indexing for some consortial reference databases like GVRL
- Other Known issues affecting student experience include: authentication, display of consortial materials (deduplication); mysterious relevancy ranking (Local results being pushed to top, title field most heavily weighted then author, but beyond that algorithm and what metadata is being searched is unclear)
- Search behaviors and how they manifest themselves within a OneSearch environment Actual student examples from all four campuses culled from interviews / chat reference queries / use studies / PRIMO reports during 2015 Ex Libris reports that the majority of these keyword searches are two words, followed by three words, which leads to potentially an overwhelming number of results (for simple keyword searches)
- Primo is supposed to ignore Boolean operators in lowercase ("and," "or," "not") but it doesn't. This is a defect that is known to Ex Libris and the issue has been with their Development team since early January 2015.
- NYCCT is teaching it, and it’s the default search on the website (stats vs how OneSearch is displayed and whether it’s taught) hence the importance of prominent display and instructing students to use facets Top level availability facets are: Peer-reviewed journals, full text online, items currently on shelf
- “The judgments embedded within these systems are often opaque and unclear for the user.” And I’m going to go a step further and say that it’s not just that these judgments are unclear for the user, but for librarians as well. And numerous articles and studies have reported librarians’ dissatisfaction with how relevancy ranking really works within discovery tools. This opacity is partly because the algorithm that Ex Libris uses is proprietary, but we don't know EBSCO's exact relevancy algorithm either, and yet we happily teach students how to use EBSCO databases all the time. }“Ex Libris uses a proprietary algorithm called ScholarRank, which takes into account elements of the search and an item's scholarly value. However, it is also possible to "boost" certain items. By default, SFX records get preference. However, OLS has made adjustments to insure that Aleph records are weighted slightly heavier than they are by default.” We know that title and then author field are weighted most heavily but beyond that it becomes less immediately clear how the algorithm is working. So what makes OneSearch so much more complex? A lot more stuff, much larger numbers of search results, coupled with a multitude of formats.
- Chat reference inquiries from Baruch QuestionPoint service (8/2015-9/2015) 12 chats about access related to inactive barcodes just in first month of school (Baruch)
- some customization possible by campus (highlight what could be customized by institution: ordering of facets vs. what could not be customized: advanced search screen) but also need to collectively agree on other aspects of customization, therefore a flattening of how these tools work e.g. many different user groups and specialties across campuses, but single interface)
- (relevance ranking (title field weighted most heavily, then author, with catalog results pushed to top, but unclear beyond that)), PCI, what’s actually being searched not apparent or consistent; need deeper understanding of source database and how that’s being queried) not easy to trace a result back to specific database to do more pointed searches
- (public services committee focuses almost exclusively on OneSearch, Webinars hosted by our central office of Library Services after much expressed need for additional training for librarians) …→ prompted some info literacy libs to reconsider teaching and reference strategies