Presentation at the BIBSYS Oria working group meeting and Alma/Oria information meeting at the University of Oslo

1. Not available, or not found?
Lessons from user queries in
the Oria catalog at UiO
BIBSYS Oria working group meeting (07/12/2016)
Alma/Oria information meeting (08/12/2016)
Hugo Huurdeman, Visual Navigation Project (@BookNavigation)

2. Visual Navigation Project
interaction methods with digital content have to be rethought
Increasing degree of digital access to materials
Loss of rich interactions in physical space
1 3
2
photo: Marina Tofting
Photo:metalocus.es

3. Introduction
• More insights needed of how current catalogs
are used in practice, what goes right and what
goes wrong
• Opportunity: analyze data gathered at UiO during
the last two years

6. Primo Analytics
• Actions
• Device usage
• Facets
• Queries (most popular, monthly)
• Timeframes: Jan-Jun 2015; Nov-Dec 2015; Jan-Sep 2016
• Sessions
• Zero result queries (daily)
• Aug 7 2015-Sep 30 2016

8. Processing & Analysis
• Basic normalization of queries
• "stanislav andreski" -- stanislav andreski
• Manual annotation of
• (1) top 50 popular queries
• (2) random selection of 50 "zero result" queries
• If derivable from query, determine
• nature of query?
• for which resource type?
• curriculum-related? [in pensum lists UiO]
• successful? [result on first results page]

9. 1. Most popular queries
• 5,776 different
queries, 115,590
searches (UiO)
• monthly totals
• 2015 & 2016
• Analyzing top 50
• Together issued
almost 20,000
times (17%)

10. Top 50
1. atekst 1425
2. pubmed 1221
3. exphil 719
4. det kvalitative forskningsintervju 711
5. nature 669
6. medical genetics 598
7. direktekrav 562
8. menneskets fysiologi 540
9. 998903677444702000 500
10. science 483
11. spesialpedagogikk 475
12. lancet 438
13. jussens venner 369
14. 991645464 364

12. Top 50
• What types of queries are most popular?
• titles (49%)
• det kvalitative
forskningsintervju
• det norske samfunn
• “topics”
• spesialpedagogikk
• neurology
• “databases”
• atekst
• pubmed
• duo

13. Top 50
• What resource types are sought for?
• books (45%)
• det kvalitative
forskningsintervju
• det norske samfunn
• journals (11%)
• science
• lancet
• “databases” (10%)
• atekst
• pubmed
• duo

14. Pensum?
• Were the queries related to pensum books?
• Quite often: at least 57%!
• menneskets fysiologi
• det kvalitative
forskningsintervju
• Or other things (9%) :)
• aftenposten;
morgenbladet
• harry potter

15. Pensum?
• Were the queries related to pensum books?
• Quite often: at least 57%!
• menneskets fysiologi
• det kvalitative
forskningsintervju
• Or other things (9%) :)
• aftenposten;
morgenbladet
• harry potter

16. Curriculum?
• Were the queries related to pensum materials?
• Quite often: at least
57%!
• menneskets fysiologi
• det kvalitative
forskningsintervju
• Or other things (9%) :)
• aftenposten;
morgenbladet

17. Were the queries successful?
• Often, yes:
• main result in first 10
results: 51%
• not (easily) found: 19%
• For example:
• pubmed
• nature
• science

18. 1. atekst 1425
2. pubmed 1221
3. exphil 719
4. det kvalitative forskningsintervju 711
5. nature 669
6. medical genetics 598
7. direktekrav 562
8. menneskets fysiologi 540
9. 998903677444702000 500
10. science 483
11. spesialpedagogikk 475
12. lancet 438
13. jussens venner 369

19. 2. “Zero result” queries
• 39,925
different
queries
• In total
52,257
searches
• Aug 2015-
now
• Annotating
random
sample (50)
al azm sadik «orientalisme og omvendt orientalisme» 28
curr eye res 27
allmenningen olaf 24
12136173x 22
am j ophthalmol 21
821017268 21
askim j. 19
900317809 19
cheng 19
(direkte krav or direktekrav) and subrogasjon 17
151582998 16
142734500 16
direkte krav or direktekrav 16
961675616 16
9780549547303 16
agirdag phalet 16
andresen steinar elin l. boasson og geir hønneland (2012) international environmental agreements 15

20. Queries without results
What is the nature of the “zero result” queries performed in Oria?
• Pasted citation (43%)
• Browning, N. (2015). The
ethics of two-way symmetry
and the dilemmas of dialogic
kantianism. Journal of Media
Ethics
• Title (31%)
• Sentralbankens oppgaver i
dag og i fremtiden
• “Folk er vanligvis ikke særlig
villige til å snakke om de
døde”
• Author (16%)
• Christopher Hotchens

21. Queries without results
• Book (26%)
• Prcopius Secret History
• Book Chapter (12%)
• Solhaug, (2006). Kapittel 13:
Strategisk læring i
samfunnsfag. I
• Article (26%)
• Browning, N. (2015). The
ethics of two-way symmetry
and the dilemmas of dialogic
kantianism. Journal of Media
Ethics
Which resource types are not found?

22. Degree of pensum queries?
At least 28% of the
unsuccessful queries
are for pensum
materials
• Fukuyama, F. (2013): What
Is Governance?
Governance, Vol. 26, No. 3,
July 2013 (s. 347–368).
• basic immubology
• Adcock + Collier /
Measurement Validity: A
Shared Standard for
Qualitative and Quantitative
Research

23. Why no results?

24. Why no results?
• Query being too specific (pasted citation, pasting quote from text)
[22%]
• Browning, N. (2015). The ethics of two-way symmetry and the
dilemmas of dialogic kantianism. Journal of Media Ethics
• Misspellings, reference mistakes (e.g. wrong year) [20%]
• why students underacheive
• svennevig j.: ledelsesretorikk i nedbemanningssituasjoner 2009
• Using incorrect query syntax [2%]
• "Psykisk helsevern" + "tvang" + "umenneskelig"
• "McLuhan" AND/OR "Understanding media"
• 978-147996410-9

25. Why no results (2)
• Searching for an ISBN number, which may just refer to a specific
edition of a book, but not the one in the library
• 9780618721566
• Searching for journal titles, ISSNs, DOIs
• journal of speech and hearing disorders
• Searching for course codes
• MED1100
• Resource not available in Oria [24%]
• Haugianerliberalistene: En analyse av haugianere som
politikere og næringslivsaktører
• "Limp Wrists and Laser Guns: Male Homosexuality and
Science Fiction"

26. 3. Learned Lessons (1)
• Importance of aiding students finding better results, e.g.
• Enhanced spell check / query corrections
• why students underacheive → 0 results
• why students underachieve → 463 results
• Query suggestions and autocomplete
• Could be based on previous (successful) queries
• especially recurring queries should be supported (50% of popular
queries!)
• ISBN suggestions
• Automatically search for the book title and author (using
information derived from ISBN number?)

27. 3. Learned Lessons (1)
• Importance of aiding students finding better results, e.g.
• Enhanced spell check / query corrections
• why students underacheive → 0 results
• why students underachieve → 463 results
• Query suggestions and autocomplete
• Could be based on previous (successful) queries
• especially recurring queries should be supported (50% of popular
queries!)
• ISBN suggestions
• Automatically search for the book title and author (using
information derived from ISBN number?)

28. 3. Learned Lessons (2)
• Importance of "pensum" queries
• Detecting curriculum queries
• E.g. widget which suggests pensum
materials directly // referring students to
UiO fagsider // etc.
• On the cataloging side:
• Course codes (e.g. INF2260)
"to catalog or not to catalog"
• More support for Database queries
• Atekst can be found, but pubmed cannot
be found...

29. 3. Learned Lessons (2)
• Importance of "pensum" queries
• Detecting curriculum queries
• E.g. widget which suggests pensum
materials directly // referring students to
UiO fagsider // etc.
• On the cataloging side:
• Course codes (e.g. INF2260)
"to catalog or not to catalog"
• More support for Database queries
• Atekst can be found, but pubmed cannot
be found...

30. Summary & Future work
• Conclusion
• Lots of Oria queries, lots of opportunities for
improvement :-)
• Plans for further analysis
• Comparing queries with most frequent loans
• Alma Analytics
• Looking at common exploratory queries
• medicine; math; united nations; economics
• how can we support those types of queries better?
• Possibility to obtain more data?
• e.g. analyzing "struggling sessions", stats per location, etc

31. ub.uio.no/om/prosjekter/the-visualisation-project
@BookNavigationFind this presentation here

32. Not available, or not found?
Lessons from user queries in
the Oria catalog at UiO
BIBSYS Oria working group meeting (07/12/2016)
Alma/Oria information meeting (08/12/2016)
Hugo Huurdeman, Visual Navigation Project (@BookNavigation)