The document summarizes the impact and assessment of implementing Summon as the discovery system at the University of Southern California (USC). It discusses two main goals: 1) providing better discoverability of subscribed content through a unified search, and 2) providing relevant search results. Summon was launched in 2010 and indexed over 85% of USC's subscribed databases and journals. This led to increased usage of full-text articles and interlibrary loan requests. However, users still complained about result relevancy. Improving relevancy and linking to full-text remains the top priority to increase use of USC's discovery system.

1. Impact
&
Assessment
of
Summon
@
USC
Beth Namei,
University of Southern California
3/5/2014

2. USC’s Two Major Discovery Service
Implementation Goals:
#1: Provide better discoverability of our subscription and
purchased content (via a unified access point)
#2: Provide relevant results

3. Goal #1: Provide better discoverability of our
subscription and purchased content (via a unified
access point)

4. 757(# of our subscription e-resources)

5. 85% (647)
of our database content is indexed by
Summon

6. 62,797# of journals indexed in Summon

7. OpenURL ClickThroughs to Full-Text
Summon added as
default search option on
the libraries’ homepage
(July 2010)
Summon rolls out Direct
Linking to Full-Text,
bypassing the OpenURL
link resolver (Nov. 2011)

8. Top Referring Sources to Full-text Content (via our
OpenURL link-resolver)

9. In March 2013, 32 A&I databases were added to
USC’s Summon instance.
This led us to revise goal #2:
Provide better discoverability and access to information, via a
single search box, regardless of whether we own or subscribe to
the content.

10. March 2013:
32 A&I
databases were
turned on in
Summon

11. New ILL Accounts Created
There was a
25% increase in
new ILL
accounts after
March 2013,
when USC
added an ILL
button to
Summon

12. Typical complaints about our catalog:
Even if you know the title of the book you're looking for, HOMER searches
often result in random/not relevant results. I find myself going to
WorldCat just to locate books in our library! We need a better search
engine.
...say I was trying to find out if the library has "The Name of War" by Jill
Lepore. If I search "Name of War" or "The Name of War" as the title,
HOMER gives me a list of results that are totally wrong. In fact, the first
result is "Domesticating Vigilantism in Africa." What?? On the other
hand, if I go to Google Books or WorldCat and type in "The Name of
War," Lepore's book is the very first result.

13. April 2010 - Pre-Summon homepage

14. July 2010 - Summon is launched as the default tab

15. July 2012 - Summon-centric homepage (Homer tab is
removed)

16. The catalog is still an option, just a less prominent one:
via a drop-down
menu
an icon below the main
Summon search box

17. Summon added
as default search
tab (July 2010)
Catalog search
tab removed
from homepage
(July 2012)

18. Over 3 Million Summon Searches in 2013

19. How Users are Getting to the Catalog (2013)

20. Success!
We have received no complaints about our
catalog’s (bad) relevancy since July 2012

21. But…users are still not completely satisfied with the
results they’re getting from Summon

22. Usability Study of the Homepage, 2013
The USC’s version of Quick Search just doesn’t generally turn up reliable
results for me. I’ll search a very obvious keyword or a very specific keyword
and it won’t turn up the most relevant results first even though I know the
highly relevant results are in there it won’t bring them up so it’s probably a
backend USC libraries problem with the Quick Search function. But because
it’s difficult to narrow down by content and by type that I want and because
the results just aren’t that relevant or they don’t turn up the best results first,
I tend not to use Quick Search. Unless I’m feeling really, really lazy and I
have to turn something in in 20 minutes and I’m like ‘whatever’s on top, I’ll
take that!

23. What impact has implementation of Summon
had on library instruction and reference service
at USC?

24. Results from instruction survey
Surveyed 47 instruction/reference librarians; 25 responded
(53%)
• 14 out of 15 librarians who teach introductory library sessions
for freshman writing courses teach Summon:
o 1 does not teach Summon at all
o 3 spend 1/4 of the class session demonstrating it (10-20 minutes), but also
demonstrate 1-2 (or more) other databases/tools.
o 6 briefly demonstrate it (no more than 5 minutes) before moving on to
demonstrating other databases/tools (most of these respondents said they
felt obliged to address it since it was so prominent on our website)
o 5 spend the majority of their teaching time demonstrating it

25. Do you teach
Summon in upper
level/subject specific
courses?
Yes: 12
No: 9
n/a: 4
Do you use Summon at
the Reference Desk?
Yes 11
No: 12
n/a: 2

26. In what ways (if any) has incorporating Summon into
your instruction sessions changed your teaching?
A wide range of responses:
• hasn’t changed my instruction at all
• allows for more time on other resources
• I spend less time with other [resources] to incorporate this one
• allowed for less lecture time spent on our webpage and what a database is and
more focus on content type, evaluations, building a search. Also: I have
significantly cut down my lecture time to allow them to figure out these things
through hands on activities.
• I spend more time talking about the information cycle and what the different
source types mean. I think it's allowed me to focus a little more on the higher-
order skills of research because they don't have to be bogged down in the tool.

27. What is the single biggest factor that would
increase use of USC’s discovery system?

28. better relevancy
and
better linking