This presentation was provided by Joyce Valenza of Rutgers University, during the NISO event "Transforming Search: What the Information Community Can and Should Build." The virtual conference was held on August 26, 2020.

1. SLIDESMANIA.COM
Things I want my
STUDENTS
to know
about search

2. SLIDESMANIA.COM
Things I want
SEARCH SYSTEMS
(and publishers)
to know about my
students

3. SLIDESMANIA.COM
(and a couple of wishlists)

4. Joyce Kasman Valenza, Ph.D.
Associate Teaching Professor, Rutgers SC&I
my blog:
http://blog.schoollibraryjournal.com/neverendingsearch/
my tweets: @joycevalenza
my deets: http://aboutme.com/jvalenza
this preso: https://tinyurl.com/NISOsearchfuture

5. Who am I?

7. END USER
Where I am in
the ecosystem

8. SLIDESMANIA.COM
two approaches
interventions

9. SLIDESMANIA.COM
stories from the field

10. SLIDESMANIA.COM
approach #1
student/user
interventions

11. SLIDESMANIA.COM
my students
high
school
LIS

12. SLIDESMANIA.COM
“
“Only librarians like to search, everyone else likes to find.”
Tennant, R. 2001. “Cross Database Search: One-Stop Shopping.” Library Journal 15 Oct.: 25.

13. SLIDESMANIA.COM
On the other hand . . .

14. 14
Again?
You’re kidding! Adam

15. 15
He’s right!
Adam

16. higher
expectations

17. Adam naming the animals. Etching. Wellcome, engraving V0034186.jpg
Children feel pride in naming things as they learn.
Giving something a name makes it real. It allows us to talk about it
and consider it explicitly in our intellectual toolkits.

18. Crossing thresholds

19. A threshold concept can be considered as akin to a
portal, opening up a new and previously
inaccessible way of thinking about something. It
represents a transformed way of understanding, or
interpreting, or viewing something without which
the learner cannot progress.
Meyer, Jan, and Ray Land. Threshold concepts and troublesome knowledge: Linkages to ways of
thinking and practising within the disciplines. Edinburgh: University of Edinburgh, 2003.

20. Authority Is Constructed and Contextual
Information Creation as a Process
Information Has Value
Research as Inquiry
Scholarship as Conversation
Searching as Strategic Exploration

21. Professional searchers today: Leadership roles in design & information experience
Tucker, Virginia M. "Sharpening the search saw: Lessons from expert searchers." SLIS Student Research Journal 5.1 (2015): Article-number.
Tucker identifies four threshold concepts for search expertise:
1. Information environment: the total information environment is perceived and
understood;
2. Information structures: content, index structures, and retrieval algorithms are
understood;
3. Information vocabularies: fluency in search behaviours related to language,
including natural language, controlled vocabulary, and finesse using proximity,
truncation, and other language based tools.
4. Concept Fusion: the integration of the other three threshold concepts. Additional
properties: visioning (anticipating next moves), being light on one’s ‘search feet’
(dancing property), and profound ontological shift (identity as searcher).

24. 24

27. infographic
table
figure
mapdata / data set
report
review
essay/op-ed
whitepaper
preprint

29. You are not just browsing and
storing promising results.
You are on the lookout for killer
articles, sentences, phrases, terms,
publications issues and authors.

34. descriptors
Controlled vocabulary
subject
headings
thesauri
authority
control
index
terms
34
And when free text matters!

35. When CV works

36. When CV doesn’t work (it does in MeSH)

38. the value of free text
emerging language
jargon
popular culture

40. SLIDESMANIA.COM
My LIS students

41. Jennifer’s checklist:
Exposure to these strategies gives us a menu of options, a sort of
checklist we can pull out when we're engaged in a complex search.
Have you conducted a research interview with yourself to properly
define the search and its facets? Check!
Can you do a search for a known item and mine it for leads? Check!
Is a particular author a rock star with her own Google Scholar
profile? Check!

42. Making
the case
for
databases

44. discovery
individual,
focused
databases
What to pick?

45. discovery
vs.
focused,
individual
databases

46. default vs. decision
Intentionality

47. domain expertise
Noun project. Icon54 images.
wizard at
everything--search &
domain knowledge
completely
clueless
I’ve got the
search chops!
Wish I had
context!!
master of my
domain, but . . .
Does discovery work equally well for both novice & expert users?
search
system
expertise

49. Did you know what “personalize” meant?

50. Celebrating contextuality!

57. Primo Resource Recommender

58. SLIDESMANIA.COM
My LIS students
(and a colleague)
on discovery

59. Megan: On bento-style results
I was skeptical about the bento-box approach until I actually took a look at one.. . . I was
afraid it would fragment the results too much.. . .
I went to the Cornell University Library webpage and searched for “homelessness” in the
discovery box. The results that appeared were in boxes for books, articles and full-text
(presumably across their databases), non-musical recordings, journals/periodicals, digital
collections, and research guides. This made it very clear to me which resources might best
apply to my search (especially such a vague search). It also gave me some ideas for other
types of sources that I might use to refine a future search. I wouldn’t have thought to
search for non-musical recordings about homelessness until I saw that was a category.

62. My students are pretty skilled in identifying the right database for the task at hand.
We spend significant instructional time on teaching kids how to use various
resources and being familiar enough to know which one(s) will most quickly meet
the tasks their teachers set. For these learners, it’s all about speed.
They do not want to browse through lots of stuff; they do not want to play with the
facets. They want to find the textual information and media that will get the job
done. Now. So, for the past few years, it’s been challenging to get students and
teachers to fully buy-in to discovery. My teachers report that students are not
finding the quality stuff they know lives in other spots. So, I’m continually rethinking
my instructional and promotional approaches to EDS; because it’s quite expensive (I
could buy 2-3 additional databases for the same cost), it may soon be on the
endangered list at my school. :-(

64. SLIDESMANIA.COM
newish things my
students like

65. Support for
reading and
language
differences

66. Visualizations
to promote
discovery

67. http://carrot2.org/

69. http://yippy.com/

71. Text Analyzer: Power of AI for
discovering connections and
addressing disciplinarity

72. Easy access to stuff inside and article and the stuff about the article & AI citation connections and impact

73. 73
https://academic.microsoft.com/

77. Google Talk to Books: Like a librarian?
AI-driven, book results for natural language questions
Great for beginning inquiry projects and lit reviews, scanning nonfiction titles

78. https://books.google.com/ngrams

79. https://books.google.com/ngrams

81. When you start in Scholar

82. Leveraging available metadata for discovering
research connections

83. Access Tricks
http://gettheresearch.org
preprints and repositories

84. SLIDESMANIA.COM
findings from our research

85. Made possible in part by the Institute of Museum and Library Services, grant number LG-81-15-015

86. RSIC RESEARCH TEAM
Tara Cataldo, MLS
Co-Principal Investigator
Rachael Elrod, M.Ed., MSLS
Investigator
Randy Graff, PhD
InvestigatorKailey Langer
Research Assistant
Lynn Silipigni Connaway, PhD
Co-Principal Investigator
Ixchel Faniel, PhD
Co-Principal Investigator
Joyce Valenza, Ph.D.
Co-Principal Investigator
Erin Hood, MLIS
Investigator
Brittany Brannon, MA
Research Assistant
Joyce Valenza, PhD
Co-Principal Investigator
Amy Buhler, MSLS
Principal Investigator
Samuel Putnam, MLS
Investigator
Chris Cyr, PhD
Investigator

87. Helpful? Cite?
Credible? Container?
Our study examined the behaviors and choices of 175 student
participants, across six educational stages, as they engaged in four
tasks in response to a grade-appropriate science research prompt
using a simulated Google SERP.

88. SLIDESMANIA.COM
What have we learned?

89. “Common Thread” resources included:
● Undergraduate essay
● Google Book
● Nature article
● National Park Service Document / Fact Sheet
● Wikipedia entry
● Ad for airboat tour
● Weebly site by environmentalist group
@RSICStudy

90. In the old days . . .

91. container collapse
Container collapse, a term coined by the RSIC research team, describes the flattening or obscuring of
information sources from the print containers that once provided visual context and cues to help
individuals identify the documents’ origins.
In digital format, a document is “decanted” from its original container and must be carefully examined
for publishing indicators to determine the journeys it took to reach the individual user.
One surprising finding was that many students did not identify Wikipedia as an encyclopedia, often
choosing to label it as a blog because of its openness.
genres and containers?

94. Students have some understanding of containers
● 86% (151 out of 175) of the students believed it was important to know
the container of online information resources.
● Many higher education students see container as an aid in credibility
assessments. “For the sake of citing, you have to know what you're citing
from.”
● They identified only an average 51% of containers correctly.
● Even at the graduate stage, none were able to correctly identify every
container.
● Some students were incorrect on almost all of their container judgments.

95. think alouds:
what they said
Wikipedia

96. On Wikipedia:
“Nobody really cares
enough about science
sources to make them
wrong.”
“I know this is a blog
because it’s by a
person that wrote
it.”
“Springer articles
are inherently
credible.”
“This one’s a blog. Oris it a journal? Or is it
a book? It’s a
journal.”
“This shouldn’t
be so hard.”
“What is a
preprint?”
“If it has a doi, it’speer-reviewed.”
“This is a journal …
because of the feel of
it looking like ascholarly article.”
But . . .

97. Journal. A scientific journal. Is that what that means? Instead of
like a diary. That would be blog. These words these days
mean-- [laughter]. That would be just a website. Journal? I feel
like that might have actually just been a website, but it's in the
form-- yeah, with the abstract, it's in the form of a journal.
Invasive, journal. I want to know if I'm wrong
community college on containers

98. This is a PDF, so it's a-- this is a-- I'm just
going to say journal. Oh, I called this a
journal . . . Actually, it could be a book.
It's not a book.
undergraduates on containers
No, I never use books. I use
books if I have to. I had a
research paper probably a
year ago, and I had to use
books so I went to the library
and got the books. But
otherwise, I probably would
never go to the library, and
actually check out books
that have anything to do
[with anything?].
Note: the ebooks were online!

99. Students have some understanding of containers
● 86% (151 out of 175) of the students believed it was important to know
the container of online information resources.
● Many higher education students see container as an aid in credibility
assessments. “For the sake of citing, you have to know what you're citing
from.”
● They identified only an average 51% of containers correctly.
● Even at the graduate stage, none were able to correctly identify every
container.
● Some students were incorrect on almost all of their container judgments.

100. G11 52:29-Totally. This is a journal. Journals are
generally [very very good], and their articles also.
I think they are generally the highest in terms of
reliability.
Springer, Nature are quite popular. They have
review committees, although of late there have
been questions arising. I don't think there are
better sources than these guys. So my trust on
them is generally higher. Their quality is good.
container and brand heuristics

101. Across grades levels--grades 4 through graduate school--we saw a progression of thinking, some hidden
behaviors, and a bit of resentment about how parents, teachers and librarians address Wikipedia as a resource.

102. what to read vs. what to cite
Note: It’s okay to do both!

104. I have high confidence in the material because I know that the
people who contribute (I’m one, as are my students) are often
quite well versed in the subject. Most importantly, the changes
are transparent – I can look through the history to read the
discussion, much the way I can in open peer reviews.
University science professor on Wikipedia’s usefulness::
Connaway, L.S., Valenza, J.K, Cyr, C. Cataldo, T. (2019). Authority, Context and Containers:
Student Perceptions and Judgments When Using Google for School Work. IFLA Libra

105. Community College talking points about Wikipedia
So I know that a lot of people say Wikipedia
is unreliable, but I always click Wikipedia first
because even though you shouldn't cite the
information it's a good way to kind of get a
broad idea of what's going on. So I just kind of
learn-- I feel like you can just learn something
off of the Wikipedia page.
I wouldn't use Wikipedia just because I know it's not
scholarly. Now, me personally, I would use it. If I was talking
to a friend of mine. I would tell them to check out this page
just because of the references. And, personally, I think that
Wikipedia kind of gets downed a little bit, but I know that
most colleges don't, and I don't think my teacher would
enjoy that, so I would not use Wikipedia in my report.
Okay. For the record, I think
Wikipedia is great, and I think
we need to change our tune
about it. I think it's highly
credible.
Stop the Wikipedia shaming!
I hate that it's shamed . . .. I've
actually found it to be really concise
and kind of like a little nugget of
information that allows me to
explore further. And poor Wikipedia,
and I'm upset that people are mean
about it.

106. graduate talking points about Wikipedia
I came for understanding.
I maybe shouldn’t love it, but
I love Wikipedia.
So Wikipedia, I would not choose. Well, I'm lying. I would
choose Wikipedia.
So, even Wikipedia, I would absolutely never cite it,
but in terms of the information that I'm reading from
it-- well, that's hard to say. So, man, that's a very hard
question because I'm thinking about it. . . .I would say
it's highly credible, because the citations, I went and
looked at the journal peer-reviewed citations, and it
backs up everything that it says. . . But that does
mean I would cite it? No, because I would go and cite
the actual papers [laughter].
Wikipedia, while a lot of people use it as a
first just understanding of a thing--
sometimes if it's something I have no
familiarity with, maybe I'll look at the
Wikipedia page, but again, it's not a
reputable source.

107. Many students confessed the covert and
embarrassed use of Wikipedia, a situation that
might best be described as Wikipedia shaming.

108. Findings:
● While experts on the study’s advisory panel were more likely to judge the Wikipedia resource at the
individual article level, taking the time to note the scope, authority and quality of references that
supported a specific entry, few students made credibility judgments at the document or article level,
● Acknowledgment of Wikipedia’s helpfulness rises across their educational stage. Acknowledgment of
its citability dips.
● In the case of Wikipedia, young students were willing to ignore their prevailing focus on aboutness
and consider a decision about authority, seeing its use as a good/bad binary choice —a choice that
parents and teachers advised them to avoid.
● Students had difficulty classifying Wikipedia as an encyclopedia. Genre bending, like other web
sources?
● Students in the higher educational levels described the importance of Wikipedia for quickly
developing critical context, discovering vocabulary in an area of new knowledge and gathering
references to scholars’ and experts’ work. Yet, their language was confessional for this use as they
described feelings that might best be described as Wikipedia shaming.

111. 111
Understanding
the news cycle

112. Beyond container awareness to
awareness of an information genre
ecosystem connected to
information/inquiry need:
reference/background
news/updates/alerts
scholarship
data
perspective/viewpoints

113. Some students come to school with knapsacks fully
packed.
Each morning they arrive at school ready to go, relying
on the presence of a set of conditions that support their
academic success. For other students, that pack may be
empty.
For educators, the recognition of information
privilege–that one student has clear advantage over
another–is a call to action, pointing to our
responsibilities to reflect on the disparities in
information access present in our own communities, to
raise awareness, and to work to close gaps.

114. Elementary/Middle School
I had access to books written in my home language
I had access to collections of books in multiple formats
I had access to books written about my culture
I had access to wifi after school and on weekends
I had a device devoted to my use for homework
I had access to a rich collection of print and digital
resources that met my personal reading interests
I learned to use digital tools to create media-rich
communication and knowledge projects
I had access to to high quality collections of ebooks,
journal, newspaper and reference databases like Gale,
Capstone, EBSCO, ProQuest, ABC-CLIO, etc.
I had instruction in safe, legal and ethical practices with
digital tools and the protection of intellectual rights and
property
I used devices on which an assortment of high quality
apps to support my creative work were curated
I had an adult to advocate for digital resources to support
my learning styles and special concerns
My access to information is not blocked by paywalls
High School
I was expected to engage in inquiry based research in my
courses
I had access to AP courses in my high school
I was expected to thoughtfully analyze and evaluate
data, media and text sources on a regular basis
I was expected to credit sources I used in my projects
I learned about how to respect and ethically use the
intellectual property of others in my own digital
communications
I had access to high quality collections of ebooks, journal,
newspaper and reference databases like JSTOR and those
offered by Gale, EBSCO, ProQuest, ABC-CLIO, etc.
I learned to use digital tools to ethically create and share
media-rich communication and knowledge projects
I learned how to use social media critically to find and
share information
I learned about Creative Commons, CC0, Public Domain
and Fair Use and the open access movement,
My access to information is not blocked by paywalls
Information
Privilege
Backpack

115. SLIDESMANIA.COM
approach #2
system
interventions

116. SLIDESMANIA.COM

117. grey
literature
OER
social
media
video
disciplinarity
traditional
peer-reviewed
journals
repositiories
& portals
Discovery Issues:
How might we unify siloed content and allow users to dig into their depths,
without losing the integrity of the lovely proliferation of interfaces, data and
formats?
dataebooks
audiobooks

118. grey
literature
OER
social
media
video
disciplinarity
traditional
peer-reviewed
journals
repositiories
& portals
Access issues:
● Users don’t always begin on library pages.
● Federated authentication/authentication solutions?
● Single sign-on systems for your browser
● Geolocation (for public libraries)?
● Broader link resolvers?
● Pointers to OA?
dataebooks
audiobooks
Free Web
search tools

119. Joyce’s wishlist
(Part 1 access)
Compensate for information silos--formats, OER, media, across
databases & repositories
Multimodal search--value of different formats for different tasks
Clear scaffolding for cross-disciplinary exploration
Search inclusive of social media, updates, feeds, etc.
Will mobile-first/consumers-first approaches serve researchers?
ADA Compliance (Immersive Reader integration for language and
reading support/accessibility)
Diversity-acknowledging algorithms (those brides in Nature!)
Geolocation links for public and school library users--remove
barcode barriers!
And . . . how will you protect my data?

120. Joyce’s wishlist (Part 2: discovery)
Assistive, predictive, proactive research inquiry that resembles my shopping experience?
Can search move beyond the individual transaction to multilayered conversations?
(Research is not linear. It’s iterative and transformative.)
Database/journal/resource recommender from Discovery systems
Automatic facet detector: identifying and remembering my search facets
Attractive visually clear identifiers for types of articles, containers
Scholarship as conversation indicators. Visualizing connections. Who’s citing whom?
Deep digging into text of books and articles (like Google Talk to Books)
Abstracts: Could they be diagrammed or sketch-noted?
Use AI to better bucket my results and browsing options (more and better bentos?)
Knowledge graph for research (for Scholar?), not just answers.
Can knowledge graphs visualize my research query?
1

121. SLIDESMANIA.COM
Finale!
My LIS students
reflect on search
in memes

135. (Librarians should act as canaries in coal mines.)

137. Joyce Kasman Valenza, Ph.D.
Associate Teaching Professor, Rutgers SC&I
my blog:
http://blog.schoollibraryjournal.com/neverendingsearch/
my tweets: @joycevalenza
my deets: http://aboutme.com/jvalenza
this preso: https://tinyurl.com/NISOsearchfuture

138. http://library.ifla.org/2508/

139. Buhler, A., Cataldo, T. T., Faniel, I. M., Connaway, L. S., Valenza, J. K., Graff, R., Elrod, R., Putnam S., Cyr C., Towler, C., Hood, E., Fowler
R., Howland S., Brannon B., Langer, K., Kirlew, S. (2015-2019). Researching students’ information choices: Determining
identity and judging credibility in digital spaces. IMLS Grant Project LG-81-15-0155. http://guides.uflib.ufl.edu/RSIC.
Buhler, A., Faniel, I., Brannon, B., Cyr, C., Cataldo, T., Connaway, L., Valenza, J., Elrod, R., Graff, R., Putnam, S. R., Hood, E. M., &
Langer, K. (2019). Container collapse and the information remix: Students’ evaluations of scientific research recast in
scholarly vs. popular sources. Paper presented at the Association of College and Research Libraries (ACRL) Conference,
Cleveland, OH. Retrieved from
http://www.ala.org/acrl/sites/ala.org.acrl/files/content/conferences/confsandpreconfs/2019/ContainerCollapseandInformati
onRemix.pdf
Connaway, L., Valenza, J., Cyr, C., Cataldo, T., Buhler, A., Faniel, I., Elrod, R., Graff, R., Putnam, S., Brannon, B., Hood, E., & Langer, K.
(2019). Authority, context, and containers: Student perceptions and judgments when using Google for school work. Paper
presented at the International Federation of Library Associations (IFLA) World Library and Information Congress (WLIC),
Athens, Greece. Retrieved from http://library.ufla.org/2508/1/207-connaway-en.pdf
Cyr, C., Cataldo, T., Brannon, B.,et al. (2020) Backgrounds and Behaviors: Which Students Successfully Identify Online Resources in
the Face of Container Collapse. [In progress]

140. http://guides.uflib.ufl.edu/RSIC/home