The document discusses ways to improve students' information literacy skills, particularly in developing search strategies and selecting relevant sources. It found that students are better at coming up with relevant keywords than quality keywords that will retrieve balanced results. Many students did not list a source, possibly due to time constraints or lack of citation skills. Students performed better with a focused research question provided. The search process should be emphasized over just keywords, focusing on revising searches. Persistence in source selection needs encouragement to avoid a "hit or miss" approach.
3. Negotiate with your group to decide which
outcomes go in which column.
Then we’ll present and discuss!
4. First-year seminar
Communications 1
Communications 2
Communications 3
Graduate students
services offered by the
library
find known items by title or
author
find known items
lifelong professional
resources (non-UW
subscriptions)
use research handbooks
resources offered by the
library
come up with a
researchable topic and
articulate a research
question
find sources to meet
information needs
citation mapping and
advanced strategies for
literature searches
citation mapping and
advanced strategies for
literature searches
how to use and/or borrow
library materials
identify useful keywords
use controlled vocabulary
use controlled vocabulary
use controlled vocabulary
library organization
find background
information, context, and
definitions
use subject specific
databases and/or
encyclopedias
use subject specific
databases
use subject specific
databases
evaluate sources for
relevance & authority
evaluate sources for
relevance & authority
evaluate sources for
relevance & authority
citation management
evaluate arguments and
research methods
evaluate arguments and
research methods
writing a literature review
explain why sources are
cited
explain why sources are
cited
introduce major journals in
area
current awareness
practice citing sources
cite sources appropriately
cite sources appropriately
5. Gain access to information
Understand
& evaluate
Document
sources
Information Literacy Curriculum Map
I=Introduce, R=Reinforce, M=Master
Students will be able to identify sources that will meet their
information needs
Students will be able to find known items via title or author on the web
or in a database.
Students will be able to identify and use article databases relevant to
their major field of study.
Students will be able to find and use resources to meet professional
information needs after leaving UW.
Students will be able to evaluate information sources for relevance and
authority.
Students will be able to evaluate a resource’s arguments and research
methods.
Students will be able to explain why sources are cited in academic
writing.
Students will be able to cite sources appropriately and consistently.
C1
C2
C3
I
R
M
I
R
M
I
R
I
I
R
M
I
R
I
R
M
I
R
M
6. Learning outcome for all first-year
information literacy classes:
• Students will be able to identify keywords
that represent a research topic and use
keywords to find useful information
sources.
7. Criteria
No results – 0
Beginning – 1
Relevance
Lists no keywords or
impossible to tell
whether keywords
are relevant.
Lists keywords that are Lists keywords that are
not relevant for the
mostly relevant OR lists not
research question.
enough keywords to
express all aspects of the
research question.
Students = 28
Students = 139
Lists keywords that are Most keywords are
meaningless (effects,
meaningful and will retrieve
impacts, etc.) and/or
results on most sides of the
keywords that will
issue. Does not use natural
retrieve biased results language.
(negative effects,
positive impacts, etc.)
OR uses all natural
language
Students=46
Students=158
Lists citation
Lists citation information
information for an
for an article that is
article that is not
somewhat relevant, but is
relevant to the
too general or broad to
research topic OR
answer research question.
citation is incomplete.
Students=37
Students=73
Quality
Results
Students = 7
Lists no keywords.
Students = 5
Lists no results or
indecipherable
results.
Students=54
Developing – 2
Exemplary –3
Lists several keywords
that express all aspects of
the research question.
Students =304
Keywords listed will lead
student to
subject/thesaurus terms.
Students=269
Lists citation information
for an article that is
relevant and
appropriately focused for
the research topic.
Students=314
8. Students are quite adept at coming up
with keywords.
They are better at coming up with relevant
keywords than coming up with quality
keywords (for a few reasons I’ll discuss).
9. • 54 out of 478 students (over 10%) left the library
session without listing one potentially useful
article. There may be a few reasons for this:
– Students didn’t have enough time to complete the
exercise
– Students didn’t know enough about the parts of a
citation to list a complete citation for an article
– Note: In many cases where students didn’t have an
article listed, they did have a good search process.
Therefore, I do not consider this the sole (or even the
most important) measure of success.
10. Broad
Topic
Keywords
Non-useful
article
Final article that is too technical or advanced
Topic: Nuclear Energy Technologies
Result: “Transfer of Elements Relevant to Radioactive
Waste From *…+ Boreal Plant Species,” from the journal
“Chemosphere,” April 2011. No author or page #’s cited.
Hit or miss approach (taking first article that comes up)
Topic: Effects of Nutrition/Food on the Body
Result: “Characterizing Whole Diets of Young Children
From Developed Countries and the Association Between
Diet and Health: a Systematic Review”
11. Focused
Topic
Keywords
Focused
article
Question with several parts; statements to prove or
disprove
Topic: The internet has opened the door to vast amounts
of information. What has been the impact of this
technology on education? How has it changed the way
we learn? Does it tend to enhance or impede critical
thinking?
Result: “The Effects of Internet Based Destruction on
Student Learning.”
12. Too many
keywords
Filler
words
Phrases
• time spent developing synonyms
• more than 2-3 main concepts
• impacts
• benefits
• advantages/disadvantages
• “checking software for errors”
• “autonomous, submersible ROV’s”
• “proofreading software”
13. •Keywords need to have
Unclear results
content
Lacks
specificity
Disadvantages
are hard to
find
•Valuable keywords vs.
filler
•Make a list of keywords
to avoid
15. • Students perform better with a research prompt or
at least plenty of direction in research question
choice. The most difficult AND the most important
part of the research process is selecting a topic and
narrowing/refining the research question.
– Questions or topics are most successful when specific
– It is often useful to have students work on solving a
problem or to have them prove/disprove a statement
– Classes/research projects with a theme
(robotics, foodways, American culture) give students
more direction for a research project.
How can librarians support the process of
coming up with a research question?
16. • We should focus more on the search process and
not just initial keyword selection. Prompting
students to search and then search again is really
important.
– More guidance on worksheet: limit initial keywords to 23, outline a process for revision
– Steer students away from quantity and instead work on
quality of keywords. (Spending too much time on
developing a long list of keywords may in fact be
counterproductive.)
– Start with broad keywords, and use phrase searching
judiciously (if at all)
How should we prioritize our time with
students?
17. • The “hit-or-miss” approach to choosing an
article for use is a common issue, and one that
is difficult (but important) to address in a 50- or
75-minute session. Students tend to take the
first somewhat relevant article that they find.
How can we encourage students to persist?
18. – How can librarians support the process of
coming up with a research question?
– How should we prioritize our time with
students?
– How can we encourage students to persist?
Melissa Bowles-Terry
mbowlest@uwyo.edu
Editor's Notes
Poll types of libraries: academic? Public? School? Other?Two parts to presentation: 1) defining learning outcomes at a programmatic level and 2) assessing one learning outcome at a programmatic level
Poll group: What age/level students do you teach? What do you teach them? Why?
Split into 4 groups—each creates table4 colored packets & tacky stuff for wall10 minutes—set timer for sorting10 minutes—discuss why certain things went in different columnsWhat if you did this at your library? Make a big list of all the things you want to teach students, then sort them into appropriate piles/columns for different levels/classes? Has anyone done this? Want to share results?
Why and how we decided
3 big learning outcomes from general education program—critical thinking skillsTalk about introduce, reinforce, masterMoving away from first-year only, inoculation model
Handout: worksheet andrubric
We evaluated students’ competency in three areas: 1) identifying relevant keywords, 2) identifying high-quality keywords, and 3) finding a relevant and focused article.We have so far evaluated a total of 26 classes and 478 students. Each student received a 0, 1, 2, or 3 in each of the three areas of criteria and we have compiled the results.
We’ll look more closely at this by examining two different classes.
We looked at how the choice of a topic or research question affected students’ overall success and learned that specificity was probably a better criteria for success than the difference between topic and question. Here are a couple of examples of students who did not demonstrate the competencies we were looking for. The first citation seems beyond the student’s intellectual comprehension for this classThe lack of a research question led to a kind of “hit or miss” approach; in other words, whatever the student found they were content with, because there was no specific goal.The most exemplary student results in UWYO 1450 began with the following topics or questions: “ADHD Chemical Nuero-Biofeedback,” “Life on Other Planets/ Non-Oxygen Dependent Lifeforms,” and even “Safer Nursing Habits.”Another thing to investigate – do students understand the elements of citation enough to record the parts on the worksheet? Can we make the worksheet clearer through different language or should we spend time on this?
The relative success of the ES 1000-07 class was because of several factors. Not only were most of them using questions instead of topics, but the questions had several parts that allowed the students to think through the issue more specifically, some parts being statements to be proved or disproved.
We looked for a correlation between relevant and high-quality keywords and a relevant and appropriately focused article listed in the results.We saw a few common problems, listed here. There is an overall sense among the ES 1000-07 results that students feel that more keywords will necessarily yield better results. It seems that a few, select (even general, at times) keywords is much more successful than using phrases. Also, when students chose too many keywords and spend too much time on this part of the assignment, this can easily lead to exasperation and frustration, and thus no result.
25 students (from both sections) with no results listed as citation – all about the process (not just the final article found)One question on worksheet: “How successful was your search?” for students who say “very” – this is not a useful questionStatements of struggle in response to above question are actually more tellingWe saw a few common problems, listed here. The phrases on the left are quotes from students who said that their search was NOT very successful.Unclear results: may come from keywords that don’t mean muchLacks specificity: same kind of problemDisadvantages are hard to find: We might need to spend more time talking about valuable key words versus non-valuable ones. One strategy that I have used in my classes is to make a list of “filler” words like “disadvantages” and leaving it on the board while they proceed with their search.
I think in the end it is best to encourage students to do a two-step process, using a limited number of terms or phrases (I don’t think it mattered tremendously in the end if it was a term or phrase), and then no matter how successful they feel the terms were (which should always be taken with a grain of salt), they must revise them and use more trial and error. Even among the best examples of choices of keywords, they all could have achieved an article that was more specific to the topic if they went through one more revision of keywords. Most students have the misconception that they should be able to come up with a result with one search. My advice: choose fewer keywords at first, carefully chosen, but not spending too much time on getting the exact right word, then revise and try again (and probably, again). The use of general words that gradually become more specific over several steps seems much more successful than trying too specific of terms in the beginning.
Concept mapping in ENGL 1010 is useful in this process. Also, the data collected in this project could be convincing if presented to the right groups.
It seems that students know how to search (not like librarians, but effectively enough). How do you think we should prioritize our time?
Changed lesson plan and worksheet to emphasize post-searching tasks of refining/evaluating sources. New assessment project.
I’m happy to continue this conversation. Thank youfor your questions and comments!