Download to read offline
More Related Content
Similar to Hawaii Library Association: What Did They Learn
Hawaii Library Association: What Did They Learn
- 1. What Did TheyLearn?: Programmatic Information Literacy Planning & Assessment Melissa Bowles-Terry
- 2. • What dowe teach? • To which groups? • Why?
- 3. Negotiate with yourgroup to decide which outcomes go in which column. Then we’ll present and discuss!
- 4. First-year seminar Communications 1 Communications2 Communications 3 Graduate students services offered by the library find known items by title or author find known items lifelong professional resources (non-UW subscriptions) use research handbooks resources offered by the library come up with a researchable topic and articulate a research question find sources to meet information needs citation mapping and advanced strategies for literature searches citation mapping and advanced strategies for literature searches how to use and/or borrow library materials identify useful keywords use controlled vocabulary use controlled vocabulary use controlled vocabulary library organization find background information, context, and definitions use subject specific databases and/or encyclopedias use subject specific databases use subject specific databases evaluate sources for relevance & authority evaluate sources for relevance & authority evaluate sources for relevance & authority citation management evaluate arguments and research methods evaluate arguments and research methods writing a literature review explain why sources are cited explain why sources are cited introduce major journals in area current awareness practice citing sources cite sources appropriately cite sources appropriately
- 5. Gain access toinformation Understand & evaluate Document sources Information Literacy Curriculum Map I=Introduce, R=Reinforce, M=Master Students will be able to identify sources that will meet their information needs Students will be able to find known items via title or author on the web or in a database. Students will be able to identify and use article databases relevant to their major field of study. Students will be able to find and use resources to meet professional information needs after leaving UW. Students will be able to evaluate information sources for relevance and authority. Students will be able to evaluate a resource’s arguments and research methods. Students will be able to explain why sources are cited in academic writing. Students will be able to cite sources appropriately and consistently. C1 C2 C3 I R M I R M I R I I R M I R I R M I R M
- 6. Learning outcome forall first-year information literacy classes: • Students will be able to identify keywords that represent a research topic and use keywords to find useful information sources.
- 7. Criteria No results –0 Beginning – 1 Relevance Lists no keywords or impossible to tell whether keywords are relevant. Lists keywords that are Lists keywords that are not relevant for the mostly relevant OR lists not research question. enough keywords to express all aspects of the research question. Students = 28 Students = 139 Lists keywords that are Most keywords are meaningless (effects, meaningful and will retrieve impacts, etc.) and/or results on most sides of the keywords that will issue. Does not use natural retrieve biased results language. (negative effects, positive impacts, etc.) OR uses all natural language Students=46 Students=158 Lists citation Lists citation information information for an for an article that is article that is not somewhat relevant, but is relevant to the too general or broad to research topic OR answer research question. citation is incomplete. Students=37 Students=73 Quality Results Students = 7 Lists no keywords. Students = 5 Lists no results or indecipherable results. Students=54 Developing – 2 Exemplary –3 Lists several keywords that express all aspects of the research question. Students =304 Keywords listed will lead student to subject/thesaurus terms. Students=269 Lists citation information for an article that is relevant and appropriately focused for the research topic. Students=314
- 8. Students are quiteadept at coming up with keywords. They are better at coming up with relevant keywords than coming up with quality keywords (for a few reasons I’ll discuss).
- 9. • 54 outof 478 students (over 10%) left the library session without listing one potentially useful article. There may be a few reasons for this: – Students didn’t have enough time to complete the exercise – Students didn’t know enough about the parts of a citation to list a complete citation for an article – Note: In many cases where students didn’t have an article listed, they did have a good search process. Therefore, I do not consider this the sole (or even the most important) measure of success.
- 10. Broad Topic Keywords Non-useful article Final article thatis too technical or advanced Topic: Nuclear Energy Technologies Result: “Transfer of Elements Relevant to Radioactive Waste From *…+ Boreal Plant Species,” from the journal “Chemosphere,” April 2011. No author or page #’s cited. Hit or miss approach (taking first article that comes up) Topic: Effects of Nutrition/Food on the Body Result: “Characterizing Whole Diets of Young Children From Developed Countries and the Association Between Diet and Health: a Systematic Review”
- 11. Focused Topic Keywords Focused article Question with severalparts; statements to prove or disprove Topic: The internet has opened the door to vast amounts of information. What has been the impact of this technology on education? How has it changed the way we learn? Does it tend to enhance or impede critical thinking? Result: “The Effects of Internet Based Destruction on Student Learning.”
- 12. Too many keywords Filler words Phrases • timespent developing synonyms • more than 2-3 main concepts • impacts • benefits • advantages/disadvantages • “checking software for errors” • “autonomous, submersible ROV’s” • “proofreading software”
- 13. •Keywords need tohave Unclear results content Lacks specificity Disadvantages are hard to find •Valuable keywords vs. filler •Make a list of keywords to avoid
- 14. Focus topic or question Search Revise keywords– more specific 2-3 concepts or keywords Search
- 15. • Students performbetter with a research prompt or at least plenty of direction in research question choice. The most difficult AND the most important part of the research process is selecting a topic and narrowing/refining the research question. – Questions or topics are most successful when specific – It is often useful to have students work on solving a problem or to have them prove/disprove a statement – Classes/research projects with a theme (robotics, foodways, American culture) give students more direction for a research project. How can librarians support the process of coming up with a research question?
- 16. • We shouldfocus more on the search process and not just initial keyword selection. Prompting students to search and then search again is really important. – More guidance on worksheet: limit initial keywords to 23, outline a process for revision – Steer students away from quantity and instead work on quality of keywords. (Spending too much time on developing a long list of keywords may in fact be counterproductive.) – Start with broad keywords, and use phrase searching judiciously (if at all) How should we prioritize our time with students?
- 17. • The “hit-or-miss”approach to choosing an article for use is a common issue, and one that is difficult (but important) to address in a 50- or 75-minute session. Students tend to take the first somewhat relevant article that they find. How can we encourage students to persist?
- 18. – How canlibrarians support the process of coming up with a research question? – How should we prioritize our time with students? – How can we encourage students to persist? Melissa Bowles-Terry mbowlest@uwyo.edu
Editor's Notes
- #2 Poll types of libraries: academic? Public? School? Other?Two parts to presentation: 1) defining learning outcomes at a programmatic level and 2) assessing one learning outcome at a programmatic level
- #3 Poll group: What age/level students do you teach? What do you teach them? Why?
- #4 Split into 4 groups—each creates table4 colored packets & tacky stuff for wall10 minutes—set timer for sorting10 minutes—discuss why certain things went in different columnsWhat if you did this at your library? Make a big list of all the things you want to teach students, then sort them into appropriate piles/columns for different levels/classes? Has anyone done this? Want to share results?
- #5 Why and how we decided
- #6 3 big learning outcomes from general education program—critical thinking skillsTalk about introduce, reinforce, masterMoving away from first-year only, inoculation model
- #7 Handout: worksheet andrubric
- #8 We evaluated students’ competency in three areas: 1) identifying relevant keywords, 2) identifying high-quality keywords, and 3) finding a relevant and focused article.We have so far evaluated a total of 26 classes and 478 students. Each student received a 0, 1, 2, or 3 in each of the three areas of criteria and we have compiled the results.
- #10 We’ll look more closely at this by examining two different classes.
- #11 We looked at how the choice of a topic or research question affected students’ overall success and learned that specificity was probably a better criteria for success than the difference between topic and question. Here are a couple of examples of students who did not demonstrate the competencies we were looking for. The first citation seems beyond the student’s intellectual comprehension for this classThe lack of a research question led to a kind of “hit or miss” approach; in other words, whatever the student found they were content with, because there was no specific goal.The most exemplary student results in UWYO 1450 began with the following topics or questions: “ADHD Chemical Nuero-Biofeedback,” “Life on Other Planets/ Non-Oxygen Dependent Lifeforms,” and even “Safer Nursing Habits.”Another thing to investigate – do students understand the elements of citation enough to record the parts on the worksheet? Can we make the worksheet clearer through different language or should we spend time on this?
- #12 The relative success of the ES 1000-07 class was because of several factors. Not only were most of them using questions instead of topics, but the questions had several parts that allowed the students to think through the issue more specifically, some parts being statements to be proved or disproved.
- #13 We looked for a correlation between relevant and high-quality keywords and a relevant and appropriately focused article listed in the results.We saw a few common problems, listed here. There is an overall sense among the ES 1000-07 results that students feel that more keywords will necessarily yield better results. It seems that a few, select (even general, at times) keywords is much more successful than using phrases. Also, when students chose too many keywords and spend too much time on this part of the assignment, this can easily lead to exasperation and frustration, and thus no result.
- #14 25 students (from both sections) with no results listed as citation – all about the process (not just the final article found)One question on worksheet: “How successful was your search?” for students who say “very” – this is not a useful questionStatements of struggle in response to above question are actually more tellingWe saw a few common problems, listed here. The phrases on the left are quotes from students who said that their search was NOT very successful.Unclear results: may come from keywords that don’t mean muchLacks specificity: same kind of problemDisadvantages are hard to find: We might need to spend more time talking about valuable key words versus non-valuable ones. One strategy that I have used in my classes is to make a list of “filler” words like “disadvantages” and leaving it on the board while they proceed with their search.
- #15 I think in the end it is best to encourage students to do a two-step process, using a limited number of terms or phrases (I don’t think it mattered tremendously in the end if it was a term or phrase), and then no matter how successful they feel the terms were (which should always be taken with a grain of salt), they must revise them and use more trial and error. Even among the best examples of choices of keywords, they all could have achieved an article that was more specific to the topic if they went through one more revision of keywords. Most students have the misconception that they should be able to come up with a result with one search. My advice: choose fewer keywords at first, carefully chosen, but not spending too much time on getting the exact right word, then revise and try again (and probably, again). The use of general words that gradually become more specific over several steps seems much more successful than trying too specific of terms in the beginning.
- #16 Concept mapping in ENGL 1010 is useful in this process. Also, the data collected in this project could be convincing if presented to the right groups.
- #17 It seems that students know how to search (not like librarians, but effectively enough). How do you think we should prioritize our time?
- #18 Changed lesson plan and worksheet to emphasize post-searching tasks of refining/evaluating sources. New assessment project.
- #19 I’m happy to continue this conversation. Thank youfor your questions and comments!