Assessing for Improvement


                Diane Harvey
                NCLA-BIG Workshop
                May 21, 2010


...
Learning outcomes for today

At the end of this session, participants will be able to:

1. Describe the learning outcomes ...
Assessment climate in
                  higher education
                             3

 Spellings Commission
 http://ww...
SACS standards
                            4

 3.3.1 …identifies expected outcomes, assesses the
 extent to which it achi...
Assessing for improvement
                 5




We don’t assess to prove…



            …but to improve.
               ...
Assessment in libraries
                          6




 Focus on collections and services (e.g. ARL
 LibQUAL)

 Focus o...
What is LOA: learning outcomes assessment?
                           7




 Systematic look at what students are learnin...
What LOA is NOT
                          8




 NOT evaluation of teaching


 NOT evaluation of program
Why assess student learning in libraries?
                           9




 Affirm commitment to student learning


 Imp...
Assessment loop


                Develop student
                   learning
                  outcomes


Work with
 resu...
Levels of assessment

 Institutional (University wide)


 Program (e.g. Department)


 Course


 Session (e.g. IL inst...
LOA in libraries: constraints
                              12




 Lack of consistent, reliable access to students


 I...
Know your IL program!
                            13


 Do you offer a for-credit IL course?


 Do you provide IL instru...
Map your IL program
             14


       First Year Writing
    and Freshman Seminars



   Subject specific instructi...
Where will you assess student learning?
                             15




 On the instruction session level?


 In a f...
Remember…
                          16




Library instruction doesn’t work the same way as
  instruction in an academic d...
17
18
Some simple approaches
                                   19



Develop learning outcomes for your instruction program, an...
Snapshots
                          20




Outcome: Students will be able to distinguish between
 popular and scholarly jo...
What is a learning outcome?

A learning outcome is one sentence that
  indicates what students should
  represent, demonst...
Good learning outcomes:

 Focus on what students will learn/know/be
    able to do
   Describe actions or behaviors
   ...
ACRL Standards

 3 levels: standards, performance
  indicators, outcomes

 Can be used for LOA


 Examples: (1)Identifi...
Writing outcomes
                     24




Today, we’ll write learning outcomes for a
  library instruction session.
Exercise: What do you cover?

Think of an information literacy instruction
 session that you teach on a regular basis.

Li...
Bloom’s Taxonomy

 Classification of educational objectives
 Published in 1956, revised in 2001 (changes: noun to
 verb,...
Bloom’s levels (lowest highest)

 Knowledge/Remembering
 Comprehension/Understanding
 Application/Applying
 Analysis/...
Verbs for information literacy

 Knowledge/Remembering: define, list, recognize
 Comprehension/Understanding:characteriz...
Verbs to avoid
                   29



Understand              Become familiar with

Appreciate              Learn about
...
Learning outcomes formula

 1. Time frame
 2. Student focus
 3. Action verb
 4. Product/process/outcome




          ...
Using the formula

 Time frame: “At the end of the library
  session…”
 Student focus: “…students will be able to…”
 Ac...
Another example:

 Time frame: “After completing the online
  tutorial…”
 Student focus: “…students will be able to…”
 ...
One more example:

 Time frame: “By the time they complete their
  undergraduate program…”
 Student focus: “…students wi...
Examples: bad and better

Bad: Students will understand how to use social science
  databases.
Better: Students will perfo...
Now, write your own!

Choose two items from the list of what you cover in
 an information literacy session.

Transform the...
Checklist for learning outcomes

 Includes a time frame?
 Focuses on students?
 Uses action verbs?
 Names a product or...
Sample size
                             37

Use any convenient sample that will give you useable
 information.

Make sure...
Assessment Methods
         38




   Ask yourself

 “How will I know?”
Assessment methods

 Knowledge test
 One Minute paper & variations
 Bibliography analysis
 Concept Inventory
 Standar...
Knowledge tests
                       40

 Tests knowledge and/or skills before and/or after
  library instruction sessi...
One minute paper & variations
                             41

Sample questions:
1. What is the most important thing about...
Bibliography analysis
                           42



 Look for citations from scholarly/peer reviewed
 journals

 Look...
Concept inventory
                            43




Make a checklist of 3-12 important concepts students need
 to master....
Standardized tests
                         44




 I-Skills


 ILAT (James Madison University)


 SAILS
I-Skills
                           45




 Developed by ETS
 7 ICT proficiencies
 Tracked to ACRL standards
 Task bas...
SAILS: Standardized Assessment of Information Literacy
                                                                   ...
ILAT – James Madison University
                         47




 Web based
 60 items
 Tests ACRL standards 1,2,3 and 5
Using assessment information

 Rewrite learning outcomes
 Change what you do/how you teach
 Work collaboratively with c...
Learning outcomes are:


 Fundamental components of a learning
  assessment program
 Focused on student learning
 Indic...
Learning Outcomes Assessment
             50




  A tool to help librarians
 improve student learning
LOA resources

 Angelo, Thomas (1993). Classroom assessment techniques.
 Maki, Peggy (2004). Assessing for learning : bu...
Let’s assess this workshop!




            52
Thank you!


      Diane Harvey
      Head, Instruction & Outreach
      Perkins Library
      Duke University
      diane...
Upcoming SlideShare
Loading in …5
×

NCLA Learning Outcomes Assessment Workshop

2,867
-1

Published on

Workshop for North Carolina Library Association Bibliographic Instruction Group, May 21, 2010, UNC-Greensboro

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,867
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
52
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

NCLA Learning Outcomes Assessment Workshop

  1. 1. Assessing for Improvement Diane Harvey NCLA-BIG Workshop May 21, 2010 1
  2. 2. Learning outcomes for today At the end of this session, participants will be able to: 1. Describe the learning outcomes assessment process in libraries. 2. Write learning outcomes for an information literacy instruction session. 3. Choose appropriate assessment methods. 4. Define ways to work with assessment results. 2
  3. 3. Assessment climate in higher education 3  Spellings Commission http://www.ed.gov/about/bdscomm/list/hiedfuture/index.ht ml  Voluntary System of Accountability (VSA) http://www.voluntarysystem.org/  Accrediting Agencies http://www.sacs.org/
  4. 4. SACS standards 4  3.3.1 …identifies expected outcomes, assesses the extent to which it achieves these outcomes, and provides evidence of improvement based on analysis of the results  3.8.2 …ensures that users have access to regular and timely instruction in the use of the library and other learning/information resources.
  5. 5. Assessing for improvement 5 We don’t assess to prove… …but to improve. D. Stufflebeam
  6. 6. Assessment in libraries 6  Focus on collections and services (e.g. ARL LibQUAL)  Focus on standards (e.g. ACRL IL Competency Standards)
  7. 7. What is LOA: learning outcomes assessment? 7  Systematic look at what students are learning  Moving from “What am I going to teach today?” to “What do I want students to learn today?”
  8. 8. What LOA is NOT 8  NOT evaluation of teaching  NOT evaluation of program
  9. 9. Why assess student learning in libraries? 9  Affirm commitment to student learning  Improve effectiveness of instructional programs  Align with campus, disciplinary, and higher ed efforts  Be recognized on campus for contributions to student learning
  10. 10. Assessment loop Develop student learning outcomes Work with results Set criteria Do assessment Devise assessment measures 10
  11. 11. Levels of assessment  Institutional (University wide)  Program (e.g. Department)  Course  Session (e.g. IL instruction) 11
  12. 12. LOA in libraries: constraints 12  Lack of consistent, reliable access to students  IL instruction: is it a “program”?  Program assessment vs. session assessment  Difficult to isolate effects of library instruction
  13. 13. Know your IL program! 13  Do you offer a for-credit IL course?  Do you provide IL instruction for large programs (e.g. freshman writing)?  Is IL integrated into the curriculum?  Do you offer IL for capstone, thesis or senior seminar courses?
  14. 14. Map your IL program 14 First Year Writing and Freshman Seminars Subject specific instruction Honors capstone seminars
  15. 15. Where will you assess student learning? 15  On the instruction session level?  In a for-credit library skills course?  Across a series of library instruction sessions?  Broadly across the student population?
  16. 16. Remember… 16 Library instruction doesn’t work the same way as instruction in an academic department. Know the shape of your program – be able to explain how and why you assess student learning.
  17. 17. 17
  18. 18. 18
  19. 19. Some simple approaches 19 Develop learning outcomes for your instruction program, and look at where they are taught – “snapshots” Focus on large programmatic efforts e.g. first year writing Highlight specific academic departments Assess individual library instruction sessions – develop shared learning outcomes, compare/contrast
  20. 20. Snapshots 20 Outcome: Students will be able to distinguish between popular and scholarly journals. Assessment: pre/post test in freshman English, one minute paper in senior capstone
  21. 21. What is a learning outcome? A learning outcome is one sentence that indicates what students should represent, demonstrate or produce as a result of what they learn. - source: Peggy Maki 21
  22. 22. Good learning outcomes:  Focus on what students will learn/know/be able to do  Describe actions or behaviors  Are results oriented  Are observable and measurable  Include a time frame 22
  23. 23. ACRL Standards  3 levels: standards, performance indicators, outcomes  Can be used for LOA  Examples: (1)Identifies keywords, synonyms and related terms for the information needed. (2) Selects an appropriate documentation style and uses it consistently to cite sources 23
  24. 24. Writing outcomes 24 Today, we’ll write learning outcomes for a library instruction session.
  25. 25. Exercise: What do you cover? Think of an information literacy instruction session that you teach on a regular basis. List what you would cover in that session. 25
  26. 26. Bloom’s Taxonomy  Classification of educational objectives  Published in 1956, revised in 2001 (changes: noun to verb, synthesis/creating becomes highest level)  Taxonomy = classification  Cognitive levels (lower  higher)  Provides way to express outcomes 26
  27. 27. Bloom’s levels (lowest highest)  Knowledge/Remembering  Comprehension/Understanding  Application/Applying  Analysis/Analyzing  Evaluation/Evaluating  Synthesis/Creating 27
  28. 28. Verbs for information literacy  Knowledge/Remembering: define, list, recognize  Comprehension/Understanding:characterize, describe, explain, identify, locate, recognize, sort  Application/Applying: choose, demonstrate, implement, perform  Analysis/Analyzing: analyze, categorize, compare, differentiate  Evaluation/Evaluating: assess, critique, evaluate, rank, rate  Synthesis/Creating: construct, design, formulate, organize, synthesize 28
  29. 29. Verbs to avoid 29 Understand Become familiar with Appreciate Learn about Know about Become aware of
  30. 30. Learning outcomes formula  1. Time frame  2. Student focus  3. Action verb  4. Product/process/outcome 30
  31. 31. Using the formula  Time frame: “At the end of the library session…”  Student focus: “…students will be able to…”  Action verb: “…identify…”  Product/process: “…a relevant database for their term paper research.” 31
  32. 32. Another example:  Time frame: “After completing the online tutorial…”  Student focus: “…students will be able to…”  Action verb: “…differentiate between…”  Product/process: “…scholarly journals and popular magazines.” 32
  33. 33. One more example:  Time frame: “By the time they complete their undergraduate program…”  Student focus: “…students will be able to…”  Action verb: “…construct…”  Product/process: “…a research question that can be investigated using primary archival resources.” 33
  34. 34. Examples: bad and better Bad: Students will understand how to use social science databases. Better: Students will perform a search in Social Sciences Abstracts that retrieves relevant items. Bad: Students will be able to search the catalog. Better: Students will use the catalog to identify a book on their topic. Bad: Students will appreciate the importance of correct citation. Better: Students will produce citations in correct APA format. 34
  35. 35. Now, write your own! Choose two items from the list of what you cover in an information literacy session. Transform them into learning outcomes, using the formula. Find a partner and critique your outcomes using the following checklist. 35
  36. 36. Checklist for learning outcomes  Includes a time frame?  Focuses on students?  Uses action verbs?  Names a product or process?  Is measurable/observable?  Prompts a measure/method?  Will be useful for you to assess? 36
  37. 37. Sample size 37 Use any convenient sample that will give you useable information. Make sure that groups of interest are represented. Sample size is important only if you plan to publish your results. Do you want to generalize to the entire student population?
  38. 38. Assessment Methods 38 Ask yourself “How will I know?”
  39. 39. Assessment methods  Knowledge test  One Minute paper & variations  Bibliography analysis  Concept Inventory  Standardized test 39
  40. 40. Knowledge tests 40  Tests knowledge and/or skills before and/or after library instruction session.  Can be given at end of library session or later in semester.  Can use clickers to gather data during library session. Sample questions: 1. What is the difference between a library catalog and a database? 2. The Boolean operator “or” narrows a search statement (true/false).
  41. 41. One minute paper & variations 41 Sample questions: 1. What is the most important thing about library research you learned today? 2. 3-2-1 (three things you learned, two things you’re still confused about, one thing you’d change about session) 3. What is one question you still have? 4. In your research, what will you do differently after today’s session?
  42. 42. Bibliography analysis 42  Look for citations from scholarly/peer reviewed journals  Look for citations for books and journals owned by your library  Look for articles retrieved from your databases.
  43. 43. Concept inventory 43 Make a checklist of 3-12 important concepts students need to master. Ask students to explain each concept in a sentence or two. If a concept is unfamiliar, they should leave blank. Count good responses for each concept, then plan future instruction.
  44. 44. Standardized tests 44  I-Skills  ILAT (James Madison University)  SAILS
  45. 45. I-Skills 45  Developed by ETS  7 ICT proficiencies  Tracked to ACRL standards  Task based, 2 sections, 75 minutes
  46. 46. SAILS: Standardized Assessment of Information Literacy Skills 46  45 questions  35-50 minutes  Multiple choice  Can require cooperation of faculty and administrators  Sample question: If you wanted to search for a topic that has several components, such as nutrition for pregnant women, which operator would you use? (and, or, not, adj) Source: Project Sails https://www.projectsails.org/abouttest/samples.php
  47. 47. ILAT – James Madison University 47  Web based  60 items  Tests ACRL standards 1,2,3 and 5
  48. 48. Using assessment information  Rewrite learning outcomes  Change what you do/how you teach  Work collaboratively with colleagues (librarians and faculty)  Revise assessment measures  Share the news in the library and on campus 48
  49. 49. Learning outcomes are:  Fundamental components of a learning assessment program  Focused on student learning  Indicate what students will know/be able to do  Measurable, observable, overt 49
  50. 50. Learning Outcomes Assessment 50 A tool to help librarians improve student learning
  51. 51. LOA resources  Angelo, Thomas (1993). Classroom assessment techniques.  Maki, Peggy (2004). Assessing for learning : building a sustainable commitment across the institution.  Middle States Commission on Higher Education(2003). Developing research & communication skills: guidelines for information literacy in the curriculum http://msche.org/publications/devskill050208135642.pdf  Neely, Teresa Y(2006). Information literacy assessment : standards-based tools and assignments.  Radcliff, Carolyn et.al (2007). A practical guide to information literacy assessment for academic librarians.  Student Learning Outcomes Assessment (University of Virginia)http://www.web.virginia.edu/iaas/assessment/outcom es.htm 51
  52. 52. Let’s assess this workshop! 52
  53. 53. Thank you! Diane Harvey Head, Instruction & Outreach Perkins Library Duke University diane.harvey@duke.edu 53
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×