Articulating Our Impact: Strategies for Holistic Library Assessment

2,343 views

Published on

Keynote at MOBIUS 2015: Articulating Our Impact: Strategies for Holistic Library Assessment

Published in: Education

Articulating Our Impact: Strategies for Holistic Library Assessment

  1. 1. A R T I C U L A T I N G O U R I M P A C T
  2. 2. A R T I C U L A T I N G O U R I M P A C T
  3. 3. H O L I S T I C library assessment s t r a t e g i e s f o r
  4. 4. s t r a t e g i e s f o r @ c h a r b o o t h s l i d e s h a r e . n e t / c h a r b o o t h H O L I S T I C library assessment
  5. 5. s t r a t e g i e s f o r @ c h a r b o o t h s l i d e s h a r e . n e t / c h a r b o o t h S A R A L O W E S E A N S T O N E N A T A L I E T A G G E D A N I B R E C H E R H O L I S T I C library assessment
  6. 6. holistic assessment of student IL learning *
  7. 7. holistic assessment of student IL learning *
  8. 8. holistic assessment of student IL learning * a u t h e n t i c
  9. 9. holistic assessment of student IL learning * a u t h e n t i c differentiated
  10. 10. holistic assessment of student IL learning * a u t h e n t i c differentiated i n t e g r a t e d
  11. 11. a c c e s s f r e e d o m a d v o c a c y i n q u i r y o p e n n e s s
  12. 12. re e d c o l l e g e
  13. 13. ✓ PROCRASTINATION
  14. 14. ✓ PROCRASTINATION ✓ COMISSERATION
  15. 15. ✓ PROCRASTINATION ✓ COMISSERATION ✓ GRAFFITING
  16. 16. ✓ PROCRASTINATION ✓ COMISSERATION ✓ SLEEPING ✓ GRAFFITING
  17. 17. O W N E R S H I P
  18. 18. R I T U A L
  19. 19. C O M M U N I T Y
  20. 20. V A L U E
  21. 21. C O N T E N T
  22. 22. C O N T A I N E R C O N T E N T
  23. 23. C O N T A I N E R C O N T E N T C O N C E P T
  24. 24. V A L U E
  25. 25. { }
  26. 26. Academic libraries have long enjoyed their status as the “heart of the university.” However, in recent decades, higher education environments have changed… Community college, college, and university librarians no longer can rely on their stakeholders’ belief in their importance. Rather, they must demonstrate their value. -Megan Oakleaf, 2010
  27. 27. { }D A T A
  28. 28. D A T A
  29. 29. D A T A
  30. 30. D A T A
  31. 31. D A T A
  32. 32. ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/val_summary.pdf - p12 D A T A
  33. 33. D A T A ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/val_summary.pdf - p12
  34. 34. { }V A L U E
  35. 35. { }I M P A C T
  36. 36. { }Wh a t ’s In It Fo r Th e m ?
  37. 37. I N S T I T U T I O N A L / E X T E R N A L A D M I N S T R A T I V E D E P A R T M E N T A L I N D I V I D U A L A U D I E N C E
  38. 38. R E S O U R C E S & S T A F F R E C O G N I T I O N C O L L A B O R A T I O N G O O D W I L L O B J E C T I V E
  39. 39. R E P U T A T I O N E F F E C T I V E N E S S S T U D E N T S U C C E S S S A T I S F A C T I O N O U T C O M E
  40. 40. holistic assessment of student IL learning * assessment
  41. 41. holistic assessment of student IL learning *
  42. 42. holistic assessment of student IL learning *
  43. 43. holistic assessment of student IL learningR U B R I C S *
  44. 44. holistic assessment of student IL learningR U B R I C S *
  45. 45. holistic assessment of student IL learningR U B R I C S * M A P P I N G
  46. 46. { }R U B R I C S M A P P I N G
  47. 47. { }c u s t o m i z e d R O A D M A P S o f l e a r n e r C O M P E T E N C I E S R U B R I C S
  48. 48. { } T H E C L A R E M O N T C O L L E G E S
  49. 49. { } T H E C L A R E M O N T C O L L E G E S
  50. 50. L I B R A R Y I L D E F I N I T I O N
  51. 51. S T U D E N T & F A C U L T Y S U R V E Y S
  52. 52. I L P E R F O R M A N C E P E R C E P T I O N S G A P 0% 20% 40% 60% 80% 100% Faculty Student Faculty Student Faculty Student Faculty Student Faculty Student Faculty Student Write annotated bibliographies Provide proper attribution to source materials in their academic work Use sources to further an argument/ thesis Evaluate sources to determine if they are authoritative Differentiate between types of information sources (e.g., scholarly v. popular literature, fact v. opinion) Effectively use Library databases, catalog(s), and other information resources to find relevant source material Excellent/Very High Above Average/High Average/Moderate Below Average/Low Poor/None
  53. 53. 0% 20% 40% 60% 80% 100% Faculty Student Faculty Student Faculty Student Faculty Student Faculty Student Faculty Student Write annotated bibliographies Provide proper attribution to source materials in their academic work Use sources to further an argument/ thesis Evaluate sources to determine if they are authoritative Differentiate between types of information sources (e.g., scholarly v. popular literature, fact v. opinion) Effectively use Library databases, catalog(s), and other information resources to find relevant source material Excellent/Very High Above Average/High Average/Moderate Below Average/Low Poor/None * * * * * * * * * * * I L P E R F O R M A N C E P E R C E P T I O N S G A P
  54. 54. How much have your instructors (students)/you (faculty) emphasized the following in the courses you teach? Student v. Faculty Mean Percentage Difference -10.38% -9.97% -9.12% -2.86% -0.97% -12% -10% -8% -6% -4% -2% 0% Using peer-reviewed or scholarly sources in assignments Using practices (terminology, procedures, writing style, etc.) of a specific major or field of study Appropriately citing the sources used in a paper or project Questioning the quality of information sources Not plagiarizing another author's work
  55. 55. C A M P U S - L E V E L I M P L E M E N T A T I O N S C R I P P S P I T Z E R P O M O N A C G U H M C K G I C M C Grassroots/ consensus governance with robust committee structure Pitzer College Consensus- based governance without much committee structure Scripps College Faculty committees with strong departmental autonomy Pomona College Top-down/ federated states governance Claremont Graduate University Faculty- committee- based governance Harvey Mudd College Small/agile/ progressive committee of core faculty Keck Graduate Institute Top-down governance/ assessment driven Claremont McKenna College
  56. 56. Grassroots/ consensus governance with robust committee structure Pitzer College Consensus- based governance without much committee structure Scripps College Faculty- committee- based with strong department decision making Pomona College Top-down/ federated states governance Claremont Graduate University Faculty- committee- based governance Harvey Mudd College Small/agile/ progressive committee of core faculty Keck Graduate Institute Top-down governance/ assessment driven Claremont McKenna College SU 2012: Director of Assessment meets with librarians to discuss IL assessment options SU 2012: Librarians edit Carleton College Rubric to create Rubric v1.1 SU 2012: Librarians score First Year Seminar papers (FYS) using Rubric v1.2 FA 2012: Faculty incorporate Rubric v1.4 into FYS FA 2012: Librarians score capstone papers with Rubric v1.4 SU 2014: Library uses AiA Rubric to evaluate student IL skills FA 2012: Writing 50 (first-year seminar) faculty score papers using Rubric 2012/2013 SU 2014: Library uses 5-part Rubric to evaluate student IL skills Future: Hopeful integration of 2013-14 IL Rubric into capstone evaluation SU 2011: Environmental Analysis (EA) Rubric created based on Drake Univ. Rubric SU 2011: Discussions with faculty improve rubric & EA program FA 2011/FA 2012: EA Rubric used to develop IL instruction for EA senior thesis students SP 2013: Librarians/ faculty use Rubric 2012/2013 to evaluate sample EA theses from three prior years, submits to WASC SU 2013: Library uses AiA Rubric to evaluate student IL skills SU/FA 2012: Rubric v1.4 discussed at all departmental meetings SP 2013: Dean requests Rubric 2012/2013 as foundation for an IL instruction session in methods course / leads to development of IL Labs pilot SP 2013: 5-Part Rubric provides foundation for developing five core learning outcomes for IL Labs pilot Future: Planning for potential integration of Rubric into a Canvas course pilot as a sample evaluation object, and a Thesis/Dissertation evaluation project FA 2012: Librarians and ALO attend WASC Accreditation Retreat on Core Competencies: Critical Thinking and Information Literacy FA 2012: Librarians present Rubric 2012/13 to Assessment Committee FA 2012: Committee interested in training upper class “peers” in using rubric to evaluate clinic work SU 2014: Library uses AiA Rubric to evaluate student IL skills Future: Hopeful integration of 2013-14 IL Rubric into clinic evaluation FA 2012: Librarians and ALO attend WASC Accreditation Retreat on Core Competencies: Critical Thinking and Information Literacy FA 2012: Map and integrate Rubric v1.? into existing rubrics for some programs SP 2013: Begin using updated rubrics as part of an overall program to increase rubric use in more classes Future: IL Rubric data needs to be gathered and assessed. Rubrics are being adopted/developed for the new BioPharmacy program FA 2012: Assessment committee adopts Rubric 2012/2013 FA 2012: Keck Science Dept. adopts Rubric 2012/2013 SP 2013: Library scores senior theses using Rubric 2012/2013 SP 2013: Keck Science Dept. scores senior theses using Rubric 2012/2013 SU 2014: Library uses AiA Rubric to evaluate student IL skills Future: Hopeful integration of 2013-14 IL Rubric into capstone evaluation
  57. 57. S P E C I F I C R U B R I C A P P L I C A T I O N S •  F i r s t - y e a r s e m i n a r p a p e r e v a l u a t i o n •  S e n i o r c a p s t o n e / t h e s i s e v a l u a t i o n •  C a m p u s e s a c c r e d i t a t i o n a s s e s s m e n t •  A s s e s s m e n t i n A c t i o n ( 2 0 1 3 - 1 4 )
  58. 58. S P E C I F I C R U B R I C A P P L I C A T I O N S •  F i r s t - y e a r s e m i n a r p a p e r e v a l u a t i o n •  S e n i o r c a p s t o n e / t h e s i s e v a l u a t i o n •  C a m p u s e s a c c r e d i t a t i o n a s s e s s m e n t •  A s s e s s m e n t i n A c t i o n ( 2 0 1 3 - 1 4 )
  59. 59. 5 C O L L E G E S = 5 F I R S T Y E A R S E M I N A R P R O G R A M S
  60. 60. Research Question “What impact (if any) does librarian intervention in first-year courses have on IL performance in student work?” Methodology 1. None = no faculty collaboration with librarian 2. Low = Traditional One-Shot 3. Moderate = multiple sessions, moderate syllabus/assignment collaboration 4. High = multiple sessions, online tutorial & quiz, significant syllabus/ assignment collaboration Collaboration Levels • First-Year Student Papers coded by Level of Librarian Collaboration in Course • Rubric Evaluation (http://bit.ly/ccl-ilrubric) of papers • 4 Liberal Arts College First-Year Seminar/ Experience programs > Claremont McKenna College, Pitzer College, Pomona College, Scripps College (coded in results for anonymity) ; Results from fifth college, Harvey Mudd, still being coded • 416 papers; 14 interrater pairs (Jan-May, 2014)
  61. 61. report and data - libguides.libraries.claremont.edu/AiA
  62. 62. { }R U B R I C S M A P P I N G
  63. 63. { } C O N T E X T U A L / C U L T U R A L I N V E S T I G A T I O N
  64. 64. { } C O N C E P T / M I N D M A P P I N G m i n d o m o . c o m
  65. 65. { } C O N C E P T / M I N D M A P P I N G K N O W L E D G E M A N A G E M E N T
  66. 66. { } C O N C E P T / M I N D M A P P I N G K N O W L E D G E M A N A G E M E N T
  67. 67. C U R R I C U L U M V I S U A L I Z A T I O N
  68. 68. c a t a l o g s s c h e d u l e s w e b s i t e s C U R R I C U L U M V I S U A L I Z A T I O N
  69. 69. c a t a l o g s s c h e d u l e s w e b s i t e s C U R R I C U L U M V I S U A L I Z A T I O N
  70. 70. C U R R I C U L U M V I S U A L I Z A T I O N
  71. 71. C A M P U S
  72. 72. P R O G R A M
  73. 73. R E Q U I R E M E N T S
  74. 74. O B J E C T I V E S / O U T C O M E S
  75. 75. A N N O T A T I O N S
  76. 76. I N S I G H T + S T R A T E G Y
  77. 77. S C A L I N G U P : I M L S S P A R K S ! G R A N T
  78. 78. T E M P L A T E S , S U P P O R T , & T R A I N I N G
  79. 79. I D E N T I F Y I N G S C A F F O L D E D I L S E Q U E N C E
  80. 80. D I S C I P L I N A R Y F A C U L T Y I L F O R U M S
  81. 81. { }R U B R I C S M A P P I N G
  82. 82. h o l i s t i c a s s e s s m e n t D E M O N S T R A T I N G I M P A C T I T E R A T I V E I M P R O V E M E N T I N V E S T I G A T I N G V A L U E
  83. 83. { }V A L U E
  84. 84. { }I M P A C T
  85. 85. R E S O U R C E S ea curriculum map: ea.pomona.edu/ea-5c-program-map/ ccl il program documents: libraries.claremont.edu/informationliteracy/ A i A p r o j e c t a n d d a t a : l i b g u i d e s . l i b r a r i e s . c l a r e m o n t . e d u / A i A Degrees of Impact: Analyzing the Effects of Progressive Librarian Course C o l l a b o r a t i o n s o n S t u d e n t P e r f o r m a n c e . W i t h M . S a r a L o w e ,  N a t a l i e T a g g e ,  a n d  S e a n M . S t o n e .  C o l l e g e &  R e s e a r c h L i b r a r i e s .  Accepted:  June 26, 2014;  Anticipated Publication Date: July 1, 2015. Slides from Tagge, Natalie; Booth, Char; Chappell, Alexandra; Lowe, M. Sara; and Stone, Sean M., "Choose Your Own Adventure: Integrating an Information L i t e r a c y R u b r i c i n t o S e v e n ( V e r y ) D i f f e r e n t C o l l e g e s " ( 2 0 1 3 ) . L i b r a r y S t a f f P u b l i c a t i o n s a n d R e s e a r c h . P a p e r 1 7 . h t t p : / / s c h o l a r s h i p . c l a r e m o n t . e d u / l i b r a r y _ s t a f f / 1 7 S l i d e s a d a p t e d f r o m L o w e , M . S a r a ; B o o t h , C h a r ; S t o n e , S e a n M . ; T a g g e , N a t a l i e ; C h a p p e l l , A l e x a n d r a ; a n d B u r r o w , G a l e , " L i b r a r i a n s M a t t e r ! I m p a c t o n F i r s t - Y e a r I n f o r m a t i o n L i t e r a c y S k i l l s a t 5 C o l l e g e s " ( 2 0 1 4 ) . L i b r a r y S t a f f P u b l i c a t i o n s a n d R e s e a r c h . P a p e r 2 2 . h t t p : / / s c h o l a r s h i p . c l a r e m o n t . e d u / l i b r a r y _ s t a f f / 2 2
  86. 86. c h a r b o o t h @ g m a i l i n f o m a t i o n a l . c o m @ c h a r b o o t h C H A R B O O T H D I R E C T O R r e s e a r c h , t e a c h i n g & l e a r n i n g S E R V I C E S c l a r e m o n t c o l l e g e s L I B R A R Y s l i d e s h a r e . n e t / c h a r b o o t h

×