Assessing Student Learning Outcomes  (Finding Out If They’re Learning What You Really Want Them  To)
Why Inputs/Outputs Don’t Work
 
In a Nutshell. . . Inputs and outputs quantify what resources you have to offer Student learning outcomes are the skills or knowledge students acquire
Student Learning Objectives Are Measurable Action-oriented Aggregated What can they DO  after your teaching that they couldn’t do  before?
Learning Objectives for This Presentation Mention how assessment must take into account learning styles Students will know a lot about writing  good learning objectives Identify instructor’s favorite clothing color
Writing Good Objectives for  Student Learning Outcomes Is this outcome measurable? Can it be taught? Can a change in the student as a result of learning be identified and/or measured? Can the results be used to improve instruction? Are you measuring the right outcome?
Learning Styles
How are SLOs measured?  (Dugan & Hernon, 2002) Direct Methods Qualitative:  developmental portfolios, directed conversations Quantitative: content analysis, evaluation of these/dissertations, tests Indirect Methods Qualitative:  focus groups, curriculum & syllabus evaluation, external reviewers Quantitative:  general surveys, satisfaction surveys
Pop Quiz True or False Use the same method of assessment for all student learning outcomes.  Direct methods of assessment analyze the work that students do. Triangulation means studying something from multiple angles using multiple methods to make the analysis stronger.
Cognitive Levels of Performance Clark’s content-performance matrix Remember: recall of content Use: application of content List the steps of limiting a MedLINE search Search MedLINE for articles on the adverse effects of smoking and limit to reviews from 2000 and later.
Bloom’s Taxonomy
Clark’s & Bloom’s Compared
 
In a Nutshell. . . It is important to design learning and assessment activities for multiple cognitive levels. It is sometimes difficult to  decide which cognitive levels a specific assignment falls into. Use common  sense and vary the activities.
Types of Assessment Activities
Pretests and Posttests  Differentiate between what you taught and what they came in knowing Use a paired samples  t -test to check for significance
One Minute Paper/Muddiest Point
Audience Response Systems High tech  Immediate response Facilitates active listening Encourages participation
Some Things to Remember Assessment should be embedded in the learning process All instructors who teach the same course need to agree on content and assessment Assessment is iterative
Pitfalls to Avoid Using tests with uncertain validity Believing information literacy is only the responsibility of the library Linking assessment to performance appraisals
The Assessment Librarian
References Anderson,  L. W., & Krathwohl, D. R. (2001).  A taxonomy for learning, teaching, and assessing .  New York: Longman.  Avery, E. F. (Ed. ). (2003).  Assessing student learning outcomes for information literacy instruction in academic institutions.  Chicago: American Library Association.  Chizmar, J. F.,  & Ostrosky, A. L. (1998). The one-minute paper: Some empirical findings.  The Journal of Economic Education, 29 (1), 1-8.  Clark, R., & Harrelson, G. L. (2002). Designing instruction that supports cognitive learning processes.  Journal of Athletic Training, 37 (4 Supplement): S-152-S-159.
Dugan, R. E., & Hernon, P.  (2002). Outcomes assessment: Not synonymous with inputs and outputs.  The Journal of Academic Librarianship, 28,  376-380. Ewell, P. (2003, July/August). The learning curve.  BizEd,  28-33. Forehand, M. (2007, July).  Bloom’s Taxonomy.  Retrieved August 6, 2007 from  http://projects.coe.uga.edu/epltt/i ndex.php?title=Bloom%27s_Taxonomy Gratch Lindauer, B. (1998). Defining and measuring the library’s impact on campuswide outcomes.  College & Research Libraries, 59,  546-570. Iannuzzi, P. (1999). We are teaching, but are they learning: Accountability, productivity, and assessment.  The Journal of Academic Librarianship, 25,  304-305.
Jenkins,  A. (1996, June 27).  How to write learning outcomes.  Retrieved August 1, 2007 from  http://www.ncgia.ucsb.edu/education/curricula/giscc/units/format/outcomes.html#top   Mann, B. L. (2006).  Selected styles in web-based educational research.  Hershey, PA: Idea Group, Inc.  McGinley, L. (2004, October).  Working definitions.  Retrieved August 1, 2007 from  http://cai.cc.ca.us/ Summer2005Institute/index.htm on 8/1/07   University of Central Florida. (n.d.).  Assessment for optimal learning: Classroom assessment.  Retrieved July 23,  2007 from  http://www.fctl.ucf.edu/assessment/ selectingmethods.html   Weinstein, D. (2006,  January). Outcomes assessment is here to stay, get faculty buy in.  Academic Leader,  1-2.
Photo Credits http://www.morguefile.com Darren Hester (slide 21) Kenn Kiser (slides 3 & 14) Derek Benjamin Lilly (slide 5)

SCC 2007 Student Learning Outcomes

  • 1.
    Assessing Student LearningOutcomes (Finding Out If They’re Learning What You Really Want Them To)
  • 2.
  • 3.
  • 4.
    In a Nutshell.. . Inputs and outputs quantify what resources you have to offer Student learning outcomes are the skills or knowledge students acquire
  • 5.
    Student Learning ObjectivesAre Measurable Action-oriented Aggregated What can they DO after your teaching that they couldn’t do before?
  • 6.
    Learning Objectives forThis Presentation Mention how assessment must take into account learning styles Students will know a lot about writing good learning objectives Identify instructor’s favorite clothing color
  • 7.
    Writing Good Objectivesfor Student Learning Outcomes Is this outcome measurable? Can it be taught? Can a change in the student as a result of learning be identified and/or measured? Can the results be used to improve instruction? Are you measuring the right outcome?
  • 8.
  • 9.
    How are SLOsmeasured? (Dugan & Hernon, 2002) Direct Methods Qualitative: developmental portfolios, directed conversations Quantitative: content analysis, evaluation of these/dissertations, tests Indirect Methods Qualitative: focus groups, curriculum & syllabus evaluation, external reviewers Quantitative: general surveys, satisfaction surveys
  • 10.
    Pop Quiz Trueor False Use the same method of assessment for all student learning outcomes. Direct methods of assessment analyze the work that students do. Triangulation means studying something from multiple angles using multiple methods to make the analysis stronger.
  • 11.
    Cognitive Levels ofPerformance Clark’s content-performance matrix Remember: recall of content Use: application of content List the steps of limiting a MedLINE search Search MedLINE for articles on the adverse effects of smoking and limit to reviews from 2000 and later.
  • 12.
  • 13.
  • 14.
  • 15.
    In a Nutshell.. . It is important to design learning and assessment activities for multiple cognitive levels. It is sometimes difficult to decide which cognitive levels a specific assignment falls into. Use common sense and vary the activities.
  • 16.
  • 17.
    Pretests and Posttests Differentiate between what you taught and what they came in knowing Use a paired samples t -test to check for significance
  • 18.
  • 19.
    Audience Response SystemsHigh tech Immediate response Facilitates active listening Encourages participation
  • 20.
    Some Things toRemember Assessment should be embedded in the learning process All instructors who teach the same course need to agree on content and assessment Assessment is iterative
  • 21.
    Pitfalls to AvoidUsing tests with uncertain validity Believing information literacy is only the responsibility of the library Linking assessment to performance appraisals
  • 22.
  • 23.
    References Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing . New York: Longman. Avery, E. F. (Ed. ). (2003). Assessing student learning outcomes for information literacy instruction in academic institutions. Chicago: American Library Association. Chizmar, J. F., & Ostrosky, A. L. (1998). The one-minute paper: Some empirical findings. The Journal of Economic Education, 29 (1), 1-8. Clark, R., & Harrelson, G. L. (2002). Designing instruction that supports cognitive learning processes. Journal of Athletic Training, 37 (4 Supplement): S-152-S-159.
  • 24.
    Dugan, R. E.,& Hernon, P. (2002). Outcomes assessment: Not synonymous with inputs and outputs. The Journal of Academic Librarianship, 28, 376-380. Ewell, P. (2003, July/August). The learning curve. BizEd, 28-33. Forehand, M. (2007, July). Bloom’s Taxonomy. Retrieved August 6, 2007 from http://projects.coe.uga.edu/epltt/i ndex.php?title=Bloom%27s_Taxonomy Gratch Lindauer, B. (1998). Defining and measuring the library’s impact on campuswide outcomes. College & Research Libraries, 59, 546-570. Iannuzzi, P. (1999). We are teaching, but are they learning: Accountability, productivity, and assessment. The Journal of Academic Librarianship, 25, 304-305.
  • 25.
    Jenkins, A.(1996, June 27). How to write learning outcomes. Retrieved August 1, 2007 from http://www.ncgia.ucsb.edu/education/curricula/giscc/units/format/outcomes.html#top Mann, B. L. (2006). Selected styles in web-based educational research. Hershey, PA: Idea Group, Inc. McGinley, L. (2004, October). Working definitions. Retrieved August 1, 2007 from http://cai.cc.ca.us/ Summer2005Institute/index.htm on 8/1/07 University of Central Florida. (n.d.). Assessment for optimal learning: Classroom assessment. Retrieved July 23, 2007 from http://www.fctl.ucf.edu/assessment/ selectingmethods.html Weinstein, D. (2006, January). Outcomes assessment is here to stay, get faculty buy in. Academic Leader, 1-2.
  • 26.
    Photo Credits http://www.morguefile.comDarren Hester (slide 21) Kenn Kiser (slides 3 & 14) Derek Benjamin Lilly (slide 5)

Editor's Notes

  • #2 This part of the class is a relatively brief overview of student learning outcomes and how to apply them to a library setting. I use a lot of examples from the academic environment, but I think this information is applicable to any library with the librarian as instructor scenario. I don’t know if you have similar backgrounds, but I made it through 2 graduate degrees without any teaching experience of any kind. When I got a job as an education librarian, it was a. what was I thinking and b. how do I do this? There weren’t any classes on teaching in library school, but that’s a big part of being a public services librarian these days. We have mostly on the job training, and it’s hard for us to know if we are putting together useful classes. What we think may be an incredibly important thing to learn, our students may already know or may never use. So, how do we figure out what will make a difference to them? Back before our new culture of assessment, at our library, every time a class was taught, we used to make the students fill out the standard class evaluation form. The instructor collected all the forms, glanced over them, and then filed them away in a drawer never to see the light of day again. Does that sound familiar to anyone else? I mean, you collected the data, sometimes you tweaked the class a little based on the comments, so that told you everything you needed to do? They showed up at the class, so they must have learned something. The point is, when we rely on outputs, we don’t really know what’s going on. An output would be every student taking a PubMed class, but our goal is not really to make every student take a library class. Out goal is to have the outcome of every student being able to successfully search PubMed to find the information he or she needs.