JulieAre students able to do what we teach them to do?“If libraries intend to remain relevant on campus, they must demonstrate their contributions to the mission of the institution by becoming involved in assessment (Oakleaf, 2009).”
Brandy:List course outcomes right in the syllabus.
Julie– Will this tool tell you what you want to learn?
Brandy“Objective”/“Traditional” – attempts to measure knowledge acquisition as proxy for skill acquisitionAuthentic – attempts to measure students’ skill acquisition during real-world experiencesPerformative – attempts to measure students’ skill acquisition in simulated real-world contexts
All questions are performative.
What we wanted to learn
More things we wanted to know. Can they use truncation effectively. Can they use Subject Headings. But had to focus on what we felt was the most important.
Scenario that mimicked the final paper they would be writing in their classes. Really wanted to see if students could combine terms using Ors. Set up a scenario that would force them to use ors.
Feedback to the students
(rubric) (emphasize value of assignment) (explain in the assignment why learning to complete these particular questions are valuable)
Which learning outcomes we should target
Nursing 402 Class—Let students create their own research question based on issues/problems they have noticed in their workplace. This encourages critical thinking and ties the assignment to real life sitations)IDS 100—Large scale assessment for Middle States (formal rubric where we assessed each learning outcome on an “Beginner, “Developing,” “Advanced.”
BrandyWork individually at first, then in pairs?(I think we could have different areas of the room for people to go depending on the standard that they pick and they can talk to others in their area)
Reaching the Summit: Using Assessment to Help You Get ThereJULIE NANAVATI, INFORMATION LITERACY COORDINATOR LOYOLA-NOTRE DAME LIBRARY BRANDY WHITLOCK, INSTRUCTION LIBRARIAN ANNE ARUNDEL COMMUNITY COLLEGE
Learning Outcomes for Today’s SessionUpon successful completion of today’s session, participants will be able to: Define key terms related to assessment Distinguish among objective, performative, and authentic measures of assessment Prepare a performative or authentic assessment for a particular learning outcome
Why Assess? Gauge the effects of our teaching on student learning Extend learning outside of the classroom Find ways to improve instruction Prove our worth
Why Assess? Good teaching is inseparable from good assessing.Source: Wiggins, G. (1996), “Creating tests worth taking”, in Blum, R.E. andArter, J.A. (Eds), A handbook for student performance in an era ofrestructuring (pp. 1-9), Association for Supervision and CurriculumDevelopment, Alexandria, VA.
Overview of Assessment Create learning activities/assessments Create rubric or other grading criteria Reflect and enact changes Learning outcomes Enact activities/ assessmentAdapted from Oakleaf, M. (2009). Theinformation literacy instruction assessment Score assessmentcycle: A guide for increasing student learningand improving librarian instructional skills. and analyze dataJournal of Documentation, 65(4), 539-560.
Overview of Assessment“Assessment Cycle at USC.” (2010). Institutionalassessment and compliance. University of SouthCarolina. Retrieved fromhttp://ipr.sc.edu/effectiveness/toolbox/cycle.htm
First Step: OutcomesWhen developing a learning experience, start withoutcomes,i.e., what should students be able to do?Upon successful completion of thecourse/class/session/assignment, students will be ableto:
First Step: OutcomesWhen designing learning outcomes, be sure they are: Clear Specific Achievable Observable Measurable
Resources for Creating Learning OutcomesYou’re not alone! Bloom’s taxonomy ACRL standards
Map to Broader OutcomesModule 12 Learning Outcomes:After completing this weeks lesson, tutorial, research log, and/ordiscussion, you should be able to: • define terms related to the World Wide Web • use appropriate criteria to evaluate sources, especially web sources • search for web sources effectively and efficiently for a particular research topic and/or research questionRelated course objectives: When you have finished the coursesuccessfully, you will be able to access needed information effectivelyand efficiently, and use information effectively to accomplish a specificgoal.
Second Step: Activities & AssessmentsNext, develop learning activities and assessments,i.e., what will students do to develop their skill(s) anddemonstrate what they’ve learned?
Learning ActivitiesThrough what experiences will students learn theknowledge and skills that you will assess? Lecture Discussion Direct observation Experimentation Problem-solving Service Research etc.
Tools for AssessmentQuestions to ask yourself:• Will this tool tell me what I want to learn about the students?• Will this tool work best for my information literacy class/program?Other factors: time, manpower, facultycollaboration, level of skills being assessed
Types of Assessment Formative – while the learning activity is happening Summative – at the end of the learning activity
Types of Assessment Formal – data collected Informal – no data collected
Sample Assessment 1Learning Outcome: Students will be able to identifyrelevant keywords for a research topic.Assessment Question: Margie is researching a careerin dog training. What are possible keywords for hertopic?Authentic?Performative?“Objective”?
Sample Assessment 1Learning Outcome: Students will be able to identifyrelevant keywords for a research topic.Assessment Question: Margie is researching a careerin dog training. What are possible keywords for hertopic?Authentic?Performative!“Objective”?
Sample Assessment 2Learning Outcome: Students will be able toidentify relevant keywords for a research topic.Assessment Question: What are keywords?Authentic?Performative?“Objective”?
Sample Assessment 2Learning Outcome: Students will be able toidentify relevant keywords for a research topic.Assessment Question: What are keywords?Authentic?Performative?“Objective”!
Sample Assessment 3Learning Outcome: Students will be able toidentify relevant keywords for a research topic.Assessment Question: List possible keywords fora topic you want or need to research.Authentic?Performative?“Objective”?
Sample Assessment 3Learning Outcome: Students will be able toidentify relevant keywords for a research topic.Assessment Question: List possible keywords fora topic you want or need to research.Authentic!Performative?“Objective”?
Sample Assessment 4Learning Outcome: Students will be able to identifyrelevant keywords for a research topic.Assessment Question: Which of the following are the bestkeywords for researching careers in dog training? a. dog trainers b. professional dog training c. dog training careers d. canine training certificationAuthentic?Performative?“Objective”?
Sample Assessment 4Learning Outcome: Students will be able to identifyrelevant keywords for a research topic.Assessment Question: Which of the following are the bestkeywords for researching careers in dog training? a. dog trainers b. professional dog training c. dog training careers d. canine training certificationAuthentic?Performative?“Objective”!
Why Performative& Authentic Assessment? Assess higher-order thinking Assess application of acquired knowledge Mimic activities that students will have to accomplish Build students’ confidence in their ability to successfully accomplish those activities Encourage student investment in the learning process
Third Step: Create Grading CriteriaStep 6: Article Evaluation*A. Evaluate the Article ContentUsing the general evaluation criteria described in module 3, explain in the boxes below how thearticle you selected meets each of the criteria (authority, reliability, coverage, and currency).B. Article RelevancyExplain why you selected this particular article (Step 5) and how it is relevant to your researchquestion. What are some specific points the article made that would assist you in writing a paperthat addresses your research question? Your response must be at least two complete sentences.How will Step 6 be graded?Unsatisfactory (0 points): Does not answer all of the evaluation criteria questions, or answer thearticle relevancy question, or both.Satisfactory (1–3 points): Minimally answers the evaluation criteria questions, or minimallyanswers the article relevancy question, or both.Excellent (4 points): Thoroughly responds to the evaluation criteria questions with details aboutthe selected articles authority, reliability, coverage, and currency, and thoroughly responds to thearticle relevancy question.*UMUC LIBS 150, Research Log Project 2
Why Use Rubrics? Students better understand instructor’s expectations Eases grading dilemmas for instructors More standardized and consistent grading Students better understand what they’ve learned and what they still have to learn
ENG 200: Question 2What did we want to learn?1. Can a student construct an effective search?2. Can a student evaluate the results that she or he generates to find results that answer her or his research question?
ENG 200: Question 2ACRL Standards: Standard 2, ACRL Information Literacy The information literate person accesses needed information effectively and efficiently. Standard 3, ACRL Information Literacy The information literate person evaluates information and its sources.
ENG 200: Question 2Learning Outcomes:1. Identify key terms from a research question2. Access database appropriate for research need (MLA Bibliography)3. Combine key terms using the connectors and&or4. Evaluate results and identify scholarly journal articles relevant to research need5. Cite a scholarly journal article using MLA style
ENG 200: Feedback Nice job pulling out the key terms from the research question. Remember, when combining two terms with “OR,” keep those terms on the same line and type the OR in between like this: 1st line: mothers or daughters 2nd line: sula or beloved Also try leaving the term “relationships” out, you will get more results this way.
ENG 200: ChallengesChallenges: We grade about 270 assignments a semester. We all have to be on board with the grading process (rubric) We have to keep faculty buy-in We have to make sure students are invested
ENG 200: Why We Do ItBenefits for Students: Prepares students for their literature class research assignment Serves as another opportunity to learn information literacy skills Provides students with feedback
ENG 200: Why We Do ItBenefits for Us: Provides us with a window into how well students are able to do what we teach (Search History) Shows us where we can strengthen our instruction Gives us data we can use to demonstrate our effectiveness
ENG 200: Changes We’ve Made Changed the way we taught how to use “ORs” in our freshman class Explained to all classes the challenges of using “TX-All Text” in a search Reemphasized the importance of evaluating results before using them in all library classes Created an interactive tutorial for students to use before completing the assignment
Assignment for Today’s Session1. Identify one ACRL learning outcome that is challenging for you to assess.2. To assess that outcome, select an optimal assessment tool that will work with your information literacy program.3. Design one question or task that uses authentic or performative assessment to evaluate whether students have met that learning outcome.4. Mark whether this question or task will be part of a formal (FM) or informal (IN) assessment, and whether the assessment will be summative (SU) or formative (FV). Write your answers on the carbons provided.
Next Steps: Score, Reflect, Change Create learning activities/assessments Create rubric or other grading criteria Reflect and enact changes Learning outcomes Enact activities/ assessmentAdapted from Oakleaf, M. (2009). Theinformation literacy instruction assessment Score assessmentcycle: A guide for increasing student learningand improving librarian instructional skills. and analyze dataJournal of Documentation, 65(4), 539-560.
We Made It!Visit Our LibGuide for an extensive bibliography, full and additional sample assignments, rubrics, and more: libguides.aacc.edu/LOTWassessment Photos courtesy of Mike Cooperstein, http://www.andesmountainguides.com