Downloaded 55 times
Uploaded byALATechSource
How to Improve Library Instruction: Assessment in Five Minutes
The document discusses quick assessment techniques that can be used in one-shot library instruction sessions. It recommends drafting learning outcomes using Bloom's Taxonomy and composing assessment questions based on the outcomes. The document reviews tools for quick assessment, such as polls, quizzes, and minute papers, and provides best practices for question drafting. The goal of assessment is to identify instructional gaps, determine how to spend class time effectively, and provide evidence of the impact of the instruction.
More Related Content
![Web based instrucitonal-strategies[1]](https://cdn.slidesharecdn.com/ss_thumbnails/web-basedinstrucitonalstrategies1-110606105250-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to How to Improve Library Instruction: Assessment in Five Minutes
How to Improve Library Instruction: Assessment in Five Minutes
- 1. How to ImproveYour Library Instruction: Assessment in Five Minutes Sarah Steiner January 10, 2013
- 2. During Today’s Class,We Will… • Analyze the purpose and benefits of quick assessment in one-shot instruction sessions. • Draft learning outcomes for a session using Bloom’s Taxonomy. • Compose assessment questions based on our learning outcomes. • Select appropriate assessment tools based on your instruction goals and population. http://www.flickr.com/photos/topgold/4858921901/
- 3. Our Focus • Classroomassessment (versus programmatic or institutional) • Fast and free assessment tools – Polls – Quizzes/Questionnaires – Minute papers http://www.flickr.com/photos/rogersg/3814863064/
- 4. What Can QuickClassroom Assessment Do? • It can identify instructional gaps or disconnects. • It can help you determine how you spend class time. • It can build your confidence. • It can provide evidence of efficacy and impact. • It should be a basis for change. http://www.flickr.com/photos/sercasey/324341982/
- 5. What Should IAssess? • To assess: – Comprehension – Retention – Achievement of learning outcomes • Not to assess: – Your personal shortcomings – Your speaking mistakes – Your hairstyle – Random things http://www.flickr.com/photos/toniblay/52445415/
- 6. Have you everwritten learning objectives or outcomes? • Nope, not yet. • Yes, a few times. • Yes, regularly. • I’m not sure. http://www.flickr.com/photos/dadaistic_fantastic/5089525186/
- 7. Learning Outcomes “A LearningOutcome is a statement of what the student should understand and be able to do as a result of what she has learned ... ‘the essential and enduring knowledge, abilities, and attitudes or dispositions’ that enable a learner to practice and apply her learning in the real world.” -Valencia Community College http://www.flickr.com/photos/gblakeley/5583354276/
- 8. A Good LearningOutcome Will… • Identify the audience. • Set a time frame and a context. • Be jargon-free. • Be measurable. • Be action-oriented. • Be brief. • Be linked to learner needs. http://www.flickr.com/photos/auntiep/4310267/
- 9. Determine the ClassPriorities • Choose three to five – What must the students accomplish? – What must they know in order to accomplish it? – What do they already know/find obvious? http://www.flickr.com/photos/paloetic/4795592340/
- 10. Write the LearningOutcomes Stem + Verb + Product/Outcome http://www.flickr.com/photos/jjpacres/3293117576/
- 11. Stem By the endof this class, you will be able to… http://www.flickr.com/photos/47108884@N07/4594962925/
- 12. Verb: Bloom’s Taxonomy http://www.flickr.com/photos/tryingmyhardist/1366874433/
- 13. Choose Verbs: Knowledge Studentswill… define, describe, identify, know, label, list, match, na me, outline, recall, recognize, reproduce, select, stat e http://www.nwlink.com/~donclark/hrd/bloom.html http://www.flickr.com/photos/reway2007/4693202935/
- 14. Choose Verbs: Comprehension Studentswill… comprehend, convert, defend, distinguish, estimate, explain, extend, generalize, give an example, infer, interpret, paraphrase, predict, rewrit e, summarize, translate http://www.flickr.com/photos/bsmith4815/112307904/
- 15. Choose Verbs: Application Studentswill… apply, change, compute, construct, demonstrate, dis cover, manipulate, modify, operate, predict, prepare , produce, relate, show, solve, use http://www.flickr.com/photos/28478778@N05/5729009434/
- 16. Choose Verbs: Analysis Studentswill… analyze, break down, compare, contrast, diagram, deconstruct, diffe rentiate, discriminate, distinguish, identify, illustrate, infer, outline, relate, select, separate http://www.flickr.com/photos/billselak/2709756134/
- 17. Choose Verbs: Synthesis Studentswill… categorize, combine, compile, compose, create, devis e, design, explain, generate, modify, organize, plan, r earrange, reconstruct, relate, reorganize, revise, rewr ite, summarize, tell, write http://www.flickr.com/photos/littleredelf/3081832532/
- 18. Choose Verbs: Evaluation Studentswill… appraise, compare, conclude, contrast, criticize, criti que, defend, describe, discriminate, evaluate, explai n, interpret, justify, relate, summarize, support http://www.flickr.com/photos/christinamatheson/108818379/
- 19. Add Product orOutcome What do they need? • A book on their topic • A thesis statement • Keyword search strategies (basic or advanced) What is the context? • A class assignment • A real-world scenario http://www.flickr.com/photos/fdecomite/2907794565/
- 20. All Together Now! Stem+ Verb + Product/Outcome By the end of this class, you will be able to identify a scholarly source. By the end of this session, you will be able to locate books on your paper topic in the library using the online catalog, GILFind. http://www.flickr.com/photos/revlimit/2186765034/
- 21. What’s Wrong Here? TodayI’ll talk about… 1. The library website. 2. Database searching for peer-reviewed journals using boolean logic and the thesaurus. 3. How you can search the catalog, search the databases, and use ILL. http://www.flickr.com/photos/mythoto/2604818537/
- 22. Question Time http://www.flickr.com/photos/sugarhiccuphiccup/5485691671/
- 23. Have you everassessed students’ achievement of pre- set learning outcomes? • Nope, not yet. • Yes, a few times. • Yes, regularly. • I’m not sure. http://www.flickr.com/photos/dadaistic_fantastic/5089525186/
- 24. Today’s Assessment Tools •Polls • Quizzes (pre and post or post only) • Minute papers http://www.flickr.com/photos/azarius/225340667/
- 25.
- 26. Poll Example Questions •How many of you have had a library instruction class here before? • Is this an article or a journal? • Is this source peer-reviewed? / Is this source scholarly or popular? • Who is the author of this book? • Are you confused about topic x? http://www.flickr.com/photos/rakka/3451170932/
- 27.
- 28.
- 29. The Good, theBad, the Ugly
- 30. Minute Papers http://www.flickr.com/photos/holtsman/4331034955/
- 31. Minute Paper ExampleQuestions
- 32. Best Practices forQuestion Drafting • Convert your learning outcomes to questions. • Include three to five “meaty” questions, two or three general questions. • Avoid overly easy or “all of the above” answers. • Avoid nebulous or complicated questions. • Work in terminology to ensure it’s understood. http://www.flickr.com/photos/djmccrady/6118626233/
- 33. Best Practices forQuestion Drafting • Get a reviewer. • If at first you don’t succeed…. More help with constructing questions: http://ets.tlt.psu.edu/learningdesign/effective_ questions/why http://www.flickr.com/photos/djmccrady/6118626233/
- 34. Review Responses • Onthe spot? Yes! • Look for… • Percentage of correct answers. • Trends in missed answers. • What to change next time. • Keep your chin up. http://www.flickr.com/photos/xingty/3421423768/
- 35. Verbal Setup http://www.flickr.com/photos/twenty_questions/2348686991/
- 36. Survey and PollTools • SurveyMonkey: http://www.surveymonkey.com • Poll Daddy: http://polldaddy.com/ • Poll Everywhere: http://www.polleverywhere.com/ • Google Forms: http://www.google.com/google-d-s/forms/ http://www.flickr.com/photos/litlnemo/5045121567/
- 37.
- 38.
- 39.
- 40.
- 41. Review • Write learningoutcomes. • Base your questions on the outcomes and on your needs. • Review and reflect on the answers you get. http://www.flickr.com/photos/demonbaby/4228327167/
- 42. Thank you! Contact meat ssteiner@gsu.edu
- 43. Other Resources • “Bloom’sTaxonomy of Cognitive Domain.” http://classweb.gmu.edu/ndabbagh/Resources/Resources2/bloomstax.htm • Educational Oasis, “Framework of Learning Outcomes.” http://www.educationoasis.com/instruction/bt/learning_objectives.htm • London Deanery, “Setting Learning Objectives.” http://www.faculty.londondeanery.ac.uk/e-learning/setting-learning- objectives • Penn State, “Writing Effective Questions to Promote Learning.” http://ets.tlt.psu.edu/learningdesign/effective_questions/ • Utah State University Library, “Assessment.” http://library.usu.edu/instruct/assessment/index.php • “Virginia Tech Instruction Clearinghouse.” http://www.lib.vt.edu/RIS/clearinghouse/assessments.html
Editor's Notes
- #2 All images are creative commons, cited in the bottom right corner.
- #3 Choose your questions and model based on the outcomes you need to meet.
- #4 This session will focus on classroom. You have 50 or 75 minutes only—how can you fit in assessment?When we think of assessment, we often think it needs to be really involved in order to be useful. Of course, you can do hugely involved assessments, but that’s not the focus today. This is about how important and beneficial quick stuff can be. Our assessment doesn’t need to have pre and post tests, doesn’t need to span an entire semester, doesn’t need to have IRB approval and pre-testers and a million steps. Formative assessment is utilized to immediately determine whether students have learned what the instructor intended. This type of assessment is intended to help instructors identify material which needs to be clarified or re-taught and should not be used to evaluate or grade students. Results of formative assessment can assist instructors to ascertain whether curriculum or learning activities need to be modified during a class session or before the next class meets.Summative assessment is cumulative in nature and is used to determine whether students have met the course goals or student learning outcomes at the end of a course or program.
- #5 Example: terminology—article, journal, databaseAssessment is not an end in itelf—it supports change and best practice in teaching.
- #6 Did they take away what you thought they were going to take away? If not, why?My favorite lesson: don’t know what a database is. Don’t know the difference between an article and a journal. They tend to be highly constructive
- #7 Two part question—have you done both of these things together? There are other kinds of assessments—observation, initial questions—but today we’re going to knit these two things together.
- #8 Our first step in good, quick assessment is to write learning outcomes. The assessment questions which you choose will, in general relate specifically to those learning outcomes. They will also provide a roadmap for you, as the instructor, in terms of what you must cover. Whatever assessment method you choose later will tie back to these. A good learning outcome will state what the student will be able to do, and WHY. It will use Bloom’s taxonomy verbs of various cognitive levels.http://valenciacc.edu/facultydevelopment/tla/Candidate/documents/ExamplesandDefinitionsofLOs.pdfDifferent from an objective. Objective states what we aim to do. Outcome describes what the student will be able to do at the end. Learning outcomes are relevant to every part of your tiny assessment plan. Help you to focusHelp to reduce students being overwhelmed
- #9 Review the outcomes for this sessionWhich one might be hard to measure?
- #10 First step: consider the top three or four class priorities. Write them down. Find a bookIdentify a scholarly articleLocate a scholarly article“Learning outcomes should flow from a needs assessment. The needs assessment should determine the gap between an existing condition and a desired condition. Learning outcomes are statements which described a desired condition – that is, the knowledge, skills, or attitudes needed to fulfill the need. They represent the solution to the identified need or issue. Learning outcomes provide direction in the planning of a learning activity.”http://www.aallnet.org/Archived/Education-and-Events/cpe/outcomes.html
- #12 Future tense—timeframeIdentifies learner (some people say “students”)
- #13 Benjamin Bloom, an American educational psychologist, helped to write these in 1956. Domains of learning. Cognitive domain. Also Affective and Psychomotor domains.I want to go through these in depth a little, because they will directly inform your assessment questions. So, as you choose verbs, consider them not just as objectives, but as assessment questions for your post test.
- #14 “Knowledge is the act of remembering previously learned material. It’s the lowest level of learning outcomes in the cognitive domain. “
- #15 “Comprehension is the ability to grasp the meaning of material. Students can demonstration comprehension through interpretation or summarization of material.These learning outcomes are a stepbeyond just remembering. They represent the lowest level of understanding.”
- #16 “Application refers to the ability to use learned material in new and concrete situations.”We do this a lot in one shots.
- #17 “Analysis is the ability to break down material so that its organizational structure can be understood. Differentiation, comparing and contrasting, identification.At this point you may notice that some of these verbs are on more than one list.
- #18 “Synthesis is the ability to put parts together to form a new whole.”It can be hard to get to this in an hour. Our students often do this at the course level—they have to write a paper, a speech, a bibliography, a literature review. Synthesis is the ultimate goal, in many cases.
- #19 “Evaluation is the ability to judge the value of material in a particular scenario. The judgments are to be based on definite criteria. Learning outcomes in this area are highest in the cognitive hierarchy because they contain elements of all the other categories, plus value judgments.
- #20 This should include context as often as possible without being too wordy or jargon-filled. So, for example, in this last I’ve said “advanced techniques” rather than saying boolean logic, subject header searches.We’ll get to those things, but not in our outcomes. You don’t have to have the context, but I like it. If you don’t include it in your written outcomes, still think about it and be able to articulate the context to the students.
- #21 Stem and outcomeDemonstrable/assessableBloom’s cognitive domain—high levelNo jargonOften INCREMENTAL—step-based. They fit together to make a whole.
- #22 Learner not identified—teacher focusNot assessablePoor verbNo verbJargon, too specificHUGE in scopeTAKE A BREAK
- #24 Two part question—have you done both of these things together? There are other kinds of assessments—observation, initial questions—but today we’re going to knit these two things together.
- #25 Just read through, no elaborationWe’re going to talk about each and which might be best for you, given your setting.
- #26 Polls are good for feedback throughout the session. They’re great if you have information that builds and is cumulative. So, you can force them to answer a few questions before you move on to the next topic. They’re also a fun active learning exercise and they give you immediate feedback.The questions have to be simpler, with no short answer or essay, but you can have more of them. You can set them up to be embedded on a site/LibGuide (if your students have computers), or use free tools to set them up as text messages. If you have a clicker system, you can use them as well. Not too expensive, but they do cost $$$. (1000 to 2000... Other?) You can take those polls and embed them into PowerPoints, and then the answers will show up there (PollEverywhere). They can also just raise their hands, of course!You can have students work together to submit a group answer, or you can give each individual a clicker. Drawback—can be distracting, can have technical problems, can take up a lot of class time.
- #27 Regardless of what method you choose to deliver the content (phone, embedded poll, clicker) you can have different question types. Multiple choice, yes/no.Substantive or friendly—can also help you to set the stage for what you’ll cover in the beginning. These can be great, because they are anonymous in the moment. So, if someone feels shy about raising a hand to say s/he is confused, this can help them to do it without drawing attention to themselves.
- #28 So, we’ve all had quizzes—they are my weapon on choice. You can set them to happen at the end, or at the beginning and end to gauge what they knew versus what they know. If you want to measure long-term retention, you can give them the same quiz at a later date. With both quizzes and polls, you can easily convert your learning outcomes to question, if they’re carefully drafted. So, “You will know how to find a book on your topic” becomes “find a book on your topic.”Not more than five “meat” questions, maybe a couple of close out questions that are a bit more free-form.
- #29 Pre and post, or just post. Pre and post can gauge learning, post can gauge what they’re walking out with. If you can get teacher cooperation, you can also do them again as a follow up, to gauge retention on various time scales.Some of these are open ended, some multiple choice. It’s a good idea to have a variety. Not too many easy ones, or you may not learn anything useful. You can also format these as quick assignments. Great for classes with one set task—evaluate and cite websites, for example. You can also give them an article citation and ask them to find it. Then describe briefly why they think it’s reputable.
- #30 You don’t need to be perfect your first time. If your questions aren’t perfect the first time or your instruction isnt perfect, that’s okay You will realize even as you get more accomplished, that you sometimes don’t say things perfectly! You can correct that on the spot. You can use answers as an opportunity to congratulate them for their awesomeness.
- #31 What is a minute paper? You have them write or type for just one minute—only two real questions (and maybe some basic information).On paper or online is your choice. It’s easier to tabulate things online though. Drawbacks:It’s hard to evaluate these in the moment, but you can. It’s also very hard to tie these in with your objectives—however, they are useful. I like to do a combination of hand-raising polling, with a post quiz that has a minute paper attached.
- #32 Gives them space to clarify and explain, if they want to. You can also invite them to leave contact info. You’ll see these final questions again in the minute paper, when we talk about that.
- #35 With polls and quizzes, if they’re brief, it’s easy to tabulate and review responses immediately. I like to do this. Skim over the answers and address problems right there. “It seems like I was unclear on Point X. Can someone describe the answer? HAVE YOUR ELEVATOR SPEECHES READY.This is the most important step. If you don’t integrate the information, it’s not useful to anyone. We’re not perfect—this kind of thing can be jarring at first. Change can relate to the way the questions are worded, ordered, or what is included, and also in the ways you teach.
- #36 Warn the students that it’s coming—but you don’t have to treat it like a test. Just say—”I’m trying to make sure that I’m explaining everything you need, so if you felt something was unclear, please do say so--please help out people who come after you!”Do it ten minutes prior to the end, if at all possible. That way they have five minutes to take it, and you have five minutes to do a final recap and address anything that looks problematic. Have your elevator speeches ready on each of the objectives—be able to talk about each one very quickly and clearly. It’s important that they know WHY this is important. You will use the information to make sure they know everything they need to know. You will use it to make your instruction more effective for future students.
- #37 Most of these can be used for both.
- #38 Basic: free10 questions per survey100 responses per surveySelect: $17 per month ($204 per year)Unlimited questionsUnlimited responsesCustom urlsSkip logic etc.
- #39 Free account: 100 responses per month LIMITED!!!Ten questions per survey or quizPRO account:$200 per year1000 survey responses per monthUnlimited questions per survey or quiz
- #40 Free version40 responses per pollPersonal$15 per month ($180 per year)50 responses per pollPresenter model $65 per month ($780 per year)250 responses per poll
- #41 Free, but somewhat clunky to use.
- #44 List of verbs:http://www.acu.edu/academics/adamscenter/course_design/syllabus/verbs.html