Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

What will they need? Pre-assessment techniques for instruction session.


Published on

Librarians all know the importance of a reference interview -- it's to make sure you're addressing what the patron actually needs. Classes take longer, and involve more people, but the fact still holds: to give the best service, you need to assess what the needs actually are.

An additional benefit of pre-assessment is that it can provide evidence of the impact of the teaching program, both to university administration and to accreditation organizations.

Presented by Gwen Exner at "Assessment Beyond Statistics" NCLA College & Universities Section/Community & Junior Colleges Section 2012 conference.

Published in: Education
  • Be the first to comment

  • Be the first to like this

What will they need? Pre-assessment techniques for instruction session.

  1. 1. What will they need?Pre-assessment techniques for instruction sessions Assessment Beyond Statistics, Nov 2, 2012 by Gwen Exner
  2. 2. “They” are…• Students in the current class… ..who need questions answered & new skills taught• Teachers of future classes… ..who need information to make classes relevant• Library administrators… ..who need to justify the library’s budget• Accreditation organizations… ..who need evidence of strategic goals met
  3. 3. Options There are three basic options for pre-assessments in a instructional session• Descriptions of planned classes• Feedback from students about expectations• Tests of pre-existing skills Each is useful is different ways, and has multiple ways it can be done.
  4. 4. Course Descriptions Cost Effort Low Low Can help students pick the course that’s right for them.• Should include examples of what will be taught. This course will teach students how to find rows & columns, how to create simple formulas, how to make a graph, and how to format a table using the templates• Should include examples of what pre-knowledge is expected. Students are expected to already know how to open the program, open files, save files, and copy & paste within files.• Should include examples of what is beyond the scope. This course will not cover how to use functions, edit graphs, or modify formatting templates.
  5. 5. Students need to know whether or not a class will answer their questions. An “Online Research” class could be answering any of these: How do I…?…find things online? …find articles …use PsycInfo? from local papers?…download ebooks? …use the catalog?…do a literature …cite onlinereview? sources?…find an essay …get thisto copy? article? Clear class descriptions allow students to self-pre-assess where they want to be.
  6. 6. Registrant Feedback Cost Effort Low Medium Can help the teacher know what to focus on.Can be conducted during course registration, or at the beginning of class.• Beginning of class limits the overall flexibility, but helps put students in the right mindset and the results can be shared.• During course registration allows the class to be planned around student needs, but can be prone to misunderstandings.Can be open-ended, selected from existing class goals, or a combination.• Open-ended can let students express what they want in their own words, but it won’t always make sense and may be outside the scope.• Selecting from existing goals will produce results that make sense, but are only as accurate as students’ understanding, and can’t introduce new ideas.• Combination can be the best of both worlds, with a list to select from, plus a brief field for “other”.
  7. 7. ExamplesAt registration At start of classPlease tell us what you want to Index cards: Hand them out asget from this class: students filter in, & ask them(type your answer here) to write their top 3 goals & return them. Flip through the answers before starting. - or - - or -Which of the following aspectsare you most interested in? Survey: After everyone’s arrived, ask what people are Boolean logic looking for from this class. Finding scholarly articles Answers can be verbal, Judging credibility clickers, or text/twitter/web Other . via services such as
  8. 8. Pre-Tests Pre-Tests Can provide very detailed, accurate information.Can serve a variety of purposes.• Improve student understanding and confidence in course self-selection.• Highlight problem areas so teachers can focus on them.• Provide data to support library funding and university accreditation.Can be varying degrees of specificity and quantifiable accuracy.• Likert scale (How much do you agree or disagree with the following?)• Multiple choice (Which would you use?)• Specific tasks (Can you/How would you do this?)Can be developed in-house, or purchased.• In-house has low cost and moderate to high development effort, but can be customized to exactly fit the library’s needs.• Purchased has high cost, but low effort, and can be easily used for accreditation documentation.
  9. 9. Developing Your Own In-house development starts with three decisionsWhat sort of test will it be?• An overall information literacy assessment?• A diagnostic test to help students decide on a course?• A general feeling of where students are at?• A detailed assessment of existing skills?How long will it be?• Short tests are more likely to be voluntarily taken.• Long tests give more detailed information.What type of results will it give?• Quantifiable evidence to show improvement?• Raw data to for teacher planning?• Analyzed data to advise students?
  10. 10. Question Types Different purposes need different typesLikert scales• Best to give a general feeling about student needs.• Details improve result accuracy, but I “strongly agree” poor phrasing/details can lead to that I’m a big fish! lower post-class ratings.Fixed Choice (Multiple choice, true/false, matching)• Best for detailed, quantifiable info about current knowledge.• Helps teachers identify specific knowledge holes.• Makes it easy to demonstrate post-class improvements.Scenario/Task Completion• Best possible proof of skills.• Difficult to develop in-house.
  11. 11. Example TestsThese sites contain samples of multiple-choice informationliteracy tests which can be used to help design your own.Appalachian State University College College Albert State College (see p79-86)CSU-Dominguez Hills University Verne University University University (see Appendix A)San Jose State University (see Appendix A)Stockton College (see Appendix B)West Chester University (Sports Med & Psychology)
  12. 12. Free Information Literacy TestsBeile Test of I. L. Test&discussion: • Assesses impact of instruction.for Education • Multiple-choice questions.(B-TILED) Validation: • Library implements & scores.NILRC I.L. Toolkit, • Lickert, multiple choice, T/F,Needs and open-ended questions.Assessment • Provides assessment summaryInstrument & composite results.SKIL - Stanford’s • Primarily a training module.Key to I.L. • Feedback after each question.TRAILS-ToolsforReal- • Gives class scores, individualTime Assessment of scores & data, and eachI.L. Skills) students own answers.
  13. 13. Commercial I.L. Testing Services iSkills TM Research strategy, tools, proper Developing search terms, use of sources, and economic, accessing database articles, and legal, & social issues. website reliability.Questions Multiple choice Scenario/Task completionNotes Based on ACRL objectves Designed in co-operation with 2- and 4-year colleges.Results Scores by individual, cohort, Raw data, and aggregated & major, and class level detailed test scores by individual & schoolLength 45 minutes 60 minutes + demographicsCosts $4/student up to 1000, $18-$20 per student. $4000 for 1000-5000 students
  14. 14. Free Computer Skills Self-Tests These self-tests all show the score & missed items to the student at the end. Very Basic: “Click on the picture of the monitor” and “Which of these is a valid email address?” Stevens-Henager Comp. Computer, word processing, Literacy Assessment internet, email & netiquette Northstar Basic Comp. Computer use, web, Windows/DIFFICULTY Skills Mac OS, email & MS Word Miami U. Basic Comp. Internet, email, MS Word, and Skills Self-Assessment operating systems Collin College Microsoft Multiple skill levels of Office Skills Assessment complete MS Office suite SkillsAssessment: Online Multiple skill levels of MS Training Needs Analysis office & general computing, gives detailed results analysis Advanced: “What would you need to do to the table below in order to use it for a mail merge?”
  15. 15. Commercial Computer Certifications Why consider certification programs?• Initial part of the process can be used as pre-evaluations• Produces quantifiable results• Students can get widely recognized certifications• Libraries may become approved testing centers to reduce costs International Computer Driving Internet & Computing Core License CertificationQuestions Multiple choice VariousEvaluates End-user computer skills Hardware, software, networks, internet skillsCosts $120 - $250 per student $99 - $420 per student.