Keiser Wuhan


Published on

Information literacy action planning

Published in: Education, Business
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Keiser Wuhan

    1. 1. Strategies for implementing successful IL action plans Barbie E. Keiser UNESCO IL TTT Workshop Wuhan University October 2008
    2. 2. What we’ll cover during this session <ul><li>Strategic planning for IL </li></ul><ul><ul><li>Environmental scanning </li></ul></ul><ul><ul><li>SWOT/TOWS analysis </li></ul></ul><ul><ul><li>Critical Success Factors (CSFs) </li></ul></ul><ul><ul><li>The team </li></ul></ul><ul><li>Needs assessment </li></ul><ul><ul><li>Methods for conducting </li></ul></ul>
    3. 3. And what we won’t <ul><li>Organizational structure and programmes within UNESCO </li></ul><ul><ul><li>Education </li></ul></ul><ul><ul><li>Communication & information (IFAP) </li></ul></ul><ul><ul><li>Bangkok ( ) </li></ul></ul><ul><li>Differentiating among education, training, and guidance </li></ul><ul><li>ACRL Information Literacy IQ (Institutional Quotient) Test ( ) </li></ul>
    4. 4. What do we know about strategic planning? <ul><li>Concerns the relationship of an organization to its environment </li></ul><ul><li>Involves wide-range scanning of external and environmental factors </li></ul><ul><li>Flexible, dynamic and continually reworked plans maximize results </li></ul><ul><li>Participative </li></ul><ul><li>Shorter and longer-range plans are interwoven into a continuous strategy </li></ul><ul><li>Forward-looking; future-oriented </li></ul><ul><li>Iterative, ongoing effort </li></ul><ul><li>Proactive; seek opportunities </li></ul><ul><li>Bottom-up decision process </li></ul><ul><li>Environment considered ever-changing and dynamic </li></ul><ul><li>Integrated focus </li></ul><ul><li>Requires creativity to deal with new opportunities and choices </li></ul><ul><li>Incentives given for overall performance of the organization </li></ul>
    5. 5. Key planning issues <ul><li>Review past performance </li></ul><ul><ul><li>Understand reasons for past failures </li></ul></ul><ul><li>Identify opportunities </li></ul><ul><li>Determine client/customer and learner preferences </li></ul><ul><li>Understand the impact of IL training on existing operations and staff function </li></ul><ul><li>Marketing </li></ul>
    6. 6. Potential planning pitfalls <ul><li>Inability to get management and/or staff involved </li></ul><ul><li>Lack of clear objectives </li></ul><ul><li>Not relating IL goals and objectives directly to those of our organization/ institution </li></ul><ul><li>Other? </li></ul>
    7. 7. What strategies can we use to assure that we do not fall into these traps? <ul><li>Establish strong partnerships (Jamaica) </li></ul><ul><ul><li>Early, strong, consistent, and growing </li></ul></ul><ul><ul><li>Be careful in selecting your champion (Quebec) </li></ul></ul><ul><ul><li>Listen </li></ul></ul><ul><ul><ul><li>Be responsive </li></ul></ul></ul><ul><ul><ul><li>Learn how to say “no” </li></ul></ul></ul><ul><li>Appreciate the art of persuasion </li></ul><ul><ul><li>Tell stories (Quebec) </li></ul></ul><ul><ul><ul><li>Storytelling trumps statistics </li></ul></ul></ul><ul><ul><li>Demonstrate value to all stakeholder groups (WIIFM and ROI) </li></ul></ul><ul><ul><li>Share the information expertise of your staff with other knowledge workers </li></ul></ul><ul><ul><ul><li>Added benefits? </li></ul></ul></ul><ul><ul><li>Employ the vocabularies of target stakeholder groups (i.e., no library jargon) </li></ul></ul><ul><ul><ul><li>Changing the message and the focus </li></ul></ul></ul><ul><li>Demonstrate the need </li></ul><ul><li>Place IL skills in context </li></ul><ul><li>Identify where IL skills are already being taught/in use </li></ul><ul><li>An added bonus : A focus for our advocacy efforts </li></ul><ul><ul><li>“ New” product(s) to “market” </li></ul></ul><ul><ul><li>Reinvented service </li></ul></ul><ul><ul><li>A case for proactivity in a way we haven’t seen before </li></ul></ul><ul><ul><ul><li>Opportunity to market </li></ul></ul></ul>Sources : Caroline Stern
    8. 8. Where are we in the strategic planning process for IL? <ul><li>Perform an environmental scan </li></ul><ul><li>Conduct a SWOT/TOWS analysis </li></ul><ul><li>Initial assumptions (and their bases in fact ) </li></ul><ul><ul><li>Never assume! </li></ul></ul><ul><ul><li>Extant data (collected and reviewed) </li></ul></ul><ul><li>Identify Critical Success Factors (CSFs) </li></ul><ul><ul><li>Indicators Measures </li></ul></ul><ul><li>Develop vision, mission, and values statements </li></ul><ul><ul><ul><li>Envisioning your IL programme </li></ul></ul></ul>
    9. 9. Environmental scanning <ul><li>Detects trends and events important to the project (IL training programmes) </li></ul><ul><li>Provides early warning of changing external conditions </li></ul><ul><li>Defines potential threats, opportunities, changes implied by trends and events </li></ul><ul><li>Promotes a future orientation in the thinking of stakeholders </li></ul><ul><li>Enables decision-makers to understand current (and potential) changes to determine organizational strategies </li></ul><ul><li>What are the triggers in your institutions/ organizations/communities (i.e., indicators that IL training is needed that will resonate with your community )? </li></ul>
    10. 10. Performing a comprehensive environmental scan <ul><li>IL models and standards (learning theories) </li></ul><ul><ul><li>Methods ? (MyBookmarks) </li></ul></ul><ul><li>Your institution/organization/community </li></ul><ul><ul><li>Methods ? </li></ul></ul><ul><li>Previous IL (and other training) efforts </li></ul><ul><ul><li>Understanding reasons for success/failure </li></ul></ul><ul><ul><li>What kind of extant data do you collect ( and review )? </li></ul></ul>
    11. 11. IL models and standards guidelines <ul><li>Country models </li></ul><ul><ul><li>National Information Literacy Framework (Scotland) </li></ul></ul><ul><ul><li>Information Literacy Framework for Schools (Hong Kong) </li></ul></ul><ul><ul><li>Australian and New Zealand IL Framework: Principles, Standards, and Practice (ANZIL) </li></ul></ul><ul><ul><li>US School Library Media Center Questionnaire ( ) </li></ul></ul><ul><li>Big6 Information Problem-Solving Process ( ) </li></ul><ul><ul><li>Task definition </li></ul></ul><ul><ul><li>Information seeking strategies </li></ul></ul><ul><ul><li>Location and access </li></ul></ul><ul><ul><li>Use of information </li></ul></ul><ul><ul><li>Synthesis </li></ul></ul><ul><ul><li>Evaluation </li></ul></ul><ul><li>AASL ( ) and CASL ( ) </li></ul><ul><li>ACRL (higher education) guidelines (competencies) - , toolkit and </li></ul><ul><ul><li>Five standards </li></ul></ul><ul><ul><li>Performance indicators (and outcomes for each) </li></ul></ul><ul><li>Seven Pillars ( ) </li></ul><ul><li>Information Inquiry, Problem-Solving and Research Process </li></ul>
    12. 12. Conducting a SWOT analysis <ul><li>Useful when you need to understand your own competitive advantages </li></ul><ul><li>Provides information helpful in matching your organization’s resources and capabilities to the environment in which it operates </li></ul><ul><li>Scan of the internal and external environment, beginning externally </li></ul><ul><li>Environmental factors internal to the institution/organization/library/community = Strengths or Weaknesses </li></ul><ul><li>Environmental factors external to the institution/organization/library/community = Opportunities or Threats </li></ul><ul><li>Sometimes can be too inward </li></ul>
    13. 13. SWOT Analysis <ul><li>Threats </li></ul><ul><li>Opportunities </li></ul><ul><li>Weaknesses </li></ul><ul><li>Strengths </li></ul>
    14. 14. TOWS <ul><li>An extension of the SWOT analysis </li></ul><ul><li>Analyze the external environment (threats and opportunities) and your internal environment (weaknesses and strengths) to help you think about the strategy of your organization </li></ul><ul><ul><li>Useful for marketing campaigns </li></ul></ul><ul><li>Threats and opportunities </li></ul><ul><ul><li>External environmental factors over which you do not have control (changing demographics) </li></ul></ul><ul><li>Weaknesses and strengths </li></ul><ul><ul><li>Internal factors (poor location; bad reputation) </li></ul></ul>
    15. 15. TOWS Strategic Alternatives Matrix WT “Mini-Mini” Strategy Strategies that minimize weaknesses and avoid threats WO “Mini-Maxi” Strategy Strategies that minimize weaknesses by taking advantage of opportunities Internal Weaknesses (W) 1. 2. 3. 4. ST “Maxi-Mini” Strategy Strategies that use strengths to minimize threats SO “Maxi-Maxi” Strategy Strategies that use strengths to maximize opportunities Internal Strengths (S) 1. 2. 3. 4. External Threats (T) 1. 2. 3. 4. External Opportunities (O) 1. 2. 3. 4.
    16. 16. What are the CSFs for your IL training programme?
    17. 17. Key questions <ul><li>Is your community ready? </li></ul><ul><ul><li>What to do if it is not </li></ul></ul><ul><li>Is your institution/ organization’s culture a barrier? </li></ul><ul><ul><li>How to deal with that </li></ul></ul><ul><li>Where did the idea for IL training originate? </li></ul><ul><ul><li>You or others </li></ul></ul><ul><li>ACRL </li></ul><ul><li>What do you want learners to be able to do? </li></ul><ul><li>What do learners need to know in order to do this well? </li></ul><ul><li>What type of instruction will best enable the learning? </li></ul><ul><li>How will the student demonstrate the learning? </li></ul><ul><li>How will you know that the learner has learned? </li></ul>Source: Ruth Pagell, SMU
    18. 18. Scope of your IL training <ul><li>Comprehensive or narrow(er)? </li></ul><ul><ul><li>Pilot project approach </li></ul></ul><ul><ul><ul><li>One subject, that can then be extended to others </li></ul></ul></ul><ul><ul><ul><li>One tool, that can then be extended to similar tools </li></ul></ul></ul><ul><ul><li>Our responsibility is to help learners make those connections! </li></ul></ul><ul><li>Focus on what the learner needs to know, teaching them how to ask the right question (if they want to get the right answer) </li></ul><ul><li>Begin by asking what the learner already knows about the subject, and then…. </li></ul>
    19. 19. Questions for the learner (Framework for IL – Scotland) <ul><li>What’s the most likely place you will find the answer?: Was this choice “the best”? </li></ul><ul><li>What words can you use to search effectively to improve on your existing knowledge?: Was the strategy “the best”? </li></ul><ul><li>How do you know when you’re finished?: Did learner assess correctly? </li></ul><ul><li>Have you learned something new? </li></ul><ul><li>Who else should know this (and how should this be shared - ethically )? </li></ul><ul><li>How will you apply this now? </li></ul><ul><li>What have you learned from this experience that you can apply elsewhere? </li></ul>
    20. 20. Overview of the process <ul><li>Goals and objectives </li></ul><ul><ul><li>Strategies and tactics </li></ul></ul><ul><li>Target population </li></ul><ul><li>Type of training </li></ul><ul><ul><li>Alternatives </li></ul></ul><ul><li>Project planning </li></ul>
    21. 21. Team approach <ul><li>Who should be included on the team? </li></ul><ul><li>Who is responsible? </li></ul><ul><li>When is it due? </li></ul><ul><li>Consider what’s needed to gain institutional commitment and stakeholder “buy-in” </li></ul>
    22. 22. Needs assessment: Knowing your “market” <ul><li>Definition : a planned, systematic approach to determining the information needs of each distinct customer group </li></ul><ul><li>Purpose : Help you develop training targeted specifically to each group and need </li></ul><ul><li>Diverse set of learners, each with distinct set of needs (and learning styles) </li></ul><ul><li>Understand your targets </li></ul><ul><ul><li>Why they need to improve their IL skills (direct impact) </li></ul></ul><ul><ul><li>What would persuade them that IL is important to their success </li></ul></ul><ul><li>Identify groups with related needs </li></ul>
    23. 23. Methods <ul><li>At-the-elbow </li></ul><ul><ul><li>Usability-lite testing </li></ul></ul><ul><li>Easter egg hunt </li></ul><ul><li>Interviews </li></ul><ul><li>Surveys </li></ul><ul><li>Focus Groups </li></ul><ul><li>Questions </li></ul><ul><li>Analysis </li></ul>
    24. 24. Interviewing techniques <ul><li>Know the lingo… enough so that you can ask sensible questions and understand the answers </li></ul><ul><li>Decide in advance what each person is likely to be able to tell you </li></ul><ul><ul><li>What he/she would be interested in talking about </li></ul></ul><ul><li>Show respect for each individual’s expertise and explain your objectives </li></ul><ul><li>Introduce yourself and why you are calling (in general terms) </li></ul><ul><ul><li>Do not lie! You are not a student doing a research project. </li></ul></ul><ul><li>You are not conducting a survey; the objective is to engage them in conversation </li></ul><ul><li>Allow enough time to do the right job: 2-3 hours to find the right source, exchange voicemails, conduct the interview, and write the summary </li></ul><ul><li>If you are covering a field on a continuing basis, make interviews part of your routine </li></ul><ul><li>Listen! </li></ul><ul><li>Benefits of a two-on-one approach </li></ul>
    25. 25. The interview <ul><li>Begin with the general and work your way towards the items in which you are really interested </li></ul><ul><ul><li>Ask the least sensitive questions first, and when they are comfortable, proceed towards the more sensitive. Ask the most sensitive questions last. </li></ul></ul><ul><ul><li>Develop models or hypotheses that address key questions </li></ul></ul><ul><ul><li>Focus on what’s most important </li></ul></ul><ul><li>Don’t take up too much precious time (< 30 minutes) </li></ul><ul><li>Write a complete summary of the conversation. Capture both facts and nuances. </li></ul><ul><li>Do not tape the interview . (Illegal in many jurisdictions, unless you notify the party, and that will put them on their guard and they’ll likely not want to continue, or at least not be as forthcoming as you’d like.) </li></ul><ul><li>Be an attentive listener </li></ul><ul><li>Get referrals: Ask who else you should contact about the subject next </li></ul><ul><ul><li>Mention who suggested that you call that individual </li></ul></ul>
    26. 26. Characteristics of …. <ul><li>Surveys </li></ul><ul><ul><li>Success based on the length, knowledge of questioner (if telephone survey is used), and list </li></ul></ul><ul><ul><li>Important to pretest items/flow </li></ul></ul><ul><ul><li>Introduction to participate/ </li></ul></ul><ul><ul><li>Intro to survey/Follow-up with non-respondents </li></ul></ul><ul><ul><li>Anonymity/confidentiality </li></ul></ul><ul><ul><li>Response rate </li></ul></ul><ul><ul><li>Online </li></ul></ul><ul><ul><li>Incentives to complete </li></ul></ul><ul><ul><li>Time </li></ul></ul><ul><ul><ul><li>Timing of release </li></ul></ul></ul><ul><ul><ul><li>Time required to complete </li></ul></ul></ul><ul><ul><li>Sampling </li></ul></ul><ul><ul><li>Significance </li></ul></ul><ul><li>Focus groups </li></ul><ul><ul><li>Less formal way of soliciting consumer feedback on products and services </li></ul></ul><ul><ul><li>6-10 people + experienced facilitator </li></ul></ul><ul><ul><li>Need for ground rules </li></ul></ul><ul><ul><li>1.5 hours </li></ul></ul><ul><ul><li>Record the session </li></ul></ul><ul><ul><li>Disadvantage: small sample, so composition is key </li></ul></ul>
    27. 27. When to use surveys and focus groups for information gathering <ul><li>Use surveys when </li></ul><ul><ul><li>You need quantitative estimates/confidence intervals </li></ul></ul><ul><ul><li>You have a clear idea as to the questions you want to ask (and how) </li></ul></ul><ul><ul><li>You have the time to develop and test the instrument, and analyze and present the results </li></ul></ul><ul><li>Use focus group when </li></ul><ul><ul><li>You need insights; quantitative estimates are less important </li></ul></ul><ul><ul><li>Statistical analysis is not a necessity </li></ul></ul><ul><ul><li>You want flexibility in pursuing issues discovered during your inquiry </li></ul></ul><ul><ul><li>Limited in terms of time and/or budget </li></ul></ul>
    28. 28. Finding participants for surveys and focus groups <ul><li>And how those methods influence the validity of your analysis </li></ul>
    29. 29. Tips for conducting successful interviews and focus groups <ul><li>Let participants talk, but facilitate the discussion </li></ul><ul><li>May have a set of questions … but do not force a slavish go-through </li></ul><ul><li>Assure complete confidentiality – notes are aggregated, no names ever given out </li></ul><ul><li>If need to “prime the pump”, refer to observations (“we noticed …”) and ask for comments </li></ul><ul><li>Be aware of interpersonal dynamics and politics </li></ul><ul><li>Recognize that participants may not want to “look bad” & may tailor comments to what they think is “correct” </li></ul><ul><li>Validate: Interesting, you are not the first to say so </li></ul><ul><li>Use “others-find” technique (you too?) </li></ul>
    30. 30. Tips for conducting effective surveys <ul><li>Short - Fast – Easy – did I mention short ! </li></ul><ul><li>Clear, unambiguous (in terms of questions posed) </li></ul><ul><ul><li>Ask only one question at a time </li></ul></ul><ul><li>Logical flow of survey sections </li></ul><ul><li>Ranking of personal priorities (What means more to you?) </li></ul><ul><li>Minimize the number of open-ended questions </li></ul><ul><ul><li> </li></ul></ul><ul><li>Do you agree with these statements made by your peers? </li></ul><ul><li>Lead with interesting questions, enticing people to respond </li></ul><ul><li>Include quick demographic questions at the end to aid in analysis </li></ul>
    31. 31. Questions to pose concerning facts <ul><li>Why should I believe it? </li></ul><ul><li>Does the claim need evidence to support it? </li></ul><ul><li>If there is evidence provided, how good is the evidence? </li></ul><ul><li>Other plausible interpretations? </li></ul><ul><li>What reasonable alternative conclusions are possible? </li></ul>
    32. 32. What we’ll cover during this session <ul><li>Moving from goals and objectives to who will do what (and when) </li></ul><ul><li>Plus a bit on…. </li></ul><ul><ul><li>How the training should be delivered </li></ul></ul><ul><ul><ul><li>Options available </li></ul></ul></ul><ul><ul><li>Monitoring performance and measuring success </li></ul></ul><ul><ul><li>Student assessments </li></ul></ul><ul><ul><li>Overall programme, including cost/benefit (ROI) and value analysis </li></ul></ul>
    33. 33. The overall goal is information fluency and developing the lifelong learner 1 Establish specific objectives for your IL training program 2 Develop IL training strategies 3 Implement IL training programme(s) 4 Evaluate IL training accomplishments Grade level; undergrad/grad/ researcher/ worker Classroom/Online BI, ICT, and more General/Basic, stand-alone Integrated, subject-specific Required or optional? One-time or semester? Credit/other reward/incentive? IMPROVE!
    34. 34. 1a. Establish objectives <ul><li>What (specific) competencies must your “audience” possess? </li></ul><ul><ul><li>Focus on the ends, not the means </li></ul></ul><ul><ul><li>From four perspectives </li></ul></ul><ul><ul><ul><li>Audience(s)/learners </li></ul></ul></ul><ul><ul><ul><li>Competence </li></ul></ul></ul><ul><ul><ul><li>Condition under which performance will be observed </li></ul></ul></ul><ul><ul><ul><li>Criteria for success </li></ul></ul></ul><ul><li>What gaps must be addressed? </li></ul><ul><ul><li>Identify required and desired proficiencies </li></ul></ul><ul><ul><li>Identify deficiencies ( and their causes ) </li></ul></ul><ul><ul><li>Identify non-training (e.g., availability equipment) and training (e.g., skills of trainers) issues </li></ul></ul><ul><ul><ul><li>What innovative approaches can be used? </li></ul></ul></ul><ul><ul><ul><li>Benefits of blended learning </li></ul></ul></ul><ul><ul><ul><ul><li>Social networks for viral spreading knowledge gained </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Web 2.0 to reinforce over time </li></ul></ul></ul></ul>
    35. 35. Worksheet 1 Deficiencies identified Competencies sought
    36. 36. The Plan <ul><li>Positioning statement : How do you want the project to be perceived? </li></ul><ul><li>Key message(s) : What is the most important message that you wish to deliver? </li></ul>
    37. 37. Creating priorities within competencies sought (ISD) <ul><li>Based on the importance of knowing what your target learners need to DO </li></ul><ul><ul><li>How frequently is the task performed? </li></ul></ul><ul><ul><li>How critical is the task to performance? </li></ul></ul><ul><ul><li>How difficult or complex is the task? </li></ul></ul><ul><ul><ul><li>If a subset of collective tasks, what is the relationship among tasks? </li></ul></ul></ul><ul><ul><li>To what extent will training for this task be encountered elsewhere? Ability to apply knowledge </li></ul></ul><ul><ul><li>What prerequisite skills, knowledge, and abilities are required to perform the task? </li></ul></ul><ul><ul><li>What is the current/desired criteria for acceptable performance? </li></ul></ul><ul><ul><li>What behaviors distinguish good performers from poor? </li></ul></ul><ul><ul><li>What behaviors are critical to the performance of the task? </li></ul></ul>
    38. 38. Non-training and training issues <ul><li>Have all library staff (faculty) been trained to facilitate IL learning? </li></ul>Knowledge/skills <ul><li>Are there mechanisms in place to measure performance and provide feedback to performers? Trainers? </li></ul><ul><li>How do achievers receive recognition? What are the consequences of not attaining IL goals? </li></ul><ul><li>What resistance exists? </li></ul>Motivational factors align incentives/remove disincentives <ul><li>Will learners have adequate access to tools/technology, resources, or information to achieve these goals? </li></ul><ul><li>What are the standards or expectations beyond the academic? </li></ul><ul><li>Do “rules” facilitate or impede goal attainment? </li></ul>Environmental factors acknowledge that training alone is unlikely to resolve performance gaps Questions to ask Types of causes
    39. 39. People learn differently! <ul><li>Doers </li></ul><ul><li>Thinkers </li></ul><ul><li>Feelers </li></ul><ul><li>Seeing is believing vs. auditory (Wharton study) </li></ul><ul><li>Importance of reinforcement </li></ul><ul><li>Storyboarding, scenario building, case studies/examples </li></ul>
    40. 40. 1b. Establish objectives <ul><li>How does each contribute to the overall strategic goal for information fluency ? </li></ul><ul><li>What approach(es) will you take? </li></ul><ul><ul><li>What has been successful for you in the past (and why)? </li></ul></ul><ul><li>What specific results (outcomes) must be accomplished so that you can get closer to your goal of information fluency? </li></ul><ul><ul><li>How those results will be achieved is explained in Step 2 </li></ul></ul><ul><li>How will you “market” this effort? </li></ul>
    41. 41. Worksheet 2 Expressed as a Noun Expressed as a Verb For each activity or task there is a result For each objective, there is an activity or task Assess-ment Result/ Outcome Rationale Objective
    42. 42. 2a. Develop IL training strategies <ul><li>What training ( content ) could address IL competence gaps? </li></ul><ul><li>What format should that training take? </li></ul><ul><ul><li>Learning environment </li></ul></ul><ul><ul><li>How do you make those decisions? </li></ul></ul><ul><ul><ul><li>Staff competencies and time available </li></ul></ul></ul><ul><ul><ul><li>Generate alternative training strategies for addressing (specific) IL gaps </li></ul></ul></ul><ul><ul><ul><li>List all trainings considered/selected (and rationale) </li></ul></ul></ul><ul><ul><ul><ul><li>What innovative approaches can be used? </li></ul></ul></ul></ul>
    43. 43. Is “classroom” training appropriate? Tool: MS Visio
    44. 44. Training worksheet Why use/discarded Types of training considered
    45. 45. 2b. Develop IL training strategies <ul><li>What are the projected (life-cycle) costs for (developing and implementing ) each type of training to be offered? </li></ul><ul><ul><li>Specific benefits anticipated, both tangible and non-tangible? </li></ul></ul><ul><li>What are the consequences to the organization and library strategic goals of not offering IL training? </li></ul>
    46. 46. 3. Implement IL training programme(s) <ul><li>Write your IL training goals and make them known </li></ul><ul><ul><li>Collaboration, teamwork, marketing/pr </li></ul></ul><ul><li>Identify performance measures and indicators (outcomes and impact) </li></ul><ul><ul><li>How will you benchmark performance prior to taking the training (e.g., pre-testing)? </li></ul></ul><ul><ul><ul><li>Mechanisms for assessing IL post-training </li></ul></ul></ul><ul><ul><ul><ul><li>Immediate and longer-term </li></ul></ul></ul></ul><ul><ul><ul><li>Continuous improvement process </li></ul></ul></ul><ul><ul><ul><li>Measuring self-sufficiency achievements </li></ul></ul></ul><ul><ul><ul><li>Understanding Top Box scores </li></ul></ul></ul><ul><li>Develop an action plan </li></ul>
    47. 47. Action plans <ul><li>Strategy </li></ul><ul><li>Tactics </li></ul><ul><li>Evaluation and control </li></ul><ul><li>Results </li></ul>
    48. 48. Develop an action plan <ul><li>What will be done? </li></ul><ul><ul><li>Key tactics to support the strategies </li></ul></ul><ul><ul><li>Identify specific tasks to be completed </li></ul></ul><ul><li>By whom? </li></ul><ul><ul><li>For whom? </li></ul></ul><ul><li>By when? </li></ul><ul><ul><li>Timelines for each objective </li></ul></ul><ul><li>What resources are required (including financial)? </li></ul><ul><li>Who should know/be involved? </li></ul><ul><ul><li>Collaborators and stakeholders </li></ul></ul><ul><li>How will you “market” the effort? </li></ul>
    49. 49. Worksheet 3 When Who What Timetable Personnel (Lead) Training Opportunity Requirements (Resources/$) Action step (initiative/task)
    50. 50. Sample Gantt chart presentation             Prepare detailed business plans 2.5         Impact analysis of different scenarios 2.4       Specify technical developments required 2.3       Review of existing facilities 2.2     Inventory of resources in selected regions 2.1                             Specification of Integrated System 2     Evaluation of methodology 1.3           Draft of methodology 1.2   Workshop on user needs 1.1               Development of Methodology 1 14 13 12 11 10 9 8 7 6 5 4 3 2 1 Description Months Task
    51. 51. 4. Evaluate IL training goal accomplishments (ISD) <ul><li>Did you achieve the training goal? </li></ul><ul><li>How much did it cost? </li></ul><ul><li>Did accomplishing your IL training goal help the organization/institution achieve larger goals? </li></ul><ul><li>What modifications should be made to the plan, based on the evaluation findings? </li></ul>
    52. 52. Training costs <ul><li>Development costs (personnel and equipment) </li></ul><ul><li>Direct implementation costs (e.g., training materials, instructor travel/per diem, facilities) </li></ul><ul><li>Indirect implementation costs (overhead, G&A) </li></ul><ul><li>Compensation for participants </li></ul><ul><li>Lost productivity or costs of “backfilling” positions during training </li></ul><ul><ul><li>Developer </li></ul></ul><ul><ul><li>Instructor </li></ul></ul><ul><ul><li>Faculty </li></ul></ul>
    53. 53. Training benefits <ul><li>Time/resource savings </li></ul><ul><li>Improved quality </li></ul><ul><li>Error reduction </li></ul><ul><li>Allow the learner to do something not possible before </li></ul>
    54. 54. Your business case <ul><li>Are the projected benefits (to the individual, library, faculty, school, organization, community) consistent with strategic performance goals? </li></ul><ul><li>What are the consequences if IL training did not occur (or did not occur here)? </li></ul><ul><li>Do the potential benefits outweigh the costs? </li></ul><ul><li>What is the value added from closing IL competency gaps? </li></ul>
    55. 55. Best practices for implementing training programmes <ul><li>Demonstrate results: Performance measures should tell each target group how well it’s achieved its goals (individual, faculty, library, school) </li></ul><ul><li>Limited to the vital few: Measures should cover key performance dimensions – Too much data may obscure rather than clarify (expensive) </li></ul><ul><li>Link to departments: Performance measures should be linked directly to offices responsible for making training work (library and faculty) </li></ul>
    56. 56. How can we assess learning and training goal achievement? <ul><li>Pre- and post-testing </li></ul><ul><ul><li>Delayed post-tests </li></ul></ul><ul><ul><li>Anonymity </li></ul></ul><ul><li>Interviews (in-person, phone) </li></ul><ul><li>Survey </li></ul><ul><li>Work samples/co-grading reports </li></ul><ul><li>Existing monitoring and reporting mechanisms (extant data) </li></ul><ul><li>Each data collecting method has advantages and disadvantages </li></ul>
    57. 57. Moving from teacher-centered to learning and learner-centered training Source: Cox and Lindsay Learning Outcome(s) Learning Objective(s) Learning Goal
    58. 58. Measuring targeted learning behaviors Source: Cox and Lindsay Proficient Low proficiency Not proficient Targeted behaviors
    59. 59. Worksheet 4 Noun Verb Assessment (method and impact) Result/ Outcome Rationale Objective
    60. 60. Types and quality of assessments (ACRL)/ Examples <ul><li>Types </li></ul><ul><li>Formal </li></ul><ul><li>Informal </li></ul><ul><li>Traditional (test) </li></ul><ul><li>Authentic (real life task) </li></ul><ul><li>Integrated </li></ul><ul><li>Knowledge/Content-based </li></ul><ul><li>Formative </li></ul><ul><li>Summative </li></ul><ul><li>Self-Assessment (pre-and post) </li></ul><ul><li>Peer </li></ul><ul><li>Portfolio </li></ul><ul><li>Quality </li></ul><ul><li>Collaborative </li></ul><ul><li>Multi-dimensional </li></ul><ul><li>Holistic </li></ul><ul><li>Assess the thinking process </li></ul><ul><li>Include critical thinking elements </li></ul><ul><li>Managed </li></ul>
    61. 61. Creating your assessments <ul><li>Create a list of KSAs: What should learners know and be able to do? </li></ul><ul><li>Identify standards that the learners need to meet </li></ul><ul><li>Design some tasks that will illustrate whether learners have grasped concepts </li></ul><ul><li>Determine what signifies “good” performance </li></ul><ul><li>Develop rubrics to be used in grading and course redesign </li></ul>
    62. 62. Assessment models and samples <ul><li>TRAILS: Tool for Real-time Assessment of Information Literacy Skills ( ) </li></ul><ul><li> </li></ul><ul><li> </li></ul><ul><li> </li></ul><ul><li>iSkills from ETS </li></ul><ul><li>Additional assessment resources ( ) </li></ul>
    63. 63. Monitoring your performance as well <ul><li>Establish a tracking system to monitor both plan execution and impact </li></ul><ul><li>For each scheduled milestone, compare the actual performance with the anticipated, and report results </li></ul><ul><li>For all variances in scheduled performance… </li></ul>
    64. 64. Ask the following: <ul><li>How does the reported performance compare with the previous performance? </li></ul><ul><ul><li>Benchmark the starting line </li></ul></ul><ul><li>Is the performance/schedule variance likely to prevent goal achievement? </li></ul><ul><ul><li>Particularly when the “ stepped approach ” has been used </li></ul></ul><ul><li>Are external factors affecting performance? Which? </li></ul><ul><li>Is the variance due to unrealistic expectations (from planning stage)? What adjustments should be made? </li></ul><ul><li>What modifications should be made to the action plan? </li></ul><ul><li>What performance information should be collected now ? </li></ul>
    65. 65. Establishing the worth <ul><li>Of training </li></ul><ul><li>Subjective method for establishing the worth of improved performance (as a result of IL training) </li></ul><ul><li>Comparing costs and benefits by calculating total Return on Investment (ROI) </li></ul><ul><li>Assessing results </li></ul>
    66. 66. A more detailed illustration of the process
    67. 67. What you should have at the end of this session <ul><li>Model worksheets for creating action plans </li></ul><ul><li>Ideas to use as the starting points for your IL training efforts </li></ul><ul><li>Resources to consult </li></ul><ul><ul><li>My bookmarks </li></ul></ul><ul><ul><li>How you can share yours </li></ul></ul>
    68. 68. Comments? Questions? Suggestions? <ul><li>Thank you! </li></ul><ul><li>Barbie E. Keiser </li></ul><ul><li>[email_address] </li></ul><ul><li>[email_address] </li></ul>
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.