Recommended
More Related Content
What's hot
What's hot (20)
Similar to Impactful Feedback to Improve eLearning
Similar to Impactful Feedback to Improve eLearning (20)
Recently uploaded
Recently uploaded (20)
Impactful Feedback to Improve eLearning
- 2. www.5leap.com Agenda Did the training have an impact? TIP OF THE ICE BERG • Qualitative vs Quantitative • Kirkpatrick model • Discussion forums Methodology • Reporting • Follow-up How to use the feedback Tricks of the trade 2
- 3. www.5leap.com Methodology 3 Qualitative deals with subjective feedback, what did they like and what did they not like Quantitative deals with numerical feedback, giving trends and patterns. Qualitative vs Quantitative
- 4. www.5leap.com Qualitative – Expressed in words 4 “How did it compare to other Sessions on this program/ on this subject?” “What did you specifically dislike?” “What did you specifically like?” “How will you use what you learned during this training?”
- 5. www.5leap.com Quantitative – Rated in numbers “How would you rate the format and structure of the training?” “How valuable do you think the concepts and skills will be to you in your work? “ " What is your overall opinion of the course that you just took?” “How would you rate the activities and exercises in the session?” 5
- 6. www.5leap.com Kirkpatrick Four levels of training evaluation (and sometimes there is a fifth…) Learning 2nd level ”What did the trainees actually learn?” Reaction 1st level "Did the trainees feel that the training was worth their time?” Behaviour 3rd level "Did the trainees put any of their learning to use?” Results 4th level ”What has changed, over time?” ROI Theoretical? ”Show me the money”
- 7. www.5leap.com Kirkpatrick – practical application 7 1st level 0 2nd level -1 3rd level 3w 4th level 3m Evaluation is given immediately after training is over. Both Qualitative and Quantitative questions Integral part of the training To measure learning, both a post and a pre- quiz is needed. Pre-quiz also helps prepare the trainee for what is important Pre and post quiz Can be done through local partners, personally or combined with questionnaires Observe Requires the before data, overview over other actions taken and ideally a control group Track results
- 8. www.5leap.com Ask the question first, then follow up with the learning Demanding style Most LMS tools support modules that have multiple paths, depending on what trainee answers in questions Multipath modules Use discussion forums, local champions and email/Yammer groups to follow up after training Learning groups
- 9. www.5leap.com Follow up after training Continuous improvements of training modules and building trainee skills Reporting Using feedback to improve training status Using the feedback
- 10. www.5leap.com Dealing with stakeholders 10 1. What is the problem the stakeholders want to solve? 1. Expected result/impact 2. Was any new problems discovered during training? 2. What are the next steps? 1. Individual follow ups 2. Progress reports 3. Building 3. Who took part and how did they score? 4. Increase your own visibility
- 11. www.5leap.com Post training follow up 11 It is not up to (only) you • Ask trainees to recommend others that would benefit from the training • Challenge trainees to support and coach others with their new knowledge/skills, e.g. be “local champions” P2P Learning • Challenge the trainees with a quiz, award best scores • Add the previous training to any rolling knowledge assessment Quiz • Post statistics related to the training • Ask for feedback on how training was applied and share anecdotes Communication
- 12. www.5leap.com Using social networks as feedback tools 12 On the same level • Exchange ideas and experiences • Faster communication • Identify high potential • Written communication is often positive for introverts TRAINEE GROUPS Invite managers to take part of the forums and social groups, increasing engagement and visibility Increase visibility • Write what they learned during training • How did they apply the new skills/knowledge • Did the training lead to expected outcomes, if so why? Or why not? Challenge trainees
- 13. www.5leap.com PORTFOLIO PROJECTS 04 13 RIGHT RANGE Asking the trainees to rate between 1-5 will lead to a higher avg. rating than 1-10 WORDS USED In the feedback, are the words negatively or positively charged? ASK “What do you want to learn” is a good start on managing expectations SET EXPECTATIONS Inform trainees in advance how they will be measured, this will lead to better scores OPEN BOOK If you use an Open book style exam, start your exam with 2-3 questions referring to specific texts
- 14. www.5leap.com 1 4 What do the trainees feel had the greatest impact and how do they rate your training overall Use questions during training, to increase engagement and drive learning (Discovery learning) Learning does not end with the trainee leaving the “classroom.” Set expectations for continued learning and continueto gather data after end of training Report both progress and final results to stakeholders MEASURE PROGRESS QUALITATIVE AND QUANTITATIVE
- 15. www.5leap.com THANK YOU There is just one more thing… How would your rate this presentation?
Editor's Notes
- Allows you to ask in-depth questions, discovering how users reacted, what they felt were particularly good. It also gives you good “sound bites” for your reports. - Gives the voice of the individual trainee
- The main benefit over quantitative feedback is that it allows for building a track record of how well a training have been received, across subjects and formats. It makes progress very visible and allows for easy reporting. Gives the voice of the overall trainee population - TIP: Ratings from 1-5 gives better overall scores, than 1-10. Especially in the Nordic countries) - TIP: Report in percentage scores
- Kirkpatrick is perhaps the most known and well respected model for training evaluation. The four levels of the training model, is progressively growing in complexity and cost to implement. Most organizations can easily implement level 1 and 2, while level 3 requires investment. The fifth level is not technically first invented by Kirkpatrick, but it is often associated with the model.
- Key for all these levels are that they are data driven, and level 2-4 should all be done in terms of Quantitative tracking, e.g. use of standard tracking sheets. Level 3 can also be measured through a trainees ability, to train another. If there is not distinct improvements, training will not have been successful.
- Integrating Kirkpatrick with eLearning – 3 tips Any other suggestions /ideas?
- This is in the order of priority when giving feedback to stakeholders, it will help build training status. Training should also have Very few trainers use the data they collect to build their reputation with the managers who sent people on the courses, and as management information to senior managers who have control of the budget We achieved 95% Excellent or Good ratings". That is so much more compelling than "We got 67% Good ratings and 28% Excellent Ratings. Discovery activity, eg “your results are 0% Poor, 4% Fair, 67% Good and 27% Excellent ratings.” How would you sell this to senior management?
- Some tricks of the trade
- Lets put this to the test
- Lets put all into practice