Recommended
Recommended
More Related Content
What's hot
What's hot (12)
Similar to RAIT Learning Evaluation - Overview
Similar to RAIT Learning Evaluation - Overview (20)
Recently uploaded
Recently uploaded (20)
RAIT Learning Evaluation - Overview
- 1. Measuring the Impact of Learning An Introduction to RAIT – Retention Application and Impact of Training
- 2. Why Do We Invest in Learning Activities? “An investment in knowledge always pays the best interest” Benjamin Franklin
- 3. Why Measure Learning Effectiveness? “Education is what survives when what has been learned has been forgotten.” B.F. Skinner
- 4. Why Measure Learning Effectiveness? “If you cannot “In God we trust. measure it, you All others must bring cannot improve it.” data” Lord Kelvin W. Edwards Deming
- 5. The Problem : Measuring Learning Effectiveness Difficult Barriers < 3%* Measurement Difficulty Results X Validity? Behaviour % of organizations X Retrospective measuring at different X Too Hard Learning evaluation levels Reactions < 89%* Easy HAPPY RESULTS Kirkpatrick Learning Evaluation Framework *Dean R. Spitzer, “Learning Effectiveness Measurement: A New Approach for Measuring and Managing Learning to Achieve Business Results”, Advances in Developing Human Resources Vol. 7, No. 1 February 2005 pp55-70
- 6. The Answer : RAIT –Scientific, Predictive & Easy RAIT is the result of 10 years of applied global research by Australian researchers into the neuroscience of learning. RAIT is a predictive measure of learning effectiveness for training, coaching and eLearning activities. RAIT is simple and easy. Two forms, 30 questions, 5 minutes each. Various reports & dashboards available. No customised data collection required.
- 7. RAIT is Based on Rigorous Applied Research • Who: Developed by Dr Stan Rodski, neuroscientist and psychologist, over the last 10 years. Scientific • Theoretical Basis: Critical incident theory, (Flanagan 1954) identifies an encounter of a learner that is particularly satisfying or dissatisfying (Bitner et al., 1990). RAIT maps variables derived from neuroscience to critical learning incidents. • Global Studies: Initial applied research. 400 variables; 50 corporate programs; 850 managers & staff; 5 multi-national companies. Tested capability of input variables to predict learning outcomes (mix of retention & application). • Results: Reduced inputs to 30 critical incident factors; 95% confidence in predicting outcomes to within +/-3% variation immediately after program and 6 months after completion. • Today: Global benchmark database of >5000 programs includes Harvard & Duke University. More recent research has provided valid and reliable ROI feedback with regard to training to Level 4 in the Kirkpatrick Learning measurement framework.
- 8. RAIT is a Predictive Measure • Learning Process: Focuses on learning process, not the specific content of the learning allows programs to be compared for learning effectiveness irrespective of content. • Predictive: Globally researched and benchmarked factors support prediction of learning effectiveness at 3 key data points: 6 months after, with coaching Immediate Impact Post- Event 6 months after, without coaching • Multiple Levels of Analysis: Reports are available at Individual, Program and various aggregate levels (dashboard, program, trainer, coach, etc).
- 9. RAIT is Easy : 5 Minutes to Administer • How: Two forms (paper or online). 5 minutes each. Participant scores 30 factors from 1-10. 2. Post-Learning: 1. Pre-Learning: How well did the How important are program perform on these factors to these factors for you? you? • No extra work: Not necessary to gather additional data on learning outcomes. Rely on the RAIT research-derived predictive factors, with 95% confidence. • Different Learning Modes: RAIT research has enabled predictive measures for training, coaching and eLearning.
- 10. In Summary
- 11. For More Information : raitinfo@navigatus.net
Editor's Notes
- Hello and welcome to this presentation on RAIT -- a tool which can be used to measure .... learning effectiveness.
- Let's start by revisiting a very basic assumption: Why do we invest in learning? <Click>Well, if we're in the business of business then we invest in learning because we believe it will contribute – in some way - to business results.<Click>And we can see Ben Franklin here -- supporting that claim from the centre of a $100 note: We invest in learning, in order to contribute to business results.
- The question then becomes: Why measure learning effectiveness.Well, there are quite a few reasons. Let’s look at two of them very quickly.<Click>The first is that learning decays – the learning inputs we provide have an inherent inefficiency.<Click>So, if we believe that learning inputs have a benefit, and yet we know that those benefits decay, then we also know that we have to take steps to improve. And that’s why we increasingly see follow-up initiatives such as coaching, reinforcement, and so forth. The challenge for us is continuous improvement.
- And that leads us to the second reason to measure learning effectiveness:<Click> If we’re not measuring, then all we’re doing is hoping, or trusting some kind of process.<Click>If we’re interested in quality though, trust is not enough ... we’re going to want to measure.
- But measuring learning effectiveness has some problems. Let’s take a look. <CLICK>Here’s the Kirkpatrick framework. It’s been around for over 50 years and we’re probably all familiar with its four levels. <CLICK>The issue is that the higher up the framework we go towards measuring results, the more difficult measurement becomes. <CLICK>And this is borne out in practice. Pretty much every organization measures at the bottom level – reaction, or satisfaction, or ... happiness. We’re essentially measuring whether people were happy with the learning activity. And we do that of course through ‘happy sheets’. The problem with that, <CLICK> is that happiness is not really correlated with results. Satisfaction is pretty easy to measure – and it’s very important to many training providers – but it’s not really helping us with our fundamental problem. <CLICK>Which lies at the other end of the spectrum, where hardly any organizations measure at the results level.In fact for all the layers above reaction, there are some barriers to measurement:<CLICK> The first is validity - how do we know that we've selected a the right measure for what we want to measure? And how can we be sure that we’ve stripped out all the confounding variables?<CLICK> Secondly, most measures are backward-looking. They’re fine for helping with justification, but not so useful for improvement.<CLICK> And finally – and perhaps it’s as simple as this – it all often feels just too hard. We have to ask people for their time; to put energy into our priorities, rather than their own. It’s a lot of work.So, what do we do?
- Well one answer – not the only answer of course – is RAIT, which has three characteristics which we’ll cover in this presentation.<Click>The first is that its based on science. Here’s a picture of a neuron! It’s based on neuroscience ... it must be wonderful!But seriously, it is based on the neuroscience of learning – the way that the brain ‘lights up’ to form memories and to motivate behaviours. It’s also based on global applied research into predicting learning effectiveness.<Click>And that’s the second main point – it’s a predictive measure. So you get to look forwards. You're not restricted to the rear-view mirror. <Click>And finally – and perhaps most importantly – it’s easy. It’s about as easy as administering a happy sheet. Only five minutes for the learner, and then you don’t need to collect any additional measurement data.Let’s take a look at each of these points in turn.
- First, the research.<Click>RAIT was developed by Dr Stan Rodski, who’s a neuroscientist, psychologist and statistician.Part of Stan’s early experience with neuroscience included working with children with brain injuries, to understand how the brain can re-map and re-learn. So he had all this detailed understanding of the neuroscience behind how our brains learn, in a very extreme environment.<Click>When Stan had the opportunity to look at adult learning, he took account of critical incident theory, which is a framework for understanding which events during a learning activity cause the brain to ‘light up’ and form memories and motivations. <Click>In the initial research studies, Stan used his knowledge of neuroscience to identify 400 potential critical incident variables.The research question was to test which factors could most reliably predict learning effectiveness, as a combination of retention (based on knowledge tests), and application (based on qualitative interviews).The initial research involved 850 learners across 50 corporate programs, globally across 5 continents.<Click>The results were that the 400 variables reduced to 30 predictive factors. And there was a 95% confidence interval in being able to predict outcomes within 3%.<Click>Today, RAIT is supported by a global database of more than 5000 programs including some well-known institutions. And the research program continues, more recently with prediction of level 4 outcomes such as Return On Investment and employee engagement.
- The second key point is that RAIT is a predictive measure.<Click>Now it’s very important to keep in mind that RAIT is interested only in the effectiveness of the learning process. RAIT is not interested in the content of the learning. So you might be teaching a sales program, or a leadership program. Or, frankly you might be trying to teach someone that the earth is flat. RAIT doesn’t care about the content – only about the learning process.<Click>RAIT is predictive. It doesn’t look backwards. It has all the predictive factors from the global research. And it predicts learning effectiveness for three time points:The first is, immediately on completion of a learning activity;The second, is 6 months after the activity, RAIT predicts the learning outcome assuming no follow-up activity;And the third is at 6 months assuming there have been follow-up learning activities.<Click>And there are reports available at individual, programme and various aggregate levels.
- The final point is that RAIT it is easy.<Click>It is two forms if it’s paper -- there are online versions available.The pre-learning form simply asks for each of the 30 factors, "How important are these factors to your learning".And the post-learning form – the red one - asks for the same 30 factors, "How well did the program perform for you".And there are different factors for training, coaching and e-learning.<Click>It's not necessary to collect any additional data. The research database has all the predictive factors. You can rely on RAIT to do all the heavy lifting for you.If you want to develop customised research parameters for a particular programme area such as sales effectiveness, or safety performance -- or for your organisation as a whole, then you can. However it is not necessary to do this in order to get started measuring with RAIT. <Click>And RAIT can measure different learning events as we have mentioned - training, coaching or e-Learning.
- So in summary, if we're simply relying on hope, then we're not sure where we might end up.<Click>The claim here of course is that, if we're measuring with RAIT, then we have a better chance of managing our learning processes towards the results that we want to achieve. Not that the measurement approach guarantees results, but it can provide guidance for improvement activities, as well as justification for past investments.<Click>RAIT is already used being used by a number of clients in Australia, and around the world.
- If you’d like to know more, please contact us.Thank you very much for your time.