The Impact of Learning Breakfast Briefing | 13 June 2013

844 views
737 views

Published on

Learning impact: avoiding the hype and proving the case

It seems the learning profession can’t agree on what learning impact is or how to measure it. The solution, according to Donald H Taylor of the Learning and Performance Institute, is to reframe the question and think about the value of learning in a different way. That isn’t to say that existing models are wrong, only that they may be the answer to the wrong question.

Join Donald and Reed Learning as they explore:
- Why learning value is essential to escape the training ghetto
- What ROI is, when it’s useful and when it isn’t
- The key question about value, and why it’s often ignored
- Re-framing conversations to approach impact the right way
- What the learning metrics debate tells us about L&D

From the Reed Learning Impact of Learning Breakfast Briefing, 13 June 2013.

Published in: Education, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
844
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
19
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Thursday 12th June 2013, VinopolisJoin us at this invitation-only event, where we will explore key L&D trends measuring the impact of learning. Find out: – how to achieve positive, measurable outcomes – what best practice looks like – the roles of the training provider, organisation and individual learner
  • Donald Kirkpatrick is Professor Emeritus of the University of Wisconsin in the United States and a past president of the American Society for Training and Development (ASTD). He is best known for creating a highly influential 'four level' model for training course evaluation. Kirkpatrick's ideas were first published in 1959 in a series of articles in the US Training and Development Journal but are better known from a book he published in 1975 entitled "Evaluating Training Programs.“Dr. Don Kirkpatrick made his final formal presentation at the ASTD International Conference and Expo on May 24, 2011.
  • Professor Emeritus, Western Michigan University Robert O. Brinkerhoff, Ed.D., is an internationally recognized expert in evaluation and training effectiveness, and winner of the ASTD 2007 Exceptional Contribution to Workplace Learning and Performance award and the 2008 Neon Elephant award for creative contributions to workplace learning The reality of learning interventionsBrinkerhoff offers two useful insights into learning interventions based on his experience of evaluating hundreds of training programmes.[1] The first insight is that there is a predictable pattern to the results of a learning event: some people will use their learning and get positive results; some people will not use their learning at all; and the majority of those left will use some learning but will notice little, if any, difference to their performance. This predictable distribution is illustrated below. Whilst Brinkerhoff does not attach any percentages to each group in this distribution, he indicates that the impact of the small number of successes normally justifies the money spent and thus produces a positive return on investment.Brinkerhoff argues that, using traditional evaluation methods, the true success of a learning event is often missed because the impact of those using the learning to achieve great results is diluted when analysed alongside those that do not use the learning at all. The Success Case Method concentrates on the most and the least successful applications of learning to highlight successes of the learning intervention and identify how it can have more of an impact in the future.The second insight that Brinkerhoff offers is that learning interventions do not impact on performance in isolation. What is different about the Success Case Method, however, is that no attempt is made to remove the impact of external factors from the results. Instead, Brinkerhoff believes that efforts to isolate the effects of learning are ineffectual and counter-productive. He recommends that the evaluation should identify the differences between the successful and unsuccessful cases and help to form the basis of recommendations to improve performance. In this way the evaluation moves away from the evaluation of the learning event and becomes an evaluation of how effectively an organisation uses learning.The Success Case MethodThe Success Case Method comprises two major phases. First, a survey of those participating in a learning event is used to identify the participants that fall into the three predictable groups identified earlier. Secondly, sample cases from the groups at the two opposing ends of the distribution are interviewed to identify and analyse the reasons for their respective successes and failures.The Success Case Method can be broken down into the following five steps:Focus and plan the evaluation. Define the potential results and benefits of the learning event. Design and conduct a survey of the participants. Conduct in-depth interviews of selected success and non-success instances. Formulate conclusions and recommendations.
  • Give example in detail hereAdd on flip chart: Training Needs Matrix (166)
  • The Impact of Learning Breakfast Briefing | 13 June 2013

    1. 1. The Impact of LearningDonald H. TaylorChairman, The Learning and Performance Institute
    2. 2. Group: reedlearning@reedlearningreedlearning#reediol
    3. 3. #reediolTypes of impact from learningWhat types of impactare there?
    4. 4. #reediolTypes of impact from learning• Better– More sales– Reduced inefficiencies• Faster– Increased productivity– Increased speed to competence• Cheaper– Reduced liability– Fewer support calls
    5. 5. #reediolQuestion...How do you show theimpact of learningin your organisation?
    6. 6. #reediol• What is ROI – really?• Alternative methods• The need for buy-in• Best practiceToday
    7. 7. #reediolTimeSalesIncreasein salesrevenue(B – A)ABTrainingintroducedhere
    8. 8. #reediolTimeSalesSales wereimprovinganywayIncrease likelyto havehappenedanywayABXTrainingintroducedhere
    9. 9. #reediolTimeSalesABXIncreaseshown bycontrolgroupTrainingintroducedhereOutsideeventhappenshereY
    10. 10. #reediolTimeSalesABXTrainingintroducedhereOutsideeventhappenshereYIncreaseattributableto training(B – Y)
    11. 11. #reediol“What‟s the ROI on that?”• Historical data collection• Good control groups• No training for B• Hawthorne effect• No new systems• „Monetizing‟ trainingIs thisreallywhat yourclient isasking for?
    12. 12. #reediol• Clothes?• Food?• Who?• Juliet!• Beer• James• Footy• Cab?It should be okay.There‟s a party at Tony‟stonight.What sort ofparty will it be?
    13. 13. #reediol“What‟s the ROI on that?”What do they really mean?
    14. 14. #reediol• I have a real need for an ROI study• Can I kill this training request?• Am I safe if I buy this?• Will this solve my problem?Possible meanings
    15. 15. #reediolQuestionAre there anyalternative methods?
    16. 16. #reediol1. Donald Kirkpatrick / Kirkpatrick Model2. Jack Phillips / ROI Institute3. Robert O. Brinkerhoff / Case study method4. Robinson, Harrison etc / Performance consultingShowing impact
    17. 17. #reediol• 1959 / 1975• 4 levels– Reaction– Learning– Behaviour– ResultsKirkpatrick modelDonald Kirkpatrick
    18. 18. #reediol• Jack Phillips ROI Institute• Kirkpatrick “5th Level”• Monetarization• Data-drivenThe Phillips ROI modelJack Phillips
    19. 19. #reediol• Robert O. Brinkerhoff• Focus on success• No isolation• Survey approachSuccess case methodRobert O Brinkerhoff
    20. 20. #reediol• Dana Gaines Robinson &James C Robinson (US)• Nigel Harrison (USA)• Many othersPerformance consultingNigel Harrison
    21. 21. #reediolPerformance relationship map1. Should 2. ShouldCausal LinkageOperational Results On-the-Job Performance4. Is 3. IsCausal Linkage5. Environmental FactorsImpacting PerformanceExternal Causes Internal CausesGap GapSource: Performance Consulting, Gaines Robinson & Robinson, Berrett-Koehler, 1995
    22. 22. #reediolSource: Improving Employee Performance, Kogan Page, 2000Seven-step process
    23. 23. #reediol1. What is the problem? Build a contract2. Who is involved?3. What are people doing now?4. What do you want them to do?5. What is the value of the gap?6. What are the causes? And potential solutions?7. Build an action planSeven-step process
    24. 24. #reediol1. What is the problem? Build a contract2. Who is involved?3. What are people doing now?4. What do you want them to do?5. What is the value of the gap?6. What are the causes? And potential solutions?7. Build an action planSeven-step process
    25. 25. #reediol• Open– “Who is involved in this?”• Engaging– “What would it look/feel like to do this?”– “What will happen if things continue like this?”• Challenging– “Could they do it in the past?”– “Does it matter to them?”– “Are they rewarded for poor performance?”– “Do they have all they need for the job?”Types of question
    26. 26. #reediol• Pain/Problem– “Is the team working well together?”– “Are you satisfied with ...?”• Implication– “What effect does that have on output?”– “Could that lead to increased costs?”– “Will it slow down your proposed plans?”• Need/Payoff Questions– “Why is it important to solve this problem?”– “What benefits do you see?”Establishing pain and needSource: SPIN Selling, by Neil Rackham, McGraw Hill, 1988
    27. 27. #reediolFacilitatingKnowledge SkillsEnvironmentMotivation
    28. 28. M L Broad & J W Newstrom (1992) | Research repeated in 1998 with similar resultsBefore During AfterManagerWorkshop Leader /InstructorLearner? ??? ?????With thanks to Charles Jennings? ??? ?????
    29. 29. Before During AfterManagerLearner1 345 67892Workshop Leader /InstructorM L Broad & J W Newstrom (1992) | Research repeated in 1998 with similar results With thanks to Charles Jennings
    30. 30. #reediolReal conversations
    31. 31. #reediolQuestionWhat skills do we need?Do we have them?
    32. 32. #reediolL&D skills• 12 months in development• Launched October 2012• 27 skills / 9 groups• 4 levels• Free to use for non-profit– bit.ly/LPICapMap• 983 assessments
    33. 33. Category and skill 1 2 3 4Analysis and StrategyAssessment and Evaluation 78 91 119 78Competency Management 94 103 86 66Learning Strategy 36 100 108 136Performance Analysis 55 132 112 90Business Skills and IntelligenceCommunication, Marketing and RelationshipManagement 73 74 107 95Financial Management 77 80 110 68Industry Awareness 50 131 103 95Procurement 67 93 98 70Collaborative LearningDeveloping Collaborative Learning Skills 132 89 85 74Supporting Communities of Practice (CoP) 112 108 77 72Supporting Content Co-creation and Curation 106 79 92 74Supporting Work Teams 52 114 146 130Learning Delivery ManagementChange Management 64 115 103 78Project Management 41 71 150 142Learning Information Management andInterpretationData Interpretation 71 99 112 37Information Architecture 67 119 109 37Learning ResourcesContent Creation 26 104 272 190Design 40 117 246 248Live DeliveryFace-to-face Learning 30 71 239 398Presentation Delivery 28 91 377 377Virtual/Online Delivery 142 132 153 141Managing the Learning FunctionPeople Management and Development 49 106 81 158Process Management and Improvement 74 66 101 119Resource Management 42 71 98 122Performance ImprovementCoaching 91 87 162 107Mentoring 108 102 110 78Performance Support 56 95 174 159L&D Skills#reediol
    34. 34. Category and skill 1 2 3 4Analysis and StrategyAssessment and Evaluation 78 91 119 78Competency Management 94 103 86 66Learning Strategy 36 100 108 136Performance Analysis 55 132 112 90Business Skills and IntelligenceCommunication, Marketing and RelationshipManagement 73 74 107 95Financial Management 77 80 110 68Industry Awareness 50 131 103 95Procurement 67 93 98 70Collaborative LearningDeveloping Collaborative Learning Skills 132 89 85 74Supporting Communities of Practice (CoP) 112 108 77 72Supporting Content Co-creation and Curation 106 79 92 74Supporting Work Teams 52 114 146 130Learning Delivery ManagementChange Management 64 115 103 78Project Management 41 71 150 142Learning Information Management andInterpretationData Interpretation 71 99 112 37Information Architecture 67 119 109 37Learning ResourcesContent Creation 26 104 272 190Design 40 117 246 248Live DeliveryFace-to-face Learning 30 71 239 398Presentation Delivery 28 91 377 377Virtual/Online Delivery 142 132 153 141Managing the Learning FunctionPeople Management and Development 49 106 81 158Process Management and Improvement 74 66 101 119Resource Management 42 71 98 122Performance ImprovementCoaching 91 87 162 107Mentoring 108 102 110 78Performance Support 56 95 174 159Live DeliveryFace-to-face Learning 30 71 239 398Presentation Delivery 28 91 377 377Virtual/Online Delivery 142 132 153 141L&D Skills#reediol
    35. 35. Category and skill 1 2 3 4Analysis and StrategyAssessment and Evaluation 78 91 119 78Competency Management 94 103 86 66Learning Strategy 36 100 108 136Performance Analysis 55 132 112 90Business Skills and IntelligenceCommunication, Marketing and RelationshipManagement 73 74 107 95Financial Management 77 80 110 68Industry Awareness 50 131 103 95Procurement 67 93 98 70Collaborative LearningDeveloping Collaborative Learning Skills 132 89 85 74Supporting Communities of Practice (CoP) 112 108 77 72Supporting Content Co-creation and Curation 106 79 92 74Supporting Work Teams 52 114 146 130Learning Delivery ManagementChange Management 64 115 103 78Project Management 41 71 150 142Learning Information Management andInterpretationData Interpretation 71 99 112 37Information Architecture 67 119 109 37Learning ResourcesContent Creation 26 104 272 190Design 40 117 246 248Live DeliveryFace-to-face Learning 30 71 239 398Presentation Delivery 28 91 377 377Virtual/Online Delivery 142 132 153 141Managing the Learning FunctionPeople Management and Development 49 106 81 158Process Management and Improvement 74 66 101 119Resource Management 42 71 98 122Performance ImprovementCoaching 91 87 162 107Mentoring 108 102 110 78Performance Support 56 95 174 159Collaborative LearningDeveloping Collaborative Learning Skills 132 89 85 74Supporting Communities of Practice 112 108 77 72Supporting Content Co-creation, Curation 106 79 92 74Supporting Work Teams 52 114 146 130L&D Skills#reediol
    36. 36. Category and skill 1 2 3 4Analysis and StrategyAssessment and Evaluation 78 91 119 78Competency Management 94 103 86 66Learning Strategy 36 100 108 136Performance Analysis 55 132 112 90Business Skills and IntelligenceCommunication, Marketing and RelationshipManagement 73 74 107 95Financial Management 77 80 110 68Industry Awareness 50 131 103 95Procurement 67 93 98 70Collaborative LearningDeveloping Collaborative Learning Skills 132 89 85 74Supporting Communities of Practice (CoP) 112 108 77 72Supporting Content Co-creation and Curation 106 79 92 74Supporting Work Teams 52 114 146 130Learning Delivery ManagementChange Management 64 115 103 78Project Management 41 71 150 142Learning Information Management andInterpretationData Interpretation 71 99 112 37Information Architecture 67 119 109 37Learning ResourcesContent Creation 26 104 272 190Design 40 117 246 248Live DeliveryFace-to-face Learning 30 71 239 398Presentation Delivery 28 91 377 377Virtual/Online Delivery 142 132 153 141Managing the Learning FunctionPeople Management and Development 49 106 81 158Process Management and Improvement 74 66 101 119Resource Management 42 71 98 122Performance ImprovementCoaching 91 87 162 107Mentoring 108 102 110 78Performance Support 56 95 174 159Analysis and StrategyAssessment and Evaluation 78 91 119 78Competency Management 94 103 86 66Learning Strategy 36 100 108 136Performance Analysis 55 132 112 90Business Skills and IntelligenceCommunication, Marketing andRelationship Management 73 74 107 95Financial Management 77 80 110 68Industry Awareness 50 131 103 95Procurement 67 93 98 70L&D Skills
    37. 37. #reediol• Right approach– Business metrics– Buy-in• Right method• Right culture• Right skillsBest practice
    38. 38. #reediol• Donald H. Taylor– Chairman, Learning and Performance Group• Jon Buttriss– Director, Reed Learning• Matthew Piddington– Head of Sales, Reed Learning• Eleanor Hudgell– Head of Learning Solutions, Reed LearningQ &A
    39. 39. #reediol• Performance Consulting– Gaines Robinson & Robinsonwww.partners-in-change.com• Improving Employee Performance– Nigel Harrisonwww.performconsult.co.uk nigel@performconsult.co.uk• What is Performance Consulting?– Nigel Harrison– Nigel will supply this PDF booklet to anyone from the ReedLearning event who mails him.References
    40. 40. Group: reedlearning@reedlearningreedlearningwww.reedlearning.com

    ×