Successfully reported this slideshow.

Astd Chicago 2010 Su210 Final

648 views

Published on

A presentation of some of my work

  • Be the first to comment

  • Be the first to like this

Astd Chicago 2010 Su210 Final

  1. 1. Impact Measurement of the Global Learning Portfolio at Vestas Wind Systems - SU210 ASTD ICE, Chicago IL May 16, 2010 Learning Objectives: Critically review your own measurement efforts and identify new areas for improvement. Design and implement large scale impact measurement on Kirkpatrick’s Levels 1 – 3. Improve dialogue with the business, based on the facts from impact measurement. Jette Nissen Specialist Impact Measurement [email_address] Lars Julin CEO [email_address]
  2. 2. Agenda <ul><li>Brief introduction to Vestas </li></ul><ul><li>Impact Measurement at Vestas </li></ul><ul><li>The Peopleway High Impact Learning Approach </li></ul><ul><li>Global Continuous Improvement of learning at Vestas </li></ul><ul><li>What does it take? </li></ul>
  3. 3. During the last 25 years... <ul><li>… Vestas has multiplied its turbine output 100 times. </li></ul>30 kW 3,000 kW
  4. 4. Global outlook - Local presence Vestas Northern Europe Vestas Mediterranean Vestas Central Europe Vestas Asia Pacific Vestas China Vestas Offshore Vestas Americas Vestas Nacelles A/S Vestas Blades A/S Vestas Spare Parts A/S Vestas Towers A/S Vestas Control Systems A/S Vestas People & Culture Vestas Technology R&D
  5. 5. Ramping up our human resources EMPLOYEES
  6. 6. The SAP Case - Background In 2008 the global implementation of SAP started - an Enterprise Resource Planning (ERP) system supporting all the processes in the Vestas Value Chain Training and measurement SAP implementation in the seven regions were completed Sept. 2009 937 courses conducted and measured during the implementation 7,657 were trained anparticipants d measured Impact Measurement of the training – a strategic decision by the Executive Vice President and CFO Service Project Production Development Sales
  7. 7. Impact Measurement at Vestas
  8. 8. The Peopleway High Impact Learning Approach Introduction to basic design and concepts | Impact Measurement at Vestas Wind Systems
  9. 9. Peopleway A/S <ul><li>Peopleway A/S was founded in 1986 in Denmark. </li></ul><ul><li>The Peopleway ® High Impact Learning Approach and the Ease® Impact Measurement Solution are results of more than 20 years of research and cooperation with numerous international companies. </li></ul><ul><li>The Peopleway Approach is greatly inspired by the work of Donald L. Kirkpatrick, Jack J. Phillips and Robert O. Brinkerhoff. </li></ul><ul><li>In 2009, Peopleway and Vestas won two international awards: </li></ul><ul><ul><li>The Bersin Award – “Learning Initiative Excellence” honored as “The best of the best” in its field. </li></ul></ul><ul><ul><li>The Brandon Hall Award - “Best use of Virtual Worlds for Learning”. </li></ul></ul> | Impact Measurement at Vestas Wind Systems
  10. 10. The Peopleway Approach – Inspiration and scope Low cost All training evaluation High cost Project evaluation Explaining cause Formative evaluation Measuring Summative evaluation Peopleway © | Impact Measurement at Vestas Wind Systems The Peopleway High Impact Learning Approach Reaction Learning Behaviour Learning Capability Transfer Peopleway® Services and Research ROI Ease® Integration Result Robert O. Brinkerhoff The Success Case Method Jack Phillips ROI Reaction Learning Behaviour Learning Capability Transfer Ease® Solution Donald Kirkpatrick Reaction Learning Behaviour Result 1 2 3 4
  11. 11. The Peopleway Evaluation Levels Seven levels of evaluation Job The Peopleway Evaluation Levels © Training | Impact Measurement at Vestas Wind Systems L5 ROI Jack Phillips L4 Result Business results L3 Job behavior Behaviour L2 Learning Knowledge, Attitude, Intention Transfer of Learning Knowledge, Attitude, Intention Peopleway Brinkerhoff Learning Capability Drivers and Resistors L1 Reaction Satisfaction Donald Kirkpatrick Level 1 and 2 Zero Correlation! Peopleway
  12. 12. Why Impact Measurement? <ul><li>In only 13% of all learning activities is there a significant positive correlation between satisfaction and learning. </li></ul><ul><li>Reaction measurements cannot reliably measure learning. </li></ul><ul><li>Subjective criteria measurements using satisfaction, self assessment or perception questions do not produce reliable and valid measures of learning or impact of learning. </li></ul><ul><li>To implement reliable and valid impact measurements, an approach using effect measurement and objective criteria in design of questions is needed. </li></ul> | Impact Measurement at Vestas Wind Systems
  13. 13. Evaluation Design Face-to-face training Peopleway © | Impact Measurement at Vestas Wind Systems 0-14 before Pre-test ? @ Learning Activity ASSESSMENT LEARNING  @ 0-14 days after Satisfaction and Post-test ? @ IMPLEMENTATION 3-6 months after Job-test ? @ Knowledge Attitude Intention Behaviour Knowledge Attitude Intention Satisfaction Knowledge Attitude Intention Behaviour
  14. 14. Increased Learning ≈ Increased Earning and ROI 50% Learning 0% 100% 10,000 hours Decreasing hours of learning ≈ Decreasing Costs | Impact Measurement at Vestas Wind Systems 80% Peopleway © 7,500 Hours Time
  15. 15. Implementation of Learning – Drivers and Resistors Implementation of Learning Time Pre Post Job Learning Learning 0% 100% Peopleway © | Impact Measurement at Vestas Wind Systems No implementation Learning is Relevant Timing is good Management involvement 90% No implementation Learning is Relevant Timing is good 75% No implementation Learning is Relevant 45% No implementation 20% 80%
  16. 16. From Learning Objectives to Measurement <ul><li>Learning Objectives should: </li></ul><ul><ul><li>Link to business needs </li></ul></ul><ul><ul><li>Be defined at an appropriate level of detail </li></ul></ul><ul><ul><li>Include a learning taxonomy </li></ul></ul><ul><li>When designing measurements, it’s important: </li></ul><ul><ul><li>To produce an optimum number of questions (not to many nor to few) </li></ul></ul><ul><ul><li>That the questions are equally distributed according to learning objectives </li></ul></ul><ul><ul><li>That questions target the right learning level taxonomy </li></ul></ul><ul><ul><li>That questions use objective criteria techniques (test, critical incident etc.) </li></ul></ul><ul><ul><li>That questions are validated </li></ul></ul> | Impact Measurement at Vestas Wind Systems
  17. 17. From Learning Objectives to Measurement Program Learning Program Module 2 Objective 2 Module 3 | Impact Measurement at Vestas Wind Systems Module Module 1 Objective Objective 1 Objective 3 Objective 4 Objective 5 Objective 6 Objective 7 Taxonomy 1 Know 2 Comprehend 1 Know 3 Apply 2 Comprehend 3 Apply 3 Apply Intention Knowledge Behaviour Knowledge Attitude Measurement Attitude Intention Knowledge Knowledge Behaviour
  18. 18. Ease® Learning Objectives | Impact Measurement at Vestas Wind Systems Program Objective Taxonomy Measurement Questions
  19. 19. The Learning Continuous Improvement Process (CIP) Pre-learning Training Development Learning Evaluation Learning Activity Peopleway © HR & Learning Developer Ease Solution Learning Provider HR/organization Question design Learning development Identify Training Needs | Impact Measurement at Vestas Wind Systems Pre-test Analysis Learning Feedback Learning activity C I P Define Learning Objectives Create Questions Develop Learning Program Proofreading Validate Questions Instructor’s improvement Developers improvement Post-test
  20. 20. Learning Analysis Objectives SAP SA04 (3 days sales course) Not the one and only truth, but an indication that here lies something worth exploring.
  21. 21. Learning Analysis - Answer distribution Process Engineering Pre Post
  22. 22. SAP SA 04, India – May ’08 Areas to improve
  23. 23. Peopleway India - May '08 China/Korea/Pacific - Sep '08 NEU - Oct '08 Tokyo - Feb '09 MED I - Mar '09 MED II - Sep ’09 29% 7% Average Improvement
  24. 24. Continuous Improvement of Vestas global SAP training Improving learning for more than 7,657 participants and produced an overall 68 percent increase in learning 937 learning activities | Impact Measurement at Vestas Wind Systems 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% USA India Asian Pacific Northern Europe MED I Central Europe MED II Learning
  25. 25. Training results by course level Northern Europe Course / Post test score 0-60% 61-80% 81-100% Grand Total SERVICE 01 15 28 57 100 SERVICE 02 - PDA 8 21 70 100 SERVICE 03 - TASK MAN. 23 46 31 100 FINANCE&CON 37 41 22 100 PRODUCT LIFECYCLE MAN. 0 56 44 100 PROJECT 28 38 33 100 SALES 9 26 65 100 Supply Chain Management 45 33 23 100 Grand Total 24 32 44 100
  26. 26. Continuous improvement of SAP training
  27. 27. What does it take to implement Continuous Improvement in your learning portfolio? <ul><li>Management decision and support </li></ul><ul><li>A scalable Effect and Impact Measurement setup </li></ul><ul><li>Detailed learning objectives </li></ul><ul><li>Objective measures of learning </li></ul><ul><li>Timing and effort </li></ul>Management decision and support
  28. 28. Thank you for your attention <ul><li>Brief introduction to Vestas </li></ul><ul><li>Impact measurement at Vestas </li></ul><ul><li>The Peopleway High Impact Learning Approach </li></ul><ul><li>Global continuous improvement of learning at Vestas </li></ul><ul><li>What does it take? </li></ul>
  29. 29. Thank you for your attention ASTD ICE, Chicago IL May 16, 2010 Copyright Notice These documents are created by Vestas Wind Systems A/S and Peopleway A/S and contain copyrighted material, trademarks, and other proprietary information. All rights reserved. No part of the documents may be reproduced or copied in any form or by any means—such as graphic, electronic, or mechanical, including photocopying, taping, or information storage and retrieval systems without the prior written permission of Vestas Wind Systems A/S and Peopleway A/S. The use of these documents by you, or anyone else authorized by you, is prohibited unless specifically permitted by Vestas Wind Systems A/S and Peopleway A/S . You may not alter or remove any trademark, copyright or other notice from the documents. The documents are provided “as is” and Vestas Wind Systems A/S and Peopleway A/S shall not have any responsibility or liability whatsoever for the results of use of the documents by you. Jette Nissen Specialist Impact Measurement [email_address] Lars Julin CEO [email_address]

×