eLearning and return on investment


Published on

A talk for the Chartered Insurance Institute

Published in: Economy & Finance, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • eLearning and return on investment

    1. 1. Designing (e)learning to meet the needs of the learner, the manager and deliver return on investment Chartered Insurance Institute 10 May 2007
    2. 2. Who, me? <ul><li>Education </li></ul><ul><li>1968 - DEC PDP8 & FORTRAN </li></ul><ul><li>1972 - BA (English Lit) </li></ul><ul><li>1986 - MPhil (Historical & Comparative Linguistics) </li></ul><ul><li>2001 - MA (Education w/Open and Distance Education) </li></ul><ul><li>2004 - PhD study at U of Southampton </li></ul><ul><ul><li>The extent to which beliefs about learning and teaching are embedded in the artefacts of learning technology </li></ul></ul><ul><li>Work & Related </li></ul><ul><li>1987 PEP Preparatory Education Project </li></ul><ul><li>1988 ACE adult community continuing education tutor </li></ul><ul><li>1989 College of Petroleum and Energy Studies </li></ul><ul><li>2000 Brookes </li></ul><ul><li>2003 Open University </li></ul><ul><li>ALT </li></ul><ul><li>Writing </li></ul>George Roberts Educational Developer [email_address]
    3. 3. Outline <ul><li>Issues </li></ul><ul><li>Objectives </li></ul><ul><li>ROI </li></ul><ul><li>Learning Design </li></ul>
    4. 4. Exploring the issues <ul><li>Consider the following images </li></ul><ul><li>What are the issues, wide and narrow? </li></ul><ul><li>What are the training issues? </li></ul>
    5. 5. The issues?
    6. 6. The issues?
    7. 7. <ul><li>1970 Equal Pay </li></ul><ul><li>1974 Rehabilitation of Offenders </li></ul><ul><li>1975 Sex Discrimination </li></ul><ul><li>1976 Race Discrimination </li></ul><ul><li>1995 Disability Discrimination </li></ul><ul><li>2000 Part-time Workers </li></ul><ul><li>2002 Fixed term workers </li></ul><ul><li>SENDA 2001 (prevents discrimination in education & training) </li></ul><ul><li>Race Relations (Amendment) Act 2001 </li></ul><ul><li>Employment Act 2002 </li></ul><ul><li>Sexual Orientation, Religion & Belief 2003 </li></ul><ul><li>Gender Recognition Act 2004 </li></ul><ul><li>Civil Partnership Act 2005 </li></ul><ul><li>Disability Discrimination Act 2006 </li></ul><ul><li>Age 2006 </li></ul><ul><li>Equality Act 2006 </li></ul>The issues?
    8. 8. <ul><li>Consider the preceding images: with your neighbour, in pairs or small groups </li></ul><ul><li>What are the issues, wide and narrow? </li></ul><ul><li>What are the training and development issues? </li></ul>Exploring the issues?
    9. 9. <ul><li>At the end of this session you should be able to explain </li></ul><ul><li>the role of continuing professional development in a knowledge-based enterprise </li></ul><ul><li>a balanced approach to ROI </li></ul><ul><li>a multi-modal approach to CPD </li></ul><ul><li>basic principles of Designing for Learning </li></ul><ul><li>and bust some sad myths </li></ul>Learning objectives
    10. 10. <ul><li>One firm training manager surmises that it will take between 50 and 200 development hours to produce one instructional hour </li></ul><ul><li>… industry insiders say that initial development costs can run from $5,000 - $35,000 </li></ul><ul><li>… lack of human contact can create a static learning environment </li></ul><ul><li>Activity </li></ul><ul><li>In pairs or small groups, identify at least one counter example to each of these statements. </li></ul>Sad myths?
    11. 11. <ul><li>Learning is not just content delivery </li></ul><ul><ul><li>Communication => Channel </li></ul></ul><ul><ul><li>Community => Relationship </li></ul></ul><ul><ul><li>Content => Topic </li></ul></ul>Busting the myths
    12. 12. <ul><li>Socio-economic-political environment </li></ul><ul><ul><li>stakeholders, shareholders, principals and agents </li></ul></ul><ul><ul><li>legislation and regulation </li></ul></ul><ul><ul><li>governance and accountability </li></ul></ul><ul><ul><li>assurance </li></ul></ul><ul><li>Cultural environment </li></ul><ul><ul><li>working practices </li></ul></ul><ul><ul><li>family patterns </li></ul></ul><ul><li>Natural environment </li></ul><ul><li>Personal environment </li></ul><ul><ul><li>your own aims, ambitions </li></ul></ul>The role of CPD: Managing change in…
    13. 13. Compliance <ul><li>Compliance is practically a full-time job by itself, not just because of the constant changes, but because of the mandatory training requirements (Cormier 2006) </li></ul>
    14. 14. <ul><li>Too hard </li></ul><ul><li>Research by Ashridge Business School has found that it is almost impossible to quantify ROI around management development because there are so many variables involve </li></ul><ul><li>net programme benefits cannot be accurately predicted and require complex assumptions concerning the future effectiveness of a programme </li></ul>ROI: two perspectives
    15. 15. <ul><li>Has to be done </li></ul><ul><li>Measuring the impact of training comes down to asking the right questions to find the data that can be linked to changed behavior </li></ul>ROI: two perspectives
    16. 16. <ul><li>In pairs/small groups produce a short list of: </li></ul><ul><ul><li>What you need to know: what data will be useful </li></ul></ul><ul><ul><li>What questions will elicit that data </li></ul></ul>ROI: Activity
    17. 17. <ul><li>Measuring the impact of training comes down to asking the right questions to find the data that can be linked to changed behavior </li></ul><ul><li>Superficial </li></ul><ul><ul><li>ask learners to rate how happy they were </li></ul></ul><ul><ul><li>list the “take homes” </li></ul></ul><ul><ul><li>Use psychometrics (Myers Briggs, Belbin) to relate multiple motivation to departmental objectives </li></ul></ul><ul><ul><li>establish Learning Transfer Climate, e.g. making sure someone's boss had been on a course before them </li></ul></ul><ul><ul><li>Apprenticeships are cost-effective and low-risk methods of identifying people with high potential: trainees AND mentors </li></ul></ul><ul><ul><ul><li>treat entry-level jobs as development opportunities. </li></ul></ul></ul><ul><ul><ul><li>hire college students to do entry level jobs part time with a view to talent spotting </li></ul></ul></ul><ul><ul><li>Formal coaching demonstrates high return-on-investment </li></ul></ul><ul><li>Significant </li></ul>ROI: has to be done
    18. 18. Philips evaluation methodology <ul><li>Reaction and planned action (near term) </li></ul><ul><ul><li>e.g. 90% of participants indicate intention to apply new knowledge within 1 month </li></ul></ul><ul><ul><li>e.g. 80% of participants indicated the training “represented good value for the company” </li></ul></ul><ul><li>Learning (near to mid term) </li></ul><ul><ul><li>compliance testing </li></ul></ul><ul><li>Application (mid term) </li></ul><ul><ul><li>formal and informal diagnosis of learning and planned action through observation and conversation </li></ul></ul><ul><li>Business impact (mid to long term) </li></ul><ul><ul><li>align departmental aims with corporate and individual aims and structure training interventions around departmental aims </li></ul></ul><ul><li>Value of Investment </li></ul><ul><ul><li>growth, development, sustainability </li></ul></ul><ul><ul><li>shareholder value </li></ul></ul><ul><li>Intangibles </li></ul>
    19. 19. <ul><li>Activity </li></ul><ul><li>In groups, suggest appropriate instruments for each stage of the Philips methodology: </li></ul><ul><ul><li>Reaction and planned action </li></ul></ul><ul><ul><li>Learning </li></ul></ul><ul><ul><li>Application </li></ul></ul><ul><ul><li>Business impact </li></ul></ul><ul><ul><li>Value of Investment </li></ul></ul><ul><ul><li>Intangibles </li></ul></ul>Philips evaluation instruments
    20. 20. Communites of Practice The community is the expert system
    21. 21. The knowledge is in the network <ul><li>Communities of Practice </li></ul><ul><li>Benchmarking clubs </li></ul><ul><li>ROI networks </li></ul><ul><li>Reframe conceptions of learning and teaching in the networked environment </li></ul><ul><li>Learner-centred experiential learning </li></ul>
    22. 22. <ul><li>Development is continuous: a conversation with the world, life, and everything </li></ul><ul><ul><li>punctuated waves </li></ul></ul><ul><ul><li>pressure boosting </li></ul></ul><ul><li>Use instruments to measure </li></ul><ul><ul><li>Matrices </li></ul></ul><ul><ul><li>Checklists </li></ul></ul><ul><ul><li>Semi-structured interviews </li></ul></ul><ul><ul><li>360 degree review </li></ul></ul><ul><ul><li>HRD Learning Management Systems (LMS) </li></ul></ul><ul><li>Value the informal along with the formal </li></ul><ul><ul><li>Pareto rules </li></ul></ul><ul><ul><li>Counting everything is not efficient or effective </li></ul></ul>Discourse technologies
    23. 23. Instruments as ways of seeing <ul><li>From individual to social/organisational learning </li></ul><ul><li>Spiral IRF </li></ul><ul><li>Kolb learning cycle and learning styles </li></ul><ul><li>Activity modelling </li></ul><ul><li>Pedagogical pragmatism </li></ul><ul><li>Five stages of learning socialisation </li></ul><ul><li>Learning Technology Support Architecture </li></ul>
    24. 24. Spiral IRF Initiation Response Feedback
    25. 25. Kolb learning cycle Experience Reflection Conceptualisation Experimentation
    26. 26. Kolb learning cycle
    27. 27. Activity model
    28. 28. Pedagogical pragmatism: the Borg cube?
    29. 29. Salmon’s 5-stage model
    30. 30. Learning Technology Support Architecture
    31. 31. Activity <ul><li>For each visualisation, develop an instrument to elicit evidence for the value of an intervention </li></ul>
    32. 32. The systematic approach <ul><li>Write aims and outcomes </li></ul><ul><li>Identify topics </li></ul><ul><li>Structure the programme </li></ul><ul><li>Design teaching and learning strategies taking into account student characteristics </li></ul><ul><li>Identify, select and design learning resources </li></ul><ul><li>Plan and design assessments </li></ul><ul><li>Evaluate </li></ul>Designing for learning (D4L)
    33. 33. <ul><li>Learning outcomes describe: </li></ul><ul><li>the kinds of knowledge the learner is expected to be able to employ </li></ul><ul><li>the kinds of problems learners may be expected to solve </li></ul><ul><li>the kinds of skills (intellectual, interpersonal, practical) which the student is expected to be able to use </li></ul><ul><li>the conditions under which the student is expected to be able to demonstrate their attainments </li></ul><ul><li>some indication of the level of performance that will be expected </li></ul>Learning outcomes
    34. 34. Why an outcomes-based approach? <ul><li>Outcomes define students knowledge, understanding, intellectual and subject specific skills at each level. </li></ul><ul><li>Outcomes clarify the purpose of the course – for you and your students </li></ul><ul><li>Outcomes help you decide and prioritise which topics to teach, and in what depth </li></ul><ul><li>Outcomes help define appropriate teaching and learning strategies </li></ul><ul><li>Thinking about how students demonstrate their learning leads naturally to purposeful assessment tasks </li></ul>Learning outcomes
    35. 35. But, <ul><li>Outcomes have been criticised for being reductionist – especially when written for lower cognitive levels </li></ul><ul><li>Outcomes usually focus on knowledge, neglecting attitudes, values, motivations and interests </li></ul><ul><li>May limit the unexpected outcomes of learning </li></ul>Learning outcomes
    36. 36. <ul><li>verb </li></ul>Learning outcomes context By the end of this course, students will be able to… object
    37. 37. Levels of learning: Bloom <ul><li>knowledge </li></ul>comprehension application analysis synthesis evaluation Learning outcomes
    38. 38. Hierarchies of learning: Biggs memorising note-taking describing explaining relating applying theorising passive active Teaching method Deep learning Surface learning Learning outcomes
    39. 39. Mix modes <ul><li>on-job coaching </li></ul><ul><li>classroom training </li></ul><ul><li>self-directed education </li></ul>
    40. 40. hi collocation hi collaboration traditional laboratory lo computerisation hi collocation whiteboards in classrooms hi collaboration virtual field trips hi computerisation lo collocation CACL, online forums hi collaboration “Learning to teach online” hi computerisation hi collocation lo collaboration video link lecture hi computerisation lo collocation lo collaboration “traditional” DL lo computerisation lo collocation lo collaboration CBT training hi computerisation Blended learning
    41. 41. <ul><ul><li>delivery </li></ul></ul><ul><ul><li>technology </li></ul></ul><ul><ul><li>locus </li></ul></ul><ul><ul><li>pedagogy </li></ul></ul><ul><ul><li>chronology </li></ul></ul><ul><ul><li>roles </li></ul></ul><ul><ul><li>focus </li></ul></ul><ul><ul><li>learning </li></ul></ul><ul><ul><li>knowledge </li></ul></ul><ul><ul><li>politics </li></ul></ul>10 Dimensions of blended learning
    42. 42. Online programs provide: Myth or magic? <ul><li>flexibility </li></ul><ul><li>accessibility </li></ul><ul><li>convenience </li></ul><ul><li>a consistent message </li></ul><ul><li>ease with updating </li></ul><ul><li>integration of learning and work </li></ul><ul><li>increased impact of the money invested in training programs, </li></ul><ul><li>significantly reduced employee travel cost and time </li></ul><ul><li>the ability to train more people, more frequently and in shorter sessions that are easier to coordinate and schedule </li></ul><ul><li>online training is scaleable because it offers the ability to add instructors and students as needed, with fewer changes </li></ul>
    43. 43. A continuum of learning <ul><li>Learning, change, and innovation are part of the same continuum and are not discrete activities </li></ul><ul><ul><li>Communication, Coaching, Consulting </li></ul></ul><ul><li>Value time </li></ul><ul><li>Plan ahead </li></ul><ul><li>Structure the programme </li></ul><ul><li>Make it mandatory </li></ul><ul><li>Keep it flexible </li></ul><ul><li>Build in reflection </li></ul><ul><li>Assess and evaluate every intervention </li></ul><ul><li>(Bates 2006, Howe & Kornfeind 2006) </li></ul>
    44. 44. Human capital: ROI = Return on Intelligence <ul><li>Long-term organizational ROI in knowledge-based industries means making routine investments in human capital </li></ul><ul><li>Value of the investment in training goes beyond the compliance requirements </li></ul><ul><li>When combined with macro-level analysis… ROI studies supply the proof of the impact of learning and development </li></ul><ul><ul><ul><li>Commit to your investment </li></ul></ul></ul><ul><ul><ul><li>Know your goal </li></ul></ul></ul><ul><ul><ul><li>Create a positive culture </li></ul></ul></ul><ul><ul><ul><li>Measure progress </li></ul></ul></ul><ul><ul><ul><li>Offer feedback </li></ul></ul></ul><ul><ul><ul><li>Structure applied learning </li></ul></ul></ul>
    45. 45. Indicative list of providers <ul><li>Brainshark, a provider of on-demand rich media communications solutions </li></ul><ul><ul><ul><li>http://presentation.brainshark.com/powerpoint-presentations-index.asp </li></ul></ul></ul><ul><li>Desire2Learn, providing innovative eLearning solutions </li></ul><ul><ul><ul><li>http://www.desire2learn.com/ </li></ul></ul></ul><ul><li>Eedo Knowledgeware, a provider of real time learning and knowledge system software </li></ul><ul><ul><ul><li>http://www.eedo.com/ </li></ul></ul></ul><ul><li>Epic: the UK market leader in e-learning, blended learning and knowledge solutions. http://www.epic.co.uk/ </li></ul><ul><li>eVergance, a provider of consulting services for customer relationship management optimization </li></ul><ul><ul><ul><li>http://www.evergance.com/ </li></ul></ul></ul><ul><li>HyperWave, c ollaborative Information Management (CIM) with a fully integrated e-learning and web conferencing environment. </li></ul><ul><ul><ul><li>http://www.hyperwave.com/e/ </li></ul></ul></ul><ul><li>NIIT Global, a learning and knowledge solutions provide </li></ul><ul><ul><ul><li>http://www.niit.com/ </li></ul></ul></ul><ul><li>OutStart, a provider of software solutions for learning and knowledge-sharing </li></ul><ul><ul><ul><li>http://www.outstart.com/ </li></ul></ul></ul><ul><li>Saba, human capital management </li></ul><ul><ul><ul><li>http://www.saba.com/index.htm </li></ul></ul></ul><ul><li>Tribal: one of the largest providers of training and CPD in the UK, clients include government, the NHS, schools, colleges, and the private sector http://www.tribalgroup.co.uk/index.php?ob=1&id=16 </li></ul><ul><li>WBT Systems, a provider of enterprise e-learning solutions </li></ul><ul><ul><ul><li>http://www.wbtsystems.com/ </li></ul></ul></ul><ul><li>Zeroed-in Technologies: provides software and services that help organizations visualize, measure, and execute effective learning and business strategies. </li></ul><ul><ul><ul><li>http://www.getzeroedin.com/ </li></ul></ul></ul>
    46. 46. Reference <ul><li>anon (2005) HR urged to use ROI for training, People Management, 21 April 2005 . </li></ul><ul><li>anon (2006) Can Customized Online Training Boost Firm ROI? Compensation and benefits for law offices, May 2006, 1, 11-13. </li></ul><ul><li>Baraban, R. and Kovaleski, D. (2006) Maximising ROI: Setting objectives for sales training, Financial and Insurance Meetings, May/June 2006, 9-10 </li></ul><ul><li>Bartley, S. J. and Golek, J. H. (2004) Evaluating the Cost Effectiveness of Online and Face-to-Face Instruction, Journal of Educational Technology & Society, 7, 167-175. </li></ul><ul><li>Brannick, M. (2005) Reporting Results: Managing Measurement, Chief Learning Officer, June, 2005, 28-33. </li></ul><ul><li>Bates, S. (2006) You can measure training ROI, Official Board Markets, November 2006, 18-19 </li></ul><ul><li>Howe, P. E. and Kornfeind, J. F. (2006) The New ROI: Return on Intelligence, Pennsylvania CPA Journal, Fall 2006 </li></ul><ul><li>Paton, Nic, Time to shift the focus from ROI? Training Magazine , Nov 2005, p4 </li></ul><ul><li>Phillips, J. J. (2005) The value of human capital: a micro-level approach, Chief Learning Officer, December, 2005, 52-54. </li></ul><ul><li>“ Training today”, Training ; Sep 2006, Vol. 43 Issue 9, p12-13 </li></ul><ul><li>Weinstein, Margery, “Measuring ROI”, Training , Jan 2006, Vol. 43, Issue 1 </li></ul>