Evaluating elearning

425 views

Published on

a avaliação da formação (em elearning) - do nível 1 ao 3 de Kirckpatrick

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
425
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Evaluating elearning

  1. 1. M-ICTE2009 Conference, Lisbon, 22-24 April José Reis Lagarto
  2. 2. evaluating elearning <ul><li>Who am I… ? </li></ul><ul><ul><li>Professor at Portuguese Catholic University </li></ul></ul><ul><ul><li>Pedagogical coordinator of the elearning master degree “ICT and Education”related to Teachers Training </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul>m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto
  3. 3. <ul><li>Initial question… what really eLearning is ? </li></ul>Distance Education? knowledge mangement? Mobile learning ? Face to face learning? m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto
  4. 4. <ul><ul><li>eLearning is the use of Internet technologies to distribute a set of solutions that allow the increasing of knowledge and abilities, either at individual level, either at institutional level. </li></ul></ul><ul><ul><ul><li>Rosenberg (2007) </li></ul></ul></ul>Formal training Knowledge management m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto
  5. 5. <ul><li>evaluating? </li></ul><ul><li>Why? </li></ul>Are the learning goals achieved ? Are there areas where improvement is possible? <ul><ul><ul><li>Can we be obtain a better position in the market? </li></ul></ul></ul>m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto
  6. 6. <ul><li>Evaluating </li></ul><ul><li>what? </li></ul>Micro vision Macro vision For a component of a course For a specific course Curriculum/program For a organization /SME For a particular activity sector On a social level m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto
  7. 7. <ul><li>O que avaliar ? </li></ul>m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto Evaluating what? Micro vision Macro vision For a component of a course For a specific course Curriculum/program For a organization /SME For a particular activity sector On a social level
  8. 8. evaluating When ? Before training During training delivery After Training Are there conditions for sucess ? Are the training strategy And materials adequate ? The impact of training in the organization m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto
  9. 9. <ul><li>Paulsen & Vieira (2006) had indicated some criteria to judge the global quality of an elearning program. </li></ul><ul><ul><li>Credibility of the training institution </li></ul></ul><ul><ul><li>Guarantee of quality criteria ( standards) </li></ul></ul><ul><ul><li>Previous information and counseling </li></ul></ul><ul><ul><li>Costs of the course </li></ul></ul><ul><ul><li>Support to e-student (technical and counseling) </li></ul></ul><ul><ul><li>Individual preferences ( learning styles) </li></ul></ul>m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto
  10. 10. The Kirkpatrick framework levels m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto 1 - satisfaction <ul><li>the students had liked the training? </li></ul><ul><li>- the drop out rate was low? </li></ul>2 - learning <ul><li>what had they learned? </li></ul><ul><li>- had the aims of learning been reached? </li></ul>3 - abilities - Are the students applying the new abilities in its workplace? 4 - results <ul><li>Is the organization improving its outcomes by the training of its workforce? </li></ul><ul><li>- how much earns the institution with the training of its collaborators? </li></ul>
  11. 11. student’s satisfaction level 1 Coordination of the course Contents related to personal objectives Technology (LMS …) Tutorial model Pedagogical model / training design evaluation criteria ? m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto - - -
  12. 12. Evaluation tools Questionnaires Interviews (singles or in focus group) Forums m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto student’s satisfaction level 1
  13. 13. m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto Satisfaction questionnaire (example)
  14. 14. Results of satisfaction quiz– Research Methodologies in Education - 2007/2008 m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto
  15. 15. Learning outcomes Level 2 m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto <ul><li>D iagnosis </li></ul><ul><ul><li>Lists of confidence </li></ul></ul><ul><ul><ul><li>minimum technical characteristics </li></ul></ul></ul><ul><ul><li>Scales of abilities </li></ul></ul><ul><ul><ul><li>formal abilities (academic or professional qualification) </li></ul></ul></ul><ul><ul><ul><li>specific abilities in some areas (previous linguistics abilities and digital literacy) </li></ul></ul></ul>
  16. 16. <ul><li>Mestrado em Ciências da Educação, especialização em Informática Educacional </li></ul><ul><li>CONDIÇÕES DE ACESSO </li></ul><ul><li>Edição 2007/2009 </li></ul><ul><li>  </li></ul><ul><li>a) Habilitações : </li></ul><ul><li>Pode candidatar-se ao curso de mestrado em Ciências da Educação (Especialização em Informática educacional (e-learning)) os candidatos com o grau de licenciado. </li></ul><ul><li>b) Competências de utilização ao nível da informática: </li></ul><ul><li>O curso de informática educacional tem como pressuposto que os alunos detenham competências médias no domínio da utilização dos softwares de utilização corrente (processador de texto, folha de cálculo, apresentações) e da utilização da Internet e da Web. </li></ul><ul><li>Deste modo, a frequência do curso está dependente da existência dessas mesmas competências. Como a UCP não fará verificação dos pré-requisitos, o candidato será responsável pelas suas capacidades no domínio das ferramentas informáticas. No anexo I define-se o perfil desejável de competências informáticas de entrada no Mestrado. </li></ul><ul><li>c) Competências linguísticas </li></ul><ul><li>Embora a língua oficial do Mestrado seja o Português, os candidatos deverão possuir competências de leitura e interpretação de textos escritos em Inglês, dado que alguns materiais pedagógicos distribuídos ou a pesquisar pelos alunos poderão estar nesse idioma. </li></ul><ul><li>d) Requisitos tecnológicos </li></ul><ul><li>Para a frequência do curso, os alunos devem dispor, em permanência, de  um computador multimédia e acesso à Internet, preferencialmente em banda larga. </li></ul><ul><li>Anexo I Pré-requisitos de competências informáticas para ac esso ao Mestrado em Informática Educacional </li></ul><ul><li>Consideram-se pré-requisitos informáticos gerais: </li></ul>Academic qualification Digital literacy Adequate knowledge foreign language Hardware requirements m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto Scale of abilities
  17. 17. m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto Learning outcomes Level 2 <ul><li>formative </li></ul><ul><ul><li>self evaluation </li></ul></ul><ul><ul><li>hetero-evaluation </li></ul></ul><ul><ul><ul><li>peers </li></ul></ul></ul><ul><ul><ul><li>tutor/professor/system </li></ul></ul></ul><ul><li>sumative </li></ul><ul><ul><li>self evaluation </li></ul></ul><ul><ul><li>hetero-evaluation </li></ul></ul><ul><ul><ul><li>peers </li></ul></ul></ul><ul><ul><ul><li>tutor/professor/system </li></ul></ul></ul>
  18. 18. online self evaluation tests assignments Project work Fóruns participation participation in sincronous sessions m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto Learning outcomes Level 2 <ul><li>Setting of a minimum number of participation </li></ul><ul><li>Setting of evaluation criteria of each message </li></ul><ul><ul><li>Interaction </li></ul></ul><ul><ul><ul><li>Confrontation </li></ul></ul></ul><ul><ul><ul><li>Lidership </li></ul></ul></ul><ul><ul><ul><li>Different approaches </li></ul></ul></ul><ul><ul><ul><li>… </li></ul></ul></ul><ul><li>Number of messages posted per day </li></ul>
  19. 19. Face to face evaluation (quality recognition) <ul><li>Testes online tests </li></ul><ul><ul><li>Múltiple choice </li></ul></ul><ul><ul><li>true - false </li></ul></ul><ul><ul><li>fill in gaps </li></ul></ul><ul><ul><li>de correspondência </li></ul></ul><ul><ul><li>… </li></ul></ul><ul><li>written testes escritos </li></ul><ul><li>projects discussion </li></ul><ul><li>Labs work </li></ul><ul><li>Chats’ participation </li></ul>m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto Learning outcomes Level 2
  20. 20. evaluating learning outcomes on-line individual works Particpation in chats forum participation on-line working groups face to face evaluation - tests / projects/ … m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto
  21. 21. % of different evaluation components in a single Curricular Unit m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto evaluation final evaluation (%) Face to face (teste ou work debate) 50 a 60 Assignments / required along the training period 15 a 20 Colaborative work - written documents sumbited to tutor. 20 a 30 Participation in online sincronous activities (according teacher strategies) 5
  22. 22. evaluation of abilities level3 external evaluation department director opinion self evaluation Less complaints Less acidents Raise off sales … m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto
  23. 23. It is possible to evaluate at this level , if The training is a consequence of clear needs The learner has strong motivation There are logistics conditions allowing th evidence of new skills m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto evaluation of abilities level3
  24. 24. Results inside organisation Nível 4 Level 4 measures the results of learning at the institutional level <ul><li>The training contributes to the objectives of the business ? </li></ul><ul><li>The training responds efficiently to the needs identified ? </li></ul><ul><li>What level of investment in training is recovered ? </li></ul>ROI m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto
  25. 25. Evaluating e-learning Evaluating learning Evaluating communication tools Conclusion : Knowledge Skills m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto
  26. 26. Thanks for your attention! [email_address] m- ICTE2009 Conference, Lisboa, 22—24 April José Reis Lagarto

×