Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Quality Assurance in Computer Supported Collaborative Learning Environments Euro-CAT Ergonomic Evaluation Dr. Niki Lambrop...
<ul><li>What is Quality? </li></ul><ul><li>Utility, usability, acceptability </li></ul><ul><ul><li>Case study : Toll staff...
What is quality?
<ul><li>Fit of purpose </li></ul><ul><li>Bergman and Klefsjö (1994):  </li></ul><ul><li>“ The quality of a product (articl...
1.  Utility, usability, acceptability Case study : Toll staff training  (learning situation + ergonomic criteria)
Tooll staff training needs <ul><li>USA Toll Tariffs: </li></ul><ul><li>No training is necessary, even children can became ...
Where and when can a toll worker could follow an elearning course? During the work hours? During the coffee break? At home?
As if the learning situation were not enough complex.... we will consider the usability criteria  Is the course  useful ? ...
<ul><li>As if the learning situation was not complex enough.... we will consider the ergonomics criteria </li></ul><ul><ul...
As if the learning situation were not enough complex.... we will consider the usability criteria  <ul><li>UTILITY </li></u...
As if the learning situation were not enough complex.... we will consider the usability criteria  Is the course  easy to u...
As if the learning situation were not enough complex.... we will consider the usability criteria  <ul><li>ACCEPTABILITY </...
As if the learning situation were not enough complex.... we will consider the usability criteria  <ul><li>SOCIAL ACCEPTABI...
Nielsen’s Taxonomy of System Acceptability
Toll staff training – Case study Conclusions <ul><li>In addition to pedagogical criteria we need to consider the utility, ...
2. CSCL ergonomic evaluation Case study: Euro-CAT Ergonomic Evaluation
HCI-Ed in 7 Steps <ul><li>Context & Learning Values - Hypothesis </li></ul><ul><li>(Iterative) Design – Requirements  </li...
Research Problem <ul><ul><li>Distributed learning teams in team-based learning situations aim to achieve organisational co...
<ul><li>Key CSCeL concept: Collaboration Awareness </li></ul><ul><ul><li>Students’ difficulties </li></ul></ul><ul><ul><li...
Collaboration Awareness <ul><ul><li>Students’ difficulties:  In addition to f2f Collaborative Learning difficulties, CSCeL...
<ul><ul><li>3C Model:  </li></ul></ul><ul><ul><li>C ommunication,  </li></ul></ul><ul><ul><li>C oordination & knowledge </...
<ul><li>If group awareness is a prerequisite for convergence then increase of group awareness will result in increase of o...
<ul><li>Euro-CAT… </li></ul><ul><li>C ollaboration  A wareness  T ool  </li></ul><ul><li>Task-independent </li></ul><ul><l...
Step2:  (Iterative) Design - Requirements The design can be defined as an individual and collective activity, finalized by...
Key Concepts for Design Users Now 1CH 2DR  3E 4D 5E 6D 7RTR Group Progress Scheduling Trail tracking Awareness Cues
Euro-CAT Screenshot 1CH 2DR  3E 4D 5E 6D 7RTR
Step3:  User/Expert Evaluation @ this workshop <ul><li>Euro-CAT Ergonomic Evaluation  </li></ul><ul><ul><li>Pedagogical Ut...
Results 1a/3: Questionnaire
Results 1b/3 Questionnaire EuroCAT Ergonomics Mean Std. Deviation Walk up Usability  4.4000 .54772 Ease of Use 4.0000 .707...
Results 3a/3
Results 3b/3
Results 3c/3
Solutions
<ul><li>Ask the e-tutors/e-learners for solutions! </li></ul><ul><li>Direct fit between educational task and the method ch...
<ul><li>Utility </li></ul><ul><li>Usability </li></ul><ul><li>Acceptability </li></ul><ul><li>Learner/user-centred design ...
Thank you! Questions? Dr. Niki Lambropoulos [email_address] Dr. Margarida Romero [email_address]
Upcoming SlideShare
Loading in …5
×

Expoelearning 2010 Virtual Campus International Quality In E Learning Lambropoulos Romero

646 views

Published on

Lambropoulos, N. & Romero, M. (2010) Quality in collaborative learning. EuroCAT ergonomic analysis. Sesión Virtual Campus International 2010. IX Congreso Internacional de e-learning ExpoElearning 2010. 24 y 25 de febrero. Madrid.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

Expoelearning 2010 Virtual Campus International Quality In E Learning Lambropoulos Romero

  1. 1. Quality Assurance in Computer Supported Collaborative Learning Environments Euro-CAT Ergonomic Evaluation Dr. Niki Lambropoulos * Dr. Margarida Romero
  2. 2. <ul><li>What is Quality? </li></ul><ul><li>Utility, usability, acceptability </li></ul><ul><ul><li>Case study : Toll staff training (learning situation + ergonomic criteria) </li></ul></ul><ul><li>CSCL ergonimic evaluation </li></ul><ul><ul><li>Case study : Euro-CAT Ergonomic Evaluation </li></ul></ul><ul><li>5 quality criteria </li></ul>Quality Assurance in Computer Supported Collaborative Learning Environments Agenda
  3. 3. What is quality?
  4. 4. <ul><li>Fit of purpose </li></ul><ul><li>Bergman and Klefsjö (1994): </li></ul><ul><li>“ The quality of a product (article or service) is its ability to satisfy </li></ul><ul><li>the needs and expectations of the customers” </li></ul><ul><li>European Foundation for Quality in eLearning (EFQUEL) </li></ul>
  5. 5. 1. Utility, usability, acceptability Case study : Toll staff training (learning situation + ergonomic criteria)
  6. 6. Tooll staff training needs <ul><li>USA Toll Tariffs: </li></ul><ul><li>No training is necessary, even children can became toll staff ! </li></ul><ul><li>European Toll Tariffs: </li></ul><ul><li>Intensive training needs to achieve 2 main learning objectives </li></ul><ul><ul><li>Vehicle category identification </li></ul></ul><ul><ul><li>Angry management </li></ul></ul>
  7. 7. Where and when can a toll worker could follow an elearning course? During the work hours? During the coffee break? At home?
  8. 8. As if the learning situation were not enough complex.... we will consider the usability criteria Is the course useful ? Is the course easy to use ? … mmmh, if the course is not useful, compatible to my workplace and out of the work life, reliable… and my colleagues don’t like the course, will I accept the course ?
  9. 9. <ul><li>As if the learning situation was not complex enough.... we will consider the ergonomics criteria </li></ul><ul><ul><li>UTILITY </li></ul></ul><ul><ul><li>USABILITY </li></ul></ul><ul><ul><li>ACCESSIBILITY </li></ul></ul>
  10. 10. As if the learning situation were not enough complex.... we will consider the usability criteria <ul><li>UTILITY </li></ul><ul><li>Relevance and efficacy (Tricot, 2007) for the performance in academic settings or the workplace </li></ul>Is the course useful ?
  11. 11. As if the learning situation were not enough complex.... we will consider the usability criteria Is the course easy to use ? <ul><li>USABILITY </li></ul><ul><li>Computer Learning Environments easy to learn and easy to use (Preece et al, 1994) </li></ul>
  12. 12. As if the learning situation were not enough complex.... we will consider the usability criteria <ul><li>ACCEPTABILITY </li></ul><ul><li>Not only consider the usefulness and usability, but also robustness, cost and reliability of ICT applications </li></ul>… mmmh, if the course is not useful, compatible to my workplace and out of the work life, reliable… and my colleagues don’t like the course, will I accept the course ?
  13. 13. As if the learning situation were not enough complex.... we will consider the usability criteria <ul><li>SOCIAL ACCEPTABILITY </li></ul><ul><li>Not only consider the usefulness and usability, but also robustness, cost and reliability of ICT applications </li></ul>Joel says the course is nuts; for me, is quite boring, and you? No, Joel is nuts, the course is just not enough challenging and interactive [Joel] The course don’t work in my computer… I needed to use my son computer to follow the course…
  14. 14. Nielsen’s Taxonomy of System Acceptability
  15. 15. Toll staff training – Case study Conclusions <ul><li>In addition to pedagogical criteria we need to consider the utility, usability and (social) acceptability criteria </li></ul><ul><li>All these criteria need to be considered for a successful elearning course </li></ul>
  16. 16. 2. CSCL ergonomic evaluation Case study: Euro-CAT Ergonomic Evaluation
  17. 17. HCI-Ed in 7 Steps <ul><li>Context & Learning Values - Hypothesis </li></ul><ul><li>(Iterative) Design – Requirements </li></ul><ul><li>Evaluation with user groups/experts </li></ul><ul><li>Development </li></ul><ul><li>Evaluation with user groups </li></ul><ul><li>Re-Design & Development </li></ul><ul><li>Study & Research - Tool Release </li></ul>1CH 2DR 3E 4D 5E 6D 7RTR
  18. 18. Research Problem <ul><ul><li>Distributed learning teams in team-based learning situations aim to achieve organisational convergence. </li></ul></ul><ul><ul><li>However, there are collaboration problems due to mediated distance & time flexibility. </li></ul></ul>1CH 2DR 3E 4D 5E 6D 7RTR
  19. 19. <ul><li>Key CSCeL concept: Collaboration Awareness </li></ul><ul><ul><li>Students’ difficulties </li></ul></ul><ul><ul><li>Group Awareness </li></ul></ul><ul><ul><li>Time convergence </li></ul></ul><ul><li>Learners’ Intentions (learning goals): Task Convergence </li></ul><ul><li>Users’ operations (actions/tasks): Pedagogical Usability </li></ul>Step1a: Context & Learning Values 1C H 2DR 3E 4D 5E 6D 7RTR
  20. 20. Collaboration Awareness <ul><ul><li>Students’ difficulties: In addition to f2f Collaborative Learning difficulties, CSCeL situation adds Computer Mediated Communication (CMC) and ILE difficulties (Kirschner, 2009) </li></ul></ul><ul><ul><li>Group Awareness: understanding of the activities of others, which provides context for your own activity (Dourish & Bellotti, 1992) </li></ul></ul><ul><ul><li>Group time awareness: Distributed learning students need to develop an awareness of their team-mates’ temporal patterns in order to regulate the Time-on-Task efficiently in the collective level (Romero, 2010) </li></ul></ul>Being Human 1C H 2DR 3E 4D 5E 6D 7RTR
  21. 21. <ul><ul><li>3C Model: </li></ul></ul><ul><ul><li>C ommunication, </li></ul></ul><ul><ul><li>C oordination & knowledge </li></ul></ul><ul><ul><li>C onvergence (Ellis et al, 1991) </li></ul></ul>How? 1C H 2DR 3E 4D 5E 6D 7RTR
  22. 22. <ul><li>If group awareness is a prerequisite for convergence then increase of group awareness will result in increase of organisational convergence and thus, collaboration awareness. </li></ul>Step1b: Hypothesis 1C H 2DR 3E 4D 5E 6D 7RTR
  23. 23. <ul><li>Euro-CAT… </li></ul><ul><li>C ollaboration A wareness T ool </li></ul><ul><li>Task-independent </li></ul><ul><li>Supports 3C via </li></ul><ul><ul><li>Mirroring capabilities </li></ul></ul><ul><ul><li>Awareness cues </li></ul></ul><ul><li>Does not replace other 3C tools </li></ul>Step2: (Iterative) Design - Requirements 1CH 2DR 3E 4D 5E 6D 7RTR
  24. 24. Step2: (Iterative) Design - Requirements The design can be defined as an individual and collective activity, finalized by a project to develop a physical and symbolic artefact. The artefact is rarely well defined in this step. 1CH 2DR 3E 4D 5E 6D 7RTR
  25. 25. Key Concepts for Design Users Now 1CH 2DR 3E 4D 5E 6D 7RTR Group Progress Scheduling Trail tracking Awareness Cues
  26. 26. Euro-CAT Screenshot 1CH 2DR 3E 4D 5E 6D 7RTR
  27. 27. Step3: User/Expert Evaluation @ this workshop <ul><li>Euro-CAT Ergonomic Evaluation </li></ul><ul><ul><li>Pedagogical Utility/Usability/Accessibility Questionnaire & System Usability Scale (SUS) </li></ul></ul><ul><ul><li>Experts/users evaluation </li></ul></ul><ul><ul><li>Think aloud protocol </li></ul></ul>1CH 2DR 3E 4D 5E 6D 7RTR
  28. 28. Results 1a/3: Questionnaire
  29. 29. Results 1b/3 Questionnaire EuroCAT Ergonomics Mean Std. Deviation Walk up Usability 4.4000 .54772 Ease of Use 4.0000 .70711 Functionality Integration 3.6000 .54772 Learnability 4.2000 .44721 Interface Attractiveness 2.4000 1.14018 Originality 3.6000 1.51658 Collaboration 3.0000 .70711 Navigation 4.8000 .44721 Task Regulation 3.8000 1.48324 Message Legibility 4.0000 1.22474 Observe Availabilities 4.2000 .83666 Facilitate Group Work 3.2000 .83666 Overall Satisfaction 3.8000 .83666 Overall Enjoyability 3.6000 1.14018 Overall Usability 3.0000 .70711 Overall Score 3.706/5.00
  30. 30. Results 3a/3
  31. 31. Results 3b/3
  32. 32. Results 3c/3
  33. 33. Solutions
  34. 34. <ul><li>Ask the e-tutors/e-learners for solutions! </li></ul><ul><li>Direct fit between educational task and the method chosen to pursue it </li></ul><ul><li>Iterative Design </li></ul><ul><ul><li>Engaging different user/expert groups </li></ul></ul><ul><li>Next step: Evaluation in context </li></ul>Conclusions & Insights
  35. 35. <ul><li>Utility </li></ul><ul><li>Usability </li></ul><ul><li>Acceptability </li></ul><ul><li>Learner/user-centred design (HCI-Ed) </li></ul><ul><li>Adaptive eLearning: </li></ul><ul><li>- Diverse Teaching/Learning Styles </li></ul><ul><li>- Learning Alone/Learning Together </li></ul><ul><li>- Learning anywhere/anytime </li></ul>5 criterios de calidad en E-learning Ergonomics
  36. 36. Thank you! Questions? Dr. Niki Lambropoulos [email_address] Dr. Margarida Romero [email_address]

×