Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Nikos Michailidis 1 , Stavros Demetriadis 1 , Yannis Dimitriadis 2 1 Multimedia Lab, Aristotle University of Thessaloniki ...
Contents ICALT 2010 (5/7/2010)   Michailidis, Demetriadis & Dimitriadis  <ul><li>Introduction </li></ul><ul><li>Collage Ed...
Introduction (I) ICALT 2010 (5/7/2010)   Michailidis, Demetriadis & Dimitriadis  <ul><li>Main objectives of the study: </l...
Introduction (II) ICALT 2010 (5/7/2010)   Michailidis, Demetriadis & Dimitriadis  <ul><li>CSCL field: </li></ul><ul><ul><l...
Collage Editor (I) (COLlaborative LeArning desiGn Editor) http://gsic.tel.uva.es/collage <ul><ul><li>Graphic-based high-le...
Collage Editor (II) ICALT 2010 (5/7/2010)   Michailidis, Demetriadis & Dimitriadis  <ul><li>Collaborative Learning Flow Pa...
Collage Editor (I ΙΙ ) <ul><li>Pattern-based design process </li></ul><ul><li>Combining & configuring CLFPs </li></ul><ul>...
Previous Collage Evaluations ICALT 2010 (5/7/2010)   Michailidis, Demetriadis & Dimitriadis   / 24 “ Collage Workshops” “ ...
Case Study:  Exploring Collage Editor Efficiency ICALT 2010 (5/7/2010)   Michailidis, Demetriadis & Dimitriadis   / 24
Context of the case study <ul><li>Postgraduate course, department of Informatics, Aristotle University of Thessaloniki (sp...
Added value of the case study <ul><li>Further explore the usability & efficiency of Collage editor </li></ul><ul><ul><li>C...
Objectives of the scenario <ul><li>Comprehend the concept of scripting collaborative learning, through the design of reali...
Description of the scenario (I) <ul><li>Students formed dyads in order to design & implement a realistic CSCL script </li>...
Description of the scenario (II) <ul><li>Support & guidance provided throughout the duration of the study </li></ul><ul><u...
Evaluation methodology <ul><li>Data sources used: </li></ul><ul><li>3 topics of interest </li></ul><ul><ul><li>Evaluation ...
Evaluation results (I) <ul><li>Students perceived the experience of the scenario as a positive improvement over their prev...
Evaluation results (II) <ul><li>Collage & the overall design process are useful & suitable for CSCL designers and especial...
Evaluation results (III) <ul><li>CLFPs in Collage facilitate & guide users through the CSCL script design process </li></u...
Evaluation results (IV) <ul><li>Collage editor could be further improved in terms of technical usability, pedagogical usef...
Conclusions (I) <ul><li>Participants highly appreciated the CL activity of the case study </li></ul><ul><ul><li>The overal...
Conclusions (II) ICALT 2010 (5/7/2010)   Michailidis, Demetriadis & Dimitriadis   / 24 <ul><li>Certain improvements could ...
Future research  / 24 <ul><li>New case studies </li></ul><ul><ul><li>In different educational & technological contexts </l...
References  / 24 <ul><li>P. Dillenbourg (2002). “Over-scripting CSCL: The risks of blending collaborative learning with in...
The end … <ul><li>Thank You! </li></ul><ul><li>Questions? </li></ul><ul><li>[email_address] </li></ul>ICALT 2010 (5/7/2010...
Upcoming SlideShare
Loading in …5
×

Scripted Collaborative Learning Based on Collaborative Learning Flow Patterns: A Case Study Using Collage Editor

1,036 views

Published on

Published in: Education, Technology

Scripted Collaborative Learning Based on Collaborative Learning Flow Patterns: A Case Study Using Collage Editor

  1. 1. Nikos Michailidis 1 , Stavros Demetriadis 1 , Yannis Dimitriadis 2 1 Multimedia Lab, Aristotle University of Thessaloniki 2 GSIC-EMIC group, University of Valladolid ICALT 2010 5 - 7 July 2010 Sousse, Tunisia Scripted Collaborative Learning Based on Collaborative Learning Flow Patterns: A Case Study Using Collage Editor 5 – 7 July, Sousse, Tunisia ICALT 2010 / 24
  2. 2. Contents ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis <ul><li>Introduction </li></ul><ul><li>Collage Editor </li></ul><ul><li>Case study: Exploring Collage Efficiency </li></ul><ul><li>Evaluation Findings & Discussion </li></ul><ul><li>Conclusions & Future Research </li></ul> / 24
  3. 3. Introduction (I) ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis <ul><li>Main objectives of the study: </li></ul><ul><ul><li>Provide further evaluation evidence regarding the strong and weak points of Collage editor for the creation of pedagogically effective CSCL scripts </li></ul></ul><ul><ul><li>Propose possible ways to enhance and improve Collage both in terms of pedagogical & technical features </li></ul></ul> / 24
  4. 4. Introduction (II) ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis <ul><li>CSCL field: </li></ul><ul><ul><li>Social interactions as key element for learning </li></ul></ul><ul><ul><li>Free collaboration does not necessarily produce fruitful interactions (Dillenbourg, 2002) </li></ul></ul><ul><ul><li>Scripting have been researched as an effective mechanism to structure collaboration in order to increase the chances of effective peer interaction </li></ul></ul><ul><li>How can teachers create “potentially effective” “computer-interpretable” collaboration scripts according to the necessities of their situation? </li></ul><ul><li>( Hernández-Leo et al., 2006) </li></ul> / 24
  5. 5. Collage Editor (I) (COLlaborative LeArning desiGn Editor) http://gsic.tel.uva.es/collage <ul><ul><li>Graphic-based high-level specialized authoring tool for collaborative learning. </li></ul></ul><ul><ul><li>Based on Reload. </li></ul></ul><ul><ul><li>IMS-LD level A compliant </li></ul></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24
  6. 6. Collage Editor (II) ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis <ul><li>Collaborative Learning Flow Patters (CLFP): </li></ul><ul><ul><li>Collection of broadly accepted techniques repetitively used by CL practitioners (best practices) when structuring the flow of activities involved in CL </li></ul></ul><ul><ul><li>Way of communicating CL expertise </li></ul></ul><ul><ul><li>Can be applied to a wide range of scenarios, since they are neutral to specific domain content </li></ul></ul> / 24 Jigsaw CLFP
  7. 7. Collage Editor (I ΙΙ ) <ul><li>Pattern-based design process </li></ul><ul><li>Combining & configuring CLFPs </li></ul><ul><li>Support for CLFP selection </li></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24 Learning objectives Type of problems / tasks Complexity (CL experience)
  8. 8. Previous Collage Evaluations ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24 “ Collage Workshops” “ Network Management” “ Planet Game” Description 2 experiences with target audience and 2 mini-cases, using a mixed evaluation method. Authentic collaborative learning scenario at the laboratory of University of Valladolid. A CLFP-based approach to the “Planet Game” scenario, during the ICALT 2006 workshop “Comparing educational modeling language on a case study”. Objectives Creating CSCL scripts based on CLFPs using Collage, focusing on topics such as the pattern-based design process, the use of Collage and the characteristics of the potential audience. Get evidence on whether the CSCL script created with Collage can be successfully used in real situations. Solving a third-party scenario using Collage and identifying the added value and/or weaknesses of this approach. Results The use of CLFPs facilitates the design task and the editor itself provides an intuitive and easy way for non-expert users to design complex collaborative learning scenarios. Need for making the description of CLFPs more flexible. Significant evidence on the suitability of Collage editor for facilitating the introduction of collaborative learning, since the tool can support the creation of meaningful and effective CSCL scripts. Collage succeeded in implementing the scenario and also pointed out some significant advantages in terms of design reusability, generality and user-friendliness.
  9. 9. Case Study: Exploring Collage Editor Efficiency ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24
  10. 10. Context of the case study <ul><li>Postgraduate course, department of Informatics, Aristotle University of Thessaloniki (spring semester 2009) </li></ul><ul><li>21 postgraduate students </li></ul><ul><li>Course “Technology-enhanced Constructivist Learning Environments” </li></ul><ul><li>2 weeks scenario </li></ul><ul><ul><li>2 weeks of collaborative activities and a synchronous laboratory session in between </li></ul></ul><ul><ul><li>Tools: Collage editor </li></ul></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24
  11. 11. Added value of the case study <ul><li>Further explore the usability & efficiency of Collage editor </li></ul><ul><ul><li>Consistent & thorough evaluation </li></ul></ul><ul><ul><ul><li>Employs non-expert users that are not biased by any previous knowledge (CLFPs or Collage) </li></ul></ul></ul><ul><ul><ul><li>Most of the users are teachers & educators in real life </li></ul></ul></ul><ul><ul><li>Focus on whether the CSCL scripts created with Collage actually reflects the teacher’s design intentions </li></ul></ul><ul><ul><ul><li>Identify possible design constraints that limit flexibility and adaptability </li></ul></ul></ul><ul><ul><ul><li>Conclude on possible ways to enhance and improve Collage both in terms of pedagogical & technical features </li></ul></ul></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24
  12. 12. Objectives of the scenario <ul><li>Comprehend the concept of scripting collaborative learning, through the design of realistic CSCL scripts </li></ul><ul><ul><li>Collage users employ the tool as a means for learning about scripts </li></ul></ul><ul><li>Students are offered the opportunity to familiarize themselves with a script authoring tool (Collage) </li></ul><ul><ul><li>Involving them in its use as much as possible </li></ul></ul><ul><li>Further evaluation evidence regarding the strong and weak points of Collage editor </li></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24
  13. 13. Description of the scenario (I) <ul><li>Students formed dyads in order to design & implement a realistic CSCL script </li></ul><ul><ul><li>First on paper and then using Collage editor </li></ul></ul><ul><ul><li>The generated CSCL scripts should be designed in such way in order to incorporate at least 2 CLFPs from those available in Collage: </li></ul></ul><ul><ul><ul><li>Jigsaw CLFP in combination with at least 1 more CLFP </li></ul></ul></ul><ul><li>Students could freely choose from any subject domain </li></ul><ul><ul><li>Elements of originality & interdisciplinary required </li></ul></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24
  14. 14. Description of the scenario (II) <ul><li>Support & guidance provided throughout the duration of the study </li></ul><ul><ul><li>Introduction to the activity and its objectives </li></ul></ul><ul><ul><li>Collage editor user guides & detailed tutorials </li></ul></ul><ul><ul><li>Collage editor live demonstration & training </li></ul></ul><ul><li>Validation of the produced CSCL scripts (Collage UoL) using a script enactment tool (i.e. a script player) </li></ul><ul><ul><li>Reload LD player was provided to students </li></ul></ul><ul><ul><li>Most of them already familiar with the tool but also extra guidance provided </li></ul></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24
  15. 15. Evaluation methodology <ul><li>Data sources used: </li></ul><ul><li>3 topics of interest </li></ul><ul><ul><li>Evaluation of the case study’s activity itself </li></ul></ul><ul><ul><li>General use & technical characteristics of Collage </li></ul></ul><ul><ul><li>Pedagogical evaluation of Collage & particularly the CLFP-based design process </li></ul></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24 Data source Type of data Hard-copy questionnaire Quantitative ratings & qualitative explanations of the students Focus group interviews Qualitative students’ opinions Student deliverables IMS-LD based CSCL produced as results of the activities
  16. 16. Evaluation results (I) <ul><li>Students perceived the experience of the scenario as a positive improvement over their previous courses </li></ul><ul><ul><li>Activity objectives were clear based on the instructions & the supporting material provided </li></ul></ul><ul><ul><li>The activity presents many enhancements with respect to previous student’s experience, in terms of utilization of technology tools </li></ul></ul><ul><ul><li>Students appreciate the use of Collage, despite minor technical problems reported (i.e. Java conflicts) </li></ul></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24 <ul><li>90.5% of students rated as positive the structure of the case study activity regarding its effectiveness to reach the objectives of the course. Need for more time [quest]. </li></ul><ul><li>‘‘ The live demo and the supporting material helped me to successfully design & </li></ul><ul><li>implement my script, [focus]’’. </li></ul>Support Data <ul><li>85.7% of students indicated that the experience of the case study introduced a lot of improvements compared to their previous ones [quest]. </li></ul><ul><li>‘‘ Authoring tools like Collage increase my efficiency and interest in the activities involving designing CSCL scripts […] I would definitely recommend it [focus]’’. </li></ul>Support Data <ul><li>“ I reckon that through the case study activity, I got familiar with most features in Collage […], however, practice makes perfect, [focus]”. </li></ul><ul><li>“ […] Collage free & open-source nature will definitely support the wider adoption of scripted CL tools to the CSCL community [focus]”. </li></ul>Support Data
  17. 17. Evaluation results (II) <ul><li>Collage & the overall design process are useful & suitable for CSCL designers and especially non-experienced users </li></ul><ul><ul><li>Collage was easy to use, user-friendly & intuitive, even for non-experienced users </li></ul></ul><ul><ul><li>Collage editor GUI facilitates the design process for creating CSCL scripts in a series of easy-to-follow steps </li></ul></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24 <ul><li>76% of the participants stated that had very little experience with CL activities as educators & 85% stated having never used another CSCL script authoring tool [quest]. </li></ul><ul><li>90.5% of students rated Collage as an easy-to-learn & u ser-friendly tool [quest]. </li></ul><ul><li>“ Collage is a great tool for facilitating the design of CSCL scripts especially for users without great experience […] the usage of buttons & graphics is quite obvious [focus]”. </li></ul>Support Data <ul><li>Overall evaluation of Collage: 5.24, deviation: 1.73 (max.: 7) [quest]. </li></ul><ul><li>“ The interface of Collage editor Window was functional, user-friendly and helped me in the development of my CSCL script design by simply editing and customizing all the provided fields in a straightforward manner […] [focus]”. </li></ul>Support Data
  18. 18. Evaluation results (III) <ul><li>CLFPs in Collage facilitate & guide users through the CSCL script design process </li></ul><ul><ul><li>CLFP selection interface helps to determine the most appropriate CLFP based on objectives, type of problems & complexity </li></ul></ul><ul><ul><li>Students successfully created CSCL scripts based on their scenarios </li></ul></ul><ul><ul><li>Needs of further flexibility & adaptability emerged, as many students found the design process of available CLFPs restrictive </li></ul></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24 <ul><li>“ The CLFP Selector helped me to choose the CLFP best suited for the desired educational and collaborative objectives of my scenario […] [focus]”. </li></ul><ul><li>Satisfaction with CLFP representation and advice: 5.38, deviation: 1.46 (max.: 7) [quest]. </li></ul>Support Data <ul><li>“ Collage helped me to structure the activities flow for my scenario […] the right-click feature for replacing a CLFP phase with another CLFP is simple & functional [focus]”. </li></ul><ul><li>8 out of 10 scripts generated were successfully validated with Reload LD Player [script]. </li></ul><ul><li>Satisfaction with produced scripts: 5.05, deviation: 1.73 (max.: 7) [quest]. </li></ul>Support Data <ul><li>45% of students found the structure of available CLFPs too coercive, [quest]. </li></ul><ul><li>Design process and CLFP structure flexibility: 4.4, deviation: 2.21 (max.:7) [quest]. </li></ul><ul><li>“ […], although my final Collage UoL reflects my original design intentions, I had to adjust my CL scenario because I couldn’t modify the number of Problem Solvers & Listeners on each phase of the TAPPS CLFP […] similar for the TPS [focus]”. </li></ul>Support Data
  19. 19. Evaluation results (IV) <ul><li>Collage editor could be further improved in terms of technical usability, pedagogical usefulness and flexibility </li></ul><ul><ul><li>Participants identified several enhancements in Collage user-interface that could further improve its usability. </li></ul></ul><ul><ul><li>Students also proposed the development of some features and operations that could further improve the pedagogical usefulness & flexibility of Collage </li></ul></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24 <ul><li>90% of the participating students stated that Collage needs to be integrated with a “mini-player” or with a method to “preliminary view” the script [quest]. </li></ul><ul><li>“ […] the lack of supporting an Undo feature is a disadvantage, […] also Collage GUI could be improved with a “Tooltip” feature or a rich text editor for inserting text [focus]”. </li></ul><ul><li>“ […] Collage usability could be further enhanced by a CLFP upload GUI [focus]”. </li></ul>Support Data <ul><li>80% of the students would prefer more CLFPs in Collage & 70% believe that proven collaborative activity patterns or well-known script templates should also be added [quest]. </li></ul><ul><li>Many participants identified the need for including assessment methods & adaptive characteristics in the Collage CLFP design process [quest]. </li></ul><ul><li>“ Need for more CLFP combining methods […] [focus]”. </li></ul>Support Data
  20. 20. Conclusions (I) <ul><li>Participants highly appreciated the CL activity of the case study </li></ul><ul><ul><li>The overall experience perceived as a positive improvement over previous non-collaborative courses. </li></ul></ul><ul><li>Collage editor is a useful tool for facilitating & guiding non-experienced designers in the creation of pedagogically effective CSCL scripts </li></ul><ul><ul><li>while reducing the complexity of manipulating the IMS-LD specification by means of a user-friendly GUI </li></ul></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24
  21. 21. Conclusions (II) ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24 <ul><li>Certain improvements could further enhance Collage potential (but also the future design of other CSCL script authoring tools): </li></ul><ul><ul><li>Support for IMS-LD levels B & C </li></ul></ul><ul><ul><li>Integration of “mini-players” or enactment tools </li></ul></ul><ul><ul><li>Integration of adaptive & assessment characteristics to tools for scripted collaboration </li></ul></ul><ul><ul><li>Adding more usability features, e.g.: Undo, tooltip, Reach HTML editor </li></ul></ul><ul><ul><li>Implementing new ways of combining the CLFPs (or other learning design patterns) </li></ul></ul>
  22. 22. Future research / 24 <ul><li>New case studies </li></ul><ul><ul><li>In different educational & technological contexts </li></ul></ul><ul><ul><li>For further evaluate the applicability & suitability of assessment & adaptation patterns (Karakostas & Demetriadis, 2009; Villasclaras-Fernández et al., 2009a) </li></ul></ul><ul><li>Investigate the possibility of adding different learning design patterns in Collage editor (apart from CLFPs) </li></ul><ul><ul><li>e.g. activity-level, resource-level, etc </li></ul></ul><ul><li>Future research may explore the usability of complementary tools </li></ul><ul><ul><li>e.g. InstanceCollage authoring tool (Villasclaras-Fernández et al., 2009b) </li></ul></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis
  23. 23. References / 24 <ul><li>P. Dillenbourg (2002). “Over-scripting CSCL: The risks of blending collaborative learning with instructional design,”. In Three Worlds of CSCL: Can We Support CSCL?, P. A. K. Heerlen, Ed. Heerlen, The Netherlands: Open University Nederland, 2002, pp. 61–91. </li></ul><ul><li>D. Hernández-Leo, E. D. Villasclaras-Fernández, I. M. Jorrín-Abellán, J. I. Asensio-Pérez, Y. Dimitriadis, I. Ruiz-Requies, & B. Rubia-Avi. (2006). “COLLAGE, a collaborative learning design editor based on patterns,” Educ. Technol. Soc., vol. 9, no. 1, pp. 58–71, 2006. </li></ul><ul><li>D. Hernández-Leo, D. Burgos, C. Tattersall & R. Koper. (2007a). “Representing Computer-Supported Collaborative Learning macro-scripts using IMS Learning Design”, Proc. of the 2nd European Conference on Technology Enhanced Learning, CEUR Workshop Proceedings, EC-TEL'07, Crete, Greece, September 2007. </li></ul><ul><li>D. Hernández-Leo, E.D. Villasclaras-Fernández, J.I. Asensio-Pérez & Y.A. Dimitriadis. (2007b). “Some findings from the evaluation of the Collage authoring tool”, TENCompetence Open Workshop on current research on IMS-LD and Lifelong Competence Development Infrastructures, pp. 27-32, Barcelona, Spain. </li></ul><ul><li>N. P. Michailidis & S. Demetriadis. (2009). “Technology tools for scripted collaborative learning: The case of the Reload LD Player”, Proc. 4th Balkan Conference on Informatics (BCI 09), Thessaloniki, Greece, September 17-19 2009, pp 163-168 </li></ul><ul><li>A. Karakostas & S. Demetriadis. (2009). “Adaptation Patterns in Systems for Scripted Collaborative Learning”, in Proc. of the International Conference on Computer Supported Collaborative Learning 2009, (CSCL 2009), pp. 477-481. </li></ul><ul><li>E. D. Villasclaras-Fernández, D. Hernández-Leo, J.I. Asensio-Pérez & Y. Dimitriadis. (2009a). “Incorporating assessment in a pattern-based design process for CSCL scripts”, Computers in Human Behavior, 2009, 25(5), pp. 1028-1039 </li></ul><ul><li>E.D. Villasclaras-Fernández, J.A. Hernández-Gonzalo, D. Hernández-Leo, J.I. Asensio-Pérez, Y.A. Dimitriadis & A. Martinez-Mones. (2009b) “InstanceCollage: A Tool for the Particularization of Collaborative IMS-LD Scripts”, Educational Technology & Society, 2009, vol.12, no. 3, pp. 56-70. </li></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis
  24. 24. The end … <ul><li>Thank You! </li></ul><ul><li>Questions? </li></ul><ul><li>[email_address] </li></ul>ICALT 2010 (5/7/2010) Michailidis, Demetriadis & Dimitriadis / 24

×