Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Participatory and Constructivistic eLearning Quality


Published on

The ongoing discussion on the issue of the quality of academic eLearning criticizes the dominating culture of “quality” linked to the industrial production, mostly based on rationalization and conformity to pre-defined standards, that requires the collection of massive quantitative data, with major interest on educational outputs as a vision of system’s productivity (Ehlers & Schneckenberg, 2010, Ghislandi, 2008, 2012). Instead, quality of education requires reflection and deep understanding of complex contextual elements, interactions and relational dimensions that are essential and often invisible to traditional assessment tools. In this paper, through the presentation of an eLearning course as case study, we attempt to show how participatory/constructivist evaluation can become a key practice to support the quality of an eLearning experience from the point of view of the learner. In fact, as it emerges from the analysis, this open form of evaluation has an enormous potential to address practices towards the values/concepts underlying meaning making processes inside a transformative learning culture. Building on this results, we contend that the evaluation of quality needs to integrate methods that open up the sense of practices and values to the participant. To this regard, we also discuss how qualitative constructivist approaches to evaluation can make an important contribution drawing on the coherence found between its epistemological and ontological assumptions and the idea of new cultures of quality evaluation where the participants build the own values and concepts of goodness.

Published in: Education
  • Be the first to comment

Participatory and Constructivistic eLearning Quality

  1. 1. 5th International Conference of Education, Research and Innovation Madrid, 19-21 November 2012 Patrizia Ghislandi – Juliana Raffaghelli University of Trento
  2. 2. A) eLearning evolution andthe problem of evaluatingqualityB) ReconceptualizingQuality of eLearning From Standards to a culture of Quality Fourth Generation Evaluation: understanding the epistemological approach to evaluation of eLearning quality.C) A Case Study The context of the research: evaluating quality inside the eLFO12 Learning Community The Methodological Approach D) Conclusions and Tools for a participatory evaluation of eLearning quality Debate! Results
  3. 3. eLearning evolution and the problem ofevaluating qualityTraditionaleLearning: deliveryof online contents Evolution: Open, networked Learning
  4. 4. Reconceptualizing Quality ofeLearning…The evolution of quality has been influenced by industrial models ○ Predefined Standards ○ Measurements on finished products ○ Quantitative Evaluation ○ Intangible aspects (relation, communication) in a second place for the evaluation tools ○ The need to accountable to external referees ○ The gap between research and educational practice ○ A vision of evaluation as certification, objectivistic quantitative paradigm 4
  5. 5. Reconceptualizing Qualityof eLearning Diverse Cultures of Quality are underpinned by diverse values:  Exceptional/Original: the value is on the uniqueness  Distinctiveness: not for all  Excellence: The highest levels of performance  Fitness for Purpose: doing what has been planned  Inclusive: all people can participate 5
  7. 7. Quality is a Complex issue 4- Diverse 2- Diverse Methods of Analys Meanings benchmarking guidelines Pedagogical standard Organizational Economical Qualitative Evaluation Technological Quantitative Evaluation Q Methodological5-Diverse Levels ofAnalysis Individual Group 3- Diverse times institutional Socio-cultural of intervention for the analysis in itinere 1-Diverse Perspectives ex ante The teacher ex post The Student The institution The external evaluators7
  8. 8. Reconceptualizing Qualityof eLearning Quality is not an intrinsic, universal value It is very much about the methodology of evaluation, And the substantial epistemological principles and values underlying the process of evaluation
  9. 9. Fourth Generation Evaluation(Guba & Lincoln, 1989) Understanding the epistemological approach to evaluation of eLearning quality Four generations of Evaluation : There is a wise, well informed, objective, aseptic, external EVALUATOR  1) Measurement There are unaware, badly  2) Observation informed, too emotionally engaged EVALUATED PEOPLE  3) Judgement (and the processes/products coming from them There is an expert, open to share,  4) Joint Reflection committed EVALUATOR There are committed, willing to and Transformation learn, EVALUATED
  10. 10. Understanding the epistemological approach to evaluation of eLearning quality Four generations of Evaluation (Guba&Lincoln, 1989) There is a wise, well informed, Quantitative  1) Measurement objective, aseptic, external Methods  2) Observation EVALUATOR  3) Judgement There are unaware, badly External informed, too emotionally Evaluation engaged EVALUATED Qualitative There is an expert, open to Methods  4) Joint Reflection share, committed EVALUATOR and Transformation There are committed, willing Self- Peer to learn, EVALUATED Evaluation External, agreed, frameworks of reference (EFQUEL, 2007)
  11. 11. The eLFO12 Learning Community
  12. 12. Defining our Case Study The decisive factor in defining a study a case study is the choice of the individual unit of study and the setting of its boundaries (…)the individual unit may be studied in a number of ways, for instance qualitatively or quantitatively, analytically or hermeneutically, or by mixed methods. This is not decisive for whether it is a case study or not; the demarcation of the unit’s boundaries is. (Flyvbjerg, 2011, p. 301)
  13. 13. Defining our Case Studya)  The decisive factor the educational process and in defining a study astrategies to improve it b) an eLearning setting Self-Peer study is the choice of the individual case Moodle platform Evaluation: eLFO12: eLearning unit of study and the setting of its interoperable boundaries (…)the individual unit may bewith Web 2.0 Trainers’ Log course for training tools AdAstra the trainers of studied in a number of ways, for instanceSocio- Rubrics Interviews operators of social qualitatively or quantitatively, analytically constructivist or approach to hermeneutically, andor by mixed methods. Thisadoption of health care the educational is not decisive for c) a professional learning whether it is a case study Technologies community composed by operators orhealth and social services willing of the unit’s of not; the demarcation boundaries trainers to improve their skills (Flyvbjerg, 2011, p. 301) Committement University of Trento –UNITN- /Direction of Health Care (Province of Trento) –PAT-
  14. 14. The Process •The log-book as part of learning activities to collect live reflections on the learning process. •A suite of rubrics as meta-evaluation instruments to support reflection on teaching and learning quality Rubrics •Interviews with participants to deepen on the emerging perceptions, ideas and reflections in the former instruments. exploring and constructing eLearning QualityThe learner’s Interviews Becoming aware of values and log-book practices for quality Making visible the invisible Transform
  15. 15. The Log-book Having introduced the course as a “socio- constructivist” oriented, the tutors asked the students to be active and collaborative ( a quality value) it was registered during the initial phases of the course, an important amount of discontent with the approach: to much “chaos” A lesson on the pedagogy of socio- constructivist approach Reflections and discussion Becoming more participant and enganged: aware of the importance of collaboration
  16. 16. Table 1 - Sloan-C pillars the connected organization of the adASTRA rubrics . Sloan-C pillars and and the connected organization of the adASTRA rubrics .The AdAstra Rubrics A set of rubrics, elaborated by the University of Trento research unit (Ghislandi & Pedroni, 2011), on the basis of SLOAN-C MODEL Sloan-C Pillars Rubric adASTRA Rubric Analysis, Design, Learning effectiveness Syllabus, Community of Practice, Screencast, Portfolio Student satisfaction Rubric Feedback Students Faculty satisfaction Rubric Feedback Teacher Cost effectiveness Not developed Access Rubric Accessibility
  17. 17. The AdAstra Rubrics Dimensions of Quality Analized Dimension Description Organization Analyses how the several course activities planned and scheduled are implemented consistently. Resources Analyses how the resources offered for learning are considered by learners, in the sense of completeness and coherence. Teaching Analyses how the teaching methods and strategies implemented are Methods and considered in the sense of effectiveness (the achievement learning Strategies goals as well as to maintain learners’ motivation). Comunication Analyses how the several synchronous and asynchronous instruments of communication are adopted (the usage is coherent with the learning goals and requirements) Collaboration Analyses how the students interact between them to achieve learning goals Technologies Analyses how the technological arrays are used, in the sense of the facilitation of learning. Assessment Analyses how the assessment is implemented, particularly taking into account the coherence between the type of assessment activities and what the course offered to learners.. The rubrics where implemented through 2 meetings and across 2 months of the course (1 month devoted to the learning design and 1 month to the delivery of the exercise module of peer learning), one at the beginning and another at a closing phase, being the researchers available to discuss with teachers and students if doubts arisen on the concepts driven.
  18. 18. Interviews: reflecting on qualityevaluationParticipatory meta-evaluation: the rubrics as instrumentsfor the reflection on course quality (excerpt on 1 of 4 dimensions) Dimension Results from Interviews (*) (*) the students transcriptions in Italian have been translated into English by the research team “It is really important to have something written that helps you to avoid mistakes when planning a course, and also to reflect on what had been done Coherence previously (…) to solve problems and to make changes” [eL5] “the level of detail has to be correlated with the dimension of course quality that you are to analyze, and this was perfectly represented into the rubrics” [eL4] “Diversely from the student side rubrics, the teacher side rubrics lack of items that recall how the teacher has reacted to critical issues, how did she feel with regard to those critical issues, so the teachers subjective perspective is missed somehow” [eL3] “It could have been introduced some question regarding the methodology adopted in the design phase or even better questions that allowe to establish if the teacher has followed coherently a pedagogical approach” [eL1] “The student rubric misses something regarding the relationships and the collaborative work…or this should be more evident….these represents a positive factor for the groups and single learning don’t they? [eL5].
  19. 19. eLFO12 Case Study: a participatory approach to theevaluation of Quality
  20. 20. Discussion of Results (1) The three moments of evaluation showed how the evaluation process worked progressively in addressing understanding and learning towards the values/concepts of the ELFO12 group The breakthrough in this process was the moment in which the learners were given the opportunity to revisit (self-evaluate) their own learning in the context of socio-constructivist theoretical framework The rubrics where considered coherent, complete, effective, though rather less efficient. But this meta- evaluation of the tools was given in the context of utility and use of the same: a socio-constructivist and technologically mediated learning activity/course.
  21. 21. Discussion of Results (2) The quality was evaluated not in the sense of external parameters of performance but in terms of the coherence and authenticity of the course learning model and the learners awareness and engagement within it. It is crucial to point out that both steps or “moments” of quality evaluation.
  22. 22. CONCLUSIONS The selection of qualitative methods,  a phenomenological approach based on narrative self-evaluation, peer- evaluation and meta-evaluation, emphasizes the interest on processes and on the empowerment of learners AS COMMITTED EVALUATED  This logic studies the topic within its context, uses an emerging design that accounts for reality as subjective and multiple, lessen the distance between “official” evaluators and participants (Denzin & Lincoln, 2011). As a result, the evaluation process encompasses a transformational (participatory/innovative) opportunity for the engaged individuals/ institutions.(Creswell, 2007; Mertens, 2009).
  23. 23. CONCLUSIONS this transformational perspective is the kernel of a quality learning culture:  a human group that take part of a learning experience as a deep, reflective experience, connected to the own professional/ personal identity ○ For which purposes do I learn? What can I do with this learning?  not just for accomplishing activities, recalling information, and obtaining credentials (course diploma). We call this become insiders of the culture of quality.
  24. 24. Thank you for your interest! For communications: