Evaluation of the IKASYS Programme


Published on

2009-07-02, Donostia (Gipuzkoa).

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Evaluation of the IKASYS Programme

  1. 1. Research, Reflections and Innovations in Integrating ICT in Education Evaluation of the IKASYS programme K. Santiago, J.F. Lukas, J. Etxeberria, and A. Gobantes UNIVERSITY OF THE BASQUE COUNTRY. Facultad FICE. Dpto. MIDE. Avda. Tolosa 70. 20018 San Sebastian, Spain. E-mail: karlos.santiago@ehu.es In this paper the IKASYS system is presented as is the methodology used in its evaluation. IKASYS is an innovative system in the field of educational research and which, with teaching activities of a wide-ranging nature, is used to develop resources that the pupil needs to achieve basic skills. It is a system whereby each pupil learns with his or her computer; it is made up of three elements: computer tools (hardware), (computer applications (software) and curricular strategies (content). Its function is to Exercise and Memorise material in an individualised process so that the pupil can, autonomously and according to his or her abilities, establish the level and rhythm of learning and do the corresponding exercises by himself or herself. The evaluation has two fundamental objectives which are, at the same time, complementary: the evaluation of the system itself and the evaluation of its impact. The evaluation of the system itself and its process of application is basically qualitative, its aim being to more precisely understand all the aspects related to the components of the system (hardware, software, curricular content) and to the procedures related to its application, as well as to lay down guidelines for enhancing both the system and its application. To this end, information and the opinions of pupils, parents, teachers and school heads are being gathered. The impact evaluation is undertaken through “quasi-experimental design with a non-equivalent control group”. In this design the dependent variable is the performance in the 5 different content areas (Basque, Spanish, English, mathematics and knowledge of the environment) and the capacity for self-regulation of the pupils Keywords: e-learning, ICT in education, Formative evaluation, Methodological complementarity 1.The Ikasys programmeTaking into consideration that each pupil has his or her own style and rhythm of learning and that one of theprincipal problems of education in this respect is to get it right as regards each student’s needs, the Federation ofIkastolas of the Basque Country have created a computer programme known as IKASYS and which has beenimplemented in a number of Primary Schools in the Basque Country. Nowadays, there are two factors accepted as needing suitable responses. The first of these refers to aregulated and contextualised undertaking of task-based exercises and which facilitates learning and enhances theresults for each pupil. From the teachers’ perspective, what is needed is strategic thinking about and a planningof educational sequences that include task-based exercises and memorising activities; these being withindedicated learning contexts and giving meaning to practical work so as to enable this type of content to beautomated, and thus facilitate the work of the student when dealing with more complex processes. The secondfactor underlines the importance of attending to the diversity of the needs of the pupils, to guide them and topersonalise the help for each (interest, skills, knowledge, and so on.) and to encourage self-regulation as anelement of self-control in learning which, in turn, is motivating for the pupil. The IKASYS programme arose from the conviction that technology, well-integrated into and contextualisedin a didactic sequence and in the curriculum of each area, could be a suitable answer and, at the same time, be amedium facilitating certain tasks for the teacher and the pupil so that each might better dedicate and optimisetheir time, efforts and resources in order to be able to attend to other needs.Some curricular content is internalised by means of doing exercises or through memorising, and it is in thesespheres where the greatest doubts arise, because it is here that the greatest differences are observed betweenpupils. To satisfactorily respond to the peculiarities of each, pupils usually have to undertake a number ofexercises or carry out memorisation. All this makes it enormously difficult for the teacher to suitably undertake a systematic monitoring of theprogress of each pupil. This is the context within which the IKASYS Project arose, motivated by the desire tofill this lacuna, and bring these resources to both pupils and the teaching staff in a personalised manner.The IKASYS programme is an innovative system in the sphere of educational research which, by means ofdidactic activities of a wide-ranging nature, is designed to develop the resources that pupils need in order toachieve basic skills. It is a system whereby each pupil can learn with his or her own PC. It has three functions: 51
  2. 2. Research, Reflections and Innovations in Integrating ICT in Education •For exercise task undertaking: a compilation of systematic and progressive activities aimed at exercising the pupil in various learning procedures. It provides, amongst other things, activities in calculations, orthography, problem solving, algorithms in physics, chemical formulae, and so on. •For memorising: a compilation of systematic activities that can help in the memorising different learning content; amongst others, historical events, and geographical-toponymic data, literary works and authors, language-lexical content, formulations, etc. •For each to work at their own pace: it is a process for the pupils, in an autonomous and individual way and based on their capabilities, to establish for themselves the level and rhythm of their learning and to do the corresponding exercises. •There are three components of the programme: •Hardware: IKASBOOK is the mobile informatics tool that responds to the needs of the school and, moreover, is easy to use and not expensive. This tool is to be integrated into the classroom, each pupil using his or her own PC. •Software: this involves three web applications: one to create and organise activities, another to carry out activities with the mobile informatics tool and, finally, an application designed for the teaching staff to monitor the progress of the work undertaken by students. •Curricular content: this includes a wide range of activities classified according to themes and level of difficulty. 45,000 exercises in each theme have been created for the 6-12 age range. In total, 225,000 exercises divided into five spheres (Basque, Spanish, English, Mathematics and Knowledge of the Environment).The procedure used by the IKASYS programme is based on the following strategies: Didactic contract: taking into account the aptitudes and capabilities of the pupil, a didactic contract will bedrawn up. In this, the teacher and the student agree on what work is to be done (the number of exercises, levelsof difficulty to reach, time given over to study, and so on). •Taking plurality into account: each pupil will work on the previously agreed exercises, on the basis of his or her level. Meantime, the teacher will supervise this work and will offer any help required. •Diversification: together with the aforementioned, the project can help in the monitoring of the pupils with special needs and also with the needs of immigrant pupils. •Evaluation: as pupils do the exercises, this computer tool corrects them simultaneously. Once the work is finished, the pupil, jointly with the teacher, carries out an assessment of the activities and the difficulties that may have arisen. And, on the basis of their evaluation, they decide between them what measures to adopt and what work commitments are to be established therefrom. 2. Evaluation of the IKASYS programmeWhen we talk about evaluation, we are referring to a process of identification, compilation and analysis ofrelevant information. This information may be either quantitative or qualitative and has to be collected andcompiled in a rigorous, planned, objective, reliable and valid manner. The final goal of this evaluation is tomake value judgements on the “educational target” so as to take decisions directed at enhancing the target itself.In this case, when we talk of “educational target” we are referring to the IKASYS programme. With this approach, the evaluation puts forward two fundamental aims: •To obtain empirical evidence on the impact of the applied programme. In concrete, what is to be evaluated is the impact of the programme on the learning of contents and skills related to areas of the Basque, Spanish and English languages, mathematics and knowledge of the environment. Moreover, the impact on the capacity of self-regulation of the pupils in certain school tasks is evaluated. •To evaluate the programme itself and the application process thereof. This objective is directly aimed at enhancing the programme and improving its application. The different assessable aspects are the components of the programme (hardware, software, curricular content) and the procedures related to its application. Although this evaluation undoubtedly has a summative dimension, in the sense that there is an evaluation ofthe results or impact, we can say that basically we are dealing with a “formative evaluation” given that itsfundamental goal is the enhancement and perfecting of the programme being evaluated. 52
  3. 3. Research, Reflections and Innovations in Integrating ICT in Education In order to achieve the targeted objectives, a methodological complementarity has been opted for in order totake full advantage of the different contributions both as regards quantitative methodology as well as qualitative. 3. Impact EvaluationThe impact evaluation is undertaken through “quasi-experimental design with a non-equivalent control group”.In this design the dependent variable is the performance in the 5 different content areas (Basque, Spanish,English, mathematics and knowledge of the environment) and the capacity for self-regulation of the pupils. During the 2008-2009 academic year, the IKASYS system was implemented in 19 schools, two parallelclasses being chosen in each of the following three levels: 2nd, 4th and 6th years. The innovative system wasimplemented in the first (experimental) group of three classes while the other three classes (the control group)followed the usual school curriculum At the end of the process the data on performance in both groups will begathered and compared to analyse the possible impact of the programme on the experimental group. Being aware, both of the existence of multiple variables that might contaminate the explanation of anydifferences in the results in performance in each of the 5 content areas, as well as of the impossibility ofcontrolling the totality of these, an effort was made to find out possible initial differences between theexperimental and the control groups. Assured that there were no significant initial differences in this respect, aquasi-experimental design with a non-equivalent control group was opted for. 4. Evaluation of the programme itselfThe evaluation of the system itself and its process of application is basically qualitative, its aim being to moreprecisely understand all the aspects related to the components of the system (hardware, software, curricularcontent) and to the procedures related to its application, as well as to lay down guidelines for enhancing both thesystem and its application. To this end, information and the opinions of pupils, parents, teachers and schoolheads are being gathered. As we explain later, this qualitative methodology is complemented with the implementation of aquestionnaire amongst all teachers participating in the experiment. The gathering of the information we analyse in this section is structured basically along the following lines: •Observers participating in the classroom In order to record the information directly and at the same time as the activities, behaviour and/or situationsthat are the target of the study, 12 Educational Science PhD or graduate students were prepared as observerswere each assigned a school. They visited the schools once a week and, following an established monitoringprotocol, they observed the implementation of the IKASYS system in the classroom. This process begins withgathering notes in situ and which, once compiled, are subjected to analysis. •Discussion groups This technique – a group of persons talking and debating on a topic of interest, contributing their specific andparticular perspectives and perceptions, was used to get to know the points of view of the school heads. Twodiscussion groups were formed in each of which approximately half of all the school heads participated,depending on their availability and professional timetable. In these discussion groups, following a previously established protocol based on the classroom observations,a comparison could be made of the experiences and an analysis undertaken of the tendencies regarding and theopinions about the programme implemented, of its functioning in the classrooms, the motivation of the teachingstaff, and so on. •Questionnaire for teachers involved in each subject area and at each level As a complement to the qualitative evaluation, a questionnaire was drawn up for the teachers at the“experiment” schools the idea being to analyse the suitability of the process from the perspective of thoseimplementing the system in the classrooms. This questionnaire, to which the teaching staff are currently responding (in the second fortnight of February2009), is structured in 5 parts: 1.- Professional data of the teacher. 2.- Opinions about the process of implementation of the IKASYS system in the school. Information is gathered about the various aspects of this implementation (training, the initiation and development of the implementation, technical problems, adaptation of the pupils, and so on), both from the point of view of the teachers themselves, as well as from their pupils and also their perception of the opinion of parents. 3.- Opinions about the implementation of the process in each of the subjects taught. 53
  4. 4. Research, Reflections and Innovations in Integrating ICT in Education 4.- Opinions about the functioning, development and optimum use of the INSPECTOR system. 5.- Overall assessment and suggestions for general improvements. •Analysis of the records A process of analysis of the recorded results s generated by the system has been undertaken. Firstly, theexercises and activities put forward for carrying out in the classroom, in each of the subjects and at each level,are analysed. Likewise the documentation related to the IKASYS programme is being analysed, and any documentthat might provide valuable information to complement that gathered by other means, is collated and analysed. References[1] Área, M. (2005). La educación en el laberinto tecnológico. De la escritura a las máquinas digitales.[2] Área, M. (2009). Introducción a la tecnología educativa. La Laguna: Universidad de La Laguna.[3] Área,M., Gros,B. & Marzal,M.A. (2008). Alfabetizaciones y TIC. Madrid: Síntesis.[4] Bartolomé, A. (1999). Nuevas tecnologías en el aula. Barcelona: Grao.[5] Cabero, J. (2002). Tecnología educativa. Diseño, producción y evaluación de medios para la enseñanza. Barcelona: Paidós.[6] Federation of Ikastolak (2009). Ikasys proiektua. http://www.ikasys.net[7] Lukas,J.F. & Santiago,K. (2004). Evaluación educativa. Madrid: Alianza.[8] TIC: Nuevas tecnologías y educación. http://www.oei.es/boltic0309.htm[9] Vivancos,J. (2008). Tratamiento de información y competencia digital. Madrid: Alianza.[10] Casamayor, C. (2008). La formación on-line. Barcelona: Grao.[11] Barberá, E. (2008) Como valorar la calidad de la enseñanza basada en las TIC. Barcelona: Grao. 54