ICT research in Africa: Towards appropriate research methodologies Duan van der Westhuizen Faculty of Education University of Johannesburg
“There is a growing concern internationally that theinvestment in ICT in schools is not impacting on literacydevelopment. This concern arises from a belief held bymany - including governments as well as schools - that ICTis beneficial to learning, and specifically literacylearning.”.Torgerson C, Zhu D (2003)
"It is generally believed that ICTs can empower teachersand learners, promote change and foster the developmentof 21st century skills, but data to support these beliefsare still limited".infoDev.org
The positive impact of ICT use in education has not beenproven In general, and despite thousands of impactstudies, the impact of ICT use on student achievementremains difficult to measure and open to much reasonabledebate.
“There is neither a strong and well-developed theoreticalcase nor much empirical evidence supporting theexpected benefits of ICT …”Contrasting evidence: BECTA (2002) and Machin etal(2006) found positive effects, while Fuchs & Woessman(2004), Leuven et al (2004) and others found no realpositive effect
Threat 1: Publish or perish• The research endeavour is a primary imperative for universities• Universities under state pressure to up research output• Universities rewarded for publication by funding model• Accredited journal articles preferred and rewarded• Cascades down – University leadership – Faculty leadership – Departmental leadership – Individual faculty members• What drives the push for increased output?
Solani Ngobeni: Fostering academic mediocrity• About 96% of research output in South Africa is journal articles• Tied to the funding model for universities• Financially lucrative for universities to publish in accredited journals• This turns universities into managerialised corporate entities• Lecturers become revenue-generating employees• Publishing becomes a Key Performance Area at all the levels• Therefore, publishing is not about scholarship anymore, but about generating revenue• It is not about quality, but about quantity• The system of journal publication is not without problems• And nor is the system of ‘rated’ journals (ISI, IBBS, DoHE)
Threat 2: Quality of Educational Research• Hargreaves (1996): Educational research is poor value for money, remote from practice, and of poor quality• Considerable amount of second rate research that is irrelevant to practice and clutters up journals that nobody reads• A study by OFSTED explored several (41) educational publications. – Partisanship in the conduct of research, the presentation of research and the argument of research – There are methodological issues – Non-empirical research did not consult primary resources – Focus was often obfuscated• Researchers “employ weak research methods, write turgid prose, and issue contradictory findings” (Reeves, 2002)• Dillon and Gabbard (1998): “statistical analyses & research methods are frequently flawed, limiting our understanding of these important issues” and limiting generalisation of findings.
Pasteur’s quadrant (Stokes) Research inspired by use No Yes Yes Research by Research by Bohr for an Pasteur for atomic model pasteurisationResearch inspired byquest for fundamentalunderstanding (theory) Discoveries and research done No by Edison
Basic Research and Applied Research• In Science—The Endless Frontier (Bush, 1990) it is argued that basic research should be performed “without the thought of practical ends”• There is a linear progression from basic research to application, to social innovation• Stokes (1997) claims that this thinking is fundamentally flawed• The motive for fundamental understanding and the drive for application are not separate or in opposition to each other• Research can be undertaken both as a quest for basic understanding (rigor) and with considerations of use (relevance).
Researching Education: The hardest science of all?• Educational Science is incredibly hard to do• Hard and soft science: a flawed dichotomy?• ‘Hard’ science is based on randomization, generalization, based on ‘empiric’ data, often in laboratory conditions• ‘Soft’ science is squishy, unreliable and imprecise• Berliner (2002) claims ‘hard’ science is easy science, and educational science is the hardest science of all • Deal with contexts that limit generalization and theory building • Challenge to replicate findings • Challenge to find the numbers for statistical analysis • Does not seek to find the regularity that nature provides
The hardest Science (cont)• Humans learners in schools are embedded in complex and changing networks of social interaction• The context can never be controlled• The variety of ‘variables’ (e.g. ability, status, motivation, teacher competence, conceptions of learning, beliefs about assessment) are continuously interacting, and the direction or reciprocity of interactions are difficult to determine• Educational science also suffers from the ‘decade by findings interactions’ phenomenon – Change in social, cultural and intellectual environments negate previous findings
Threat 3: Quality of Educational ICT Research• ICT research lacks a sense of history• Longitudinal studies lag behind ICT developments• The focus on ICT (as an object) research, not on learning impact• Over-reliance on Qualitative methods• Quantitative work is generally on ‘attitude’ and ‘usage’• Simple descriptive analyses• Few comparative analyses (e.g. that of Lim, 2003, and Pelgrum, 2001)• Small-scale studies that fail to produce findings that are reliable and generalisable• Much of the research is dispersed at sub-discipline level
Threat 4: Lack of large-scale meta- analyses• Most prior meta-analyses were conducted on Distance Education• Kulik & Kulik pre-1980• Most recently by US DoE: Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies• Project RED: THE TECHNOLOGY FACTOR
Threat 5: Insufficient local research• Van der Westhuizen (2002), there is little research addressing and locating specifically local concerns• Annotated Bibliography on e-Learning and Application of Educational Technology in African Countries, or in Contexts Relevant to Africa (Carnegie)• Deficits in Academic Staff Capacity at African Universities (Partnership for Higher Education in Africa, 2010)• Growth in student numbers not met by growth in staff• Lack of female staff• Low numbers of PhD and M students• Staff qualifications• SA: CHE Report on Higher Education Landscape
Case Study M & D ICT Research in SA• Finding congruence between the purpose of the inquiry, the theoretical framework in which the study was conducted, the research design and the interpretation of findings.• Not simply mapping the nature of theoretical frameworks in the inventory, but to interpret and critique the way in which the authors used theoretical framing as a scholarly device.
Method• Seven leading universities ito publication output• Sample studies were identified through library keyword searches• Samples were either taken off the shelves or obtained digitally• Research instrument was applied to 103 studies• Instrument captured summaries of – Research question – Supporting theories – Research paradigms – Findings (conclusions)
Discussion: towards theoretical emancipation (Best cases)• a small insignificant portion of the selected studies had theoretical frameworks as epistemological guides• these studies were conceptualised coherently and the various aspects of the inquiries were congruent• studies that were theoretically developed yielded data that could be interpreted in more depth
Discussion: towards theoretical emancipation (2)• A substantial majority of authors employed theoretical frameworks in a very limited way• theoretical frameworks in literature reviews without using them as epistemological guides• findings that were descriptive in nature, contrary to a claimed interpretivist stance.
So ……• Educational ICT is a developing research field and the disciplinary training is not yet cohesive• There is evidence that the researchers remained practitioners of Educational ICT and did not manage to become scholars• It raises the concern that supervisors in M & D programmes are not equipping their students with the necessary skills to “find an epistemological home” for their studies and to analyse and interpret results.
Possible causes• The demands of transformation and standardisation have been many• Academic and social “fall out” of mergers• Certainly, the state of supervision in the country is generally not sufficient, as is evident in the drive of an organisation such as SANTRUST (ex SANPAD)• supervisors are increasingly expected to have a high and accelerated output and ‘through put’• The demand to increase a university’s income is seductive for an academic and linked to promotion
How?• Appropriatestudies Abandon case Designs?• Abandon “user experience” ‘research’• •Course evaluation is not research Experimental/quasi-experimental research• (considering the ‘random assignment’ and have Research projects must be theoretically founded, theoretical outcomes• requirement) Research-competency training• •Select appropriate research designs Large-scale studies• •Encourage research teams Longitudinal studies – Supervisor 1 (Senior) – PhD student •Repeated-measures studies 1 - PhD Student 2 - Supervisor 2 (Junior) Master Student 1, 2, 3 •Design-based Research – Work on (dimensions of) focussed, authentic problems – (which addresses all of the above) International collaborative research