Chapter 5 - Theory and methodology in learn- ing design research Chapter 5 - Theory and methodology in learning design research..............................1 Introduction.................................................................................................................2 Definitions....................................................................................................................2 Researchers’ home disciplines......................................................................................4 The nature of theory ..................................................................................................5 Theoretical perspectives ............................................................................................6Cultural Historical Activity Theory (CHAT)Communities of PracticeActor Network TheoryCybernetics and systems thinking Methodological approaches........................................................................................7Content analysisEthnographyCase studiesAction researchEvaluationChoosing an appropriate methodology Influences, beliefs and theoretical perspectives........................................................8 Conclusion...................................................................................................................9
IntroductionThis chapter will provide an overview of the theoretical perspectives and associatedmethodologies that und2erpin learning design. It will articulate how learning design isdistinct from, but relates to, related research such as Instructional Design and LearningSciences. Discussing learning technology as a field, Oliver et al. argue that the object ofinvestigation is the knowledge-technology-society nexus. (Oliver, Roberts, et al., 2007).They describe how the study of each of these, knowledge, technology and society, drawson a rich range of research fields across the Social Sciences; including InstructionalDesign, Education, Philosophy and Sociology. Clearly which research fields are drawn onhas implications for both the methodologies used and the theoretical perspectives chosenby the researchers. It will locate learning design within the broader field of e-learning,drawing on the findings of a study which looked at the nature of interdisciplinarity inTechnology-Enhanced Learning (TEL) (Conole, Scanlon, Mundin, & Farrow, 2010) anda Networked Learning conference hotseat on theory and methodology in NetworkedLearning (Conole, 2010).DefinitionsResearch into the use of technology in an educational context had a long history withchanging labels over the years, each indicating evolving trends in the field and emphas-ising different types of foci of inquiry. Commonly used terms include: Educational Tech-nology, Learning Technology, E-learning, Computer Supported Collaborative Learning(CSCL) Networked Learning and more recently Technology-enhanced Learning (TEL).The focus of this chapter is on theories and methodologies used across these fields. Manybooks have been written on research methods in Social Science. Cohen et al. is one of thestandard texts for educational research (Cohen, Manion, & Morrison, 2007). The Re-search Methods Knowledge Base1 provides covers the entire research process including:formulating research questions; sampling; measurement; research design; data analysis;and, writing the research paper. It also addresses the major theoretical and philosophicalunderpinnings of research including: the idea of validity in research; reliability of meas-ures and ethics. The ESRC National Centre for Research Methods2 provides a compre-hensive site for collating research methods activities across the Social Sciences, alongwith the latest in innovations in research methods. Early work carried out by the centreincluded a review of research methods and the generation of a typology of research meth-ods (Beissel-Durrant, 2004), which illustrates the rich variety of research methods beingused reflecting the breadth of different epistemological perspectives in the field.Oliver et al. (2007) argue that there are a range of different epistemological positions ad-opted by researchers in the field and that these have implications for how the field is re-searched. They argue that this is often explained in terms of the ‘paradigm debate’, andframed as a contrast between qualitative and quantitative methods; although they go on toqualify that this is a rather crude distinction; i.e. qualitative data can be interpreted in apositivist way and quantitative data can be used to yield understandings beyond the spe-cific numerical data. They argue that:1 http://www.socialresearchmethods.net/kb/2 http://www.ncrm.ac.uk/
We need to consider how different philosophical positions would interpret the kinds of data generated by particular empirical methods. ‘Methodology’ describes this relationship, and must be understood separately from ‘methods’, which are the techniques used to collect and analyse data (This will include things like inter- views, questionnaires, observation etc.) Methodology determines whether the im- plementation of particular methods is successful or credible. Indeed, according to Agger (2004, p. 77), “methodologies can’t solve intellectual problems but are simply ways of making arguments for what we already know or suspect to be true. To do this, methodology codifies beliefs about the world, reflecting ‘out there’ or ‘in here’ positions. The view that knowledge is hard, objective and tangible will demand of research- ers an observer role, together with an allegiance to methods of natural science; to see knowledge as personal, subjective and unique, however, imposes on research- ers an involvement with their subjects and a rejection of the ways of the natural scientist. To subscribe to the former is to be positivist; to the latter, anti-positivist (Cohen, Manion and Morrison, 2000: 6). Such commitments and interests arise from historical, cultural and political influ- ences, which collectively shape traditions of research that provide the context for current work (Conole, 2003). These have profound implications for the topics that people study and the kinds of conclusions they are willing to draw (Oliver, Roberts, et al., 2007, p. 9).Therefore methods are the techniques used to collect and analyse data, whereas methodo-logy align with different epistemological beliefs and views of the world. The term theoryis contested and is used in a variety of different ways; here are some definitions that arethe closest to how it is used in an e-learning research context: •Theory, in the scientific sense of the word, is an analytic structure designed to ex- plain a set of empirical observations. A scientific theory does two things: i) it identi- fies this set of distinct observations as a class of phenomena, and ii) makes assertions about the underlying reality that brings about or affects this class. In the scientific or empirical tradition, the term theory is reserved for ideas which meet baseline require- ments about the kinds of empirical observations made, the methods of classification used, and the consistency of the theory in its application among members of the class to which it pertains. These requirements vary across different scientific fields of knowledge, but in general theories are expected to be functional and parsimonious: i.e. a theory should be the simplest possible tool that can be used to effectively ad- dress the given class of phenomena.3 •A set of statements or principles devised to explain a group of facts or phenomena, especially one that has been repeatedly tested or is widely accepted and can be used to make predictions about natural phenomena.43 http://en.wikipedia.org/wiki/Theory4 http://www.thefreedictionary.com/theory
The relationship between theory and empirical data can be defined as follows: Social research is theoretical, meaning that much of it is concerned with develop- ing, exploring or testing the theories or ideas that social researchers have about how the world operates. But it is also empirical, meaning that it is based on obser- vations and measurements of reality -- on what we perceive of the world around us. You can even think of most research as a blending of these two terms -- a com- parison of our theories about how the world operates with our observations of its operation.5Researchers’ home disciplinesOne of the 2010 Networked Learning conference hotseats focused on theory and method-ology for networked learning (Conole, 2010). The hotseat was initiated with a positionalpaper and then a forum was moderated for a month to discuss aspects of the paper. In par-allel a series of 18 interviews were conducted with key TEL researchers, as part of theInterdisciplinarity study (Conole, et al., 2010). The reseachers participating in the hotseatand the TEL reearchers interviewed were asked to indicate their home discipline. Theycited a broad range of disciplines, including: Computer Science, Education, Plant Sci-ence, Veterinary Science, Ethnology cultural studies, Psychology, HCI, Philosophy, FineArt, Philosophy, Electronic Engineering, Chemistry, History of Art, Geology, HPS, Lin-guistics, Artificial Intelligence, Philosophy, Sociology, Maths and Physics. Hence e-learn-ing researchers bring with them a rich variety of theoretical perspectives and methodolo-gies. Clearly such diversity brings with it strengths, but it also results in tensions - differ-ences in definitions and understandings and even fundamentally opposed epistemologicalbeliefs:6•Some researchers recognise the underlying influence their ‘home discipline’ has on theirresearch approach. However others argued that their perspectives around e-learning havebeen shaped far more by the experiences they have had working in the area than by priorstudies in an unrelated discipline many years ago.•The transition to an educational perspective for researchers originally from a Sciencebackground is hard, requiring a complete rethinking of underlying epistemological be-liefs. However having an understanding of both Science and Social Science perspectivesis incredibly useful. Similarly transitional processes are evident from those coming intothe research from managerial or business backgrounds.•Many researchers are drawn into research into the use of technologies in an educationalcontext from a practical perspective, i.e. what can these technologies offer? What are theissues? This pragmatic stance is coupled with a desire to understand and describe emer-gent theoretical perspectives.•Irrespective of the theoretical and methodological lenses used to study technologicalphenomena, it is important to take account of the contextual and in particular the humandimension where e-learning takes place.5 http://www.socialresearchmethods.net/kb/naturres.php6 http://cloudworks.ac.uk/cloud/view/2806
The nature of theoryIn the introduction to a special issue of JIME, Oliver provides an overview of the positionof theories in the emergent field of learning technologies in 2002 (Oliver, 2002): I was struck by the diversity of theories that people were drawing upon, and the very different ways in which they were using them. For some, a theory was a touchstone, a guiding set of principles, and the foundation on which their work built. For others, theories were tools, and the important thing was having the right one for the job. What, I wondered, was the right way to use theory here? Should we believe in them, live them, and risk being dogmatic — or should we be plural- istic, tied to none, and risk being superficial?The papers included in the special issue were very varied. Approaches vary considerably— from theory as tool, to theory as principle; from theory building, to theory using; fromdisciplines as diverse as Film Studies, Psychology, Sociology and Education. So too werethe objects of investigation — software tools, logic learning, metadata, multimedia andexploration of issues around the mainstreaming of the use of technologies in education.Masterman and Manton (2009) considered the role of theory with respect to e-learningposing the following questions: what is the value of theory to teachers? What do we meanby theory? And how has theory has been embedded? They drew on Lawes work (Lawes,2004), and in particular the notion that theory gives a framework of understanding thatultimately improves the quality of practice and leads to the transformation of subjectiveexperience. They argued that theory could act as the glue between technology and prac-tice. They then went to make a distinction between theories, models and frameworks: Theories provide a means of understanding and predicting something (Cook 2002). In the original article Cook expands this ‘A theory or model can be used as a means for understanding and predicting some aspect of an educational situation. Theories are not the same as models. A theory can posses an explanatory power and can consist of a set of: ...general assumptions and laws ... that are not themselves intended to be directly (in)validated (for that, the theory must engender a model). Theo- ries are foundational elements of paradigms, along with shared problems and methods. Model are abstract representations that helps us understand something we can’t see or experience directly (Conole, Oliver et al., 2007)), models include things like Kolb’s leaning cycle, A framework is a structure and vocabulary that supports the explication of con- cepts and issues (Conole & Oliver, 2002) such as Laurillard’s Conversational Framework (Laurillard, 2002).They argue that theory is a cornerstone of professional practice and an antidote to techno-logical determinism. However, teachers generally do not consciously espouse formal the-ories and are driven much more by prior experience and reflective practice.
Theoretical perspectivesThis section articulates some of the main theoretical perspectives that are evident in e-learning research. Reviewing the research literature, the following range of theoreticalperspectives are evident: Social Constructivism, Actor Network Theory, Constructivism,Critical Theory, Action Research, Communities of Practice, Scientific Enquiry, Conversa-tional Framework, Philosophy of Technology, Anthropological views on tools artefactsand technology, and Activity Theory. A sample of these is discussed below.Cultural Historical Activity Theory (CHAT)Despite the range of theories listed above, arguably socio-cultural perspectives are a pre-dominate discourse in the field. In particular, Cultural Historical Activity Theory (CHAT)(Cole, Engeström, & Vasquez, 1997; Daniels, Cole, & Wertsch, 2007; Engeström,Punamäki-Gitai, & Miettinen, 1999). A key idea in CHAT is the notion of mediation byartefacts (Kutti, 1996; Wertsch, 1991), which are broadly defined as including instru-ments, signs, language, and machines (Nardi, 1995). As discussed elsewhere in this book,Conole (2008) describes the range of mediating artefacts that practitioners can use to sup-port the learning design process (Conole, 2008). Engeström’s so-called ‘triangle’ repre-sentation (Engeström, et al., 1999) has been used extensively to described particular in-stances of e-learning interventions, as it helps consider a focus on subject-object with as-sociated outcomes, supported through mediating tools in the context of a wider communi-ty context and associated rules and divisions of labour (Joyes, 2008; Karasavvidis, 2008;Waycott, Jones, & Scanlon, 2005).Communities of PracticeWenger’s notion of Communities of Practice (Wenger, 1998) has been picked up and usedextensively in the field of e-learning (Breuleux, Laferrière, & Bracewell, 1998; Cousin &Deepwell, 2005; Guldberg & Pilkington, 2006). Its appeal is probably a combination ofthe fact that it is relatively easy concept to grasp and that it offered a means of explainingsome of the more socially situated interactions arising in networked learning.Actor Network TheoryDeveloped by Callon (1999) Callon and Latour (2005), Actor Network Theory considersboth people and technologies as ‘actants’ in a connected network, emphasising that it isthe relationship between these actants that is important. Although called a theory it does-n’t explain a phenomenon but focuses more on why a network takes the form that it does.It is much more interested in exploring how actor-networks get formed, hold themselvestogether, or fall apart.Cybernetics and systems thinkingCybernetics and systems thinking provide a means of understanding complex systems(Capra, 1996; Gharajedaghi, 1999) and has been applied to a limited extend in an e-learn-ing context. Liber (2004) for example draws on the work of Illich and Beer as a means ofdescribing in modern learning environments and systems (Beer, 1959; Illich, 1973). Re-lated work swhich also apply systems thinking include the work of Friesen, Stankov et al.and Cantoni et al. (Cantoni, Cellario, & Porta, 2004; Friesen, 2004; Stankov, Grubišić, &Žitko, 2004).
Methodological approachesThis section describes some of the key methodological approaches used in e-learning.The choice of methodology tends to reflect both the individual’s epistemological stanceand their focus of inquiry. Oliver et al. (2007) argue that the kinds of data that are avail-able to e-learning researchers may suggest particular kinds of interpretation. This hints atthe suggestion there is a complex inter-relationship between research in the field and theaffordances of the technologies themselves.It is not possible to provide a comprehensive review of all the different methodologicalapproaches used in e-learning. Methodologies are predominantly interpretive in nature;although experimental approaches are still used extensively in North America. In terms ofmethods a range are evident – interviews, focus groups, observation, surveys, studentjournals, video and audio diaries, document analysis, and web tracking. In-depth casestudies are popular, as are large-scale surveys. The use of learning analytics and webtracking as a means of data collection is still in its infancy but is a growing area of re-search; indeed the first international conference on learning analytics was held this yearin Banff.7Content analysisEarly research in the field was dominated by analysis of asynchronous discussion forums(Mason and Kay, 1989). Coding schemes such as those developed by Henri and Gun-awadena et al. were used extensively (Gunawardena, Lowe, & Anderson, 1997; Henri,1992). Henri identified following five dimensions, which can be used to evaluate Com-puter-Mediated Conferences: participative, social, interactive, cognitive and metacognit-ive. Garrison et al. (2000) developed a community of learning model, which assumesthat learning occurs through the interaction of three core components: cognitive presence,teaching presence, and social presence (Arbaugh, et al., 2008; D. R. Garrison, Anderson,T., and Archer, W. , 2000). Gunawadena et al. divided content into the types of cognitiveactivities the participant engaged with (questioning, clarifying, negotiating, synthesising,etc), the types of arguments they put forward, the resources used and any evidence ofchanges in understanding (Gunawardena, et al., 1997). In this early work arguably therewas a naïve assumption that focusing on the content in the treaded messages was enoughto capture the whole event. Whereas in reality the level of detail/object of focus will nat-urally have a significant impact on results and it was soon realised that taking account ofthe broader context within which discussion forums were taken place was important.Jones for example reports students simulating collaboration online whilst actually beingco-present seated around four computers (Jones, 1999). A number of approaches havebeen used to take account of the broader perspective. For example, De Laat et al. use amulti-method approach using Social Network Analysis with content analysis and criticalevent recall (De Laat, Lally, Lipponen, & Simons, 2007; M. F. De Laat, 2006; M. F. DeLaat, Lally, V., Lipponen, L., and Simons, P.R.J., 2006). In this work, they used SocialNetwork Analysis (Hawthornthwaite, 2002) to visualise the social structures and dynam-ics of the course, content analysis is used to identify the learning and teaching processesand critical event recall is used to elicit teachers’ experiences and perceptions.7 https://tekri.athabascau.ca/analytics/
EthnographyEthnography has been used extensively in e-learning (Hodgson & Watland, 2004; Kruger,2006; Rice-Lively, 1994). The approach is qualitative based on ‘systematic descriptionof human behaviour and organisational culture based on first-hand observation’ (Howard,2002).Case studiesRich, situated case studies are a very popular and common form of studying e-learning. Acase study is an in-depth investigation/study of a single individual, group, incident, orcommunity (Yin, 2009). The nature and scope of the cases can vary significantly and theapproach often overlaps with other methodological approaches (such as action research,evaluation and ethnography). Critics of the case study approach argue that the findingsare not generalisable or transferable. Proponents argue that the case-based approach en-ables the researcher to gather a rich, contextual understanding of a situation in context.Action researchAs might be expected, given the educational nature of e-learning as a research field, ac-tion research is often used as a methodological approach, particularly by practitionerswho are trialling out the use of technologies in their classroom and want a frameworkwithin which to study the interventions (Derntl & Motschnig-Pitrik, 2004; D. R.Garrison, 2003; Sloman, 2001).EvaluationThe importance of evaluation has grown in recent years; as new learning technologiesemerge there is a need to evaluate how these are used to support an increasingly diversestudent population. The relationship between evaluation and research more generally re-mains contested. Both processes may use the same methods and study the same things.However, one way to distinguish them is to consider how findings are used. If they areinterpreted by an immediate, local audience and used to support decision-making, thestudy was probably an evaluation; if findings are interpreted in terms of theories and arepresented as a contribution to knowledge, it was probably research. Oliver et al. contendthat approaches in evaluation range from positivist approaches focussed upon objectivedata collection (typically using quantitative methods) to interpretivist ones more rooted inconstructivism (typically using qualitative methodologies) (Oliver, Harvey, Conole, &Jones, 2007).Choosing an appropriate methodologySo which methodology should be used when, and are some methodologies better thanothers? Oliver et al. considers how five different methodological approaches (action re-search, behaviourist, activity theory-based, and a perspective based on power) are used totackle the same research problem (Oliver, Roberts, et al., 2007). This provides a nice il-lustration of how different theoretical perspectives would explain this situation differ-ently, and how each can contribute to our understanding of this field.Influences, beliefs and theoretical perspec-tivesAnalysis of the discussion on the Networked Learning hotseat and the interviews with the18 TEL researchers, indicates that researchers in the area area drawing on a broad group
of influential thinkers. However, it is also evident that there does appear to be a commonshared discourse underpinning the field. Socio-cultural approaches – in particular thework of Vygotsy (1978), Engeström (1999) and others around Activity Theory seem tobe particularly influential. Laurillard’s ‘Rethinking university teaching and learning’ hasbeen extensively cited (Laurillard, 2002) and act as somewhat of a watershed in the fieldas it was published at a key time.Listing others mentioned gives some indication of the theoretical perspectives these re-searchers are drawing on: Alan Collins; work on design-based research (Collins, 1992),Michael Patton’s work on utilisation-focussed evaluation (Patton, 2008), Barbara Rogoffand Lave’s work on Cultural Psychology (Lave & Wenger, 1991; Rogoff, 2003), MaggieBoden’s work on Artificial Intelligence (Boden, 1989), Lave and Wenger’s work on Com-munities of Practice (Lave & Wenger, 1991, 1998), Alan Blackwell at al.’s work on inter-disciplinarity (Blackwell, Wilson, Street, Boulton, & Knell, 2009), Howard Gardner’swork on multiple intelligences (Gardener, 1993), James Wertsch’s work on mediatingartefacts (Wertsch, 1991) and Michael Cole’s work on Activity Theory (Cole, et al.,1997).Looking at some of the specific texts that were cited as influences is also insightful.These included ‘Educating the Reflective Practitioner’ (Schön, 1987), ‘Academic Tribesand ‘Territories: Intellectual Enquiry and the Cultures of Discipline’ (Becher & Trowler,2001), ‘Distributed Cognition’ (Salomon, 1993), ‘Rethinking university teaching’(Laurillard, 2002), ‘Plans and situated actions: the problem of human-machine commu-nication’ (Suchman, 1987), ‘A dynamic medium for creative thought’ (Kay, 1972)(Kay,1972), ‘‘Doing Research/Reading Research Re-interrogating Education’ (Dowling &Brown, 2010) and ‘Common and Border Lands’ (Strathern, 2004).These individuals and texts give a flavour of what is shaping the field and the broader lit-erature that is being drawn on. It demonstrates that the field is indeed interdisciplinary,because these texts are drawn from a broader set of disciplines than research that can bepurely labelled TEL research. However, there is an additional important aspect to thenature of interdisciplinarity in TEL research, both in terms of the actual processes in-volved and how individuals react with and benefit from the other researchers.ConclusionThis chapter has considered the nature of theory and methodology in e-learning research.It has described the range of feeder disciplines and associated approaches, as well as ar-ticulating some of the key research perspectives that e-learning researchers draw on. Con-ole et al. (2010) conclude that e-learning is by nature an interdisciplinary field and sum-marise with the follow observations.Firstly, as a relatively new field, TEL research has attracted people from different discip-lines, each bringing with them different theoretical and methodological perspectives. Seealso Conole and Oliver (2007: 1-15).Secondly, TEL research by its nature is complex, and is concerned with improving educa-tion through the use of technology. It therefore needs to draw both on subject areas con-cerned with learning and teaching (Education, Psychology, etc.) and those concerned with
technology (Computer Sciences, Information Sciences etc.), as well as understanding thelocal nuances and cultural differences across different subject domains. Bringing thesedifferent aspects together effectively is a key challenge of TEL research and therefore itneeds the different interdisciplinary perspectives to understand it; i.e. interdisciplinarity isa core facet of TEL research. If TEL research is going to work, it has to be interdisciplin-ary and people need to bring a wide range of different skills, perspectives and researchtools to bear upon a particular problem.Third, there are huge and interesting cognitive, technical and social questions surround-ing the delivery of technology-enhanced learning. For example, how should the cognit-ive and the social be integrated? How should knowledge be organised? How shouldlearning interventions be orchestrated and managed? These are highly complex questionsand need more technical resources than other areas of educational research. Also theproducts or artefacts produced then need an interdisciplinary approach to evaluation.Fourth, a number of strategies need to be in place to support TEL research practices. Re-searchers need to be helped to develop the skills needed to undertake interdisciplinary re-search. Institutions need to have in place appropriate career paths to foster and promoteinterdisciplinarity. This has not always been the case and some TEL researchers havefound that they had reached a ceiling in their institution in terms of promotion, having toeither revert to more traditional roles or move into managerial positions. Some felt thatoften the value of TEL research groups in terms of institutional support remains to befully exploited and, that interdisciplinary research groups could be playing a more proact-ive role within institutions, helping them make strategic decisions on the effective use oftechnologies to support learning and teaching. It seems that TEL research groups oftenfind themselves outside of formal institutional decision-making mechanisms.Fifth, some tensions are evident between the disciplines. TEL research has to meet theresearch agenda of the disciplines involved, and, in particular, the needs of both Com-puter Scientists and Educationalists, arguably two of the core disciplines underpinning e-learning.