What Can We Hope To Achieve with eLearning Research?


Published on

This is a review of different approaches to eLearning research. The presentation makes the point that we need to conduct research that makes a difference in the classroom and it favors action research in this respect.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

What Can We Hope To Achieve with eLearning Research?

  1. 1. 1 Dr. Iain Doherty Director, eLearning Pedagogical Support Unit Centre for the Enhancement of Teaching and Learning The University of Hong Kong December 6, 2013 The Educational Gold Standard What Can We Hope To Achieve with eLearning Research?
  3. 3. 3 WHY QUESTION? Good question . . . HKU is developing three Massive Open Online Courses (MOOCS), one of which is being offered by the Li Ka Shing Faculty of Medicine. MOOCS are throwing up a lot of questions and whilst past research offers some answers there is clearly a need for MOOC specific research. I sit in lots of meetings discussing research and I ask myself . . . What kind of research should we be conducting?
  4. 4. 4 GOLD STANDARD The idea of a gold standard brings to mind notions of clarity of hypotheses, adequacy and rigor of research methods, robustness of conclusions etc. These are not the issues that are addressed in this presentation . . . At least not directly. The issue for this presentation really concerns the goals of eLearning / educational research. But we should note that quality of research has been an issue in eLearning . . .
  6. 6. 6 Reeves (2000) talks about the generally shoddy nature of eLearning research and cites substantial evidence for poor methodology and lack of impact, for example: “After reviewing nearly 500 papers related to hypermedia and learning, Dillon and Gabbard (1998) identified 118 studies that appeared to meet their criteria for quantitative studies examining the effectiveness of hypermedia in education . . . only 30 studies published between 1990 and 1996 met the minimal criteria of scientific merit for inclusion in the literature review”. Hypermedia and Learning 500 papers Effectiveness of Hypermedia 118 studies Minimal Criteria - Scientific Merit 30 studies (Reeves, 2000) Review showed little impact from use of hypermedia
  7. 7. 7 Even Worse… Reeves says that eLearning / educational research tends to occupy the bottom left quadrant:. Research inspired by considerations of use No Yes Basic Research Design Research Yikes Action Research Yes Research inspired by quest for fundamental understanding No So, we need to remain cognizant of the quality of eLearning research. Assuming quality – for now at least – we can take a straightforward perspective on the gold standard for educational research.
  8. 8. 8 The Easy Answer There is a straightforward answer to the question ANSWER ONE The gold standard is research that improves the teaching and learning experience ANSWER TWO This has to be true not just in a local context but in multiple contexts The research has to provide “theoretical” insights as a basis for further research????? ANSWER THREE The improvements also have to be sustained over time and they have to be scalable
  9. 9. 9 The Easy Answer made Difficult Theory – “An idea or set of ideas that is intended to explain facts or events” – “the general principles or ideas that relate to a particular subject” Keep in mind the question whether educational research has to contribute to educational theory.
  10. 10. 10 Direction For Reeves we need to attend both to what we are doing and to why we are doing it . . . (Reeves, 2000)  Assuming clarity of goals, appropriate methodology, rigorous application of method, clarity of findings, clear conclusions  We need research that is immersed in the classroom / comes out of the classroom Teaching is messy (Brydon-Miller et al, 2003) and if we want to research it we need to get our hands dirty.
  12. 12. 12 Action Research “In general, the demand from academic staff is for help with design – for customisable, re-usable ideas, not fixed, pre-packaged solutions. However, demand is also expressed in ways which emphasise academics’ strong sense of being time-poor. There is no visible demand for complex methodologies, approaches which require substantial revision of existing work practices, or methods which require mastery of complex skills or specialised language”. (Goodyear, 2005, p.2) “Action research can change current classroom practice. Certainly it is better than merely presenting the research results of professionals, as teachers must become personally involved in changing their classroom behavior, and few teachers become personally involved in an issue of the Review of Educational Research”. (Hodgkinson, 1957)
  13. 13. 13 Action Research is in the classroom and involves systematic observation and data collection (quantitative and / or qualitative) that can be used by the practitioner-researcher to improve what is going on in the classroom. 1 Identify a problem 2 Create a solution 3 Implement 4 Evaluate 5 Revise accordingly
  14. 14. 14 University of Wollongong Template Designs Action Research can be shared quickly and easily through e.g. a database of action research designs. (Laurillard, 2008) Improvements are sustainable in the local context by the individual teacher and scalable through offering the template designs to other teachers. Learning Activity Management System (LAMS)
  15. 15. 15 Some people claim that action research is not really research but just a form of evaluation. However, it can be considered to be research if results are shared so that others may draw their own inferences. (Reeves, 2000) Unlike “science” this is not controlled experimentation that takes variables into account ACTION RESEARCH (Hodgkinson, 1957) Unlike “science” work stops at the practical solution with no attempt to generalize (Hodgkinson, 1957) It sometimes theorizes against knowledge base
  16. 16. 16 EPSU Action Research EPSU has engaged in some action research with teachers: FLIPPED CLASSROOM Faculty of Medicine ONLINE CASES Department of Pathology LANGUAGE LEARNING GRADEMARK School of Chinese Faculty of Social Sciences, Centre for Applied English Studies
  17. 17. 17 Remember the Question Mark Improve the teaching and learning experience Can be applied in multiple contexts 2009) (Chandler & Torbet 2003) However, educational action research is not generally concerned with drawing substantially on theory. Action Research It can provide theoretical insights as a basis for further research (Friedman & Rogers, There are many varieties of action research and some do aim for theoretical insights. Can be sustained over time and is scalable It is useful but not the right approach for researching the complexity of MOOCS.
  19. 19. 19 Researchers with . . . [design research] goals are focused on the dual objectives of developing creative approaches to solving human teaching, learning, and performance problems while at the same time constructing a body of design principles that can guide future development efforts” . (Reeves, 2000) Design Research – like action research – is carried out in the classroom and this is important in terms of testing designs with respect to what is really happening.
  20. 20. 20 Empirical Versus Design Research – Design research works off of what Sandoval refers to as design conjectures (2004) = hypotheses. – Based on familiarity with a body of knowledge + the actual teaching situation researchers, teachers, students etc. come together to create a learning design that is expected to behave in certain ways in the classroom. – The “conjectures” are tested in the real world to determine if what was conjectured is in fact the case.
  21. 21. 21 Example of Design Research – Conjecture that computer mediated situated learning environment will lead to knowledge that students [pre-service teachers in this case] can apply. (Herrington & Oliver, 2000; Herrington & Oliver, 1999) – Students develop mathematics assessment strategy. – Herrington and Oliver identify 9 elements to a situated learning environment and they ground these elements in the relevant research literature.
  22. 22. 22 Example of Design Research – So the first point is that design research utilizes a lot of theory. Very time intensive to put a research based design into practice. – In this case the design research approach also required the creation of a complex learning environment rich with resources. – Take a look at the situated learning element again, this time with the resources mapped against the element.
  23. 23. 23 Run the design experiment and capture data from observing students interacting with the multimedia environment, from reports produced by the students and from questionnaires. Carry out data analysis and report findings.
  24. 24. 24 Design Research The clear benefit here is that principles are being developed and tested out in the environment that matters Research is complex and intensive with a lot of data gathering The actual situation is incredibly complex and messy and multiple variables will impact on the learning design However, there is a positive here because this is “real world” and unexpected insights occur!
  25. 25. 25 Design Research can Improve the teaching and learning experience Can be applied in multiple contexts [but not generalizable to multiple contexts] Provides theoretical insights as a basis for further research Can be sustained over time and is scalable? Requires a huge commitment of time / resources and tends not to be scalable because others do not have the inclination / time to put results into practice / further the research.
  26. 26. 26 CONCLUSIONS
  27. 27. 27 OK, so the gold standard depends on perspective but there is one key issue – the research has to make a difference in practice. – Action research suits teachers + instructional designers + management to some degree. – Design research suits teaching intensive teachers + instructional designers + management to some degree. – Basic / theoretical research suits, well, researchers + instructional designers who draw on it + management who have to report research outputs.
  28. 28. 28 Q&A
  29. 29. 29 References • Brown, A. L. (1992). Design Experiments: Theoretical and Methodological Challenges in Creating Complex Interventions in Classroom Settings. The Journal of the Learning Sciences, 2(2), 141–148. Retrieved from http://www.jstor.org/stable/1466837 • Brydon-Miller, M., Greenwood, D., & Maguire, P. (2003). Why Action Research? Action Research, 1(1), 9–28. doi:10.1177/14767503030011002 • Chandler, D., & Torbert, B. (2003). Transforming Inquiry and Action: Interweaving 27 Flavors of Action Research. Action Research, 1(2), 133–152. doi:10.1177/14767503030012002
  30. 30. 30 References • Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design Research: Theoretical and Methodological Issues. The Journal of Learning Sciences, 13(1), 15–42. Retrieved from http://www.jstor.org/stable/1466931 . • Friedman, V. J., & Rogers, T. (2009). There is Nothing so Theoretical as Good Action Research. Action Research, 7(1), 31–47. doi:10.1177/1476750308099596 • Goodyear, P. (2005). Educational Design and Networked Learning: Patterns, Pattern Languages and Design Practice. Australasian Journal of Educational Technology. Retrieved from http://www.ascilite.org.au/ajet/ajet21/goodyear.html
  31. 31. 31 References • Herrington, J., & Oliver, R. (1999). Using Situated Learning and Multimedia to Investigate HigherOrder Thinking. Journal of Educational Multimedia and Hypermedia, 8(4), 401–422. Retrieved from http://www.aace.org/pubs/jemh/v8n4.htm • Herrington, J., & Oliver, R. (2000). An Instructional Design Framework for Authentic Learning Environments. Educational Technology Research and Development, 48(3), 23–48. doi:10.1007/BF02319856 • Hodgkinson, H. L. (1957). Action Resarch - A Critique. Journal of Educational Sociology, 31(4), 137–153. Retrieved from http://www.jstor.org/stable/2264741
  32. 32. 32 References • Laurillard, D. (2008). The Teacher as Action Researcher: Using Technology to Capture Pedagogic Form. Studies in Higher Education, 33(2), 139–154. Retrieved from http://dx.doi.org/10.1080/03075070801915908 • Oh, E., & Reeves, T. (2008). Design Research vs. Instructional Systems Design: Implications for Educational Technologists. In World Conference on Educational Multimedia, Hypermedia and Telecommunications 2008 (pp. 2119–2127). Vienna, Austria: AACE. Retrieved from http://www.editlib.org/index.cfm?fuseaction=Reader. ViewAbstract&paper_id=28664
  33. 33. 33 References • Reeves, T. C. (2000). Enhancing the Worth of Instructional Technology Research through “Design Experiments” and Other Development Research Strategies. In Annual Meeting of the American Educational Research Association (pp. 1–15). New Orleans, LA, USA. Retrieved from http://it.coe.uga.edu/~treeves/ • Sandoval, W. A. (2004). Developing Learning Theory by Refining Conjectures Embodied in Educational Designs. Educational Psychologist, 39(4), 213–223. doi:10.1207/s15326985ep3904_3