Current challenges for educational technology research


Published on

Current challenges for educational technology research

Mayes described educational technology research as being like the film, 'Groundhog Day', with "cycles of high expectation [...] followed by proportionate disappointment", and "a cyclical failure to learn from the past". Fifteen years on, this experience still rings true.

Is this pattern inevitable and inescapable? This paper identified several challenges faced by work in this area. Together, they go some way towards explaining this pattern, and identifying what will need to change if we are to break out of this.

These challenges include the strategic difficulty of maintaining research work across cycles of new technology; the methodological challenge of studying things people have forgotten they are using; the epistemological challenge of reconceptualising the relationship between technology, users and effects; the practical challenge of knowing our learners; and the political challenge of securing funding for anything other than instrumental, applied work.


Seminar at Oxford education department, 17/11/10. Cited papers listed in the speaker's notes.

Published in: Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Mayes, J T (1995) Learning Technology and Groundhog Day. In Strang W, Simpson V B, Slater D (Eds): 'Hypermedia at Work: Practice and Theory in Higher Education', University of Kent Press, Canterbury.
  • Alsop, G., & Tompsett, C. (2007). From Effect to Effectiveness: the Missing Research Questions. Educational Technology &
    Society, 10 (1), 28-39.
    Oliver, M. & Conole, G. (2003) Evidence-based practice and E-Learning in Higher Education: Can We and Should We? Research papers in education, 18 (4), 385-397.
  • Friesen, N. (2008) Re-thinking e-learning research, 6-7.
  • Cf. Latour, B. (1987) Science in Action:  How to Follow Scientists and Engineers Through Society. Harvard: Harvard University Press.
  • Johnson, E. & Berner, B. (Eds, 2010) Technology & Medical Practice: Blood, Guts and Machines. Farnham: Ashgate.
  • Mol, A. (2002) The body multiple: ontology in medical practice. Durham, NC: Duke University Press.
  • Oliver, M. & Dempster, J. (2003) Embedding E-Learning Practices. In Blackwell, R. & Blackmore, P. (Eds), Towards Strategic Staff Development in Higher Education, 142-153 Buckingham: SRHE/OU Press.
  • Price, S. & Oliver, M. (2007) A Framework for Conceptualising the Impact of Technology on Teaching and Learning. Educational Technology & Society, 10 (1), 16-27.
  • Price, S. & Oliver, M. (2007) A Framework for Conceptualising the Impact of Technology on Teaching and Learning. Educational Technology & Society, 10 (1), 16-27.
  • Prensky, M. (2001) Digital Natives, Digital Immigrants, Part II: Do They Really Think Differently? On the Horizon, 9 (6).,%20Digital%20Immigrants%20-%20Part2.pdf
    Bennett, S.; Maton, K.; Kervin, L. (2008). "The ‘digital natives’ debate: A critical review of the evidence". British Journal of Educational Technology 39 (5): 775–786.
  • e.g. Benfield, G., Ramanau, R., & Sharpe, R. (2009) Student learning technology use: preferences for study and contact. Brookes eJournal of learning and teaching.
  • Eynon, R. (2009) Mapping the Digital Divide in Britain: Implications for Learning and Education. Learning, Media and Technology, 34 (4), 277-290.
    Jones, C. Ramanau, R., Cross, S. & Healing, G. (2010) Net generation or Digital Natives: Is there a distinct new generation entering university? Computers & Education, 54 (3), 722-732.
    Czerniewicz, L, Williams, K. & Brown, C. (2009) Students make a plan: understanding student agency in constraining conditions. ALT-J: Research in Learning Technology, 17 (2), 75-88.
  • Selwyn, N. and Grant, L. [ed.] (2009) Special issue of Learning, Media and Technology – issue theme ‘Learning and social software - researching the realities’ , 34, 2
    Luckin, R., Clark, W., Graber, R.,Logan, K., Mee, A., Oliver, M. (2009) 'Do Web 2.0 tools really open the door to learning: practices, perceptions and profiles of 11-16 year old learners ', Learning Media and Technology Volume 34, Issue 2 June 2009 , pages 87 – 104
    Selwyn, N. (2009) ‘Faceworking: exploring students’ education-related use of Facebook’ Learning, Media and Technology 34, 2, pp.157-174
  • Holley, D. & Oliver, M. (2010) Student engagement and blended learning: portraits of risk. Computers & Education 54, 693-700.
  • Friesen, N. (2008) Re-thinking e-learning research, 6-7.
  • Atkins, D.; Seely Brown,J., Hammond, A. (2007) A Review of the Open Educational Resources (OER) Movement: Achievements, Challenges, and New Opportunities. Menlo Park, CA: The William and Flora Hewlett Foundation.
  • Smith, H. & Oliver, M. (2002) University teachers' attitudes to the impact of innovations in ICT on their practice. In Rust, C. (Ed), Proceedings of the 9th International Improving Student Learning Symposium, 237-246. Oxford: Oxford Centre for Staff and Learning Development. ISBN 1-873576-68-4.
  • PriceWaterhouseCoopers (2000) Annex 3: Learning products and services for the e-U.
  • Cuban L. (2001) Oversold and Underused: Computers in the Classroom, Cambridge, MA: Harvard University Press.
    Oliver, M. (2004) Metadata vs. educational culture: roles, power and standardisation. In Land, R & Bayne, S. (Eds) Education in Cyberspace, 112-138. London: RoutledgeFalmer.
  • Millen, J. (1997) Par for the Course: designing course outlines and feminist freedoms. Curriculum Studies, 5 (1), 9-27.
    Oliver, M., Vogel, M., Carr, D (2009) Representing Pedagogy. In iPED Research Network (Eds), Academic Futures: Inquiries into Higher Education and Pedagogy, 144-159. Newcastle upon Tyne: Cambridge Scholars Publishing.
  • Current challenges for educational technology research

    1. 1. Current challenges for educational technology research Martin Oliver
    2. 2. What I’m going to cover… • Technology as revolution • Technology as cause • The myth of learners • Technology as solution • Hope…?
    3. 3. • Technology as revolution
    4. 4. In the film "Groundhog Day", the protagonist is forced to experience the events of a single day over and over again. He is free to act in any way he chooses, but whatever he does the day always finishes in the same way. Part of the fascination of this predicament is the awful familiarity of this experience: so often one feels caught in a flow of events which will unfold in an entirely predictable way.
    5. 5. People who have been involved over any length of time with educational technology will recognise this experience, which seems characterised by a cyclical failure to learn from the past. We are frequently excited by the promise of a revolution in education, through the implementation of technology. We have the technology today, and tomorrow we confidently expect to see the widespread effects of its implementation. Yet, curiously, tomorrow never comes. We can point to several previous cycles of high expectation about an emerging technology, followed by proportionate disappointment, with radio, film, television, teaching machines and artificial intelligence.
    6. 6. Revolutions • Multimedia (CD roms) • Hypertexts • World Wide Web • Virtual Learning Environments • Wikis, blogs • Social networks, virtual worlds, games, twitter…
    7. 7. Revolutions or cycles? • Proof of concept (needed!) • Replication • Replication in disciplinary contexts …increasingly “me too” studies very few, “yes, but” or “no, not really” studies
    8. 8. What do I see getting published? • Lots of (positive) case studies • A fair amount of studies of organisational development (inc. academic/staff development) • Some studies of design/development • Some quasi-experimental studies (quantitative but still case-based) • A few multi-site comparative studies (qualitative and quantitative) • Some national surveys • Some theory, position papers etc
    9. 9. So how should we be looking at this? • Do ‘medical’/“agro-botanical” studies better (Alsop & Tompsett, 2007) 1 Can a new drug be shown to have an effect? Effect 2 Ditto, with an RCT in a selected population that takes drug properly? Efficacy 3 Ditto, for a typical population that behaves normally Effectiveness 4 Any previously unobserved outcomes post release? Side effects
    10. 10. …or…? • But… – Still dose/response assumption – Assumes aggregation is unproblematic • Do something different – Design research – Thesis/antithesis/synthesis theory building – Something else…?
    11. 11. None of it makes any difference anyhow… • • The phrase refers to the innumerable quasi experimental “media comparison” studies […] These studies have shown, in other words, that technology-based courses, when compared to those taught in the classroom, do not result in a statistically significant difference in student performance or educational efficiency. From print-based correspondence to courses taught via radio, television, and the Web, the use of new media in each case was not found to result in a statistically significant improvement in educational efficiency.
    12. 12. • Technology as cause
    13. 13. The problem with dose/response models • “black box” trials – “does it work?” – Attribution is the object of study – Causation is problematic – Explanation is unlikely • What happens if you open the “black box”…?
    14. 14. Why the medical model never worked anyhow …analyses [that] comprise an attach on the philosophical principles behind the evidence-based Medicine movement. [… It] undermines the assumption that results from random controlled trials can be applied across varying context without considering the social and structural specifics of the context in which medicine is practiced. […] ‘technological guidelines’ can be problematic if they are posited to be universal while the practice they are meant to guide is very place and culture specific. (Johnson & Berner, 2010: 76-7)
    15. 15. Perhaps we should be studying practices? • Social studies of science and medicine – It’s not just “stuff” acting on “stuff” unproblematically – Scientific knowledge is socially constructed – But saying it’s all just talk misses out the hands, bodies and so on of medical practice …so how can we bind them back together? • Mol’s ‘Praxiology’ – the study of practices – Talk, action and their ongoing negotiation
    16. 16. Why are there (apparently) no successes? • Activity theory – Activity – Action – Operation • Routine, breakdown and repair • Tacit knowledge • Anything that works, we become unable to talk about
    17. 17. How does something “work”? • Studying teachers (HE) starting to use a virtual learning environment I’m looking for some kind of contribution, any contribution, I look for basically and if I don’t get that then I know there’s probably something wrong. It’s when people are chipping in their bits and then all of a sudden it goes quiet. That’s the danger sign. You do pick up on odd stuff like that – its just transferring what you normally do in normal situations to a virtual environment.
    18. 18. • Activity (Strategy) – no real change …to responsibilities, values, role • Action (Tactics) – some differences …changes to tools used to ‘look’ for participation, changes to pace and time of ‘looking’ • Operation – completely different …scan the room, listen for pauses… …click to generate list of contributors, look for 0’s…
    19. 19. What this reveals is how the move to teaching online renders the role of the teacher both the same and different simultaneously. The purpose and strategic direction may remain unchanged, but the methods of achieving this alter in significant ways. (Price & Oliver, 2007: 24)
    20. 20. • The myth of learners
    21. 21. Who’s practicing what, exactly? • Prensky, Tapscott, Oblinger & Oblinger – We know our learners, and they’re not us (even down to how their brains work) “rather than being empirically and theoretically informed, the debate can be likened to an academic form of a 'moral panic’” (Bennett et al)
    22. 22. We don’t know our learners… …but at least we’re trying. • UK-based studies of learner experiences of e-learning – JISC learner experience projects – – Institutional studies: what are our learners like?
    23. 23. • National/international studies – In the UK, it’s not so simple, although there are trends (Eynon, 2009; Jones et al, 2010) – In South Africa, there’s sacrifice to get there and there’s important pockets of non-engagement (Czerniewicz et al, 2009)
    24. 24. …and what are they doing? • Not what the headlines suggest… – Selwyn’s “state of the actual” – Creativity isn’t the norm (Luckin et al 2009); only a few children do ‘interesting’ things like media production – Not using social networks in a collective wisdom, ‘connectivist’ sense; instead, gossip, banter and ‘identity work’ (Selwyn, 2009) …or not using it at all? Average age of social networker is 35-44
    25. 25. Open and flexible learning? • Well intentioned, but flexibility can cause as well as solve problems (Holley & Oliver, 2010) • Learners struggling with new barrier • Unfamiliar expectations • New cultural norms to engage with
    26. 26. • Charles – ‘ideal’ student – colonisation of home space for study, with notes “as far as you can see”; no mention of family, constraints; purchases resources “even when they’re not essential” • Kwame – non-engaged – struggled, but tried to avoid bothering those “in authority” (merely another resources to Charles); struggled to use shared access machines; knew answers but not how to communicate them; eventually helped when he finds a friend
    27. 27. Those with social advantage find it easier to take advantage of new opportunities; advantage can be perpetuated, not eroded, by introducing new forms of learning and teaching
    28. 28. • Refocusing on practice, not technology: People aren’t using it because it’s technology; they’re using it because they see it as useful in the context of something they’ve chosen to do
    29. 29. • Technology as solution
    30. 30. Back to determinism… • Given all this complexity, why have we spent so much on this? • The appealing myth of technology as a solution • A policy desire? – More technology means more learning – I want to improve education, but it’s tough – But I can buy technology… – So I can buy improvements to education
    31. 31. Not just a caricature… • New labour policy frames education as an economic endeavour (not necessarily cultural or political) • Technology is presented as being able to make systems more efficient • Policy drives… – Purchase (but now saturation) – Some innovation (but the exception not the norm) – Scapegoating of teachers for non-use (it’s not that just buying tech didn’t work, honest…) (Saima Rana)
    32. 32. A massive influence on research too • Funding from JISC, The Higher Education Academy – Proof of concept, implementation projects, quality enhancement (and learner experience) (This explains a lot about the technological emphasis/patterns of publication earlier…) – Even medical model fails: roll-out isn’t a (fundable) research issue
    33. 33. “To use the words of educational technologist Rob Koper […] this research tends not to be “theory-oriented,” but rather “technology-oriented” in character. E-learning research, Koper (2007) explains, is not focused on “predicting or understanding events [in] the world as it exists” (p. 356); it instead seeks to “change the world as it exists” (p. 356; emphasis added). E-learning or technology- oriented research, in other words, attempts “to develop new technological knowledge, methods, and artifacts” for practical ends or purposes (p. 356). It is this applied, practical, and technological research that Koper (2007) says is ideally suited to e-learning.”
    34. 34. • Knowledge-constituative interests – Technical (prediction and control) – Practical (mutual and self understanding) – Emancipatory (New ALT SIG: Politics, activism and critical theory)
    35. 35. To illustrate some of this… • Open Educational Resources OER are teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use or re-purposing by others. Open educational resources include full courses, course materials, modules, textbooks, streaming videos, tests, software, and any other tools, materials, or techniques used to support access to knowledge.
    36. 36. How does this fit with Higher Education? • Echoing moves such as MIT Open Courseware initiative • In the UK, part of a growing tradition of research and policy – Projects, e.g. OU Learning Design Initiative – Policy about how content should be created and shared, and how learning should be organised
    37. 37. Dearing report (1997) • Analysis of technology-related excerpts – Students portrayed as passive, apart from when choosing a course (described in terms of costs and outcomes), then ‘developed’ – Lecturers not talked about as teachers; expectation of giving up teaching to focus on materials development – Technology described in terms of access to information
    38. 38. The new interactive media, offering adaptive feedback and student control have the potential to support independent study, but only if fully developed, tested and maintained. […] Many staff would seek to spend some of their time on development of learning materials, because these will enshrine the core of their teaching. […] IT methods must achieve their promise of greater efficiency both by improving the quality of student learning, and by amortising the cost of development over large student numbers. (Dearing report, Appendix 2)
    39. 39. What constitutes the curriculum here? • Curriculum as syllabus (a thing to be bought) • Curriculum as materials (to be consumed) • Some pedagogy (Resource-based learning) • Gives no real sense of the experience of learning
    40. 40. The e-University As the learner progresses through the courseware, there is the opportunity to ask questions by selecting the associated ‘chat’ channel in the toolbar. In response, a chat window opens and the learner is greeted and invited to describe the assistance sought, in text form. The person who answers the questions is part of a call centre and is specifically trained to answer questions about the courseware. […] If the mentor is unable to answer a question, it is referred to a tutor with superior subject expertise, who returns a full answer to the learner by e-mail within a set period.
    41. 41. Not-so-intelligent tutoring? • Intelligent tutoring systems – Selecting and sequencing of materials for each learner – Built on a (sophisticated) student model – Big in the 70s/80s, largely abandoned in the 90s as not really worth the effort (Est. half as good, at best, as tutor support) • Now, available materials ‘personalised’ using learning styles…
    42. 42. Cook recommendation to Denham committee …a new approach to virtual education based on a corpus of open learning content: the UK must have a core of open access learning resources organised in a coherent way to support on-line and blended learning by all higher education institutions and to make it more widely available in non-HE environments. This needs to be supported by national centres of excellence to provide quality control, essential updating, skills training, and research and development in educational technology, e-pedagogy and educational psychology. All HEIs should be encouraged and helped to exploit virtual education technologies as appropriate to their student’s requirements and their strategies.
    43. 43. Building an effective and competitive on-line learning capacity at both undergraduate and postgraduate level will help meet the changing needs of students and stimulate growth in both higher education and the skills sector. A coherent collection of learning resources can also be exploited to save staff time. Failure to do so will reduce the UK’s ability to exploit e-learning; an aspect of learning and teaching where the UK should aspire to, and gain, a world leading position.
    44. 44. • The UK must have a core of open access learning resources – Why? If they’re open, why not use US ones, etc? • This needs to be supported by national centres of excellence – Not quite the monolithic e-University, but still about centralisation, control, and (arguably) preservation of advantage • research and development in educational technology, e-pedagogy and educational psychology – But not education, ethics, cultural studies… all instrumental • help meet the changing needs of students – Do we really know what these are? • Failure to do so will reduce the UK’s ability to exploit e-learning – It’s all about the money
    45. 45. What have we learnt? • Studies about production and tagging of resources, design of repositories – Relatively little about successful re-use • Better ways to reify practice – Work of formalisation (e.g. LDSE) and pedagogic patterns – Relatively little about how teachers have made use of these • Processes: of excellent value to researchers, but perhaps of less impact for teachers
    46. 46. • Like Cuban’s studies of technology in schools: – Belief in the power of technology – A moral imperative to change – An economic argument – Scapegoating: staff have ‘failed’ to do this • Systematic separation of course production from academic role – A power grab in the name of efficiency – Can be traced back to instructional design and “so called subject matter experts (SMEs)”
    47. 47. What could we learn? • Millen’s studies of course reading lists – The politics of curricula • Small-scale work about the co-ordination of course texts by ‘exemplary’ VLE users – Design as performance – Agility and adaptivity vs. transparency and encoding; practice, spaces and reification • Where are the studies of how teachers work with textbooks?!
    48. 48. What could we do? • Cape Town Open Education Declaration This […] combines the established tradition of sharing good ideas with fellow educators and the collaborative, interactive culture of the Internet. It is built on the belief that everyone should have the freedom to use, customize, improve and redistribute educational resources without constraint. [… P]art of a worldwide effort to make education both more accessible and more effective.
    49. 49. These resources include openly licensed course materials, lesson plans, textbooks, games, software and other materials that support teaching and learning. They contribute to making education more accessible, especially where money for learning materials is scarce. They also nourish the kind of participatory culture of learning, creating, sharing and cooperation that rapidly changing knowledge societies need. However, open education is not limited to just open educational resources. It also draws upon open technologies that facilitate collaborative, flexible learning and the open sharing of teaching practices
    50. 50. Hype, hope and disappointment • Revolutions as repetitive cycles • Difficulty of shifting naïve accounts of the influence of technology • Little (funded) interest in critical, emancipatory work • Where now?
    51. 51. No funding - is there a bright side? • UK cutbacks marginalising technology in education – As a very ‘new labour’ thing, is it realistic to expect it to be supported? • Less research in the area… but will the research that does happen be better as a consequence? – Tapping into established concerns – Working through other disciplines (any discipline is better than none!)
    52. 52. • Consolidation? – Less innovation, more valuing (and understanding) of established practice? – More scope to synthesise research and genuinely address gaps, not just follow fashions? • Less instrumentalism? – If it’s no longer ‘the solution’, perhaps we can think differently about it? – Growing interest in critical engagement – can this be sneaked in…?
    53. 53. Fin.