Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Current challenges for educational
technology research
Martin Oliver
What I’m going to cover…
• Technology as revolution
• Technology as cause
• The myth of learners
• Technology as solution
...
• Technology as revolution
In the film "Groundhog Day", the protagonist is
forced to experience the events of a single day
over and over again. He is...
People who have been involved over any length of time
with educational technology will recognise this
experience, which se...
Revolutions
• Multimedia (CD roms)
• Hypertexts
• World Wide Web
• Virtual Learning Environments
• Wikis, blogs
• Social n...
Revolutions or cycles?
• Proof of concept (needed!)
• Replication
• Replication in disciplinary contexts
…increasingly “me...
What do I see getting published?
• Lots of (positive) case studies
• A fair amount of studies of organisational
developmen...
So how should we be looking at this?
• Do ‘medical’/“agro-botanical” studies better
(Alsop & Tompsett, 2007)
1 Can a new d...
…or…?
• But…
– Still dose/response assumption
– Assumes aggregation is unproblematic
• Do something different
– Design res...
None of it makes any difference anyhow…
• http://www.nosignificantdifference.org/
• The phrase refers to the innumerable q...
• Technology as cause
The problem with dose/response models
• “black box” trials – “does it work?”
– Attribution is the object of study
– Causat...
Why the medical model never worked anyhow
…analyses [that] comprise an attach on the
philosophical principles behind the e...
Perhaps we should be studying practices?
• Social studies of science and medicine
– It’s not just “stuff” acting on “stuff...
Why are there (apparently) no successes?
• Activity theory
– Activity
– Action
– Operation
• Routine, breakdown and repair...
How does something “work”?
• Studying teachers (HE) starting to use a
virtual learning environment
I’m looking for some ki...
• Activity (Strategy) – no real change
…to responsibilities, values, role
• Action (Tactics) – some differences
…changes t...
What this reveals is how the move to
teaching online renders the role of the
teacher both the same and different
simultane...
• The myth of learners
Who’s practicing what, exactly?
• Prensky, Tapscott, Oblinger & Oblinger
– We know our learners, and they’re not us
(even ...
We don’t know our learners…
…but at least we’re trying.
• UK-based studies of learner experiences
of e-learning
– JISC lea...
• National/international studies
– In the UK, it’s not so simple, although there are trends
(Eynon, 2009; Jones et al, 201...
…and what are they doing?
• Not what the headlines suggest…
– Selwyn’s “state of the actual”
– Creativity isn’t the norm (...
Open and flexible learning?
• Well intentioned, but flexibility can cause
as well as solve problems
(Holley & Oliver, 2010...
• Charles – ‘ideal’ student – colonisation of home
space for study, with notes “as far as you can
see”; no mention of fami...
Those with social advantage find it easier to
take advantage of new opportunities;
advantage can be perpetuated, not erode...
• Refocusing on practice, not technology:
People aren’t using it because it’s
technology; they’re using it because they
se...
• Technology as solution
Back to determinism…
• Given all this complexity, why have we
spent so much on this?
• The appealing myth of technology as...
Not just a caricature…
• New labour policy frames education as an
economic endeavour
(not necessarily cultural or politica...
A massive influence on research too
• Funding from JISC, The Higher Education
Academy
– Proof of concept, implementation p...
“To use the words of educational technologist Rob Koper
[…] this research tends not to be “theory-oriented,” but
rather “t...
• Knowledge-constituative interests
– Technical (prediction and control)
– Practical (mutual and self understanding)
– Ema...
To illustrate some of this…
• Open Educational Resources
OER are teaching, learning, and research resources
that reside in...
How does this fit with Higher Education?
• Echoing moves such as MIT Open
Courseware initiative
• In the UK, part of a gro...
Dearing report (1997)
• Analysis of technology-related excerpts
– Students portrayed as passive, apart from
when choosing ...
The new interactive media, offering adaptive feedback
and student control have the potential to support
independent study,...
What constitutes the curriculum here?
• Curriculum as syllabus (a thing to be
bought)
• Curriculum as materials (to be con...
The e-University
As the learner progresses through the courseware, there
is the opportunity to ask questions by selecting ...
Not-so-intelligent tutoring?
• Intelligent tutoring systems
– Selecting and sequencing of materials for
each learner
– Bui...
Cook recommendation to Denham committee
…a new approach to virtual education based on a
corpus of open learning content: t...
Building an effective and competitive on-line learning
capacity at both undergraduate and postgraduate level
will help mee...
• The UK must have a core of open access learning resources
– Why? If they’re open, why not use US ones, etc?
• This needs...
What have we learnt?
• Studies about production and tagging of
resources, design of repositories
– Relatively little about...
• Like Cuban’s studies of technology in schools:
– Belief in the power of technology
– A moral imperative to change
– An e...
What could we learn?
• Millen’s studies of course reading lists
– The politics of curricula
• Small-scale work about the c...
What could we do?
• Cape Town Open Education Declaration
This […] combines the established tradition of sharing
good ideas...
These resources include openly licensed course
materials, lesson plans, textbooks, games, software and
other materials tha...
Hype, hope and disappointment
• Revolutions as repetitive cycles
• Difficulty of shifting naïve accounts of the
influence ...
No funding - is there a bright side?
• UK cutbacks marginalising technology in
education
– As a very ‘new labour’ thing, i...
• Consolidation?
– Less innovation, more valuing (and understanding) of
established practice?
– More scope to synthesise r...
Fin.
Upcoming SlideShare
Loading in …5
×

Current challenges for educational technology research

2,855 views

Published on

Current challenges for educational technology research

Mayes described educational technology research as being like the film, 'Groundhog Day', with "cycles of high expectation [...] followed by proportionate disappointment", and "a cyclical failure to learn from the past". Fifteen years on, this experience still rings true.

Is this pattern inevitable and inescapable? This paper identified several challenges faced by work in this area. Together, they go some way towards explaining this pattern, and identifying what will need to change if we are to break out of this.

These challenges include the strategic difficulty of maintaining research work across cycles of new technology; the methodological challenge of studying things people have forgotten they are using; the epistemological challenge of reconceptualising the relationship between technology, users and effects; the practical challenge of knowing our learners; and the political challenge of securing funding for anything other than instrumental, applied work.

----

Seminar at Oxford education department, 17/11/10. Cited papers listed in the speaker's notes.

Published in: Education
  • Be the first to comment

Current challenges for educational technology research

  1. 1. Current challenges for educational technology research Martin Oliver
  2. 2. What I’m going to cover… • Technology as revolution • Technology as cause • The myth of learners • Technology as solution • Hope…?
  3. 3. • Technology as revolution
  4. 4. In the film "Groundhog Day", the protagonist is forced to experience the events of a single day over and over again. He is free to act in any way he chooses, but whatever he does the day always finishes in the same way. Part of the fascination of this predicament is the awful familiarity of this experience: so often one feels caught in a flow of events which will unfold in an entirely predictable way.
  5. 5. People who have been involved over any length of time with educational technology will recognise this experience, which seems characterised by a cyclical failure to learn from the past. We are frequently excited by the promise of a revolution in education, through the implementation of technology. We have the technology today, and tomorrow we confidently expect to see the widespread effects of its implementation. Yet, curiously, tomorrow never comes. We can point to several previous cycles of high expectation about an emerging technology, followed by proportionate disappointment, with radio, film, television, teaching machines and artificial intelligence.
  6. 6. Revolutions • Multimedia (CD roms) • Hypertexts • World Wide Web • Virtual Learning Environments • Wikis, blogs • Social networks, virtual worlds, games, twitter…
  7. 7. Revolutions or cycles? • Proof of concept (needed!) • Replication • Replication in disciplinary contexts …increasingly “me too” studies very few, “yes, but” or “no, not really” studies
  8. 8. What do I see getting published? • Lots of (positive) case studies • A fair amount of studies of organisational development (inc. academic/staff development) • Some studies of design/development • Some quasi-experimental studies (quantitative but still case-based) • A few multi-site comparative studies (qualitative and quantitative) • Some national surveys • Some theory, position papers etc
  9. 9. So how should we be looking at this? • Do ‘medical’/“agro-botanical” studies better (Alsop & Tompsett, 2007) 1 Can a new drug be shown to have an effect? Effect 2 Ditto, with an RCT in a selected population that takes drug properly? Efficacy 3 Ditto, for a typical population that behaves normally Effectiveness 4 Any previously unobserved outcomes post release? Side effects
  10. 10. …or…? • But… – Still dose/response assumption – Assumes aggregation is unproblematic • Do something different – Design research – Thesis/antithesis/synthesis theory building – Something else…?
  11. 11. None of it makes any difference anyhow… • http://www.nosignificantdifference.org/ • The phrase refers to the innumerable quasi experimental “media comparison” studies […] These studies have shown, in other words, that technology-based courses, when compared to those taught in the classroom, do not result in a statistically significant difference in student performance or educational efficiency. From print-based correspondence to courses taught via radio, television, and the Web, the use of new media in each case was not found to result in a statistically significant improvement in educational efficiency.
  12. 12. • Technology as cause
  13. 13. The problem with dose/response models • “black box” trials – “does it work?” – Attribution is the object of study – Causation is problematic – Explanation is unlikely • What happens if you open the “black box”…?
  14. 14. Why the medical model never worked anyhow …analyses [that] comprise an attach on the philosophical principles behind the evidence-based Medicine movement. [… It] undermines the assumption that results from random controlled trials can be applied across varying context without considering the social and structural specifics of the context in which medicine is practiced. […] ‘technological guidelines’ can be problematic if they are posited to be universal while the practice they are meant to guide is very place and culture specific. (Johnson & Berner, 2010: 76-7)
  15. 15. Perhaps we should be studying practices? • Social studies of science and medicine – It’s not just “stuff” acting on “stuff” unproblematically – Scientific knowledge is socially constructed – But saying it’s all just talk misses out the hands, bodies and so on of medical practice …so how can we bind them back together? • Mol’s ‘Praxiology’ – the study of practices – Talk, action and their ongoing negotiation
  16. 16. Why are there (apparently) no successes? • Activity theory – Activity – Action – Operation • Routine, breakdown and repair • Tacit knowledge • Anything that works, we become unable to talk about
  17. 17. How does something “work”? • Studying teachers (HE) starting to use a virtual learning environment I’m looking for some kind of contribution, any contribution, I look for basically and if I don’t get that then I know there’s probably something wrong. It’s when people are chipping in their bits and then all of a sudden it goes quiet. That’s the danger sign. You do pick up on odd stuff like that – its just transferring what you normally do in normal situations to a virtual environment.
  18. 18. • Activity (Strategy) – no real change …to responsibilities, values, role • Action (Tactics) – some differences …changes to tools used to ‘look’ for participation, changes to pace and time of ‘looking’ • Operation – completely different …scan the room, listen for pauses… …click to generate list of contributors, look for 0’s…
  19. 19. What this reveals is how the move to teaching online renders the role of the teacher both the same and different simultaneously. The purpose and strategic direction may remain unchanged, but the methods of achieving this alter in significant ways. (Price & Oliver, 2007: 24)
  20. 20. • The myth of learners
  21. 21. Who’s practicing what, exactly? • Prensky, Tapscott, Oblinger & Oblinger – We know our learners, and they’re not us (even down to how their brains work) “rather than being empirically and theoretically informed, the debate can be likened to an academic form of a 'moral panic’” (Bennett et al)
  22. 22. We don’t know our learners… …but at least we’re trying. • UK-based studies of learner experiences of e-learning – JISC learner experience projects – ELESIG.ning.com – Institutional studies: what are our learners like?
  23. 23. • National/international studies – In the UK, it’s not so simple, although there are trends (Eynon, 2009; Jones et al, 2010) – In South Africa, there’s sacrifice to get there and there’s important pockets of non-engagement (Czerniewicz et al, 2009)
  24. 24. …and what are they doing? • Not what the headlines suggest… – Selwyn’s “state of the actual” – Creativity isn’t the norm (Luckin et al 2009); only a few children do ‘interesting’ things like media production – Not using social networks in a collective wisdom, ‘connectivist’ sense; instead, gossip, banter and ‘identity work’ (Selwyn, 2009) …or not using it at all? Average age of social networker is 35-44
  25. 25. Open and flexible learning? • Well intentioned, but flexibility can cause as well as solve problems (Holley & Oliver, 2010) • Learners struggling with new barrier • Unfamiliar expectations • New cultural norms to engage with
  26. 26. • Charles – ‘ideal’ student – colonisation of home space for study, with notes “as far as you can see”; no mention of family, constraints; purchases resources “even when they’re not essential” • Kwame – non-engaged – struggled, but tried to avoid bothering those “in authority” (merely another resources to Charles); struggled to use shared access machines; knew answers but not how to communicate them; eventually helped when he finds a friend
  27. 27. Those with social advantage find it easier to take advantage of new opportunities; advantage can be perpetuated, not eroded, by introducing new forms of learning and teaching
  28. 28. • Refocusing on practice, not technology: People aren’t using it because it’s technology; they’re using it because they see it as useful in the context of something they’ve chosen to do
  29. 29. • Technology as solution
  30. 30. Back to determinism… • Given all this complexity, why have we spent so much on this? • The appealing myth of technology as a solution • A policy desire? – More technology means more learning – I want to improve education, but it’s tough – But I can buy technology… – So I can buy improvements to education
  31. 31. Not just a caricature… • New labour policy frames education as an economic endeavour (not necessarily cultural or political) • Technology is presented as being able to make systems more efficient • Policy drives… – Purchase (but now saturation) – Some innovation (but the exception not the norm) – Scapegoating of teachers for non-use (it’s not that just buying tech didn’t work, honest…) (Saima Rana)
  32. 32. A massive influence on research too • Funding from JISC, The Higher Education Academy – Proof of concept, implementation projects, quality enhancement (and learner experience) (This explains a lot about the technological emphasis/patterns of publication earlier…) – Even medical model fails: roll-out isn’t a (fundable) research issue
  33. 33. “To use the words of educational technologist Rob Koper […] this research tends not to be “theory-oriented,” but rather “technology-oriented” in character. E-learning research, Koper (2007) explains, is not focused on “predicting or understanding events [in] the world as it exists” (p. 356); it instead seeks to “change the world as it exists” (p. 356; emphasis added). E-learning or technology- oriented research, in other words, attempts “to develop new technological knowledge, methods, and artifacts” for practical ends or purposes (p. 356). It is this applied, practical, and technological research that Koper (2007) says is ideally suited to e-learning.”
  34. 34. • Knowledge-constituative interests – Technical (prediction and control) – Practical (mutual and self understanding) – Emancipatory (New ALT SIG: Politics, activism and critical theory)
  35. 35. To illustrate some of this… • Open Educational Resources OER are teaching, learning, and research resources that reside in the public domain or have been released under an intellectual property license that permits their free use or re-purposing by others. Open educational resources include full courses, course materials, modules, textbooks, streaming videos, tests, software, and any other tools, materials, or techniques used to support access to knowledge.
  36. 36. How does this fit with Higher Education? • Echoing moves such as MIT Open Courseware initiative • In the UK, part of a growing tradition of research and policy – Projects, e.g. OU Learning Design Initiative – Policy about how content should be created and shared, and how learning should be organised
  37. 37. Dearing report (1997) • Analysis of technology-related excerpts – Students portrayed as passive, apart from when choosing a course (described in terms of costs and outcomes), then ‘developed’ – Lecturers not talked about as teachers; expectation of giving up teaching to focus on materials development – Technology described in terms of access to information
  38. 38. The new interactive media, offering adaptive feedback and student control have the potential to support independent study, but only if fully developed, tested and maintained. […] Many staff would seek to spend some of their time on development of learning materials, because these will enshrine the core of their teaching. […] IT methods must achieve their promise of greater efficiency both by improving the quality of student learning, and by amortising the cost of development over large student numbers. (Dearing report, Appendix 2)
  39. 39. What constitutes the curriculum here? • Curriculum as syllabus (a thing to be bought) • Curriculum as materials (to be consumed) • Some pedagogy (Resource-based learning) • Gives no real sense of the experience of learning
  40. 40. The e-University As the learner progresses through the courseware, there is the opportunity to ask questions by selecting the associated ‘chat’ channel in the toolbar. In response, a chat window opens and the learner is greeted and invited to describe the assistance sought, in text form. The person who answers the questions is part of a call centre and is specifically trained to answer questions about the courseware. […] If the mentor is unable to answer a question, it is referred to a tutor with superior subject expertise, who returns a full answer to the learner by e-mail within a set period.
  41. 41. Not-so-intelligent tutoring? • Intelligent tutoring systems – Selecting and sequencing of materials for each learner – Built on a (sophisticated) student model – Big in the 70s/80s, largely abandoned in the 90s as not really worth the effort (Est. half as good, at best, as tutor support) • Now, available materials ‘personalised’ using learning styles…
  42. 42. Cook recommendation to Denham committee …a new approach to virtual education based on a corpus of open learning content: the UK must have a core of open access learning resources organised in a coherent way to support on-line and blended learning by all higher education institutions and to make it more widely available in non-HE environments. This needs to be supported by national centres of excellence to provide quality control, essential updating, skills training, and research and development in educational technology, e-pedagogy and educational psychology. All HEIs should be encouraged and helped to exploit virtual education technologies as appropriate to their student’s requirements and their strategies.
  43. 43. Building an effective and competitive on-line learning capacity at both undergraduate and postgraduate level will help meet the changing needs of students and stimulate growth in both higher education and the skills sector. A coherent collection of learning resources can also be exploited to save staff time. Failure to do so will reduce the UK’s ability to exploit e-learning; an aspect of learning and teaching where the UK should aspire to, and gain, a world leading position.
  44. 44. • The UK must have a core of open access learning resources – Why? If they’re open, why not use US ones, etc? • This needs to be supported by national centres of excellence – Not quite the monolithic e-University, but still about centralisation, control, and (arguably) preservation of advantage • research and development in educational technology, e-pedagogy and educational psychology – But not education, ethics, cultural studies… all instrumental • help meet the changing needs of students – Do we really know what these are? • Failure to do so will reduce the UK’s ability to exploit e-learning – It’s all about the money
  45. 45. What have we learnt? • Studies about production and tagging of resources, design of repositories – Relatively little about successful re-use • Better ways to reify practice – Work of formalisation (e.g. LDSE) and pedagogic patterns – Relatively little about how teachers have made use of these • Processes: of excellent value to researchers, but perhaps of less impact for teachers
  46. 46. • Like Cuban’s studies of technology in schools: – Belief in the power of technology – A moral imperative to change – An economic argument – Scapegoating: staff have ‘failed’ to do this • Systematic separation of course production from academic role – A power grab in the name of efficiency – Can be traced back to instructional design and “so called subject matter experts (SMEs)”
  47. 47. What could we learn? • Millen’s studies of course reading lists – The politics of curricula • Small-scale work about the co-ordination of course texts by ‘exemplary’ VLE users – Design as performance – Agility and adaptivity vs. transparency and encoding; practice, spaces and reification • Where are the studies of how teachers work with textbooks?!
  48. 48. What could we do? • Cape Town Open Education Declaration This […] combines the established tradition of sharing good ideas with fellow educators and the collaborative, interactive culture of the Internet. It is built on the belief that everyone should have the freedom to use, customize, improve and redistribute educational resources without constraint. [… P]art of a worldwide effort to make education both more accessible and more effective.
  49. 49. These resources include openly licensed course materials, lesson plans, textbooks, games, software and other materials that support teaching and learning. They contribute to making education more accessible, especially where money for learning materials is scarce. They also nourish the kind of participatory culture of learning, creating, sharing and cooperation that rapidly changing knowledge societies need. However, open education is not limited to just open educational resources. It also draws upon open technologies that facilitate collaborative, flexible learning and the open sharing of teaching practices
  50. 50. Hype, hope and disappointment • Revolutions as repetitive cycles • Difficulty of shifting naïve accounts of the influence of technology • Little (funded) interest in critical, emancipatory work • Where now?
  51. 51. No funding - is there a bright side? • UK cutbacks marginalising technology in education – As a very ‘new labour’ thing, is it realistic to expect it to be supported? • Less research in the area… but will the research that does happen be better as a consequence? – Tapping into established concerns – Working through other disciplines (any discipline is better than none!)
  52. 52. • Consolidation? – Less innovation, more valuing (and understanding) of established practice? – More scope to synthesise research and genuinely address gaps, not just follow fashions? • Less instrumentalism? – If it’s no longer ‘the solution’, perhaps we can think differently about it? – Growing interest in critical engagement – can this be sneaked in…?
  53. 53. Fin.

×