• Save


Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

Like this presentation? Why not share!

Like this? Share it with your network


Exploring the map of distance and e-learning research

Uploaded on

Keynote presented at the 6th EDEN Research Workshop in Budapest, 25 October 2010

Keynote presented at the 6th EDEN Research Workshop in Budapest, 25 October 2010

More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide
  • Since this is the EDEN Research Workshop, I would like to explore with you and describe the field of DE research – the map or geography of distance and e-learning research – by means of a bibliographic analysis.
  • My presentation has three parts: In the first part we will explore the international network of DE journals and the relations between them. Then we will look INTO the journals: I will present a classification systems of research areas in DE. And finally we will look at publication and authorship patters in a literature review to identify gaps and priority areas. The presentation is based on a series of four studies that are already published. I did these studies together with colleages from the FernUniversity in Hagen, which is Germany's Open University, among them Christine von Prümmer. And with Terry Anderson from the Athabasca University, which is Canada's Open University.
  • OK, now I start to analyse the international network of DE journals by covering the following questions….
  • Journals are important carriers of knowledge within a research discipline. Research builds upon previous research and previous research is acknowledged through citations. So the journals and the relations between them in terms of citations can be seen as a network of information exchange that reveals the "intellectual structure" of a discipline. To investigate the network, we can use Social Network Analysis (SNA): The nodes are the journals and the ties in the network are citations.
  • We selected 12 journals for our study…. Between 2003 and 2008 a total number of 1,416 full papers were published in these journals. The highest number of papers were published in TOJDE (255), the lowest number in AJDE (74) and DE (68) – they seem to be much more selective.
  • This is the journal citation network in a 12x12 matrix. In the rows you see the citing journals and in the columns are the cited journals. They generated a total number of 6,292 citations which yields a mean citation rate accross all journals for the 144 entries in this table of 44. This we term the overall tie density. The journal with the highest number of citations is AJDE (898), followed by DE (437), JDE (385), and OL (335). OJDLA seems to constitute a separate cluster with a distinct community in distance education administration and management. This community refers to itself, 62 % (240 of 386) are self-citations. For the further analysis and a comparison between journals we have to standardize the citation values. The absolute level of citation depends on the size of the journal (larger journals with more issues and papers per volume probably receive more citations than smaller journals). So the structure of the journal network must be viewed without any confounding effects due to size. To control for the different journal sizes, the giving and receiving citations are standardized by dividing them by the number of papers in each journal. This yields an overall tie density of 0.46 cites per paper.
  • With SNA tools you can produce these buitiful network graphs… The nodes in this network diagram are the journals and the ties between them are incoming and outgoing citations. The biggest node here is AJDE. This is the most central journal in terms of incoming citations. A prominent, central journal in the network is a journal that is extensively involved in relationships with other journals in terms of citations. The prestige or status of a journal in the network increases as the journal becomes the object of more ties. So the simple idea is that prestigious journals receive more citations.
  • This is a simplified image of the journal network showing only those ties that are above the mean citation rate or tie density. Dashed lines indicate ties close the tie density of 0.46 cites per paper. The stars in the network are the most established journals: AJDE, JDE, DE and OL. They receive the highest number of citations above the average. Then we have journals in the periphery: These journals are much younger (mean publication of 12 years) and interestingly 4 of the 5 are open access journals. They are active members of the network in terms of giving citations . So they refer to the core, top journals, however their citations are not reciprocated. This finding is supported by Doreian and Fararo who found that l eading authors who publish in top journals pay little or no attention to work contained in less prestigious journals and they do not tend to publish in such journals. The Asian JDE, EURODL and IJDTE are isolates, at least on the international level. IJDET has a strong focus on computer science and the papers are very technically oriented, suggesting a focus on a different network outside of the other distance education journals. Note that there is also NO link between the two European journals EURODL and OL.
  • What about the perceived quality of the journals by experts in the field. We sent a journal prestige survey with a rating scale according to Nelson, Buss & Katzko to all 277 Editorial Board Members of the 12 journals. 83 of them (= 30%) responded.
  • Based on the Editor's ratings re: how they evaluate the quality and prestige of the journals we see that we have a top group of 5 journals with… Then a group in the middle with… And a group at the bottom The highest rated journal is IRRODLE, however I don't believe that IRRODL is that much different from the other 4 top journals, because editors are biased towards their 'own' journal as you can see on the next slide…
  • We tested with a series of t-tests if the ratings of editors and non-editors are significantly different from each other. All editors rate the quality of their own journal higher than non editors. And for example for IRRODL this difference is statically significant: 4.52 compared to 4.02.
  • We now described the overall structure of the distance education journal network – but now let's look INTO the top journals in the field research trends and to identify gaps and priority areas…
  • The first thing we need to do that, is a validated classification system of research areas in distance education. So I carried out an international Delphi study. 25 individual were on the expert panel. They came from 11 different countries, reporting an average of 27 years of professional experience in distance education. In the first round the experts were asked to list 10 important research areas.
  • Three major lines of research or broad meta-levels of distance education research were derived from the expert's responses in the Delphi study… Which is the level of the individual learner
  • Within these three levels, the research issues that are considered important by the experts were categorised into the following 15 research areas …
  • Based on this classification system we can now look at publication and authorship patters in a literature review.
  • Based on this classification system we can now look at publication and authorship patters in a literature review.
  • We reviewed all full papers that were published in the 5 top DE journals… We selected the stars of the journal network plus the highest ranked journal in the journal prestige survey (IRRODL) In the time period between 2000 and 2008, 695 articles were published in the five journals.
  • Looking at the frequencies of research articles on each topic, reveals a strong imbalance of research areas covered in the publications: The micro-perspective ( Teaching and learning in distance education ) is highly over-represented. As you can see here in this nicely edited chart… This is not very surprising since I would guess that the field of DE is of course dominated by educators who are concernced with research into individual learning processes.
  • This table shows a ranking of research areas by number of articles. Here in the column with the cummulative percentages you see that over 50 % of articles deal with topics on the micro-level, whereas issues that were considered as neglected research areas are at the bottom of the list. This supports the expert‘s opinion in the Delphi study.
  • This diagram shows the percentages within gender in relationship to research areas. For this diagram only 302 single author papers were taken into consideration…. There is a highly significant association between research areas and gender. I am sorry that I have to confirm common stereotypes now… Male researchers often deal with topics stereotypically associated with them: management and technology. Whereas female researchers are more concerned with the social aspects of learning and teaching, interaction and communication in learning communities, learner support services, and learner characteristics. The only field of educational management where women contributed more articles than men is learner support services and professional development and faculty support, often perceived as a 'softer' discipline of management.
  • This table shows the frequency of applied research methods by journal. Maybe there is a trend towards more empirical research. In 2001 Berge & Mrozowski classified 76 % of 727 articles published in AJDE, DE, JDE and OL between 1990 and 1999 as purely descriptive. Whereas in our analysis for the time period between 2000 and 2008, only 38 % of papers were descriptive in nature. The table furthermore reveals that AJDE seems to prefer quantitative studies. 63,4% followed a quantitative design. And the journal which accepted by far the highest percentage of qualitative studies is DE (29,5%).
  • OK, it is also very interesting to look at who are the leading contributors in DE research and where do they come from? 1,148 different authors contributed to the 695 articles in this study, and 48 authors publish three or more papers. The vast majority of them comes from Canada. Canada seems to be the center of international distance education research. Congratulation! Where are the Canadians? Canada is followed by…
  • The top author is Allan Jeong, a professor at the Florida State University. His research is focused on diversity in online interaction patterns using quantitative sequential analysis to explore different message-response sequences and group interaction patterns in the online learning environment. Now check out the list. Here are the leading authors in our field. Show up, how many of them are here today? I know Dianne Conrad, number 2 on the list, is here. Where are you Dianne? Congratulations!
  • For the analysis of country-wise distribution of articles the origin of the first author was taken into consideration. They came from 54 different countries. The research literature is dominated by only 5 countries. The vast majority (over 80 %) comes from USA, Canada, UK, Australia, and China. The UK is the only European country among the top 10 countries! The crosstabulation of countries and journals furthermore reveals that journals tend to publish more from their country of origin… Especially AJDE has a strong North-American focus – or maybe we can say a North-American bias – over 80 % of articles published in AJDE are contributed by US-American or Canadian authors. However, another Canadian Journal, IRRODL, is the most international Journal with authors coming from 34 different countries and only 19% of authors coming Canada.
  • Research questions should be posed within a theoretical framework and embedded in a holistic structure of research areas within a discipline. I hope my work is a good contribution towards such a map of DE research which helps to organize the body of knowledge in our field. Current research is clearly dominated by studies on the micro level (teaching and learning in DE), whereas major gaps were found on the macro and meso level, especially on the institutional level of educational management. I just came across a very interesting project, the DEHub project in Australia. A research consortium was established in 2009 between 5 Australian universities: University of New England (UNE), Charles Sturt University (CSU), Central Queensland University (CQUniversity), the University of Southern Queensland (USQ), and Massey University. They received a massive grand from the Australien Government (I heard about 20 Million Dollars) to establish a distance education research institute and to develop a research agenda.
  • As you can see here, for their research agenda they used the classification system developed in our Delphi Study. The research levels and areas are almost excactly the same, they just added the theme of OER on the meso level.
  • The European DE research community is weak compared to North American institutions, at least in terms of the quantitative research outputs in leading DE journals. The UK is the only European country among the top 10 countries. I think, we need a European research agenda for the next decade with a clear cut research profile to contribute to the consolidation of the research-based professional field of DE to inform practice, Governments and other stakeholders who might provide the project funds. The priority areas in this research agenda could be based on – among other resources – on the Delphi study and the literature review I presented here – this might be a good starting point. Maybe a European Association such as EDEN could be the facilitator of this process. European DE journals should be strengthened, especially EURODL. EURODL is an isolated journal in terms of citations. EURODL should be better integrated into the international journal network by publishing research papers with higher impact obligated to academic rigour. The goal has to be that EURODL is listed in major citation databases such as SCOPUS or the Web of Science.


  • 1.   Exploring the map of distance and e-learning research – a bibliographic analysis EDEN Research Workshop, Budapest, 25 October 2010 Olaf Zawacki-Richter University of Oldenburg, Germany Center for Lifelong Learning Faculty of Educational and Social Sciences
  • 2. Overview
    • Distance Education Journals
    • Validated structure of distance education research areas
    • Review of research between 2000 and 2008: gaps and priority areas
    • Perspectives for future research
    Zawacki-Richter, O. (2009). Research areas in distance education – a Delphi study. International Review of Research in Open and Distance Learning, 10 (3), 1-17. Zawacki-Richter, O., Bäcker, E. M., & Vogt, S. (2009). Review of distance education research (2000 to 2008) – analysis of issues, methods and authorship patterns. International Review of Research in Open and Distance Learning, 10 (6), 21-50. Zawacki-Richter, O., & von Prümmer, C. (2010). Gender and collaboration patterns in distance education research. Open Learning, 25 (2), 95-114. Zawacki-Richter, O., Anderson, T., & Tuncay, N. (2010). The growing impact of open access distance education journals - a bibliometric analysis. Journal of Distance Education, 24 (3).
  • 3. Part I: Distance Education Journals
    • What are the major journals in the field and how can they be grouped in terms of impact and perceived quality?
    • How do the journals relate to each other in a network of giving and receiving citations?
    • What are the most central and prestigious journals in the network and which journals are peripheral or isolated?
  • 4. Structure of Distance Education Journal Network
    • Journals as important carriers of knowledge in a discipline
    • Research builds upon and refers to previous research
    • Previous research is acknowledged by citations
    • Social Network Analysis (SNA): The nodes are journals and the ties in the network are citations.
    • The journal network reveals relationships and patterns of scientific information exchange, the "intellectual structure" of a discipline (Liu, 2007).
    Liu, Z. (2007). Scholarly communication in educational psychology: a journal citation analysis. Collection Building . doi: 10.1108/01604950710831915
  • 5. Selected Journals
  • 6. Journal Citation Network
    • Mean citation rate across all journals: 44 ("tie density")
    • Standardized mean citation rate: 0.46 cites/paper
  • 7.  
  • 8. Journal Citation Network – Image Matrix Doreian, P., & Fararo, T. J. (1985). Structural equivalence in a journal network. Journal of the American Society for Information Science , 36 (1), 28-37.  
  • 9. Perceived quality
    • Journal Prestige Survey : Editorial Board Members (83 of 277).
    • Rating scale according to Nelson, Buss & Katzko (1983): Please assign numbers to the journals listed below according to how you would rate them, where 5 = outstanding, 4 = excellent, 3 = good, 2 = adequate, 1 = poor. You may leave blanks if you do not recognize a journal.
    Nelson, T. M., Buss, A. R., & Katzko, M. (1983). Rating of scholarly journals by chairpersons in the social sciences. Research in Higher Education , 19 (4), 469-497.  
  • 10. Perceived quality and prestige
  • 11. Perceived quality and prestige
  • 12. Part II: Research Areas in Distance Education
    • What are the most important research areas in DE and how can they be categorized?
    • What are the most neglected research areas?
  • 13. Research Areas: Delphi Study
    • Expert panel: 25 individuals from 11 countries (average of 27 years of professional experience in distance education)
    • 1. round: Experts were asked to list 10 important research areas
    • 2. round: Experts were asked to rate research areas on a scale of importance; What are the most neglected research areas?
  • 14. Classification of Research Areas
    • Three broad research perspectives or levels with 15 research areas:
      • Macro level: Distance education systems and theories -> global system level: 5 research areas
      • Meso level: Management, organisation and technology -> institutional level: 7 research areas
      • Micro level: Teaching and learning in distance education -> individual level: 3 research areas
  • 15. Classification of Research Areas
    • Meso level: Management, organisation and technology
    • Management and organisation
    • Costs and benefits
    • Educational technology
    • Innovation and change
    • Professional development and faculty support
    • Learner support services
    • Quality assurance
    • Macro level: Distance education systems and theories
    • Access, equity and ethics
    • Globalization of education and cross-cultural aspects
    • Distance teaching systems and institutions
    • Theories and models
    • Research methods in distance education and knowledge transfer
    • Micro level: Teaching and learning in DE
    • Instructional design
    • Interaction and communication in learning communities
    • Learner characteristics
  • 16. Neglected Research Areas
    • Macro level: cross-cultural aspects of transnational education
    • Meso level: innovation and change management, educational management and organisation, costs and funding of DE
    • Micro level: added value of Web 2.0 applications (instructional design)
  • 17. Part III: Publication and Authorship Patterns
    • What are the topics with the highest and lowest number of papers?
    • What kind of research methods are applied and do DE journals prefer to publish qualitative or quantitative studies?
    • Do female and male researchers tend to choose different research topics? If yes, what are their priority areas?
    • Who are the leading authors and where do they come from?
  • 18. Literature Review
    • Review of articles that were published in 5 prominent DE journals
      • 9 years: 2000 to 2008
      • N=695 (675 English, 20 French)
    • Journals:
      • Open Learning (UK)
      • Distance Education (Australia)
      • American Journal of Distance Education (USA)
      • Journal of Distance Education (Canada)
      • IRRODL (Canada)
  • 19. N=695
  • 20.  
  • 21. χ 2 = 38.07, df=14, p < .001 Cramer's V = .36, p < .001 N=302 159 (M) 143 (F)
  • 22. Research Methods
    • Trend towards more empirical research?
    • Berge & Mrozowski (2000): 76 % (N=727 articles; AJDE, DE, JDE and OL, between 1990 and 1999).
    • 2000-2008: 38 %
  • 23. Authorship Patterns: Leading Contributors
    • 1,148 different authors contributed to the 695 articles in this study.
    • 48 authors who published at least 3 articles:
      • Canada (17)
      • USA (9)
      • UK (8)
      • Australia (5), China (5)
      • Israel (2)
      • Japan/South Korea (1)
      • New Zealand (1)
  • 24.  
  • 25. Authorship Patterns: Countries
    • The origin of the first author was taken into consideration (54 countries)
    • The vast majority (>80%) comes from only 5 countries
    • AJDE: strong North-American focus
  • 26. So quo vadis? Perspectives for future research
    • Current research is clearly dominated by studies on the micro level.
    •  Major gaps were found on the macro and meso level.
    • DEHub Project in Australia: http://www.dehub.edu.au/
    • Research consortium between 5 Australian universities established 2009
  • 27. So quo vadis? Perspectives for future research
  • 28. So quo vadis? Perspectives for future research
    • European research community is weaker compared to North American institutions (research outputs in leading DE journals).
    • European research agenda for the next decade.
    • Clear research profile for the professional field of DE.
    • EDEN: Focus Group or launch a project?!
    • EURODL: better integration into the international journal network needed
  • 29. Thanks for your attention! Olaf Zawacki-Richter, Ph. D. Professor of Educational Technology University of Oldenburg Faculty of Educational and Social Sciences Center for Lifelong Learning (C3L) Germany [email_address]