Presented at SOLES conference in San Diego this was the first in a series of presentations deliverd as the meta-analysis on published action research developed.

1. Ongoing Meta-Analysis: The View After 150 Papers Have Been Analyzed
Introduction
Early action research (AR) literature set itself apart from the rest of the standard methodologies in two
ways: 1) rather than being neutral observers researchers were to take actions with the intent of
progressing on the issues that they studied, and 2) by developing reflection cycles (later called double-
loop learning or reflexion) researchers considered their own ideas as data and changed their own
practice accordingly (Argyris & Schon, 1974, Cunliffe, 2004). Often considered within a pragmatic
framework, AR studies could be considered good candidates for the development and evaluation of
best practices (Authors study, 2006). Interpersonally the early literature also demonstrated increased
communication across power lines with emancipatory results (Argyris & Schon, 1978).
This study considers whether and to what extent those, and other pragmatic or emancipatory outcomes
consistently exist in AR as demonstrated by studies published in journals 50 years later. It develops out
of a vague sense of dissatisfaction. After finishing an overview of recent Action Research Journals a
reader might ask "Where is the action in action research?" In a similar fashion moved by a recent blog
post on the Sage Action Research blog from a young graduate student who professed, “I admire your
(AR) ability to adapt to every situation without losing sight of your values” (Nylander, 2011) this
researcher wondered if that was true? And so a meta-analysis was born.
The scope of this project covers literature of AR in business, education, and healthcare. Committed to
continue until 300 articles are contained in the database, by the time of AERA, 2012, 150 articles will
be included in the study. The first set of 50 articles established the baseline information and the
findings and conclusion of this paper. By fall 2011, another 50, this set focused on business and
organizational development writing, were included. By spring 2012 the third set of 50, focused on
education, finished the first half of the study. Those data will be analyzed for the presentation.
Propelled by Kemmis, 2006, who cajoled readers to consider their own potential growth and new
stances for the AR community, this study measures AR articles published in journals across the
continuums he discussed. Kemmis suggested that action researchers:
1. Reach beyond the focus on improving techniques of practice to address the broader questions
about how work influences the context in society, becoming a vehicle for critical debate to
improve our world.
2. Reach beyond a focus on improving efficiency to one of taking a critical view on the “irrational,
unjust or alienating consequences of contextual issues that cause the issues” AR projects
address (p 460).
3. Use AR to improve policy beyond something that creates conformity to build flexible structures
that can be used to address everyone's needs.
4. Reach beyond improved understanding of the situation solely from the professional’s point of
view to one that is inclusive of the needs and ideas of community members and the broader
public.
5. Reach beyond acting alone to becoming part of the wider AR community so that it adds to the
discursive commentary on what we do and how to make AR work better, expanding the
knowledge about AR in the world.
The research questions which drive this meta-analysis are first, whether and to what extent does the

2. work of action researchers, as it is reported in journals, display clear actionable outcomes that are
rigorously measured? Second, looking at the Kemmis suggestions as a continuum of potential focus,
where do studies fall? Finally, to what extent does the literature produced in action research focus on
personal transformation, or the methodology more than on the issue of study?
The Basics
This is a report of findings for an ongoing meta-analysis of 150 years of action research literature. The
purpose of this study is to investigate what has been published and to rank it against whether it meets
the criteria of the early writers on action research, delineating AR as different because people using AR
take action towards an outcome rather than measuring as a neutral observer. Each article has also been
ranked for placement on each of the five continuums of Kemmis (2006). The final analysis was
whether or not the article was overall more focused on personal transformation or a discussion of
subtleties within the methodology as opposed to the development of strong, pragmatic, action oriented
outcomes. The process of analysis involves the article being placed in a spreadsheet based on the
industry about which it reports, noted as to whether it is published in a journal devoted to AR, and then
read and analyzed on a simple ranking of 0-3. Zero is used when the article had no evidence of that
measure. A rating of one equated to the left side of the continuum, three to the right and two is used
when a case could be made for either or both. Notes were taken as to the reasoning behind the score.
This meta-analysis by the spring of 2012 will contains 150 entries. The findings in this article were
generated after the first 50 and therefore any conclusions herein are extremely preliminary and can only
hint at those which may be derived by presentation at AERA. This study contributes to positioning AR
as a mature methodology, evokes discussion within the academy as to future guidelines, and allows the
editors of journals, the students and practitioners, and anyone else interested in these methods, to plot
their course with different considerations than prior to the availability of the overview.
Initial Findings (from the first 1/3rd of these data)
#1 The difference between AR studies published in AR journals versus journals focused on the
field of study that contained the issues/circumstances/practices studied.
Out of the 21 articles that ranked one focused on personal or methodological subtleties, (almost half)
eight or 38% were in AR journals. This compares to the fact that among the 50 articles studied, 28%
overall were published in these same journals. We can therefore say that from this small sample, it
appears that AR journals are more likely to publish articles focused on the personal or methodological
to the exclusion of actionable outcomes.
It is also instructive to look at the reverse data. Out of the 11 articles that were ranked a 2.5 or 3 which
had results that were outcome oriented and pragmatic, only 1 was published in an AR journal. This
equates to only 9% because of the small sample size used for initial findings.
#2 There appears to be a correlation between a study that develops best practices and one that
starts out with a strong clarity of the issue they face/action they desire.
An incredibly strong correlation exists through all ranking levels within the articles from 0-3 on these
two types of ranking. For instance: 23 (46%) have the same ranking on both best practices and action,
or with a variance of .5 or less. In addition, 15 (30%) are only a point apart and of those rated on both
scales, only 5 (10%) vary more than 1 point.
#3 There appears to be a correlation between different variables on the continuums developed

3. through Kemmis.
For the states of acting as a Lone Researcher to being part of a Community /AR Community and those
focused on Professionalism versus developing a Community Vision there is a strong correlation
between 1:1 and 3:3 rankings. Thirty three (66%) were in exact agreement or less than 1 point
difference between them, and only 4 (8%) had more than 1 point variation between them.
In a similar fashion, the ranking of 1 for focused on technique vs. 3 for a focus on a life enhancing
topic also show strong correlation with 1 for improving efficiency vs. 3 for discussion with a critical
view with 25 (50%) an exact match or .5 difference between them. Only 8 (16%) had a more than 1
point difference.
Conclusions
Of course it is difficult to draw firm conclusions from initial findings. This discussion will be much
more robust by the fall, 2011 when the next iteration of this article is uploaded and then even still more
robust for the presentation in April 2012. Nevertheless, four conclusions seemed to develop naturally
from this study after the initial stage:
1. A call for action to AR journals to take these data and the meta-analysis seriously. Preliminary
results suggest that action researcher jounals might consider the norm they are establishing and
choose instead to publish a different ratio of articles that demonstrate clear actions and
measurable results.
2. A call for action to those doing AR to pay attention to these results, placing their study
consciously on these continuums. As was suggested by Kemmis, this would increase evidence
of AR's effect on democracy, community vision, and the building of an AR community across
silos within academic settings.
3. AR researchers should consider whether and to what extent they want their work to strive for
the Kemmis emancipatory goals. Friere, Kemmis, and other emancipatory writers are widely
quoted, but initial findings do not confirm that they are widely emulated.
Further Study
What questions does this lead us to for future considerations against these and more robust data?
 Are some industries more likely to be in one place on the continuums than others?
 What can be said about the reality that emerges in the loop of relations between the edges of
each continuum?
 How will the outcomes or inferences change as I add: more from each category, older research,
etc.
 How will more articles reviewed in public administration change the perspective of that set of
literature?

4. Impact on Personal Practice (the story in first person)
Based upon the work of Cunliffe (2004) I asked myself a series of reflexive questions. Cunliffe
suggests practitioners should: develop ways and means to question their own behavior, their
culpability in the outcomes of their research, their own emotional intelligence, and to be open
to their own fallibility.
Improved practice due to being open to personal fallibility: In the course of reading and analyzing
articles, one article was accidentally ranked twice – most of the ranking ended up to be the
same but a couple did not. The outcome of this embarrassing circumstance was an increase in
attention and note taking as to why/how articles were ranked.
Improved practice through asking “What deeper insights are available?” Reflecting on the part of my
nature that can be judgmental, I questioned my initial choice to call the non-pragmatic articles
“tail chasers.” Was that just my annoyed and elitist vision or could I consider it a valid
continuum for judgment? A deeper part of my nature is to “tell the whole truth and nothing but
the truth” as my father always admonished me to do, with the potential that an unpopular truth
may surface others who agree and result in a movement towards improving the situation and so
I kept it through my first presentation. After I presented my initial findings for the first time
and found that they were well received, I was able to down from the need to discuss “tail
chasing” and future findings were ranked on the continuum of whether they focused more on
the subtleties of personal or methodological transformation as opposed to actionable outcomes.
Perhaps also there is a lesson from ranking outcomes across continuums that we can learn from Morin
as quoted by Bataille & Clanet (1981)
“In order to explain a complex phenomenon, it is not enough to associate opposing
notions in a concurrent and complementary way, linking them in the abstract by a
“master word” dialectic. It is essential to define the association of opposites within a
system of loops such as theory – practice, action – thought, subject – object, etc).
Reality emerges in the loop of relations between the edges.”
This came to me at a time when I was annoyed with the opposites I was seeing, and at myself for
constantly judging them. Defining the association of opposites as loops gave the whole process/project
a critical lift that I needed – a higher ground if you would- past the judgment of who did what on what
continuum and into a vision of its long term importance as part of the Yin/Yang of action research.
Improved practice through asking how the deeper part of my nature was/and can be more apparent?
This study is basically analytical, yet the heart of AR and the reason it is my methodology of
choice is the personal. Since I consider the deepest part of my nature to be loving and
supportive, this question reads: how can my analysis of AR be used for loving support of the
people doing AR and to make this methodology stronger within the world of research? This
requires not a muck-raking attitude but one of questioning support. Neither side of the
yin/yang of the continuums of this study is right or better than the other. The evolutionary path
will consider the whole rather than the parts.
Improved practice through considering whether my behavior models the norms I want to create? When
I rank my own AR articles and writing I see that they are strong in action and outcome, emergent in

5. reflexion and humanity. I also see that I am young in the understanding of paradox and looking
forward to the 'ah ha' moments that point to the reality that emerges in the loop of relations between
the edges.
Final considerations: Is this Action Research and Where Do We Go From
Here?
Bridget Somekh mentioned that articles in the Educational Action Research Journal (EAR) were not
being cited at the CARN conference in Utrecht. Is this still the case? If so, how can the academy
consider what guidelines are appropriate to help ensure that action research is providing access of
our learning to others not engaged with this methodology yet working across all the fields of
healthcare, education, business, and organizational development? Are we providing clear
examples of how this methodology improves practice? In order to build towards the global future
we envision, we need a concrete idea of where we currently stand.
References
Argyris, C. (2002). Double-loop learning, teaching and research. Academy of Management Learning
and Education 1(2), 206.
Argyris, C., & Schön, D. A. (1974). Theory in practice : increasing professional effectiveness (1st ed.).
San Francisco: Jossey-Bass Publishers.
Argyris, C., & Schön, D. A. (1978). Organizational learning. Reading, Mass.: Addison-Wesley Pub.
Co.
Cunliffe, A. L. (2004). On becoming a critically reflexive practitioner. Journal of Management
Education, 28(4), 407-426.
3 Studies by the Author deleted for peer review
Kemmis, S. (2006). Participatory action research and the public sphere. Educational Action Research,
14(4), 459-476.
Morin (1977) as discussed in Bataille, M., & Clanet, C. (1981). Elements contributing to a theory and a
methodology of action research in education. International Journal of Behavioral Development,
4 (1981), 271-291.