DR-AR-ICIS-2005

Slide 1: BEING PROACTIVE: WHERE ACTION RESEARCH MEETS DESIGN RESEARCH Robert Cole Sandeep Purao School of Information Sciences and School of Information Sciences and Technology Technology Penn State University Penn State University University Park, PA U.S.A. University Park, PA U.S.A. rcole@ist.psu.edu spurao@ist.psu.edu Matti Rossi Maung K. Sein Department of Management School of Management Helsinki School of Economics Agder University College Helsinki, Finland Kristiansand, Norway matti.rossi@hse.fi maung.k.sein@hia.no Presentation at ICIS 2005, Nevada, Las Vegas

Slide 2: Motivation Our Premise: To be relevant, IS research community needs  to take a proactive stance and conduct research that: adds to theory i.e. make a scientific contribution  solves current or anticipated problems of practitioners  Research approaches with this dual orientation  Action Research (AR) and Design Research (DR).  Our belief: despite their traditional separation:  They are fundamentally similar  They provide unique strengths and can inform each other 

Slide 3: This Research Objective: Investigate mapping between AR and DR  Research Approach: cross-application of research criteria  Select an exemplar study of each type of research  Examine each type according to the criteria of the other type  Question: Can an exemplar of one type be “recognized” by  applying the criteria of the other type. Anticipated Outcome: Exemplars in each type will satisfy  the criteria from the other type. Anticipated Interpretation: The two research types are  fundamentally similar.

Slide 4: Characterizing DR Design Research  Construction and evaluation of technology artifacts to meet  organizational needs Development of associated theories  Axiological emphasis on utility  Artifacts produced include (adapted, March and Smith  1995) Conceptual designs  Methods  Models and Systems (prototypes)  Better theories 

Slide 5: Design Research Criteria (From A. R. Hevner, S. T. March, J. Park, and S. Ram, “Design Science in Information Systems Research,” MIS Quarterly (28:1), 2004, p. 83. Copyright © 2004, Regents of the University of Minnesota. Used with permission.) Criterion Description 1. Design as an artifact Design research must produce a viable artifact in the form of a construct, a model, a method, or an instantiation. 2. Problem Relevance The object of design research is to develop technology-based solutions to important and relevant business problems. 3. Design Evaluation The utility, quality, and efficacy of a design artifact must be rigorously demonstrated via well-executed evaluation plans. 4. Research Effective design research must provide clear and verifiable contributions in Contributions the areas of the design artifact, design foundations, and/or design methodologies. 5. Research Rigor Design research relies upon the application of rigorous methods in both the construction and evaluation of the design artifact. 6. Design as a search The search for an effective artifact requires utilizing available means to process reach desired ends while satisfying laws in the problem environment. 7. Communication of Design research must be presented effectively both to technology-oriented Research as well as management-oriented audiences.

Slide 6: Characterizing AR Action Research (adapted, Baskerville 1999)  Fundamentally a change-oriented approach  Central assumption: complex social processes can  best be studied by introducing change into these processes and observing their effects Focuses on practical problems with theoretical  relevance Produces highly relevant results while simultaneously  informing theory There are various flavors of AR  Our analysis will use criteria of canonical action research 

Slide 7: Canonical Action Research Criteria (Adapted from Davison, R. M., Martinsons, M. G., and Kock, N. “Principles of Canonical Action Research,” Information Systems Journal (14:1), 2004, pp. 65-86.) Criterion Description 1. Principle of Researcher-Client The RCA provides the basis for mutual commitment and role Agreement (RCA) expectations. 2. Principle of Cyclical Process The CPM consists of the stages diagnosing, action planning, action Model (CMP) taking, evaluating, and specifying learning. 3. The Principle of Theory Theory must play a central role in action research. 4. The Principle of Change Action and change are indivisible research elements related Through Action through intervention focused on producing change. 5. The Principle of Learning Considered reflection and learning allow a researcher to make both Through Reflection a practical and theoretical contribution.

Slide 8: A ‘DR’ Exemplar Markus, M. L., Majchrzak, A., and Gasser, L. “A Design  Theory for Systems That Support Emergent Knowledge Processes,” MIS Quarterly (26:3), 2002, pp. 179-212. This study presents the design and implementation  of an IT system called technology organization and people integration modeler (TOP Modeler) for the support of an emergent knowledge process of organizational design. Detailed analysis of criteria 4 and 5: ‘Change through action’ ‘Learning through reflection’

Slide 9: Applying AR Criterion 4 to DR Criterion 4: The Principle of Change through Action  Expectation:  Evidence of practitioner motivation for change  Evidence of change resulting from design outputs - the  improvement of a previously undesirable problem situation. Findings:  Client motivation is clearly present.  Evidence of behavioral change is apparent:  Individual level - users learning about their organizations,  achieving consensus on design issues etc. Organizational level - cancellation of a plant operation  relocation based on identified weaknesses, postponement of international joint venture based on strategic differences uncovered through using TOP Modeler.

Slide 10: Applying AR Criterion 5 to DR Criterion 4: The Principle of Learning Through  Reflection Expectation:  Evidence of outcome reflection.  Reporting of research results and implications.  Findings:  No explicit reflection  Evidence of strong client engagement  Research outcomes were clearly reported to the  research community through articulation of several aspects of an activity area (EKP)

Slide 11: Applying AR Criteria to DR Exemplar AR Criterion Evidence found in the DR Exemplar 1. The Principle of Researcher- No explicit RCA but clear evidence of motivational factors Client Agreement (RCA) 2. The Principle of Cyclical Iterative design/evaluate process followed Process Model (CPM) 3. The Principle of Theory Theory played central role in artifact development and theoretical contribution was made 4. The Principle of Change through Behavioral change evident at both the individual and Action organizational levels 5. The Principle of Learning No explicit evidence of progress reporting but evidence of through Reflection strong client engagement; reporting of research outcomes

Slide 12: An ‘AR’ Exemplar Iversen, J. H., Mathiassen, L., and Nielsen, P. A.  “Managing Risk in Software Process Improvement: An Action Research Approach,” MIS Quarterly (28:3), 2004, pp. 395-433. The research was part of a larger research program  and the specific aim of the project was to improve the implementation of software process initiative (SPI) practices. Detailed analysis of criteria 1 and 3: ‘Design as an Artifact,’ ‘Design Evaluation’ 

Slide 13: Applying DR Criterion 1 to AR Criterion 1: Design as an Artifact  Expectation:  Intervention in the organization will at least implicitly  result in the creation of artifacts Findings:  No explicitly stated artifacts, But,  Two implicit primary contributions:  an SPI risk management framework and process  an approach to tailor risk management to specific  contexts.

Slide 14: Applying DR Criterion 3 to AR Criterion 3: Design Evaluation  Expectation:  Evidence of evaluation of organizational interventions  due to the prominent role played by the evaluation stage in the CPM. Findings:  No specific evidence of explicit application of systematic  evaluation criteria to the research outputs, but, The SPI approach developed was evaluated according to  the standard of utility to practitioners. Through several iterations, the SPI framework was  refined based on feedback until it reached a practically usable and stable form

Slide 15: Applying DR Criteria to AR Exemplar DR Criterion Evidence found in the AR Exemplar 1. Design as an Artifact Instantiation of SPI models and methods (implicit) 2. Problem Relevance Clear evidence of relevance due to high resource commitment by organizations involved 3. Design Evaluation Evaluation based on utility to practitioners 4. Research Contributions Several theoretical contributions present 5. Research Rigor Explicit discussion of adherence to canonical criteria and logic behind SPI 6. Design as a Search Four CPM cycles executed before the risk management approach Process was evaluated as stable and usable 7. Communication of Results were communicated to both practitioners and researchers Research

Slide 16: Implications Commonalities between AR and DR:  Ontology: the phenomenon of interest does not remain  static through the research process. DR criterion 6, AR criterion 1.  Epistemology: knowledge is created through  intervening to effect change, and reflecting on this intervention. DR criterion 1 AR, AR criterion 4.  Axiology: both value the relevance of the research  problem, and emphasise practical utility and theoretical knowledge. DR criteria 2 and 3 and AR criterion 5 are applicable to  one another.

Slide 17: A Common Paradigm? Pragmatism  Applying the four tenets of pragmatism (that characterize AR) to  DR: Consequences defining concepts: In DR, there is a need to establish the  purpose of the resultant artefact. Practical outcome embodying truth: The focus of DR is practical action,  which ensures that the notion of truth lies in the utility of the produced artefact. Logic of controlled inquiry: The essence of DR is that designing must  inform theory in that the produced artefact should embody a theoretical premise or a “new idea,” which can be evaluated by evaluating the artefact. Social context of action: In DR, the act of designing is socially and  organizationally situated, specifically in our conceptualization.

Slide 18: Cross-Fertilization Adding Reflection to Augment Learning from DR  Interjecting an AR cycle at the last stage of the DR process  A DR project may be framed as an AR project if an  organizational problem needs to be solved, and the action involves building a system Concretizing Learning from AR by Adding Build  Frame the output of AR as a DR artifact, such as prototypes,  frameworks, or models Enhancing the AR action taking phase by including the building  of a design artefact. Envisioning an AR-DR Integrated Research Process 

Slide 19: An Integrated Research Process AR1 - Diagnosing Problem DR1 - Identifying a need a problem Definition AR2 - Action planning Intervention DR2 - Building AR3 - Action taking AR4 – Evaluating DR3 - Evaluating Evaluation reflecting Reflection DR4 – Learning AR5 – Specifying and Learning learning DR5 - Theorizing

Slide 20: Future Work Investigate incommensurability, if any, between the  assumptions underlying AR and DR Deeper analysis of the epistemological roots of the two  approaches Examine the historical and institutional roots of the two  approaches to explore why the disconnect exists Explore other methods of cross-fertilization between  the two approaches Apply the integrated research process in a project 

Slide 21: BEING PROACTIVE: WHERE ACTION RESEARCH MEETS DESIGN RESEARCH Robert Cole rcole@ist.psu.edu Sandeep Purao spurao@ist.psu.edu Matti Rossi matti.rossi@hse.fi Maung Sein maung.k.sein@hia.no Presentation at ICIS 2005, Nevada, Las Vegas