Information Systems Action design research method


Published on

Published in: Education, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Information Systems Action design research method

  1. 1. Raimo Hälinen (2012) Lahti 14.12.2012
  2. 2.  12.00 – 13.30 Action Design Research  13.30 – 14.00 Cafee break  14.00 – 15.00 Using Action Design Reseach method  Discussion how to apply ADR-method  15.00 – 16.00 Comparison DSR, AR and ADR methods  how to generalize results using by deduction and induction  Summary and conclusion
  3. 3. ISDSR Framework Activity Framework for DSR Dominant paradigm Design science Utility theory, Problem theories Focus Solution focused Problem-oriented, solution focused Perspective Researcher as experimenter (intervener) Researcher as theory developer and experimenter Logic Intervention-outcome Enhancement or creation of a method, product, system, practice or technique Research question Alternative IS interventions for classes of problems Problem space understanding and causes and consequences Research product Tested and grounded technological rules (design knowledge) Tested and evaluated product by Field studies, Experiments Action research, Simulations Nature of research product Heuristic Iterative cyclical process from theoretical basis to evaluated and implemented product Justification Saturated evidence Meta-design and meta-requirements Type of resulting theory Practical and abstract IS design theory and knowledge Practical and scientific IS design theory and knowledge Source: Walls et al (1992), van Aken (2004), Carlsson (2006), Venable (2006)
  4. 4. Paradigmatic dimension Action research Design science research Ontology Anti-realism Realism or anti-realism Epistemology Mainly anti-positivism Mainly positivism but also anti-positivism especially in evaluation Methodology Idiographic Constructive (building) Nomotethic (evaluation) Idiographic (evaluation) Ethics Meand-end Possible interpretive Unlikely critical Means-end Possible interpretive Possible critical Source: Iivari and Venable (2009) “Considering the practical implications of this analysis, we further identify that the employment of AR in the conduct of DSR needs to be done with care, especially where there is the potential for significant risk to the client or other stakeholders.”
  5. 5. Action design research approach Action research criterion Design research criterion Problem identification and definition The principle of research-client agreement Design as an artifact Problem relevance Build, intervention and evaluation (BIE) The principle of cyclical process model The principle of theory Design evaluation Research rigor Design as a search process Reflection and learning The principle of change through action The principle of learning through action Communication of research results for practitioners and researchers. Formalization of learning Research contribution Source: Cole, Purao, Rossi and Sein (2005)
  6. 6. Action research Design science research Action research emphasizes the utility aspect of the future system from the people’s point of view. Design science’s products are assessed against criteria of valua or utility. Action research produces knowledge to guide practice in modification. Design science produces design knowledge (concepts, constructs, models, and methods.) Action research means both action taking and evaluating. Buildin and evaluation are the two main activities of design science. Action researc is carried out in collaboration between action researcher and and the client system. Design science research is initiated by the researcher(s) interest in developing technological rules for a certain type of issue. (Each individual case is primarily oriented at solvin the local problem in close collaboration wtih the local people.) Action research modifies a given reality or develops new system. Design science solves construction problems (producing new innovations) and improvement problems (improving the performance of existing entities). The researcher intervenes in the problem setting. Design science research is initiated by the researcher (s) interest in developing technological rules for certain type of issue. Knowledge is generated, used, tested adn modified in the course of the action research project. Knowledge is generated, used and evaluated through the building action. Source: Järvinen P: (2005, 2007)
  7. 7. 1. Problem Formulation Principle 1. Practice-Inspired research Principle 2. Theory-Ingrained Artifact Principle 3. Data-inspired research 2. Building, Intervention, and Evaluation Principle 3. Reciprocal Shaping Principle 4. Mutually Influential Roles Principle 5. Authentic and Concurrent Evaluation 3. Relfection and learning Principle 6. Guided Emergence 4. Formalization of Learning Principle 7. Generalized Outcomes
  8. 8. 1. Identify and concepualize the research opportunity 2. Formulate initial research questions 3. Cast the problem as an instance of a class of problems 4. Indentify contributing theoretical bases and prior technology advances 5. Secure long-term organizational commitment 6. Set up roles and responsibilities Orlikowski and Iacono (2001) defined IT artifacts as the “bundles of material and cultural properties packaged in some socially recognizable form such as hardware and/or software”.
  9. 9.  Principle 1. Practice-Inspired Research The principle emphasizes to consider field problems (real-world problems) Action design research method is applied to study intersection of technological and organizational domains. The researcher can investigate how technological solutions can be used to improve organizational processes and achieve better results. An action design Researchers should try to generate knowledge that can be applied at the class of problems. (The cases are examples of the recognized class of problem). The result from the research activity is problem-inspired.
  10. 10.  Principle 2. Theory-Ingrained Artifact The principle emphasizes that created artifacts are based on theories. Gregor (2006) explored systems of statements that allow generalization and abstraction to be theories. Gregor’s theory of Type IV (explanation and prediction theories) or Type V (design theories) The applied theories can be universal laws of natural science or ADR- researcher can utilize the specific theory (e.g. TAM, IS-succes) . 1. Identify problem, 2. Identify potential solutions, 3. design guide. Ingrained artifact is subject to organizational practice and it provides basis for cycles of intervention, evaluation and further reshaping.
  11. 11. ADR team The Generic Schema for IT-dominant building, intervention and evaluation Practitioners End-users Artifact Design principles Contribution to the specific ensemble being designed Utility for the users Researcher(s) Alpha version Beta version
  12. 12. Researcher(s) Practitioners End-users Alpha version Beta version Implementation of Artifact Design principles Contribution to the specific ensemble being designed Utility for the users ADR team The Generic schema for Organization-Dominant BIE 1. Discover initial knowledge creation target 2. Select or customize BIE form 3. Execute BIE cycle(s) 4. Assess need for additional cycles, repeat Implementation of an artifact can lead to modification of organizational processes and work flows.
  13. 13. Researchers Practitioners End-users ARD team Alpha version Beta versin The Generic Schema for Data-Dominant BIE Database Utility for the users Contribution to the specific database being designed Design principles
  14. 14.  Principle 3: Reciprocal Shaping The iterative process that are described is based on DeGrace and Stahl (1990) recursive cycles. Scrum process Source:, DeGrace P and Stahl L. (1990), Sutherland J. (2010) Scrum meeting 30 min. 1. What did you do yesterday? 2. What will you do today? 3. What obstacles got in your way? The roles of Scrum project 1. The product owner 2. The Team 3. The Scrum Master The basic concecpt 1. The product backlog 2. The Sprint 3. Sprint planning 4. Daily scrum meeting 5. Sprint review and retrospective
  15. 15.  Principle 4: Mutually Influential Roles The action design researcher’s role is to share the knowledge of theory and technological advances. The practitioner’s role is to consider practical hypotheses and knowlegde of organizational work practices. Researchers and practitioners’ role are complementary and complate each others. The clear assigment of these responsibilities is important that reflection and experience can be utilized during the research projects.
  16. 16.  Principle 5: Authentic and Concurrent Evaluation The evaluation is essential activities in building, intervention and evaluation process. The evaluation is not a separate stage as it is e.g. March and Smith (1995) state-gate models or compared to Peffers et al.’s (2008) model. The evaluation is ongoing and continues from start to the end of the project. The style of evaluation is formative (Remenyi and Sherwood-Smith (1999). The summative evaluation is utilized for beta version evaluation. The controlled evaluation may be difficult to achieve, so it should be carried as part of natural controls where possible.
  17. 17.  Principle 6: Guided Emergence The designed artifact will reflect preliminary design and ongoing shaping by organizational use, perspective and participants. Component of ISDT Design product 1. Meta-requirement 2. Meta-design Describes the class of goals to which the theory applies. Describes a class of artifacts hypothesized to meet meta-requirements. 3. Kernel theories Theories from natural or social sciences governing design requirements. 4. Testable design product hypotheses Used to test whether the meta-design satisfies the meta-requirements. Design process 1. Design method A description of procedure(s) for artifact construction. 3. Kernel theories Theories from natural or social sciences governing design process itself. 4. Testable design process hypotheses Used to verify whether the design method results in an artifact which is consistent with meta-design. Source: Walls et al. (1992, 2004) Kernel theories Meta- requiremens Meta-design Testable desing product Kernel theories Desing method Testable design process
  18. 18.  The objective is to formalize the learning.  According to Van Aken (2004), the situated learning should develop general solution concepts for a class of field problems.  Principle 7: Generized Outcomes 1. Generalization of the problem instance 2. Generalization of the solution instance 3. derivation of design principles from the design research outcomes.
  19. 19. 1. Absract the learning into concepts for a class of real- world problems. 2. Share outcomes and assessment with practitioners 3. Articulate outcomes as design principles 4. Articulate learning in light of theories selected 5. Formalize results for dissemination Tsang and Williams (2012) Definition of statements: a statement is empirical when and only when it cannot be ascertained to be true or false without experience or observation. a statement is theoretical when and only when it is a generalization that purports to predict and explain the phenomena to which it refers.
  20. 20. Evidence Cause Effect Prescribed action Explanatory statement Goal Prescribtive statement Articact evaluation Kernel theories Mid-range theories Design theories Theory development Might lead to Is installed to lead to Source: Kuechler and Vaishnavi (20008) can be confirmed byleads to revision of Kuechler and Vaishnavi (2008) pointed out to need for to create mid-range theories. The mid-range theories act as a bridge between kernel theories and design theories.
  21. 21. Design Theory Design Principles Design process Class of Problems Problem Solution as an artifact in use Class of Solutions Four levels in conceptual process: 1. Generalization of the problem instance 2. Generalization of the solution instance 3. Emerging design knowledge in the form of design principles 4. Feedback to design theory Level 1 Level 2 Level 3 Level 4 Rossi, Purao and Sein (2012)
  22. 22.  Trying to generalize results of the research project can be characterized using by Yin’s process(1994, p.31). Theory Rival theory Population characteristics Sample of research Case study findings Experimental findings Subjects Level-2 inference Level-1 inference
  23. 23.  Lee and Baskerville (2003) considered generalization problem and proposed the framework. Generalizing empirical statements Generalizing to theoretical statements Generalizing from empirical statements (EE) Generalizing from data to description. Measurement, observation or other description. It may be Inductive analogy. (ET) Generalizing from desription to theory. Measurement, observation or other description to a theory Generalizing from theoretical statements (TE) (emprical testing) Generalizing from theory to description. Confirmed in one setting, to descriptions of other settings. (Deduction) (TT) ? EE = emprical to emprical, ET = empirical to theory TE = theory to empirical, TT = theoretical to theory Accroding to Järvinen P review this framework may be problematic?
  24. 24. Theory Same population Different population Different context Different time Within-population generalization Cross-population generalization Contextual generalization Temporal generalization Sample Particular instance Theoretical generalization Inductive analogy Statistical syllogism Empirical level Source: modfied from Tsang and Williams (2012)
  25. 25. Types Definition Theoretical Generalization from research findings to theories Within- population Generalizing from the characteristics of a sample to those of the corresponding population. Cross-population Generalizing from a sample in one population to members of another population, with both population existing in a similar context and a similar period of time. Contextual Generalizing from a sample in one population to members of another population, with both population existing in a significantly different context but a similar period of time. Temporal Generalizing from a sample in one population at one point in time to members of the same or a different population at another point in time, assuming that the context remains more or less the same. Statistical syllogism An inference of the form P1 N % of Fs are Gs. P2 X is an F. C X is a G. where “N” denotes a precise statistic or a vague range of statistics as in “Most” or “Nearly all.” Inductive analogy An inference of the form P1 X has properties a, b, c … and z P2 Y has properties a, b, c … C Y has property z Source: adopted from Tsang and Williams (2012
  26. 26. Property Design research Action research Action design research Artifact Central Peripheral Central Organizational impact Peripheral Central Central Subject participation in research design Possible Mandatory Mandatory Subject Feedback Discrete Continous Continous Transferability Explicit Implicit Explicit Success measure Quantifiable measures of artifact behaviour Organizational impact Organizational learning and artifact generalizability Source: Henfidsson (2011)
  27. 27. Orientation Quantitative Qualitative Assumption about the world A single reality and specified real- world phenomenon. Multiple realities and real-world phenomena. Purpose of the research Trying to establish relationships between measured variables. Trying to understand social situation from participants’ perspective. Methods and processes Research procedures and activities are established before research begins. Hypotheses are formulated before study can begin. Deductive approach. Research procedures and activities varies depending on data gathering and how the study is proceeding. Saturation point is essential for data collection process. Inductive approach. Researcher’s role Ideally an objective observator. The researcher participates in the study activities, and it can have different roles during the research project. Participant’s role No active role during the research project. Participants’ role is the active participant from start to the end of the project. Research results and generalizability Generalizations are context-free depending on sample size and population. Generalizations are based on context-detailed data and analysis. Source: Modified from Thomas (2010)
  28. 28. Contingency variables Strategy Structure Size Environment Technology Individual Task Management information system Design Management implementation investment Use Implementation Management information system Performance Firm Performance Use Satisfaction Success Effectiveness Perception Financial Source: Weil and Olson (1987)
  29. 29. Intensive cases Comparative cases Purpose Developing theory from intensive exploration. Developing concepts based on case comparison. Assumption Creativity through comparison with existing theories. Comparison of cases leads to more useful theory. Examples Situation Usually evolves out of a researcher’s intensive experience with culture or organization Usually concepts are developed from one case compared with another case. Types Narratives, tabulation, explanatory or interpretative Case comparison, case survey, interpretative comparison. Source: Järvinen (2011)
  30. 30. Stage Key Charasteristics Identify research problem and question(s). Phenomenon is examined in a natural setting. The focus is on contemporary events. Determine type of case study. The complexity of the unit is studied intensively. Select participants or groups. One or view entities are examined (person,group, organization) Collect data. Data are collected by multiple means. Analyze data. Independent and dependent variables are not specified in advance. Within-case or cross-case analysis. Compose the report. Why and How questions are useful in case studies. Evaluate the validity and reliability. No experimental controls or manipulation are involved. Write proposals. The resultss derived depend heavily on the intergrative powers of investigator. Write contributions. The research situation Problem analysis Field insights and data gathering Resourse development evaluate the validity and reliability Write Proposals Real-world situation A single case Source: modified from Benbasat et al. (1987), Eisenhardt K.M. (1989) Järvinen P. (2012)
  31. 31.  How and Why questions are important  The focus is contemporal  The purpose of case studies:  Description of phenomena  Grounded theory can be applied to explore phenomena  Explorative approach is used to validate, confirm or falsify developed theory or model based on collected data.
  32. 32.  Conceptual requirements:  Construct validity  Internal validity  External validity  Reliability  Structural requirements:  Relevance for the audience  Voyage of discovery  Controversies  Include all necessary data but not too much  Formal structure and elaboration
  33. 33. Plan the human centred process Specify the context of use Specify user requirements Specify organizational requirements Specify main features of an application Design and develop an application Evaluate appication against user requirements Evaluate application against organizational requirements Demonstrate application against speficied features (Field experiment) Information related work systems Test processes module test Incremental Acceptance test verification Practitioners Users, managers and executices participates to define needed features.
  34. 34.  Baskerville and Wood-Harper’s strategies:  Establish an ethical client-system infrastructure  Design data gathering process carefully  Follow the planned iterative phases:  plan action  take action  evaluate action  Promote collaboration by the subject and support their subjects’ learning cycles.  Write the report that disseminate the scientific knowledge in a way that it is possible to carry out future research and by this way confirm or refute any causal suggestions or claims of generilized theory.
  35. 35.  The following slides include examples, how ADR-method has been applied by other researchers.  Saarinen L. (2012) Dissertation in Aalto University  Rothengatter D. (2012) Dissertation  Modified action design reserch method:  Bilandzic and Venable (2011): Participatory Action Design Research method, a new method applied to urban informatics in Australia  Wieringa and Morali (2012) Technical Action Research as a Validation Method in ISDS.  Papas, O’Keefe and Seltsikas (2012) The Action research vs design science debate: reflection from an intervention in eGovernment. They point out that ADR-method is meta- method.
  36. 36. Source: Saarinen (2012) Dissertation
  37. 37. Source: Rothengatter (2012) Dissertation
  38. 38. Literature review Initial design Problem formulation Domain analysis Implementation of improved design Evaluation of improved desing Building, intervention and evalluaution Donain analysis Implementation of initial design in case Evaluation Design updating based on evaluation Building, implementing and evaluation Analysis of initial design Analysis of evaluation of desing Analysis of improved desing Reflection and learning Update of underlying theories Formalization of final design Formalization of learning Rothengatter’s meta-level process in ADR included two building, intervention and evaluation states. First iteration stage focus is to develop version 1. Second iteration stage includes an improvement of version one after evaluation is carried out, and results are available. Reflection and learning include three analyses. Rothengatter applied contingency theory to achieve IS performance and organizational performance of Information system design (ISD).
  39. 39. Source: Bilandzic and Venable (2011) Diagnosis and Problem formulation Action planning Action taking: design Impact evaluation Reflection and learning Participative problem setting Ethnographic study Opportunity identification Participative planning Participative design Prototyping and installation Ethnographic study Participative evaluation Participative client learning Design theorizing for UI Urban informatics Community, Urban dwellers Social good Well-being, healt, social connectedness Government, public institution Open, mobile, diverse Public, access for all Public Completely discretionary Essential
  40. 40. IT-artefact problem investigation IT-artefact design It-artefact design validation It-artefact implementation Implementation evaluation Research problem investigation Research design Research design validation Research execution Analysis of research results and publishing Client-system problem investigation Treatment design Design validation Implementation in the Client- system Implentation evaluation and applying results
  41. 41. Engineering cycle Engineering cycle Idealizing assumptions Realistic assumptions o o o o Framework for IS design science Environ- ment IS design sccience Knowledge base Improvement problem solving Knowledge question investigation Goals Artifact Knowledge
  42. 42. Real-world practices and values Local Practices Design Research Practice Reserch Practices Knowledge base and values Contribution to general practice Contribution to knowledge base Practical Realm Academic Realm Purely practical contribution Practical and scientific contribution Purely scientific contribution Practical advancement Scientific advancement No contribution Source: modified from Sjöström and Donellan (2012
  43. 43. Research object Questions AR DR ADR The artefact What is the role of the artefact? Is the design of the artefact to improve organizational practice? Is the use cases needed to before artefact development? ? ? ? The process and research cycles How is the research problem going to be determined and agreed? Is a predetermined cycle of activity going to be followed? Is a software development method a necessary part of the activities? ? ? ? The focus of evaluation Can evaluation be a by-product of the research cycle? Is it necessary to carry out explicit evaluation activities? (e.g. ex-ante, ongoing, ex-post, verifaction and validation) How are acceptance tests included to research cycle? ? ? ? The role of knowledge How is the role of knowledge concerning researchers understanding of real-world phenomena? How is the role of knowledge about practitioners understanding of the research processes? ? ? ? The role of learning What expectations are in question of learning in organizational practices? ? ? ? The role of practical results How important is the artefact or improvement of the organizational process during the research process? ? ? ? The role of scientific results To what extent is to develop scientific knowledge during the research process? ? ? ? Source: modified from Papas et al. (2012)
  44. 44. Identify real-world situations (P) and desired outcomes (O) Review (kernel theories) and previous research Propose/refine design theory Test design theory Source: Carlsson et al. (2007), Hrastinski et al. (2007) O=f(p,i,c,m), where P = problem I = IS initiative M = Mechanism C = Context O = Outcomes A realist Information Systems intervention
  45. 45. Research area Research objects Design science research Design research method are used more int the future. Design and designers are selected to the research objects. Design science theory is developed and studied in the future. Healt care and IT Safety and efficiency healt-care systems. e-recipes, electronic medical records. clinical decision support systems. Design challengies are multiple. Green technology and IT Carbon footprints: 1. Primary 2. Secondary Renowable energy and energy usage in IS. Green computing: Algritm efficiency Computer virtualization Telecommuting Collaboration Two-way information channels Collaboratice design Education Web 2.0 (3.0) Semantic web Social media Voice over IP Game industry Designer Software developers Animators Source: modified from Hevner et al. (2010)
  46. 46.  Bilandzic, M., & Venable, J. (2011). Towards Participatory Action Design Research: Adapting Action Research and Design Science Research Methods for Urban Informatics. Journal of Community Informatics. Special Issue: Research in Action: Linking Communities and Universities, 7(3).  Carlsson, S.A. (2006): Towards an Information Systems Design Research Framework: A Critical Realist Perspective, Proceedings of the First International Conference on Design Science in Information Systems and Technology (DESRIST 2006), 192-212.  Henfridsson O. (2010) Action Design Research, presentation slides, University of Oslo  Hevner A and Chatterjee S. (2010) Design Research in Information Systems, Theory and Practice, Integrated Series in Information Systems 22, Springer.  Järvinen P. (2012) On baundaries between field experiment, action research and design research, University of Tamper, School of Information Sciences, Reports in Information Sciences 14, Tampere  Järvinen P. (2010) IS reviews, Department of Computer Sciences, University of Tampere, DE-2010- 16, Tampere  Lee, A. S., and Baskerville, R. L. 2003. “Generalizing Generalizability in Information Systems Research,” Information Systems Research (14:3), pp. 221-243.  Rothengatter D. (2012) Engineering situational methods for professional service organization, An action design research approach, CTIT Ph D. Thesis Series No. 11-225, Enchede.  Saarinen L. (2012) Enhancing ICT Supported Distributed Learning through Action Design Research, Department of Information and Service Economy, Aalto University, Doctoral Dissertation.  Sein, Henfridsson, Purao, Rossi and Lindgren (2011), Action Design research, MIS Quarterly, Vol. 35, No. 1, pp. 37-56  Sjörström and Donellan (2012) Design research practice: A product semantics interpretation, The International workshop on IT Artefact Design & Workpractice Intervention, 10 June 2012, Barcelona  Tsang E. W. K. and Williams J.N. (2012) Generalization and induction: Clarifications, and a classification of induction, MIS Quarterly, Vol. 36, No. 3, pp. 729-748.  Yin R.K, (2003 ) Case Study Resaarch, 3rd edition, Sage Publication.