Information Systems design science research


Published on

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Information Systems design science research

  1. 1. Raimo Hälinen (2012) Lahti 11.10.2012
  2. 2. Source: Christensen and Sundahl (2001) Paradigm Theory Classification Observe, describe & measure phenomena Confirm Theory is a statement of what causes why, what, and under what circumstances Anomaly
  3. 3. Source: Venable (2006) Technology Invention/Design Enhancement creation of a method, product, system, practice or techniques Theory Building Solution space and Problem theories Utility theories or hypotheses Technology Evaluation Field studies Experiments Action research Simulation Problem Diagnosis Problem space understanding Problem causes and consequences Information Systems Design theory (ISDT) (Walls et al. (1992) 1. Design theories must deal with goals as contingencies 2. A design theory can never involve pure explanation or prediction 3. Desing theories are prescriptive 4. Design theories are composite theories with encompass kernel theories from natural, social science and mathematics.
  4. 4. Components Software development Managing risks Tarkoitus ja ala Purpose and scope Tavoitteena on kehittää vaiheittain rakentuvan tietosysteemin kehittämis- ja testauspolitiikka. Sovellusten kehitystyöhön sisältyvien riskien hallinnan lähestymistavan kehittäminen. Käytetyt ilmaisut Constructs Vaiheittaisen kehittämisprosessin virheiden tunnistaminen ja testattavuus Riskitekijöiden tunnistaminen, riskin aiheuttajien tunnistaminen ja riskin eliminoiminen Muodon ja toiminnan periaatteet Prinsciples of form and function Dynaamisten ohjeiden kehittäminen ja järjestelmäintegraation tunnistaminen Riskien tunnistamisprosessin ja sen aktiviteettien kehittäminen Artefaktin kehitysluonne Artefact mutability Tiimioppisiseen liittyvien vaikutusten tunnistaminen kehitystyön erivaihessa. Sovellusten ja tietosysteemien käytettävyys ja itse-ohjautuvuuden huomioon ottaminen Testattavat väittämät Testable propositions Sovelluksen tai tietosysteemien tuotosten (tulosten) arviointi ja simulointi perustuen suunnitteluteorian tosilauseisiin. Suunniteltu riskien lähestymistapa on sovellettavissa yleisesti sovellusten kehittämistyöhön Todistava tietämys Justifactory knowledge Luonnon- ja sosiaalitieteiden antama tietämys, johon suunnittelutyö perustuu Yleisestä riskienhallinnan johtamismalleista johdetut periaatteet. Toteutusperiaatteet Principles of implementation Sovelluksen tai tietosysteemin käyttöönottamisen periaatteet Käyttöönottoon liittyvien riskien hallinta ja osaaminen Konkreettinen toteutus Expository instantiation Varsinaisen käyttöönottoprosessin vaiheistus ja ohjeistus Riskitekijöiden tunnistaminen ja arviointi käyttöönottovaiheessa.
  5. 5. Awareness of problem Suggestion Development Evaluation Conclusion Proposal Tentative desing Artifact Performance measures Results Knowledge Flows Process Steps Outputs Circumscription Operation and Goal knowledge Source: Vaishnavi & Kuehler, (2004,2011)
  6. 6. Rigor Cycle Grounding Additions to KB Relevance Cycle Requirements Field testing Application Domain People Orgnizational Systems Technical Systems Problems and Opportunities Environment Design Science Research Knowledge Base Build Design Artifacts Processes Evaluate Foundations Scientific Theories and Methods Experience Expertise Meta-Artifacts Design Products and Design Processes Design Cycle Source: Hevner (2007) Hevner: ” Design science research is essentially pragmatic in nature.” It emphasizes relevance. It makes clear contribution into the application environment.
  7. 7. Theory and models of IS It-artefact Outcomes (Reflection) Hypotheses (Technological rules) Observations (Testing) why, what, how, who, when What might work for whom and why Context-mechanism-outcomes (CMO) Source: Pawson and Tilley(1997) and Kazi (2003), Carlsson (2006) Outputs Description Constructs The conceptual vocabulary of a domain Models A set of propositions or statement expressing relationships between constructs Methods A set of steps to perform a task – how- to knowledge Instantiation The operational of constructs, models and methods. Better theories Artifact construction as analogous to experimental natural science, coupled with reflection and abstraction
  8. 8. Source: van de Ven (2006)
  9. 9. Researchers Consultants Managers Executives Users New theory New practical solution
  10. 10. Guideline Descriptions Design as an artifact Tutkimuksen tavoitteena on sovelluksen, menetelmän tai tietosysteemin suunnittelu. Tarkoituksena ratkaista jokin tietosysteemiin tai sen käyttöön liittyvä ongelma. Problem relevance Suunnittelu perustuu organisaation saamaan hyötyyn (taloudellinen ja toiminnallinen) Desing evaluation Suunnittele, miten artifaktin hyöty ja tarkoitus voidaan arvioida suunnitteluprosessin aikana ja sen jälkeen. Research contribution Tutkimuksen tavoitteena on tuottaa suunittelutieteen käyttöön uutta tietoa: 1) uusi artifakti, 2) suunnittelutieteen perusteita, 3) arviointiin ja arviointimetodeihin liittyvää tietoa. Research rigor Tutkimusote perustuu täsämälliseen metodiin, jota käytetään suunnittelussa ja /tai arvioinnissa. (Tutkimuksen toistettavuus). Design as a search process Suunnitteluprosessi on etsintää, missä pyritään löytämään parempia tapoja tuottaa artifakteja. (iteratiivinen prosessi) Communication of research Tutkimuksen tulokset julkaistaan ja välitetään sekä tutkijoille että käytännön toimijoille (johto, suunnittelijat, käyttäjät). Source: Hevner et al. (2004)
  11. 11. Research Process Focus Contribution Problem identification Relevant research problems Formulation of research problem and questions Research Proposal development Context and scope Research stragegy Answer the questions: why, what, how, who and when Literature Review Current state of theory and application Analysis of focal theory, application and supporting technology Conceptualization Concept discovery and potential empirical generalization Conceptual model, variables and relationships and constraints Experimentation Design the approach and selecting methods and how to demonstrate Confirmation of theoretical conjecture, propositions and hypotheses Source: Steenkamp and McCord (2007)
  12. 12. Needs for new artifact Assump- tions Initiative (First hunch) Requirements and assumptions Structural specifications Protype Implementation Evaluation Evaluation Verschuren and Hartog (2005, p.739 Plan of evaluation 1. Evaluation of goals 2. Evaluation of means 3. Evaluation of relationship between goals and means. Types of evaluation 1. summative 2. formative
  13. 13. Possible Research Entry Points Source: Peffers et al. (2008), Design Science Methodology Process Model (DSRM) Identify problem and motivate Show importance Define objectives of a solution What would be a better artifact to accomplish? Design and development Artifact Demonstration Find suitable context Use artefact to solve problem Evaluation Obserce how effective, efficient Iterate back to desing Communication Scholarly publications Professional publications Problem- centered initiation Objective- centered solution Design & developm ent- centered initiation Client/ context initiated Process iteeration Nominal process sequence Inference Theory How to Knowledge Metrics, Analysis Knowledge Disiplinary knowledge
  14. 14. Design science research approach (Educational) approach Defining features of an application or a process Defining stakeholders’ preferences and requirements Designing research settings and selecting methods Data gathering and analysis Results and discussion Feedback and possible contirubutions
  15. 15. First version project Change projects Requirements and Features Desining and programming Requirements and Features Implementation Usage of apps New version Use Maintenance Source: modified from Järvinen P. (2011)
  16. 16. The searech process The design process Ex-ante evaluation The construction process The artefact Ex-post evaluation Design science Desing research Source: Pries-Heje and Baskerville, (2008)
  17. 17. Identify evaluation problem and select evaluation method Defining evaluation criteria Develop and specify an evaluation method Data gathering and demonstrating evaluation Results New method Improvement Failure Time Knowledge flows Feedback and knowldge flows
  18. 18. 1. Plan the human centred process 2. Specify the context of use 3. Specify user and organizational requirements 4. Produce design solutions 5. Evaluate design against user requirements Feedback
  19. 19. Stages Questions of the research and -paper 1. Title of research proposal What is the main results? 2. Problem area How precise are the claims? 3. Research question How could the outcomes be used? 4. Answer of the research question What is evidence? 5. Reason for this answer How was the evidence gathered? 6. Research situation How were measurements taken? 7. Key concepts that are investigated How carefylly are the algoritms and experiments described? 8. The research design Why is the paper (research) trustworthy? 9. The research results will address the research problem Has the background literature been discussed? What would reproduction of the results involve?Source: Van de Ven (2006)
  20. 20. Design product Definition Description Meta-requirement Goals and main aims of the designed product Describe the class of problems. Meta-design A plan for system to meet meta-requirements Describe the class of artefacts that will meet the meta- requirements. Kernel theories Theories from natural or social sciences that provides a conceptual bridge between meta- requirements and meta- design Are applied to describe conceptual connections between meta-requirements and meta-design. IT-artefacts are used IT- related work systems (Alter). Testable design product hypotheses Propositions to be examined regarding satisfaction of meta- requirements and meta- design How developed or planned artefact can be tested to see if hypotheses can be verified. How meta-design satisfies the meta-requirements.
  21. 21. Design process Definition Description Design method Procedures to be used for construction of system Design method defines the procedures, how artefact is constructed. Kernel theories Theries that support the design method Applied theories from natural or social sciences that used to develop the design process. Testable design process hypotheses Propositions to be examined regarding whether the design method produces an acceptable system Defined hypotheses, which are used to verify, if the designed process is consistent with meta- desing.
  22. 22. Variance Method (Poole et al.(2000) The Process Approach 1. The world is made up of fixed entities with varying attributes 1. The world is made up of entities that participate in events 2. Necessary and sufficient causality is requisite for explanation 2. Necessary causality is requisite for explanation 3. Efficient causalisity is the basis of explanation 3. Final and formal causality, supplemented by efficient causality is the basis for explanation 4. The generality of explanation depends on their ability to apply uniformly 4. The generality of explanation depends on their versality 5. The temporal sequince in which independent variables influence the dependent variables is immaterial to the outcome 5. The temporal sequince of event is critical 6. Explanations should emphasize immediate causation. 6. Explanation should incorporate layers of explantion ranging from immediate to distal 7. Attributes have one and only one causal meaning over the course of time. 7. An entity, attribute, or event may change in meaning over time.
  23. 23. Stages Decision Description 1. The research question and perspective What is the causal conditional proposal or question? Try to take potential users, your own interest and organization’s goal into account. Reflective viewpoint may help to define research questions. 2. Unit of analysis What individual or collective properties are being studied ? Individual, group, organization, technology, it-artefact, process, data. Global, relational or analytical properties. 3. Causal model What is the variance research model? (see e.g. Whetten (2002, Modeling-as-theoryzing: A systematic methodology for theory development, Thousand Oaks, Sage) List variables, draw arrows, list assumptions and boundary conditions. List alternative factors that may be rival explanations. 4. Experimental design Is this a randomized, quasi or non- experimental design? Randomized, quasi-experiment (survey), or non- experiment study (no control-group). 5. Sampe selection and size What criteria are used to select units, constructs, observations & settings? construct validity, external validity. Sample size must meet statistical significance. 6. Measurement How to measure variables? systematic and unsystematic measurement biases 7. Data analysis What data gathering technique will be used? Select technique that fit the research guestions and model. 8. Validity What are the threats to validity of study findings? Statistical conclusion, internal, construct and external validity. Van de Ven: ”A variance research model represents the theory as a causal relationship among variables of units are sampled, measured and analyzed in accordance with expremimental design procedures.”
  24. 24. Phases Decision Research ideas and proposals 1. Meaning of process A category of concepts or developmental sequence? Are researchers interested in concepts or how a process itself can be described and analyzed (narrative, longitudinal). 2.Theories of process Examine one or more models? Select a process model or two models that can be compared. 3.Reflexity Whose viewpoint is featured? Researchers, practitioner’s or user’s viewpoint to consider research outcomes. 4.Mode of inquiry Deductive, inductive or retrodeductive? Is the research based on the model of reality? Is the research based on how it-artefact process is going on at the real time? 5.Observation al method Real-time or historical observations? Is a research process exploring active it-artefact development process? Is a research process observing how it-artefact process was carried out? 6.Source of change Age, cohort or transient sources? Try to develop parallel, synchronic research design. 7.Sample diversity Homogenous or hetereogenous? Try to gather different types of events. 8.Sample size Number of events and cases? Try to find out more than one case or more events inside the one case. 9.Process research designs What data analysis methods to use? Match data analysis methods to number of cases and events.
  25. 25. Task Description Objective setting Traditionally, system objectives have been set by top management and computer technologists. But increasingly other groups are asking or demanding to be involved in this process, in particular, line management, white and blue collar workers and the trade unions. Adaptation The process of adaptation is concerned with moving from one kind of organizational structure and state to another and the means by which this change is assisted to take place smoothly and successfully. It is what normally happens during the implementation phase of a new system. Rapid adaptation does not easily come about of its own accord and there is a need for philosophies, facilities and strategies to assist the process. Integration Technology People Tasks Integration is the action taken, once the system has been designed and is being implemented, to ensure a new situation reaches a state of equilibrium. Stabilization Once integration has been achieved it has to be maintained. To avoid "one man's job enrichment becoming another man's job impoverishment" imaginative solutions or even compromises may be required. The maintenance of a state of equilibrium into the future requires processes for socializing and educating new group members. Source: Mumford (1983)
  26. 26. Category Structure Behaviour Motivation Instantiation Analytical Salient properties and features Process supported Major generic business/ organizational motivations Mock-up prototypes Synthetic Overall logical organization The dynamic behaviour of IS; its working principles Motivations for the type of IS solution as related to requirements Working prototypes that illustrated the concept Technological Refined structural design for the type of IS Refined methods, algoritms, interactions employed by IS Objectives for the design Generig packages, frameworks, shells. Implementation Architecture design of implemented systems Dynamic aspects of implemented systems Specific business motivation Implemented functionin systems Source: Vahidov (2006)
  27. 27. Phases the researcher the practitioner Agreement develop an agreement agree agreement Start of research process create a research plan agree research plan During the real process Develop a new artifact based on requirements and needed features of the artifact. Participate in development process and agree the requirements and features. Demonstration Carry out laboratory test for artifact simulating a reality. Carry out simulation and lab- test for new artifact at the real environment. Evaluation Develop evaluation plan and define needed criteria and combine results to the practitioner’s evaluation results. Carry out evaluation process and give feedback to the researcher. Conclusion Publish what is learned during the research process. Discuss with the researcher concerning the research process. Source: Hevner et al. (2004), March and Smith (1995), Järvinen,(2012)
  28. 28. Research item Questions Design science approach The artefact What is the role of the artefact? Is design of the artefact or improving organizational practice the primary goal of the intervetion? The artefact is central of the research. Collaborators will focus on the artefacts, but success may need some changes to process and organizational practice The process and research cycles How the research process problem going to determined and agreed? Is a predetermined cycle of activity to be followed? Is softwarte development method a necessary part of the activities? Problem must be identified and design criteria may be used to establish potential solutions. The focus of evaluation Can evaluation be a by-product of the research cycle, or must explicit evaluation activites be used? Does user acceptance testing and similar activities have a role to play? Does the artefact require the generation of use cases and some level of validation? (verification?) Evaluation of the designed artefacts is crucial. Evaluation steps are included to the approach. Software testing is vital andis part of the research process. Design as a serch process suggest that use cases and testing are included to the research process.
  29. 29. Research item Questions Design science approach The role of knowledge Is actionable knowledge a realistic goal of the research? To what extent is informing theory and objective? To some extent knowledge emerges from the design. Explicit theory is less prevalent in DS. Design theory is employed as a framework and a reference point for outcomes. The role of learning What expectations are there with regards to learning? Will learning emerge from the design of the artefact or the actions undertaken by the researcher and collaborators? Expectations for learning will exist. Expectations of learning from the artefact will perdominate. Researchers might be surprised by what they learn. Source: Papas et al. (2012)
  30. 30. Semiotic level Evaluation criteria Helfert and Donnellan Pragmatic Relevance, usability, completeness, timeliness, actuality, efficiency. Semantic Precise definitions and terminology, easy to understand, interpreditability, accuracy (free-of error), consistent content. Syntax Consistent and adequate syntax, syntactical correctness, consistent representation, accessability. Source: Helfert M. and Donnellan B. (2012) Ex-ante Ex-post Naturalistic Design process/ product Design process / product Artificial Design process / product Design process / product Source: Pries-Heje and Baskerville (2008) Method Evaluation process Observantional Case or field study Analytical Static analysis, Architecture analysis, Optimization, Dynamic Experimental Controlled experiment Testing Functional (black box) Structural (white box) Descriptive Informed argument Scenarious Souce: Hevner et al. (2004)
  31. 31. Evaluation type Information system in use Invormation system as such Goal-based Data source Evaluator’s role Participators Has the IT-system fulfilled the desired goals? IT-system, goal definitions, requirements, , interaction between users and the it-system. Deductive Evaluator and users, managers Has the IT-system fulfilled the desired goals? What is the contribution? IT-system, goal definitions, requirements. Deductive Evaluator expert Goal-free Data source Evaluator’s role Participators Try to gain a deeper and broader understanding of the IT-systems. IT-system, description of IT-system. Inductive Evaluator expert Try to gain a broader understanding of the IT-systems. IT-system, description of IT-system. Inductive Evaluator expert Criteria-based Data source Evaluator’s role Participators Try to gain a deeper and broader understanding of the IT-system. IT-system, observation, users’ perceptions. Inductive Evaluator expert and user, manager Try to evaluate quality of the IT- system. IT-system, description of IT-system, defined evaluation criteria. Deductive Evaluator expert Source: modified from Cronholm and Goldkuhl (2003)
  32. 32. Evaluation object Relevant evalution questions 1. Design of an artefact Produce the research an it-artefact, that includes a construct, a model or an instantiation. 2. Problem relevance Is the research problem and objective of the it-artefact important? Is the it- artefact or solution relevant business problem? 3. Design evaluation Could the results be verified? Are the evaluation method rigorous and well-defined? Is the utility, quality and efficacy rigorously demonstrated? 4. Research contribution Is there a contribution?Is it significant? Is the contribution timely interest? 5. Research rigor Are the results correct? Are the all technical detail correct? Are they sensible? 6. Design as a search process Are the researchers utilized available means? Do the results satisfy laws on the problem environment? 7. Communication of research Are the appropriate conclusions drawn from the results? Can the paper be understand? Is it clearly written? Is the results presented effectively both to the technology-oriented and management oriented audiences?
  33. 33.  Carlsson S.A. (2006), Towards and Information Systems Design Research Framework: A Critical Realist Perspective, Destrist 2006, February 24-25 2006, Claremont CA.  Crossan M.M. and M. Apaydin (2010), A Multi-Dimensional Framework of Organizational Innovation: A Systematic Review of the Literature, Journal of Management Studies 47, No 6, 1154-1191.  Hevner A.R. (2007), Three Cycle View of Design Science Research, Scandinavian Journal of Information Systems, Vol. 19, No.2, pp. 87-92.  Hevner A.R., March S.T., Park J, and Ram S. (2004) Design science in information systems research, MIS Quarterly, Vol. 28, Issue 1, March 2004.  Järvinen A & P. (2011) Tutkimustyön metodeista, Opinpajan kirja, Tampere.  Järvinen P. (2012), On boundaries between field experiment, action research and design research, University of Tampere, Shcool of Infomation Sciences, Reports in Information Sciences 14, Tampere.  Iivari J. (2005), Information Systems as a design science, Information Systems Development: Adnvances in Theory, Practice and Education, Edited bgy O. Vasilegas et atl. Springer.  Papas N., O´Keefe R.M. and Seltsikas P. (2012), The action research vs design research science depate: reflections from an intervention in eGovernment, European Journal of Information Systems, Vol. 21, pp- 147-159.  Peffers K., Tuunanen T., Gengler C.E., Rossi M., Hui W., Virtanen V. and Bragge J.(2006), The Design Science Research Process: A model for producing and presenting information systems research, Destrist 2006, February 24-25 2006, Claremont, CA.  Vahidov R. (2006), Design Researcher’s IS Artifact: a Representational Framework, Destrit 2006, February 24-25 2006 Clamenont CA (CGU2006).  Vaishnavi V. and Kuechler W. (2004), Design Science Research in Information Systems, research-in-information-systems/, last updated September 2011  van Aken Joan E. (2001), Management Research based on the paradigm of the Design Sciences; The Quest for tested and grounded technological rules.  Van de Ven (2007), Engaged Scolarship: A Guide for Organizational and Social Research, Oxword University Press Inc., New York  Venable J.R. (2006), The Role of Theory and Theorising in Design Science Research, Destrist 2006, Proceeding/2A_1pdf.  Verschuren P. and Hartog R. (2005), Evaluation in Design-Oriented Research, Quality & Quantitity, Vol. 39, pp. 733-762.  Zobel J. (2004), Writing for computer science second edition, Springer,-Verlag, London Limited.