Research methodology


Published on


Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Research methodology

  1. 1. Example.. Let ’s publish a paper f or t his nice Bahamas conf er ence… What about a hashing f unct ion f or sear ching lar ge image dat abases (int er net -> $ $ $ $ $ !) Research methods and Ef f icient hashing f or lar ge image dat abases methodology 1. Intro (there are so many images!) •Research methodology? 2. The Algorithm (pseudo-code) 3. Example (it works) •Objective? Hans Weigand 4. Conclusions (it really works!) •Related research? Univ of Tilburg •Validation? Nov 2006 Hans Weigand 1 Nov 2006 Hans Weigand 2 Knowledge and scientific Science fiction and fact knowledge • Objective knowledge • Opinion, taste, hype • Repeatable experiments • handwaiving intuitive knowledge • Valid reasoning • rhetoric knowledge of the • Unbiased • university politics scientific knowledge concrete • Revealing • dull, uninteresting • Researcher as the shy • researchers highly religious knowledge and cooperative genius competitive and over-self- confident Focuses on - the abstract - using logical reasoning Nov 2006 Hans Weigand 3 Nov 2006 Hans Weigand - acquired methodologically 4 Course overview Overview of this introduction • Introduction (Hans Weigand) • Basic research methodology (Verschuren/Doorewaard) • Philosophy of empirical sciences (Richard Starmans) • Philosophy of formal sciences (John-Jules Meyer) • What is IS/CS research? Wieringa, March • Research design (Hans Weigand, Hans Heerkens) • Research paradigms in IS • Research methods (Hans Akkermans) • Research methods in • Research as communication – Agent Systems (Catholijn Jonker) – Machine Learning (Antal vd Bosch) – IR (Djoerd Hiemstra) • Examples from two Ph.D. students • How to write an article (Hans Akkermans) Nov 2006 Hans Weigand 5 Nov 2006 Hans Weigand 6 1
  2. 2. Research Methodology Empirical cycle • Types of research • Hypothesis • Conceptual research design • Experiment – Research objective • Evaluation – Research questions – Definition of concepts • Research planning “is it true? (and why?) Nov 2006 Hans Weigand 7 Nov 2006 Hans Weigand 8 Design cycle H. Simon, The Sciences of the Artificial • Design • Simulation/implementation The engineer, and more generally the designer, is concerned with how things ought to be - how they ought to be in order to attain goals, and to • Evaluation function … With goals and "oughts" we also introduce into the picture the dichotomy between normative and descriptive. Natural science has “is it possible?” (and how?) found a way to exclude the normative and to concern itself solely with how things are … Artificial things can be characterized in terms of functions, goals and adaptation. Note: In CS, “implementation” can range from model to Research methods: simulation and decomposition proof-of-concept prototype to full implementation Nov 2006 Hans Weigand 9 Nov 2006 Hans Weigand 10 Theory/practice oriented Research quadrant • Theory-oriented: develop or test theory – Involves experimentation empirical physics, political poll sociology • Practice-oriented: solve a problem – Involves intervention. “develop incremental “develop web-site” design NL parser” theory-oriented practice-oriented Nov 2006 Hans Weigand 11 Nov 2006 Hans Weigand 12 2
  3. 3. Research Methodology Research objective • Start exploring the project context • Types of research • Formulate research objective • Conceptual research design – useful (added-value) – Research objective – feasible – Research framework – clear – Research questions • The objective is to … by … – Definition of concepts • Research planning your contribution your overall result Nov 2006 Hans Weigand 13 Nov 2006 Hans Weigand 14 Theory oriented Practice-oriented • Theory development • Problem finding • Theory testing (cf. Glaser & Strauss) • Diagnosis • Design • Intervention Different kinds of research A theory is more than accumulated knowledge, it provides a and research questions • Evaluation perspective (abstraction, interpretation) and is explanatory/ predictive Nov 2006 Hans Weigand 15 Nov 2006 Hans Weigand 16 Research framework Research framework visualized (example master project) • Specify short objective Interviews with consultants • Determine the object Proposal evolutionary • Establish the nature of the research design method A/D method for • Determine ingredients Theory of IS design evolutionary design methods • Visualize Application of method (case study) NB: often, a research project combines several research types Nov 2006 Hans Weigand 17 Nov 2006 Hans Weigand 18 3
  4. 4. Example: vd Heuvel (2002) Research questions “a methodology for objectifying legacy systems” 1. Problem definition Explorative, desk research • The questions that you state yourself for getting at your objective 2. Related research Literature review • Main question/subquestions 3. Design of solution Meta-modeling • Requirements: 4. Validation of - Logical consistency metamodel – Effectiveness (do you reach your objective?) proposed solution - Implementability prototype – Efficiency (balance between goal and means) - Plausibility field experiment 5. Assessment of – Steering function • What type of knowledge is required? research results • What material needs to be gathered? Nov 2006 Hans Weigand 19 Nov 2006 Hans Weigand 20 How to get at research questions? Research Methodology • By exploring the research framework • Types of research • By unravelling key concepts • Conceptual research design • Use knowledge hierarchy: – Research objective prescriptive – Research framework evaluative – Research questions What is the best? predictive – Definition of concepts (tomorrow) Is A better then B? explanatory Is it possible? • Research planning descriptive Nov 2006 Hans Weigand 21 Nov 2006 Hans Weigand 22 Research planning : HOW? Research objective • Research material – People, documents, observation research research research – Data sources, knowledge sources question I question II question III • Research strategy – Survey, experiment, case study, grounded theory approach, desk research, … r esear ch met hod A r esear ch met hod B (e.g. case st udy) (e.g. mat hemat ical • Time planning pr oof ) r esear ch met hod C (e.g. exper iment ing) Arrows represent your choices Nov 2006 Hans Weigand 23 Nov 2006 Hans Weigand 24 4
  5. 5. Concluding remarks on research What is IS/CS research? methodology • Research needs preparation (GIGO) • Be aware of the choices that you make Is CS a science? • During execution, the framework should always indicate the purpose of what you are IS CS an engineering discipline? doing at that moment. Is it a formal science, branch of mathematics? • As research is a creative process, adapting the framework on the way is natural and should always be possible. Nov 2006 Hans Weigand 25 Nov 2006 Hans Weigand 26 Wieringa/Heerkens March/Smith • Characterizes CS as design science • CS/IS research should integrate design and • Design is aimed at solving some problem (action “natural science research” problem, world problem) • Design is aimed at creating things • The design itself is not research (contra Popper), • Basic design activities: build and evaluate but the other steps in the intervention cycle give rise to knowledge problems that can be addressed • Natural science is aimed at developing theory – by scientific research not restricted to natural phenomena, can also apply • Sometimes solving a knowledge problem requires to artifacts (contra Simon) solving an action problem • Basic science activities: theorize and justify Nov 2006 Hans Weigand 27 Nov 2006 Hans Weigand 28 Theory and Action March/Smith research framework in balance build evaluat e t heor ize j ust if y theory practice const r uct s knowledge problem knowledge problem knowledge problem model theory problem world problem met hod -lack of t heor y -lack of validit y action problem action problem inst ant iat ion -f alsif icat ion at t empt (wor king pr ogr am) -anomaly Nov 2006 Hans Weigand 29 Nov 2006 Hans Weigand 30 5
  6. 6. Ramesh et al Why mathematics? • Based on empirical study of publications • Conceptual clarity – like precise definitions in Law • Focus in CS is on formulating (80%) rather – Could perhaps be reached also by UML diagram than describing or evaluating • As a research method • Research methods: mainly conceptual/ – to validate certain claims mathematical • For the development of a (predictive) theory • Low reference to other discipline (some – not really different from e.g. game theory in Economics mathematics) – Example: relational algebra, complexity theory, .. Nov 2006 Hans Weigand 31 Nov 2006 Hans Weigand 32 Research paradigms in IS Interpretivism • Positivism (Comte) and rationalism (Descartes) • Post-modernism (Foucault, Derrida), cf. The aim of interpretive research is to understand how members of a social community, through their participation in interpretivism, constructivism, … social processes, enact their particular realities and endow • Pragmatism (Peirce, Dewey, Rescher) them meaning, and to show how these meanings, beliefs and • Critical theory (Habermas) intentions of the members help to constitute their social action (Orlowski & Baroudi, 1991) Cf debate between rationalist AI tradition (Schank, Stefik/Bobrow) and Winograd & Flores, to which also Suchman contributed. Nov 2006 Hans Weigand 33 Nov 2006 Hans Weigand 34 Pragmatism Not in the sense of opportunism, but concerned with Critical theory action rather than with being Ch. Peirce “There is no such thing as genuine knowledge and fruitful • Knowledge is not neutral - be critical about understanding except as the offspring of doing… Only by assumptions such as the technical wrestling with the conditions of het problem at first hand, imperative to improve efficiency seeking and finding his own way, does he think“ John Dewey • Knowledge must be legitimated by “Thinking is not different in kind from the use of natural materials and energies ..” consensus and democratic discourse Knowledge of a thing is knowledge of what you can do with it. • Knowledge is not neutral – science should Tries to bridge empirical science and design science. serve social goals such as emancipation Nov 2006 Hans Weigand 35 Nov 2006 Hans Weigand 36 6
  7. 7. Research as communication So … • Communicate! (publish, discuss, review,…) • Researchers communicate via papers, • Be clear! (Grice’s conversational maxims) presentations, discussions • Take communication partners seriously! • In communicating, they make descriptive – Read related research – Try to understand and normative statements (validity claims) – Refer • These claims may be challenged by others, – Consider their goals and expectations leading to discussion • Don’t “drop an idea”; try to convince! • Discussion is as essential as research • Justify your claims! activities itself Nov 2006 Hans Weigand 37 Nov 2006 Hans Weigand 38 Maxim of Quantity: 1. Make your contribution to the conversation as informative as necessary. How to justify your claims? 2. Do not make your contribution to the conversation more informative than necessary. Maxim of Quality: "Ik ben er nog steeds van overtuigd dat 1. Do not say what you believe to be false. zoals ik het doe je het moet doen want 2. Do not say that for which you lack adequate evidence. anders zou ik het niet doen." Maxim of Relevance: Be relevant (i.e., say things related to the current topic of the conversation). Maxim of Manner: 1. Avoid obscurity of expression. 2. Avoid ambiguity. 3. Be brief (avoid unnecessary wordiness). 4. Be orderly. Nov 2006 Hans Weigand 39 Nov 2006 Hans Weigand 40 Literature How to justify your claims? • Designing a research project – P. Verschuren, H. Doorewaard. Utrecht, 1999. • Grounding (empirical, simulation, …) • Desiging Information Technology in the postmodern age – R. Coyne, 1997 • Careful reasoning • Qualitative methods in management research – E. Gummesson • Examples (not just for explanation, but to put your claim to the test) • The sciences of the artificial. H. Simon, 1967 • Delimitation • Research in Computer Science: an empirical study -Ramesh et al, 2004 • Be clear about the status of your claim. • Design and natural science research on information technology - March, Smith, 1995 Nov 2006 Hans Weigand 41 Nov 2006 Hans Weigand 42 7