Let ’s publish a paper f or t his nice Bahamas conf er ence…
What about a hashing f unct ion f or sear ching lar ge
image dat abases (int er net -> $ $ $ $ $ !)
Research methods and
Ef f icient hashing f or lar ge image dat abases
1. Intro (there are so many images!)
2. The Algorithm (pseudo-code)
3. Example (it works) •Objective?
Hans Weigand 4. Conclusions (it really works!)
Univ of Tilburg •Validation?
Nov 2006 Hans Weigand 1 Nov 2006 Hans Weigand 2
Knowledge and scientific
Science fiction and fact
• Objective knowledge • Opinion, taste, hype
• Repeatable experiments • handwaiving intuitive knowledge
• Valid reasoning • rhetoric
knowledge of the
• Unbiased • university politics scientific knowledge
• Revealing • dull, uninteresting
• Researcher as the shy • researchers highly religious knowledge
and cooperative genius competitive and over-self-
- the abstract
- using logical reasoning
Nov 2006 Hans Weigand 3 Nov 2006 Hans Weigand
- acquired methodologically
Course overview Overview of this introduction
• Introduction (Hans Weigand) • Basic research methodology (Verschuren/Doorewaard)
• Philosophy of empirical sciences (Richard Starmans)
• Philosophy of formal sciences (John-Jules Meyer)
• What is IS/CS research? Wieringa, March
• Research design (Hans Weigand, Hans Heerkens) • Research paradigms in IS
• Research methods (Hans Akkermans)
• Research methods in
• Research as communication
– Agent Systems (Catholijn Jonker)
– Machine Learning (Antal vd Bosch)
– IR (Djoerd Hiemstra)
• Examples from two Ph.D. students
• How to write an article (Hans Akkermans)
Nov 2006 Hans Weigand 5 Nov 2006 Hans Weigand 6
Research Methodology Empirical cycle
• Types of research • Hypothesis
• Conceptual research design • Experiment
– Research objective • Evaluation
– Research questions
– Definition of concepts
• Research planning “is it true? (and why?)
Nov 2006 Hans Weigand 7 Nov 2006 Hans Weigand 8
Design cycle H. Simon, The Sciences of the Artificial
• Simulation/implementation The engineer, and more generally the designer, is concerned with how
things ought to be - how they ought to be in order to attain goals, and to
• Evaluation function … With goals and "oughts" we also introduce into the picture
the dichotomy between normative and descriptive. Natural science has
“is it possible?” (and how?) found a way to exclude the normative and to concern itself solely with
how things are … Artificial things can be characterized in terms of
functions, goals and adaptation.
Note: In CS, “implementation” can range from model to Research methods: simulation and decomposition
proof-of-concept prototype to full implementation
Nov 2006 Hans Weigand 9 Nov 2006 Hans Weigand 10
Theory/practice oriented Research quadrant
• Theory-oriented: develop or test theory
– Involves experimentation empirical physics, political poll
• Practice-oriented: solve a problem
– Involves intervention.
“develop incremental “develop web-site”
design NL parser”
Nov 2006 Hans Weigand 11 Nov 2006 Hans Weigand 12
Research Methodology Research objective
• Start exploring the project context
• Types of research
• Formulate research objective
• Conceptual research design
– useful (added-value)
– Research objective
– Research framework
– Research questions
• The objective is to … by …
– Definition of concepts
• Research planning your contribution your overall result
Nov 2006 Hans Weigand 13 Nov 2006 Hans Weigand 14
Theory oriented Practice-oriented
• Theory development • Problem finding
• Theory testing (cf. Glaser & Strauss) • Diagnosis
• Intervention Different kinds of research
A theory is more than accumulated knowledge, it provides a and research questions
perspective (abstraction, interpretation) and is explanatory/
Nov 2006 Hans Weigand 15 Nov 2006 Hans Weigand 16
Research framework Research framework visualized
(example master project)
• Specify short objective Interviews with
• Determine the object
• Establish the nature of the research design method
A/D method for
• Determine ingredients Theory of IS design evolutionary design
Application of method
NB: often, a research project combines several research types
Nov 2006 Hans Weigand 17 Nov 2006 Hans Weigand 18
Example: vd Heuvel (2002) Research questions
“a methodology for objectifying legacy systems”
1. Problem definition Explorative, desk research • The questions that you state yourself for getting at
2. Related research Literature review
• Main question/subquestions
3. Design of solution Meta-modeling
4. Validation of - Logical consistency metamodel
– Effectiveness (do you reach your objective?)
proposed solution - Implementability prototype
– Efficiency (balance between goal and means)
- Plausibility field experiment
5. Assessment of – Steering function
• What type of knowledge is required?
research results • What material needs to be gathered?
Nov 2006 Hans Weigand 19 Nov 2006 Hans Weigand 20
How to get at research questions? Research Methodology
• By exploring the research framework
• Types of research
• By unravelling key concepts
• Conceptual research design
• Use knowledge hierarchy: – Research objective
prescriptive – Research framework
evaluative – Research questions
What is the best? predictive – Definition of concepts (tomorrow)
Is A better then B? explanatory
Is it possible? • Research planning
Nov 2006 Hans Weigand 21 Nov 2006 Hans Weigand 22
Research planning : HOW? Research objective
• Research material
– People, documents, observation research research research
– Data sources, knowledge sources question I question II question III
• Research strategy
– Survey, experiment, case study, grounded
theory approach, desk research, … r esear ch met hod A r esear ch met hod B
(e.g. case st udy) (e.g. mat hemat ical
• Time planning pr oof ) r esear ch met hod C
(e.g. exper iment ing)
Arrows represent your choices
Nov 2006 Hans Weigand 23 Nov 2006 Hans Weigand 24
Concluding remarks on research
What is IS/CS research?
• Research needs preparation (GIGO)
• Be aware of the choices that you make Is CS a science?
• During execution, the framework should
always indicate the purpose of what you are IS CS an engineering discipline?
doing at that moment. Is it a formal science, branch of mathematics?
• As research is a creative process, adapting
the framework on the way is natural and
should always be possible.
Nov 2006 Hans Weigand 25 Nov 2006 Hans Weigand 26
• Characterizes CS as design science • CS/IS research should integrate design and
• Design is aimed at solving some problem (action “natural science research”
problem, world problem) • Design is aimed at creating things
• The design itself is not research (contra Popper), • Basic design activities: build and evaluate
but the other steps in the intervention cycle give
rise to knowledge problems that can be addressed • Natural science is aimed at developing theory –
by scientific research not restricted to natural phenomena, can also apply
• Sometimes solving a knowledge problem requires to artifacts (contra Simon)
solving an action problem • Basic science activities: theorize and justify
Nov 2006 Hans Weigand 27 Nov 2006 Hans Weigand 28
Theory and Action
March/Smith research framework
build evaluat e t heor ize j ust if y
const r uct s knowledge problem
theory problem world problem
-lack of t heor y
-lack of validit y action problem
inst ant iat ion -f alsif icat ion at t empt
(wor king pr ogr am) -anomaly
Nov 2006 Hans Weigand 29 Nov 2006 Hans Weigand 30
Ramesh et al Why mathematics?
• Based on empirical study of publications • Conceptual clarity
– like precise definitions in Law
• Focus in CS is on formulating (80%) rather
– Could perhaps be reached also by UML diagram
than describing or evaluating
• As a research method
• Research methods: mainly conceptual/ – to validate certain claims
mathematical • For the development of a (predictive) theory
• Low reference to other discipline (some – not really different from e.g. game theory in Economics
mathematics) – Example: relational algebra, complexity theory, ..
Nov 2006 Hans Weigand 31 Nov 2006 Hans Weigand 32
Research paradigms in IS Interpretivism
• Positivism (Comte) and rationalism (Descartes)
• Post-modernism (Foucault, Derrida), cf. The aim of interpretive research is to understand how
members of a social community, through their participation in
interpretivism, constructivism, …
social processes, enact their particular realities and endow
• Pragmatism (Peirce, Dewey, Rescher) them meaning, and to show how these meanings, beliefs and
• Critical theory (Habermas) intentions of the members help to constitute their social action
(Orlowski & Baroudi, 1991)
Cf debate between rationalist AI tradition (Schank, Stefik/Bobrow) and
Winograd & Flores, to which also Suchman contributed.
Nov 2006 Hans Weigand 33 Nov 2006 Hans Weigand 34
Not in the sense of opportunism, but concerned with
action rather than with being Ch. Peirce
“There is no such thing as genuine knowledge and fruitful • Knowledge is not neutral - be critical about
understanding except as the offspring of doing… Only by assumptions such as the technical
wrestling with the conditions of het problem at first hand, imperative to improve efficiency
seeking and finding his own way, does he think“
John Dewey • Knowledge must be legitimated by
“Thinking is not different in kind from the use of natural
materials and energies ..” consensus and democratic discourse
Knowledge of a thing is knowledge of what you can do with it. • Knowledge is not neutral – science should
Tries to bridge empirical science and design science. serve social goals such as emancipation
Nov 2006 Hans Weigand 35 Nov 2006 Hans Weigand 36
Research as communication So …
• Communicate! (publish, discuss, review,…)
• Researchers communicate via papers, • Be clear! (Grice’s conversational maxims)
presentations, discussions • Take communication partners seriously!
• In communicating, they make descriptive – Read related research
– Try to understand
and normative statements (validity claims)
• These claims may be challenged by others, – Consider their goals and expectations
leading to discussion • Don’t “drop an idea”; try to convince!
• Discussion is as essential as research • Justify your claims!
Nov 2006 Hans Weigand 37 Nov 2006 Hans Weigand 38
Maxim of Quantity:
1. Make your contribution to the conversation as informative as necessary. How to justify your claims?
2. Do not make your contribution to the conversation more informative than
Maxim of Quality:
"Ik ben er nog steeds van overtuigd dat
1. Do not say what you believe to be false.
zoals ik het doe je het moet doen want
2. Do not say that for which you lack adequate evidence.
anders zou ik het niet doen."
Maxim of Relevance:
Be relevant (i.e., say things related to the current topic of the conversation).
Maxim of Manner:
1. Avoid obscurity of expression.
2. Avoid ambiguity.
3. Be brief (avoid unnecessary wordiness).
4. Be orderly.
Nov 2006 Hans Weigand 39 Nov 2006 Hans Weigand 40
How to justify your claims? • Designing a research project – P. Verschuren, H.
Doorewaard. Utrecht, 1999.
• Grounding (empirical, simulation, …) • Desiging Information Technology in the postmodern age – R.
• Careful reasoning • Qualitative methods in management research – E.
• Examples (not just for explanation, but to put your claim to the test)
• The sciences of the artificial. H. Simon, 1967
• Delimitation • Research in Computer Science: an empirical study
-Ramesh et al, 2004
• Be clear about the status of your claim. • Design and natural science research on information
technology - March, Smith, 1995
Nov 2006 Hans Weigand 41 Nov 2006 Hans Weigand 42