SOME PROBLEMS RELATED TO ACCUMULATION OF KNOWLEDGE IN CONTEMPORARY SCIENTIFIC PRACTICE. HUMBOLDT-KOLLEG WISSENSCHAFT UND BILDUNG IM WANDEL/ SCIENCE AND EDUCATION IN TRANSITION Belgrade, October 28-30, 2010.

SCIENCE AND KNOWLEDGE
MANAGEMENT:
SOME PROBLEMS RELATED TO ACCUMULATION OF
KNOWLEDGE IN CONTEMPORARY SCIENTIFIC PRACTICE
Goran S. Milovanović
PhD Candidate
Department of Psychology
Faculty of Philosophy
University of Belgrade
HUMBOLDT-KOLLEG
WISSENSCHAFT UND BILDUNG IM WANDEL/ SCIENCE AND EDUCATION IN TRANSITION
Belgrade, October 28-30, 2010.
1

HUMBOLDT-KOLLEG
THE ACCUMULATION OF
KNOWLEDGE
2

HUMBOLDT-KOLLEG
THE SCIENTIFIC PRODUCTION:
POWER LAW FOR CITATIONS
3
1 or 2 references (~30%)
Less than 10% receives
100 or more references
Note: data set is for ilustrative purposes only. Source: Google Scholar, 17. October 2010.
Citations
Frequency

HUMBOLDT-KOLLEG
SCIENCE AND KNOWLEDGE MANAGEMENT:
“AN ENGINEERING APPROACH” TO SCIENTIFIC KNOWLEDGE
How does this affect our “perception of science”: are we really able to manage all this information
effectively?
HOW MANY RESEARCH PAPERS ARE YOU ABLE TO READ IN THE COURSE OF ONE WORKING WEEK?
HOW MANY OF THESE PAPERS WILL YOU CONSIDER RELEVANT?
HOW MANY WILL YOU CITE?
4

HUMBOLDT-KOLLEG
INCOMEASUREABLE
METHODOLOGIES AS SOURCES OF
INCREASED PRODUCTION
5

HUMBOLDT-KOLLEG
INCOMEASUREABLE METHODOLOGIES AND EMPIRICAL GROUNDING
6
T h e o r y A
C1A
Theoretical Concepts of A:
C2A C3A
Empirical scopes of the concepts of the Theory A
ES(C1A) ES(C2A) ES(C3A)
Note: the illustration presents a rude simplification, obviously.

HUMBOLDT-KOLLEG
7
C1A
Empirical scope of C1A
ES(C1A)
e1 e2 e3 e4 e5 e6 e7 e8
m1 m2 m8. . . Methods used to empirically
ground C1A in ES(C1A)
Empirical phenomena:
ES(C1A) is the manifestation
set of C1A.
Concept

HUMBOLDT-KOLLEG
8
C1A
ES(C1A)
e1 e2 e3
m1
m2
m3
C1A
ES(C1A)
e1 e2 e3
mA
mB
mC
Your Lab My Lab
PROBLEM!
When we communicate on C1A,
do we report on the same
phenomenon?

HUMBOLDT-KOLLEG
9
IT HAPPENS ALL THE TIME.

HUMBOLDT-KOLLEG
CASE STUDY:
MODEL SELECTION IN
CHOICE UNDER RISK
10

HUMBOLDT-KOLLEG
MODELING CHOICE UNDER RISK
11
Ticket A Ticket B
win EUR 50
with 25%
win EUR 25
with 75%
win EUR 500
with 12%
lose EUR 50
with 88%
What do you prefer: A or B?

HUMBOLDT-KOLLEG
NORMATIVE THEORY (axiomatized, von Neumann & Morgnestern, 1948, following Bernoulli’s early
analysis of choice under risk from 1738).
PROBLEM: Normative theory does not predict all choices correctly – there are systematic behavioral
violations from normative predictions; anomalies of rational choice).
Consequence: the development of ...
BEHAVIORAL THEORIES (many different models of choice under risk; most famous and most
commonly used is Tversky and Kahneman’s Prospect Theory, Kahneman & Tversky, 1979, Tversky &
Kahneman, 1992).
12

HUMBOLDT-KOLLEG
Two methods are prevalent in model selection in this field:
DETERMINATION OF CERTAINTY EQUIVALENTS (CE) OF RISKY GAMBLES
• Ask a participant to determine a certain amount of value that he or she would
accept in exchange for the offered risky ticket.
13
win EUR 50
with 25%
win EUR 25
with 75%
EUR 30
Accept

HUMBOLDT-KOLLEG
Two methods are prevalent in model selection in this field:
14
Ticket A Ticket B
win EUR 50
with 25%
win EUR 25
with 75%
win EUR 500
with 12%
lose EUR 50
with 88%
THEORIES OF CHOICE UNDER RISK MUST BE ABLE TO PREDICT
BOTH CE AND CHOICE DATA
CHOICE EXPERIMENTS

HUMBOLDT-KOLLEG
Model selection in this field: direct hypothesis testing based on experimental data is very
rare.
1. Tversky & Kahneman (1992): CE dataset, estimation of their model (Cummulative
Prospect Theory); no direct comparisons against Expected Utiliy (or any other) model.
This CE dataset exhibits properties that cannot be explained in the normative
framework of Expected Utility.
15

HUMBOLDT-KOLLEG
rare.
2. Gonzales & Wu (1999): CE dataset, estimation of the Prospect Theory model; no direct
comparisons with other models.
This CE dataset exhibits properties that cannot be explained in the normative
framework of Expected Utility.
16

HUMBOLDT-KOLLEG
rare.
3. Hay & Orme (1994): Choice dataset, 100 pairwise choices; likelihood ratio tests -
direct comparison of Expected Utility against various alternatives.
Conclusion: there is no superiority over Expected Utility. 39% of subjects are best
fitted by Expected Utility model; in 61% of cases, other models fit better.
No single behavioral model is consistently superior to normative theory.
17

HUMBOLDT-KOLLEG
rare.
4. Harrison & Rutstrom (2006): Choice dataset, join estimation of Expected Utility and
Prospect Theory, including modeling of the proportion of choices better explained by
the former or the later theory.
Conclusion: there is no representative cognitive agent for a single theory; some choices
are better explained by Expected Utility while others are better explained by Prospect
Theory.
18

HUMBOLDT-KOLLEG
HOW MANY THEORIES OF CHOICE UNDER RISK?
Different methods (that were expected to provide convergent results) resulted in different
conclusions.
In the meanwhile, the number of theories of choice under risk keeps on growing.
How many explanations? Here we go:
Prospect Theory Yarri’s Dual Model
Cummulative Prospect Theory Subjective Expected Utility Theory
Dissapointment Aversion Theory Prospective Reference Theory
Quadratic Utility Regret Theory (w/wo independence)
Weighted Utility Theory TAX Model
Perceived Relative Argument Model
There’s more on the market!
19

HUMBOLDT-KOLLEG
HOW MANY EMPIRICAL PHENOMENA TO EXPLAIN IN CHOICE UNDER RISK?
While the number of theories and models increases...
Birnbaum, 2008 (Psych.Review): There are 11 (eleven) new paradoxes that the normative
theory (Expected Utility) cannot explain.
Guess what happens with the number of publications in this field...
20

HUMBOLDT-KOLLEG
WHAT IS HAPPENING IN THIS
FIELD?
21

HUMBOLDT-KOLLEG
INCOMEASUREABLE
METHODOLOGIES
AND
HYPOTHESIS TESTING
22

HUMBOLDT-KOLLEG
INCOMEASURABLE METHODOLOGIES
The problem seems obvious: full hypothesis testing procedures are very rarely employed in
this field.
CONSEQUENCES:
• The establishment of empirical phenomena that are not properly positioned in the
context of other phenomena.
• The establishment of many models of choice under risk that sometimes work better
and sometimes worse, probably as a function of what particular experimental design
was used to test them.
23

HUMBOLDT-KOLLEG
All models of choice under risk are “naturally” expected to predict both CE and CHOICE
data.
CE datasets  most often exhibit properties that are not consistent with the normative
theory.
CHOICE datasets  most often support the normative theory or do not allow for
empirical demarcation of models.
How is this possible?
24

HUMBOLDT-KOLLEG
A. INADEQUATE SAMPLING OF EXPERIMENTAL DESIGNS
Experimental designs in CE/CHOICE methodologies combine:
Levels of VALUE: 10, 20, 50, 100, 125, 150 EUR etc.
Levels of PROBABILITY: .01 .05 .10 .25 .50 .75 .95 .99 etc.
1. The choice of levels on both factors in CE/CHOICE designs is arbitrary. No one
provides a clear motivation for the design they use!
2. Experimental designs are very complicated. The number of all possible combinations
of values and probabilities is huge. An experiment encompassing all possible choices is
practically impossible.
3. Why there are no attempts to sample the universe of stimuli properly and work with
random designs involving a large number of subjects?
25

HUMBOLDT-KOLLEG
B. THE STORY OF A COMPLEX MODEL MAKING SPECIFIC PREDICTIONS
1. One develops a complex model of a (prima facia simple) cognitive process.
2. Then, one’s model makes a very specific empirical prediction.
3. Then, one tests that specific prediction in an experiment specifically designed to test that
prediction and gains empirical support for his/her model.
That empirical finding and the method used to establish it remain without context in a set
of all possible experimental designs and their possible outcomes.
26

HUMBOLDT-KOLLEG
27
C1
ES(C1A)
e1 e2
Your Lab
My Lab
This can go on forever.
C1m1
mA mB
m2
Your publication
Our publication

HUMBOLDT-KOLLEG
28
SOCIAL, ECONOMIC AND
PSYCHOLOGICAL PHENOMENA ARE
VERY COMPLICATED, SAMPLING
FROM COMPLEX DESIGNS IS VERY
DIFFICULT ETC, ETC.

HUMBOLDT-KOLLEG
29
YES, I KNOW IT’S
DIFFICULT.

HUMBOLDT-KOLLEG
30
THAT IS WHAT SCIENCE IS
ABOUT: SOLVING
NOTORIOUSLY DIFFICULT
PROBLEMS.

HUMBOLDT-KOLLEG
31
The golden plate attached to the Voyager space probe.
You know, the E.T.s might find it one day.

THANK YOU FOR YOUR
ATTENTION!
HUMBOLDT-KOLLEG
32
SCIENCE AND KNOWLEDGE MANAGEMENT:
SOME PROBLEMS RELATED TO ACCUMULATION OF KNOWLEDGE IN
CONTEMPORARY SCIENTIFIC PRACTICE
Goran S. Milovanović
University of Belgrade
Contact: goran.s.milovanovic@gmail.com
Phone: +381 69 16 86 298

SOME PROBLEMS RELATED TO ACCUMULATION OF KNOWLEDGE IN CONTEMPORARY SCIENTIFIC PRACTICE. HUMBOLDT-KOLLEG WISSENSCHAFT UND BILDUNG IM WANDEL/ SCIENCE AND EDUCATION IN TRANSITION Belgrade, October 28-30, 2010.

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

More from Goran S. Milovanovic

More from Goran S. Milovanovic (18)

Recently uploaded

Recently uploaded (20)

SOME PROBLEMS RELATED TO ACCUMULATION OF KNOWLEDGE IN CONTEMPORARY SCIENTIFIC PRACTICE. HUMBOLDT-KOLLEG WISSENSCHAFT UND BILDUNG IM WANDEL/ SCIENCE AND EDUCATION IN TRANSITION Belgrade, October 28-30, 2010.