1.We don’t necessarily know ‘what works’ – “confident predictions about policy made by experts often turn out to be incorrect. RCTs have demonstrated that interventions which were designed to be effective were in fact not” 2. RCTs don’t have to cost a lot of money – “The costs of an RCT depend on how it is designed: with planning, they can be cheaper than other forms of evaluation.” 3. There are ethical advantages to using RCTs – “Sometimes people object to RCTs in public policy on the grounds that it is unethical to withhold a new intervention from people who could benefit from it.” “If anything, a phased introduction in the context of an RCT is more ethical, because it generates new high quality information that may help to demonstrate that an intervention is cost effective.” 4. RCTs do not have to be complicated or difficult to run – “It is much more efficient to put a smaller amount of effort [than a post-intervention impact evaluation] into the design of an RCT before a policy is implemented.”

1. T H E B E AU T Y O F A R A N D O M I Z E D T R I A L I S T H AT T H E R E S E A RC H E R
D O E S N OT N E E D TO U N D E R S TA N D A L L T H E FA C TO R S T H AT
I N F LU E N C E O U TC O M E S . S AY T H AT A N U N D I S C OV E R E D G E N E T I C
VA R I AT I O N M A K E S C E R TA I N P E O P L E U N R E S P O N S I V E TO
M E D I C AT I O N . T H E R A N D O M I Z I N G P RO C E S S W I L L E N S U R E — O R
M A K E I T H I G H LY P RO BA B L E — T H AT T H E A R M S O F T H E T R I A L
C O N TA I N E Q UA L N U M B E R S O F S U B J E C T S W I T H T H AT VA R I AT I O N .
T H E R E S U LT W I L L B E A FA I R T E S T. K R A M E R ( 2 0 1 6 )

2. RANDOM
Statisticians and economists use randomization because when
enough people are randomly chosen to participate in a survey,
conveniently, the characteristics of those chosen individuals are
representative of the entire group from which they were
chosen. In other words, what is discovered about them is
probably true about the larger group.

3. WE CAN SEE THE IMPACT, THE DIFFERENCE, HERE, AND WILL THEREFORE BE NEGATIVE.
BUT THIS IS WRONG.
IN FACT, THE PROJECT DID INCREASE THE PARTICIPANTS' HEALTH.
BUT WE ARE COMPARING A RICH AREA TO A POOR AREA THAT WILL OBVIOUSLY HAVE
LOWER HEALTH OUTCOMES AS THEY HAVE LOWER INCOME AND WEALTH.
SEE, THERE ARE A VARIETY OF FACTORS THAT MIGHT MAKE THE CONTROL GROUP'S
HEALTH OUTCOMES HIGHER THAN THE PARTICIPATION GROUP'S OUTCOMES.
AND ATTRIBUTING THIS TO THE PROGRAM IS WRONG.
-20
-10
0
10
20
30
40
50
1 2 3 4
health
Time
a
b
Start of the program

4. What is a randomized experiment?
In a randomized experiment, a sample of people is assigned to groups by
pure lottery, by flipping a coin, by casting a die, a treatment group where
individuals receive the program and a control group, where the individuals
do not receive the program.
The important a point here is that these people are assigned randomly to
one group or the other by pure lottery.
So how does that solve our previous program of having to find a comparison
group that is similar to the participation group?
When two groups are randomly selected from the same population, they both
represent the larger group.
They are not only statistically equivalent to the larger group.
They are also statistically equivalent to each other.
This means that on average, both groups have very similar
characteristics.

5. When two or more groups are selected in this way, we can say that
individuals have been randomly assigned to groups.
This is called random assignment. And random assignment is the key
element of randomized experiments.
If we randomly assign people to two groups and we take the average of
the
ages in each group, the averages will be very similar in both groups.
a simple randomized evaluation is that one
group receives the program that is being evaluated.
We call it the treatment.
And the other group does not.

6. • The advantages of controlled experimentation over other methods of
analysis are easy to describe. Because experimental subjects are randomly
assigned to alternative treatments, the effects of the treatments on behavior can
be measured with high reliability. The assignment procedure assures us of the
direction of causality between treatment and outcome: differences in average
outcomes among the several treatment groups are caused by differences in
treatment, and differences in average outcome are not the cause of the observed
differences in treatment.
• Random assignment also removes any systematic correlation between treatment
status and both observed and unobserved participant characteristics.
Estimated treatment effects are therefore free from the selection bias that
potentially taints all estimates based on non-experimental sources of information.
Advantages of controlled experimentation

7. CIRCUMSTANTIAL CAUSALITY
RCTs show is that there exists some z ε Z, such that if we have the
condition in state(x, z) ε X instead of (y, z) ε X, the condition in the
next period will be in state a ε X instead of state b ε X. This is like
saying, other things being the same (that is, z), if you vaccinate
people, in the next period, there will be no influenza. But if you do
not vaccinate them, there will be influenza. If we accept the
determinist axiom, as many do,then this demonstration means that
whenever we switch from (y, z) to (x, z), the condition will switch in
the next period from b to a. It is the “whenever” that makes this a
causal claim. This is what Kaushik Basu is referring to as
“circumstantial causality”. Given a certain set of circumstances,
changing y to x has a predictable consequence.

8. DATED DESCRIPTION
one in which the date or period of the experiment is a part of the
description of what is being held constant. Because a date never
recurs, we can never use this description in the future. So we will
never be able to say that the same conditions now hold and so the
same result is to be expected (recall the determinist axiom). What is
needed, at a minimum, is a description of what is being held
constant when a particular experiment is being run without
reference to a date. RCTs do not give us this and this is what makes
the findings of RCTs non-portable.

9. LIMITATIONS
RCTs have given us numerous valuable descriptions of what happened in the past and numerous
instances of causes in the past(provided of course that one is willing to accept the determinist
axiom) but, what they show is very limited. This is because when they show that it was the switch
from y to x that caused the switch from b to a, what they are saying is that this was true; under
certain historical conditions (z), but they cannot tell you what those historical conditions are. RCT
discoveries never graduate from something “was a cause” of something else to something “is a
cause”.
RCTs do provide insights into circumstantial causality and are the gold standard for
describing large populations over time. At the same time, however, it fails to demonstrate
any form of universal causality. That show us using the law of large numbers, we can
describe the average characteristics of a large population and changes over time, by
appropriately studying a small sample drawn from the population. But average
characteristics are not the only pertinent features of populations.
If we would recognize that the proportion of knowledge that we absorb
informally is substantially greater than what we know based on scientific
enquiries. To dismiss the former out of hand would greatly deplete our
knowledge. Only one incomplete answer relates to evolution.

10. “policies should be evidence based”, the hardest part of
the rule is contained in the word “based”. This is the
reason why we so often hear glib references to some
evidence and then a hand-waving switch to one’s favored
policy, ignoring the fact that the policy should be based on
the best available evidence.

11. For Bengal, in a certain period, electing a woman leader of
the local government caused water provisioning to be
better. This is no guide to the future because we do not
fully know what Bengal in a certain period is like.
Henceforth, a reference to causality without a qualifying
characterization should be taken to be a reference to
universal causality because for policy purposes, that is
what is of essence.