Mol evol jc.151216-amb.bayesianstats-teaser

Bayesian Statistics
Made Simple
Álvaro Martínez Barrio, PhD
Alvaro.Martinez.Barrio@scilifelab.se
linkedin.com/in/ambarrio
@ambarrio
!
Uppsala, Dec 16th 2015

2
Think Bayes
Bayesian Statistics Made Simple
Version 1.0.3
Allen B. Downey
Green Tea Press
Needham, Massachusetts

Notation: Probability
3
• p(A): the probability that A occurs
!
• p(A|B): the probability that A occurs, given
that B has occurred
!
• p(A and B) = p(A) p(B|A): Conjoint probability

The cookie problem
5
Suppose there are two bowls of cookies. Bowl 1 contains 30 vanilla
cookies and 10 chocolate cookies. Bowl 2 contains 20 of each.
Now suppose you choose one of the bowls at random and, without
looking, select a cookie at random. The cookie is vanilla. What is the
probability that it came from Bowl 1?

The cookie problem
6
p(B1|V) = p(B1) p(V|B1) / p(V)

The cookie problem
7
p(B1|V) = p(B1) p(V|B1) / p(V)
p(B1|V) = (1/2) (3/4) / 5/8

History: Bayes’ Theorem
8
Thomas Bayes,
(b. 1702, London - d. 1761,
Tunbridge Wells, Kent)
In the early 18th century, the mathematicians
of the time knew how to find the probability
that, say, 4 people aged 50 die in a given year
out of a sample of 60 if the probability of any
one of them dying was known.
But they did not know how to find the
probability of one 50-year old dying based on
the observation that 4 had died out of 60.

History: Bayes’ Theorem
9
Thomas Bayes,
(b. 1702, London - d. 1761,
In the early 18th century, the mathematicians
of the time knew how to find the probability
that, say, 4 people aged 50 die in a given year
out of a sample of 60 if the probability of any
one of them dying was known.
But they did not know how to find the
probability of one 50-year old dying based on
the observation that 4 had died out of 60.
the question of inverse probability

The “diachronic” interpretation
10
Thomas Bayes,
(b. 1702, London - d. 1761,
p(H|D) = p(H) p(D|H) / p(D)
• p(H) is the probability of the hypothesis before we see the data,
called the prior probability, or just prior.
• p(H|D) is what we want to compute, the probability of the
hypothesis aer we see the data, called the posterior.
• p(D|H) is the probability of the data under the hypothesis, called
the likelihood.
• p(D) is the probability of the data under any hypothesis, called
the normalizing constant.

History: Syllogism
11
4th century BC
!
!
• Major premise
!
!
• Minor premise
!
!
• Conclusion
A rhetorical syllogism (a 3-part deductive
argument) used in oratorial practice.

History: Syllogism
12
4th century BC
• Major premise: “All
humans are mortal”
!
!
• Minor premise: “All
Greeks are human”
!
• Conclusion: “All Greeks
are mortal”

History: Syllogism
13
4th century BC
• Major premise: “All
mortals die”
!
!
• Minor premise: “All men
are mortals”
!
• Conclusion: “All men
die”

History: Enthymeme
14
4th century BC
!
• “Socrates is mortal because he’s
human”
!
• Major premise (unstated): “All humans
are mortal.”
!
• Minor premise (stated): “Socrates is
human.”
!
• Conclusion (stated): “Therefore,
Socrates is mortal.”

History: Enthymeme
15
4th century BC
!
• "He is ill, since he has a cough.”
!
!
• “Since she has a child, she has
given birth."

History: Enthymeme
16
4th century BC
• He started to propose that
enthymemes are based on
probabilities (eikos), examples,
tekmêria (i.e., proofs,
evidences), and signs (sêmeia).

History: Enthymeme
17
4th century BC
• Carol Poster argues that
enthymemes as truncated
syllogisms was invented by
British rhetoricians (such as
Richard Whately) in the XVIII
century.
Poster, Carol (2003). "Theology, Canonicity, and
Abbreviated Enthymemes". Rhetoric Society
Quarterly 33 (1): 67–103.

Chapter 1: Bayes’ Theorem
• Mutually exclusive: At most one hypothesis in the set
can be true
!
• Collectively exhaustive: There are no other
possibilities; at least one of the hypotheses has to be
true

can be true
!
true
p(D) = p(B1) p(D|B1) + p(B2) p(D|B2)

can be true
!
true
p(D) = p(B1) p(D|B1) + p(B2) p(D|B2)
p(D) = (1/2) (3/4) + (1/2) (1/2) = 5/8

can be true
!
true
p(D) = p(B1) p(D|B1) + p(B2) p(D|B2)
p(D) = (1/2) (3/4) + (1/2) (1/2) = 5/8
If p(A|B) is hard to compute, or hard to measure experimentally, check whether it
might be easier to compute the other terms in Bayes’s theorem, p(B|A), p(A) and p(B).

Chapter 2: Computational Statistics
• Distribution:

• Distribution: set of values and their corresponding
probabilities.

• Distribution: set of values and their corresponding probabilities.
• Probability mass function: way to represent a distribution
mathematically.

mathematically.
• When talking about probabilities, you need to normalise (they
should add up to 1)

mathematically.
• When talking about probabilities, you need to normalise (they should
add up to 1)
• This distribution, which contains the priors for each hypothesis, is called
(wait for it) the prior distribution.
• To update the distribution based on new data (a vanilla cookie!), we
multiply each prior by the corresponding likelihood.
• The distribution is no longer normalized, you need to renormalize
• The result is a distribution that contains the posterior probability for
each hypothesis, which is called (wait again!) the posterior distribution.

mathematically.
• When talking about probabilities, you need to normalise (they should
add up to 1)
• This distribution, which contains the priors for each hypothesis, is called
(wait for it) the prior distribution.
• To update the distribution based on new data (a vanilla cookie!), we
multiply each prior by the corresponding likelihood.
• The distribution is no longer normalized, you need to renormalize
• The result is a distribution that contains the posterior probability for
each hypothesis, which is called (wait again!) the posterior distribution.
Terminology and design patterns of python programs that you can use during the rest
of the course

Chapter 3: Estimation
Suppose I have a box of dice that contains a 4-sided die, a 6-sided
die, an 8-sided die, a 12-sided die, and a 20-sided die. If you have
ever played Dungeons & Dragons, you know what I am talking
about.
Suppose I select a die from the box at random, roll it, and get a 6.
What is the probability that I rolled each die?

Suppose I have a box of dice that contains a 4-sided die, a 6-sided
die, an 8-sided die, a 12-sided die, and a 20-sided die. If you have
ever played Dungeons & Dragons, you know what I am talking
about.
Suppose I select a die from the box at random, roll it, and get a 6.
What is the probability that I rolled each die?
Let me suggest a three-step strategy for approaching a problem like this:
1. Choose a representation for the hypotheses. 
2. Choose a representation for the data. 
3. Write the likelihood function.

Mosteller’s Fifty Challenging Problems in Probability with Solutions
“A railroad numbers its locomotives in order 1..N. One day you see a
locomotive with the number 60. Estimate how many locomotives
the railroad has.”

Part I of Statistical Inference.

Part I of Statistical Inference.
There are two ways to proceed:
!
• Get more data. 
• Get more background information.

• Credible interval: For intervals we usually report two values
computed so that there is a 90% chance that the unknown value
falls between them (or any other probability).
• The width of this interval suggests how uncertain we are about
the conclusion based in our unknown value.
• There are two approaches to choosing prior distributions:
• i) informative: best represents background information
• ii) uninformative: intended to be as unrestricted as possible

• Credible interval: For intervals we usually report two values
computed so that there is a 90% chance that the unknown value
falls between them (or any other probability).
• The width of this interval suggests how uncertain we are about
the conclusion based in our unknown value.
• There are two approaches to choosing prior distributions:
• i) informative: best represents background information
• ii) uninformative: intended to be as unrestricted as possible
In real world you have two ways to proceed:
!
If you have a lot of data, the choice of the prior doesn’t matter very much; informative
and uninformative priors yield almost the same results.
!
If you don’t have much data, using relevant background information makes a big
difference.

Differences between Bayesians and Non-Bayesians
According to Jeff Gill (Center for Applied Statistics, WashU)

ACCP 37th Annual Meeting, Philadelphia, PA [2]
Diﬀerences Between Bayesians and Non-Bayesians
According to my friend Jeﬀ Gill
Typical Bayesian Typical Non-BayesianTypical Bayesian

Typical Bayesian Typical Non-BayesianTypical Bayesian
Typical Bayesian Typical Non-BayesianTypical Non-Bayesian

Conclusions
• Importance of modelling
• Follow a discrete approach: correct first, and
expand later

General Approach
1. Start with simple models and implement them in clear,
readable and demonstrably correct code. Focus should be on
good modelling decisions, not optimisation

General Approach
2. Identify the biggest sources of error. Perhaps increase the
number of values in a discrete approximation, increase the
number of iterations in a MC simulation, or add details to the
model

General Approach
2. Identify the biggest sources of error. Perhaps increase the
number of values in a discrete approximation, increase the
number of iterations in a MC simulation, or add details to the
model
3. Is performance good? If not, try optimising then

REFERENCES
• PyCon tutorials (by Allen Downey)
https://sites.google.com/site/simplebayes/
!
• “Probably Overthinking It” (by Allen Downey)
http://allendowney.blogspot.se/
!
• Mark A. Beaumont & Bruce Rannala (2004)
Nature Rev Genetics
Monument to members of the Bayes and Cotton families, including Thomas Bayes
and his father Joshua, in Bunhill Fields burial ground

Mol evol jc.151216-amb.bayesianstats-teaser

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (9)

Recently uploaded

Recently uploaded (20)

Mol evol jc.151216-amb.bayesianstats-teaser