This document discusses the key differences between Bayesian and classical statistics, focusing on how Bayesian statistics updates beliefs based on new data. It explains concepts such as prior, data, and posterior beliefs using an example about pregnancy. The text also highlights the implications of both statistical approaches, emphasizing the advantages of Bayesian methods in incorporating prior knowledge and updating beliefs, while classical methods are described as more objective.

1. How is Bayesian Statistics
Different?
by Wayne Tai Lee

2. Goal
● Clarify the difference between “classical and
Bayesian Statistics
● Lay out the pro/con with this “attitude”

3. One sentence definition
Bayesian statistics is a mathematical
framework to update beliefs as you observe
more data.

4. Bayesian Update in Movies
Recall movies where a female character
realizes her period is late?

5. Movie cliché: Am I pregnant?
● What did I do in the past month?

6. Movie cliché: Am I pregnant?
● What did I do in the past month?
– Forms a prior belief of whether I am pregnant

7. Movie cliché: Am I pregnant?
● What did I do in the past month?
– Forms a prior belief of whether I am pregnant
● The missing period
– Data!

8. Movie cliché: Am I pregnant?
● What did I do in the past month?
– Forms a prior belief of whether I am pregnant
● The missing period
– Data!
● Belief is updated as more data is observed!

9. Bayesian terminology
● Prior: your belief about pregnancy before
seeing new data
● Data: missing period
● Posterior: your belief that is updated after
seeing the data

10. How do we formalize this update?
● Pregnant is a uncertain event with two
outcomes: Yes or No

11. How do we formalize this update?
● Pregnant is a uncertain event with two
outcomes: Yes or No
● “Days delayed of period” is a data point
– If (Pregnant = Yes), delayed ~ 30*9 days
– If (Pregnant = No), it might come sooner

12. Mathematical framework
● “Pregnant” is a random variable:
– P(Pregnant = Yes) = X
– P(Pregnant = No) = (1 - X)

13. Mathematical framework
● “Pregnant” is a random variable:
– P(Pregnant = Yes) = X
– P(Pregnant = No) = (1 - X)
● “Days delayed of period” is another random
variable!
– P(days delay >= 7 days | Pregnant) = 1
– P(days delay >= 7 days | Not Pregnant) = Y

14. Simplify
● Start with the objective:
Am I pregnant?
i.e. P(Pregnant | Data)?

15. Simplify
● Start with the objective:
Am I pregnant?
i.e. P(Pregnant | Data)?
● Note all the numbers we know are the form of
P( **** | Pregnant)

16. Conditional Probability!
P(Pregnant | Data)
= P(Data | Pregnant) P(Pregnant) / P(Data)

17. Conditional Probability!
P(Pregnant | Data)
= P(Data | Pregnant) P(Pregnant) / P(Data)
Immediate implication:
● If your prior says you cannot be pregnant,
your belief cannot be changed!

18. “Bayes Rule”
P(Pregnant | Data)
= P(Data | Pregnant) P(Pregnant) / P(Data)
= P(Data | Pregnant) P(Pregnant) /
[ P(Data | Pregnant) P(Pregnant) +
P(Data | Not Pregnant) P(Not Pregnant) ]

19. “Bayes Rule”
P(Pregnant | Data)
= P(Data | Pregnant) P(Pregnant) / P(Data)
= P(Data | Pregnant) P(Pregnant) /
[ P(Data | Pregnant) P(Pregnant) +
P(Data | Not Pregnant) P(Not Pregnant) ]
Why add more numbers?
P(Data) was hard to compute, so chop it into
pieces we know!

20. P(Data): Big Issue for Bayesians
● Pregnant is binary which made this realllllly
easy
● In general, a lot of “tricks” are trying to
– solve for P(Data)
● Belief propagation in graphical models
– getting around it
● Sampling: MCMC
● Approximation: Variational Bayes

21. Back to the key question:
P(Pregnant | Data)
= P(Data | Pregnant) P(Pregnant) /
[ P(Data | Pregnant) P(Pregnant) +
P(Data | Not Pregnant) P(Not Pregnant) ]
= 1 * X / [ 1 * X + Y * (1 - X) ]

22. Back to the key question:

23. Can add more data
….....almost for free!
● Notice “Data” is quite general:
– Can add pregnancy strips data to further
update beliefs!
– Treat previous outputs as priors then update
similarly!

24. So.....what's the big deal?
● Your belief matters a lot!
– Your prior changes the outcome
● Your prior and my prior may be different

25. What “could” a bad Frequentist
Do?
● Calculate the p-value for you, i.e.
P(Late period | Not Pregnant)
● Declare that you're Pregnant if this is <= 5%

26. What “could” a bad Frequentist
Do?
● Calculate the p-value for you, i.e.
P(Late period | Not Pregnant)
● Declare that you're Pregnant if this is <= 5%
● Declaration has 5% false positive and a
certain false negative rates

27. What “could” a bad Frequentist
Do?
● Calculate the p-value for you, i.e.
P(Late period | Not Pregnant)
● Declare that you're Pregnant if this is <= 5%
● Declaration has 5% false positive and a
certain false negative rates
● Issue: Not as relevant to you! Rates are for all
the people using this procedure...not specific
to your case!

28. “not as relevant”?
● There's no consideration of your specific case
– There was no P(Pregnant) in the p-value
calculation
– You could be really sure that you're not
pregnant....doesn't change the calculation!

29. What would a Frequentist say?
● P(Pregnant) = 100% or 0%
– Fixed but unknown
– NOT uncertain
● …Not actually interested in a single event
– Probabilities are defined for repeated events
– Will not write down P(Pregnant | Data)
– For your one case, anything could be true

30. What would a Frequentist say?
● P(Pregnant) = 100% or 0%
– Fixed but unknown
– NOT uncertain
● …Not actually interested in a single event
– Probabilities are defined for repeated events
– Will not write down P(Pregnant | Data)
– For your one case, anything could be true
● Would say “Go talk to a doctor”

31. Key difference
● “Attitude”
– What can be a random variable?
● Bayesian: Uncertain events
● Frequentist: Repeatable events

32. Implications of this attitude
● Bayesian:
– Can incorporate prior knowledge easily
– Can update beliefs easily
– Can tackle a wider class of problems since
probabilities are “beliefs”

33. Implications of this attitude
● Bayesian:
– Can incorporate prior knowledge easily
– Can update beliefs easily
– Can tackle a wider class of problems since
probabilities are “beliefs”
– Must specify a model
– Your belief can be different from mine
● Our answers will be different!

34. Implications of this attitude
● Frequentist:
– Probabilities are more objective
– Harder to cheat
– Has non-parametric methods

35. Implications of this attitude
● Frequentist:
– Probabilities are more objective
– Harder to cheat
– Has non-parametric methods
– Focused on repeatable events
– Prior knowledge is introduced in an ad hoc
format
– Usually need lots of data

36. In the end...
● Frequentist and Bayesian use the same rules
of probabilities
● Difference exists in set-up: “What is random?”
– Bayesians: uncertainty in knowledge
– Frequentist: intrinsic randomness

37. Take Home
● Different problems should use different
approaches!
– Both schools are awesome!~
● Be aware of what you're using and be
consistent!