SlideShare a Scribd company logo
1 of 50
An
Introduction
to Bayesian
Statistics
Paul Herendeen
April 2013
0
2000
4000
6000
8000
1960 1970 1980 1990 2000 2010
WOS "Bayesian"
Citations by Year
The rise of Bayesian
statistics
So…What are Bayesian
Statistics?
So…What are Bayesian
Statistics?
1) A fundamentally different approach to
probability
So…What are Bayesian
Statistics?
1) A fundamentally different approach to
probability
2) An associated set of mathematical tools
Frequentists vs. Bayesian
Round 1
Parameters
fixed
Data
varies
Data
fixed
Parameters
Vary
Frequentists vs. Bayesian
Round 1
Frequentists vs. Bayesian
Round 1
Confidence
Interval
Credible
Interval
Conditional Probability
in 2 minutes
Conditional Probability
in 2 minutes
All possible outcomes
Conditional Probability
in 2 minutes
Conditional Probability
in 2 minutes
Conditional Probability
in 2 minutes
Conditional Probability
in 2 minutes
Conditional Probability
in 2 minutes
Bayes’ Theorem
Bayes’ Theorem
Prior
Bayes’ Theorem
Likelihood
Prior
Bayes’ Theorem
Likelihood
Prior
Evidence
Bayes’ Theorem
Likelihood
Prior
Posterior
Evidence
Frequentist vs. Bayesian
Round 2
“The Strength of the Prior”
Sparse
Data
Abundant
Data
Uniform
Prior
Where do Priors Come
From?
So…What are Bayesian
Statistics?
1) A fundamentally different approach to
probability
2) An associated set of mathematical tools
How do you actually
do this?
So how do you actually
do this?
1. Analytical methods
So how do you actually
do this?
1. Analytical methods
2. Grid approximation
So how do you actually
do this?
1. Analytical methods
2. Grid approximation
3. Markov Chain Monte Carlo
MCMC
• Algorithm for exploring parameter space
MCMC
• Algorithm for exploring parameter space
1.Pick a starting point
MCMC
• Algorithm for exploring parameter space
1.Pick a starting point
2.Propose a move
MCMC
• Algorithm for exploring parameter space
1.Pick a starting point
2.Propose a move
3.Accept or decline move based on
probability
MCMC
• Algorithm for exploring parameter space
1.Pick a starting point
2.Propose a move
3.Accept or decline move based on
probability
• Time spent at each point approximates
parameter distribution
MCMC
• Algorithm for exploring parameter space
1.Pick a starting point
2.Propose a move
3.Accept or decline move based on
probability
• Time spent at each point approximates
parameter distribution
• E.g. Metropolis-Hastings, Gibbs sampling
MCMC
2D example
MCMC
2D example
So what does all this
get us?
Bayesian methods
really shine in complex
(hierarchical) models…
For example,
Individual
Fecundity
Group
Effect
Population
Effect
Foraging
success
Environment
or…
Individual
Fecundity
Group
Effect
Population
Effect
Environment
Many benefits to this
approach
• Simultaneously estimate parameters
• …as well as parameter relationships
• “Borrow” strength across studies
• Model comparison
So, is it a Bayesian
Revolution?
Bayesian stats can do
most things
frequentist,
Bayesian stats can do
most things
frequentist, but…
• Many simple models don‟t gain much
• Better do something „boring‟ well than
something exciting poorly
Bayesian stats can do
most things
frequentist, but…
• Many simple models don‟t gain much
• Better do something „boring‟ well than
something exciting poorly
• Don‟t be this guy
DO use Bayesian
methods if
• You have a complex model with many
interacting parameters
• You have „messy‟ data
• You don‟t want to make assumptions
about distributions
In Conclusion
• Bayesian methods are powerful tools
for ecological research
• Like most things statistical, they are
no substitute for thinking
• They are here to stay, and you should
at least be familiar with them
Great, I want to learn
more!
JAGS
(Just Another Gibbs Sampler)

More Related Content

What's hot

Curse of dimensionality
Curse of dimensionalityCurse of dimensionality
Curse of dimensionality
Nikhil Sharma
 
Bayesian decision making in clinical research
Bayesian decision making in clinical researchBayesian decision making in clinical research
Bayesian decision making in clinical research
Bhaswat Chakraborty
 

What's hot (20)

Bayesian Networks - A Brief Introduction
Bayesian Networks - A Brief IntroductionBayesian Networks - A Brief Introduction
Bayesian Networks - A Brief Introduction
 
Principal component analysis and lda
Principal component analysis and ldaPrincipal component analysis and lda
Principal component analysis and lda
 
Missing Data and data imputation techniques
Missing Data and data imputation techniquesMissing Data and data imputation techniques
Missing Data and data imputation techniques
 
Logistic regression
Logistic regressionLogistic regression
Logistic regression
 
Bayesian networks
Bayesian networksBayesian networks
Bayesian networks
 
Bayesian networks
Bayesian networksBayesian networks
Bayesian networks
 
Naive Bayes Presentation
Naive Bayes PresentationNaive Bayes Presentation
Naive Bayes Presentation
 
Logistic regression
Logistic regressionLogistic regression
Logistic regression
 
Machine learning session4(linear regression)
Machine learning   session4(linear regression)Machine learning   session4(linear regression)
Machine learning session4(linear regression)
 
Bayesian classification
Bayesian classificationBayesian classification
Bayesian classification
 
Belief Networks & Bayesian Classification
Belief Networks & Bayesian ClassificationBelief Networks & Bayesian Classification
Belief Networks & Bayesian Classification
 
Curse of dimensionality
Curse of dimensionalityCurse of dimensionality
Curse of dimensionality
 
Change Point Analysis
Change Point AnalysisChange Point Analysis
Change Point Analysis
 
Naive Bayes
Naive BayesNaive Bayes
Naive Bayes
 
Bayesian decision making in clinical research
Bayesian decision making in clinical researchBayesian decision making in clinical research
Bayesian decision making in clinical research
 
Pca ppt
Pca pptPca ppt
Pca ppt
 
NAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIERNAIVE BAYES CLASSIFIER
NAIVE BAYES CLASSIFIER
 
Statistical tests for data involving quantitative data
Statistical tests for data involving quantitative dataStatistical tests for data involving quantitative data
Statistical tests for data involving quantitative data
 
Logistic regression
Logistic regressionLogistic regression
Logistic regression
 
3. Regression.pdf
3. Regression.pdf3. Regression.pdf
3. Regression.pdf
 

Similar to An introduction to bayesian statistics

D. Mayo: Philosophical Interventions in the Statistics Wars
D. Mayo: Philosophical Interventions in the Statistics WarsD. Mayo: Philosophical Interventions in the Statistics Wars
D. Mayo: Philosophical Interventions in the Statistics Wars
jemille6
 
What should we expect from reproducibiliry
What should we expect from reproducibiliryWhat should we expect from reproducibiliry
What should we expect from reproducibiliry
Stephen Senn
 
Applied bayesian statistics
Applied bayesian statisticsApplied bayesian statistics
Applied bayesian statistics
Springer
 
The replication crisis: are P-values the problem and are Bayes factors the so...
The replication crisis: are P-values the problem and are Bayes factors the so...The replication crisis: are P-values the problem and are Bayes factors the so...
The replication crisis: are P-values the problem and are Bayes factors the so...
StephenSenn2
 
The replication crisis: are P-values the problem and are Bayes factors the so...
The replication crisis: are P-values the problem and are Bayes factors the so...The replication crisis: are P-values the problem and are Bayes factors the so...
The replication crisis: are P-values the problem and are Bayes factors the so...
jemille6
 

Similar to An introduction to bayesian statistics (20)

Berger 2000
Berger 2000Berger 2000
Berger 2000
 
Thinking statistically v3
Thinking statistically v3Thinking statistically v3
Thinking statistically v3
 
tjosullivanThesis
tjosullivanThesistjosullivanThesis
tjosullivanThesis
 
Machine learning clisification algorthims
Machine learning clisification algorthimsMachine learning clisification algorthims
Machine learning clisification algorthims
 
D. Mayo: Philosophical Interventions in the Statistics Wars
D. Mayo: Philosophical Interventions in the Statistics WarsD. Mayo: Philosophical Interventions in the Statistics Wars
D. Mayo: Philosophical Interventions in the Statistics Wars
 
Mayod@psa 21(na)
Mayod@psa 21(na)Mayod@psa 21(na)
Mayod@psa 21(na)
 
On being Bayesian
On being BayesianOn being Bayesian
On being Bayesian
 
beyond objectivity and subjectivity; a discussion paper
beyond objectivity and subjectivity; a discussion paperbeyond objectivity and subjectivity; a discussion paper
beyond objectivity and subjectivity; a discussion paper
 
Crisis of confidence, p-hacking and the future of psychology
Crisis of confidence, p-hacking and the future of psychologyCrisis of confidence, p-hacking and the future of psychology
Crisis of confidence, p-hacking and the future of psychology
 
What should we expect from reproducibiliry
What should we expect from reproducibiliryWhat should we expect from reproducibiliry
What should we expect from reproducibiliry
 
Probabilistic modeling in deep learning
Probabilistic modeling in deep learningProbabilistic modeling in deep learning
Probabilistic modeling in deep learning
 
Resolving e commerce challenges with probabilistic programming
Resolving e commerce challenges with probabilistic programmingResolving e commerce challenges with probabilistic programming
Resolving e commerce challenges with probabilistic programming
 
Discussion a 4th BFFF Harvard
Discussion a 4th BFFF HarvardDiscussion a 4th BFFF Harvard
Discussion a 4th BFFF Harvard
 
Signal and noise
Signal and noiseSignal and noise
Signal and noise
 
Applied bayesian statistics
Applied bayesian statisticsApplied bayesian statistics
Applied bayesian statistics
 
probability.pptx
probability.pptxprobability.pptx
probability.pptx
 
The replication crisis: are P-values the problem and are Bayes factors the so...
The replication crisis: are P-values the problem and are Bayes factors the so...The replication crisis: are P-values the problem and are Bayes factors the so...
The replication crisis: are P-values the problem and are Bayes factors the so...
 
The replication crisis: are P-values the problem and are Bayes factors the so...
The replication crisis: are P-values the problem and are Bayes factors the so...The replication crisis: are P-values the problem and are Bayes factors the so...
The replication crisis: are P-values the problem and are Bayes factors the so...
 
0 methods for multiple treatment comparisons
0 methods for multiple treatment comparisons0 methods for multiple treatment comparisons
0 methods for multiple treatment comparisons
 
Software estimation is crap
Software estimation is crapSoftware estimation is crap
Software estimation is crap
 

Recently uploaded

The basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxThe basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptx
heathfieldcps1
 
1029-Danh muc Sach Giao Khoa khoi 6.pdf
1029-Danh muc Sach Giao Khoa khoi  6.pdf1029-Danh muc Sach Giao Khoa khoi  6.pdf
1029-Danh muc Sach Giao Khoa khoi 6.pdf
QucHHunhnh
 
Seal of Good Local Governance (SGLG) 2024Final.pptx
Seal of Good Local Governance (SGLG) 2024Final.pptxSeal of Good Local Governance (SGLG) 2024Final.pptx
Seal of Good Local Governance (SGLG) 2024Final.pptx
negromaestrong
 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdf
ciinovamais
 

Recently uploaded (20)

On National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan FellowsOn National Teacher Day, meet the 2024-25 Kenan Fellows
On National Teacher Day, meet the 2024-25 Kenan Fellows
 
The basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxThe basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptx
 
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
Explore beautiful and ugly buildings. Mathematics helps us create beautiful d...
 
1029-Danh muc Sach Giao Khoa khoi 6.pdf
1029-Danh muc Sach Giao Khoa khoi  6.pdf1029-Danh muc Sach Giao Khoa khoi  6.pdf
1029-Danh muc Sach Giao Khoa khoi 6.pdf
 
Measures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and ModeMeasures of Central Tendency: Mean, Median and Mode
Measures of Central Tendency: Mean, Median and Mode
 
Seal of Good Local Governance (SGLG) 2024Final.pptx
Seal of Good Local Governance (SGLG) 2024Final.pptxSeal of Good Local Governance (SGLG) 2024Final.pptx
Seal of Good Local Governance (SGLG) 2024Final.pptx
 
Key note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdfKey note speaker Neum_Admir Softic_ENG.pdf
Key note speaker Neum_Admir Softic_ENG.pdf
 
Activity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdfActivity 01 - Artificial Culture (1).pdf
Activity 01 - Artificial Culture (1).pdf
 
microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introduction
 
Web & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdfWeb & Social Media Analytics Previous Year Question Paper.pdf
Web & Social Media Analytics Previous Year Question Paper.pdf
 
Introduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The BasicsIntroduction to Nonprofit Accounting: The Basics
Introduction to Nonprofit Accounting: The Basics
 
How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17How to Give a Domain for a Field in Odoo 17
How to Give a Domain for a Field in Odoo 17
 
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptxBasic Civil Engineering first year Notes- Chapter 4 Building.pptx
Basic Civil Engineering first year Notes- Chapter 4 Building.pptx
 
Holdier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdfHoldier Curriculum Vitae (April 2024).pdf
Holdier Curriculum Vitae (April 2024).pdf
 
Z Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot GraphZ Score,T Score, Percential Rank and Box Plot Graph
Z Score,T Score, Percential Rank and Box Plot Graph
 
Role Of Transgenic Animal In Target Validation-1.pptx
Role Of Transgenic Animal In Target Validation-1.pptxRole Of Transgenic Animal In Target Validation-1.pptx
Role Of Transgenic Animal In Target Validation-1.pptx
 
Class 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdfClass 11th Physics NEET formula sheet pdf
Class 11th Physics NEET formula sheet pdf
 
Micro-Scholarship, What it is, How can it help me.pdf
Micro-Scholarship, What it is, How can it help me.pdfMicro-Scholarship, What it is, How can it help me.pdf
Micro-Scholarship, What it is, How can it help me.pdf
 
ICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptxICT Role in 21st Century Education & its Challenges.pptx
ICT Role in 21st Century Education & its Challenges.pptx
 
Python Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docxPython Notes for mca i year students osmania university.docx
Python Notes for mca i year students osmania university.docx
 

An introduction to bayesian statistics