SlideShare a Scribd company logo
How meta-analysis
advances knowledge
Stephen S Holden
Risks of eating bacon?
• colorectal cancer
risk +18% risk /
50g of processed
meat / day
• overall risk = 1 in
17 = 5.8%
Risks of eating bacon?
5.8%
6.9%
0%
2%
4%
6%
8%
No bacon 50g / day
% risk cancer
Theory of Signal Detection (TSD)
Noise Signal
Dangers of NHST
Hit ! False alarm
Miss
Correct
rejection
Effect No effect
Think so
Think not
Overlooking effect sizes
“I’ve heard it before”…
…belies the uncertainty of science
Open Science Collaboration Science 2015;349:aac4716
The price of oversight
• The Reproducibility
Project
• p-hacking
Advancing knowledge
Theatre of theory
• top-down
• theoretical
• ideas
• hypothetico-deductive
• NHST
Empire of empirical
• bottom-up
• empirical
• data
• fishing trip
• effects
Effect size measures
• absolute measures
• correlational
• r, R2, eta-sq
• differences
• SMDs, Cohen’s d, Hedge’s g
• categorical: odds-ratio, relative-risk
• raw-mean score differences
• relative measures
• % change, elasticities
Sales
SoM
Choice
Purchase
Consump’n
Learning
Memory
Perception
- - - -
Motivation
Beliefs
Attitudes
Intentions
Psychology Behavior Outcome
Advertising
Cons’r bhvr
Channels
Pricing
NPD
Sales
Strategy
Interventions
Chain of effect
Effect of attitudes on behaviour?
• r=.5
• k=128, N=4,598
• Glasman & Albarracin (2006)
• stronger when
• attitudes are more accessible
• direct experience and/or reported frequently
• attitudes are more stable over time
• confident, based on bhvr relevant information, one-sided info
Effect of anti-depressants?
• Cohen’s d < .2
• for HDRS < 23 (mild or moderate), placebo vs ADM (anti-depressant
medication)
• HDRS scale runs 0 to 52
• pax treated with ADMs average 2 points higher on HDRS than
placebo!
• statistical significance vs clinical significance
• https://www.scientificamerican.com/article/antidepressants-do-
they-work-or-dont-they/
SMD
Standardized
Mean
Differences
Cohen’s
d
Pearson’s
r
“small” .3 .1
“medium” .5 .3
“large” .8 .5
Cohen 1992 “A power primer”
Effect of anti-depressants
Effect of advertising on sales
• advtg elasticity = .12
• ie, a 10% increase in advertising  1.2%
increase in sales
• price elasticity = -2.3
• ie, a 10% increase in price  23% reduction
in sales
• banner click-through-rate (CTR)
• .3 to .05% (PWC report)
Effect of portion-size on obesity?
• effect on behaviour
• d = .45
• elasticity = .35
• effect on weight
• .56kg/mo
Effect of health supplements
Meta-meta-analysis of advance in
knowledge
• 176 meta-analyses
• >7,500 primary studies (43 studies per meta-analysis)
published between 1918 and 2012
• >54,000 effect sizes (307 effect sizes per meta-analysis)
• sample of 8,337,096 subjects (based on primary studies of 131
meta-analyses)
• Eisend 2015 JM
Meta-meta-analytic effect sizes
Corr (r) r2
All effect sizes .24 5.6%
Advertising .21 4.6%
Channels .24 5.7%
Consumer behaviour .28 8.1%
Method .15 2.4%
New product development .20 4.1%
Pricing .36 13.0%
Sales .23 5.2%
Strategy .25 6.1%
Advancing knowledge
Theatre of theory
• top-down
• theoretical
• ideas
• hypothetico-deductive
• NHST
Empire of empirical
• bottom-up
• empirical
• data
• fishing trip
• effects
It’s not the end
of the world if
you’re wrong !
How meta-analysis has
helped to advance
knowledge
Stephen S Holden
• research designs
• preferential treatment of experiments (RCT)
• selection bias
• file drawer
• publication bias
• heterogeneity / moderator analyses
• non-random sample of domain
• fixed vs random effects
• collinearity
Contentious issues

More Related Content

Similar to How meta-analysis advances knowledge

Clinical prediction models: development, validation and beyond
Clinical prediction models:development, validation and beyondClinical prediction models:development, validation and beyond
Clinical prediction models: development, validation and beyond
Maarten van Smeden
 
Exposure Assessment, George Gray
Exposure Assessment, George GrayExposure Assessment, George Gray
Exposure Assessment, George Gray
OECD Governance
 
Technology Assessment, Outcomes Research and Economic Analyses
Technology Assessment, Outcomes Research and Economic AnalysesTechnology Assessment, Outcomes Research and Economic Analyses
Technology Assessment, Outcomes Research and Economic Analyses
evadew1
 
Clinical trial bms clinical trials methodology 17012018
Clinical trial bms   clinical trials methodology 17012018Clinical trial bms   clinical trials methodology 17012018
Clinical trial bms clinical trials methodology 17012018
SoM
 
Call on Congress 2014 John Marshall, Treatment
Call on Congress 2014 John Marshall, TreatmentCall on Congress 2014 John Marshall, Treatment
Call on Congress 2014 John Marshall, Treatment
Fight Colorectal Cancer
 
Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016
Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016
Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016
evadew1
 
26 Oct 2017 Genetics and precision medicine Milano Cornel.ppt
26 Oct 2017 Genetics and precision medicine Milano Cornel.ppt26 Oct 2017 Genetics and precision medicine Milano Cornel.ppt
26 Oct 2017 Genetics and precision medicine Milano Cornel.ppt
Animikh Ray
 
Day 1 (Lecture 3): Predictive Analytics in Healthcare
Day 1 (Lecture 3): Predictive Analytics in HealthcareDay 1 (Lecture 3): Predictive Analytics in Healthcare
Day 1 (Lecture 3): Predictive Analytics in Healthcare
Aseda Owusua Addai-Deseh
 
Food safety and antimicrobial resistance research: A One Health perspective
Food safety and antimicrobial resistance research: A One Health perspectiveFood safety and antimicrobial resistance research: A One Health perspective
Food safety and antimicrobial resistance research: A One Health perspective
ILRI
 
randomized clinical trials II
randomized clinical trials IIrandomized clinical trials II
randomized clinical trials II
IAU Dent
 
Designing Causal Inference Studies Using Real-World Data
Designing Causal Inference Studies Using Real-World DataDesigning Causal Inference Studies Using Real-World Data
Designing Causal Inference Studies Using Real-World Data
InsideScientific
 
Food safety and antimicrobial resistance research: a One Health perspective
Food safety and antimicrobial resistance research:  a One Health perspectiveFood safety and antimicrobial resistance research:  a One Health perspective
Food safety and antimicrobial resistance research: a One Health perspective
ILRI
 
H2O World - H2O for Genomics with Hussam Al-Deen Ashab
H2O World - H2O for Genomics with Hussam Al-Deen AshabH2O World - H2O for Genomics with Hussam Al-Deen Ashab
H2O World - H2O for Genomics with Hussam Al-Deen Ashab
Sri Ambati
 
BioVariance - Pediatric Pharmacogenomics in Drug Discovery
BioVariance - Pediatric Pharmacogenomics in Drug DiscoveryBioVariance - Pediatric Pharmacogenomics in Drug Discovery
BioVariance - Pediatric Pharmacogenomics in Drug Discovery
Josef Scheiber
 
Pich Deck for Pepper Bio, for TechCruch's Pitch Deck Teardown series
Pich Deck for Pepper Bio, for TechCruch's Pitch Deck Teardown seriesPich Deck for Pepper Bio, for TechCruch's Pitch Deck Teardown series
Pich Deck for Pepper Bio, for TechCruch's Pitch Deck Teardown series
HajeJanKamps
 
Increasing precision in survey experiments without introducing bias
Increasing precision in survey experiments without introducing biasIncreasing precision in survey experiments without introducing bias
Increasing precision in survey experiments without introducing bias
Wilte Zijlstra
 
Modeling the cost effectiveness of two big league pay-for-performance policies
Modeling the cost effectiveness of two big league pay-for-performance policiesModeling the cost effectiveness of two big league pay-for-performance policies
Modeling the cost effectiveness of two big league pay-for-performance policies
cheweb1
 
Health IT Summit Austin 2013 - Presentation "The Impact of All Data on Health...
Health IT Summit Austin 2013 - Presentation "The Impact of All Data on Health...Health IT Summit Austin 2013 - Presentation "The Impact of All Data on Health...
Health IT Summit Austin 2013 - Presentation "The Impact of All Data on Health...
Health IT Conference – iHT2
 

Similar to How meta-analysis advances knowledge (20)

Clinical prediction models: development, validation and beyond
Clinical prediction models:development, validation and beyondClinical prediction models:development, validation and beyond
Clinical prediction models: development, validation and beyond
 
Exposure Assessment, George Gray
Exposure Assessment, George GrayExposure Assessment, George Gray
Exposure Assessment, George Gray
 
Technology Assessment, Outcomes Research and Economic Analyses
Technology Assessment, Outcomes Research and Economic AnalysesTechnology Assessment, Outcomes Research and Economic Analyses
Technology Assessment, Outcomes Research and Economic Analyses
 
Clinical trial bms clinical trials methodology 17012018
Clinical trial bms   clinical trials methodology 17012018Clinical trial bms   clinical trials methodology 17012018
Clinical trial bms clinical trials methodology 17012018
 
Call on Congress 2014 John Marshall, Treatment
Call on Congress 2014 John Marshall, TreatmentCall on Congress 2014 John Marshall, Treatment
Call on Congress 2014 John Marshall, Treatment
 
Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016
Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016
Technology Assessment/Outcome & Cost-Effectiveness Analysis 2016
 
26 Oct 2017 Genetics and precision medicine Milano Cornel.ppt
26 Oct 2017 Genetics and precision medicine Milano Cornel.ppt26 Oct 2017 Genetics and precision medicine Milano Cornel.ppt
26 Oct 2017 Genetics and precision medicine Milano Cornel.ppt
 
Day 1 (Lecture 3): Predictive Analytics in Healthcare
Day 1 (Lecture 3): Predictive Analytics in HealthcareDay 1 (Lecture 3): Predictive Analytics in Healthcare
Day 1 (Lecture 3): Predictive Analytics in Healthcare
 
Food safety and antimicrobial resistance research: A One Health perspective
Food safety and antimicrobial resistance research: A One Health perspectiveFood safety and antimicrobial resistance research: A One Health perspective
Food safety and antimicrobial resistance research: A One Health perspective
 
randomized clinical trials II
randomized clinical trials IIrandomized clinical trials II
randomized clinical trials II
 
Designing Causal Inference Studies Using Real-World Data
Designing Causal Inference Studies Using Real-World DataDesigning Causal Inference Studies Using Real-World Data
Designing Causal Inference Studies Using Real-World Data
 
Food safety and antimicrobial resistance research: a One Health perspective
Food safety and antimicrobial resistance research:  a One Health perspectiveFood safety and antimicrobial resistance research:  a One Health perspective
Food safety and antimicrobial resistance research: a One Health perspective
 
H2O World - H2O for Genomics with Hussam Al-Deen Ashab
H2O World - H2O for Genomics with Hussam Al-Deen AshabH2O World - H2O for Genomics with Hussam Al-Deen Ashab
H2O World - H2O for Genomics with Hussam Al-Deen Ashab
 
MyRISQ
MyRISQMyRISQ
MyRISQ
 
BioVariance - Pediatric Pharmacogenomics in Drug Discovery
BioVariance - Pediatric Pharmacogenomics in Drug DiscoveryBioVariance - Pediatric Pharmacogenomics in Drug Discovery
BioVariance - Pediatric Pharmacogenomics in Drug Discovery
 
Iadh innovation in healthcare r4
Iadh innovation in healthcare r4Iadh innovation in healthcare r4
Iadh innovation in healthcare r4
 
Pich Deck for Pepper Bio, for TechCruch's Pitch Deck Teardown series
Pich Deck for Pepper Bio, for TechCruch's Pitch Deck Teardown seriesPich Deck for Pepper Bio, for TechCruch's Pitch Deck Teardown series
Pich Deck for Pepper Bio, for TechCruch's Pitch Deck Teardown series
 
Increasing precision in survey experiments without introducing bias
Increasing precision in survey experiments without introducing biasIncreasing precision in survey experiments without introducing bias
Increasing precision in survey experiments without introducing bias
 
Modeling the cost effectiveness of two big league pay-for-performance policies
Modeling the cost effectiveness of two big league pay-for-performance policiesModeling the cost effectiveness of two big league pay-for-performance policies
Modeling the cost effectiveness of two big league pay-for-performance policies
 
Health IT Summit Austin 2013 - Presentation "The Impact of All Data on Health...
Health IT Summit Austin 2013 - Presentation "The Impact of All Data on Health...Health IT Summit Austin 2013 - Presentation "The Impact of All Data on Health...
Health IT Summit Austin 2013 - Presentation "The Impact of All Data on Health...
 

More from Stephen S Holden

Evaluation of social inclusion programs for Persons with Disability
Evaluation of social inclusion programs for Persons with DisabilityEvaluation of social inclusion programs for Persons with Disability
Evaluation of social inclusion programs for Persons with Disability
Stephen S Holden
 
Getting men to talk (14nov18)
Getting men to talk (14nov18)Getting men to talk (14nov18)
Getting men to talk (14nov18)
Stephen S Holden
 
Obesity research agenda: a reflection & a refresh
Obesity research agenda: a reflection & a refreshObesity research agenda: a reflection & a refresh
Obesity research agenda: a reflection & a refresh
Stephen S Holden
 
Overcoming obesity: small, slow and steady steps
Overcoming obesity: small, slow and steady stepsOvercoming obesity: small, slow and steady steps
Overcoming obesity: small, slow and steady steps
Stephen S Holden
 
Ethics of marketing influence (v3 23nov17)
Ethics of marketing influence (v3 23nov17)Ethics of marketing influence (v3 23nov17)
Ethics of marketing influence (v3 23nov17)
Stephen S Holden
 
Practical wisdom: share the wisdom
Practical wisdom: share the wisdomPractical wisdom: share the wisdom
Practical wisdom: share the wisdom
Stephen S Holden
 
Battling the bulge some immodest proposals
Battling the bulge   some immodest proposalsBattling the bulge   some immodest proposals
Battling the bulge some immodest proposals
Stephen S Holden
 
Battling the bulge an immodest proposal
Battling the bulge   an immodest proposalBattling the bulge   an immodest proposal
Battling the bulge an immodest proposal
Stephen S Holden
 
Sleeping with the enemy (final)
Sleeping with the enemy (final)Sleeping with the enemy (final)
Sleeping with the enemy (final)
Stephen S Holden
 

More from Stephen S Holden (9)

Evaluation of social inclusion programs for Persons with Disability
Evaluation of social inclusion programs for Persons with DisabilityEvaluation of social inclusion programs for Persons with Disability
Evaluation of social inclusion programs for Persons with Disability
 
Getting men to talk (14nov18)
Getting men to talk (14nov18)Getting men to talk (14nov18)
Getting men to talk (14nov18)
 
Obesity research agenda: a reflection & a refresh
Obesity research agenda: a reflection & a refreshObesity research agenda: a reflection & a refresh
Obesity research agenda: a reflection & a refresh
 
Overcoming obesity: small, slow and steady steps
Overcoming obesity: small, slow and steady stepsOvercoming obesity: small, slow and steady steps
Overcoming obesity: small, slow and steady steps
 
Ethics of marketing influence (v3 23nov17)
Ethics of marketing influence (v3 23nov17)Ethics of marketing influence (v3 23nov17)
Ethics of marketing influence (v3 23nov17)
 
Practical wisdom: share the wisdom
Practical wisdom: share the wisdomPractical wisdom: share the wisdom
Practical wisdom: share the wisdom
 
Battling the bulge some immodest proposals
Battling the bulge   some immodest proposalsBattling the bulge   some immodest proposals
Battling the bulge some immodest proposals
 
Battling the bulge an immodest proposal
Battling the bulge   an immodest proposalBattling the bulge   an immodest proposal
Battling the bulge an immodest proposal
 
Sleeping with the enemy (final)
Sleeping with the enemy (final)Sleeping with the enemy (final)
Sleeping with the enemy (final)
 

Recently uploaded

一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
slg6lamcq
 
社内勉強会資料_LLM Agents                              .
社内勉強会資料_LLM Agents                              .社内勉強会資料_LLM Agents                              .
社内勉強会資料_LLM Agents                              .
NABLAS株式会社
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
u86oixdj
 
一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单
一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单
一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单
ewymefz
 
一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单
enxupq
 
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
yhkoc
 
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
oz8q3jxlp
 
Q1’2024 Update: MYCI’s Leap Year Rebound
Q1’2024 Update: MYCI’s Leap Year ReboundQ1’2024 Update: MYCI’s Leap Year Rebound
Q1’2024 Update: MYCI’s Leap Year Rebound
Oppotus
 
Criminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdfCriminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdf
Criminal IP
 
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
nscud
 
Machine learning and optimization techniques for electrical drives.pptx
Machine learning and optimization techniques for electrical drives.pptxMachine learning and optimization techniques for electrical drives.pptx
Machine learning and optimization techniques for electrical drives.pptx
balafet
 
Sample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdf
Sample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdfSample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdf
Sample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdf
Linda486226
 
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
axoqas
 
Empowering Data Analytics Ecosystem.pptx
Empowering Data Analytics Ecosystem.pptxEmpowering Data Analytics Ecosystem.pptx
Empowering Data Analytics Ecosystem.pptx
benishzehra469
 
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
AbhimanyuSinha9
 
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Subhajit Sahu
 
一比一原版(CBU毕业证)卡普顿大学毕业证成绩单
一比一原版(CBU毕业证)卡普顿大学毕业证成绩单一比一原版(CBU毕业证)卡普顿大学毕业证成绩单
一比一原版(CBU毕业证)卡普顿大学毕业证成绩单
nscud
 
【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】
【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】
【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】
NABLAS株式会社
 
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
ahzuo
 
一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单
一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单
一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单
ewymefz
 

Recently uploaded (20)

一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
一比一原版(UniSA毕业证书)南澳大学毕业证如何办理
 
社内勉強会資料_LLM Agents                              .
社内勉強会資料_LLM Agents                              .社内勉強会資料_LLM Agents                              .
社内勉強会資料_LLM Agents                              .
 
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
原版制作(Deakin毕业证书)迪肯大学毕业证学位证一模一样
 
一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单
一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单
一比一原版(UPenn毕业证)宾夕法尼亚大学毕业证成绩单
 
一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单一比一原版(YU毕业证)约克大学毕业证成绩单
一比一原版(YU毕业证)约克大学毕业证成绩单
 
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
一比一原版(CU毕业证)卡尔顿大学毕业证成绩单
 
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
一比一原版(Deakin毕业证书)迪肯大学毕业证如何办理
 
Q1’2024 Update: MYCI’s Leap Year Rebound
Q1’2024 Update: MYCI’s Leap Year ReboundQ1’2024 Update: MYCI’s Leap Year Rebound
Q1’2024 Update: MYCI’s Leap Year Rebound
 
Criminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdfCriminal IP - Threat Hunting Webinar.pdf
Criminal IP - Threat Hunting Webinar.pdf
 
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
一比一原版(CBU毕业证)不列颠海角大学毕业证成绩单
 
Machine learning and optimization techniques for electrical drives.pptx
Machine learning and optimization techniques for electrical drives.pptxMachine learning and optimization techniques for electrical drives.pptx
Machine learning and optimization techniques for electrical drives.pptx
 
Sample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdf
Sample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdfSample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdf
Sample_Global Non-invasive Prenatal Testing (NIPT) Market, 2019-2030.pdf
 
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
哪里卖(usq毕业证书)南昆士兰大学毕业证研究生文凭证书托福证书原版一模一样
 
Empowering Data Analytics Ecosystem.pptx
Empowering Data Analytics Ecosystem.pptxEmpowering Data Analytics Ecosystem.pptx
Empowering Data Analytics Ecosystem.pptx
 
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...Best best suvichar in gujarati english meaning of this sentence as Silk road ...
Best best suvichar in gujarati english meaning of this sentence as Silk road ...
 
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
Algorithmic optimizations for Dynamic Levelwise PageRank (from STICD) : SHORT...
 
一比一原版(CBU毕业证)卡普顿大学毕业证成绩单
一比一原版(CBU毕业证)卡普顿大学毕业证成绩单一比一原版(CBU毕业证)卡普顿大学毕业证成绩单
一比一原版(CBU毕业证)卡普顿大学毕业证成绩单
 
【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】
【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】
【社内勉強会資料_Octo: An Open-Source Generalist Robot Policy】
 
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
一比一原版(UIUC毕业证)伊利诺伊大学|厄巴纳-香槟分校毕业证如何办理
 
一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单
一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单
一比一原版(IIT毕业证)伊利诺伊理工大学毕业证成绩单
 

How meta-analysis advances knowledge

  • 2. Risks of eating bacon? • colorectal cancer risk +18% risk / 50g of processed meat / day • overall risk = 1 in 17 = 5.8%
  • 3. Risks of eating bacon? 5.8% 6.9% 0% 2% 4% 6% 8% No bacon 50g / day % risk cancer
  • 4.
  • 5. Theory of Signal Detection (TSD) Noise Signal
  • 6. Dangers of NHST Hit ! False alarm Miss Correct rejection Effect No effect Think so Think not
  • 7. Overlooking effect sizes “I’ve heard it before”… …belies the uncertainty of science
  • 8. Open Science Collaboration Science 2015;349:aac4716 The price of oversight • The Reproducibility Project • p-hacking
  • 9. Advancing knowledge Theatre of theory • top-down • theoretical • ideas • hypothetico-deductive • NHST Empire of empirical • bottom-up • empirical • data • fishing trip • effects
  • 10. Effect size measures • absolute measures • correlational • r, R2, eta-sq • differences • SMDs, Cohen’s d, Hedge’s g • categorical: odds-ratio, relative-risk • raw-mean score differences • relative measures • % change, elasticities
  • 11. Sales SoM Choice Purchase Consump’n Learning Memory Perception - - - - Motivation Beliefs Attitudes Intentions Psychology Behavior Outcome Advertising Cons’r bhvr Channels Pricing NPD Sales Strategy Interventions Chain of effect
  • 12. Effect of attitudes on behaviour? • r=.5 • k=128, N=4,598 • Glasman & Albarracin (2006) • stronger when • attitudes are more accessible • direct experience and/or reported frequently • attitudes are more stable over time • confident, based on bhvr relevant information, one-sided info
  • 13. Effect of anti-depressants? • Cohen’s d < .2 • for HDRS < 23 (mild or moderate), placebo vs ADM (anti-depressant medication) • HDRS scale runs 0 to 52 • pax treated with ADMs average 2 points higher on HDRS than placebo! • statistical significance vs clinical significance • https://www.scientificamerican.com/article/antidepressants-do- they-work-or-dont-they/
  • 16. Effect of advertising on sales • advtg elasticity = .12 • ie, a 10% increase in advertising  1.2% increase in sales • price elasticity = -2.3 • ie, a 10% increase in price  23% reduction in sales • banner click-through-rate (CTR) • .3 to .05% (PWC report)
  • 17. Effect of portion-size on obesity? • effect on behaviour • d = .45 • elasticity = .35 • effect on weight • .56kg/mo
  • 18. Effect of health supplements
  • 19. Meta-meta-analysis of advance in knowledge • 176 meta-analyses • >7,500 primary studies (43 studies per meta-analysis) published between 1918 and 2012 • >54,000 effect sizes (307 effect sizes per meta-analysis) • sample of 8,337,096 subjects (based on primary studies of 131 meta-analyses) • Eisend 2015 JM
  • 20. Meta-meta-analytic effect sizes Corr (r) r2 All effect sizes .24 5.6% Advertising .21 4.6% Channels .24 5.7% Consumer behaviour .28 8.1% Method .15 2.4% New product development .20 4.1% Pricing .36 13.0% Sales .23 5.2% Strategy .25 6.1%
  • 21.
  • 22.
  • 23. Advancing knowledge Theatre of theory • top-down • theoretical • ideas • hypothetico-deductive • NHST Empire of empirical • bottom-up • empirical • data • fishing trip • effects
  • 24. It’s not the end of the world if you’re wrong !
  • 25. How meta-analysis has helped to advance knowledge Stephen S Holden
  • 26. • research designs • preferential treatment of experiments (RCT) • selection bias • file drawer • publication bias • heterogeneity / moderator analyses • non-random sample of domain • fixed vs random effects • collinearity Contentious issues

Editor's Notes

  1. 10 cohort studies 5.8% risk background risk vs 6.9% risk if eating 50g of bacon a day ! +17% risk / 100g of red meat / day cf men who smoke have 20x risk of lung cancer
  2. SIMULATED 95% CIs of +/- 0.5% what does it tell us if the CIs are smaller? larger?
  3. statistically significant vs clinically (or practical) significance statistical significance is a function of sample size, p-value, and effect-size (bigger effects are more likely to be “statistically significant”)
  4. Tversky & Kahneman (1971) “Belief in the law of small numbers” Psych Bull expt n=20 generated a significant result, z=2.23, p<.05 Well (1991) “The perils of N=1” JCR Bass (1995) “Empirical generalizations in marketing science: a personal view” Mktg Science McShane & Gal (2015) “Blinding Us to the Obvious? The Effect of Statistical Training on the Evaluation of Evidence” Mgt Science NHST encouraging dichotomous view
  5. Original study effect size versus replication effect size (correlation coefficients). Diagonal line represents replication effect size equal to original effect size. Dotted line represents replication effect size of 0. Points below the dotted line were effects in the opposite direction of the original. Density plots are separated by significant (blue) and nonsignificant (red) effects.
  6. the stage is set and the play requires hypotheses, test, conclusions so we create hypotheses to explain the data - after they are collected !
  7. Hamilton Depression Rating Scale (HDRS) k=6, 718 patients
  8. breakage rate of condoms is 2% you have more chance of a condom failing than someone clicking on your banner ad! highlighting that mktrs don’t control powerful effects, but are playing in the ultimate numbers game
  9. the stage is set and the play requires hypotheses, test, conclusions so we create hypotheses to explain the data - after they are collected !
  10. Go fishing, let your suspicions run wild, choose those which have evidence, accept that there may be some false alarms