The document discusses several issues with commonly used statistical methods and p-values. It notes that p-values test more than just the null hypothesis and can be unreliable indicators. When effects are rare or experiments are underpowered, false discovery rates may be much higher than the significance level of 0.05. Multiple testing corrections and determining what constitutes a "negative result" can also be problematic. Widespread practices like p-hacking and non-publication of non-significant results undermine the reliability of published findings. Overall, the document argues that statistical testing and interpretation requires careful consideration of assumptions, biases, and limitations to avoid "making a fool of yourself".
A comment in Nature, signed by over 800 researchers, called for a rise up against statistical significance. This was followed by a special issue of The American Statistician aimed at halting the use of the term "statistically significant", and new guidelines for statistical reporting in the New England Journal of Medicine. These slides discuss the broader context of the "p-value crisis" and alternatives for communicating the conclusions after statistical analyses.
Target audience: Medical researchers; Scientists involved in conducting or interpreting analyses and communicating the results of scientific research, as well as readers of scientific publications.
Learning objectives:
To understand the context of the reproducibility crisis in medical research.
To learn about problems with p-values and alternatives to report findings.
To understand how (not) to interpret significant and insignificant findings.
To learn how to communicate research findings in a modest, thoughtful, and transparent way.
What is the significance of p value while reporting statistical analysis. Is there an alternate approach for Fisher, if so what is that approach. These are some of the issues addressed here.
This document discusses statistical principles for writing scientific manuscripts, including how to describe sampling uncertainty, present results using measures of central tendency and variability, and report findings using confidence intervals and p-values. It emphasizes quantifying and conveying measurement uncertainty and effect sizes rather than relying solely on hypothesis testing. Guidelines are proposed for systematically reporting the design, methods, results and interpretation of laboratory experiments to improve transparency and enable verification.
Check out this brief paper if you want to know more about P value calculations. There is a misconception that a very small p value means the difference between groups is highly relevant. Looking at the p value alone deviates our attention from the effect size. Consider an experiment in which 10 subjects receive a placebo, and another 10 receive an experimental diuretic. After 8 h, the average urine output in the placebo group is 769 mL, versus 814 mL in the diuretic group—a difference of 45 mL. How do we know if that difference means the drug works and is not just a result of chance? Read on and let me know if you have any questions...
P-values the gold measure of statistical validity are not as reliable as many...David Pratap
This is an article that appeared in the NATURE as News Feature dated 12-February-2014. This article was presented in the journal club at Oman Medical College , Bowshar Campus on December, 17, 2015. This article was presented by Pratap David , Biostatistics Lecturer.
This document provides an overview of basic probability concepts and probability distributions. It defines probability as the likelihood of an event occurring between 0 and 1, and explains the roles of probability and statistics. It then covers various ways to calculate probability using classical, relative frequency and subjective approaches. Other topics discussed include events, sample spaces, conditional probabilities, the law of large numbers, rules of probability, and random variables.
Excursions into the garden of the forking paths Ulrich Dirnagl
The document discusses issues with reproducibility and replication in biomedical research. It notes decades of failed translational stroke research due to biases, low statistical power, and other issues. It argues that both exploration and confirmation are needed, but they are often conflated. Exploration aims to generate new theories while confirmation aims to demonstrate treatment effects. The document provides suggestions to reduce biases, increase power, practice open science through preregistration and data sharing, and consider effect sizes over sole reliance on p-values. Failure to replicate may be due to cuting-edge research pushing boundaries rather than a false positive original result. Both replication and non-replication can provide valuable scientific insights when properly interpreted.
This document discusses various statistical concepts related to analyzing results from multiple studies, including effect sizes, p-values, meta-analysis, and issues that can arise such as publication bias, multiple hypothesis testing, and heterogeneity. It provides examples of how effect sizes are calculated, how meta-analyses combine results across studies, and statistical methods that can help address problems like multiple comparisons, including Bonferroni corrections and false discovery rate analysis. The document cautions that when analyzing large datasets, there is a risk of finding spurious correlations due to chance that may have no predictive value.
A comment in Nature, signed by over 800 researchers, called for a rise up against statistical significance. This was followed by a special issue of The American Statistician aimed at halting the use of the term "statistically significant", and new guidelines for statistical reporting in the New England Journal of Medicine. These slides discuss the broader context of the "p-value crisis" and alternatives for communicating the conclusions after statistical analyses.
Target audience: Medical researchers; Scientists involved in conducting or interpreting analyses and communicating the results of scientific research, as well as readers of scientific publications.
Learning objectives:
To understand the context of the reproducibility crisis in medical research.
To learn about problems with p-values and alternatives to report findings.
To understand how (not) to interpret significant and insignificant findings.
To learn how to communicate research findings in a modest, thoughtful, and transparent way.
What is the significance of p value while reporting statistical analysis. Is there an alternate approach for Fisher, if so what is that approach. These are some of the issues addressed here.
This document discusses statistical principles for writing scientific manuscripts, including how to describe sampling uncertainty, present results using measures of central tendency and variability, and report findings using confidence intervals and p-values. It emphasizes quantifying and conveying measurement uncertainty and effect sizes rather than relying solely on hypothesis testing. Guidelines are proposed for systematically reporting the design, methods, results and interpretation of laboratory experiments to improve transparency and enable verification.
Check out this brief paper if you want to know more about P value calculations. There is a misconception that a very small p value means the difference between groups is highly relevant. Looking at the p value alone deviates our attention from the effect size. Consider an experiment in which 10 subjects receive a placebo, and another 10 receive an experimental diuretic. After 8 h, the average urine output in the placebo group is 769 mL, versus 814 mL in the diuretic group—a difference of 45 mL. How do we know if that difference means the drug works and is not just a result of chance? Read on and let me know if you have any questions...
P-values the gold measure of statistical validity are not as reliable as many...David Pratap
This is an article that appeared in the NATURE as News Feature dated 12-February-2014. This article was presented in the journal club at Oman Medical College , Bowshar Campus on December, 17, 2015. This article was presented by Pratap David , Biostatistics Lecturer.
This document provides an overview of basic probability concepts and probability distributions. It defines probability as the likelihood of an event occurring between 0 and 1, and explains the roles of probability and statistics. It then covers various ways to calculate probability using classical, relative frequency and subjective approaches. Other topics discussed include events, sample spaces, conditional probabilities, the law of large numbers, rules of probability, and random variables.
Excursions into the garden of the forking paths Ulrich Dirnagl
The document discusses issues with reproducibility and replication in biomedical research. It notes decades of failed translational stroke research due to biases, low statistical power, and other issues. It argues that both exploration and confirmation are needed, but they are often conflated. Exploration aims to generate new theories while confirmation aims to demonstrate treatment effects. The document provides suggestions to reduce biases, increase power, practice open science through preregistration and data sharing, and consider effect sizes over sole reliance on p-values. Failure to replicate may be due to cuting-edge research pushing boundaries rather than a false positive original result. Both replication and non-replication can provide valuable scientific insights when properly interpreted.
This document discusses various statistical concepts related to analyzing results from multiple studies, including effect sizes, p-values, meta-analysis, and issues that can arise such as publication bias, multiple hypothesis testing, and heterogeneity. It provides examples of how effect sizes are calculated, how meta-analyses combine results across studies, and statistical methods that can help address problems like multiple comparisons, including Bonferroni corrections and false discovery rate analysis. The document cautions that when analyzing large datasets, there is a risk of finding spurious correlations due to chance that may have no predictive value.
"The Statistical Replication Crisis: Paradoxes and Scapegoats”jemille6
D. G. Mayo LSE Popper talk, May 10, 2016.
Abstract: Mounting failures of replication in the social and biological sciences give a practical spin to statistical foundations in the form of the question: How can we attain reliability when Big Data methods make illicit cherry-picking and significance seeking so easy? Researchers, professional societies, and journals are increasingly getting serious about methodological reforms to restore scientific integrity – some are quite welcome (e.g., preregistration), while others are quite radical. Recently, the American Statistical Association convened members from differing tribes of frequentists, Bayesians, and likelihoodists to codify misuses of P-values. Largely overlooked are the philosophical presuppositions of both criticisms and proposed reforms. Paradoxically, alternative replacement methods may enable rather than reveal illicit inferences due to cherry-picking, multiple testing, and other biasing selection effects. Popular appeals to “diagnostic testing” that aim to improve replication rates may (unintentionally) permit the howlers and cookbook statistics we are at pains to root out. Without a better understanding of the philosophical issues, we can expect the latest reforms to fail.
Research Sample size by Dr Allah Yar Malikhuraismalik
The document discusses sample size calculations and hypothesis testing. It provides formulas for calculating sample sizes for estimating proportions, means, and comparing two groups. The key factors that influence sample size calculations are the level of significance, power, anticipated effect size, and standard deviation. Hypothesis testing involves stating a null hypothesis, choosing a suitable statistical test, calculating a test statistic and p-value, and deciding whether to reject or fail to reject the null hypothesis based on a cutoff p-value of 0.05 typically. Websites for online sample size calculators are also provided.
Statistics and experimental design are important for drawing valid conclusions from research. Well-designed experiments produce unbiased comparisons, precise estimates, and account for variability. Hypothesis tests answer yes/no questions about population values and aim to reject false null hypotheses. P-values indicate the likelihood of obtaining extreme data if the null is true. Multiple testing increases chances of false positives, requiring adjustments. Sample size impacts power to detect effects and precision of estimates. Both statistical and practical significance must be considered.
This presentation discusses the following topics:
Hypothesis Test
Potential Outcomes in Hypothesis Testing
Significance level
P-value
Sampling Errors
Type I Error
What causes Type I errors?
What causes Type II errors?
4 possible outcomes
This document provides guidance on how to read, analyze, and critique a scientific study. It discusses key concepts like the null hypothesis, statistical significance, and types of data and appropriate statistical tests. It also outlines important steps to follow when reviewing a study, including understanding the study design, evaluating the data collection and analysis, interpreting graphs and statistics, and carefully considering the discussion and conclusions. Finally, it identifies several common pitfalls to watch out for related to statistical analysis and presentation of results.
This document discusses the appropriate use and interpretation of p-values. It defines what a p-value represents and outlines some common misuses, such as using p-values to describe the strength of an effect or double dipping in the data. Alternative approaches like confidence intervals and effect sizes are presented that provide more meaningful information about study results than p-values alone. Examples from clinical studies are provided where p-values were inappropriately emphasized over other important results.
The document provides guidance on improving the chances of getting a manuscript accepted for publication. It discusses key methodological concepts like uncertainty of measurement and sampling, statistical assumptions, multiplicity, and proper reporting of different study types including case reports, experiments, observational studies, and randomized trials. The key recommendations are to 1) clearly describe statistical methods and sample sizes, 2) present both data and interpretation of results considering clinical and statistical significance, and 3) properly adjust for confounding and comply with reporting standards for different study designs.
Hypothesis testing involves stating a null hypothesis (H0) and an alternative hypothesis (H1). H0 assumes there is no effect or relationship in the population. H1 states there is an effect. A study is conducted and statistics are used to determine if the data supports rejecting H0 in favor of H1. The p-value indicates the probability of obtaining results as extreme as the observed data or more extreme if H0 is true. If p ≤ the predetermined significance level (α = 0.05), H0 is rejected in favor of H1. Otherwise, H0 is retained but not proven true. Type I and II errors can occur when the true hypothesis is incorrectly rejected or retained.
This document discusses key concepts in research methods and biostatistics, including hypothesis testing, random error, p-values, and confidence intervals. It explains that hypothesis testing involves determining if study findings reflect chance or a true effect. The p-value represents the probability of observing results as extreme or more extreme than what was observed by chance alone. A p-value less than 0.05 indicates statistical significance. Confidence intervals provide a range of values that are likely to contain the true population parameter.
These slides were presented on November 22 2016 during the Annual Julius Symposium, organised by the Julius Center for Health Sciences and Primary Care, University Medical Hospital Utrecht.
Only a few months ago, the American Statistical Association authoritatively issued an official statement on significance and p-values (American Statistician, 2016, 70:2, 129-133), claiming that the p-value is: “commonly misused and misinterpreted.”
In this presentation I focus on the principles of the ASA statement.
This document provides an overview of biostatistics. It defines biostatistics and discusses variables that can be studied, including discrete and continuous variables. It describes common software used for analysis and summarizes typical descriptive measures like mean, median, standard deviation, etc. The document outlines common types of comparisons between continuous and categorical variables, including t-tests, ANOVA, and chi-square tests. It also discusses concepts like alpha, beta, power, and cautions around hypothesis testing and interpreting statistical significance.
- A/B testing involves randomized controlled experiments comparing a treatment group to a control group. However, there are various sources of variability beyond just the treatment that must be accounted for.
- Good experiment design aims to minimize bias and convert it to random noise through randomization. The role of statistics is to quantify the magnitude of the treatment effect compared to the noise.
- Classical hypothesis testing approaches the problem as "assuming no difference and seeing if the data contradicts that". However, concerns with this approach include overreliance on p-values and not addressing multiple testing.
- Bayesian approaches consider the probability of there being a difference given the data, but require specifying a prior probability which is challenging. Alternatives like multi-
Novelties in social science statisticsJiri Haviger
Jiří Haviger from the University of Hradec Kralove discusses key topics in social science statistics including:
1) Determining sample size by specifying the acceptable type I error rate (α), power (1- β), and estimated effect size to calculate the required sample size.
2) Using machine learning in social research for prediction rather than theory testing, with supervised models trained on data and evaluated on held-out samples.
3) Structural equation modeling to evaluate theoretical models describing relationships between observed and latent variables, with software like JASP used to estimate models and evaluate fit.
Slides given for Deborah G. Mayo talk at Minnesota Center for Philosophy of Science at University of Minnesota on the ASA 2016 statement on P-values and Error Statistics
Hypothesis testing is a formal statistical procedure used to investigate ideas about the world. There are 5 main steps: 1) state the null and alternate hypotheses, 2) collect representative data, 3) perform an appropriate statistical test comparing within- and between-group variances, 4) decide whether to reject or fail to reject the null based on the p-value, and 5) present the findings. The document provides an example where a t-test found the average height difference between men and women was 13.7cm with a p-value of 0.002, allowing the researcher to reject the null hypothesis that there is no difference.
The document provides guidelines for improving statistical analysis and reporting in research manuscripts submitted for publication. It discusses key methodological concepts like measurement uncertainty, sampling uncertainty, p-values, confidence intervals, and assumptions of statistical tests. It provides special recommendations for different study designs including case reports, experiments, observational studies, and randomized trials. The document emphasizes presenting details of statistical methods, quantifying findings with precision measures, avoiding sole reliance on p-values, adjusting for confounding, and following reporting guidelines for randomized trials.
Psbe2 08 research methods 2011-2012 - week 2Vlady Fckfb
The document discusses null hypothesis significance testing (NHST) and power. It explains that NHST is used to determine if mean differences between groups are greater than would be expected by chance. A statistically significant result is one that has a low probability of occurring if the null hypothesis is true. The level of significance is typically set at p<.05. Limitations of NHST include that small sample sizes can lead to a lack of power to detect effects. Power refers to the likelihood of correctly rejecting the null hypothesis and depends on sample size, significance level, and effect size. Prospective power analyses estimate the needed sample size while retrospective analyses determine a study's power based on its parameters.
Psbe2 08 research methods 2011-2012 - week 2Vlady Fckfb
The document discusses null hypothesis significance testing (NHST) and power. It explains that NHST is used to determine if mean differences between groups are greater than would be expected by chance. A statistically significant result is one that has a low probability of occurring if the null hypothesis is true. The level of significance is typically set at p<.05. Limitations of NHST include that small sample sizes may lack power to detect real effects. The document also discusses how power and effect size determine the likelihood of correctly rejecting the null hypothesis and avoiding Type II errors.
"The Statistical Replication Crisis: Paradoxes and Scapegoats”jemille6
D. G. Mayo LSE Popper talk, May 10, 2016.
Abstract: Mounting failures of replication in the social and biological sciences give a practical spin to statistical foundations in the form of the question: How can we attain reliability when Big Data methods make illicit cherry-picking and significance seeking so easy? Researchers, professional societies, and journals are increasingly getting serious about methodological reforms to restore scientific integrity – some are quite welcome (e.g., preregistration), while others are quite radical. Recently, the American Statistical Association convened members from differing tribes of frequentists, Bayesians, and likelihoodists to codify misuses of P-values. Largely overlooked are the philosophical presuppositions of both criticisms and proposed reforms. Paradoxically, alternative replacement methods may enable rather than reveal illicit inferences due to cherry-picking, multiple testing, and other biasing selection effects. Popular appeals to “diagnostic testing” that aim to improve replication rates may (unintentionally) permit the howlers and cookbook statistics we are at pains to root out. Without a better understanding of the philosophical issues, we can expect the latest reforms to fail.
Research Sample size by Dr Allah Yar Malikhuraismalik
The document discusses sample size calculations and hypothesis testing. It provides formulas for calculating sample sizes for estimating proportions, means, and comparing two groups. The key factors that influence sample size calculations are the level of significance, power, anticipated effect size, and standard deviation. Hypothesis testing involves stating a null hypothesis, choosing a suitable statistical test, calculating a test statistic and p-value, and deciding whether to reject or fail to reject the null hypothesis based on a cutoff p-value of 0.05 typically. Websites for online sample size calculators are also provided.
Statistics and experimental design are important for drawing valid conclusions from research. Well-designed experiments produce unbiased comparisons, precise estimates, and account for variability. Hypothesis tests answer yes/no questions about population values and aim to reject false null hypotheses. P-values indicate the likelihood of obtaining extreme data if the null is true. Multiple testing increases chances of false positives, requiring adjustments. Sample size impacts power to detect effects and precision of estimates. Both statistical and practical significance must be considered.
This presentation discusses the following topics:
Hypothesis Test
Potential Outcomes in Hypothesis Testing
Significance level
P-value
Sampling Errors
Type I Error
What causes Type I errors?
What causes Type II errors?
4 possible outcomes
This document provides guidance on how to read, analyze, and critique a scientific study. It discusses key concepts like the null hypothesis, statistical significance, and types of data and appropriate statistical tests. It also outlines important steps to follow when reviewing a study, including understanding the study design, evaluating the data collection and analysis, interpreting graphs and statistics, and carefully considering the discussion and conclusions. Finally, it identifies several common pitfalls to watch out for related to statistical analysis and presentation of results.
This document discusses the appropriate use and interpretation of p-values. It defines what a p-value represents and outlines some common misuses, such as using p-values to describe the strength of an effect or double dipping in the data. Alternative approaches like confidence intervals and effect sizes are presented that provide more meaningful information about study results than p-values alone. Examples from clinical studies are provided where p-values were inappropriately emphasized over other important results.
The document provides guidance on improving the chances of getting a manuscript accepted for publication. It discusses key methodological concepts like uncertainty of measurement and sampling, statistical assumptions, multiplicity, and proper reporting of different study types including case reports, experiments, observational studies, and randomized trials. The key recommendations are to 1) clearly describe statistical methods and sample sizes, 2) present both data and interpretation of results considering clinical and statistical significance, and 3) properly adjust for confounding and comply with reporting standards for different study designs.
Hypothesis testing involves stating a null hypothesis (H0) and an alternative hypothesis (H1). H0 assumes there is no effect or relationship in the population. H1 states there is an effect. A study is conducted and statistics are used to determine if the data supports rejecting H0 in favor of H1. The p-value indicates the probability of obtaining results as extreme as the observed data or more extreme if H0 is true. If p ≤ the predetermined significance level (α = 0.05), H0 is rejected in favor of H1. Otherwise, H0 is retained but not proven true. Type I and II errors can occur when the true hypothesis is incorrectly rejected or retained.
This document discusses key concepts in research methods and biostatistics, including hypothesis testing, random error, p-values, and confidence intervals. It explains that hypothesis testing involves determining if study findings reflect chance or a true effect. The p-value represents the probability of observing results as extreme or more extreme than what was observed by chance alone. A p-value less than 0.05 indicates statistical significance. Confidence intervals provide a range of values that are likely to contain the true population parameter.
These slides were presented on November 22 2016 during the Annual Julius Symposium, organised by the Julius Center for Health Sciences and Primary Care, University Medical Hospital Utrecht.
Only a few months ago, the American Statistical Association authoritatively issued an official statement on significance and p-values (American Statistician, 2016, 70:2, 129-133), claiming that the p-value is: “commonly misused and misinterpreted.”
In this presentation I focus on the principles of the ASA statement.
This document provides an overview of biostatistics. It defines biostatistics and discusses variables that can be studied, including discrete and continuous variables. It describes common software used for analysis and summarizes typical descriptive measures like mean, median, standard deviation, etc. The document outlines common types of comparisons between continuous and categorical variables, including t-tests, ANOVA, and chi-square tests. It also discusses concepts like alpha, beta, power, and cautions around hypothesis testing and interpreting statistical significance.
- A/B testing involves randomized controlled experiments comparing a treatment group to a control group. However, there are various sources of variability beyond just the treatment that must be accounted for.
- Good experiment design aims to minimize bias and convert it to random noise through randomization. The role of statistics is to quantify the magnitude of the treatment effect compared to the noise.
- Classical hypothesis testing approaches the problem as "assuming no difference and seeing if the data contradicts that". However, concerns with this approach include overreliance on p-values and not addressing multiple testing.
- Bayesian approaches consider the probability of there being a difference given the data, but require specifying a prior probability which is challenging. Alternatives like multi-
Novelties in social science statisticsJiri Haviger
Jiří Haviger from the University of Hradec Kralove discusses key topics in social science statistics including:
1) Determining sample size by specifying the acceptable type I error rate (α), power (1- β), and estimated effect size to calculate the required sample size.
2) Using machine learning in social research for prediction rather than theory testing, with supervised models trained on data and evaluated on held-out samples.
3) Structural equation modeling to evaluate theoretical models describing relationships between observed and latent variables, with software like JASP used to estimate models and evaluate fit.
Slides given for Deborah G. Mayo talk at Minnesota Center for Philosophy of Science at University of Minnesota on the ASA 2016 statement on P-values and Error Statistics
Hypothesis testing is a formal statistical procedure used to investigate ideas about the world. There are 5 main steps: 1) state the null and alternate hypotheses, 2) collect representative data, 3) perform an appropriate statistical test comparing within- and between-group variances, 4) decide whether to reject or fail to reject the null based on the p-value, and 5) present the findings. The document provides an example where a t-test found the average height difference between men and women was 13.7cm with a p-value of 0.002, allowing the researcher to reject the null hypothesis that there is no difference.
The document provides guidelines for improving statistical analysis and reporting in research manuscripts submitted for publication. It discusses key methodological concepts like measurement uncertainty, sampling uncertainty, p-values, confidence intervals, and assumptions of statistical tests. It provides special recommendations for different study designs including case reports, experiments, observational studies, and randomized trials. The document emphasizes presenting details of statistical methods, quantifying findings with precision measures, avoiding sole reliance on p-values, adjusting for confounding, and following reporting guidelines for randomized trials.
Psbe2 08 research methods 2011-2012 - week 2Vlady Fckfb
The document discusses null hypothesis significance testing (NHST) and power. It explains that NHST is used to determine if mean differences between groups are greater than would be expected by chance. A statistically significant result is one that has a low probability of occurring if the null hypothesis is true. The level of significance is typically set at p<.05. Limitations of NHST include that small sample sizes can lead to a lack of power to detect effects. Power refers to the likelihood of correctly rejecting the null hypothesis and depends on sample size, significance level, and effect size. Prospective power analyses estimate the needed sample size while retrospective analyses determine a study's power based on its parameters.
Psbe2 08 research methods 2011-2012 - week 2Vlady Fckfb
The document discusses null hypothesis significance testing (NHST) and power. It explains that NHST is used to determine if mean differences between groups are greater than would be expected by chance. A statistically significant result is one that has a low probability of occurring if the null hypothesis is true. The level of significance is typically set at p<.05. Limitations of NHST include that small sample sizes may lack power to detect real effects. The document also discusses how power and effect size determine the likelihood of correctly rejecting the null hypothesis and avoiding Type II errors.
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Round table discussion of vector databases, unstructured data, ai, big data, real-time, robots and Milvus.
A lively discussion with NJ Gen AI Meetup Lead, Prasad and Procure.FYI's Co-Found
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...sameer shah
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
Learn SQL from basic queries to Advance queriesmanishkhaire30
Dive into the world of data analysis with our comprehensive guide on mastering SQL! This presentation offers a practical approach to learning SQL, focusing on real-world applications and hands-on practice. Whether you're a beginner or looking to sharpen your skills, this guide provides the tools you need to extract, analyze, and interpret data effectively.
Key Highlights:
Foundations of SQL: Understand the basics of SQL, including data retrieval, filtering, and aggregation.
Advanced Queries: Learn to craft complex queries to uncover deep insights from your data.
Data Trends and Patterns: Discover how to identify and interpret trends and patterns in your datasets.
Practical Examples: Follow step-by-step examples to apply SQL techniques in real-world scenarios.
Actionable Insights: Gain the skills to derive actionable insights that drive informed decision-making.
Join us on this journey to enhance your data analysis capabilities and unlock the full potential of SQL. Perfect for data enthusiasts, analysts, and anyone eager to harness the power of data!
#DataAnalysis #SQL #LearningSQL #DataInsights #DataScience #Analytics
The Ipsos - AI - Monitor 2024 Report.pdfSocial Samosa
According to Ipsos AI Monitor's 2024 report, 65% Indians said that products and services using AI have profoundly changed their daily life in the past 3-5 years.
Codeless Generative AI Pipelines
(GenAI with Milvus)
https://ml.dssconf.pl/user.html#!/lecture/DSSML24-041a/rate
Discover the potential of real-time streaming in the context of GenAI as we delve into the intricacies of Apache NiFi and its capabilities. Learn how this tool can significantly simplify the data engineering workflow for GenAI applications, allowing you to focus on the creative aspects rather than the technical complexities. I will guide you through practical examples and use cases, showing the impact of automation on prompt building. From data ingestion to transformation and delivery, witness how Apache NiFi streamlines the entire pipeline, ensuring a smooth and hassle-free experience.
Timothy Spann
https://www.youtube.com/@FLaNK-Stack
https://medium.com/@tspann
https://www.datainmotion.dev/
milvus, unstructured data, vector database, zilliz, cloud, vectors, python, deep learning, generative ai, genai, nifi, kafka, flink, streaming, iot, edge
State of Artificial intelligence Report 2023kuntobimo2016
Artificial intelligence (AI) is a multidisciplinary field of science and engineering whose goal is to create intelligent machines.
We believe that AI will be a force multiplier on technological progress in our increasingly digital, data-driven world. This is because everything around us today, ranging from culture to consumer products, is a product of intelligence.
The State of AI Report is now in its sixth year. Consider this report as a compilation of the most interesting things we’ve seen with a goal of triggering an informed conversation about the state of AI and its implication for the future.
We consider the following key dimensions in our report:
Research: Technology breakthroughs and their capabilities.
Industry: Areas of commercial application for AI and its business impact.
Politics: Regulation of AI, its economic implications and the evolving geopolitics of AI.
Safety: Identifying and mitigating catastrophic risks that highly-capable future AI systems could pose to us.
Predictions: What we believe will happen in the next 12 months and a 2022 performance review to keep us honest.
3. 1,000 tests
real effect in 10%
100 tests
no effect in 90%
900 tests
Pr(real) = 0.1
power = 80%
significance level = 5%
80% detected
80 true positives
20% not detected
20 false negatives
95% tested negative
855 true negatives
5% “detected”
45 false positives
False discovery rate
𝐹𝐷𝑅 =
false positives
discoveries
𝐹𝐷𝑅 =
45
45 + 80
= 0.36
Colquhoun D., 2014, “An investigation of the false discovery rate
and the misinterpretation of p-values”, R. Soc. open sci. 1: 140216.
False positive rate
𝐹𝑃𝑅 =
false positives
no effect
𝐹𝑃𝑅 =
45
900
= 0.05
4. If you publish a 𝑝 < 0.05
result, you have a 36%
chance of making a fool
of yourself
Colquhoun D., 2014, “An investigation of the false discovery rate
and the misinterpretation of p-values”, R. Soc. open sci. 1: 140216.
1,000 tests
real effect in 10%
100 tests
no effect in 90%
900 tests
Pr(real) = 0.1
power = 80%
significance level = 5%
80% detected
80 true positives
20% not detected
20 false negatives
95% tested negative
855 true negatives
5% “detected”
45 false positives
5. What’s wrong with p-values?
Hand-outs available at http://is.gd/statlec
Marek Gierliński
Division of Computational Biology
7. Statistical testing
Statistical model
Null hypothesis
H0: no effect
All other assumptions
Significance level
𝛼 = 0.05
p-value: probability that the
observed effect is random
𝑝 < 𝛼
Reject H0
(at your own risk)
Effect is real
𝑝 ≥ 𝛼
Insufficient evidence
Statistical test against H0
Data
8. p-value:
Given that H0 is true, the probability of
observed, or more extreme, data
It is not the probability that H0 is true
9. P-value is the degree to which the data are
embarrassed by the null hypothesis
Nicholas Maxwell
10. “All other assumptions”
Null hypothesis
H0: no effect
Significance level
𝛼 = 0.05
𝑝 < 𝛼
Reject H0
𝑝 ≥ 𝛼
Do not reject H0
Experimental
protocols followed
Instruments
calibrated
Data collected
correctly
All other assumptions
about biology are
correct No other effects
No other effects
𝑝 < 𝛼
Reject H0
No silly misakes
11. 1
p-values test not only the null hypothesis,
but everything else in the experiment
12. 𝐹𝐷𝑅 =
45
45 + 80
= 0.36
1,000 tests
real effect in 10%
100 tests
no effect in 90%
900 tests
Pr(real) = 0.1
power = 80%
significance level = 5%
80% detected
80 true positives
20% not detected
20 false negatives
95% tested negative
855 true negatives
5% “detected”
45 false positives
Why large false discovery rate?
13. Simulated population of mice
13
No effect (97%)
𝜇C = 20 g
𝜎 = 5 g
Real effect (3%)
𝜇F = 30 g
𝜎 = 5 g
Power analysis
effect size Δ𝑚 = 10 g
power 𝒫 = 0.9
significance level 𝛼 = 0.05
sample size 𝑛 = 5
Null hypothesis H0: 𝜇 = 20 g
one-sample t-test
23. Gedankenexperiment: reliability of p-values
23
Normal population, 100% real effect
One-sample t-test
p-values can be
unreliable
Sample size = 3, power = 0.18 Sample size = 10, power = 0.80
28. Neuroscience: most studies underpowered
Button et al. (2013) “Power failure: why small sample size undermines the
reliability of neuroscience”, Nature Reviews Neuroscience 14, 365-376
30. The effect size
With sample size large
enough everything is
“significant”
Effect size is more important
Looking at whole data is
even more important
𝑝 = 0.004
𝑛F = 775 𝑛Q = 392
33. Multiple test corrections can be tricky
10,000 genes 10,000 tests
Benjamini-Hochberg
correction
RESULT
34. Multiple test corrections can be tricky
10,000 genes 10,000 tests
Benjamini-Hochberg
correction
RESULT
Complex experiment
Multi-dimensional data
Searching...
Nothing Searching...
Nothing
Searching...
Nothing
Searching...
RESULT
Batch effects? No
35. 8
It is not always obvious how to correct
p-values
36. What’s wrong with p-values?
P-values test not only the
targeted null hypothesis,
but everything else in the
experiment
The chance of making a
fool of yourself is much
larger than 𝛼 = 0.05
When you get a
𝑝 ~ 0.05, you are
screwed
When your experiment is
underpowered, you are
screwed
Multiple test corrections
are tricky
A lot, because
statistics
When the effect is rare,
you are screwed
Statistical significance
does not imply biological
relevance
When you have lots of
replicates, p-values are
useless
P-values test not only the
targeted null hypothesis,
but everything else in the
experiment
When the effect is rare,
you are screwed
When you get a
𝑝 ~ 0.05, you are
screwed
When your experiment is
underpowered, you are
screwed
Statistical significance
does not imply biological
relevance
When you have lots of
replicates, p-values are
useless
Multiple test corrections
are tricky
The chance of making a
fool of yourself is much
larger than 𝛼 = 0.05
37.
38. The plain fact is that 70 years ago
Ronald Fisher gave scientists a
mathematical machine for turning
baloney into breakthroughs, and flukes
into funding. It is time to pull the plug.
Robert Matthews, Sunday Telegraph,
13 September 1998.
Null hypothesis significance testing is a potent
but sterile intellectual rake who leaves in his
merry path a long train of ravished maidens
but no viable scientific offspring.
Paul Meehl, 1967, Philosophy of Science, 34,
103-115
The widespread use of “statistical
significance” as a license for
making a claim of a scientific
finding leads to considerable
distortion of the scientific process.
ASA statement on statistical
significance and p-values (2016)
39. By Jim Borgman, first published by the Cincinnati Inquirer 27 April 1997
41. Journal of the American Statistical Association,
Vol. 54, No. 285 (Mar., 1959), pp. 30-34
“There is some evidence that [...] research which yields nonsigificant results is not
published. Such research being unknown to other investigators may be repeated
independently until eventually by chance a significant result occurs [...] The
possibility thus arises that the literature [...] consists in substantial part of false
conclusions [...].”
42. Canonization of false facts
Nissen S.B., et al., “Research: Publication bias and the
canonization of false facts”, eLife 2016;5:e21451
43. Canonization of false facts
Negative publication rate
Probability
of
canonizing
false
claim
as
fact
Nissen S.B., et al., “Research: Publication bias and the
canonization of false facts”, eLife 2016;5:e21451
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
𝛼 = 0.05
Power = 0.8
44. If you don’t publish negative results,
science is screwed
but...
there is a thin line between “negative
result” and “no result”
45. Data dredging, p-hacking
Searching until you find the
result you were looking for
Massaging data
Post-hoc hypothesis
Unaccounted multiple
experiments/tests
𝑝 = 0.06?
Let’s try again
Ignoring
confounding effects
Not reporting non-
significant results
46. Evidence of p-hacking
46
Head M.L., et al. “The Extent and Consequences of P-Hacking
in Science”, PLoS Biol 13(3)
Distribution of p-values reported in publications
Evidence of p-hacking
𝑛F 𝑛Q H0: 𝑛F = 𝑛Q
47. Reproducibility crisis
Open Science Collaboration, “Estimating the reproducibility
of psychological science”, Science, 349 (2015)
Managed to reproduce only 39% results
51. Are referees more likely to give red cards to black players?
51
• one data set
• 29 teams
• 61 scientists
• task: find odds ratio
Silberzahn et al., “Many analysts, one dataset: Making
transparent how variations in analytical choices affect
results”, https://osf.io/j5v8f
55. Before you do the experiment
talk to us
The Data Analysis Group
http://www.compbio.dundee.ac.uk/dag.html
56. Specify the null
hypothesis
Design the experiment
• randomization
• statistical power
Quality control
some crap comes out in statistics
We assumed the null hypothesis
Never, ever say that large 𝑝 supports H0
Ditch the 𝜶 limit
use p-values as a continuous measure of
data incompatibility with H0
𝑝 ~ 0.05 only means ‘worth a look’
Reporting a discovery based only on
𝑝 < 0.05 is wrong
Use the three-sigma rule
that is 𝑝 < 0.003, to demonstrate a
discovery
Reporting
• Always report the effect size and its confidence limits
• Show data (not dynamite plots)
• Don’t use the word ‘significant’
• Don’t use asterisks to mark ‘significant’ results in
figures
Validation
Follow-up experiments to
confirm discoveries
Publication
Publish negative results
57. ASA Statement on Statistical Significance and P-Values
1. P-values can indicate how incompatible the data are with a specified statistical
model
2. P-values do not measure the probability that the studied hypothesis is true, or
the probability that the data were produced by random chance alone
3. Scientific conclusions and business or policy decisions should not be based only
on whether a p-value passes a specific threshold
4. Proper inference requires full reporting and transparency
5. A p-value, or statistical significance, does not measure the size of an effect or
the importance of a result
6. By itself, a p-value does not provide a good measure of evidence regarding a
model or hypothesis
https://is.gd/asa_stat