Representing and generating uncertainty effectively presentatıonAzdeen Najah
Prof. Frank H Knight (1921) proposed that "risk" is randomness with knowable probabilities, and "uncertainty" is randomness with unknowable probabilities. However, risk and uncertainty both share features with randomness. The illustration here explains the relationship of the concepts better than words...
In this talk I discuss our recent Bayesian reanalysis of the Reproducibility Project: Psychology.
The slides at the end include the technical details underlying the Bayesian model averaging method we employ.
Representing and generating uncertainty effectively presentatıonAzdeen Najah
Prof. Frank H Knight (1921) proposed that "risk" is randomness with knowable probabilities, and "uncertainty" is randomness with unknowable probabilities. However, risk and uncertainty both share features with randomness. The illustration here explains the relationship of the concepts better than words...
In this talk I discuss our recent Bayesian reanalysis of the Reproducibility Project: Psychology.
The slides at the end include the technical details underlying the Bayesian model averaging method we employ.
The history of p-values is covered to try and shed light on a mystery: why did Student and Fisher agree numerically but disagree in terms of interpretation.?
The Seven Habits of Highly Effective StatisticiansStephen Senn
If you know why the title of this talk is extremely stupid, then you clearly know something about control, data and reasoning: in short, you have most of what it takes to be a statistician. If you have studied statistics then you will also know that a large amount of anything, and this includes successful careers, is luck.
In this talk I shall try share some of my experiences of being a statistician in the hope that it will help you make the most of whatever luck life throws you, In so doing, I shall try my best to overcome the distorting influence of that easiest of sciences hindsight. Without giving too much away, I shall be recommending that you read, listen, think, calculate, understand, communicate, and do. I shall give you some example of what I think works and what I think doesn’t
In all of this you should never forget the power of negativity and also the joy of being able to wake up every day and say to yourself ‘I love the small of data in the morning’.
Statistics for UX Professionals - Jessica CameronUser Vision
Are you looking to expand your research toolkit to include some quantitative methods, such as survey research or A/B testing? Have you been asked to collect some usability metrics, but aren’t sure how best to go about that? Or do you just want to be more aware of all of the UX research possibilities? If your answer to any of those questions is yes, then this session is for you.
You may know that without statistics, you won’t know if A is really better than B, if users are truly more satisfied with your new site than with your old one, or which changes to your site have actually impacted conversion rates. However, statistics can also help you figure out how to report satisfaction and other metrics you collect during usability tests. And they’re essential for making sense of the results of quantitative usability tests.
This session will focus on the statistical concepts that are most useful for UX researchers. It won’t make you a quant, but it will give you a good grounding in quantitative methods and reporting. (For example, you will learn what a margin of error is, how to report quantitative data collected during a usability test - and how not to - and how many people you really need to fill out a survey.)
Are you looking to expand your research toolkit to include some quantitative methods, such as survey research or A/B testing? Have you been asked to collect some usability metrics, but aren’t sure how best to go about that? Or do you just want to be more aware of all of the UX research possibilities? If your answer to any of those questions is yes, then this session is for you.
You may know that without statistics, you won’t know if A is really better than B, if users are truly more satisfied with your new site than with your old one, or which changes to your site have actually impacted conversion rates. However, statistics can also help you figure out how to report satisfaction and other metrics you collect during usability tests. And they’re essential for making sense of the results of quantitative usability tests.
This session will focus on the statistical concepts that are most useful for UX researchers. It won’t make you a quant, but it will give you a good grounding in quantitative methods and reporting. (For example, you will learn what a margin of error is, how to report quantitative data collected during a usability test - and how not to - and how many people you really need to fill out a survey.)
Systems Thinking offers methods for breaking down any problem into its component parts so that you can truly understand its causes and start to formulate solutions. Learn how four concepts—Distinctions, Systems, Relationships, and Perspectives—can help you tackle any professional challenge.
Presidents' invited lecture ISCB Vigo 2017
Discusses various issues to do with how randomised clinical trials should be analysed. See also https://errorstatistics.com/2017/07/01/s-senn-fishing-for-fakes-with-fisher-guest-post/
These slides were presented on November 22 2016 during the Annual Julius Symposium, organised by the Julius Center for Health Sciences and Primary Care, University Medical Hospital Utrecht.
Only a few months ago, the American Statistical Association authoritatively issued an official statement on significance and p-values (American Statistician, 2016, 70:2, 129-133), claiming that the p-value is: “commonly misused and misinterpreted.”
In this presentation I focus on the principles of the ASA statement.
The history of p-values is covered to try and shed light on a mystery: why did Student and Fisher agree numerically but disagree in terms of interpretation.?
The Seven Habits of Highly Effective StatisticiansStephen Senn
If you know why the title of this talk is extremely stupid, then you clearly know something about control, data and reasoning: in short, you have most of what it takes to be a statistician. If you have studied statistics then you will also know that a large amount of anything, and this includes successful careers, is luck.
In this talk I shall try share some of my experiences of being a statistician in the hope that it will help you make the most of whatever luck life throws you, In so doing, I shall try my best to overcome the distorting influence of that easiest of sciences hindsight. Without giving too much away, I shall be recommending that you read, listen, think, calculate, understand, communicate, and do. I shall give you some example of what I think works and what I think doesn’t
In all of this you should never forget the power of negativity and also the joy of being able to wake up every day and say to yourself ‘I love the small of data in the morning’.
Statistics for UX Professionals - Jessica CameronUser Vision
Are you looking to expand your research toolkit to include some quantitative methods, such as survey research or A/B testing? Have you been asked to collect some usability metrics, but aren’t sure how best to go about that? Or do you just want to be more aware of all of the UX research possibilities? If your answer to any of those questions is yes, then this session is for you.
You may know that without statistics, you won’t know if A is really better than B, if users are truly more satisfied with your new site than with your old one, or which changes to your site have actually impacted conversion rates. However, statistics can also help you figure out how to report satisfaction and other metrics you collect during usability tests. And they’re essential for making sense of the results of quantitative usability tests.
This session will focus on the statistical concepts that are most useful for UX researchers. It won’t make you a quant, but it will give you a good grounding in quantitative methods and reporting. (For example, you will learn what a margin of error is, how to report quantitative data collected during a usability test - and how not to - and how many people you really need to fill out a survey.)
Are you looking to expand your research toolkit to include some quantitative methods, such as survey research or A/B testing? Have you been asked to collect some usability metrics, but aren’t sure how best to go about that? Or do you just want to be more aware of all of the UX research possibilities? If your answer to any of those questions is yes, then this session is for you.
You may know that without statistics, you won’t know if A is really better than B, if users are truly more satisfied with your new site than with your old one, or which changes to your site have actually impacted conversion rates. However, statistics can also help you figure out how to report satisfaction and other metrics you collect during usability tests. And they’re essential for making sense of the results of quantitative usability tests.
This session will focus on the statistical concepts that are most useful for UX researchers. It won’t make you a quant, but it will give you a good grounding in quantitative methods and reporting. (For example, you will learn what a margin of error is, how to report quantitative data collected during a usability test - and how not to - and how many people you really need to fill out a survey.)
Systems Thinking offers methods for breaking down any problem into its component parts so that you can truly understand its causes and start to formulate solutions. Learn how four concepts—Distinctions, Systems, Relationships, and Perspectives—can help you tackle any professional challenge.
Presidents' invited lecture ISCB Vigo 2017
Discusses various issues to do with how randomised clinical trials should be analysed. See also https://errorstatistics.com/2017/07/01/s-senn-fishing-for-fakes-with-fisher-guest-post/
These slides were presented on November 22 2016 during the Annual Julius Symposium, organised by the Julius Center for Health Sciences and Primary Care, University Medical Hospital Utrecht.
Only a few months ago, the American Statistical Association authoritatively issued an official statement on significance and p-values (American Statistician, 2016, 70:2, 129-133), claiming that the p-value is: “commonly misused and misinterpreted.”
In this presentation I focus on the principles of the ASA statement.
This year marks the 70th anniversary of the Medical Research Council randomised clinical trial (RCT) of streptomycin in tuberculosis led by Bradford Hill. This is widely regarded as a landmark in clinical research. Despite its widespread use in drug regulation and in clinical research more widely and its high standing with the evidence based medicine movement, the RCT continues to attracts criticism. I show that many of these criticisms are traceable to failure to understand two key concepts in statistics: probabilistic inference and design efficiency. To these methodological misunderstandings can be added the practical one of failing to appreciate that entry into clinical trials is not simultaneous but sequential.
I conclude that although randomisation should not be used as an excuse for ignoring prognostic variables, it is valuable and that many standard criticisms of RCTs are invalid.
Topic Learning TeamNumber of Pages 2 (Double Spaced)Num.docxAASTHA76
Topic: Learning Team
Number of Pages: 2 (Double Spaced)
Number of sources: 1
Writing Style: APA
Type of document: Essay
Academic Level:Master
Category: Psychology
VIP Support: N/A
Language Style: English (U.S.)
Order Instructions:
I will attach the instruction. On this paper please follow the instructions carefully. Thank you
Correlation
PSYCH/610 Version 2
1
University of Phoenix Material
Correlation
A researcher is interested in investigating the relationship between viewing time (in seconds) and ratings of aesthetic appreciation. Participants are asked to view a painting for as long as they like. Time (in seconds) is measured. After the viewing time, the researcher asks the participants to provide a ‘preference rating’ for the painting on a scale ranging from 1-10. Create a scatter plot depicting the following data:
Viewing Time in Seconds
Preference Rating
10
3
12
4
24
7
5
3
16
6
3
4
11
4
5
2
21
8
23
9
9
5
3
3
17
5
14
6
What does the scatter plot suggest about the relationship between viewing time and aesthetic preference? Is it accurate to state that longer viewing times are the result of greater preference for paintings? Explain. Submit your scatter plot and your answers to the questions to your instructor.
LEARNING OBJECTIVES
· Explain how researchers use inferential statistics to evaluate sample data.
· Distinguish between the null hypothesis and the research hypothesis.
· Discuss probability in statistical inference, including the meaning of statistical significance.
· Describe the t test and explain the difference between one-tailed and two-tailed tests.
· Describe the F test, including systematic variance and error variance.
· Describe what a confidence interval tells you about your data.
· Distinguish between Type I and Type II errors.
· Discuss the factors that influence the probability of a Type II error.
· Discuss the reasons a researcher may obtain nonsignificant results.
· Define power of a statistical test.
· Describe the criteria for selecting an appropriate statistical test.
Page 267IN THE PREVIOUS CHAPTER, WE EXAMINED WAYS OF DESCRIBING THE RESULTS OF A STUDY USING DESCRIPTIVE STATISTICS AND A VARIETY OF GRAPHING TECHNIQUES.In addition to descriptive statistics, researchers use inferential statistics to draw more general conclusions about their data. In short, inferential statistics allow researchers to (a) assess just how confident they are that their results reflect what is true in the larger population and (b) assess the likelihood that their findings would still occur if their study was repeated over and over. In this chapter, we examine methods for doing so.
SAMPLES AND POPULATIONS
Inferential statistics are necessary because the results of a given study are based only on data obtained from a single sample of research participants. Researchers rarely, if ever, study entire populations; their findings are based on sample data. In addition to describing the sample data, we want to make statements ab.
Hypothesis TestingIn doing research, one of the most common actiNarcisaBrandenburg70
Hypothesis Testing
In doing research, one of the most common activities is testing hypotheses. The Afrobarometer data set below is a survey of African citizens’ attitudes on democracy, governance, the economy, and other related topics (www.afrobarometer.org). Using this data set, you might want to examine hypotheses related to whether rural and urban citizens differ, on average, in how much they trust the government. The tables below present results from an independent samples t-test to examine these hypotheses using a random sample of 44 participants from the complete data set. Each respondent’s score is a value between 0 and 15 with a higher score indicating greater trust. You can see that the mean for the urban group is 7.00 ( SD = 4.17) and the mean for the rural group is 7.74 ( SD = 4.38). The observed value of the t-statistic is -.564 and the p-value equals 0.576 (see the column labeled “Sig. (2-tailed)”).
African Citizens' Attitudes on Democracy
The tables below present results from an independent samples t-test to examine these hypotheses using a random sample of 44 participants from the complete data set. Each respondent’s score is a value between 0 and 15 with a higher score indicating greater trust. You can see that the mean for the urban group is 7.00 ( SD = 4.17) and the mean for the rural group is 7.74 ( SD = 4.38). The observed value of the t-statistic is -.564 and the p-value equals 0.576 (see the column labeled “Sig. (2-tailed)”).
t
df
Sig.
(2-tailed)
Mean Difference
Std. Error Difference
Trust in Government Index
(higher scores = more trust)
-.564
41
.576
-.73913
1.30978
Group Statistics
Urban or Rural Primary
Sampling Unit
N
Mean
Std. Deviation
Std. Error Mean
Trust in Government Index
(higher scores = more trust)
Urban
20
7.000
4.16754
.93189
Rural
30
7.7391
4.38196
.91370
The p-value is the probability of obtaining a value more extreme than .564 (less than -.564 or greater than +.564) if you were to repeat the test with a new sample of data and if the null hypothesis is true. You will see in this Skill Builder that the p-valuecan easily be used to make statistical decisions in hypothesis testing. However, while the p-valueis important in determining statistical significance, it does not tell the whole story.
Steps of Hypothesis Testing
To interpret p-values, let's review the key steps in hypothesis testing. Use the < and > icons to navigate between the steps.
Step 1
State the null and alternative hypotheses
Recall that hypotheses are statements about population parameters. For the Trust in Government example from the Afrobarometer data set, the null (HO) and alternative hypotheses (HA) is seen in the above image.
The Greek letter, µ, indicates a population mean, and the subscripts indicate levels of the independent variable (“urban” and “rural”). Here the null is saying that the mean for the urban population on the Trust In Government variable is the same as the mean for the rural population. The alternative hypothe ...
Page 266LEARNING OBJECTIVES· Explain how researchers use inf.docxkarlhennesey
Page 266
LEARNING OBJECTIVES
· Explain how researchers use inferential statistics to evaluate sample data.
· Distinguish between the null hypothesis and the research hypothesis.
· Discuss probability in statistical inference, including the meaning of statistical significance.
· Describe the t test and explain the difference between one-tailed and two-tailed tests.
· Describe the F test, including systematic variance and error variance.
· Describe what a confidence interval tells you about your data.
· Distinguish between Type I and Type II errors.
· Discuss the factors that influence the probability of a Type II error.
· Discuss the reasons a researcher may obtain nonsignificant results.
· Define power of a statistical test.
· Describe the criteria for selecting an appropriate statistical test.
Page 267IN THE PREVIOUS CHAPTER, WE EXAMINED WAYS OF DESCRIBING THE RESULTS OF A STUDY USING DESCRIPTIVE STATISTICS AND A VARIETY OF GRAPHING TECHNIQUES. In addition to descriptive statistics, researchers use inferential statistics to draw more general conclusions about their data. In short, inferential statistics allow researchers to (a) assess just how confident they are that their results reflect what is true in the larger population and (b) assess the likelihood that their findings would still occur if their study was repeated over and over. In this chapter, we examine methods for doing so.
SAMPLES AND POPULATIONS
Inferential statistics are necessary because the results of a given study are based only on data obtained from a single sample of research participants. Researchers rarely, if ever, study entire populations; their findings are based on sample data. In addition to describing the sample data, we want to make statements about populations. Would the results hold up if the experiment were conducted repeatedly, each time with a new sample?
In the hypothetical experiment described in Chapter 12 (see Table 12.1), mean aggression scores were obtained in model and no-model conditions. These means are different: Children who observe an aggressive model subsequently behave more aggressively than children who do not see the model. Inferential statistics are used to determine whether the results match what would happen if we were to conduct the experiment again and again with multiple samples. In essence, we are asking whether we can infer that the difference in the sample means shown in Table 12.1 reflects a true difference in the population means.
Recall our discussion of this issue in Chapter 7 on the topic of survey data. A sample of people in your state might tell you that 57% prefer the Democratic candidate for an office and that 43% favor the Republican candidate. The report then says that these results are accurate to within 3 percentage points, with a 95% confidence level. This means that the researchers are very (95%) confident that, if they were able to study the entire population rather than a sample, the actual percentage who preferred th ...
A lecture in HEVGA research summer school at the University of Skövde, Sweden (Aug 21-23 2019) on Game Analysis. The focus of the lecture is in the formal analysis and some applications of it.
My level design intro course lecture: assignment description
Video at slide 16: https://youtu.be/w_x5wI3PNZA
Video at slide 25: https://youtu.be/OPIwVcOe3k0
Over view to a book about research methods edited by Petri Lankoski and Staffan Björk (2015). http://press.etc.cmu.edu/content/game-research-methods-overview
Slides introduces Escape package that is developed for teaching Unity. The package contains building blocks for a simple first person sneaking game.
Contents:
- Introduction to prefabs in package
- Level design assignment
Download the Unity package:
http://www.mediafire.com/download/2t49ajxl6n7xq3z/escape_new.unitypackage
Slides revised Mar 23, 2015.
The course intro for level design course with an introduction to some surrealist methods and development project aiming to use those those techniques. This is part of an experiment in design teaching to extend student design understanding outside of tradition methods.
The slides contains the course intro, instructions to a development assignment and description of prefabs that are offered for the project (the Unity Project will be available later after I I fixed all the details and removed assets that I cannot redistribute).
A lecture on game system design. Introduction to concepts for describing and discussing designs with examples. Some notes about evaluating game system behavior.
- Course description and assignments (slides 3-10)
- Game system design
-- Game elements (slides 12-29)
-- Hints for design (slides 30-32)
- References + reading list (slides 33-34)
Techniques to optimize the pagerank algorithm usually fall in two categories. One is to try reducing the work per iteration, and the other is to try reducing the number of iterations. These goals are often at odds with one another. Skipping computation on vertices which have already converged has the potential to save iteration time. Skipping in-identical vertices, with the same in-links, helps reduce duplicate computations and thus could help reduce iteration time. Road networks often have chains which can be short-circuited before pagerank computation to improve performance. Final ranks of chain nodes can be easily calculated. This could reduce both the iteration time, and the number of iterations. If a graph has no dangling nodes, pagerank of each strongly connected component can be computed in topological order. This could help reduce the iteration time, no. of iterations, and also enable multi-iteration concurrency in pagerank computation. The combination of all of the above methods is the STICD algorithm. [sticd] For dynamic graphs, unchanged components whose ranks are unaffected can be skipped altogether.
Analysis insight about a Flyball dog competition team's performanceroli9797
Insight of my analysis about a Flyball dog competition team's last year performance. Find more: https://github.com/rolandnagy-ds/flyball_race_analysis/tree/main
Chatty Kathy - UNC Bootcamp Final Project Presentation - Final Version - 5.23...John Andrews
SlideShare Description for "Chatty Kathy - UNC Bootcamp Final Project Presentation"
Title: Chatty Kathy: Enhancing Physical Activity Among Older Adults
Description:
Discover how Chatty Kathy, an innovative project developed at the UNC Bootcamp, aims to tackle the challenge of low physical activity among older adults. Our AI-driven solution uses peer interaction to boost and sustain exercise levels, significantly improving health outcomes. This presentation covers our problem statement, the rationale behind Chatty Kathy, synthetic data and persona creation, model performance metrics, a visual demonstration of the project, and potential future developments. Join us for an insightful Q&A session to explore the potential of this groundbreaking project.
Project Team: Jay Requarth, Jana Avery, John Andrews, Dr. Dick Davis II, Nee Buntoum, Nam Yeongjin & Mat Nicholas
Unleashing the Power of Data_ Choosing a Trusted Analytics Platform.pdfEnterprise Wired
In this guide, we'll explore the key considerations and features to look for when choosing a Trusted analytics platform that meets your organization's needs and delivers actionable intelligence you can trust.
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Round table discussion of vector databases, unstructured data, ai, big data, real-time, robots and Milvus.
A lively discussion with NJ Gen AI Meetup Lead, Prasad and Procure.FYI's Co-Found
Learn SQL from basic queries to Advance queriesmanishkhaire30
Dive into the world of data analysis with our comprehensive guide on mastering SQL! This presentation offers a practical approach to learning SQL, focusing on real-world applications and hands-on practice. Whether you're a beginner or looking to sharpen your skills, this guide provides the tools you need to extract, analyze, and interpret data effectively.
Key Highlights:
Foundations of SQL: Understand the basics of SQL, including data retrieval, filtering, and aggregation.
Advanced Queries: Learn to craft complex queries to uncover deep insights from your data.
Data Trends and Patterns: Discover how to identify and interpret trends and patterns in your datasets.
Practical Examples: Follow step-by-step examples to apply SQL techniques in real-world scenarios.
Actionable Insights: Gain the skills to derive actionable insights that drive informed decision-making.
Join us on this journey to enhance your data analysis capabilities and unlock the full potential of SQL. Perfect for data enthusiasts, analysts, and anyone eager to harness the power of data!
#DataAnalysis #SQL #LearningSQL #DataInsights #DataScience #Analytics
Enhanced Enterprise Intelligence with your personal AI Data Copilot.pdfGetInData
Recently we have observed the rise of open-source Large Language Models (LLMs) that are community-driven or developed by the AI market leaders, such as Meta (Llama3), Databricks (DBRX) and Snowflake (Arctic). On the other hand, there is a growth in interest in specialized, carefully fine-tuned yet relatively small models that can efficiently assist programmers in day-to-day tasks. Finally, Retrieval-Augmented Generation (RAG) architectures have gained a lot of traction as the preferred approach for LLMs context and prompt augmentation for building conversational SQL data copilots, code copilots and chatbots.
In this presentation, we will show how we built upon these three concepts a robust Data Copilot that can help to democratize access to company data assets and boost performance of everyone working with data platforms.
Why do we need yet another (open-source ) Copilot?
How can we build one?
Architecture and evaluation
06-04-2024 - NYC Tech Week - Discussion on Vector Databases, Unstructured Data and AI
Discussion on Vector Databases, Unstructured Data and AI
https://www.meetup.com/unstructured-data-meetup-new-york/
This meetup is for people working in unstructured data. Speakers will come present about related topics such as vector databases, LLMs, and managing data at scale. The intended audience of this group includes roles like machine learning engineers, data scientists, data engineers, software engineers, and PMs.This meetup was formerly Milvus Meetup, and is sponsored by Zilliz maintainers of Milvus.
Adjusting OpenMP PageRank : SHORT REPORT / NOTESSubhajit Sahu
For massive graphs that fit in RAM, but not in GPU memory, it is possible to take
advantage of a shared memory system with multiple CPUs, each with multiple cores, to
accelerate pagerank computation. If the NUMA architecture of the system is properly taken
into account with good vertex partitioning, the speedup can be significant. To take steps in
this direction, experiments are conducted to implement pagerank in OpenMP using two
different approaches, uniform and hybrid. The uniform approach runs all primitives required
for pagerank in OpenMP mode (with multiple threads). On the other hand, the hybrid
approach runs certain primitives in sequential mode (i.e., sumAt, multiply).
2. You should be familiar with following
• Mean (medelvärde), for a normal distribution
• Median (median)
• Mode (typvärde)
• Line chart (linjediagram)
• Bar chart (stapeldiagram)
Petri Lankoski, 2018 2
3. Is the Die Loaded?
11st throw
12st throw
43st throw
14st throw
25st throw
We cannot say for certain, but we can estimate how
likely or unlikely the perceived sequence is
In long run we expect to see equal amount of 1s, 2s,
3s, 4s, 5s and 6s
16st throw
Chance to get 1 is 1/6, but as first throw, this is as
likely as any other result. We do not have enough
information to say anything more about this
six throws is probably still too little to estimate the
die, so we would need to roll more…
Petri Lankoski, 2018 3
4. Is the Die Loaded?
1
1
4
1
2
1
3
6
1
1
1
5
Testing this sequence against expected sequence
indicate that the die is loaded
• But we have around 1% change to be wrong
We roll following sequence: 2 6 2 6 6 4 6 5 4 1 3 4
4 6 5 3 5 3 2 5
• Amounts of 6s and 1s does not match to
expected amounts
• We would have 70% likelihood of being wrong
if we claim that the die is load
Petri Lankoski, 2018 4
6. density and violin plot
Violin plot is a form
of density plot
Petri Lankoski, 2018 6
Density plot and data points
7. Scatter plot
-2 -1 0 1 2
-3-2-10123
Variable 1
Variable2
Scatter plot shows values of two variables
• For example how a participant answered
to questions
Petri Lankoski, 2018 7
8. Random sampling Predicting election results
- It is not practically possible to ask all what they will vote
- Picking a sample of people randomly & asking them
However, we know that there is uncertainty here
If random sample again, we might get something else
We get:
A: 37.6%
B: 12.3%
C: 33.1%
D: 5.2%
…
We get:
A: 36.9%
B: 13.0%
C: 32.7%
D: 6.1%
…
We can estimate uncertainty, but we need to make some
assumptions
Petri Lankoski, 2018
8
We get:
A: 38.7%
B: 11.0%
C: 31.7%
D: 6.3%
…
9. Normal distribution
1𝜎 2𝜎-2𝜎 -1𝜎 0𝜎
68.3%
95.4% of data
9
𝜎 = standard deviation
• describes the width of distribution
10. Back to polling
1.96𝜎-1.96𝜎 0𝜎
95% of population is in the
area of ∓1.96𝜎; sample
distribution behaves similarly
However, within 95% certainty
what we observed falls in area
between -1.96𝜎 and 1.96𝜎.
We cannot know where in
population distribution what
we observed was (red vertical
lines).
10
We do not know true
population value (black
vertical line).
Support for A
36.1%
38.7%
37.6%
11. Random sampling
Instead of uncertainty, confidence is usually used.
Confidence interval (CI), usually 95%, is function of sample
size and probability of someone choosing a candidate.
0.376 ∓ 1.96 ∗ √
0.376(1 − 0.376)
𝑁
𝜎95%A
Petri Lankoski, 2018 11
We can backtrack from the sample distribution and estimate
the uncertainty in what we observed when polling
• When we poll next time within 95% certainty what we
observed falls in area between -1.96𝜎 and 1.96𝜎
12. Are two means different, t-test?
A B∆
We have two sample means A and B
Their difference is ∆=B-A
Mean is calculated based on sampled values
Mean(A) =
∑𝑎
𝑛
(for normally distruted variables)
To extrapolate if the there is difference between
groups A and B in population level (from witch A and
B were sampled) we need to account uncertainty.
Again population mean and sample mean can be
different.
Petri Lankoski, 2018 12
13. Are two means different, t-test?
A B∆
We have two sample means A and B
Their difference is ∆=B-A
t statistic describes difference so that it takes into
account variance (𝜎2) and sample size
p describes probability that perceived data deviates
from null hypothesis; in case null hypothesis of t-test, is
the means are not different.
p depends on t-value and sample size; high t-value
means lower p.
p = 0.05 means that there is 5% change that observed
data did not deviate from expected, there is no
difference. P<0.05 is a typical statistically significant
result criterion.
Petri Lankoski, 2018 13
14. Are tree means different, one-way ANOVA
• One-way ANOVA is similar to t-test
• F-statistic describes difference so that it takes into account variance
and sample size
• p describes probability that perceived data deviates from null
hypothesis; in case null hypothesis of ANOVA, is the means are not
different
• A significant result (p<0.05) tells that at least one mean differ from
others
• But not which
• Post hoc comparisons are needed to determine which variable differs from
which
Petri Lankoski, 2018 14
15. Correlation
Correlation (r) describes the strength of
association between two variables
p describes the likelihood that the observed
correlation deviates from what is expected
under null hypothesis (which is that there is no
relation between the two variables)
Correlation does not tell if v1 causes v2 or vice
versa
• There is a strong correlation between ice
cream sales and drowning
• Either is causing another
• Third variable, temperature, related to both
Petri Lankoski, 2018 15
Editor's Notes
https://stats.stackexchange.com/questions/3194/how-can-i-test-the-fairness-of-a-d20/3735#3735
chisq.test(table(c(1,1,4,1,2,1,3,6,1,1,1,5)), p = rep(1/6,6))
Chi-squared test for given probabilities
data: table(c(1, 1, 4, 1, 2, 1, 3, 6, 1, 1, 1, 5))
X-squared = 15, df = 5, p-value = 0.01036
Note that we cannot test if the die is not biased. We can only test if behaves enough unexpectly
rolls = sample(1:6, 20, replace=TRUE) # 20 times d6
chisq.test(table(rolls), p = rep(1/6,6))
Polling is done via random sampling using telephone catalog. However, people owning a phone and people voting are not the same populations and the poll results are systematically off; however, there are techniques counter the sampling bias, especially in the case of voting when it is possible to compare results to poll results.
𝜎=standard deviation, describes the width of distribution
Black vertical line: population value
Red vertical line: sample values
𝜎=standard deviation, describes the width of distribution
Black vertical line: population value
Red vertical line: sample values
The standard deviation is the square root of the variance.