Chapter 5
Producing Data
5.1 DESIGNING SAMPLES
Some notes before we begin
• We are entering the second part of the
statistics course “Experimental Design”
• In most real...
Population and Sample
Population-
• The entire group of individuals for which
information is produced
Sample-
• A subset o...
Sample vs. Census
Census
• Information gathered from the entire
population (no exceptions!)
• Produces the most accurate d...
Samples
• By their nature, the success or failure of a
study or experiment depends on good
technique in sampling
• We want...
Some ‘bad’ sampling techniques
Voluntary Response Sampling
• Most often seen as a ‘call-in’ poll or an
‘internet poll’
• P...
Some ‘bad’ sampling techniques
Convenience Sampling
• Individuals in the sample consist of those who are
easiest to reach
...
Bias
• In statistics, bias refers to the systematic
favoring of one outcome over another
• Try not to confuse this definit...
Some notation
• The lowercase script ‘n’ always denotes the
number of individuals in a sample
• The capital ‘N’ denotes th...
Simple Random Samples
• This is THE sampling technique for this
statistics course
– Other sampling techniques exist, but o...
Choosing an SRS
1. Label Individuals
Assign each individual in the population a
unique “ID”
Each ID should have the same #...
Probability Samples
• Samples are chosen by chance
• All possible samples are known
• The probability of choosing each sam...
Stratified Random Sample
• Population is divided into strata
– These strata are segments of the population that are
simila...
Stratified Random Sample
Suppose the population contains 100
juniors and 50 seniors
• We would like our samples to reflect...
Cluster Sampling
1. The population is divided into clusters or
groups
Each cluster must be representative of
the populatio...
Multistage sampling
• Used when the population is very large
• Take samples from the samples
repeatedly until the sample s...
Cautions about Sample Surveys
Undercoverage
• Sample does not include all segments of the
population, or systematically fa...
Cautions about Sample Surveys
Nonresponse
• Many people contacted for a survey choose not to
participate
• Extremely signi...
Cautions about Sample Surveys
Response Bias
• Respondents answer in a way that is
different from the actual opinion
• Can ...
Cautions about Sample Surveys
Wording of Questions
• Questions that are “confusing”
– Complicated wording affects response...
Sample Survey Wisdom
• Insist of knowing the following before
trusting results:
1. The exact questions asked
2. Rate of no...
Assignment 5.1A
#2, 6, 7, 9, 11, 24, 26, 32
5.2 DESIGNING EXPERIMENTS
Definitions
An experiment is conducted to reveal the
response of one variable (response
variable) to changes in other vari...
Definitions
Experimental Units
• The individuals upon whom the
experiment is conducted
• Human experimental units are call...
Definitions
Factors
• Another term for explanatory variables in
an experiment
• An experiment can examine the effects
of m...
Principles of Design
• Control
– Minimize effect confounding variables
– Obtain and apply treatments to exp. units
• Repli...
Control
• Try to detect and separate effects from the
treatment from effects from other variables
• Control Group
– Repres...
Replication
• We would like exp. units within each
treatment group to respond similarly to the
treatment, and differently ...
Replication
• Use enough experimental units to
eliminate “chance variation”
• Replication (in terms of experimental
design...
Randomization
• Assign experimental units to treatments
using a randomized design (SRS)
• Minimize bias due to individual’...
Statistical Significance
• After experimentation, we hope to see a
difference in response level that is
large/measurable
•...
Assignment 5.2A
• Pg 357 #33, 35, 37, 39, 40, 43, 45, 46, 67
Randomized Comparative
Experiments
• Completely Randomized Design
– Most basic
• Block Design
– Used when we believe there...
Completely Randomized Design
• Can be used for many treatments
• Exp units assigned to treatment group
randomly
• Response...
Completely Randomized Design
(Example Diagram)
Block Design
• This is an instance of control
• Exp Units are known to have similar
response level groups (i.e. gender
dif...
Block Design
• Each treatment group is averaged an
compared within the block
• Each block may (or may not) have a
control ...
Block Design
(Example Diagram)
Matched Pairs Design
• Exp units are matched into pairs that are
similar in terms of the experiment
• Each of two experime...
Matched Pairs Design
• Randomization-
– Randomized which member of the pair
receives which treatment
– Randomize the order...
Matched Pair Design
(example diagram – single subject)
Subject #1
treatment
control
compare
Subject #2
control
treatment
c...
Matched Pair Design
(example diagram – paired subjects)
Subject #1 treatment
control compareSubject #2
Subject #3 treatmen...
Cautions about Experimentation
Double Blind Experiment
• Sometimes bias is produced unconsciously
• Sometimes a subject wi...
Cautions about Experimentation
Lack of realism
• Experimental results are produced under
conditions that cannot be realist...
Assignment 5.2B
• #45-49, 55, 57, 62, 63, 67, 68
Upcoming SlideShare
Loading in...5
×

Stats chapter 5

5,229

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
5,229
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
45
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Stats chapter 5"

  1. 1. Chapter 5 Producing Data
  2. 2. 5.1 DESIGNING SAMPLES
  3. 3. Some notes before we begin • We are entering the second part of the statistics course “Experimental Design” • In most real life applications, experimental design begins the process of statistics • Provided experiments (and surveys) are carefully designed, we can use the techniques of statistics to analyze the results with increased “significance” • Much of this material is covered in social science courses (i.e. psychology)
  4. 4. Population and Sample Population- • The entire group of individuals for which information is produced Sample- • A subset of the population that is examined in greater detail • Results of the sample are generalized to the population.
  5. 5. Sample vs. Census Census • Information gathered from the entire population (no exceptions!) • Produces the most accurate description of the population • Usually expensive or impossible
  6. 6. Samples • By their nature, the success or failure of a study or experiment depends on good technique in sampling • We want our sample to “look like” our population – We would like to minimize the effect of outlier observations – We would like to decrease ‘variability’ in our sample – We would like to decrease ‘bias’
  7. 7. Some ‘bad’ sampling techniques Voluntary Response Sampling • Most often seen as a ‘call-in’ poll or an ‘internet poll’ • People with strong, often negative opinions are most likely to respond • Polls are easily “fixed” • This sampling technique and its’ results are not to be trusted!
  8. 8. Some ‘bad’ sampling techniques Convenience Sampling • Individuals in the sample consist of those who are easiest to reach • Mall interviews – The sample is only valid for people who visit the mall (this is not everyone!) – The sample tends to consist of the “easiest targets” • Some telephone studies • This is not to say that samples must be difficult to construct, they just cannot consist of only the easiest individuals to sample
  9. 9. Bias • In statistics, bias refers to the systematic favoring of one outcome over another • Try not to confuse this definition with a non-statistical definition • Bias is enemy #1 for sampling technique
  10. 10. Some notation • The lowercase script ‘n’ always denotes the number of individuals in a sample • The capital ‘N’ denotes the size of the population • ‘Table B’ (inside back cover) is the table of random digits • A random integer can be produced from a TI with the command “RandInt(a, b, n)” – a = smallest number, b = largest number, n = number of digits to produce (optional)
  11. 11. Simple Random Samples • This is THE sampling technique for this statistics course – Other sampling techniques exist, but our course is focused on the results of an SRS • Every possible sample of size n has an equal chance of being selected • This is analogous to placing “names in a hat” or “drawing straws”
  12. 12. Choosing an SRS 1. Label Individuals Assign each individual in the population a unique “ID” Each ID should have the same # of digits 2. Select Individuals Use table B or your calculator to select individuals 3. Stopping rule Indicate when you will stop sampling 4. Identify Sample Indicate which individuals/ID#’s are included your sample
  13. 13. Probability Samples • Samples are chosen by chance • All possible samples are known • The probability of choosing each sample is known • SRS is one example of a probability sample
  14. 14. Stratified Random Sample • Population is divided into strata – These strata are segments of the population that are similar in an important way • Each stratum undergoes an SRS • The samples from each stratum are combined to form the full sample • A stratified sample ensures that all groups are represented at the appropriate proportion – Would a sample that consists of 50% boys and 50% girls make sense for a population of IT consultants?
  15. 15. Stratified Random Sample Suppose the population contains 100 juniors and 50 seniors • We would like our samples to reflect this proportion between juniors and seniors 1. Choose an SRS n=10 from the juniors 2. Choose and SRS n=5 from the seniors 3. The 15 individuals chosen will be the sample for our Stratified Random Sample
  16. 16. Cluster Sampling 1. The population is divided into clusters or groups Each cluster must be representative of the population (no bias!) 2. One cluster is randomly chosen Random ID selection (table B, names in a hat, calculator) 3. The entire cluster that is chosen becomes the sample
  17. 17. Multistage sampling • Used when the population is very large • Take samples from the samples repeatedly until the sample size is “manageable” • Refer to pg 341
  18. 18. Cautions about Sample Surveys Undercoverage • Sample does not include all segments of the population, or systematically favors one segment of the population • Many telephone samples will contain an undercoverage bias simply because many people do not have telephones – (yes, it’s true) • This is most serious when the “undercovered” individuals differ significantly from the rest of the population.
  19. 19. Cautions about Sample Surveys Nonresponse • Many people contacted for a survey choose not to participate • Extremely significant if the non-responders differ from the responders • Simply “sampling more people” will not eliminate bias, esp. if the bias is systematically linked to the nonresponse – We are likely to get more nonresponse! • We should either: (1) redesign the survey, or (2) follow up on the nonresponders
  20. 20. Cautions about Sample Surveys Response Bias • Respondents answer in a way that is different from the actual opinion • Can be caused by the interviewer – Appearance and gender sensitive questions can be influenced by the appearance and gender of interviewer
  21. 21. Cautions about Sample Surveys Wording of Questions • Questions that are “confusing” – Complicated wording affects responses • Questions that are “leading” – Present a scenario that can influence a response before prompting for a response – Use words that color the respondent's opinions
  22. 22. Sample Survey Wisdom • Insist of knowing the following before trusting results: 1. The exact questions asked 2. Rate of nonresponse 3. Date and method of survey • Larger samples produce more accurate results than smaller samples
  23. 23. Assignment 5.1A #2, 6, 7, 9, 11, 24, 26, 32
  24. 24. 5.2 DESIGNING EXPERIMENTS
  25. 25. Definitions An experiment is conducted to reveal the response of one variable (response variable) to changes in other variables (explanatory variable/s)
  26. 26. Definitions Experimental Units • The individuals upon whom the experiment is conducted • Human experimental units are called “subjects” Treatment • The specific experimental condition applied to the experimental units
  27. 27. Definitions Factors • Another term for explanatory variables in an experiment • An experiment can examine the effects of multiple factors Levels • Factors can be applied to experimental units in different amounts or levels
  28. 28. Principles of Design • Control – Minimize effect confounding variables – Obtain and apply treatments to exp. units • Replication – Minimize effects of outlier observations – Use multiple exp units • Randomization – Minimize effects of variability from individual responses
  29. 29. Control • Try to detect and separate effects from the treatment from effects from other variables • Control Group – Represents the population with no treatment – Often applied a placebo treatment – Provides a “baseline” for comparison • Don’t confuse “Control” (the principle) with “Control Group” (the treatment group)
  30. 30. Replication • We would like exp. units within each treatment group to respond similarly to the treatment, and differently from exp. units in other treatment groups • BUT variability (and outliers) exists throughout each treatment group • If the experiment is replicated many times (many exp. units), the effects of variability (and outliers) will “average out”
  31. 31. Replication • Use enough experimental units to eliminate “chance variation” • Replication (in terms of experimental design) does not mean “repeat the entire experiment” • Remember: larger samples produce more accurate results than smaller samples
  32. 32. Randomization • Assign experimental units to treatments using a randomized design (SRS) • Minimize bias due to individual’s response level to different treatments
  33. 33. Statistical Significance • After experimentation, we hope to see a difference in response level that is large/measurable • A difference that is too large to have happened “by chance” is called statistically significant • We try to produce statistically significant results! • We will discuss how large the difference must be in future chapters.
  34. 34. Assignment 5.2A • Pg 357 #33, 35, 37, 39, 40, 43, 45, 46, 67
  35. 35. Randomized Comparative Experiments • Completely Randomized Design – Most basic • Block Design – Used when we believe there is a difference in response levels of different groups • Matched Pairs Design – Compares only two treatments – Measures effect of treatment on two very similar exp units
  36. 36. Completely Randomized Design • Can be used for many treatments • Exp units assigned to treatment group randomly • Response in each treatment group is averaged • Average of each treatment group is compared
  37. 37. Completely Randomized Design (Example Diagram)
  38. 38. Block Design • This is an instance of control • Exp Units are known to have similar response level groups (i.e. gender differences) • Exp units are “blocked” according to these groups • Each block undergoes an SRS into treatment groups
  39. 39. Block Design • Each treatment group is averaged an compared within the block • Each block may (or may not) have a control group • Form blocks based on the most important unavoidable sources of variability among exp units • “Control what you can, block what you can’t control, randomize the rest”
  40. 40. Block Design (Example Diagram)
  41. 41. Matched Pairs Design • Exp units are matched into pairs that are similar in terms of the experiment • Each of two experimental units will receive a different treatment • Many times, the subjects in the pair are the same person • The effect of the response from the matched pair is measured with a simple subtraction
  42. 42. Matched Pairs Design • Randomization- – Randomized which member of the pair receives which treatment – Randomize the order the treatments are applied – Often randomization can be done with a coin flip! – Sometimes, it is important to have a length of time between treatment applications
  43. 43. Matched Pair Design (example diagram – single subject) Subject #1 treatment control compare Subject #2 control treatment compare Subject #3 treatment control compare Subject #n control treatment compare Randomize order compare
  44. 44. Matched Pair Design (example diagram – paired subjects) Subject #1 treatment control compareSubject #2 Subject #3 treatment control compare Subject #n treatment control compare Randomize treatment Subject #4 Subject #n-1 Match Pairs
  45. 45. Cautions about Experimentation Double Blind Experiment • Sometimes bias is produced unconsciously • Sometimes a subject will produce bias if he knows he as receiving placebo treatment • Effects can be controlled if neither the experimenter nor the subject know which treatment was administered • Typically, the treatment is given an ID number and only the researcher will know which treatment corresponds to which ID. • Controls the placebo effect
  46. 46. Cautions about Experimentation Lack of realism • Experimental results are produced under conditions that cannot be realistically duplicated • Subjects who know they are exp units may behave differently than the population • The laboratory setting itself may be a variable of the experiment!
  47. 47. Assignment 5.2B • #45-49, 55, 57, 62, 63, 67, 68
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×