Your SlideShare is downloading. ×
Sampling
Sampling
Sampling
Sampling
Sampling
Sampling
Sampling
Sampling
Sampling
Sampling
Sampling
Sampling
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Sampling

2,319

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,319
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
117
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. G3658-3Program Development and Evaluation Sampling Ellen Taylor-Powell Program Development and Evaluation Specialist May 1998
  • 2. Adapted from Sampling in ExtensionEllen Taylor-PowellTexas Agricultural Extension ServiceThe Texas A&M University SystemCollege Station, Texas, 1988
  • 3. s s s TABLE OF CONTENTS The sample: who and what . . . . . . . . . . . . . . . 2 Sampling for generalizability: probability sampling . . . . . . . . . . . . . . . . . . . . . 3 Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Simple random sample . . . . . . . . . . . . . . . . . . . 3 Systematic sample . . . . . . . . . . . . . . . . . . . . . . 4 Stratified sample . . . . . . . . . . . . . . . . . . . . . . . 4 Sample size . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Sampling for other purposes: nonprobability sampling . . . . . . . . . . . . . . . . . . 6 Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Quota sample . . . . . . . . . . . . . . . . . . . . . . . . . 6 Purposeful sampling . . . . . . . . . . . . . . . . . . . . . 6 Sample size . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Appendix 1. Table of random numbers. . . . . . . . . . . . . 9 Appendix 2. Recommended sample sizes for two different precision levels. . . . . . . . . . . . . . . . 10
  • 4. 2 s s s P R O G R A M D E V E L O P M E N T A N D E V A L U A T I O N A sample is a portion or a subgroup of a larger group called a population. The two standard The sample: ways to draw a sample are by probability and who and what? nonprobability sampling. If the evaluationÕs In Extension, we are usually interested in purpose is to generalize to the whole group, or peopleÑproducers, families, youth. We can to provide a statistical basis for saying that the take samples of almost anything, however, sample is representative, a probability sample such as acres, result demonstrations, livestock, is appropriate. If the aim of the evaluation is to lakes or 4-H records. learn about individuals or cases for some To identify who or what to include in a sample, purpose other than generalizing to the popula- start by clearly defining the population of tion, or if random selection is not feasible, then interest. The term population does not necessar- nonprobability sampling is appropriate. ily refer to all the people in the town, county or Sampling is not always necessary. When a pop- state. Rather, it refers to the group or units of ulation is small, you may choose to survey all interest (married couples, livestock owners, its members. Decisions about sampling watersheds, participants in an extension depend upon population size, what you want program, or recipients of a newsletter series) to know, and the resources available. who are found in the geographic area of inter- est (town, county, region) during the time of interest (since 1995, during 1990-1995, and so on). Clearly defining the population is the essential first step in selecting a sample. This process includes three parts: s identifying the group of interest; s naming the geographic area where the group is found; and s indicating the time period of interest, as necessary. Identifying the population from which to select the sample depends upon what you want to know. If the purpose is to show the impact of a financial management program on family financial well-being, all participants in the program would be the population of interest. If the purpose is to assess interest among the countyÕs single parents in a prospective finan- cial management program, the population is all single parents in the county.
  • 5. S A M P L I N G s s s 3 Sampling for Strategies generalizability: Simple random sample In a simple random sample, each unit of theprobability sampling population has an equal chance of beingIf you want to generalize from the sample to selected. This requires a complete list of the totalthe population, random sampling is necessary. populationÑall participants in the program,A probability sample provides for random all homeowners, all county residents, and soselection. This means that each unit has the on. One of the following methods may besame or a known chance of being selected so selected for drawing a random sample:that it is possible to confidently make estimates 1. When the population is small, numbers orabout the total population based on the sample names can simply be drawn from a hat.results. Random sampling increases the likeli- Record the number or name that is drawn.hood that the information collected is repre- Put the slip back into the container andsentative of the entire group. continue drawing until the requiredThe use of probability samples requires that sample size is obtained. If the sameeach individual in the population be identifi- number is drawn again, disregard it, put itable on a list or at a location. This so-called back in the container and continue.sampling frame must meet several criteria. It 2. For larger populations and to ensure equalneeds to be: selection opportunities, a random number 1. accurateÑincluding only those individuals table may be used such as those found in of interest; most statistics textbooks (see example, Appendix 1). After assigning consecutive 2. complete and currentÑincluding all indi- numbers to the names on the population viduals of interest; list, a number is randomly selected on the 3. free of duplicate names; and table (close eyes and point). Proceed either 4. absent any pattern in the way the names vertically or horizontally. For example, if are listed. the total population is a three-digit number (100), use the last three digits ofInformation gained from a sample can be gen- the random table number that corresponderalized only to the larger population from to a number on the population list, (anywhich the sample is taken. Conclusions drawn number between 1 and 100) until thefrom a representative sample of teenagers in needed sample size is obtained. The corre-Adams County may be generalized to all sponding names on the population listteenagers in Adams County. They should not form the sample.be generalized to teenagers in other counties oracross the state.
  • 6. 4 s s s P R O G R A M D E V E L O P M E N T A N D E V A L U A T I O N Systematic sample Stratified sample This process is easier and may be substituted In stratified sampling, the total population is for random sampling with large populations. divided into separate groups (strata) which Sampling starts from some randomly selected differ along selected characteristics such as point on the population list (again, close your gender, age, size of operation, or geographical eyes and point) and proceed by selecting indi- location. A random sample is drawn from each viduals or units at intervals thereafter. The subgroup or stratum. interval is determined by dividing the total This method is especially appropriate when number on the population list by the required particular subgroups are known to vary or sample size. For example, if a sample of 200 is when some characteristic, such as age, is desired from a population of 800 people, the known to be related to the outcome of interest. interval would be 800/200 = 4. Randomly For example, we might want to draw a sample select a starting point and select every fourth from each age grouping in the program to name on the list thereafter. If you come to the ensure that each is fully represented. To do so, end of the list and donÕt have the required determine what percentage each age group sample size, go to the beginning of the list and makes up of the total number of participants. keep counting and choosing until the required Use this percentage to determine the propor- number is obtained. tion of the sample that must come from each A potential problem with this method is the age group. Then, randomly select an equal order in which names appear on the popula- number of participants from those enrolled in tion list. Individuals may be listed in a certain the corresponding age group. The example order such as husbandsÕ names preceding that follows shows the sample size for each wivesÕ; groupings by age; groupings by times age stratum in a nutrition program where the and days of the week, or some other way. total number of participants is 250 and the Even alphabetical listings may result in biased desired sample size is 154. selection when ethnic names are alphabeti- cally clustered, resulting in under- or overrep- resentation. Systematic sampling works best if the names on the list are randomized first. Enrollment and stratified sample of a nutrition program Age 20–30 31–40 41–50 50+ TOTAL Number enrolled 50 112 68 20 250 % of total group 20% 45% 27% 8% 100% Sample size 31 69 42 12 154
  • 7. S A M P L I N G s s s 5Sample size population adopted it. If results show that 95%When your purpose is to generalize or show of the sample improved in parenting skills,representativeness, the size of the sample you can feel comfortable saying that 93.5% tobecomes an issue. How many individuals or 97.5% of all participants improved in thoseunits should you include so that the sample skills.provides a fair representation? Often, a propor- The risk of being wrong within the margin oftion of the population is used to determine the error is known as the confidence level. Thatsample size. For example, you might sample is, a 95% confidence level (C = .95) means that10% of all producers in the county, or 20% of there is a 5% chance that the results will notall program participants. Whether you use pro- fall within the specified margin of error. Usingportions depends on the size of the total popu- the previous example, there is a 5% chance thatlation. In a program with 200 participants, a this interpretation is not correctÑthat 93.5% to20% sample would produce a sample of 40 97.5% of all participants did not improve theirpeopleÑunderrepresenting the population skills. We can say that we are Ò95% sureÓ thatsince there is a large chance for respondent our conclusions accurately reflect the totalvariability. On the other hand, a 20% sample of population.50,000 county inhabitants produces a sample of Most studies in Extension allow for a 5Ð10%10,000Ñan unnecessary oversampling. You do margin of error with a confidence level of 95%.not gain much more precision than you would Appendix 2 gives estimates of suggestedusing a sample of 400. You can use the table in sample sizes at different margins of error.Appendix 2 as a guide to recommended Extension practitioners recommend that whensample sizes for various population sizes. greater accuracy is needed, all individuals beNotice that the smaller the population, the included when the population is 100 or less. Inlarger the sample size. This is because smaller this manner, the problem in sampling smallpopulations exhibit greater variability. populations is overcome: Because small popu-Variability, known as sampling error, exists lations exhibit greater variability, a greater pro-within any sample. The larger the sample, the portion of the population must be sampled.smaller the sampling error. But no sample will Anticipating nonresponseyield exactly the same information as if allpeople in the population had been included. Generally, a certain number of people will notInformation collected from a sample is used to respond for one reason or another. The samplemake estimates about the population. The size needs to be large enough to compensateintent is to produce as close an approximation for these nonresponses. A way to do this is toof the population as possible within the con- estimate the rate of nonresponse (30% nonre-straints of time and money. sponse may be realistic) and increase the sample size accordingly. For example, if theTo estimate how closely the sample approxi- total group of Extension participants is 150 andmates the population, two parameters are setÑ we set a margin of error at 10%, we need athe margin of error and the confidence level. sample size of 61 participants (Appendix 2). IfThe margin of error indicates the range of we think that 70% of the individuals willvalues that can result when you use a sample respond, we actually need to begin with ato estimate the population. For example, a 5% sample of 87 (61/.70) to ensure that we end upmargin of error means that if 50% of the with 61 people.sample adopted a recommendation, you can befairly certain that 47.5% to 52.5% of the whole
  • 8. 6 s s s P R O G R A M D E V E L O P M E N T A N D E V A L U A T I O N Sampling for Sample size When your purpose is to examine selected other purposes: cases in greater depth, the previous recommen- nonprobability dations for sample size do not apply. samples Often, in purposeful sampling the sample size is In some instances, probability sampling may very smallÑpossibly even just one case study be impossible, unnecessary or even undesir- (n=1). For example, you may wish to conduct a able. You may be limited to only those partici- single case study of a low-income participant pants or those counties that agree to be and the difference a program made in his or her included. Or you may want more in-depth life. Or, your purpose may require an in-depth information regarding a particular program, analysis of successful community collabora- participants or delivery method. In nonproba- tions, highlighting the factors affecting success. bility sampling, there is no expectation that Another option is to ask knowledgeable people each unit has an equal chance of being to identify such collaborations and then select a included in the sample. Since the sample does certain number to include in your sample. If the not intend to represent the population, find- purpose of the evaluation is to document diver- ings should not be generalized to the whole. sity or variation, a larger number of cases may be necessary to capture the variety. What is the Strategies recommended sample size? There are no rules. It depends upon what you want to know, what Quota sample will be useful, what will be credible, and what A quota sample divides the population being can be accomplished within the time and studied into subgroups such as male and resources you have available. female, younger and older. You estimate the proportion of people in each group based on When sampling for the purpose of generaliz- what you know about the population in ability, the sample size is set in advance. With general. For example, if you know that 20% of purposeful sampling, on the other hand, the the population is headed by a single-parent sample size may change as the study pro- family, then search for respondents until 20% gresses. For instance, based on early investiga- of your sample is single parents. This is not the tions, you may identify other Òinformation- same as the stratified random sample because richÓ cases to include. Or, you may terminate not every single parent is identified and has an data collection when no new information is equal opportunity of being included. forthcoming from the new sampled units. Purposeful sampling More than likely, the factors determining Patton (1990) describes a number of sampling sample size will be time and resources. Select strategies that serve purposes other than repre- the individual cases carefully to choose those sentativeness or randomness (see table 1). Basic that are most likely to provide the information to all these is the importance of selecting Òinfor- you are seeking. Finally, explain and justify mation-richÓ cases from which you can learn your sampling procedures and decisions so much about issues that are important to the that information users and decisionmakers study. Focus on the specific rather than the understand your logic. As a professional, you general; for example, if an evaluationÕs purpose are obliged to describe the strengths and weak- is to increase a programÕs effectiveness in reach- nesses of the sampling procedures that are rel- ing low-income families, you may learn more evant for understanding the findings. Be by conducting an in-depth query of the few careful not to generalize but to focus on the poor families in the program than by gathering intention and strengths of a purposeful standardized information from a random sample. sample of all participants.
  • 9. S A M P L I N G s s s 7Table 1. Purposeful sampling strategiesExtreme or deviant case sampling Learning from highly unusual cases, such as outstanding successes/notable failures; top of the class/dropouts; crises.Intensity sampling Information-rich cases that manifest the phenomenon intensely, but not extremely, such as good students/ poor students; above average/below average.Maximum variation sampling Purposefully pick cases to illustrate a wide range of variation on dimensions of interest. Identifies important common patterns that cut across variations.Homogeneous sampling Pick cases that are alike. Focuses, reduces variation, simplifies analysis, faclitates group interviewing.Typical case sampling Illustrates or highlights what is typical, normal, average.Stratified purposeful sampling Illustrates characteristics of particular subgroups of interest; facilitates comparisons.Critical case sampling Permits logical generalization and maximum application of information to other cases because if itÕs true of this one case, itÕs likely to be true of all other cases.Snowball or chain sampling Relies on people identifying other people or cases to investigate. As they identify new names, the snowball gets bigger. A few key names may be mentioned repeatedly indicating their special importance. Particularly useful when there is no population list or you want to draw a sample based on recommendations.Criterion sampling Picking all cases that meet some criterion, such as all children abused in a treatment facility.Theory-based or operational Finding manifestations of a theoretical construct ofconstruct sampling interest so as to elaborate and examine the construct.Confirming and disconfirming cases Elaborating and deepening initial analysis, seeking exceptions, testing variation.Opportunistic sampling Following new leads during fieldwork, taking advantage of the unexpected, requires flexibility.Random purposeful sampling Uses random selection to select limited(small sample size) number of cases from a larger purposeful sample; adds credibility (not for generalizations or representativeness).Politically important or sensitive Attracts attention to the study (or avoids attractingcase sampling undesired attention by eliminating politically sensitive cases from the sample).Convenience sampling Takes individuals who are available or cases as they occur. Saves time, money and effort. Poorest rationale; lowest credibility.Combination or mixed Combination of above approaches, meets multiplepurposeful sampling interests and needs.Patton, 1990: 182Ð183 (Adapted with permission by author. Reprinted with permission of Sage Publications.)
  • 10. 8 s s s P R O G R A M D E V E L O P M E N T A N D E V A L U A T I O N References Summary Abbey-Livingston, Diane and Davis S. Abbey. 1982. Enjoying Research? A ÔHow- Sampling is the selection of a smaller number toÕ Manual on Needs Assessment. Ontario: of units from among a larger group or popula- Ministry of Tourism and Recreation. tion. To generalize from the sample to the pop- ulation, probability or random sampling is Cochran, W.G. 1963. Sampling Techniques, 2nd needed to ensure that the sample is representa- Ed. N.Y.: John Wiley and Sons, Inc. tive. If generalizations are not desired or neces- Fink, Arlene. 1995. How to Sample in Surveys. sary, nonprobability sampling is appropriate Thousand Oaks, Calif: Sage Publications. and can yield very useful information if cases are selected thoughtfully. Ilvento, Tom W., Paul D. Warner, Richard C. Maurer. 1986. Sampling Issues for Many people are concerned about how many Evaluations in the Cooperative Extension cases to include in their samples. There is no Service. Kentucky Cooperative Extension simple answer. Determining the sample size Service, Lexington, KY. depends upon: 1. The purpose of the study Isaac, Stephen and William B. Michael. 1981. Handbook in Research and Evaluation, 2nd 2. Expectations of information users, and Ed. San Diego, EdITS Publishers. 3. The resources available. Kerlinger, Fred N. 1973. Foundations of Behavioral Research, 2nd Ed. N.Y.: Holt, Rinehart, and Winston, Inc. Patton, Michael Q. 1990. Qualitative Evaluation and Research Methods. 2nd ed. Newbury Park, Calif: Sage Publications. Rohs, F. Richard. 1985. Sampling Techniques for Extension Program Planners and Evaluators. The Cooperative Extension Service, University of Georgia, Athens, Georgia. Sabrosky, Laurel K. 1959. ÒSamplingÓ in Evaluation in Extension, pg. 37-45. ES- USDA, Washington, D.C. Smith, M.F. 1983. Sampling Considerations in Evaluating Cooperative Extension Programs. Florida Cooperative Extension Service, Gainesville, Florida.
  • 11. S A M P L I N G s s s 9Appendix 1. Table of random numbers59391 58030 52098 82718 87024 82848 04190 96574 90464 2906599567 76364 77204 04615 27062 96621 43918 01896 83991 5114110363 97518 51400 25670 98342 61891 27101 37855 06235 3331686859 19558 64432 16706 99612 59798 32803 67708 15297 2861211258 24591 36863 55368 31721 94335 34936 02566 80972 0818895068 88628 35911 14530 33020 80428 39936 31855 34334 6486554463 47237 73800 91017 36239 71824 83671 39892 60518 3709216874 62677 57412 13215 31389 62233 80827 73917 82802 8442092494 63157 76593 91316 03505 72389 96363 52887 01087 6609115669 56689 35682 40844 53256 81872 35213 09840 34471 7444199116 75486 84989 23476 52967 67104 39495 39100 17217 7407315696 10703 65178 90637 63110 17622 53988 71087 84148 1167097720 15369 51269 69620 03388 13699 33423 67453 43269 5672011666 13841 71681 98000 35979 39719 81899 07449 47985 4696771628 73130 78783 75691 41632 09847 61547 18707 85489 6994440501 51089 99943 91843 41995 88931 73631 69361 05375 1541722518 55576 98215 82068 10798 86211 36584 67466 69373 4005475112 30485 62173 02132 14878 92879 22281 16783 86352 0007780327 02671 98191 84342 90813 49268 95441 15496 20168 0927160251 45548 02146 05597 48228 81366 34598 72856 66762 1700257430 82270 10421 00540 43648 75888 66049 21511 47676 3344473528 39559 34434 88596 54086 71693 43132 14414 79949 8519325991 65959 70769 64721 86413 33475 42740 06175 82758 6624878388 16638 09134 59980 63806 48472 39318 35434 24057 7473912477 09965 96657 57994 59439 76330 24596 77515 09577 9187183266 32883 42451 15579 38155 29793 40914 65990 16255 1777776970 80876 10237 39515 79152 74798 39357 09054 73579 9235937074 65198 44785 68624 98336 84481 97610 78735 46703 9826583712 06514 30101 78295 54656 85417 43189 60048 72781 7260620287 56862 69727 94443 64936 08366 27227 05158 50326 5956674261 32592 86538 27041 65172 85532 07571 80609 39285 6534064081 49863 08478 96001 18888 14810 70545 89755 59064 0721005617 75818 47750 67814 29575 10526 66192 44464 27058 4046726793 74951 95466 74307 13330 42664 85515 20632 05497 3362565988 72850 48737 54719 52056 01596 03845 35067 03134 7032227366 42271 44300 73399 21105 03280 73457 43093 05192 4865756760 10909 98147 34736 33863 95256 12731 66598 50771 8366572880 43338 93643 58904 59543 23943 11231 83268 65938 8158177888 38100 03062 58103 47961 83841 25878 23746 55903 4411528440 07819 21580 51459 47971 29882 13990 29226 23608 1587363525 94441 77033 12147 51054 49955 58312 76923 96071 0581347606 93410 16359 89033 89696 47231 64498 31776 05383 3990252269 45030 96279 14709 52372 87832 02735 50803 72744 8820816738 60159 07425 62369 07515 82721 37875 71153 21315 0013259348 11695 45751 15865 74739 05572 32688 20271 65128 1455112900 71775 29845 60774 94924 21810 38636 33717 67598 8252175086 23537 49939 33595 13484 97588 28617 17979 70749 3523499495 51434 29181 09993 38190 42553 68922 52125 91077 4019726075 31671 45386 36583 93459 48599 52022 41330 60651 9132113636 93596 23377 51133 95126 61496 42474 45141 46660 42338
  • 12. 10 s s s Appendix 2. Recommended sample sizes for two different precision levels Sample size Sample size Population Size +5% 10% Population size 5% 10% 10 10 275 163 74 15 14 300 172 76 20 19 325 180 77 25 24 350 187 78 30 28 375 194 80 35 32 400 201 81 40 36 425 207 82 45 40 450 212 82 50 44 475 218 83 55 48 500 222 83 60 52 1000 286 91 65 56 2000 333 95 70 59 3000 353 97 75 63 4000 364 98 80 66 5000 370 98 85 70 6000 375 98 90 73 7000 378 99 95 76 8000 381 99 100 81 51 9000 383 99 125 96 56 10,000 385 99 150 110 61 15,000 390 99 175 122 64 20,000 392 100 200 134 67 25,000 394 100 225 144 70 50,000 397 100 250 154 72 100,000 398 100 Source: Isaac and Michael, 1981; Smith, M. F., 1983 Author: Ellen Taylor-Powell is a program development and evaluation specialist for Cooperative Extension, University of WisconsinÐExtension. An EEO/Affirmative Action employer, University of WisconsinÐExtension provides equal opportunities in employment and programming, including Title IX and ADA requirements. Requests for reasonable accommoda- tion for disabilities or limitations should be made prior to the date of the program or activity for which they are needed. Publications are available in alternative formats upon request. Please make such requests as early as possible by contacting your county Extension office so proper arrangements can be made. If you need this infor- mation in an alternative format, contact the Office of Equal Opportunity and Diversity Programs or call Extension Publications at (608)262-2655. © 1996 by the Board of Regents of the University of Wisconsin System doing business as the division of Cooperative Extension of the University of WisconsinÐExtension. Send inquiries about copyright permission to: Director, Cooperative Extension Publications, 201 Hiram Smith Hall, 1545 Observatory Dr., Madison, WI 53706. You can obtain copies of this publication from Cooperative Extension Publications, Room 170, 630 W. Mifflin Street, Madison, WI 53703, or by phoning (608)262-3346. G3658-3 Program Development and Evaluation, Sampling RP-4-98-.5M-200

×