Quantitative designs

5,257 views

Published on

Published in: Technology
1 Comment
2 Likes
Statistics
Notes
  • Great presentation: I'm refining my ability to design studies and having to review which methods are appropriate/ a lot of this is still marinating ... :)
    I'm working on my DBA (dreams of PHD!) with a MPA (Tribal Governance).....
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
5,257
On SlideShare
0
From Embeds
0
Number of Embeds
1,869
Actions
Shares
0
Downloads
65
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Quantitative designs

  1. 1. Quantitative Research Designs and Methods Notes fromNotes from McMillan & SchumacherMcMillan & Schumacher EDUU600EDUU600
  2. 2. Credibility of Research Design  Goal of sound research designGoal of sound research design  Provide results that approximate realityProvide results that approximate reality  Judged trustworthy and reasonableJudged trustworthy and reasonable  Eliminate or reduce sources of errorEliminate or reduce sources of error  Extraneous effects need to beExtraneous effects need to be controlledcontrolled
  3. 3. Sources of Variability Observations take on different valuesObservations take on different values  SystematicSystematic  Two variables have high variabilityTwo variables have high variability MaximizeMaximize  ErrorError  Sampling and Measurement ErrorsSampling and Measurement Errors MinimizeMinimize
  4. 4. Design Validity Degree to which scientific explanations ofDegree to which scientific explanations of phenomena match the realities of the world.phenomena match the realities of the world.  Internal ValidityInternal Validity  Extent to which extraneous variables have beenExtent to which extraneous variables have been controlled and accounted forcontrolled and accounted for  External ValidityExternal Validity  Generalizability of the resultsGeneralizability of the results  The extent to which the results and conclusionsThe extent to which the results and conclusions can be generalized to other people and othercan be generalized to other people and other settingssettings
  5. 5. Designing Quantitative Research  Who will be assessed? –Who will be assessed? – SUBJECTSSUBJECTS  What will they be assessed by –What will they be assessed by – INSTRUMENTSINSTRUMENTS  How will they be assessed –How will they be assessed – PROCEDURES FOR DATAPROCEDURES FOR DATA COLLECTIONCOLLECTION  In Experimental Designs – How areIn Experimental Designs – How are treatments administered?treatments administered?
  6. 6. Plausible Rival Hypotheses Is there anything that hasIs there anything that has occurred or was done thatoccurred or was done that could provide an explanationcould provide an explanation that is in addition to thethat is in addition to the stated hypothesis or intent ofstated hypothesis or intent of the research?the research?
  7. 7. Possible Sources of Error  Does the researcher have an existing bias aboutDoes the researcher have an existing bias about subjects or topic researched?subjects or topic researched?  Are the subjects aware that they are being studied?Are the subjects aware that they are being studied?  Are the subjects responding honestly?Are the subjects responding honestly?  Did both groups receive treatment as described?Did both groups receive treatment as described?  Does the sex of the interviewer matter?Does the sex of the interviewer matter?  Did very many subjects drop out before the end ofDid very many subjects drop out before the end of the study?the study?  Did the time of day the research was done affect theDid the time of day the research was done affect the results?results?
  8. 8. Population  Group of elements or cases (individuals,Group of elements or cases (individuals, objects, events) that conform to specificobjects, events) that conform to specific criteriacriteria  Group to which we intend to generalize theGroup to which we intend to generalize the results of the researchresults of the research  Target population or UniverseTarget population or Universe
  9. 9. Sample of Population- Probability Sampling  Subjects drawn from a larger populationSubjects drawn from a larger population  Estimate what is true from a smaller sampleEstimate what is true from a smaller sample of the general populationof the general population  Representative of the populationRepresentative of the population  Unbiased random sampleUnbiased random sample  Each member of the population has theEach member of the population has the same chance of being selected as othersame chance of being selected as other members of the groupmembers of the group See page 178-179
  10. 10. Simple Random Sampling  Drawing names out of a hatDrawing names out of a hat  Table of random numbers – setTable of random numbers – set of randomly assorted digitsof randomly assorted digits  Each person randomlyEach person randomly assigned a number from 001assigned a number from 001 to ?to ?  Computer software randomComputer software random selectionselection
  11. 11. Systematic Sampling  EveryEvery nnthth element selected from listelement selected from list  Weakness in sampling techniqueWeakness in sampling technique  when numbers arranged inwhen numbers arranged in systematic patternsystematic pattern  rank order,rank order,  by school, age, scores, etc.by school, age, scores, etc. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
  12. 12. Stratified Random Sampling  Population divided into subgroups on basis of strataPopulation divided into subgroups on basis of strata (age, gender, level of education, etc.)(age, gender, level of education, etc.)  Proportional sampling – according to number inProportional sampling – according to number in stratastrata  Nonproportional sampling – equal numbers betweenNonproportional sampling – equal numbers between stratastrata  Cluster SamplingCluster Sampling  Groups of individuals are identified fromGroups of individuals are identified from populationpopulation  Subjects are drawn from this groupSubjects are drawn from this group
  13. 13. Nonprobability Sampling  Convenience SamplingConvenience Sampling  Accessible subjectsAccessible subjects  Convenience group – classroom of students, members ofConvenience group – classroom of students, members of a groupa group  Purposeful SamplingPurposeful Sampling  Judgment samplingJudgment sampling  Population that will be informative or representativePopulation that will be informative or representative about a topicabout a topic  Quota SamplingQuota Sampling  Select subjects on a basis of certain characteristics of theSelect subjects on a basis of certain characteristics of the populationpopulation  Profiles of groups identified and selected non-randomlyProfiles of groups identified and selected non-randomly
  14. 14. Sample Size  Number of subjects in a studyNumber of subjects in a study  What size of sample will provide sufficientWhat size of sample will provide sufficient data to answer research question?data to answer research question?
  15. 15. Factors to take into consideration  Type of researchType of research  Correlational – minimum of 30Correlational – minimum of 30  Comparing Groups – at least 15Comparing Groups – at least 15  Survey – At least 100Survey – At least 100  Research hypothesisResearch hypothesis  Small differences – large sampleSmall differences – large sample  Relatively small – but perhaps significantRelatively small – but perhaps significant differencesdifferences  Financial ConstraintsFinancial Constraints
  16. 16. Other Considerations  Importance of ResultsImportance of Results  Exploratory – smaller sample sizeExploratory – smaller sample size  When major potential effect on people or $$When major potential effect on people or $$ spent – need large enough sample to minimizespent – need large enough sample to minimize errorerror  Number of variables studied – more variables meanNumber of variables studied – more variables mean larger samplelarger sample  Methods of data collection – accuracy orMethods of data collection – accuracy or consistencyconsistency  Larger the population, the smaller the percentageLarger the population, the smaller the percentage neededneeded
  17. 17. Test Validity  Extent to which inferences and uses made on theExtent to which inferences and uses made on the basis of scores from an instrument are reasonable andbasis of scores from an instrument are reasonable and appropriateappropriate  Appropriateness of measure for specific inferences,Appropriateness of measure for specific inferences, decisions, consequences or uses that results fromdecisions, consequences or uses that results from scores are generatedscores are generated  Dependent on the purpose, population, situationalDependent on the purpose, population, situational factorsfactors  Mental Measurements reports technical informationMental Measurements reports technical information about instrument validity in terms of generalizationabout instrument validity in terms of generalization
  18. 18. Test Reliability  Consistency of the measurementConsistency of the measurement  Extent to which scores are similar overExtent to which scores are similar over different forms of the same instrumentdifferent forms of the same instrument  Scores consistent over different occasionsScores consistent over different occasions of data collectionof data collection  Minimize the influence on the scores ofMinimize the influence on the scores of change or other variables unrelated to intentchange or other variables unrelated to intent of the measureof the measure
  19. 19. Internal Validity of Design  Control over extraneous variablesControl over extraneous variables  Controls for errorControls for error  Source of errors – “threats”Source of errors – “threats”  Threats may invalidate the study’s findingsThreats may invalidate the study’s findings  Various Sources of ThreatsVarious Sources of Threats
  20. 20. Threats to Internal Validity HistoryHistory  Incidents and events that occur during theIncidents and events that occur during the researchresearch  Can occur within the study whenCan occur within the study when something outside or unrelated happenssomething outside or unrelated happens during the treatment in an experimentduring the treatment in an experiment  Outside the research settingOutside the research setting
  21. 21. Threats  SelectionSelection  Systematic difference in groupsSystematic difference in groups  Existing differencesExisting differences  Volunteers – different characteristicsVolunteers – different characteristics  Statistical Regression – tendency for very high orStatistical Regression – tendency for very high or low scores in pretest to regress to the mean onlow scores in pretest to regress to the mean on posttestposttest  Pretesting – Test itself has impact on subjectsPretesting – Test itself has impact on subjects  Stimulate thoughtStimulate thought  Change attitude toward topicChange attitude toward topic
  22. 22. More Validity Threats  InstrumentationInstrumentation  The way the instrument is used to collect dataThe way the instrument is used to collect data  The person who uses the instrumentThe person who uses the instrument  Fatigue, boredom, way of recording dataFatigue, boredom, way of recording data  Subject Attrition – mortality or loss of subjectsSubject Attrition – mortality or loss of subjects during course of studyduring course of study  Maturation –Maturation –  Changes in subjects over a period of extendedChanges in subjects over a period of extended timetime  Fatigue, boredom, concentration, hunger, time ofFatigue, boredom, concentration, hunger, time of dayday
  23. 23. Threats, continued  Diffusion of Treatment - When controlDiffusion of Treatment - When control group/experimental group becomes aware ofgroup/experimental group becomes aware of differences in treatmentdifferences in treatment  Experimenter EffectsExperimenter Effects  Deliberate and intentional effect on subjectsDeliberate and intentional effect on subjects  Tone of voice, gender, race, educational level,Tone of voice, gender, race, educational level, clothing, attitude of experimenterclothing, attitude of experimenter
  24. 24. More Threats to Internal Validity  Treatment ReplicationsTreatment Replications  Number of subjects in study is not the same asNumber of subjects in study is not the same as the number of treatmentsthe number of treatments  Comparing 2 classes – n=2 but number ofComparing 2 classes – n=2 but number of students may be 60.students may be 60.  Group assignment - but individual treatment forGroup assignment - but individual treatment for each studenteach student  Statistical Conclusion – “use of impressive-Statistical Conclusion – “use of impressive- sounding statistics does not guarantee validsounding statistics does not guarantee valid results”results” (pp. 192-193)(pp. 192-193)
  25. 25. Subject Effects Change behavior because they are being studiedChange behavior because they are being studied  Hawthorne effect – increase positive behaviorHawthorne effect – increase positive behavior because they know they are being studiedbecause they know they are being studied  John Henry effect – Group tried harder to competeJohn Henry effect – Group tried harder to compete with other group (compensatory rivalry)with other group (compensatory rivalry)  Resentful demoralization – not selected for theResentful demoralization – not selected for the group with preferred treatmentgroup with preferred treatment  Novelty effect – React with increased motivationNovelty effect – React with increased motivation because they are doing something new and differentbecause they are doing something new and different

×