Research design and methods, Dr. William Kritsonis
RESEARCH DESIGN AND METHODS Research Design William Allan Kritsonis, PhD Educational research is the application of the scientific and disciplinedinquiry approach to the study of educational problems and the primary goalof educational research is to explain and help understand educational issues,questions, and processes. Secondary goals of educational research are tohelp others understand, predict future outcomes, improve future research andpractice, and raise new questions to research. Scientific and disciplined inquiry approach is distinguished by fourcrucial steps. These are: Recognize and identify a question or a problem tobe studied, Describe and execute procedures to collect information about thequestions and problems being studied, Analyze the collected information,and state the results or implications based on the analysis of the information. Basic research is conducted to develop or refine theory, not to solveimmediate practical problems. Applied research is conducted to find
solutions to current practical problems. Most of the educational researchwould be conducted for and classified towards the applied end of the basic-applied research continuum. The purpose of evaluation research is to aiddecision making about educational programs and practices. The general purpose of Quantitative research is to generalize about orcontrol phenomena. Quantitative research has been conducted since theearly 1920s and is the primary way educational researchers have sought tobring about an understanding of educational issues. Quantitative methodsinvolve the collection and analysis of numerical data that is obtained fromtest, questionnaires, checklist, and surveys. Important features ofquantitative research are: defining the problem or questions to be studied anddeveloping a hypotheses that predicts the results of the research before thestudy begins; controlling contextual factors that might influence the resultsof the study; collecting data from samples of participants; and usingnumerical, statistical approaches to analyze the collected data. An importantassumption that underlines the quantitative approach is that the world thatwe live in and carry out our research in is relatively stable, uniform, andcoherent; therefore, it can be measured, understood, and classified. Qualitative research methods involve the collection and analysis ofprimarily nonnumerical data obtained from observation, interviews, taped
information, documents. The important points of qualitative research are thefollowing: defining the problem or question to be studied; theaforementioned point does not have to be stated at the start of the study. Theresearcher also takes into account contextual factors of the researchparticipants; data is collected from a small number of purposely selectedresearch participants; and using nonnumerical, interpretive approaches toprovide narrative descriptions of the participants and their contexts. Theunderlining belief of qualitative research is that the world is neither stable,coherent, nor uniform, and therefore, "truth" as sought by quantitativeresearchers cannot be obtained because perspectives and understandingsdiffer from group to group. Descriptive research and survey research is synonymous in that theyboth collect numerical data to answer questions. Most descriptive studiesobtain information about the preferences, attitudes, practices, concerns, orinterest of some group. The data are collected by a self-administeredinstruments and or telephone polls. The difficulty in conducting descriptiveresearch is constructing clear, consistent descriptive instruments and tryingto acquire instruments from interviewees along with cooperation intelephone interviews.
Correlational research examines the degree of relationship that existbetween two or more variables. A correlation is a quantitative measure ofthe degree of correspondence between two or more variables. The degree ofrelationship is measured by a correlation coefficient of .00 indicating norelationship between the variables. The researcher must keep in mind that ifthere is a high relationship between variables, it does not mean that one is acause of the other. Causal-Comparative research seeks to investigate cause and effectrelationships between two or more programs, methods, or groups. Theactivity thought to make a difference; program, method, or group is calledthe causal factor, treatment, or independent variable. The "effect" is calledthe dependent variable. In most casual-comparative research studies theresearcher does not have control over the causal factor because it has alreadyoccurred or cannot be manipulated. This makes cause-effect conclusionstenuous and tentative. Causal-comparative research is useful in thosecircumstances when it is impossible or unethical to manipulate the causalfactor. Experimental research investigates the cause-effect relationship byselecting participants from a single pool and assigning them at random todifferent causal treatments. The researcher controls contextual variables that
might interfere with the study and because of the randomness of selectingand assigning participants into different treatments cause-effect statementsare considered to be true statements. Historical research is a form of qualitative research that involvesinterpreting past events of individuals, important social issues, links betweenthe old and the new, and reinterpretations of prior historical works. Theprimary sources of data are first-person eyewitnesses or authors. Secondarysources are non-first person accounts. Historians use external criticism toassess the authenticity of their data and use internal criticism to assess thetruthfulness of their data. Qualitative research may include a number of specific methods suchas ethnology, ethnomethodology, case study phenomenology, and symbolicinteraction. These methods focus on the deep descriptions of aspects ofpeoples everyday perspectives and context. Qualitative research providesfield-focused, interpretive, detailed descriptions and interpretations ofparticipants and their settings. Long-term immersion into the researchsetting is also very common. Common methods of data collection includeobservation, interviewing, tape and video recording, examining artifacts, andparticipant observation. Data analysis is based on categorizing andinterpreting the observations, conversations with participants, documents,
tape recordings, and interviews collected to provide a description andexplanation of the participants and their experiences. The qualitativeresearcher writes from the perspective of the participants. There are guidelines that a researcher can follow to determine theresearch method needed to be used for a particular study. The same generalproblem can be investigated using many types of research. The moreinformation available about the nature and procedures of a study, the easierto classify it. A structured way to classify a study is to first determine if it isa quantitative or qualitative study. If a study is quantitative, try to identifythe purpose of the study to determine whether it is descriptive, correlational,causal-comparative, or experimental. If the study is qualitative, determinewhether it is historical or qualitative. Look for key words in the title of thestudy: survey, description, relationship, historical, culture, and the like. Identifying a research topic may come from three main sources:theory may be composed of generalizations and concepts, personalexperience, and replication of an existing study. A basic characteristic of agood research problem is that it is researchable. The stating of a research topic for a quantitative study generallyindicates variables and a well-written topic that defines relevant variables,
either directly or operationally. The topic statement is the first component inthe introductory section and will provide the direction for the study. The review of literature involves the systematic identification,location, and analysis of documents containing information related to theresearch topic. The major reasons for the review of literature are to put yourstudy in context with what information already exist and it also points outresearch strategies, procedures, and instruments that may be used. Identifying your sources of information is the first step in acomprehensive research project. Most libraries use a computer catalogsystem and computer databases such as: ERIC, Educational Index,Psychological Abstract, and Dissertation Abstracts. The Internet is acomputer link to the world -wide web that can be utilized through searchengines such as Yahoo or Lycos. Abstracting involves locating, reviewing, summarizing, andclassifying your references. For each reference write a completebibliographic record including authors name, date of publication, title,journal name or book title, volume number, issue number, page number, andlibrary call number. Formulation of a statement of a hypothesis is based on the theory orreview of literature. A characteristic of a good hypothesis is that it is based
on sound rational, a reasoned prediction. A good hypothesis states as clearlyand concisely as possible the expected relationship (or difference) betweentwo variables and defines those variables in measurable terms. A well-statedand well-defined hypothesis must be testable. The six types of hypothesis discussed in the Educational Researchbook are as follows: An inductive hypothesis is a generalization made froma number of observations, a deductive hypothesis is derived from theory andis aimed at providing evidence that supports, expands, or contradicts aspectsof a given theory, A research hypothesis states the expected relationship (ordifference) between two variables, a nondirectional hypothesis indicates thata relationship or difference exists but does not indicate the difference, adirectional hypothesis indicates that a relationship or difference exist andindicates the direction of the difference, and a null hypothesis states thatthere will be no significant relationship (or difference) between variables. A research plan is a detailed description of a proposed study. Itincludes justification for the study, description of the steps that will befollowed in the study, and information about the analysis of the collecteddata. They are always ethical considerations that must be thought of andthere are several sources that a researcher can go to for information. TheNational Research Act of 1974 and The Ethical Principles of Psychologists
are two documents that have guidelines as to the rules and regulations thatone should follow while conducting research. A design is a general strategy for conducting a research study, stepsthat will take place and the order that they will take place. Sampling is the process of selecting a number of individuals for astudy in such a way that the individuals represent the larger group fromwhich they were selected. Random sampling is the process by which all individuals in thedefined population have an equal and independent chance of being selectedfor the sample. Stratified Sampling is the process of selecting a sample insuch a way that the sample represents an identified subgroup in the sameproportion as they exist in the general population. Cluster Sampling is aprocess in which groups, not individuals, are randomly selected. SystematicSampling is a process of sampling by which individuals are selected from alist by taking every Kth name. That is K= number of individuals on the listdivided by the number of participants need for the sample. Instruments of measurement are as follows: Constructs are mentalabstractions such as personality, creativity and intelligence that in mostinstances cannot be measured directly. Constructs become variables whendifferent levels or scores can be used to measure the construct.
Measurement scales describe four different levels of measurement: nominal,which categorize, ordinal which rank, interval variables have equal intervals,and ratio variables have a defined zero point. Cognitive Test are as follows: achievement test that measure thecurrent status of an individual on a particular subject matter, Aptitude testare used to predict how well an individual is likely to do in future test oroccupations, Affective test measure characteristics such as interest, values,attitude, and personality. The five basic types of scales used to measureattitudes are Likert, Semantic, Differential, Rating, Thurstone, and Guttman.Projective test takes into account an ambiguous situation and is preformedthrough the use of association. Validity is the most important quality of a test. It refers to the degreeto which a test measures what it is suppose to measure. Validation is also amatter of degree: highly valid, moderately valid, or generally valid. Contentvalidity is the degree to which a test measures an intended content area.There is also Criterion -Related Validity that is divided into two forms:Concurrent validity is the degree to which scores on a test are related toscores on another test administered at the same time and Predictive validitywhich refers to the degree to which scores on a test are related to scores onanother test to be administered in the future.
Reliability refers to the degree to which a test consistently measureswhatever it is suppose to measure. The five general approaches to reliabilityare stability, equivalence, equivalence and stability, internal consistency, andscore/rater. The Standard Error of Measurement is an estimate of how often onecan expect test score errors of a given size. Qualitative research has characteristics that are common features inthis type of research. There is reliance on interpretation involving intensivefield participation, rich data collected in the field, and the researcher as theprimary synthesizing and interpretive agent of data analysis. Data analysistypically involves a six-step process. They data managing;reading/memoing; describing; classifying interpreting; and representing theresults. Descriptive research involves collecting data in order to testhypotheses or answer questions about opinions. This can be done throughself-report or through observation and is categorized in terms of cross-sectional or longitudinal. Categories to consider are: statement of theproblem, selection of participants, construction of the questionnaire,preparation of the cover letter, pre-testing the questionnaire, follow-up to thequestionnaire, dealing with non-response, and analysis of the results.
Correctional research involves collecting data to determine whetherand to what degree a relationship exists between two or more variables. Thedegree of the relationship is expressed as a correlation coefficient. Theprocess is composed of the following: problem selection, participant andinstrument selection, design and procedure, data analysis and interpretation Causal-comparative research or ex post facto research is used todetermine the cause or reason for existing differences in the behavior in agroup. The basic design involves selecting two groups differing on someindependent variable and comparing them on some dependent variable.Data analysis involves a variety of descriptive and inferential statistics. Experimental research utilizes the manipulation of at least oneindependent variable, controls the other independent variables, and observesthe effect on one or more independent variables. The steps in theexperimental research are basically the same as the steps for other types ofresearch: selection and definition of a problem, selection of participants andmeasuring instruments, selection of a design, execution of a procedure,analysis of data, and formulations of conclusions. Inferential statistics deals with the inferences about populations basedon the results of samples. This type of research allows the researcher togeneralize to a population of individuals based on the information obtained
from a limited number of research participants. There is an expectedstandard error of chance variation among the means. The Null Hypothesissays that there is no true difference or relationship between parameters in thepopulation. The levels of significance are four fold and along with this thereare Type I and Type II errors. When the null hypothesis is true and theresearcher rejects it and says there is a difference, he/she makes an incorrectdecision, this is a type I error. If the null hypothesis is false and theresearcher says it is true and does not reject it, the researcher also makes anincorrect decision referred to as a type II error. Probably the most foremost rule of research report writing is to relateaspects of the study in a manner that accurately reflects what was done andwhat was found.
GLOSSARY OF RESEARCH-RELATED TERMSA-B design A single-subject design in which baseline measurements are repeatedly madeuntil stability is presumably established, treatment is introduced, and an appropriatenumber of measurements are made during treatment.A-B-A design A single-subject design in which baseline measurements are repeatedlymade until stability is presumably established, treatment is introduced, and an appropriatenumber of measurements are made, and the treatment phase is followed by a secondbaseline phase.A-B-A-B design A single-subject design in which baseline measurements are repeatedlymade until stability is presumably established, treatment is introduced, and an appropriatenumber of measurements are made, and the treatment phase is followed by a secondbaseline phase, which is followed by a second treatment phase.abstract A summary of a study, which appears at the beginning of the report anddescribes the most important aspects of the study, including major results andconclusions.accessible population Refers to the population from which the researcher canrealistically select participants.accidental sampling See convenience sampling.achievement test An instrument that measures the current status of individuals withrespect to proficiency in given areas of knowledge or skill.action research An approach in which teachers study their own problems or concerns intheir own classrooms.additive designs Refers to variations of the AB design which involve the addition ofanother phase or phases in which the experimental treatment is supplemented withanother treatment.alternating treatments design A variation of a multiple-baseline design which involvesthe relatively rapid alternation of treatments for a single participant.analysis of covariance A statistical method of equating groups on one or more variablesand for increasing the power of a statistical test; adjusts scores on a dependent variablefor initial differences on some variable such as pretest performance or IQ.analytic induction A method of identifying regularities in qualitative data, determiningtheir explanation, and finding other contexts to determine whether the explanations holdup.applied research Research conducted for the purpose of applying, or testing, theory andevaluating its usefulness in solving problems.aptitude test A measure of potential used to predict how well someone is likely toperform in a future situation.artificial categories Categories which are operationally defined by the researcher.assumption Any important "fact" presumed to be true but not actually verified;assumptions should be described in the procedures section of a research plan or report.
attenuation Refers to the principle that correlation coefficients tend to be loweredbecause less-than-perfectly reliable measures are used.basic research Research conducted for the purpose of theory development or refinement.case study The in-depth investigation of one "unit," e.g., individual, group, institution,organization, program, or document.category The classification of ideas and concepts in qualitative data analysis.causal-comparative research Research that attempts to determine the cause, or reason,for existing differences in the behavior or status of groups of individuals; also referred toas ex post facto research.census survey Descriptive research that attempts to acquire data from each and everymember of a population.changing criterion design A variation of the A-B-A design in which the baseline phaseis followed by successive treatment phases, each of which has a more stringent criterionfor acceptable behavior level.chi square A nonparametric test of significance appropriate when the data are in the formof frequency counts; it compares proportions actually observed in a study withproportions expected to see if they are significantly different.clinical replication Refers to the development and application of a treatment package,composed of two or more interventions which have been found to be effectiveindividually, designed for persons with complex behavior disorders.cluster sampling Sampling in which intact groups, not individuals, are randomlyselected.coefficient alpha (a) See Cronbachs alpha.common variance The variation in one variable that is attributable to its tendency to varywith another variable.concurrent validity The degree to which the scores on a test are related to the scores onanother, already established test administered at the same time, or to some other validcriterion available at the same time.constant comparison A qualitative method for identifying similarities and differences bycomparing new evidence to prior evidence.construct validity The degree to which a test measures an intended hypotheticalconstruct, or nonobservable trait, which explains behavior.contamination The situation that exists when the researchers familiarity with theparticipants affects the outcome of the study.content analysis The systematic, quantitative description of the composition of the objectof the study.content validity The degree to which a test measures an intended content area; it isdetermined by expert judgment and requires both item validity and sampling validity.control Efforts on the part of the researcher to remove the influence of any variableother than the independent variable that might affect performance on a dependentvariable.control group The group in a research study that either receives a different treatmentthan the experimental group or is treated as usual.control variable A nonmanipulated variable, usually a physical or mental characteristicof the participants (such as IQ).
convenience sampling The process of using as the sample whoever happens to beavailable, e.g., volunteers. (Also referred to as accidental sampling and haphazardsampling.) correlational research Research that involves collecting data in order todetermine whether, and to what degree, a relationship exists between two or morequantifiable variables.correlation coefficient A decimal number between .00 and :!:1.00 that indicates thedegree to which two variables are related.counterbalanced design A quasi-experimental design in which all groups receive alltreatments, each group receives the treatments in a different order, the number of groupsequals the number of treatments, and all groups are post tested after each treatment.credibility A term used in qualitative research to indicate that the topic was accuratelyidentified and described.criterion In a prediction study, the variable that is predicted.criterion-related validity Validity which is determined by relating performance on a testto performance on another criterion; includes concurrent and predictive validity.Cronbachs alpha (a) The general formula for estimating internal consistency based on adetermination of how all items on a test relate to all other items and to the total test. (Also referred to as coefficient alpha and Cronbachs coefficient alpha.) cross-validationValidation of a prediction equation with at least one group other than the group on whichit was based; variables that are no longer found to be related to the criterion measure areremoved from the equation.curvilinear relationship A relationship in which increase in one variable is associatedwith a corresponding increase in another variable to a point, at which point furtherincrease in the first variable is associated with a corresponding decrease in the othervariable (or vice versa).data saturation A point in qualitative research when so much data are collected that it isvery unlikely that additional data will add to what is already collected.deductive hypothesis A hypothesis derived from theory which provides evidence whichsupports, expands, or contradicts the theory.dependent variable The change or difference in behavior that occurs as a result of theindependent variable; also referred to as the criterion variable, the effect, the outcome, orthe post test.descriptive statistics Data analysis techniques enabling the researcher to meaningfullydescribe many scores with a small number of numerical indices.developmental studies Studies concerned with behavior variables that differentiatechildren at different levels of age, growth, or maturation.diagnostic test A type of achievement test yielding multiple scores for each area ofachievement measured that facilitate identification of specific areas of deficiency.differential selection of participants Refers to the fact that groups may be differentbefore a study even begins, and this initial difference may at least partially account forpost test differences.direct replication Refers to the replication of a study by the same investigator, with thesame participants or with different participants, in a specific setting.ecological validity The degree to which results can be generalized to environmentsoutside of the experimental setting.
educational research The formal, systematic application of the scientific and disciplinedinquiry approach to the study of educational problems.environmental variable A variable in the setting in which a study is conducted thatmight cause unwanted differences between groups (e.g., learning materials).equivalent forms Two tests identical in every way except for the actual items included.equivalent-forms reliability Indicates score variation that occurs from form to form of atest; also referred to as alternate-forms reliability.ethnographic research A qualitative approach that studies the cultural patterns andperspectives of participants in their natural setting.ethnomethodology A qualitatiTe approach that studies how participants make sense oftheir everyday activities to act in a social way.evaluation The systematic process of collecting and analyzing data in order to makedecislons.experimental group The group in a research study that typically receives a new, ornovel, treatment, a treatment under investigation.experimental research Research in which at least one independent variable ismanipulated, other relevant variables are controlled, and the effect on one or moredependent variables is observed.experimenter bias A situation in which the researchers expectations concerning theoutcomes of the study actually contribute to producing various outcomes.ex post facto research See causal-comparative research.external criticism The analysis of data to determine their authenticity.external validity The degree to which results are generalizable, or applicable, to groupsand environments outside of the experimental setting.factorial analysis of variance The appropriate statistical analysis if a study is based on afactorial design and investigates two or more independent variables and the interactionsbetween them; yields a separate F ratio for each independent variable and one for eachinteraction.factorial design An experimental design that involves two or more dependent variables(at least one of which is manipulated) in order to studv the effects of the variablesindividually a~d in interaction with each other.fieldwork A qualitative research strategy that involves spending considerable time in thesetting under study, immersing oneself in this setting, and collecting as much relevantinformation as possible as unobtrusively as possible.follow-up study A study conducted to determine the status of a group of interest aftersome period of time.generosity error The tendency to give an individual the benefit of the doubt wheneverthere is insufficient knowledge to make an objective judgment.grounded theory Theory based on data collected in real-world settings which reflectwhat naturally occurred over an extended period of time.halo effect The phenomenon whereby initial impressions concerning an individual(positive or negative) affect subsequent measurements.haphazard sampling See convenience sampling.hardcopy Refers to computer output that is printed out on paper.hardware Refers to the actual equipment, the computer itself and related accessoriessuch as printers.
Hawthorne effect A type of reactive arrangement resulting from the participantsknowledge that they are involved in an experiment, or their feeling that they are in someway receiving "special" attention.historical research The systematic collection and evaluation of data related to pastoccurrences in order to describe causes, effects, or trends of those events which may helpto explain present events and anticipate future events.history Any event which is not part of the experimental treatment but which may affectperformance on the dependent variable.hypothesis A tentative, reasonable, testable explanation for the occurrence of certainbehaviors, phenomena, or events.independent variable An activity or characteristic believed to make a difference withrespect to some behavior; also referred to as the experimental variable, the cause, and thetreatment.inductive hypothesis A generalization based on observation.inferential statistics Data analysis techniques for determining how likely it is that resultsbased on a sample or samples are the same results that would have been obtained for anentire population.instrumentation Unreliability in measuring instruments that may result in invalidassessment of participants performance.interaction Refers to the situation in which different values of the independent variableare differentially effective depending upon the level of the control variable.interiudge reliability The consistency of two (or more) independent scorers, raters, orobservers.internal criticism The analysis of data to determine their accuracy which takes intoconsideration the knowledge and competence of the author, the time delay between theoccurrence and recording of events, biased motives of the author, and consistency of thedata.internal validity The degree to ,hich obse!ed differences on the dependent variable area direct result of manipulation of the independent variable, not some other variable.interval scale A measurement scale that classifies and ranks participants, is based uponpredetermined equal intervals, but does not have a true zero point.intervening variable A variable which intervenes between, or alters the relationshipbetween, an independent variable and a dependent variable, which cannot be directlyobserved or controlled (e.g., anxiety) but which can be controlled for.intrajudge reliability The consistency of the scoring, rating, or observing of anindividual.item validity The degree to which test items represent measurement in the intendedcontent area.John Henry effect The phenomenon whereby if for any reason members of a control groupfeel threatened or by being in competition with an experimental group, they mayout do themselves and perform way beyond what would normally be expected.judgment sampling The process of selecting a sample which is believed to berepresentative of a given population. (Also referred to as purposive sampling.) Likertscale An instrument that asks an individual to respond to a series of statements byindicating whether she or he strongly agrees (SA), agrees (A), is undecided (U), disagrees(D), or strongly disagrees (SD) with each statement.
limitation An aspect of a study which the researcher knows may negatively affect theresults or generalizability of the results, but over which he or she has no control.linear relationship The situation in which an increase (or decrease) in one variable isassociated with a corresponding increase (or decrease) in another variable.logical validity Validity which is determined primarily through judgment; includescontent validity.matching A technique for equating groups on one or more variables, resulting in eachmember of one group having a direct counterpart in another group.maturation Physical or mental changes which occur within participants over a period oftime and which may affect their performance on a measure of the dependent variable.mean The arithmetic average of a set of scores.measures of Indices representing the average or typical score attained by a group ofparticipants.measures of variability Indices indicating how spread out the scores are in adistribution.median That point in a distribution above and below which are 50% of the scores.menu-driven Refers to computer programs which allow the user to select desiredanalyses from a list, or menu, of options.meta-analysis A statistical approach to summarizing the results of many studies whichhave investigated basically the same problem.mode The score that is attained by more pa;..ticipants in a group than any other score.modem A device which permits telephone communication between two computers byconverting computer language to audiotones.mortality Refers to the fact that participants who drop out of a study may share acharacteristic such that their absence has a significant effect on the results of the study.multiple-baseline design A single-subject design in which baseline data are collected onseveral behaviors for one participant or one behavior for several participants andtreatment is applied systematically over a period of time to each behavior (or eachparticipant) one at a time until all behaviors (or participants) are under treatment.multiple comparisons Procedures used following application of analysis of variance todeterrnine which means are significantly different from which other means.multiple regression equation A prediction equation using two or more variables thatindividually predict a criterion to make a more accurate prediction.multiple time-series design A variation of the time-series design that involves theaddition of a control group to the basic design.multiple-treatment interference Refers to the carry-over effects from an earliertreatment that make it difficult to assess the effectiveness of a later treatment.naturalistic observation Observation in which the observer purposely controls ormanipulates nothing, and in fact works very hard at not affecting the observed situation inany way.negatively skewed distribution A distribution in which there are more extreme scores atthe lower end than at the upper, or higher, end.nominal scale The lowest level of measurement which classifies persons or objects intotwo or more categories; a person can only be in one category, and members of a categoryhave a common set of characteristics.
nonequivalent control group design A quasiexperimental design involving at least twogroups, both of which are pretested; one group receives the experimental treatment, andboth groups are posttested.nonparametric test A test of significance appropriate when the data represent an ordinalor nominal scale, when a parametric assumption has been greatly violated, or when thenature of the distribution is not known.nonparticipant observation Observation in which the observer is not directly involvedin the situation to be observed, i.e., the observer does not intentionally interact with oraffect the object of the observation.nonprobability sampling The process of selecting a sample using a technique whichdoes not permit the researcher to specify the probability, or chance, that each member ofa population has of being selected for the sample.novelty effect A type of reactive arrangement resulting from increased interest,motivation, or participation on the part of participants simply because they are doingsomething different.null hypothesis States that there is no relationship (or difference) between variables andthat any relationship found will be a chance relationship, the result of sampling error, nota true one.observational research Descriptive research in which the desired data is obtained not byasking individuals for it but through such means as direct observation.observer bias The phenomenon whereby an observer does not observe objectively andaccurately, thus producing invalid observations.observer effeds The phenomenon whereby persons being observed behave atypicallysimply because they are being observed, thus producing invalid observations.one-group pretest-posttest design A pre-experimental design involving one groupwhich is pretested, exposed to a treatment, and post tested.one-shot case study A pre-experimental design involving one group which is exposed toa treatment and then post-tested.operational definition One which defines concepts in terms of processes, or operations.ordinal scale A measurement scale that classifies participants and ranks them in terms ofthe degree to which they possess a characteristic of interest.organismic variable A characteristic of a participant, or organism (e.g., sex), which can-not be directly controlled but which can be controlled for.parameter A numerical index describing the behavior of a population.parametric test A test of significance appropriate when the data represent an interval orratio scale of measurement and other assumptions have been met.participant A person who provided data for a research study.participant observation Observation in which the observer actually becomes a part of, aparticipant in, the situation to be observed.participant variable A variable on which participants in different groups in a studymight differ, e.g., intelligence.pattern The connection of categories in qualitative analysis.Pearson r A measure of correlation appropriate when the data represent either interal orratio scales; it takes into account each and every score and produces a coefficient between.00 and ::::1.00.
percentile rank A measure of relative position indicating the percentage of scores thatfall at or below a given score.phenomenology The experience of an activity or concept from participants perspectivespilot study A small-scale study conducted prior to the conducting of the actual study;the entire study is conducted, every procedure is followed, and the resulting data areanalyzed-all according to the research plan.placebo effect Refers to the discovery in medical research that any "medication" couldmake participants feel better, even sugar and water.population The group to which the researcher would like the results of a study to begeneralizable.positively skewed distribution A distribution in which there are more extreme scores atthe upper, or higher, end than at the lower end.posttest-only control group design A true experimental design involving at least tworandomly formed groups; one group receives a new, or unusual, treatment and bothgroups are post tested.power The ability of a significance test to avoid making a Type II error.prediction study An attempt to determine which of a number of variables are most highlyrelated to a criterion variable, a complex variable to be predicted.predictive validity The degree to which a test is able to predict how well an individualwill do in a future situation.predictor In a prediction study, the variable upon which the prediction is based.pretest-postest control group design A true experimental design which involves at leasttwo randomly formed groups; both groups are pretested, one group receives a new, orunusual treatment, and both groups are post tested.pretest sensitization See testing.pretest-treatment interaction Refers to the fact that participants may respond or reactdifferently to a treatment because they have been pretested.primary source Firsthand information such as the testimony of an eyewitness, anoriginal document, a relic, or a description of a study written by the person whoconducted it.probability sampling The process of selecting a sample using a sampling techniquewhich permits the researcher to specify the probability, or chance, that each member of adefined population has of being selected for the sample.problem statement A statement which indicates the variables of interest to theresearcher and the specific relationship behveen those variables which is to be, or was,investigated.prospective causal-comparative research A variation of the basic approach tocausalcomparative research which involves starting with the causes and investigatingeffects.purposive sampling See judgment sampling.qualitative approach The collection of extensive narrative data in order to gain insightsinto phenomena of interest.qualitative research The collection of extensive narrative data on many variables overan extended period of time, in a naturalistic setting, in order to gain insights not possibleusing other types of research.
quantitative research The collection of numerical data in order to explain, predict and/or control phenomena of interest.quartile deviation One-half of the difference between the upper quartile (the 75thpercentile) and the lower quartile (the 25th percentile) in a distribution.quota sampling The process of selecting a sample based on required, exact numbers, orquotas, of persons of varying characteristics.random sampling The process of selecting a sample in such a way that all individuals inthe defined population have an equal and independent chance of being selected for thesample.range The difference between the highest and lowest score in a distribution.rationale equivalence reliability An estimate of internal consistency based on adetermination of how all items on a test relate to all other items and to the total test.ratio scale The highest level of measurement that classifies participants, ranksparticipants, is based upon predetermined equal intervals, and has a true zero point.reactive arrangements Threats to the external validity of a study associated with theway in which a study is conducted and the feelings and attitudes of the participantsinvolved.readiness test A test administered prior to instruction or training in a specific area inorder to determine whether and to what degree a student is ready for, or will profit from,instruction.relationship study An attempt to gain insight into the variables, or factors, that arerelated to a complex variable such as academic achievement, motivation, and self-concept.reliability The degree to which a test consistently measures whatever it measures.replication Refers to when a study is done again; the second study may be a repetition ofthe original study, using different participants, or it may represent an alternative approachto testing the same hypothesis.research The formal, systematic application of the scientific and disciplined inquiryapproach to the study of problems.research hypothesis A statement of the expected relationship (or difference) betweentwo variables.research plan A detailed description of a proposed study designed to investigate a givenproblem.response set The tendency of an observer to rate the majority of observees the sameregardless of the observees actual behavior.retrospective causal-c:omparative research The basic approach to causal-<:omparativeresearch which involves starting with effects and investigating causes.review of literature The systematic identification, location, and analysis of documentscontaining information related to a research problem.sample A number of individuals selected from a population for a study, preferably insuch a way that they represent the larger group from which they were selected.sample survey Research in which information about a population is inferred based on theresponses of a sample selected from that population.sampling The process of selecting a number of individuals (a sample) from a population,preferably in such a way that the individuals selected represent the larger group fromwhich they were selected.
sampling bias Systematic sampling error; two major sources of sampling bias are the useof volunteers and the use of available groups.sampling error Expected, chance variation in variables that occurs when a sample isselected from a population.sampling validity The degree to which a test samples the total intended content area.Scheffe test A conservative multiple comparison technique appropriate for making anyand all possible comparisons involving a set of means.secondary source Secondhand information, such as a brief description of a study writtenby someone other than the person who conducted it.selection-maturation interaction Refers to the fact that if already-formed groups areused in a study, one group may profit more (or less) from treatment or have an initialadvantage (or disadvantage) because of maturation factors; selection may also interactwith factors such as history and testing.selection-treatment interaction Refers to the fact that if nonrepresentative groups areused in a study the results of the study may hold only for the groups involved and maynotbe representative of the treatment effect in the population.self-report research Descriptive research in which information is solicited fromindividuals using, for example, questionnaires or interviews.semantic differential scale An instrument that asks an individual to give a quantitativerating to the participant of the attitude scale on a number of bipolar adjectives such asgoodbad, friendly-unfriendly, positive-negative. shrinkage Refers to the tendency of aprediction equation to become less accurate when used with a different group, a groupother than the one on which the equation was originally formulated.simple analysis of variance (ANOVA) A parametric test of significance used todetermine whether there is significant difference between or among two or more means ata selected probability level.simulation observation Observation in which the researcher creates the situation to beobserved and tells the participant what activities they are to engage in.simultaneous replication Refers to when replication is done on a number of participantswith the same problem, at the same location, at the same time.single-subject experimental designs Designs applied when the sample size is one; usedto study the behavior change which an individual exhibits as a result of someintervention, or treatment.single-variable designs A class of experimental designs involving only one independentvariable (which is manipulated).single variable rule An important principle of single-subject research which states thatonly one variable should be manipulated at a time.skewed distribution A nonsymmetrical distribution in which there are more extremescores at one end of the distribution than the other.sociometric study A study that assesses and analyzes the interpersonal relationshipswithin a group of individuals.software Refers to the programs which give instructions to the computer concerningdesired operationsSolomon four-group design A true experimental design that involves randomassignment of participants to one of four groups; two groups are pretested, two are not;
one of the pretested groups and one of the unpretested groups receive the experimentaltreatment, and all four groups are post tested.Spearman rho A measure of correlation appropriate when the data for at least one of thevariables are expressed as ranks; it produces a coefficient between .00 and :!:1.00.specificity of variables Refers to the fact that a given study is conducted with a specifickind of participant, using specific measuring instruments, at a specific time, under aspecific set of circumstances, factors that affect the generalizability of the results.split-half reliability A type of reliability that is based on the internal consistency of a testand is estimated by dividing a test into two equivalent halves and correlating the scoreson the two halves.standard deviation The most stable measure of variability which takes into account eachand every score in a distribution.standard error of the mean The standard deviation of sample means which indicates byhow much the sample means can be expected to differ if other samples from the samepopulation are used.standard error of measurement An estimate of how often one can expect errors of agiven size in an individuals test score.standard score A derived score that expresses how far a given raw score is from somereference point, typically the mean, in terms of standard deviation units.stanines Standard scores that divide a distribution into nine parts.static group comparison A pre-experimental design that involves at least twononrandomly formed groups; one receives a new, or unusual, treatment and both areposttested.statistic A numerical index describing the behavior of a sample or samples.statistical regression The tendency of participants who score highest on a pretest toscore lower on a post test, and of participants who score lowest on a pretest to scorehigher on a post test.statistical significance The conclusion that results are unlikely to have occurred bychance; the observed relationship or difference is probably a real one.stratified sampling The process of selecting a sample in such a way that identifiedsubgroups in the population are represented in the sample in the same proportion thatthey exist in the population or in equal proportion.structured interview Interview questions that provide options for participants to selectfrom.structured item A question and a list of alternative responses from which the responderselects; also referred to as a closed-form item.survey An attempt to collect data from members of a population in order to determine thecurrent status of that population ith respect to one or more variables.systematic replication Refers to replication which follows direct replication, and whichinvolves different investigators, behaviors, or settings.systematic sampling Sampling in which individuals are selected from a list by takingevery Kth name, where K equals the number of individuals on the list divided by thenumber of participants desired for the sample.T score A standard score derived from a z score by multiplying the z score by 10 andadding 50.
t test for independent samples A parametric test of significance used to determinewhether there is a significant difference between the means of two independent samplesat a selected probability level.t test for nonindependent samples A parametric test of significance used to determinewhether there is a significant difference between the means of two matched, ornonindependent, samples at a selected probability level.target population Refers to the population to which the researcher would ideally like togeneralize results.terminal A device for communicating with a computer which consists of a display screenand a keyboardtest A means of measuring the knowledge, skill, feelings, intelligence, or aptitude of anindividual or group.testing A threat to experimental validity which refers to improved scores on a post testwhich are a result of participants having taken a pretest. (Also referred to as pretestsensitization.) test objectivity Refers to a situation in which an individuals score is thesame, or essentially the same, regardless of who is doing the scoring.test of significance A statistical test used to determine whether or not there is asignificant difference between or among two or more means at a selected probabilitylevel.test-retest reliability The degree to which scores on a test are consistent, or stable, overtime.time-series design A quasi-experimental design involving one group which is repeatedlypretested, exposed to an experimental treatment, and repeatedly post tested.triangulation The use of multiple methods, data collection strategies, and/ or datasources, in order to get a more complete picture and to cross-check information.true categories Categories into which persons or objects naturally fall, independently ofthe research study.Type I error The rejection by the researcher of a null hypothesis which is actually true.Type II error The failure of a researcher to reject a null hypothesis which is really false.unobtrusive measures Inanimate objects (such as school suspension lists) which can beobserved in order to obtain desired information.munstructured ite A question giving the responder complete freedom of response.validity The degree to which a test measures what it is intended to measure; a test is validfor a particular purpose for a particular group.variable A concept that can assume anyone of a range of values, e.g., intelligence,height, aptitude.z score The most basic standard score that expresses how far a score is from a mean mterms of standard deviation.Z score See T score.Link: http://www.artsci-ccwin.Concordia.ca/edtech/et664.html http://www.ed.gov/offices/oeri/nerppb/ http://www.oeri/research.html http://www./resCtr.html http://www.crede.ucsc.edu/