Lecture 8

300 views

Published on

  • Be the first to comment

  • Be the first to like this

Lecture 8

  1. 1. Lecture 8: ResearchLecture 8: Research Methods 2Methods 2 PSY 3617, Tuesday, February 13thPSY 3617, Tuesday, February 13th
  2. 2. TodayToday  ActivityActivity  Lecture 8: Research Methods 2Lecture 8: Research Methods 2  Sample exam, review/questionsSample exam, review/questions
  3. 3. Left to coverLeft to cover  Incremental validityIncremental validity  Construct validityConstruct validity  Biological researchBiological research  Psychosocial researchPsychosocial research
  4. 4. Types of reliability & validityTypes of reliability & validity  Inter-rater reliabilityInter-rater reliability  Test-retest reliabilityTest-retest reliability  Construct validityConstruct validity  Convergent & Discriminant ValidityConvergent & Discriminant Validity  Incremental validityIncremental validity  Book defines reliability & validity…in brief:Book defines reliability & validity…in brief:  Reliability: consistency of measurementReliability: consistency of measurement  Validity: are you measuring what you think you are??Validity: are you measuring what you think you are??  Analogous to…Analogous to…  reliability: precisenessreliability: preciseness  validity: accuracyvalidity: accuracy
  5. 5. Convergent & Discriminant ValidityConvergent & Discriminant Validity  Select 3+Select 3+ traitstraits  Extraversion (“EV”), Constraint and Stress-ReactivityExtraversion (“EV”), Constraint and Stress-Reactivity  Assess all 3 traits with 3+ differentAssess all 3 traits with 3+ different methodsmethods::  EV 1= interview measure of extraversionEV 1= interview measure of extraversion  EV 2= self-report checklist measure of extraversionEV 2= self-report checklist measure of extraversion  EV 3 = behavioral measure of extraversionEV 3 = behavioral measure of extraversion  same for Constraint & Stress Reactivitysame for Constraint & Stress Reactivity
  6. 6. Convergent/DiscriminantConvergent/Discriminant Validity, cont’dValidity, cont’d  Administer all 9 measures toAdminister all 9 measures to heterogeneousheterogeneous samplesample  GetGet multitrait, multimethodmultitrait, multimethod correlation matrixcorrelation matrix  Want correlations within a trait to be high (all methods ofWant correlations within a trait to be high (all methods of assessment give similar rating for individual):assessment give similar rating for individual): monotraitmonotrait  Want correlations between different traits to be low (singleWant correlations between different traits to be low (single method differentiates those traits we expect to be different):method differentiates those traits we expect to be different): monomethodmonomethod
  7. 7. Hypothetical Multitrait MultimethodHypothetical Multitrait Multimethod MatrixMatrix EV1EV1 CN1CN1 SR1SR1 EV2EV2 CN2CN2 SR2SR2 EV3EV3 CN3CN3 SR3SR3 EV1EV1 .89.89 CN1CN1 .51.51 .89.89 SR1SR1 .38.38 .37.37 .76.76 EV2EV2 .57.57 .22.22 .09.09 .93.93 CN2CN2 .22.22 .57.57 .10.10 .68.68 .94.94 SR2SR2 .11.11 .11.11 .46.46 .59.59 .58.58 .84.84 EV3EV3 .56.56 .22.22 .11.11 .67.67 .42.42 .33.33 .94.94 CN3CN3 .23.23 .58.58 .12.12 .43.43 .66.66 .34.34 .67.67 .92.92 SR3SR3 .11.11 .11.11 .45.45 .34.34 .32.32 .58.58 .68.68 .60.60 .85.85
  8. 8. Reliability diag (MonoT-MonoM)Reliability diag (MonoT-MonoM) EV1EV1 CN1CN1 SR1SR1 EV2EV2 CN2CN2 SR2SR2 EV3EV3 CN3CN3 SR3SR3 EV1EV1 .89.89 CN1CN1 .51.51 .89.89 SR1SR1 .38.38 .37.37 .76.76 EV2EV2 .57.57 .22.22 .09.09 .93.93 CN2CN2 .22.22 .57.57 .10.10 .68.68 .94.94 SR2SR2 .11.11 .11.11 .46.46 .59.59 .58.58 .84.84 EV3EV3 .56.56 .22.22 .11.11 .67.67 .42.42 .33.33 .94.94 CN3CN3 .23.23 .58.58 .12.12 .43.43 .66.66 .34.34 .67.67 .92.92 SR3SR3 .11.11 .11.11 .45.45 .34.34 .32.32 .58.58 .68.68 .60.60 .85.85
  9. 9. Convergent Val diag (MonoT-Convergent Val diag (MonoT- HeteroM)HeteroM) EV1EV1 CN1CN1 SR1SR1 EV2EV2 CN2CN2 SR2SR2 EV3EV3 CN3CN3 SR3SR3 EV1EV1 .89.89 CN1CN1 .51.51 .89.89 SR1SR1 .38.38 .37.37 .76.76 EV2EV2 .57.57 .22.22 .09.09 .93.93 CN2CN2 .22.22 .57.57 .10.10 .68.68 .94.94 SR2SR2 .11.11 .11.11 .46.46 .59.59 .58.58 .84.84 EV3EV3 .56.56 .22.22 .11.11 .67.67 .42.42 .33.33 .94.94 CN3CN3 .23.23 .58.58 .12.12 .43.43 .66.66 .34.34 .67.67 .92.92 SR3SR3 .11.11 .11.11 .45.45 .34.34 .32.32 .58.58 .68.68 .60.60 .85.85
  10. 10. Discriminant val (HeteroT-MonoM)Discriminant val (HeteroT-MonoM) EV1EV1 CN1CN1 SR1SR1 EV2EV2 CN2CN2 SR2SR2 EV3EV3 CN3CN3 SR3SR3 EV1EV1 .89.89 CN1CN1 .51.51 .89.89 SR1SR1 .38.38 .37.37 .76.76 EV2EV2 .57.57 .22.22 .09.09 .93.93 CN2CN2 .22.22 .57.57 .10.10 .68.68 .94.94 SR2SR2 .11.11 .11.11 .46.46 .59.59 .58.58 .84.84 EV3EV3 .56.56 .22.22 .11.11 .67.67 .42.42 .33.33 .94.94 CN3CN3 .23.23 .58.58 .12.12 .43.43 .66.66 .34.34 .67.67 .92.92 SR3SR3 .11.11 .11.11 .45.45 .34.34 .32.32 .58.58 .68.68 .60.60 .85.85
  11. 11. Incremental ValidityIncremental Validity  Additional utility above & beyond measuresAdditional utility above & beyond measures already usedalready used  Does assessment giveDoes assessment give incrementincrement inin validityvalidity ofof judging/ predicting criterion?judging/ predicting criterion?  Is what it tells us NEW, and TRUE?Is what it tells us NEW, and TRUE?  Practical questions:Practical questions:  How big an increment?How big an increment?  Relative to what prior info?Relative to what prior info?  Importance in cost-effective health care? (TimeImportance in cost-effective health care? (Time spent & by whom = cost)spent & by whom = cost)
  12. 12. Incremental Validity Designs:Incremental Validity Designs: (Within-Judge)(Within-Judge)  Give judge/group info a piece at a time, e.g.:Give judge/group info a piece at a time, e.g.:  No info (stereotype Average Patient)No info (stereotype Average Patient)  Bio (biographical data sheet)Bio (biographical data sheet)  Bio + MMPIBio + MMPI  Bio + MMPI + Rorschach (etc…)Bio + MMPI + Rorschach (etc…)  Compare prediction accuracy at each stepCompare prediction accuracy at each step
  13. 13. Incremental Validity Design:Incremental Validity Design: (Between-Judge)(Between-Judge)  If 4 info sources, 2If 4 info sources, 244 = 16 different info sets:= 16 different info sets:  Bio onlyBio only  Bio + Rorschach + interviewBio + Rorschach + interview  MMPI + Rorschach + interviewMMPI + Rorschach + interview  Bio + MMPI + Rorschach + interviewBio + MMPI + Rorschach + interview  etc.etc.  Randomize info sets to judgesRandomize info sets to judges  Compare prediction of accuracy across setsCompare prediction of accuracy across sets
  14. 14. Research in Clinical PsychologyResearch in Clinical Psychology  Research in etiology of major mental disordersResearch in etiology of major mental disorders (costly/common problems):(costly/common problems):  Schizophrenia, other chronic psychosesSchizophrenia, other chronic psychoses  Major affective disordersMajor affective disorders  Alcoholism, drug abuseAlcoholism, drug abuse  Criminality, antisocial behaviorCriminality, antisocial behavior  Other disordersOther disorders  Biological factorsBiological factors  Psychosocial factorsPsychosocial factors
  15. 15. Research on BiologicalResearch on Biological Etiological FactorsEtiological Factors  Three questions about factor X:Three questions about factor X:  (1) Does X make a difference?(1) Does X make a difference?  Risk for disorderRisk for disorder  Development of disorder (e.g., age of onset)Development of disorder (e.g., age of onset)  Phenomenology (symptoms) of disorderPhenomenology (symptoms) of disorder  Course of disorder over timeCourse of disorder over time  (2) How much difference does X make?(2) How much difference does X make?  (3) What is mechanism for X to influence disorder?(3) What is mechanism for X to influence disorder?
  16. 16. Classes of Biological FactorsClasses of Biological Factors  GeneticsGenetics  NeurochemistryNeurochemistry  Structural brain abnormalitiesStructural brain abnormalities  Gross anatomyGross anatomy  Microstructural changes/deficitsMicrostructural changes/deficits  Neuropsychological (functional brain)Neuropsychological (functional brain) abnormalitiesabnormalities
  17. 17. Designs for Studying FactorsDesigns for Studying Factors  Case-controlCase-control  Depressives vs. normals, compared on serum cortisolDepressives vs. normals, compared on serum cortisol  Schizophrenics vs. depressives, compared on dopamine receptor countsSchizophrenics vs. depressives, compared on dopamine receptor counts  Experimental psychopathology: heighten abnormality in disordered group vs.Experimental psychopathology: heighten abnormality in disordered group vs. controlscontrols  tryptophan depletion in depressivestryptophan depletion in depressives  frontal lobe-demanding tasks in schizophrenicsfrontal lobe-demanding tasks in schizophrenics  Family-genetic studiesFamily-genetic studies  nuclear family (no control)nuclear family (no control)  twin & twin-family (DZ twin = control), adoptiontwin & twin-family (DZ twin = control), adoption  Pedigree linkage (control provided by null hypothesis)Pedigree linkage (control provided by null hypothesis)  Animal models, where possibleAnimal models, where possible  Feeding & satiety---eating disordersFeeding & satiety---eating disorders  Drug-seeking---psychoactive substance use disordersDrug-seeking---psychoactive substance use disorders  Stereotyped behavior in monkey/rat---OCD?Stereotyped behavior in monkey/rat---OCD?
  18. 18. Types of Genetic StudiesTypes of Genetic Studies  Family studies: yield ambiguous resultsFamily studies: yield ambiguous results  liability = hypothetical (unobserved) trait, with genetic ANDliability = hypothetical (unobserved) trait, with genetic AND environmental causes, that controls risk for developing disorderenvironmental causes, that controls risk for developing disorder  Heritability = degree to which liability is influenced by additiveHeritability = degree to which liability is influenced by additive genetic factors in the populationgenetic factors in the population  How to estimate influence of genetic/environmental factors on aHow to estimate influence of genetic/environmental factors on a given liability:given liability:  Twin studiesTwin studies  Adoption studiesAdoption studies
  19. 19. Twin StudiesTwin Studies  Identical = Monozygotic MZ = 1 “zygote”Identical = Monozygotic MZ = 1 “zygote”  Paternal = Dizygotic DZ = 2 “Zygotes”Paternal = Dizygotic DZ = 2 “Zygotes”  Zygote = a fertilized egg (one set genes)Zygote = a fertilized egg (one set genes)  MZ share 100% genes; DZ share 50% (on average)MZ share 100% genes; DZ share 50% (on average)  Concordance is % of time one twin matches the other on trait or diagnosisConcordance is % of time one twin matches the other on trait or diagnosis  MZ/DZ correlations index similarity between twins and their co-twinsMZ/DZ correlations index similarity between twins and their co-twins  Compare MZ & DZ twin correlations:Compare MZ & DZ twin correlations:  hh22 = 2(r= 2(rMZMZ – r– rDZDZ))
  20. 20. Twin StudiesTwin Studies  Twin studies from 1920’s to todayTwin studies from 1920’s to today  EveryEvery study shows rstudy shows rMZMZ > r> rDZDZ  Average heritability about .42 (bipolar & Schiz. are higher,Average heritability about .42 (bipolar & Schiz. are higher, about .7)about .7)  Older studies (inpatients) give higher heritability than newerOlder studies (inpatients) give higher heritability than newer studies (more outpatients), e.g., hstudies (more outpatients), e.g., h22 = .7= .7  Severity predicts greater risk to co-twinSeverity predicts greater risk to co-twin
  21. 21. Adoption StudiesAdoption Studies  Adopted away offspringAdopted away offspring  Relatives of disordered adoptees, vs. those ofRelatives of disordered adoptees, vs. those of normal adopteesnormal adoptees  Cross-fostering (animal studies)Cross-fostering (animal studies)  Examine biological parents of disorderedExamine biological parents of disordered adoptees, vs. bio parents of normal adopteesadoptees, vs. bio parents of normal adoptees  Need adoptees reared by non-relatives, fromNeed adoptees reared by non-relatives, from early ageearly age
  22. 22. Heston adoption studyHeston adoption study  Design: adopted-away offspringDesign: adopted-away offspring  Schizophrenic mothersSchizophrenic mothers  Psychiatrically normal controlsPsychiatrically normal controls  Dependent variable: rate of schizophreniaDependent variable: rate of schizophrenia  Result: high rate in at-risk adoptees, zero inResult: high rate in at-risk adoptees, zero in controlscontrols  Criticism: diagnoses not blindCriticism: diagnoses not blind
  23. 23. Types of Neurological StudiesTypes of Neurological Studies  NeuroimagingNeuroimaging  CT, MRI (structural)CT, MRI (structural)  SPECT, PET, fMRI (functional)SPECT, PET, fMRI (functional)  Neuroanatomy (gross, microscopic)Neuroanatomy (gross, microscopic)  Neurotransmitter (receptor) studiesNeurotransmitter (receptor) studies  Neuropsychological (functional) studiesNeuropsychological (functional) studies
  24. 24. Findings in schizophreniaFindings in schizophrenia  Skull, brain (esp. frontal) of schizophrenics (&Skull, brain (esp. frontal) of schizophrenics (& bipolars?) smaller on average (CT, MRI)bipolars?) smaller on average (CT, MRI)  As skull forms around brain in utero, this is developmentalAs skull forms around brain in utero, this is developmental processprocess  Enlarged cerebral ventricles (& sulci?) in schizophreniaEnlarged cerebral ventricles (& sulci?) in schizophrenia —many replications (CT,MRI)—many replications (CT,MRI)  Esp. 4Esp. 4thth ventricleventricle  Not artifact of institutionalization or meds—present inNot artifact of institutionalization or meds—present in teenage schizophrenicsteenage schizophrenics  Brains of schizophrenics relatively more hypoactiveBrains of schizophrenics relatively more hypoactive frontally, when engaged in tasks calling on frontal lobefrontally, when engaged in tasks calling on frontal lobe functionfunction
  25. 25. Psychosocial Etiology ResearchPsychosocial Etiology Research  Does a psychosocial variable X affect a disorderDoes a psychosocial variable X affect a disorder Y?Y?  RiskRisk  Age of onset, presentation, courseAge of onset, presentation, course  How much difference does X make?How much difference does X make?  What’s the mechanism?What’s the mechanism?
  26. 26. Suitable DesignsSuitable Designs  Case-control comparisonCase-control comparison  RetrospectiveRetrospective (very common)(very common)  ProspectiveProspective (rarely used but better)(rarely used but better)  Longitudinal: developmental prospective studyLongitudinal: developmental prospective study  Behavior-genetic study designs (essentially neverBehavior-genetic study designs (essentially never used)used)
  27. 27. Problems with RetrospectiveProblems with Retrospective StudiesStudies  Sampling problemsSampling problems  How do they find cases & controls?How do they find cases & controls?  Who’s willing/likely to participate?Who’s willing/likely to participate?  Measurement problemsMeasurement problems  How do they measure factor X? (recall)How do they measure factor X? (recall)  Causal inference problemsCausal inference problems  Nature-nurture confoundsNature-nurture confounds  Other 3Other 3rdrd variable confounds (environmental)variable confounds (environmental)
  28. 28. Measurement ProblemsMeasurement Problems  Quasi-random recall errorsQuasi-random recall errors  Substantial error rate in recalling even major events in pastSubstantial error rate in recalling even major events in past year (e.g., medical hospital stays)year (e.g., medical hospital stays)  Recall biasesRecall biases  Over-recalling (GPA, possible false memories of sexualOver-recalling (GPA, possible false memories of sexual abuse)abuse)  Under-recalling (income)Under-recalling (income)  Reporting biasesReporting biases  Under-reporting sensitive topicsUnder-reporting sensitive topics  Potential over-reporting (e.g., borderline pts)Potential over-reporting (e.g., borderline pts)
  29. 29. Causal Inference ProblemsCausal Inference Problems  Nature-nurture confoundsNature-nurture confounds  ““Schizophrenogenic” mother, or schizophrenia-predisposingSchizophrenogenic” mother, or schizophrenia-predisposing genes?genes?  Criminality in the home:Criminality in the home:  direct cause of child’s criminal behavior?direct cause of child’s criminal behavior? vs.vs.  criminal parents pass on “criminal genes”?criminal parents pass on “criminal genes”?  Or are both genetic and environmental factors important, andOr are both genetic and environmental factors important, and to what extent?to what extent?
  30. 30. Causal Inference Problems,Causal Inference Problems, cont’dcont’d  One environmental factor as proxy (like “3One environmental factor as proxy (like “3rdrd variable” we discussed) for another. Examples:variable” we discussed) for another. Examples:  ““Poverty” may be proxy for a broad range ofPoverty” may be proxy for a broad range of associated factors (e.g., chaotic home,associated factors (e.g., chaotic home, malnutrition)malnutrition)  ““Abuse” may be proxy for neglect, or vice versaAbuse” may be proxy for neglect, or vice versa
  31. 31. Common Research TopicsCommon Research Topics  Childhood adversity variablesChildhood adversity variables  Chronic: poverty, single parent, ‘bad’ neighborhood,Chronic: poverty, single parent, ‘bad’ neighborhood, neglect, physical illnessneglect, physical illness  Episodic: abuse, death of a parent, divorce ofEpisodic: abuse, death of a parent, divorce of parentsparents  As causes of:As causes of:  Childhood maladjustment (acting out, poor schoolChildhood maladjustment (acting out, poor school performance, peer relations)performance, peer relations)  Psychopathology (e.g., ADHD, CD, depression)Psychopathology (e.g., ADHD, CD, depression)
  32. 32. Longitudinal-DevelopmentalLongitudinal-Developmental StudiesStudies  Find cohort at high riskFind cohort at high risk  Impoverished kids, abused/neglected kidsImpoverished kids, abused/neglected kids  Chronically ill kidsChronically ill kids  Suitable control group (what’s suitable?)Suitable control group (what’s suitable?)  Follow over time (to old age?)Follow over time (to old age?)  (Relatively) frequent follow-ups(Relatively) frequent follow-ups  Comprehensive assessments of domainsComprehensive assessments of domains  Challenge: keep track of families, keep in studyChallenge: keep track of families, keep in study
  33. 33. Ex. Life events in DepressionEx. Life events in Depression  Older studies indicated associationOlder studies indicated association  Causal ambiguity conceptCausal ambiguity concept  Could (premorbid) depression be causing adverseCould (premorbid) depression be causing adverse life events? (time order)life events? (time order)  Possible reporting biases (search for meaning)Possible reporting biases (search for meaning)  Nature/nature confoundNature/nature confound
  34. 34. Ex. Poverty and Mental IllnessEx. Poverty and Mental Illness  Older studies indicated association (e.g.,Older studies indicated association (e.g., schizophrenia)schizophrenia)  Causal inference problem: “downward drift” vs.Causal inference problem: “downward drift” vs. direct causationdirect causation  Best evidence (Kohn): 2/3 downward driftBest evidence (Kohn): 2/3 downward drift  Need prospective study to sort out causationNeed prospective study to sort out causation  ECA substudy on poverty: 2X increase in rates ofECA substudy on poverty: 2X increase in rates of essentially all diagnoses, associated with povertyessentially all diagnoses, associated with poverty

×