Measurement II<br />What did we catch?<br />Was it the same as the others?<br />How do we know we caught it?<br />
Properties of Instruments<br />Reliability<br />The stability and consistency of a measure<br />Validity<br />How we know that our measure captures what we're interested in<br />
AN INSTRUMENT MUST HAVE RELIABILITY BEFORE IT CAN HAVE VALIDITY!<br />
Reliability<br />The stability and consistency of an instrument<br />Stability<br />Similar results over time<br />Consistency<br />Test items will be consistent with each other<br />Items will be consistent with the overall score on the test<br />
Stability<br />Measured using test-retest reliability checks<br />A participant completes the test at one point in time<br />After a period of time elapses they complete the test again<br />Usually at least 6 weeks<br />Assessed using correlation coefficients<br />Pearson r<br />Spearman <br />
Consistency<br />Internal consistency<br />The items all measure the same core construct<br />Measured using the Cronbach's Alpha Coefficient <br />A form of multiple correlation<br />Measures how much the individual items in a test or scale have in common statistically<br />
First Question<br />The Juvenile Court Administrator wants to evaluate a new assessment tool to find out if you can tell which kids, once released from Detention, will recidivate or not.<br />What is the first thing that must be done to solve your groups research problem?<br />How do you propose to do it?<br />
Validity<br />How we know that our instrument captures what we're interested in<br />Concept translation<br />How well our instrument translates a concept into a variable<br />Validity is reported as correlations of varying complexity<br />
Face Validity<br />How well an instrument apparently accomplishes concept translation<br />How completely an instrument samples all of the components of a concept<br />Panels of experts<br />In the domain being measured<br />Who have experience with the target<br />
Convergent / Concurrent Validity<br />How well an instrument relates to other measures of the same or similar concepts<br />Compare an instrument with:<br />Other existing instruments<br />Expert opinions<br />Behavioral observations<br />Assessed when the measure is given to subjects<br />
Discriminant Validity<br />How well an instrument discriminates between the presence or absence of concept components<br />An instrument can discriminate between known groups, assessed when the measure is given to subjects<br />
Predictive Validity<br />How well an instrument predicts future events or behavior<br />Useful concepts will predict phenomena<br />Predictive validity assessment - examines how we know that our measure captures what we are interested in<br />Compare the instrument with other predictive factors<br />Test prediction of the measure alone<br />Assessed longitudinally, after the instrument is given to subjects<br />
Second Question<br />The DCFS Area Regional Director wants to be able to tell how many future CPS referrals families are likely to have after their first CPS referral. He devises a questionnaire for the child’s mother to complete.<br />What type of validity is involved in your research problem?<br />What data might you use to address the validity question?<br />
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.