Your SlideShare is downloading. ×
0
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
251109 rm-c.s.-assessing measurement quality in quantitative studies
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

251109 rm-c.s.-assessing measurement quality in quantitative studies

1,926

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,926
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
64
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Assessing Measurement quality in quantitative studies. Presented By: Mrs. Christy Simpson Professor, Maternity nursing
  • 2.
    • Definition:
    • Quantitative Data:
    • Information collected in a quantified (numeric ) form.
    • Quantitative Research :
    • The investigation of phenomena that lend themselves to precise measurement and quantification, often involving a rigorous and controlled design.
    • Quantitative Analysis:
    • Manipulation of numeric data through statistical procedures for the purpose of describing phenomena or assessing the magnitude and reliability of relationships among them.
  • 3.
    • Measurement:
    • Quantitative studies derive data through the measurement of variables.
    • Measurement involves the assignment of numbers to represent the amount of an attribute present in an object or person using a specified set of rules.
    • Principles of Measurement:
    • Classical measurement theory e.g Psychosocial constructs such as depression or social support.
    • Alternative measurement theory or Item Response theory e.g Cognitive constructs, Achievement or ability.
  • 4.
    • Advantages of measurement:
    • - Measurement is a language of communication.
    • - Numbers are less vague than words and therefore can communicate information more correctly. e.g Obese than 80Kg
  • 5.
    • Errors of Measurement:
    • Instruments that are not perfectly accurate yield measurements containing some error.
    • With in classical measurement theory, any observed (Obtained) score can be decomposed conceptually in to two parts :
    • a) An error component
    • b) A true component
    • Obtained score = true score ± error
  • 6.
    • Many factors contribute to errors of measurement:
    • Some are random or variable, others are systematic, which represent bias.
    • The most common influences on measurement error are:
    • Situational contaminants:
    • Scores can be affected by the conditions under which they are produced.e.g. A participant’s awareness of an observer’s presence (reactivity).
    • Other environmental factors are:Temperature , lighting etc.
  • 7.
    • 2.Transitory personal factors:
    • A person’s score can be influenced by such temporary personal states as fatique, hunger, anxiety or mood.
    • 3. Response set biases:
    • Relatively enduring characteristics of respondents can interfere with accurate measures. E.g social desirability, acquiescence
    • 4. Administration Variations:
    • Alterations in the method of collecting data from one person to the next.
  • 8.
    • Errors cont’d
    • 5. Instrument Clarity:
    • If the directions for obtaining measures are poorly understood, then scores may be affected by misunderstanding. E.g. Self - report instrument may be interpreted differently by different respondents.
    • 6. Item Sampling:
    • Errors can be introduced as a result of the sampling of items used in the measure.
    • 7. Instrument format: Technical characteristics of an instrument. E.g open ended questions yield different information than closed ones.
  • 9.
    • Criterion to assess the quality of quantitative instrument:
    • Reliability:
    • An instrument’s reliability is the consistency with which it measures the target attribute.
    • The less variation an instrument produces in repeated measurements, the higher its reliability.
    • The three key aspects of reliability:
    • Stability , Internal consistency and equivalence
  • 10.
    • Stability:
    • The stability of an instrument is the extent to which similar results are obtained on two separate occasions.
    • Assessments of an instrument’s stability involve procedures that evaluate test – retest reliability.
    • e.g. Administer the same measure to a sample twice and then compare the scores by computing a reliability coefficient, which is an index of the magnitude of the test’s reliability. Statistical analysis is correlation –coefficient.
  • 11.
    • How to read a correlation coefficient:
    • Two relationships:
    • Positive relationship:
    • The possible values for a correlation coefficient ranges from – 1.00 through .00 to + 1.00.
    • Positive relationship value should be 1
    • e.g: Anxiety scale - Administer the scale twice with 2 weeks duration
  • 12.
    • Negative Relationship:
    • When two variables are inversely related, increases in one variable are associated with decreases in the second variable.
    • The value of negative relationship is -1.
    • e.g: IQ is more in tall person.
    • The higher the coefficient, the more stable the measure.
    • The reliability coefficient is higher for short term retests than longterm retests
  • 13.
    • Internal consistency:
    • Scales designed to measure an attribute ideally are composed of items that measure that attribute and nothing else.
    • An instrument may be said to be internally consistent or homogeneous to the extent that its measure the same trait.
    • e.g Depression scale
    • The most widely used method for evaluating internal consistency is coefficient alpha or Cronbach’s alpha.Normal range of value is .00 and +1.00
  • 14.
    • Equivalence:
    • The degree to which two or more independent observers or coders agree about the scoring on an instrument.
    • Inter rater reliability can be assessed. When ratings are dichotomus, Following equation is used to calculate the proportion of agreements.
    • Number of agreement
    • Number of agreement + disagreements
    • The statistics used is Cohen’s Kappa which adjust for chance agreements. Multi rater Kappa when more than two raters.
  • 15.
    • Factors affecting reliability:
    • More items tapping the same concept should be added.
    • Items that have no discriminating power should be removed
  • 16.
    • Validity:
    • It is the degree to which an instrument measures what it is supposed to measure.
    • A measuring device that is unreliable cannot possibly be valid.
    • Validation efforts should be viewed as evidence gathering enterprises.
    • The more evidence gathered, using various methods to assess validity, the stronger the inference.
  • 17.
    • Types of validity:
    • Face validity:
    • Refers to whether the instrument looks as though it is measuring the appropriate.
    • Scale is established by consulting the experts and person with a same disease
    • 2. Content Validity:
    • Concerns the degree to which an instrument has an appropriate sample of items for the construct being measured and adequately covers the construct domain.
    • Content validity is relevant for both affective and cognitive measures
  • 18.
    • Content Valid cont’d
    • An content validity is necessarily based on judgement.No objective methods to ensure content validity.
    • Use a panel of substantive experts to evaluate and document the content validity of new instruments.Validation by minimum of three.
  • 19.
    • Calculate the Content Validity index,(CVI) Experts rate items on a 4 – point scale of relevance, the item(I) CVI is computed as the number of raters giving a rating of either 3 or 4 , divided by the number of experts.I-CVI of .80 is considered an acceptable value.
    • Scale CVI (S) CVI can be also done.
  • 20.
    • 3. Concurrent Validity:
    • Concurrent Validity refers to a measurement device’s ability to vary directly with a measure of the same construct or indirectly with a measure of an opposite construct. It allows you to show that your test is valid by comparing it with an already valid test.
  • 21.
    • 4. Criterion – Related validity:
    • Determines the relationship between an instrument and an external criterion.
    • The instrument is said to be valid if its scores correlate highly with scores on the criterion.
    • Two types of criterion related validity:
    • a) Predictive validity : Refers to the adequacy of an instrument in differentiating between people’s performance on some future criterion. e.g , High school grades for nursing school performance
  • 22.
    • b) Construct validity:
    • It is a key criterion for assessing the quality of a study.
    • sometimes also called factorial validity, has to do with the logic of items which comprise measures of social concepts.
    • The key construct validity questions:
    • - What is this instrument really measuring?
    • - Does it adequately measure the abstract concept of interest
  • 23.
    • Construct cont’d
    • A good construct has a theoretical basis which is translated through clear operational definitions involving measurable indicators.
    • It involves logical analysis and hypothesis test.
  • 24.
    • Methods of construct validity:
    • 1.Known groups Technique:
    • The instrument is administered to groups hypothesized to differ on the critical attribute because of some known characteristics.
    • E.g Anxiety among primi & Multi in labour.
    • 2. Hypothesized Relationship:
    • Testing hypothesized relationships, often on the basis of theory. E.g Smoking ---Cancer
  • 25.
    • 3. Convergent and Discriminant Validity:
    • An important construct validation tool is a procedure known as the Multitrait – multimethod matrix method which involves convergence and Discriminiability.
    • Convergence is evidence that different methods of measuring a construct yield similar results.e.g Self report,Observation etc.
    • Discriminiability is the ability to differentiate the construct from other similar constructs.
    • e.g. Psychological & Physical problems (HIV)
  • 26.
    • 4. Factor Analysis:
    • It is a method for identifying clusters of related variables – that is ,dimensions underlying a central construct.
    • It is a statistical procedure for identifying unitary clusters of items.
    • e,g Assess nursing students confidence in caring mentally ill patients.
  • 27.
    • Criteria for screening and diagnostic instruments:
    • Sensitivity and Specificity
    • Sensitivity is the instrument’s ability to identify a case correctly.(Its rate of yielding true positives)
    • True positives divided by positives, (Smokers who had high cotinine / all real smokers)
    • Specificity is the instrument’s ability to identify non cases correctly.(Its rate of yielding true negatives)Teenagers reported that they did not smoke,True negatives / all negatives.
  • 28. Urinary cotininie level Sensitivity = A/(A+C) = .50, Specificity = D/ (B+D) = .83 ( Percentage) Positive predictive value = A/(A+B) =.67 Negative predictive value =D/(C=D)=.71 Likelihood ratio –Positive (LR+) = Sensitivity/(1- Specificity) = 2.99 Likelihood ratio – Negative(LR_) = (1- sensitivity) / specificity =.60 LR Summarizes the relationship specificity and sensitivity in a single number. Self Reported smoking Positive Cotinine Negative Cotinine Total Yes , Smoked A (True positive) 20 B (False positive)10 A+B =30 No,Did not smoke C (False negative)20 D(True negative)50 C+D = 70 A+C=40 B+D=60 A+B+C+D 100
  • 29.
    • Other criteria to assess quantitative measures:
    • Efficiency
    • One aspect of efficiency is the number of items on the instrument. Long instruments tend to be more reliable than shorter ones.
    • Spearman – Brown formula , to estimate how reliable the scale would be with fewer items
    • There are other 6 criteria to check the quality and it is related to reliability and validity.
  • 30.
    • Six Criteria
    • Comprehensibility:
    • Subjects and researchers should be able to comprehend the behaviors required to secure accurate and valid measures.
    • 2. Precision:
    • An instrument should discriminate between people with different amounts of an attribute as precisely as possible.
    • 3. Speededness:
    • Researchers should allow adequate time to obtain complete measurements without rushing the measuring process.
  • 31.
    • Criteria cont’d
    • 4. Range:
    • The instrument should be capable of achieving a meaningful measure from the smallest expected value of the variable to the largest.
    • 5. Linearity:
    • A researcher normally strives to construct measures that are equally accurate and sensitive over the entire range of values.
    • 6. Reactivity:
    • Instrument should avoid affecting the attribute being measured.
  • 32.
    • Conclusion:
    • Quantitative Research studies are more common
    • Easy to do and analyze
    • Quality of the instrument must be assessed.
    • Reliability and validity are the main qualities.
    • Measure carefully to make the study findings more relevant to use it for nursing or midwifery practice.

×