These slides shed a light on the types of research study instruments used in pharmacoepidemiological research

1. GENERIC and SPECIFIC Instruments in
Pharmacoepidemiological Research

2.  Schron and Shumaker defined HRQOL as “a
multidimensional concept referring to a person’s
total well-being, including his or her psychological,
social, and physical health status.”
 Commonly measured domains of HRQoL include….
 Physical health and functioning
 Mental health and functioning
 Social and role functioning
 Perceptions of general well-being
 4 possible QoL outcomes associated with
pharmacotherapy:
(1) QoL is improved
(2) QoL is decreased,
(3) QoL is actively maintained, or
(4) QoL remains unaffected

3.  2 types of instruments: Generic and Specific
Generic instruments: Focus is on…
 Health profiles
 Preference-Based Measures (PBM)
Specific instruments:
 Population-based (e.g., geriatric, pediatric cases)
 Disease-specific (e.g., DM, HTN, BA etc.)
 Function-specific (e.g., Pulmonary function, Liver function
etc.)
 Condition or problem-specific (e.g., pain, allergies etc.)

4. GENERIC INSTRUMENTS (general HRQoL instruments )
 Designed to be used in…
• all diseases or conditions;
• across different medical interventions;
• and across a wide variety of populations;
 Egs., EuroQol group‟s EQ-5D; Nottingham Health Profile;
Quality of Well-Being Scale; Sickness Impact Profile; Health
Utilities Index;
 Based on: Health profiles and Preference-based measures
HEALTH PROFILES
 Provides multiple outcome scores in different dimensions or
domains of HRQoL.
 Adv.: These scores are useful to clinicians and/or researchers
attempting to measure differential effects of a condition or its
treatment in various QoL sections.

5. Medical Outcomes Study 36-Item Short-Form
Health Survey (SF-36)
 Includes nine health concepts or domains or scales
 Can be self-administered or administered by a trained
interviewer (face-to-face or via telephone).
 Aggregates the items into Physical Component Scores (PCS)
and Mental Component Scores (MCS)
 Advantages.:
• It is brief (it takes about 5 –10 minutes to complete),
• Its reliability and validity have been documented in many clinical
situations and disease states.
• SF-12: Is an abbreviated version of the SF-36 containing only 12
items. But, the scale scores and mental and physical component
summary scores derived from the SF-12 are based on fewer items
and fewer defined levels of health and, as a result, are estimated
with less precision and less reliability.

7. PREFERENCE-BASED MEASURES
 This approach combines the measurement of an individual‟s
current health status with relative desirability (or
preference) for a better health state, based on his / her
disease condition(s).
 HRQOL assessed by preference-based measures is a single
overall index score on a scale of 1.0 (full health) and 0.0
(dead).
 Health states considered worse than dead are reflected
by negative numbers on the scale.
 These measures are very useful in pharmacoeconomic research
analyses focusing on QALYs (Quality Adjusted Life Years).

8. Extra info. on PBM
 Preference-Based Measures (PBM) or Generic PBM are
increasingly used in health economic evaluations to calculate
Quality-Adjusted Life Years (QALYs).
 Such measures usually comprise a number of domains (or
descriptive set) that patients can use to describe various aspects
of their health (e.g. limitations in daily activities and mobility,
pain and discomfort).
 These patient-reported values (profile scores) are then
converted to an index score using a selected algorithm
(sometimes country-specific).
 These algorithms are based on surveying the general public’s
preferences for different combinations of health states, which is
why these measures are referred to as ‘preference-based’.

9. Extra info. on PBM (contd.’)
 The index scores (also called ‘utilities’) usually range between 0
to 1.
 ‘1’ indicates a valuation of ‘perfect health’;
 ‘0’ refers to valuation of ‘death’.
 Values < ‘0’ may be possible, representing health states
perceived to be worse than death.
 Examples of PBMs: EQ-5D, SF-6D, and the Health Utilities Index.
 The EQ-5D is NICE’s (National Institute for health and Care
Excellence) preferred instrument for cost-utility evaluations in
healthcare technology assessments.

10. SPECIFIC INSTRUMENTS (Targeted instruments)
 Provide greater detail concerning particular outcomes, in terms
of functioning and wellbeing, uniquely associated with a
condition and/or its treatment.
 E.g., Asthma Quality of Life Questionnaire (AQLQ)
• Self- or interviewer-administered;
• 4-domains, 32-item instrument developed to assess the impact of
asthma on patients‟ everyday functioning and well-being.
• 4-domains: Symptoms (11 items); Activity Limitation (12 items);
Emotional Function (5 items);and Environmental Exposure (4
items)
• Scaling: 7-point Likert scale (7 = not impaired at all - 1 = severely
impaired)

11.  Results from research in which the AQLQ was used have
appeared in promotional materials for Salmeterol inhaler
(GlaxoSmithKline). This was one of the first times a
pharmaceutical firm has promoted a product based on data
from trials involving QoL as a primary outcome measure.
 Mostly, disease- or condition-specific instruments can be more
sensitive than a generic measure to particular changes in
HRQOL secondary to the disease or its treatment.

12.  A concern regarding the use of only specific instruments is that
by focusing on the specific impact, the general or overall impact
on functioning and well-being may be overlooked.
 In studies involving pharmacotherapy, the use of both a
generic and a specific instrument is the best approach.
• Psychometrics: the measurement of psychological domains, such as
QoL.
• Conceptual model: the rationale for, and description of the concepts
that a study instrument is intended to assess, and the
interrelationships of those concepts.

13. Conceptual Models
 A conceptual framework can guide research by providing a
visual representation of theoretical constructs (inclusive
of the variables) of interest.
 Theoretical construct refers to the theory of the concept
being studied or analyzed or evaluated or studied.
 Designing a conceptual model begins with conducting a
thorough review of the literature (Search peer-reviewed journal
articles, books/monographs, conference papers, and other
relevant references).
 Next, organize the literature around the topic. Begin to develop
research questions or objectives based on theory.
 “Theories present a systematic view of phenomena by
specifying relations among variables using a set of interrelated
constructs/variables, definitions and propositions (Kerlinger,
1979).”

16. RELIABILITY
 It is the extent to which the instrument gives consistent results.
 Purpose of evaluating the reliability of a QOL instrument is to
estimate how much of the variation in a score is real as
opposed to random.
• E.g.1, Reliable cars (cars that start every time we turn on the
ignition);
• E.g. 2, : Imagine a kitchen scale. If you weigh five kg of potatoes in the
morning, and the scale is reliable, the same scale should register five
kg for the potatoes an hour or any time later (unless, of course, you
peeled and cooked them).
• E.g.3: Reliable, consistent instruments (data collection formats,
survey forms) should be reliable – it should not make any difference
whether a respondent takes the assessment in the morning or
afternoon; one day or the next.

17. RELIABILITY ASSESSMENT METHODS
 3 most common reliability assessment methods:
• Internal consistency
• Test-retest reliability
• Parallel (Alternate) form reliability

18. Internal consistency
 Assessment of the performance of „items‟ within a scale
(domain).
 Whether all the items in a domain are relevant to that
particular domain.
 Is commonly measured using Cronbach‟s alpha coefficient.
 Alpha coefficients values…
• > 0.90 (for making comparisons between individuals)
• > 0.70 (for comparisons between groups).

19. Test-retest reliability
 Refers to the relationship between scores obtained from the
same instrument on two or more separate occasions when all
important conditions remain relatively unchanged.
 However, QOL is not assumed to be constant over the course of
time.
 Most clinical studies attempt to assess how QOL changes.
 Test-retest reliability estimates may have limited value in
evaluating measures that are designed to assess a dynamic
(changing) process.
 Reliability is necessary but not sufficient for valid measurement.

20. Parallel form reliability
 Also known as „Alternate form reliability‟;
 Is a measure of reliability obtained by administering different
versions of an assessment tool (both versions must contain
items that probe or examine the same construct, skill,
knowledge base, etc.) to the same group of individuals.
 The scores from the two versions are then correlated in order
to evaluate the results‟ consistency of both the versions.
 E.g.,: Designing a study tool to evaluate students‟ knowledge
of Bronchial asthma.
• To evaluate the reliability, create a large set of items that all
pertain to every aspect of bronchial asthma and then
randomly divide the questions (equal weightage for each
domain) into two sets. The correlation between the scores
in the two parallel versions of the study instrument is then
performed.

21. Inter-rater reliability (inter-observer reliability)
 It is considered when any aspect of the measurement process
(scoring of an instrument) involves human judgment (degree of
subjectivity will arise).
 It is a measure of reliability used to assess the degree to which
different judges or raters agree in their assessment decisions.
 Inter-rater reliability is useful because human observers will not
necessarily interpret answers the same way.
 Raters may disagree as to how well certain responses or
material demonstrate knowledge of the construct or skill being
assessed.
 E.g., Inter-rater reliability might be employed when different
judges are evaluating whether the course syllabus of
B.pharmacy programme meets certain standards. Inter-rater
reliability is especially useful when judgments can be
considered relatively subjective.

22. VALIDITY
 Estimation of the extent to which the instrument is measuring
what it is supposed to be measuring.
 Even if a test is reliable, it may not provide a valid measure.
• E.g., Imagine a weighing scale that consistently tells you that you
weigh 60 kg. The reliability (consistency) of this scale is very good, but
it may not be accurate (valid) [because you actually weigh 50 kg].
• The validity inferred from the study instruments is essential (even
more crucial than the reliability).
 If a test is valid, it is almost always reliable.
 3 types of validities: Criterion, Content, and Construct

23. Criterion validity
• The extent to which the scores (data) is in agreement with an
external criterion (or the outcome).
• 2 types: Concurrent and Predictive
• Concurrent validity is demonstrated when a test correlates
well with a measure (criterion) at the same time.
• E.g.: This is a common method of developing validity evidence
for employment tests. A test is administered to employees, then
a rating of these employees' job performance is also obtained
(independently of the test) by the supervisor rating, to assess if
the scores obtained on the test and the performance rating by
the supervisor for each employee correlates.

24. • Predictive validity involves testing a group of subjects for a
certain construct, and then comparing them with results
obtained at some point in the future.
• Tests are administered to job applicants, and then after those
individuals work in the job for a year, their test scores are
correlated with their first year job performance scores.
Content validity
• Refers to how adequately the questions/items capture the
relevant aspects of the domain or concept being measured.
• Whether questions in a particular section are valid and relevant
to that section.

25. Construct validity
• Refers to the appropriateness of the inferences made based on
the test scores.
• This validity explores whether the „test‟ actually measures the
intended construct.
• The „construct‟ here refers to the hypothetical concept which
the researcher is actually investigating.
• E.g.: As a researcher, you might investigate whether an
interventional programme (counselling) increases respondents‟
confidence levels. Construct validity is a measure of whether
your research actually measures the increase in confidence
levels of the respondents (which might be quite abstract).

26. Face validity
 Refers to the perception of the potential respondents to the
appropriateness of the study instrument in assessing the issue
of interest.
 It is important as it‟s a simple first step to measure the overall
validity of a test or technique.
 It‟s a relatively intuitive, quick, and easy way to start checking
whether a new measure seems useful at first glance.
 Good face validity means that anyone who reviews your study
instrument says that it seems to be measuring what it‟s
supposed to.
 With poor face validity, someone reviewing your study
instrument may be left confused about what you‟re measuring
and why you‟re using this method.

27. Face validity
 An instrument with good face validity is characterized by:
 Clearly relevant for what it‟s measuring
 Appropriate for the participants
 Adequate for its purpose

30. References:
 Textbook of Pharmacoepidemiology; 2007 ed.
Editors: Brian L. Strom and Stephen E. Kimmel.
Publishers: John Wiley & Sons Ltd. The Atrium,
Southern Gate, Chichester, West Sussex PO19 8SQ,
England(England) ; ISBN 978-0-470-02925-1
 Understanding Pharmacoepidemiology; 1st Edition.
Eds. Yang Y, West-Strum D. McGraw Hill, New York,
2011.
 Modern Epidemiology; 3rd Edition. Eds. Rothman
KJ, Greenland S, Lash TL. Lippincott Williams &
Wilkins, Philadelphia, 2008.
 https://www.scribbr.com/methodology/face-
validity/

31. THE END