This document discusses patient-reported outcomes (PROs) and how to develop a valid PRO instrument. It defines PROs as reports coming directly from patients about their health without interpretation. PROs are important for evaluating treatment outcomes from the patient's perspective. The document outlines the steps to develop a PRO instrument, including defining the concept, creating questions, testing validity and reliability. Validity ensures the instrument accurately measures the concept, while reliability confirms it produces consistent results. Adhering to standards helps PRO instruments gain regulatory approval.

1. Copyright © 2020 pubrica. All rights reserved 1
How to Develop and Evaluate a PRO Instrument?
Dr. Nancy Agens, Head,
Technical Operations, Pubrica
sales@pubrica.com
I. WHAT IS PRO?
PRO or Patient-reported outcome
is a relatively novel method to evaluate the
outcome of any treatment by clinician or
researcher (in clinical trial) purely based
on the inputs got from the patient alone.
The United States Food and Drug
Administration (US-FDA) defines PRO as
‘any report of the patient’s health
condition that comes directly from the
patient, without interpretation of the
patient’s response by a clinician or anyone
else.’
II. ASK THE DOCTOR, WHY
ASK THE PATIENT?
Objective data regarding outcome
can be given by the treating physician
(cure/survival/disability limitation), by
investigations (return of deranged
biochemistry to normal) and by the
caregiver (return to functional normality).
These are Observer-reported outcomes
(OROs) and medicine has long relied on
these for evidence-based management.
Any illness followed by its
treatment weighs physically, mentally,
emotionally, economically on the patient.
Patients perceive the outcome of the
treatment in comparison to their
expectations based on prior knowledge.
Patient response and compliance to
treatment are intricately linked to the
ultimate treatment outcome.
Since long, clinicians have
recorded patient symptoms in the daily
clinical notes and modified their treatment
accordingly. PRO goes beyond just
recording symptoms. Central to this is the
concept of Health-related Quality of Life
(HRQOL), a multi-dimensional approach
that records patient’s perceived effect of
the illness and treatment on quality of life
in his/her physical, mental, emotional,
economical, and social space. By
standardizing the questions and
quantitating them as far as possible, data
can be analysed, interpreted, and
compared. PRO data is used to inform and
guide patient-centered care, to assess risk
and benefit from treatment, to decide on
treatment protocols and to guide health
policy makers.
III. WHAT INFORMATION DO I
WANT FROM THE PATIENT? –
THE PRO INSTRUMENT
The outcome of interest that we
want to study is the PRO Concept, while
the PRO instrument is the means to collect
the data related to the PRO concept, in the
form of questionnaire and the information
and documentation that supports its use.
In other words, PRO instrument is
a questionnaire that compiles data related
to the PRO concept of interest which can
be summarized as a score.
Content validity is the extent to
which an instrument measures the concept
of interest that is desired from both the
developer’s and user’s perspective.
IV. DESIGNING A VALID PRO
INSTRUMENT
Steps involved are: -

2. Copyright © 2020 pubrica. All rights reserved 2
1. Defining the concept – concept should
be clearly defined based on review and
literature gap analysis of existing PRO
instruments as well as taking inputs
from scientific research services,
domain specialists, developers and
most importantly the patients or the
users and should focus on the core
characteristics.
2. Conceptual Framework – Breakdown
the concept to sub-concepts and finally
to items or the questions that will be
included in the questionnaire. Each
item should be crisp and clear with
focus on only one aspect. It should not
unnecessarily cause strain to the
patient in recalling.
3. End-point Model– Endpoint is the final
measured outcome of the treatment
that will be compared, and an endpoint
model summarises all the endpoint
based on OROs and PROs.
4. Review of Preliminary Instrument –
Literacy review is done to understand
the proportion of people who may or
may not have the literacy level to take
the questionnaire. Translation review is
done to ensure conceptual equivalence
when translated to another language.
Cognitive interview is the process
where the user is made to take the
questionnaire by thinking aloud to
understand the approach of the user to
the questions and that it follows the
intention of the item. This is done until
saturation is reached on the
information received for reviewing the
instrument.
5. Reliability Testing – Measure to
estimate reproducibility and consistent
true estimation of outcomes under

3. Copyright © 2020 pubrica. All rights reserved 3
consideration. It is further described
below.
6. Validation –
It is an ongoing never-
ending process of improving the
instrument by repeated feedback
mechanism.
V. EVALUATING PRO
INSTRUMENT
As clinical validation data
accumulates, a PRO instrument undergoes
review and revision to make them robust.
It includes the evaluating for the
following: -
1. Validity– is the extent to which the
scores and their interpretations,
supported by theoretical evidence,
correlate with the proposed use of the
instrument. It needs to be tested for the
entire range of the proposed
application.
a. Content Validity- It is a measure of
how much the instrument measures the
concept of interest. US-FDA proposes
using qualitative and quantitative
methods.
 Qualitative estimation – include
items involved, their recall period,
response options, scoring pattern,
method of conducting the
questionnaire etc.
 Quantitative estimation – e.g.
 Item response theory (IRT) – A
powerful tool to assess whether
the items used measure the
entire spectrum or continuum
of the patient responses.
 Rasch analysis - Only IRT tool
to use the total of all the items
in the PRO instrument making
it a simple model.
b. Construct Validity- It is based on
comparison with convergent and
divergent groups with known validity
and showing that the instrument under
study conforms to similar
interpretations.
c. Criterion Validity- Comparison with
known criteria or known gold standard
for this concept.
2. Reliability-
a. Reproducibility or test-retest
reliability- It is assessed under
same conditions in same user in
a close span of time to assess
any variations in recording
results that can be attributed to
the instrument. It is calculated
by intra-class correlation
coefficient (ICC).
b. Internal consistency- The items
should deliver same scores in
similar situations consistently.
It is determined by Cronbach’s
alpha or coefficient of
reliability.
c. Floor and ceiling effect – i.e.
the instrument’s ability to
measure extremes of construct
helps choose the proper
instrument for the study.
d. Inter-interviewer reliability-
there should not be significant
difference in scores when
interviewed by two different
interviewers.
e. Variability –
Instrument should have ability
to detect changes when
outcomes are different, else it is
no good.
Adhering to good practices while
designing and validating a PRO instrument
is primary for it to be effective and receive
regulatory approvals. Readiness to
continuously revisit and revise items is
needed to do better justice to the concept
of interest in the long run.
REFERENCES:
1. FDA Guidance for Industry. Patient-Reported
Outcome Measures: Use in Medical Product
Development to Support Labeling Claims. 2009.
Available at:
http://www.fda.gov/downloads/Drugs/Guidances/U
CM193282.pdf.
2. Patrick DL, Burke LB, Gwaltney CJ, et al. Content
validity--establishing and reporting the evidence in

4. Copyright © 2020 pubrica. All rights reserved 3
newly developed patient-reported outcomes (PRO)
instruments for medical product evaluation: ISPOR
PRO good research practices task force report: part
1--eliciting concepts for a new PRO
instrument. Value Health. 2011;14(8):967‐977.
doi:10.1016/j.jval.2011.06.014.
3. Rothrock, N & Kaiser, Karen & Cella, David.
(2011). Developing a Valid Patient-Reported
Outcome Measure. Clinical pharmacology and
therapeutics. 90. 737-42. 10.1038/clpt.2011.195.
4. Deshpande, P. R., Rajan, S., Sudeepthi, B. L., &
Abdul Nazir, C. P. (2011). Patient-reported
outcomes: A new era in clinical
research. Perspectives in clinical research, 2(4),
137–144. https://doi.org/10.4103/2229-3485.86879