This presentation provides an overview of risk minimisation activities for medicines and the tools or activities that may be used in a risk management plan.

1. Risk minimisation activities associated with Risk Management Plans
Evaluating effectiveness of risk minimisation activities
Dr Jane Cook
Branch Head
Post-market Surveillance Branch
Monitoring and Compliance Division, TGA
ARCS Scientific Congress 2015
6 May 2015

2. What are risk minimisation activities?
• Initiatives that attempt to positively influence patients or health care professional
behaviours, and through these changes lead to improved patient outcomes
• Focuses on minimising harms
• Need to plan the level of change or achievement of outcome indicators that the
intervention was successful.
• A protocol for how the evaluation will occur needs to be developed and this
should specify the metrics and thresholds by which the programme success will
be measured and audited
• Should be supplied with the Risk Management Plan (RMP)
Risk minimisation activities associated with
Risk Management Plans
1

3. Types of risk minimisation activities
Focus on:
• Risk prevention – identifying those at particular risk through testing and not
prescribing, adjusting dose etc.
• Risk mitigation – usually relies on monitoring during therapy to identify increase
risk and allow intervention
• Neither preventable or able to be mitigated – may be acceptable depending on
risk-benefit profile
Risk minimisation activities associated with
Risk Management Plans
2

4. Tool selection
Will depend on the following factors:
• Level of risk
• Objectives of the risk minimisation activity
• The target or targets of the activity
• The characteristics of the prescribers (generalists versus specialists)
• Characteristic of potential patients – vulnerability access
• Potential variability in standards of care across regions – activity may already be embedded as
part of patient selection
A comprehensive strategy may require a range of interventions
Risk minimisation activities associated with
Risk Management Plans
3

5. Types of risk minimisation activities
• Communication:
- DHCP letters/information
- Prescribing dispensing guidelines
- Patient brochures, alert cards
- Education
• Restricted access:
- Registration programmes for patients
- Certification programmes for prescribers/dispensers
- Limited pack size, repeats, dosage forms
- Dispensing linked to test confirmation
- Certain types of prescribers
- Limited to patient specific group – limited pharmacogenomic profile
• Controlled regulatory framework - scheduling
Risk minimisation activities associated with
Risk Management Plans
4

6. Risk Minimisation Programme
• You have identified the risks to be mitigated
• Selected the tools (activities) to be used in the programme
• You now need to ensure the activities will meet their objectives and be successful
in mitigating the identified risks
• You now need to develop an evaluation framework or proto
Risk minimisation activities associated with
Risk Management Plans
5

7. Areas to consider when developing a framework
Area Description
Structure  Reason for and description of the interventions and tools and their objectives
Design  Key aspects, including any comparison, and duration and measures of success
Process
indicators
 Those to address adequacy of the content
 Those to assess adequacy of coverage, utilisation and maintenance
Outcome
indicators
 Knowledge and awareness, attitudes, actual behaviours – depend on nature of activity
 Those related to patient outcomes such as avoidance of C/I drugs, morbidity or mortality
measures
Analyses  How measure of performance will be undertaken against pre-identified measures of
success
 When will data be collected
 What type of analyses will be performed
Risk minimisation activities associated with
Risk Management Plans
6

8. Programme evaluation
• Need to measure effectiveness of overall programme as well as each individual
component
• Indicators selected to evaluate the performance of the programme need to be:
– relevant
– well-defined
– sensitive
– reliable
– evidence-based
– objective
– tailored to the programme
Risk minimisation activities associated with
Risk Management Plans
7

9. Evaluation
• Comprehensive evaluation of overall effectiveness should involve measuring
performance in several aspects or domains
• One methodology is the RE-AIM framework:
– Reach
– Efficacy/effectiveness
– Adoption
– Implementation
– Maintenance
Risk minimisation activities associated with
Risk Management Plans
8

10. RE-AIM
Dimensions Definition
Reach Number and representativeness of participants selected for inclusion in
evaluation the protocol
Efficacy/effectiveness What are the changes expected – behavioural, patient outcome have these
any negative consequences?
Adoption Number and representativeness of those that agree to participate – do these
continue to be representative – bias?
Implementation The degree to which the intervention has been delivered as intended (could
also consider cost)
Maintenance The extent to which the intervention has been delivered overtime as
intended
Risk minimisation activities associated with
Risk Management Plans
9

11. How did the intervention succeed in the
‘real world’
• Implementation fidelity
• Poor design or poor implementation?
• Good outcomes could they be better?
Risk minimisation activities associated with
Risk Management Plans
10

12. Evaluating implementation fidelity
Area Description Examples of assessment methods
Exposure Were all aspects delivered to
the target recipients?
Survey, review administrative records of delivery of
elements to participants. Samples need to be
representative
Content Was what delivered the same
as originally designed? Was
the interpretation of the
materials as expected? I.e.
was messaging ‘fit for
purpose’?
Focus groups
Frequency Was it delivered consistently at
the frequency in the initial
activities plan?
Stakeholder survey, patient clinical reviews, drug utilisation
study
Duration Was it delivered consistently
throughout the activity period
Sampling as above
11

13. Evaluating effectiveness of the risk minimisation programme
• Selecting appropriate outcome indicators
– Specific
– Measurable
– Time-bound
• Consistent with overall aim of the risk minimisation programme
• Meaningful targets from both clinical and public health perspective
• Precisely defined – hepatotoxicity versus liver failure vs hepatitis, contraindication
• Will influence sample size
• Consideration of surrogate or composite endpoints
Risk minimisation activities associated with
Risk Management Plans
12

14. Characteristics of suitable outcome measures
Characteristic Description
Preventable or mitigatable
Specific Clearly defined and requires little judgement
Easy to diagnose/detect Easy to identify and confirm
Rich in information Consider periodic rather than one off sampling
Responsive Is sensitive the proposed activity
Reliable Precise, reproducible, unlikely to be variable on repeated measures
Internally validity Linked to use
External validity Generalizable to general population
Clinical relevance Likely to influence treatment choices
Practical Easily implementable and low cost
13

15. Designs
• Comparator – can be difficult where has been required as part of registration
• Staggered roll out
• Before and after comparison if available
• Trend analysis through interrupted time series (periodic sampling) and use of
other existing databases e.g. health claims data
• Should confirm persistence and maintenance of the programme impact over time
Risk minimisation activities associated with
Risk Management Plans
14

16. Other considerations
• When would evaluation cease?
• When could the risk minimisation activity cease?
Risk minimisation activities associated with
Risk Management Plans
15

17. Summary
• Life-cycle approach
• Evaluation of the parts and the whole of the risk minimisation programme
• Should be simple, pragmatic and user friendly to prevent undue clinician and
patient workload
• Measures should be clearly defined, easy to use, reproducible and practical
• Risk management programme should aim to in the real world ensure that ‘the
right prescriber provides the right medicine to the right patient, at the right dose
and right time’
Risk minimisation activities associated with
Risk Management Plans
16