Risk Behaviors, Morbidity and Mortality

1. Risk Behaviors,
Morbidity and Mortality
Presenters:
• Peter Kreiner, PhD, Senior Scientist, Brandeis University
• Christopher Ringwalt, DrPH, MSW, Senior Scientist, Injury
Prevention Center, University of North Carolina - Chapel Hill
• Sharon Schiro, PhD, Associate Professor/Data Scientist,
University of North Carolina - Chapel Hill
PDMP Track
Moderator: John J. Dreyzehner, MD, MPH, FACOEM,
Commissioner, Tennessee Department of Health, and
Member, Rx and Heroin Summit National Advisory Board

2. Disclosures
• Peter Kreiner, PhD; Christopher Ringwalt, DrPH,
MSW; and Sharon Schiro, PhD, have disclosed no
relevant, real, or apparent personal or professional
financial relationships with proprietary entities that
produce healthcare goods and services.
• John J. Dreyzehner, MD, MPH, FACOEM – Ownership
interest: Starfish Health (spouse)

3. Disclosures
• All planners/managers hereby state that they or their
spouse/life partner do not have any financial
relationships or relationships to products or devices
with any commercial interest related to the content of
this activity of any amount during the past 12 months.
• The following planners/managers have the following to
disclose:
– John J. Dreyzehner, MD, MPH, FACOEM – Ownership
interest: Starfish Health (spouse)
– Robert DuPont – Employment: Bensinger, DuPont &
Associates-Prescription Drug Research Center

4. Learning Objectives
1. Identify indicators of risk behaviors by
prescribers.
2. Demonstrate the prescriber risk behavior
indicators used by the nation’s Prescriber
Behavior Surveillance System.
3. Explain how North Carolina shares its PDMP
data with multiple agencies to reduce morbidity
and mortality related to Rx drug abuse.
4. Provide accurate and appropriate counsel as
part of the treatment team.

5. A Validation Study of Prescriber Risk
Measures from the Prescription Behavior
Surveillance System
PDMP Track: Risk Behaviors, Morbidity and Mortality
March 29, 2016
Peter Kreiner, Ph.D.
PDMP Center of Excellence, Brandeis University

6. Disclosure Statement
• Peter Kreiner, PhD, has disclosed no relevant,
real or apparent personal or professional
financial relationships with proprietary
entities that produce health care goods and
services.

7. Learning Objectives
• Identify indicators of risk behaviors by
prescribers.
• Demonstrate the prescriber risk indicators used
by the Prescription Behavior Surveillance System.
• Describe a validation study of these indicators in
one state, using actions taken by the state
Medical Board.
• Provide accurate and appropriate counsel as part
of the treatment team.

8. Overview
• High-level summary of Prescription Behavior
Surveillance System (PBSS) project
• Validation study
– Analytic strategy
– Methods
– Results
– Conclusions/Limitations
– Next steps

9. The Prescription Behavior Surveillance System
(PBSS)
A longitudinal, multi-state database of de-identified PDMP
data, to serve as:
1. An early warning public health surveillance tool
2. An evaluation tool, in relation to state and local laws,
policies and initiatives
Info available at: http://www.pdmpexcellence.org/content/
prescription-behavior-surveillance-system-0
Publication: Paulozzi LJ, Strickler GK, Kreiner PW, Coris CM.
Controlled substance prescribing patterns – Prescription Behavior
Surveillance System, eight states, 2013. MMWR Surveillance
Summaries. Oct. 16, 2015; 64(9): 1-14.

10. PBSS Continued
• Began in FY2012 with support from CDC and FDA,
administered through BJA
• Guided by Oversight Committee:
– Federal partners: CDC, FDA, BJA, SAMHSA, ASPE
– State partners to date: CA, DE, FL, ID, KY, LA, ME, OH, TX,
VA, WA, WV
– Additional state partners in process
– No release of data or findings without Oversight
Committee approval

11. PBSS Measures
• Prescribing measures
– Rates of opioid, benzodiazepine, stimulant
prescriptions
• By quarter and year, by drug class, sex, and age group
• By quarter and year, by major opioid, benzodiazepine, and
stimulant drug category
• Patient risk indicators
– Average daily dosage of opioids (MMEs)
– Days of overlapping prescriptions
– Multiple provider episode rates
• By drug class, age group, and drug category

12. PBSS Measures Continued
• Prescriber risk indicators
– Prescriber percentile ranking, based on daily
prescribing volume
• By quarter, year, and drug class
– Average daily dosage for opioid patients (MMEs)
– Median distance in miles, patient to prescriber
– Percentage of patients with MPE
– Percentage of prescriptions by payment type
– Percentage of patients prescribed LA/ER opioids who
were opioid-naïve
• Pharmacy risk indicators
– Analogous to prescriber risk indicators

13. Prescriber Risk Indicators
• Prescribing volume by prescriber decile: Proportion
of total prescriptions accounted for by prescriber
10% groupings
• Average daily opioid dosage (MMEs) by prescriber
decile (volume)
• Distance patients travel to prescriber and
proportion of prescriber practice who meet MPE
threshold

14. 0
10
20
30
40
50
60
70
1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th
California, 2012: Proportion of Total Opioid,
Stimulant, and Benzodiazepine Prescriptions Written
by Prescriber Deciles
Percent of total opioid
prescriptions
Percent of total stimulant
prescriptions
Percent of total
benzodiazepine
prescriptions

15. 0
10
20
30
40
50
60
70
80
1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th
AveragedailydosageofpatientsinMMEs
Prescriber decile based on volume of opioid prescriptions
California 2012: Average Daily Dosage of Patients
by Prescriber Decile
Based on Volume of Opioid Prescriptions

16. 0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0 20 40 60 80 100 120 140
PercentageofprescriberpatientswithMPE
Average distance patients travel in miles
California 2012: Prescriber Deciles Based on Average
Distance Patients Travel, Compared with Percentage of
Prescriber Patients with an MPE
Percentage of patients
with MPE
Log. (Percentage of
patients with MPE)

17. Validation Studies
Purpose:
1. Examine frequency of prescribers highest on
prescriber risk indicators having actions taken
against them (Medical Board, DEA, law
enforcement)
– Vs. prescribers lower on these indicators
2. Develop predictive models of actions taken to
estimate relative risk of different prescriber
behaviors

18. Validation Studies: Analytic Strategy
• Prescriber outcomes
– Identify prescribers against whom actions have been taken
• By the state Medical Board/Board of Osteopathic
Medicine
• By the DEA
• By other law enforcement
– Categorize types of offense and types of action taken
– Examine/take into account prescriber license type and
physician specialty

19. Validation Studies: Analytic Strategy
• Logistic regressions of actions taken (2010-2014)
• Predictor variables
– Prescriber risk indicators
• In 2010; vs. action(s) taken in 2010-2014
• Examine factor structure of prescriber indicators
• Trajectory analysis: identify different groups/patterns
over time
– Measure of prescribing complexity?
• Pattern of drugs prescribed, in relation to peers
• Control variables
– Prescriber sex, specialty (categorical variable)

20. Medical Specialties Used in the Analysis:
Maine ProvidersMedical Specialty Frequency Share Share (exc. missing)
Anesthesiology 26 0.3% 0.5%
Dermatology 10 0.1% 0.2%
Ear, nose, & throat (ENT) 33 0.4% 0.6%
Emergency medicine 263 3.2% 4.8%
GP/FM/DO a 845 10.4% 15.4%
Internal Medicine 446 5.5% 8.1%
Obstetrics and Gynecology (OB/GYN) 135 1.7% 2.5%
Oncology b 65 0.8% 1.2%
Ophthalmology 48 0.6% 0.9%
Orthopedics c 132 1.6% 2.4%
Pain Medicine d 32 0.4% 0.6%
Pediatricians 198 2.4% 3.6%
Physical Medicine and Rehabilitation 52 0.6% 0.9%
Podiatrist 54 0.7% 1.0%
Psychiatry & Neurology 330 4.1% 6.0%
Radiology 35 0.4% 0.6%
Surgery 198 2.4% 3.6%
Dentist 489 6.0% 8.9%
Veterinarian 195 2.4% 3.6%
Othere 474 5.8% 8.6%
Missing f 2621 32.4%
Physician Assistants g 536 6.6% 9.8%
Non-physician prescriber h 886 10.9% 16.2%

21. Types of Maine Medical Board Actions
Action Brief Description
Revocation of License Physician’s license is terminated; individual can no longer practice
medicine within the state or territory.
Suspension of License Physician may not practice medicine for a specified period of time,
perhaps due to disciplinary investigation or until other state board
requirements are fulfilled.
Probation of License Physician’s license is monitored by a state board for a
specified period of time.
Probation of License with Condition(s) Physician must fulfill certain conditions to avoid further sanction by
the state board.
Examples: Subject to urine test/ substance abuse surveillance; require
to complete training/ course
Restricted or Conditional License Physician's ability to practice medicine is limited (e.g., loss of
prescribing privileges).
Denial of License or
Denial of License Renewal
Physician’s application for a medical license or renewal of a current
license is denied.
Fine (monetary) A monetary penalty against a physician.
Reprimand/ Warning Physician is issued a warning or letter of concern.
Voluntary Surrender or Withdrawal of
License
Physician voluntarily surrenders or withdraws medical license,
sometimes during the course of a disciplinary investigation.
Amendment or Removal of Adverse
Action
Amendment or Removal of adverse action
Reinstatement of License Reinstatement of License

22. Board Actions/Decisions and Individuals Disciplined
Maine, 2010 - 2014
Total 2010 2011 2012 2013 2014
n % n % n % n % n % n %
Number of Board Actions/Decisions
(an individual can receive more than one action a year) 199 100 36 18 34 17 41 21 42 21 46 23
Number of unique individuals disciplined:
Unique individuals sanctioned by year
(with replacement across years)
164 100 30 18 29 18 37 23 35 21 33 20
N individuals placed on probation or probation with
conditions 27 100 5 19 7 26 7 26 6 22 2 7
N individuals with a license suspension
18 100 3 17 4 22 5 28 2 11 4 22
N individuals whose license was revoked or voluntarily
surrendered license 24 100 6 25 4 17 7 29 4 17 3 13
N individuals whose license application was denied or
denied renewal of license 12 100 2 17 4 33 2 17 1 8 3 25
Unique individuals sanctioned 2010-2014
(without replacement)
119 100 30 25 25 21 23 19 23 19 18 15

23. Share of prescribers in ME PDMP dataset with criminal records (2002-
2011) and medical board disciplinary actions taken against them (2010-
2014), by specialty

24. Prescriber Indicators: Factor Structure
Factor 1: # of patients
# opioid prescriptions
# benzodiazepine prescriptions
# prescriptions written daily
# opioid prescriptions written daily
Factor 2: Average daily MME
% of patients with average daily MME > 100
Factor 3: # stimulant prescriptions
(-) % of prescriptions paid for by cash
Factor 4: % of patients meeting MPE threshold
Average distance travelled by patients

25. Conclusions
• Being in the top 1% of Maine prescribers on number of
patients issued a CS prescription, prescriptions per day,
opioid prescriptions per day, or average daily dosage of
opioids prescribed (MMEs) is associated with being
subject to a Medical Board action for inappropriate
prescribing and with prescription restrictions
• Being in the top 1% for MMEs per day is associated
with being subject to a severe action by the Board
• Being in the top 2% of prescribers for average distance
traveled by their patients is also associated with being
subject to a severe action by the Board

26. Study Limitations
• Findings may not be generalizable to other states: rural
state; relatively small number of Board actions taken
– We have validation studies in process with Ohio and
Washington
• Board actions reactive rather than proactive
• Unknown lead time before Board takes action
– We have follow-up study of Board actions to address this
question; also effects of Board actions and Board decision-
making process

27. Next Steps
• Examine Ringwalt measures of co-prescribing of
benzodiazepines and > 100 MME opioids, and
overlapping CS prescriptions
• Explore measure interactions (e.g., top 1% of
prescribers on volume and top X% on risk indicator)
• Examine prescribing profiles based on prescriber –
drug networks, in relation to specialty

28. Maine Prescribers Above 90th Percentile: Prescriber – Drug
Network with Medical Board and Criminal Justice Actions

29. Contact Information
Peter Kreiner, Ph.D.
Principal Investigator
PDMP Center of Excellence
Brandeis University
781-736-3945
pkreiner@brandeis.edu
www.pdmpexcellence.org

30. Reducing morbidity and mortality from
prescription drug abuse through
partnerships with NC’s PDMP
Sharon Schiro, Chris Ringwalt, Rachel Seymour, Scott Proescholdbell, Joseph
Hsu, David Henderson, Alex Asbun

31. Disclosure statement
• Sharon Schiro, PhD, has disclosed no relevant, real or
apparent personal or professional financial relationships
with proprietary entities that produce health care goods
and services.
• Chris Ringwalt, DrPH, has disclosed no relevant, real or
apparent personal or professional financial relationships
with proprietary entities that produce health care goods
and services.

32. Learning objectives
• Explain how North Carolina shares its PDMP data with
multiple agencies to reduce morbidity and mortality
related to Rx drug abuse.
• List other sources of data that augment the PDMP for the
purpose of addressing problematic prescribing
• Identify methodologies utilized to overcome technical and
policy-related hurdles.
• Provide accurate and appropriate counsel as part of the
treatment team.

33. Introduction
• Three projects
• Identifying prescribers who prescribe very high levels of controlled
substances
• Identifying prescribers with multiple patients with opioid-related
death
• Immediate feedback to prescribers on potentially high risk patients
through health care system electronic prescription records and PDMP
data
• Partners
• Injury andViolence Prevention Branch, NC Division of Public Health
• Injury Prevention Research Center (IPRC), University of North
Carolina-Chapel Hill
• Department of Surgery, University of North Carolina-Chapel Hill
• Carolinas Healthcare Department of Orthopedics
• NC Medical Board (NCMB)
• Drug Control Unit, Division of Mental Health, Developmental
Disabilities and Substance Abuse Services (DMH/DD/SAS)

34. Identifying prescribers who prescribe very
high levels of controlled substances

35. Identifying prescribers who prescribe very high
levels of controlled substances: Partners
• Partnership between DMH/DD/SAS, UNC IPRC, and UNC
Department of Surgery to develop algorithms
• Partnership with NC Medical Board for possible
investigation of identified prescribers

36. Identifying prescribers who prescribe very high
levels of controlled substances: Methodology
• Project goal: Develop and validate algorithms using PDMP (NC
CSRS) data to identify prescribers with unusual prescribing
patterns.
• Algorithms evaluated:
• Rates of prescriptions for daily dose > 100 MME
• High average daily MME*
• High total MME per prescription
• Prescription rates for opioids, benzodiazepines, stimulants*
• Rates of co-prescribed opioids (> 100 MME) and benzodiazepines*
• Temporally overlapping prescriptions*
• Long travel distance by patient to prescriber or pharmacy
• Multiple prescribers for controlled substances
• Multiple pharmacies for controlled substances
* Algorithms most closely associated with deaths from overdose

37. Identifying prescribers who prescribe very high
levels of controlled substances: Methodology
• NC Medical Board receives lists of prescribers who are in:
• Top 1% of those prescribing 100 milligrams of morphine
equivalents (“MME”) per patient per day
• OR
• Falls within the top 1% of those prescribing 100 MME’s per patient
per day in combination with any benzodiazepine and who are
within the top 1% of all controlled substance prescribers by
volume

38. Identifying prescribers who prescribe very high
levels of controlled substances: Next steps
• Identify additional data elements to be added to metrics
to improve sensitivity of metrics and reduce false
positives. Concern is time required for investigations and
impact of investigations on legitimate prescribers.
• Work with NCMB Advisory Group (2 Board members and 3
consultants) to develop objective criteria to decide
whether or not to open an investigation

39. Identifying prescribers with more than one
patient with an opioid-related death

40. Identifying prescribers with more than one
patient with an opioid-related death: Partners
• Partnership between NC Division of Mental Health,
Developmental Disabilities and Substance Abuse Services
and NC Injury andViolence Prevention Branch of the
Division of Public Health to develop algorithms
• Partnership with NC Medical Board for possible
investigation of identified prescribers

41. Identifying prescribers with more than one
patient with an opioid-related death: Methods
• Identified deaths attributed to unintentional or
undetermined controlled substance-related poisoning
usingVital Records Death Certificate file.
• Linked to CSRS data to identify those who prescribed the
decedent a controlled substance prescription within 60
days prior to death.
• Linked to CSRS andVital Records to determine whether
the prescriber(s) had any other patient deaths due to
substance-related poising during the prior 12 months
• Prescribers with two or more controlled-substance related
deaths in the past 12 months referred to NCMB

42. Identifying prescribers with more than one
patient with an opioid-related death: Results
• Validation results:
• 465 opioid overdose-related deaths identified in 2012
• 651 prescribers prescribed opioids to these patients within 30 days
of their death
• Match to prescribers who fall in top 1% of all CS prescribers and
100+ MME prescribers
• 30-46% of prescribers who fell in top 1% of metrics also had
prescribed an opioid to a patient within 30 days of their death
• Publication:The use of a prescription drug monitoring program to
develop algorithms to identify providers with unusual prescribing
practices for controlled substances. Journal of Primary Prevention
October 2015.

43. Identifying prescribers with more than one
patient with an opioid-related death: Next steps
• Identify additional data elements to be added to metrics
to improve sensitivity of metrics and reduce false
positives. Concern is time required for investigations and
impact of investigations on legitimate prescribers
• Working with NCMB Advisory Group (2 Board members
and 3 consultants) to develop objective criteria to decide
whether or not to open an investigation

44. Immediate feedback on high risk patients
through healthcare system prescription
records and PDMP data

45. Immediate feedback on high risk patients
through healthcare system prescription
records and PDMP data: Partners
• Partnership between NC Division of Mental Health,
Developmental Disabilities and Substance Abuse Services
and Carolinas Healthcare System (CHS).

46. Immediate feedback on high risk patients
through healthcare system prescription
records and PDMP data: Methods
• Integration of decision support in to electronic medical
record (EMR) using 5 criteria to trigger an alert.
• Existing prescription with >50% prescription duration remaining
• 2+ visits to ED with on-site administration of controlled substance
• 3+ controlled substance prescriptions in last 30 days
• Positive screen for blood alcohol, cocaine, or marijuana
• Previous presentation to ED for overdose

47. Immediate feedback on high risk patients
through healthcare system prescription
records and PDMP data: Results
• Two silent phases used to tune alert triggers to have
acceptable number of relevant alerts at point of care.
• Alerts generated for 5.9% of prescribing encounters
• Majority of prescribing encounters in outpatient setting.
• Inpatient discharges and ED/Urgent Care encounters have
disproportionately high rates of opioid and
benzodiazepine prescriptions.

48. Immediate feedback on high risk patients
through healthcare system prescription
records and PDMP data: Next steps
• Assess burden on prescribers.
• Evaluation of impact on prescribing patterns.
• Integration of PDMP data in to EMR and alerts

49. Navigating policy hurdles: Lessons learned
• Access to PDMP (CSRS) data: sensitive data requires high
level of security and confidentiality. Striking the balance
between legal protection, legal authority, and clinical
interest.
• Investigating prescribers is a big deal, so need to ensure
that metrics identify only prescribers truly warranting
investigation.
• Need to balance vigorous regulatory efforts with potential
chilling effect on other prescribers and the right for
adequate pain control for many patients

50. Navigating technical hurdles: Lessons
learned
• Storage and processing power requirements are
significant for analysis of large databases.
• Extensive cleaning of PDMP (CSRS) data required.
• Clear, complete data dictionaries needed for all datasets
to avoid errors due to assumptions on how data are
collected and defined.

51. Importance of the partnerships
• Utilization of research, evaluation, and data science skill
sets from an academic institution (UNC), a large
healthcare system (CHS), the NC Division of Public Health,
and the NC Division of Mental Health/Developmental
Disabilities/Substance Abuse Services
• Integration by research partners of topic-specific
knowledge and experience from DMH/DD/SAS and NCMB
staff, as well as CHS prescribers

52. Importance of the partnerships, continued
• Translation of research into practice through investigative
efforts of NCMB
• Because this is a multi-disciplinary, multi-jurisdictional
problem, increased coordination and communication are
essential. NC continues to identify ways in which key
stakeholders can collaborate and share knowledge to
address common problems.

53. Questions?
• Contact information:
• Sharon Schiro: Sharon_Schiro@med.unc.edu
• Chris Ringwalt: cringwal@email.unc.edu
• Alex Asbun: Alex.Asbun@dhhs.nc.gov
• David Henderson: David.Henderson@NCMedBoard.org
• Joseph Hsu: Joseph.Hsu@CarolinasHealthcare.org
• Scott Proescholdbell: Scott.Proescholdbell@dhhs.nc.gov
• Rachel Seymour:
Rachel.Seymour@CarolinasHealthcare.org

54. Risk Behaviors,
Morbidity and Mortality
Presenters:
• Peter Kreiner, PhD, Senior Scientist, Brandeis University
• Christopher Ringwalt, DrPH, MSW, Senior Scientist, Injury
Prevention Center, University of North Carolina - Chapel Hill
• Sharon Schiro, PhD, Associate Professor/Data Scientist,
University of North Carolina - Chapel Hill
PDMP Track
Moderator: John J. Dreyzehner, MD, MPH, FACOEM,
Commissioner, Tennessee Department of Health, and
Member, Rx and Heroin Summit National Advisory Board