Dr. Nabarun Dasgupta presents "Using Social Media to Support Safety Surveillance Efforts" at M-CERSI: University of Maryland Center of Excellence in Regulatory Science and Innovation

1. NabarunDasgupta, MPH, PhD
Epidemico, Inc.
September 11, 2015– M-CERSIWorkshop
Using Social Media to Support
Safety Surveillance Efforts
Slides available now
@epidemico

2. R E A L - W O R L D E X P E R I E N C E
00
Disclosure
13
PatientBenefits
06
SupplyChain
16
DEMO
10
Post-marketing
01
Data
Taming
09
Clinical Trials
11
Space+ Time
14
Limitations
20
Minutes

3. 01. USA
A public-privatepartnershipenteringYear 2.
TheWEB-RADR projectis supportedbythe
InnovativeMedicinesInitiativeJoint
Undertaking(IMI JU)undergrant agreementn°
115632,resourcesof which arecomposedof
financialcontributionfrom theEuropean
Union'sSeventhFrameworkProgramme
(FP7/2007-2013)andEFPIAcompanies’in kind
contribution.
02. EUROPE
GSK hasbeena development
partnertoextendthetechnology
platformtomeetthestandardsof
regulatedsoftwareforthelastyear
andhalf.Epidemico providesdatato
otherpharmaceuticalcompanies
andCROs.
03. INDUSTRY
Disclosures Funding Sources
FDAhasbeenfundingthe
researchintothetechnologyfor
3years.
OfficeoftheChiefScientist
(OCS)
CenterforTobacco Products
(CTP)
OfficeofInternationalPrograms
(OIP)
EPIDEMICO, Inc.+ EPIDEMICO, Ltd.
Authors areemployees or consultants. Epidemico is commercializing aspects ofthis software platform. Epidemico is a wholly
own subsidiary ofBooz Allen Hamilton.

4. How toTame SocialMedia

5. The Drug I’m TakingThe Drug I’m Responsible for Monitoring

6. Remove false
positives
(optional)
Hypothesis
generation
Automated processes
to identifyevents and
clean data
Collect public
social mediadata
Synthesis of
information from
other sources
Acquire Curate Causation
StatisticsFilter
D A T A P R O C E S S O V E R V I E W

7. SCRANTONB,DEVRIESJ,BERINATOS,BORTZC.HARVARDBUSINESSREVIEW,JUNE2013

8. Geotag
Geographyidentified
if availableusing
GoogleAPI
4
3
2
1
Consolidate
Multiplicatecopies&
postsfrom sameuser
consolidated
(PIIremoved)
Translate
Internetvernacularto
MedDRAv18 preferred
terms
Identify Events
AutomatedNLP Bayesian
classifier
Machinelearning
A U T O M A T E D P R O C E S S E S

9. Spelling VariationsCombined or invented wordsImplied phrasesInternet Vernacular Translation
lost their eyesight
seeing weird color
seeing weird colour
doublevision
uldn’t see
double vision
blind
googley eyed
blurry vision
changes in viscross eyed
seeing weird
vision change
blindness
cross visionvisual snow
googly eyed
seeing doub
making me eat like a mouse
anorexic
st appetite
#notevenhungry
appetite is nonexistent
apetite surpressed
didn’t get hungry
dont want to eat
killed my apetite
miss feeling hung
killed my appetite
can’t eat
lost apetite
lost my appetite
no appetitey
lack of apetite
stomach small
lost teh appetite
never hnever want to eat
cant eat
coucrosseyed
blurry
apetite surpressed
killed my apetite
lost apetite
lack of apetite
lost teh appetite
Typos
seeing weird color
seeing weird colour
googley eyed
googly eyed
#notevenhungry
no appetitey
making me eat like a mouse
never want to eat

10. Visual impairment
MedDRA 10047571
Visual impairment
SNOMED 397540003
Decreased appetite
MedDRA 10061428
Loss of appetite
SNOMED 79890006
making me eat like a mouse
anorexic
st appetite
#notevenhungry
appetite is nonexistent
apetite surpressed
didn’t get hungry
dont want to eat
miss feeling hung
killed my appetite
can’t eat
lost apetite
lost my appetite
no appetitey
lack of apetite
stomach small
lost teh appetite
never hnever want to eat
cant eat
lost their eyesight
seeing weird color
seeing weird colour
doublevision
uldn’t see
double vision
blind
googley eyed
blurry vision
changes in viscross eyed
seeing weird
vision change
blindness
cross visionvisual snow
googly eyed
seeing doub
coucrosseyed
blurry
killed my apetite

11. P r o t o - A E
P o s t s w i t h R e s e m b l a n c e t o A d v e r s e E v e n t s

12. ML classifier
trained by manual
review of posts
Precision
-
Positive predictive
value
Recall
-
Sensitivity
Posts mentioning
medical products
60M 380k 68% 88%
S Y S T E M P E R F O R M A N C E
~100% with manual curation

13. Supply Chain Safetyfor Drugs & Food

15. OIP investigated further on Tingyi Holding Corp:
- The company actively ships to the U.S. with
increasing shipments from 2011-2014, two
refused entry lines in 2012 and four detained or
reconditioned lines from 2011-2013
- Products include instant noodles, tea, fruit
juices, and prepared baked goods
- Multiple distributors of its products with the
U.S.
- OIP presented its findings to the Centers with a
mapping of potential networks of suppliers
identified using FDA internal systems.

16. Counterfeit Products: Weibo
Consumer purchased
extremely cheap ointment sold
under brand name Bactroban
but was not manufactured by
GSK.
GSK’s Bactroban

17. ClinicalTrial Unblinding and Drop-out

18. 18
CLINICAL TRIAL DISCUSSION
I'm in a [Company] double blind ######### trial and could receive
the placebo up until week 16 at which point I'm guaranteed to
receive the drug.
What I don't understand is that I can't be told if I was on the
placebo for the first 16 weeks. I'm not seeing any positive benefit
from whatever I've been taking for 8 weeks so far, and I would
really like to know at week 16 if it's the drug because then I can
bail from the study at that time and move on to something else…

19. 19
CLINICAL TRIAL DISCUSSION
It sounds like [your father] very well may be on the real thing.
Those are definite improvements.
…I rather doubt my bro is on the real thing, even with the physical
things that have happened to him (constipation, UTI and prostate
infections.) Either that, or it's not a medication that's going to work
for him. I'm so afraid the latter is the case. If so, what next??
…If he's been on it, I can't see any reason to continue…If he
hasn't been getting it, then certainly it's worth continuing with the
real thing. Problem is we won't find out until this study is
completed for everyone currently in it…

20. 20
CLINICAL TRIAL DISCUSSION
He started on Thursday. He told me yesterday that he tried to chew a
capsule (I'm not sure why, I think he wanted to see if it had a flavor) & it
was hard as a rock. I'm assuming maybe it was eneric coated or
something. He took the no flavor to mean he's on the placebo, but I
actually wonder if they would bother enteric coating a placebo.
Probably doesn't matter & I know we're not supposed to know anyway,
just wondered if anyone else's was enteric coated.

21. Rectifying A False Negative in Spontaneous
Report Data

22. Submit to Regulator
Review Company’s
Safety Database
Routine PortfolioSurveillance
Twitter Post Identified
“Didthisproductcausex, y,z? I’ve
had3administrationsofthis
productandeversince thenhave
felta, b, c.”
SOCIAL DATA
HELPING
SPONTANEOUS DATA
172

23. Real-timeGeo-temporal Surveillance for Adverse
Events
AskingNewKinds ofQuestions inPharmacovigilance

24. RemoveSpam,news, coupons
67% ofpostswerespam
Consolidatemultiplicatecopies
35% Excludingliteralduplicatesandretweets
Keepposts within-scope geography
KeeppostsfromUS,UK,CA,AU
Identifiablegeographyfor55% ofProto-AEs
ImportTwitterPosts
AllDTCproducts
LimittoEnglish Language
Only6 postswithunknownlanguage
Identify Posts with Proto-AEs
Automatedclassifier
Exclude Chantix/Champix
DTCadvertisingallowedinnon-UScountries
1,245,530
408,867
338,263
218,512
11,265
5,535
4,499
D A T A P R O C E S S I N G
18 month study period 58 Rx Drugs w/DTC ads

26. Tweetsper15MinuteIncrement
0
50
100
150
200
250
300
350
0:00
4:00
8:00
12:00
16:00
20:00
Proto-AEs
0:00 24:00
Hour of the Day (local time)
Daytime Television Prime Time
18 month study period 58 Rx Drugs w/DTC ads

27. ResultsW h a t H a p p e n s a t 1 0 O ’ C l o c k ?
"Most AEs occurred between 08:00
and 12:00a.m. with a peak around
10:00 a.m."
“The majority of postings [about
migraine] occurred in the United
States, peaking on weekdays at
10:00h and then gradually again at
22:00h…”

28. Can we quantifypatient benefit?

29. 12 month study period 15 GSK Rx & OTC Products
Source: Powell GE, Duke SP, Bell HG, Anderson LS, Metcalf MA. In Their Own Words: Social Listening for “Real-World
Benefits” from Prescription and OTC Products. Poster, ISPOR 2015.

30. DEMO

31. LIMITATIONS OF SOCIAL MEDIA
CAUSALITY & VERACITY
Patients may not correctly assess
causality. Can system be gamed?
Develop tools to assess
manipulation.
VOLUME
Volume of reports likely to be large.
Reduce false positives and create
automated tools to triage
information.SIGNAL DETECTION
Very limited statistical methods to
detect problems. Collaborate with
academia, industry and regulators
to refine methods.
PRIVACY & GENERALIZABILITY
Patient privacy expectations and
fear of government oversight. Use
publicly available data only.
REGULATION UNCLEAR
When is there an obligation to monitor
or report? Work with regulators and
industry to clarify guidance.
01
02
03
04
05

32. CarrieMike Aranka Ciara Nabarun Wenjie
Harold John Robin Chi Sue
Kelly Kate Clark Chris Robin
Derek
Matt
Essam
Melissa
Kyra
Nico
Lou
Carly
Lucas

33. @epidemico @WEBRADR demo.medwatcher.org nabarun@epidemico.com
W E W A N T 2 H E A R F R O M U