SlideShare a Scribd company logo
Using Online Crowd-Sourcing Platform (mTurk) to
Prioritize Research Topics Among Patient Populations:
Ghauri MS, Truitt AR, Malashanka S, Lawrence SO, Lavallee DC. Department of Surgery. University of Washington
Introduction
 Lower back pain is one of the leading causes of work disability in the
United States, estimating 240 billion dollars in annual medical care
costs (WHO 2015)
 Clinicians and researchers have largely directed research priorities,
creating a potential misalignment between what patients want to know
and current research trajectories
 Research funders are looking into ways of allocating funds better
 To address this, we explore how to include patient perspectives about
back pain research topic priorities
Methods
Results
Additional Prioritization topics by
frequency
1. Identifying complementary and
alternative to surgery and medication
6. Education on drugs and medication
use
2. Comorbidities
7. Identifying prophylactic measures to
prevent LBP and pain management
3. Healthcare System Reform
(referrals, cross-specialty)
8. Improving working disability insurance
and return to work benefits
4. Improved communication between
physicians and patients to help
patients better interpret
treatment/prevention strategies
9) Addiction and mental health counseling
5. Home-based self-care strategies
10)Treatment efficacy and outcomes
research
Future Directions
BOLD
2
3
4
MTurk
6 7
8 9
Discussion
Conclusion
 The ranking and frequency analysis conveys a discrepancy
between researchers/clinicians and patients’ perspectives of
important research topics
 MTurk had more than twice the response rate showing its efficacy
as a recruitment strategy for low back pain research and the
willingness of participants to incorporate their perspectives
 Both populations reveal new areas of focus for improving patient
centered outcomes
1
4
10
 Highest ranked research topics are the same between BOLD and
MTurk, suggesting both strategies yield similar priorities.
 BOLD participants supplied additional topics about co-morbities
and strategies to improve healthcare processes
 MTurk participants supplied additional topics about information
about medication side effects, including addiction, and work
disability insurance and return to work benefits
 Using MTurk with incentives cost a $600 and approximately 2
months of staff time in comparison to BOLD which took over a year
and a half to obtain similar results
• What can be done to contain and reverse the epidemic of
LBP in developed countries
• What are the “best” strategies for treating LBP
• How can we improve self-care strategies and stimulate self-
reliance among persons with chronic or recurrent LBP
Researcher/Clinician
Highest Ranked Research Topics
• What are the causes of low back pain?
• What are effective tests to diagnose lower back pain
• How should lower back pain be defined?
BOLD
• What are the causes of low back pain?
• What are effective tests to diagnose lower back pain
• How can the burden of lower back pain be reduced?
MTurk
 Patient registries provide an opportunity for getting patients involved
in identifying and prioritizing research topics. However, further work
is needed to improve the translation of patient priorities into clinical
practice
 Online crowd-sourcing platforms may serve as a possible venue for
cheap and efficient research topics identification
 Further work should explore strategies to incorporate diverse
perspectives
Recruitment and Data Collection
 Participants were recruited in two ways: Back Pain Outcomes Using Longitudinal Data
(BOLD) registry and MTurk
 Each recruitment method utilized the same survey that asked participants to rank research
topics from a list of priorities previously established (cite) and add any additional topics, as
well as demographic information
 Addressed the limitation of the BOLD registry : wider demographic under the age of 65
BOLD
 BOLD is a patient registry of adults 65 and older with lower back pain recruited from
two major healthcare systems
Amazon Mechanical Turk
 (MTurk) is an online crowd-sourcing platform
 Completed a screening survey to identify back pain, paralleling BOLD processes
Data Analysis
 Using descriptive statistics, we compared the participants from each recruitment method
 We categorized the prioritized the topics by frequency for each group
 We qualitatively analyzed the additional topics identified
Participants
 351 of the 480 MTurk participants (74%) completed the prioritization survey
 931 of the total 3162 (29%) eligible BOLD participants responded
Demographics
Acknowledgements
This project was supported by contract number ME-1310-07328 from
the Patient-Centered Outcomes Research Institute (PCORI). The
content is solely the responsibility of the authors and does not
necessarily represent the official views of the PCORI.
Worker characteristics BOLD (N=4131) MTurk (N=351)
Age 73 (mean) 37 (mean)
Sex
Female 3098 (74%) 234 (67%)
Male 1033 (26%) 114 (33%)
Ethnicity/Race
Caucasian 69 (%) 299 (83%)
Non-Hispanic 93 (%) 322 (92%)
Education
HS grad or more 3820 (70%) 344 (86%)
RMDQ 12.8 (mean) 11.3 (mean)
Pain Duration
> 5 yr 17 (%) 44 (%)

More Related Content

What's hot

AITRP eHIT 101 tutorial
AITRP eHIT 101 tutorialAITRP eHIT 101 tutorial
Initial Medical Policy and Model Coverage Guidelines
Initial Medical Policy and Model Coverage GuidelinesInitial Medical Policy and Model Coverage Guidelines
Initial Medical Policy and Model Coverage Guidelines
Center for Medical Technology Policy
 
Patient Recruitment Insights
Patient Recruitment InsightsPatient Recruitment Insights
Patient Recruitment Insights
Dora Calderon
 
Medical Simulation 2.0: Improving value-based healthcare delivery
Medical Simulation 2.0:  Improving value-based healthcare deliveryMedical Simulation 2.0:  Improving value-based healthcare delivery
Medical Simulation 2.0: Improving value-based healthcare delivery
Yue Dong
 
Growth of Research Centers Focused on Quality Improvement report_FINAL
Growth of Research Centers Focused on Quality Improvement report_FINALGrowth of Research Centers Focused on Quality Improvement report_FINAL
Growth of Research Centers Focused on Quality Improvement report_FINAL
Natalie Fahlberg
 
Defining the Outcomes that Matter for Perioperative Pain Medicine
Defining the Outcomes that Matter for Perioperative Pain MedicineDefining the Outcomes that Matter for Perioperative Pain Medicine
Defining the Outcomes that Matter for Perioperative Pain Medicine
Colin McCartney
 
Outcomes research
Outcomes research Outcomes research
Outcomes research
Marilyn Alvarez, BSN, RN
 
My Top 5 articles from 2015-16 about Informatics and Digital Health in Physio...
My Top 5 articles from 2015-16 about Informatics and Digital Health in Physio...My Top 5 articles from 2015-16 about Informatics and Digital Health in Physio...
My Top 5 articles from 2015-16 about Informatics and Digital Health in Physio...
Samantha Plumb
 
Operational research in Public Health in India
Operational research in Public Health in IndiaOperational research in Public Health in India
Operational research in Public Health in India
Dr. Dharmendra Gahwai
 
Visual Analytics in Healtcare
Visual Analytics in HealtcareVisual Analytics in Healtcare
Visual Analytics in Healtcare
jetweedy
 
SafePatientHandlingLiteratureReviewMCPHIsAbstract
SafePatientHandlingLiteratureReviewMCPHIsAbstractSafePatientHandlingLiteratureReviewMCPHIsAbstract
SafePatientHandlingLiteratureReviewMCPHIsAbstract
Pamela McNamara RN BSN
 
Clinical Trial Recruitment
Clinical Trial RecruitmentClinical Trial Recruitment
Clinical Trial Recruitment
Clinical List America
 
After Fellowship
After FellowshipAfter Fellowship
After Fellowship
MedicineAndHealthUSA
 
OTC Electronic Clinical Trial Recruitment
OTC Electronic Clinical Trial RecruitmentOTC Electronic Clinical Trial Recruitment
OTC Electronic Clinical Trial Recruitment
TECHdotMN
 
Machine Learning applied to heart failure readmissions
Machine Learning applied to heart failure readmissionsMachine Learning applied to heart failure readmissions
Machine Learning applied to heart failure readmissions
John Frias Morales, DrBA, MS
 
Using NHS Electronic Health Records for Research
Using NHS Electronic Health Records for ResearchUsing NHS Electronic Health Records for Research
Using NHS Electronic Health Records for Research
London School of Hygiene and Tropical Medicine
 
Clinical Trial Recruitment & Retention
Clinical Trial Recruitment & RetentionClinical Trial Recruitment & Retention
Clinical Trial Recruitment & Retention
Asijit Sen
 
Healthcare Communications Study Among Physicians: Medical Monitor 2013
Healthcare Communications Study Among Physicians: Medical Monitor 2013Healthcare Communications Study Among Physicians: Medical Monitor 2013
Healthcare Communications Study Among Physicians: Medical Monitor 2013
Joshua Spiegel
 
Results of an Online Survey of Stakeholders Regarding Barriers and Solutions ...
Results of an Online Survey of Stakeholders Regarding Barriers and Solutions ...Results of an Online Survey of Stakeholders Regarding Barriers and Solutions ...
Results of an Online Survey of Stakeholders Regarding Barriers and Solutions ...
John Reites
 
2010: Time for Minimum Standards for Health Facilities
2010:  Time for Minimum Standards for Health Facilities 2010:  Time for Minimum Standards for Health Facilities
2010: Time for Minimum Standards for Health Facilities
MEASURE Evaluation
 

What's hot (20)

AITRP eHIT 101 tutorial
AITRP eHIT 101 tutorialAITRP eHIT 101 tutorial
AITRP eHIT 101 tutorial
 
Initial Medical Policy and Model Coverage Guidelines
Initial Medical Policy and Model Coverage GuidelinesInitial Medical Policy and Model Coverage Guidelines
Initial Medical Policy and Model Coverage Guidelines
 
Patient Recruitment Insights
Patient Recruitment InsightsPatient Recruitment Insights
Patient Recruitment Insights
 
Medical Simulation 2.0: Improving value-based healthcare delivery
Medical Simulation 2.0:  Improving value-based healthcare deliveryMedical Simulation 2.0:  Improving value-based healthcare delivery
Medical Simulation 2.0: Improving value-based healthcare delivery
 
Growth of Research Centers Focused on Quality Improvement report_FINAL
Growth of Research Centers Focused on Quality Improvement report_FINALGrowth of Research Centers Focused on Quality Improvement report_FINAL
Growth of Research Centers Focused on Quality Improvement report_FINAL
 
Defining the Outcomes that Matter for Perioperative Pain Medicine
Defining the Outcomes that Matter for Perioperative Pain MedicineDefining the Outcomes that Matter for Perioperative Pain Medicine
Defining the Outcomes that Matter for Perioperative Pain Medicine
 
Outcomes research
Outcomes research Outcomes research
Outcomes research
 
My Top 5 articles from 2015-16 about Informatics and Digital Health in Physio...
My Top 5 articles from 2015-16 about Informatics and Digital Health in Physio...My Top 5 articles from 2015-16 about Informatics and Digital Health in Physio...
My Top 5 articles from 2015-16 about Informatics and Digital Health in Physio...
 
Operational research in Public Health in India
Operational research in Public Health in IndiaOperational research in Public Health in India
Operational research in Public Health in India
 
Visual Analytics in Healtcare
Visual Analytics in HealtcareVisual Analytics in Healtcare
Visual Analytics in Healtcare
 
SafePatientHandlingLiteratureReviewMCPHIsAbstract
SafePatientHandlingLiteratureReviewMCPHIsAbstractSafePatientHandlingLiteratureReviewMCPHIsAbstract
SafePatientHandlingLiteratureReviewMCPHIsAbstract
 
Clinical Trial Recruitment
Clinical Trial RecruitmentClinical Trial Recruitment
Clinical Trial Recruitment
 
After Fellowship
After FellowshipAfter Fellowship
After Fellowship
 
OTC Electronic Clinical Trial Recruitment
OTC Electronic Clinical Trial RecruitmentOTC Electronic Clinical Trial Recruitment
OTC Electronic Clinical Trial Recruitment
 
Machine Learning applied to heart failure readmissions
Machine Learning applied to heart failure readmissionsMachine Learning applied to heart failure readmissions
Machine Learning applied to heart failure readmissions
 
Using NHS Electronic Health Records for Research
Using NHS Electronic Health Records for ResearchUsing NHS Electronic Health Records for Research
Using NHS Electronic Health Records for Research
 
Clinical Trial Recruitment & Retention
Clinical Trial Recruitment & RetentionClinical Trial Recruitment & Retention
Clinical Trial Recruitment & Retention
 
Healthcare Communications Study Among Physicians: Medical Monitor 2013
Healthcare Communications Study Among Physicians: Medical Monitor 2013Healthcare Communications Study Among Physicians: Medical Monitor 2013
Healthcare Communications Study Among Physicians: Medical Monitor 2013
 
Results of an Online Survey of Stakeholders Regarding Barriers and Solutions ...
Results of an Online Survey of Stakeholders Regarding Barriers and Solutions ...Results of an Online Survey of Stakeholders Regarding Barriers and Solutions ...
Results of an Online Survey of Stakeholders Regarding Barriers and Solutions ...
 
2010: Time for Minimum Standards for Health Facilities
2010:  Time for Minimum Standards for Health Facilities 2010:  Time for Minimum Standards for Health Facilities
2010: Time for Minimum Standards for Health Facilities
 

Similar to URP_AT (1)

Example Dissertation Proposal Defense Power Point Slide
Example Dissertation Proposal Defense Power Point SlideExample Dissertation Proposal Defense Power Point Slide
Example Dissertation Proposal Defense Power Point Slide
Dr. Vince Bridges
 
Healthier Longevity: Personalizing the Aging Process Through Connected Health...
Healthier Longevity: Personalizing the Aging Process Through Connected Health...Healthier Longevity: Personalizing the Aging Process Through Connected Health...
Healthier Longevity: Personalizing the Aging Process Through Connected Health...
Personal Connected Health Alliance
 
Cadth 2015 a5 1 pt engage symp marshall cadth apr 13 2015 present
Cadth 2015 a5 1  pt engage symp marshall cadth apr 13 2015 presentCadth 2015 a5 1  pt engage symp marshall cadth apr 13 2015 present
Cadth 2015 a5 1 pt engage symp marshall cadth apr 13 2015 present
CADTH Symposium
 
Week 12 discussion research
Week 12 discussion researchWeek 12 discussion research
Week 12 discussion research
HaleyKnutson1
 
Mbareng601week12
Mbareng601week12Mbareng601week12
Mbareng601week12
Mitchell Bareng
 
Initial post week 12
Initial post week 12 Initial post week 12
Initial post week 12
rraquedan
 
Transitional Care Workgroup
Transitional Care WorkgroupTransitional Care Workgroup
Improving Healthcare Systems Program
Improving Healthcare Systems ProgramImproving Healthcare Systems Program
Improving Healthcare Systems Program
Patient-Centered Outcomes Research Institute
 
Latino Roundtable Discussion
Latino Roundtable Discussion Latino Roundtable Discussion
Latino Roundtable Discussion
Patient-Centered Outcomes Research Institute
 
Example Final Defense Power Point Slide
Example Final Defense Power Point SlideExample Final Defense Power Point Slide
Example Final Defense Power Point Slide
Dr. Vince Bridges
 
Week 12 discussion power point
Week 12 discussion power pointWeek 12 discussion power point
Week 12 discussion power point
Jamie Lewis
 
MARKETING PRINCIPLESMKTG 305SWOT Assignment – CSUSB.docx
MARKETING PRINCIPLESMKTG 305SWOT Assignment – CSUSB.docxMARKETING PRINCIPLESMKTG 305SWOT Assignment – CSUSB.docx
MARKETING PRINCIPLESMKTG 305SWOT Assignment – CSUSB.docx
alfredacavx97
 
Weitzman 2013: PCORI: Transforming Health Care
Weitzman 2013: PCORI: Transforming Health CareWeitzman 2013: PCORI: Transforming Health Care
Weitzman 2013: PCORI: Transforming Health Care
CHC Connecticut
 
Outcomes Research
Outcomes ResearchOutcomes Research
Outcomes Research
Ana Manalili
 
Outcomes Research
Outcomes ResearchOutcomes Research
Outcomes Research
Ana Manalili
 
Amia2016 pghd-panel-v8
Amia2016 pghd-panel-v8Amia2016 pghd-panel-v8
Amia2016 pghd-panel-v8
Pei-Yun Sabrina Hsueh
 
Chapter 4 Knowledge Discovery, Data Mining, and Practice-Based Evi.docx
Chapter 4 Knowledge Discovery, Data Mining, and Practice-Based Evi.docxChapter 4 Knowledge Discovery, Data Mining, and Practice-Based Evi.docx
Chapter 4 Knowledge Discovery, Data Mining, and Practice-Based Evi.docx
christinemaritza
 
Paper Assignment # 2 Using your exploration from paper 1, .docx
Paper Assignment # 2 Using your exploration from paper 1, .docxPaper Assignment # 2 Using your exploration from paper 1, .docx
Paper Assignment # 2 Using your exploration from paper 1, .docx
bunyansaturnina
 
My Research Projects
My Research ProjectsMy Research Projects
My Research Projects
rpunekar
 
krista hamlin research post
krista hamlin research postkrista hamlin research post
krista hamlin research post
fnpstudent
 

Similar to URP_AT (1) (20)

Example Dissertation Proposal Defense Power Point Slide
Example Dissertation Proposal Defense Power Point SlideExample Dissertation Proposal Defense Power Point Slide
Example Dissertation Proposal Defense Power Point Slide
 
Healthier Longevity: Personalizing the Aging Process Through Connected Health...
Healthier Longevity: Personalizing the Aging Process Through Connected Health...Healthier Longevity: Personalizing the Aging Process Through Connected Health...
Healthier Longevity: Personalizing the Aging Process Through Connected Health...
 
Cadth 2015 a5 1 pt engage symp marshall cadth apr 13 2015 present
Cadth 2015 a5 1  pt engage symp marshall cadth apr 13 2015 presentCadth 2015 a5 1  pt engage symp marshall cadth apr 13 2015 present
Cadth 2015 a5 1 pt engage symp marshall cadth apr 13 2015 present
 
Week 12 discussion research
Week 12 discussion researchWeek 12 discussion research
Week 12 discussion research
 
Mbareng601week12
Mbareng601week12Mbareng601week12
Mbareng601week12
 
Initial post week 12
Initial post week 12 Initial post week 12
Initial post week 12
 
Transitional Care Workgroup
Transitional Care WorkgroupTransitional Care Workgroup
Transitional Care Workgroup
 
Improving Healthcare Systems Program
Improving Healthcare Systems ProgramImproving Healthcare Systems Program
Improving Healthcare Systems Program
 
Latino Roundtable Discussion
Latino Roundtable Discussion Latino Roundtable Discussion
Latino Roundtable Discussion
 
Example Final Defense Power Point Slide
Example Final Defense Power Point SlideExample Final Defense Power Point Slide
Example Final Defense Power Point Slide
 
Week 12 discussion power point
Week 12 discussion power pointWeek 12 discussion power point
Week 12 discussion power point
 
MARKETING PRINCIPLESMKTG 305SWOT Assignment – CSUSB.docx
MARKETING PRINCIPLESMKTG 305SWOT Assignment – CSUSB.docxMARKETING PRINCIPLESMKTG 305SWOT Assignment – CSUSB.docx
MARKETING PRINCIPLESMKTG 305SWOT Assignment – CSUSB.docx
 
Weitzman 2013: PCORI: Transforming Health Care
Weitzman 2013: PCORI: Transforming Health CareWeitzman 2013: PCORI: Transforming Health Care
Weitzman 2013: PCORI: Transforming Health Care
 
Outcomes Research
Outcomes ResearchOutcomes Research
Outcomes Research
 
Outcomes Research
Outcomes ResearchOutcomes Research
Outcomes Research
 
Amia2016 pghd-panel-v8
Amia2016 pghd-panel-v8Amia2016 pghd-panel-v8
Amia2016 pghd-panel-v8
 
Chapter 4 Knowledge Discovery, Data Mining, and Practice-Based Evi.docx
Chapter 4 Knowledge Discovery, Data Mining, and Practice-Based Evi.docxChapter 4 Knowledge Discovery, Data Mining, and Practice-Based Evi.docx
Chapter 4 Knowledge Discovery, Data Mining, and Practice-Based Evi.docx
 
Paper Assignment # 2 Using your exploration from paper 1, .docx
Paper Assignment # 2 Using your exploration from paper 1, .docxPaper Assignment # 2 Using your exploration from paper 1, .docx
Paper Assignment # 2 Using your exploration from paper 1, .docx
 
My Research Projects
My Research ProjectsMy Research Projects
My Research Projects
 
krista hamlin research post
krista hamlin research postkrista hamlin research post
krista hamlin research post
 

URP_AT (1)

  • 1. Using Online Crowd-Sourcing Platform (mTurk) to Prioritize Research Topics Among Patient Populations: Ghauri MS, Truitt AR, Malashanka S, Lawrence SO, Lavallee DC. Department of Surgery. University of Washington Introduction  Lower back pain is one of the leading causes of work disability in the United States, estimating 240 billion dollars in annual medical care costs (WHO 2015)  Clinicians and researchers have largely directed research priorities, creating a potential misalignment between what patients want to know and current research trajectories  Research funders are looking into ways of allocating funds better  To address this, we explore how to include patient perspectives about back pain research topic priorities Methods Results Additional Prioritization topics by frequency 1. Identifying complementary and alternative to surgery and medication 6. Education on drugs and medication use 2. Comorbidities 7. Identifying prophylactic measures to prevent LBP and pain management 3. Healthcare System Reform (referrals, cross-specialty) 8. Improving working disability insurance and return to work benefits 4. Improved communication between physicians and patients to help patients better interpret treatment/prevention strategies 9) Addiction and mental health counseling 5. Home-based self-care strategies 10)Treatment efficacy and outcomes research Future Directions BOLD 2 3 4 MTurk 6 7 8 9 Discussion Conclusion  The ranking and frequency analysis conveys a discrepancy between researchers/clinicians and patients’ perspectives of important research topics  MTurk had more than twice the response rate showing its efficacy as a recruitment strategy for low back pain research and the willingness of participants to incorporate their perspectives  Both populations reveal new areas of focus for improving patient centered outcomes 1 4 10  Highest ranked research topics are the same between BOLD and MTurk, suggesting both strategies yield similar priorities.  BOLD participants supplied additional topics about co-morbities and strategies to improve healthcare processes  MTurk participants supplied additional topics about information about medication side effects, including addiction, and work disability insurance and return to work benefits  Using MTurk with incentives cost a $600 and approximately 2 months of staff time in comparison to BOLD which took over a year and a half to obtain similar results • What can be done to contain and reverse the epidemic of LBP in developed countries • What are the “best” strategies for treating LBP • How can we improve self-care strategies and stimulate self- reliance among persons with chronic or recurrent LBP Researcher/Clinician Highest Ranked Research Topics • What are the causes of low back pain? • What are effective tests to diagnose lower back pain • How should lower back pain be defined? BOLD • What are the causes of low back pain? • What are effective tests to diagnose lower back pain • How can the burden of lower back pain be reduced? MTurk  Patient registries provide an opportunity for getting patients involved in identifying and prioritizing research topics. However, further work is needed to improve the translation of patient priorities into clinical practice  Online crowd-sourcing platforms may serve as a possible venue for cheap and efficient research topics identification  Further work should explore strategies to incorporate diverse perspectives Recruitment and Data Collection  Participants were recruited in two ways: Back Pain Outcomes Using Longitudinal Data (BOLD) registry and MTurk  Each recruitment method utilized the same survey that asked participants to rank research topics from a list of priorities previously established (cite) and add any additional topics, as well as demographic information  Addressed the limitation of the BOLD registry : wider demographic under the age of 65 BOLD  BOLD is a patient registry of adults 65 and older with lower back pain recruited from two major healthcare systems Amazon Mechanical Turk  (MTurk) is an online crowd-sourcing platform  Completed a screening survey to identify back pain, paralleling BOLD processes Data Analysis  Using descriptive statistics, we compared the participants from each recruitment method  We categorized the prioritized the topics by frequency for each group  We qualitatively analyzed the additional topics identified Participants  351 of the 480 MTurk participants (74%) completed the prioritization survey  931 of the total 3162 (29%) eligible BOLD participants responded Demographics Acknowledgements This project was supported by contract number ME-1310-07328 from the Patient-Centered Outcomes Research Institute (PCORI). The content is solely the responsibility of the authors and does not necessarily represent the official views of the PCORI. Worker characteristics BOLD (N=4131) MTurk (N=351) Age 73 (mean) 37 (mean) Sex Female 3098 (74%) 234 (67%) Male 1033 (26%) 114 (33%) Ethnicity/Race Caucasian 69 (%) 299 (83%) Non-Hispanic 93 (%) 322 (92%) Education HS grad or more 3820 (70%) 344 (86%) RMDQ 12.8 (mean) 11.3 (mean) Pain Duration > 5 yr 17 (%) 44 (%)