The Evidence Gap
•  Is there evidence to show that mHealth
strategies lead to improved health?
•  How strong is that evide...
mHealth is not monolithic:
This impacts -- evaluation, indicators, business models,
opportunities and strategies for scale.
INTERVENTION(
OF(KNOWN(
EFFICACY(
EFFECTIVE(
COVERAGE(
mHEALTH:((
A(Health(Systems(Catalyst(
Jo(Y,(Labrique(AB(et(al.(In#P...
2011:
< 13% of primary evidence = High Strength
41% “inconclusive”
Source: PubMed, GSMA Literature Review of State of Evid...
mHealth Technical Evidence
Review Group for RMNCH
“m-TERG”
“Providing governments and implementing agencies
objective, evi...
Two Initial Areas Selected
Do mHealth strategies:
!  improve provider adherence to care and
treatment guidelines ?
!  redu...
Bridging the Evidence Gap - Methods
1.  Document Search “beyond the peer-reviewed”:
articles, reports, blogs, presentation...
•  A(flexible(approach(that(allows(the(grading(of(
reporWng(and(methodology(for(varied(study(
designs(
•  &Domain&1:&ReporW...
Two Initial Areas Selected
Do mHealth strategies:
!  improve provider adherence to care and
treatment guidelines ?
!  redu...
Provider Competence,
Accountability,
Effectiveness.
Client Knowledge
and Self-Efficacy
Availability of
Commodities, Health...
Provider Competence,
Accountability,
Effectiveness.
Client Knowledge
and Self-Efficacy
Availability of
Commodities, Health...
Search Strategy
PROVIDER(
ADHERENCE(
Ar#cles(iden#fied(through(database(
search(and(use(of(search(engines(using(a(
combina#...
Completeness of description of methodology
Completeness of description of technology, intervention
Quality of Information
...
!  Domain(1(score(ranged(from(5(to(37(points(out(of(a(
maximum(47apoint(score((11(to(79%)m(suggesWng(large(
variaWons(in(q...
!  The studies that were methodologically sound showed significant
improvement of HW performance when using mHealth
Exampl...
Two Initial Areas Selected
Do mHealth strategies:
!  improve provider adherence to care and
treatment guidelines ?
!  redu...
Provider Competence,
Accountability,
Effectiveness.
Client Knowledge
and Self-Efficacy
Availability of
Commodities, Health...
Provider Competence,
Accountability,
Effectiveness.
Client Knowledge
and Self-Efficacy
Availability of
Commodities, Health...
Search Strategy STOCKOUTS(
Completeness of description of methodology
Completeness of description of technology, intervention
STOCKOUTS(Quality of In...
•  (Overall,(a(majority(of(the(evidence(was(
anecdotal(and(the(evidence(to(suggest(
effecWveness(of(use(of(mHealth(strategi...
The Evidence Gap - Overall
!  Across both domains, there are many interventions but
most are incompletely described
!  Alm...
Labrique mitchell grading the-evidence-combined_v4 #mh4mh
Labrique mitchell grading the-evidence-combined_v4 #mh4mh
Labrique mitchell grading the-evidence-combined_v4 #mh4mh
Labrique mitchell grading the-evidence-combined_v4 #mh4mh
Labrique mitchell grading the-evidence-combined_v4 #mh4mh
Labrique mitchell grading the-evidence-combined_v4 #mh4mh
Labrique mitchell grading the-evidence-combined_v4 #mh4mh
Upcoming SlideShare
Loading in...5
×

Labrique mitchell grading the-evidence-combined_v4 #mh4mh

229

Published on

#mh4mh #mHealth

Published in: Government & Nonprofit
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
229
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Labrique mitchell grading the-evidence-combined_v4 #mh4mh

  1. 1. The Evidence Gap •  Is there evidence to show that mHealth strategies lead to improved health? •  How strong is that evidence?
  2. 2. mHealth is not monolithic: This impacts -- evaluation, indicators, business models, opportunities and strategies for scale.
  3. 3. INTERVENTION( OF(KNOWN( EFFICACY( EFFECTIVE( COVERAGE( mHEALTH:(( A(Health(Systems(Catalyst( Jo(Y,(Labrique(AB(et(al.(In#Press#2013(( ( ShiF(focus(from(“Does(mHealth(work?”(to(“Does( mHealth(op.mize#what(we(know(works(?”( (
  4. 4. 2011: < 13% of primary evidence = High Strength 41% “inconclusive” Source: PubMed, GSMA Literature Review of State of Evidence on mHealth 2011
  5. 5. mHealth Technical Evidence Review Group for RMNCH “m-TERG” “Providing governments and implementing agencies objective, evidence-based guidance for the selection and scale of mHealth strategies across the reproductive, maternal, newborn and child health continuum”
  6. 6. Two Initial Areas Selected Do mHealth strategies: !  improve provider adherence to care and treatment guidelines ? !  reduce stockouts of essential maternal health drugs and commodities? PROVIDER( ADHERENCE( STOCKOUTS( mTERG&
  7. 7. Bridging the Evidence Gap - Methods 1.  Document Search “beyond the peer-reviewed”: articles, reports, blogs, presentations using extended strategy, ‘sleuthing’ references and links 2.  Included documents which describe: !  Implementation of an mHealth intervention !  Systematic Evaluation Methodology 3.  Tag document using WHO mTERG taxonomy 4.  Grade document quality using WHO mTERG instrument 5.  Summary / Synthesis of overall direction, magnitude and consistency of effect mTERG&
  8. 8. •  A(flexible(approach(that(allows(the(grading(of( reporWng(and(methodology(for(varied(study( designs( •  &Domain&1:&ReporWng(and(Methodology(–(This(is( indicaWve(of(the(quality(of(methodological(rigor( employed(by(the(studies(under(consideraWon,(as( well(as(the(reporWng(standards(that(have(been( adhered(to.( •  Domain&2:(EssenWal(mHealth(criteria(–(Classifies( the(studies(under(consideraWon(based(on(the( quality(of(informaWon(presented(about(the( mHealth(intervenWon.(( Grading&quality&of&evidence
  9. 9. Two Initial Areas Selected Do mHealth strategies: !  improve provider adherence to care and treatment guidelines ? !  reduce stockouts of essential maternal health drugs and commodities? PROVIDER( ADHERENCE( STOCKOUTS(
  10. 10. Provider Competence, Accountability, Effectiveness. Client Knowledge and Self-Efficacy Availability of Commodities, Health Workers, Equipment Improved Information about individuals, populations, providers, facilities, outcomes, Improved HEALTH Outcomes Improved Quality of Care Improved Health Behaviors Improved Efficiency + Coverage Provider Client System Remote Client to Provider (Telemedicine) Provider Training and Education Provider Work Planning and Scheduling Disease Prevention Provider to Provider Communication Human Resource Management Electronic Decision Support Appointment Reminders Treatment Adherence Mass Messaging Campaigns Health Education or Promotion Hotlines and Information Services Registries/Vital Events Tracking Insurance Payment for Services Savings Accounts Electronic Health Records Data Collection and Reporting Point of Care Diagnostics Disease Management Referrals Cold Chain Management Performance Based Incentives Stock Out Prevention Counterfeit Prevention Maintenance of Equipment Commodity Tracking
  11. 11. Provider Competence, Accountability, Effectiveness. Client Knowledge and Self-Efficacy Availability of Commodities, Health Workers, Equipment Improved Information about individuals, populations, providers, facilities, outcomes, Improved HEALTH Outcomes Improved Quality of Care Improved Health Behaviors Improved Efficiency + Coverage Provider Client System Remote Client to Provider (Telemedicine) Provider Training and Education Provider Work Planning and Scheduling Disease Prevention Provider to Provider Communication Human Resource Management Electronic Decision Support Appointment Reminders Treatment Adherence Mass Messaging Campaigns Health Education or Promotion Hotlines and Information Services Registries/Vital Events Tracking Insurance Payment for Services Savings Accounts Electronic Health Records Data Collection and Reporting Point of Care Diagnostics Disease Management Referrals Cold Chain Management Performance Based Incentives Stock Out Prevention Counterfeit Prevention Maintenance of Equipment Commodity Tracking PROVIDER( ADHERENCE(
  12. 12. Search Strategy PROVIDER( ADHERENCE( Ar#cles(iden#fied(through(database( search(and(use(of(search(engines(using(a( combina#on(of(search(terms( n=#1,499# Poten#ally(eligible(ar#cles(remaining( n#=#1,459# Titles(and(abstracts(of(1,459(poten#ally( eligible(ar#cles(screened( Applica#on(of(inclusion(and(exclusion( criteria(to(1,459(ar#cles( 53#ar,cles#tagged#using#the#taxonomy# tool# 21#ar,cles#graded#using#the#grading# tool# 1,406#ar,cles#excluded# •  161(were(not(mobile(health( related(( •  1,117(did(not(meet(health( domain(criteria( •  105(did(not(meet(mHealth( applica#on(criteria( •  11(did(not(meet(target(user( criteria( •  12(only(briefly(men#oned(the(use( of(mHealth(for(decision(support(( # Iden#fica#on(Screening(Eligibility(Inclusion( 40#duplicate#ar,cles# removed# # 32#ar,cles#did#not#meet#grading# criteria# •  7(did(not(provide(descrip#on(of( evalua#on(of(mHealth(program( •  2(did(not(provide(descrip#on(of( implementa#on(of(mHealth( program(( •  13(did(not(provided(descrip#on( of(evalua#on(or(implementa#on( of(mHealth(program( #
  13. 13. Completeness of description of methodology Completeness of description of technology, intervention Quality of Information PROVIDER( ADHERENCE(
  14. 14. !  Domain(1(score(ranged(from(5(to(37(points(out(of(a( maximum(47apoint(score((11(to(79%)m(suggesWng(large( variaWons(in(quality(and(methodology(of(reporWng( ( !  Only(three(of(the(21(graded(studies(achieved(a(score( >75%(for(Domain(2((essenWal(mHealth(criteria),(which( indicates(that(most(studies(do(not(adequately(describe( the(mHealth(intervenWon(they(are(reporWng(on.(( & Can mHealth strategies be used to improve provider adherence to care and treatment guidelines? PROVIDER( ADHERENCE(
  15. 15. !  The studies that were methodologically sound showed significant improvement of HW performance when using mHealth Examples: o  In(South(Africa,(Rhode(and(colleagues(randomized(24(primary(care(nurses(to( be(trained(in(IMCI(using(tradiWonal(paperabased(guidelines,(and(concluded(that( nurses(who(used(the(IMCI(decisionasupport(tool(were(significantly(more(likely( to(make(an(accurate(diagnosis((p<0.001).( o  (In(rural(Kenya,(Zurovac(et(al(randomized(outpaWent(health(workers(with( mobile(phones(to(receive(text(message(reminders(about(naWonal(pediatric( malaria(guidelines(to(improve(and(sustain(protocol(adherence.(For(health( workers(who(received(the(SMS(reminders,(correct(management(of(pediatric( malaria(increased(by(almost(25%(at(both(6(months(and(12(months(( !  The use of mHealth varied: e-training, peer to peer, SMS reminders, decision support PROVIDER( ADHERENCE(
  16. 16. Two Initial Areas Selected Do mHealth strategies: !  improve provider adherence to care and treatment guidelines ? !  reduce stockouts of essential maternal health drugs and commodities? PROVIDER( ADHERENCE( STOCKOUTS(
  17. 17. Provider Competence, Accountability, Effectiveness. Client Knowledge and Self-Efficacy Availability of Commodities, Health Workers, Equipment Improved Information about individuals, populations, providers, facilities, outcomes, Improved HEALTH Outcomes Improved Quality of Care Improved Health Behaviors Improved Efficiency + Coverage Provider Client System Remote Client to Provider (Telemedicine) Provider Training and Education Provider Work Planning and Scheduling Disease Prevention Provider to Provider Communication Human Resource Management Electronic Decision Support Appointment Reminders Treatment Adherence Mass Messaging Campaigns Health Education or Promotion Hotlines and Information Services Registries/Vital Events Tracking Insurance Payment for Services Savings Accounts Electronic Health Records Data Collection and Reporting Point of Care Diagnostics Disease Management Referrals Cold Chain Management Performance Based Incentives Stock Out Prevention Counterfeit Prevention Maintenance of Equipment Commodity Tracking
  18. 18. Provider Competence, Accountability, Effectiveness. Client Knowledge and Self-Efficacy Availability of Commodities, Health Workers, Equipment Improved Information about individuals, populations, providers, facilities, outcomes, Improved HEALTH Outcomes Improved Quality of Care Improved Health Behaviors Improved Efficiency + Coverage Provider Client System Remote Client to Provider (Telemedicine) Provider Training and Education Provider Work Planning and Scheduling Disease Prevention Provider to Provider Communication Human Resource Management Electronic Decision Support Appointment Reminders Treatment Adherence Mass Messaging Campaigns Health Education or Promotion Hotlines and Information Services Registries/Vital Events Tracking Insurance Payment for Services Savings Accounts Electronic Health Records Data Collection and Reporting Point of Care Diagnostics Disease Management Referrals Cold Chain Management Performance Based Incentives Stock Out Prevention Counterfeit Prevention Maintenance of Equipment Commodity Tracking STOCKOUTS(
  19. 19. Search Strategy STOCKOUTS(
  20. 20. Completeness of description of methodology Completeness of description of technology, intervention STOCKOUTS(Quality of Information
  21. 21. •  (Overall,(a(majority(of(the(evidence(was( anecdotal(and(the(evidence(to(suggest( effecWveness(of(use(of(mHealth(strategies(to( prevent(stockaouts(is(weak( •  (A(number(of(projects(are(being(currently( implemented((but(have(limited(published( results(on(effecWveness( The Evidence Gap STOCKOUTS(
  22. 22. The Evidence Gap - Overall !  Across both domains, there are many interventions but most are incompletely described !  Almost none have design that allow rigorous evaluation !  Almost none have control groups !  It is impossible to compare or combine study results due to lack of standardization !  The nature of the exact interventions differs !  Differing indicators make meta-analyses difficult( PROVIDER( ADHERENCE( STOCKOUTS(
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×