The document discusses two initial areas being evaluated by mTERG (mHealth Technical Evidence Review Group for RMNCH) to determine if mHealth strategies can improve provider adherence to care guidelines and reduce stockouts of essential maternal health drugs and commodities. For provider adherence, the studies reviewed showed mHealth improved health worker performance, though quality varied. For stockouts, evidence was weak and anecdotal, with few published results on effectiveness. Overall, there is a lack of rigorous evaluation and standardization across mHealth interventions, making it difficult to draw conclusions about impacts on health.

2. The Evidence Gap
• Is there evidence to show that mHealth
strategies lead to improved health?
• How strong is that evidence?

3. mHealth is not monolithic:
This impacts -- evaluation, indicators, business models,
opportunities and strategies for scale.

4. INTERVENTION(
OF(KNOWN(
EFFICACY(
EFFECTIVE(
COVERAGE(
mHEALTH:((
A(Health(Systems(Catalyst(
Jo(Y,(Labrique(AB(et(al.(In#Press#2013((
(
ShiF(focus(from(“Does(mHealth(work?”(to(“Does(
mHealth(op.mize#what(we(know(works(?”(
(

6. 2011:
< 13% of primary evidence = High Strength
41% “inconclusive”
Source: PubMed, GSMA Literature Review of State of Evidence on mHealth 2011

10. mHealth Technical Evidence
Review Group for RMNCH
“m-TERG”
“Providing governments and implementing agencies
objective, evidence-based guidance for the
selection and scale of mHealth strategies
across the reproductive, maternal,
newborn and child health continuum”

11. Two Initial Areas Selected
Do mHealth strategies:
! improve provider adherence to care and
treatment guidelines ?
! reduce stockouts of essential maternal health
drugs and commodities?
PROVIDER(
ADHERENCE(
STOCKOUTS(
mTERG&

12. Bridging the Evidence Gap - Methods
1. Document Search “beyond the peer-reviewed”:
articles, reports, blogs, presentations using extended
strategy, ‘sleuthing’ references and links
2. Included documents which describe:
! Implementation of an mHealth intervention
! Systematic Evaluation Methodology
3. Tag document using WHO mTERG taxonomy
4. Grade document quality using WHO mTERG
instrument
5. Summary / Synthesis of overall direction, magnitude
and consistency of effect
mTERG&

13. • A(flexible(approach(that(allows(the(grading(of(
reporWng(and(methodology(for(varied(study(
designs(
• &Domain&1:&ReporWng(and(Methodology(–(This(is(
indicaWve(of(the(quality(of(methodological(rigor(
employed(by(the(studies(under(consideraWon,(as(
well(as(the(reporWng(standards(that(have(been(
adhered(to.(
• Domain&2:(EssenWal(mHealth(criteria(–(Classifies(
the(studies(under(consideraWon(based(on(the(
quality(of(informaWon(presented(about(the(
mHealth(intervenWon.((
Grading&quality&of&evidence

14. Two Initial Areas Selected
Do mHealth strategies:
! improve provider adherence to care and
treatment guidelines ?
! reduce stockouts of essential maternal health
drugs and commodities?
PROVIDER(
ADHERENCE(
STOCKOUTS(

15. Provider Competence,
Accountability,
Effectiveness.
Client Knowledge
and Self-Efficacy
Availability of
Commodities, Health
Workers, Equipment
Improved Information
about individuals,
populations, providers,
facilities, outcomes,
Improved
HEALTH
Outcomes
Improved
Quality
of Care
Improved
Health
Behaviors
Improved
Efficiency +
Coverage
Provider
Client
System
Remote Client to Provider (Telemedicine)
Provider Training and Education
Provider Work Planning and Scheduling
Disease Prevention
Provider to Provider Communication
Human Resource Management
Electronic Decision Support
Appointment Reminders
Treatment Adherence
Mass Messaging Campaigns
Health Education or Promotion
Hotlines and Information Services
Registries/Vital Events Tracking
Insurance
Payment for Services
Savings Accounts
Electronic Health Records
Data Collection and Reporting
Point of Care Diagnostics
Disease Management
Referrals
Cold Chain Management
Performance Based Incentives
Stock Out Prevention
Counterfeit Prevention
Maintenance of Equipment
Commodity Tracking

16. Provider Competence,
Accountability,
Effectiveness.
Client Knowledge
and Self-Efficacy
Availability of
Commodities, Health
Workers, Equipment
Improved Information
about individuals,
populations, providers,
facilities, outcomes,
Improved
HEALTH
Outcomes
Improved
Quality
of Care
Improved
Health
Behaviors
Improved
Efficiency +
Coverage
Provider
Client
System
Remote Client to Provider (Telemedicine)
Provider Training and Education
Provider Work Planning and Scheduling
Disease Prevention
Provider to Provider Communication
Human Resource Management
Electronic Decision Support
Appointment Reminders
Treatment Adherence
Mass Messaging Campaigns
Health Education or Promotion
Hotlines and Information Services
Registries/Vital Events Tracking
Insurance
Payment for Services
Savings Accounts
Electronic Health Records
Data Collection and Reporting
Point of Care Diagnostics
Disease Management
Referrals
Cold Chain Management
Performance Based Incentives
Stock Out Prevention
Counterfeit Prevention
Maintenance of Equipment
Commodity Tracking
PROVIDER(
ADHERENCE(

17. Search Strategy
PROVIDER(
ADHERENCE(
Ar#cles(iden#fied(through(database(
search(and(use(of(search(engines(using(a(
combina#on(of(search(terms(
n=#1,499#
Poten#ally(eligible(ar#cles(remaining(
n#=#1,459#
Titles(and(abstracts(of(1,459(poten#ally(
eligible(ar#cles(screened(
Applica#on(of(inclusion(and(exclusion(
criteria(to(1,459(ar#cles(
53#ar,cles#tagged#using#the#taxonomy#
tool#
21#ar,cles#graded#using#the#grading#
tool#
1,406#ar,cles#excluded#
• 161(were(not(mobile(health(
related((
• 1,117(did(not(meet(health(
domain(criteria(
• 105(did(not(meet(mHealth(
applica#on(criteria(
• 11(did(not(meet(target(user(
criteria(
• 12(only(briefly(men#oned(the(use(
of(mHealth(for(decision(support((
#
Iden#fica#on(Screening(Eligibility(Inclusion(
40#duplicate#ar,cles#
removed#
#
32#ar,cles#did#not#meet#grading#
criteria#
• 7(did(not(provide(descrip#on(of(
evalua#on(of(mHealth(program(
• 2(did(not(provide(descrip#on(of(
implementa#on(of(mHealth(
program((
• 13(did(not(provided(descrip#on(
of(evalua#on(or(implementa#on(
of(mHealth(program(
#

18. Completeness of description of methodology
Completeness of description of technology, intervention
Quality of Information
PROVIDER(
ADHERENCE(

19. ! Domain(1(score(ranged(from(5(to(37(points(out(of(a(
maximum(47apoint(score((11(to(79%)m(suggesWng(large(
variaWons(in(quality(and(methodology(of(reporWng(
(
! Only(three(of(the(21(graded(studies(achieved(a(score(
>75%(for(Domain(2((essenWal(mHealth(criteria),(which(
indicates(that(most(studies(do(not(adequately(describe(
the(mHealth(intervenWon(they(are(reporWng(on.((
&
Can mHealth strategies be used to improve
provider adherence to care and treatment
guidelines?
PROVIDER(
ADHERENCE(

21. ! The studies that were methodologically sound showed significant
improvement of HW performance when using mHealth
Examples:
o In(South(Africa,(Rhode(and(colleagues(randomized(24(primary(care(nurses(to(
be(trained(in(IMCI(using(tradiWonal(paperabased(guidelines,(and(concluded(that(
nurses(who(used(the(IMCI(decisionasupport(tool(were(significantly(more(likely(
to(make(an(accurate(diagnosis((p<0.001).(
o (In(rural(Kenya,(Zurovac(et(al(randomized(outpaWent(health(workers(with(
mobile(phones(to(receive(text(message(reminders(about(naWonal(pediatric(
malaria(guidelines(to(improve(and(sustain(protocol(adherence.(For(health(
workers(who(received(the(SMS(reminders,(correct(management(of(pediatric(
malaria(increased(by(almost(25%(at(both(6(months(and(12(months((
! The use of mHealth varied: e-training, peer to peer, SMS reminders,
decision support
PROVIDER(
ADHERENCE(

22. Two Initial Areas Selected
Do mHealth strategies:
! improve provider adherence to care and
treatment guidelines ?
! reduce stockouts of essential maternal health
drugs and commodities?
PROVIDER(
ADHERENCE(
STOCKOUTS(

23. Provider Competence,
Accountability,
Effectiveness.
Client Knowledge
and Self-Efficacy
Availability of
Commodities, Health
Workers, Equipment
Improved Information
about individuals,
populations, providers,
facilities, outcomes,
Improved
HEALTH
Outcomes
Improved
Quality
of Care
Improved
Health
Behaviors
Improved
Efficiency +
Coverage
Provider
Client
System
Remote Client to Provider (Telemedicine)
Provider Training and Education
Provider Work Planning and Scheduling
Disease Prevention
Provider to Provider Communication
Human Resource Management
Electronic Decision Support
Appointment Reminders
Treatment Adherence
Mass Messaging Campaigns
Health Education or Promotion
Hotlines and Information Services
Registries/Vital Events Tracking
Insurance
Payment for Services
Savings Accounts
Electronic Health Records
Data Collection and Reporting
Point of Care Diagnostics
Disease Management
Referrals
Cold Chain Management
Performance Based Incentives
Stock Out Prevention
Counterfeit Prevention
Maintenance of Equipment
Commodity Tracking

24. Provider Competence,
Accountability,
Effectiveness.
Client Knowledge
and Self-Efficacy
Availability of
Commodities, Health
Workers, Equipment
Improved Information
about individuals,
populations, providers,
facilities, outcomes,
Improved
HEALTH
Outcomes
Improved
Quality
of Care
Improved
Health
Behaviors
Improved
Efficiency +
Coverage
Provider
Client
System
Remote Client to Provider (Telemedicine)
Provider Training and Education
Provider Work Planning and Scheduling
Disease Prevention
Provider to Provider Communication
Human Resource Management
Electronic Decision Support
Appointment Reminders
Treatment Adherence
Mass Messaging Campaigns
Health Education or Promotion
Hotlines and Information Services
Registries/Vital Events Tracking
Insurance
Payment for Services
Savings Accounts
Electronic Health Records
Data Collection and Reporting
Point of Care Diagnostics
Disease Management
Referrals
Cold Chain Management
Performance Based Incentives
Stock Out Prevention
Counterfeit Prevention
Maintenance of Equipment
Commodity Tracking
STOCKOUTS(

25. Search Strategy STOCKOUTS(

26. Completeness of description of methodology
Completeness of description of technology, intervention
STOCKOUTS(Quality of Information

28. • (Overall,(a(majority(of(the(evidence(was(
anecdotal(and(the(evidence(to(suggest(
effecWveness(of(use(of(mHealth(strategies(to(
prevent(stockaouts(is(weak(
• (A(number(of(projects(are(being(currently(
implemented((but(have(limited(published(
results(on(effecWveness(
The Evidence Gap
STOCKOUTS(

29. The Evidence Gap - Overall
! Across both domains, there are many interventions but
most are incompletely described
! Almost none have design that allow rigorous evaluation
! Almost none have control groups
! It is impossible to compare or combine study results due
to lack of standardization
! The nature of the exact interventions differs
! Differing indicators make meta-analyses difficult(
PROVIDER(
ADHERENCE(
STOCKOUTS(