Preliminary results from a survey on the use of metrics and evaluation strategies among mHealth projects

by jehill3 on May 04, 2010

1,054 views

Preliminary results from a survey on the use of metrics and evaluation strategies among mHealth projects...

Preliminary results from a survey on the use of metrics and evaluation strategies among mHealth projects

Patricia Mechael, Nadi Kaonga
Center for Global Health and Economic Development at the Earth Institute, Columbia University

CORE Group Spring Meeting, April 30, 2010

Less

3 Embeds 57

http://mywebcollection.blogspot.com	48
http://www.slideshare.net	8
http://mywebcollection.blogspot.co.uk	1

Statistics

Favorites: 1
Downloads: 29
Comments: 0
Embed Views: 57
Views on SlideShare: 997
Total Views: 1,054

Preliminary results from a survey on the use of metrics and evaluation strategies among mHealth projects — Presentation Transcript

Preliminary results from a survey on the use of metrics and evaluation strategies among mHealth projects Garrett Mehl, Franz Allmayer, Heli Bathija Department of Reproductive Health and Research, WHO, Switzerland Patricia Mechael, Nadi Kaonga Center for Global Health and Economic Development at the Earth Institute, Columbia University

Survey Instrument

50 multiple choice and open ended questions

~ 15 minutes for each project

Survey topics

Health focus areas, strategic approaches

Specific objectives, types of monitoring, approach to evaluation

Project phase, duration, level of support

Self-identified areas of need

Survey Methodology

Jointly developed with Columbia U.

The survey is still being carried out.

Available as a web-survey or soft-copy for printing

Survey was announced to various electronic forums

Additionally, individuals were sent targeted emails.

Twitter

USA:5 Peru: 7 Haiti:2 Mexico: 3 India:6 Pakistan:4 Kenya:11 Uganda:10 South Africa:4 Tanzania:5 Ghana:3 Nigeria:5 Malawi: 2 Philipines:4 All countries highlighted contain an mHealth project covered by the survey. Countries with multiple projects are specified. Jordan Survey Reach

Project duration in sample

Projects by health focus

Current Phase of projects Current Phase Needs assessment 8.8% Usability testing 8.8% Pilot not for scaling 7.4% Pilot for scaling 57.4% Large scale implementation 17.6%

Frequency of project objectives

Client information

Health provider information

Increasing access to services

Improve service quality and/or safety

Increase client service demand

Increase time savings

Improve intra-provider communication

Increase provider skills

Reduce unneeded referrals

Increase patient treatment compliance

Reduce health service costs

Improve ability to respond to crises

Reduce stock-outs

Client focus Provider focus Efficiencies

mHealth Strategies

Clustering of mHealth objectives Stock Crisis response Time savings Reduce referral Communication Provider Skills Lower costs Quality/Safety Compliance Service Demand Service access Client info Provider info

Domains of Measurement

Measurement domains of focus

Level of monitoring

Focus on Monitoring and Evaluation

Focus of Evaluation Assessment Did the intervention result in improvements in:

Number of evaluation questions tracked Costs Sustainability Behavior changes Health outcomes Knowledge, attitudes Performance Quality of Care Service Utilization

Type of evaluation approach Descriptive 29.8% Cross-sectional 44.7% Longitudinal 40% Case-control 12.8% Wedge 8.5%

Rigor of Evaluation Design

Drivers of Monitoring Level of Monitoring Significance Performance Accounting correlation coefficient = .392 (spearman’s rho) sig. (2-tailed) .002 Number of funding sources correlation coefficient = .456 (spearman’s rho) sig. (2-tailed) .000

Project phase and monitoring level

Evaluation rigor by project start date

Evaluation rigor by project phase

Drivers of evaluation rigor Rigor of evaluation Significance Performance Accounting correlation coefficient = .501 (spearman’s rho) sig. (2-tailed) .000 Number of funding sources correlation coefficient = .392 (spearman’s rho) sig. (2-tailed) .001

Assistance requested

Assistance need by project phase

“ We need a systematic approach to analyzing the data we have collected over the past 3 years. “ Assistance Requested

“ We also need to learn what is the norm for "success" in this field and how we stack up to normal interventions vs. other mHealth projects working on [similar] technology.”

“ We need guidance on evaluation methods for mHealth”

“ We are interested in collaborative approaches and standard indicators that will be measured across the different mHealth programs.”

“ How to assess the impact of [our] mHealth tool.” mHealth tool.”

Thank you. For more information, or to submit your mHealth project to the survey, please send an email to: Dr. Garrett Mehl [email_address]

Preliminary results from a survey on the use of metrics and evaluation strategies among mHealth projects

by jehill3 on May 04, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 57

Statistics

Preliminary results from a survey on the use of metrics and evaluation strategies among mHealth projects — Presentation Transcript