MobileActive08 presentation: Metrics and Evaluation

870 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
870
On SlideShare
0
From Embeds
0
Number of Embeds
9
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

MobileActive08 presentation: Metrics and Evaluation

  1. 1. MobileActive 08: Mobile metrics and Evaluation Patty Mechael Peter Benjamin
  2. 2. <ul><li>OUTLINE OF THE SESSION </li></ul><ul><li>Introductions of all </li></ul><ul><li>Metrics & evaluation framework </li></ul><ul><li>Case study: “Mobile 4 HIV” @ SA </li></ul><ul><li>Group work </li></ul><ul><li>Report back </li></ul><ul><li>Next steps  Mobile Metrics Work Group? </li></ul>
  3. 3. Introductions <ul><li>Name </li></ul><ul><li>Organisation </li></ul><ul><li>Interest in mobile metrics? (Research question, techniques, sector) </li></ul>
  4. 4. Counting SMS is not evaluation! <ul><li>Input - Budget, tech, people … </li></ul><ul><li>Output - Usage, User info, TGI, SMS … </li></ul><ul><li> Outcomes - Action from info </li></ul><ul><li> Impacts - Wider social change </li></ul>
  5. 5. Case study: “Mobile 4 HIV” <ul><li>Project in SA to use mobile for range of services for HIV sector </li></ul><ul><li>Working with HIV-related organisations: Treatment Action Campaign, Soul City, DTHF, HIVAN, SANAC </li></ul><ul><li>Low costs, language, low-tech, local </li></ul><ul><li>Pilots, content, develop tech, research </li></ul>
  6. 6. Start with stated aim of the project – don’t be blinded by the tech. Adherence indictor, Clinic stats, patient QoC Adherence Loss to follow-up Patient – clinic link Adherence indicator, focus group. Uptake of services Improved positive living, adherence Messaging: Treatment Behaviour change indic. Self-report. Uptake of services Ideally – reduced incidence. Behaviour change Messaging: Prevention Eval Tools (e.g.) Goal Area
  7. 7. Eval Tools (e.g.) Goal Area Self-perception of reduced isolation, Improved question response Reduced isolation, self-help, respond to need Peer-peer, councelling Chosen indicators (eg branch reports) Feeling of inclusion and organisational capacity Improved info flow inside an organisation Org capacity Length of time to complete, data validation techniques Improved data collection (data quality, faster) M&E
  8. 8. Cellphone as evaluation tool <ul><li>Also, cellphones can be used as a tool for evaluation (data collection) </li></ul><ul><li>Can random sample users and call / SMS back (ethics, opt-in and prior approval) </li></ul><ul><li>Fairly easy to do longitudinal and follow-up studies </li></ul><ul><li>Quiz on cellphones (esp SMS or USSD) for pre- and post-questions </li></ul>
  9. 9. Other questions <ul><li>Social meaning of messages (not neutral, language, literacy). Ethnography & sociology issues </li></ul><ul><li>Pulling out contribution of mobile to intervention (confounding variables) </li></ul><ul><li>Scientific studies: randomised, double blind, controlled </li></ul>
  10. 10. Group discussion <ul><li>Business case, ROI </li></ul><ul><li>Use of mobile as research tool </li></ul><ul><li>Assess impact, monitoring & eval </li></ul><ul><li>Topic areas: HIV, Health, water & an, education, youth, pricing, finance, universal birth registration </li></ul>

×