sician order entry systems have been shown to ibility of all studies identified in our search. Areduce medical errors,15 but they can also in- second reviewer confirmed all relevant articlescrease error rates if not well designed and and retrieved full-text articles. Supplementaryimplemented.16 methods of finding evaluations included a review of article reference lists, informatics conference proceedings, information provided by primaryStudy Data And Methods study authors, requesting submissions fromSTUDIES ELIGIBLE FOR REVIEW In our survey of stud- other researchers and implementers, andies for review, we included any qualitative or searching the RHINO Literature Database20 andquantitative evaluation of information technol- other recent reviews.7,21–23ogy affecting health care in developing coun- DATA ABSTRACTION AND SYNTHESIS We extractedtries. We did not include telemedicine because data according to recurring themes, defined be-other recent reviews exist.9,17 Developing countries low.We summarized these findings using tabularwere defined as those in the Emerging and De- techniques and descriptive statistics. Reportedveloping Economies List in the International analyses were too disparate to be pooled in aMonetary Fund’s World Economic Outlook Report. meta-analysis.Evaluations were excluded if (1) data complete- The systems described in the articles wereness of the system was the only outcome, (2) the placed into one of eight categories correspond-evaluation method was not described, (3) the ing to the typical applications used in developingarticle only described the feasibility or technical countries. The order of these categories does notevaluation of a system, (4) the evaluation was on infer any priority:attitudes toward or knowledge of e-health (not (1) Electronic health record: an electronic rec-an actual system), or (5) it was only an educa- ord of health-related information on an indivi-tional tool.18,19 In the case of the Uganda Health dual that can be created, managed, or consultedInformation Network, we report on the e-health by clinicians or staff. In literature, the term elec-component of the system. If an article did not tronic medical record is used interchangeably andhave an abstract, we attempted to find the article is used as a synonym in this paper.through the Harvard or Massachusetts Institute (2) Laboratory information management sys-of Technology (MIT) library systems. tem: a system for laboratory-specific activities or FINDING RELEVANT STUDIES We conducted a for reporting results to administrators andworldwide review of the literature and requested health care personnel.submissions from researchers and those imple- (3) Pharmacy information system: any systemmenting e-health in developing countries. Lit- used to order, dispense, or track medications orerature searches were completed through Octo- medication orders including computerized or-ber 2009 without language restrictions through der entry systems.MEDLINE, EMBASE, Science Citation Index (4) Patient registration or scheduling system:(Web of Science), Social Sciences Citation Index, any system used to monitor and manage thethe Cochrane Library, and the Latin American movement of patients through multistep proc-and Caribbean Health Science Literature Data- esses or to maintain a census.24 An example isbase (LILACS). To find reports not in scientific admissions-discharge-transfer systems.journals or conferences, we also used Google (5) Monitoring, evaluation, and patient track-Scholar. For MEDLINE and EMBASE searches, ing system: any system used for aggregate report-terms were derived from the MeSH database and ing of information, program monitoring, andEMTREE tool, respectively. We searched for tracking of patients’ status. Examples includemore than forty commonly used terms to de- district health information systems or healthscribe e-health applications, found the broadest management information systems.term within each tool that maintained its con- (6) Clinical decision support system: systemtext, and then used that term for the search to designed to improve clinical decision making, inensure that we included all possible studies. which characteristics of individual patients areAmong the terms used in the final strategies were matched to a computerized knowledge base andmedical informatics applications, reminder system, software algorithms generate patient-specificgeographic information system, hospital informa- recommendations.25tion systems, outcome and process assessment (7) Patient reminder system: a system used to(Health Care), evaluation studies, attitude, costs prompt patients to perform a specific action—forand cost analysis, developing countries, poverty, example, take medications or attend the clinic.Africa, Latin America, eastern Europe, and central (8) Research/data collection system: any sys-or southeastern Asia (complete strategies are tem used for collecting data from different loca-available from the authors on request). An initial tions or for storing, managing, or reporting onreviewer read the abstracts to evaluate the elig- data used for research purposes. F E B R UA RY 2 0 1 0 29 : 2 HE A LT H A FFA IRS 245
POLICIES & POTENTIAL Evaluations were classified into two major and abstracts, we found 126 articles that ap- categories—qualitative and quantitative—as peared relevant. An additional five articles were shown in Exhibit 1. Qualitative evaluations were identified by hand-searching bibliographies of those where users gave opinions regarding a eligible articles and prior reviews. Of these, system. These could be through questionnaires, forty-five fulfilled the inclusion criteria after full focus groups, or interviews. (This definition is review of their abstracts. They are listed by type different from the one proposed by Anselm of system and evaluation in Exhibit 1 and are Strauss and Juliet Corbin of “any type of research categorized by systems in Appendix Exhi- that produces findings not arrived at by statisti- bits 2a–5a.32 We included an evaluation from cal procedures or other means of quantifica- the U.S. Indian Health Service, although it is tion.”)26 Quantitative evaluations were those not in a developing country, because socioeco- whose outcomes were data quality, administra- nomic and infrastructure conditions among the tive changes, patient care, or economic assess- population treated are similar to those in devel- ment. Evaluation designs were grouped accord- oping countries. If a system had multiple evalua- ing to the definition by Charles Friedman and tions, only those with different outcomes are Jeremy Wyatt:27(1) descriptive (uncontrolled) listed. If they had the same outcome, we took study; (2) historically controlled (before-after) the one with the largest sample size. There were study; (3) case-control (retrospective) study; two articles reporting an evaluation that did not (4) prospective self-controls (subjects perform- occur because of a failed system implementa- ing the same action in both systems; this cate- tion.33,34 These are not part of the results, but gory was added by the authors); (5) simultaneous we considered them relevant to list because ar- nonrandomized controls; (6) simultaneous ran- ticles on unsuccessful systems are not commonly domized controls; and (7) externally and intern- published. ally controlled before-after study. Two cost stud- Fifteen articles performed qualitative evalua- ies and two studies modeling future medication tions, and forty performed quantitative evalua- requirements were categorized as self-controls tions. If an evaluation performed both types, it because they compared the impact of the system was counted in both categories. Two qualitative against the same situation without the system. evaluations and sixteen quantitative performed As a result of the inherent limitation of perform- statistical analysis. Of all evaluations, two ing a case-control, descriptive, or qualitative (13 percent) of the qualitative and seven (18 per- study without statistics, we do not comment cent) of the quantitative were performed by an on the limitations of these studies. outside evaluator. The number of evaluations has more than tripled comparing periods before and after 2002. Study Results ELECTRONIC HEALTH RECORDS Because EHRs are Searches retrieved 2,043 citations. Five articles the core clinical application, they usually encom- were excluded because they did not have ab- pass a variety of functionalities, which makes stracts and full-text versions were not avail- their implementations complex35 and prone to able.28–31 After the initial screening of article titles failure.36 This complexity provides an additional EXHIBIT 1 Number Of Articles Included In Analysis, By E-Health Category And Evaluation Type Quantitative E-health category Qualitative Descriptive studies Controlled studies Electronic health record 5 1 5 Laboratory information management systems 0 1 2 Pharmacy information systems 4 2 3 Patient registration or scheduling systems 1 0 2 Monitoring, evaluation, and patient tracking systems 0 2 4 Clinical decision support systems 1 0 3 Patient reminder systems 0 1 3 Research/data collection systems 5 1 11 Total 15 8 32 SOURCE Authors’ analysis. NOTES The articles (n ¼ 45) are classified by e-health category and by type of evaluation. If an article had both qualitative and quantitative studies or multiple types of systems, it was counted in both categories. Details about the evaluated projects are in Appendix Exhibits 2a–5a, available online as in Note 32.246 HEA LT H AF FA IR S F E B R UA RY 2 0 1 0 29:2
challenge in their evaluation. Most evaluations in training and technical support and the need tofound provided insight into possible impacts of maintain a parallel paper system.these systems, but had limited scientific rigor, as MONITORING , EVALUATION , AND PATIENT TRACKINGseen in Appendix Exhibit 2a.32,27 SYSTEMS Evaluations of systems to track and The Indian Health Service’s Vista system was monitor patients’ status are limited to twothe most complete system we reviewed, and its case-control studies performed by the same or-rigorous qualitative evaluation showed that a ganization in Haiti (Appendix Exhibit 4a).32majority of clinicians viewed its implementation Both of these studies suggest that an electronicpositively and hence used it more. The Mosoriot system can effectively alert staff of patients whoMedical Record System evaluation in Kenya pro- have “fallen through the cracks” and preventvides data on the impact that an EHR can have on rates of patients lost to follow-up, which wereimproving staff productivity and reducing pa- found to be as high as 76 percent (after twotient wait times. All other evaluations were qual- years) as reported in some HIV programs.3itative and provided insights into EHRs’ ability Two randomized controlled trials looked atto improve staff satisfaction, providing higher- the effect of Global Positioning Systems (GPS)quality data to relevant personnel and ultimately in finding households once a patient has beenimproving patient care. identified. An evaluation from South Africa LABORATORY INFORMATION MANAGEMENT SYSTEMS showed that GPS reduced the time to find aThere were only three evaluations of laboratory household by 20–50 percent, whereas one frominformation management systems, all quantita- Nicaragua showed no difference between the pa-tive, with only one having a control group (Ap- per and GPS systems. Both the South African andpendix Exhibit 3a).32 However, they suggest two Nicaraguan systems were tested in similar urbanmajor benefits that such systems can provide: settings with novice users, so no immediate(1) decreasing times for communication of re- reason for the difference can be found. Bothsults, and (2) improving the productivity of the studies had small sample sizes (identifyinglaboratory. An additional impact, reduction in ten to fifty households) and lacked statisticalerrors, has not yet been studied, although there analysis.are groups currently performing such trials.37 Two evaluations, one descriptive and one cost PHARMACY INFORMATION SYSTEMS Computerized analysis, looked at monitoring departmentsorder entry can provide a key incentive for clin- within a hospital in Cambodia and health estab-ical staff, especially clinicians, to use an informa- lishments nationwide in Tanzania. They suggesttion system, because such systems can reduce the that electronic systems can help allocate re-time to order medications (especially repeat or- sources efficiently and improve infection controlders) and provide easy access to past informa- and can be relatively low cost, respectively. How-tion. The four qualitative evaluations shown in ever, both evaluations lacked detail on the tasksAppendix Exhibit 3a32 cite these as their system’s affected, as well as control groups.main advantages. The two quantitative evalua- CLINICAL DECISION SUPPORT SYSTEM Decisiontions with a control group (Socios en Salud in support systems have received attention for de-Peru and Hamadan University of Medical veloping countries as a possible solution to theSciences in Iran) showed a reduction in errors, lack of trained clinical personnel, especially inwhich is a main outcome cited in developed rural areas. The three quantitative evaluationscountry studies. An additional benefit from some seen in Appendix Exhibit 4a32 were of high rigor.pharmacy systems in developing countries is The expert system for mechanically ventilatedtheir ability to forecast medication requirements newborns showed that nurses performed better(Socios en Salud in Peru). This is useful if a on a standardized test and felt that they hadcountry or organization needs to order medica- better judgment after receiving training on thetions months in advance to get lower prices, system. The evaluation of the personal digitalwhich is currently the case for drug-resistant assistant (PDA) device to perform the ElectronicTB medications. Integrated Management of Childhood Illness ap- PATIENT REGISTRATION AND SCHEDULING The two proach in Tanzania showed that more clinicalquantitative evaluations of registration systems, staff completed the electronic questionnaireseen in Appendix Exhibit 4a,32 showed that fin- compared to the paper booklet. It also showedgerprint scanners and barcode readers de- that it took the same amount of time (12.5 min-creased the time to locate records by 74 percent utes) to fill out the questionnaire by either meth-and 97 percent, respectively. The small sample od. The evaluation of the Early Diagnosis andsize of thirty in these randomized controlled Prevention System in India showed higher satis-trials was their biggest limitation. In the quali- faction among patients if they were seen by atative evaluation of the Baobab system in Mala- computer operator before their clinical visitwi, users preferred it to paper despite limitations and that there was a large increase in new pa- F E B R UA RY 2 0 1 0 2 9 :2 HE A LT H A FFA IR S 247
POLICIES & POTENTIAL tients at health centers with the system. compared the PDA system to paper and not to However, the two studies with simultaneous a gold standard. The study performed by Socios controls had major limitations. The evaluation of en Salud had a small number of users (n ¼ 4), the Electronic Integrated Management of Child- and the study performed by the London School of hood Illness was performed by the developers of Economics was performed seventeen years ago. the systems, and because the technology was The organizations that implemented the PDA- new to the users, the novelty rather than its use- based systems in Uganda and South Africa have fulness could account for the additional comple- experience with hundreds of users and more teness. In the case of the Early Diagnosis and than a dozen implementations combined, which Prevention Systems, the increased attendance empirically shows the feasibility of such systems. and patients’ opinions could have been easily The cost analyses show that these systems are biased by the presence of the computers, the able to recoup the high initial costs by providing motivation of computer operators, and the increased efficiency and continuous material length of time spent with operator, none of costs. The Uganda system showed a cost savings which were present at control sites. of 91 percent over the paper system. The South PATIENT REMINDER SYSTEMS The quantitative African analysis calculated that after using the evaluation of the South African text messaging PDA system for data collection in eight studies of system (Appendix Exhibit 5a)32 found that after medium scale, it would equal the costs of paper. the system was implemented, there were higher The PDA system in Peru would pay for expansion completion rates of TB treatment. However, the to other health districts in three months as a comparison was made to the city’s TB program result of increased efficiency. register, for which the data quality was not ver- ified and the data were different from the source of the prospective data. A randomized trial in Discussion Malaysia found that both text messaging and This review shows that with the exception of mobile phone reminders significantly increased PDA-based data collection, there are still few attendance (by 21 percent) over the control scientifically rigorous data on the effectiveness group. Although they both had similar effective- and cost-effectiveness of e-health systems in de- ness, the text messaging system was half the cost veloping countries. Further, the evaluations of the mobile phone reminders. This evaluation have mostly been performed by organizations had no major limitations. connected to academic settings and not by other, The Malaysian study performed a well- larger recipients of donor funding.When looking designed cost-effectiveness study showing that at the software systems included in the U.S. Pres- text messaging, implemented correctly, can be a ident’s Emergency Plan for AIDS Relief (PEP- cost-effective method to increase clinic atten- FAR) Anti-Retroviral Therapy (ART) Software dance. This is especially important since both Inventory Report5 and EngenderHealth–Open- TB and HIV treatments require constant super- Society software tools38 comparison, only three vision of patients and strict adherence to a daily systems, the Partners in Health—Electronic regimen of medications. Such systems can help Medical Record/HIV—Electronic Medical Rec- patients in resource-poor settings who encoun- ord in Kenya, Mosoriot Medical Record System ter many obstacles that can prevent them from in Kenya, and Vista in the U.S. Indian Health getting their medications. Service, have had any evaluations performed. RESEARCH / DATA COLLECTION SYSTEMS Research/ Although a few studies have been commissioned data collection systems was the group with the by the U.S. Centers for Disease Control and Pre- largest number and most rigorous evalua- vention (CDC), it is particularly important that tions (Appendix Exhibit 5a).32 All systems, ex- large funders such as the U.S. Agency for Inter- cept the Mexican National Institute of Public national Development or PEPFAR include re- Health’s Audio Computer-Assisted Self-Inter- sources for the evaluation of e-health systems view (ACASI) system, were on PDAs. Four ran- developed and deployed in developing countries domized trials showed that the main benefits of and perhaps make them a requirement for con- PDA-based systems were data qual- tinued funding. This could include ity similar to paper systems or high- standard designs for studies that er, less time taken to perform inter- are suitable for resource-poor en- views, and decreased time to collect vironments, that minimize biases, data. However, many of the studies and that are easily comparable to had major limitations. The systems the results from other projects. from the Universidad Peruana The overall pattern of e-health Cayetano Heredia and the South evaluations in developed countries African Medical Research Council reflects an initial focus on qualita-248 HE A LT H A FFA IRS F E B R UARY 2 0 10 2 9 :2
tive and descriptive evaluations, with an increase health and cell phone–based tools, because thesein the number of quantitative and larger evalua- devices are also playing an increasing role intions published in the past decade. Developing communication directly with patients.countries seem to be following this pattern as Evaluations of e-health systems are chal-well, so in this study we found mostly qualitative lenging and require significant resources in ad-and descriptive studies but saw an increase in the dition to funds creating and implementing sys-number of randomized trials performed in the tems. Implementations should have evaluationspast few years. This suggests that as e-health built into the process. This will provide usefulimplementations become more robust in devel- feedback to improve the project (formative eval-oping countries, we can expect more rigorous uations) and will also demonstrate the impact ofstudies, such as randomized trials or cost-effec- the system in the long term (summative evalua-tiveness studies. tions). Evaluations in resource-poor environ- Initial evaluations suggest that the following ments face many challenges when compared tofunctions are of positive impact in developing those in developed countries, such as the physi-countries: cal environment, power, networking, and avail- (1) Ability to track patients through the treat- ability of technical staff. Measures of short- andment initiation process, monitor adherence, and long-term system usage and data completenessdetect those at risk for loss to follow-up. (2) Tools are important and a necessary prerequisite to ato decrease communication times of information full evaluation study. Poor data quality is a con-within and between institutions. (3) Tools to stant problem in health projects, whether theylabel or register samples and patients. (4) Ability use paper or electronic systems, so tools that canto electronically monitor and remind patients of reduce errors as well as benefiting other aspectshealth care needs or treatment. (5) Collection of of care are likely to be well received.clinical or research data using PDA applications. Some benefits of electronic systems are diffi-(6) Reductions in errors in laboratory and med- cult to quantify. One is the ability to performication data. operational research with greatly reduced costs. Important findings include the user prefer- During our search we found eight studies thatence for the Baobab touch-screen system in used electronic databases and probably couldMalawi, one of the only fully electronic point- not have been performed if manual data collec-of-care systems in use in Africa, which is now tion was required. Another is the impact of emer-in daily use for more than 35,000 HIV patients. gency communication across large distances,The benefit shown for patient tracking and such as in the cholera outbreak in India or refu-reminders is also important, given the loss to gee situations.39 The strongest evidence for ben-follow-up rate of up to 76 percent for HIV pa- eficial impact of e-health on health care will cometients in Africa.3 The Malaysian systems that from long-term follow-up of this sort carried outtexted patient reminders showed a significant by independent evaluators.decrease in missed visits, at a reasonably lowcost, and the On Cue Compliance Service inSouth Africa was well liked by users several years Conclusionsafter implementation and, perhaps more impor- With the rapid growth of e-health in developingtant, by an independent evaluation team. These countries, there is clearly an urgent need forsystems can be of high value because intermit- solid evidence of its impact to justify and guidetent treatment puts patients at grave risk of the investment of resources in such systems.deterioration and death, as well as causing in- Despite major increases in evaluations in recentcreased drug resistance and further transmis- years, most large e-health implementations havesion of disease to the wider community. little or no evaluation data. To date, most studies Tools to store and communicate such data with have been small; focused on process indicatorslow error rates have been early successes in de- rather than patient outcomes, or on the attitudesveloped countries, and the positive evaluations of users and patients; and performed mostly bydescribed here should drive their use in the de- academic groups. An increased focus on includ-veloping world. Evaluations of PDAs and mobile ing evaluations as part of e-health implementa-devices were particularly rigorous, and they con- tions is necessary and should be adopted by or-vincingly demonstrate that such devices can be ganizations implementing or funding suchvery effective in improving data collection time systems. One method is for large funders to in-and quality. An additional benefit is their light clude resources for evaluations or make them aweight and lack of printing costs compared to requirement for implementation.large paper forms, which is crucial in remote Although evaluations of important indicatorsareas with poor infrastructure. These results of care are difficult to do well, this review hasare important for the growing field of mobile confirmed that they are feasible even in very F EB R UARY 2 0 1 0 29:2 H E ALT H AF FAI RS 249
POLICIES & POTENTIAL challenging environments. Initial benefits were medications. Because of the lack of infrastruc- shown in systems that track patients through ture and backup systems in resource-poor envir- treatment initiation, monitor adherence, and de- onments, well-designed e-health solutions may tect those at risk for loss to follow-up; tools to have a much larger impact on quality of care than decrease information communication times in more developed areas. As e-health becomes within and between institutions, as well as errors widespread in developing countries, these and in reporting laboratory data; barcoding for pa- other benefits will need to be identified by more tient identification cards and laboratory sam- rigorous evaluations that include long-term ples; handheld devices for collecting and acces- follow-up and are carried out by independent sing data; and the ordering and management of evaluators. ▪ An initial version of this paper was Chilean company that provides health Veronica Rojas, Adesina Iluyemi, requested by the Rockefeller Foundation informatics consulting and technology in Mauricio Soto, Waldo Ortega, Chris for the Making the eHealth Connection Latin America. The authors acknowledge Bailey, Patrick Whitaker, Gerry Douglas, conference held in Bellagio, Italy, in July those who took the time to provide Natasha Kanagat, Steve Yoon, Zach 2008. This paper was funded by the additional information: Holly Ladd and Landis Lewis, Joel Selanikio, and Neal Rockefeller Foundation. Joaquin A. Blaya Berhane Gebru from AED-Satellife, Lesh. Finally, the authors thank Claire is cofounder of eHealth Systems, a Libby Levison, Heather Zornetzer, Mack for her invaluable editing. NOTES 1 World Health Organization. 58th tively collected data. BMC Med In- lingual online physician education World Health Assembly Report; 16– form Decis Mak. 2007;7(1):38. about electronic medical records. 25 May 2005. Geneva: WHO; 2005. 12 Garrido T, Jamieson L, Zhou Y, AMIA Annu Symp Proc. 2005:946. 2 Edworthy SM. Telemedicine in de- Wiesenthal A, Liang L. Effect of 20 Routine Health Information Net- veloping countries. BMJ. 2001;323 electronic health records in ambu- work. RHINO Literature Database (7312):524–5. latory care: retrospective, serial, [Internet]. Boston (MA): Routine 3 Rosen S, Fox MP, Gill CJ. Patient cross sectional study. BMJ. Health Information Network retention in antiretroviral therapy 2005;330(7491):581. (RHINO); 2008 [cited 2010 Jan 4]. programs in sub-Saharan Africa: a 13 Wang SJ, Middleton B, Prosser LA, Available from: http://www systematic review. PLoS Med. 2007 Bardon CG, Spurr CD, Carchidi PJ, .iphealth.info/refbase/index.php Oct 16;4(10):e298. et al. A cost-benefit analysis of elec- 21 Fraser HS, Biondich P, Moodley D, 4 World Health Organization. Elec- tronic medical records in primary Choi S, Mamlin BW, Szolovits P. tronic health records: a manual for care. Am J Med. 2003;114 Implementing electronic medical developing countries. Geneva: (5):397–403. record systems in developing coun- WHO; 2007. 14 Evans DC, Nichol WP, Perlin JB. tries. Inform Prim Care. 2005;13 5 U.S. President’s Emergency Plan for Effect of the implementation of an (2):83–95. AIDS Relief. PEPFAR Software In- enterprise-wide electronic health 22 Forster M, Bailey C, Brinkhof MW, ventory Report. Washington (DC): record on productivity in the Veter- Graber C, Boulle A, Spohr M, et al. PEPFAR; 2004. ans Health Administration. Health Electronic medical record systems, 6 Rigby M. Impact of telemedicine Econ Policy Law. 2006;1(Pt data quality and loss to follow-up: must be defined in developing 2):163–9. survey of antiretroviral therapy pro- countries. BMJ. 2002;324 15 Bates DW, Teich JM, Lee J, Seger D, grammes in resource limited set- (7328):47–8. Kuperman GJ, Ma’Luf N, et al. The tings. Bull World Health Organ. 7 Tomasi E, Facchini LA, Maia MF. impact of computerized physician 2008;86(12):939–47. Health information technology in order entry on medication error 23 Fraser HS, Allen C, Bailey C, Douglas primary health care in developing prevention. J Am Med Inform Assoc. G, Shin S, Blaya J. Information sys- countries: a literature review. Bull 1999;6(4):313–21. tems for patient follow-up and World Health Organ. 2004;82 16 Koppel R, Metlay JP, Cohen A, chronic management of HIV and (11):867–74. Abaluck B, Localio AR, Kimmel SE, tuberculosis: a life-saving technology 8 Mitchell E, Sullivan F. A descriptive et al. Role of computerized physician in resource-poor areas. J Med In- feast but an evaluative famine: sys- order entry systems in facilitating ternet Res. 2007;9(4):e29. tematic review of published articles medication errors. JAMA. 2005;293 24 Shortliffe EH, Perreault LE, editors. on primary care computing during (10):1197–203. Medical informatics: computer ap- 1980–97. BMJ. 2001;322 17 Hersh WR, Hickam DH, Severance plications in health care and bio- (7281):279–82. SM, Dana TL, Pyle Krages K, Helfand medicine. 2nd edition. New York 9 Roine R, Ohinmaa A, Hailey D. As- M. Diagnosis, access, and outcomes: (NY): Springer; 2001. sessing telemedicine: a systematic update of a systematic review of 25 Garg AX, Adhikari NK, McDonald H, review of the literature. CMAJ. telemedicine services. J Telemed Rosas-Arellano MP, Devereaux PJ, 2001;165(6):765–71. Telecare. 2006;12(Suppl 2):S3–31. Beyene J, et al. Effects of compu- 10 Whitten PS, Mair FS, Haycox A, May 18 Mallapaty G, Kim S, Astion ML. terized clinical decision support CR, Williams TL, Hellmich S. Sys- Using interactive software to teach systems on practitioner performance tematic review of cost effectiveness image-based clinical laboratory tests and patient outcomes: a systematic studies of telemedicine interven- in developing countries: a pilot trial review. JAMA. 2005;293 tions. BMJ. 2002;324(7351):1434–7. in Nepal. Clin Chem Lab Med. (10):1223–38. 11 Pollak VE, Lorch JA. Effect of elec- 2003;41(5):711–3. 26 Strauss A, Corbin J. Basics of quali- tronic patient record use on mor- 19 Edmonson SR, Esquivel A, tative research: grounded theory tality in end stage renal disease, a Mokkarala P, Johnson CW, Phelps procedures and techniques. New- model chronic disease: retrospective CL. Using technology to teach tech- bury Park (CA): Sage; 1990. analysis of nine years of prospec- nology: design and evaluation of bi- 27 Friedman CP, Wyatt JC. Evaluation250 H E A LT H A FFA IRS F E B R UA RY 2 01 0 2 9 :2
methods in medical informatics. 2nd online; see the Appendix Exhibits cess, and local improvisations. In- edition. New York: Springer- link in the box to the right of the formation Society. 2002;18 Verlag; 2005. article online. (2):101–12.28 Halbwachs H. The technical and fi- 33 Littlejohns P, Wyatt JC, Garvican L. 37 Blaya JA, Shin SS, Yagui MJ, Yale G, nancial impact of systematic main- Evaluating computerised health in- Suarez CZ, Asencios LL, et al. A Web- tenance and repair services within formation systems: hard lessons still based laboratory information system health systems of developing coun- to be learnt. BMJ. 2003;326 to improve quality of care of tuber- tries. Health Estate. 1999;53(4):6– (7394):860–3. culosis patients in Peru: functional 8,10–1. 34 Iluyemi A, Briggs J, Fitch T. Elec- requirements, implementation, and29 Desikan P, Koram MR, Trivedi SK, tronic health records in developing usage statistics. BMC Med Inform Jain A. An evaluation of the effec- countries, integrating with mobile Decis Mak. 2007;7:33. tiveness of the laboratory informa- technology and legacy systems for 38 EngenderHealth–Open Society In- tion system (LIS) with special refer- community based health workers: stitute. Health toolkit: information ence to the microbiology laboratory. organisational and end-users’ issues. management challenges and oppor- Indian J Path Microbiol. Proceedings of the European Con- tunities for community-based orga- 2005;48(3):418. ference on Information Manage- nizations serving people living with30 Janecki J, Podsiadly T. Computer- ment and Evaluation. Montpellier, HIV/AIDS. New York: Engender- assisted analysis of patients’ medical France; 2007 Sep 20–21. Health–Open Society Insti- records. Pol Tyg Lek. 1992;47(20– 35 Brender J, Ammenwerth E, Nykanen tute; 2004. 21):470–2. P, Talmon J. Factors influencing 39 Babille M, Decolombani P, Guerra R,31 Swaminathan R, Black RJ, success and failure of health infor- Zagaria N, Zanetti C. Post- Sankaranarayanan R. Database on matics systems—a pilot Delphi emergency epidemiological surveil- cancer survival from developing study. Methods Inf Med. 2006;45 lance in Iraqi-Kurdish refugee camps countries. IARC Sci Publ. 1998; (1):125–36. in Iran. Disasters. 1994;18(1):58–75. (145):19–25. 36 Heeks R. Information systems and32 The Appendix Exhibits are available developing countries: failure, suc- ABOUT THE AUTHORS Africa, and Asia. Blaya, 31, implementing an electronic Institute in the United United Kingdom. He also who was born in Chile, is a health record for use in States, the Medical completed a fellowship in Harvard and Massachusetts managing multidrug-resistant Research Council in South clinical decision making and Institute of Technology TB patients in Peru. He and Africa, and others, have cardiology at MIT and the (MIT)–trained Ph.D. in health Blaya teamed up to produce developed an “open source,” New England Medical Center. sciences and technology. a Palm Pilot–based system or nonproprietary, electronic Blaya, who today is a Fraser, age 47, was born in to collect laboratory results health record system for research fellow at Partners Scotland and was educated on behalf of these patients. developing countries, called in Health, is also a National and trained in medicine and In a study published in 2009 OpenMRS. The system is Library of Medicine Fellow Joaquin A. Blaya cardiology in the United in the International Journal used by more than forty-five at Harvard Medical School. Kingdom. They met in 2004 of Infectious Diseases, the organizations in twenty- In addition, he recently when Blaya was at a joint system was shown to three countries and is cofounded a company, Harvard-MIT program decrease delays in getting available for download at eHealth Systems, which aims working on his Ph.D. and those results from thirty http://www.openmrs.org. to implement open-source Fraser became his days to eight days, and to “My focus has been on technologies, including supervisor. Then, as now, reduce errors in the practical systems that are OpenMRS, in health systems Fraser was an assistant communication of these useful for doctors and other in Latin America. Having professor of medicine at tests to clinicians by 59 health care staff,” says emigrated from Chile to Harvard Medical School and percent. Fraser, who is also an Miami, Florida, twenty-two director of informatics and Since then, the two have associate physician at the years ago, he plans to move telemedicine at the worked on implementing a Brigham and Women’s back to Chile in 2010. His nonprofit organization Web-based system to Hospital in Boston. In five-year goal is for a Partners in Health, which communicate laboratory addition to his medical majority of public health Hamish Fraser focuses on providing health results to TB clinicians in degree, he trained in the centers in Chile to use Coauthors and frequent care for the poor in a more than 220 health development and use of so- OpenMRS and to expand collaborators Joaquin Blaya number of developing centers throughout Peru. called knowledge-based their use in Nicaragua, and Hamish Fraser share a countries, including Haiti, Fraser’s group (the systems—computer systems Argentina, Brazil, and other passion for using e-health Rwanda, and Peru. Electronic Medical Records to diagnose and analyze countries. technologies to improve Back then, Fraser was Team at Partners in Health), real-world data—at health care in Latin America, working on developing and with the Regenstrief Edinburgh University in the F E B R UA RY 2 0 1 0 2 9 :2 HE A LT H A FFA IRS 251
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50.APPENDIX Exhibit 1a: Additional References[1a] Merrell RC, Merriam N, Doarn C. Information support for the ambulant health worker.Telemed J E Health. 2004 Winter;10(4):432-6.[2a] Singh AK, Kohli M, Trell E, Wigertz O, Kohli S. Bhorugram (India): revisited. A 4 yearfollow-up of a computer-based information system for distributed MCH services. Internationaljournal of medical informatics. 1997 Apr;44(2):117-25.[3a] Llido LO. The impact of computerization of the nutrition support process on the nutritionsupport program in a tertiary care hospital in the Philippines: report for the years 2000-2003.Clin Nutr. 2006 Feb;25(1):91-101.[4a] Chae YM, Kim SI, Lee BH, Choi SH, Kim IS. Implementing health managementinformation systems: measuring success in Koreas health centers. Int J Health Plann Manage.1994 Oct-Dec;9(4):341-8.[5a] Al Farsi M, West DJ, Jr. Use of electronic medical records in Oman and physiciansatisfaction. J Med Syst. 2006 Feb;30(1):17-22.[6a] Weinhara M, Stoicu-Tivadar L, Dagres C. Early stage testing of users satisfaction afterimplementation of a central electronic health record (EHR) system in Serbia. Journal onInformation Technology in Healthcare. 2009;7(2):127-33.[7a] Sequist TD, Cullen T, Hays H, Taualii MM, Simon SR, Bates DW. Implementation anduse of an electronic health record within the Indian Health Service. J Am Med Inform Assoc.2007 Mar-Apr;14(2):191-7.[8a] Ndira SR, Rosenberger KD, Wetter T. Assessment of data quality of and staff satisfactionwith an electronic health record system in a developing country (Uganda): A qualitative andquantitative comparative study. Methods of Information in Medicine. 2008 2008;47(6):489-98.[9a] Rotich JK, Hannan TJ, Smith FE, Bii J, Odero WW, Vu N, et al. Installing andimplementing a computer-based patient record system in sub-Saharan Africa: the MosoriotMedical Record System. J Am Med Inform Assoc. 2003 Jul-Aug;10(4):295-303.[10a] Pourasghar F, Malekafzali H, Koch S, Fors U. May not fit Factors influencing the qualityof medical documentation when a paper-based medical records system is replaced with anelectronic medical records system: an Iranian case study. Int J Technol Assess Health Care. 2008Fall;24(4):445-51.[11a] Ayyagari A, Bhargava A, Agarwal R, Mishra SK, Mishra AK, Das SR, et al. Use oftelemedicine in evading cholera outbreak in Mahakumbh mela, Prayag, UP, India: Anencouraging experience. Telemedicine Journal and E-Health. 2003;9(1):89-94.[12a] Alvarez Flores MG, Guarner J, Terres Speziale AM. [Productivity before and afterinstalling a computerized system in a clinical laboratorya]. Rev Invest Clin. 1995 Jan-Feb;47(1):29-34.[13a] Turhan K, Kayikcioglu T. Implementation of a virtual private network-based laboratoryinformation system serving a rural area in Turkey. Laboratory Medicine. 2006;37(9):527-31.[14a] Cassiani SH, Freire CC, Gimenes FR. [Electronic medical prescription at a universityhospital: writing failures and users opinions]. Rev Esc Enferm USP. 2003 Dec;37(4):51-60.[15a] Costa AL, de Oliveira MM, Machado Rde O. An information system for drugprescription and distribution in a public hospital. International journal of medical informatics.2004 May;73(4):371-81.[16a] Gimenes FRE, Miasso AI, De Lyra Jr DP, Grou CR. Electronic prescription ascontributing factor for hospitalized patients safety. Pharmacy Practice. 2006;4(1):13-7. 1
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50.[17a] Tan WS, Phang JS, Tan LK. Evaluating user satisfaction with an electronic prescriptionsystem in a primary care group. Ann Acad Med Singapore. 2009 Jun;38(6):494-7.[18a] Kazemi A, Ellenius J, Tofighi S, Salehi A, Eghbalian F, Fors UG. CPOE in Iran--a viableprospect? Physicians opinions on using CPOE in an Iranian teaching hospital. Internationaljournal of medical informatics. 2009 Mar;78(3):199-207.[19a] Kazemi A, Ellenius J, Pourasghar F, Tofighi S, Salehi A, Amanati A, et al. The Effect ofComputerized Physician Order Entry and Decision Support System on Medication Errors in theNeonatal Ward: Experiences from an Iranian Teaching Hospital. Journal of Medical Systems.2009.[20a] Fraser H, Jazayeri D, Choi S, Blaya J, Bayona J, Levison L, et al. Forecasting three yearsdrug supply for a large MDR-TB treatment program in Peru. Int J Tuber Lung Dis. 2006;10(11Suppl. 1):S245.[21a] Yamanija J, Durand R, Bayona J, Blaya J, Jazayeri D, Fraser H. Comparing actualmedication consumption against the quantities ordered and a prediction using an informationsystem. Int J Tuber Lung Dis. 2006;10(11 Suppl. 1):S69-S70.[22a] Choi SS, Jazayeri DG, Mitnick CD, Chalco K, Bayona J, Fraser HS. Implementation andinitial evaluation of a Web-based nurse order entry system for multidrug-resistant tuberculosispatients in Peru. Medinfo. 2004;11(Pt 1):202-6.[23a] CDC Global AIDS Program. Responses to the Touchscreen System User Survey: QueenElizabeth Central Hospital. Malawi: CDC Global AIDS Program; 2007.[24a] Aviles W, Ortega O, Kuan G, Coloma J, Harris E. Quantitative assessment of the benefitsof specific information technologies applied to clinical studies in developing countries. Am JTrop Med Hyg. 2008 Feb;78(2):311-5.[25a] Fabre-Teste B, Sokha O. [Calmette Hospital, Phnom Penh, Cambodia. Assessment of theimplementation of the Medical Information System (SIM). Global analysis of the 1998 results].Sante. 1999 Nov-Dec;9(6):367-75.[26a] Rommelmann V, Setel PW, Hemed Y, Angeles G, Mponezya H, Whiting D, et al. Costand results of information systems for health and poverty indicators in the United Republic ofTanzania. Bull World Health Organ. 2005 Aug;83(8):569-77.[27a] Fraser HSF, Allen C, Bailey C, Douglas G, Shin S, Blaya J. Information systems forpatient follow-up and chronic management of HIV and tuberculosis: A life-saving technology inresource-poor areas. Journal of Medical Internet Research. 2007;9(4):38.[28a] Dwolatzky B, Trengove E, Struthers H, McIntyre JA, Martinson NA. Linking the globalpositioning system (GPS) to a personal digital assistant (PDA) to support tuberculosis control inSouth Africa: a pilot study. International journal of health geographics. 2006;5:34.[29a] Jirapaet V. A computer expert system prototype for mechanically ventilated neonatesdevelopment and impact on clinical judgment and information access capability of nurses.Comput Nurs. 2001 Sep-Oct;19(5):194-203.[30a] DeRenzi B, Lesh N, Parickh T, Sims C, Mitchell M, Maokola W, et al. e-IMCI:Improving Pediatric Health Care in Low-Income Countries. CHI. Florence, Italy 2008.[31a] Peters DH, Kohli M, Mascarenhas M, Rao K. Can computers improve patient care byprimary health care workers in India? International Journal for Quality in Health Care.2006;18(6):437-45.[32a] Bridges.org. Evaluation of the On Cue Compliance Service Pilot: Testing the use of SMSreminders in the treatment of Tuberculosis in Cape Town, South Africa. Cape Town: City of 2
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50.Cape Town Health Directorate and the International Development Research Council (IDRC);2005.[33a] Leong KC, Chen WS, Leong KW, Mastura I, Mimi O, Sheikh MA, et al. The use of textmessaging to improve attendance in primary care: a randomized controlled trial. Fam Pract. 2006Dec;23(6):699-705.[34a] Shirima K, Mukasa O, Schellenberg JA, Manzi F, John D, Mushi A, et al. The use ofpersonal digital assistants for data entry at the point of collection in a large household survey insouthern Tanzania. Emerg Themes Epidemiol. 2007;4:5.[35a] Bridges.org. Evaluation of the SATELLIFE PDA Project, 2002: Testing the use ofhandheld computers for heathcare in Ghana, Uganda, and Kenya. Boston, MA: Satellife; 2003.[36a] Satellife and Uganda Chartered HealthNet. Uganda Health Information Network, Phase-III: June 9, 2006 – June 8, 2007. Boston: Satellife and Uganda Chartered HealthNet; 2007.[37a] Kinkade S, Verclas K. Wireless Technology for Social Change. Washington, DC: UNFoundation-Vodafone Group Foundation Partnership; 2008.[38a] Missinou MA, Olola CH, Issifou S, Matsiegui PB, Adegnika AA, Borrmann S, et al.Short report: Piloting paperless data entry for clinical research in Africa. Am J Trop Med Hyg.2005 Mar;72(3):301-3.[39a] Gutierrez JP, Torres-Pereda P. Acceptability and reliability of an adolescent risk behaviorquestionnaire administered with audio and computer support. Revista Panamericana De SaludPublica-Pan American Journal of Public Health. 2009 May;25(5):418-22.[40a] Bernabe-Ortiz A, Curioso WH, Gonzales MA, Evangelista W, Castagnetto JM, CarcamoCP, et al. Handheld computers for self-administered sensitive data collection: a comparativestudy in Peru. BMC medical informatics and decision making. 2008;8:11.[41a] Cheng K, Ernesto F, Truong K. Participant and Interviewer Attitudes toward HandheldComputers in the Context of HIV/AIDS Programs in Sub-Saharan Africa. CHI: Healthcare inthe Developing World. Florence, Italy 2008.[42a] Zwarenstein M, Seebregts C, Mathews C, Fairall L, Flisher AJ, Seebregts C, et al.Handheld Computers For Survey and Trial Data Collection in Resource-Poor Settings:Development and Evaluation of PDACT, a Palm™ Pilot Interviewing System. unpublished.[43a] Blaya JA, Gomez W, Rodriguez P, Fraser H. Cost and implementation analysis of apersonal digital assistant system for laboratory data collection. Int J Tuberc Lung Dis. 2008Aug;12(8):921-7.[44a] Blaya JA, Cohen T, Rodriguez P, Kim J, Fraser HS. Personal digital assistants to collecttuberculosis bacteriology data in Peru reduce delays, errors, and workload, and are acceptable tousers: cluster randomized controlled trial. Int J Infect Dis. 2009 May;13(3):410-8.[45a] Forster D, Behrens RH, Campbell H, Byass P. Evaluation of a computerized field datacollection system for health surveys. Bull World Health Organ. 1991;69(1):107-11. 3
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50. APPENDIX Exhibit 2a Electronic Health Record Evaluations System or Evaluation Institution Country Type Outcome Virginia System costs were US$750 for satellite Commonwealth communication, and a fixed cost of a satellite University phone (US$500), and monthly fees. They provided [1a] Kenya Cost for 2700 patients. Bhorugram Over 4 years immunizations increased from 45.4% Rural Case- to 81.9% and 46.1% to 77.7% in DPT and polio Dispensary control vaccines; antenatal registration increased from [2a] India study 384 to 705 patients. Decreased percentages of wrong entries and non- St. Lukes Case- entries either of weight or height; Increases Medical Philipp control seen in nutrition support services referrals to Center [3a] ines study clinical dietitians and dietician productivity. Kwonsun Staff & Increased staff productivity and satisfaction. Health Center patient Did not increase staff decision abilities. [4a] Korea surveys Increased visitors satisfaction with services. Advantages: physicians recorded improved communication (95%); improved quality of care (85%); accurate entry and retrieval of data (80%); easy access to data (70%); usable in physician liability cases (64%); reduced medical errors (67%); enhanced productivity (59%); Disadvantages: disease coding is a problem Sur Hospital Physician (70%); system is time consuming (67% agree); and [5a] Oman survey too slow (60%). Advantages: improve clinical documentation, consistency of health maintenance, access to patients data and research opportunities. Euro Health Staff Disadvantages: negative impact on physician- Group [6a] Serbia survey patient consultation time. Advantages: EHR implementation was viewed positively (66%); improved quality of care (35%); 34% self-reported that EHRs improved quality, this was associated with increased utilization (odds ratio 3.03). IT could improve quality of care in underserved settings (87%) Indian Health Physician Disadvantages: decreased quality of patient– Service [7a] USA survey doctor interaction (39%). Higher availability of reports at district Tororo health office compared to paper (79% vs. 100%), District Before- no difference in quality, majority of staff Hospital[8a] Uganda after interviewed appreciated system. Hospital matron noticed a cluster of sexually transmitted disease and therefore dispatched a team to investigate. Also noted lack of child Mosoriot immunizations and dispatched nurses to that Medical site. Reports that previously took a clerk two Record System User weeks, now take minutes; allowed the director to [9a] Kenya opinion reassign two clerks to other duties Mosoriot Duration of visits dropped from 41 to 31 Medical minutes; providers time with patients dropped Record System Before- from a third to a sixth of workday; providers [9a] Kenya after spent two thirds less time interacting with
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50. other staff and tripled their time spent in personal activities; clerks spent two thirds less time interacting with other staff and almost doubled their time registering patients. The EMR had higher overall completeness than the paper system. High workloads, shortage of Karolinska Random bedside hardware and lack of software features Institute selection were prominent influential factors in the [10a] Iran of records quality of documentation. SOURCE: Authors’ Analysis NOTES: Evaluations are in increasing order of strength with multiple evaluations of a single system placed together. References can be found in Appendix Exhibit 1a
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50. APPENDIX Exhibit 3a Laboratory Information Management Systems (LIMS) and Pharmacy Information System Evaluations System or Evaluation Institution Country Type Outcome Laboratory Information Management Systems (LIMS) Cholera was isolated in 22.6% (7/31) of samples sent to a central laboratory. Information was relayed to hospital and Sanjay Gandhi health authorities, who took strict measures Post Graduate to improve hygiene at a festival. Institute of Subsequently, the number of diarrhea cases Medical during festival decreased and an epidemic was Sciences [11a] India Descriptive averted. Case- Productivity indexes showed an increase by control 41% in number of patients handled and 28% in Tesilab [12a] Mexico study number of tests processed. Turn around times for routine samples Karadeniz decreased from 1 to half day; number of Technical samples processed increased a factor of 2; University, Before- annual laboratory revenue increased 4 times, [13a] Turkey after from 55,000 to 220,000 euro per month. Pharmacy Information Systems In 28.2% of medication orders there was dubious or misleading information Advantages: ease of data access and ordering. Disadvantages: repetition of Universidade de Descriptiv orders from previous days without a review São Paulo [14a] Brazil e and incorrectly typed information. Advantages: user-friendly interface; quickness and clarity of information; ease of use; reduction of time between drug Hospital das prescription and administration; believed to Clínicas da result in a drastic reduction in the risk of Faculdade de error. Medicina de Disadvantages: insufficient number of Ribeirão Preto Staff terminals; system got stuck; technical [15a] Brazil survey support was unsatisfactory. Advantages: legibility (37.5%); less time to order (20.5%); more practical and organized (8%). Disadvantages: repetition of previous prescriptions (34%); typing mistakes (17%); University of Staff dependence on computers (11%); alterations São Paulo [16a] Brazil survey made manually (7%) Over 70% of users preferred system over paper, felt that it reduced the number of prescription errors, and knew what to do when system was down. Its limitations were with system speed and functionality in processing prescriptions. National Satisfaction was more associated with Healthcare Staff perceived impact on productivity than with Group [17a] Singapore survey patient care. Ekbatan Staff Clinician users of the prescribing system Hospital [18a] Iran interviews were found to mostly rely on their memories
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50. and be overconfident which could lead to errors. Advantages: increased confidentiality, reduction of medication errors and educational benefits. Disadvantages: high cost, social and cultural barriers, data entry time and problems with technical support. Before intervention (Period 1), error rate was 53%, which did not significantly change after the implementation of CPOE without Hamadan decision support (Period 2). However, errors University of were significantly reduced to 34% after the Medical Before- decision support was added to the CPOE Sciences [19a] Iran after (Period 3). Accuracy of prediction per medication was Socios En Salud Model vs. 117% over-estimate in 2002, 5% underestimate [20a] Peru actual use in 2003 and to 2% under-estimate 2004. Model, For subgroup of 58 patients on order individualized treatment, model predicted Socios En Salud placed vs. 99% of actual use, the actual order placed [21a] Peru actual use was 145% of actual use. Externally 17.4% error rate fell significantly in the controlled study group to 3.1% per patient. Error rate Socios En Salud before- did not differ statistically in control [22a] Peru after group (8.6% to 6.9%). SOURCE: Authors’ Analysis NOTES: Evaluations are in increasing order of strength with multiple evaluations of a single system placed together. References can be found in Appendix Exhibit 1a.
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50. Appendix Exhibit 4a Patient Registration and Scheduling, Monitoring and Evaluation, and Clinical Decision Support System Evaluations System or Evaluation Institution Country Type Outcome Patient Registration and Scheduling Most of the users (70%) expressed a clear preference for the touch screen over the paper system. However, every respondent Baobab Clinical also identified on-going problems that need Health[23a] Malawi user survey to be addressed. Sustainable Mean time to locate record with fingerprint Sciences Simultaneous scan was 7.0 (SD 3.5) seconds, versus 27.3 Institute randomized (SD 7.1) seconds using the traditional [24a] Nicaragua controls method. Average time to locate a patient’s chart Sustainable using traditional methods was 2.9 (SD 2.1) Sciences Simultaneous minutes, whereas using barcode-based Institute randomized methods the average was 0.09 minutes, or [24a] Nicaragua controls 5.5 (SD 1.2) seconds. Monitoring, Evaluation, and Patient Tracking Systems Data are invaluable for the short-term Calmette management of the hospital. SIM helped set Hospital [25a] Cambodia Descriptive up infection control committee. Tanzanian Total annual systems cost was US$2,119,941, Ministry of $0.13 per participant, and $0.06 per Health [26a] Tanzania Cost capita. For patients with CD4 counts between 101 and 350, those entered into the system within 14 days had an odds ratio of 3.2 for Case-control starting treatment within 14 days compared HIV-EMR [27a] Haiti study to those without early CD4 entry. Logged patient follow-up visits allowed staff to rapidly identify a decline among patients who had stopped receiving food supplementation. New strategies were HIV-EMR2.0 implemented within 3 weeks, and clinic (OpenMRS) Case-control attendance returned to original level of [27a] Haiti study over 90%. University of Time taken to locate ten households was the Simultaneous reduced by 20% and 50% in each of two Witwatersrand South randomized communities using the PDA/GPS device [28a] Africa controls compared to paper. Sustainable Sciences Simultaneous Institute randomized GIS did not significantly decrease the time [24a] Nicaragua controls necessary to locate a home. Clinical Decision Support System (CDSS) Chulalongkorn Nurses perceived they had better judgment University Before-after and information access, all participants [29a] Thailand qualitative wanted permanent installation. Chulalongkorn University Before-after Mean judgment performance score for case [29a] Thailand quantitative simulations increased by 42%.
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50. Electronic Integrated 84.7% of e-IMCI investigations had IMCI Management of completed compared to 61% with the chart Childhood Simultaneous booklet. Amount of time for both IMCI and Illness (e- nonrandomize e-IMCI sessions averaged 12.5 minutes for IMCI) [30a] Tanzania d controls the one clinician tested. Increase of 430 new patient visits per month at intervention sites, increase from Early baseline of 18% at intervention sites Diagnosis and compared with decline of 5% at control Prevention sites. Intervention was associated with System (EDPS) Longitudinal significant improvements in Global Patient [31a] India RCT Assessment of Care Index. SOURCE: Authors’ Analysis NOTES: Evaluations are in increasing order of strength with multiple evaluations of a single system placed together. References can be found in Appendix Exhibit 1a.
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50. Appendix Exhibit 5a Patient Reminder and Research/Data Collection Systems Evaluations System or Country Evaluation Institution Type Outcome Patient Reminder Systems On Cue Compliance South Cost of 120 SMS reminders were [32a] Africa Cost R13.90/patient/month (US$2.43). Intervention had higher completion rate (10.6 vs. 3%), but similar cure rate (62.3 vs. 66.4%) and treatment success rate (73 vs. On Cue 69%) compared to data from City of Cape Compliance South Before- Towns TB Control Program for same clinic in [32a] Africa after 2003. It cost RM 0.45 per attendance for text International messaging reminder as compared with RM 0.82 Medical Cost- per attendance for mobile phone reminder. The University effectivene ratio of cost per unit attendance of text Puchong [33a] Malaysia ss messaging versus mobile phone was 0.55. Attendance rates of control, text messaging and mobile phone reminder groups were 48.1, 59.0 and 59.6%, respectively. The text messaging group was significantly higher than International Simultaneou control group, no difference between text Medical s messaging and mobile phone group. Text University randomized messaging reminder system cost less than half Puchong [33a] Malaysia controls of the mobile phone reminder per attendance. Research/Data Collection Systems There were no problems with the PDAs while Ifakara collected data on 83,346 individuals over Health seven weeks. Dataset was available within 24 Research & hours. Median time to form completion was 14 Development minutes during training and nine minutes Centre [34a] Tanzania Descriptive during survey. 87% reported that health content received Uganda helped them make faster more accurate Health diagnoses. 86% integrated PDA into other Information activities. 73% able to solve problems; 68% Network reported problems with 41% of them being [35a, 36a] Uganda User survey resolved due to lack of technical support. System provides up to 91% saving per unit Uganda spending compared to paper-based HMIS data Health collection and reporting approaches. Information Reporting compliance to MOH improved from Network Cost national average of 63% to 94-100% for [35a, 36a] Uganda analysis districts using UHIN. Advantages: time savings (95 percent); the ability to quickly mobilize or organize individuals (91 percent); reaches audiences previously difficult or impossible to reach UN-Vodafone (74 percent); transmit data more quickly and Partnership Multiple accurately (67 percent); gather data more [37a] countries User survey quickly and accurately (59 percent). Albert Gabon Self- Rate of discrepant entries was 1.7%.
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50. Schweitzer controls Categorical data were more commonly Hospital discrepant than were continuous “typed in” [38a] data (2.4% versus 1.2. In 558 patient interviews accuracy of PDA and Sustainable paper methods was 97.1% and 97.6%, Sciences respectively. For 1,543 field visits, Institute Self- accuracy rate for PDA and paper methods was [24a] Nicaragua controls 98.9% and 99.3%, respectively. Mexican National Institute of Public Focus A majority of adolescents preferred the Health [39a] Mexico groups computer-assisted system to interviews. Mexican Adolescents were 2-8 times more likely to National report smoking, alcohol, and sexual behavior Institute of using the audio computer-assisted self- Public Self- interview system compared to face-to-face Health [39a] Mexico controls interviews. Before- First survey, almost perfect agreement Universidad after between paper and PDA. Second survey, rates Peruana (first of responses to sensitive questions were Cayetano survey), similar between both kinds of questionnaires. Heredia RCT (second PDA had 96% less inconsistencies and 66% less [40a] Peru survey) missing values than paper. Charles R. There was no difference between participants’ Drew self-reported comfort across handheld and University paper conditions. However, participants in of Medicine the handheld condition were more likely to and Science give socially desirable responses to the [41a] Angola Block RCT sexual behavior questions . Cost of PDA survey is slightly less than South paper when cost of hardware is annualized African over four studies and the programming cost Medical excluded. When programming cost is included, Research upfront costs need to be discounted over Council South Cost eight studies to obtain a comparative cost [42a] Africa analysis with paper. South African Medical 85% of PDA users preferred PDA and 7% Research preferred paper for answering questions about Council South User sex. 53% of paper users preferred PDA and 28% [42a] Africa surveys preferred paper. South African Medical Simultaneou Research s Comparing the data collected by the paper and Council South randomized PDAs using intra-scale and the test-retest [42a] Africa controls reliability found them to be similar. User satisfaction higher for PDA (mean 5 of 5) than paper (3.5 of 5). PDA reduced mean work-time per result from 6.75 to less than 2 minutes. Mean 1.13 technical problems per Socios En month which could be fixed in the field (2 Salud[43a] Peru User Survey users) or back at the office (2 users). Socios En Peru Cost Work hours required decreased by 60%. Total
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50. Salud [43a] analysis cost and time to develop and implement was US$26,092 and 22 weeks. Cost to expand to 9 districts was $1,125 and to implement collecting patient weights $4,107. PDA-based system decreased mean processing time from 23 to 8 days, and results with Socios En times greater than 90 days from 9.2% to 0.1%. Salud [44a] Peru Cluster RCT It reduced errors by 57.1%. A custom-built PDA system showed a 30% improvement for collection of identification data and a 100% improvement for dates and Cross-over times [system automatically time stamped]. London simultaneou Significant reduction in inter-individual School of s variability in data accuracy. By the third Economics randomized week the average interview times were 31% [45a] Gambia controls shorter for field workers who used handheld. SOURCE: Authors’ Analysis NOTES: Evaluations are in increasing order of strength with multiple evaluations of a single system placed together. References can be found in Appendix Exhibit 1a.
Blaya JA, Fraser HSF, Holt B. E-health technologies show promise in developing countries. Health Aff (Millwood). 2010;29(2):243-50.