The core objectives of the collaboration between the MoH and MEASURE Evaluation were to:
1.	describe the pro...
Upcoming SlideShare
Loading in …5

Botswana’s Integration of Health Data Quality Assurance Into Standard Operating Procedures


Published on

Presented at the 2014 CUGH Conference.

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Botswana’s Integration of Health Data Quality Assurance Into Standard Operating Procedures

  1. 1. 2 METHODOLOGY The core objectives of the collaboration between the MoH and MEASURE Evaluation were to: 1. describe the process for ensuring data quality at each level of the health system, including level specific responsibilities, and 2. provide guidelines for data quality monitoring procedures. KEY DELIVERABLES IN THE PROCESS INCLUDED: 1. Data Quality Standard Operating Procedure (SOP)—General, high level protocol for ensuring data quality at the service delivery, district, and national levels 2. Routine Data Quality Assessment (RDQA) SOP—Protocol for the implementation of RDQAs as a monitoring tool to routinely review the quality of data at the service delivery, district, and national levels 3. Customized RDQA Tool for Botswana (B-RDQA Tool)—Excel worksheet consisting of data entry forms and graphical output 4. B-RDQA Tool User Manual—Detailed guidance on implementing and disseminating results using the B-RDQA Tool for conducting an RDQA of any health program 5. Data Quality Curriculum—Curriculum for use in MoH trainings on data quality, including presen- tations, exercises, and a full participant’s guide. Curriculum covers the content of the two SOPs and the collection and use of data from RDQAs. The development of the protocols and curriculum occurred over the course of a year (2012), followed by training workshops (2012/2013). January Established scope of work & objectives February Developed B-RDQA Tool March Drafted SOPs April Pre-tested B-RDQA Tool in the field Reviewed SOPs May Finalized B-RDQA Tool Revised SOPs June Finalized SOPs July-Oct. Developed the data quality curriculum November Conducted 1st data quality training workshop April Conducteddataqualitytrainingoftrainers(ToT)workshop Supported 2nd data quality workshop The development of this poster was supported by funds from the USAID MEASURE Evaluation project. RESOURCE REQUIREMENTS Time & cost: The process of developing the SOPs, customized tool, training materials, and conducting the trainings took about 16 months. The total cost to MEASURE Evaluation, primarily in staff time and travel for in-country consultative workshops and trainings, was $300,000, funded by the United States Agency for International Development. The MoH was responsible for funding the in-country costs, including workshop venues, per diems, and transportation costs. Staff: The newly formed Department of Health Policy Development, Monitoring and Evaluation (DHPDME) at the MoH initiated the activities with MEASURE Evaluation. The Principal Health Officer was a key champion for the process, supported by the Chief Health Officer. The MEASURE Evaluation team that worked with the MoH included three Senior M&E Advisors and two M&E Associates. Travel: Five trips were made to work in-country with the MoH and other stakeholders including: 1. January 2012—Planning visit to develop the scope of work 2. April 2012—B-RDQA Tool customization workshop and pilot testing 3. June 2012—Consultative workshops to finalize SOPs and user manual 4. November 2012—Training of M&E officers 5. April 2013—Training of trainers and training of M&E officers 7 MEASURE Evaluation is funded by USAID through cooperative agreement GHA-A-00-08-00003-00 and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill, with Futures Group, ICF International, John Snow, Inc., Management Sciences for Health, and Tulane University. The views expressed in this publication do not necessarily reflect the views of USAID or the United States government. Botswana’s Integration of Health Data Quality Assurance Into Standard Operating Procedures STANDARD OPERATING PROCEDURES The SOPs for data quality and RDQA, as well as the B-RDQA Tool User Manual were drafted for review while the B-RDQA Tool was being customized. • The data quality SOP was written as a high level document on the various dimensions and considerations of data quality, intended for senior MoH officials, other policymakers, and M&E Officers. • The RDQA SOP was written as a general protocol for conducting a RDQA, including responsibilities by level, intended for any MoH or district official responsible for initiating, managing or conducting routine assessments. • The B-RDQA Tool User Manual was written specifically for those staff using the B-RDQA Tool to conduct RDQAs in the field. Draft SOPs and a draft user manual were reviewed and discussed with stakeholders at consultative workshops. Both Ministry and external stakeholders participated in the consultations, and documents were finalized based on the recommendations from the workshops. Final documents were printed in country for distribution by the MoH. 5 BACKGROUND To support improved data quality throughout the health system, the Botswana Ministry of Health (MoH) collaborated with experts from MEASURE Evaluation to develop a national procedure for routine monitoring of data quality, as well as provide specific guidance on developing action plans to address challenges using a bottom-up approach. Because information is a key building block of a health system, efforts to improve data quality directly support improvements in a country’s information system across program areas. 1 The quality of health data impacts a government’s ability to make strategic health-related decisions. At the national level, data ultimately inform budget and policy decisions. In the District Health Management Teams (DHMTs) and Service Delivery Sites, data enable providers and Monitoring and Evaluation (M&E) Officers to understand the broader health activities and priorities in their respective areas. SERVICE DELIVERY HEALTH WORKFORCE INFORMATION MEDICAL PRODUCTS, VACCINES & TECHNOLOGIES FINANCING LEADERSHIP/GOVERNANCE IMPROVED HEALTH (LEVEL AND EQUITY) RESPONSIVENESS SOCIAL AND FINANCIAL RISK PROTECTION IMPROVED EFFICIENCY THE WHO HEALTH SYSTEM FRAMEWORK SYSTEM BUILDING BLOCKS OVERALL GOALS / OUTCOMES ACCESS COVERAGE QUALITY SAFETY FIGURE 1: INFORMATION IN THE WHO HEALTH SYSTEM BUILDING BLOCKS Health data are collected at service delivery sites Health data are aggregated at district & national levels Assessment of data impacts policy & budgets Policy & budget decisions impact health outcomes Data collection Aggregation & Analysis Impact on health FIGURE 2: DATA & HEALTH IMPACT ADAPTATION OF THE B-RDQA TOOL The B-RDQA Tool is an Excel dataset with multiple worksheets for a user to complete to verify data on up to four indicators at various levels of the health system, as well as conduct a system assessment to evaluate the key functional components of the M&E system. The tool was customized for the Botswana country context, with changes to the language used to describe the various levels of the health system to reflect the Botswana data flow from service delivery sites to health districts to National M&E. One of the significant changes to the tool was the addition of the “use of data for decision making” functional area in the system assessment component of the tool. The importance of this component was reinforced in subsequent consultative workshops and during the first training, where district M&E officers identified the use of data for decision making as a key challenge. 4 M&E structures, functions & capabilities Use of data for decision making Indicator definitions and reporting guidelines Training Data management processes Data collection and reporting forms and tools Six functional areas of an M&E system FIGURE 5: FUNCTIONAL AREAS OF THE M&E SYSTEM DATA QUALITY CURRICULUM & TRAINING A comprehensive curriculum was developed by MEASURE Evaluation to train national and district M&E officers on how to implement and use the SOPs and the B-RDQA Tool. The curriculum supports a two and a half day training with a balance of presentations and hands-on exercises that give attendees first-hand experience using the tool, interpreting outputs, and developing action plans. The 1st training of 22 M&E Officers was conducted in November 2012, followed by a ToT workshop in April 2013 with select participants from the first data quality training. The trainers then conducted the 2nd training with support from MEASURE Evaluation. Overall feedback on the trainings was very positive as indicated by the quotes to the left. 6 “This [process] will really reduce work burden… very exciting, can’t wait to implement. This was one of the best trainings which will really address our district data quality problems.” – District Health Officer “A very good training that came at the right time, providing skills that are sustainable and very easy to use…Bringing out very valuable results to improving health information systems, important to system improvement and decisions making.” - Trainee KEYS TO SUCCESS Country ownership: This activity was initiated by the MoH, who requested technical assistance to develop and implement protocols for improving data quality and adapt global tools to the Botswana context. The country-led foundation of this process was essential in connecting with the correct stakeholders to give input and insight. Furthermore, responsibility for the in-country expenses proved beneficial with respect to the MoH taking ownership of the process. Going forward, the MoH will conduct an annual national M&E forum where data quality issues will be addressed by M&E Officers based on findings from their RDQAs. The MoH also has identified a consultant to develop the National Health M&E Plan for the ministry, which will include RDQAs as a routine activity for districts and at national level. Champions: Also key to the entire process was having a strong champion for data quality activities at the MoH. Without a strong technical voice, it would have been challenging to find the momentum to support the development and implementation of the protocols. Decentralization: Finally, the protocols decentralize the process of planning targeted activities to improve data quality, allowing service delivery sites and district-level officials to take ownership of data quality in a systematic and structured way. Service delivery sites and districts develop their own recommendations and action items, putting the power in local hands. Evidence of the success of the RDQA process was found at a district outside the capitol. A District M&E Officer implemented a log book (see left) to track data quality indicators after attending the training workshop in April 2013. 8 Authors: Suzanne Cloutier, Sergio Lins, Amanda Makulec, David Boone, Ernest Fetogang, Rosinah T. Dialwa, Segametsi Duge and Sophia Magalona With growing interest and investment in health system strengthening measures, the Botswana adaptation of global data quality tools operationalizes a system for health information system improvements that could be adopted by other countries facing data quality challenges. Having conducted this adaptation in Botswana, the customization approach has been tested and streamlined, such that the deliverables could be adapted to other country contexts. CONCLUSIONS: 3 BOTSWANA’S APPROACH The data quality protocols in Botswana were developed to ensure the accuracy, timeliness and completeness of health data being transmitted throughout the health information system. The Botswana adaptation of the RDQA is unique as it provides a national protocol for routine data quality assessment across program areas. The ideal data flow for Botswana health data is illustrated below. Botswana’s health data currently flow through more than 39 different information systems, including both electronic and paper-based systems that feed into various data management systems. With the creation of a national M&E unit, the MoH is working to streamline processes and move towards this ideal flow. FIGURE 3: IDEAL BOTSWANA HEALTH DATA FLOW FIGURE 4: BOTSWANA CONCEPTUAL FRAMEWORK FOR DATA QUALITY JOHN SNOW, INC. A global conceptual framework for data quality was adapted to reflect the Botswana data flow and priorities within the country’s M&E system.