Botswana Adaptation of the RDQA

Uploaded on

Authored by Ernest Fetogang, Suzanne Cloutier, Sergio Lins, Amanda Makulec, Rosinah T. Dialwa, and Tom Achoki and presented at the 2013 Global Maternal Health Conference.

Authored by Ernest Fetogang, Suzanne Cloutier, Sergio Lins, Amanda Makulec, Rosinah T. Dialwa, and Tom Achoki and presented at the 2013 Global Maternal Health Conference.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Botswana 1 1/7/2013 10:28:57 AM BOTSWANA ADAPTATION OF THE RDQA Global Maternal Health Conference Authors: Ernest Fetogang, Suzanne Cloutier, Sergio Lins, Amanda Makulec, Rosinah T. Dialwa, and Tom Achoki. Ministry of Health Botswana John Snow Inc. John Snow Inc. John Snow Inc. Ministry of Health Botswana Ministry of Health Botswana I Background II Methodology/Approach III Botswana’s Approach Information is a key building block of a health system, and efforts to improve The core objectives of the collaboration between the MoH and MEASURE Basic measures to ensure standards of data quality should be taken to ensure data quality directly support improvements in a country’s information system Evaluation were to both accuracy and reliability at all levels of the health system. The data quality across programme areas. (1) describe the process for ensuring data quality at the service delivery, protocols in Botswana were developed to address this challenge, and are appli- district, and national levels, and cable at all levels of the health system. Figure 1: Information in the WHO Health System Building Blocks (2) provide guidelines for data quality monitoring procedures. Figure 3: Ideal Botswana Health Data Flow The ideal data flow for the Ultimately, these activities aimed to insure accuracy, completeness, and timeli- Botswana MoH is illustrated MoH External National THE WHO HEALTH SYSTEM FRAMEWORK ness of health data being transmitted in Botswana and define responsibilities Management, Stakeholders Programs, in Figure 3. Botswana health Policy, Planners HSU for data quality at each level of the health information system. data currently flow through SYSTEM BUILDING BLOCKS OVERALL GOALS/OUTCOMES more than 39 different infor- SERVICE DELIVERY Other ministries The development of the protocols and curriculum was conducted over the Consolidation; Analysis; National [e.g. Ministry of mation systems, including Data Dissemination M&E course of one year. Home Affairs, etc.] both electronic and paper- HEALTH WORKFORCE ACCESS IMPROVED HEALTH (LEVEL AND EQUITY) based systems that feed COVERAGE into various data manage- INFORMATION RESPONSIVENESS January Establish scope of work & objectives Consolidation DHMT ment systems. With the MEDICAL PRODUCTS, VACCINES & TECHNOLOGIES SOCIAL AND FINANCIAL RISK PROTECTION February Develop B-RDQA Excel tool creation of a national M&E March Draft Standard Operating Procedures (SOPs) unit, the Ministry is working QUALITY FINANCING SAFETY IMPROVED EFFICIENCY April Pre-test B-RDQA tool Hospitals, to streamline processes Data Private Clinics, NGOs, Review SOPs Producers Sector Mobile CBOs, and move towards this FBOs LEADERSHIP/GOVERNANCE Stops ideal flow. May Finalize B-RDQA tool Revise SOPs A global conceptual framework for data quality was adapted to reflect the The ability of health system stewards to make strategic decisions is impacted by the quality of health data. At the national level in Botswana, data ultimately June Finalize SOPs Botswana data flow and priorities within the country’s M&E system. inform budget and policy decisions. In the Health Districts and Service Delivery July-October Develop data quality curriculum Sites, data enables providers and monitoring and evaluation (M&E) officers to November Conduct data quality training workshop understand the broader health activities and priorities in their respective areas. Figure 4: Botswana Conceptual Framework for Data Quality Figure 2: Data & Health Impact Key deliverables in the process included: Dimensions of Quality 1. Data Quality SOP—General, high level protocol for ensuring data quality QUALITY DATA Accuracy, Completeness, Relability, Timeliness, Policy & at the service delivery, district, and national levels Confidentiality, Precision and Integraity Health data Health data Assessment budget is collected aggregated of data 2. Routine Data Quality Assessment SOP— Protocol for the implementation Functional Components of a Data Management decisions System Needed to Ensure Data Quality at service at district & impacts of RDQAs as a monitoring tool to routinely review the quality of data at the impact delivery national policy & service delivery, district, and national levels I M&E Structure, Functions & Capabilities health sites levels budgets Data Management and outcomes 3. Customized RDQA Tool for Botswana (B-RDQA Tool) National M&E II Indicator Definitions & Reporting Guidelines Reporting System REPORTING LEVELS Data collection Aggregation & Analysis Impact on health 4. B-RDQA Tool User Manual—Detailed guidance on the implementation III Data-collection & Reporting Forms/Tools and results dissemination from conducting an RDQA using the B-RDQA Tool Health Districts IV Data Management Processes To support improved data quality throughout the health system, the Botswana on up to four indicators within any health programme V Training Ministry of Health (MoH) collaborated with experts from MEASURE Evaluation to 5. Data Quality Curriculum—Curriculum for use in MoH trainings on data Service Delivery Sites develop a national procedure for routine monitoring of data quality and provid- quality, including presentations, exercises, and a full participant’s guide. VI Use of Data for Decision Making ing specific guidance on developing action plans to address challenges using a Curriculum covers the content of the two SOPs and the collection and use bottom-up approach. Botswana is the first country to adapt the RDQA method- of data from routine data quality assessments. ology and tools for national use. Figure 5: Timeline for implementation of RDQA Suggested follow up Suggested follow up Full RDQA Baseline full RDQA full RDQA Data verifications & Data verifcations & Data verifcations & System assessment System assessment System assessment Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Data verifications Data verifications Data verifications Data verifications Data verifications Data verifications IV Adaptation of the B-RDQA Tool VI Year 1 Year 2 Data Quality Curriculum & Training Note that if a follow-up full RDQA was not conducted at the end of Year 1, a full RDQA must be conducted at the end of Year 2 inorder to have an up-to-date system assessment. C Figure 6: Functional Areas The B-RDQA Tool is an Excel tool Finally, a complete curriculum was developed by MEASURE Evaluation to train M Y of the M&E System with multiple worksheets for a user national and district M&E officers on how to implement and use the SOPs andCM to complete to verify data at various the B-RDQA Tool. The curriculum underpins a two and a half day training with aMY levels of the health system and con- balance of presentations and hands-on exercises that give attendees first-handCY duct a system assessment to evalu- experience using the tool, interpreting outputs, and developing action plans. VIICMY M&E ate the key functional compo- structures, Resource Requirements K functions & nents of the M&E system. The A training of 22 M&E officers was conducted in November 2012, and a training of capabilities tool was customized with trainers is planned for March 2013. Overall feedback on the first training was very Indicator changes to the language positive, indicating that the SOPs and RDQA process would be useful for use both Use of data definitions for decision used to describe the various at the district and national levels as a routine tool for improving data quality. Time & cost: The process of developing the SOPs, customized tool, training ma- and reporting making quidelines levels of the health system to terials, and conducting the first training took approximately one year. The total Six functional reflect the Botswana data flow. cost, primarily in staff time and travel for in-country consultation workshops areas of an and training, was US$250,000, funded by the United States Government M&E system The adaptation of the tool through MEASURE Evaluation. Data collection included a customization and reporting Training workshop in April 2012, forms and “This training makes our DQA knowledge Staff: At the MoH, the newly formed Department for Health Policy, Monitoring, tools where specific recommenda- clear & improved; developed skills that and Evaluation (DHPME) initiated the activities with MEASURE Evaluation. A Data tions were made on content Principal Health Officer was a key champion for the process, supported by the management changes to reflect the needs of we expected for over a decade.” Chief Health Officer and Monitoring & Evaluation Advisor. The MEASURE Evalua- processes the MoH. One of the significant - Trainee tion team that worked with the MoH included three Senior M&E Advisors and additions to the tool was the addition of the two M&E Associates. “use of data for decision making” functional area in the system assessment component of the tool. “This [process] will really reduce work burden… Travel: A total of four trips were made to work in-country with the MoH and The importance of this component was reinforced in subsequent consultative other stakeholders including: workshops and at the November training, where district M&E officers identified very exciting, can’t wait to implement. This was 1) January 2012—Planning visit to develop the scope of work. the use of data for decision making as a key challenge. one of the best trainings which will really address 2) April 2012—B-RDQA Tool customization workshop and pretesting. our district data quality problems.” 3) June 2012—Consultative workshops to finalize SOPs and user manual. Following the customization workshop, the B-RDQA tool was pretested in the 4) November 2012—Training for M&E officers, at the invitation of the MoH. field with the Family Planning and Reproductive Health Programme. – District Health Officer VIII. Keys to Success Country ownership: The development and implementation of protocols for im- “A very good training that came at the right time, proving data quality was initiated by the MoH, who approached MEASURE Evaluation for technical assistance to adapt global tools to the Botswana con- providing skills that are sustainable and very easy text. The country-led foundation of this process has been essential in connect- to use…Bringing out very valuable results to ing with the correct stakeholders to give input and insight. V improving health information systems, important Standard Operating Procedures to system improvement and decisions making.” Champions: Also key to the entire process was having a strong champion for data quality activities at the MoH. Without a strong technical voice supporting - Trainee the investment in protocols to improve data quality, it would have been chal- lenging to find the momentum to support the development and implementa- The SOPs for data quality and RDQA, as well as the B-RDQA User Manual were tion of the protocols. drafted for review while the final B-RDQA Tool was being customized. Originally, the team envisioned one comprehensive SOP and user guide. To make the docu- Decentralization: Finally, the protocols decentralize the process of planning ments more user-friendly, the SOP was divided into two SOPs and a user manual. targeted activities to improve data quality, allowing service delivery sites and district level officials to take ownership of data quality in a systematic and struc- The data quality SOP was written as a high level document on the various tured way. Service delivery sites and districts develop their own recommenda- dimensions and considerations of data quality, intended for senior MoH officials, tions and action items, putting the power in local hands. other policymakers, and M&E officers. The RDQA SOP was written as a general protocol for conducting an RDQA, including responsibilities by level, intended A challenge, moving forward, will be the need for continued implementation for any MoH or district official responsible for initiating, managing or conducting support. The sustainability of these protocols and the use of the customized routine assessments. The B-RDQA user manual was written specifically for those tool will rely on continued support—both technical and financial—to train staff staff using the B-RDQA tool to conduct assessments in the field at service deliv- and encourage use of the B-RDQA Tool for monitoring data quality. ery sites. Draft SOPs and a draft user manual were reviewed and discussed with stakeholders IX at consultative workshops in June 2012. Both Ministry and external stakeholders partici- Conclusions pated in the consultations, and documents were final- Over the next year and beyond, results of regular system assessments and rou- ized based on the recom- tine data verification exercises could be analyzed to evaluate the impact of the mendations from the work- SOPs and use of the RDQA process. With growing interest and investment in shops. Final documents were health system strengthening measures, the Botswana adaptation of global data printed in country for distri- quality tools operationalizes a system for health information system improve- bution by the MoH. ments that could be adopted by other countries facing data quality challenges. The development of this poster was supported by funds from the USAID MEASURE Evaluation project. MEASURE Evaluation is funded by USAID through cooperative agreement GHA-A-00-08-00003-00 and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill, with Futures Group, ICF International, John Snow, Inc., Management Sciences for Health, and Tulane University. The views expressed in this publication do not necessarily reflect the views of USAID or the United States government.