Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Exploring data quality in Community Health Information Systems in Kenya
Similar to Exploring data quality in Community Health Information Systems in Kenya (20)
More from REACHOUTCONSORTIUMSLIDES
More from REACHOUTCONSORTIUMSLIDES (20)
Recently uploaded
Recently uploaded (20)
Exploring data quality in Community Health Information Systems in Kenya
- 1. 1 Exploring data quality in Community Health Information Systems in Kenya Regeru Njoroge Regeru 1st International Symposium on Community Health Workers 23rd February 2017 Kampala, Uganda.
- 2. Background CHWs have emerged as a means to achieve Universal Health Coverage - Kok et al., 2016 CHWs collect data from the households they visit on a routine basis - Mireku et al., 2014 Data collection at community level is critical in assessing the performance of CHW programmes - Perry et al., 2014 2
- 3. Kenya’s Healthcare System 3 National Referral Health Services County Referral Health Services Primary Care Services Community Health ServicesMinistry of Health (Kenya), 2014 Referral Referral Referral
- 4. Structure of the Community Health Strategy 4Ministry of Health (Kenya), 2006
- 5. Community Health Services in Kenya 5 Community Health Volunteer (CHV) Community Health Extension Worker (CHEW) Facility In-Charge Link Healthcare Facility Sub-County Community Health Strategy Focal Person Community Health Committee Facility Health Management CommitteeMinistry of Health (Kenya), 2014
- 6. The structure of a Health Information System 6Pact Inc., 2014
- 7. Service Delivery Log Books Community Health Extension Worker Summary National Health Information System Community Health Volunteers Community Health Extension Worker Sub-County Health Records Information Officer 7
- 8. Health Information System performance “data quality and continuous use of routine information for decision-making” - Hotchkiss et al., 2012 8
- 9. What does data quality refer to? Accuracy Reliability Precision Completeness Timeliness Integrity Confidentiality 9 MEASURE Evaluation, 2008
- 10. Why explore data quality? Data should be used for decision- and policy-making Data collected at community level in Kenya is not used in decision-making because it is considered to be low quality 10 Ekirapa et al., 2012
- 11. Research objectives PERCEPTIONS OF ENABLERS AND BARRIERS TO GENERATING HIGH QUALITY DATA DATA QUALITY ASSESSMENT 11
- 12. Methods • Focus Group Discussions • In-depth Interviews • Calculation of data verification ratios for 7 indicators; March – May 2016 • Purposive selection of FGD and IDI participants • Data collection and reporting tools used for DQA • 2 Counties – Nairobi (urban) and Kitui (rural) • 4 Community Units • Cross-sectional • Mixed Methods Study Design Study Sites Data collection Sample selection 12
- 13. Qualitative Data Collection • Participants Community Health Volunteers (FGDs), Community Health Extension Workers (IDIs) and Sub-County Health Records Information Officers (IDIs) Key Informants: Facility In-Charges (IDIs) and Sub-County Community Health Strategy Focal Persons (IDIs) • Topics explored Understanding of data quality Data flow Data source Data collection Data collation Data analysis Data reporting Data use 13 Referrals
- 14. Data Quality Assessment – 7 indicators selected for calculation of data verification ratios • Pregnant woman referred for ANC • Pregnant woman referred for skilled delivery • Delivered by skilled attendant • Child 0-11 months referred for immunization • Child 0-59 months participating in growth monitoring • Child 6-59 months with mid-upper arm circumference (Red) indicating severe malnutrition • Child 6-59 months with mid-upper arm circumference (Yellow) indicating moderate malnutrition 14
- 15. Data Quality Assessment – calculation of data verification ratios Level 1: Value reported in Community Health Extension Workers Summary Reaggregated total of values recorded in Service Delivery Log Books Level 2: Value reported in National Health Information System Value reported in Community Health Extension Workers Summary 15
- 16. Data Quality Assessment – interpretation of data verification ratios 100% = PERFECT MATCH < 100% = UNDER-REPORTING > 100% = OVER-REPORTING * Admon et al., 2013, Otieno et al., 2012 90-110%* 16
- 17. Results - Data Quality Assessment Community Units Township (Kitui) Museve (Kitui) Maili Saba (Nairobi) Bangladesh (Nairobi) Number of Community Health Workers at time of study 45 40 50 14 Number of Service Delivery Log Books reaggregted 0 0 33 13 17
- 18. Results - Data Quality Assessment • No data reported for at least 12 months • Data Quality Assessment not possible • Partly attributed to devolution and establishment of a new County community health programme Kitui County • Data verification ratios level 1: 0 – 260% • Data verification ratios level 2: 0 – 100% Nairobi County 18
- 19. Qualitative Results - Barriers 19 • Data source Lack of data collection and reporting tools/referral tools Sub-optimal design of tools • Data collection Inadequate training on data management Inconsistent understanding of indicators • Data collation Incomplete data collection Lack of guidelines for data verification
- 20. 20 • Data analysis Lack of guidelines for data analysis • Data reporting Late submission or no submission of data to the higher level • Data use Lack of feedback to CHEWs and CHVs on performance Lack of feedback to communities on their health status • Referrals Poor linkage between Community Units and Primary Healthcare facilities
- 21. 21 Conclusions and Recommendations • Availability of data collection and reporting tools is a prerequisite - PROVIDE • Regular training is necessary to increase reliability of data collection and reporting - TRAIN • Accountability and ownership is fostered via regular feedback and supportive supervision - REGULAR DATA QUALITY ASSESSMENTS
- 22. 22 Acknowledgements • Meghan Bruce-Kumar • Robinson Karuga • Millicent Kiruki • Maryline Mireku • Robbie Mulwa • Nelly Muturi • Dorothy Njeru • Lilian Otiso • Miriam Taegtmeyer • All CHVs, CHEWs, Sub-County CHS Focal Persons, Sub-County Health Record Information Officers and Facility In-Charges that participated in this study
- 23. 23 Regeru Njoroge Regeru Technical Officer LVCT Health Rregeru@lvcthealth.org @rnregeru The USAID SQALE CHS Program is made possible by the generous support of the American people through the United States Agency for International Development (USAID) and is implemented under cooperative agreement number AID-OAA- A-16-00018. The program is managed by prime recipient, Liverpool School of Tropical Medicine www.lvcthealth.org www.reachoutconsortium.org www.usaidsqale.reachoutconsortium.org
Editor's Notes
- In low- and middle-income countries such as Kenya, there is an inadequate number and maldistribution of skilled health professionals. Community Health Workers have emerged as a response to this human resources for health crisis by (in theory) increasing equitable access to essential services by serving disadvantaged populations thus contributing to Universal Health Coverage. In 2006, as a response to a high maternal mortality rate and rising infant and under-five mortality rates, Kenya developed the Community Health Strategy programme to reverse worsening trends in health outcome indicators and contribute towards achievement of Universal Health Coverage. Similar to Community Health Workers elsewhere, Community Health Workers in Kenya collect data from the households they visit on a routine basis. Such data is essential in providing information on the health status of specific communities, health determinants and the performance of the Community Health Strategy programme.
- There are six levels of healthcare service delivery in Kenya. Level 1 refers to Community Health Services and these are delivered by the Community Health Strategy programme. Primary Care Services comprise dispensaries (level 2) and Health Centres (level 3), including those managed by non-state action. They provide: disease prevention and health promotion services; linkage to community units; basic outpatient diagnostic, medical, surgical and rehabilitative services; ambulatory services; and inpatient services for emergency clients awaiting referral, clients for observation, and normal delivery services. County Referral Health Services comprise primary (level 4) and secondary (level 5) hospitals and services in the country. These form the County Health System together with those managed by non-state actors. They provide: comprehensive in-patient diagnostic, medical, surgical, habitative and rehabilitative care, including reproductive health services; specialized outpatient services; and facilitate, and manage both vertical and horizontal referrals. National Referral Services are comprised of all tertiary (level 6) referral hospitals, national reference laboratories and services, government-owned entities, blood transfusion services, research and training institutions. These provide highly specialized services including: general specialization; discipline specialization; and geographical/regional specialization. The focus is on high specialized healthcare, for area/region of specialization as well as training and research services on issues of cross-country importance.
- Following the declaration of a new Constitution of Kenya in 2010, a devolved system of governance came into place in 2013 with the establishment of 47 counties each with its own locally-elected County Government. Within each county are sub-counties that represent the administrative units through which County Governments provide functions and services. This means that within the Community Health Strategy programme, it is sub-county health management teams that coordinate community health services (McCollum et al., 2015). Community health services within a sub-county are organised around Community Units. A Community Unit consists of 5000 people (approximately 1,000 households) living in a specified location. There are two cadres of healthcare worker that provide community health services: CHVs and CHEWs. According to the Community Health Strategy programme: • One CHV should serve 20 households or 100 people • One CHEW should supervise and support 25 CHVs Therefore, a Community Unit of 5000 people requires 50 CHVs being supervised by two CHEWs (Ministry of Health (Kenya), 2006)
- CHVs live in and are chosen by members of the Community Units in which they serve; they provide community health services on a voluntary basis. CHEWs are trained, government-employed healthcare workers. Each Community Unit has an allocated link primary healthcare facility that is where community members should be referred to for further care when necessary (Ministry of Health (Kenya), 2006).
- The components of a Health Information System can be conceptualised by considering how data cycle through an organisation. There are six key stages that data cycle through: “data source, data collection, data collation, data analysis, data reporting and data usage” (Pact Inc., 2014). This data management cycle represents the process through which raw data is converted into information that can be used for planning and management (Lippeveld et al., 2000, Pact Inc., 2014). In addition, before any Health Information System can begin operating, the information needs of stakeholders must be decided upon. This involves selecting indicators that are action-oriented and that serve as the basis for decision-making by healthcare providers and healthcare managers (Lippeveld et al., 2000).
- The Health Information System of a Community Unit should collect, collate, analyse and report data based on a uniform list of indicators that reflect the health services provided by CHVs (Ministry of Health (Kenya), 2006).
- The term performance when describing any routine Health Information System refers to “data quality and continuous use of routine information for decision-making” (Hotchkiss et al., 2012). It is important to be clear then on how the quality of data is defined. Data quality is “recognised as a multi-dimensional concept” (Chen et al., 2014). A dimension of data quality refers to a particular attribute of data. A review study of 39 publications assessing Health Information System performance identified a total of 49 attributes of data quality (Chen et al., 2014). This study focussed on seven of the most commonly used attributes of data quality according to MEASURE Evaluation: Accuracy – Also known as validity. Accurate data are considered correct – the data measure what they are intended to measure. Accurate data minimize errors (e.g. recording or interview bias, transcription error, sampling error) to a point of being negligible. Reliability – Reliable data are generated by a Health Information System based on protocols and procedures that do not change according to who is using them and when or how they are used. The data are reliable because they are measured and collected consistently. Precision – This means that the data have sufficient detail. A Health Information System lacks precision if it is not designed to record the sex of the individual who received a particular health service. Completeness – Completeness means that an information system from which the results are derived in appropriately inclusive; it represents the complete list of eligible persons or units and not just a fraction of the list. Timeliness – Data are timely when they are up-to-date (current), and when the information is available on time. Integrity – Data have integrity when the system used to generate them is protected from deliberate bias or manipulation for political or personal reasons. Confidentiality – Confidentiality means that clients are assured that their data will be maintained according to national and/or international standards for data. This means that personal data are not disclosed inappropriately, and that data in hard copy and electronic form are treated with appropriate levels of security (e.g.: kept in locked cabinets and in password-protected files).
- In low- and middle-income countries, concerns about the quality of data reported by Community Health Workers are frequently cited. Because of these concerns, these data are not regularly used when it comes to decision- and policy-making (Wagenaar et al., 2015b). The same is true of Kenya in which qualitative findings from a study exploring data demand and use in Kenya revealed that stakeholders believe the quality of data reported by Community Health Volunteers and Community Health Extension Workers is low. To date however, there are only a small number of studies that have formally appraised the quality of data collected and reported by CHVs and CHEWs. Furthermore, only a few studies have focused on factors affecting the quality of data reported by CHVs and CHEWs in Kenya. This study contributes to the existing field of knowledge by assessing data quality in Community Units in which such assessment has never taken place and by identifying factors that contribute positively or negatively to data quality. Thereafter steps can be taken to strengthen the CHISs in these sites such that the data reported by CHWs and CHEWs can be used for planning and action at the source of collection and at higher levels, for example sub-county level.
- The research questions of this study were: • What is the quality of data reported by CHVs, CHEWs and Sub-County Health Records Information Officers for selected Community Units in Kitui and Nairobi counties? • What are the enablers and barriers to reporting high-quality data in these Community Units? The aims of this study were twofold: • to determine the quality of data reported by CHVs, CHEWs and Sub-County Health Records Information Officers for selected Community Units; and • to identify factors that contribute positively or negatively to data quality The specific objectives of this study that answered these questions and met these aims are: To perform a Routine Data Quality Assessment of the Health Information Systems of selected Community Units in Kitui and Nairobi counties 2. To explore perceptions of enablers and barriers to generating high-quality data from multiple perspectives within and outside the Community Health Strategy programme in these Community Units
- To calculate data verification ratios, it was necessary to select indicators from the source documents being assessed. The source documents for this study were: Service Delivery Log Book, CHEW Summary and the National Health Informaiton System. The goal of the research and support that LVCT Health is providing to this study’s selected Community Units is reduction in maternal and child mortality. For this reason, indicators were chosen from the Service Delivery Log Book that correspond to four fundamental areas of maternal and child healthcare: antenatal care (ANC); delivery by skilled healthcare professionals; immunization; and nutrition. These indicators are also found in the CHEW Summary and National Health Information System.
- The data verification ratio is a measure of the consistency of reporting from one level to the next. It is expressed as a percentage. A data verification ratio of 100% indicates complete consistency of the value reported by one level and the next. A data verification ratio less than 100% indicates underreporting (at the higher level) whereas a data verification ratio greater than 100% indicates overreporting (at the higher level). This study defined the quality of the data reported by CHEWs and sub-county HRIOs as high if the data verification ratio was 90-110%. That is to say at Level 1, the data reported was considered high quality if the value reported by the CHEW Summary was ≤10% or ≥10% than the reaggregated total of the data recorded in the Service Delivery Log Books. At Level 2, the data reported was considered high quality if the value reported in the National Health Information System was ≤10% or ≥10% than the value reported by the CHEW Summary. This threshold was decided upon because previous studies that have assessed data quality reported by Community Health Workers suggest that this is the level of consistency in reporting necessary to be trustworthy for use in decision-making in programme management (Admon et al., 2013, Otieno et al., 2012).
- The complete deficit of Service Delivery Log Books from Township and Museve to reaggregate is because Community Health Volunteers in these Community Units do not have Service Delivery Log Books. The partial deficit of Service Delivery Log Books from Maili Saba and Bangladesh prompted a review of two particular pieces of information captured by the CHEW Summary. These are ‘Number of CHWs’ and ‘Total CHWs reported’. ‘Total CHWs reported’ refers to the number of Service Delivery Log Books that were aggregated to complete the CHEW Summary for a particular month. This showed that in Maili Saba, for all three months, CHEW Summaries were completed using only a fraction of the total number of Service Delivery Log Books that should have been available. This was also the case in Bangladesh for two months out of three.
- In the Community Units in Nairobi County: Data reported by Community Health Volunteers does not match with data reported by Community Health Extension Workers Data reported in National Health Information System more frequently matches the data reported by Community Health Extension Workers
- Data reported by Community Health Workers does not match with data reported by Community Health Extension Workers Data reported in National Health Information System more frequently matches the data reported by Community Health Extension Workers
- Data reported by Community Health Workers does not match with data reported by Community Health Extension Workers Data reported in National Health Information System more frequently matches the data reported by Community Health Extension Workers