1. AAROHI
Evaluation of trained Accredited Social Health Activist
(ASHA) workers and Supervisors associated with
‘Aarogya Project’ regarding their knowledge attitude
and practices about Health Management Information
System.
Evaluation Report
Dr. Tanmay Singh, MPH (Candidate), BDS
7/1/2015
Abstract: The study was conductedwith purpose of evaluating the knowledge attitude and practices about
health management and information system of ASHAs, Supervisors and Area Coordinator associated with
the ‘Aarogya Project’implemented by Aarohi in the remote Okhalkanda blockof Nainital, Uttarakhand.
Aarohi. A cross-sectional study was conducted in the three field areas Karyal, Khansyu and Patlot. In the
evaluation the performance of ASHAs, supervisors and area coordinator was assessed using PRISM
frameworkquestionnaires whichwere modified accordingto the study. It was observed 18% of ASHAs
and 23% of supervisors obtained ‘A’ grade. Chi square statistics of ASHAs and supervisors was significant
whichshows that workexperience is an important factordetermining the performance in the evaluation.

2. Evaluation of trained Accredited Social Health Activist (ASHA) workers and
Supervisors associated with ‘Aarogya Project’ regarding their knowledge attitude
and practices about Health Management Information System.
EXECUTIVE SUMMARY
Health management and information system is a corecomponent of both preventive and curative
health systems, aimed towards monitoring, evaluation, planning and management. This study was
conducted withpurpose of evaluating the knowledge attitude and practices about health
management and information system of ASHAs, Supervisors and Area Coordinator associated with
the ‘Aarogya Project’implemented by Aarohi in the remote Okhalkanda blockof Nainital,
Uttarakhand.
Aarohi has been implementing the Aarogya projectwhich aims to strengthen community-driven
processes that will enable households to increase controlover and improve the health of mothers
and children. To achieve this goal essential health care based on practical, scientifically sound &
socially acceptable methods & technology is being made accessible to individuals and families.
A cross-sectional study was conducted in the three field areas Karyal, Khansyu and Patlot. In the
evaluation the performance of ASHAs, supervisors and area coordinator was assessed using PRISM
frameworkquestionnaires whichwere modified accordingto the study. The participants in the
study were evaluated fortheir knowledge, attitude and perception about HMIS. Three grade scales
was used for evaluation. It was observed 18% of ASHAs and 23% of supervisors obtained ‘A’ grade.
Chi square statistics of ASHAs and supervisors was significant which shows that workexperience is
an important factor determining the performance in the evaluation.
Despite the continuous capacity building of ASHAs and supervisors lacunae still exist in the
knowledge about the importance of data collectionand objectives of data collectionprocess. There
is need of specific capacity building in the arena of HMIS.

3. SUMMARY MATRIX OF FINDINGS, EVIDENCE AND RECOMMENDATIONS
Findings
1. 18% of ASHAs and 23% of supervisors obtained ‘ A’ grade.
2. The maximum percentage of ‘A’ grade forASHAs has been obtained in Karayal area whichis
11%.
3. Khansyu field area has maximum percentage of ‘A’ grade forsupervisors which are 18%.
4. 99% match of data in pen and paper, monthly progress report and computer.
5. 50% of supervisors in Karayal, 28% of supervisors in Khansyu and 43% of supervisors in
Patlotwere able to correctly calculatepercentage and rates.
Evidence
1. Analysis of data collectedvia questionnaires on MicrosoftExcel Professional2010 edition
2. Manual matching of data in monthly progress report withASHA records and computer.
Recommendation
1. There is a need to increase the understanding of importance data collectionand processing.
2. Capacity building in the arena of HMIS at all levels of project.
INTRODUCTION
Background and context
Expenses on health systems are an area of rapid growth. Health management and information
systems are worldwide implemented for decision making, monitoring and evaluation. The issue of
data quality is of major concern to most of the organizations. Aarohi has a well-developedHMIS but
there is a lackin effectivenessand efficiency due to lackof knowledge and perception. The data
collectedis accurate but strategy forits implementation in decision making is lacking. This
evaluation is undertaken to understand the knowledge, attitude and perception among the field
level workers at all level.

4. Evaluation methodology
Across-sectional study was conductedat 70 villages of Okhlakanda block divided into Karyal,
Khansyu and Patlot area. All the ASHA, supervisors and area coordinator workingin the ‘Aarogya
project’ were included in the study. 46 ASHAs out of 60 and 22 supervisors out of 24 participated
in the evaluation and 14 ASHAs and 2 supervisors were excluded as they were absent on the day of
evaluation. Self- administered questionnaires of the PRISMframework were used in evaluation
whichwas modified according tothe study design and need. The collected data was analyzed with
MicrosoftExcel2010 Professional edition and applying Chi square test. The observational
component of the study includes manual matching of data in monthly progress reports, ASHA
records and data entered in the computer. Eachof the study participants was assured by the
authors that the study findings wouldnot have any impact on their job.
Limitations to the evaluation
In the evaluation the pre-test of questionnaire was not performed due to remoteness, logistics and
time constraints. The sample size is based on convenience and is not applicable to the complete
blockas the additional training is only being done in the coverage area of Aarohi only. The study is
performed only in coveragearea of ‘Aarogya Project’and lacks overall validity.
EVALUATION FINDINGS
Complete Results of ASHA evaluation
questionnaire
Complete Results of Supervisor evaluation
questionnaire
Grade
A
Grade
B
Grade
C
Grade A Grade B Grade C
Total 8 19 17 Total 5 5 12
Percentage 18% 44% 38% Percentage 23% 23% 54%

5. The bar graph below shows the area wise comparison on the basis of grades. The questionnaire was
administered to 44 ASHA's. The questionnaire was administered to 15 ASHAs in Karayal, 22 ASHAs
in Khansyu and 9 ASHAs in Patlot.
The bar graph below shows the area wise comparison on the basis of grades. The questionnaire was
administered to 22 Supervisors. The questionnaire was administered to 8 Supervisor in Karayal, 7
supervisors in Khansyu and 7Supervisors in Patlot
Bar Charts of Supervisors grades compared with work experience.

6. Bar Charts of ASHA grades compared with work experience
CONCLUSIONS
The ASHA is an important link between the community and the service provider. The
HMIS depends on primary data collected on the field level. The capacity building in HMIS is
very important to efficiently implement HMIS.
Aarohi implements a system of HMIS based on data collected by ASHAs which is then
reported on monthly basis to the area coordinator level. In an attempt to decrease the
workload a new monthly progress report has been introduced three months back. This
reporting format is very well adapted to the needs of ASHA and supervisors. On close
observation the reporting format provides concise monthly reporting of all indicators for
the villages under the specific supervisor. The area coordinator then sends the collected
monthly reports to the assistant coordinator, who is responsible to present this data to
Project advisor. The mechanism of feedback is present at all levels is present but its
implementation suffers the issues which can be attributed to remoteness and connectivity
issues present in field area. All the participant ASHAs get feedback from the supervisors
and similarly feedback is provided at all levels.
Evaluation clearly shows work experience is an important determinant of the performance
for both ASHA and Supervisors. Data accuracy is 100% in all the areas. In the current study,
100% correct knowledge was observed regarding responsibilities of ASHA and 100%
correct knowledge of data accuracy checking methods was observed in the supervisors
who participated in the study.
In the area wise comparison it was observed 25% supervisor and 33% ASHAs in Krayal
obtained grade ‘A’. This performance is attributed to more no of experienced participants.
In Khansyu, 28% of supervisors and 9% of ASHAs obtained grade ‘A’. In Patlot, 14% of
supervisors and none of the ASHAs obtained grade ‘A’. The observed reason is, less work

7. experience, the area came in the coverage of ‘Aarogya Project’ recently and level of
education in the ASHAs. It is also important to talk about the work burden of ASHAs as they
are frontline workers for many government related services; this fact is important
determinant of efficiency of ASHAs.
In the research, it has been observed that Aarohi trained ASHAs are performing at a high
level but comparative study between ASHAs only receiving government provided training
is needed to concretely define the performance. There is further scope of comparative
study at the level of ASHAs.
RECOMMENDATIONS
1. The HMIS has been introduced with a top to bottom approach and the grass root
level workers have limited capacity in the HMIS which demands focused capacity
building for ASHAs
2. The capacity building of supervisors on detecting of errors, possible reasons of these
errors and possible solutions for elimination of errors.
3. The feedback loop is based on monthly feedback and suffers issue related
remoteness, connectivity and time constraints. There is a requirement to focus on
real time data monitoring like Commcare.
4. The data collected has the required accuracy but capacity building on why collect
data can help in decision making and increasing the efficiency for data focused
planning at level of supervisors
5. The new monthly reporting format is user friendly but the column of expected has
been removed. I recommend its re-introduction as it is a simple yet powerful
measure of performance which requires no knowledge of statistical calculation.
6. The sharing mechanism of monthly reports between the area coordinators is
present. This is an important strength but needs to be mainstreamed with
introduction of measures for inter area feedback. As per evaluators observation the
different areas just email the monthly reports to each other.

8. Acknowledgements
I will like to forward my sincere thanks to Aarohi for giving me the opportunity to perform
this evaluation. In particular, I wish to thanks Dr. Sushil Sharma (Vice Chairman, Aarohi),
Nain Singh (Assistant Coordinator Aarogya Project), Ganga Singh (Area Coordinator
Aarogya Project) and Chandra Shekhar (Office Assistant, Aarohi). Finally, I would like to
thank management and staff of Aarohi for providing support at all levels and this
evaluation had not being possible without their support.
References
1. PRISM framework: a paradigm shift for designing, strengthening and evaluating
routine health information system. Anwer Aqil, Theo Lippeveld and Dairiku Hozumi
2. Monitoring and Evaluation Manual Prepared for ADRA International Food Security
Department.
3. Saving Mothers, Giving Life: Monitoring and Evaluation Overview.
4. PRISM: Performance of Routine Information System Management
PRISM Tools for Assessing, Monitoring, and Evaluating RHIS Performance
5. A Review of Data Quality Assessment Methods for Public Health Information
Systems Hong Chen David Hailey, Ning Wang and Ping Yu
Annexures
1. Questionnaires for ASHA and Supervisors
2. Comprehensive data sheet.