This document discusses selecting appropriate data collection instruments for project evaluations. It provides an overview of various instruments including interviews, focus groups, surveys, observations, tests, concept maps, documents/product reviews, and case studies. It notes the strengths and weaknesses of each instrument and considers what types of audiences and projects each instrument is best suited for. The key aspects to match are the type of information needed, the audience being assessed, and the nature of the project or activity being evaluated. Selecting the right instrument requires understanding these context factors.

1. Socio-educational project
Collaborative Work
Second Term
Andrea Vallejo
Edison Yepez
Antonio Rodriguez
DATA COLLECTION
INSTRUMENTS

2. Project coordinators and evaluators have an array of data collection
instruments available to them. Although selecting the most appropriate
instrument requires thought and
careful consideration, the selection process is also shaped by a number of
decisions that have already been made. The type of evaluation being
considered (i.e., needs assessment, process, or outcome) will determine, to
some extent, the most appropriate data collection
instrument(s). Likewise, the level of evaluation being conducted (i.e., KASA
changes, practices, SEE conditions), the audience involved (e.g., children vs.
adults, casual visitors
vs. organized groups), and the amount of resources available (e.g., time,
money) will all help determine which data collection instruments should be
used. Each data collection strategy comes with strengths and weaknesses.
INTRODUCTION

3. Matching data Collection Instruments to
what is being Assessed
• an evaluation would want to
address the degree to which
participants increased their level
of understanding, developed a
particular set of skills,
• or further considered their
attitudes on a topic
1
• Some data collection
instruments are
particularly adept at
assessing knowledge
gain
2 • Others are
appropriate for
documenting skill or
attitude development.
3

5. VALIDITY
Before you begin to collect data for
your project evaluation, make sure
the instruments
or tools you plan to use have been
validated.
Two factors that
should
be taken into
consideration when
validating your
evaluation
instruments are
validity
and reliability.
Test the validity of
your evaluation
instruments by
asking a panel
of experts if your
questions adequately
sample the content
you wish to convey
he reliability of an
instrument is the
extent to which it
yields consistent
responses each time
it is administered.

6. pilot testing means trying out
your
instruments under conditions
similar to those you expect to
have for the evaluation
conduct the field test on a
representative sample of
people who are likely to
participate in the actual
project
Is the questionnaire providing
the information needed to
answer my evaluation
questions?
Are multiple interviewers
collecting information in the
same way?
How consistent is the
information obtained with the
questionnaire?
How accurate is the
information obtained with the
observation rubric?
PILOT TESTING

13. Understand impressions or
experiences about their answer or
questionnaires.
Provides a full range and depth of
information
 Can take much time , hard to
analyze and compare , and
generalized is limited
To explore a topic in depth and
it is useful in evaluation and
marketing.
An efficient way to gather
information in short time.
Hard to analyze responses, need
a good moderator and generate
other answer as stronger individual
INTERVIEW FOCUS GROUP
TYPES OF DATA COLLECTION
INSTRUMENTS

14. Easy to obtain a lot of
information from people.
Obtain lots of data, easy to
compare and analyze, and it is
inexpensively.
Don`t get careful feedback and
wording can bias client’s responses
To gather accurate information
about how a project operate and
the process.
View operation of a project and
then they are adapted to events.
Difficult to interpret,
categorize and can influence on
participant behaviors.
QUESTIONNAIRES AND
SURVEYS OBSERVATION

15. To gathers information about
audience and/or the issues.
Can provide a lot of information
in short time and provide evidence
about the problem.
Not address specific questions, not
flexible means data and reports can
be incomplete

To determine a current state of
knowledge or skill about regarding
the issue.
Help to identify a problem and
result are easily quantified and
compared
Limited available of validity test for
specific situation
LITERATURE REVIEW TEST

16. To gather information about
someone`s understanding of
complex subject or topic
 A visual tool for learners that
produce qualitative and quantitative
data.
 To take and training to administer
and complete properly, difficult to
analyze and interpret.
To gather information
on how the project
operate ,for example, review
of applications, finances,
memos, minutes etc.
Propose historical
information and don`t
interrupt project or client’s
routine in the project.
To take much time, the
information could
be incomplete, and not a
flexible means to obtain data
result
CONCEPT MAPS DOCUMENTS OR PRODUCT
REVIEW

17. Full understanding and experience
in a project to conduct a
comprehensive examination.
Full depict client`s experience in
project input, process, and result.
To involve quite time to collect,
organize and describe information
and may not be generalized.
CASE STUDY OR PEER
REVIEW

18. To provide the general observation
about the types of data gathering
instruments.
 The use of any particular method
will depend on particular context and
type of information needed
SELECTING THE RIGHT DATA
COLECTION INSTRUMENT

19. Evaluation Instrument
Audience
Intervie
w
Focus
group
Survey Observ
ation
Test Concept
Maps
Case
study
Adults/who Know you or
your organization
Good Good Good Good Good Fair Good
Adult`s/who don`t Know
you or your organization
Good Good Good Good Fair to
fair
Fair Fair
Decisions-makers/policy
makers/community leaders
Good Fair Good Good
to fair
Fair Poor Good
Cultural group (oher than
your own)
Fair to
poor
Good
to fair
Fair to
poor
Good
to fair
Poor Poor Fair to
fair
Teachers Good Good Good Good Good Good Good
Teens Good Fair Fair Good Good Good Good
Eight to twelve years
old
Fair Fair Fair Good Good Good Fair
Three to seven years old Poor Poor N/A Good N/A N/A Fair to
poor
A. INSTRUMENT VERSUS
AUDIENCE

20. Evaluation Instrument
Project/Activity
Intervie
w
Focus
Group
Survey test Observat
ion
Concept
Map
Take lectute (short single
event)
Poor Poor Fair Good to
fair
Poor fair
Workshop(single event) Fair Fair Fair Good Fair Fair
Series (multiple meetings) Good Fair Good Good Fair Fair
Training (skill buildings Good Good Fair Fair Good Fair
Tour (adults) Fair Fair Good Good Fair Fair
Tour (3-6 years old) Good Fair Fair Fair Fair Fair
Event/festival Good Fair N/A Good Fair N/A
Interpretive sign Good Fair FAir Good Fair N/A
Exhibit Good Good Good Fair Good Good
Curriculum packet /materials Good Fair Good Fair Fair Fair
Kit /activities Good Fair Good Fair Good Fair
Publications Good Fair Fair Fair Poor Poor
Media Interactive Media( e.g.
CD)
Good Good Good to
fair
Fair Good Good
Website Good Fair Good to
Fair
Fair Good Good
B. INSTRUMENT VERSUS
ACTIVITY PROJECT