Evaluators can use various data collection instruments for assessments depending on the type of evaluation, level of evaluation, audience, and available resources. Some key considerations when selecting an instrument include what is being assessed, such as knowledge, skills, attitudes, or behaviors. It is important for instruments to be valid and reliable, which can be ensured through field testing with a sample of participants. Using mixed methods with both quantitative and qualitative instruments provides more accurate and valid evaluation results by minimizing the weaknesses of individual instruments. Common data collection instruments include interviews, focus groups, questionnaires, observations, tests, and reviews of various documents. The right instrument depends on factors like the audience and type of project activities.

1. UNIVERSITY OF THE ARMED FORCES
SOCIO-EDUCATIONAL PROJECT
DESIGNING EDUCATION PROJECTS
Part IV.
DATA COLLECTION INSTRUMENTS
DAYANA CRISTINA TAMAYO

2. DATA COLLECTION INSTRUMENTS
INTRODUCTION
Evaluators can use different data collection instruments according some considerations in
order to select the most appropriate
These are some of the considerations to take into account:
 Type of evaluation (need assessment, process, outcome)
 Level of evaluation (KASA changes, practices, SEE conditions)
 The audience (children, adults, casual or organized groups)
 The amount of resources available (time, money)

3. DATA COLLECTION INSTRUMENTS
Matching Data Collection Instruments to What is Being
Assessed
Some specific data collection instruments can evaluate or document the level of skills,
behavior, attitude development or knowledge gain. For example:
 Interview: evaluates knowledge, attitude, and maybe behavior
 Observation: assesses skills and behavior
 Questionnaire: evaluates knowledge, skills, attitude, and
maybe behavior

4. DATA COLLECTION INSTRUMENTS
Matching Data Collection Instruments to What is Being
Assessed
 Literature Review: assesses knowledge, skills, attitude,
and behavior
 Test: evaluates knowledge and skills
 Concept maps: evaluates knowledge, and maybe attitude

5. DATA COLLECTION INSTRUMENTS
Matching Data Collection Instruments to What is Being
Assessed
 Focus group: evaluates attitude, and maybe knowledge
 Document or product review: evaluates knowledge,
skills, and maybe attitude
 Case study: assesses knowledge, skills, attitude, and
behavior

6. When selecting the data collection instruments for project evaluation, it is necessary to validate them
before to use, taking steps to ensure instruments are valid and reliable through two considerations:
DATA COLLECTION INSTRUMENTS
Validity and Reliability of Evaluation Instruments
CONSIDERATIONS
Validity Reliability

7.  Validity: it is the extent to which it measures what it pretends to measure
 Reliability: it is the extent to which it yields responses when is administered
DATA COLLECTION INSTRUMENTS
Validity and Reliability of Evaluation Instruments

8. DATA COLLECTION INSTRUMENTS
Validity and Reliability of Evaluation Instruments
Field or Pilot
testing
Evaluators
• can measure validity and reliability of evaluation
instruments
• test a representative sample of project participants
with instruments selected to gather information

9. DATA COLLECTION INSTRUMENTS
Validity and Reliability of Evaluation Instruments
Field or Pilot Testing
The field test should answer to:
 Necessary information to answer evaluation questions
 The way of collecting information
 Consistency and accuracy in information obtained
Although there are other techniques to validate instruments using statistical methods, but they depend
of:
 expert evaluation team
 data collection methods
 project’s longevity
 and practicality

10. DATA COLLECTION INSTRUMENTS
MIXED METHODS
The using of a combination of data gathering like quantitative and qualitative in
evaluation. Using both type of instruments can provide more accuracy and validity of the
information and results from evaluation, because each instrument has individual strengths
and weaknesses, and using together, the weaknesses are minimized.
QUANTITATIVE QUALITATIVE
MIXED
METHODS

11. DATA COLLECTION INSTRUMENTS
MIXED METHODS
QUANTITATIVE
METHODS
Count events and number of
participants
Determine cost/participants
Perform statistical analyses
Generalize findings beyond
actual respondents

12. QUALITATIVE
METHODS
Record participants’ own words
about experiences (perceptions,
descriptions)
Create narratives in-depth view
and understanding of the
evaluation context
Participate few participants and
cannot generalize results
DATA COLLECTION INSTRUMENTS
MIXED METHODS

13. DATA COLLECTION INSTRUMENTS
Types of Data Collection Instruments
Many of the data collection instruments are oriented to a specific purpose and project,
having advantages and challenges of using them.
There are 9 data collection instruments:
understand
someone’s
experiences
provides full
range of
information
be hard to
analyze and
compare
INTERVIEWS
explore a
topic in-depth
quickly obtain
common
impressions
hard to
analyse
responses
FOCUS
GROUP obtain lot of
information
from people
complete
anonymously
doesn’t yield
full story
QUESTIONNAIRES

14. DATA COLLECTION INSTRUMENTS
Types of Data Collection Instruments
gather
accurate
information
adapt to
events as
they occur
be difficult to
interpret
behaviors
OBSERVATION gather
information
on the
audience
provides
evidence
about the
problem
may not
address
specific
questions
LITERATURE
REVIEW
determine
audience’s
current state
of knowledge
results are
easily
quantified
limited
availability
TESTS

15. gather
information of
someone’s
understanding
be a better tool
for visual
learners
be challenging
and time
consuming to
score
CONCEPT
MAPS
gather with
information
already exists
yields
historical
information
takes much
time
DOCUMENT OR
PRODUCT
REVIEW
understand
client’s
experiences in
a project
powerful
means to
portray project
to outsiders
Information
cannot be
generalized
CASES STUDIES
OR PEER
REVIEW
DATA COLLECTION INSTRUMENTS
Types of Data Collection Instruments

16. DATA COLLECTION INSTRUMENTS
Selecting the Right Data Collection Instrument
According the type of project, activities and the information needed the project coordinator can select the
appropriate evaluation data collection instrument.
By the audience:
 Adults (interview, focus group, survey, observation, test, case study)
 Cultural groups (focus group, observation)
 Teens (interview, observation, test, concept map, case study)
 Children (observation, test, concept maps)
By the project/activity:
 Talk/lecture (test, survey, concept map)
 Workshop (survey, test, interview, focus group, concept map)
 Training (interview, focus group, observation, survey, test)
 Exhibit (interview, focus group, survey, observation, concept map)
 Media (interview, focus group, survey)
 Interactive media (interview, focus group, observation, concept map)