Data Collection Tools: Validity & Reliability. Objectives: Discuss types of measurement tools for collecting data for quantitative, qualitative and outcome research. Differentiate between interview guide and interview schedule Discuss reliability and validity of questionnaires. Data: The set of values collected for the variable of each of the elements belonging to the sample Data sources include (Quantitative) Surveys where there are a large number of respondents (esp where you have used a Likert scale) Questionnaires, data collection tools/ instruments Observations (counts of numbers and/or coding data into numbers) Secondary data (government data; SATs scores etc) Analysis techniques include hypothesis testing, correlations and cluster analysis. Data sources include (Qualitative) Interviews (structured, semi-structured or unstructured) Focus groups Questionnaires or surveys Secondary data, including diaries, self-reporting, written accounts of past events/archive data and company reports; Direct observations – may also be recorded (video/audio) Ethnography Data analysis; thematic or content analysis . Data Collection: “The process of gathering and measuring information on variables of interest, in an established systematic fashion that enables one to answer queries, stated research questions, test hypotheses, and evaluate outcomes.” Data Collection Methods: Surveys, quizzes, and questionnaires Interviews Focus groups Direct observations Documents and records. Data Collection Tools for Quantitative Research: Closed-ended Surveys and Online Quizzes Closed-ended surveys and online quizzes are based on questions that give respondents predefined answer options to opt for. There are two main types of closed-ended surveys – those based on categorical and those based on interval/ratio questions. Categorical survey questions can be further classified into dichotomous (‘yes/no’), multiple-choice questions, or checkbox questions and can be answered with a simple “yes” or “no” or a specific piece of predefined information. Interval/ratio questions, on the other hand, can consist of rating-scale, Likert-scale, or matrix questions and involve a set of predefined values to choose from on a fixed scale. Data Collection Tools for Qualitative Research: 1. Open-Ended Surveys and Questionnaires Opposite to closed-ended are open-ended surveys and questionnaires. The main difference between the two is the fact that closed-ended surveys offer predefined answer options the respondent must choose from, whereas open-ended surveys allow the respondents much more freedom and flexibility when providing their answers. 2. In-depth Interviews/ Face to Face Interviews One-on-one (or face-to-face) interviews are one of the most common types of data collection methods in qualitative research. Here, the interviewer collects data directly from the interviewee.

2. Shakir Rahman
BScN, MScN, MSc Applied Psychology, PhD Nursing (Candidate)
University of Minnesota USA
Principal & Assistant Professor
Ayub International College of Nursing & AHS Peshawar
Visiting Faculty
Swabi College of Nursing & Health Sciences Swabi
Nowshera College of Nursing & Health Sciences Nowshera
Data Collection Tools
Validity & Reliability

3. objectives
 Discuss types of measurement tools for collecting
data for quantitative, qualitative and outcome
research.
 Differentiate between interview guide and
interview schedule
 Discuss reliability and validity of questionnaires

4.  Data:
 The set of values collected for the variable of each of the
elements belonging to the sample
 Data sources include (Quantitative)
 Surveys where there are a large number of respondents
(esp where you have used a Likert scale)
 Questionnaires, data collection tools/ instruments
 Observations (counts of numbers and/or coding data into
numbers)
 Secondary data (government data; SATs scores etc)
 Analysis techniques include hypothesis testing, correlations
and cluster analysis
Data

5.  Data sources include (Qualitative)
 Interviews (structured, semi-structured or unstructured)
 Focus groups
 Questionnaires or surveys
 Secondary data, including diaries, self-reporting, written
accounts of past events/archive data and company reports;
 Direct observations – may also be recorded (video/audio)
 Ethnography
 Data analysis; thematic or content analysis
Data

6. Data Collection
 “The process of gathering and measuring
information on variables of interest, in an
established systematic fashion that enables one
to answer queries, stated research questions, test
hypotheses, and evaluate outcomes.”

7. Data Collection Methods
 Surveys, quizzes, and questionnaires
 Interviews
 Focus groups
 Direct observations
 Documents and records

8. Data Collection Tools for Quantitative
Research
 Closed-ended Surveys and Online Quizzes
 Closed-ended surveys and online quizzes are
based on questions that give respondents
predefined answer options to opt for. There are
two main types of closed-ended surveys – those
based on categorical and those based on
interval/ratio questions.

9. Cont….
 Categorical survey questions can be further
classified into dichotomous (‘yes/no’), multiple-
choice questions, or checkbox questions and can
be answered with a simple “yes” or “no” or a
specific piece of predefined information.
 Interval/ratio questions, on the other hand, can
consist of rating-scale, Likert-scale, or matrix
questions and involve a set of predefined values
to choose from on a fixed scale.

10. Data Collection Tools for Qualitative
Research
 1. Open-Ended Surveys and Questionnaires
 Opposite to closed-ended are open-ended
surveys and questionnaires. The main difference
between the two is the fact that closed-ended
surveys offer predefined answer options the
respondent must choose from, whereas open-
ended surveys allow the respondents much more
freedom and flexibility when providing their
answers.

12. Cont….
 2. In-depth Interviews/ Face to Face Interviews
 One-on-one (or face-to-face) interviews are one
of the most common types of data collection
methods in qualitative research. Here, the
interviewer collects data directly from the
interviewee. Due to it being a very personal
approach, this data collection technique is perfect
when you need to gather highly personalized
data.

13. Cont….
 Depending on your specific needs, the interview
can be informal, unstructured, conversational,
and even spontaneous (as if you were talking to
your friend) – in which case it’s more difficult and
time-consuming to process the obtained data – or
it can be semi-structured and standardized to a
certain extent (if you, for example, ask the same
series of open-ended questions).

14. Cont….
 3. Focus group discussion
 The focus group data collection method is essentially
an interview method, but instead of being done 1-on-
1, here we have a group discussion.
 Whenever the resources for 1-on-1 interviews are
limited (whether in terms of people, money, or time) or
you need to recreate a particular social situation in
order to gather data on people’s attitudes and
behaviors, focus groups can come in very handy.
 Ideally, a focus group should have 3-10 people, plus a
moderator. Of course, depending on the research
goal and what the data obtained is to be used for,
there should be some common denominators for all
the members of the focus group.

15. Cont….
 4. Direct observations
 Direct observation is one of the most passive
qualitative data collection methods. Here, the data
collector takes a participatory stance, observing the
setting in which the subjects of their observation are
while taking down notes, video/audio recordings,
photos, and so on.
 Due to its participatory nature, direct observation can
lead to bias in research, as the participation may
influence the attitudes and opinions of the researcher,
making it challenging for them to remain objective.
Plus, the fact that the researcher is a participant too
can affect the naturalness of the actions and
behaviors of subjects who know they’re being
observed.

16. Validity & Reliability
of data collection
tool

17.  Validity:
 Validity is defined as the extent to which an
instrument accurately measuring what it has
intended to measure.
 Example, weight machine for measuring weight of an
object.
 It’s important to consider validity and reliability of the
data collection tools (instruments) when either
conducting or critiquing research.
 Reliability:
 The extent to which a research instrument consistently
has the same results if it is used in the same situation
on repeated occasions.
 Also called as Internal Consistency
 Example is an alarm clock that rings at 7:00 each
morning, but is set for 6:30. It is very reliable (it

18. Types of Validity
Type of Validity Description
Content Validity The extent to which a research instrument
accurately measures all aspects of a
construct
Construct
Validity
The extent to which a research instrument
(or tool) measures the intended construct
Criterion Validity The extent to which a research instrument
is related to other instruments that
measure the same variables

19. Attributes of Reliability
Attribute Description
Homogeneity
(or internal
consistency)
The extent to which all the items
on a scale measure one
construct
Stability The consistency of results using
an instrument with repeated
testing
Equivalence Consistency among responses of
multiple users of an instrument,
or among alternate forms of an
instrument

20. Reliability measurement
 Cronbach’s α is the most commonly used test to
determine the internal consistency of an
instrument.
 Instruments with questions that have more than
two responses can be used in this test.
 The Cronbach’s α result is a number between 0
and 1. An acceptable reliability score is one that
is 0.7 and higher.

21. Interview Guide & Interview
Schedule
 In quantitative interviews, an interview schedule is
used to guide the researcher as he or she poses
questions and answer options to respondents. An
interview schedule is usually more rigid than an
interview guide.
 It contains the list of questions and answer options that
the researcher will read to respondents.
 For qualitative research, An interview guide is a
document that enables organizations to structure the
way they conduct their candidate interviews. It helps
interviewers to know what to ask about and in what
order and it ensures a candidate experience that is the