This document discusses instruments used in quantitative research, specifically questionnaires and tests. It provides details on developing and administering questionnaires, including reviewing literature, constructing question items, pretesting questionnaires. Guidelines are given for developing standardized tests as well, such as defining objectives and the target population. Common types of standardized tests are also outlined, including intelligence, aptitude, achievement, and diagnostic tests. The document emphasizes establishing validity and reliability of research instruments.

1. Instruments in Quantitative
Research
Presented by:
Narottam Dev Sharma
BA (Hons), MA in English Language & Literature
Eastern University
MA in TESOL
BRAC Institute of Languages (BIL)
BRAC University
Dhaka, Bangladesh

2. Questionnaire
‡ After the careful formulation of research
questions/hypothesis and sample
selection next step in research chain is
developing data collection instrument.

3. The two most commonly used research
instruments in quantitative research
studies include
‡ Questionnaire and
‡ Tests.

4. Validity and reliability of
instruments:
‡ Validity is the degree to which an
instrument measure what it is purports to
measure. Invalid instruments can lead to
erroneous research conclusions, which in
turn can influence educational decisions.
‡ Reliability is the internal consistency or
stability of the measuring device over time
(Gay, 1996).

5. ‡ A questionnaire is a research instrument
consisting of a series of questions and
other prompts for the purpose of gathering
information from respondents.
Questionnaire is widely used esp. in
descriptive survey studies(Borg & Gall,
1983).

6. Advantages of Questionnaire
‡ Can reach a large number of people
relatively easily and economically.
‡ Provide quantifiable answers.
‡ Relatively easy to analyze.
‡ Less time consuming than interview or
observation.
(Bailey,1982)

7. Steps in Questionnaire
Construction:
A. Reviewing the Literature
B. Deciding what information should be
sought
C. Knowing respondents
D. Constructing Questionnaire Items
E. Reexamination and revision of the questions
F. Pretesting questionnaires
G. Editing the questionnaire and specifying
procedure for use

8. A. Reviewing the Literature:
‡ Before constructing the questionnaire the
researcher must review all the related literature
to see if an already prepared questionnaire is
available to similar his/her topic of study. This
will save time and effort required to construct an
entirely new questionnaire. Changes can be
made as the study demands.

9. B. Deciding what information should be
sought:
‡ List specific objectives to be achieved by
the questionnaire.
‡ Methods of data analysis that will be
applied to the retuned questionnaire
should also be kept in mind

10. C. Knowing respondents:
‡ Researcher must know his target
population in relation to occupation,
special sensitivities, education, ethnicity
language etc.

11. D. Constructing Questionnaire Items:
‡ Each item on the questionnaire must be
developed to measure a specific aspect of
objectives or hypothesis.
‡ Researcher must be able to explain in detail why
a certain question is being asked and how the
responses will be analyzed.
‡ Making ³dummy tables´ that show how item-by-
item results of the study will be analyzed is a
good idea(Borg & Gall, 1983).

12. Rules for constructing items
1.Both open as well as closed ended
questions can be used however closed
ended questions are preferred
2.Clarity of all the items is necessary to
obtain valid results.
3.Short items are preferable to long items as
they are easier to understand.
4.Negative items should be avoided as they
are often misread by many respondents.

13. 5. Avoid ³double-barreled´ items, which require the
subject to respond to two separate ideas with a
single answer.
6. Avoid using technical terms, jargons or big
words that some respondents may not
understand.
7. When a general and a related specific question
are to be asked together it is preferable to ask
general question first. Otherwise it will narrow
the focus of the general question if specific
question is asked first.
8. Avoid biased or leading questions.
(Borg & Gall, 1983)

14. Question Formats
Basic
Open-ended
Questions
Probing
Questions
Clarifying
Questions
Open-ended
Questions
Dichotomous
Questions
Single-coded
Multiple-coded
Multiple-Choice
Responses
Closed-ended
Questions
Labeled
Questions
Unlabeled
Questions
Scaled
Questions
Basic
Question Formats

15. Open ended questions:
Basic open-ended questions
Q. What do you particularly like about Lipton Tea?
_______________________________________________________________________
Probing Questions:
What do you particularly like about Johnson & Johnson baby oil?
____________________________________________________________________
PROBE: Anything else
Close ended questions:
A. Dichotomous questions with no neutral response
Q Do you have a cellular phone?
1 Yes ...................................... without neutral
2 No ...................................... response
b. Dichotomous questions With Neutral Response
Is it likely that you will purchase a cellular phone in the next six months?
1 Yes ...................................... with neutral
2 No ...................................... response
3 Not Sure ..............................
http://www.authorstream.com

16. ‡ Single- and multi-coded multiple choice question
On an average, how much do you spend on newspapers, books and
magazines in a month? (Please check one from the following
responses.)
‡ 1 Less than $15 ...................................
Single-coded
‡ 2 Between $16 & $30 ......................«.
question
‡ 3 Between $31 & $45 .......................«
‡ 4 Between $46 & $60 .....................«..
‡ 5 $60 or more .................................«..
Which of the following household appliances does your household
have?
‡ (Please check as many responses that are applicable to you.)
‡ 1TV «««« 5 VCR «« « Multi coded question
‡ 2 LCD ««««. 6 Microwave «.««
‡ 3 PC «««« 7 Cellular phone«««.
‡ 4 Fax «««« 8 Others «««.
‡ Specify
______________________________________________________
______________________________________________________
____

17. Scaling questions
Scaling questions are special types of closed-
ended questions. They include following
categories of questions.
‡ Behavioral/Attitudinal questions
‡ Agree-Disagree questions
‡ Preference questions
‡ Ranking questions
The questions can be labeled or unlabeled

18. Labeled and Unlabeled Scaling Questions:
Unlabeled scaled-response question
‡ On a scale of 1 to 7 how would you rate the IBM Thinkpad on ease of operation?
Advantages:
‡ Allows respondents to express the degree of his/her intensity of feelings.
‡ Easy to administer and code
Diasadvantage:
‡ Respondent may not relate to the scale well.
Labeled scaled-response question
e.g ³Do you disagree strongly, disagree, agree, or agree strongly with the statement,
µIBM laptops are a better value than Compaq laptops¶?´
Advantages:
‡ Allows a respondent to express the degree of his/her intensity of feelings.
‡ Easy to administer and code.
‡ Respondents can relate to the scale.
Disadvantage:
‡ Scale may be ³forced´ or overly detailed.
http://www.authorstream.com

19. Questionnaire format
Borg and Gall (1983) state following rules for questionnaire
format:
‡ Attractive for the respondent.
‡ Organize such that it¶s easy to complete.
‡ Number questionnaire items and pages.
‡ Instructions should be brief, clear & in bold type.
‡ Organize questions in a logical sequence.
‡ Name and address of the person to whom questionnaire
is to be returned should be mentioned in the beginning
as well as in the end.

20. ‡ Use examples before any item that might be
confusing or difficult to understand.
‡ Begin with a few interesting and non-threatening
items.
‡ Do not put important items at the end of a long
questionnaire
‡ Use of words like ³questionnaire´ and ³checklist´
should be avoided as some people might be
prejudiced against these words.
‡ as short as possible.
‡ for questionnaire items to be meaningful to
respondents enough information should be
included.

21. E. Reexamination and revision of the questions:
After questionnaire formulation revise it which involves:
‡ supplementation of one¶s effort by the critical opinion of
experts (should represent different approaches and
belong to different social backgrounds)
‡ reviewing by representative of different groups such as
minorities, racial groups, women etc.
‡ questionnaire should be scrutinized for any technical
defects (Selltiz et al., 1976).

22. F. Pretesting questionnaires:
‡ According to Selltiz et al ., (1976) pretest
helps in identifying and solving the
unforeseen problems in administration of
questionnaire such as phrasing, sequence
of questions or its length, identifying the
need for any additional questions or
elimination of undesired ones.

23. ‡ sample from the population used to pretest
questionnaire should be similar in characteristics
to those who will be included in actual study.
‡ Data collection technique should be same as
planned for actual study.
‡ After making necessary changes second pretest
should be conducted. Sometimes, in fact a
series of three or four or even more revisions
and pre-testing is required (Selltiz et al., 1976)

24. G. Editing the questionnaire and
specifying procedure for use:
‡ A final editing by the research staff is
done to ensure that every element passes
inspection: the content, form and question
sequence, spacing arrangement and
appearance(Selltiz et al., 1976).

25. Tests as Research Instrument
‡ A means of measuring the knowledge,
skill, feeling, intelligence or aptitude of an
individual or group.
‡ Produce numerical scores that can be
used to identify, classify or evaluate test
takers(Gay,1996).

26. Types of tests
‡ Norm-reference and domain-reference test:
Norm-reference test produces a score that tells how
individual¶s performance compares with other
individuals. Tables of norms based on scores obtained
by relevant group of subjects tested by the test
developer are provided by manual. Interpretations based
on relative performance are very useful for most of the
characteristics studied in behavioral sciences such as
anxiety, creativity, dogmatism or racial attitudes. It
describes performance, such as achievement, in relative
terms.
Example: How does the overall achievement of student
in class A compare to the students in class B? (Borg &
Gall, 1983)

27. ‡ Domain-reference test measures the learner¶s
absolute level of performance in a precisely
defined content area or domain. It is being used
increasingly to measure achievement ±related
performance. Domain reference tests estimate
individual¶s domain status i.e. precisely what is
his level of performance and specific
deficiencies in the domain covered by the test.
Example: what percentage of addition problems
john can solve correctly? (Borg & Gall, 1983)

28. Individual versus group tests:
‡ A group test is designed so that a sample of subjects
can take the test all at one time whereas individual test
measures one individual at a time.
‡ Group tests are more of objective type. Individually
administered measures are used when researcher is
interested in studying the process rather than the
product and should be used only if they make an
important contribution to the research.
‡ Example: Tests developed in clinical settings such as
Rorschach Inkbolt technique and Thematic Apperception
Test often are individually administered so that clinicians
can measure not only a subject¶s response but also learn
why subject gave a particular response.

29. Development of tests:
‡ Reviewing literature
‡ Define objectives
‡ Define target population
‡ Review related measures
‡ Develop an item pool
‡ Prepare a prototype
‡ Evaluate the prototype
‡ Revise measure

30. Reviewing literature:
‡ Before developing a new test review the available
literature in order to seek a test already available that
can be used for the study as test development is an
extremely complex process and require training.
Define objectives:
‡ Give careful thought to the specific outcomes that
measure is to achieve. E.g. construction of achievement
tests requires careful description of the knowledge or
skill that the test should measure.

31. Define target population:
‡ definition of target population is required as characteristics of target
population must be considered in many of the decisions on such
questions as item type, reading level, test length and type of
directions.
Review related measures:
‡ A careful review of tests that measure similar characteristics provide
ideas on methods for establishing validity, application of different
type of items, expected level of validity and reliability and possible
formats.
Develop an item pool:
‡ Before starting to write test items researcher needs to make
decisions regarding type of items that should be used and amount of
emphasis that should be given to each aspect of characteristics or
content area to be measured.

32. Prepare a prototype:
‡ The first form of the test puts into effect the earlier decision made regarding
the format, item type etc. through tryouts.
Evaluate the prototype:
‡ Obtain a critical review by at least three or more experts in test construction.
This review identifies needed changes and the prototype is field-tested with
a sample from target population.
‡ After collecting data item analysis is conducted to identify good and bad
items. Analysis and interpretation depends upon nature of test. E.g. in
developing a norm-referenced achievement test item analysis is usually
concerned with difficulty level of each item and its ability to discriminate
between good and poor students.
Revise measure:
‡ On the basis of field test experience and results of item analysis prototype is
revised and again field tested. This cycle may be repeated several times in
order to develop an effective test.Collect data on test reliability and validity.

33. Standardized test:
‡ Constructed by experts
‡ Individual test items revised and analyzed
to meet standards of quality
‡ Directions for carrying out test available
‡ Objective
‡ Existence of validity and reliability test

34. Types of standardized test
Test Example
Intelligence test Wechsler Adult Intelligence
Scale(suitable for testing late
adolescents and adults)
Aptitude test Modern Language Aptitude
Test
Achievement test Wide Range Achievement
Test(in areas of reading,
spelling and arithmetic)
Diagnostic test Stanford DIAGNOSTIC
reading Test

35. Measures of Creativity Word Fluency(person writes word
each containing a specified letter)
Projective techniques Holtzman Inkbolt Technique and
Rorschach Test.
Measures of self concept Tennessee Self Concept
Scale(include areas as self-
criticism, physical self, personal
self and social self)
Attitude scale Thurston Type Scale(individual
expresses agreement or dis
agreement with a series of
statements about attitude object)
Measures of vocational interest Career Assessment Inventory

36. Self report measures of personality:
a. General inventories Minnesota Multiphasic Personality
Inventory (most suitable in late
adolescents and adults, measure
variables as response sets, scales of
ego strength, anxiety and repression-
sensitization in addition to original
scales e.g. depression, schizophrenia)
b. Specific inventories Rokeach Dogmatism Scale (designed
to measure the variable of closed-
mindedness, often used in educational
and psychological research as a
measurement of authoritarianism)
c. Checklist The Adjective Checklist (measure
adjectives such as imaginative,
stubborn, relaxed )
(Gay,1996; Borg & Gall, 1983)

37. Use of tests in research:
‡ Tests are used in co relational studies e.g. to determine
relationship between quantitative ability and
achievement in science among high school students, the
researcher may decide the quantitative ability will be
defined as scores School and College Ability Test Series
III and science achievement will be defined as scores on
science sections of sequential Test of Educational
Progress (STEPIII).
‡ Experimental studies also use different tests such as
Youth Program Quality Assessment, Kansas City Youth
Net Standards etc.

38. References
‡ Bailey, K. D. (2nd Ed.). (1982). Methods of social research. London: Collier
‡ Macmillan Publishers.
‡ Borg, W.R. & Gall, M .D. (4th Ed.). (1983). Educational research: an
‡ introduction. NewYork: Longman Inc.
‡ Gay, L. R. (5th Ed.).(1996). Educational research: competencies for
analysis and
‡ application. New Jersey: Merrill
‡ Selltiz, C., Wrightsman, L.S., & Cook, S.W. (3rd Ed.). (1976).Research
methods
http://www.authorstream.com/Presentation/aSGuest37105-315810-concepttionn
aire-research-education-ppt-powerpoint/