This document discusses instrumentation in research, including defining instruments and the instrumentation process. It outlines different types of instruments such as questionnaires, interviews, and tests. The document also discusses open-ended and closed-ended questions, and provides examples of different question structures like checklists, scales, and rankings. It emphasizes the importance of validity and reliability in instrument development.

1. INSTRUMENTATION
ADIBAH ABDUL LATIF, PhD
Faculty of Education

2. INSTRUMENT & INSTRUMENTATION
• Instrument is the generic term that
researchers use for a measurement device
(checklist, test, questionnaire).
• Instrumentation is the course of action (the
process of developing, testing, and using the
device).

3. INSTRUMENT
• Instruments fall into two broad categories,
researcher-completed and subject-
completed, distinguished by those
instruments that researchers administer
versus those that are completed by
participants.

4. INSTRUMENT
Researcher-completed Instruments Subject-completed Instruments
Rating scales Questionnaires
Interview schedules/guides Self-checklists
Tally sheets Attitude scales
Flowcharts Personality inventories
Performance checklists Achievement/aptitude tests
Time-and-motion logs Projective devices
Observation forms Sociometric devices

5. INSTRUMENT
Basic
Questions
Open ended
Closed-ended

6. INSTRUMENT
• Open ended questions :- when there are a great
number of possible answers or researchers
cannot predict all the possible answers
• For example – a question about students reason
for selecting particular university would probably
be open ended or a question about a college
major would be open ended because researchers
don’t want to the long possible majors.

7. INSTRUMENT
• Things to remember when writing open-ended
questions:
– Write questions clearly and concisely.
– Write questions by thinking about the reading
level of the target population.
– Avoid double negatives.
– Avoid double-barreled questions.
Example: Are you satisfied with the place and
time of the program?

8. INSTRUMENT
 Open-ended questions are useful to explore a
topic in depth however,
open-ended questions are difficult to
(i) Respond and (ii) Analyze.
 Therefore, limit the number of open-ended
questions to the needed minimum.

9. INSTRUMENT
• Close ended questions:- all the possible,
relevant responses to a question can be
specified and the number of possible
responses is limited
• Example- gender, ethnic, educational level

10. INSTRUMENT
Close ended questions
 Make sure that your answer categories are mutually
exclusive.
 Example: What is your age group?
a) Less than 20 years
b) 20-30 years
c) 31-40 years
d) 41-50 years
e) Above 50 years

11. INSTRUMENT
Close ended questions
1. Are you a male or female? (fixed alternative)
2. Are you a smoker? (‘yes’ or ‘no’ answer)
3. Please indicate your annual income level for the previous
year:
Ʒ RM 0.00 to RM10,000.00
Ʒ RM10,000.01 to RM 35,000.00
Ʒ RM 35,000.01 and above
(requires the researcher to include all possible responses and to make sure
that the responses are mutually exclusive, i.e. only one answer is possible
and there is no overlapping categories of income)

12. INSTRUMENT
Advantages of Close-ended Questions:
• Limiting the answer options makes data analysis easier
• Ensure desired information is obtained
• Can increase the reliability of the study
Disadvantages
• Other equally important information may not be retrieved
• Some respondents may become frustrated with limited ‘yes’ and ‘no’
responses
• Difficult to know whether the answers were guessed or actually
known
• If a question not answered – difficult to know whether the
respondent purposely did not answer or it was inadvertently missed

13. INSTRUMENT
• Both formats can be used in the same questions.
1. What type of writing assignment do you typically
require in your course?
a. Reports
b. Themes or essays
c. Research papers
d. Take home test
e. Other (Please specify) ____________________

14. DEVELOPING INSTRUMENT
• Use research questions, theoretical framework
and literature as a guide in constructing
instrument
• Researchers are communicating with
respondents using questions from the
instrument. So, respondents must really
understand what is the “heart” of the
instrument. This called communication
validity

15. DEVELOPING INSTRUMENT
• Review literature in the domain which you
wish to measure (i.e., “generic skills").
• Conceptual Definition - CD is an element of
the scientific research process, in which a
specific concept is defined as a measurable
occurrence. It basically gives you the meaning
of the concept.

16. DEVELOPING INSTRUMENT
• Construct formation - Develop a list of
subscales also known as constructs that you
wish to sample from the domain.
• e.g. : Conceptual definition = generic skills
• Construct Formation = communication skills,
problem solving, ethics, adaptability and team
working

17. DEVELOPING INSTRUMENT
• Operational Definitions – An OD is a process
by which the characteristics of a concept can
be defined, including identification and
classification. Also known as item
• Example for OD (i) I prefer to work in group
compared by individually.
• It is advisable to write at least 5
items/statements for each construct for the
first phase.

18. DEVELOPING INSTRUMENT
• Draw blueprint / meta data analysis and give
sources for each item to guarantee the
content validity
• Give the instrument to at least 3 experts to
validate (content and language experts)
• Calculate the level of agreement among the
experts

19. DEVELOPING INSTRUMENT
• Do pilot test on the instrument after
validation
• You can run many statistical analysis to check
the validity and reliability of the item
• E.g. : Exploratory factor analysis, Confirmatory
factor analysis, Principal Component Analysis,
Unidimensionality etc.

20. DEVELOPING INSTRUMENT
• Factor analysis can help to check construct
validity.
• Review whether each item conceptually
belongs with its factor (subscale) and remove
those which do not.
• Check the item fit or factor loading for each
item either its belong to the given construct
or not.

21. DEVELOPING INSTRUMENT
• Run reliability test such as item and person
reliability or Croanbach's Alpha for each
factor/category (subscale) to investigate
internal consistency reliability.
• Modify and retest the instrument if necessary
(alpha<.70).
• Check the scaling that you used either it is
understandable for the respondents or not

22. STRUCTURE OF QUESTIONS
1. Completion, or fill in - open ended questions
to which respondents must supply their own
answers in their own words
Example – What is the major weakness you have
observed in your students preparation for
college?

23. STRUCTURE OF QUESTIONS
2. Checklists - present a number of possible answers and
the respondents are asked to check those are apply
Example ; What type of teaching aids do you use in your
class? Check as many as apply
1) Chalkboard
2) Overhead projector
3) Computer projector
4) Video tapes
5) Other (please specify)_________________________

24. STRUCTURE OF QUESTIONS
3. Scaled Items- ask respondents to rate a concept, event, or
situation on such dimensions as quantity or intensity,
indicating how much, how well or how often?
Example : How would you rate the writing skills of students
you are teaching this semester? (check one)
1. Very poor
2. Less than adequate
3. Adequate
4. More than adequate
5. Excellent
6. Insufficient information

25. STRUCTURE OF QUESTIONS
4. Ranking Items – ask respondents to indicate
the order of their preference among a number
of options. Ranking should not involve more
than six options.
Example: Do your students have more difficulty
with some types of reading than with other
types? Please rank the order of difficulty of the
following materials with 1 the most difficult and
4 the least difficult

26. STRUCTURE OF QUESTIONS
_____1) Textbooks
_____2) Other reference books
_____3) Journal articles
_____4) other (Please specify)______________

27. STRUCTURE OF QUESTIONS
5. Likert Scales– lets subjects indicate their responses to
selected statements on a continuum from strongly
disagree to strongly agree.
Example : The students who typically enroll in my course
are underprepared in basic math skills. (Circle one)
1. Strongly Disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly Agree

28. STRUCTURE OF QUESTIONS
Example for items in Likert Type Scale format
(Agreement with 4 point scale)
1. Strongly Disagree
2. Disagree
3. Agree
4. Strongly Agree

29. 29
6. Semantic Differential measures attitudes toward a concept by
asking the respondent to rate qualities such as evaluation,
potency, and activity on a 5 to 9 point scale anchored by the
bipolar adjectives (Osgood et al 1957)
E.g. Your attitude towards visiting the dentist is:
Active…………………………Passive (activity)
Powerful………………………… Powerless (potency)
Favourable…………………………Unfavourable (evaluation)
1 2 3 4 5
Semantic Differential Scale

30. 30
7. Visual Analog Scale
• Useful for assessing perception of physical stimuli such
as pain, sleep quality, shortness of breath etc.
• Consists of a linear scale of 100mm in length, anchored
by two words or phrases.
e.g. Please mark an X on the line to indicate the amount of pain
you are experiencing now:
Pain as bad as it can be No pain
_____________________________________________________

31. VALIDITY
Types of Validity
• Content validity – the extent to which it adequately
covers the various dimensions of the concept under
studied (i.e. physical, psychological and theoretical)
• Criterion validity – assess a measure against another
criterion (or indicator) of the same concept
– Concurrent validity uses an already existing and tested tool
– Predictive validity assesses the degree to which a measure can predict
some future event of interest
• Construct validity – tests the link between a measure
and the theory underlying it

32. RELIABILITY
Types of Reliability
• Test-retest reliability – demonstrates the stability of a
measure over time
• Internal consistency reliability – most of the items within
a rating scale of a concept show a consistency of scoring
• Interrater reliability – the extent to which two or more
independent researchers are consistent in observing,
recording and scoring data (should be 70% or higher
agreement)
• Intrarater reliability – relies on one researcher to rate an
object or event twice (70% or higher of agreement)

33. 33
ADMINISTRATION OF QUESTIONNAIRE
• Postal surveys
• Face to face interviews
• Group administration
• Telephone surveys
• Direct observation
• Computer / Internet

34. WRITING SURVEY QUESTIONS
1. Question should be short, simple and direct.
A useful rule of thumb is that most of the
items should have fewer that 10 words (one
line) and should be under twenty words
2. Phrase items so that they can be understood
by every respondents. Avoid too technical
words, not to use slang, abbreviations or
acronyms may not be familiar to all.

35. WRITING SURVEY QUESTIONS
3. Phrase items so as to elicit unambiguous
answers.
Example :
1. Did you vote for last election?
2. I often go to library

36. WRITING SURVEY QUESTIONS
4. Avoid bias that may predetermined a
respondents answers
Example
1. Have you exercised your Malaysian right and
registered to vote?

37. WRITING SURVEY QUESTIONS
5.Avoid double barreled questions
Example
1. Do you feel that university should provide
basic skills courses for students and give credit
for them?