Test Construction and Instrument Validation

1. Construction and Validation of
Research Instrument
Flerida S. Pajarillaga
EPS II- HRD

2. By the end of this seminar-workshop, the
participants should be able to:
 Understand the usual steps in crafting a
data-gathering tool
 Be able to apply the principles in question
construction
 Conduct basic validation of created
instrument for action research

3. Instrument Design
 Designing an instrument
is challenging and a
complex process.
 Consider whether your
variables are
measurable
 Consider modifying an
existing instrument

4. DESIGN
1
2
3

5. Write Different Types of
Questions
1
 Personal, Attitudinal
and Behavioral
Questions
 Sensitive Questions
 Open-and Close-
Ended Questions

6. Personal, Attitudinal and
Behavioral Questions
 Three popular Types

7. Other possible
descriptors include:
• Exemplary, Proficient,
Marginal, Unacceptable
• Advanced, High,
Intermediate, Novice
• Beginning, Developing,
Accomplished, Exemplary
• Outstanding, Good,
Satisfactory,
Unsatisfactory

8. Personal, Attitudinal and
Behavioral Questions
 Three popular Types
Behavior Did you take a semester
off during any of your
four years in college?
______ Yes
______ No

9. Sensitive Questions
This is usually written when the respondent
has “warmed up” answering neutral
questions Place leading questions like:
 Instead of :
Have you taken drugs?
 Ask:
In past surveys, many men reported at
some point in their lives they have taken
marijuana. This could happen during
adolescence or as an adult. Have you
ever smoked Marijuana?

10. Open- and Closed-Ended
Questions
 CLOSE ENDED QUESTIONS
The researcher poses a question and
provides preset responses or options
EXAMPLE:
There are many reasons why adults wish to get more
education. What is your most important reason for
coming to adult basic education classes (Check one.)
___________ To be able to help my children with their
schoolwork
___________ To get a better job
___________ To improve myself
___________ To get a high school equivalent diploma

11. Open- and Closed-Ended
Questions
 OPEN ENDED QUESTIONS
questions for which researchers do not
provide options, participants provide
responses
EXAMPLE:
Why are you attending
adult education classes?
_________________________________
___
_________________________________
___
_________________________________

12. Open- and Closed-Ended
Questions
 SEMI-CLOSED-ENDED QUESTIONS
has the advantage of open-and close-ended
questions
EXAMPLE:
There are many reasons why adults wish to get more
education. What is your most important reason for
coming to adult basic education classes (Check one.)
___________ To be able to help my children with their
schoolwork
___________ To get a better job
___________ To improve myself
___________ To get a high school equivalent diploma
___________ Other (please comment)

13. Use Strategies for Good
Question construction
2
 Pay attention to the quality of your
questions
 It should be clear/ unambiguous that it
does not confuse participants
 Consider cultural background and
avoid gender-biased words
 Eliminate the problems

14. Problem no. 1:
Do you support gun
control?
_____ YES
_____ NO
_____ Don’t Know
Do you believe that
guns do not belong
in school?
_____ YES
_____ NO
_____ Don’t Know
UNCLEAR QUESTIONS

15. Problem no. 2:
Do you believe that
guns and knives do
not belong in school
?
_____ YES
_____ NO
_____ Don’t Know
Do you believe that
knives do not belong
in school?
_____ YES
_____ NO
_____ Don’t Know
MULTIPLE QUESTIONS

16. Problem no. 3:
Whenever violence
occurs in schools,
weapons are typically
found in school
lockers. Do you
believe that students
should have gun in
the lockers?
_____ YES
_____ NO
Should students have
guns in their lockers?
_____ YES
_____ NO
_____ Don’t Know
WORDY QUESTIONS

17. Problem no. 4:
Students should not
carry weapons and
not have them in
their lockers. Do you
Agree?
_____ Strongly Agree
_____ Agree
_____ Undecided
_____ Disagree
_____ Strongly Disagree
Should students have
guns in their
lockers?
_____ YES
_____ NO
_____ Don’t Know
NEGATIVELY WORDED
QUESTIONS

18. Problem no. 5:
Should students pack a
.45 at school?
_____ YES
_____ NO
_____ Don’t Know
Should students have
handgun in their
lockers?
_____ YES
_____ NO
_____ Don’t Know
QUESTIONS WITH JARGON

19. Problem no. 6:
How many times have
you seen a student
carry a handgun?
_____ 0 times
_____ 1-2 times
_____ 2-3 times
_____ More than 3
times
How many times have
you seen a student
carry a handgun?
_____ 0 times
_____ 1-2 times
_____ 3-4 times
_____ More than 4
times
OVERLAPPING RESPONSES

20. Problem no. 7:
To what extent do you
feel that handguns are
a problem at your
school
_______ A great extent
_______ Some
_______ Not very
important
_______ Not a problem at all
To what extent do you
feel that handguns
are a problem at your
school
_______ A great
extent
_______ Some extent
_______ Little extent
UNBALANCED RESPONSE OPTION

21. Problem no. 8:
How often have you
seen students carry a
semi-automatic
weapon?
_____ None
_____ 1 time
_____ 2 times
_____ 3 or more times
How many times have
you seen a student
carry a riffle at
school?
_____ None
_____ 1 time
_____ 2 times
_____ 3 or more times
MISMATCH OF THE QUESTION AND
THE ANSWERS

22. Problem no. 8:
How many students
have you seen
carrying guns at
school?
_____ 1 student
_____ 2 students
_____ 3 students
______ More than 3
students
MISMATCH OF THE QUESTION AND
THE ANSWERS
Have you seen students
carrying guns at
school?
_____ Yes _____ No
If Yes, how many
students?
_____ 1 student
_____ 2 students
_____ 3 students
______ More than 3 students

23. Problem no. 9 & 10:
OVERLY TECHNICAL LANGUAGE
QUESTIONS NOT APPLICABLE TO
ALL PARTICIPANTS
Example:
What would be the appropriate care for children with Progeria?
What database would be most user-friendly for school-base
paperless database system?
____FoxPro _____ Lotus123 _____Access

24. Pilot Test Questions
3
 Pilot test is a procedure in which the
researcher makes changes in an instrument
based on feedback from small number of
participants who evaluated the questions.
 The participants makes comments while
taking the test and the researcher makes
changes
 Exclude this participants on the final sample of
the study

25. ARE YOUR INSTRUMENTS
RELIABLE AND VALID?

26. Individual scores should be nearly the same
or stable over repeated administration of
the instrument and should be free from
measurement errors
RELIABILITY
Types Reliability : Test re-test, Alternate Forms,
Interrater and the split half

28. The intended test interpretation of the concept
or construct that the test assumes to measure
matches the proposed purpose.
VALIDITY
Types of Validity: Criterion-related (predictive
and concurrent) validity, Content validity and
construct validity

30. STEPS IN CONSTRUCTING A
STANDARDIZED INSTRUMENT (Benson &
Clark, 1983)
PHASE
1
Planning
PHASE
2
Constructi
on
 Identify purpose of test and target
groups
 Define Domains of the Test
 Review literature on construct/
variable
 Select item format
 Develop Table of Specifications
 Validate content – have judges
complete qualitative evaluation
 Revise Items

31. PHASE
3
Quantitativ
e
Evaluation
PHASE
4
Validatio
n
 Pilot Test
 Calculate reliability and item
analysis
 Revise Items
 Administer second pilot test
 Run item analysis
 Repeat steps of revision
 Validate (until satisfied!)
STEPS IN CONSTRUCTING AN
INSTRUMENT (Benson & Clark, 1983)

32. LET’S CREATE OUR TO

33. WORKSHOP
1. Participants will be divided into groups
according to research theme assigned
2. Select a type of tool and construct the
instrument (at least 5-10 items) with the
following details:
(a) Research Title/ Question
(b) Participants of the Study
(c) Question Items

34. WORKSHOP
4. Each group will have a partner
who will critique (face /content
validate ) their work
5. Revise the items

35. School Principals’ Influence
on Trust
.
RESEARCH STUDY No. 1
Reference:
Sheldon, D.L, Angell, M.E., Stoner, J.B. & Roseland, B.D., (2010) .
School Principals’ Influence on Trust: Perspective of Mothers of
Children with Disabilities. Journal of Educational Research 103, 159-
170

36. Trust Worthy Leadership Matrix
(Mishra, 1996; Tschannen-Moran & Hoy, 1998, 2000)
 Trustworthiness is the willingness to be
vulnerable to another on the basis that the
other is benevolent, honest, open, reliable and
competent

37. Trust Worthy Leadership Matrix
(Mishra, 1996; Tschannen-Moran & Hoy, 1998, 2000)
 5 Functions of a Leader
Visioning creating a trustworthy environment
Modeling Continuity between word and example.
Lead quietly, exercising restraint and
modesty
Coaching assist people to move forward through
conversation - know when to push and to
back off.
Managing delegate control, high-trust= fewer rules,
Trust with parents = less time explaining
actions.
Mediating Dealing with betrayal and conflict skillful
in conflict management

38. Sample: Table of Specification
Visioning Modelling Coachin
g
Managing Mediatin
g
I
encourage
my team to
participate
when it
comes
decision
making
time and I
try to
implement
their ideas
and
suggestion
I will be the
first one to
follow the
rules set.
I enjoy
coaching
people
on new
tasks and
policies
Nothing is
more
important
than
accomplishin
g a goal or
task.
When
correcting
mistakes,
I do not
worry
about
jeopardizi
ng
relationshi
ps.