Psychometric Theory Course Syllabus

THE AMERICAN SCHOOL OF PROFESSIONAL PSYCHOLOGY
DALLAS
COURSE SYLLABUS
PP7230: Psychometric Theory
Wednesday 6:30 PM to 9:20 PM
Fall 2010
Room 218
Faculty Information
Instructor’s Name: Mark W. Matthews, PhD
Office Location: 203
Office Hours: Mondays: 1:00 PM to 3:00 PM
Tuesdays: 1:00 PM to 4:00 PM
Wednesdays: by appointment
Thursdays: 10:00 AM to Noon
Phone Number(s): Office: (214) 459-2221
Email Address: mmatthews@argosy.edu
Short Faculty Bio: Dr. Matthews is an Assistant Professor in Argosy’s PsyD program and a
Licensed Clinical Psychologist. He has an academic and practical history
in psychological assessment, psychometric theory, and the application of
psychometric theory to assessment issues. As an instructor, he has
taught personality assessment for over 15 years. As a researcher he has
investigated graduate student opinions about Sigmund Freud and
Psychoanalysis. In his private practice, he has spent over a decade
conducting psychological assessments. Furthermore, beyond his
undergraduate and graduate education in the subject, he taught
psychometric theory as a graduate TA. His knowledge, skills, and
abilities in this area impact how he teaches assessment, conducts
research, and how he utilizes psychological measures in his practice. For
example, in assessment courses, he spends a substantial amount of time
not just teaching how to administer, score, and interpret a measure, but
ensuring the students are knowledgeable about item development, text
construction, reliability, validity, potential for bias, and generalizability.
As a researcher, he has applied item response theory (IRT) to his
aforementioned work on student opinions. Moreover, in a previous role
as director of clinical training, he analyzed the program’s comprehensive
exam. This analysis included assessment of item response distributions
and item difficulty. Finally, in his practice, he has developed a set of
local clinical norms for the Personality Assessment Inventory (PAI) and the
Parenting Stress Index (PSI). Specifically, the norms are for parents
referred for psychological evaluations by Child Protective Services.
When the university decided to offer a course in psychometric theory,
Dr. Matthews volunteered to create the syllabus and presently teaches
the course.

PP7230 Fall 2010 2
Syllabus is subject to change by instructor
Course Catalogue Description:
This course is designed to provide students an introduction to the key concepts of
psychometric theory (e.g., assessment theory, reliability, validity, item development, norm
referencing, ethical and professional issues in assessment) as they apply to the practice of
clinical psychology. To this end, the course is designed to provide students the skills necessary
to evaluate, choose, implement, and interpret psychological tests and measures commonly
employed in the practice of clinical psychology. Furthermore, students will be introduced to
the foundational knowledge of test development theory. To this end, the course will examine
classical test theory views of reliability and item analysis, validity, and test development
principles. Finally, modern approaches to test theory, including item response theory and
generalizability theory are covered.
Course Prerequisites:
None
Technology:
Access to MS Office (i.e., Word, Access, Excel, and PowerPoint) for word processing, slide
shows, and database programs is required. MS Office 2007 required
Access to Microsoft Internet Explorer 6.0 or higher to access class website
Adobe Acrobat Reader 7.0 or higher to download and print protocols and other handouts
XMind to use mind maps
You can obtain a free open-source copy at http://www.xmind.org/. You do NOT need to
purchase the Pro version to utilize the mind maps from class.
Course Length, Contact Hours, Credit Value:
Course: 15 weeks, 45 hours, 3 units
Required Textbooks:
Crocker, L., & Algina, J. (2006). Introduction to Classical and Modern Test Theory
Embretson, S. E., & Reise, S. P. (2000). Item Response Theory for Psychologists, 2nd Ed.
Required Journal Articles:
Campbell, D. P., & Fiske, D. W. (1959). Convergent and discriminant validity in the multitrait-
multimethod matrix. Psychological Bulletin, 56, 81-105.
Comrey, A. L. (1988). Factor-Analytic Methods of Scale Development in Personality and
Clinical Psychology. Journal of Consulting and Clinical Psychology, 56(5), 754-761.
Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological
Bulletin, 52, 281-302.
Embretson, S. E. (1996). The new rules of measurement. Psychological Assessment, 8, 341-349.
Schmitt, N., & Stults, D. M. (1986). Methodology review: Analysis of the multitrait-
multimethod matrices. Applied Psychological Measurement, 10, 1-22.
Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54,
93-105.

PP7230 Fall 2010 3
Shavelson, R. J., Webb, N. M., & Rowley, G. L. (1989). Generalizability theory. American
Psychologist, 44, 922-932.
Recommended Textbook:
Morey, L. C. (1991). Personality Assessment Inventory professional manual. Odessa, FL: Psychological
Assessment Resources.
Program Outcomes:
2. Assessment:
Students will deliver assessment and diagnostic services in a manner consistent with
professional and ethical standards by effectively assessing and conceptualizing the strengths and
problems of clients through a variety of appropriate assessment techniques and accurately
communicating findings in a professional manner.
Course Goals (Learning Outcomes):
1. Students will learn how to evaluate psychological assessment measures’ reliability, validity, and
other psychometric characteristics.
2. Students will develop a measure designed to assess a psychological construct.
Course Evaluation:
These skills will be evaluated by grading and assessing the students’:
1. Attendance, discussion, and participation
2. Quizzes or exams as necessary
3. Developing a measure designed to assess a psychological construct by evaluating construct
development, item selection, measuring reliability, and assessing validity.
All written assignments are expected to be submitted in APA-style unless otherwise specified by the instructor.
All work will be evaluated according to the syllabus and any other appropriate handout(s).
Grading Scale Equivalents:
Grade Percentage GPA
A 100-93 4.0
A- 92-90 3.7
B+ 89-87 3.3
B 86-83 3.0
B- 82-80 2.7
C+ 79-77 2.3
C 76-73 2.0
C- 72-70 1.7
F 69-below 0.0
Attendance Policy:
Argosy University Attendance Policy
Students are expected to be punctual to all classes and practicum. Absences should occur only
for such urgent reasons as ill health or critical emergency. Whenever possible, students should
notify the faculty of these absences in advance. Excessive late arrivals or absences, regardless of
the reason, may jeopardize a student’s academic standing.
Class Attendance Policy

PP7230 Fall 2010 4
Attendance is important for both the acquisition of knowledge and the demonstration of your
professional readiness to become a responsible clinician. Your willingness and ability to get to
class on time and consistently attend reflects on your investment in the class, your education,
and professional identity. You are allowed to miss two classes for personal emergencies. Don’t
ask for permission to miss class. If you miss a third class, even for a personal emergency, you
will not pass the class.
If you miss class, you are responsible to get notes and information from your peers. Please do
not ask, “Did I miss anything?” or “What did I miss?” If I lectured, you missed something.
You are responsible to learn what you missed from someone who attended. I am also able to
meet with you during regular office hours to answer any questions after you get notes from
your peers.

SCHEDULE OF ASSIGNMENTS
Week Date Assignment Due Readings Due Lecture/Discussion Homework/Assignment
1 9/8/10 None None Introduction to Psychometric Theory Reading
Choose psychological construct
2 9/15/10 Psychological
Construct
Crocker & Algina Chapter 2 Statistical Concepts for Test Theory Reading
3 9/22/10 None Crocker & Algina Chapters 3
& 4
Scaling
Test Construction Part 1
Reading
Steps 1 – 3 of Chapter 4
Start: “Test Conceptualization” of Test
Development Project
4 9/29/10 Steps 1 – 3 Cronbach & Meehl (1955)
Schwartz (1999)
Morey Chapter 5
Test Construction Part 2
Construct Validity
Self-Reports
Step 4 of Chapter 4
Finish “Test Conceptualization” of Test
Development Project
5 10/6/10 “Test
Conceptualization
Step 4
Crocker & Algina Chapters 5,
6, & 7
Morey Chapter 8
Reliability Start: “Test Construction” of Test Development
Project
6 10/13/10 Crocker & Algina Chapters 8
& 9
Shavelson et al. (1989)
Morey Chapter 6
Generalizability Finish: “Test Construction” of Test Development
Project
7 10/20/10 “Test
Construction”
Crocker & Algina Chapter 10
Campbell & Fiske (1959)
Morey Chapter 9
Validity Reading
Continue revision of Test Development Project
8 10/27/10 Crocker & Algina Chapter 11
Morey Chapter 9
Prediction and Classification Reading
9 11/3/10 Project Update Crocker & Algina Chapter 12
Morey Chapter 7
Bias in Selection Reading
Morey Chapter 7
Item Analysis Reading
Embretson, S. E. (1996)
Morey Chapter 5 (again)
Introduction to Item Response Theory
(IRT)
Reading
12 11/24/10 Project Update Embretson & Reise
Chapters 1 & 2
More New Rules Reading
13 12/1/10 Embretson & Reise
Chapters 6 & 12
IRT and Personality Assessment Reading
14 12/8/10 Crocker & Algina Chapters
17 & 18
Corrections and Setting Standards Reading
15 12/15/10 Completed Project Crocker & Algina Chapters
19 & 20
Norms and Standards None

Special Policies:
Extenuating Circumstances
Students must inform the instructor immediately of extenuating circumstances that may affect
course performance. Instructors are not obligated to, but may, solely at their discretion, make
adjustments in due dates for extenuating circumstances. Instructors may request supporting
documentation of extenuating circumstances.
American’s with Disabilities Act Policy
It is the policy of Argosy University to make reasonable accommodations for qualified students
with disabilities, in accordance with the Americans with Disabilities Act (ADA). If a student
with disabilities needs accommodations, the student must notify the Director of Student
Services. Procedures for documenting student disability and the development of reasonable
accommodations will be provided to the student upon request.
Students will be notified by the Director of Student Services when each request for
accommodation is approved or denied in writing via a designated form. To receive
accommodation in class, it is the student’s responsibility to present the form (at his or her
discretion) to the instructor. In an effort to protect student privacy, the Department of Student
Services will not discuss the accommodation needs of any student with instructors. Faculty may
not make accommodations for individuals who have not been approved in this manner.
Policy regarding Academic Dishonesty
In an effort to foster a spirit of honesty and integrity during the learning process, Argosy
University requires that the submission of all course assignments represent the original work
produced by that student. All sources must be documented through normal scholarly
references/citations and all work must be submitted using the Publication Manual of the American
Psychological Association, 5th
Edition (2001). Washington DC: American Psychological Association
(APA) format. Please refer to Appendix A in the Publication Manual of the American Psychological
Association, 5th
Edition for thesis and paper format. Students are encouraged to purchase this
manual (required in some courses) and become familiar with its content as well as consult the
Argosy University catalog for further information regarding academic dishonesty and
plagiarism. Resubmission of work in more than one class is prohibited. If you have any
questions about academic honesty, ask the instructor before submission of your work. A
finding of academic dishonesty or plagiarism may result in disciplinary action, up to and
including dismissal.
Scholarly Writing
The faculty at Argosy University is dedicated to providing a learning environment that supports
scholarly and ethical writing, free from academic dishonesty and plagiarism. This includes the
proper and appropriate referencing of all sources. You may be asked to submit your course
assignments through “Turnitin,” (www.turnitin.com), an online resource established to help
educators develop writing/research skills and detect potential cases of academic dishonesty.
Turnitin compares submitted papers to billions of pages of content and provides a comparison
report to your instructor. This comparison detects papers that share common information and
duplicative language including work previously submitted to Turnitin.
Professional Conduct
Argosy University requires that all students meet the standards of the profession for which they
are preparing. Students are required to do more than complete certain academic and field

PP7230 Fall 2010 7
training requirements. Students are expected to conduct themselves in a manner consistent
with professional ethics at all times. Professional conduct requires the faithful discharge of all
responsibilities undertaken during practicum and internships, as well as the maintenance of
respectful interpersonal relationships with all individuals.
Professional conduct is required in classroom and campus behavior as well as interpersonal
behaviors and relationships with faculty, staff, and students.
The Argosy University Statement Regarding Diversity
The Argosy University provides equitable access through its services and programs to students
of any social, geographic and cultural background, regardless of gender, and strives to prepare
all candidates to work with and provide services to diverse populations. Argosy demonstrates
its commitment to diversity through the development and support of a diverse educational
community.
Use of Argosy University Dallas Video and Audio Equipment:
There are two video cameras available for students to use on school premises for mock
sessions. You can check these out from the library for up to four hours. They use VHS-C
tapes which have a recording time of approximately 1.5 hours The library DOES NOT provide
the tapes so the students must bring one themselves.
After you create a video you can view it by checking out an adapter to use in one of the library’s
two study rooms. Argosy University/Dallas DOES NOT provide equipment to dub VHS-C
tapes onto a regular video tape.
There are also three audio cassette recorders available for student use. Again you can check
these out for up to four hours. They use standard audio tapes. Again the library DOES NOT
provide the tapes.
The students must check out both a video camera/audio tape recorder AND a clinical training
room. You can reserve each for up to a week in advance. You cannot use any equipment
unless you have a clinical training room reserved.

PP7230 Fall 2010 8
Test Development Project
I. Test Conceptualization
A. What is the purpose of your measure?
B. What construct will you measure?1
1. How does your construct relate to other measures?
2. How does your construct relate to other outcome variables?
3. Are there sub-constructs?
4. How does your construct meet an unmet need?
5. Include a “nomological network” (See Cronbach & Meehl [1955]).
6. What population is this measure designed to assess?
7. What is already known about this construct?
a. Previous research?
b. Diagnostic criteria?
8. Will this scale typically be employed in conjunction with other measures?
a. If yes, which measures?
b. If no, why?
9. What are the potential positive implications of using this scale?
10. What are the potential negative implications of using this scale?
II. Test Construction
A. Item Development Preparation
1. How were items developed?
a. What behaviors represent the construct? (See Crocker & Algina, pp. 67 – 28).
b. Discuss the following as related to developing items for your measure:
i. Content analysis
ii. Review of research
iii. Critical incidents
iv. Direct observations
v. Expert judgment
vi. Instruction objectives
2. What are the “item specifications?” (See Crocker & Algina, pp. 72 – 75)
B. Item Construction
1. What is the item format?
1 For all of the following discussion points, include sufficient literature to support your assertions.

PP7230 Fall 2010 9
a. Optimal Performance Test?
b. Inventory?
2. Is the proposed format feasible for the intended examinees?
3. How will you select and train item writers?
4. Write items and recruit classmates and other peers to help.
5. How would you monitor the progress of the item writers and evaluate the quality of the
items they write?
6. How did you choose to scale your measure? (See Cocker & Algina, Chapter 3)
a. Why?
b. Advantages?
c. Disadvantages?
7. How do you score your measure?
a. Why?
b. Advantages?
c. Disadvantages?
III. Item Review, Item Revision, and Item Tryout
A. Item Review
1. How will you assess item accuracy?
2. How will you assess item appropriateness or relevance to test specifications?
3. How will you assess technical item-construction flaws?
4. How will you assess item grammar?
5. How will you assess item offensiveness or appearance of “bias?”
6. How will you assess item level of readability?
7. How many initial items do you want to develop?
a. Why not fewer?
b. Why not more?
B. Item Revision
1. How will you go about deciding which items to retain?
2. How do you plan to address missing data?
3. Do you have sufficient domain coverage?
a. Why?
C. Preliminary Item Tryout
1. How do you propose to tryout items on a preliminary basis?

PP7230 Fall 2010 10
a. What methodologies will you use?
b. What population will you use?
c. How does this population apply to the population you want to eventually use the
measure to assess?
2. What techniques will you use after trying out the items? (See Cocker & Algina, pp. 82 –
83).
3. What expert reviews will you use?
4. What bias reviews will you use?
IV. Item Analysis
A. Test Revision
1. How will you go analyzing responses to items?
a. How will you decide which items to retain?
2. How will you assess reliability and validity?
a. Explain why the various forms of reliability chosen
b. Explain why the various forms of validity chosen
i. Be comprehensive – consider the various issues and concerns discussed in class
and in the readings
ii. Include an MTMM matrix if applicable
B. How many test iterations might you need to complete?
C. How do you plan to cross-validate your findings, if appropriate?
V. Final Measure
A. How many items do you plan to have for your final measure?
B. What additional studies need to be conducted?
C. How do you propose to interpret scores?
VI. Discussion
A. What are the theoretical and practical implications of your scale?
B. Limitations and Future Research
1. What plans might you implement to examine your measure further?
2. Thoroughly discuss the limitations associated with the particular approach you used.
VII. References
VIII. Figures/Tables as needed to support your discussion and work

Psychometric Theory Course Syllabus

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Psychometric Theory Course Syllabus

Similar to Psychometric Theory Course Syllabus (20)

Recently uploaded

Recently uploaded (20)

Psychometric Theory Course Syllabus