1. Validating Practice ComprehensiveTests
For Student And Institutional Bar Success
ProfessorYaira S. Ortiz-Medina
Pontifical Catholic University of Puerto Rico
School of Law
Professor Laurie Zimet
UC Hastings College of the Law

2. Goals
By the end of this session participants will:
 identify the key elements of a test validation process;
 be able to design a test validation process for their schools;
 be able to interpret the results and make decisions based in the
process’s outcomes.

4. Reliability
Is the degree to which an assessment tool
produces stable and consistent results.
Phelan andWren (2005-06)

5. Validity
Refers to how well a test measures what
it is purported to measure.
Phelan andWren (2005-06)

6. Reliability Coefficient KR20
 Measures test reliability and is an overall measure of internal consistency.
A higher value indicates a stronger relationship between items on the test.
Kelley (1939)

7. Response Frequencies
The percentage of
times that the
alternative was
selected.
Kelley (1939)

8. Non Distractor
 Item options that are not chosen by
any student.Therefore, they do not
provide any information to distinguish
different levels of student
performance.When an item has too
many non-distractors, it needs to be
revisited and possibly revised.
Kelley (1939)

9. Point-Biserial
 Indicates if a question was a good discriminator between better students
and poorer students. Point Biserial ranges from -1 to 1. A positive value
indicates that the students who did well on the test answered the question
correctly.
Kelley (1939)

10. Checklist Of Findings
 Example
O = Observed N = Not observed NA = Not applicable
O N NA Criteria Comments
X
Results allow me to
identify the mean
score.
Mean score is
15.73.

11. Checklist Of Findings (Part A)
O = Observed N = Not observed NA = Not applicable
O N NA Criteria Comments
X
Results allow me to identify the maximum
scores.
Maximum score is 21.00.
X
Results allow me to identify the minimum
scores.
Minimum score is 9.00.
X The Reliability Coefficient (KR20) is above .70. KR20 are .38.
X
Detailed item analysis provided allows me to
identify weak questions.
Weak questions are 8, and 9.
Questions 2, 3, and 5 should be
revised.
X
The amount of items is adequate for the test’s
purpose.
The amount of items per
subject is not representative.

12. Checklist Of Findings (Part B)
O = Observed N = Not observed NA = Not applicable
O N NA Criteria Comments
X
Subjects Learning Objective Report
allows me to identify subjects that need
more attention.
The weaker subject is General
Theory of Obligations.
Civil Procedural Law, Evidence,
Ethics, and Torts might need
reinforce.
X
Procedures before, during and after the
test are specified.
Are specified, but might need
more structure.
X
Results report allows ASP staff to design
an action plan.
ASP could develop an action
plan that will impact peer-
tutoring program and
workshops.