- Traditional tests have fixed forms with all examinees answering the same items, which is inefficient and leads to differences in precision. - Computer adaptive testing (CAT) tailors the difficulty and number of items to each examinee based on their responses to previous questions. CAT aims to maximize precision by selecting subsequent questions based on the examinee's estimated ability level. - CAT requires fewer items than traditional tests to arrive at equally accurate scores while providing a more personalized experience for each examinee.

1. Computer Adaptive
Testing

2. Traditional testing
• In traditional paradigm of test design, tests are assembled into one or more
fixed forms. We have all experienced such tests: here are 200 multiple choice
items, please answer them all. This approach ha s a number of problems.

3. Problems of traditional testing
• Inefficient: all 200 items are not necessary
• Large differences in precision among examinees: medium difficulty items

4. Computer Adaptive Testing
.
• CAT works by tailoring both difficulty and number of items to an examinee

5. • CAT successively selects questions for the purpose of maximizing the
precision of the exam based on what is known about the examinee from
previous questions. From the examinee's perspective, the difficulty of the
exam seems to tailor itself to their level of ability. For example, if an
examinee performs well on an item of intermediate difficulty, they will then
be presented with a more difficult question. Or, if they performed poorly,
they would be presented with a simpler question. Compared to static
multiple choice tests that nearly everyone has experienced, with a fixed set of
items administered to all examinees, computer-adaptive tests require fewer
test items to arrive at equally accurate scores.

6. What is a CAT?
COMPUTER ADAPTIVE TEST
?
?
?
?
?
?
?

7. What is a CAT?
How will
it work
for me?

8. How will it work for the student?
Question Selection
The student receives questions
based on his or her response to
the previous questions. The
computer program quickly selects
a new question after reviewing how
well the student performed on all
the previous questions. Based on
the responses, the program selects
a question that fits the test
blueprint and gives the best
information about what the student
knows. This is how a CAT can
customize each test so it is an
accurate measure of any student’s
skills.

9. How will it work for the student?
The test starts with
an item from a pre-
determined difficulty
level. It may be
based on average
difficulty for the
grade, or it may be
based on
information known
about the student
from previous tests.

10. How will it work for the student?
Responses
Because the program knows
which questions are harder and
which are easier, several
students may have answered a
similar number of questions
correctly, but the student who
has answered the more
challenging questions correctly
will achieve a higher score.

11. How will it work for the student?
That's it!
A student’s score is based on the difficulty of the items that
were right or wrong, not on the total number of correct
answers.
Of course, it all seems so simple when described like this. But
there is a very complex process happening within the CAT
engine to make all of these determinations simultaneously
with predictability built in.

12. CAT components
• Calibrated Item Pool
• A pool of items must be available for the CAT to choose from.
• Starting Point
• In CAT, items are selected based on the examinee's performance up to a given point in the test. However, the CAT is
obviously not able to make any specific estimate of examinee ability when no items have been administered. So some
other initial estimate of examinee ability is necessary. If some previous information regarding the examinee is known, it
can be used, but often the CAT just assumes that the examinee is of average ability - hence the first item often being of
medium difficulty.
• Item Selection Algorithm
• Therefore, if the CAT has an estimate of examinee ability, it is able to select an item that is most appropriate for that
estimate. Technically, this is done by selecting the item with the greatest information at that point.

13. •Scoring Procedure
• After an item is administered, the CAT updates its estimate of
the examinee's ability level. If the examinee answered the item
correctly, the CAT will likely estimate their ability to be
somewhat higher, and vice versa. This is done by using the
item response function to obtain a likelihood function of the
examinee's ability.
• Termination Criterion
• The CAT algorithm is designed to repeatedly administer items
and update the estimate of examinee ability. This will continue
until the item pool is exhausted unless a termination criterion is
incorporated into the CAT.

14. • Practical Constraints of Adaptivity
• CAT exams must usually meet content specifications; a verbal exam may
need to be composed of equal numbers of analogies, fill-in-the-blank
and synonym item types. CATs typically have some form of item
exposure constraints, to prevent the most informative items from being
over-exposed. Also, on some tests, an attempt is made to balance surface
characteristics of the items such as gender of the people in the items or
the ethnicities implied by their names. Thus CAT exams are frequently
constrained in which items it may choose and for some exams the
constraints may be substantial and require complex search strategies to
find suitable items.
For examinees with true scores very close to the passing score,
computerized classification tests will result in long tests while those with
true scores far above or below the passing score will have shortest exams.

15. Some advantages of CAT
• Frequent retesting: the high number of permutations also enables more
frequent receives a few weeks of instruction, by the time they take a CAT
again their ability has increased somewhat, ad they will receive a completely
different test.
• More precision: because CATs are more efficient, the organization has the
option to design the CAT to actually be more precise than a conventional
test while still using fewer items.
Shorter tests: CAT can reduce testing time by 50% or more. This can
obviously translate into huge financial benefits.

16. • Examinee experience: A CAT will provide an appropriate challenge for each
examinee. Low examinees are not discouraged or intimidated. High examinees enjoy
receiving difficult items.
• Increased motivation: because of the better experience, there is likely an increase in
examinee motivation. Low examinees feel better, and high examinees feel
challenged. Both will try harder than with a conventional test.
• Equiprecision: CATs can be designed so that examines are all measured with the
same level of precision, even though they all potentially see different items. This
makes t4est extremely fair from a psychometric perspective.
• Security: because the CAT algorithm is very flexible and can adapt with potentially
millions of permutations, there is much greater security than if veryone was
administered the same set of 200 items.

17. Disadvantages of CAT
Recovery of poor starts: CATs are susceptible to issues with examinee test anxiety, as the
elimination of item review prevents someone from going back to the first few items. If they
answered all those items incorrectly due to severe test anxiety, the test cannot correct itself.
Requirements: CAT require large sample size and extensive expertise.
Item exposure: CATs are designed to select the best items in the bank, and these items often
become overexposed if a control algorithm is not implemented.
No review: CATs rarely allow for examinees to return to items already administered as the CAT
has since adapted and it cannot updated.
Public relation: because of the complexity and the departure from the familiarity of the
traaitional exam paradigm, an organization must put forth more effort into public relations,
explaining CAT and the reasons for using it.

18. Thanks for your
attention
Niloufar Yazdni Majd