This document provides guidance on selecting a test delivery vendor. It emphasizes the importance of choosing a platform with tight integration between the item banking tool, candidate registration system, and delivery engine. It also recommends prioritizing security and controlling item exposure to protect the integrity of high-stakes certification tests.

1. Choosing
a
Test
Delivery
Vendor
–
Part
1:
The
Basics
Selection
of
your
testing
partner
will
either
free
you
to
focus
on
content
and
enforcement,
or
drag
you
in
to
bondage
that
sucks
the
life
from
your
certification
program.
No
(false)
pressure
here.
1
Program
Design
Assuming
the
intent
of
your
program
is
to
measure
learned
ability
through
the
use
of
written
assessments
and
simulations,
you
must
start
with
addressing
the
fundamental
issue
of
test
security.
Will
the
results
of
the
program
allow
individuals
to
practice
in
an
area
that
could
put
the
public
at
risk
if
done
by
an
unqualified
individual?
If
the
answer
is
yes,
yours
is
a
high
stakes
exam.
Test
Design
High
stakes
tests
must
be
valid
and
reliable.
The
test
design
will
be
based
on
a
responsible
practice
analysis
and
test
content
that
is
carefully
crafted
and
reviewed
by
experts
in
the
field
of
practice.
Unless
you
have
unlimited
resources
to
develop
and
maintain
a
huge
item
bank,
you
will
need
to
limit
item
exposure.
For
the
best
control
of
item
exposure
you
probably
want
to
use
CAT
to
reduce
the
number
of
items
required
for
a
valid
test.
CAT
is
sexy,
but
be
realistic
with
your
goals.
If
you
deliver
fewer
than
5,000
tests
a
year,
CAT
is
not
for
you.
If
your
program
isn’t
ready
for
CAT
yet,
consider
LOFT.
2
LOFT
is
Linear
On
the
Fly
Testing.
It
offers
a
measured
approach
that
selects
a
pre-­‐
defined
number
of
items
from
each
domain,
but
not
necessarily
the
same
ones
for
each
candidate.
Linear
fixed
form
remains
a
popular,
if
not
innovative,
standard
for
high
stakes
tests.
Forms
may
be
pre
or
post
administration
equated
by
your
psychometric
team.
Not
all
platforms
support
all
models.
If
you
don’t
have
a
psychometricians
on
staff
(or
very
handy)
don’t
try
to
learn
IRT
along
the
way.
There
are
programs
that
will
run
the
statistics
for
you
so
you
can
at
least
know
how
your
items
and
your
forms
are
performing,
but
if
you
are
moving
to
equated
forms,
or
CAT,
bring
in
the
big
guns.
Test
Assembly
Don’t
underestimate
the
effort
required
to
move
items
between
systems.
QTI
comes
in
many
flavors
and
if
anyone
says
you
can
just
put
your
items
in
a
spreadsheet
and
port
them
over,
RUN
and
run
FAST.
2016
Test
Delivery
LOREM
Test
Assembly
Fixed
form,
randomized
order,
LOFT,
or
CAT
Candidate
Registration
Requirements
screening,
individual
and/or
batch,
ecommerce
Test
Delivery
Public
or
private
sites,
online
and/or
paper
Reporting
Individual,
aggregate
TRACE
CONSULTING
3115
Anita
Circle
NE
Iowa
City,
Iowa
52240
CONTACT
319-­‐855-­‐8690
terry@tracetests.com
WEB
www.tracetests.com

2.
Select
a
test
delivery
platform
with
tight
linkage
between
the
banking
tool,
the
candidate
registration
system,
and
the
delivery
engine.
1
Candidate
Registration
Know
who
will
be
looking
at
your
tests,
perhaps
even
limiting
the
lookie-­‐loo
factor
by
confirming
each
test
taker
is
otherwise
qualified
for
the
certification
BEFORE
they
see
the
test
content.
Several
vendors
offer
tools
to
manage
certification
requirements
as
part
of
the
registration
process.
When
your
items
are
ready
and
your
forms
are
assembled,
don’t
waste
time
trying
to
register
test
takers
and
manage
rosters.
The
automation
to
do
this
is
inexpensive
and
powerful.
E
commerce
is
a
little
trickier
if
you
don’t
already
have
a
system
to
collect
payment,
but
most
test
vendors
today
are
PCI
compliant,
and
support
collection
of
their
fees
and
yours
through
their
registration
system.
Test
Delivery
The
only
really
safe
way
to
protect
your
intellectual
property
is
to
keep
it
from
ever
being
seen.
That
doesn’t
work
if
you
need
to
let
people
take
the
tests.
Be
responsible
about
how
you
expose
the
tests.
If
you
send
them
out
in
paper
form,
expect
that
someone
will
make
a
copy
before
sending
them
back.
It
happens
in
the
best
of
settings
with
the
highest
level
of
control.
Out
of
sight
paper
forms
should
be
considered
out
of
circulation
(in
a
short
time
at
least)
and
are
probably
the
best
excuse
you
can
find
for
a
short
and
infrequent
window
of
administration.
Decide
how
much
inconvenience
you
are
willing
to
tolerate
as
you
design
your
security
features.
As
you
publish
an
online
test,
don’t
get
crazy
protecting
against
the
end
case
unless
you
are
SURE
you
have
covered
the
basics.
1.
Tell
your
test
takers
what
cheating
looks
like
and
make
them
agree
that
they
won’t
cheat
before
you
let
them
start
each
test.
Let
everyone
know
that
if
a
cell
phone
with
a
camera
(or
any
other
camera
or
audio
recording
device)
seen
in
the
testing
room
it
is
a
breach
of
security
procedures
that
will
2
invalidate
the
scores
of
the
test
and
may
lead
to
legal
prosecution.
2.
Of
course
you
want
your
proctors
to
verify
test
taker
identification.
You
probably
want
to
be
sure
the
test
is
only
released
with
an
authorized
proctor
in
the
room.
Most
vendors
offer
proctor
authentication
as
part
of
the
service,
and
some
actually
train
and
certify
those
proctors.
The
proctor
is
probably
the
weakest
link
in
your
security
chain
unless
you
work
with
a
vendor
that
either
hires
their
own,
or
signs
agreements
with
each
test
center
and
audits
against
the
requirements
in
that
agreement.
3.
Lock-­‐down
browsers
are
a
good
idea,
unless
they
cause
your
test
administration
to
fail
30%
of
the
time.
Installing
a
lock
down
browser
on
third
party
testing
devices
may
be
painful.
There
are
test
platform
vendors
that
prevent
cut
and
paste
function,
and
stop
the
test
if
a
candidate
leaves
the
screen,
WITHOUT
the
pain
of
an
extra
layer
of
lock
down
browser.
Look
for
those.
4.
Once
a
test
starts,
industry
standard
for
Internet
interruptions
in
a
testing
event
is
about
5%.
Make
sure
your
proctors
know
how
to
restart
the
test,
and
make
sure
your
vendor
saves
responses
after
each
submission
and
turns
the
clock
off
if
the
heartbeat
stops.
5.
Standards
of
fair
testing
require
that
test
takers
know
why
they
are
taking
the
test,
what
score
is
required
to
pass
the
test,
and
if
their
performance
met
the
minimum
requirements.
If
you
choose
to
report
domain
and
even
sub
domain
level
scores,
make
sure
they
are
not
over
interpreted.
And
be
ready
to
do
a
lot
of
explaining.
Your
vendor
is
working
for
you.
Listen
to
their
suggestions,
but
don’t
be
held
hostage
by
a
hostile
vendor.
Your
candidates
and
your
field
of
practice
are
depending
on
you
to
manage
the
process.
Test
Assembly
Candidate
Registration
Test
Delivery
Reporting
Whether
you
are
looking
for
the
right
vendor
or
trying
to
figure
out
how
to
make
your
vendor
right,
we
are
here
to
help.
With
years
of
experience
delivering
high
stakes
tests,
we
offer
a
no-­‐nonsense
approach
to
supporting
your
program.
TRACE
CONSULTING
3115
Anita
Circle
NE
Iowa
City,
Iowa
52240
CONTACT
319-­‐855-­‐8690
terry@tracetests.com
WEB
www.tracetests.com
Fixed
form,
randomized
order,
LOFT,
or
CAT
Requirements
screening,
individual
and/or
batch,
ecommerce
Public
or
private
sites,
online
and/or
paper
Individual,
aggregate