This report presents data derived from a comparative analysis that indicates gaps and security risks between multi-day test administrations with smaller numbers of test takers and single-format testing of large numbers of test takers. Additionally, the report highlights the use of standard best practices adopted around the world by leading test sponsor organisations.

1. 1www.prometric.com |
|
Copyright © 2016 Prometric, Inc. All Rights Reserved.
Third Quarter 2016 | Industry White Paper
Executive Summary
Technologically advanced societies rely on the
development of numerous specialised professions in
order to ensure the proper and safe operation of
critical societal functions, and the continuing discovery
of new knowledge and advancement of technology.
Fundamental to all professions is some technique or
mechanism for admitting new students and
practitioners and for making sure they are current with
the state of their art or science. Usually this takes the
form of a test that assesses an individual student’s or
practitioners’ current knowledge.
The practice of simply replicating a paper test on a
computer, however, should still adhere to best-practice
approaches and standards learned over decades of
experience by testing and assessment organisations.
This becomes a critical path to follow for emerging
computer-based testing programs, especially when the
subject of single-day testing has become such a topic
of great interest in India recently.
This report presents data derived from a comparative
analysis that indicates gaps and security risks
between multi-day test administrations with smaller
numbers of test takers and single-format testing of
large numbers of test takers. Additionally, the report
highlights the use of standard best practices adopted
around the world by leading test sponsor
organisations.
Industry White Paper
Five Key Findings for Best-in-Class Computerised Testing

2. 2www.prometric.com |
|
Copyright © 2016 Prometric, Inc. All Rights Reserved.
Third Quarter 2016 | Industry White Paper
Comparative Analysis
Delivering a high-stakes test to large numbers of test
takers in a single-day format can pose challenges in
delivering valid and reliable exams, managing security
risks and ensuring the testing environment allows
honest test takers the ability to demonstrate their
knowledge and qualifications accurately and fairly.
Alternatively, delivering tests over a test administration
window with a smaller test taker footprint enables a
number of quality measures, security safeguards and
test taker advantages.
In July of 2016, Prometric conducted a comparative
analysis between these two types of testing models to
identify the critical components required for a best-in-
class testing and assessment program. Five key
findings are detailed in this report:
1. Test Administrations Require a Robust Lab Set Up.
If the technical infrastructure of a testing environment is comprised of third-party, “rented” test workstations, then
precautionary steps should be taken to mitigate the risks involved. A best-practices approach entails that all third-
party test workstations be completely isolated and rebuilt to ensure an absolute security equivalent to official,
corporate test sites. For example, critical security threats to the exam program would occur if laptops were simply
brought in and connected to workstations by browsers.
Lab Set Up and Security Multi-Day
Testing
Single Day
Testing
Value
Ensure that the central testing
network is completely and
physically isolated from test
workstations.
Yes No The physical separation of test workstations from the central testing
network mitigates security risks from problems that could spread beyond
the existing network into the rental facility.
Install proxy servers at testing
sites.
Yes No The test workstations are further protected by being connected to a
specific and dedicated proxy server built from scratch at the testing site.
Install dedicated servers for test
workstations stations at sites.
Yes No Instead of a laptop-configured server brought to the site, a dedicated test
workstation should be built on site. Such an approach provides extra
security because laptop servers can be compromised when moved from
location to location.
Incorporate best-practices when
formatting workstations and the
operating system (OS) as well
as the installation of additional
software.
Yes No All test workstations should be flattened, and licensed operating systems
(Windows) should be installed from scratch – including the installation of
anti-virus software.
Set a limit of up to 40 test
workstations per testing site.
Yes No To deliver better performance and security, a proxy server and a server
per 40 workstations should be provided instead of up to 200 test
workstations with only one server.

3. www.prometric.com
Third Quarter 2016 | Industry White Paper
2. Staff Recruitment/Management Call for Careful Governance.
When organised cheating occurs, it is dependent upon complicit people being in place who are able to act with little
oversight. For some organisations, there is a tendency to send in one or two staff members to work with a testing
site’s established teams. If cheating is well organised and well financed, though, there is a need for additional
layers of protection to be put in place. By having numerous staff recruited from two different entities – and by using
corporate staff – organisations can minimize the possibility of testing staff engaged in bad and unlawful practices.
Staff and Security Multi-Day
Testing
Single Day
Testing
Value
Avoid having a heavy reliance
upon third-party, “rental” test
sites and provided staff.
No Yes The heavy reliance of staff connected to the rental site can provide a
dangerous exposure to malfeasance. A best-practices approach typically
includes using a consultancy company to help recruit third-party staff.
This practice places additional staff on site to reduce the opportunity for
site staff to be compromised.
Employ dedicated, auditing staff
and a robust process.
Yes No In addition to corporate staff and third-party staff, it is a good business
practice to employ yet another redundant layer of auditors who are
recruited through an unbiased agency as a deterrent to proxy schemes. In
order to run a proxy scheme at a testing center, culprits must be able to
negotiate with staff. Having staff recruited by three different entities
greatly reduces the chance that tests can be “bought” from unscrupulous
site staff.
Ensure there is a high proctor-
to-candidate ratio.
Yes No Best-in-class test delivery providers offer a 1:10 ratio of proctors to test
takers versus a 1:30 ratio offered by second-tier providers. The greater
the ratio, the greater the chance of candidate misbehavior that cannot be
observed nor reported.
Have unbiased test experts
process test results in secure
servers outside of India.
Yes No The use of outside resources in the processing of the score results
provides yet another layer of protection. There is a much less likelihood
for in-market bad actors to influence professional test experts located in
other countries.
3. Best-in-Class Processes Need to Be Measured for Performance Improvement.
High-stakes tests around the world call for clear and robust processes that can be audited and managed for
performance improvement to reduce the probability of organised or even opportunistic foul play.
Process and Security Multi-Day
Testing
Single Day
Testing
Value
Set up a secure zone at the
testing location.
Yes No A secure zone is a dedicated area that is demarcated and enforced. The
secure zone creates a buffer between the facility and the testing
workstations. A secure zone also helps to prevent access by
unauthorised personnel.
Require test takers to check in
and check out – leveraging the
secure zone.
Yes No Test takers should be checked in and out of the testing labs to ensure
that the right person is taking the right test at the right time. This helps
eliminate the risk of proxy testers completing a portion of the exam and
also acts as a deterrent by being a visible reminder of enforced security
measures.

4. www.prometric.com
Third Quarter 2016 | Industry White Paper
Inspect test takers carefully. Yes No Test takers are inspected through frisking, wanding and other checks to
ensure that no foreign materials such as electronic devices, chunky
jewelry and slips of paper are brought into the testing area.
Set video monitoring at a low
ratio of test takers to cameras.
Yes No Fixed cameras that feed to DVR storage should be used to eliminate the
possibility of malfeasance. Using a handycam for “passes through” will
not provide adequate protection. Additionally, fixed cameras and DVR
storage should be combined with a high-proctor ratio and walkthroughs in
the testing area to provide robust security at the time of testing as well as
post-test forensic evidence for all matters that need further investigation.
Perform biometric capture with
real-time feedback.
Yes No With advanced ID management, all test taker biometric data are captured
and securely managed to identify suspected proxy testers and prevent
“blacklisted” test takers from entering the testing labs.
4. Operational Convenience Should Benefit Test Takers When Delivering
High-Stakes Exams.
The technology capabilities of computer-based testing and longer testing administration windows offer test takers
greater conveniences.
Test Taker Convenience Multi-Day
Testing
Single Day
Testing
Value
Offer test takers a range of dates
from which to select.
Yes No Frequently, test takers may have challenges in their personal life that
could interfere with a single date. This can impact them in many ways –
even though they may be able to physically attend, they may be faced
with emotional hardships that can affect their performance.
Take into account accidents and
unforeseen instances that may
impact test takers.
Yes No In every test administration window, there are test takers who have
accidents or encounter weather/public action inconveniences that can
make them miss the testing day. Multiple sessions help to mitigate these
risks so that they can realise a successful testing event.
Afford test takers flexibility so
that they have the best chance to
recover from a disaster.
Yes No There is always the chance that significant events can disrupt a testing
schedule, including everything from typhoons, strikes and social unrest.
Wider testing administration windows offer greater flexibility for test
takers so that they can test successfully despite such instances.
5. Valid, Reliable and Fair Tests Start with a Knowledge of Test Content Development.
High-stakes tests must be of the highest quality, accurately measure the necessary knowledge and skills and be
fair to all test takers. Developing psychometrically sound, legally defensible tests requires significant investment in
time, knowledge and resource.
Valid, Reliable and Fair Tests Multi-Day
Testing
Single Day
Testing
Value
Develop test content in
accordance with industry-
recognised standards.
Yes No The need for stricter content, statistical guidelines and validity
evidence increases as the stakes regarding test use increase. Test
development practices should meet the high standards of the ETS

5. www.prometric.com
Third Quarter 2016 | Industry White Paper
Standards for Quality and Fairness. The core of any test
development process should adhere to industry best practices and
support content validity and reliability of results.
Conduct thorough psychometric
analyses to keep exam
performance sound.
Yes No Psychometric analyses determine how effective test items are and
how they perform. Each test and its items should be subjected to
various analyses aimed at determining the quality of the assessment.
Adopt an equating model when
assessing the comparability of
scores across different test forms.
Equating Normalization The comparability of scores across different test forms of a
standardised assessment has been a focus of educational
measurement and the testing industry for the past 90 years.
Normalisation is done based on the assumption that "in all multi-
sessions of a test, the distribution of abilities of test takers is the
same across all the sessions." There is no evidence supporting that
test taker groups who take different forms are of equal ability.
Equated scores are not affected by the problems that occur with the
normalisation because equating adjusts for differences in the
difficulty of test forms and test taker abilities. Item response theory
(IRT) anchoring and calibration is used for hundreds of high-stakes
testing programs around the world and has been demonstrated to be
highly defensible to legal challenges of fairness in scoring.
Secure all test development
processes and exam content.
Yes Yes Every precaution should be taken to prevent unauthorised disclosure
of examination content and assure the integrity of examinations. All
test items and item banks should be securely created, stored,
accessed and delivered with full encryption to ensure maximum
security of intellectual property.
Executive Summary
High-stakes examinations help derive important, life-changing decisions – assisting individuals to realize their human
potential, be it a certification, assessment, licensure or entrance exam.
While India has begun to embrace the digital era, it is critical as never before that licensing, certification and academic
organisations rely on best-in-class testing and assessment processes to ensure the integrity and reliability of their
testing programs.
Testing and assessment service providers have the responsibility to ensure that the exam content they deliver on
behalf of a test sponsor organisation as well as their test centers give candidates the opportunity to demonstrate their
knowledge and qualifications accurately and fairly.
Copyright©
2016 Prometric, Inc. All Rights Reserved.