The need for global workforce skills credentials continues to grow. At the same time, the global workforce is shrinking. It is imperative that skill recognition be accurate and the level of test security be appropriate for the skills being assessed. The Security subcommittee of the new Workforce Skills Credentialing division of ATP created a new test security framework that will provide guidance to testing organizations when selecting the level of security needed for their assessments. Join our guest presenters, Rachel Schoenig and Jennifer Geraets of ACT, as they discuss the challenge of identifying global workforce skills and how this new test security framework will help to align the expectations of those involved with workforce credentialing (e.g., test publishers, examinees, and employers). Rachel and Jennifer will also provide a call to action, requesting your comments on this new framework.

1. The Test Security Framework:
Why different tests need different
test security requirements
June 29, 2016
Caveon Webinar Series

2. www.caveon.com2
Today’s Presenters
Rachel Schoenig, ACT
AVP and Head of Test
Security
Jennifer Geraets, ACT
Senior Test Security
Analyst
Jamie Mulkey,
Caveon
VP, Client Services

3. Agenda
• Division and Project Background
• Process
• Work to Date
• Next Steps
3 www.caveon.com

4. ATP Workforce Skills
Credentialing Division
ATP Workforce Skills Credentialing Division
1. Security and Privacy Subcommittee
2. Initiative Outreach Taskforce
4 www.caveon.com

5. ATP Workforce Skills Credentialing
Division
• Workforce: The people engaged in or
available for work, either in a country or
area or in a particular company or industry.
• Credential: The evidence of mastery,
competence, or achievement of a specific
topic or skill that is under the control of the
individual.
5 www.caveon.com

6. Security & Privacy Subcommittee
1. Survey of workforce skills
credentials and test security
2. Framework for evaluating
security and privacy of
credential offerings
6 www.caveon.com

7. Workforce Shifts
Shrinking Workforce
7
Sources: Pew Research Center; United Nations; Organisation for Economic
Cooperation and Development; International Politics and Society
www.caveon.com

8. Workforce Shifts
Reliance on younger
generations
8
Sources: Worldwatch Institute; Organisation for
Economic Cooperation and Development
www.caveon.com

9. Workforce Shifts
Migration
9
Sources: Organisation for Economic Cooperation and Development
www.caveon.com

10. Workforce Skills are Important
10 www.caveon.com

11. Now what?
11
Access to
education
Better skills
and jobs
matches
Formal
recognition
of skills
www.caveon.com

12. Incomplete Equation
www.caveon.com12
Skills /
Competency
Content
Trustworthy
Portable
Credentials

13. Access to Opportunity
13 www.caveon.com

14. www.caveon.com14

15. Access to Opportunity
$$ Thousands
$$
15 www.caveon.com

16. Access to Opportunity
FAKE Credentials
are the OPPOSITE
16
Source: Business Insider

17. Complete Equation
www.caveon.com17
Content
Trustworthy
Portable
Credentials
Test
Security

18. ATP Workforce Skills Division
Framework of Test
Security
18 www.caveon.com

19. Process
19
Testing
Company
Employee Employer
www.caveon.com

20. Examinee Perspective
• Value
– What does this credential say about my skills?
• Fairness
– What does everyone else have to do to earn this
credential?
– Is this a fair process?
– What happens if someone cheated?
20 www.caveon.com

21. Credential User Perspective
• Trustworthiness
– Did the individual earn the credential and
have the required skills?
– How can I authenticate the credential?
– How will I know if a credential is revoked?
• Portability and Value
– Who administered the test and what brand
is attached to the credential?
– Who else relies on the credential?
21 www.caveon.com

22. Methodology
1. Test Taker Identification
2. Test Administration
3. Test Design
4. Results Validation and Authentication
5. Investigation and Remediation
6. Sustainability
22 www.caveon.com

23. Methodology
23
One Size
Fits All
www.caveon.com

24. Methodology
• Surveys
– Employees
– Employers
– Test publishers
24 www.caveon.com

25. Work to Date
• Framework
– 6 categories
– 4 tiers
– Capabilities / tools at each tier
• Transparency
• Self-identification
25 www.caveon.com

26. Applying the Framework
Hypotheticals
26 www.caveon.com

27. Credential Security –
Pre-Employment testing
• JRJ Manufacturing company needs entry
level workers to operate a simple piece of
machinery.
• They offer candidates a pre-employment
reading test to assess comprehension and
the ability to follow directions.
– The test is given online, without any
proctoring
– Score results are used with other factors to
determine if the candidate moves to the
next round of employment interviews.
27 www.caveon.com

28. Using the Framework
Test Administration – Test Taker
28
I know my credential has value and
that results will be reliable if I follow
the rules
 I have been told the testing rules in
advance and know I am expected
to follow all rules concerning
appropriate testing conduct
 I am able to take the test on my
own
 I may or may not be proctored
during testing
I know my credential is valuable and that
testing is fair because there are strong
standardized test administration
requirements.
• I have to test at a designated, secure
testing location
• I have been told the rules in advance
and know I am expected to follow all
rules concerning appropriate testing
conduct
• I am prohibited from bringing
unauthorized aids and electronics
(digital devices) into the testing room
and expect additional security to
prevent me or others from doing so
Level 1 Level 4

29. Credential Security –
Medical Device Certification
• The new JRJ hand-held medical sonogram
is being marketed in the US and Europe
but a credential is required for use
– Individuals taking the sonogram device training
already have medical credentials (MD, RN,
RDMS)
– A computerized test is completed at the end of
training course completion. Scores are
recorded and tracked.
– The test is monitored by the instructor who
taught the course.
29 www.caveon.com

30. Using the Framework
Test Taker Identification – Test Publisher
30
Level 1 Level 4
Minimal identification and
authentication tools
 Examinee self-identifies
 Credential issuer may seek to
match identification through
other tools (such as Facebook
or Twitter accounts)
 Credential issuer will seek to
verify identification if questions
arise
Strong identification and authentication
tools
• Identification is established by
requiring government-issued photo
identification
• Identification is further established via
biometrics tools, challenge questions
and presentation of government-
issued photo identification
• Test taker image is captured at time of
testing

31. Challenges
• Communicating the Framework
• Getting credential users and examinees to
request and use the Framework levels
• Will users see the Framework’s flexibility?
• Consulting the Framework before test
development
31 www.caveon.com

32. Next Steps
Work in Process
• Completion & publication by ATP 2017
• Privacy Framework is next
32 www.caveon.com

33. Next Steps
Reviewers needed!
33 www.caveon.com

34. Complete Equation
www.caveon.com34
Content
Trustworthy
Portable
Credentials
Test
Security

35. Q&A
www.caveon.com35

36. Thank You!
www.caveon.com36
Follow Caveon on twitter @caveon
Check out our blog
www.caveon.com/blog
LinkedIn Group “Caveon Test Security”
Rachel Schoenig
Rachel.Schoenig@act.org
Jennifer Geraets
Jennifer.Geraets@act.org
Dr. Jamie Mulkey
Jamie.Mulkey@caveon.com