Education & Skills Online is an assessment tool designed to provide individual-level results that are linked to the OECD Survey of Adult Skills (PIAAC) measures of literacy, numeracy and problem solving in technology-rich environments. All results are comparable to the measures used in PIAAC and can be benchmarked against the national and international results available for the participating countries. In addition, the assessment contains non-cognitive measures of skill use, career interest, health and well-being, and soon also behavioral competencies.

1. 0
Education & Skills Online

2. What is Education & Skills Online?
• The OECD Education & Skills Online offers individuals,
businesses, community based and government organizations a
powerful new set of tools to assess their adult populations.
• It is specifically designed to provide individual level data that
is linked to PIAAC and, therefore, shares the same validity
evidence
• It measures both cognitive and non-cognitive skills
• It uses an Internet delivered platform that is available 24 / 7
and includes technical support
• It was developed and field tested in 9 language versions in
2013, and there will be a subsequent field test for additional
countries who may want to join
1

3. Education & Skills Online Contents
• Core background questionnaire
• Cognitive modules:
• Literacy and numeracy assessments with linking
PIAAC items and new items developed
specifically for Education & Skills Online
• 53% linking items and 47% new items
• Reading Components module to evaluate the
literacy skills of low-performing readers
• Evaluates print vocabulary, sentence processing, and passage
comprehension
• Problem Solving in Technology Rich
Environments module
2

4. Education & Skills Online
Contents Continued
Noncognitive modules:
• Career Interest and Intentionality - This module measures
an individual’s preferences for different types of work activities and environments and
the level of an individual’s intention to seek out new job opportunities and career-
and job-related training.
• Skill Use – This module utilizes items from PIAAC to assess the specific skills
that respondents use in both their work and daily lives as important drivers of skill
acquisition as well as critical outcomes affecting their lives. The questions in this
module focused on skills associated with reading, writing, use of mathematical
information and ideas, and information and communications technology (ICT).
• Subjective Well-Being and Health - This module examines
the main components of subjective well-being: life evaluation and positive and
negative affect, in addition to subjective health and well-being indicators.
• Behavioral Competencies (Forthcoming)- This module
measures selected personality facets based on their high relevance and utility for
academic and workforce readiness and success.
3

5. Uses and Benefits
• Individual scores in each measured domain provided
automatically and in real time
• Incorporates item response theory, multi-stage adaptive
testing and automated scoring
• Individuals and organizations have flexibility to determine
which skills to assess
• Organizations purchasing tests will have access to their
database of results for further analysis
4

6. Package Types
Customers have the option to purchase codes in three package types,
described here.
5
Package Contents
Cognitive • Background questionnaire
• Literacy and Numeracy assessments with adaptive
testlets
• Reading components
• Problem solving in technology-rich environments
Noncognitive • Background questionnaire
• Career Interest and Intentionality
• Skill Use
• Subjective Well-Being and Health
Bundled Cognitive
and Noncognitive
• The contents of both the cognitive and
noncognitive packages with results linked by the
authorization code

7. Workflow of Education & Skills Online
6

8. How Test Takers Access Assessment
 Each test taker will receive a unique 10-digit test authorization
code from the customer who purchased the codes.
 The customer will direct the test taker to the Education &
Skills Online log-in page (next slide) where the test taker will
enter the test code twice to log in.
 In the upper right hand corner of the log-in page, test takers
or administrators can use the drop-down menu to change the
language of the log-in page. The language of the test is
determined by the test authorization code.
 Test takers MUST use the Firefox browser to take the
assessment.
7

9. 8
Change language

10. Home Page
9

11. Background Questionnaire
1. Age
2. Gender
3. Highest Education Level
4. Were you born in [Country]?
 If no, in what year did you come to live in [Country]?
5. What is your first language?
 Language typed in if answer is “other”
6. Current employment status
 Current occupation (ISCO-08 code)
10

12. Starting the Core Cognitive Assessment
11

13. Core Assessment
Workflow Detail
12

14. Literacy and Numeracy
Testlet Formation
• Each test taker will be presented with two testlets for literacy and two
testlets for numeracy
• A testlet is a set of units grouped to target the low, middle, and
high end of the PIAAC literacy or numeracy scale
• Testlets overlap so that a few items are shared by two testlets
13
Testlet
1
Testlet
2
Testlet
3
◄ Easier - - - - - - - - - - - - - - - Harder ►

15. Literacy and Numeracy
Assessment
• The literacy and numeracy assessment is expected to take 60
minutes.
• The test uses the test taker’s response to certain background
questions to determine which stage 1 testlet to present.
• There are 9 questions in stage 1 for both literacy and numeracy.
• The test taker’s score on the first testlet is used to choose the
stage 2 testlet.
• There is no transition screen between the first and second testlet
– it appears seamless to the test taker.
• There are between 11 and 13 questions in the stage 2 testlets.
14

16. Resuming an Assessment
15

17. Score Reports
• The customer decides whether the test taker will see a score
report at the end of the test or whether the score report will
be saved in the online system that can be accessed by test
administrators.
16

18. Score Reports for Literacy
and Numeracy
• Include an explanation of the skills that were measured based
on PIAAC proficiency level standards and how those skills are
used in everyday life
• Characterize the strengths and weaknesses in the skill areas
assessed
• Provide a scale score for the individual test taker
• Provide a comparison of the test taker’s score to the national
and international data from PIAAC by education level,
occupation, and age group
17

19. 18
Literacy and
Numeracy
Score Report

20. Optional Cognitive
Assessments
19

21. Optional Cognitive Modules
Workflow Detail
20

22. Reading Components
• Includes three sections:
• Vocabulary (38 items)
• Sentence Comprehension (22 items)
• Passage Comprehension (44 items)
• Test takers are given two scores for each section:
one for their rate and one for their accuracy
21

23. 22
Reading
Components
Score Report

24. Problem Solving in
Technology-Rich Environments
• Begins with a short orientation for the test taker
• 9 questions from PIAAC
• Test takers see a score report after completing the
test
23

25. 24
Problem Solving
in Technology-
Rich
Environments
Score Report

26. All Assessments Completed
25
• When the test taker completes all the modules in the workflow, all the
modules will appear in the Completed Assessment table.
• Clicking on the name of the completed module will open the score report for
that particular module (if the score reports are turned on for the institution).

27. Noncognitive Module
Workflow Detail
26
Skill Use
+
Score Report
Career Interest
and
Intentionality
+
Score Report
Subjective Well-
Being and
Health
+
Score Report
If noncognitive modules are included in the purchased
product package, then the test taker has the option to
take the following modules in any order:

28. Noncognitive Modules
27

29. Skill Use
• 57 items (from the PIAAC background questionnaire)
• Takes approximately 5 minutes to complete.
• Assesses the specific skills that respondents use in both their
work and daily lives as important drivers of skill acquisition as
well as critical outcomes affecting their lives.
• The questions in this module focused on skills associated with
reading, writing, use of mathematical information and ideas,
and information and communications technology (ICT).
28

30. Career Interest and
Intentionality (CII)
• 92 questions
• Takes approximately 10 minutes to complete.
• This module measures an individual’s preferences for different
types of work activities and environments and the level of an
individual’s intention to seek out new job opportunities and
career- and job-related training.
29

31. Subjective Well-Being
and Health
• 27 questions
• Takes approximately 5 minutes to complete
• Research has shown the predictiveness of health for education
and work-related outcomes, with healthy individuals being
more productive and making less use of health care.
• This module examines the main components of subjective
well-being: life evaluation and positive and negative affect in
addition to subjective health and health indicators in line with
the measures described in the World Health Organization’s
agenda.
30

32. Pricing and
Test Administration Portal
31

33. Pricing for Package Types
• Package 1: Cognitive Modules
• Package 2: Noncognitive Modules
• Package 3: Bundled Cognitive and Noncognitive
32
Quantity
Purchased
Unit Price
Package 1
Unit Price
Package 2
Unit Price
Package 3
<5,000 €9.00 €2.00 €11.00
5,000 – 10,000 €8.25 €2.00 €10.25
10,001 – 25,000 €7.75 €2.00 €9.75
25,001 – 75,000 €7.00 €2.00 €9.00
75,001 – 149,999 €6.00 €2.00 €8.00
150,000+ €5.00 €2.00 €7.00

34. Test Administration Portal
• Purchasers will have access to an online portal to:
• Monitor test completion
• Download result data in Excel spreadsheets for
analysis
• Background questions, scale scores in literacy,
numeracy, and problem solving, and noncognitive
measures
• Create additional users for their organization
• Create up to two organizational levels below the
purchasing institution
• Read technical documentation
33

35. Functionality in Test
Administration Portal
• Create new agencies and locations and assign codes to
those groups
• Download Excel spreadsheets with lists of activated
codes
• View available/used code inventory for each agency or
locations
• Reclaim unused inventory from an agency or location
that can be reallocated to another agency or location
• View and print score reports
• Create additional users at each level:
• Administrators – access to all functionality, including downloading data and
creating new users
• Operations Support – ability to download authorization code spreadsheets
and assign test codes to sub-levels, but not to download data or create
new users
34

36. Distribution of Test
Authorization Codes and Data Access
• Purchasers can choose to create sub-levels to manage test authorization
codes
• Purchasers can distribute codes either by allowing these levels to draw down
codes automatically or requiring them to request codes through the level
above them
• Users at each level can be either administrators with full access to all features
or operations support with the ability to distribute codes but not download
data
• Each level can download data for its institution and the levels below its
institution
35
Location
Agency
Organization
(Purchaser)
Organization
(1,000 codes)
Agency
(500 codes)
Location
(250 codes)
Location
(250 codes)
Agency
(500 codes)
Location
(250 codes)
Location
(250 codes)

37. Data Download
• Only users with administrator rights are able to download data
in an Excel file format.
• Data can be downloaded on demand. Users do not need to
request data from ETS.
• When downloading data, users can use search features to
specify a date range or use pre-designated time rages such as
“all data from the last week.”
• The authorization code is the unique identifier of the individual
test-taker. No personal information (names, email addresses,
phone numbers) is collected by Education & Skills Online;
therefore, no personal information is stored in the system.
36

38. Data Sharing
• The purchaser will own the data and therefore has the
responsibility of notifying individuals of their rights and
obtaining any permission necessary for sharing the data that
has been collected.
• Any entity (country, institution, etc.) can become a purchaser
in order to maintain control over data and privacy concerns.
• The purchaser may send data to other entities, with proper
permission from the individual test takers, but must do so
outside of the test delivery system.
• A purchaser may arrange to share data with the OECD, but
any agreements would be the responsibility of the purchaser
in cooperation directly with the OECD. It will be the
purchaser’s responsibility to supply the data to the OECD in
the agreed upon format and conditions.
37

39. Data Download Fields
General Information Data
Fields
(for all package types)
• Authorization Code
• Institution Name & ID
• Language
• Background questionnaire
answers (9)
• Date and time the code was
first used
• Start date and time (for each
module)
• Last accessed date and time
(for each module)
Core Cognitive Data Fields
• Literacy and Numeracy scale
scores
• Literacy and Numeracy level
• Reading Components scores
(rate & accuracy) for
vocabulary, sentence
comprehension, and passage
comprehension
• Reading Components levels for
rate and accuracy for the three
components
• Problem Solving scale score
• Problem Solving level
38

40. Noncognitive Data
Download Fields
Subjective Well-Being and
Health
• Life Satisfaction, Positive
Affect, and Negative Affect
levels (low/medium/high)
• Subjective Health level
(good/fair/poor)
• BMI class
• Sleep, Diet, Smoking, and
Physical Activity levels
Skill Use
• Levels for Reading, Writing,
Numeracy, and ICT at work
and at home (8 scores)
• Responses to 6 questions
about computer use
Career Interest &
Intentionality
• Scores for each of the 6
interest categories
• ISCO codes and job fit
level for current and
intended jobs
• ISCO codes for the 20
best-fitting jobs
• ISCO codes for the 10
poorest-fitting jobs
• Levels for questions about
intention of finding a new
job/seeking training
39

41. Find Out More About
Education & Skills Online at:
www.oecd.org/skills/ESonline-
assessment/
Follow us on:
www.facebook.com/OECDSkillsSurveys
Education & Skills Online is a joint initiative of the OECD, participating
countries and the European Commission. The development of the assessment
was possible thanks to the financial support of the European Commission.
The delivery of and user support for the assessment is provided by Education
Testing Services (ETS) on behalf of the OECD

42. Questions?
Contact Ann Kennedy or Luisa Langan
at ETS
at ESonline-support@ets.org