eduworks-network.eu
facebook.com/eduworksnetwork
@EduworksNetwork

Presentation for Textkernel,
17 January 2014
Gábor Kismihók, Stefan Mol
Amsterdam Business School
http://abs.uva.nl/ and www.jobknowledge.eu

Kea Tijdens, AIAS (Amsterdam Institute of Advanced
Labour Studies) http://www.uva-aias.net/39

University of Amsterdam
Outline
• Ongoing Projects
•
•
•
•
•
•
•

WageIndicator
InGRID
Center of Job Knowledge Research
OntoHR
med-assess
OntoTech
UvAInform

• Eduworks www.eduworks-network.eu/
•
•
•
•
•

Introduction
Studies in Personnel Selection
Studies in training
Labour market driven learning analytics
Eduworks organisation
WageIndicator web survey
• Data collection: WageIndicator web-survey
• WageIndicator web-survey posted on 70 national WageIndicator websites
(20 mln visitors 2013)
in Netherlands known as Loonwijzer.nl, in Germany as Lohnspiegel.de, etc.
• Web-survey has questions about work and wages in national languages in 70 countries (soon
75)
• Web-survey is posted continuously, inviting visitors to complete the survey with prize incentive
• approx. 200,000 observations per year

• Databases for „long-list questions‟ in web survey
• „What is your occupation?‟: web-survey uses an API for this question, offering visitors a choice
between semantic matching or search tree, using the WISCO database of occupations
• „In which industry do you work?‟: API for 300 industries (coded NACE2.0)
• „What is your education/region/language spoken at home?‟: API has labels in national language

• WISCO database of occupations
• with 1700 occupational titles in languages of 70 countries (coded ISCO-08)

• www.wageindicator.org
Researching occupations
• Tasks in occupations
• WageIndicator web-survey used for testing features of occupations: asking
jobholders how often they perform a task, using a list of approx. 10 tasks per
occupation, specified for 433 occupations in 13 countries (aim: N=50,000)

• Comparing vacancies and jobholders
• confronting required skill levels in vacancies and attained skill levels of jobholders
in same occupations
• using export of EURES vacancy data and WageIndicator web survey data
• currently for Czech Rep, more countries foreseen

• https://inclusivegrowth.be/
• Funded from EU‟s FP7 research program for research infrastructures (2013-17)
Center of Job Knowledge Research
• Why CJKR? Because job knowledge…
• Is among the most important yet least understood individual
level drivers of the knowledge economy
• Is imperative to successfully match people to jobs but largely
neglected in personnel selection research
• Is an ultimate outcome against which both education and
training must be evaluated
• Is a means to ensure public spending on education enhances
graduates‟ opportunities, fit, and accomplishment on the labor
market
www.jobknowledge.eu
Job Knowledge
“…people who are more intelligent learn
more job knowledge and learn it faster, the
major determinant of job performance is
not GMA but job knowledge” (Schmidt and
Hunter, 2001).
GMA

Job knowledge

Job
Performance
Undeniably Job General

Unequivocally Job Specific (albeit more portable for more related occupations)

Job
Knowledge

GMA

Education
Job Specific – Job
General

Job
Performance
A personnel selection and training platform that takes an
individualized approach to the assessment and development
of job knowledge of ICT Systems analysts
www.ontohr.eu
Transfer of innovation: Development of job knowledge test
and training platform for nurses
www.med-assess.eu
OntoTech
Extending portfolio of ICT Job Knowledge Tests for
purposes of curriculum diagnostics and development
UvAInform
Learning analytics is “the measurement, collection, analysis,
and reporting of data about learners and their contexts, for
purposes of understanding and optimizing learning and the
environments in which it occurs” (SOLAR 2012).
Internal UvA project aimed at the pan-university
development of an educational data informed student
feedback interface.
•
•
•
•
•
•
•
•
•
•
•
•
•

GITP
Randstad Holding
ECORYS
Aristotle University of Thessaloniki (AUT)
Central European Labour Studies Institute (CELSI)
Corvinus University of Budapest (CUB)
Ericsson
European Distance and E-Learning Network (EDEN)
European Foundation for the Improvement of Living and Working Conditions (EUROFOUND)
Labour Asociados
Netpositive
University of Alicante (UAL)
Wageindicator
ER 5 & 6: Occupations and
skills (2 postdoc researchers)
• Dynamics within and across occupations
• testing theories concerning the clustering of tasks into occupations, using the tasks
data for 433 occupation for 13 countries, using web survey data
• testing theories how employers define the required skill levels for vacancies, using
EURES vacancy data

• Building an occupational information system
• improving the measurement of job titles and their coding into classifications
-> using semantic matching for surveys
-> using statistical likelihood estimates given industry, education, gender, age
(from previous data collections)
• expanding the WISCO database of 1,700 occupations for 75 countries with more
occupational titles and more languages
• building an occupational and educational information system
ESR1: Leveraging the potential of job
knowledge to fit individuals to jobs:
Studies in Personnel Selection
Research
• State of the art on job knowledge based personnel selection: A (quantitative)
literature review
• Development and psychometric validation of job knowledge tests for purposes
of personnel selection
• Field and (quasi) experimental studies providing support for the (causal) job
knowledge mediated relationship between GMA and Job Performance

Data
• Primary Studies, interviews with job incumbents (N>50) and HR Managers
(N>20), content analysis of vacancies and job databases, multisource surveys
• We foresee close collaboration with ESR#2.
ESR2: Leveraging the potential of job
knowledge to fit individuals to jobs:
Studies in training
Research
• State of the art on job knowledge based training: A (quantitative) literature
review
• Development and psychometric validation of job knowledge tests for training
needs analysis and training validation
• Field and (quasi) experimental studies providing support for the (causal) job
knowledge mediated relationship between training and Job Performance
• Temporal dynamism in the co-development of job knowledge and job
performance over time.

Data
• Primary Studies, interviews with job incumbents (N>50) and HR Managers
(N>20), content analysis of vacancies and job databases, multisource surveys
• We foresee close collaboration with ESR#1.
ESR7: Labour market driven
learning analytics
Research
• State of the art on labour market context educational outcome measures:
A literature review
• Exploring Person-Education-Labour market (mis)matches by merging big data
from education (HvA, UvA), Randstad, GITP, etc.
• Identifying educational context predictors of labour market success
• Provision of labour market validated feedback to current students to enhance
their study success

Data
• Secondary data obtained from HvA and UvA student data warehouses,
including grades, behaviors in online learning environments), labour market
data from associate partners (GITP, Randstad, etc).

Eduworks presentation at Textkernel 17-01-2014

  • 1.
    eduworks-network.eu facebook.com/eduworksnetwork @EduworksNetwork Presentation for Textkernel, 17January 2014 Gábor Kismihók, Stefan Mol Amsterdam Business School http://abs.uva.nl/ and www.jobknowledge.eu Kea Tijdens, AIAS (Amsterdam Institute of Advanced Labour Studies) http://www.uva-aias.net/39 University of Amsterdam
  • 2.
    Outline • Ongoing Projects • • • • • • • WageIndicator InGRID Centerof Job Knowledge Research OntoHR med-assess OntoTech UvAInform • Eduworks www.eduworks-network.eu/ • • • • • Introduction Studies in Personnel Selection Studies in training Labour market driven learning analytics Eduworks organisation
  • 3.
    WageIndicator web survey •Data collection: WageIndicator web-survey • WageIndicator web-survey posted on 70 national WageIndicator websites (20 mln visitors 2013) in Netherlands known as Loonwijzer.nl, in Germany as Lohnspiegel.de, etc. • Web-survey has questions about work and wages in national languages in 70 countries (soon 75) • Web-survey is posted continuously, inviting visitors to complete the survey with prize incentive • approx. 200,000 observations per year • Databases for „long-list questions‟ in web survey • „What is your occupation?‟: web-survey uses an API for this question, offering visitors a choice between semantic matching or search tree, using the WISCO database of occupations • „In which industry do you work?‟: API for 300 industries (coded NACE2.0) • „What is your education/region/language spoken at home?‟: API has labels in national language • WISCO database of occupations • with 1700 occupational titles in languages of 70 countries (coded ISCO-08) • www.wageindicator.org
  • 4.
    Researching occupations • Tasksin occupations • WageIndicator web-survey used for testing features of occupations: asking jobholders how often they perform a task, using a list of approx. 10 tasks per occupation, specified for 433 occupations in 13 countries (aim: N=50,000) • Comparing vacancies and jobholders • confronting required skill levels in vacancies and attained skill levels of jobholders in same occupations • using export of EURES vacancy data and WageIndicator web survey data • currently for Czech Rep, more countries foreseen • https://inclusivegrowth.be/ • Funded from EU‟s FP7 research program for research infrastructures (2013-17)
  • 5.
    Center of JobKnowledge Research • Why CJKR? Because job knowledge… • Is among the most important yet least understood individual level drivers of the knowledge economy • Is imperative to successfully match people to jobs but largely neglected in personnel selection research • Is an ultimate outcome against which both education and training must be evaluated • Is a means to ensure public spending on education enhances graduates‟ opportunities, fit, and accomplishment on the labor market www.jobknowledge.eu
  • 6.
    Job Knowledge “…people whoare more intelligent learn more job knowledge and learn it faster, the major determinant of job performance is not GMA but job knowledge” (Schmidt and Hunter, 2001). GMA Job knowledge Job Performance
  • 7.
    Undeniably Job General UnequivocallyJob Specific (albeit more portable for more related occupations) Job Knowledge GMA Education Job Specific – Job General Job Performance
  • 8.
    A personnel selectionand training platform that takes an individualized approach to the assessment and development of job knowledge of ICT Systems analysts www.ontohr.eu
  • 9.
    Transfer of innovation:Development of job knowledge test and training platform for nurses www.med-assess.eu
  • 10.
    OntoTech Extending portfolio ofICT Job Knowledge Tests for purposes of curriculum diagnostics and development
  • 11.
    UvAInform Learning analytics is“the measurement, collection, analysis, and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs” (SOLAR 2012). Internal UvA project aimed at the pan-university development of an educational data informed student feedback interface.
  • 14.
    • • • • • • • • • • • • • GITP Randstad Holding ECORYS Aristotle Universityof Thessaloniki (AUT) Central European Labour Studies Institute (CELSI) Corvinus University of Budapest (CUB) Ericsson European Distance and E-Learning Network (EDEN) European Foundation for the Improvement of Living and Working Conditions (EUROFOUND) Labour Asociados Netpositive University of Alicante (UAL) Wageindicator
  • 15.
    ER 5 &6: Occupations and skills (2 postdoc researchers) • Dynamics within and across occupations • testing theories concerning the clustering of tasks into occupations, using the tasks data for 433 occupation for 13 countries, using web survey data • testing theories how employers define the required skill levels for vacancies, using EURES vacancy data • Building an occupational information system • improving the measurement of job titles and their coding into classifications -> using semantic matching for surveys -> using statistical likelihood estimates given industry, education, gender, age (from previous data collections) • expanding the WISCO database of 1,700 occupations for 75 countries with more occupational titles and more languages • building an occupational and educational information system
  • 16.
    ESR1: Leveraging thepotential of job knowledge to fit individuals to jobs: Studies in Personnel Selection Research • State of the art on job knowledge based personnel selection: A (quantitative) literature review • Development and psychometric validation of job knowledge tests for purposes of personnel selection • Field and (quasi) experimental studies providing support for the (causal) job knowledge mediated relationship between GMA and Job Performance Data • Primary Studies, interviews with job incumbents (N>50) and HR Managers (N>20), content analysis of vacancies and job databases, multisource surveys • We foresee close collaboration with ESR#2.
  • 17.
    ESR2: Leveraging thepotential of job knowledge to fit individuals to jobs: Studies in training Research • State of the art on job knowledge based training: A (quantitative) literature review • Development and psychometric validation of job knowledge tests for training needs analysis and training validation • Field and (quasi) experimental studies providing support for the (causal) job knowledge mediated relationship between training and Job Performance • Temporal dynamism in the co-development of job knowledge and job performance over time. Data • Primary Studies, interviews with job incumbents (N>50) and HR Managers (N>20), content analysis of vacancies and job databases, multisource surveys • We foresee close collaboration with ESR#1.
  • 18.
    ESR7: Labour marketdriven learning analytics Research • State of the art on labour market context educational outcome measures: A literature review • Exploring Person-Education-Labour market (mis)matches by merging big data from education (HvA, UvA), Randstad, GITP, etc. • Identifying educational context predictors of labour market success • Provision of labour market validated feedback to current students to enhance their study success Data • Secondary data obtained from HvA and UvA student data warehouses, including grades, behaviors in online learning environments), labour market data from associate partners (GITP, Randstad, etc).