vacancy mining

1. eduworks-network.eu
facebook.com/eduworksnetwork
@EduworksNetwork
This project has been funded with support from the European Commission.
This communication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be
made of the information contained therein.
Automatic Extraction of Job
Information from Job
Vacancies
Vladimer Kobayashi
Advisers: Stefan Mol, Gábor Kismihók, and Deanne den
Hartog

2. Online Vacancies

3. What are in a
vacancy*?
• Worker-oriented domain
1. Worker characteristics
2. Worker requirements
3. Experience Requirements
Worker oriented
Job oriented domain
• Job-oriented domain
1. Occupational requirements
2. Workforce characteristics
3. Occupation-specific information
*Based on O*NET’s Content Model

4. Job Information Extraction from
Vacancies
• XML
• Databases
Data
Integration &
Management
• Part-of-
Speech
tagging
• Classification
Automatic
Extraction
• Summarisation
• Visualization
• Analytics
Presentation

5. Method – Automatic Extraction
Sentences Feature
matrix
Preprocessing
and
Segmentation
Feature
Extraction
Classification
model
Random
Forest,
SVM, and
Naive Bayes
Hard to
classify
sentences
Query by
committee
Newly expert
labelled
sentences
Retrain
Vacancies
Expert
Classified
sentences
Validation

6. Preprocessing
• Punctuation removal
• Lower case
• Sentence segmentation
• Stopword removal
• We do not remove these stopwords
“to", "have", "has", "had", "must","can", "could", "may","might",
"shall","should","will", and "would"

7. Feature Type Number of derived
features
Variable Type
Part of speech (POS) tag of the
first word
1
Categorical (actual POS)
Is the first word in this sentence
unique in work activity sentences
(based from the labelled data)
1
Numeric
Is the first word in this sentence
unique in worker attribute
sentences (based from the
labelled data)
1
Numeric
Is the last word in this sentence
unique in work activity sentences
(based from the labelled data)
1
Numeric
Is the last in this sentence unique
in worker attribute sentences
(based from the labelled data) 1
Numeric

8. Feature Type
Number of derived features
Variable Type
Proportion of adjectives 1 Numeric
Proportion of verbs 1 Numeric
Proportion of word “to” 1 Numeric
Proportion of modal verbs
1
Numeric
Proportion of numbers 1 Numeric
Proportion of adverbs 1 Numeric

9. Feature Type Number of derived features Variable Type
Proportion of nouns 1 Numeric
Proportion of nouns, verbs,
adjectives, adverbs, and other
part of speech tags followed by
another verb
5
Proportion of unique words
found only in work activity
sentences (based from the
labelled data)
1
Numeric
Proportion of unique words
found only in worker attributes
sentences (based from the
labelled data)
1
Numeric
Frequency of keywords for work
activity and worker attributes
sentences
149
Numeric

10. Validation
• Compare with independent expert
• Compare with task inventory

11. Key results
• We identified
• 270,000 work activity sentences
• 317,000 work attribute sentences
• Classifier is at least 90 percent accurate (10-fold cross
validation)

12. Topic 100
development
software
agile
methodologies
application
scrum
design
life
Topic 86
new
learn
quickly
willingness
adapt
technologies
internet
desire
Topic 132
travel
willingness
willing
work
time
needed
internationally
international
Topic 20
sales
selling
salesforcecom
outside
crm
success
account
inside
Topic 75
communication
written
oral
verbal
interpersonal
presentation
effective
listening
Topic 18
highly
motivated
oriented
self
driven
organized
starter
selfstarter
Key worker attributes from topic
modeling

13. Topic 61
license
valid
drivers
driving
record
transportation
reliable
vehicle
Topic 16
data
analysis
quantitative
research
statistics
economics
statistical
modeling
Topic 60
scripting
python
linux
programming
java
perl
languages
unix

14. Word2vec – word similarity
Word Cosine similarity
interpersonal 0.90
verbal 0.90
skills 0.88
written 0.85
strong 0.84
excellent 0.83
good 0.83
communicator 0.81
ability 0.80
organisational 0.80
Words similar to communication

15. Job clusters according to worker
attributes

16. Sample application

17. 17

18. 18

19. 19

20. 20

21. Job Vacancy Information
Classifier
https://youtu.be/vVVL3teMqeY

22. Applications
• Task Analysis (with Expert Validation)

23. Applications
• Hybrid Teachers
Job Profession Group Example Vacancies (Titles) Total vacancy
matches (min)
Minimum
matchscore
match score
expert
(minimum)
total matching
vacancies
considering
match score
expert
AK Verkoop en handel Controller bij Unique Uitzendbureau in Maassluis; Intercedent (ervaren) 10 0,3 0,3 4458 11
Informatie- en communicatietechnologie Applicatiebeheerder sociaal domein; 10 0,3 0,3 4888 11
Administratie en klantenservice Planners Thuiszorg 10 0,3 0,3 2733 9
Beleid en bestuur Beleidsmedewerker sociaal domein; Projectleider WMO Sociale Teams 10 0,3 0,3 1628 20
Communicatie, marketing en PR Communicatie Adviseur 10 0,3 0,3 1104 8
Recht, arbeid en maatschappij Vrijwilligerscoördinator; HR Adviseur bij KMO Team Focus 10 0,3 0,3 1304 8
Natuur- en milieuwetenschappen Gis specialist 10 0,3 0,3 151 2
WIS Productie (Meewerkend) Werkplaats Chef - Almere;Lasser Mig/Mag;Assistent kredieten 100 0,4 0,1 554 32
Engineering Werkvoorbereider Gww; Engineer HVAC Utiliteit; Calculator/Engineer
Installatietechniek
100 0,4 0,1 513 16
Verkoop en handel Verkeerscoördinator a.i.; Callcenter Agents Outbound; Sales engineer;
Controller
100 0,4 0,1 554 14
Informatie- en communicatietechnologie Robot Software Engineer; IT Consultants; Informatiemanager 100 0,4 0,1 751 12
Bouw en delfstoffenwinning Graduation Thesis Mechanical Engineering; Werkvoorbereider 100 0,4 0,1 551 12
Installatie, reparatie en onderhoud Allround monteur technische dienst; scheepsschilders 100 0,4 0,1 639 15
Overig Inkoop Traineeship (intern) 100 0,4 0,1 434 10
NL Engineering 100 0,4 0,2 1272 60
Verkoop en handel junior sales manager; Kassamedewerker; Callcenter medewerk(st)er
(Nederlands of Vlaamstalig); Commercieel Manager; Assistent store manager
100 0,4 0,2 2033 100
Informatie- en communicatietechnologie project bekijken Storage engineer (netwerk en systemen); Implementatie
Specialist
100 0,4 0,2 1487 59
Administratie en klantenservice Managementassistente (nr. 4107); Senior Receptionist(e)/Gastvrouw/-heer;
Programma Coördinator; Customer Service Medewerker
100 0,4 0,2 2332 133
Overig Junior Consultant/Trainee; Allround Binnendienst Medewerker; TRAINEESHIP
(MULTINATIONAL, TREASURY, HBO / WO)
100 0,4 0,2 842 76
Gezondheidszorg en welzijn Groepshulp 0,4 0,2 225 12
Politie, brandweer en beveiliging medewerker beveiliging 0,4 0,2 41 4

24. Other Applications
• Job Test Validation

25. Job Information Dashboard for
the ICT Job Profession Group
https://youtu.be/4TJ-Z-_Uyi8

26. Key Worker attributes

27. Challenges
• Labeling data is time consuming
• Choose which data to label
• Make use of unlabeled data
• Crowd source the labeling

28. • Job vacancies as source of job information
• Apply techniques from text mining and machine learning
to perform the job information extraction
• Contribution to Job Analysis, Job Test Validation, and
Career planning
• Benefits job-seekers and recruiters.
Summary

29. Key Publications
2017
Kobayashi, V. B., Berkers, H. A., Mol, S. T., Kismihok, G., & Den Hartog, D. N. (2017). Text
Mining in Organizational Research. Organizational Research Methods. Manuscript in
Preparation.
Kobayashi, V. B., Mol, S. T., Berkers, H. A., Kismihok, G., & Den Hartog, D. N. (in press). Text
Classification for Organizational Research: A Tutorial. Organizational Research Methods.
2016
Kobayashi, V., Mol, S. T., Kismihok, G., & Hesterberg M. (2017). Automatic Extraction of
Nursing Tasks from Online Job Vacancies In M. Fathi, M. Khobreh, & F. Ansari (Eds),
Professional Education and Training through Knowledge, Technology and Innovation (pp. 51-
56). Siegen, Germany: Universitatsverlag Siegen.

30. This work was supported by the
European Commission through the
Marie-Curie Initial Training
Network EDUWORKS (grant
number PITN-GA-2013-608311)