Towards Open LMI Data: Principles, Users and Context

LABOUR MARKET INFORMATION COUNCIL
CONSEIL DE L’INFORMATION SUR LE MARCHÉ DU TRAVAIL
Big Data & Analytics for the Public Sector
2 October 2019
Tony Bonen (tony.bonen@lmic-cimt.ca)
Director, Research, Data and Analytics
Towards Open LMI Data
Principles, Users and Context

1 Introduction to LMIC
2 Principles for Establishing Open LMI
3 Focus on Use Cases
4 Contextualizing Skills Data
5 Conclusion

Who We Are
National
Stakeholder
Advisory
Panel (NSAP)
Labour Market
Information
Experts Panel
Board
of Directors
(13 PTs, ESDC,
and Statistics
Canada)
NSAP Chair
(David Ticoll)

Strategic Goals
COLLECT ANALYZE DISTRIBUTE
Gather and improve
the availability of
relevant LMI
Undertake insightful and
high-quality analyses of
LMI
Provide Canadians with
timely, relevant and
reliable LMI

Our Values
•Client Centric and Demand Driven
•Inclusive and Collaborative
•Integrity and Transparency
•Innovative and Evolutionary

LMIC Data
Hub
Refine LMI
needs
Map
delivery
system
Re-
Structure
data
Guidelines
+
Metadata
Take stock
of existing
LMI
Understan
d LMI
needs
Open LMI is a Process
Phase I Phase II Phase III Phase IV

Principles: Users and Information
User (demand)
Relevant Address specific questions of individuals
Contextualized Data and insights placed in broader
context
Findable Easy to obtain through standard means
(e.g., googling, navigable website)
Accessible Different channels to access
Understandable Described in plain language with clearly
articulated connections between data
points
Information (supply)
Reliable High level of accuracy and
representativeness
Comprehensive Available for largest set of areas,
populations and indicators possible
Validated Rigorous processing system
Comparable Consistently applied descriptors

Principles: Underlying Data Standards
Data characteristics
Open Ease with which data can be
accessed in machine-readable format
Localness Smallest geographic area
Granular Number and specificity of grouping
variables (e.g., demographics)
Frequent Rate at which data are updated
Timely Delay between data reference period
and when it becomes available
Metadata characteristics
Open Ease with which metadata can be
accessed in machine-readable format
Consistent Similarity of underlying methods for
producing data across sources and
through time
Annotated Detailed information, caveats and
commentary of data (“meta-metadata”)

LMIC Hub: Separating data from use-case
Statistics
Canada
F/P/T (admin
data,
occupational
outlook, etc.)
Other, e.g.
private sources
LMIC
API

Statistics
Canada
F/P/T (admin
data,
occupational
outlook, etc.)
Other, e.g.
private sources
New LMI
LMIC
API

Statistics
Canada
F/P/T (admin
data,
occupational
outlook, etc.)
Other, e.g.
private sources
New LMI
LMIC
API
Restructure
data
Partnerships
to generate
new LMI

Job
outlooks
Statistics
Canada
F/P/T (admin
data,
occupational
outlook, etc.)
Other, e.g.
private sources
New LMI
Other
data
Salaries
by field
of study
Skills in
demand
LMIC
API
Restructure
data
Partnerships
to generate
new LMI

Flows to
Intermediaries
Job
outlooks
Statistics
Canada
F/P/T (admin
data,
occupational
outlook, etc.)
Other, e.g.
private sources
New LMI
Other
data
Salaries
by field
of study
Skills in
demand
LMIC
Intermediary:
Education/
Career choice
API
Intermediary:
Investment
decision
Restructure
data
Partnerships
to generate
new LMI

Flows to
Intermediaries
Job
outlooks
Statistics
Canada
F/P/T (admin
data,
occupational
outlook, etc.)
Other, e.g.
private sources
New LMI
Other
data
Salaries
by field
of study
Skills in
demand
LMIC
Intermediary:
Education/
Career choice
API
Intermediary:
Investment
decision
Restructure
data
Partnerships
to generate
new LMI
Other LMI
Sources

Flows to
Intermediaries
Job
outlooks
Statistics
Canada
F/P/T (admin
data,
occupational
outlook, etc.)
Other, e.g.
private sources
New LMI
Other
data
Salaries
by field
of study
Skills in
demand
LMIC
Intermediary:
Education/
Career choice
API
Intermediary:
Investment
decision
Restructure
data
Partnerships
to generate
new LMI
Other LMI
Sources
Other
non-LMI
Sources

Flows to
Intermediaries End Users
Job
outlooks
Current College/
University students
Industry/sector
Statistics
Canada
F/P/T (admin
data,
occupational
outlook, etc.)
Other, e.g.
private sources
New LMI
Other
data
Salaries
by field
of study
Skills in
demand
LMIC
Intermediary:
Education/
Career choice
API
Intermediary:
Investment
decision
Restructure
data
Partnerships
to generate
new LMI
Other LMI
Sources
Other
non-LMI
Sources

Develop Based on Use Cases
• Who will use the data?
• What decisions are they trying to make?
• What is their current level of understanding?
• What does the existing ecosystem look like, and how can it
be leveraged?

Use Cases Help Identify Data Needs
What?
Career
decisions
How LMI is
consumed
How career
decisions are
made
❶ Why?❷
Data Needs
• Type (e.g., wages)
• Structure (e.g., take-home
pay vs. annual gross
salary or hourly wages)
Best practices
• Distributing LMI (e.g.,
what is best form of
dissemination, frequency,
etc.)
How?❸
Qualitative
research
Literature
review
International
experiences
Test 1 use case
Repeat &
expand

Data Architecture Will Follow
Important questions to tackle around architecture:
• Costs and functionality trade off
• Data Warehouse vs Data Lake
• Scalability
• Geographic location
Put Ecosystem, Use Cases and Target Groups first

Establishing a Project Team
1. Composition:
i. Project design/management
ii. Representatives from both pilot use-cases
iii. Technology experts
2. Role:
i. Oversee design architecture
ii. Provide technical guidance/support
iii. End-user perspective

Bridging Skills & Occupations
COLLECT ANALYZE DISTRIBUTE
Skills data gap identified
• Education level/type
used as proxy
Linking skills to occupations
• Learning from others
(O*NET, ESCO)
• Exploring new techniques
with big data
Will publish data and analyses
• LFS data linked to skills and
downloadable
• Report methodological details
and ongoing updates

A Phased Approach
1 2 3
4
Consult & improve the
Taxonomy
Identify and evaluate
mapping approaches
Pilot tests
Assess and validate
tests
Disseminate, administer, and
implement
5

ESDC’s Skills and Competencies Taxonomy
7 Foundational skills
9 Analytical
9 Technical
13 Resource management
9 Interpersonal
Total: 47
skills
500 National
Occupational
Classifications
(NOC)

Mapping Guided by 7 Criteria
Criteria Description
Flexible
Responds to changing labour market conditions and captures
emerging skills.
Sustainable and cost
effective
Adequate resources to maintain and update the mapping
Representative Reflects the different ways people express skill requirements
Granular Greater specificity of skills and occupation-specific data
Responsive
Enables better informed decisions about skills training and
education
Measurable Allows for reasonable measurement of skills
Statistically sound Estimated skill levels representative of labour markets

Mapping Approaches Being Explored
Potential
Approaches
Exampl
es
Advantages Drawbacks
Consult occupational
experts
O*NET • High quality linkages to well-
defined skills taxonomy
• Standardized review process
ensures consistency
• Slow adaptation to emerging
skills
• Unnatural skills language
Survey workers
directly
O*NET • Obtain “front line” knowledge
• Linkages to skills taxonomy of
choice
• Requires expert validation
• Risk of misunderstanding
• Closed vs open-ended questions
Leverage web-scraped
data
Nesta,
LinkedIn
• Draws on large pool of data
• Natural language in job postings
• Responsive to emerging skills
• Inexpensive to maintain
• Requires vetting / validation
• Skewed market segment
• Inconsistency of skills language
• Omission of implied skills
Hybrid of the above • Balance natural vs consistent
skills language
• Expensive to maintain

Nature of Skill-Occupation Linkage
Importance and level ratings (O*NET)
O*NET: 1 = not important
2 = somewhat important
3 = important
4 = Very important
5 = Extremely important
Binary classification (ESCO)
ESCO: “essential” or “non-essential”
Alternatives?

Approach 1: Job analysts
Skill Importance Level
1. Critical thinking 78 64
2. Mathematics 78 61
3. Reading comprehension 78 68
4. Active listening 75 57
5. Judgement and decision
making
75 57
6. Speaking 75 61
7. Writing 75 61
8. Active learning 72 57
9. Complex problem solving 72 59
Skill Importance Level
10. Instructing 63 45
11. Systems analysis 60 55
12. Systems evaluation 56 57
13. Learning strategies 53 50
14. Monitoring 53 52
15. Coordination 50 45
16. Persuasion 50 52
17. Service orientation 50 41
18. Time management 50 43
Example: O*NET and US SOC codes: 19-3011 (”Economists”)

Approach 1: Considerations
• Complexity: Leveraging O*NET taxonomy of skills
requires translation into local occupational categories
• Limited: O*NET taxonomy is fixed (35 unique skills)
• Slow responsiveness: 100 occupations updated per year

Approach 2: Web-Scraping
Item Type Incidence
1. Communication skills skill 53%
2. Teamwork skill 47%
3. English language Work requirement 38%
4. Forecasting Work requirement 34%
5. Data Analysis Work requirement 22%
6. Decision making Skill 19%
7. EViews Work requirement 9%
8. Writing Skill 6%
9. MATLAB Work requirement 3%
Example: Vicinity Jobs: NOC code 4162 (Economists, etc.)

Approach 2: Considerations
• Measure: Incidence in job postings does not equal level
of importance or frequency of requirements
• Complexity: Translating to rigorous skills taxonomies
challenging

Challenges to Web-Scraped Skills Mapping
• Linking natural language on skills to formal taxonomy
• Distinguishing between “skills” and “work requirements”
• Capturing implicit skills
• Lack of equally comprehensive supply-side data

Conclusion
•Open access that meets the needs of multiple
users / use cases
•Always design with use cases in mind
•Leverage existing information distributions
systems as much as possible
•Share your data!

Conclusion
Source: http://xkcd.com/

Questions?
For additional information visit
our website lmic-cimt.ca

Towards Open LMI Data: Principles, Users and Context

Recommended

Recommended

More Related Content

Similar to Towards Open LMI Data: Principles, Users and Context

Similar to Towards Open LMI Data: Principles, Users and Context (20)

More from Labour Market Information Council | Conseil de l’information sur le marché du travail

More from Labour Market Information Council | Conseil de l’information sur le marché du travail (20)

Recently uploaded

Recently uploaded (20)

Towards Open LMI Data: Principles, Users and Context