The document discusses using different types of user interaction signals (clicks, bookmarks, replies) to improve job recommendations. It finds that replies most accurately reflect user preferences for job recommendations, followed by bookmarks, while clicks may not always be reliable. The conclusion recommends implementing a recommendation strategy that learns from user replies through feedback to better match users and jobs.

1. Using Interaction Signals for Job Recommendation
Benjamin Kille, Fabian Abel, Balázs Hidasi, Sahin Albayrak| SIREMTI | 13 November 2015

2. Agenda
– Looking for a Job: now and then
– Data Description
– User Inquiry
– Findings
– Conclusion and Outlook
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 2/26

3. Problem Description
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 3/26
http://peacechild.org/wp-content/uploads/2015/09/Youth-unemployment.jpg

4. Traditional Method to Look for a Job
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 4/26
http://www.businessreviewaustralia.com/public/uploads/large/large_article_im640_newspaper_ad_2.jpg

5. Tends in Job Seeking
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 5/26
Use of print media decreases
Professionals predominantly use
– Online job offer collectors
– Online business networks
Trend leads to
– Higher volume of job offers to
process for professionals
– Higher volume of candidates to
deal with for employers
– Reciprocal selection problem
Weitzelet al. (2015). Bewerbungspraxis 2015 – Eine empirische Studie mit 7000 Stellensuchenden und Karriereinteressierten im Internet.
http://3.bp.blogspot.com/-Tm8S-4mJcdQ/Tx2Y-ND1flI/AAAAAAAACcM/l6fPkkR9RMI/s1600/information_overload_hydrant.jpg

6. Reciprocal Selection
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 6/26
Professional:
– reduce job offers to manageable size
– remove irrelevant job offers
– keep relevant job offers
Recruiter:
– reduce list of candidates to
manageable pool
– keep candidates with required skills
– keep candidates likely to respond
– remove candidates lacking necessary
skills
Ideally: match needs of both parties

7. How do we select job offers/candidates?
1. learn a modell representing professionals’ requirements
– curriculum vitae/skills
– location
– preferences
2. apply modell to available job offers
3. present suggestions to professionals
4. observe how professionals react
5. adjust modell to improve suggestions (repeat)
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 7/26

8. Types of Feedback
We track users ...
... clicking on
... bookmarking
... replying to
suggested job offers
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 8/26

9. Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 9/26

10. Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 10/26

11. Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 11/26

12. Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 12/26

13. Which Type of Feedback should we use?
What can a click tell us?
What can a bookmark tell us?
What can replies tell us?
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 13/26

14. A User Inquiry
– How satisfied are users with their job recommendations?
– Collect ratings for job recommendations
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 14/26

15. Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 15/26

16. Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 16/26

17. Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 17/26

18. Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 18/26

19. What Type of Feedback Tells us Most?
Idea: Check which kind of Feedback correlates best with ratings:
– ratings ~ clicks
– ratings ~ bookmarks
– ratings ~ replies
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 19/26

20. Relation: ratings ~ clicks
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 20/26
clicks
ratings
1 2 3 4 5 6 7 8 9 10
12345

21. Relation: ratings ~ bookmarks
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 21/26
Ratings for Bookmarked Jobs (μ = 3.6)
rating
Density
0 1 2 3 4 5
0.00.10.20.30.40.5

22. Relation: ratings ~ replies
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 22/26
replies
ratings
1 2 3 4 5 6 7 8
12345

23. Signal Comparison
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 23/26
clicks
ratings
1 2 3 4 5 6 7 8 9 10
12345
Ratings for Bookmarked Jobs (μ = 3.6)
rating
Density
0 1 2 3 4 5
0.00.10.20.30.40.5
replies
ratings
1 2 3 4 5 6
12345

24. Conclusion and Outlook
feedback is necessary to improve recommendations
analysis of three signals:
– clicks à few clicks might be misleading
– bookmarks à filter bad suggestions; concentrate on medium preferences
– replies à most accurately reflect preferences
next steps
– implement a recommendation strategy that learns with replies
– A/B testing to verify suitability
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 24/26

25. Questions?
Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 25/26

26. Using Interaction Signals for Job Recommendation | Benjamin Kille | SIREMTI 2015
slide 26/26
Benjamin Kille (TU Berlin)
Competence Center Information Retrieval & Machine Learning
Institute of Commercial Information Technology and Quantitative Methods
benjamin.kille@tu-berlin.de
@bennykille
http://crowdrec.eu
http://xing.com