Slides presented at the ACM International Conference on Communities and Technologies (C&T '15), Limerick, Ireland, June 27–30, 2015 Expertise identification is important for various kinds of online and offline organizations, with practical applications such as supporting question answering, problem-solving, and team formation. Using developers as the target population, we demonstrate that it is possible to identify novices and experts of programming by examining the types of programming related websites they visit.

1. Discount Expertise Metrics for
Augmenting Community
Interaction
Pei-Yao Hung1
, Mark S. Ackerman1, 2
School of Information1
and Dept. of EECS 2
, University of Michigan, USA
1

2. 2

3. 3

4. 4
He has done almost nothing…

5. Estimating Expertise
• Profile: curation => needs maintenance
!
(Farrell, Lau, Nusser, Wilcox, & Muller, 2007)
• Artifact: production => needs contribution
!
(Nam & Ackerman, 2007)
• Interaction: participation => needs contribution
!
(Hanrahan, Convertino, & Nelson, 2012; Zhang & Ackerman, 2005; Zhang,
Ackerman, Adamic, & Nam, 2007)
5

6. Estimating Expertise
• Profile: curation => needs maintenance
!
(Farrell, Lau, Nusser, Wilcox, & Muller, 2007)
• Artifact: production => needs contribution
!
(Nam & Ackerman, 2007)
• Interaction: participation => needs contribution
!
(Hanrahan, Convertino, & Nelson, 2012; Zhang & Ackerman, 2005; Zhang,
Ackerman, Adamic, & Nam, 2007)
6
You need to do/contribute a lot
of work before we can estimate
your expertise!

7. !
A lot of people consume, but do not contribute.
7

8. Q: Can we use the browsing history to
estimate levels of technical expertise?
8
docs.python.org
tutorialspoint.com
docs.ggplot2.org
cyclismo.org/tutorial/R
...
github.com
stackoverflow.com
pypi.python.org
ruby-doc.org
...

9. How do we analyze browsing history?
• Intuition: programmers at different levels might visit
different type of web pages.
• Library/Repository: https://github.com/
• Tutorial: http://www.tutorialspoint.com/python/
• Q&A: http://stackoverflow.com/
• …
9

10. Q: Can we use the browsing history to
estimate levels of technical expertise?
10
Document
Tutorial
Document
Tutorial
...
Library/Repository
Q&A
Library/Repository
Document
...

11. Recruiting Participants
• Recruiting: presentations, email lists, and snowball
referrals
• 26 participants who are ‘actively’ programming
• 11 male and 15 female.
• 24 students (undergrad ~ Ph.D.)
• Diverse majors (e.g., Russian, economics, to
computer science)
11

12. 12

13. Rating Scheme for Expertise
!
!
!
!
!
!
Level Experience
+1 Learning programming for the first time/year
+2
Electrical Engineering (EE) training, 1 year professional
programming experience, or 3 - 4 years assistant/part-
time programming experience
+3
Computer Science (CS) training, or 2 - 3 years
professional programming experience
+4 4+ year professional programming experience
+5 6+ year professional programming experience
13

14. Analysis
Programming
Relevant Web
Page Visits
(distribution)
Level of
Expertise
Level 1 ~ 5Tutorial, Library/Repository,
Q&A, Document, Blog,
Forum, Search, Other
Logistic
Regression
InterviewBrowsing History
14

15. If everything goes as expected…
15

16. 16
Tutorial

17. 17
Library/Repository

18. Logistic Regression (N=26): the relationship between
page visits and expertise isn’t that straight forward.
18

19. Conservative Classifiers using
Heuristics
• Beginner (Lv 1 or 2) <- over 80% of programming relevant
visits on “Tutorial”
• Expert (Lv 4 or 5) <- over 80% of programming relevant
visits on “Library/Repository and Q&A”
!
!
!
19

20. 20
Well, but he is probably an
expert pythonist…

21. How can this discount expertise measure
augment community interaction?
• “Inclusive”
• Provides initial expertise estimation to smooth the
process of “blending in” a new community.
• Tracks expertise development in a learning
community (e.g, MOOC).
• Allows ad-hoc network formation.
21

22. Future Work
• Distinguishes expertise development for different
programming languages.
• Monitors changes of expertise through a
longitudinal study (e.g., 6-12 months).
!
!
22

23. Takeaways
• Browsing history could be a source for a discount
expertise metric.
• Discount expertise metric has the potential to
argument community interaction.
!
!
23
Contact: Pei-Yao Hung, peiyaoh@umich.edu