This document summarizes a study of mining email social networks in open source software projects. It finds that developers who are most active in communication and coordination on mailing lists, as measured by social network metrics like in-degree and betweenness centrality, tend to be the most active contributors to the source code as well. Communication and coordination activity is strongly correlated with software development work. The study analyzed aliases, clustered identities, and relationships between developers' email and code contribution activities.

1. Mining Email Social
Networks
Christian Bird, Alex Gourley,
Prem Devanbu, Michael Gertz, Anand Swaminathan
University of California, Davis
Presented By:
Arnamoy Bhattacharyya

2. Communication & Co-ordination (C&C) activities are central to large software
projects

3. Communication & Co-ordination (C&C) activities are central to large software
projects
Difficult to observe and study in traditional (closed-source, commercial)
settings

4. Communication & Co-ordination (C&C) activities are central to large software
projects
Difficult to observe and study in traditional (closed-source, commercial)
settings
the email archives of OSS projects provide a useful trace of the
communication and co-ordination activities of the participants

5. CHATTERERS & CHANGERS
A mailing list in an OSS project is a public forum

6. CHATTERERS & CHANGERS
A mailing list in an OSS project is a public forum
Anyone can post messages to the list.

7. CHATTERERS & CHANGERS
A mailing list in an OSS project is a public forum
Anyone can post messages to the list.
Posted messages are visible to all the mailing list
subscribers.

8. CHATTERERS & CHANGERS
A mailing list in an OSS project is a public forum
Anyone can post messages to the list.
Posted messages are visible to all the mailing list
subscribers.
Posters include developers, bug-reporters, contributors (who submit
patches, but don't have commit privileges) and ordinary
users.

9. A response b to a message a is an indication That –
the sender of b; (Sb) found that the sender of a; (Sa) had something
interesting to say

10. A response b to a message a is an indication That –
the sender of b; (Sb) found that the sender of a; (Sa) had something
interesting to say
It is also an indication of Sa’s status, i.e., Sb indicates that s/he
found Sa's email worth reading, and worthy of response.

11. A response b to a message a is an indication That –
the sender of b; (Sb) found that the sender of a; (Sa) had something
interesting to say
It is also an indication of Sa’s status, i.e., Sb indicates that s/he
found Sa's email worth reading, and worthy of response.
However, the vast majority of individuals participating on the email
list sent very few messages, and received very few replies to their
messages

12. OF DOGS AND DEVELOPERS
“On the Internet, no one knows if you're a Dog“ - Peter Steiner

13. OF DOGS AND DEVELOPERS
“On the Internet, no one knows if you're a Dog"
The same individual
can use different email aliases

14. OF DOGS AND DEVELOPERS
“On the Internet, no one knows if you're a Dog"
The same individual
can use different email aliases
developer Ian Holsman uses 7 different email
aliases

15. OF DOGS AND DEVELOPERS
“On the Internet, no one knows if you're a Dog"
The same individual
can use different email aliases
developer Ian Holsman uses 7 different email
aliases
Ignoring these aliases would confound later
steps of data analysis

16. Unmasking Aliases
Most emails include a header that identifies the sender, of this form:
From: "Bill Stoddard" <reddrum@attglobal.net>

17. Unmasking Aliases
Most emails include a header that identifies the sender, of this form:
From: "Bill Stoddard" <reddrum@attglobal.net>
Crawl messages and extract all
headers to produce a list of
<Name,email> identifiers (IDs)
Execute a clustering algorithm
that measure the similarity
between every pair of IDs
Manually Post Process the
clusters formed to remove
further aliases

18. Unmasking Aliases
Most emails include a header that identifies the sender, of this form:
From: "Bill Stoddard" <reddrum@attglobal.net>
Crawl messages and extract all
headers to produce a list of
<Name,email> identifiers (IDs)
Execute a clustering algorithm
that measure the similarity
between every pair of IDs
Manually Post Process the
clusters formed to remove
further aliases
set the cluster similarity threshold quite low:
easier to split big clusters than to unify two disparate clusters from a very
large set.

19. THE CLUSTERING ALGORITHM
1. Normalize name
remove all punctuation, suffixes
(“jr")
turn all whitespace into a single space
Remove generic terms like “admin", “support", from the name
split the name into first name and last name (using whitespace
and commas as cues)

20. THE CLUSTERING ALGORITHM
2. Name Similarity:
Use a scoring algorithm between –
The full names
The first name and last name separately
Consider names similar if the full names are similar, or
if both first and last names are similar
e.G Andy Smith <-> Andrew Smith
Deepa Patel !<-> Deepa Ratnaswamy

21. THE CLUSTERING ALGORITHM
3. Names-email Similarity:
If the email contains both first and last names – match
Arnamoy Bhattacharyya <-> ar.bhat@yahoo.com
if the email contains the initial of one part of the name and entirety
of the other part – match
Erin Bird <-> ebird
Erin Bird <-> erinb

22. THE CLUSTERING ALGORITHM
4. Email Similarity:
If the Levenshtein edit distance between two email address bases (not
including the domain, after the "@") is small – Match

23. THE CLUSTERING ALGORITHM
5. Cumulative ID similarity:
The similarity between two IDs is the maximum of the all mentioned
above
E.G
Name Similarity – 3
Names-email similarity – 5
Email Similarity – 2
If the threshold is 4, it would be considered as a match

25. vast majority of people send only one message, and
there are some who send a great many

27. Out-degree - # of different people from whom an individual has
received responses
Higher out-degree <-> higher status

28. In-degree - # of different people to whom an individual has
replied-to
Indicates the level of engagement of an
individual in the mailing list and the breadth of
his/her interests

29. In-degree - # of different people to whom an individual has
replied-to
Indicates the level of engagement of an
individual in the mailing list and the breadth of
his/her interests
The distributions show a small-world character

30. High correlation between messages sent and replies got(out order) -
0.97

31. Correlation may not be true-
1. People who only post relevant messages get large responds to
messages
2. Only people who receive replies from several people keep sending
messages (Survival Effect)

32. Each link indicates at
least 150 messages
sent

33. C&C ACTIVITY AND DEVELOPMENT
ACTIVITY
How does email activity relate to software development activity?
73 committers-
1. A correlation of 0.80 between the number of messages sent by an
individual, and number of source changes they make –
more software development work <-> more C&C activity

34. C&C ACTIVITY AND DEVELOPMENT
ACTIVITY
How does email activity relate to software development activity?
73 committers-
1. A correlation of 0.80 between the number of messages sent by an
individual, and number of source changes they make –
more software development work <-> more C&C activity
2. A correlation of 0.57 between the number
of messages sent by an individual, and number
of document changes they make
source code activities require much more co-
ordination effort
than documentation effort

35. Are developers more likely to play the role of gatekeepers or brokers in the
complete email social network?

36. Are developers more likely to play the role of gatekeepers or brokers in the
complete email social network?
Betweenness (BW)---

37. Are developers more likely to play the role of gatekeepers or brokers in the
complete email social network?
Betweenness (BW)---
High betweenness <-> that the person is a kind of broker, or gatekeeper

38. mean

39. mean
Developers are higher in status than non-developers

40. Relative Status of Developers
Do the most active developers have the highest status among developers ?

41. Relative Status of Developers
Do the most active developers have the highest status among developers ?
Source changes are not as highly correlated with document changes <-> not all
developers are engaged in both to the same degree

42. Relative Status of Developers
Do the most active developers have the highest status among developers ?
Source changes are not as highly correlated with document changes <-> not all
developers are engaged in both to the same degree
Source changes shows the strongest rank correlation with the social network
status <-> the most active developers play the strongest role of
communicators, brokers, and gatekeepers

43. Conclusion
The level of activity on the mailing list is strongly correlated with source code
change activity, and to a lesser extent with document change activity.

44. Conclusion
The level of activity on the mailing list is strongly correlated with source code
change activity, and to a lesser extent with document change activity.
Social network measures such as in-degree, out-degree and betweenness
indicate that developers who actually commit changes, play much more
significant roles in the email community than non-developers.

45. Conclusion
The level of activity on the mailing list is strongly correlated with source code
change activity, and to a lesser extent with document change activity.
Social network measures such as in-degree, out-degree and betweenness
indicate that developers who actually commit changes, play much more
significant roles in the email community than non-developers.
Even within the select group of developers, there is a strong correlation
between the social network importance and level of source code change activity.

46. Questions?