A Preliminary Analysis on the Effects of Propensity to Trust in Distributed Software Development
1. A Preliminary Analysis on the
Effects of Propensity to Trust in
Distributed Software Development
Fabio Calefato, Filippo Lanubile, Nicole Novielli
University of Bari
Italy
ICGSE’17
May 22-23, 2017 – Buenos Aires, Argentina
2. A Bit of Theory on Trust
INTEGRITY
The adherence to intrinsic moral
norms which makes a trustee
reliable
BENEVOLENCE
The perceived level of
courtesy and positive
attitude
ABILITY
Capability of a trustee
(based on knowledge,
competence, skills) to
perform tasks within a
specific domain
PREDICTABILITY
The degree to which a person is
liable and accountable and meets
the expectation of another
person
Trustee’s
antecedents to trust
Trustor’s
antecedent to trust
PROPENSITY TO TRUST
A general, not experience-based inclination to
display faith and adopt a trusting attitude
toward others
4. Why We Care About Trust
• Trust fundamental to build “sense of teamness”
• Lack of trust can mean:
– Unwillingness to cooperate or share information
– Perception of being on separate teams
– Decreased goodwill towards others
• The detrimental effects of lack of trust indirectly
affect project performance (costs, speed, …)
– Especially in case of distributed collaboration (no
F2F interaction)
ICGSE'17
5. Success Factors in
Collaborative Development
• ~13% of reviewed PRs in GitHub rejected for
purely technical reasons
Pull Request
acceptance
Contributor’s Experience &
Social Connections
• track record
• connection strength
• @mentions
• # followers
PRs technical
characteristics
• test cases
• commit size
• review comments
+
Gousios et al., ICSE’14, ICSE’15
Tsay et al., ICSE’14, FSE’14
Vasilescu et al., MSR’15
ICGSE'17
6. Research Question
Trust ≈ Project performance
• How does individual propensity to trust facilitate the
successful collaborations in globally-distributed software
projects?
Successful collaboration ≈ Accepted Pull Request (PR)
ICGSE'17
8. Study
6 integrators
(20+ emails)
218 Pull Requests
(Accepted / Not accepted)
~5K emails
GH Torrent
mlstats
Tone Analyzer
Propensity to Trust
Scores (Low / High)
Agreeableness
ICGSE'17
9. • Logistic regression model
– Outcome: PR acceptance
– Predictor: Integrator’s propensity to trust
Findings
+34%Integrator with
High propensity
to trust
ICGSE'17
Integrator with
Low propensity
to trust
Control vars:
# emails:
# PRs reviewed
10. Limitations & Future work
• Limited generalizability
– 1 project
– 6 integrators
– 218 pull requests
• Construct validity
– Relied on
agreeableness as a
proxy of propensity to
trust
– Relied on IBM Watson
Tone Analyzer tout
court
• Study replications on
more projects with
larger datasets
• Exploit other
personality theories
(e.g., MMPI)
• Replication with trust-
specific linguistic
resource (e.g., NRC
emotion lexicon)
ICGSE'17
11. Big 5 model & Trust
ICGSE'17
Limitations & Future work
• Limited generalizability
– 1 project
– 6 integrators
– 218 pull requests
• Construct validity
– Relied on
agreeableness as a
proxy of propensity to
trust
– Relied on IBM Watson
Tone Analyzer tout
court
• Study replications on
more projects with
larger datasets
• Exploit other
personality theories
(e.g., MPPI)
• Develop a trust-specific
linguistic resource from
SE corpora
ICGSE'17
Study
6 integrators
(20+ emails)
218 Pull Requests
(Accepted / Not accepted)
~5K emails
GH Torrent
mlstats
Tone Analyzer
Propensity to Trust
Scores (Low / High)
agreeableness
• Logistic regression model
– Outcome: PR acceptance
– Predictor: Integrator’s propensity to trust
Findings
11
+34%
Integrator with
Low propensity
to trust
Integrator with
High propensity
to trust
Thanks!
@fcalefato
Editor's Notes
Trust is a complex matter to study since it involves both interpersonal relationships (e.g., cultural issues between trustee and trustor) and facets of human behavior (e.g., personal traits). To date several definitions of trust have been given. A widely used and concise definition is provided by Jarvenpaa et al.
“The expectations of one (the trustor) that the others (the trustees) will behave as predicted” [1]
In other words, positive trust emerges when others’ actions meet our expectation; otherwise, negative trust, or mistrust, arises.
When we assess someone’s trustworthiness, we do it along judging others along several dimensions or personal characteristics called antecedents of trust, that is, the properties of others that trigger the evaluation when one assessing their trustworthiness.
Therefore, sharing these properties/antecedents about someone fosters trust building, although this is also influenced by one’s intrinsic propensity to trust, a natural disposition towards trusting others in general
propensity to trust (i.e., (a dispositional willingness to rely on others) Trust propensity is a general disposition to trust people in life (this is a consumer trait that is relatively stable. People differ in this trait; some are always distrusting others, whereas others believe that people can be trusted. Disposition to trust is a general, i.e. not situation specific, inclination to display faith in humanity and to adopt a trusting stance toward others (Mayer et al. 1995; McKnight et al. 1998), and is perceived as a personality trait (Grabner-Kräuter et al., 2003). This tendency is not based upon experience with or knowledge of a specific trusted party (ref), but is the result of an ongoing lifelong experience and socialization. As an antecedent of trust, disposition to trust is most effective in initial phases of a relationship when the parties are still mostly unfamiliar with each other (Mayer et al. 1995) and before extensive ongoing relationships provide a necessary background for the formation of other trust-building beliefs, such as integrity, benevolence and ability.
ability (e.g., skills, knowledge), benevolence (e.g., courtesy, availability), integrity relates to a set of moral norms and trustee’s characteristics usually considered as good as, for example, honesty, fairness, loyalty and discretion, and predictability (e.g., reliability, consistent behaviors) are the personal characteristics of a trustee that facilitate the establishment of the trust relationship with a trustor. More specifically, the ability and predictability dimensions are assessed by means of cognitive elaboration of personal and professional information. At the same time, affective-based appraisal leads to trust building along the dimensions of benevolence and integrity is adherence to a set of
principles (such as study/work habits) thought to make the trustee dependable and reliable, according to the trustor
Previous research on psycholinguistic has confirmed strong connections between language use and personality and that the personality traits can be successfully derived from the analysis of written text [13] such as emails [14].
In fact, Tausczik & Pennebaker [15] found that every trait in the Big-Five model is strongly and significantly associated with theoretically-appropriate patterns of word use, indicating.
Why is trust so important in SE and to SE researchers?
Connection strength = • prior interactions with core devs
One common limitation identified in prior empirical-research findings is that there is no explicit measure of ‘how much’ improving trust contributes to a project performance.
Difficult to measure trust, it is sensed self-reported data (read questionnaires)
We seek for a more operative, quantitative measure, approximate trust by means of exchange of affective lexicon between a pair
Performance is intuitively intended as a software project that delivers a quality product staying on budget and time schedule
We approximate it as successful requests accepted PRs that cause the code base to grow, adding features, fixing bugs, one step closer to the project final goal
You can look at the paper to check the formula with the coefficient estimates.
For a given PR k, there’s a +34% increase in the chance of being merged when the propensity to trust score of the integrator shifts from Low to High.
Big5 model: trust is just one facet of agreeableness
Is it any reliable, we have seen with Sentiment Analysis tool how their lexicon is domain dependent.
Minnesota Multiphasic Personality Inventory
NRC lexicon