Ahmed Shah Mashiyat, Michalis Famelis,
Rick Salay, Marsha Chechik
University of Toronto
RSSE@ICSE 2014
Using Developer Conversations
to Resolve Uncertainty in
Software Development:
A Position Paper

Software Development is Social
2
Conversation between developers on
jazz.net about an enhancement ticket

Software Development is Social
3
Conversation between developers on
jazz.net about an enhancement ticket
Ideal conversation: Question, Debate, Solution

The Manager’s Perspective
Goals:
• Product quality
• Speedy delivery
So, conversations should:
• cover all issues
• include all stakeholders
• not take longer than
necessary
• But that is not always the
case…
4
Manager

Software Development is Social
5
(The conversation has been abstracted and
simplified for presentation purposes. The
listing to the left is not intended to be legible.

Software Development is Social
6
“Ali” is leading the
development of a
component.
(The conversation has been abstracted and
simplified for presentation purposes. The
listing to the left is not intended to be legible.

Software Development is Social
7
He seeks input from
stakeholders.
` Color?
Orientation?

Software Development is Social
8
Stakeholders start
discussing.
?
?
“Christine”

Software Development is Social
9
Stakeholders start
discussing.
?
?
“Bao”

Software Development is Social
10
Stakeholders start
discussing.
?
?

Software Development is Social
11
Ali thinks he has found
the answers he needs
=
=
Bao agrees

Software Development is Social
12
Ali marks the issue as
resolved
=
=

Software Development is Social
13
=
=
But “Dimitra” thinks that
the issue is not settled
Size??

Software Development is Social
14
=
=
But nobody takes her
objection into account
Size??
Guys?

Conversations are Messy
• The decision about
orientation was not debated
• The question about size
was left unanswered
• Stakeholder Dimitra was not
consulted until late
• Changing the status to
“resolved” was premature
15
Manager

Proposed Solution
16
Mine conversations
Gather Analytics
Generate Action
Recommendations

Proposed Solution
17
Mine conversations
Gather Analytics
Generate Action
Recommendations

Generating Recommendations
Use metrics to support managers in facilitating conversations:
18
Intervene when questions are left
unanswered for a “long” period of time
Alert when questions are prematurely
marked as resolved
Report when a question has been
discussed enough to be considered mature
If a question is re-opened, create a context
reminder of previously examined alternatives

Proposed Solution
19
Mine conversations
Gather Analytics
Generate Action
Recommendations

• Online vs Offline (post-hoc)
• Main challenge: determining resolution of a point of
uncertainty:
• Trusting status changes not necessarily a good idea
• Who should be part of discussion?
• People’s stance may not be obvious
• E.g. does elaboration imply endorsement?
• What does absence of comment (a person’s “default”) mean?
• For now:
Compute metrics from the structure of the conversation
20
Gathering Analytics about Uncertainty

Dashboard Mock-ups
21

Proposed Solution
22
Mine conversations
Gather Analytics
Generate Action
Recommendations

Mining the Conversations
23
`
• Identify unique Points of Uncertainty
from question utterances

Mining the Conversations
24
• Identify unique Points of Uncertainty
from question utterances
• Associate them with Proposed
Alternatives

Mining the Conversations
25
• Identify unique Points of Uncertainty
from question utterances
• Associate them with Proposed
Alternatives
• Track the stakeholders’ arguments

Mining the Conversations
• Identify unique Points of Uncertainty
from question utterances
• Associate them with Proposed
Alternatives
• Track the stakeholders’ arguments
• Recognize the Resolution of
Uncertainty
26

Construct a Model of the Conversation
27
• Twente Argumentation Schema (TAS)
• Time: left-to-right, top-to-bottom

Constructing the TAS Model
28
Automation Challenges
• Identifying unique points of
uncertainty among many
utterances of questions
• Identifying proposals when they
are not stated explicitly
• e.g. cultural differences in
assertiveness
• Mapping utterances of proposals
to points of uncertainty
Benefits
• Associates points of
uncertainty with proposed
answers
• Easy to compute structural
metrics
• Can be used to trace
uncertainty to software
artifacts

Summary
29
quality
speed

Future Work
• How to best automate the creation of the TAS model?
• How can we safely determine that a point of uncertainty
has been resolved?
• What are the most useful metrics, analytics and action
recommendation for managers?
• How to connect uncertainty in conversations with relevant
organizational, social and software models?
30

Questions?
icons by:
31