End-user Driven Feedback Prioritization

1. End-user Driven Feedback Prioritization
Norbert Seyff, Melanie Stade, Farnaz Fotrousi, Martin Glinz, Emitza Guzman, Martina Kolpondinos-Huber, Denisse Munante Arzapalo,
Marc Oriol, and Ronnie Schaniel
1

2. Motivation and Goal
End-user feedback gathering allows software companies to
elicit end-users’ experiences and requirements.
Several feedback channels allow a large group of end-users to
continuously communicate feedback, which can lead to large
numbers of feedback.
Most channels invite everybody to communicate feedback.
This includes end-users with low reputation and therefore
feedback can be misleading.
Software companies need to decide which feedback issues are
of high priority and should be considered in a next release.
The goal of our work is to provide approaches engaging end-
users in feedback prioritization. 2

3. Three areas of concern
3
Feedback
Gathering
Feedback
Analysis
Decision
Making
End-user Driven Feedback Prioritization
“It is so
difficult to
upload files
with this
app!”

4. Prioritization within Feedback Gathering
4
Feedback
Gathering
Feedback
Analysis
Decision
Making
End-user Driven Feedback Prioritization
“It is so
difficult to
upload files
with this
app!”

5. Feedback approaches supporting the
communication of priorities
Provision of feedback approaches which allows end-users
to express the importance of their feedback using
 Textual description of priority
 Ratings (e.g., “How urgent is this for you?”, “How
important is this for you?”, How important is this for
project success?”, 1-10 rating)
 Selection of a category (e.g., “Feature shortcoming”,
“Bug report”, “Praise”)
5
“It is so difficult to
upload files with this
app!”
8 of 10
“Feature shortcoming”

6. “It is so difficult to
upload files with this
app!”
Feedback approaches supporting the
discussion of feedback among end-users
Provision of feedback approaches which allow end-users
to prioritize existing feedback.
 Social networks allow for a communication of
(dis)agreements (“I like”), but interpretation is tricky
 Tailored solutions such as the “Garuso” (Game-based
Requirements Elicitation) platform provide more
sophisticated methods (relevance, popularity).
 End-users earns points for rating feedback provided by
others (Gamification).
6
1
1 2
5 9
0
0 1 4 2

7. Prioritization within Feedback Analysis
7
Feedback
Gathering
Decision
Making
End-user Driven Feedback Prioritization
“It is so
difficult to
upload files
with this
app!”Feedback
Analysis

8. “It is so difficult to
upload files with this
app!”
Number, Sentiment and Type
Number of feedback issues discussing the same issue
 Automatic clustering of feedback
 Number of feedbacks within a cluster
Sentiment of the feedback issue
 A sentiment reflects the feeling or view in a feedback
text
 Automatic classification: positive, neutral or negative
sentiment
Type of feedback issue
 Automatic classification: e.g., feature shortcoming, bug
reports, feature requests
8
similar feedback received:
21 times
sentiment:
negative
classification:
feature shortcoming

9. “It is so difficult to
upload files with this
app!”
Monitoring Data and Feedback Issues
Use of other information to automatically define the
priority of a feedback issue.
 Use of monitoring to identify the number of affected
end-users
 Use of monitoring to gather more accurate data about
the feedback description
 Use of monitoring to better understand a particular
end-user
9
User A needs on average
3 min and 28 taps to
upload a file
All users need on average
30 seconds and 8 taps to
upload a file.
All users need on average
2 min 34 seconds and 23
taps to upload a file.
OR

10. Prioritization within Decision Making
10
Feedback
Gathering
Feedback
Analysis
End-user Driven Feedback Prioritization
“It is so
difficult to
upload files
with this
app!”Decision
Making

11. “It is so difficult to
upload files with this
app!”
Other Sources and Validation
11
different reputations
different sources
Other Requirements and Feedback Issues
 Requirements from other sources might discuss the
same need as incoming feedback issues
Validation of End-user Feedback Priority
 Decision maker can finally decide and prioritize a
feedback based on evidence from different sources
(different weights).
Validation of requirements from other sources with the
help of feedback channels
 Decision maker can ask end-users about their opinion
on requirements gathered from other sources

12. Conclusion
Prioritized end-user feedback can
help to build software systems (release
planning)
Most end-user feedback is text-based
that is hard to interpret in terms of priority
Ideas to elicit information about
prioritization
 Motivation of end-users (Gamification,
inform end-users about decisions)
 Validity of information
12
Feedback
Gathering
Feedback
Analysis
Decision
Making
End-user Driven Feedback Prioritization