RE tutorial user stories

User story best practices
requirements in agile context
Garm Lucassen and Sjaak Brinkkemper, 12 September 2016 
Utrecht University, The Netherlands

Lucassen & Brinkkemper 12 September 2016
Today
• User story basics
• Improving user story quality
• Embedding user stories in agile practice
• Advanced analysis
2

Garm Lucassen
• PhD @ Utrecht University
- Software product management
• students
• professionals
- User stories
• Contact: g.lucassen@uu.nl
• http://garmlucassen.nl
• http://softwareproductmanagement.org
3

Sjaak Brinkkemper
• Professor Software Production @ Utrecht University
- Leads research group of  
35 staff and PhDs
- Product Software:  
Methodology of Development,  
Implementation and Entrepreneurship
• Contact: s.brinkkemper@uu.nl
• http://www.uu.nl/staff/SBrinkkemper/0
4

How about you?
• Name
• Company
• Role
• Experience with user stories
• What do you hope to learn today?
5

6
Schedule Time
1 Introduction 8.30
2 Familiarize with user story concept 9.00
3 Exercise 1: creating user stories 9.15
4 Varieties in industry 9.45
5 Coffee break 10.00
6 Industry survey 10.30
7 User story quality basics 11.30
8 Lunch 12.00
9 Quality User Story Framework 13.30
10 Exercise II: Grimm Tool 14.30
12 User stories in agile practice 15.30
13 Advances in user story analysis 16.15
14 Exercise 3: Visual Narrator 16.45
15 Contribute to ongoing research and closing 17.00

What is a user story?
• “As a Visitor, I want to purchase an event ticket”
• “As a visitor, I want to search for new events by
favorited organizers so that I am the ﬁrst to know of
new events”
• “As a Visitor, I want to be notiﬁed when an event is
close to becoming sold out, so that I do not miss
the event”

8
• User stories represent customer requirements in a card,
leading to conversation and conﬁrmation (Jeffries, 2001)
• User stories only capture the essential elements of a
requirement: 
- who it is for  
- what it expects from the system 
- why it is important (optional?)
• Simple format used by 70% of practitioners
who what why
(Connextra)As a role I want to action (so that beneﬁt, , )
(Lucassen et al., 2016)

• “As a Visitor, I want to purchase an event ticket”
• “As a visitor, I want to search for new events by
favorited organizers so that I am the ﬁrst to know of
new events”
the event”
“As a Visitor I want to search for new events by favorited organizers
so that I am the ﬁrst to know of new events”
, ,

History
• First mention in Kent Beck’s 1999 book  
Extreme Programming Explained
- Unstructured text
- Similar to use cases
- Restricted in size
• Jeffries 2001: card, conversation, conﬁrmation
• Widespread popularity after Mike Cohn’s  
User Stories Applied in 2004
10

Popularity
• 45% of practitioners employ user stories (Kassab, 2015)
• In agile: 90%! (Wang, 2015)
11

Exercise 1 
Creating user stories
• We are going to create user stories for an  
event ticketing service: TicketExpert
• The fundamental stories are:
- “As an Organizer, I want to distribute tickets, so that visitors can attend my event”
- “As a Visitor, I want to attend an event, so I can see an artist perform”
- “As a Visitor, I want to one click purchase tickets, so that I do not need to supply my
personal information with every purchase”
- “As a TicketExpert Employee, I want to manage events”
• Expand upon this list by creating 6 user stories:
- Create 3 simple user stories
- Create 2 more advanced user stories
- How about a quality attribute?
12

Functional architecture
On-line Event Ticketing
Product
scope
ticketing
contract
event requestbooking
ticket types
banking
details
payee
interaction
optionsterms of
services
customer
details
event details
Contract management
Ticket sales
Pay-
ment
Acqui-
sition
Module
Information flow

FA on module level: Ticket sales
Module
scope
Event  
registration
Event reporting
templates
booking
reports
ticket types
final ticket
payment 
details
Visitor management
event details
Ticket selection
Pre-payment  
arrangement
Ticket sales
Type management
package  
structures
ticket overview
visitor details
Sub-module

Exercise 1 
• We are going to create user stories for an  
event ticketing service: TicketExpert
• The fundamental stories are:
- “As an Organizer, I want to distribute tickets, so that visitors can attend my event”
- “As a Visitor, I want to attend an event, so I can see an artist perform”
- “As a Visitor, I want to one click purchase tickets, so that I do not need to supply my
personal information with every purchase”
- “As a TicketExpert Employee, I want to manage events”
• Expand upon this list by creating 6 user stories:
- Create 3 simple user stories
- Create 2 more advanced user stories
- How about a quality attribute?
15

Exercise evaluation
• How did it go?
• Things to consider
- Why part is very important!
- Don’t force a story into its format when its unnatural
- Remember: business/domain/application language,
- No technical details!
• What did you observe for your own stories?
16

Exercise evaluation
• How did it go?
• Things to consider
- Is your role the actual role?
- Why part is very important!
- Don’t force a story into its format when its unnatural
- Remember: business/domain/application language,
- No technical details!
• What did you observe for your own stories?
• Save stories as input for exercise 2!
17

18
Schedule Time
1 Introduction 8.30
8 Lunch 12.00

Industry survey
• Survey w/182 responses & 21 follow-up interviews
• Use of user stories
- Development Methods
- Templates
• Perception of user story effectiveness
- Impact on productivity?
- Impact on work deliverable quality?
19
(Lucassen et al. 2016a)
http://bit.ly/us_effective

Usage
20
As a role I want to action (so that benefit, , )
User Story Quality
Pragmatic
Complete
Independent
Uniform
Unique
Estimatable
Full sentence
Semantic
Conflict-free
Unambiguous
Problem-oriented
Conceptually sound
Syntactic
Minimal
Atomic
Well-formed
Independent 
Negotiable
Valuable
Estimable
Simple
Testable
• 70% use the Connextra template (10% w/o
optional) 
• Minority uses pre-defined quality guidelines
like INVEST (23,5%) or QUS

How are user stories applied?
• Huge diversity
• User story is the most granular representation
of a requirement developers use to build new
features
• Strong connection to Scrum (94%)
21

“For me, user stories and
Scrum are interconnected”
22
- Requirements engineer

23
Schedule Time
1 Introduction 8.30
4 Industry survey introduction 9.45
6 Industry survey continued 10.30
8 Lunch 12.00

24
Schedule Time
1 Introduction 8.30
8 Lunch 12.00

User Story Effectiveness
- Perception of all practitioners
- The role of using a template
- The impact of quality guidelines
- Technical vs. non-technical roles
25
Lucassen, G., Dalpiaz, F., van der Werf, J. M. E., & Brinkkemper, S. (2016a). The use and
effectiveness of user stories in practice. In REFSQ (pp. 205-222).
http://bit.ly/us_effective

Garm Lucassen12 September 2016
9%
8%
13%
14%
7%
10%
68%
61%
54%
53%
48%
40%
23%
31%
32%
33%
45%
51%
S6. Quality − guidelines
S5. Productivity − guidelines
S4. Quality − template
S3. Productivity − template
S2. Quality
S1. Productivity
Percentage
Strongly Disagree Disagree Neither Agree nor Disagree Agree Strongly Agree
source_data_role_r
Perception of all
practitioners
26
11%
15%
15%
33%
22%
37%
52%
52%
33%
30%
30%
22%
37%
33%
52%
37%
48%
41%
idelines
idelines
emplate
emplate
Quality
ductivity
Percentage
non_template
9%
8%
13%
14%
7%
10%
68%
61%
54%
53%
48%
40%
23%
31%
32%
33%
45%
51%

Perception of all
practitioners
11%
15%
15%
33%
22%
37%
52%
52%
33%
30%
30%
22%
37%
33%
52%
37%
48%
41%
idelines
idelines
emplate
emplate
Quality
ductivity
Percentage
non_template
9%
8%
13%
14%
7%
10%
68%
61%
54%
53%
48%
40%
23%
31%
32%
33%
45%
51%
9%
8%
13%
14%
7%
10%
68%
61%
54%
53%
48%
40%
23%
31%
32%
33%
45%
51%
S2. Quality
S1. Productivity
Percentage
source_data_role_r
27
Practitioners agree, productivity and work deliverable quality
increases when using:
- user stories
- templates

Perception of all
practitioners
11%
15%
15%
33%
22%
37%
52%
52%
33%
30%
30%
22%
37%
33%
52%
37%
48%
41%
idelines
idelines
emplate
emplate
Quality
ductivity
Percentage
non_template
9%
8%
13%
14%
7%
10%
68%
61%
54%
53%
48%
40%
23%
31%
32%
33%
45%
51%
9%
8%
13%
14%
7%
10%
68%
61%
54%
53%
48%
40%
23%
31%
32%
33%
45%
51%
S2. Quality
S1. Productivity
Percentage
source_data_role_r
28
Respondents are ambivalent about quality guidelines

Perception of all
practitioners
11%
15%
15%
33%
22%
37%
52%
52%
33%
30%
30%
22%
37%
33%
52%
37%
48%
41%
idelines
idelines
emplate
emplate
Quality
ductivity
Percentage
non_template
9%
8%
13%
14%
7%
10%
68%
61%
54%
53%
48%
40%
23%
31%
32%
33%
45%
51%
9%
8%
13%
14%
7%
10%
68%
61%
54%
53%
48%
40%
23%
31%
32%
33%
45%
51%
S2. Quality
S1. Productivity
Percentage
source_data_role_r
29
Very few practitioners are negative (7 - 14%)

“Why do you think user stories improve productivity or quality?”
30
In follow-up interviews:

“The right software”
31
- 10 interviewees

“User stories optimize
for happiness”
32

33

Template
12%
8%
52%
62%
36%
30%
12%
8%
52%
71%
36%
21%
36%
11%
32%
56%
32%
33%
36%
10%
24%
59%
40%
31%
24%
8%
32%
41%
44%
52%
16%
6%
36%
50%
48%
44%
S1. Productivity
S2. Quality
Non−Template
Template
Non−Template
Template
Non−Template
Template
Non−Template
Template
Non−Template
Template
Non−Template
Template
34

12%
8%
52%
62%
36%
30%
12%
8%
52%
71%
36%
21%
36%
11%
32%
56%
32%
33%
36%
10%
24%
59%
40%
31%
24%
8%
32%
41%
44%
52%
16%
6%
36%
50%
48%
44%
S1. Productivity
S2. Quality
Non−Template
Template
Non−Template
Template
Non−Template
Template
Non−Template
Template
Non−Template
Template
Non−Template
Template
n = 155
n = 27
35
Those who use templates are more positive about templates

12%
8%
52%
62%
36%
30%
12%
8%
52%
71%
36%
21%
36%
11%
32%
56%
32%
33%
36%
10%
24%
59%
40%
31%
24%
8%
32%
41%
44%
52%
16%
6%
36%
50%
48%
44%
S1. Productivity
S2. Quality
Non−Template
Template
Non−Template
Template
Non−Template
Template
Non−Template
Template
Non−Template
Template
Non−Template
Template
n = 155
n = 27
36
More positive, but no statistical conﬁrmation.

37
“Why do you use
this template?”

“A template
not
the template”
38
- 3 interviewees

“A template
not
the template”
“It’s not the template that improves quality, it’s what we’re
doing - we’re sharing requirements and a template makes that
easy to do and more likely that we’ll do it”.
39 - VP Engineering

“The why is essential”
“Typically, the why question is correctly answered if after the
initial answer, you ask ‘why?’ again for three more times”
41
- Agile coach

42

8%
7%
10%
49%
74%
67%
43%
19%
23%
12%
5%
8%
62%
77%
68%
25%
19%
23%
17%
14%
12%
49%
58%
55%
35%
28%
33%
19%
7%
10%
47%
72%
53%
33%
21%
37%
11%
7%
10%
24%
67%
38%
65%
26%
52%
6%
5%
10%
25%
84%
52%
69%
12%
38%
S1. Productivity
S2. Quality
Yes, self−defined guidelines
Yes, INVEST
No
Yes, INVEST
No
Yes, INVEST
No
Yes, INVEST
No
Yes, INVEST
No
Yes, INVEST
No
Quality guidelines
43

8%
7%
10%
49%
74%
67%
43%
19%
23%
12%
5%
8%
62%
77%
68%
25%
19%
23%
17%
14%
12%
49%
58%
55%
35%
28%
33%
19%
7%
10%
47%
72%
53%
33%
21%
37%
11%
7%
10%
24%
67%
38%
65%
26%
52%
6%
5%
10%
25%
84%
52%
69%
12%
38%
S1. Productivity
S2. Quality
Yes, INVEST
No
Yes, INVEST
No
Yes, INVEST
No
Yes, INVEST
No
Yes, INVEST
No
Yes, INVEST
No
Quality guidelines
n=72 
n=43
n=60
44
Respondents that follow quality guidelines are more positive.  
Respondents applying INVEST are even more positive.

45
“Why do you think quality guidelines improve  
work deliverable quality and productivity?”

“INVEST is not a
checklist”
46
Independent 
Negotiable
Valuable
Estimable
Simple
Testable
- 3 interviewees

“INVEST is useful for
inexperienced teams”
47
Independent 
Negotiable
Valuable
Estimable
Simple
Testable
- 2 interviewees

48

Technical vs. non-technical
8%
9%
61%
60%
31%
31%
6%
15%
74%
55%
20%
31%
13%
16%
56%
45%
31%
38%
9%
22%
62%
36%
28%
42%
10%
9%
48%
20%
42%
71%
6%
11%
56%
31%
39%
58%
S1. Productivity
S2. Quality
Non−Technical
Technical
Non−Technical
Technical
Non−Technical
Technical
Non−Technical
Technical
Non−Technical
Technical
Non−Technical
Technical
49

8%
9%
61%
60%
31%
31%
6%
15%
74%
55%
20%
31%
13%
16%
56%
45%
31%
38%
9%
22%
62%
36%
28%
42%
10%
9%
48%
20%
42%
71%
6%
11%
56%
31%
39%
58%
S1. Productivity
S2. Quality
Non−Technical
Technical
Non−Technical
Technical
Non−Technical
Technical
Non−Technical
Technical
Non−Technical
Technical
Non−Technical
Technical
50
Non-technical stakeholders are:
- On average 25% more positive
- Strongly in favor of quality guidelines

51

Key take aways
a. The simple structure of user stories enables
developing the right software, for it facilitates
creating a common understanding concerning the
requirement
b. Specifying the why part of a user story is essential
for requirements quality.
c. Practitioners who use the INVEST quality
guidelines are signiﬁcantly more positive about the
impact of user stories on productivity and the
impact of templates on work deliverable quality.
52

Garm Lucassen12 September 2016 53
We call for an increase in the diffusion of
knowledge concerning quality guidelines

54
Schedule Time
1 Introduction 8.30
8 Lunch 12.00

User story quality
55
Lucassen, G., Dalpiaz, F., van der Werf, J. M. E., & Brinkkemper, S.
(2016b) Improving agile requirements: the Quality User Story
framework and tool. Requirements Engineering Journal, 1-21.
http://bit.ly/improving_us

User story quality
• Conceptual model of user stories
• INVEST
• Quality User Stories Framework
• Prevalence of errors in real user story sets
• Exercise 2: Grimm Tool
56

Meet Anne
• Project manager web dev team
• Creates high quality user stories
• Developers do not
• Confused clients
57

In this tutorial section
Foundations for:
• Solving Anne’s problem
• Helping practitioners write high quality user stories
• Conducting advanced analyses in ﬁnal section
58

User story template
59
As a role I want to action (so that beneﬁt, , )

Conceptual model
60
Subject
User Story
EndMeans
Role
1 1..*
1
0..*
Format0..1
1..*
has_parent
Action Verb
Direct
Object
Indirect
object
Adjective
1
Epic
1..*
has
QualityClariﬁcation
0..* 0..*
11
1 1 10..* 0..*
Dependency
0..*

“As a User, I want to search for new events by favorited organizers,
Conceptual model
61
User Story
EndMeans
Role
1 1..*
1
0..*
Format0..1
1..*
has_parent
1
11

Conceptual model
62
User Story
EndMeans
Role
1 1..*
1
0..*
Format
1
11
0..1
1..*
has_parent
“As a User I want to search for new events by favorited organizers
, ,

Means concepts
63
Subject
Means
Action Verb
Direct
Object
Indirect
object
Adjective
1 1 10..* 0..*
“I want to search for new events by favorited organizers”I search eventsnew
Subject Action Verb
Direct
Object
Adjective
Indirect
object
favorited organizers

Ends concepts
“so that I am the first to know of new events”
64
End
0..* 0..*
Dependency
0..*
“so that I am the first to know of new events”new events
DependencyClarification Quality
the first

Conceptual model
65
Subject
User Story
EndMeans
Role
1 1..*
1
0..*
Format0..1
1..*
has_parent
Action Verb
Direct
Object
Indirect
object
Adjective
1
Epic
1..*
has
0..* 0..*
11
1 1 10..* 0..*
Dependency
0..*

66
Schedule Time
1 Introduction 8.30
8 Lunch 12.00

67
Schedule Time
1 Introduction 8.30
8 Lunch 12.00

Quality problems in practice
• Model captures correct
stories
• Stories from practitioners:
- Too long
- Unnecessary information
- Too little information
- Inconsistent
- Irrelevant
- Ambiguous
68
Subject
User Story
EndMeans
Role
1 1..*
1
0..*
Format0..1
1..*
has_parent
Action Verb
Direct
Object
Indirect
object
Adjective
1
Epic
1..*
has
0..* 0..*
11
1 1 10..* 0..*
Dependency
0..*

INVEST?
• Most popular and simple reminder for
characteristics of a good user story:
- Independent:
- Negotiable
- Valuable
- Estimable
- Simple
- Testable
• Drawbacks: non-speciﬁc, generic, qualitative
69

Slides available online
70
http://bit.ly/re16tut

Quality User Story Framework
71
Based on critical analysis of:
- Hundreds of user stories
- Existing quality frameworks
- Problems in practice

72
User Story Quality
Pragmatic
Complete
Independent
Uniform
Unique
Estimatable
Full sentence
Semantic
Conﬂict-free
Unambiguous
Problem-oriented
Conceptually sound
Syntactic
Minimal
Atomic
Well-formed
RQ1
RQ1
RQ2
RQ3
RQ4
RQ5
RQ6
RQ7
RQ8
RQ9
RQ10
RQ11
RQ12
RQ13

73
Individual
RQ1 Well-formed A user story includes at least a role and an action
RQ2 Atomic A user story expresses a requirement for exactly one feature
RQ3 Minimal A user story contains nothing more than role, action and benefit
RQ4 Conceptually sound The action expresses a feature and the benefit expresses a rationale
RQ5 Problem-oriented A user story only specifies the problem, not the solution to it
RQ6 Unambiguous A user story avoids terms or abstractions that lead to multiple interpretations
RQ8 Full sentence A user story is a well-formed full sentence
RQ9 Estimatable A story does not denote an unrefined requirement that is hard to plan and prioritize
Set
RQ7 Conflict-free A user story should not be inconsistent with any other user story
RQ10 Unique Every user story is unique, duplicates are avoided
RQ11 Uniform All user stories in a specification employ the same template
RQ12 Independent The user story is self-contained and has no inherent dependencies on other stories
RQ13 Complete Implementing a set of user stories creates a feature-complete application, no steps are missing
Quality User Story
Framework

• Not all quality criteria immediately critical
• Focus on:
RQ1. Well-formed
RQ2. Atomic
RQ3. Minimal
RQ4. Conceptually sound
RQ5. Problem oriented
RQ8. Full sentence
RQ11. Uniform
• Avoid premature optimization!
74

A user story includes at least a role and an action
I want to revoke access for problematic event organizers
➡ add role
As a TicketExpert Employee, I want to revoke access for problematic event
organizers
RQ1 - well-formed

A user story expresses a requirement for exactly one feature/problem
As a Visitor, I want to register for an event and create a personal account, so that I
can quickly register for more events in the future
➡ split
1. As a Visitor, I want to register for an event, so that I am admitted to the event 
2. As a Visitor, I want to create a personal account during event registration, so that I
can quickly register for more events in the future
RQ2 - atomic

RQ3 - minimal
A user story contains nothing more than role, action and beneﬁt
As an Event Organizer, I want to see the personal information of attendees (split into
price levels). See: Mockup by Alice NOTE: - First create the overview screen
➡ (re)move unnecessary information
As an Event Organizer, I want to see the personal information of attendees

RQ4 - conceptually sound
The action expresses a feature and the beneﬁt expresses a rationale
As an Event Organizer, I want to open the event page, so that I can see the personal
information of attendees
➡

RQ4 - conceptually sound
The action expresses a feature and the beneﬁt expresses a rationale
As an Event Organizer, I want to open the event page,  
so that I can see the personal information of attendees
➡ ends becomes separate means
1. As an Event Organizer, I want to open the event page, so that I can review
event related information
2. As a User, I want to see personal information of attendees, so that I know the
demographical distribution of the event

RQ5 - problem oriented
A user story only speciﬁes the problem, not the solution to it
As a Visitor, I want to download an event ticket. - Add download button on top right
(never grayed out)
➡

RQ5 - problem oriented
A user story only speciﬁes the problem, not the solution to it
As a Visitor, I want to download an event ticket. - Add download button on top right
(never grayed out)
➡ remove solution
As a Visitor, I want to download an event ticket

RQ8 - full sentence
A user story is a well-formed full sentence
update proﬁle
➡ add ‘want to’
As a Visitor, I want to update my proﬁle

RQ 11 - uniform
All user stories follow roughly the same template
1. As a Visitor, I want to create an account 
2. As a Visitor, I want to reset my password 
3. As a TicketExpert Manager, I receive an email notiﬁcation when a new user is
registered
➡ add ‘want to’
As an TicketExpert Manager, I want to receive an email notiﬁcation when a new user
is registered

84
Schedule Time
1 Introduction 8.30
8 Lunch 12.00

Grimm Tool Technology:
Automatic Quality User Story Artisan
• Automatically assess user story quality
• Restrict ourselves to criteria with potential for 100% recall
• The Berry Recall Criterion
• Syntactic: 
Well-formed 
Atomic 
Minimal
• Semantic 100% recall unachievable
• Selection of pragmatic: 
Explicit dependencies 
Uniform 
Unique
86
(Daniel Berry et al., 2012)
(Ryan, 1993)
User
stories
AQUSA
Linguistic
parser
Enhancer
Analyzer
Synonyms Homonyms Ontologies
Error
report
Atomic Independent UniqueMinimal Uniform
Report
generator
User story
base
Corrections

Grimm Tool
87
Well-formedAtomic Minimal Uniform Unique
AQUSA
√
only one feature no unnecessary text follows the template has a role and meansno duplicates
(with my name)
“As a Visitor, I want to supply my personal details, so that
the ticket is personalized ”
√√√⤫
the ticket is personalized (with my name)”
error
!
none found
√Explicit
dependencies

Grimm Tool
88
Well-formedAtomic Minimal Uniform Unique
AQUSA
√
only one feature no unnecessary text follows the template has a role and meansno duplicates
√√√
the ticket is personalized”
none found
√Explicit
dependencies
√ “As a Visitor, I want to supply my personal details, so that
the ticket is personalized”
√
perfect
story

Jira integration
automatically posts comments
89

Grimm Tool
90

Grimm Tool evaluation
• Measure: 
Precision 
Recall 
Most common violations
• Do we achieve the Berry Recall Criterion?
• Apply mechanics to seventeen datasets
91

Table 3 Detailed results split per data sets, showing number of defects correctly detected (Def), false positives (FP), and false negatives (FN)
1: ResearchComp 2: ExpenseComp 3: EnterpriseComp 4: DataComp 5: RealProd
# Def # FP # FN # Def # FP # FN # Def # FP # FN # Def # FP # FN # Def # FP # FN
Atomic 5 2 1 10 4 0 1 1 0 6 0 1 6 3 2
Minimal 6 3 0 3 1 0 25 5 0 4 2 0 16 6 0
Well-formed 6 4 0 1 1 0 33 21 0 2 0 0 0 0 0
Uniform 17 8 0 27 9 0 38 17 0 7 0 0 9 0 1
Unique 2 0 0 0 0 0 0 0 0 0 0 0 2 0 0
SUM 36 17 1 41 15 0 97 45 0 19 2 1 33 9 3
N, precision, recall 50 53% 95% 50 63% 100% 50 55% 100% 23 89% 94% 51 73% 89%
6: E-ComComp 7: EmailComp 8: ContentComp 9: CMSComp 10: HealthComp
Atomic 7 5 0 12 6 0 9 2 3 1 0 1 8 1 2
Minimal 20 6 0 6 0 0 6 0 0 10 0 0 5 1 0
Well-formed 8 8 1 8 0 0 0 0 0 2 0 0 0 0 0
Uniform 33 4 1 36 0 0 34 0 0 35 0 0 11 0 7
Unique 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
SUM 68 23 2 62 6 0 49 2 3 48 0 1 24 2 9
11: AccountancyComp 12: PharmacyComp 13: SupplyComp 14: IntegrationComp 15: HRComp
Atomic 12 2 0 10 3 1 4 2 1 3 2 0 21 7 6
Minimal 0 0 2 1 0 4 2 0 0 1 0 0 52 28 0
Well-formed 0 0 0 0 0 0 0 0 0 0 0 0 44 26 1
Uniform 11 0 0 14 0 9 0 0 0 46 0 0 41 17 0
Unique 18 0 0 0 0 0 0 0 0 0 0 0 0 0 0
SUM 41 2 2 25 3 14 6 2 1 50 2 0 158 78 7
16: FilterComp 17: FinanceComp Public 1: Duke University
# Def # FP # FN # Def # FP # FN # Def # FP # FN
Atomic 42 39 0 13 4 0 10 1 2
Minimal 5 0 0 25 5 0 4 3 0
Well-formed 0 0 0 0 0 0 0 0 0
Uniform 38 0 0 29 0 0 18 0 0
Unique 2 0 0 6 0 0 0 0 0
SUM 87 39 0 73 9 0 32 4 2
N, precision, recall 51 55% 100% 55 88% 100% 48 88% 93%

Grimm Tool evaluation
• 56% of selected stories violate one or more quality
criteria
• These data sets: 93.8% recall and 72.2% precision
93
14
Table 4 Overall recall and precision of AQUSA v1, com-
puted using both the micro- and the macro average of the
data sets
Recall Precision
Macro 92.1% 77.4%
Micro 93.8% 72.2%
Table 5 Number of defects, false positives, false negatives,
recall and precision per quality criterion
n = 1023 Totals
# Def # FP # FN Rec Prec
Atomic 170 83 18 82.9% 51.18%
Minimal 187 57 6 95.5% 69.52%
Well-formed 104 60 2 95.7% 42.31%
Uniform 426 55 18 95.4% 87.09%
Unique 30 0 0 100% 100%
SUM 917 255 44 93.8% 72.2%
Looking
and the tot
false negati
out. With t
the absolut
another. Re
atomic stan
as detected
the number
atomic, min
section, we
Atomic. Th
quently occu
‘&’ within a
and “As an
tion 6, this c
main types

Exercise 2: Grimm Tool
• Who brought user stories?
• Upload to: aqusa.nl
- Create an account
- Example CSV ﬁles: bit.ly/us_csvs
- Put your stories in a CSV, UTF-8 encoding
- One story per row
- Add “ and ” to beginning and end of each row
• Pre-submitted user stories
94

1. Create an account
95

2. Create CSV of your stories
96

3. Upload a CSV
97

4. Review output
98

4. Review output
99
Sometimes
necessary!

Example output
100

Exercise 2: Grimm Tool
• Who brought user stories?
• Upload to: aqusa.nl
- Create an account
- Example CSV ﬁles: http://tinyurl.com/us-csv
- Put your stories in a CSV, UTF-8 encoding
- One story per row
- Add “ and ” to beginning and end of each row
• Pre-submitted user stories
101

Output
• What kind of errors does the tool detect?
• How would you ﬁx the user stories?
• Things to ask yourself:
- Well-formed -> are these quality requirements?
- Atomic -> impact on estimation?
- Minimal -> too little detail?
- Uniform -> how many styles do you use?
- Unique -> where do these duplicates come from?
102

Estimating and developing
• Remaining criteria gradually become relevant
• Focus on:
RQ6. Unambiguous
RQ7. Conﬂict-free
RQ9. Estimatable
RQ12. Independent
RQ10. Unique
RQ13. Complete
• Keep iterating!
103

RQ6 - unambiguous
A user story avoids terms or abstractions that lead to multiple
interpretations
As an Event Organizer, I want to edit the content that I added to an event’s page
➡

RQ6 - unambiguous
A user story avoids terms or abstractions that lead to multiple
interpretations
As an Event Organizer, I want to edit the content that I added to an event’s page
➡ clarify the content
As an Event Organizer, I want to edit video and text content that I added to an event’s
page

RQ7 - conﬂict-free
A user story should not be inconsistent with any other user story
1. As an Event Organizer, I’m able to edit any event 
2. As an Event Organizer, I’m able to delete only the events that I added
➡

RQ7 - conﬂict-free
A user story should not be inconsistent with any other user story
1. As an Event Organizer, I’m able to edit any event 
2. As an Event Organizer, I’m able to delete only the events that I added
➡ change 1
1. As an Event Organizer, I’m able to edit events that I added

RQ9 - estimatable
A story does not denote a coarse-grained requirement that is difﬁcult to
plan and prioritize
As an Event Organizer, I want to see my task list during the event, so that I can
prepare myself (for example I can see at what time I should start traveling)
➡ split
1. As an Event Employee, I want to see my task list during the event, so that I can
prepare myself 
2. As an Event Organizer, I want to upload a task list for event employees

RQ10 - unique
Every user story is unique, duplicates are avoided
1. As a Visitor, I’m able to see a list of new events, so that I stay up to date  
2. As a Visitor, I’m able to see a list of new events, so that I stay up to date
➡ remove one
1. As a Visitor, I’m able to see a list of news items, so that I stay up to date

???
RQ12 - independent
The user story is self-contained and has no inherent dependencies on
other user stories
1. As an Event Organizer, I am able to add a new event
2. As a Visitor, I am able to view an event page
➡

• Try to avoid dependencies as much as possible
• Impossible to be fully independent
• But… always remain ﬂexible!
111
RQ12 - independent
The user story is self-contained and has no inherent dependencies on
other user stories
1. As an Event Organizer, I am able to add a new event
2. As a Visitor, I am able to view an event page
➡???

RQ13 - complete
112
Implementing a set of user stories creates a feature-complete
application, no steps are missing
1. As an Event Organizer, I want to update an event 
2. As an Event Organizer, I want to delete an event
➡add story
As an Event Organizer, I want to create an event

113
Schedule Time
1 Introduction 8.30
8 Lunch 12.00

114
Schedule Time
1 Introduction 8.30
8 Lunch 12.00

User stories in agile practice
• The context of user stories in systems development
• Epics for high-level planning
• Creating user stories as a team
• Industry experiences
115

The context of user stories in
systems development
116

User story
Deﬁnition:
A User story describes a functionality valuable to a user of a
software system
• Not just a description
• Assist in discussion on details of the story
• No attributes, conditions, special cases are included in the description
• Determines tests to state when the story is completed.
• Reference: Sonja Dimitrijevic, Jelena Jovanovic, Vladan Devedzic (2015). A Comparative Study of Software Tools for
User Story Management. Information and Software Technology, vol 57, pp. 352-368.
117

User Story processes
• There is no unique US process.
• Applied in any systems development process  
(waterfall, agile, iterative)
• Standard in agile processes: Scrum, XP
• Key US processes
1. Gathering from relevant sources
2. Role modeling
3. Acceptance testing
4. Estimating for releases and sprints
5. Planning in releases and sprints
6. Tracking and communicating
118

Waterfall, Scrum, or …?
119

US in Waterfall
User stories are written during Requirements stage as reﬁnements of high level
speciﬁcation. User stories are input for:
• Design: design of functionality; stakeholder interaction; quality
• Code: assignment of coding tasks
• Test: acceptance tests for user functionality
120
Requirements
Design
Code
Integration
Test
Deploy

US in Waterfall
121
Requirements
Design
Code
Integration
Test
Deploy
User Stories

US in Waterfall
122
Requirements
Design
Code
Integration
Test
Deploy
User Stories

US in Scrum
User stories are assigned to sprints during Release planning stage  
as refinements of high level specification, called epics. User stories are input for:
• Sprint planning: Refinement into Backlog items
• Test: acceptance criteria
123
Sprint 1 Sprint 2 Sprint 3 Sprint 4
refine
design code
test
review deploy

US in Scrum
124
refine
design code
test
review deploy
User Stories

US in Scrum
125
refine
design code
test
review deploy
User Stories

Gathering US
• All kinds of sources
• Active involvement of users and customers for the whole
duration of the project/release in Scrum
• Customer team can write user stories; and perform the
prioritization
• Language of application domain must be used
• Non-intrusive lightweight techniques: interviews, questionnaires,
observation, and story-writing workshops
• High level US, so-called epics, are detailed in other USs.
126

User Role modeling
Definition
A user role represents a collection of attributes the characterize a population of
users and their intended interactions with the system
• Examples: Event organizer, event visitor, payment bank, entertainer, ticket
fulfillment officer
• Identification of user roles should be done before writing of user stories
• Stories without a user role are apparently not essential
• Technique: Persona as used in User experience design, (introduced by Alan
Cooper: The Inmates Are Running the Asylum (1998)). A persona is a play-
acted fictitious characters in order to explain design questions. Personas are
more figurative, whereas user role is more formally structured.
127

Acceptance testing
• Veriﬁcation of complete development of user
stories
• Criteria: expectations of customer team
• Provide details for design and developments
• Written on the back of the story card
• Test in 2 ways
1. Manually by customer representative
2. Automatically by test tool
128

Estimating User stories
• Estimation is done collaboratively by team: measured in story points
• Using mixture of expert opinion, analogy, decomposition
• Simple process for estimating US
1. Identify base stories Identify base stories for relative sizing of the backlog.  
Story done earlier. Understanding this story is same among everyone on the team.
2. Walk through the requirements of the story Product owner explains story.
3. Discuss any details ScrumMaster adds details to story.
4. Sample questions the team should ask:
1. Design: What to learn before start work on this story?
2. Coding: What will be coding effort?
3. Unit testing: Special setup required for unit test?
4. Acceptance testing: Help customer automate acceptance tests?
5. Integration points: Does the story have any external dependencies?
6. Expertise: Does anyone have prior experience on similar story?
5. Reach a consensus Team to come up with the best estimation that everyone accepts.
6. Validation of estimated story points Estimates are internally consistent among : the
1's are about the same, all of the 2's match, etc.. Also, agree 4-point story is twice a 2-
point story.
129
Source: scrumalliance.org:  
A Practical Guide: Story Points-Based Estimation

Planning releases and
sprints
Velocity: number of story points per sprint
• Experience data: total number of story points of project/ sprints per project
• Assign USs over different sprints: Planning Game
• USs of sprint do not exceed velocity
• Prioritization: maximize value delivered to customer
• Technique: MoSCoW, cost/value, relative weigths
• Value: ﬁnancial, development cost, amount of learning, amount of risk
• Planning USs in sprint: decomposition in tasks
• Changes due to corrections in plan
130

Tracking and
communicating
• Monitoring and discussion of progress
• Reﬁne plan according to observations
131
Task board Burndown chart

Refinement of User Stories
• User stories need to be fit into the Sprint planning
• User Stories are formulated as Product Backlog Items, and refined in Sprint
tasks
• Some companies define Sprint tasks also in User Stories
132

Epics
▪ It is common to formulate Product Backlog Items (PBI) in User Story form
▪ Oversized PBIs are called epics.
▪ Traditional planning breaks features into several tasks that are hard to
prioritize as they lack business value
▪ Most customers don’t use most features of most products
▪ Wise to split epics to deliver the most valuable stories ﬁrst.
▪ Delivering lower-value features later is likely to involve some rework.
▪ Backlog Reﬁnement Meeting: also called “Backlog Grooming” or
“Backlog Maintenance”.
133

Epic splitting
▪ Large User Stories are called epics.
▪ Agility requires to split large epics into user stories representing small
product features.
▪ Example: “As a theatre visitor I want to display the available seats in a theatre so
I can make a good decision on which seat to select”
▪ Example ticketing application:
▪ Epic “display the available seating in a theatre”
▪ US1 “display available seats per row”
▪ US2 “display left, middle or right with empty seats”
▪ US3 “display theatre seating plan“
▪ US4 “display available seats on seating plan”
▪ Visualizing the theatre chairs in various forms is technical challenging, 
where the number of seats and the distance to the stage are most  
important to know for booking.
▪ So US4 can be left to a later release with some rework.
134

Creating user stories 
as a team
• Many approaches to creation process
• Quality should concern everyone
• Poker planning
• Team Estimation Game
• Different quality criteria apply
135

Garm Lucassen25 April 2016 137
Estimating user stories using
poker planning
• Consensus-based and extremely simple:
1. Project manager reads user story
2. Developers discuss user story (change if necessary)
3. Each developer scores using story points  
0, 1, 2, 3, 5, 8, 13, 20, 40
4. Debate score differences
5. Repeat 3 and 4 until developers reach consensus

Garm Lucassen25 April 2016 138
Poker planning impact
• Bring a QUS cheat sheet to poker planning
• Use question mark to indicate you don’t know
• If applicable, use QUS terminology to convey the
problem 
 
• Discuss, change story if necessary, repeat
1. Unambiguous 2. Estimatable  
3. Conﬂict-free 4. Independent 5. Unique

Industry experiences
• Many different approaches to working with user
stories. Currently exploring types!
• Want to contribute? Contact us!
139

Introducing QUS and
Grimm Tool
First evaluation with three companies
- 40% reduction in user story defects
- Unexpected beneﬁts
140
“As a Visitor, I want to get messages from the system (tricks etc)”
“I think the conversation effectiveness did change positively,  
but the Grimm Method training made me more critical of  
what to expect from user story conversation in terms of effectiveness”

Topics
• Introduction
• User story basics
• User story quality
• User stories in agile practice
• User story analysis
141

Topics
• Extracting models from user stories
• Advanced analysis through visualization:
- Conﬂict detection
- Duplicate prevention
- Ambiguity resolution
- Incompleteness mitigation
• Exercise 3: visualize your user stories
142

143
Schedule Time
1 Introduction 8.30
8 Lunch 12.00

Automated Extraction of Conceptual
Models from User Stories via NLP
Marcel Robeer, Garm Lucassen, Jan Martijn E.M. van der Werf,
Fabiano Dalpiaz and Sjaak Brinkkemper 
September 16th 2016c @ RE’16
ticket
visitor
account
system
create,
rename
log in,
log out
event
type
filter on
event
see
keep track of
account password
has
password
change purchase

• “As a Visitor, I want to buy an event ticket”
the event”
“As a Visitor I want to search for new events by favorited organizers
, ,
145
Going from…

.. to
146

C. Heuristics
148
Noun
Subject
Verb (including
preposition)
Concept
Relationship
Verb ‘to be’
Hierarchical
relationship

C. Heuristics
149
Noun
Subject
Verb (including
preposition)
Concept
Relationship
Verb ‘to be’
Hierarchical
relationship
Noun-noun
compound

As a visitor
2. Functional role
150
I want to choose an event
so that I can book a ticket for that event
Role
Means
End

As a visitor
3. Simplify the means
151
Role
Means
End

As a visitor
3. Simplify the means
152
Role
Means
End

As a visitor
4/5. Main verb & main object
153
Role
Means
End

As a visitor
6. Main relationship
154
Role
Means
End
visitor event
choose

As a visitor
7. Remaining information
155
Role
Means
End
visitor event
choose

As a visitor
156
Role
Means
End
visitor event
choose
ticketbook

As a visitor
157
Role
Means
End
visitor event
choose
ticketbook
has

However…
Models can get BIG

Just 32
stories!

Currently exploring
zoomand filter
techniques

First step: role ﬁlter
161
hasInstalla...
canRun
canMigrate
hasAdminis...
hasNode
System
Installation
System Admini...
Neurohub
Administrator
Script
Neurohub Insta... Data
Neurohub Node
Node

Conceptual: colors
162

Practitioner perception
163
1. Training new employees
2. Active documentation, finding inconsistent
terminology
3. Discussing with stakeholders to detect
incompleteness
4. Requirements prioritization
In their current state, the models can have the
following practical applications:

Advanced analysis
• Possible inconsistencies
- Conﬂict detection
- Duplicate prevention
- Ambiguity resolution
• Incompleteness mitigation
164

Conﬂict detection
165
hasMachine
hasDepen...
canInstall
hasAdminis...
Neurohub De...
LTS
Neurohub
Administrator
System
LTS Machine
Machine
Dependency
System Admini...
hasInstalla...
canRun
canMigrate
hasAdminis...
hasNode
System
Installation
System Admini...
Neurohub
Administrator
Script
Neurohub Insta...
Data
Neurohub Node
Node
Systems administrator System administrator

Duplicate prevention
166
Separate stories for:
- find flight
- search flight number
- look flight name
canKnow5
canSee6
canKnow4
canMeasure
canKnow6canAlertTo
canSee2
canSearc...
canSee3
canSee4
canSearc...
canSee5
canKnow2
hasNumber
canLookAt
hasName
canKnow...
canKnow10
canPreview
canRead
canLook
hasRecord
canAlert
canFind
canSee1
Flight Name
Turbulence
Amount
Air
Temperature
Flight Number
Longer
Flight
Graph
User
World
pectation
Turbulence Re...

Ambiguity resolution
167
canDelete
canEdit
canUse
canGet
canBe
hasLocation2canSeehasAddress
canClick
canFind
canAdd4
canAdd5
canView2
canRequest
canAdd2
canView1
canAdd3
canAdd6
canOpen
canAdd1
hasReset
canSet
canRemove
User
Map
Video Description
Plot Location
Text
Event
Fragment
Address
Location
Password Reset
Email Event Location
Access
Information
Email Address
Content
Password
Image

Incompleteness mitigation
168
hasFile
hasType
canSearchBy
canAccess
canUploadcanAttach1
canSearch...
canKeep
canBulk
canShare
canForm
hasData
canDownload
canLocate
canIndicat...
canCollect
canTag
canAttach2
canAttach3
Write Ups
File
Directory
Type
File Type
Data File
Meta Data
Link
Researcher
Data
Can search for
data by type. But
researcher cannot
search?

169
Schedule Time
1 Introduction 8.30
8 Lunch 12.00

Exercise 3
• Go to WebVowl or OWLGrED
• Upload .omn or paste link to .omn on the web
• Can you identify inconsistencies or
incompleteness?
• Your user stories available in email
• Example visualizations tinyurl.com/us-vis
170

174
Schedule Time
1 Introduction 8.30
8 Lunch 12.00

Contribute to research!
• Supply user stories
• Integrate aqusa.nl with Jira
• Beta test future visualization tools
• Participate in a survey on user story practice
• To indicate interest: bit.ly/us-research  
or contact: g.lucassen@uu.nl
175

Did we succeed?
• Did you learn about:
➡ User story basics
➡ Improving user story quality
➡ Embedding user stories in agile practice
➡ Advanced analysis
176

User Story Quality
Pragmatic
Complete
Independent
Uniform
Unique
Estimatable
Full sentence
Semantic
Conﬂict-free
Unambiguous
Problem-oriented
Conceptually sound
Syntactic
Minimal
Atomic
Well-formed
User Story Research 
Utrecht University
• Why are user stories effective?
• How to create better user stories?
• Create RE support tools using NLP
• Less is more!
- QUS Framework
- Grimm Method
- Visual/Interactive Narrator
- Next: connecting code
• Contribute: bit.ly/us-research!

Thank you!
Thank you!
178

References
- Cohn, M. (2004). User stories applied: For agile software development. Addison-Wesley Professional.
- Jeffries, R. (2001). Essential XP: Card, conversation, conﬁrmation. XP Magazine, 30.
- Wake, B. (2003). INVEST in good stories, and SMART tasks. h ttp://xp123. com/articles/invest-in-good-stories-
and-smart-tasks.
- Beck K (1999) Extreme programming explained: embrace change. Addison-Wesley, Boston
- Berry D, Gacitua R, Sawyer P, Tjong S (2012) The case for dumb requirements engineering tools. In:
Proceedings of international conference on requirements engineering: foundation for software quality
(REFSQ), LNCS, vol 7195. Springer, pp 211–217
179
- Lucassen, G., Dalpiaz, F., van der Werf, J. M. E., & Brinkkemper, S. (2016a). The use and
effectiveness of user stories in practice. In REFSQ (pp. 205-222). Springer International Publishing.
- Lucassen, G., Dalpiaz, F., van der Werf, J. M. E., & Brinkkemper, S. (2016b) Improving agile
requirements: the Quality User Story framework and tool. Requirements Engineering Journal, 1-21.
- Lucassen, G., Dalpiaz, F., van der Werf, J. M. E. M., & Brinkkemper, S. (2016c). Visualizing User
Story Requirements at Multiple Granularity Levels via Semantic Relatedness. Proc. of ER 2016
(forthcoming)
- Robeer, M., Lucassen, G., Van der Werf, J. M. E. M., Dalpiaz, F., & Brinkkemper, S. (2016).
Automated Extraction of Conceptual Models from User Stories via NLP. Proc. of RE 2016  
Friday: 11.30 @ Modeling and Simulations track

RE tutorial user stories

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (16)

Similar to RE tutorial user stories

Similar to RE tutorial user stories (20)

More from Garm Lucassen

More from Garm Lucassen (7)

Recently uploaded

Recently uploaded (20)

RE tutorial user stories