At the IEEE International Requirements Engineering 2015, I gave a presentation about my work on user stories as presented in the paper: "Forging High Quality User Stories: Towards a Discipline for Agile Requirements" Download paper: https://goo.gl/Q3rGTp

1. Forging High-Quality
User Stories
Garm Lucassen, Fabiano Dalpiaz, Jan Martijn E.M. van der Werf, Sjaak Brinkkemper
August 27 2015
Towards a Discipline for Agile Requirements (R13)

2. Garm LucassenRE’15 August 27 2015
Meet Anne
• Project manager web dev team
• Writes high quality user stories
• Developers do not
• Confused clients
2

3. Garm LucassenRE’15 August 27 2015
In this presentation
Foundations for:
• Solve Anne’s problem
• Help practitioners write higher quality user stories
• Conduct advanced analyses in future work
3

4. Garm LucassenRE’15 August 27 2015
What is a user story?
• User stories represent customer requirements in a card,
leading to conversation and confirmation (Jeffries, 2001)
• User stories only capture the essential elements of a
requirement:
- who it is for
- what it expects from the system
- why it is important (optional)
• Simple format:
4
who what why
(Mike Cohn)
(Yu and Mylopoulos, 1994)
As a role I want to action (so that benefit, , )

5. Garm LucassenRE’15 August 27 2015
What is a user story?
• “As a User, I want to purchase an event ticket”
• “As a User, I want to be notified when an event is
close to becoming sold out, so that I do not miss
the event”
5
“As a User I want to search for new events by favorited organizers
so that I am the first to know of new events”
, ,

6. Garm LucassenRE’15 August 27 2015
Overview
1. Quality User Story Framework
2. Conceptual Model of User Stories
3. Cross-Story Relationships
4. Automatic Quality User Story Artisan (AQUSA)
6

7. 1. What is user story quality?
2. Conceptualizing a user story
3. Cross-story relationships
4. Automating the QUS Framework

8. Garm LucassenRE’15 August 27 2015
Quality User Story Framework
Introduction
• Defines 14 quality criteria for individual and sets
of user stories
• All derivable from user story text
• Ignores requirements management concerns
such as effort estimation and comments
8

9. Garm LucassenRE’15 August 27 2015
Quality User Story Framework
Overview
9

10. Garm LucassenRE’15 August 27 2015 10
Atomic
A user story expresses a requirement for exactly one feature
Quality defect example
As a User, I want to click a particular location from the map and thereby perform a
search of landmarks associated with that latitude longitude combination
➡
1. As a User, I want to click a particular location from the map
2. As a User, I want to search landmarks associated with a location based on lat long
split
Example 1- syntactic

11. Garm LucassenRE’15 August 27 2015
Example 2 - semantic
11
Problem-oriented
A user story only specifies the problem, not the solution to it
Quality defect example
As a care professional I want to save a reimbursement. - Add save button on top
right (never grayed out)
➡
As a care professional I want to save a reimbursement
remove solution

12. Garm LucassenRE’15 August 27 2015
Example 3 - pragmatic
12
Uniform
All user stories follow roughly the same template
Quality defect example
1. As a User, I want to create an account
2. As a User, I want to reset my password
3. As an Administrator, I receive an email notification when a new user is registered
➡
As an Administrator, I want to receive an email notification when a new user is
registered
add want to

13. 2. Conceptualizing a user story
1. What is user story quality?
3. Cross-story relationships
4. Automating the QUS Framework

14. Garm LucassenRE’15 August 27 2015
Conceptual model
14
Subject
User Story
EndMeans
Role
1 1..*
1
0..*
Format0..1
1..*
has_parent
Action Verb
Direct
Object
Indirect
object
Adjective
1
Epic
1..*
has
QualityClarification
0..* 0..*
11
1 1 10..* 0..*
Dependency
0..*

15. Garm LucassenRE’15 August 27 2015
“As a User, I want to search for new events by favorited organizers,
so that I am the first to know of new events”
Conceptual model
15
User Story
EndMeans
Role
1 1..*
1
0..*
Format0..1
1..*
has_parent
1
11

16. Garm LucassenRE’15 August 27 2015 16
User Story
EndMeans
Role
1 1..*
1
0..*
Format
1
11
0..1
1..*
has_parent
“As a User I want to search for new events by favorited organizers
so that I am the first to know of new events”
, ,

17. Garm LucassenRE’15 August 27 2015
Means concepts
17
Subject
Means
Action Verb
Direct
Object
Indirect
object
Adjective
1 1 10..* 0..*
“I want to search for new events by favorited organizers”I search eventsnew
Subject Action Verb
Direct
Object
Adjective
Indirect
object
favorited organizers

18. Garm LucassenRE’15 August 27 2015
Ends concepts
“so that I am the first to know of new events”
18
End
QualityClarification
0..* 0..*
Dependency
0..*
“so that I am the first to know of new events”new events
DependencyClarification Quality
the first

19. Garm LucassenRE’15 August 27 2015
Conceptual model
19
Subject
User Story
EndMeans
Role
1 1..*
1
0..*
Format0..1
1..*
has_parent
Action Verb
Direct
Object
Indirect
object
Adjective
1
Epic
1..*
has
QualityClarification
0..* 0..*
11
1 1 10..* 0..*
Dependency
0..*

20. 1. What is user story quality?
2. Conceptualizing a user story
4. Automating the QUS Framework
3. Cross-story relationships
- Complete
- Independent
- Uniform
- Unique

21. Garm LucassenRE’15 August 27 2015
Cross-story relationships (1)
Complete
When user stories imply necessity of functionality not yet captured in
another user story, the set is incomplete
Example: to read, update or delete an item you must first create it
Example:
US1: As a User, I want to delete an event
US2: As a User, I want to update an event
US3: As a User, I want to create an event
21
Requires➡

22. Garm LucassenRE’15 August 27 2015
Cross-story relationships (2)
Independent
There are many ways in which user stories are dependent on another user
story. Two examples are causality and superclasses:
Example: ‘content’ can refer to different types of multimedia. Audio, video,
photographs, annotated text, …
Example:
US1: As a User, I want to edit profile content
US2: As a User, I want to upload a profile photo
US3: As a User, I want to provide profile text
22
refers to➡

23. Garm LucassenRE’15 August 27 2015
Cross-story relationships (3)
Uniform
A user story has a format that is consistent with the format of all other
user stories
Example: all user stories (roughly) follow the template
Example:
US1. As a User, I want to create an account
US2. As a User, I want to reset my password
US3. As an Administrator, I receive an email notification when a new user is registered
US3’. As an Administrator, I want to receive an email notification when a new user is
registered
23
change to➡
As a role I want to action (so that benefit, , )

24. Garm LucassenRE’15 August 27 2015
Cross-story relationships (4)
Unique
No other user story is identically or semantically the same or too
similar. Many different ways: exact or semantic duplicate,
different role or ends = means
Example:
US1: As a User, I want to edit an event
US2: As a User, I want to change an event
24

25. 4. Automating the QUS Framework
1. What is user story quality?
2. Conceptualizing a user story
3. Cross-story relationships

26. Garm LucassenRE’15 August 27 2015
Automatic Quality User
Story Artisan
• Automatically assess user story quality
• Restrict ourselves to criteria with potential for 100% recall
• Syntactic:
Well-formed
Atomic
Minimal
• Semantic 100% recall unachievable
• Selection of pragmatic:
Explicit dependencies
Uniform
Unique
26
(Daniel Berry et al., 2012)
User
Stories
AQUSA
Linguistic
Parser
Enhancer
Analyzer
Synonyms Homonyms Ontologies
Error
Report
Atomic
Explicit
dependencies UniqueMinimal Uniform
Report
generator
User story
base
(Ryan, 1993)

27. Garm LucassenRE’15 August 27 2015 27
Automatic Quality User
Story Artisan
Well-formedAtomic Minimal Uniform Unique
AQUSA
√
only one feature no unnecessary text follows the template has a role and meansno duplicates
(with my name)
“As a Visitor, I want to supply my personal details, so that
the ticket is personalized ”
√√√⤫
“As a Visitor, I want to supply my personal details, so that
the ticket is personalized (with my name)”
error
!
none found
√Explicit
dependencies

28. Garm LucassenRE’15 August 27 2015 28
Automatic Quality User
Story Artisan
Well-formedAtomic Minimal Uniform Unique
AQUSA
√
only one feature no unnecessary text follows the template has a role and meansno duplicates
√√√
“As a Visitor, I want to supply my personal details, so that
the ticket is personalized”
none found
√Explicit
dependencies
√ “As a Visitor, I want to supply my personal details, so that
the ticket is personalized”
√
perfect
story

29. Garm LucassenRE’15 August 27 2015
Automatic Quality User
Story Artisan
29

30. Garm LucassenRE’15 August 27 2015
AQUSA evaluation
• Measure:
Accuracy
Precision
Recall
Most common violations
• Apply mechanics to three datasets
1. Tailor made software project from small NL comp (98 US)
2. POS prod by Belgian team outsources to Romania (124 US)
3. Advanced health care product recently adopted US (24 US)
30

31. Garm LucassenRE’15 August 27 2015
AQUSA evaluation
• 25% of selected stories violate one or more quality criteria
• These data sets: 100% recall and 71% precision
31
Set 1 (n=96) Set 2 (n=24) Set 3 (n=124)
V FP V FP V FP
Atomic 7 5 10 3 17 12
Minimal 0 - 17 - 16 -
Well-formed 8 - 2 - 6 -
Explicit
dependencies 0 - 1 - 0 -
Uniform 14 4 2 - 1 -
Unique 2 - 0 - 0 -
Total US
with errors 27 9 19 3 37 12

32. Garm LucassenRE’15 August 27 2015
AQUSA evaluation
• Violation types vary: uniform, minimal, atomic
• Sneak preview: in new tests more diverse violations
and larger number of edge cases
32
Set 1 (n=96) Set 2 (n=24) Set 3 (n=124)
V FP V FP V FP
Atomic 7 5 10 3 17 12
Minimal 0 - 17 - 16 -
Well-formed 8 - 2 - 6 -
Explicit
dependencies 0 - 1 - 0 -
Uniform 14 4 2 - 1 -
Unique 2 - 0 - 0 -
Total US
with errors 27 9 19 3 37 12

33. Garm LucassenRE’15 August 27 2015
Conclusion
• Presented the theoretical foundation for improving
user story quality using NLP:
• QUS Framework
• Conceptual model
• Cross-story relationships
• Applying AQUSA shows
• Relevant quality defects present in real-world
data
• Feasible to automatically detect quality defects
33

34. Garm LucassenRE’15 August 27 2015
Future work
• Realize and evaluate AQUSA (started)
Please contribute: https://github.com/gglucass/AQUSA
• Explore role of domain & foundational ontologies
• How do practitioners use user stories? (ongoing)
• Longitudinal study w/ AQUSA in multiple software
companies
34

35. Garm Lucassen: g.lucassen@uu.nl
Fabiano Dalpiaz: f.dalpiaz@uu.nl
Jan Martijn van der Werf: j.m.e.m.vanderwerf@uu.nl
Sjaak Brinkkemper: s.brinkkemper@uu.nl
Thank you!

36. Garm LucassenRE’15 August 27 2015
References
• M. Cohn, User Stories Applied: for Agile Software Development. Red- wood City, CA,
USA: Addison Wesley Longman Publishing Co., Inc., 2004.
• O. I. Lindland, G. Sindre, and A. Sølvberg, “Understanding Quality in Conceptual
Modeling,” IEEE Software, vol. 11, no. 2, pp. 42–49, 1994.
• D. Berry, R. Gacitua, P. Sawyer, and S. Tjong, “The Case for Dumb Requirements
Engineering Tools,” in Proc. of Requirements Engineer- ing: Foundation for Software
Quality. Springer, 2012, vol. 7195, pp. 211–217.
• IEEE Computer Society, “IEEE Recommended Practice for Software Requirements
Specifications,” IEEE Std 830-1993, 1994.
• Jeffries, Ron. “Essential XP: Card, Conversation, and Confirmation.” XP Magazine
(August 30, 2001)
• P. Heck and A. Zaidman, “A Quality Framework for Agile Requirements: A
Practitioner’s Perspective,” CoRR, vol. abs/1406.4692, 2014. [Online]. Available: http://
arxiv.org/abs/1406.4692
36

37. Garm LucassenRE’15 August 27 2015
AQUSA Architecture
37
User
Stories
AQUSA
Linguistic
Parser
Enhancer
Analyzer
Synonyms Homonyms Ontologies
Error
Report
Atomic
Explicit
dependencies UniqueMinimal Uniform
Report
generator
User story
base