4. the use of the ai technique of natural language processing for user story analysis and estimation simon wright

Dr. Simon Wright
simon@symtech.ltd.uk

An Agile Functional Size Estimate should be:
precise, accurate and repeatable
 Agile Estimate Precision is a function of User Story completeness
 If a user story is missed, then the estimate is imprecise
 Agile Estimate Accuracy is a function of User Story correctness
 If a user story is incorrectly expressed, then the estimate is inaccurate
 Agile Estimate Repeatability is a function of User Story consistency
 If a user story uses terms inconsistently then different estimators are likely
to use different interpretations and thereby produce different estimates.
Copyright © 2017-2018 Symtech Ltd, All rights reserved.

NLP Analysis for User Stories
 A Correct user story
 A Complete sets of user stories
 A Consistent set of user stories
Text analysis Syntax analysis Semantic analysis
Incompleteness by
comparison - CRUDL
Incompleteness between
High level -Low level US
Duplicated/
Redundant
User Stories
Inconsistent
Terms/Units
Inconsistent
Properties.

1. It will implement the same set
of functions as its
predecessor
2. Relevant sets of project data
will be sorted as quickly as
possible
3. Appropriate standards shall
be used when necessary
4. Careful consideration should
be given to using a high-
bandwidth channel
5. Windows 10 shall be installed
on all PCs
 Badly phrased User Needs
often contain key indicators
such as specific words,
constructs or phrases
 It is not difficult to define a set
of terms to avoid.
We all can spot “poor” statements of user need

1. It will implement the same set
of functions as its
predecessor
2. Relevant sets of project data
will be sorted as quickly as
possible
3. Appropriate standards shall
be used when necessary
4. Careful consideration should
be given to using a high-
bandwidth channel
5. Windows 10 shall be installed
on all PCs
 5 words to change
 1 word to change.
We all can spot “poor” statements of user need

Text Analysis Metric
 Definition:
 TA Correctness Metric = the number of times a word or phrase occurs in
the “avoid” list
 Definition:
 Quality Level Low = Quality Score more than or equal to 5
 Quality Level Medium = Quality Score more than 1 and less than 5
 Quality Level High = Quality Score more than 0 and less than 1
1. It will implement the same set of functions as
its predecessor
2. Relevant sets of project data will be sorted as
quickly as possible
3. Appropriate standards shall be used when
necessary
4. Careful consideration should be given to using
a high-bandwidth channel
5. Windows 10 shall be installed on all PCs
TA C Metric Q. Level
5 Low
6 Low
3 Medium
4 Medium
1 Medium
Total 19 Av. 3.8
The next
step is
syntax.

The syntax of effective User Stories
THE ACTOR IMPERATIVE AN ACTION AN OBJECT and/or
CONDITION
As a US Author I want to maintain User Stories
As a Product
Owner
I need to list User Stories alphabetically
<ACTOR> <ACTION> <OBJECT> and
<TARGET> <PERFORMANCE> <TRIGGER> <CONSTRAINT> <RATIONALE>*
*Writing Effective Natural Language Requirements Specifications by William M. Wilson The Journal of Defense Software Engineering
February 1999

Effective User Stories follow a syntax
 As a [Actor – who/what does the action]
 I want to [Action – what happens e.g. store, update, send data]
 for [Direct Object – what is acted upon]
 the [Indirect Object – what is acted upon]
 on the [Target – where the output is sent]
 with [Performance - frequency and/or quality characteristic]
 when [Trigger – causes of action]
 unless / even if [Constraint – business rule or limiting factor]
 So that [Rationale - description of value or benefit is achieved]
There are other User Story syntaxes but a uniform syntax of user
story expression increases clarity, consistency and completeness.

Example User Story
As a library_user I want to search for books in the library catalogue on
the computer screen with speed and ease-of-use when I provide a
book title even if the book title is incomplete so that I can find all books
with similar titles
<ACTOR><ACTION><OBJECT(s)> //
<TARGET><PERFORMANCE><TRIGGER><CONSTRAINT><RATIONALE>
<TRIGGER><ACTOR><ACTION><OBJECT><TARGET> <CONSTRAINT>
I provide a book title and I begin a search for books in the library
catalogue on the computer screen even if the book title is incomplete.
Example Use Case

Syntax Analysis of a good User Story
 As a library user I want to search for books
 on the computer screen
 with speed and ease-of-use
 when I provide a book title
 even if the book title is incomplete
 so that I can find all books with similar titles
What
When & how
Why
The next
step is
semantics.

Semantic Analysis of 3 words
The three words ‘book’, ‘library’
and ‘a’ can be ordered in six ways
Only three are syntactically correct
“A library book”
“A book library”
“Book a library”
An object
A descriptor
An action.

Measure: Book the book from the book library
NLP: Correctness and Consistency
Metric
Phrase Syntactically Correct? Semantically Consistent? Total
“A library book”
“A book library”
“Book a library”
0
0
0
0
1
1
0
1
1
“Library a book”
“Library book a”
“book Library a”
1
1
1
0
0
1
1
1
2
Book the book from the
book library
0 2 2
Reserve the book from
the document library
0 0 0
How does this apply to:
“Agile Deliveries; #Estimates or #NoEstimates? That is the question”.

What if Estimation is easy, valid, (precise, accurate
and repeatable), so maybe it’s worth a try?

A Strategy for NLP of User Stories
 NLP Analysis to identify Users in the User Stories
 NLP Analysis to identify Entities in the User Stories
 NLP Analysis to identify Transactions in the User Stories
 Correctness analysis to identify incorrectly expressed user stories
 Completeness analysis to identify incomplete epics
 Consistency analysis to identify inconsistent terms in an epic
 Functional size can be estimated as a by product.

NLP Analysis to identify Users
 Users come in many
shapes and sizes
 Normal Operator
 Maintenance Operator
 Operational Support
 Interfacing System / API
 Clock
 Other Software etc

NLP Analysis to identify Users
 As a library user I want to search for books
Subject
What
When & how
Why

NLP Analysis to identify Transactions
Process
User(s)
Humans
Other Systems
Other Software
DATA DATA
Storage
Transactional / Functional

Catalogue
SEARCH TERM
DATAMATCHING DATA
Search

 As a library user I want to search for a book
Verb
What
When & how
Why

NLP Analysis to identify Entities
Book
Catalogue
BOOK SEARCH TERM
BOOK DATAMATCHING BOOK DATA
Search

NLP Analysis to identify Entities
Object
What
When & how
Why
VerbSubject

An automatable process
 NLP Analysis to identify the Users in the User Stories
 NLP Analysis to identify the Transactions in the User Stories
 NLP Analysis to identify the Entities in the User Stories
 Correctness analysis to identify incorrectly expressed user stories
 Completeness analysis to identify incomplete epics
 Consistency analysis to identify inconsistent terms in user stories

An automatable process
 NLP Analysis to identify the Subjects in the User Stories
 NLP Analysis to identify the Verbs in the User Stories
 NLP Analysis to identify the Objects in the User Stories
 Correctness analysis to identify incorrectly expressed user stories?
 Completeness analysis to identify incomplete epics?
 Consistency analysis to identify inconsistent terms in user stories?

Correctness analysis
 Has every User Story at least
 one User, (subject)?
 one Transaction, (verb)? and
 one Entity, (object)?
Consistency analysis
 Have Users been named consistently?
 Have Transactions been named
consistently?
 Have Entities been named consistently?
Completeness analysis
 Has every Users been the subject of a user story?
 Does every Entity have a “create” verb in a user story?
 Does every Entity have a “read” verb in a user story?
 Does every Entity have a “update” verb in a user story?
 Does every Entity have a “delete” verb in a user story?

NLP Analysis of our User Story
Subject Verb Object
What
When & how
Why
Primary Intent
Essential Details
}

Automatic NLP Analysis of our User Story
Subject Verb Object
Primary Intent
Essential Details
}

Users and
Objects found
by Scopemaster
Candidate
Users and
Objects found
by NLP

Data Movements
found by NLP

Automatic NLP Analysis
of our User Story
Potential
Defects
But what have we missed?
Correct and Consistent Yes
But incomplete!

But what about an epic?
 Here is an example of an epic from the
internet
 How long would it take to make a precise,
accurate and repeatable functional size
estimate?
 This is how long it took me …

Scrum Alliance Web Site Profiles v1
 Upload of 14 User Stories – 15 minutes
 NLP analysis of original stories – 5 seconds
 Summary Result

 Specific Result – 8th User Story
 Need to separate primary intent from the essential details – v2

 Split the PI from ED of 14 User Stories – 10 minutes
 NLP re-analysis of user stories – 5 seconds
 Summary Result

Scrum Alliance Web Site Profiles
Version 1 Version 2
Primary Intent
Essential Details
}
}
But what about the un-estimated ones?

Scrum Alliance Web Site Profiles
Version 2
Primary Intent
Essential Details
}
}
But what about the un-estimated ones – reword?

 Reword 6 User Stories – 12 minutes
 NLP re-analysis of user stories – 5 seconds
 Summary Result

Primary Intent
Essential Details
}
}

But what have I
missed?
 These may be covered in one of the
other epics – but they may not
 There were 5 more pages of User
Stories – probably containing 14 US
per page – 70 US total?
 So how long to do a full analysis?

Results and Summary
 v1 took 15 mins 5 sec
 Total 37 mins 15 sec
for 14 User Stories
 So maybe 3.5 hours
for 70 User Stories
 Precise - all User Stories sized ✓
 Accurate - same size for same function ✓
 Repeatable – not tested but probably ✓
 14 Correct and Consistent User Stories in 37 min
 But not Complete.
 About 3.5 hours to estimate

What if Estimation is easy, valid, (precise, accurate and
repeatable), so maybe it’s worth 3.5 hours of your time?

4. the use of the ai technique of natural language processing for user story analysis and estimation simon wright

Recommended

Recommended

More Related Content

Similar to 4. the use of the ai technique of natural language processing for user story analysis and estimation simon wright

Similar to 4. the use of the ai technique of natural language processing for user story analysis and estimation simon wright (20)

More from Nesma

More from Nesma (20)

Recently uploaded

Recently uploaded (20)

4. the use of the ai technique of natural language processing for user story analysis and estimation simon wright