Measure COSMIC Size of Requirements Automatically

Copyright (c) 2018 Pentad-SE Ltd
Automatic measurement of COSMIC size
by syntactical/semantic analysis of text
requirements
20/06/2018 1COSMIC-SIG 07/06/2018
Peter Fagg
Pentad-SE Ltd
md@pentad.co.uk

Objectives of this work
Copyright (c) 2018 Pentad-SE Ltd20/06/2018 2COSMIC-SIG 07/06/2018
Set the context by explaining briefly the concepts of Natural Language
(NL)
Explain a type of NL called Controlled Natural Language (CNL) which
conforms to a defined syntax, then explain its use to state requirements
To explain in more detail the process for formally defining a CNL in a form
suitable for automation
To demonstrate how two NCLs have been implemented and how a COSMIC
functional size may be measured automatically.
To describe in outline four implementations of a CNL

Natural Language
‘Natural Language’ refers to a human language such as English, Russian,
German, or Japanese as distinct from a command or programming language
which can be understood by a machine
It is language that has evolved as a method of communicating between
people
It is only Natural when used within a community that understands it.
e.g. the language used by a doctor to describe a prescription is understandable
by a pharmacist who must dispense the prescription, e.g.
“2 tab b.i.d. p.c”
However, the patient receiving the prescription needs to be told that this means:
“2 tablets twice per day after meals”

Natural Language
Fortunately in the field of Requirements Engineering , there is a specific audience,
and a generally-understood language for communication.
Sometime third parties have to analyse NL so as to understand the meaning and
where the information comes from, in many forms and formats and from many
sources, in languages that are Natural only to the respective intended listeners or
readers.
Examples:
The NL Processing techniques used to make sense of the body of information
gathered are very sophisticated and diverse.
Even in this field there are several types of stakeholder, each with its own NL
National security services monitor phone calls, texts, e-mails, etc.,
to detect and intercept possible security threats;
Google analyses web search pattern to determine interests so they
can target advertising.

Requirement Types
Requirements emerge at various stage of the lifecycle. ISO/IEC 29148
(Requirements Engineering) has defined the type of requirements that
can be expected and is illustrated as follows:-

Controlled Natural Language
Even a language directed to a specific audience does not guarantee that statements
of requirements are clear and unambiguous.
To help overcome that we can use what is called a ‘Controlled Natural Language’.
Use of a Controlled Natural Language makes the automatic analysis to understand
meaning much easier that if a NL is used.

A subset of Natural Language where the grammar and vocabulary have
been restricted so as to remove or reduce ambiguity and complexity.
CNL’s are constructed so as to satisfy two objectives ...
• To improve readability, e.g. for non-native speakers by
constraining vocabulary and grammar
• To enable reliable and automatic analysis of the language
Two important concepts to achieve objectives: Syntax and Semantics
Syntax concerns the structure and rules of the language. It is referring
to grammar and vocabulary, which in a CNL, is constrained
Semantics deal with the meaning assigned to the Keywords and
Words used by the Syntax.

Examples
I am researching four forms of expressing requirements expressed in CNL
Easy Approach to Requirements Analysis (EARS):
COSMIC User Story Standard (CUSS)
ISO IEC/29148 Requirements Engineering.
The Haoues, Sellami, Ben-Abdallah Use Case.

The structure and content is controlled by Keywords followed by
placeholders containing specific information, appearing in a strict order.
There is only one requirement per sentence.
Requirements are separated by a full stop.
EARS

CUSS - Cosmic User Story Standard
General Form
As a <who/role>, I want to <what>, linked to <connections>; so/then be
notified about operation status.
Example
“As a Librarian, I want to register a new book, connected to author and
publisher; then be informed about operation status”
Keywords and placeholders again, but more punctuation rules. The Comma
Semi-Colon and Full-stop have special meaning

ISO/IEC 29148
Keywords and placeholders and fewer punctuation rules. The Comma and Full-stop
have special meaning. The Role of the User is less obvious but it is there.

29148 - Conclusions
• This is included as it is something I have researched and will definitely progress in
due course as it is a good subject for automation.
• It is in line with my policy not to invent but to use a ready-made solution, where
others are doing the selling and just add value to it.
• IS 29148 will be issued shortly, there must be an audience for it. It would be good
for COSMIC to be ready for it.

Haoues, Sellami, Ben-Abdallah
A paper in 2017 by M. Haoues, A. Sellami, and H. Ben-Abdallah proposes a format for
Use Cases based on a limited form of Markup.
Markup involves using tags of different kinds to identify the different elements of the
statement. For example the Use Case header has the following information
Number:<unique ID assigned to a use case>
Name:<unique name assigned to a use case>
Level:<level of use case description>
Description:<a summary of use case purpose>
Actors:<Primary actor: actor that initiates the use case>
<Secondary actor: actor that participates within the use case>

Haoues, Sellami, Ben-Abdallah
Pre-condition:<a list of conditions that must be true to initialize the use case>
Post-condition (success):<state of the system if goal is achieved>
Post-condition (failure):<state of the system if goal is abandoned>
Relationship
[Include:<use cases in relation with this use case by "include">, Extend:<use cases in relation with this use case by "extend">,
Super use case:<list of subordinate use cases of this use case>, Sub use case: <list of all use cases that specialize this use case>]
Begin
MS /* Main scenario */
<Steps of the scenario from trigger to goal>
Begin
<NumAction> [<Pre-condition>] <Actor|System><Type: Request, DataPersist, Expletive, Response, DataRecovery><Action
Description> [<Int-Parameter>] [<Out-Parameter>]
End
AS /* Alternative scenario */
Begin<Event, begin at Num "action number">
The main scenario back to NUM
End
ES /* Error scenario */
Begin<Event, begin at Num "action number">
End
End Use Case

Apply these ideas to the ‘Resto-Sys’ case study
Add an Order
1. when the waiter logged on, the list of restaurant options is displayed
2. The waiter presses the option “Add a new order”
3. The system displays the list of tables
4. The waiter chooses the table where the customer is installed. The system
changes the order state from active to closed
5. The system displays the list of item families
6. The waiter chooses the item families based on the items requested by the
customer
7. The system displays the list of items according to the selected Item families
8. The waiter selects the items requested by the customer and specify for each
item the recommended quantity and customer's comment. The system creates a
new order
COSMIC Case Study – Sizing Natural Language/Use Case uses Resto-Sys as its
case study. (Downloadable from Cosmic Website) An example case is ...

Resto-Sys
Add an Order
Begin
1. <System><Expletive><when the waiter logged on, the list of restaurant options is
displayed>.
2. <Waiter><Request><The waiter presses the option “Add a new order”> [<Order Data>].
3. <System><Response & Request &Response ><The system displays the list of tables> [<Order
Data>] & [<Table Data>&<Table Data>].
4. <Waiter><Request & Response><The waiter chooses the table where the customer is
installed. The system changes the order state from active to closed > [<Table ID>&<Table ID>].
/*If a new order for the same table is entered, the previous order (if any) should be closed. In
this case, there is always at most one order ‘active’)*/
5. <System><Request & Response><The system displays the list of item families> [<Item family
data>&<Item family data>].
6. <Waiter><Request & Response><The waiter chooses the item families based on the items
requested by the customer> [<Item family ID>&<Item familyID>].
7. <System><Request & Response><The system displays the list of items according to the
selected Item families> [<Item Data>&<Item Data>].
8. <Waiter><Request& Response><The waiter selects the items requested by the customer and
specify for each item the recommended quantity and customer's comment. The system
creates a new order> [<Order Items>] & [<Order Items>].
End

Resto-Sys
Add an Order
Begin
1. <System><Expletive><when the waiter logged on, the list of restaurant options is displayed>.
2. <Waiter><Request><The waiter presses the option “Add a new order”> [<Order Data>].
3. <System><Response & Request &Response ><The system displays the list of tables> [<Order
Data>] & [<Table Data>&<Table Data>].
4. <Waiter><Request & Response><The waiter chooses the table where the customer is
installed. The system changes the order state from active to closed > [<Table ID>&<Table ID>].
/*If a new order for the same table is entered, the previous order (if any) should be closed. In
this case, there is always at most one order ‘active’)*/
5. <System><Request & Response><The system displays the list of item families> [<Item family
data>&<Item family data>].
6. <Waiter><Request & Response><The waiter chooses the item families based on the items
requested by the customer> [<Item family ID>&<Item familyID>].
7. <System><Request & Response><The system displays the list of items according to the
selected Item families> [<Item Data>&<Item Data>].
8. <Waiter><Request& Response><The waiter selects the items requested by the customer and
specify for each item the recommended quantity and customer's comment. The system creates
a new order> [<Order Items>] & [<Order Items>].
End

Resto-Sys – Conclusion (1)
The addition of Markup adds greatly to the burden of expressing stories. Will
requirements writers do this? Given that the benefit will be perceived as of value
only to the measurement community, it could be hard to sell.
Markup also reduces readability for those who are not trained in the language. I see
it is as intermediate language, translating from plain to structured text in a form to
enable automation.
I have automated the measurement of size of requirements expressed as Markup
languages in another domain, so I am sure it can be done in this case, but it is not a
priority for me.
This is just an illustration of the extra information that needs to be added when a
Markup approach is adopted, and the structure that has to be imposed to be able to
make it machine readable. It is included here to draw attention to other approaches
in this field of automation using COSMIC

Resto-Sys – Conclusion (2)
The authors of the paper indicate a software tool may be developed and perhaps
demonstrated at the next IWSM in China.
It will be interesting to see how it deals with semantics.
Note however, this problem of dealing with the semantics is not unique to Markup.
It is common to any other method of size measurement automation.

EARS and CUSS
The EARS approach relies on sentence structure, Keywords, Placeholders and
Vocabulary. The result is both Human and machine readable.
The CUSS approach is the same , but in addition it uses punctuation marks with
specific meanings, so is more complicated.
These will now be explained in more detail

EARS-FSM
EARS was originally devised for specifying REAL-Time systems in Marvin et al at Rolls
Royce. There is an active EARS community. The first International Conference and
Workshop is in Banff in August, co-resident with IEEE conferences
My main motivation for using EARS is to apply COSMIC to measure size of Real Time
systems specified using EARS.
I am working on specifying three case studies (an Elevator, the Rice Cooker and a
Liquid Mixing system) in EARS for training and demonstration purposes
To make it applicable to business systems, I have also compiled a Vocabulary specific
to the Data Rich Domain.
The syntax/vocabulary combination is called EARS-FSM, and this part of the
presentation describes how the EARS-FSM syntax/vocabulary is defined.
The same definition process is applied to EARS and CUSS. 29148 and marked-up
Use Case are not discussed further here.

EARS-FSM
The EARS Syntax comprises 5 basic boilerplates called Event Driven, Ubiquitous, State
Driven, Conditional and Complex. Any statement of requirement can be expressed
using a combination of these Boilerplates.
The term “Boilerplate” is used because that is the language of this community. It is
simply another word for Template
The Complex boilerplate is simply a container for combinations of the other four.
The process of defining any boilerplate is additive in that a Boilerplate item itself can
in turn require another Boilerplate to define it.
The full set of EARS-FSM Boilerplates has been completed and will be made available
for download for those interested in knowing more.
For this presentation, the Event Driven boilerplate is described in detail in order
to explain the process. The other boilerplates are defined in outline.

EARS-FSM
General Form
WHEN <optional preconditions> <trigger> the<system name> shall <system
response>
Event Driven
An event-driven requirement is initiated only when a triggering event is detected at
the system boundary. The keyword When is used for event-driven requirements.
EBNF
event-driven ::= 'WHEN' precondition+ Trigger 'the' SystemName 'shall' SystemResponse
Railroad Diagram

EARS-FSM
When. A keyword to indicate the type of pattern
Precondition. Optional. Condition bounded by IF .. THEN
Trigger. The task the User requires the system to perform
followed by the Username. The set of values for the
Trigger is defined in the Vocabulary
Event Driven

EARS-FSM
The Keyword indication start of the system name
SystemName. The system or application performing the task
Shall Keyword indicating the start of the mandatory System
Response
SystemResponse. What the system does in response to the task it has
been requested to perform by the User. The values for
SystemResponse is a set of domain-dependent
Keywords, each Keyword is further defined using its
own EBNF.
Event Driven

EARS-FSM
Get
Primary purpose is recover an object from persistent storage for
processing
EBNF and Railroad
get ::= 'get' ObjectName ('selector' ( ',' attribute)* )?

EARS-FSM
Get: the Keyword
ObjectName: The subject of the get, i.e. where information
about the object is stored
Selector: where applicable, the selection criteria
Attribute: the set of attributes describing the Object to be
retrieved

EARS-FSM
Ubiquitous
A ubiquitous requirement has no preconditions or trigger. It is not
invoked by an event detected at the system boundary or in response
to a defined system state, but is always active.
EBNF and Railroad
Ubiquitous ::= 'the' SystemName 'shall' SystemResponse

EARS-FSM
Unwanted Behaviour or Conditional
“Unwanted behaviour” is a general term used to cover all situations
that are undesirable. A “Conditional” state is only performed when the
pre-condition is satisfied
General Form
IF <optional preconditions> <trigger>, THEN the <system name> shall <system
response>
EBNF and Railroad
Unwanted ::= 'if' Precondition+ Trigger 'then' 'the' SystemName 'shall'
SystemResponse

EARS-FSM
Placeholder = Trigger
EBNF and Railroad
Trigger ::= (enters| requests)

EARS-FSM
State-driven
A state-driven requirement is active while the system is in a defined
state. The keyword While is used to denote state-driven
requirements.
General Form
WHILE <in a specific state> the <system name> shall <system response>
EBNF and Railroad
stateDriven ::= 'WHILE' state 'the' SystemName 'shall' SystemResponse

EARS – Conclusions (1)
• EARS is already used in the Real Time world. My research shows it can be adapted
for Data Rich domain, however the adaptation had to be formally specified. The
slides illustrate what was done to achieve this. The result is called EARS-FSM.
• The full EARS-FSM Specification is complete in draft form. This will be completed
shortly. It will be downloadable from the VisualFSM website and will be offered for
download from the Cosmic website
• As proof of concept, the EARS-FSM Specification has been successfully
implemented in a software tool called VisualFSM Autosize Workbench. This takes
an EARS-FSM conformant set of requirements, and automatically measures and
documents them.
• A free Community Edition of the software is in final preparation for release
Please see http://www.VisualFSM.com/cosmicautosize.asp.

EARS – Conclusions (2)
• This experience of implementation showed that It is reasonably straight-forward to
implement the EARS-FSM syntax and that the Semantic rules are easily derived.
The reason for this is the use of a CNL constrains the Syntax and Vocabulary used
making it easier for a parser to extract meaning
– E.g. the parser assumes the EARS reserved-word ‘show’ indicates the start of
an Exit’ statement ; reserved-word ‘enters’ indicates the start of an ‘Entry’
statement
• The EARS for Real Time (EARS-FSM-RT) is being specified and a module will be
added to the Workbench and an update will be made available.

CUSS – COSMIC User Story
Standard
• current requirements specification techniques used in agile
software development are customer-oriented and, from the
developers point of view, have proven to be insufficient
• there is more information from the customer point of view,
written in a too-high level, than from the developers perspective,
with some implementation details.
Miguel Ecar, Fabio Kepler, and Jo˜ao Pablo S. da Silva published a paper in
May 2018, available from the COSMIC website in which they conclude:
They present a template that captures more information than a normal User
Story. They call this the COSMIC User Story Standard (CUSS).

Standard
The CUSS template is the following:
“As a <who/role>, I want to <what>, linked to <connections>; so/then
be notified about operation status.”
where:
– <who/role> is the Functional User
– <what> is the verb that represents the action or the functional
process
– <connections> represents other data groups involved in this
functional process
– “so/then be notified about operation status” is optional
statement and represents the functional user feedback.

Standard
PLINK -> DG
SLINK -> COMMA PLINK | COMMA PLINK SLINK
TLINK => and DG
DOT -> .
COMMA -> ,
SEMICOLON -> ;
USER -> <;ist of roles>
METHOD -> <list of verbs/actions>
DG -> list of data groups
STRUCTURE and SYNTAX

Standard
– As a Manager, I want to remove a book.
– As a user, I can update books; so be notified about operation status.
– As a Manager, I want to add a new book, connected to author.
– As the Manager, I want to save books, connected to author and
publishing company.
– As a Manager, I want to create books, connected to author and
publishing company; then be notified about operation status.
Examples

Standard
CUSS
“As a <who/role>, I want to <what>, linked to <connections>; so/then be
notified about operation status.”
EBNF and Railroad
CUSS ::= "as a" role "i want to" action ("so"|"then") "be notified about" state

Standard
EBNF
As ::= 'as a' | 'as an' | 'as the'
Keyword AS

Standard
EBNF
role ::= ("user"|"manager"|"operator")
Placeholder Who/Role

Standard
EBNF
action ::= (register|create|update|list|send)
Placeholder Action

Standard
EBNF
state ::= (status|any)
Placeholder State

CUSS - Conclusions
CUSS is an extension to the traditional User Story designed to replace the human
consensus approach used in Agile (planning poker etc)
The additional information required by CUSS better reflects the intent of the
function better than traditional User Story conventions so less expert opinion is
involved. CUSS is still missing detail required for measurement and the remaining
semantics have to be assumed using expert opinion. If you change the experts, the
result may change due to different semantic interpretation by individuals.
If the Syntactical and Semantic analysis of the same input are automated using
CUSS it would result in a repeatable result. Although the semantic element is still
subject to expert opinion, as it is embedded, it is repeatable reflecting the
opinion of the original experts.
A tool with a different Semantic element may yield a different result. You need to
understand the Semantics embedded in the tool to be able to evaluate its
usefulness for the purpose.

Implementation
The final step is to consider Semantics. The Syntax imposes a structure, whereas
the Vocabulary constrains the Keywords and Words that can be used. Semantic
analysis makes sense of the whole.
The Syntax and Semantic rules have been successfully implemented in a software
tool I use for research purposes called VisualFSM Autosize-WB which I use for
research
EARS/CUSS/etc Measurement
Model
Autosize
Syntax Semantic
Syntax Model
DMSD
(size aCFP)
It takes in “well formed” requirements, performs Syntactical analysis to break out
the Syntax Model, then performs Semantic analysis to calculate the size.

VisualFSM-Autosize-WB
EARS Analysis. Text to Data Movement Sequence Diagram

Conclusions (1)
CUSS. The survey approach of gaining consensus of experts is only valid when a
group of local experts is used. The result is not transportable. CUSS overcomes this
by reducing the need for expert opinion by defining a syntax and requiring more
detail .
The syntax itself uses punctuation marks to convey meaning, this could be difficult
to enforce.
CUSS itself used the survey technique to evaluate the effectiveness of the syntax by
inviting expert opinion as the size of a set of CUSS requirements. The semantics
embedded in the proposed tool will depend on expert opinion.
As User Stories are used early in the life cycle where detail may be missing, CUSS
yields a more useful and repeatable result then wholly expert opinion.
Users should just be aware that the subjective element is still there. This is to be
expected given the information available

Conclusions (2)
EARS. the additional detail it contains compared to User Stories and CUSS yields
a result closer to the actual size. The semantic element still relies on expert
opinion but to a lesser extent because a Data Movement can be derived directly
from a statement with fewer assumptions.
The one sentence/one statement enforces the one object – one subject approach
which is accepted best practice. The ability to combine several different
statements means more complete and complex requirements can be defined.
The need for subjective assessment by human experts of what might be expected
to happen, due to lack of detail, is very much reduced.
The attention to detail in the EARS syntax for RT applications came about because
of the need to specify more clearly due to the consequence of error.
Business systems should really adopt the same approach and avoid the
consequences, which in this case is rework due to lack of clarity.
EARS-FSM is one way of enforcing this. IMO!!!

Conclusions (2)
IS 29148
Given that it is an International Standard, it is worth giving it some serious
consideration
I have done some preliminary work on the formal Specification of the Syntax
using the same method as for EARS and CUSS. From that work I am sure it can be
automated .
It uses the same strategy as EARS so I expect it to involve minor changes to adapt
to the different Syntax and hopefully minimal changes to the Semantic analysis.
A Module will be added to VisualFSM Autosize-WB in due course

Conclusions (3)
Finally
•CUSS can be used very early
•EARS when more information is available (or required by contract)
•29148 same as EARS but being an International Standard it could have more
exposure
•actual measurement of size when all is known which can be used to
validate and improve the automation results
Each can be automated and will yield a result that is better than the consensus of
Expert opinion.
Which approach to use depends on the purpose and when the information
becomes available Primarily it depends on the motivation for people to do it

Thank You
Questions
What is EBNF?
See slide (A1) which follows for a brief overview
How do I read a Railroad Diagram?
See slide (A2) that follows for a brief overview

(A1) -Extended Backus Naur Form
EBNF is a formal mathematical way to describe a language. I use it to define
the syntax because it is relatively easy to parse elements of the statement
described in this way.
It is too big a subject to cover in a few minutes but a simple example will give
an insight. Characters, symbols, quotes, punctuation and brackets have
specific meanings: Consider the definition of the
Get statement: get ::= 'get' ObjectName ('selector' ( ‘,' attribute)* )?
the GET statement is started by the keyword ‘get’ followed by the name of the
object. The opening and closing bracket () define a group and the final ? means
it is optional. ‘selector’ is a keyword followed by a group of attributes to filter
the selection; the * means it is a repeating group; the “,” means the attributes
are separated by a comma.
From this simple explanation you can imagine it is far easier to parse the EBNF
I have used Extended Backus-Naur Form (EBNF) to define Syntax in previous
slides.

(A2) Railroad Diagram
A Railroad Diagram (RRD) is a way of visualizing the processing paths of a
program, so-called because it is similar to moving carriages in a railway siding,
following all possible paths, left to right, to the destination.
Consider the GET Statement again
get ::= 'get' ObjectName ('selector' ( ‘,' attribute)* )?
Submitting this the Railroad Diagram generator we get
Reading left to right, follow the lines; these are processing paths. Where the
lines loop back, as before Attribute, it means go round until all attributes are
processed.
The combination of EBNF and RRD defines the structure of the parser required
to process the get statement. A parser is required for each statement
To supplement the EBNF statements, I use RRDs to validate the EBNF: the
Diagram Utility I use takes EBNF as input and outputs the RRD.

Measure COSMIC Size of Requirements Automatically

Recommended

Recommended

More Related Content

Similar to Measure COSMIC Size of Requirements Automatically

Similar to Measure COSMIC Size of Requirements Automatically (20)

Recently uploaded

Recently uploaded (20)

Measure COSMIC Size of Requirements Automatically

Editor's Notes