Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Accoun&ng  for  Non-­‐Func&onal  and  Project  
requirements  
in  so9ware  project  performance  measurement,  benchmarki...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Agenda
•  IntroducJon:	
  Why	
  the	
  need	
  for	
  standards	
  for...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
The  ac&vi&es  of  so9ware  project  performance  
measurement,  benchm...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Func&onal  size  measurements  are  used  
consistently  across  all  a...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
There  is  no  good  exis&ng  defini&on  of  
Non-­‐Func&onal  Requireme...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Next,  each  ac&vity  has  defined  its  own  
data  and  terminology  f...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Past  surveys  have  found  varying  
numbers  of  NFR  terms
•  ‘at	
 ...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Past  aSempts  to  measure  a  collec&ve  
size  of  NFR  are  now  dis...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Conclusions
•  There	
  is	
  no	
  common	
  understanding	
  in	
  th...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Agenda
•  IntroducJon:	
  Why	
  the	
  need	
  for	
  standards	
  for...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Words
What	
  do	
  we	
  mean	
  when	
  saying	
  
•  A	
  date	
  is...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
We  have  agreed  many  aspects  of  
NFR  and  PRC  over  the  past  y...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
We  have  clarified  the  terms    
Requirements  and  Constraints
Thesa...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
The  scope  of  requirements  and  
constraints
Requirements	
  and	
  ...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Data
Technology
Hardware/Software
System
Project
Environment
Other
Deli...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Requirements	
  for	
  a	
  
SoTware	
  System	
  Project	
  
Project	
...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
System	
  and	
  SoTware	
  Non-­‐FuncJonal	
  Requirements	
  (NFR)	
 ...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
COSMIC  and  IFPUG  defini&ons
FuncAonal	
  User	
  Requirements	
  (FUR...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
COSMIC  and  IFPUG  defini&ons
Non-functional requirement – COSMIC and I...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
COSMIC  and  IFPUG  defini&ons
	
  Project Requirements and Constraints ...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
67	
  Terms	
  
Examples
COSMIC	
  /	
  IFPUG	
  
DefiniJon	
  
NFR	
  s...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
COSMIC  and  IFPUG  strongly  
recommend  their  joint  Glossary
•  Dev...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Agenda
•  IntroducJon:	
  Why	
  the	
  need	
  for	
  standards	
  for...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
The  COSMIC  Guideline  builds  on  
the  joint  COSMIC/IFPUG  Glossary...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Requirements  that  first  appear  as  NFR  
may  evolve,  wholly  or  p...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
The  ‘addi&onal’  FUR  can  be  measured  by  
approximate  or  standar...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
The  Guideline  gives  examples  for  all  
Quality  NFR  how  they  ma...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
COSMIC  has  no  plans  to  develop  a  
collec&ve  size  measure  for ...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
1.  Collec&ve  size  measures  are  
common  and  o9en  valuable
Exampl...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
2.  There  is  a  large  number  and  variety  of  
NFR.  Many  overlap...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
3.  How  to  capture  the  effort  
associated  with  implemen&ng  NFR?
...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Instead,  we  propose  basic  sets  of  NFR/PRC  to  
record  and  meas...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
….  and  advise  how  to  deal  with  NFR  in  
performance  measuremen...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Guideline  Conclusions
The	
  COSMIC	
  ‘Guideline	
  for	
  dealing	
 ...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Agenda
•  IntroducJon:	
  Why	
  the	
  need	
  for	
  standards	
  for...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
FPA and SNAP covers functional and
non-functional requirements
PRC	
  –...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
PRC  affects  produc&vity  
and  not  size
•  Effort	
  depends	
  on	
  ...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
SNAP  
•  SNAP	
  idenJfies	
  fourteen	
  non-­‐funcJonal	
  characteri...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
SNAP  
•  Parameters	
  can	
  be	
  used	
  as	
  a	
  table	
  of	
  ...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
SNAP  sub  categories  are  independent  of  
the  way  NFR  are  class...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
SNAP  sub-­‐categories  are  independent  
of  the  way  NFR  are  clas...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Data  Func&ons  and  Transac&onal  
Func&ons  are  not  sufficient  to  s...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Example  1  –  The  requirements
NFR:	
  Improve	
  CRM	
  system	
  pe...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Example  1  -­‐  the  design
1.  Increase	
  the	
  virtual	
  memory	
...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Example  1  –  SNAP  count
•  Increasing	
  virtual	
  memory:	
  	
  
...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
IFPUG  Coun&ng  Process
Gather	
  available	
  
documentaJon	
  	
  
De...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Es&ma&on  and  benchmark
Es&ma&on
𝑬𝒇𝒇𝒐𝒓𝒕=
​[ 𝑷𝑫𝑹↓ 𝟏 (𝑷𝒓𝒐𝒋𝒆𝒄​ 𝒕↑′ 𝒔     ...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Agenda
•  IntroducJon:	
  Why	
  the	
  need	
  for	
  standards	
  for...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
COSMIC  and  IFPUG  have  come  
closer  over  the  past  year
Close	
 ...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Conclusions
Whilst	
  COSMIC	
  &	
  IFPUG	
  sJll	
  have	
  different	...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
Next  steps
•  Enhance	
  the	
  Glossary	
  to	
  take	
  into	
  acco...
Thank	
  you	
  for	
  your	
  
a^enAon	
  
	
  
Talmon	
  Ben-­‐Cnaan	
  (www.ifpug.org	
  );	
  
talmonbc@amdocs.com	
  ...
IWSM	
  MENSURA	
  
Krakow,	
  October	
  2015	
  
References
1.  ISO/IEC/IEEE	
  24765:2010	
  ‘Systems	
  and	
  soTware...
Upcoming SlideShare
Loading in …5
×

Accounting for non functional and project requirements - cosmic and ifpug development - talmon ben-cnaan - charles symons

525 views

Published on

Presentation from the IWSM Mensura 2015 conference held October 5-7 in Cracow, Poland

Published in: Software
  • Be the first to comment

  • Be the first to like this

Accounting for non functional and project requirements - cosmic and ifpug development - talmon ben-cnaan - charles symons

  1. 1. Accoun&ng  for  Non-­‐Func&onal  and  Project   requirements   in  so9ware  project  performance  measurement,  benchmarking  and  es&ma&ng:     COSMIC  and  IFPUG  developments   IWSM/Mensura  Conference,  Krakow,  October  2015   Talmon  Ben-­‐Cnaan  (IFPUG),  Charles  Symons   (COSMIC)  
  2. 2. IWSM  MENSURA   Krakow,  October  2015   Agenda •  IntroducJon:  Why  the  need  for  standards  for  Non-­‐ FuncJonal  Requirements  (NFR)  and  for  Project   Requirements  and  Constraints  (PRC)?   •  The  joint  COSMIC/IFPUG  development  of  a  Glossary   of  NFR  and  PRC  terms   •  The  COSMIC  Guideline  on  how  to  consider  NFR  and   PRC   •  The  IFPUG  SoTware  Non-­‐FuncJonal  Assessment   Process  (SNAP)  to  measure  NFR   •  Conclusions  and  next  steps   •  QuesJons  and  debate  
  3. 3. IWSM  MENSURA   Krakow,  October  2015   The  ac&vi&es  of  so9ware  project  performance   measurement,  benchmarking  and  es&ma&ng   need  consistent  data  and  terminology Measuring   project   performance Project   estimating Benchmarking Project data & benchmark repository Recording  &  measuring   software  system   project  requirements
  4. 4. IWSM  MENSURA   Krakow,  October  2015   Func&onal  size  measurements  are  used   consistently  across  all  ac&vi&es •  There  are  three  types  of  requirements  for  a   soTware  system  project   •  FuncJonal  User  Requirements  (FUR)   •  Non-­‐funcJonal  Requirements  (NFR)   •  Project  Requirements  and  Constraints  (PRC)   •  FuncJonal  size  measurement  methods,  e.g.  the   COSMIC  or  IFPUG  methods,  are  used  consistently   to  measure  the  size  of  FUR  across  all  acJviJes   But  what  about  NFR  and  PRC?  
  5. 5. IWSM  MENSURA   Krakow,  October  2015   There  is  no  good  exis&ng  defini&on  of   Non-­‐Func&onal  Requirements  (NFR) Example:   ISO  24756  definiJon  1):  “A  so%ware  requirement  that   describes  not  what  the  so%ware  will  do  but  how  the   so%ware  will  do  it.   Example:  so%ware  performance  requirements,  so%ware   external  interface  requirements,  so%ware  design   constraints,  and  so%ware  quality  a>ributes.”     Comment:  only  ‘design  constraints’  define  ‘how  the   soTware  will  do  it’  
  6. 6. IWSM  MENSURA   Krakow,  October  2015   Next,  each  ac&vity  has  defined  its  own   data  and  terminology  for  NFR  and  PRC  2) Requirements Recording (50) IEEE 804, ISO 9126 Wikipedia Requirements Sizing (36) VAF, TCA, SNAP Benchmarking (48) ISBSG, SEI Project Estimating (39) COCOMO II One common NFR (= “Interfaces”)
  7. 7. IWSM  MENSURA   Krakow,  October  2015   Past  surveys  have  found  varying   numbers  of  NFR  terms •  ‘at  least’  161  terms  (Chung  et  al  3))   •  122  terms  in  a  structured  hierarchy  (Saito  et  al  4))   •  108  terms  (Symons  2))   •  ISO/IEC  SQuaRE  5)  standard  lists  32  Quality  terms     A  complete,  standard  list  of  NFR  and  PRC  terms  may  be   impossible  
  8. 8. IWSM  MENSURA   Krakow,  October  2015   Past  aSempts  to  measure  a  collec&ve   size  of  NFR  are  now  discredited Albrecht’s  10  components  of  the  ‘Value  Adjustment  Factor’  6)              14  components  (IFPUG)  7)              19+  components  (MkII  FPA)  8)     …..  but  they  did  have  value  when  they  were  first  invented.    
  9. 9. IWSM  MENSURA   Krakow,  October  2015   Conclusions •  There  is  no  common  understanding  in  the  soTware   industry  of  what  is  meant  by  NFR   •  ExisJng  standards  for  NFR  and  PRC  are  not  coherent   •  These  issues  handicap  acceptance  of  methods  for:   •  quanJfying  requirements,   •  developing  project  performance  benchmarks,   •  project  esJmaJng.   These  were  the  drivers  for  COSMIC  and  IFPUG  to   develop  the  joint  Glossary  of  NFR  and  PRC  terms  
  10. 10. IWSM  MENSURA   Krakow,  October  2015   Agenda •  IntroducJon:  Why  the  need  for  standards  for  Non-­‐ funcJonal  Requirements  (NFR)  and  for  Project   Requirements  and  Constraints  (PRC)?   •  The  joint  COSMIC/IFPUG  development  of  a  Glossary   of  NFR  and  PRC  terms   •  The  COSMIC  Guideline  on  how  to  consider  NFR  and   PRC   •  The  IFPUG  SoTware  Non-­‐FuncJonal  Assessment   Process  (SNAP)  to  measure  NFR   •  Conclusions  and  next  steps   •  QuesJons  and  debate  
  11. 11. IWSM  MENSURA   Krakow,  October  2015   Words What  do  we  mean  when  saying   •  A  date  is  a  date.   •  Right!     •  It  is  not  my  type  
  12. 12. IWSM  MENSURA   Krakow,  October  2015   We  have  agreed  many  aspects  of   NFR  and  PRC  over  the  past  year •  Scope  of  the  Glossary  9)   •  Clarity  on:   •  ‘Requirements’  versus  ‘constraints’     •  The  ‘things’  that  NFR  and  PRC  apply     •  DisJnguishing  and  defining  NFR  and  PRC   •  Structuring  NFR  (aligned  with  SQuaRE)  and  PRC  to   classes  à  groups  à  terms   •  The  terms:   •  67  NFR  terms,  mainly  from  ISO  and  ISBSG  (27   COSMIC/IFPUG)   •  19  PRC  terms,  mainly  from  PMI  and  ISBSG  
  13. 13. IWSM  MENSURA   Krakow,  October  2015   We  have  clarified  the  terms     Requirements  and  Constraints Thesaurus:   Requirement:  “a  thing  demanded  or  obligatory”     Constraint:  “limitaJon  or  restricJon”   The  difference  is  not  always  clear:       •  A  requirement  that  the  so%ware  shall  use  C#  is   also  a  constraint   •  ‘Latency’  can  be  a  requirement  for  some  real-­‐Fme   systems  but  a  constraint  for  the  Mars  Rover   system   •  Staffing  skills  can  be  a  project  requirement  or  a   constraint   Constraints   Requirements   We  use  requirements  for  convenience.  We  use  constraints  only   when  it  is  helpful  to  disAnguish  constraints  from  requirements.  
  14. 14. IWSM  MENSURA   Krakow,  October  2015   The  scope  of  requirements  and   constraints Requirements  and  constraints  can  apply  to   •  The  soCware;     •  The  data  maintained  or  used  by  the  soTware;   •  The  technology  to  be  used,  e.g.  the  planorms;   •  Other  deliverables,  e.g.  documentaJon  or  training;   •  The  combined  hardware/soCware  system,  e.g.  a   response  Jme;   and  some  constraints  come  from  the  environment  
  15. 15. IWSM  MENSURA   Krakow,  October  2015   Data Technology Hardware/Software System Project Environment Other Deliverables Software Project,  System,  Product A  project:  a  temporary  endeavor  to  achieve  defined   objecJves  of  delivering  a  product  by  defined  dates.   Following  a  Project,  there  is  a  Product  in  place  (A  new   Product  or  a  Product  that  was  changed  by  the  project).     A  product  is  a  hardware/   soTware  system  or  an     item  of  soTware  such  as  a     soTware  package     Product  A   Product  C   Product  B  
  16. 16. IWSM  MENSURA   Krakow,  October  2015   Requirements  for  a   SoTware  System  Project   Project  Requirements   and  Constraints  (PRC)   System  and  SoTware   Non-­‐FuncJonal   Requirements  (NFR)   SoTware   FuncJonal  User   Requirements  (FUR)   System   Environment   Requirements   Technical   Requirements   Quality   Requirements   (SoTware  System)   Quality   Requirements   (Data)   Classifica&on  of  Requirements
  17. 17. IWSM  MENSURA   Krakow,  October  2015   System  and  SoTware  Non-­‐FuncJonal  Requirements  (NFR)   System  Environment   Requirements   Technical   Requirements   Quality   Requirements   (SoTware  System)   Quality   Requirements   (Data)   Classifica&on  of  NFR 1  Quality  of  the  data     maintained  by  the  soTware   2  System  performance   3  CompaJbility   4  Ease  of  use   5  System  reliability   6  Control    of  access   7  Maintainability   8  Ease  of  deployment   9  System  or  soTware  architecture  or  design   1  Context   2  ApplicaJon  Domain   3  ImplementaJons   4  User  Base   1  OperaJonal  Planorm   2  Database   3  OperaJonal  Planorm   constraints   4  Development   requirements   Class   Group   Type  
  18. 18. IWSM  MENSURA   Krakow,  October  2015   COSMIC  and  IFPUG  defini&ons FuncAonal  User  Requirements  (FUR)  -­‐  ISO/IEC  14143-­‐1    DefiniAon   Adopted  by  COSMIC  and  IFPUG   A  sub-­‐set  of  the  user  requirements.    Requirements  that  describe  what  the   soCware  shall  do,  in  terms  of  tasks  and  services.   NOTE:  FuncJonal  User  Requirements  relate  to  but  are  not  limited  to:   •  data  transfer  (for  example  Input  customer  data;  Send  control  signal)   •  data   transformaJon   (for   example   Calculate   bank   interest;   Derive   average  temperature)   •  data   storage   (for   example   Store   customer   order;   Record   ambient   temperature  over  Jme)   •  data   retrieval   (for   example   List   current   employees;   Retrieve   latest   aircraT  posiJon)  
  19. 19. IWSM  MENSURA   Krakow,  October  2015   COSMIC  and  IFPUG  defini&ons Non-functional requirement – COSMIC and IFPUG  Definition Any  requirement  for  a  soCware  system  or  for  a  soCware  product,   including  how  it  should  be  developed  and  maintained,  and  how  it  should   perform  in  operaAon,  except  any  funcAonal  user  requirement  for  the   soCware.     Non-­‐funcJonal  requirements  concern:   •  the  soTware  system  or  soTware  product  quality;         •  the  environment  in  which  the  soTware  system  or  soTware  product   must  be  implemented  and  which  it  must  serve;         •  the  processes  and  technology  to  be  used  to  develop  and  maintain  the   soTware  system  or  soTware  product,  and  the  technology  to  be  used   for  their  execuJon.  
  20. 20. IWSM  MENSURA   Krakow,  October  2015   COSMIC  and  IFPUG  defini&ons  Project Requirements and Constraints - Definition Requirements  that  define  how  a  soCware  system  project   should  be  managed  and  resourced,  or  constraints  that  affect   its  performance.     Requirements  may  include:   •  The  targets  the  project  should  achieve  (e.g.  budget,   delivery  date,  product  quality);   •  The  project  management  processes  that  should  be  used;   •  How  the  project  should  be  governed  and  resourced.   Constraints  may  include:   •  LimitaJons  on  the  project  resources;   •  Dependencies  on  other  projects  outside  the  control  of   the  project  concerned.  
  21. 21. IWSM  MENSURA   Krakow,  October  2015   67  Terms   Examples COSMIC  /  IFPUG   DefiniJon   NFR  sub-­‐type  (Quality,  System   Environment  Requirements,   Technical  Requirements)   Quality:                      9  groups   Environment:  4  groups   Technical:                4  groups   ISO  DefiniJon  
  22. 22. IWSM  MENSURA   Krakow,  October  2015   COSMIC  and  IFPUG  strongly   recommend  their  joint  Glossary •  Developed  through  >  20  iteraJons  over  a  year   •  Reviewed  by  many  experts  (academic  and   pracJJoners)  from  around  the  world   •  Available  for  free  download  from:   •  www.cosmic-­‐sizing.org     •  www.ifpug.org     •  We  welcome  feedback  and  comments  
  23. 23. IWSM  MENSURA   Krakow,  October  2015   Agenda •  IntroducJon:  Why  the  need  for  standards  for  Non-­‐ funcJonal  Requirements  (NFR)  and  for  Project   Requirements  and  Constraints  (PRC)?   •  The  joint  COSMIC/IFPUG  development  of  a  Glossary   of  NFR  and  PRC  terms   •  The  COSMIC  Guideline  on  how  to  consider  NFR  and   PRC   •  The  IFPUG  SoTware  Non-­‐FuncJonal  Assessment   Process  (SNAP)  to  measure  NFR   •  Conclusions  and  next  steps   •  QuesJons  and  debate  
  24. 24. IWSM  MENSURA   Krakow,  October  2015   The  COSMIC  Guideline  builds  on   the  joint  COSMIC/IFPUG  Glossary Contents   •  Terminology,  NFR/PRC   definiJons   •  ClassificaJon  of  NFR/PRC  terms   •  Glossary  of  NFR/PRC  terms   •  EvoluJon  of  NFR  in  a  project   •  How  to  deal  with  NFR/PRC  in   performance  measurement,   benchmarking  &  esJmaJng   •  Measurement  of  NFR  (interim)   Glossary9)   ü      ü      ü        Guideline10)   ü      ü      ü      ü      ü      ü       
  25. 25. IWSM  MENSURA   Krakow,  October  2015   Requirements  that  first  appear  as  NFR   may  evolve,  wholly  or  partly,  into  FUR Outline Funct- -ional Requts. Outline NFR Project Requts. & Constraints Require- ments Analysis Definition & Design Build, Test & Implement A r c h i t e c t u r e Approx. Funct- -ional Requts. Detailed NFR Imple- mented software system or software product Detailed FUR  ……  as  demonstrated  by  Al  Sarayreh,  Abran  and  others,  e.g.  [11]    
  26. 26. IWSM  MENSURA   Krakow,  October  2015   The  ‘addi&onal’  FUR  can  be  measured  by   approximate  or  standard  COSMIC  sizing Outline Funct- -ional Requts. Outline NFR Project Requts. & Constraints Require- ments Analysis Definition & Design Build, Test & Implement A r c h i t e c t u r e Imple- mented software system or software product Approx. Funct- -ional Requts. Detailed NFR Detailed FUR Size by analogy or expert judgement Approx. COSMIC size measurement Precise COSMIC size measurement
  27. 27. IWSM  MENSURA   Krakow,  October  2015   The  Guideline  gives  examples  for  all   Quality  NFR  how  they  may  evolve  as  a   project  progresses FuncAonality  that  may  evolve  from   the  NFR,  that  can  be  measured     ‘Middleware’  funcJonality  to  enable   portability  across  mulJple  DBMS  or  OS   or  hardware     Graphical  User  Interface  funcJons;   and   FuncJonality  to  assist  users,  e.g.  ‘Help’   funcJons     FuncJonality  to  import  external  data   in  real-­‐Jme  so  that  it  is  available  for   immediate  use   IniAal  NFR  term       Portability         Usability           Response  Jme         ‘Residual’  or  ‘Real’  NFR   statement     The  specific  environments   across  which  the  soTware   must  be  portable     The  specific  usability   requirements  (e.g.  ‘must  be   usable  by  public  with  no   training’)     -­‐  The  response  Jme  target;   -­‐  Specific  hardware;   -­‐  Low-­‐level  programming   language.   Examples:  
  28. 28. IWSM  MENSURA   Krakow,  October  2015   COSMIC  has  no  plans  to  develop  a   collec&ve  size  measure  for  NFR Inherent  difficulJes:   1.  How  to  form  a  collecJve  size  measure  that  will   add  pracJcal  value,  long-­‐term?   2.  The  large  number  and  variety  of  NFR   3.  How  to  collect  effort  data  related  to  NFR  when   NFR  evolve  into  FUR?  
  29. 29. IWSM  MENSURA   Krakow,  October  2015   1.  Collec&ve  size  measures  are   common  and  o9en  valuable Examples  of  valuable  collecJve  size  measures.  (All  are   mathemaJcally  invalid,  but  they  work.)   •  The  IFPUG  Value  Adjustment  Factor  (when  first  introduced)   •  Indices  that  affect  stock  market  senJment,  e.g.  the  PMI   •  Measures  of  ‘biological  age’   •  Total  Factor  ProducJvity   CondiJons  for  a  usable  collecJve  size  measure:   •  A  finite,  stable,  well-­‐defined  set  of  components   •  If  possible,  a  common  unit  of  measure  for  all  components   •  A  simple  formula  for  the  index  (avoid  complex  maths)  
  30. 30. IWSM  MENSURA   Krakow,  October  2015   2.  There  is  a  large  number  and  variety  of   NFR.  Many  overlap  in  meaning How  many  separate  NFR  should  we  include  in  a  collecJve   size  measure?  (10,  14,  19+,  67,  108,  122,  161+  …?)     Maintain-­‐   ability   Test-­‐   ability   Flex-­‐   ibility   Adapt-­‐   ability   Extend-­‐   ability   Evolv-­‐   ability   Modula-­‐   rity   Modifia-­‐   ability   Reus-­‐   ability    Port-­‐    ability  
  31. 31. IWSM  MENSURA   Krakow,  October  2015   3.  How  to  capture  the  effort   associated  with  implemen&ng  NFR? ….  given  that  NFR  evolve  as  a  project  progresses  into   FUR  …..  ?     ....  &  noJng  that  effort  to  implement  NFR  varies  with:   •  soTware  domain,   •  type  of  NFR,   •  the  parJcular  ‘soluJon’  for  the  NFR,   •  technology,  re-­‐use,  etc.  
  32. 32. IWSM  MENSURA   Krakow,  October  2015   Instead,  we  propose  basic  sets  of  NFR/PRC  to   record  and  measure  for  performance   measurement,  benchmarking,  es&ma&ng NFR and PRC Class Terms Quality NFR Response time, Transaction rate Security, Privacy Maintainability, Reusability Interfaces, Operational processing mode System Environment NFR Application type, sub-type Implementations (numbers of) Maximum number of concurrent users Technical NFR Operational platform type Programming language DBMS software Project Requirements and Constraints (PRC) Many, but not all of the 19 project requirements and constraints are worth recording. Example: if staff experience levels, processes and methods and tools are the same for all projects, then they need not be recorded for internal purposes.
  33. 33. IWSM  MENSURA   Krakow,  October  2015   ….  and  advise  how  to  deal  with  NFR  in   performance  measurement,   benchmarking  and  es&ma&ng •  EsJmate  and  measure  total  funcJonal  size,   including  funcJonality  that  evolves  from  NFR   •  Within  a  defined  ‘environment’,  i.e.   •  soTware  domain  (business,  real-­‐Jme)   •  soTware  architecture,  technology   •  project  type  (new,  enhancement,  etc)   ….  collect  and  record  data  on  ‘real’  NFR  and  PRC  so   that  you  can  make  like-­‐for-­‐like  comparisons   (This  is  normal  pracJce  for  benchmarking  acJviJes,   plus  a  few  addiJonal  parameters.)  
  34. 34. IWSM  MENSURA   Krakow,  October  2015   Guideline  Conclusions The  COSMIC  ‘Guideline  for  dealing  with  Non-­‐ FuncFonal  Requirements  in  project  performance   measurement  &  esFmaFng’   •  builds  on  the  joint  COSMIC/IFPUG  Glossary   •  provides  pracJcal  advice  on  how  to  deal  with   manageable  sub-­‐sets  of  NFR  and  PRC    and  will  be  available  in  October  for  free  download   from  www.cosmic-­‐sizing.org        
  35. 35. IWSM  MENSURA   Krakow,  October  2015   Agenda •  IntroducJon:  Why  the  need  for  standards  for  Non-­‐ funcJonal  Requirements  (NFR)  and  for  Project   Requirements  and  Constraints  (PRC)?   •  The  joint  COSMIC/IFPUG  development  of  a  Glossary   of  NFR  and  PRC  terms   •  The  COSMIC  Guideline  on  how  to  consider  NFR  and   PRC   •  The  IFPUG  SoTware  Non-­‐FuncJonal  Assessment   Process  (SNAP)  to  measure  NFR   •  Conclusions  and  next  steps   •  QuesJons  and  debate  
  36. 36. IWSM  MENSURA   Krakow,  October  2015   FPA and SNAP covers functional and non-functional requirements PRC  –  do  not  impact   soTware  size   NFR  –  measured   with  SNAP  Points   FUR  –  measured   with  FP   Guidelines  to   prevent  doubled   counJng  
  37. 37. IWSM  MENSURA   Krakow,  October  2015   PRC  affects  produc&vity   and  not  size •  Effort  depends  on  PRC  classes.  No  standard  /  industry   mathemaJcal  formula   •  Examples:  Project  type;  (same  size  different  effort)   •  Effort  may  be  formulated  for  some  PRC  (“producJvity   drivers”)   •  Examples:  Skill  level;    LocaJon;  Schedule  compression   (“crashing”)  (same  size  different  effort)   •  Effort  (or  deviaJon  from  esJmaJon)  can  be  explained   by  some    PRC   •  Examples:  Risk  that  was  materialized;  scope  creep  
  38. 38. IWSM  MENSURA   Krakow,  October  2015   SNAP   •  SNAP  idenJfies  fourteen  non-­‐funcJonal  characterisJcs   (“sub  categories”)  that  classify  the  way  NFR  are  met.   •  In  each  sub-­‐category,  the  size  is  assessed  within  a   counJng  unit  (SCU).  The  SCU  is  part  of  the  definiJon  of   the  sub-­‐category.   •  Non  funcJonal  size  is  determined  by  a  set  of   parameters  /  complexity  levels.  The  parameters  that   define  each  sub-­‐category  answer  the  quesJon  “what   factors  affect  the  effort  to  deliver  a  NFR”   •  Example:  The  effort  to  add  input  methods  (e.g.  barcode   reader,  fax  server,  reading  CSV  file)  with  same  funcJonality   depends  on  number  of  added  input  methods  
  39. 39. IWSM  MENSURA   Krakow,  October  2015   SNAP   •  Parameters  can  be  used  as  a  table  of  values  (e.g.   “High,  medium,  low”,  “<10,  10  to  15,  16+”)  or  as  a   mulJplier  (“SP  =  3*#  addiJonal  input  methods),  or   a  combinaJon  of  the  above   •  Once  the  parameters  are  assessed,  the  size  of  the   SCU  is  calculated.  SP  size  is  the  sum  of  the  SPs  of   the  SCUs  
  40. 40. IWSM  MENSURA   Krakow,  October  2015   SNAP  sub  categories  are  independent  of   the  way  NFR  are  classified  /  defined Accuracy   Performance   Ease of use / customer satisfaction   Response Time   Requirements   CharacterisJcs  
  41. 41. IWSM  MENSURA   Krakow,  October  2015   SNAP  sub-­‐categories  are  independent   of  the  way  NFR  are  classified Accuracy   Ease of use / customer satisfaction   Response Time   ISO9126   ISO25010   COSMIC/IFPUGGlossary   Requirements   CharacterisJcs   Time Behavior   Performance  
  42. 42. IWSM  MENSURA   Krakow,  October  2015   Data  Func&ons  and  Transac&onal   Func&ons  are  not  sufficient  to  size  NFR The  effort  to  deliver  NFR  is  also  affected  by •  Database  ac&vi&es •  Dealing  with  UI  proper&es •  Extensive  Logical  /  Mathema&cal  opera&ons •  Batch  processes  that  do  not  cross  the  boundaries   and  are  not  exposed  to  users •  Mul&ple  inputs/output •  …..  
  43. 43. IWSM  MENSURA   Krakow,  October  2015   Example  1  –  The  requirements NFR:  Improve  CRM  system  performance  for   “Retrieve  and  View  Customer  Details“-­‐  by  loading   the  ‘customer  details’  screen  in  3  seconds  or  less.   (Currently  it  takes  0.5  sec  to  6  sec.)     Problem  analysis:  The  system  is  slow  when  the  user  (call   center  representaJve)  opens  a  big  customer  with  many   products  and  many  interacJons.  It  also  happens  when  users’   virtual  memory  is  low  
  44. 44. IWSM  MENSURA   Krakow,  October  2015   Example  1  -­‐  the  design 1.  Increase  the  virtual  memory  of  ‘Windows’  to  maximum;   2.  Create  a  database  view,  which  includes  data  from  three  tables   (Assigned  Products,  TransacJons,  and  Payments)  (Assuming  30   different  DETs  are  fetched).     3.  Add  Logic  to  idenJfy  which  customers  will  be  loaded  to  the  new   view   4.  Add  a  field  to  ‘Customers’  table  to  indicate  whether  recent   informaJon  moved  to  the  new  view  and  add  logic  to  idenJfy  which   transacJons  will  be  copied  (not  seen  by  the  users,  hence  non-­‐ funcJonal  change).   5.  A  batch  process  runs  in  the  background  every  2  hours,  refreshing   this  view  with  large  and  most  acJve  users    
  45. 45. IWSM  MENSURA   Krakow,  October  2015   Example  1  –  SNAP  count •  Increasing  virtual  memory:     InstallaJon  of  a  larger  size  virtual  memory  is  a  hardware   configuraJon  and  not  a  soTware  change.  It  is  not  counted  using   soTware  sizing  methods.   •  CreaJng  a  database  view  (CUST_DETAIL_SUMMARY_  TEMP)   and  adding  an  indicator  (field)  to  ‘Customer’  table:       •  Count  SP  using  3.2  Database  changes;  SNAP  count  is  based  on   two  database  changes,  20-­‐50  DETs,  2-­‐5  RETs   •  Adding  a  batch  process:  (this  batch  job  does  not  cross  the   boundary,  hence  it  is  not  counted  using  FP)     •  Count  SP  using  3.3  Batch  Processes.  SNAP  count  is  based  on  30   DETs  and  3  FTRs   •  Adding  logic  to  meet  the  NFR:    Count  SP  using  1.2  Logical   and  MathemaJcal  OperaJons.  SNAP  count  is  based  on  3  FTRs,   type  =  ‘Logical’  and  10  DETs  used  for  the  added  logic  
  46. 46. IWSM  MENSURA   Krakow,  October  2015   IFPUG  Coun&ng  Process Gather  available   documentaJon     Determine  counJng   purpose,  scope,   boundaries  and   parJJons   IdenJfy  requirements   as  FUR,  NFR.  Separate   mixed  requirements   to  FUR  and  NFR   Measure  Data   FuncJons   Measure   TransacJonal   FuncJons   Associate  NFR  to  sub-­‐ categories   Calculate   funcJonal  size   Determine  SNAP   size  of  each  sub   category   Calculate  SNAP  size   Document   and  report   FuncJonal   Non-­‐FuncJonal  
  47. 47. IWSM  MENSURA   Krakow,  October  2015   Es&ma&on  and  benchmark Es&ma&on 𝑬𝒇𝒇𝒐𝒓𝒕= ​[ 𝑷𝑫𝑹↓ 𝟏 (𝑷𝒓𝒐𝒋𝒆𝒄​ 𝒕↑′ 𝒔     𝑪𝒉𝒂𝒓𝒂𝒄𝒕𝒆𝒓𝒊𝒔𝒕𝒊𝒄𝒔)× 𝑭𝒖𝒏𝒄𝒕𝒊𝒐𝒏𝒂𝒍   𝑺𝒊𝒛𝒆]+[​ 𝑷 𝑫𝑹↓ 𝟐 (𝑷𝒓𝒐𝒋𝒆𝒄​ 𝒕↑′ 𝒔     𝑪𝒉𝒂𝒓𝒂𝒄𝒕𝒆𝒓𝒊𝒔𝒕𝒊𝒄𝒔)× 𝑵𝒐𝒏 −𝒇𝒖𝒏𝒄𝒕𝒊𝒐𝒏𝒂𝒍   𝑺𝒊𝒛𝒆] PDR  –  ProducJvity  Delivery  Rate   Note:  Feedback  from  users  show  that  PDR2  diverges  in  3  of  the  14   sub-­‐categories.  NSFFC  will  reformulate  them  based  on  collected  data   Benchmarking •  Based  on  linear  regression  to  extract  PDR1  and  PDR2:     •  Two  different  baselines,  two  business  values  to  customers  
  48. 48. IWSM  MENSURA   Krakow,  October  2015   Agenda •  IntroducJon:  Why  the  need  for  standards  for  Non-­‐ funcJonal  Requirements  (NFR)  and  for  Project   Requirements  and  Constraints  (PRC)?   •  The  joint  COSMIC/IFPUG  development  of  a  Glossary   of  NFR  and  PRC  terms   •  The  COSMIC  Guideline  on  how  to  consider  NFR  and   PRC   •  The  IFPUG  SoTware  Non-­‐FuncJonal  Assessment   Process  (SNAP)  to  measure  NFR   •  Conclusions  and  next  steps   •  QuesJons  and  debate  
  49. 49. IWSM  MENSURA   Krakow,  October  2015   COSMIC  and  IFPUG  have  come   closer  over  the  past  year Close  exchange  of  ideas  has  led  to:   •  Agreeing  to  disJnguish  NFR  and  PRC   •  AdopJng  PMI  definiJons  for  PRC  terms   •  Many  refinements  to  the  classificaJon  structure   and  Glossary  contents  –  through  >  20  iteraAons   Further,  we  agree  that:   •  NFR  increase  the  size  of  the  soTware,  and  should   be  measured   •  Performance  measurement,  benchmarking  and   project  esJmaJng  must  consider  both  FUR  and  NFR  
  50. 50. IWSM  MENSURA   Krakow,  October  2015   Conclusions Whilst  COSMIC  &  IFPUG  sJll  have  different  views  on   how  to  measure  NFR,  we  have  collaborated  very  well   to  agree:   •  DefiniJons  and  classificaJon  of  NFR  and  PRC   •  The  Glossary  of  NFR  and  PRC  terms,  v1.0     We  strongly  recommend  these  to  the  soTware   engineering  community  
  51. 51. IWSM  MENSURA   Krakow,  October  2015   Next  steps •  Enhance  the  Glossary  to  take  into  account  ‘Data   Quality’  requirements   •  Enhance  the  Glossary  (and  define  measurements?)   to  take  into  account  ‘Other  Deliverables’  such  as   training  and  documentaJon       This  is  a  quesJon  to  YOU:  Should  we?????  
  52. 52. Thank  you  for  your   a^enAon     Talmon  Ben-­‐Cnaan  (www.ifpug.org  );   talmonbc@amdocs.com       Charles  Symons  (www.cosmic-­‐sizing.org  );   cr.symons@bJnternet.com    
  53. 53. IWSM  MENSURA   Krakow,  October  2015   References 1.  ISO/IEC/IEEE  24765:2010  ‘Systems  and  soTware  engineering  –  vocabulary’.   2.  Symons,  C.R.,  ‘AccounJng  for  Non-­‐FuncJonal  Requirements  in  ProducJvity  Measurement,  Benchmarking  and   EsJmaJng’,  UKSMA/COSMIC  InternaJonal  Conference  on  SoTware  Metrics  &  EsJmaJng,  London,  27/28   October  2011.   3.  L.  Chung,  B.  Nixon,  E.  Yu,  and  J.  Mylopoulos,  "Non-­‐funcJonal  Requirements  in  SoTware  Engineering,“  Kluwer   Academic  Publishing,  2000.   4.  Saito,  Y.,  Monden  A.,  Matsumoto  K.,  ‘EvaluaJon  of  non-­‐funcJonal  requirements  in  a  request  for  proposal   (RFP)’,  Nara  InsJtute  of  Science  &  Technology,  Japan,  at  InternaJonal  Workshop  on  SoTware  Measurement   (IWSM),  Nara,  2012..   5.  ISO/IEC  FCD  25010:  Systems  and  soTware  engineering,  –System  and  soTware  product  Quality  Requirements   and  EvaluaJon  (SQuaRE)  –System  and  soTware  quality  models     6.  Albrecht,  A.  J.,  ‘Measuring  applicaJon  development  producJvity’,  In  Proc.  of  the  IBM  ApplicaJons   Development  Symposium,  pp.  83-­‐-­‐92.  Monterey,  California  (1979)   7.  IFPUG,  CounJng  PracJces  Manual,  Release  4.3.1  –  Appendix  C   8.  Symons,  C.R.,  SoTware  sizing  &  esJmaJng:  Mk  II  FPA’,  John  Wiley  &  Sons,  1991,  ISBN  0  471  92985  9   9.  ‘Glossary  of  terms  for  Non-­‐FuncJonal  Requirements  and  Project  Requirements  used  in  soTware  project   performance  measurement,  benchmarking  and  esJmaJng’,  Version  1.0,  September  2015   10.  ‘Guideline  on  Non-­‐FuncJonal  &  Project  Requirements:  How  to  consider  non-­‐funcJonal  and  project   requirements  in  soTware  project  performance  measurement,  benchmarking  and  esJmaJng’,  version  1.0,   October  2015   11.  Khalid  T.    Al-­‐Sarayreh,  Alain  Abran    and  Juan  J.  Cuadrado-­‐Gallego,  ‘A  Standards-­‐based  model  of  system   maintainability  requirements’,  Journal  of  SoTware:  EvoluJon  and  Process,  2013,  Vol.  25,  no.  5,  pp.  459-­‐505    

×