Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
Emerging Adult Study
Next
Download to read offline and view in fullscreen.

Share

BA_FCaballero

Download to read offline

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Be the first to like this

BA_FCaballero

  1. 1. Bachelorarbeit Comparative evaluation of template-systems for requirements documentation Francisco Jos´e Caballero Cerezo September 12, 2016 Gutachter: Prof. Dr. Jan J¨urjens M. Sc. Katharina Großer Prof. Dr. Jan J¨urjens Institut f¨ur Softwaretechnik Institut f¨ur Informatik Universtit¨at Koblenz Universit¨atsstraße 1 56070 Koblenz https://rgse.uni-koblenz.de
  2. 2. Francisco Jos´e Caballero Cerezo francaballero@uni-koblenz.de Matrikelnummer: 215202495 Studiengang: Bachelor Informatik Pr¨ufungsordnung: PO2012 Thema: Comparative evaluation of template-systems for requirements doc- umentation Eingereicht: September 12, 2016 Betreuer: Prof. Dr. Jan J¨urjens M. Sc. Katharina Großer Prof. Dr. Jan J¨urjens Institut f¨ur Softwaretechnik Institut f¨ur Informatik Universtit¨at Koblenz Universit¨atsstraße 1 56070 Koblenz
  3. 3. i
  4. 4. ii Ehrenw¨ortliche Erkl¨arung Ich erkl¨are hiermit ehrenw¨ortlich, dass ich die vorliegende Arbeit selbst- st¨andig angefertigt habe; die aus fremden Quellen direkt oder indirekt ¨uber- nommenen Gedanken sind als solche kenntlich gemacht. Die Arbeit wurde bisher keiner anderen Pr¨ufungsbeh¨orde vorgelegt und auch noch nicht ver¨offentlicht. Koblenz, den September 12, 2016 Francisco Jos´e Caballero Cerezo
  5. 5. iii Abstrakt Die vorliegende Bachelorarbeit umfasst die Auswertung verschiedener Tem- plate-Systeme f¨ur die Anforderungsdokumentation. Die Auswahl der richtigen Template-Systeme beeinflusst den Erfolg jedes Softwareprojektes. Zun¨achst wurde eine theoretische Evaluation durchge- f¨uhrt, um die St¨arken und Schw¨achen der Template-Systeme zu analysieren. Um an der Anforderungsdokumentation mitwirken zu k¨onnen, wurde an- schließend eine Benutzeroberfl¨ache f¨ur Template-Systeme implementiert. Die vergleichende Evaluation wurde schlussendlich von einem Experiment begleitet, welches mithilfe von Industriefachpersonal und Studenten durchge- f¨uhrt wurde. Die Ergebnisse weisen darauf hin, dass Template-Systeme eine gute Verbesse- rungsstrategie f¨ur die Anforderungsdokumentation sind und MASTeR das passendere darstellt. This thesis deals with the evaluation of different template-systems for re- quirements specification. The selection of the right template-system influences the success of any software project. As a first step, a theoretical evaluation was conducted to analyse the strengths and weaknesses of the template-systems. Second, in order to assist in the requirements documentation, a user interface for template-systems was implemented. Finally, the comparative evaluation was supported through an experiment performed with industrial profes- sionals and students. The results suggest that template-systems are a good improvement for re- quirements specification and MASTeR would be the more suitable one.
  6. 6. iv Acknowledgements Firstly, I would like to express my sincere gratitude to Prof. Dr. Jan J¨ur- jens for his support and for providing me with the opportunity to write this Bacherlor’s thesis. I would also like to thank M. Sc. Katharina Großer for her useful comments, remarks and patience. Her guidance has helped me in all the time of research and writing of this thesis. My sense of gratitude to one and all, who directly or indirectly, have sup- ported me in this venture. Especially, I would like to thank everyone who has participated in the experiment. Without their help, the test could not have been successfully conducted. I also take this opportunity to thank my parents and my friends for sup- porting me from a distance. Lastly, I am also grateful to my partner who has backed me throughout the entire process by keeping me harmonious and encouraged. Este trabajo est´a dedicado a mi abuelo Paco, una persona que ha sido mi referente y ejemplo durante toda mi vida y que me ha hecho ser quien soy a d´ıa de hoy.
  7. 7. CONTENTS v Contents 1 Introduction 1 1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Targets of the Thesis . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.4 Structure of the thesis . . . . . . . . . . . . . . . . . . . . . . . 4 2 State of the Art in Requirements Specification 5 2.1 Requirements Engineering . . . . . . . . . . . . . . . . . . . . 5 2.1.1 Software Requirements . . . . . . . . . . . . . . . . . . 5 2.1.2 Requirement Engineering Processes . . . . . . . . . . 9 2.2 Requirements Specification . . . . . . . . . . . . . . . . . . . . 12 2.2.1 MASTeR . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.2.2 EARS . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 2.2.3 Planguage . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3 Comparative Evaluation 23 3.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.1.1 First Approach . . . . . . . . . . . . . . . . . . . . . . . 23 3.1.2 Second Approach . . . . . . . . . . . . . . . . . . . . . 25 3.1.3 Formula variations . . . . . . . . . . . . . . . . . . . . 28 3.2 Execution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.3 Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . . 30 4 User Interface for Template-Systems 32 4.1 Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 4.1.1 Technologies . . . . . . . . . . . . . . . . . . . . . . . . 32 4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 5 Users Experiment 38 5.1 Experiment design . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.1.1 Introduction section . . . . . . . . . . . . . . . . . . . . 39 5.1.2 Reading section . . . . . . . . . . . . . . . . . . . . . . 40 5.1.3 Writing section . . . . . . . . . . . . . . . . . . . . . . . 40 5.2 Test subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 5.3 Results and Analysis . . . . . . . . . . . . . . . . . . . . . . . . 41 5.3.1 Reading requirements . . . . . . . . . . . . . . . . . . . 42 5.3.2 Writing requirements . . . . . . . . . . . . . . . . . . . 44
  8. 8. CONTENTS vi 5.3.3 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . 46 6 Summary 48 A Checklists 49 A.1 Inspecting Quality . . . . . . . . . . . . . . . . . . . . . . . . . 49 A.2 Inspecting Formality . . . . . . . . . . . . . . . . . . . . . . . . 54 A.3 Inspecting Participants’ Requirements . . . . . . . . . . . . . 54 B Experiment 55 B.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 B.2 Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 B.3 Writing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Bibliography 59
  9. 9. LIST OF FIGURES vii List of Figures 2.1 Classification of requirements types, extracted from [Poh96]. 6 2.2 RE four processes, adapted from [Poh96]. . . . . . . . . . . . 9 2.3 Users of a requirements document, extracted from [Som11]. 11 2.4 Example of schema represented in Z notation, adapted from [WD96]. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.5 Requirements are specified by a combination of templates and attributes. . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 2.6 FunctionalMASTeR template, adapted from [RdS14b]. . . . 14 2.7 PropertyMASTeR template, adapted from [RdS14b]. . . . . 16 2.8 EnvironmentMASTeR template, adapted from [RdS14b]. . 16 2.9 ProcessMASTeR template, adapted from [RdS14b]. . . . . . 17 2.10 ConditionMASTeR template, adapted from [RdS14b]. . . . 17 2.11 LogicMASTeR template, adapted from [RdS14b]. . . . . . . 18 2.12 EventMASTeR template, adapted from [RdS14b]. . . . . . . 18 2.13 TimeMASTeR template, adapted from [RdS14b]. . . . . . . 19 3.1 Checklist-based inspection results (detailed). . . . . . . . . . 26 3.2 Results of the formula application for first and second vari- ations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 4.1 W3C HTML successful validation. . . . . . . . . . . . . . . . 34 4.2 W3C CSS successful validation. . . . . . . . . . . . . . . . . . 34 4.3 Main page, containing MASTeR and EARS introduction. . 34 4.4 MASTeR view, with the control panel. . . . . . . . . . . . . 35 4.5 How requirements can be specified with the tool. . . . . . . . 35 4.6 Glossary view, where terms can be defined. . . . . . . . . . . 36 4.7 Results can be exported at glossary view. . . . . . . . . . . . 36 4.8 The responsive design adapts the page layout to different resolutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 5.1 Component diagram of the web-based test. . . . . . . . . . . 39 5.2 The user interface developed in Chapter 4 was adapted to the web-based experiment. . . . . . . . . . . . . . . . . . . . . 41 5.3 Correctness of participants’ answers in reading section, grouped by notation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 5.4 How the test subjects evaluated the reading facilities. . . . . 43 5.5 Quality of participants’ requirements, grouped by notation. 44
  10. 10. LIST OF FIGURES viii 5.6 How the test subjects evaluated the writing facilities. . . . . 45 5.7 Comparison between the comparative evaluation and the ex- periment results. . . . . . . . . . . . . . . . . . . . . . . . . . . 46 B.1 Questionnaire about participant’s previous experience and occupation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
  11. 11. CHAPTER 1. INTRODUCTION 1 1 Introduction This Bachelor’s thesis belongs to the T-Reqs project, a project of the Soft- ware Engineering research group of the Institute for Software Technology (IST) at the University of Koblenz-Landau in collaboration with the Euro- pean Space Agency (ESA). The T-Reqs project is focused on technologies that can improve the quality of requirements relating to precision, accuracy and completeness by the use of ontologies. Ontologies are information models that make descriptions of concepts of the real word readable and accessible for humans and machines. Template-systems are semi-formal notations for requirements specification that support this ontology approach. The goal of this thesis is to select and evaluate template-systems. 1.1 Motivation In 1996, the ESPITI (European Software Process Improvement Training Initiative) project stated that one of the most important reasons for soft- ware projects’ cancellations, cost overruns or delays were related to poor requirements specifications [Eak96]. Semi-formal notations, like template-systems, make it possible to construct requirements following a plan (a template) and assigning a similar structure to each requirement. This approach helps to avoid mistakes from the early stages of the development process by specifying high-quality requirements that are cost and time efficient [RdS14a]. Notations for requirements specification have been studied for decades and many semi-formal notations have been designed and developed. Some of them are as follows: • Template-systems, like: – MASTeR (Musterg¨ultige Anforderungen - die SOPHIST Tem- plates f¨ur Requirements) [RdS14b]. – EARS (Easy Approach to Requirements Syntax) [MWHN09].
  12. 12. CHAPTER 1. INTRODUCTION 2 – Adv-EARS (Advanced-EARS) [MSKB11]. – Boilerplates [HJD05]. – Extensions to Boilerplates [Far12]. – RLS (Requirements Specification Language) of CESAR project [DSP10]. – Gherkin [WH12]. – Spider (Specification Pattern Instantiation and Derivation En- vironment) [KC05]. • Keywords-driven languages, like Planguage [Gil05] or Volatile [RR06]. • Controlled syntaxes, like ISO/IEC/IEEE 29148 syntax recommenda- tions [ISO11]. Every system provides different outputs, strengths and weaknesses. Before starting a project, an analysis to stipulate the most appropriate notation should be performed. This could get very complex depending on the num- ber of notations analysed and will greatly affect the results of the quality tests and the development process. 1.2 Targets of the Thesis The main target of this thesis is to select and compare different template- systems by evaluating which strengths and weaknesses they present over other templates and unrestricted natural language. A theoretical evaluation of the selected semi-formal notations will be per- formed. Later on, this will be analysed and contrasted by conducting an experiment on users. A second target is to develop a user interface for template-systems that can assist researchers and industry professionals in the usage of the notations. The required steps to accomplish this goals are as follows: 1. Search and select a variety of different template notations, by exam- ining the existing literature. 2. Conduct a theoretical evaluation of the selected notations, based on human readability, use of natural language, formality, expressiveness, etc.
  13. 13. CHAPTER 1. INTRODUCTION 3 3. Choose a selection of the most promising template-systems for the experiment, based on the results of the comparative evaluation. 4. Develop a User Interface for the selected systems. 5. Design, implement, execute and analyse the experiment. 6. Summarise and give an insight of the future work. 1.3 Related Work In recent years, many research about template-systems has been made. Some exemplary works are the one of Farfeleder [Far12] and Majumdar et al. [MSKB11]. The PhD conducted by Farfeleder contributed a framework for require- ments specification and requirements analysis. In there, Farfeleder pre- sented, among other things, an extension of the original Boilerplates set of Hull et al. [HJD05] that includes role and type attributes, allows several strings at one single attribute and defines a set of default attributes. Majumdar et al., for their part, proposed a syntax called Adv-EARS, based on EARS [MWHN09]. That proposal is focused on identifying the elements of use case diagram1 from which the latter would be generated automat- ically. They accomplished it by extending the existing requirements syn- taxes of EARS and also introducing a new requirement type (hybrid re- quirement). However, not many researches have been focused on comparing the differ- ent template-systems. Johannessen, in his Master’s thesis [Joh12], presented a comparison be- tween natural language and the Boilerplates of Hull et al. by carrying out a test with university students. His experiment concluded without a clear answer, but the results suggested that requirements’ errors are easier to find in Boilerplates. On the presentation of EARS [MWHN09], Mavin et al. expounded a case study that compared natural language requirements from the CS-E (Certifi- cation Specification for Engines) document and their EARS interpretation. Quantitative results for the case study were provided, which stated that EARS presents a number of advantages over the use of natural language. 1 ”A UML diagram that shows actors, use cases, and their relationships” [ISO10].
  14. 14. CHAPTER 1. INTRODUCTION 4 Nevertheless, no publication has been found by the author that selects and evaluates different template-systems. As mentioned in the beginning of the chapter, a selection of the most suitable notation is essential for the future of a project. This thesis is the first study in this area. It will evaluate the strengths and weaknesses of different template-systems based on metrics such as reading and writing facilities, expressiveness, etc. This evaluation will be tested and contrasted with the results of an experiment performed with test subjects (industry professional, scientific researchers and students). 1.4 Structure of the thesis After the introduction described in the current chapter, the document has the following structure: Chapter 2 This chapter will explain the state of the art of requirements specifications, from the basic concepts of Requirements Engineering, to the description of template-systems for requirements specification. Chapter 3 The comparative evaluation of the selected template-systems will be detailed in this chapter. Chapter 4 A user interface for the specification of requirements through template-systems will be developed. This chapter explains the aim and the results of this tool. Chapter 5 This chapter will describe the experiment design, the test sub- jects and the analysis of results of the test performed to evaluate the different template-systems. Chapter 6 The final chapter will contain a summary of the realized work and an insight about the future work.
  15. 15. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 5 2 State of the Art in Requirements Specification In this chapter, the bases for this thesis will be exposed. First, the basics of Requirements Engineering will be briefly described. After that, the re- quirements specifications techniques will be explained in more detail and the selected template-systems will be expounded and described. 2.1 Requirements Engineering Requirements Engineering (RE) refers to the process of discovering, analy- sing and verifying the requirements of a system [Som11]. The correct understanding (elicitation), documentation (specification) and validation of the customer needs is an essential part to make the system meet the user needs [Poh96]. Corrections on requirements after delivering the product are normally between 60 and 100 times more expensive than changes at early stages of the development [Dav95]. 2.1.1 Software Requirements A software requirement is a service or a constraint that has to be satis- fied by a system. At a high abstraction level, it is a property that must be demonstrated by a software in order to solve some problem in the real world [BF14]. According to ISO/IEC/ IEEE 24765 Systems and software engineering - Vocabulary [ISO10], a requirement is: ”A condition or capability that must be met or possessed by a system, system component, product, or service to satisfy an agreement, standard, specifica- tion, or other formally imposed documents”.
  16. 16. CHAPTER2.STATEOFTHEARTINREQUIREMENTSSPECIFICATION6 Figure 2.1: Classification of requirements types, extracted from [Poh96].
  17. 17. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 7 Many authors have proposed different classifications for requirements types. In 1996, Pohl [Poh96] stated a classification of requirements types, shown in Figure 2.1. According to his work, the different types of requirements are as follows: Functional Requirements Also known as capabilities or features, func- tional requirements represent the functions that the software has to provide (e.g., formatting text, modulating a signal...) [BF14]. They describe the services that the system should provide and how the system should react in particular situations. As Figure 2.1 shows, external interfaces for users, hardware and software express function- ality. Non-Functional Requirements Non-functional requirements, someti- mes known as constraints or quality requirements, do not describe functionalities, but properties that the system has to fulfill. They constrain the solution [BF14]. According to the classification of Pohl (Figure 2.1), they can be clas- sified as requirements of portability, maintainability, error situation, design constraints, error description, error handling, user supply, safety-security, test cases, performance, backup/recovery, cost con- straints, time constraints and documentation [Poh96]. Requirements Quality Criteria According to Wiegers [Wie99], a high-quality requirement should be: Correct The requirement should accurately describe the functionality to be delivered. In addition, there should not be any conflicts with other requirements of the corresponding system. A source of the requirement (e.g., a stakeholder1 or a higher-level specification) should be provided as a reference of correctness. Re- garding requirements inspections, representative users can determine the correctness of user requirements. Feasible It must be possible to implement the requirement according the capabilities and limitations of the system. In order to avoid infeasi- ble features (e.g., requirements technologically overly complex or too costly), domain experts should participate during the elicitation and 1 ”Individual or organization having a right, share, claim, or interest in a system or in its possession of characteristics that meet their needs and expectations” [ISO10].
  18. 18. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 8 the negotiation process (Section 2.1.2). Necessary The requirement should reflect something that the customer/- user really needs or something that is required for conformance to an external requirement, external interface, or standard. In the case that a requirement is not traceable back to its source, it may be not really necessary. Prioritized Each requirement should have an implementation priority. An agreement with the customer should be made in order to identify the most important requirements, being prepared for changes during development, budget cuts, schedule overruns or any other project management risk. An example of priority levels is the following: • High priority: The requirement must be in next product re- lease. • Medium priority: The requirement is necessary, but can be postponed to later release. • Low priority: The requirement is nice to have, but it may be dropped if there are insufficient time or resources. Modal verbs, as shall, should or will, are used to express liability (Section 2.2.1) and also provide priorities. Unambiguous Requirements should have only one possible interpreta- tion by the reader. Natural language is highly prone to ambiguity; for this reason, subjective words (such as user-friendly, easy, simple, rapid, efficient, several, improved, maximize, minimize...) should be avoided. The requirement should be written in a simple and straight- forward language of the user domain. Example: “The HTML Parser shall produce an HTML markup error report which allows quick resolution of errors when used by HTML novices.”. Ambiguity is produced by the word quick. Verifiable It must be possible to determine if the requirement is properly implemented in the product. A requirement needs to be correct, fea- sible, and unambiguous in order to be verifiable. This property can also be found in the literature named as testable.
  19. 19. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 9 2.1.2 Requirement Engineering Processes Many authors, like Bourque et al. [BF14], Sommerville [Som11] or Pohl [Poh96], have proposed different definitions for requirements engineering processes. In 1996, Pohl stated four tasks that were to be performed dur- ing the RE process: the elicitation, the negotiation, the specification, and the verification/validation of requirements. These fours tasks, the interac- tions and the relationship with the stakeholders, are presented in Figure 2.2. Figure 2.2: RE four processes, adapted from [Poh96]. Requirements Elicitation Requirement elicitation is the process of gathering information about the required system and the pre-existing systems. This process uses the in- formation obtained from stakeholders, documentation and specifications of similar systems to establish user and system requirements2 [Som11]. The interaction with participants is developed through elicitation tech- niques. The most common are: interviews, scenarios, prototypes, facili- tated meetings, observation and user stories [BF14]. 2 ”System requirements are more detailed descriptions of the software system’s func- tions, services, and operational constraints [...] It may be part of the contract between the system buyer and the software developers” [Som11].
  20. 20. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 10 As RE processes are iterative (Figure 2.2), the understanding of the re- quirements by the analysts team gets better with every round of the cycle. Requirements Negotiation Also known as conflict resolution, requirements negotiation concerns re- solving problems with requirements [BF14]. Some of the conflicts that can arise are related to: • Requirements and resources; • Functional and non-functional requirements; • Disagreements between stakeholders. It is important to make every decision traceable back to its source, in or- der to avoid the reoccurrence of these problems. During this negotiation process, tradeoffs can be agreed as alternatives. Requirements Specification & Documentation Often called the ”software requirements document”, the Software Require- ments Specification (or SRS) is an official statement which specifies every requirement that has to be implemented in the system [Som11]. As Fig- ure 2.3 shows, many different stakeholders like system customers, managers, system engineers, system test engineers or system maintenance engineers can access and use this document. While the requirements’ quality criteria (Section 2.1.1) are focused on in- dividual requirements, the IEEE Std 830-1998 Recommended Practice for Software Requirements Specifications [ISO98] extends the quality criteria from individual requirements to an SRS level with some new properties: Complete An SRS is complete if, and only if, it includes the following elements: 1. Every significant requirement relating to functionality, perfor- mance, design constraints, attributes, external interfaces and any external requirement imposed by a system specification. 2. Description of the software response to every valid or invalid class of input data in all realizable classes of situations.
  21. 21. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 11 Figure 2.3: Users of a requirements document, extracted from [Som11]. 3. Labels and references to every figure, table and diagram, as well as definitions of all terms and units of measure. Consistent An SRS is not consistent if it does not agree with some higher- level document or with itself. Furthermore, two or more requirements should not describe the same need. Ranked for importance and/or stability Each requirement of the SRS should have an identifier to indicate its importance or stability. Modifiable An SRS is complete if, and only if, any changes to the re- quirements can be made easily, completely, and consistently without changing the style or the structure. Traceable The origin of each requirement should be clear and it should facilitate the referencing of each requirement in future development or enhancement documentation. These quality criteria may vary according to the authors [Che13] [Ebe08], but each project should always state a systematic approach to follow in
  22. 22. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 12 order to guarantee high-quality requirements. Requirements Validation & Verification According to Boehm [Boe79], an informal interpretation of validation and verification is: Verification ”Am I building the product right?”. Validation ”Am I building the right product?”. This means that verification is focused on checking if a system matches its specification, while validation checks if the system presents the functional- ity expected by the customer/user. During the validation process, different types of checks, like validity, con- sistency, completeness, realism and verifiability checks, should be applied to the requirements document [Som11]. 2.2 Requirements Specification There are many possible notations that exist and could be used to write requirements. Natural language is the most flexible notation. It allows software require- ments to be written in a quick way, making them easy to read and under- stand by the reader. Nevertheless, there can be some problems that could arise, and therefore hamper the compliance of the quality criteria (Section 2.1.1). According to Mavin et al. [MWHN09], some of these problems are: Ambiguity Requirements could have more than one interpretation. Untestable Requirements could be unverifiable once the system is imple- mented. Inconsistency Several requirements could define the same need. Vagueness Requirements could present a lack of precision, detail or struc- ture. Complexity Some requirements could contain complex sub-clauses and/or several interrelated statements.
  23. 23. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 13 Omission Missing requirements, especially requirements to manage un- wanted behaviour. Wordiness Requirements could contain an unnecessary number of words. Inappropriate implementation details Requirements could describe a- bout how the system should be built, rather than what it should do. Requirements are often written in natural language; however, this may be supplemented by formal or semiformal descriptions [Som11]. Graphical notations, like the Unified Modeling Language (UML) [Lav03], and Formal notations, based upon elementary mathematics, can be used to specify precise, unambiguous and structured requirements. Some examples of formal notations are Event-B [Abr10], Alloy [Jac01] and Z [WD96]. Z is a mathematical language with a powerful structuring mechanism. An example of usage is presented in Figure 2.4. Figure 2.4: Example of schema represented in Z notation, adapted from [WD96]. In contrast to natural language, formal notations require a high level of training by the development team and the customers/users in order to write and understand requirements. Semi-formal notations, like template-systems and keywords-driven languages, are approaches that facilitates the satisfaction of the quality criteria (Sec- tions 2.1.1 and 2.1.2) specifying requirements by a combination of templates (syntax) and attributes (semantics), as shown in Figure 2.5. Three promising notations have been selected from the list presented in Sec- tion 1.1: MASTeR template-system, EARS template-system and PLan- guage keywords-driven language. Their widespread use in the industry, as well as there complexity and there documentation support, have made them valuable to be compared and evaluated on this thesis. This selection is described in more detail in the following sections.
  24. 24. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 14 Figure 2.5: Requirements are specified by a combination of templates and attributes. 2.2.1 MASTeR In 2014, Rupp et al. presented MASTeR (Musterg¨ultige Anforderungen - die SOPHIST Templates f¨ur Requirements) [RdS14b], a template-system that is currently used by many well-know german companies3. MASTeR allows writing requirement specifications through some basic types: functional and non-functional requirements, divided in property, environment and process requirements. Functional requirements Functional requirements can be specified with the FunctionalMASTeR tem- plate, shown in Figure 2.6. The uppercase words represent the template fixed values and the lowercase words with angle brackets represent the at- tributes that have to be filled in [RdS14b], as mentioned in [Gro15]. Figure 2.6: FunctionalMASTeR template, adapted from [RdS14b]. The elements are: 1. Condition: Optional conditions can be specified with the Condition- MASTeR, LogicMASTeR, EventMASTeR or TimeMASTeR tem- plates (described below in this section). 3 Details can be consulted at: https://www.sophist.de/referenzen/unsere-kunden/
  25. 25. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 15 2. System: Represents the system or component that should provide the functionality. 3. Liability: Modal verbs are used to express liability: (a) Shall: Statement is legal binding and mandatory. (b) Should: Statement is desired, but not mandatory. (c) Will: Statement is recommended, but not mandatory. 4. Activity type: The different types of system activities are the fol- lowing [RdS14a]: (a) Independent system action: (-) the system performs the process by itself. (b) User interaction: (PROVIDE <actor> WITH THE ABILITY TO) the system provides the user with some process functional- ity. (c) Interface requirement: (BE ABLE TO) the system performs the process dependent on a third factor. 5. Process verb: Represents the process: procedures (functionality) and actions to be provided by the system [RdS14a]. 6. Object: Characterizes the process word. It’s the answer to the ques- tion what? [RdS14a]. Example: The document editor shall provide the user with the ability to create new documents. Property requirements Property requirements describe properties of the system. PropertyMASTeR template (Figure 2.7) is proposed. The specific elements of the PropertyMASTeR template are: 1. Characteristic: Is the property of the subject matter. 2. Subject matter: Represents the system, sub-system, component, function, etc. 3. Qualifying expression: Specifies the range of the value.
  26. 26. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 16 Figure 2.7: PropertyMASTeR template, adapted from [RdS14b]. 4. Value: Is connected with the qualifying expression through the verb to BE. Example: The design of the website should be responsive. Environment requirements Technological requirements of the system’s environment are described with EnvironmentMASTeR template (Figure 2.8). The fixed values are designed in a way that make the requirement belongs to the system and not the en- vironment [RdS14b], as mentioned in [Gro15]. Figure 2.8: EnvironmentMASTeR template, adapted from [RdS14b]. Example: The charger of the device shall be designed in a way the system can be operated in a range 100-240V/50-60Hz. Process requirements Process requirements can be worded with the ProcessMASTeR template (Figure 2.9), which is based on the independent system action version of
  27. 27. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 17 the FunctionalMASTeR template. Process requirements are related to ac- tivities or legal-contractual requirements, as well as non-functional require- ments. In this template, the subject of the requirement is an actor and not the system [RdS14b], as mentioned in [Gro15]. Figure 2.9: ProcessMASTeR template, adapted from [RdS14b]. Example: The software developers should work according to the Personal Software Process (PSP). Conditions In the case that the functionality is only given or provided under certain logical or temporal conditions, the ConditionMASTeR template can be applied (Figure 2.10) [RdS14a]. Figure 2.10: ConditionMASTeR template, adapted from [RdS14b]. Example: If the server could not found the document requested. A more precise specification can be obtained by the use of LogicMASTeR, EventMASTeR or TimeMASTeR templates.
  28. 28. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 18 LogicMASTeR (Figure 2.11) is used to specify logical conditions. The log- ical statement is made through a compared object, an actor or a system [RdS14b]. Figure 2.11: LogicMASTeR template, adapted from [RdS14b]. With EventMASTeR (Figure 2.12), requirements are initiated as soon as the event condition is satisfied. The term event summarises the possible events that may affect the system [RdS14b]. Figure 2.12: EventMASTeR template, adapted from [RdS14b]. TimeMASTeR (Figure 2.13) is used to specify a certain period of time when a system or object may have temporary behaviours. Both, condi- tions and requirements, end at the same time [RdS14b]. 2.2.2 EARS EARS (Easy Approach to Requirements Syntax) is a template-system cre- ated by Mavin et al. [MWHN09] and currently used by companies like Intel Corporation or Rolls-Royce [Ter13].
  29. 29. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 19 Figure 2.13: TimeMASTeR template, adapted from [RdS14b]. The simplest structure of EARS system is the generic requirement syntax (shown in Listing 2.1). <optional preconditions> <optional trigger> the <system name> shall <system response> Listing 2.1: Generic requirement syntax. According to Marvin et al. [MWHN09], the elements are: 1. Precondition: Necessary condition to invoke the requirement. 2. Trigger: Event that initiates the requirement. 3. System response: Represents the system behaviour, which is acti- vated if and only if, the preconditions and trigger are activated. Mavin et al. stated that the generic requirement syntax is specialized into five types (Ubiquitous, Event-driven, Unwanted behaviours, State-driven and Optional features), described in the following subsections. Ubiquitous requirements An ubiquitous requirement defines a fundamental property of the system and has no preconditions or trigger. The format is shown in Listing 2.2. The <system name> shall <system response> Listing 2.2: Ubiquitous requirements format. Example: ”The software shall be written in Java” [Ter13].
  30. 30. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 20 Event-driven requirements An Event-driven requirement is activated only when a trigger occurs or is detected. It uses the keyword WHEN. Listing 2.3 shows its format. WHEN <optional preconditions> <trigger> the <system name> shall <system response> Listing 2.3: Event-driven requirements format. Example: ”When a DVD is inserted into the DVD player, the OS shall spin up the optical drive” [Ter13]. Unwanted behaviours Requirements for unwanted behaviours (failures, error conditions, distur- bances, deviations...) use IF and THEN keywords and have the format shown in Listing 2.4. IF <optional preconditions> <trigger>, THEN the <system name> shall <system response> Listing 2.4: Unwanted behaviours format. Example: ”If the memory checksum is invalid, then the software shall dis- play an error message” [Ter13]. State-driven requirements State-driven requirements are active while the system is in a specific state. They use the keywords WHILE or DURING (Listing 2.5). WHILE <in a specific state> the <system name> shall <system response> Listing 2.5: State-driven requirements format. Example: ”While the autopilot is engaged, the software shall display a vi- sual indication to the pilot” [Ter13]. Optional requirements Optional requirements are invoked only if the system includes a special fea- ture. They use WHERE keyword, as shown in Listing 2.6.
  31. 31. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 21 WHERE <feature is included> the <system name> shall <system response> Listing 2.6: Optional requirements format. Example: ”Where a HDMI port is present, the software shall allow the user to select HD content for viewing” [Ter13]. Complex requirements Requirements with complex conditional clauses can be defined with a com- bination of the keywords WHEN, IF and THEN, WHILE and WHERE. Example: ”When the landing gear button is depressed once, if the software detects that the landing gear does not lock into position, then the software shall sound an alarm” [Ter13]. 2.2.3 Planguage Planguage is a keyword-driven language created by Gilb in 1998 [Gil05]. The name Planguage is a combination of the words planning and language. Planguage provide a diverse set of keywords; the most common are shown in Table 2.1. In addition, Planguage is an extensible language. Sub-keywords can be used to add precision and specifity. An example is shown in Table 2.2. Quali- fiers add richness, precision, and utility by allowing precise description of conditions and events. Listing 2.7, adapted from [Sim01], shows an example of qualifiers usage. PLAN [Q1 00]: 20,000 units sold MUST [First year]: 120,000 units sold WISH [First release, entrp. version]: 1 Dec. 2000 PLAN [US market, first 6 months of production]: Defects Per Million < 1,000 METER [Prototype]: Survey of focus group METER [Release Candidate]: Usability lab data Listing 2.7: Example of qualifiers usage.
  32. 32. CHAPTER 2. STATE OF THE ART IN REQUIREMENTS SPECIFICATION 22 Keyword Definition TAG Unique and persistent identifier GIST Statement’s short and simple description STAKEHOLDER Person or organization affected by the requirement SCALE Scale of measurement used to quantify the statement METER Process or device used to measure using SCALE MUST Minimum level required to avoid failure PLAN Level at which is considered successful STRETCH Goal if everything goes perfectly WISH Desirable level of achievement PAST Previous results for comparison TREND Historical data or extrapolation of this RECORD Best-known achievement DEFINED Official definition of a term AUTHORITY Person, group or level of authorization Table 2.1: Planguage Keywords, adapted from [Sim01]. Keyword Definition METHOD Method for measuring to determine on the SCALE FREQUENCY Frequency of measurements SOURCE Responsible entity for making the measurement REPORT Where and when the measure will be reported Table 2.2: Sub-keywords for the METER Keyword, adapted from [Sim01].
  33. 33. CHAPTER 3. COMPARATIVE EVALUATION 23 3 Comparative Evaluation In this chapter, the comparative evaluation of the selected template-systems, from the preparation of the evaluation to the execution, results and analy- sis is described. 3.1 Preparation This evaluation compares different template-systems based on metrics. The- re exists many metrics that can be obtained from semi-formal notations. Some of them are measured by automated processes (e.g., length of the requirements), while other metrics can be measured by human revision of the notation’s syntax. Table 3.1 presents some of these metrics, a brief definition of them and the type of measurement. 3.1.1 First Approach In the first place, this evaluation will aim to look at a generic software project, a project that is not focused on any specific field and includes any type of requirement. In order to find the notation that is more suitable to specify requirements in that project, a first approach which attempts to identify the notation that provides higher-quality requirements is proposed. In this case, Suitability is defined as: Suitability = Quality Where the Quality attribute is measured by the satisfaction of the quality criteria (Sections 2.1.1 and 2.1.2). The template-system or keyword-driven language that provides higher-quality requirements would be the most suit-
  34. 34. CHAPTER 3. COMPARATIVE EVALUATION 24 Metric Description Measurement Number of words Length of the final requirements. Automated Number of process verbs Numbers of process verbs of the requirements. Automated Number of keywords Length of the keywords set. Automated Number of decision branches Number of possible branches in a template. Automated Expressiveness Capability of the notation to cover the domain problem. Inspection Formality Level of structuring of the notation. Inspection Human readability Capability of the requirements to be understood by humans. Inspection Computer readability Capability of the requirements to be analyzed and processed by computer programs. Inspection Quality Level of quality, according to the satisfaction of the quality criteria (Sections 2.1.1 and 2.1.2). Inspection Table 3.1: Notations’ metrics. able for the project. In order to measure the quality criteria compliance, a checklist-based in- spection can take place. To accomplish this, a requirements-focused check- list [Mos97] has been adapted to create a checklist which enables to measure the quality criteria satisfaction based on the notation’s syntax. The answers to these checks could be subjective, because they depend on the judgment or perception of the evaluator, so, in order to obtain accurate results, sev- eral people should perform this inspection. In the current work, the author has performed the inspection twice. Both, checklist and full results, are available in the section A.1 of Appendix A. The result of this inspection will be a number in the range between 0 and 100, which will indicate the level of quality that a notation provides, ac- cording to the compliance of the quality criteria. To add some semantic meaning to this value, a scale has been designed and is presented in Table 3.2. The results of the inspection carried out by the author are shown in Table 3.3. These results suggest that Planguage is the template-system which pro-
  35. 35. CHAPTER 3. COMPARATIVE EVALUATION 25 Level of Quality Quality total score Excellent 80 to 100 Good 60 to 79 Fair 40 to 59 Poor 20 to 39 Bad 0 to 19 Table 3.2: Scale for quality values. vides higher-quality requirements, followed closely by MASTeR and, lastly, by EARS, which presents a poor quality. However, subjectivity problems, previously discussed, may have affected the results of the inspection, so no significant difference between Planguage and MASTeR qualities should be interpreted. Notation Score Level of Quality MASTER 81 Excellent EARS 40 Fair Planguage 84 Excellent Table 3.3: Checklist-based inspection results. For a proper analysis of the results, a deeper insight into the checklist evaluation is needed. The check results corresponding to the main quality properties have been collected and displayed in Figure 3.1. These results in- dicate that the main difference established between EARS, Planguage and MASTeR is set by the completeness property, which influences other prop- erties like reliability. This could be attributed to the scope of EARS, which is basically focused on functional requirements. MASTeR and Planguage, for their part, present a more complete set of templates, which allows to specify more variety of non-functional requirements. 3.1.2 Second Approach The results of the first approach do not yet fulfill the aims of this evalua- tion, since it can not resolve issues such as: Can untrained stakeholders read and understand the specifications?. Can requirements engineers specify requirements without a significant amo- unt of training hours?. In order to measure the reading and writing facilities, a second approach is needed. To accomplish that goal, some of the metrics presented at the be-
  36. 36. CHAPTER 3. COMPARATIVE EVALUATION 26 Figure 3.1: Checklist-based inspection results (detailed). ginning of the chapter have been now selected: Expressiveness, (Average) Number of words and Formality. Expressiveness Refers to how well the notation’s syntax covers the do- main phenomena [HAL+14]. The purpose of this metric is to measure the level of expressiveness, which will affect positively to the final suitability. This metric is to be calculated by all the possible combinations that the notation can provide to cover the domain phenomena: Expressiveness = Ncombinations∗ * Measuring the number of combinations can depend on the nota- tion’s syntax. This is the reason why the author has not provided a specific formula to calculate this value. In the special case of a notation allows nesting1, only one level of nest- ing will be taken into account for the measurement. This is due to higher levels of nesting represent an exponential growth of the num- ber of combinations and, according to the author’s opinion, it will not properly represent the value of expressiveness and will distort the final result. The results of applying of the formula will be numeric, so, in order to get a better picture of them, a scale has been designed and is pre- sented at Table 3.4. Average Number of Words The Average Number of Words (NoW) in- dicates the average length of the requirements. For untrained stake- holders, too long requirements could be unreadable, while too shorts 1 Ability to specify requirements by combining different templates or keywords.
  37. 37. CHAPTER 3. COMPARATIVE EVALUATION 27 Expressiveness scale Expressiveness value Excellent >300 Good 200 to 300 Fair 100 to 199 Poor 0 to 99 Table 3.4: Scale for expresiveness values. requirements could not be complete. High values will affect negatively to the suitability. In this metric, an algorithm can be used to automate the count of the number of words, based on the next formula: NoW = ∑ n req=1 Nwords(req) n Formality The purpose of this metric is to measure the level of formaliza- tion that a notation presents (e.g., mathematical notations could be identified as high-formalized, while natural language could be consid- ered as low-formalized). High formality levels usually present a lack of flexibility and hardly can be understood by untrained stakeholders, so they affect negatively to the suitability. A low level of formality, however, could imply specification problems (Section 2.2). This metric will be measured by a checklist-based inspection. The checklist, partly based on [MR13] and [AYI04], and its application are available at the section A.2 of the Appendix A. The results of the inspection will be in the range between 1 and 6. For a better understanding of these results, the scale shown in Table 3.5 can be utilised. Formality scale Formality score Over-Formalized 6 Extremely-Formalized 5 Highly-Formalized 4 Well-Formalized 3 Fair-Formalized 2 Under-Formalized 1 Table 3.5: Scale for formality values. For this second approach, some of the metrics presented at the beginning of the chapter have been finally discarded. It is the case of the number of
  38. 38. CHAPTER 3. COMPARATIVE EVALUATION 28 process verbs, which can be obtained in notations like MASTeR, but is not obvious in notations like EARS or Planguage, due to their lack of atomicity. The number of decision branches, for its part, is to calculate the possible combinations. Finally, the computer readability has been remained out of the scope of this evaluation. This is a point that could be added to future researches. As a first approach, it could be measured taking into account the analysed formality and the atomicity that the notations provide. 3.1.3 Formula variations At this point of the evaluation, two positive metrics, Quality and Expres- siveness, and two negative metrics, Average Number of Words and Formal- ity have been identified. They can be used to identify the most appropiate template-system according to the properties of every particular project. However, there is still a lack of order (a formula) that can sort the nota- tions according to the suitability for a generic project. In order to achieve this goal, two variations are proposed: First variation Suitability = Quality ∗ Expressiveness NoW ∗ Formality In this variation, positive properties are multiplied and negative properties are divided. Second variation Suitability = Quality + Expressiveness − NoW − Formality In this second variation, positive properties are added and negative prop- erties are subtracted. Thus, our second approach is based on the idea that the template-system which provides requirements with higher quality and more expressiveness with less number of words and less formality would be the most appropiate one. Positive metrics contribute to the suitability while negative metrics penalize it. Both formula variations have been applied to the selected notations and the results are shown at Figure 3.2. Both provide the same sort: MASTeR is the most suitable notation followed by EARS and, lastly, Planguage.
  39. 39. CHAPTER 3. COMPARATIVE EVALUATION 29 Figure 3.2: Results of the formula application for first and second variations. However, no sufficient data is analyzed at the current evaluation that can support a decision of which variation to use. The evaluation will continue by using the first variation, but, as mentioned, the second variation would also be valid according to the current data. 3.2 Execution Summarising, the steps followed to run the formula have been: Quality A checklist-based inspection has taken place to collect the quality score for each notation. In order to accomplish it, a checklist based on [Mos97] has been used. Both the checklist and the results of the inspection are available in the section A.1 of Appendix A. Expressiveness All combinations that each notation can provide have been calculated. MASTeR’s combinations have been calculated by the combinations of conditions and decision branches. For EARS, combinations have been calculated according to all the decision bran- ches and one level of nesting. For Planguage, the number of keywords, as well as the qualifiers and the number of keywords that can be nested have been taken into account. Average Number of Words An algorithm has been used to count the number of words of the requirements. Some of the requirements have been created from scratch and others have been based on examples found in [Mav10] and [Sim11].
  40. 40. CHAPTER 3. COMPARATIVE EVALUATION 30 Formality A checklist, partly based on [MR13] and [AYI04], has been created to measure this metric. Both checklist and results of the in- spection are available at the section A.2 of the Appendix A. 3.3 Results and Analysis The results of the first formula variation, previously presented in Section 3.1.2, suggest that MASTER is the most suitable template-system, followed by EARS and Planguage. Extended information about the formula appli- cation has been collected. This is shown in Table 3.6. MASTeR EARS Planguage Avg Quality 81 40 84 68.33 Expressiveness 324 196 126 215.33 NoW 24 18.27 94.33 45.53 Formality 3 2 5 3.33 Suitability 364.50 214.46 22.43 200.76 Table 3.6: Detailed results of the formula application. These results suggest that MASTeR is the most ideal template-system, due to its excellent quality (a score of 81/100 points) in concise and well- structured requirements.. EARS, for its part, is the second notation at the results’s sort. The fair- formalized and concise specifications are compromised by a fair quality (40 points). Finally, results indicate that Planguage is the less ideal notation. This no- tation provides excellent quality, but it is highly penalized by the level of formality and the length of its specifications. Thus, as results indicate, MASTeR would be an ideal notation to specify requirements in a generic project due to the great possibilities and expres- siveness that this notation offers to cover the domain’ phenomena. On the other hand, if a project has mostly functional requirements, results mark EARS as an ideal option, due to its reading and writing facilities. Finally, if users are working on a critical project that is focused in the quality of their requirements leaving aside the ease of reading and writing, Planguage is suggested as an ideal option.
  41. 41. CHAPTER 3. COMPARATIVE EVALUATION 31 As mentioned, extended results of this evaluation can be utilised to select the most ideal notation according to the project’s properties. In addition, it is important to state that the judgement of the evaluator could have influenced a percentage of the measurements. In addition, not enough data has been analysed that can validate the formula. Due to this, an experiment will be conducted to test the template-systems usability on users (Chapter 5).
  42. 42. CHAPTER 4. USER INTERFACE FOR TEMPLATE-SYSTEMS 32 4 User Interface for Template-Systems In this chapter, the aim and the results of a software application developed for requirements specification are related. This tool allows users to write requirements using MASTeR and EARS template-systems. 4.1 Preparation The main purpose of this user interface is to assist in the experiment, pro- viding the test subjects with a tool where they can specify requirements using template-systems. In addition, this tool is also designed for scientific researchers that investigate template-systems and semi-formal notations. Finally, this tool can help to industry professionals who want to specify their requirements by the use of template-systems. The only public available MASTeR tool that has been found at the date of publication is the one developed by Eckey on his Master’s thesis [Eck15]. The tool that he presents is focused on analyzing the consistency of require- ments transformed from BPMN1 models. Regarding EARS and Planguage systems, no public available tool has been found by the author that can support the process of specification. The formality that MASTeR and EARS presented at the comparative eval- uation (Chapter 3) suggests that they are the easiest notations to learn by untrained stakeholders. For this reason, as well as the similarities between their complexity and format, MASTeR and EARS have been selected to be evaluated at the experiment (Chapter 5). This user interface will support both notations. 4.1.1 Technologies The user interface will be designed as a web-based application. The main advantage of this approach is the accessibility: the tool will be platform independent, accessible anytime and anywhere from any device which has 1 Business Process Modeling Notation [GDW09].
  43. 43. CHAPTER 4. USER INTERFACE FOR TEMPLATE-SYSTEMS 33 Internet connectivity and a compatible web browser. This helps to improve the coverage of the tool, increasing the number of users that can make use of the developed software. In order to accomplish this goal, the following technologies are used: Markup language HTML5 as the markup language to code the views of the application. Presentation layer The presentation of the documents is developed with LESS2, a CSS pre-processor. On the other hand, Bootstrap3, a CSS framework, will help to develop a responsive design, making the tool more user-friendly for a wider range of devices. Client-side scripting The controller will be coded with AngularJS4. In this JavaScript framework developed by Google, the models are spec- ified in JSON format, which can help to improve the interoperability with other tools. In addition, AngularJS facilitates an asynchronous communication with the server (AJAX-based) that improves the user experience. Storage HTML5 Web storage will be used to storage information at the client’s web browser. This will allow to save in memory all require- ments in which the user is working, although he closes the browser or turns off the device. 4.2 Results The source code of the developed application is available at the CD-ROM and at GitHub5. A public demo is also available online6. The markup’s validity has been checked according to the W3C (World Wide Web Consortium) Markup Validation Service7. Figure 4.1 shows the successful output of the validation. For its part, CSS code has been validated according to the W3C CSS Val- idation Service8 and the output of such validation is shown in Figure 4.2. 2 http://lesscss.org 3 http://getbootstrap.com 4 https://angularjs.org 5 https://github.com/francaballero/requirements_generator 6 https://userpages.uni-koblenz.de/~francaballero/tool/ and also http:// francaballero.net/requirements_generator/ 7 https://validator.w3.org 8 https://jigsaw.w3.org/css-validator/
  44. 44. CHAPTER 4. USER INTERFACE FOR TEMPLATE-SYSTEMS 34 Figure 4.1: W3C HTML successful validation. Figure 4.2: W3C CSS successful validation. Finally, JSHint tool9, a static code analysis tool for JavaScript, has been used to validate the JavaScript code. The main page of the application is shown at Figure 4.3. In this view, users are introduced to MASTeR and EARS template-systems by an explanation of their syntax and examples. Figure 4.3: Main page, containing MASTeR and EARS introduction. Then, users are able to navigate through four different views: MASTeR, EARS, Glossary and Results. At the MASTeR and EARS views, the tool provides users with the ability to specify requirements. A control panel is available on the right side of the page, as shown in Figure 4.4, where users can create new requirements (for MASTeR and EARS) and conditions (for MASTeR). When the user click on one of these buttons, he/she will be guided to the view shown in Figure 4.5. By clicking in the attributes, the user is able to 9 http://jshint.com
  45. 45. CHAPTER 4. USER INTERFACE FOR TEMPLATE-SYSTEMS 35 Figure 4.4: MASTeR view, with the control panel. select a term from the list of terms. The ”New Term” button (highlighted in green) is used to create new terms, which are shared between MASTeR and EARS views. Figure 4.5: How requirements can be specified with the tool. The glossary view is shown at Figure 4.6. In this view is where users are able to define their terms. At the right side, the control panel will allow, again, the creation of new terms. Results view provides users with the ability to save their work by printing or saving a PDF document containing all their progress. This is shown at Figure 4.7. In addition, users can remove terms and requirements from the local storage. On the other hand, the responsive design of the webpage adapts the page layout to different devices and resolutions. A sample of this functionality is shown at Figure 4.8.
  46. 46. CHAPTER 4. USER INTERFACE FOR TEMPLATE-SYSTEMS 36 Figure 4.6: Glossary view, where terms can be defined. Figure 4.7: Results can be exported at glossary view. Figure 4.8: The responsive design adapts the page layout to different resolu- tions.
  47. 47. CHAPTER 4. USER INTERFACE FOR TEMPLATE-SYSTEMS 37 This tool offers industry professionals and researchers the possibility of documenting requirements by the usage of MASTeR and EARS notations. Furthermore, this tool could be later improved by integrating a JSON- based interoperability with existing tools that automate the processing of requirements. Finally, EARS’s complex requirements could be improved by enabling multiple nesting levels.
  48. 48. CHAPTER 5. USERS EXPERIMENT 38 5 Users Experiment In this chapter, all the steps of the design, the test subjects selection and the analysis of results of the experiment will be described. 5.1 Experiment design The aim of this experiment is to test and evaluate the results of the compar- ative evaluation, described in Chapter 3, with real users. The experiment shall measure the reading and writing facilities of the template-systems. To accomplish this goal, the test subjects will be asked to solve tasks related to requirements specification and provide some feedback about their expe- rience. The test group will be composed of industry professionals and students. The experience of industry professionals makes them ideal test subjects for this evaluation. However, it will not be possible to recruit a significant number of participants. This lack of participants will be compensated with students of advanced years of Software Engineering, Computer Engineering, Computer Science and Web Science. This test group will have presumably less experience in Requirements Engineering, so the expectations for their performance should be adjusted. However, this may actually be considered as an ad- vantage at measuring the template-systems’ learnability. In addition, their lack of experience may affect similarly to both, template-systems and nat- ural language, making their results more reliable. It is important to state that it will not be possible to perform the experiment in a laboratory. In addition, the test group will not homogeneous, because the test subjects will have different backgrounds and experience. At last, the test is not planned to be performed by a really significant number of users. All these reasons make the sample not representative enough and it should be taken as a first approximation that can be improved in the future. In order to accomplish all these goals and reach as many participants as possible, the test is planned to be web-based. This approach will also allow
  49. 49. CHAPTER 5. USERS EXPERIMENT 39 the use of algorithms to automate part of the analysis of the results. The same technologies described in Section 4.1.1 will be used to implement the experiment. Moreover, the user interface designed for MASTeR and EARS (Chapter 4) will be integrated. However, a database is now needed to store and consult the results. For this purpose, OpenWS1, a web-service that provides a REST API to store JSON objects, will be used. The compo- nent diagram of Figure 5.1 shows the physical structure of the application. Figure 5.1: Component diagram of the web-based test. The source code of the web-based experiment is available at the CD-ROM and at GitHub2. A public demo of the application is also available online3. The experiment operates with the requirements of NBDiff, an open source project that provides a means of viewing changes made to IPython Note- book files and resolving merge conflicts. The requirements of the project4 have been changed and adapted for the use of the experiment. The test consists of three sections, introduction, reading and writing, de- scribed in the following sections. 5.1.1 Introduction section First of all, at the introduction section (Appendix B.1), test subjects will be asked to complete a short questionnaire about their occupation and their 1 https://openws-app.herokuapp.com 2 https://github.com/francaballero/requirements_user_testing 3 https://userpages.uni-koblenz.de/~francaballero/ and also http://francaballero.net/requirements_user_testing/ 4 http://nbdiff-docs.readthedocs.io/en/latest/SRS.html
  50. 50. CHAPTER 5. USERS EXPERIMENT 40 previous experience, used for statistics purposes. Next, if the test sub- ject has indicated that he/she has no previous experience on MASTeR or EARS, he/she will be conducted to a video that explains the aim and usage of the template-systems. This explanation can be reviewed in text-format or can be extended with the mentioned references. Later on, the NBDiff system is briefly described and the test subjects are asked to send any possible question to the author before proceeding with the reading section. 5.1.2 Reading section At the reading section (Appendix B.2), participants will be asked to an- swer questions regarding some requirements that they shall read. Two of the requirements presented have been written using MASTeR, another two have been written using EARS and one requirement have been expressed in natural language. Questions are related to quality criteria’ properties (Sections 2.1.1 and 2.1.2) like completeness, ambiguity and inconsistency. No definition of the terms has been provided. This will enable to see and measure the need of a glossary. At the end, they will be asked to complete a questionnaire about their ex- perience with the template-systems’ readability. For this, five statements are presented for which they have to decide between: strongly disagree, dis- agree, agree or strongly agree. This four-options questionnaire is designed in a way the test subjects are forced to make decisions. The test subjects could learn from reading and evaluating these require- ments. This is another of the purposes of the reading section, so they will be able to perform the writing section. 5.1.3 Writing section In this section of the test (Appendix B.3), the test subjects will be intro- duced to the tool designed in Chapter 4 through a video tutorial. Later on, they will be asked to implement requirements using the tool, which is integrated in the test (Figure 5.2). The tasks are introduced through four short descriptions of what the requirements should represent. Participants will be able to use MASTeR, EARS or natural language, de- pending on the restrictions of each task. Some terms are already predefined in order to encourage the use of the template-systems. This part of the ex- periment is designed in such a way the subjects should use MASTeR and
  51. 51. CHAPTER 5. USERS EXPERIMENT 41 EARS at least once. Figure 5.2: The user interface developed in Chapter 4 was adapted to the web-based experiment. At the end, a questionnaire is displayed and the test subjects are asked to complete it according to their experience. 5.2 Test subjects The experiment was carried out between between August 23 and Septem- ber 9, 2016. The link of the test5 was spread in scientific groups, working groups of ESA and students groups. A total of 20 participants completed the test: 5 industry professionals and 15 students. The background of the participants is shown in Table 5.1. It is important to emphasise that the large majority of participants, espe- cially the students group, were not native English speakers, so this may have influenced the reading and writing of requirements in English. 5.3 Results and Analysis The data obtained from the experiment has many forms. Some algorithms can be applied to process part of the data. However, the the answers of the reading and writing sections are not easily processable and they should be manually evaluated to judge the correctness. 5 https://userpages.uni-koblenz.de/~francaballero/
  52. 52. CHAPTER 5. USERS EXPERIMENT 42 Experience in RE Experience in template systems Experience in MASTeR Experience in EARS Industry professionals 60% 40% 0% 0% Students 20% 0% 0% 0% Total average 40% 10% 0% 0% Table 5.1: Information about the test subjects’ background. 5.3.1 Reading requirements At the reading section, test subjects answered five questions related to five requirements that they read. The questions had multiple answers that could be considered as correct, so they had been manually evaluated by the author of this thesis. The results of the evaluation have been grouped by notation and listed in Figure 5.3. Participants completed 92% of the questions. The 77% of the answers related to MASTeR requirements were correct, the 88% related to EARS were correct and the 55% related to natural language were correct. Figure 5.3: Correctness of participants’ answers in reading section, grouped by notation. At the same time, response times to every question were collected. The average times have been calculated and grouped by notation. The results are shown at Table 5.2. MASTeR and natural language answers took 1.37 times and 1.20 times longer, respectively, than EARS. These results also show how industry professionals took 2.42 times longer than students to answer the questions, which could have highly influenced to their better
  53. 53. CHAPTER 5. USERS EXPERIMENT 43 performance on this section of the test. Notation MASTeR EARS Natural language Industry professionals 4.44 mins 3.96 mins 1.80 mins Students 1.42 mins 0.808 mins 1.96 mins Total of participants 2.18 mins 1.59 mins 1.92 mins Table 5.2: Average response times to reading questions, grouped by notation. Finally, the participants gave their feedback about their experience at read- ing requirements. The results are listed in MASTeR Figure 5.4. Figure 5.4: How the test subjects evaluated the reading facilities. Thus, these results suggest that MASTeR and EARS provide more read- ing facilities than natural language. This is supported by the participants performance and feedback. However, it should be considered as just a tendency, due to the participants not solving enough natural requirements tasks in this test. Regarding MASTeR and EARS, EARS has got a better performance rate averaging less time, but MASTeR has got a better feedback from the par- ticipants. At this point of the analysis, both notations present a similar
  54. 54. CHAPTER 5. USERS EXPERIMENT 44 readability rate and no significant differences can be identified between them. In addition, the manual evaluation of the answers has served to support the hypothesis that the existence of a correct glossary would have significantly increased the quality requirements. 5.3.2 Writing requirements At the writing section, participants wrote requirements to specify four func- tionalities of the NBDiff system. Due to the nature of the template-systems, the analysis could have been automated. However, in this analysis, the an- swers have been manually evaluated by the author. The evaluation criteria was based on six quality properties: ambiguity, priority, vagueness, com- pleteness, absence of implementation details and verifiability, as well as on the correct use of the notation’s syntax. The checklist used for this purpose is available in the section B.3 of Appendix B. The 76% of the proposed tasks were completed by the participants. Re- sults of the evaluation based on 7 quality points are shown at Figure 5.5. It shows how the quality of MASTeR requirements was 1.48 times higher than EARS’s and 3.43 times higher than natural language’s. In the tasks where participants could decide which notation to use, the 57.5% chose MASTeR, the 10% chose EARS, the 7.5% chose natural language and the 25% did not solve the task. Figure 5.5: Quality of participants’ requirements, grouped by notation.
  55. 55. CHAPTER 5. USERS EXPERIMENT 45 The average writing times have been collected and grouped in Table 5.3. This shows how writing requirements using MASTeR took to participants 1.39 times longer than using EARS and 1.5 times longer than using natural language. Once again, the results indicate how industry professionals spent more time (1,23 times more) to specify better requirements. Notation MASTeR EARS Natural language Industry professionals 14.06 mins 6.54 mins 0.46 mins Students 8.02 mins 2.83 mins 6.35 mins Total of participants 8.84 mins 6.35 mins 5.89 mins Table 5.3: Average writing times, grouped by notation. According to the experience that participants have gained during this sec- tion of the test, they gave the feedback collected in Figure 5.6. Figure 5.6: How the test subjects evaluated the writing facilities. The results indicate a significant difference between MASTeR writing fa- cilities and the facilities the other notations provide. Participants have specified their best requirements using MASTeR, which also has the best feedback rate. However, test subjects took longer to specify requirements
  56. 56. CHAPTER 5. USERS EXPERIMENT 46 using this notation. This suggests that some interest and training from part of the participants is necessary to specify high-quality requirements. On the other hand, the results suggest that EARS also significantly im- proves the specification of requirements. However, the overall requirements written by students contained many mis- takes and not all of them managed to write all the solutions. Part of this results can be attributed to the complexity of the test and their lack of experience specifying requirements. This again suggests that some level of training is needed to specify high-quality requirements. 5.3.3 Conclusions The results of the experiments have not indicated any significant difference on notations readability. It is not so with the ease of writing, which has presented a significant improvement by using MASTeR. The collected information about answers, response times and feedback has been combined and scaled to be comparable with the comparative evalua- tion described in Chapter 3. This comparison is shown at Figure 5.7. Figure 5.7: Comparison between the comparative evaluation and the experi- ment results. The scaled results of the comparative evaluation and the experiment results show a similar tendency: MASTeR is more suitable than EARS. As it is
  57. 57. CHAPTER 5. USERS EXPERIMENT 47 said, some factors, like a sample that is not significant enough and the lack of experience of part of the test subjects, could have influenced to the results of the experiment. However, it is important to emphasise that the results of the comparative evaluation and the experiment coincide in the same tendency.
  58. 58. CHAPTER 6. SUMMARY 48 6 Summary The purpose of this thesis was to evaluate template-systems for require- ments specification. The results of a theoretical evaluation and an ex- periment conducted with industrial professional and students suggest that MASTeR is the notation that provides higher-quality specifications and the best reading and writing facilities. However, this should be considered as a tendency. The theoretical evalua- tion needs more information to be contrasted with an experiment that has not been performed with a significant amount of experienced users. The current work is the first one in this field and can be taken as a first ap- proximation that could be improved upon. More notations can be analysed and more experienced test subjects should perform the experiment.
  59. 59. APPENDIX A. CHECKLISTS 49 A Checklists In this chapter of the appendix, the checklists utilised during the compar- ative evaluation are expounded. A.1 Inspecting Quality The following Table shows the checklist used to evaluate the satisfaction of the quality criteria. Results are presented in columns 2, 3 and 5, represent- ing MASTeR, EARS and Planguage scores, respectively. Quality Checklist. MASTeR, EARS, Planguage scores Clarity Can all requirements be written in non-technical understandable language? 1 1 1 Can any requirement have more that one interpretation? 2 2 2 Can be each characteristic of the final product described with an unique terminology? 3 3 3 Is there a glossary in which the specific meaning(s) of each term can be defined? 4 4 4 Could the requirements be understood and implemented by an independent group? 5 5 5 Completeness Can be all terms be defined? 6 6 6 Can all units of measure be defined? 7 7 6 Can the missing information be defined in the requirement? 7 7 7 Should any requirement be specified in more detail? 7 7 8 Should any requirement be specified in less detail? 8 8 9 Can all of the requirements be included? 8 8 9 Can all the requirements related to functionality be included? 9 9 10 Could be any requirement which make you feel uneasy? 10 10 11 To be continued on next page
  60. 60. APPENDIX A. CHECKLISTS 50 Quality Checklist. MASTeR, EARS, Planguage scores Can all requirements related to performance be included? 11 10 12 Can all requirements related to design constraints be included? 12 12 13 Can all requirements related to attributes be included? 12 12 14 Can all requirements related to external interfaces be included? 13 12 15 Can all requirements related to databases be included? 14 12 16 Can all requirements related to software be included? 15 12 17 Can all requirements related to communications be included? 16 12 18 Can all requirements related to hardware be included? 17 12 19 Can all requirements related to inputs be included? 18 12 20 Can all requirements related to outputs be included? 19 12 21 Can all requirements related to security be included? 20 12 22 Can all requirements related to maintainability be included? 21 12 23 Can all requirements related to installation be included? 21 12 24 Can all requirements related to criticality be included? 22 12 25 Can all requirements related to the permanency limitations be included? 23 12 26 Can all possible changes to the requirements be specified? 23 12 27 Can be the likelihood of change be specified for each requirement? 24 13 28 Consistency Could be any requirements describing the same object that conflict with other requirements with respect to terminology? 24 13 28 Could be any requirements describing the same object that conflict with respect to characteristics? 25 14 29 Could be any requirements that describe two or more actions that conflict logically? 26 15 30 Could be any requirements that describe two or more actions that conflict temporally? 27 16 31 To be continued on next page
  61. 61. APPENDIX A. CHECKLISTS 51 Quality Checklist. MASTeR, EARS, Planguage scores Traceability Can be all requirements traceable back to a specific user need? 27 16 32 Can be all requirements traceable back to a specific source document or person? 27 16 33 Can be all requirements traceable forward to a specific design document? 27 16 34 Can be all requirements traceable forward to a specific software module? 27 16 34 Verifiability Could be any requirements included which are impossible to implement? 28 17 35 For each requirement, could exist a process that can be executed by a human or a machine to verify the requirement? 29 18 36 Modifiability Could be any redundancy in the requirements? 30 19 37 Content General Can be each requirement relevant to the problem and its solution? 31 20 38 Can any of the defined requirements contain design details? 32 20 30 Can any of the defined requirements contain verification details? 33 22 40 Can any of the defined requirements contains project management details? 34 23 41 Specific Inputs Can all input sources be specified? 35 23 42 Can all input accuracy requirement be specified? 36 23 43 Can all input range values be specified? 37 23 44 Can all input frequencies be specified? 38 23 45 Can all input formats be specified? 39 23 46 Outputs Can all output destinations be specified? 40 24 47 Can all output accuracy requirements be specified? 40 24 48 Can all output range values be specified? 41 24 49 Can all output frequencies be specified? 42 24 50 Can all output formats be specified? 43 24 51 Functions Can all software functions be specified? 44 25 52 Can all system throughput rates be specified? 45 24 53 To be continued on next page
  62. 62. APPENDIX A. CHECKLISTS 52 Quality Checklist. MASTeR, EARS, Planguage scores Reliability Can the consequences of software failure be specified for each requirement? 46 25 53 Can the information to be protected from failure be specified? 46 25 54 Can a strategy for error detection be specified? 47 26 54 Can a strategy for correction be specified? 48 27 54 Tradeoffs Can acceptable tradeoffs be specified for competing attributes? 48 28 54 Hardware Can the minimum memory be specified? 49 28 55 Can the minimum storage be specified? 50 28 56 Can the maximum memory be specified? 51 28 57 Can the maximum storage be specified? 52 28 58 Software Can the required software environments/OS’s be specified? 53 28 59 Can all of the required software utilities be specified? 54 28 60 Can all purchased software products that are to be used with the system be specified? 55 28 61 Communication Can the target network be specified? 56 29 62 Can the required network protocols be specified? 57 30 63 Can the required network capacity be specified? 58 31 64 Can the required/estimated network throughput rate be specified? 59 31 64 Can the estimated number of network connections be specified? 60 31 64 Can the minimum network performance requirements be specified? 61 31 64 Can the maximum network performance requirements be specified? 62 31 65 Can the optimal network performance requirements be specified? 63 31 66 Can all aspects of the processing be specified for each function? 64 32 67 Can all outputs be specified for each function? 65 33 68 Can all performance requirements be specified for each function? 66 34 69 To be continued on next page
  63. 63. APPENDIX A. CHECKLISTS 53 Quality Checklist. MASTeR, EARS, Planguage scores Can all design constrains be specified for each function? 67 35 69 Can all attributes be specified for each function? 68 36 70 Can all security requirements be specified for each function? 69 37 71 Can all maintainability requirements be specified for each function? 70 37 72 Can all database requirements be specified for each function? 71 37 73 Can all operational requirements be specified for each function? 72 37 74 Can all installation requirements be specified for each function? 72 37 75 External interfaces Can all user interfaces be specified? 73 38 75 Can all hardware interfaces be specified? 73 38 76 Can all software interfaces be specified? 74 38 77 Can all communications interfaces be specified? 75 38 78 Can all interface design constraints be specified? 76 38 79 Can all interface security requirements be specified? 77 38 80 Can all interface maintainability requirements be specified? 78 38 81 Can all human-computer interactions be specified for user interfaces? 79 39 82 Timing Can all expected processing times be specified? 80 40 83 Can all data transfer rates be specified? 81 40 84 Table A.1: Quality Criteria Checklist, based on [Mos97] .
  64. 64. APPENDIX A. CHECKLISTS 54 A.2 Inspecting Formality Table A.2 shows the checklist utilised to measure the formality of the nota- tions in the comparative evaluation (Section 3.2). Results of the application for MASTeR, EARS and Planguage templates are represented in columns 2, 3 and 4, respectively. Formality Checklist. MASTeR, EARS and Planguage scores Does the notation present rules about the structure of the elements?1. 1 1 1 Does the notation present rules related to tokens2?. 1 1 2 Are the tokens represented in symbolic form3?. 1 1 2 Is it necessary to parse4 the resulting requirement so a human can understand its semantics?. 1 1 3 Is the notation free of idioms?. 2 1 4 Is the notation free of poetry and metaphors?. 3 2 5 Table A.2: Formality Checklist, partly based on [MR13] and [AYI04]. A.3 Inspecting Participants’ Requirements Table A.3 shows the checks utilised to measure the quality of the require- ments written by the participants of the experiment (Section 5.3.2). Checklist for participants’ requirements Is the requirement free of ambiguities? Is the requirement complete? Is the requirement free of vagueness? Is the requirement prioritised? Is the requirement verifiable? Is the requirement free of inappropriate implementation details? Is the requirement free of error in the usage of template-systems? Table A.3: Checklist utilised to test the quality of participants’ requirements. 1 Any language will present structure of its elements, even natural languages like english or german. 2 Allowed elements of the language. 3 Use of symbols and logical, arithmetic, ..., connectors. 4 Analyze and understand the structure of the requirement.
  65. 65. APPENDIX B. EXPERIMENT 55 B Experiment The content of the web-based experiment is summarised in this chapter. B.1 Introduction Figure B.1 shows the questionnaire about the occupation and previous ex- perience of the participants. This is placed at the introduction of the test (Section 5.1.1) and is used for statistical purposes. Figure B.1: Questionnaire about participant’s previous experience and occu- pation. B.2 Reading Table B.1 lists the requirements and questions that participants had to read and answer at the reading section of the experiment (Section 5.3.1). Table B.2 lists the feedback statements of which the participants had to to agree or disagree. B.3 Writing Table B.3 lists all the descriptions that the test subjects had to specify at the writing section of the experiment (Section 5.3.2). Table B.4 shows all the statements of the writing questionnaire.
  66. 66. APPENDIXB.EXPERIMENT56 Requirement Question Validate notebook file format. Is it necessary to specify the file format so that the requirement is complete? Why? The system shall provide users with the ability to resolve merge conflicts manually. Is the requirement complete? If a merge is not intentioned, the system should provide users with the ability to undo the merge. Could the requirement have more than one interpretation? Why? If an IPython Notebook file is corrupted, then the system shall display a message to the user. Should the requirement be specified in more/less detail? Why? While the system is requesting a notebook from remote servers, the system shall not accept parallels requests to the same notebook. Is the requirement testable? Table B.1: Questions to be answered at the reading section.
  67. 67. APPENDIXB.EXPERIMENT57 Statement Do you agree or disagree? Natural language requirements are easy to understand. ◻ Strongly Disagree ◻ Disagree ◻ Agree ◻ Strongly Agree MASTeR requirements are easy to understand. ◻ Strongly Disagree ◻ Disagree ◻ Agree ◻ Strongly Agree EARS requirements are easy to understand. ◻ Strongly Disagree ◻ Disagree ◻ Agree ◻ Strongly Agree MASTeR requirements are more complete than natural language requirements. ◻ Strongly Disagree ◻ Disagree ◻ Agree ◻ Strongly Agree EARS requirements are more complete than natural language requirements. ◻ Strongly Disagree ◻ Disagree ◻ Agree ◻ Strongly Agree Table B.2: Content of the reading section’s feedback questionnaire. Description Notation This functionality allows the user to diff and display modifications made between two versions of the same IPython notebook, allowing the user to view the added, removed or modified elements. MASTeR EARS Natural language This functionality allows the user to generate a diff of a notebook from two local IPYNB files, provided by the user. MASTeR This functionality generates a diff of a notebook on a line based level for text based cells, indicating to the user when a cell has been modified and which lines were added or deleted. EARS This functionality generates a diff of a notebook on a line based level for text based cells, indicating to the user when a cell has been modified and which lines were added or deleted. MASTeR EARS Natural language Table B.3: Descriptions of the system functionalities to be specified in the writing section.
  68. 68. APPENDIXB.EXPERIMENT58 Statement Do you agree or disagree? Descriptions were easy to specify in MASTeR. ◻ Strongly Disagree ◻ Disagree ◻ Agree ◻ Strongly Agree Descriptions were easy to specify in EARS. ◻ Strongly Disagree ◻ Disagree ◻ Agree ◻ Strongly Agree Descriptions were easy to specify in natural language. ◻ Strongly Disagree ◻ Disagree ◻ Agree ◻ Strongly Agree MASTeR contains a full set of templates. ◻ Strongly Disagree ◻ Disagree ◻ Agree ◻ Strongly Agree EARS contains a full set of templates. ◻ Strongly Disagree ◻ Disagree ◻ Agree ◻ Strongly Agree Table B.4: Content of the writing section’s feedback questionnaire.
  69. 69. BIBLIOGRAPHY 59 Bibliography [Abr10] Jean-Raymond Abrial. Modeling in Event-B: System and Soft- ware Engineering. Cambridge University Press, New York, NY, USA, 1st edition, 2010. [AYI04] Yunus Adanur, Oktay Yagiz, and Ahmet I¸sik. Mathematics and language. Journal of the Korea Society of Mathematical Education Series, 8:31–27, Mar 2004. [BF14] Pierre Bourque and R. E. Fairley. SWEBOK: guide to the software engineering body of knowledge, volume 3. IEEE Com- puter Society, 2014. [Boe79] Barry Boehm. Guidelines for verifying and validating software requirements and design specifications. Euro IFIP 79, pages 711–719, 1979. [Che13] Lianping Chen. Characterizing architecturally significant re- quirements. IEEE Software, 30(2):38–45, Apr 2013. [Dav95] A. M. Davis. Remontando: Una Necesidad Simple Descuid´o. IEEE Software, 1995. [DSP10] Definition and Exemplication of RSL and RMM, CESAR Project. Report, EADS Deutschland GmbH - IW, Germany, 2010. [Eak96] Kevin Eakin. Implementing espiti on a european scale. Journal of Systems Architecture, 42(8):579–582, 1996. [Ebe08] Christof Ebert. Systematisches Requirements-Engineering und Management: Anforderungen ermitteln, spezifizieren, analysieren und verwalten. dpunkt-Verl., 2008. [Eck15] Florian Eckey. Konzeption und entwicklung einer software zur konsistenzhaltung zwischen bpmn modellierung und textueller anforderungsdokumentation, 2015. [Far12] S. Farfeleder. Requirements Specification and Analysis for Em- bedded Systems. PhD thesis, Vienna University of Technology, 2012.
  70. 70. BIBLIOGRAPHY 60 [GDW09] Alexander Grosskopf, Gero Decker, and Mathias Weske. The process: business process modeling using BPMN. Meghan- Kiffer Press, 2009. [Gil05] Tom Gilb. Competitive engineering: a handbook for systems engineering, requirements engineering, and software engineer- ing using Planguage. Butterworth-Heinemann, 2005. [Gro15] Katharina Großer. Investigating the use of ontology techniques as a support for on-board software requirement engineering, 2015. [HAL+14] Jennifer Horkoff, Fatma Basak Aydemir, Feng-Lin Li, Tong Li, and John Mylopoulos. Conceptual modeling: Evaluating Mod- eling Languages, An Example from the Requirements Domain, page 260–274. 33rd International Conference, 2014. [HJD05] Elizabeth Hull, Ken Jackson, and Jeremy Dick. Requirements engineering. Springer, 2005. [ISO98] IEEE recommended practice for software requirements speci- fications. Standard, International Organization for Standard- ization, Switzerland, 1998. [ISO10] Systems and software engineering — vocabulary. Standard, International Organization for Standardization, Switzerland, 2010. [ISO11] Systems and software engineering - life cycle processes - re- quirements engineering. Standard, International Organization for Standardization, Switzerland, 2011. [Jac01] Daniel Jackson. Alloy: A lightweight object modelling nota- tion. Laboratory of Computer Science, Massachusetts Institute of Technology, November 2001. [Joh12] V. Johannessen. Cesar - text vs. boilerplates: What is more efficient - requirements written as free text or using boilerplates (templates), 2012. [KC05] Sascha Konrad and Betty HC Cheng. Facilitating the con- struction of specification pattern-based properties. 13th IEEE International Requirements Engineering Conference, 2005. [Lav03] L. Lavazza. Rigorous Description of Software Requirements with UML. 15th International Conference on Software Engi- neering and Knowledge Engineering (SEKE2003), July 2003. [Mav10] Alistair Mavin. EARS Tutorial. Rolls-Royce, 2010.
  71. 71. BIBLIOGRAPHY 61 [Mos97] Daniel J. Mosley. Software requirements specification walk- through checklist. Technical report, CSST Technologies, Inc, 1997. [MR13] Brad Miller and David Ranum. Formal and natural languages, 2013. [MSKB11] D. Majumdar, S. Sengupta, A. Kanjilal, and S. Bhattacharya. Automated Requirements Modelling With Adv-EARS. Inter- national Journal of Information Technology Convergence and Services, 2011. [MWHN09] A. Mavin, P. Wilkinson, A. Harwood, and M. Novak. EARS (Easy Approach to Requirements Syntax). 17th IEEE Interna- tional Requirements Engineering Conference, 2009. [Poh96] Klaus Pohl. Requirements engineering: an overview. RWTH, Fachgruppe Informatik, 1996. [RdS14a] Chris Rupp and die SOPHISTen. Requirements Templates - The Blueprint of your Requirement. SOPHIST GmbH, 2014. [RdS14b] Chris Rupp and die SOPHISTen. Schablonen f¨ur alle F¨alle. SOPHIST GmbH, 2014. [RR06] Suzanne Robertson and James Robertson. Mastering the re- quirements process. Addison-Wesley, 2 edition, 2006. [Sim01] Erik Simmons. Quantifying quality requirements using plan- guage. Technical report, Intel Corporation, Hillsboro, 2001. [Sim11] Erik Simmons. 21st Century Requirements Engineering: A Pragmatic Guide to Best Practices. PNSQC, 2011. [Som11] Ian Sommerville. Software engineering. Pearson, 2011. [Ter13] John Terzakis. Ears: The easy approach to requirements syn- tax, Jul 2013. [WD96] Jim Woodcock and Jim Davies. Using Z: specification, refine- ment, and proof. Prentice Hall, 1996. [WH12] Matt Wynne and Aslak Hellesoy. The Cucumber Book: Behaviour-Driven Development for Testers and Developers. The Pragmatic Bookshelf, 2012. [Wie99] Karl E. Wiegers. Writing quality requirements. Software De- velopment, May 1999.

Views

Total views

196

On Slideshare

0

From embeds

0

Number of embeds

5

Actions

Downloads

3

Shares

0

Comments

0

Likes

0

×