Usability requirements and their elicitation
Upcoming SlideShare
Loading in...5

Usability requirements and their elicitation






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds



Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial LicenseCC Attribution-NonCommercial License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Usability requirements and their elicitation Usability requirements and their elicitation Document Transcript

  • Usability requirements and their elicitation Lucas Machado, Menglin Xu, Muhammad Qasim, Silvia P. H. Rubio (SIS) School of Information Sciences – University of Tampere October 25, 2013 Abstract understanding and knowledge of the purpose leads us to requirements engineering; an approach to procure useful specifications (requirements) which aid in deciding on how to build an artefact. Requirements engineering is, as its name suggests, the engineering discipline of establishing user requirements and specifying software systems (Sutcliffe, 2013). It involves finding out what people expect from a software system and specifying their expectations (purpose of software system) in terms of software system design. Nuseibeh and Easterbrook (2000) give a more comprehensive definition as ”software systems requirements engineering (RE) is the process of discovering that purpose, by identifying stakeholders and their needs, and documenting these in a form that is amenable to analysis, communication, and subsequent implementation”. Requirements of a software system are gathered and implemented to produce a usable system. To ensure the easiness of use and acceptance of this system, its usability must be validated. Usability is a quality attribute of a system. It is the extent to which specific users to their satisfaction can perform a specific goal effectively and efficiently in a specified context (ISO, 1998). The usability of a system is stated by usability requirements. Usability requirements are the specifications that incorporate five usability factors: This essay is a result of a group work about usability requirements, testing and their elicitation for the Requirements Engineering course. The content of this article is organized as follows: Section 1 introduces concepts and general ideas, Section 2 is about usability requirements styles and testing, and is based mainly in the article by Lauesen and Younessi (1998). Section 3 is based mainly in the article by Jos Trienekens (2012) and is about categorizing usability requirements elicitation methodologies and exploring the most suitable of them for determined project needs. In Section 4, some practical experience with requirements engineering of one of the authors is reported. Finally, Section 5 gives the conclusions. Key concepts: usability requirements, methodologies for eliciting and analyzing usability requirements, usability testing. 1 Introduction Engineering involves building of purposeful artefacts often known as machines. Software engineering as a subdiscipline of engineering, concerns with the configuration of a general-purpose machine (the computer) to work for a specific purpose. Before building a machine or artefact to fulfil a specific purpose, it is important to understand what that purpose is. This 1. Ease of learning: Both novice and experienced users must quickly learn the system. 1
  • 2. Task efficiency: The system must be swift and responsive for user tasks. the lifecycle when modifications are often difficult, impractical, or even impossible.” There exist many issues regarding the usability requirements elicitation, their methodologies, and their validation. In this essay, we will discuss two emergent issues. First issue is “What is the relation between usability requirements elicitation and usability testing? ” and the second issue is “How to identify the most suitable usability requirements elicitation methodology according to the needs of different projects/situations? ” With regard to first issue, six styles for usability requirements elicitation are discussed with focus on their verification, using them during development process and how usability testing is useful in eliciting the data for specification. It will help to understand at what extent theses styles address usability issues in a software system. There are also many methodologies for the elicitation and analysis of usability requirements available in the literature. But the challenge is to select the best methodology for specific needs. So, in second question we argue which method/methodologies are the best suited according to the particular demands of the project for gathering usability requirements. The results of this discussion help non-experts of usability to select the best suited method in accordance to the software system requirements or project needs. 3. Ease of remembrance: The system functionality must be easy to remember by casual users. 4. Understandability: The user must understand the system effortlessly. 5. Subjective Satisfaction: The user must be satisfied with the system after using it. These requirements may interact with each other, and so should be prioritized for a particular system. These requirements are elicited and analysed by various approaches, for example, by analysing the current systems, by observing the interaction of a user with a system, or by following guidelines and doing heuristics analysis. Usability requirements play crucial factor in determining the success of a software system. Accurateness of usability requirements being transformed into usability features of a system is directly proportional to the acceptance of the system by the users. In order to guarantee the acceptance of the system, validation of the usability requirements is required which is done by conducting usability testing. Usability testing is a technique to evaluate a system for usability problems by testing it with real users. The users with relevant background are provided with the tasks to test a system or a prototype. This testing ensures the ease of use, learnability and satisfaction of a user from the system. Usability testing should not be delayed into the later stages of the software development process but it should be done concurrently with production of the components of a software system. Addressing usability at the requirements stage has the same benefits as considering other quality attributes early on in the development process (Mario R. Barbacci, 2003): “The earlier key quality attribute requirements are identified and prioritized, the more likely it is that the essential quality attributes will be built into the system. It is more cost-effective to reason about quality attribute trade-offs early in the lifecycle than later in 2 What is the relation between usability testing and usability requirements elicitation? The concept of usability requirements has been described previously, but it is important to note that there are several ways of writing these requirements. In their paper “Six Styles for usability requirements” Lauesen and Younessi (1998) explain the different styles in which usability requirements can be represented according to the part of the development process that they are focused on and it is also described 2
  • be changed or refined and most importantly if any new usability requirements emerge. Usability testing in this style covers most of the usability factors, however, subjective satisfaction cannot be analyzed with this kind of requirements as it only provides the information on how well the user performs a task and even though they do well in one particular task, their overall impression on the system can be different. Defect-style requirements are very similar to performance-based ones, they also work with tasks but the main focus is on the limit of usability problems that users can encounter while performing this task. A usability problem arises when a user makes mistakes or finds difficulties while using the system. For instance, if we think of the ATM scenario this could be an example of a defect-style requirement: how usability testing is involved in the elicitation of these requirements. These six styles mentioned in the paper are: performance, defect, process, subjective, design and guideline. Performance-based requirements specify different user groups and a set of tasks- a work that needs to be completed by the user with help of the system - that the users must undertake. A performance objective is established for each task. If we consider an ATM system, the following could be an example of a performance-based requirement: Customers with ATM experience from other banks: In their first attempt, they must be able to withdraw a preset amount of cash within an average of 2 minutes. (Lauesen and Younessi, 1998) In this example, we can identify the user (customers with ATM experience from other banks), the task (withdrawing a preset amount of money) and the performance objective (taking an average of 2 minutes). The initial elicitation of performance-based requirements follows the same steps as any other functional requirement by evaluating the stakeholders’ needs and possibly some previous systems. However, usability testing is used to verify and modify (if needed) these initial requirements. There are several techniques in usability testing. If we consider the previous example of a performancebased requirement, the testing could be done by recording a group of ATM users (randomly chosen) while using the new system and evaluating the way they interact with it. These users could also be asked to perform their task while saying what they are thinking (think aloud usability testing technique) or even a combination of the two techniques described previously. Regardless of the testing technique, the goal of usability testing in this case scenario is to verify the usability requirements initially elicited and evaluate whether the performance objectives need to Customers with ATM experience from other banks: In their first attempt, no more than 0.2 task failures can be encountered. (Lauesen and Younessi, 1998) The same kind of usability tests proposed to verify performance-based requirements could be used in this case. However, since the focus is in usability problems, these have to be counted and then the data obtained can be analyzed. The importance of counting usability problems is that they can reveal usability defects which are the defects in the system design that cause the usability problems. To reveal usability defects feedback needs to be collected from users, an example of this would be interviewing a user and asking: “What did you find difficult in this particular task?” when referring to a task where the user failed or had difficulties with. Once again, usability testing provides a way to get feedback from users that allows the validation and refinement of the initial requirements and even the elicitation of new ones. It is also important to point out that these tests can be executed even before the system is ready with the use of prototypes, this is 3
  • For instance, one subjective-style requirement in the ATM case scenario would be: important to guide the development process in the right direction. As with performance-based requirements, the subjective satisfaction cannot be tested with this kind of requirements because the fact that users do not encounter problems in certain tasks does not mean that they will be satisfied with the complete system. Depending on the system being implemented, performance objectives and usability defects may be difficult to identify. In these cases, it is better to specify the requirements that will ensure usability and how they will be addressed during the design process rather than the usability requirements on the product itself. An illustration of this process-style requirement is the following: 80% of customers having tried the ATM at least once must find the system helpful. 60% must recommend it to friends if asked. (Lauesen and Younessi, 1998) In this example, we can observe that the requirement is not related to any task in particular but for the satisfaction of the system as a whole. Usability testing is again required to validate that the requirements are being met; during the development period and most importantly after the product is ready, feedback from the users has to be collected to verify that users are satisfied with the product. This requirement style is, as its name states, particularly good for analyzing the subjective satisfaction During design, a sequence of 3 prototypes factor. However, this is a very complex factor to anahas to be made. Each prototype must be uslyze because satisfaction can vary among users (even ability tested and the most important defects in the same situations) depending on the culture and corrected. (Lauesen and Younessi, 1998) even their personality. In this example we can observe instructions that Design-style requirements specify what the must be followed during the design process to achieve prototype design should look like. These requireusability. Even though these requirements do not al- ments are set by the requirements engineer and unways specifically request usability testing, most expe- like other usability requirements, which are considrienced developers will request usability testing dur- ered as non-functional requirements, design-style reing the design and development process as getting quirements are part of the functional requirements of feedback from users is the only way to make sure the system as they specify what the prototype should that usability is achieved. have. With process style requirements, the usability facAn example of this style would be the following: tors measured depend on the requirements chosen by The menu points and push buttons shall the developers, the downside of this type of requirefunction as shown in App. yy ments is that they only address usability testing durWith this approach, inspections and validations are ing the design process and usability factors could not necessary throughout the development process. Simibe tested after this stage. Another way of eliciting the requirements is based larly to previous requirements styles, usability testing on measuring the user experience as a whole. This is used in design-style requirements to get feedback style of requirements is subjective and its measure- from users and evaluate if the requirements are corment can be imprecise as different users may have rect and complete. Moreover, usability testing in design-style requirecompletely different perspectives of the same issue, however, a general idea of the usability of the prod- ments is also used to elicit the initial usability requirements. In order to do so an initial prototype is uct can be obtained. 4
  • 3 built and tested, after that, the feedback obtained in these tests is used as the initial requirements for the design. How to identify the most suitable Usability Requirements Elicitation Methodology according to the needs of different projects/situations? Experts have suggested many guidelines to achieve usability, some companies even have their own guidelines that they follow when developing a new system. The last requirements described in the paper are guideline-style requirements, which are basi- Usability Requirements Elicitation is a complex task cally focused on how the final product must meet the that should not be performed without following a specifications of certain preset guidelines. structured methodology. There are several different The following is an illustration of a guideline-style methodologies to elicit usability requirements and as more options become available to those project manrequirement: agers and developers there is an emerging need to identify which of the methodologies is more suitable for a project in terms of cost, time and effort. Jos Trienekens (2012) propose a framework to comThe system shall follow the MS-Windows pare what they consider the four most important apstyle guide. (Lauesen and Younessi, 1998) proaches in the requirements elicitation and analysis (i.e. The Usability Engineering Lifecycle (UEL), The Delta Method, Approach Centered on Usability and Driven by Use Cases (ACUDUC) and GuideFollowing this kind of requirement style is a challines for Eliciting Usability Functionalities (GEUF)). lenge because inspections and verifications can reHowever, in this paper we will focus in understandquire too much work (guidelines are usually too long) ing how the framework works and how it can help and still not lead to usability. Once again, a solution project managers identify the best methodology for to this problem is usability testing, after selecting the their project. In order to do so, only the Delta guidelines that the system will follow a prototype can Method and the Usability Engineering Lifecycle will be built and tested. Testing is used to refine this first be addressed. prototype according to the user’s needs and then this The framework divides each methodology into prototype can be used as the guideline that developseveral methods, which are single functions of the ers need to follow. methodology, and then evaluates each one of these To sum up, there are several styles in which usabil- methods according to some preset criteria. Conseity requirements can be elicited. Each style focuses quently, an overall score is assigned to the methodolon different aspects of the development process but ogy based on the individual scores of all the methods all of them have usability as a common goal. Also, for a specific criterion. usability testing is closely involved in the elicitation The criteria are divided into four categories: exprocess of these usability requirements, it is generally ternal factors, characteristics, maintaining the inused as a way to validate that the initial requirements tegrity of the specifications and quality and effectiveare being met, to refine initial requirements and in ness. Each category encircles the criteria related to some cases even to elicit new or initial usability re- that area, criterion are basically questions about the quirements. method. The set of categories and their criteria are 5
  • D. Quality and effectiveness: The criteria in this category address the level of detail that is elicited using the methodology/method. the following: A. External factors: This refers to the environment in which the project is to be developed. Three main questions are the subcategories in this criterion: • C4.1. Does a methodology/method elicit enough information to help the developer specify the fit criterion? • C1.1. Does a methodology/method need a human computer interaction (HCI) expert? A score of (+) is assigned if the method needs an HCI expert, (-) if not needed. • C4.2. Does a methodology/method elicit the dependencies between the usability requirements and other functional and nonfunctional requirements? • C1.2. Does a methodology/method need access to representative users? A score of Now that all the criteria have been explained, it (++) is assigned if the method needs a deep of involvement from the users, (+) if needs involve- is important to understand each methodology to anment from users, (-) if involvement is not needed. alyze their representation in the framework. First, we will describe the Usability Engineering Lifecycle • C1.3 Does a methodology/method work (UEL) methodology which was one of the first to prowith non-experienced users? A score from vide methods to incorporate usability engineering in 1 to 5 is assigned to represent how much experi- the product development process. UEL methods can ence a user must have for this particular method. also be applied in the usability requirements elicitaThis may not be applicable to some methods. tion and analysis, these methods are the following: Know the user refers to obtaining information B. Characteristics: This refers to the character- about the users (e.g. job, gender, computer experiistics of each methodology. There are two criteria in ence, etc.), observing users do their tasks and their this category: interaction with the system, as well as follow their • C2.1. Does a methodology/method give learning of the system. Competitive analysis refers to doing heuristic strict guidance to help the developers to analysis (using given usability guidelines) and protocarry it out? typing on existing (competition) products. Products • C2.2. Does a methodology/method take from the competition are a very good source for prothe user feedback into account for further totyping as they are currently working products and improvement? analysis on their strengths and weaknesses can contribute to a good design. C. Maintaining the Integrity of the SpecificaSetting usability levels indicates establishing a tions: This category encircles the criteria related set of usability goals beyond the main five usability to the effort required by methods. Two criteria are characteristics (e.g. learnability, subjective user satspecified in this category: isfaction, etc.). To do so, levels of usability should be • C3.1. Is a methodology/method time con- defined (i.e. worst acceptance level, planned usability level, current level and best possible level) that suming? indicate specify measurable usability goals. • C3.2. Is a methodology/method common Participatory design means involving the reprein the software development process? sentative users in the design by having regular meet6
  • Figure 1: Analysis results for the UEL methodology. (Jos Trienekens, 2012) ings with the designers. Users can ask questions and help to discover problems with the designers’ vision of the task and the real task. Coordinated design means having one (or more) coordinator(s) that can supervise the user interface design in order to achieve consistency not only in the current design but in further development of the product (e.g. future versions). Agreements on the user interface design, prototyping and following standards are also recommended to achieve consistency. Guidelines and heuristic analysis encircles following pre-established guidelines (e.g. usability, userinterface design, product-specific, etc.) and performing heuristic evaluations based on such guidelines. Prototyping indicates building and testing different prototypes throughout the development process, it is recommended to prototype early and often to avoid extra effort in implementation of unnecessary or erroneous assumptions of the user needs. Empirical testing refers to users performing tests on the product and then analyzing the collected results. Testing can be done with early or late versions of the product. Collect feedback from field use is a method similar to empirical testing. Figure 1 shows the extract of the framework proposed in Jos Trienekens (2012) paper that corresponds to the UEL methodology. We can observe that each method has been evaluated with all of the criteria and then an overall score was assigned to the methodology based on the combination of all the individual scores. Based on the information in the table the UEL methodology: • Needs advice from a HCI expert. • Needs frequent access to the representative users of the product in most of the methods. • Does not require experienced users • Does not provide strict guidance to developers. • Gets users’ feedback to improve usability. • Requires a lot of effort from the developer team. • Does not provide a lot of methods commonly used in the software engineering process. • It provides the developer with enough information to elicit requirements. • It provides enough information to elicit the dependencies between requirements. As an example, we can think of the manager of a new mobile app project who has his/her customers spread in five different countries and needs to launch 7
  • Figure 2: Analysis results for the Delta method. (Jos Trienekens, 2012) this product in less than two months with a small group of novice programmers. Just by using this framework in less than five minutes he/she could decide that this methodology is not suitable for this particular project for several reasons: it demands a lot of time and effort, it requires constant access to the users and it provides no guidance for the novice developers. Prototyping and Usability Testing indicates that a first paper-based prototype is built and tested (using the usability specification) and based on the feedback collected from users are the foundations for the design process. These steps are repeated through the entire development process with several prototypes until the goals are achieved. Figure 2 contains the extract of the framework proposed in Jos Trienekens (2012) paper that corresponds to the Delta method. Once again, each method was evaluated with all the criteria and an overall score for the methodology was derived from the combination of all the methods and criteria. Based on the information in the table the UEL methodology: The second methodology that we will be discussed in this paper is the Delta method. This methodology is a task-based and usability-oriented approach used in requirements engineering. It consists of the five following methods: Pre-study addresses the evaluation of the “hard” requirements (e.g. policies, industry standards, etc.) and the agreement on a vision scope from all parts involved. • Does not need advice from a HCI expert. • Needs access to the representative users in all the methods. User profiling involves recording an overview of the users by means of a questionnaire containing users’ preferences, problems, main tasks, etc. • Requires moderate experience from users. Task Analysis refers to interviewing the different representative users of the system in order to identify and describe their working tasks in detail including information about the current problems they have in performing these tasks. • Provides strict guidance to developers (through questionnaires, usability specifications, etc.). • Gets users’ feedback to improve usability in some of the methods. Usability specification refers to agreeing on the desired level of usability which is established in a measurable form that can be tested later on in the process; the inputs for this process are the conceptual design and the previous documentation. • Does not require a lot of effort from the developer team. • Some methods are commonly used in the software engineering process. 8
  • • It provides the developer with enough (not quanIn the third stage, when the user interfaces were betitative) information to elicit requirements. ing coded and tested, some of them were not needed anymore, and there was the need for other interfaces • It does not provide enough information to elicit that were not specified too. Also, in the developed the dependencies between requirements. interfaces, a lot of problems of usability were found because the requirements were not correctly managed If we consider the same example from above (the and there was no much attention to them. project manager dilemma), he/she might consider With all of that, the user interfaces of the system this a more suitable methodology in this particular had to be rebuilt, but at this time, guidelines, checkproject for the following reasons: it does not require lists of common features, and usability testing were a lot of effort (less time-consuming), it requires less established, so the system could finally have an acexperience from the developers as it provides guidceptable services quality and easy of use. However, ance and even though it still needs involvement from the project used much more resources and time than the representative users it can be completed faster. what was originally planned. Finally, there is probably not a “perfect” methodThis example clearly shows the importance of the ology for a project but since each project has its own requirements engineering, and the need to have clear priorities this framework can be a precise and reliable tool to analyze the benefits and drawbacks of and well-developed requirements. Also, in the last each methodology, it can help managers make the stages the usability requirements suffered the same necessary trade-offs and ultimately choose the most problems as the functional ones, then needing ansuitable methodology to elicit usability requirements. other set of rework to fix them. With well-designed functional and usability requirements, and using usability testing, the project was finished. If a methodology of usability requirements elicitation was used 4 Practical experience since the beginning along with good processes of funcIt is possible to observe the software engineering tech- tional requirements development, the project could niques, as well as the usability requirements elicita- have saved extra expenses. tion in the practical work experience of one of the authors. As an example, there was a system being developed that went through by three different stages 5 Conclusion in the company, these stages are described as follows: In the first stage, the company was not using the Usability is an important feature in a software, and proper requirements-engineering techniques in the can be determining in its success. It influences the project. So the objective of the requirements engineer system performance, adoption and easy to use, but was actually just generating documentation, and the often does not receive the deserved attention in the development and it is hard to analyze, recognize the requirements quality was poor. In the second stage, when some parts and compo- gaps and implement them. nents of the system were being be integrated, a lot of To help solve this issue, there are methodologies problems and conflicts were found due to the lack of in the requirements engineering that can be used to specification and poor interfaces. With this problem gather and establish usability requirements. Usabilin hands, the team managed to redo all the require- ity testing can also be used to detect usability probments specification of the components that were in lems in the system and correct or prevent them (depending on the style, testing can be done with a functrouble, but did not update all the documentation. 9
  • tional software or even with prototypes or design). 6 Division of work There are differences between the methodologies in a way that we need to choose a proper one based in the needs of the project. The first addressed methodology of the described UREAM framework by Jos Trienekens (2012) (Usability Engineering Lifecycle) is more suitable to large scale projects in large companies as it requires several resources and a great amount of effort and time. The second addressed methodology, the Delta Method, is easier to apply as it not requires many resources (like experts or user involvement) and is less time-consuming. Then it should be appropriate for smaller and simpler projects. The work division for writing this essay was mainly based in its sections. Section 1 was written by Muhammad, Section 2 was written by Silvia, Section 3 was written by Menglin with Silvia’s help, and finally Sections 4 and 5 were written by Lucas. Additionally, all the group members revised the entire document and made corrections. References P. Carlshamre and J. Karlsson. A usabilityoriented approach to requirements engineering. In Requirements Engineering, 1996., Proceedings of the Second International Conference on, It is also possible to skip some of the methods pages 145–152, 1996. doi: 10.1109/ICRE. within a methodology, as well as adding others de1996.491439. pending on the project or system needs. The methodxpl/articleDetails.jsp?arnumber=491439. ologies should be interpreted as good practices and ISO. ISO 9241-11:1998 Ergonomic requirements for not as strict instructions. office work with visual display terminals (VDTs) – Part 11: Guidance on usability. Technical The different styles of usability requirements elicreport, International Organization for Standarditation are all aimed at improving the usability of a ization, 1998. system in different ways and by complementing all resources/iso9241/part11.html. of them we can have a wide coverage over the main usability problems that can occur in a system. Rob Kusters Jos Trienekens. A framework for characterizing usability requirements elicitation Finally, usability testing can be closely involved and analysis methodologies (uream). In ICSEA with the requirements elicitation as a way to improve 2012, The Seventh International Conference on the general usability. When a system is already funcSoftware Engineering Advances, page 308 to tional, they can refine or create new requirements 313, Lisbon, Portugal, November 2012. IARIA. based on feedback, or they can even prevent usabil ity problems during the design or prototyping stages article&articleid=icsea_2012_11_30_10254. by eliciting initial requirements. Søren Lauesen and Houman Younessi. Six styles for usability requirements. In Eric Dubois, With a proper selection of the usability requireAndreas L. Opdahl, and Klaus Pohl, ediments methodology and/or methods, along with intors, REFSQ, pages 155–166. Presses Univertegrated usability testing, depending on the style of sitaires de Namur, 1998. ISBN 2-87037-272these methods of requirements elicitation, it is possi8. ble to minimize the problems and improve or achieve refsq1998.html#LauesenY98. a high level of usability in a system. 10
  • Anthony J. Lattanze Judith A. Stafford Charles B. Weinstock William G. Wood Mario R. Barbacci, Robert J. Ellison. Quality attribute workshops (qaws), third edition. Technical report, Software Engineering Institute, 2003. asset-view.cfm?assetID=6687. J. Nielsen. The usability engineering life cycle. Computer, 25(3):12–22, 1992. ISSN 0018-9162. doi: 10. 1109/2.121503. xpl/articleDetails.jsp?arnumber=121503. Bashar Nuseibeh and Steve Easterbrook. Requirements engineering: a roadmap. In Proceedings of the Conference on The Future of Software Engineering, ICSE ’00, pages 35–46, New York, NY, USA, 2000. ACM. ISBN 1-58113-253-0. doi: 10. 1145/336512.336523. 1145/336512.336523. Alistair G. Sutcliffe. Requirements Engineering from an HCI Perspective. The Interaction Design Foundation, Aarhus, Denmark, 2013. http:// requirements_engineering.html. 11