Kaleem urrehman managing risks and quality in agile methods

Table of Contents List of Tables .........................................................................................................................................................3 List of Figures ........................................................................................................................................................3 Abstract.................................................................................................................................................................41. INTRODUCTION ............................................................................................................................................. 5 PROPOSAL SUMMARY .....................................................................................................................................................5 RESEARCH QUESTION(S)..................................................................................................................................................6 STRUCTURE OF THIS DISSERTATION ....................................................................................................................................6 SUMMARY OF THE PROBLEM DOMAIN................................................................................................................................72. LITERATURE REVIEW ..................................................................................................................................... 9 SOFTWARE QUALITY .......................................................................................................................................................9 What is quality ......................................................................................................................................................9 What is software quality.......................................................................................................................................9 Quality Attributes................................................................................................................................................10 Quality Assurance ...............................................................................................................................................12 SQA Phases and Activities ...................................................................................................................................12 REVIEWS (QUALITY ASSURANCE).....................................................................................................................................13 In-Process Review................................................................................................................................................13 Phase-end Review ...............................................................................................................................................14 SOFTWARE TESTING......................................................................................................................................................14 Testing Lifecycle ..................................................................................................................................................15 V – Model (Testing Framework)..........................................................................................................................16 RISK...........................................................................................................................................................................18 ITERATIVE LIFE CYCLE ....................................................................................................................................................19 Agile Development..............................................................................................................................................20 AGILE MANIFESTO........................................................................................................................................................24 Agile Requirement Documentation.....................................................................................................................24 Agile Customers ..................................................................................................................................................26 Agile Sprints/Small Releases ...............................................................................................................................26 Agile Team Effort ................................................................................................................................................27 Agile Refactoring.................................................................................................................................................28 Agile Multiple Testing Teams..............................................................................................................................28 Agile Exploratory Testing ....................................................................................................................................29 SUMMARY ..................................................................................................................................................................303. RESEARCH METHODS .................................................................................................................................. 31 CASE STUDIES AS RESEARCH METHOD ..............................................................................................................................32 SUMMARY ..................................................................................................................................................................324. CASE STUDIES IMPLEMENTATION................................................................................................................ 33 FIRST CASE STUDY: INTEGRATING TESTING IN AGILE PROJECTS ..............................................................................................33 Overview of the Company...................................................................................................................................33Kaleem Urrehman Page 1

Challenges...........................................................................................................................................................33 Solution ...............................................................................................................................................................33 Result ..................................................................................................................................................................33 SECOND CASE STUDY: AGILE SOFTWARE DEVELOPMENT IN LARGE ORGANIZATION ...................................................................34 Overview of the Company...................................................................................................................................34 Challenges...........................................................................................................................................................34 Solution ...............................................................................................................................................................34 COMPARATIVE STUDIES .................................................................................................................................................34 SUMMARY ..................................................................................................................................................................355. ANALYSIS..................................................................................................................................................... 366. CONCLUSION ............................................................................................................................................... 407. REFERENCES ................................................................................................................................................ 42Kaleem Urrehman Page 2

List of TablesTable 1: SQA Phases and Activities vs. Software Development Phases ..................................................... 12Table 2: In-Process Review Characteristics ................................................................................................ 13Table 3: Phase-end Review Subject Documents ........................................................................................ 14Table 4: Agile QA vs. Traditional QA .......................................................................................................... 21Table 5: Agile quality techniques as applied in extreme programming ..................................................... 22Table 6: Comparison between two case studies ........................................................................................ 35List of FiguresFigure 1: Cost of Fixing Bugs ...................................................................................................................... 10Figure 2: Quality Attributes ........................................................................................................................ 11Figure 3: Waterfall Phases with V&V Model Extension ............................................................................. 18Figure 4: Agile Methods and QA ................................................................................................................ 22Figure 5: Documentation throughout Software Development Phases ..................................................... 25Figure 6: Interaction of Roles ..................................................................................................................... 27Kaleem Urrehman Page 3

AbstractQuality is impossible (rather a challenge) to measure as it is an ongoing process. There is any way tokeep check on quality of process/product in less time and resources, is all about my dissertation.Software with minor bugs annoys the customers and at the same time, software with major bugs late itsrelease. Software quality, everybody seems to understand it, everybody wants it and yet everyone has adifferent perception of quality.This dissertation involved in two research aims through a far-reaching study of relevant literature andthe two case studies implementation. This research produces a number of key findings: why software isdelayed from its release date, why software is not producing results as per customer expectations, whycustomer involvement is necessary on regular basis, how can quality be improved and impact of riskscan be reduced.The main conclusion drawn from this dissertation are quality and risk aspects in agile environment i.e.team effort, multiple testing teams, testers’ training, exploratory testing, minimal documentation,continuously changing requirements, refactoring, sprints and customer regular feedback. Thisdissertation argues for the customer involvement on regular basis especially after every agile sprint. Onethat takes into account is to minimize documentation effort along with change in code behavior asrequirements tends to change continuously and last but not the least, need for team effort.Kaleem Urrehman Page 4

1.IntroductionProposal SummaryQA processes during the software development are means of identifying bugs in planning, analysis anddesign activities before the actual development activity starts (The earlier in iteration, a defect isdetected, the cheaper it is to fix it). After product is developed, testing activity is implemented withintent to find a bug. How reviews (QA) should be done manually and how testing should be donemanually/automated is a question and will be discussed properly in this document.Quality has been part of human life, culture, and history from its earliest beginnings and it has alwayshad two aspects. One aspect, represented by the 11,000-year-old Sphinx at Giza, is beauty. Indefinableand alluring, beauty draws us, adding richness to our lives. Another aspect is represented at Giza as well,the 5000-year-old Great Pyramid—still standing—represents the functional quality of great engineering.The Sphinx is still standing because definable and measurable functional quality brings stability to themore ephemeral quality of beauty. In recent centuries, we have been able to define more and more ofwhat quality is, and, in defining it, make it more susceptible to engineering, make it reproducible, andbring it under management. But there will always be an indefinable side to quality—what we call beauty.(Kemp, 2006)Many software fail in the market due to its bad quality. We have several examples where qualityassurance/software testing is not done for BIG HUGE projects and those projects were not successfuland then everybody admits the importance of QA/testing.As the complexities of software development have evolved over the years, the demands placed onsoftware engineering, information technology (IT), and software quality professionals have grown andtaken on greater relevance. We [QA and Tester Professionals] are expected to check whether thesoftware performs in accordance with its intended design and to uncover potential problems that mightnot have been anticipated in the design. We are expected to develop and execute more tests, faster,and more often. Test groups are expected to offer continuous assessment on the current state of theprojects under development. At any given moment, we must be prepared to report explicit details oftesting coverage and status, the health or stability of the current release, and all unresolved errors.(Nguyen, 2003)Ahmed (2010) reveals that software testing management is one of the most difficult tasks out there.There are many reasons for this. First, a software product is not a tangible thing that can be measured,physically felt, or sampled. So it is difficult to test a software product. Second, software testing is still notconsidered a recognized trade and so finding professionally qualified people for the testing job isdifficult. Third, unlike well-defined and standardized processes for product design, productdevelopment, quality control, and so on, which exist for any product development activity, similarstandardized processes have yet to be defined for software testing. Fourth, tools for automation ofsoftware testing activities are still in their nascent stage, and it will take time to have sophisticatedautomation tools available for software testing activities. Fifth, effort estimation techniques for softwareKaleem Urrehman Page 5

testing activities are still being evolved, and currently effort estimation is done mostly on an ad hocbasis.The importance of software testing is so immense! Any failure of the software product or applicationcan cause damages to the tune of millions of dollars [pounds] for any company. Even if the softwaredefect is not so big, the support cost can run in the thousands of dollars over the life of the softwareproduct. (Ahmed, 2010)Research Question(s)Mainly focusing question for this dissertation is – Why software quality is important and how can wemanage risks to get optimal quality in context of agile methods?Q. 1 – Practical insight with clear examples for Process Testing (quality assurance – QA, reviews andverification) i.e. requirement gathering and building software design? Each process gives an output as a product. Software processes if managed in a good way canenhance the software product’s quality. Agile methods allows developers to directly take therequirements from the client and start building the software where from the start agile testing alsoinvolves in.Q. 2 – Practical insight with clear examples for Agile Product Testing (validation, types, levels,techniques)? Testing is part of process quality assurance. Good testers always do work in systematic ways toapproach best test practices in identifying bugs of the products. What agile test approaches areavailable in the market concluded from the past experiences will be discussed with examples.Q. 3 – What steps should be taken to control change requests that are affecting the budget and totalfunctionality of the system, hence increasing the system failure risk? Client requests so many change requirements that sometimes; cost required to develop thatchange without affecting the overall system functionality is much higher than the overall systemdevelopment cost, we will look into this matter, what process can control the change requests?Structure of this DissertationI have developed a path divided into 7 chapters which will help me in completing this dissertationdescribed as follows:Introduction – It tells why I believe quality and risks factors are important in agile software developmentis important that I have as dissertation topic, what are the key areas which I will cover and totalstructure of the dissertation.Literature Review – It gathers all sorts of possible information through different sources who havealready worked on agile quality and risks factors in software development.Kaleem Urrehman Page 6

Research Methods – It explains what 8 possible methods I can choose and why I am choosing thatparticular research method as dissertation method i.e. case studies.Case Studies Implementation – This area covers 2 case studies in detail sharing two different scenariosthat 2 agile companies were facing in relevant to quality and risk factors in software development. Bycomparing the findings of two case studies with the pool of information mentioned in chapter 2 ieLiterature Review hence helping to conclude useful ideas in next chapterAnalysis – This chapter covers the analysis made between case studies comparison and literature reviewmaterial so it helps me to come up with my own conclusion and recommendations.Conclusion – This chapter tells the recommendations and true findings of this dissertation afterthorough research done in above chapters.References – This chapter reveals all the information sources from where (websites, books, journals,case studies, articles), what kind of information is shared in this dissertation to avoid plagiarism.Summary of the Problem DomainAnon. (2011) tells that suppose you are assigned to develop custom software. Each block belowrepresents a step required to develop the software. Gather as much information as possible about thedetails & specifications of the desired software from the client. This is nothing but the Requirementsgathering stage. Plan the programming language like java, php, .net; database like oracle, mysql etcwhich would be suited for the project. Also some high level functions & architecture. This is the DesignStage. Actually code the software. This is the Built Stage. Next you, Test the software to verify that it isbuilt as per the specifications given by the client. This is the TEST stage. Once your software product isready, you may to do some code changes to accommodate enhancements requested by the client. Thiswould be Maintenance stage. All these levels constitute the waterfall method of software developmentlifecycle.Anon. (2011) reveals that if you are working in large project, where the systems are complex, it’s easy tomiss out key details in the requirements phase itself. In such cases, an entirely wrong product will bedelivered to the client. You will have to start a fresh with the project. Or if you manage to note therequirements correctly but make serious mistakes in design and architecture of you software you willhave to redesign the entire software to correct the error. Assessments of thousands of projects haveshown that defects introduced during requirements & design make up close to half of the total numberof defects. Also, the costs of fixing a defect increases across the development life cycle. The earlier in lifecycle a defect is detected, the cheaper it is to fix it. As it is said, "A stitch in time saves a nine"Stamelos (2007) reveals that agile software development methodologies have taken the concepts ofsoftware quality assurance further than simply meeting customer requirements, validation, andverification. Agility innovatively opens new horizons in the area of software quality assurance…Agilesoftware development is not just about meeting customer requirements (because even process-drivenmethodologies do that), but it is about meeting the changing requirements right up to the level ofproduct deployment.Kaleem Urrehman Page 7

I will first look into basic concepts of Quality Assurance, Testing and Risks and then see it how it isadopted in agile methods taking care of change requests at the same time.Kaleem Urrehman Page 8

2. Literature ReviewSoftware QualityWhat is quality – It is obvious that quality cannot be ignored and now for instance, gives a try todefine quality as written in different famous dictionaries  An essential and distinguishing attribute of something or someone  A degree or grade of excellence or worth  A characteristic property that defines the apparent individual nature of something  The inherent or distinctive characteristics or properties of a person, object, process or other thing  Totality of features and characteristics of a product or service that bear on its ability to satisfy stated or implied needsWhat is software quality – Quality of software cannot be defined, but still some mathematicians andcomputer engineers have shared their thoughts on that  It is hard to define, impossible to measure, easy to recognize (Kitchenham).  It is not the “goodness” or “elegance” but conformance to requirements (Crosby).  Fitness for use which means the following two things: “(1) quality consists of those product features that meet the needs of the customers and thereby provide product satisfaction. (2) Quality consists of freedom from deficiencies” (Juran)  The value to some people (Weinberg)  Quality is associated with human assessment, and cost and benefit (Hendrickson, 2004)  Quality is correctness, robustness, extendibility, reusability, compatibility, efficiency, portability, integrity, verifiability, and ease of use. (McCall)  Conformance to explicitly stated functional requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software (Pressman, 2001)  Software quality as a management process concerned with ensuring that software has a low number of defects and that it reaches the required standards of maintainability, reliability, portability, and so on. (Sommerville, 2006)  He views quality from five different perspectives namely; the transcendental meaning that quality can be recognized but not defined, user view meaning that quality is fitness for purpose, manufacturing meaning that quality is conformance to specification, product view meaning that quality is tied to inherent product characteristics, and the value-based view meaning that quality depends on the amount the customer is willing to pay for the product. (Pfleeger)  The degree to which a system, component, or process meets customer or user needs or expectations. (IEEE)  The totality of features and characteristics of a product or service that bear on its ability to satisfy specified or implied needs. (ISO)Although above all are the best try to define a quality and software quality, still it is hard to definequality. Quality like love is a thing to feel that cannot be described in words.Kaleem Urrehman Page 9

Consider time and budget are enough for the software development, but still there are examples wheresoftware fails as per client expectations. There is a need to identify those loop holes and address themin proper ways.Selby (2007) tells that calculating and understanding the value of a single overall metric for softwarequality may be more trouble than it is worth. The major problem is that many of the individualcharacteristics of quality are in conflict; added efficiency is often purchased at the price of portability,accuracy, understandability, and maintainability; added accuracy often conflicts with portability viadependence on word size; conciseness and conflict with legibility. Users generally find it difficult toquantify their preferences in such conflict situationsError vs Bug vs Failure - People developing mistakes do make mistakes. People make mistakes due topressures. Hambling (2007) reveals that Pressures such as deadlines, complexity of systems andorganizations, and changing technology all bear down on designers of systems and increase thelikelihood of errors in specifications, in design and in software code. An error (or mistake) leads to adefect, which can cause an observed failure.Error (Mistake)  Defect (Bug)  FailureFigure 1: Cost of Fixing Bugs (Hambling, 2007)One thing is clear: if we wish to influence errors with testing we need to begin testing as soon as webegin making errors – right at the beginning of the development process – and we need to continuetesting until we are confident that there will be no serious system failures – right at the end of thedevelopment process (Hambling, 2007).For a complete SDLC either in agile methods or waterfall methodology, all phases and its respectiveactivities are scheduled and implemented to get fruitful results, how these activities are defined anddealt with is a detailed discussion and will be the part of this dissertation.Quality AttributesGoodyear (2011) reveals that McCall identified three main perspectives for characterizing the qualityattributes of a software product. These perspectives are o Product revision (ability to change) o Product transition (adaptability to new environments) o Product operations (basic operational characteristics)Kaleem Urrehman Page 10

Product revisionThe product revision perspective identifies quality factors that influence the ability to change thesoftware product, these factors are:-  Maintainability, the ability to find and fix a defect  Flexibility, the ability to make changes required as dictated by the business  Testability, the ability to Validate the software requirementsFigure 2: Quality Attributes (Rizwan, n.d.)Product transitionThe product transition perspective identifies quality factors that influence the ability to adapt thesoftware to new environments:-  Portability, the ability to transfer the software from one environment to another  Reusability, the ease of using existing software components in a different context  Interoperability, the extent, or ease, to which software components work togetherProduct operationsThe product operations perspective identifies quality factors that influence the extent to which thesoftware fulfils its specification:-  Correctness, the functionality matches the specification  Reliability, the extent to which the system fails  Efficiency, system resource (including cpu, disk, memory, network) usage  Integrity, protection from unauthorized access  Usability, ease of useIn total McCall identified the 11 quality factors broken down by the 3 perspectives, as listed above.Kaleem Urrehman Page 11

Barbacci (1995) tells that systems often fail to meet user needs (i.e., lack quality) when designersnarrowly focus on meeting some requirements without considering the impact on other requirementsor by taking them into account too late in the development process. For example, it might not bepossible to meet dependability and performance requirements simultaneously: 1) Replicatingcommunication and computation to achieve dependability might conflict with performancerequirements (e.g., not enough time) 2) Co-locating critical processes to achieve performance mightconflict with dependability requirements (e.g., single point of failure).Quality AssuranceHass (2008) reveals that it is not possible to test quality into a product when the development is close tobeing finished. The quality assurance activities must start early and become an integrated part of theentire development project and the mindset of all stakeholders. Quality assurance comprises fouractivities:Quality Criteria These criteria are the expression of the quality level that must be reached or an expression of “what is sufficiently good.” These criteria can be very different from product to product.Validation Assessment of the correctness of the product (the object) in relation to the users’ needs and requirements (”Are we building the correct product?”)Verification Assessment of whether the object fulfills the specified requirements. (“Are we building the product correctly?”)Quality reporting Finding and results should be produced.SQA Phases and ActivitiesAccording to Rizwan (n.d.), a comparison between software quality assurance phases and itscorresponding activities with software development phases is shown below:Table 1: SQA Phases and Activities vs. Software Development Phases (Rizwan, n.d.)Software Development Phases SQA Phases SQA ActivitiesP&C Sign off Initialization of SQA Process QA Resources are assignedInitial Project Plan Initial SQA Plan –RS & Project Planning Reviews & SQA Planning Detail SQA Plan is preparedFS & Design Reviews –Coding/Unit Testing Test Planning Test Plan Preparation Test Case Preparation Environment SetupRework Integration Testing – System TestingDeployment Release Management Process Release Number Process Release Planning Build Process Release ConfigurationKaleem Urrehman Page 12

Management Process Shipment Assurance TestingPost Implementation Post Implementation Client Requests Testing(Maintenance)Reviews (Quality Assurance)Reviews are the first and primary form of quality control activity [i.e. Verification]. Quality control isconcerned with the search for faults or defects in the various products of software development.Reviews are also more cost-effective because they consume fewer resources than do tests. They areshort, require small groups of reviewers, and can be scheduled and rescheduled with minimum impact.Reviews are conducted during the process of the development, not at the end, as is the case withtesting. Quality control has as its mission the detection and elimination of defects in the product.Reviews are the front line in this mission. Costs of defects rise very steeply the longer they remain in theproducts. Reviews are aimed at finding the defects as they are created, rather than depending on thetest and operation phases to uncover them. (Horch, 2003)Horch (2003) tells that Reviews take on various aspects depending on their type. The two broad types ofreviews are as follows:  In-process (generally considered informal) review  Phase-end (generally considered formal) reviewIn-Process ReviewTable 2: In-Process Review Characteristics (Horch, 2003)Review Type Records Participants Stress LevelOne-on-one None Coworker Very lowWalk-through Marked-up copy or defect Interested project Low to reports members mediumStructured walk-through Defect reports Selected project Medium(Informal) membersInspection (Formal) Defect report database Specific role players HighOne-on-one reviews as table shows are not recorded and Configuration Management is not needed atthe same time. Coworkers and peers help the producer in indentifying the bugs if present and its stresslevel is very low. Producer can then remove those bugs before handing it over to the professional testerteam.Walk-through is enhanced type of one-on-one review where a group of coworkers/peers sit togetherand pin points the bugs. Its stress level is low to medium means bugs are recorded publicly and canincrease the stress for the producer to resolve them before handing it over for UAT. Participants areproject members who are interested in the project. As it is informal review which is not scheduled butstill its agenda is clear and product to be reviewed is shared with the project members for their betterKaleem Urrehman Page 13

understanding before start working on finding the bugs, which is why its defects are recorded for theproducer help.Inspection is very much similar to walk-through but requires more specific cast of participants, moredetailed minutes and action item reporting. There are usually two meetings i.e. mini walkthrough inwhich producer walks through the product with the participants and moderator confirms that eachparticipant has got necessary document for review and then second meeting where participants canoffer their suggestions to improve the product. Its defect list is recorded for the producer help.In-process Audits – can be informal as the Software Development Life Cycle progresses and informalaudits are used to assess the product’s compliance with those requirements for the various products.Phase-end ReviewPhase-end reviews are formal reviews that usually occur at the end of the SDLC phase, or work period,and establish the baselines for work in succeeding phases. Unlike the in-process reviews, which dealwith a single product, phase-end reviews include examination of all the work products that are to havebeen completed during that phase. A review of project management information, such as budget,schedule, and risk, is especially important. (Horch, 2003)Table 3: Phase-end Review Subject Documents (Horch, 2003)Review RequiredSoftware requirements Software requirements specification Software test plan Software development plan Quality system plan CM plan Standards and procedures Cost/schedule status reportPreliminary design Software top-level design Software test description Cost/schedule status reportCritical design Software detailed design Cost/schedule status reportTest readiness Software product specification Software test procedures Cost/schedule status reportSoftware TestingAnon. (2011) tells that software testing is an activity to check whether the actual results match theexpected results and to ensure that the software system is defect free. Software bugs can potentiallycause monetary and human loss, history is full of such examples: China Airlines Airbus A300 crashingdue to a software bug on April 26, 1994 killing 264 innocent lives. In 1985, Canadas Therac-25 radiationtherapy machine malfunctioned due to software bug and delivered lethal radiation doses to patients,leaving 3 people dead and critically injuring 3 others. In April of 1999, a software bug caused the failureof a $1.2 billion military satellite launch, the costliest accident in history. In May of 1996, a software bugKaleem Urrehman Page 14

caused the bank accounts of 823 customers of a major U.S. bank to be credited with 920 million USdollars.Anon. (2011) reveals that there are 7 principles of testing as follows: 1) Testing shows presence of defects 2) Exhaustive testing is impossible - Consider a scenario where you are moving a file from folder A to Folder B. Think of all the possible ways you can test this. Apart from the usual scenarios, you can also test the following conditions. Try to move the file when it is Open. You do not have the security rights to paste the file in Folder B. Folder B is on a shared drive and storage capacity is full. Folder B already has a file with the same name; in fact the list is endless. Or suppose you have 15 input fields to test, each having 5 possible values, the number of combinations to be tested would be 5^15. If you were to test the entire possible combinations project EXECUTION TIME & COSTS will rise exponentially. Hence, one of the testing principle states that EXHAUSTIVE testing is not possible. Instead we need optimal amount of testing based on the risk assessment of the application. 3) Early Testing - Testing should start as early as possible in the Software Development Life Cycle. So that any defects in the requirements or design phase are captured as well more on this principle in a later training tutorial. 4) Defect Clustering - a small number of modules contain most of the defects detected. 5) Pesticide Paradox - If the same tests are repeated over and over again, eventually the same test cases will no longer find new bugs. To overcome this, the test cases need to be regularly reviewed & revised, adding new & different test cases to help find more defects. 6) Testing is context dependent - Testing is context dependent which basically means that the way you test a e-commerce site will be different from the way you test a commercial off the shelf application 7) Absence of errors is a fallacy - Finding and fixing defects does not help if the system build is unusable and does not fulfill the user’s needs & requirementsMyers (1979) tells that the testing of even a trivial program is not an easy task. And if this is true,consider the difficulty of testing a 100,000-statement air-traffic-control system, a compiler, or even amundane payroll program.Testing LifecycleRizwan (n.d.) reveals that the testing phases typically associated with a project:  Test Strategy  Test Planning  Test Execution  Testing may also be involved in the Post-Implementation Review phase.Test Strategy – This phase will define the scope and requirements for testing a project. Specifically, thestrategy will answer questions about what types of testing will be used on a project, who will beperforming testing, how much will be done, when it will occur, and which testing standards, procedures,and tools will be used by the project team. The primary deliverable from this phase is a test strategydocument.Kaleem Urrehman Page 15

Test Planning – This phase will consist of the necessary organization and preparation work required foreach level of testing (i.e. unit, system, etc.). As part of this phase, the project’s requirements arecollected, organized, and reviewed for preparing test cases, identifying sources of test data, setting uptest areas (i.e. libraries, databases, etc.), determining test runs, and writing actual test plans and testcases.Test Execution – This phase will be the actual execution, verification, and resolution activities requiredin performing and carrying out the testing. As part of this phase, test data and expected results are setup, test runs are executed, actual test results are reviewed, and any implementation activities areperformed. Included in this phase are any activities related to identifying, resolving, re-testing, andrecording any discrepancies found in the testing.Post-Implementation Review – This phase will be performed a few weeks or months after systeminstallation. It will address the following questions:  What fixes or requirements are needed to the system?  How can the next project be more successful?  Have the business objectives been met?  How satisfied are the clients or users?  Have the claimed benefits (e.g. market share, customer satisfaction, productivity, efficiency, cost reduction and user morale) been achieved, or are they likely to be achieved anytime soon?  How satisfied or comfortable are the groups who have been given the responsibility to operate and maintain the on-going system?  Did the transition to the new system proceed smoothly?  How effective was the overall development process, and how should it be improved?  Is the delivered result sufficiently flexible and robust to accommodate both ongoing and evolving business needs?  How effective is the on-going support for the installed system?  Have standards been complied with?  Did the project meet time, cost and quality goals?  What are the nature, frequency and severity of post-implementation defects?  Are there any post-implementation defects to fix or fine-tuning improvements worth making at this time?  Have the implementation test results been retained and archived?  Did the project turnover include an on-going, re-usable regression test facility?V – Model (Testing Framework)There are different software development life cycles depending upon the context of theproject/product. Ahmed (2010) tells that V-model is a framework to describe the software developmentlife cycle activities from requirement specification to maintenance. The V-model illustrates how testingactivities can be integrated into each phase of the software development life cycle.Let’s take the example of waterfall model, the earliest model with sequential stages; to betterunderstand the different levels of testing involved in each phase of development life cycle.Kaleem Urrehman Page 16

V model of testing is developed where for every phase, in the Development life cycle there is acorresponding Testing phase. The left side of the model is Software Development Life Cycle – SDLC. Theright side of the model is Software Test Life Cycle – STLC. The entire figure looks like a V, hence thename V – model (Anon., 2011)There are four levels of testing i.e. unit testing and integration testing (both also are known as Structuraltesting) and system testing and acceptance testing (both also are known as Functional testing)Test Plan Development Acceptance Testing - When user requirements are taken from the client and properly documented to be approved by both parties (user and development team) e.g. Requirements Specification etc. Only then Acceptance testing plan and writing acceptance test cases starts. This testing phase is usually done by the client. (Anon., 2011) tells that focus of Acceptance test is not to find defects but to check whether meet client requirements .Since this is the first time, the client sees their requirements which is plain text into an actual working system. Acceptance Testing can be done in two ways. Alpha Testing – A small set of employees of the client in our case employees of the bank will use the system as the end user would. Beta Testing – A small set of customers in our case bank account holders will use the software and recommend changes System Testing - When System requirements are documented i.e. use cases and system sequence diagrams etc. then system testing plan and system test cases are written. This testing is done by professional Testers and it is known as System Testing. (Anon., 2011) tells that system testing checks complete end to end scenarios, as the way a customer would use the system. Integration Testing - When detailed design documents are finished i.e. deployment diagram etc then Integration testing plan is made. This testing is done by the developers and it is known as Integration Testing. Unit Testing - When components design is finished i.e. packages, sub-system diagram and class diagram then unit/component testing plan is made. This testing is done is by developers on their side before handing the product over to the testing team for proper testing and it is known as Unit Testing.Test Plan Execution Above model shows that testing starts from the first phase of development cycle and continues till the end of the development cycle --- these all four testing plans and testing documents are created from top to bottom and are then executed from bottom to top.Waterfall model follows V-model framework. This model has a number of advantages anddisadvantages. Ould (1999) tells that the advantage are that it does give a structure to the whole of aproject in the form of a well-defined sequence of phases – well defined in that we can define clear exitcriteria for them. It recognizes that there is no perfect solution and that it is not commercially sensibleKaleem Urrehman Page 17

to look for one. It might be commercially preferable to accept a sub-optimal solution in order to make aprofit and/or to satisfy the client. As such the model makes a good management tool.Ould (1999) also reveals that the disadvantage is that a compromise can in the limit become an errorand project can end up discovering, too late in the day, that it took a wrong turn earlier on. The V-modelperpetuates this disadvantage. Because there is no explicit admission that there can be risks, theireffects are always felt at the last moment rather than anticipated as early as possible.Figure 3: Waterfall Phases with V&V Model Extension (Huo et al., 2004)V Process model should be used when there is a good system specification and a ‘predictable’ solutionleading towards less risk or measurable risks/controlled risks.RiskAccording to Hambling (2007), risk is a factor that could result in future negative consequences, usuallyexpressed as impact and likelihood… Calculation of the risk will be:Kaleem Urrehman Page 18

Level of risk = probability of the risk occurring * impact if it did happenBoehm (2003) tells that methodologies such as Extreme Programming (XP), Scrum, and agile softwaredevelopment promise increased customer satisfaction, lower defect rates, faster development times,and a solution to rapidly changing requirements… However, both agile and planned approaches havesituation-dependent shortcomings that, if left unaddressed, can lead to project failure.Hambling (2007) reveals that in a project a test leader will use risk in two different ways: project risksand product risks.Project risks include:  Supplier issues o Failure of a third party to deliver on time or at all o Contractual issues, such as meeting acceptance criteria  Organizational factors o Skills, training and staff shortages o Personal issues o Political issues, such as problems that stop testers communicating their needs and test results. o Failure by the team to follow up on information found in testing and reviews (e.g. not improving development and testing practices). o Improper attitude toward or expectations of testing (e.g. not appreciating the value of finding defects during testing)  Technical issues o Problems in defining the right requirements o The extent that requirements can be met given existing project constraints o Test environment not ready on time. o Late data conversion, migration planning and development and testing data conversion/migration tools o Low quality of the design, code, configuration data, test data and testsPotential failure areas (adverse future events or hazards) in software are known as product risks, as theyare risk to the quality of the product. In other words, the potential of a defect occurring in the liveenvironment is a product risk. Examples of product risks are (Hambling, 2007):  Error – prone software delivered  The potential that a defect in the software/hardware could cause harm to an individual or company  Poor software characteristics (e.g. functionality, security, reliability, usability, performance)  Poor data integrity and quality (e.g. data migration issues, data conversion problems, data transport problems, violation of data standards)  Software that does not perform its intended functionsIterative Life Cycle – Most of the development life cycles are not sequential e.g. iterative life cyclewhich has many small/sub life cycles known as iteration. Anon. (2011) tells that apart from V model,there are iterative development models, where development is carried in phases where each phase isadding functionality to the software. Each phase comprises of, its own independent set of developmentKaleem Urrehman Page 19

and testing activities. Good examples of Development lifecycles following iterative method are RapidApplication Development, Agile DevelopmentOnce one small lifecycle is properly tested and second small iteration testing phase is completed, thenintegration testing is taken place i.e. combining both life cycles’ product together and again tests it as anintegrated system. Here comes the concept of Regression Testing as well which becomes the ongoingand important part for quality of the product. Regression testing is done to validate new changerequirement(s) has not affected the previously tested component.Agile Development – Now we take under consideration of agile development. XP extremeprogramming is one of the renowned agile development life cycle models. Graham (2008) reveals thatthe methodology claims to be more human friendly than traditional development methods. Below arebasic advantages that XP offers:  It takes help from business stories to understand the product functionality.  It requires regular feedback from the clients and client should do user acceptance testing for this.  It involves pair programming and share ownership in between the developers.  It requires test scripts should be written before code is going to start.  It requires test scripts should be automated.  It says that integration testing should be done many times in a day.  It says about simple solutions to meet today’s challenges.In XP, there are several small iterations where iteration is tested individually, after that, in integratedmanner and then regression testing is done on continuous basis whenever new change is done.Agile Model Feasibility  For each work product, there is a testing process.  Testing process starts from first phase till the finish phase of iteration  Should start creating tests plans and test documents during its respective development phase  Testers should be involved in reviewing documents.Ahmed (2010) reveals that if we need to deliver more than 120,000 lines of source code per year thenAgile models do not suite.This shows that applications of code size less than 120,000 can adopt agile methods as the best optionof development.Ahmed (2010) tells that one more requirement of agile models is that the team should consist ofbrilliant coders.That means development team should have extensive experience of development. It is developers’responsibility to understand the requirements and then convert it into code in such a way that less bugsshould appear at the end of the iteration.Kaleem Urrehman Page 20

Ahmed (2010) tells that In the middle of your project, your customers want something different thanwhat they originally thought they want. You tried to accommodate as much as possible. The result issomething which is neither this nor that. Agility has its own limitations.It shows that project life cycle cannot be changed in the middle of development and result can bemixture of previous lifecycle and new method i.e. agileAgile Quality AssuranceTheunissen et al. (2003) tells that the Quality Assurance Process is a process for providing adequateassurance that the software products and processes in the project life cycle conform to their specifiedrequirements and adhere to their established plans. To be unbiased, quality assurance needs to haveorganizational freedom and authority from persons directly responsible for developing the softwareproduct or executing the process in the project. Quality assurance may be internal or externaldepending on whether evidence of product or process quality is demonstrated to the management ofthe supplier or the acquirer. Quality assurance may make use of the results of other supportingprocesses, such as Verification, Validation, Joint Reviews, Audits, and Problem Resolution.Table 4: Agile QA vs. Traditional QA (Gupta, 2008)Criteria Traditional QA Agile QA Upfront Constant interaction within the team and with the Product OwnerUnderstandingRequirements Plan one time delivery which Continuous sprint by sprint planning, has to happen way later deliver important features firstPlanningDesigning of Test All Upfront Evolving and Iteration WiseCases Prohibit Change Adapt and adjust at every release and iteration boundaryManage Change Milestone document review See working software every iteration and every releaseProgress reviewQuality Responsibility Quality Engineer Entire teamStamelos (2007) reveals that, from an agile perspective, quality has been defined by some practitionersas follows:Agile quality assurance as the development of software that can respond to change, as the customerrequires it to change, this implies that the frequent delivery of tested, working, and customer-approvedsoftware at the end of each iteration is an important aspect of agile quality assurance.Kaleem Urrehman Page 21

Figure 4: Agile Methods and QA (Huo et al., 2004)Agile quality [is] a result of practices such as effective collaborative work, incremental development, anditerative development as implemented through techniques such as refactoring, test-drivendevelopment, modeling, and effective communication techniques. (Ambler, 2005)Agile Quality TechniquesStamelos (2007) reveals that, these following aspects of agile quality have eliminated the need for heavydocumentation that is prescribed in traditional processes as a requirement for quality.Table 5: Agile quality techniques as applied in extreme programming (Stamelos, 2007)Technique DescriptionRefactoring Make small changes to code, Code behavior must not be affected, Resulting code is of higher quality (Ambler, 2005).Test-driven development Create a test, Run the test, Make changes until the test passes (Ambler, 2005).Acceptance testing Quality assurance test done on a finished system, Usually involves the users, sponsors, customer, etc. (Huo et al., 2004).Continuous integration Done on a daily basis after developing a number of user stories. Implemented requirements are integrated and tested to verify them. This is an important quality feature.Pair programming Two developers work together in turns on one PC, Bugs are identified as they occur, Hence the product is of a higher quality (Huo et al.,Kaleem Urrehman Page 22

2004).Face-to-face communication Preferred way of exchanging information about a project as opposed to use of telephone, email, etc. Implemented in the form of daily stand-up meetings of not more than twenty minutes (Huo et al., 2004). This is similar to the daily Scrum in the Scrum method. It brings accountability to the work in progress, which vital for quality assurance.On-site customer A customer who is a member of the development team, Responsible for clarifying requirements (Huo et al., 2004)Frequent customer feedback Each time there is a release the customer gives feedback on the system, and result is to improve the system to be more relevant to needs of the customer (Huo et al., 2004). Quality is in fact meeting customer requirements.System metaphor Simple story of how the system works (Huo et al., 2004), Simplifies the discussion about the system between customer/ stakeholder/ user and the developer into a non-technical format. SimplicityAgile Quality ParametersAbove mentioned agile techniques are used to address the following agile quality parameters. A briefdefinition of each of agile quality parameters is given according to Meyer (2000):Correctness: The ability of a system to perform according to defined specification.Robustness: Appropriate performance of a system under extreme conditions. This is complementary tocorrectness.Extendibility: A system that is easy to adapt to new specification.Reusability: Software that is composed of elements that can be used to construct different applications.Compatibility: Software that is composed of elements that can easily combine with other elements.Efficiency: The ability of a system to place as few demands as possible to hardware resources, such asmemory, bandwidth used in communication and processor time.Portability: The ease of installing the software product on different hardware and software platforms.Timeliness: Releasing the software before or exactly when it is needed by the users.Integrity: How well the software protects its programs and data against unauthorized access.Verifiability: How easy it is to test the system.Ease of use: The ease with which people of various backgrounds can learn and use the software.Huo et al. (2004) reveal that the agile approach turns the traditional software process sideways. Basedshort releases, agile methods go through all development stages a little at a time, throughout theirsoftware development life cycle. In one agile release, the steps may not be clearly separated, as they arein a waterfall development model, but the requirements recognition, planning, implementation andintegration sequences are the same as in waterfall model… There is an important need for developers toknow more about the quality of the software produced. Developers also need to know how to revise ortailor their agile methods in order to attain the level of quality they require.Huo et al. (2004) reveal the following aspects of agile quality assurance:  Agile methods do have practices that have QA abilities, some of them are inside the development phase and some others can be separated out as supporting practicesKaleem Urrehman Page 23

 The frequency with which these agile QA practices occur is higher than in a waterfall development  Agile QA practices are available in very early process stages due to the agile process characteristics.Agile ManifestoBeck et al. (2001) tells that we follow these agile manifesto principles:  Customer Satisfaction: our highest priority is to satisfy the customer through early and continuous delivery of valuable software.  Continuous Changing Requirements: welcome changing requirements, even late in development. Agile processes harness change for the customers competitive advantage.  Sprints/ Small Releases: deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.  Team Effort: business people and developers must work together daily throughout the project.  Fast paced environment: Build projects around motivated individuals.  Give and take: give them the environment and support they need, and trust them to get the job done.  Face-to-face conversation: the most efficient and effective method of conveying information to and within a development team is face-to-face conversation.  Software is working: working software is the primary measure of progress.  Development at ease: agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.  Refactoring: continuous attention to technical excellence and good design enhances agility  Simplicity: the art of maximizing the amount of work not done--is essential.  Self-organizing teams: the best architectures, requirements, and designs emerge from self- organizing teams.  Regular Reviews: at regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.Agile Requirement DocumentationAmbler (2001) reveals that comprehensive documentation does not ensure project success; in fact, itincreases your chance of failure. Documentation should be concise: overviews/roadmaps are generallypreferred over detailed documentation. The best documentation is the simplest that gets the job done.Don’t create a fifty-page document when a five page one will do. Don’t create a five-page documentwhen five bullet points will do. Don’t create an elaborate and intricately detailed diagram when asketch will do. Don’t repeat information found elsewhere when a reference will do. Write in pointform. Document only enough to provide a useful context. Start with a document thats minimal enoughfor the needs of its customers then augment it as needed. To determine what is truly the minimumamount of documentation required by my customers I will actively explore how they intend to use thedocumentation and why they are using it that way.Kaleem Urrehman Page 24

Figure 5: Documentation throughout Software Development Phases (Ambler, 2001)Sehlhorst (2010) reveals that some collaboration is transient – communication happens right now, and isonly important right now. Other communications are persistent – the collaboration happens right now,but we need to remember later what we agreed upon and why. There are variations of “right now” andvarying degrees of “later”, but slightly oversimplifyingThere is communication for nowThere is communication for later (Both are important)Sehlhorst (2010) reveals that documentation, in the vision from your nightmares – giant requirementsdocuments, architectural analyses, and detailed psychographic profiles – is very inefficient whencommunicating for now. These massive documents, while getting in the way of a conversation, doprovide the opportunity to remember (in the future) why we make the decisions we make (today).Agilemostly believe in building software component first and then start documenting what is developed.Through this way, time is saved plus minimal effort is required to build the right product.Boehm (2002) tells that Highsmith and Cockburn emphasize that agile approach “are most applicable toturbulent, high change environments.” According to their world view, organizations are complexadaptive systems in which requirements are emergent rather than prespecifiable.1 However, while agilemanifesto tenets such as the second principle—welcome changing requirements, even late indevelopment— offer great potential for success, developers can misapply them, with disastrous results.Boehm (2002) tells that plan-driven methods work best when developers can determine therequirements in advance— including via prototyping—and when the requirements remain relativelystable, with change rates on the order of one percent per month. In the increasingly frequent situationsin which the requirements change at a much higher rate than this, the traditional emphasis on havingcomplete, consistent, precise, testable, and traceable requirements will encounter difficult toinsurmountable requirements-update problems. Yet this emphasis is vital for stable, safety-criticalembedded software.Kaleem Urrehman Page 25

Agile CustomersBoehm (2002) tells that in USC’s rapid-development electronic services projects, we have found thatunless customer participants are committed, knowledgeable, collaborative, representative, andempowered, the developed products generally do not transition into use successfully, even though theymay satisfy the customer. Participants in the XP 2001 “Customer Involvement in XP” workshop reachedsimilar conclusions. 8 Agile methods work best when such customers operate in dedicated mode with thedevelopment team, and when their tacit knowledge is sufficient for the full span of the application.Again, though, these methods risk tacit knowledge shortfalls, which the plan-driven methods reduce viadocumentation and use of architecture review boards and independent expert project reviews tocompensate for onsite customer shortfalls.Crispin et al. (2009) tells that the customer team includes business experts, product owners, domainexperts, product managers, business analysts, subject matter experts—everyone on the “business”side of a project. The customer team writes the stories or feature sets that the developer team delivers.They provide the examples that will drive coding in the form of business-facing tests. They communicateand collaborate with the developer team throughout each iteration, answering questions, drawingexamples on the whiteboard, and reviewing finished stories or parts of stories.Testers are integral members of the customer team, helping explicit [and implicit] requirements andexamples and helping the customers express their requirements as tests (Crispin et al., 2009)Agile Sprints/Small ReleasesCohn et al. (2003) tells that in Scrum [agile method], projects progress in a series of month longiterations called sprints Development teams meet with the client, or product owner, before each sprintto prioritize the work to be done and select the tasks the team can complete in the upcoming sprint.During the sprint, the team stays on track by holding brief daily meetings. At the end of each sprint, theteam delivers a potentially shippable product increment. Scrum is ideally suited for projects with rapidlychanging or highly emergent requirements such as Web projects or product development for newmarkets.Chowdhary (2011) tells that in today’s hasty world, stakeholders and customers want quick return ontheir investments. They don’t want to wait for longer periods for a full featured product. As a result,nowadays new software testing paradigm is catching momentum, i.e., Scrum approach. In agile scrum(sprint) process, projects are divided into small components to be developed and then to be tested inspecific time-slice called as sprint (small cycles). Each feature should get developed and tested in aspecified small time-slice.Kaleem Urrehman Page 26

Figure 6: Interaction of Roles (Crispin et al., 2009)Crispin et al. (2009) tells that the customer and developer teams work closely together at all times.Ideally, they are just one team with a common goal. That goal is to deliver value to the organization.Agile projects progress in iterations, which are small development cycles that typically last from one tofour weeks. The customer team, with input from the developers, will prioritize stories to be developed,and the developer team will determine how much work they can take on. They will work together todefine requirements with tests and examples, and write the code that makes the tests pass. Testershave a foot in each world, understanding the customer viewpoint as well as the complexities of thetechnical implementation.Agile Team EffortChowdhary (2011) tells that in the conventional development process which is based on phases, eachphase goes through thorough a lengthy validation before triggering the next stage while Agile testingdoes not emphasize rigidly defined testing procedures, but rather focuses on testing iteratively againstnewly developed code until quality is achieved from an end customers perspective. In other words, theemphasis is shifted from "Testers as Quality Police" to something more like "entire project team workingtoward demonstrable quality."Cross functional Team work is at the heart of Agile Testing. There is no “my work”, “I have finished mywork” and “your work”. On an Agile team, we find only “Our work”, “we have completed our Sprint”.Individuals will have helping tendency for sharing technical knowledge. Agile Members are alwaysavailable to team members rather than locked away behind closed doors. Team Lead will alwaysmotivate the teams and create a supporting learning environment. Team will always be sprint-orientedand often discuss smooth run of the sprint. An Agile team’s job is to self-organize around the challengesand management’s job is to remove impediments to self-organization. (Chowdhary, 2011)Crispin et al. (2009) tells that an agile team must possess all the skills needed to produce quality codethat delivers the features required by the organization. While this might mean including specialists onthe team, such as expert testers, it does not limit particular tasks to particular team members. Any taskmight be completed by any team member, or a pair of team members. This means that the team takesresponsibility for all kinds of testing tasks, such as automating tests and manual exploratory testing. ItKaleem Urrehman Page 27

also means that the whole team thinks constantly about designing code for testability. The whole-teamapproach involves constant collaboration. Testers collaborate with programmers, the customer team,and other team specialists—and not just for testing tasks, but other tasks related to testing, such asbuilding infrastructure and designing for testability.Agile RefactoringMake small changes to code, Code behavior must not be affected, Resulting code is of higher quality(Ambler, 2005).Boehm (2002) tells that with great developers and small systems, the assumption that refactoring isessentially free is valid… Empirical evidence indicates that with less-than-great developers, refactoringeffort increases with the number of requirements or stories. For very large systems, our Pareto analysisof rework costs at TRW indicated that the 20 percent of the problems causing 80 percent of the reworkcame largely from “architecture-breakers,” such as architecture discontinuities to accommodateperformance, fault-tolerance, or security problems, in which no amount of refactoring could put HumptyDumpty back together again.Anon. (2005) reveals that well known but often ignored in software development is the fact that thecosts of changes and repairs are significantly less during the design phase than at the testing phase.Since full advance requirement specification is almost never possible a realistic approach of conductingspecification, development and testing in synchronization is most effective. QA management expertisein the business problems for which the software solution is intended can translate into an effectivecontribution to early cost saving in the design process. The primary mission of QA activity must beconsistent with the goal of improving the overall productivity and profitability of the firm. Day to day QAmanagement is about the ability to support business trade-offs with good product information. It isimportant to find the balance between theoretical and practical solutions and to effectivelycommunicate that balance to decision makers. This balance is also required in determining appropriateprocess steps within QA to achieve cost effective testing and quality management of software solutions.Agile Multiple Testing TeamsKniberg (2007) tells that I hear both objections:“But that’s obvious! Scrum teams are supposed to be cross functional!”“Scrum teams are supposed to be role-less! We can’t have a guy who is only a tester!”Let me clarify, [tester is] “A guy whose primary skill is testing”, rather than “a guy whose role is to doonly testing”. The tester is the “signoff guy” – in addition to being “just” a team member, the tester hasan important job. He is the signoff guy. Nothing is considered “done” in a sprint until he says it’s done.I’ve found that developers often say something is done when it really isn’t. Even if you have a very cleardefinition of “done”, developers will frequently forget it. We programmers are impatient people andwant to move on to the next item ASAP (Kniberg, 2007).If the team is experienced and comfortable with Scrum, and there is a logical way of splitting theroadmap into two distinct tracks, and those two tracks don’t both involve the same source code, thenKaleem Urrehman Page 28

I’d say it’s a good idea to split the team. Otherwise I’d consider sticking to one team, despite thedisadvantage of big teams (Kniberg, 2007)My experience is that it is better to have fewer teams that are too big than to have many small teamsthat keeping interfering with each other. Make small teams only when they don’t need to interfere witheach other (Kniberg, 2007)Agile Exploratory TestingRay (2010) reveals that exploratory testing is simultaneous test design, execution and learning…Exploratory testing is not constrained in this way. The objective is to expose information that is of valueto stakeholders and assess the quality of the software, of which compliance with the specification is onlyone factor… Exploratory testers are always looking for emergent behaviors, which are behaviors thatcould not have been predicted prior to the start of testing.The exploratory tester starts with an outline test plan in mind, but can adapt it in response to changingcontexts. If one part of an application is not yielding bugs, it may make sense to test in other areasinstead. Alternatively the tester may consider using different test techniques in the same area.Exploratory testing is an efficient and effective approach to testing software. It produces useful resultsfrom the first hours of testing, and finds the biggest bugs faster than any other approach. Furthermore,it finds bugs that other approaches wont find at all. (Ray, 2010)Rothman (2008) tells that exploratory testing is helpful when working with the product owner and theproject team, trying to define user stories. Imagine youre working on a banking system and want toallow between-account transfers. The developers have a scheme to use checksums to verify the Fromand To account numbers. Take an example of insufficient checking for input and confirmation failure. Ifexploratory testers had been involved at the time of user story definition, testers might have asked,“What happens if we put in a longer account number than the system expects?” Or, during the modelingstage (which I certainly would expect on an agile project with a financial transaction system comprisedof databases), an exploratory tester could ask questions such as how many ways can we make thetransaction "fail?" Testers who ask questions like that all the time and explore the answers to thosequestions before the code is written may help the developers write better code. And, theyll have thebasis for some great, nasty tests to see what the system really does. The questions lead to the testdesign, which leads to the test execution, which provides learning for everyone on the project.You can see that each question leads to learning or test design. In the same way, each test design leadsto more questions or learning. The three pillars--test design, test execution, and learning--reinforce eachother. You dont need to differentiate among the activities; which activity a tester performs is notrelevant. What is relevant is that the tester performs all of them, and feeds back information to the restof the project team. (Rothman, 2008)Run high level tests allow testers to learn about an aspect of the applications behavior, and each testprovides information that allows ever more powerful tests to be devised. This knowledge is invariablylost in structured testing projects. Rather than ask does the application meet this requirement, askquestions like what happens if... (Ray, 2010)Kaleem Urrehman Page 29

Agile Regression TestingAnon. (n.d.) reveals that any time you modify an implementation within a program; you should also doregression testing. You can do so by rerunning existing tests against the modified code to determinewhether the changes break anything that worked prior to the change and by writing new tests wherenecessary. Adequate coverage without wasting time should be a primary consideration whenconducting regression tests. Try to spend as little time as possible doing regression testing withoutreducing the probability that you will detect new failures in old, already tested code. Some strategiesand factors to consider during this process include the following:  Test fixed bugs promptly. The programmer might have handled the symptoms but not have gotten to the underlying cause.  Watch for side effects of fixes. The bug itself might be fixed but the fix might create other bugs.  Write a regression test for each bug fixed.  If two or more tests are similar, determine which is less effective and get rid of it.  Identify tests that the program consistently passes and archive them.  Focus on functional issues, not those related to design.  Make changes (small and large) to data and find any resulting corruption.Winkels (2010) reveals that the target area of the regression test is the complete set of (end-to-end)business functions that a system encompasses. The same tests are executed repeatedly over time toensure that the required behaviors of the system remain stable. In an agile context this means thatregression test results are of great value to the product owner at the end of each sprint, when he has toevaluate and accept not only the newly developed functionality, but the overall functionality of thesystem after the changes have been made. Ideally, there should be no regression in the functionality ofthe system. In real-life, complex systems however, a trade off can be made between the immediatevalue of the new functionality and regression bugs that have (unintentionally) been introduced.SummaryTesting and Quality Assurance are two important activities involved in agile software development.Quality can be improved and risks impact can be reduced using quality and testing techniques. Thischapter deals with commonly used QA and Testing methodologies and techniques which can becompared with dissertation research method i.e. case studies (next chapter) to come up with my ownconclusion and recommendations.Kaleem Urrehman Page 30

3. Research MethodsThere are 8 research methods where I can choose anyone for my dissertation, which are as follows:Case Studies: It is detailed study of the fewer cases or examples. Data collection technique used in thismethod ranging from interviews and observations or quasi-experiments; Because of the sensitivity andnumber of the context, there are somehow limitations to conclude on one point. But at the same time,as we are only focusing on particular side of the topic, its intensive nature can help you to concludesomehow fruitful.Quasi-experiments: It is an observational study done on groups that possess similar properties. Casualfactors affecting the groups are manipulated and correlation is determined on the basis of study. Thistype of observational study may not produce optimal result as some other hidden/not studied factorsaffect the study output. Hence I am not using it as dissertation research method.Automated Logging: Computer dependant study which records sequence of electronic communicationsor activity logs that can be precise and accurate with proper timings; this gives better Performancecomparison of activities in terms of speed; but it cannot record other factors that may be of any interesthence I am not using it as dissertation research methodObservation: It is a broader scheme of investigation. The basic idea behind it is Watch and Record ofbehavior in context in natural environment. There are two important parts affecting this study i.e. levelof observant interest and observing power of the observer. Findings can be kept in form of descriptionsas notes or audio files or video files for future references as well. Fundamentally this kind of studytotally depends upon quality of the observer. I feel this type of study is pretty much time taking and notfruitful to the type of the project which I am dealing with, hence I am not using it as dissertationresearch method.Interview: Already revised questions are planned which should be asked to the interviewee. The qualityof the output depends upon the chemistry between both the parties but potential for bias anddistortion is high at the same time. Interview can be done face to face or electronically or eventelephonic interview can provide useful data. I am not using it as research method as it requires fullyinvolvement of the interviewee.Survey Search & Questionnaire: It is a structured interview where already planned questions should beasked in such a way where one answering the question should not find it difficult for one self. It requiresthe honesty and involvement of the person answering that can affect the output statistics. Hence, I amnot using it as my research method.Controlled Experiments: It is the study of testing hypothesis generated from theories by doingsystematic manipulation of variables under controlled conditions. One dependant variable and manyindependent variables are calculated and changing the values of independent variables, affect on theKaleem Urrehman Page 31

dependant variable is recorded. As it is still an experiment, not a natural way of depiction, hence due toits limitation, I am not using it dissertation research method.Document Studies: Last but not the least here comes the Document Studies which provide authenticityand a powerful way of concluding the final result. One has to study the books, journals, websites andnewspaper to gather important information to let one conclude useful data.Case Studies as Research MethodCase study research method is an empirical inquiry that investigates a contemporary phenomenonwithin its real-life context; when the boundaries between phenomenon and context are not clearlyevident; and in which multiple sources of evidence are used (Yin, 1984) Soy (1996) reveals that case study research excels at bringing us to an understanding of a complex issueor object and can extend experience or add strength to what is already known through previousresearch. Case studies emphasize detailed contextual analysis of a limited number of events orconditions and their relationships. Critics of the case study method believe that the study of a smallnumber of cases can offer no grounds for establishing reliability or generality of findings. Others feelthat the intense exposure to study of the case biases the findings. Some dismiss case study research asuseful only as an exploratory tool. Yet researchers continue to use the case study research method withsuccess in carefully planned and crafted studies of real-life situations, issues, and problems. Reports oncase studies from many disciplines are widely available in the literature.SummaryI have chosen case study out of 8 available research methods where I will find couple of companies indifferent scenarios and then work on them to lead to my dissertation conclusion.Kaleem Urrehman Page 32

4. Case Studies ImplementationAs research method chosen by me is Case Studies, I have opted two case studies as follows:Let’s take first case study:First Case Study: Integrating Testing in Agile ProjectsOverview of the CompanyCompany A is the worlds premier insurance brokerage, consulting services and consumer insuranceunderwriting organization. The IT group builds integrated solutions to link human resources or peoplestrategies with business strategies. (SoftO2, 2001)ChallengesTask in this project was to set up software testing in an agile software development environment.As in agile, development is very fast and developers are getting requirements from the client directly.Through this way, developers have better understanding of the requirements and can then negotiatewith the client regarding what is possible practically and what is not. With regular correspondence ofthe client, development speed automatically increases and here now challenge of product testing before‘client tests it’ comes. How should that challenge be addressed is really a big question to deal with.SolutionApproach was to involve - besides the QA team - the product owner, Scrum Master, business analystsand developers in building an effective environment with best practices and sprints that allow forsmooth incorporation of software testing. Along with new practices, training of the QA team on the keyagile concepts relevant for testing, agile test planning, and close collaboration with developers, and onexploratory testing techniques.Agile process where testing was not introduced properly and developers are supposed to do testing ontheir own will obviously increase the chances of appearing bugs in the product. The solution provided inthis case study was to build an environment where developers and testers can communicate moreclosely. Also client + business analysts + Test Manager and obviously developers working in a teamenvironment can lead to better requirement understanding hence preventing bugs appearing in theproduct.Agile tester new team training is mandatory step taken into scenario so that testers can be on samelevel as any other member of the team. Best testing techniques were taken into consideration e.g.exploratory or monkey testing. Exploratory testing helps to start testing from any point of the productand see if any bugs appear.ResultThe result of the work was a shift in the IT teams perception and participation in building quality withfewer cycles. The QA team mastered the skills to strategize and drive effective testing activities in theKaleem Urrehman Page 33

Sprint planning meetings and during the iterations and this with active involvement of colleagues withintheir respective cross-functional teams.Encouraging each team member to work together in scrum agile process where scrum master (testmanager) can plan each sprint (small iteration, may be ranging from one week to four weeks) and callthe meetings for each sprint so that can perceive where the team is lacking and can overcome it.Second Case Study: Agile Software Development in Large OrganizationOverview of the CompanyCompany B was founded in 1975 is the worldwide leader in software, services, and solutions that helppeople and businesses realize their full potential.ChallengesParihar (2008) reveals that the challenges are:  Requirements were changing on daily basis  Customer wanted to be part of the development closely  Budge was a not a constraint, but delivering lot of functionality within fixed deadline was a challenge  End date to deliver the project was fixedSolution  Everyone to tell his 8 hours work done yesterday, roadblocks, learning in daily meetings…meeting documented by scrum master  Daily scrum to be completed in 15 minutes  Buddy Testing, very effective  Work closely with Dev team, till bugs get fixed  Three teams within testing team: o White box test team – Code review, writing testing scripts, o Acceptance test team – Writing acceptance features and UI tests o System Test teamComparative StudiesNow we will compare both the case studies in a way that what challenge one case study was facingsomehow differs from the second case study. What quality and risk factors affecting the company forcesthem to go for proper planned agile testing is mentioned below in tabular form:Kaleem Urrehman Page 34

Table 6: Comparison between two case studies Factors affecting Case Study A Case Study B Quality and RisksWas agile testing No Yesintroduced already Team Effort + Exploratory Team Effort + Three testing teams Testing + Testers Training for [code review + UI/acceptance +Quality Systems system testing]Requirements Minimal Documentation Continuously ChangingChange Control Refactoring Refactoring Regular feedback + Sprints Client Direct Involvement + Small ReleasesCustomersComparison shows which quality factors were addressed in Case Study A and how it is dealt with andsame scenario is repeated for Case Study B as well.As we can see, Company A was not implementing agile testing before, so it was a challenge to newlyintroduce this whole testing process in the organization. That is why testers’ training was required whoare good in exploratory testing. Also it was enforced in testing department and other departments towork mutually to achieve success in agile environment. Obviously continuously changing requirementsresult in refactoring making developers change in the design or code of the software to meet userrequirements. Customer’s feedback is proved to be an essential input with sprints where eachrelease/sprint is going to released after regular interval (2-4 weeks) for customer review. Other benefitof agile methods is minimal documentation, as software component is developed first and then itsdocumentation can be started properly.Now, Company B was actually dealing with agile environment already but one concern was continuouslychanging requirements which were better dealt with refactoring (changing in the code so that behaviorof the code is not affected) and at the same time customer wants to involve in the project which in turnproves to be good thing, as agile needs customer interaction/feedback on regular basis specially for eachsprint. As each sprint is scheduled after fixed period of time, on-time delivery challenge was also fixed.SummaryCase Studies for two different agile companies are taken into consideration and look into the challengesthey were facing and then trying to compare them on the basis of different quality and risk factors.Kaleem Urrehman Page 35

5. Analysis Quality and Literature Review Risk FactorsQualitySystems Team Effort As it is shown in above chapter “Case Studies Implementation” that case study A is dealing with new agile testing setup and case study B was facing daily requirement changing challenge and solution provided for both cases was Team Effort. In Literature Review, I can read that not only QA/Tester is responsible for process/product quality but everybody in the team has responsibility is to ensure quality. There is no ‘my work’ term used like I have done my work or I have done your work but instead it is our work done used. Start helping and start sharing technical/project knowledge to each other. It is team leader role to make it sure that agile friendly environment is running smoothly and each member is ready to help others, if needed. Tasks are not always dedicated to only one particular team member, it can be assigned to any team member or pair of members. Also testers work with developers not just for testing, but also focusing on test-related tasks likewise building infrastructure and designing testability. Scrum Mater, Business Analyst, Product Owner, Tester and Developers have to work in a team to help setting up a new agile testing environment and to deal with daily requirement changing challenge. Testing Teaming Case study B gives an idea about multiple testing teams for different scenarios of tasks i.e. code review testing team, acceptance testing team and system testing team. In literature review, It is said that tester is not who does testing but tester is who has a skills of testing. Actually tester is a sign-off guy. It is his responsibility to pass the current patch/release/sprint to the customer. Developers are very impatient persons and they want to move to next step and that is why they said task is done but actually it is not done. Making teams in testing totally depends upon the nature of tasks where each team is dealing with its own developers’ code. If there are more number of testing teams with each team having small number of resources sometimes leads to each team communicating with each other because they are sharing partial development code to test. It is not a good idea that is why testing teams should have a clear line that distinguishes them with each other. Having 3-4 teams of 10-15 testers in each team is good where every team is communicating with its team members and all the teams are working independently and inter-team communication frequency is very low.Kaleem Urrehman Page 36

Testers Training Case study A is dealing with setting up a new testing agile environment where new testers should be trained properly so that they can easily adjust to the agile environment. In literature review, it is said quality assurance/testing is different in agile as compared to traditional testing or quality assurance techniques. In agile, for each iteration/sprint/small release, it is team work who leads to handle agile challenges like: requirement changing and team communication etc. For agile testers it is necessary to understand the meaning of agile i.e. team effort, critical and analytical thinking, keep tracking of test related things on regular basis (daily 15 minutes meeting) and regular communication with developers, scrum master, business analysts, project managers and all other stakeholders. Exploratory Cast study A was dealing with exploratory testing. In literature Testing review, it can be seen that exploratory testing let the tester to execute high level test cases which gives him better understanding about the application. For exploratory testing, no special planning is needed, tester just start executing test cases which are in his mind. At the same time, if tester is not able to find bugs then he should start doing exploratory testing. It gives him the starting point for testing. It finds bugs that no other testing approach can find. It can find major and severe bugs in first hour of testing. In regular testing, where it is verified that application meets customer requirements, in contrast to that exploratory testing ask questions like what happens if I do this or that. Regression testing is also important aspect where tester can see the impact of one fixed bug on other modules of the application where some other bugs have now been introduced which were not present before. Agile promotes exploratory testing as it is the fastest way of start capturing the bugs and also helps to build the application understanding.Requirement Minimal Case study A is dealing with requirements challenge which is solved Documentation with minimal documentation so that more time can be saved and be consumed in development. In literature review, it is said that detailed documentation does not ensure project success actually it contributes in failure of the project. There is no need to write 5 page documents when 5 bullet points can serve the same purpose. Requirement documentations should be simple and concise. At the same time, if nothing is developed yet, there is no need to create long detailed documentation.Kaleem Urrehman Page 37

Documentation is for customer, make is simple and then augment it with the passage of time according to customer expectations. Agile gives customer feedback in 2-4 weeks and as requirements are amending on regular basis, there is no need to spend time in writing documents but work on user stories and save this time in start developing something. Continuously Case study B is facing continuous changing requirements challenge Changing which, in a way, is handled with minimal documentation and also start developing something and then progress further with regular customer feedback. In literature review, it is said that it is agile methods beauty that it is flexible and can accommodate the new or changing requirements. Each release has duration of 2-4 weeks. Any new requirements can then be entertained after 2-4 weeks which in turn does not affect the previous releases so much and change is coding is possible. In traditional development methodologies, change requirements have to wait for 3 – 9 months, until current release is not completed. On the other hand, agile provides small releases of 2 – 4 weeks where if requirements are changing, still there is not enough work you now need to do in changing the design and coding of the current release. Changing the old design for new requirements require QA testers to review it after developers have built it. To achieve good quality, testing (reviews) plays an important role.ChangeControl Refactoring Both the case studies deal with change control challenge which is solved with refactoring. In literature review, refactoring is the process of change in code and resultant code should be of higher quality. In traditional way of development, we have seen that cost of change in requirements is less in design phase rather than in testing phase. But at the same time it is agile beauty that provides flexibility to change in requirements. New requirements are dealt with simple change in design and simple change in code which fulfills the customer’s requirements. New requirements are of high risk to the current software build. As soon as new requirements are merged with the current code, from nowhere software failure risks arise. To overcome this risk factor, special reviews are done in design phase by QA resources to see if in any way it will result in system failure. Refactoring is the process which helps to change in design/code behavior that new design/code is of better quality.Customer Sprints/Small Both the case studies deal with small releases which alternatively Releases are also known as sprints. In literature review, team memberKaleem Urrehman Page 38

prioritizes the tasks of each team and assign them for each sprint. Daily 15 minutes brief meeting is called to support requirement changing phenomena. Developers are the first contact to the customers most of the time and they are communicating each other on regular basis for better understanding of the application and developer can then develop the application as per customer expectation. Regular Both the case studies involve with customer interaction with the Feedback development team. In literature review, it is said that customer is reviewing each sprint/release output and can then communicate with the developers and scrum master to tell them about feedback. No customer wants its return on investment late, through sprints customer get something developed and he can see himself the progress. Customers want to involve in the software development, agile gives customer this opportunity to start reviewing small releases and start giving the feedback back to developers hence customers can see actually where software stands as per expectations.Kaleem Urrehman Page 39

6. ConclusionThe basic dissertation question was: “How can we manage risk and quality aspects in agile methods”,and after detailed research done (Literature Review) and research method used (Case Studies),comparing the pool of information with each other – I can conclude that agile methods have thefollowing quality aspects and risk factors which causes low quality and high risk impact on the agilesoftware:  Requirement Change Control – customer is changing requirements rapidly  Team Effort – Less communication within the department and inter-departments can lead to less quality  Tester Resources Management – QA and testing team members should aware of agile standards and agile testing skillsWith the research done this dissertation, I can conclude that we can overcome the above quality andrisk factors – below are the answers:  Requirement Change Control with Refactoring + Minimal Documentation  Team Effort with good communication level in between developers, testers, business analyst, scrum master  Tester Resources Management – training is given if required + testing techniques introduced like exploratory testing if needed + different testing teams can be made for better testing resultsAfter the research done for this dissertation, recommendations are:Agile Training – a company which is going to start implementing agile environment or a company whichis already working in agile environment, both should train its resources properly as per the agilestandards.Make reasonable documentation – company working in agile environment should not waste its time inmaking long documentation and start investing that time into developing system. To make therequirements understandable, there is no need to write 5 pages, where 5 bullet points can serve thesame objective.Customer Interaction – Let the customer test the application. Sometimes customers want to be the partof the software development that is a good sign. But if customer is not playing an active role related toregular feedback, then customers should be encouraged to do so. Because ultimately it is customer whois going to use that applicationQA/Tester Teams – Onsite testers (who are working on behalf of the customer) should be trainedproperly as per agile environment standards. It should be taken into consideration that it is a team effortgame, where each member of the team should work closely to get fruitful results out of the softwaredevelopment process.Kaleem Urrehman Page 40

Making a team of different testers sometimes is a good idea but one thing should be taken care of isthat those teams should have minimum level of communication in between each other as their tasksand duties are different from each other. Each testing team should have its own set of developmentcode to test. Even if the team size is increased to 10-12 members that is fine unless there is cleardistinguished line of that team with other testing teams. On the other hand, making multiple teams of 3-5 members who are actually corresponding to other teams does not make sense and it is better tomerge them in a single team.Refactoring – The risk of continuous changing requirements can be dealt with refactoring so that changein design or code brings better change in software overall behavior and also at the same time, managingthe software in scheduled time of 2-3 weeks per sprint does not let the customer to wait for a longerperiod of time. Through this way, agile says welcome to new changes.In total, I can conclude that agile methods deal with team effort, testers’ multiple team, testers’ training,new testing techniques (exploratory testing) and refactoring and customer regular involvement.Kaleem Urrehman Page 41

7. References Ahmed A (2010) Software Quality Assurance and Testing USA: Auerbach Ahmed A (2010) Testing as a Service USA: Auerbach Ahmed A (2010). Agile model suited for which projects Available: http://ahmedashfaque.wordpress.com/2010/03/27/agile-model-suited-for-which-projects Last accessed 15th May 2011 Amber S (2005) Quality in an Agile World Available: http://www.ambysoft.com/downloads/agileQuality.pdf Last accessed 12th July 2011 Ambler S (2001) Best Practices for Agile/Lean Documentation. Available: http://www.agilemodeling.com/essays/agileDocumentationBestPractices.htm Last accessed 12th July 2011 Anon. (2005) Cost Effective Software QA Available: http://testdesigners.com/softwareqa/costeffectiveqa.html Last accessed 30th July 2011 Anon. (2011) My Quality testing: Manual Testing. Available: http://myqualitytesting.blogspot.com/2011/02/manual-testing.html Last accessed 13th June 2011 Anon. (n.d.) Regression Testing Available: http://msdn.microsoft.com/en- us/library/aa292167%28v=vs.71%29.aspx Last accessed 30th July 2011 Barbacci M (1995) Quality Attributes USA: Carnegie Mellon University Beck K, Beedle M and Bennekum A (2001) Principles behind the Agile Manifesto Available: http://agilemanifesto.org/principles.html Last accessed 25th July Boehm B (2002) Get Ready for Agile Methods, with Care USA: IEEE p64-69 Boehm B (2003) Using Risk to Balance Agile and Plan-driven Methods USA: IEEE Computer Society Chowdhary H (2011) Agile Testing: Best to Keep Customers Happy Available: http://www.codeproject.com/KB/testing/AgileTesting.aspx Last accessed 27th July 2011 Cohn M and Ford D (2003) Introducing an Agile Process to an Organization USA: IEEE Computer Society Crispin L and Gregory J (2009) Agile Testing - A Practical Guide to Testers and Agile Teams USA: Pearson Education Inc. Goodyear (2011) Software Quality Attributes. Available: http://www.sqa.net/softwarequalityattributes.html Last accessed 26th June 2011Kaleem Urrehman Page 42

Graham D, Evans I, Veenandaal E and Black R (2008) Foundations of Software Testing (2nd edition) USA: Cengage Gupta M (2008) Role of QA and Testing in Agile Available: http://www.slideshare.net/nashjain/role- of-qa-and-testing-in-agile-presentation Last accessed 30th July 2011 Hambling B (2007) Software Testing: An ISEB Foundation London: British Computer Society Hass A (2003) Configuration Management Principles and Practice USA: Addison-Wesley Hass A (2008) Guide to Advanced Software Testing USA: Artech House Horch W (2003) Practical Guide to Software Quality Management USA: Artech House Huo M, Verner J, Zhu L, & Babar A (2004) Software quality and agile methods Proceedings of the 28th Annual International Computer Software and Applications Conference (COMPSAC04) IEEE Computer Kemp S (2006) Quality Management Demystified USA: McGraw-Hill Professional Kniberg H (2007) Scrum and XP from the Trenches Sweden: C4Media Meyer B (2000) Object-oriented software construction Prentice Hall PTR Myers G (1979) The Art of Software Testing New York: Wiley Nguyen H (2003) Testing Application on the web: Test Planning for Mobile and Internet-based Systems (2nd Edition), Canada: Wiley Ould M (1999) Managing Software Quality and Business Risk USA: John Wiley & Sons Parihar N(2008) Case Study of Agile Testing Available: http://www.slideshare.net/nashjain/case- study-of-agile-testing Last accessed 12th July 2011 Ray (2010) Exploratory Testing - An Agile Approach Available: http://www.testertroubles.com/2009/11/exploratory-testing-agile-approach.html Last accessed 10th July Rizwan A (n.d.) Testing Lifecycle unpublished Rothman J (2008) Does Exploratory Testing Have A Place On Agile Teams Available: http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=COL&ObjectId=13860&tt h=DYN&tt=siteemail&iDyn=2 Last accessed 26th July Sehlhorst S (2010) Agile Documentation Available: http://tynerblain.com/blog/2010/11/16/agile- documentation Last accessed 10th JulyKaleem Urrehman Page 43

Selby R (2007) Software Engineering: Barry W. Boehm’s Lifetime contributions to Software Development, Management and Research Canada: John Wiley & Sons SoftO2 (2001) Integrating Testing in Agile Projects Available: http://www.softo2.com/clientcases/agiletesting.htm Last accessed 12th July 2011 Sommerville I (2006) Software Engineering (8th edition) United Kingdom: Addison-Wesley Soy S (1996) The Case Study as a Research Method. Available: http://www.ischool.utexas.edu/~ssoy/usesusers/l391d1b.htm Last accessed 9th July 2011 Splaine S and Jaskiel S (2001) The Web Testing Handbook USA: STQE Stamelos G (2007) Agile Software Development Quality Assurance UK: Idea Group Theunissen W, Kouried D AND Watson B (2003) Standards and Agile Software Development South Africa: SAICSIT p178-188 Whittaker J (2010) Exploratory Software Testing New Jersey: Addison-Wesley Winkels M (2010) Regression Testing with an Agile Mindset Available: http://blog.xebia.com/2010/11/regression-testing-with-an-agile-mindset Last accessed 5th November 2010 Yin K (1984) Case study research: Design and methods Newbury Park, CA: SageKaleem Urrehman Page 44

Kaleem urrehman managing risks and quality in agile methods

by Kaleem Urrehman

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics

Kaleem urrehman managing risks and quality in agile methods Document Transcript