So how to execute the QA framework in Agile context? There are few guidelines, and the key is balance and adaptive. As mentioned previously, every scrum team is unique. Scrum team has to find the best suitable approach based on the actual status of each scrum team.
So far QA is the major user of QC. This is an example process which involves all roles in Scrums.
Automation in Sprint also will incremental
There is no doubt that Automation is important. But how do we know how are we doing? How to measure automation testing? One of the common way is automation coverage.Code coverage: cover how many line of codes and possible code pathTest case coverage: percentage test cases that are automatedRequirement coverage: cover how much percentage of user scenariosOne execution of test case is one test run.How is test coverage defined? Are we measuring test cases against requirements (generally during system testing), or are we measuring test cases against all possible paths taken through the units and components (generally used for unit testing)? In other words, are we looking at unit testing coverage, code coverage, or requirements coverage?Another measurement could be how many bugs are found by automation? So can automation testing find more bugs? In next slide of defect derivative model, we can look into it.
Which type of defects can be found by Automation?From the cost effective perspective, the best place to use automation is to increase the chance of correct implementation during coding. Example, plain password in the log file. So UT automation, component level automation is very important. These type of automation can help us find problem before it becomes defect.Regression testing also is another type of testing can leverage automation. Actually I believe regression has to use automation.But no matter what, we don’t think today we are over-automated, given the same amount of investment how can we increase automation coverage? The answer is BPT and Data driven.
So what’s BPT? BPT stands for Business Process Testing. In the end of the this slide, there are more details information about BPT. It also fits into whole team approach and incremental approach.Less test cases; ASBU used to have more than 10 thounds of test case, one of the reason is we mixed test data with our function test case. Sankar and I actually discussed it previously, we all agree we should reduce the number of test cases. And using BPT is one of the effective way to reduce test cases.
BPT is the automation framework which can leverage both strength of DQA and AQA. It also can free AQA’s hand so AQA can focus on the most important thing: coding, instead of maintain. Once component is turned over to DQA, DQA will take the ownership of automation, create, execute and monitor automation test cases will be handled by DQA. Moving forward, we also expect DQA can pick up some of the tasks from AQA, moving up the bar.Benefits of BPT:Fully leverage strength of AQA and DQAIt fits into Agile development and QA processMaximize Automation ROIKeep track automation usages and provide comprehensive reportsNo additional cost incurred
Agile Manifesto<br />We Value the items on the left more !<br />
Agenda<br />Goal of QA Framework<br />Roles and Role Interchangeability in Scrum<br />Key success factor: Whole Team Approach<br />QA Framework and Execution Guideline<br />Infrastructure<br />Process<br />Automation in Agile<br />Whole team and Incremental Approach Automation<br />Defects Derivative Model<br />BPT Automation Framework<br />What’s Next<br />Recap and Q&A<br />
Roles in Scrum<br />DEV<br />Model system design<br />Knowledge of product internals<br />Focus on how it can work<br />QA<br />Model user behavior<br />Domain knowledge<br />Focus on how it can go wrong<br />AQA<br />Mixed both DEV/QA<br />
Role Interchangeability in Scrum<br />Sprint planning<br />Sprint Review<br />Design, Coding, Debug<br />Unit testing, code review<br />Test cases execution<br />Translate User Stories into Test Cases<br />Automation feasible analysis<br />Automation testing<br />
Whole Team Approach<br />Whole team is responsible for quality<br />Testers are not quality Police<br />
QA Framework: Process<br />Test plan management (Quality Center)<br />Iterations and Requirements<br />Mapping user story to test plan<br />Test data management<br />Efficient Test Data Management process benefits manual and automated testing and it directly impacts testing effectiveness<br />Obtaining/creating test data could be time consuming<br />Defect management<br />Continuous Integration<br />Data collection and reporting<br />Defect trending/Test progress/Automation coverage<br />
QA Framework: Automation in Agile<br />Automation is MUST in Agile<br />Purpose of Automation<br />Improve testing effectiveness and efficiency<br />Ultimate goal: Improve Quality<br />Automation v.s. Manual<br />Maximize ROI<br />Balance: what can automated, what cannot or should not<br />Whole team approach<br />Incremental approach<br />
Whole Team Approach Automation in Scrum<br />End User<br />Code<br />
Automation Measurement<br />Define Automation Coverage<br />Code coverage = UT<br />Test case coverage<br />Unique case (with or without iteration)= Automated test cases/Total test cases<br />Run coverage= Automated test runs/Total test runs<br />Requirement coverage<br />Automation can find more bugs?<br />
Business Process Testing Process<br />Separate automation script from test data and business logic<br />Less test cases, many iterations (testing data)<br />Central management: testing plan, testing data, testing result<br />QC<br />Application<br />Being Tested<br />QTP<br />Login<br />Data-Driver<br />BPT Test Cases<br />BPT<br />Backup<br />BPT<br />Testing data<br />Job Status<br />BPT<br />
BPT Automation Framework<br />Automation<br />Define components<br />Create Function Libraries<br />Create Object Repositories<br />Create Business Components<br />Create Business Components<br />Design user scenarios<br />SME<br />Drag Components to create test plan <br />Configure Input/Output parameters<br />Add cases to test set in Test Lab and Execute<br />