Test Less is More

1. FLEXIBLE FRAMEWORK FOR QUALITY ASSURANCE AND TESTING
OF SOFTWARE
Prepared by BonnieYau
email: bonnie.yau.hk@gmail.com
facebook page: https://www.facebook.com/mobileapptest/?ref=hl

2. STRATEGY AND PROCESS
• Baked in Quality from the start - research Best Practise / risk and compare with competitor for new feature
• Define Test Plan after user requirement confirm
• Draft Test suite for all case / scenario that fit general requirement while frequent change on wireframe / navigation flow still keep the suite stably apply for function test
• Execute different type of Test and refine process, user requirement, test case as there must be something still missing while planning
• Deliver Test Result with complete issue list that giving issue table with priority
• Provide testing support during debug
• Keep tracking with issue open and closed should have a chart dropping down
• Loop regression test to Kill all Critical and Major Issue for UATTest
• Fix High Priority UAT issue and try to Kill all other issue before release deadline
• Test Production in 3 Days and check if any critical issue found for hot fix release again
• Having Retrospective meeting with teams to refine better process for next release cycle
2

3. TEST FLOW
1. Acceptance Test - build verification test to prove whether or not a given build is worth deeper testing
2. Functional Test - test all case of positive and negative case - user stories
3. Performance Test - give us measure on how the application responds to requests and how the application behaves under load
4. UI Test - Screen shot for all platform with wireframe
5. UX Test - Navigation record for action change match design / platform guideline (e.g. iOS design guide)
6. Localisation Test - ensure language can switch and every text show as per define
7. Retesting and Regression Testing - test for issue solved and closed issue report
8. Backward compatibility test / update test - ensure switching backend service with same application don’t affect key function
9. UAT test - collect issue from user which less than 20% of QA reported issue
10.Production test - check the 5% issue that only happen in production
3

4. MORETHAT HELPS…..
1. CreateTesting tools specific to product that anyone can use
2. ExploratoryTest
3. Create test automation
4. Define requirement and acceptance criteria with product owner
4

5. iPhone iPad
Android
Phone
Android
Tablet
Web - PC
Web -
Mac
Web -
mobile
Brand 1 1 5* 3* 1* 1* 4*
OS 2 2 8* 8* 6* 6* 6*
Screen size 3 3 4* 4* Dynamic Dynamic Dynamic
Test Case 500 * 500 * 700* 700* 400* 400* 400*
Network 5* 5* 5* 5* 4* 4* 4*
TotalTest 15,000 15,000 560,000 336,000 9,600 9,600 9,600
* more if count for minor influence (less than 1%)
TEST COVER
5

6. QATEAM WORK
Lead Senior Experienced
Strategy 3 1 0
Planning 3 1 0
Test Case written 1 3 2
Test Environment 0 2 3
Test Execution 1 2 3
Issue reported 1 3 4
Test result 2 1 0
Retrospecitve 3 1 0
Number stand for the weight for work
6

7. QA FOCUS
Lead Senior Experienced
AcceptanceTest o o o
FunctionalTest o o o
PerformanceTest x o x
UITest o x o
UXTest o x o
LocalisationTest o x o
RegressionTest o x o
UpgradeTest o x o
UATTest x x o
ProductionTest o o o
7

8. ISSUE STANDARDISATION
• Objective: Report issue that developer had enough evidence to solve in fast pace which avoid duplicate issue
report (one root cause with different scenario)
• Priority: Blocker, Critical, Major, Minor,Trival
• Frequency:Always,Ad Hoc, Sometimes
• Specific issue happen in : iOS, Device brand, Logic,Account status
• Investigate by repeating different test to screen out the only root cause of issue which help direct bug fix
• Steps, expected results,Actual results, attach issue screen shot, attach original user requirement (e.g. wireframe)
8

9. QUALITY PROGRESSTRACKING
with JIRA to work on reporting
9

10. AUTOMATED WORKFLOW

11. LESSVERBAL, MORE COMMUNICATION
• 80% of issue or work transfer to developer or team member through
JIRA system
• 20% of issue further clarify in email , meeting, face-to-face
conversation
11

12. QAVS UAT
• QA assists on dev team side and UAT is only on the client side
• QA helps the dev team in having a fast feedback (quality) cycle for new features
and to save guard older features with automated and manual test cases.
• QA shorten the debugging period while UAT burning the dev team
efficiency to investigate issue with not enough analyst to solve the case, or even
worst clients give different direction on how the issue be fixed which finally
crack the application without organising the logic
12