In next 3 min …Renata Čičman, TietoAistė Stikliūtė, WebmediaUAT WITHSALTThe taste of UAT in large projects  © 2011 Tieto C...
Aistė Stikliūtė                Renata ČičmanLinkedIn:http://lt.linkedin.com/in/aistestikliute   LinkedIn: http://lt.linked...
IN THE NEXT 30 MIN …                             COMPLAINTS FINDINGS                        SARCASM        GOOD EXAMPLES  ...
HOW TO START?Do not wait for someone to tell You what to do  Determine company culture and    Collect existing processes  ...
WHAT TO BE CONSIDERED?                            Context in which the UAT testing                            is to be per...
BEST PRACTICESThere are good practices incontext, but there are no bestpractices.Understand your context andthen make your...
ENVIRONMENT –Human resourcesPOSSIBLE ON TIME? How to make environment come on time: • Rehearsals • Set up the environment ...
CASE STUDY2 environments with different versions of the system   • Same test data (periodic loads from production)   • Sam...
HOW MANY /Human resourcesWHAT METRICS TO USE Per team / area: • Percentage of test cases executed • Defects found (per pri...
HOW MANY /WHAT METRICS TO USE                            How many: depends on what                            decisions yo...
CASE STUDY (2 METRICS) © 2011 Tieto Corporation
CASE STUDY (MANY METRICS) © 2011 Tieto Corporation
TEST CASES: WHO SHOULDDEFINE?• Customer knows what’s important to him• Vendor knows how to use the system• Solution:   • c...
HOW TO SELECT WHAT TO TEST?                                            Critical                                           ...
TRAININGS FOR UATTEST ANALYSTS • Purpose of UAT • Ownership of the testing • Rely on "End-User" documentation • Use of the...
SUPPORT FOR UATTEST ANALYSTS • Inexperienced Testers • Compliance Issues • Poor Release Control • Unstable Test environmen...
BUSINESS SUPPORTFOR TEST ANALYSTS • Gaps in requirements • 3rd parties performing acceptance testing © 2011 Tieto Corporat...
‘PILOT’ – THE WAYTO PREVENT FAILURE • Is our UAT strategy working? • Are our trainings effective? • Gaps in testing script...
PHASES FOR UAT?                                ITER.        WTF. How to split: • By functionality to be rolled-out first •...
CASE STUDYFrom staged                                UAT1     UAT2         UAT3    Reh.        Go-Liveto continuous       ...
COMMUNICATION• How to communicate with UAT test analysts?• How to communicate progress?• What channels should be used?   ©...
DEFECTS TRACKING • There shouldn’t be many new defects during UAT. • But they need special attention. © 2011 Tieto Corpora...
DEFECT ANALYSISWhy we find defects in UAT still?  • Business-knowledge related defects,      like missed requirements  •  ...
POSTPONING FIXESACCEPTABLE OR NOT?• Should UAT defects be of higher  priority by default?• When should UAT defects be clos...
CASE STUDY                              Why these bugs are                              found only now???!Why are you test...
CHALLENGES • Environmental (technical) issues • Scope control • Business users engagement • Plan changes © 2011 Tieto Corp...
ENVIRONMENTAL(TECHNICAL) PROBLEMS• super-fast communicating  and escalating• Rescheduling• planning pre-prod or prod  test...
SCOPE CONTROL• Keep scope unchanged as much as possible• Apply bureaucratic view for any potential changes• Testing scope ...
SCOPE TENDS TO GROW…© 2011 Tieto Corporation
CASE STUDYScope change during UAT preparation   © 2011 Tieto Corporation
CASE STUDYScope change during UAT execution   © 2011 Tieto Corporation
BUSINESS ENGAGEMENTNeed whipped cream or a whip? :)                              Vs.   © 2011 Tieto Corporation
BUSINESS ENGAGEMENTUAT timeframe vs.UAT scope vs.business availability   © 2011 Tieto Corporation
SIGN-OFF • Sign-off time causes sudden diligence :) • The key: sign-off strategy confirmed in advance © 2011 Tieto Corpora...
CONDITIONAL SIGN-OFF • Reduces the number of sign-off iterations • Planning the follow-up:       - Due date       - Resour...
YOUR QUESTIONS?                            Aistė Stikliūtė                Renata Čičman  LinkedIn:http://lt.linkedin.com/i...
Upcoming SlideShare
Loading in …5
×

UAT_in_large_projects

1,638 views

Published on

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,638
On SlideShare
0
From Embeds
0
Number of Embeds
272
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

UAT_in_large_projects

  1. 1. In next 3 min …Renata Čičman, TietoAistė Stikliūtė, WebmediaUAT WITHSALTThe taste of UAT in large projects © 2011 Tieto Corporation
  2. 2. Aistė Stikliūtė Renata ČičmanLinkedIn:http://lt.linkedin.com/in/aistestikliute LinkedIn: http://lt.linkedin.com/in/renatacicman Blog: http://luni.net84.net/ Blog: http://reci7422.projektas.in Twitter: @RenataCicman
  3. 3. IN THE NEXT 30 MIN … COMPLAINTS FINDINGS SARCASM GOOD EXAMPLES STOLEN IDEAS ADVANTAGES OF LESSONS LEARNED OUR IDEAS © 2011 Tieto Corporation
  4. 4. HOW TO START?Do not wait for someone to tell You what to do Determine company culture and Collect existing processes existing systems and procedures Understand the Identify Business Case stakeholders, requirements, ri sks, constraints, assumptions © 2011 Tieto Corporation
  5. 5. WHAT TO BE CONSIDERED? Context in which the UAT testing is to be performed Consistency in testing across the testing teams Schedule Methods and techniques, metrics, tools Communication © 2011 Tieto Corporation
  6. 6. BEST PRACTICESThere are good practices incontext, but there are no bestpractices.Understand your context andthen make your approach(Michael Bolton @ DevelopSense,http://www.developsense.com/presentations/User%20Acceptance%20Testing%20-%20STAR%20East%202006.pdf) © 2011 Tieto Corporation
  7. 7. ENVIRONMENT –Human resourcesPOSSIBLE ON TIME? How to make environment come on time: • Rehearsals • Set up the environment in advance • Pre-test the environment • Good configuration management is the key! © 2011 Tieto Corporation
  8. 8. CASE STUDY2 environments with different versions of the system • Same test data (periodic loads from production) • Same schedule • No other systems integrated, no other teams involved © 2011 Tieto Corporation
  9. 9. HOW MANY /Human resourcesWHAT METRICS TO USE Per team / area: • Percentage of test cases executed • Defects found (per priority/severity) • Defects found (per root cause) •… © 2011 Tieto Corporation
  10. 10. HOW MANY /WHAT METRICS TO USE How many: depends on what decisions you’ll want to make No ambiguities! Performance metrics? Probably not, but... © 2011 Tieto Corporation
  11. 11. CASE STUDY (2 METRICS) © 2011 Tieto Corporation
  12. 12. CASE STUDY (MANY METRICS) © 2011 Tieto Corporation
  13. 13. TEST CASES: WHO SHOULDDEFINE?• Customer knows what’s important to him• Vendor knows how to use the system• Solution: • customer gives guidelines • vendor writes test cases • customer can add test cases © 2011 Tieto Corporation
  14. 14. HOW TO SELECT WHAT TO TEST? Critical success Typical factors??? scenarios High risk scenarios Business benefits??? Existing requirement documentation © 2011 Tieto Corporation
  15. 15. TRAININGS FOR UATTEST ANALYSTS • Purpose of UAT • Ownership of the testing • Rely on "End-User" documentation • Use of the Testing Tool/Repository • Activities involved © 2011 Tieto Corporation
  16. 16. SUPPORT FOR UATTEST ANALYSTS • Inexperienced Testers • Compliance Issues • Poor Release Control • Unstable Test environment • Poor Planning • Incomplete Test Scripts • ... © 2011 Tieto Corporation
  17. 17. BUSINESS SUPPORTFOR TEST ANALYSTS • Gaps in requirements • 3rd parties performing acceptance testing © 2011 Tieto Corporation
  18. 18. ‘PILOT’ – THE WAYTO PREVENT FAILURE • Is our UAT strategy working? • Are our trainings effective? • Gaps in testing scripts? In requirements?! • Should we use UAT environment for ‘Pilot’? • Should testing results from ‘Pilot’ be considered as part of UAT? © 2011 Tieto Corporation
  19. 19. PHASES FOR UAT? ITER. WTF. How to split: • By functionality to be rolled-out first • By resources availability When to split: • Time frame is long enough • By diversity of activities What ever approach you choose it must be thoroughly planned in advance © 2011 Tieto Corporation
  20. 20. CASE STUDYFrom staged UAT1 UAT2 UAT3 Reh. Go-Liveto continuous UAT2 cont. Defects retesting UAT1 cont. No env. UAT3 cont. © 2011 Tieto Corporation
  21. 21. COMMUNICATION• How to communicate with UAT test analysts?• How to communicate progress?• What channels should be used? © 2011 Tieto Corporation
  22. 22. DEFECTS TRACKING • There shouldn’t be many new defects during UAT. • But they need special attention. © 2011 Tieto Corporation
  23. 23. DEFECT ANALYSISWhy we find defects in UAT still? • Business-knowledge related defects, like missed requirements • Technical issues because of newly setup environment • Defects were missed in functional testing © 2011 Tieto Corporation
  24. 24. POSTPONING FIXESACCEPTABLE OR NOT?• Should UAT defects be of higher priority by default?• When should UAT defects be closed? © 2011 Tieto Corporation
  25. 25. CASE STUDY Why these bugs are found only now???!Why are you testing things Why TC that I tested 5 times that are not in TC ??? now suddenly FAILS?! © 2011 Tieto Corporation
  26. 26. CHALLENGES • Environmental (technical) issues • Scope control • Business users engagement • Plan changes © 2011 Tieto Corporation
  27. 27. ENVIRONMENTAL(TECHNICAL) PROBLEMS• super-fast communicating and escalating• Rescheduling• planning pre-prod or prod tests• postponing UAT• releasing without testing in UAT © 2011 Tieto Corporation
  28. 28. SCOPE CONTROL• Keep scope unchanged as much as possible• Apply bureaucratic view for any potential changes• Testing scope can be reduced but more often vice versa © 2011 Tieto Corporation
  29. 29. SCOPE TENDS TO GROW…© 2011 Tieto Corporation
  30. 30. CASE STUDYScope change during UAT preparation © 2011 Tieto Corporation
  31. 31. CASE STUDYScope change during UAT execution © 2011 Tieto Corporation
  32. 32. BUSINESS ENGAGEMENTNeed whipped cream or a whip? :) Vs. © 2011 Tieto Corporation
  33. 33. BUSINESS ENGAGEMENTUAT timeframe vs.UAT scope vs.business availability © 2011 Tieto Corporation
  34. 34. SIGN-OFF • Sign-off time causes sudden diligence :) • The key: sign-off strategy confirmed in advance © 2011 Tieto Corporation
  35. 35. CONDITIONAL SIGN-OFF • Reduces the number of sign-off iterations • Planning the follow-up: - Due date - Resources for verification © 2011 Tieto Corporation
  36. 36. YOUR QUESTIONS? Aistė Stikliūtė Renata Čičman LinkedIn:http://lt.linkedin.com/in/aistestikliute LinkedIn: http://lt.linkedin.com/in/renatacicman Blog: http://luni.net84.net/ Blog: http://reci7422.projektas.in Twitter: @RenataCicman © 2011 Tieto Corporation

×