UAT_in_large_projects
- 1. In next 3 min …
Renata Čičman, Tieto
Aistė Stikliūtė, Webmedia
UAT WITH
SALT
The taste of UAT in large projects
© 2011 Tieto Corporation
- 2. Aistė Stikliūtė Renata Čičman
LinkedIn:http://lt.linkedin.com/in/aistestikliute LinkedIn: http://lt.linkedin.com/in/renatacicman
Blog: http://luni.net84.net/ Blog: http://reci7422.projektas.in
Twitter: @RenataCicman
- 3. IN THE NEXT 30 MIN …
COMPLAINTS FINDINGS
SARCASM GOOD EXAMPLES
STOLEN IDEAS ADVANTAGES OF
LESSONS LEARNED
OUR IDEAS
© 2011 Tieto Corporation
- 4. HOW TO START?
Do not wait for someone to tell You what to do
Determine company culture and Collect existing processes
existing systems and procedures
Understand the Identify
Business Case stakeholders, requirements, ri
sks, constraints, assumptions
© 2011 Tieto Corporation
- 5. WHAT TO BE CONSIDERED?
Context in which the UAT testing
is to be performed
Consistency in testing across
the testing teams
Schedule
Methods and
techniques, metrics, tools
Communication
© 2011 Tieto Corporation
- 6. BEST PRACTICES
There are good practices in
context, but there are no best
practices.
Understand your context and
then make your approach
(Michael Bolton @ DevelopSense,
http://www.developsense.com/presentations/
User%20Acceptance%20Testing%20-
%20STAR%20East%202006.pdf)
© 2011 Tieto Corporation
- 7. ENVIRONMENT –
Human resources
POSSIBLE ON TIME?
How to make environment come on time:
• Rehearsals
• Set up the environment in advance
• Pre-test the environment
• Good configuration management is the key!
© 2011 Tieto Corporation
- 8. CASE STUDY
2 environments with different versions of the system
• Same test data (periodic loads from production)
• Same schedule
• No other systems integrated, no other teams involved
© 2011 Tieto Corporation
- 9. HOW MANY /
Human resources
WHAT METRICS TO USE
Per team / area:
• Percentage of test cases executed
• Defects found (per priority/severity)
• Defects found (per root cause)
•…
© 2011 Tieto Corporation
- 10. HOW MANY /
WHAT METRICS TO USE
How many: depends on what
decisions you’ll want to make
No ambiguities!
Performance metrics?
Probably not, but...
© 2011 Tieto Corporation
- 13. TEST CASES: WHO SHOULD
DEFINE?
• Customer knows what’s important to him
• Vendor knows how to use the system
• Solution:
• customer gives guidelines
• vendor writes test cases
• customer can add test cases
© 2011 Tieto Corporation
- 14. HOW TO SELECT WHAT TO TEST?
Critical
success
Typical factors???
scenarios
High risk
scenarios
Business
benefits???
Existing
requirement
documentation
© 2011 Tieto Corporation
- 15. TRAININGS FOR UAT
TEST ANALYSTS
• Purpose of UAT
• Ownership of the testing
• Rely on "End-User" documentation
• Use of the Testing Tool/Repository
• Activities involved
© 2011 Tieto Corporation
- 16. SUPPORT FOR UAT
TEST ANALYSTS
• Inexperienced Testers
• Compliance Issues
• Poor Release Control
• Unstable Test environment
• Poor Planning
• Incomplete Test Scripts
• ...
© 2011 Tieto Corporation
- 17. BUSINESS SUPPORT
FOR TEST ANALYSTS
• Gaps in requirements
• 3rd parties performing acceptance testing
© 2011 Tieto Corporation
- 18. ‘PILOT’ – THE WAY
TO PREVENT FAILURE
• Is our UAT strategy working?
• Are our trainings effective?
• Gaps in testing scripts? In requirements?!
• Should we use UAT environment for ‘Pilot’?
• Should testing results from ‘Pilot’ be considered as part
of UAT?
© 2011 Tieto Corporation
- 19. PHASES FOR UAT?
ITER. WTF.
How to split:
• By functionality to be rolled-out first
• By resources availability
When to split:
• Time frame is long enough
• By diversity of activities
What ever approach you choose it must be
thoroughly planned in advance
© 2011 Tieto Corporation
- 20. CASE STUDY
From staged
UAT1 UAT2 UAT3 Reh. Go-Live
to continuous
UAT2 cont. Defects retesting
UAT1 cont.
No env. UAT3 cont.
© 2011 Tieto Corporation
- 21. COMMUNICATION
• How to communicate with UAT test analysts?
• How to communicate progress?
• What channels should be used?
© 2011 Tieto Corporation
- 22. DEFECTS TRACKING
• There shouldn’t be many new defects during UAT.
• But they need special attention.
© 2011 Tieto Corporation
- 23. DEFECT ANALYSIS
Why we find defects in UAT still?
• Business-knowledge related defects,
like missed requirements
• Technical issues because of newly
setup environment
• Defects were missed in
functional testing
© 2011 Tieto Corporation
- 24. POSTPONING FIXES
ACCEPTABLE OR NOT?
• Should UAT defects be of higher
priority by default?
• When should UAT defects be closed?
© 2011 Tieto Corporation
- 25. CASE STUDY
Why these bugs are
found only now???!
Why are you testing things Why TC that I tested 5 times
that are not in TC ??? now suddenly FAILS?!
© 2011 Tieto Corporation
- 26. CHALLENGES
• Environmental (technical) issues
• Scope control
• Business users engagement
• Plan changes
© 2011 Tieto Corporation
- 28. SCOPE CONTROL
• Keep scope unchanged as much as possible
• Apply bureaucratic view for any potential changes
• Testing scope can be reduced but more often vice
versa
© 2011 Tieto Corporation
- 34. SIGN-OFF
• Sign-off time causes sudden diligence :)
• The key: sign-off strategy confirmed in advance
© 2011 Tieto Corporation
- 35. CONDITIONAL SIGN-OFF
• Reduces the number of sign-off iterations
• Planning the follow-up:
- Due date
- Resources for verification
© 2011 Tieto Corporation
- 36. YOUR QUESTIONS?
Aistė Stikliūtė Renata Čičman
LinkedIn:http://lt.linkedin.com/in/aistestikliute LinkedIn: http://lt.linkedin.com/in/renatacicman
Blog: http://luni.net84.net/ Blog: http://reci7422.projektas.in
Twitter: @RenataCicman
© 2011 Tieto Corporation
