This document summarizes information about testing for a large university system upgrade project called Project FOCUS. It provides details about: - The scope of the project including upgrading various systems, migrating data, and rewriting business processes - The testing team structure and responsibilities including testing various phases like data conversion, processes, enhancements, integration, performance, security, usability, and user acceptance testing - Examples of test scenarios and approaches for different test types - Tips and recommendations for effective testing based on the presenter's experience

1. Testing for 20 months – how cost
effective is that?
Session #31028
01 December, 2011
Farid Vaswani
University of Auckland
Twitter: @FVaswani

2. Experience
o 12 years IT experience in Development & Testing
o Conference Speaker
o Techie / Grumpy / Unsatisfied Tester
Responsibilities
• Testing Manager, University of Auckland
o Team of 8 - 20 Testers and Test Engineers
• Associate Board Member, ANZ Testing Board
o Enhancing Career Opportunities for Testing Professionals

5. o 40,000 students
o 5,000 staff
o 8 Faculties
o 2 large-scale
research institutes
o 4 main campuses

6. o PeopleSoft CS
o PeopleSoft HR
o PeopleSoft FIN
o Databases
o WebLogic Server
o RightNow (CRM)

7. CCv7.6
(nDEVA)
JAVA
Batch
Processes
AFA
EPRv1 (Portal)

8. CS
v9
WLI DSS
EPR AFA
v2 v2

9. o Oracle support
o SOA architecture
o Real-time updates
o Identity & Access Management
o Usability
o Hardware virtualisation
o Performance
o Students
o http://www.youtube.com/user/MassInTheAss#p/a/f/5/CsMfvIEUFkk

10. Project FOCUS
Focus On Changing University Services

11. o Budget: eight-figure $ amount
o 63 business process, to be rewritten
o 500+ staff affected
o 40,000+ student records
o 400,000 rows in one of the tables
o 10+ years of data to be migrated
o Project team size: 50+ FTE
o + UAT testers

12. It’s just another
PeopleSoft upgrade
project

13. Programme Manager
Process Manager
Data Conversion
Manager
Development Manager
Integration Lead
EA Lead
Security Lead
Testing Manager

14. 1. Testing Types
2. Testing Responsibilities
3. Test Team
4. Testing Approach
5. Testing Schedule
6. Testing Tools
7. Test Reporting

15. o Data Conversion Testing
o Process Testing
o Enhancement Testing
o Integration Testing
o Performance Testing
o Security Testing
o Usability
o Compatibility
o User Acceptance Testing
o Go-live

16. Process Testing Manager DSS Manager
Manager
Process Team Test Team Reporting Team
(15) (6) (4)
DC Tester
Security Tester
(2)
Data Conversion Security Manager
Manager

17. Process Testing Manager DSS Manager
Manager
• Functional
Process Team • Integration Reporting Team
(12) • Performance (5)
• Automation
• Compatibility
• Security Scans
DC Tester Security Tester
Data Conversion Security Manager
Manager

18. Tip #1:
Way in to other
teams’ time is
through their
managers.

19. Data Conversion Testing
CC v7.6 International -> CS v9.0 Asia-Pac
o CC v7.6 -> CC v8.0 -> CS v9.0
o Vanilla upgraded by Oracle lab
o Customisations, converted
(implemented) in-house

20. Data Conversion Testing
Testing
o Field mappings
o Field counts
o Record counts
o GUI testing
o EPR-CS9 reconciliation

21. Data Conversion Testing
Tip #2:
Data Conversion testers
are good functional
testers during their
down time.

22. Data Conversion Testing
Tip #3:
Never
underestimate the
value of manual GUI
checking after DC.

23. Process Testing
10 Mega processes, 63 sub-processes
o Admissions
o Enrolment
o Manage Campus Community
o Financial Services
o Financial Support
o Teach Learn Evaluate
o Transition Student
o Advisement
o Self Service
o Reporting

24. Process Testing
Testing
o 6 Process Leads + 9 team members
o Written Test Cases
o Exploratory Scenarios
o Mostly positive testing for staff based functionality

25. Process Testing
Tip #4:
Attend Process
workshops; all of
them, if possible.

26. Enhancement Testing
Scope + Testing
o 54 enhancements
o PeopleSoft only
o Split between QA & Process team
o Written Test Cases
o Exploratory Scenarios
o Some negative testing included

27. Enhancement Testing
Tip #5:
Good to keep a tap
on business testers
when writing test
scripts.

28. Integration Testing
*1* Interfaces
o WLI
o Data Extracts
o Third party
o Studylink (student loans)
o National Student Index
o Payment Gateway
o Technical Testers responsible for these

29. Integration Testing
*2* Process Integration
o Student lifecycle
o Various start & end points
o Multiple processes
o Multiple systems

30. Integration Testing
*2* Process Integration
o Scenario 1 (e.g.)
o Admission > Enrolment > Fees > Exam > Completion
> Graduation
o Domestic student
o NCEA
o NSI
o Undergrad
o Student loan
o 1st year student

31. Integration Testing
*2* Process Integration
o Scenario 4 (e.g.)
o Admission > Enrolment > Fees > Visa Processing
o International student
o Post grad
o 120 points
o Paying full fees

32. Integration Testing
*2* Process Integration
o 7 scenarios
o 45 (out of 63) sub-processes
o 10 external systems
o 240 transitions
o One person responsible for these

33. Integration Testing
Tip #6:
Book some extra
time slots for testing
with external-party,
in advance.

34. Performance Testing
Challenge
CS9 AFA WLI
• Staff • Student • World using it
• Batch • EOI • Relatively new
• Transactions • EPR • Real-time
• Reports
• Self Service
Infrastructure

35. Performance Testing
Testing: Hybrid approach
o 6 batch processes: manual
o eg: Generate Invoice, Advisement, Transcript
o Student Self-Service: automated
o Staff transactions
o Manual and automated data generation scripts
o 14 reports, manual
o AFA, automated
o 9 WLI interfaces, scripted

36. Performance Testing
Tip #7:
Pre-Prod is NOT Test

37. Security
Testing
o Application Penetration Tests
o User roles
o Staff vs Students
o Positive & Negative
o Infrastructure Scan
o Security certificates
o Split URLs
o https://student.auckland.ac.nz/
o https://student.admin.auckland.ac.nz/

38. Security
Tip #8:
Make an effort to
understand
security-speak

39. Usability
Usability
Testing

40. Usability
Usability
o Review…
o Personas…
o Wire-frames…
o Mock screens…
o Real people…
o Iterate ---^

41. Usability
Tip #9:
Consult Steve
Alexander from
PurpleShirt.co.nz

42. Compatibility Testing
We Tested ALL
Users
want
We
support
Oracle
supports

43. User Acceptance Testing
Testing
o Subject Matter Experts…
o …supported by Process Leads
o 3 cycles…
o …on rotation basis
o MS-Word templates

44. Security
Tip #10:
SMEs are busy. Take
their BAU schedule
into consideration
when planning.

45. Security
Tip #11:
UAT is not a Training
opportunity.

46. Go-live Testing
Testing
o Business critical scenarios
o Integration scenarios
o Reports
o Self-Service
o People: QA, Process, SMEs
o Google Docs
o allowed simultaneous editing

47. Go-Live Testing
Tip #12:
Include SMEs.

48. o 4 test cycles
o Process & SMEs
o Priority based
o Partial processes
o Enhancements (non-PS)
o Agile, continuous testing
o Non-functional
o Performance – last 3-4 months
o Security – 3 scans

49. o HP Quality Centre
o Test case management
o Jmeter
o Performance
o soapUI
o Web – services
o Functional & load
o HermesJMS
o WLI queues
o AppScan
o MS-Word, MS-Excel
o Google Docs
o Confluence wiki
o Jira

50. Defects Raised per Week
250
Low
200
150 Medium
100
High
50
0
Week Ending

51. Defect Status
2500
2000
Opened
1500 Cumulative
Total
1000 Closed
Cumulative
500
Nett Open
0
Reopened
Week Ending
2500 Defects Forecast
Actual
2000
Expected
Cumulative Defects
1500
1000
500
0
-500
Week Ending

52. Tip #0:
Start Early. Test Early.
Deliver on time.

53. Video: http://www.youtube.com/watch?v=uobcj9A9Mck

55. Farid Vaswani
Testing Manager
ITS, Group Applications
University of Auckland
E-mail: f.vaswani@auckland.ac.nz
Twitter: @FVaswani

56. This presentation and all
Alliance Down Under 2011
presentations are available
for download from the
Conference site at
www. heug.org
Presentations from previous meetings are also available