Critical functionality testing


Published on

A pre-UAT round that quantifies and qualifies the application under test for User Acceptance Testing (UAT) CFT is a strategy derived from our Risk Based Testing Framework. It includesTesting of Business Critical Functionalities across modules with emphasis on Go-live functionalities. It indicates what trouble may lie ahead in UAT and can cover about 20-30% of cases planned for UAT

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Critical functionality testing

  1. 1. © 2013 Maveric Systems Limited Critical Functionality Testing (CFT)
  2. 2. © 2013 Maveric Systems Limited 2 ▸ Introduction – CFT ▸ Need for CFT ▸ CFT Framework ▸ Benefits of CFT ▸ Advantages of CFT over Traditional Approach ▸ Case Study Presentation Outline
  3. 3. © 2013 Maveric Systems Limited 3 ▸ A pre-UAT round that quantifies and qualifies the application under test for User Acceptance Testing (UAT) ▸ Test evolution – from our experiences across multiple engagements ▸ Is a strategy derived from our Risk Based Testing Framework ▸ Testing of Business Critical Functionalities across modules with emphasis on Go-live functionalities ▸ CFT will not replace UAT, it only ensures a better UAT and helps meeting overall project timelines ▸ CFT – cannot pick all showstoppers issues, it only indicates what trouble may lie ahead in UAT ▸ Can cover about 20-30% of cases planned for UAT Introduction – Critical Functionality Testing
  4. 4. © 2013 Maveric Systems Limited 4 ▸ Assesses the stability of the application ▸ Quantifies the percent of critical functions which have defects across modules ▸ Irons out differences if any on functionalities with the business ▸ Missing or wrong parameter set-ups can be identified before UAT thereby negating unnecessary re-running of scripts in UAT rounds, especially batch dependent cases due to missing set-ups ▸ Interfaces – understanding the compatibility of the application with 3rd party interfaces Need for CFT
  5. 5. © 2013 Maveric Systems Limited 5 Plan Specify Build Test Implement Application Lifecycle Cost International Software Benchmarking Standard Group (Data Sample 404 projects) Real time Industry Maveric Approach 5% 23% 41% 21% 9%
  6. 6. © 2013 Maveric Systems Limited 6 Specify Define Design Code Test Deploy Production Early Detection StrategyNoofdefects (Increasing) time and cost to fix defects Time & Cost Prevention strategy Traditional detection strategy Early detection strategy (CFT)
  7. 7. © 2013 Maveric Systems Limited 7 Best Practices – CFT Framework CFT framework Critical functional scenarios Parameter set-ups Interfaces connectivity and compatibility Batch processes Test bed readiness and database sync up across all systems
  8. 8. © 2013 Maveric Systems Limited 8 Best Practices – CFT Framework CFT framework Test bed readiness and database sync up across all systems Critical functional scenarios Parameter set-ups Interfaces connectivity and compatibility Batch processes
  9. 9. © 2013 Maveric Systems Limited 9 ▸ Transactions with high financial impact ▸ Transactions involving critical functional flow ▸ Batch dependent scenarios ▸ Focused discussions and meetings with business to identify critical functional points ▸ Identifying any changes to Business Requirements at an early stage thereby avoiding the need to raise CRs Critical Functional Scenarios – Rationale
  10. 10. © 2013 Maveric Systems Limited 10 ▸ Compatibility of interfaces with the core system ▸ Communication between different 3rd party interfaces ▸ Helps in optimizing the Run Plan covering only functionality of interfaces which are test ready for UAT Interface Connectivity and Compatibility
  11. 11. © 2013 Maveric Systems Limited 11 ▸ Batch critical reports – MIS reports » Financial reports » Regulatory and compliance reports ▸ Focuses more on the data generated in the report than cosmetic ones like alignment etc. ▸ Example: Premium collection report » Claim process report Reports
  12. 12. © 2013 Maveric Systems Limited 12 ▸ Ensures all parameter set-up are completed before starting UAT ▸ Special focus on escalations on authorizations and approvals ▸ Negates unnecessary need to re-run cases due to missing set-up Set-up Parameters
  13. 13. © 2013 Maveric Systems Limited 13 ▸ End of day and end of month process are run ▸ End of year process is run by moving dates/defining holidays ▸ Ensures there are no hiccups in execution of test cases as per the Run Plan during UAT ▸ Defects in batch process adversely effect the schedule of testing, CFT helps in negating such high risks entering into UAT Batch Processes
  14. 14. © 2013 Maveric Systems Limited 14 ▸ Ensures different databases are in sync with the core ▸ Access to different databases for migrated test data ▸ Gives an idea on the quality of migrated data Test Bed Readiness and Database Sync-up
  15. 15. © 2013 Maveric Systems Limited 15 Entry criteria for CFT – Done when SIT is complete and signed off ▸ Business critical functional points have been identified after discussion and sign-off by Business Line Managers CFT exit criteria ▸ 70% of the Critical Functions should pass ▸ Number of showstopper defects to be less than 20% ▸ All interfaces should be up and running ▸ All databases should be made in Sync ▸ EOD should be successful CFT Entry and Exit
  16. 16. © 2013 Maveric Systems Limited 16 Insurance ▸ Customer Creation (CIF) ▸ Opening of Customer & Loan Accounts ▸ Cash Deposits and Withdrawals ▸ Fund Transfers ▸ End of Day Processing ▸ Customer Creation ▸ New Business ▸ Underwriting and Rating Factors ▸ Premium Computation ▸ Mid Term Endorsements – Financial ▸ Mid Term Endorsements – Non Financial ▸ Collections and Refunds ▸ Claim Handling ▸ Renewals ▸ Reports and Accounting ▸ Batch Processing Examples
  17. 17. © 2013 Maveric Systems Limited 17 Observations and Risk Classification Key process/activities Risk classification Potential risk Critical functional scenarios Under control, however non conformance by business users can lead to risk Test environment readiness Missing set-ups and a sync databases may lead to re-running cases thereby increasing timelines and effort Test readiness of interfaces Knowing incompatibility or non-availability of major functionality during UAT may cause major set back to timelines Reports Non-availability of compliance or regulatory reports during UAT impacts Go-live date Batch process Abnormal timelines or showstopper here will affect the entire testing schedule Low Risk Medium Risk High Risk
  18. 18. © 2013 Maveric Systems Limited 18 ▸ 70% of the critical functions should pass ▸ Number of showstopper defects to be less than 20% ▸ All interfaces should be up and running ▸ All databases should be made in sync ▸ EOD should be successful CFT Exit – UAT Entry
  19. 19. © 2013 Maveric Systems Limited 19 ▸ Reduce costs – Identifying defects at an early stage reduces costs and also only 50-60% of the testing team is present onsite for CFT ▸ CFT design is done parallel with UAT design hence separate effort is not required ▸ Change requests on critical functions if any can be identified at an early stage by the business ▸ Reduces the effort of business users during UAT ▸ Same design pack can be re-used with minor modifications to do a quick sanity check after releases or enhancements ▸ Helps the business and also the testing team to understand the differences between legacy and the new system Benefits of CFT for Business
  20. 20. © 2013 Maveric Systems Limited 20 ▸ Identifying most of the business critical defects at an early stage gives the development team more time to fix ▸ Helps the development team to reassess priorities and areas to focus on as the development team is not bombarded with cosmetic and minor defects earlier Benefits of CFT for the Developers’
  21. 21. © 2013 Maveric Systems Limited 21 ▸ Gives a metrics based report on defects thereby helping to identify areas to focus on during UAT ▸ Helps project management team identifying areas to focus-on and reschedule timelines if necessary ▸ Exactly identify when to start and bring in the full team for UAT Benefits of CFT for the Testing Team
  22. 22. © 2013 Maveric Systems Limited 22 ▸ Covers both critical and non-critical transactions hence priorities may get misplaced on non-critical functions ▸ Critical business defects may be found at the end of the round 1 which will take time to fix ▸ Defects on critical function may get overlooked due to overwhelming number of defects ▸ May need to stop UAT mid-way due to large showstoppers and critical defects thereby wasting time and money Advantage CFT Traditional Approach CFT ▸ Focus on critical functions from a very early stage of the testing lifecycle ▸ Helps the business to know the percent and the status of defects on Go-Live critical functions right from the start ▸ Cuts down on time spent on non- critical functions at the start of the testing assignment ▸ Focused discussions and meeting with business to exactly identify the critical functions
  23. 23. © 2013 Maveric Systems Limited 23 CFT – Where Does it Fit in the Testing Calendar CFT runs parallel to UAT Design and CFT uses 30% of UAT design CFT ensures a high quality UAT is met CFT sets entry criteria for UAT Entry
  24. 24. © 2013 Maveric Systems Limited 24 Case Study ▸ A leading commercial insurance organization in the Middle East was migrating from a legacy core insurance system to a new core insurance solution with specific business customization and enhancements ▸ The core-insurance solution was to be implemented phase-wise (in 3 phases) across 73 branches through out the country ▸ The phase I rollout was being planned with aggressive timelines as the insurance company had to do catch-up with its competitors in converting to new technology solution which was of paramount importance to sustain its market share Client Context Challenges ▸ Documentation was very minimal and not updated ▸ Frequent changes to requirements ▸ Need to collaborate with the insurance company on one side, and the application vendor’s on the other side with stiff timelines ▸ An application architecture that required extensive batch runs for EODs etc. ▸ Domain competence in core insurance solution to address country specific requirements and regulations especially for reports (compliance) ▸ An approach to ensure production quality at the same time keeping cost and timelines at a tight leach
  25. 25. © 2013 Maveric Systems Limited 25 ▸ Maveric adopted CFT based testing approach to meet timelines at the same time ensuring product quality ▸ Only 60% of the team was positioned onsite to cut down the project cost ▸ The team comprised of domain experts who organized daily meetings with Business Line Managers to get clear cut understanding of the customizations ▸ Weekly two meetings were conducted involving both the development team and the Business Line Managers to ensure that everyone was in the same page with the respect to the functional specs of the customizations ▸ Minutes of the meeting were documented and circulated to all the parties involved and were used as BRS documents for reference Maveric Solution – CFT
  26. 26. © 2013 Maveric Systems Limited 26 ▸ The end of day was taking 12hrs to complete during CFT, this was immediately worked upon and brought to 5hrs during UAT Round 1. Further, analysis were done by the development team and fixing was too complex and took time, but at the end of Round 2, it was brought down to 1 hour 30 minutes ▸ Due to extensive customization the solution had around 40% defects in business critical functionalities ▸ Project Management company delayed the start of UAT by 10 days to give more time for developers’ to fix the critical defects. The remaining of the test team was not brought board during this time thereby reducing the cost ▸ The development team suggested work around solutions for few defects as the fix was complex and required more time. Those fixes as promised were delivered for Round 2 Maveric Solution – CFT (Cont’d)
  27. 27. © 2013 Maveric Systems Limited 27 ▸ Gap Analysis – Identified that only five interfaces out of the 7 were test ready ▸ Priorities were reassessed and only 5 interfaces namely ATM, I-Score, CBE Files Electronic Cheque Clearing and Covered Visa were taken up for Phase I ▸ Reports – One resource from Maveric was dedicated to test CBE reports in CFT Maveric Solution – CFT (Cont’d)
  28. 28. © 2013 Maveric Systems Limited 28 ▸ To Introduce Early Defect Detection (EDD) Strategy, Essential Start -Stop- Continue Strategies are: ▸ CFT is a Win-Win strategy for the client, development teams and also for testing team to mutually ensure a quality solution goes into production within the timelines planned Conclusion Start Stop Continue Planning ahead and release application for EDD early Introducing last minute requirement changes Focus on Good Quality UAT Engage specialized testing partners and build high toll gate for UAT Entry Defining poor UAT Entry Criteria's To reject poorly built application Plan for CFT to be included in the overall project implementation Plan Don’t engage Users till CFT toll gates are met Engage Users in UAT and build lifecycle cases
  29. 29. © 2013 Maveric Systems Limited Thank You