T12: Exploitation testingFor testing Service Level Agreements                  based on a best practice at1 december 2009
Agenda• Understanding Exploitation  testing• Best practice at APG• Service Level  Management and Testing• Using State Tran...
Understanding Exploitation testing• Based on Dutch term ‘Exploitatie test’• Definition:       Check whether the agreed or ...
Service Level Agreement (SLA)• A negotiated agreement between two parties   • customer and service provider• Containing me...
Trends in IT service providingFormalization of SLA’s, due to:   • Increased Integration of systems   • Globalization (high...
Agenda• Understanding Exploitation  testing• Best practice at APG• Service Level  Management and Testing• Using State Tran...
All Pension Group• Pension provider for international pension funds• Approximately 4000 staff provide pensions  for around...
Test optimization• One of the optimization targets:       Involve ES in an earlier stage in project and incorporate       ...
Agenda• Understanding Exploitation  testing• Best practice at APG• Service Level  Management and Testing• Using State Tran...
Short view on ITIL v3                    Service Validation and Testing
ITIL version 2                                      Service Support                             Incident             Opera...
Exploitation test within the V-model                                          Exploitation           User  Business       ...
Stakeholders
SLA items versus Software testingSLA Items               Quality attributes         Test approachAvailability            M...
Agenda• Understanding Exploitation  testing• Best practice at APG• Service Level  Management and Testing• Using State Tran...
Use of State Transition TestingTest specification steps:1. Specify the system components influencing the   availability of...
Example State Transition Testing (1)              1.   Critical system component: application and database servers        ...
Example State Transition Testing (2)                5.     Specify test cases              TC1                   TC2      ...
Questions
Upcoming SlideShare
Loading in …5
×

T12 exploitation testing - presentation 1.2

249 views

Published on

Published in: Technology
  • Be the first to comment

  • Be the first to like this

T12 exploitation testing - presentation 1.2

  1. 1. T12: Exploitation testingFor testing Service Level Agreements based on a best practice at1 december 2009
  2. 2. Agenda• Understanding Exploitation testing• Best practice at APG• Service Level Management and Testing• Using State Transition Test to test Availability
  3. 3. Understanding Exploitation testing• Based on Dutch term ‘Exploitatie test’• Definition: Check whether the agreed or expected Service level related to IT infrastructures can be achieved.• Those agreements and/or expectations are formalized in Service Level Agreements (SLA)• Represents a group of quality attributes
  4. 4. Service Level Agreement (SLA)• A negotiated agreement between two parties • customer and service provider• Containing measurable KPI’s, such as: • 99.9% uptime per month => max. downtime: 43.2 minutes • 95% of transactions completed within 1 second • Mean Time To Recovery of 4 hours • 98% of batch jobs executed within batch window • Maximum response time of 30 minutes
  5. 5. Trends in IT service providingFormalization of SLA’s, due to: • Increased Integration of systems • Globalization (higher availability demands) • Third party involvement • Growing complexity of infrastructures and architectures • Etc…
  6. 6. Agenda• Understanding Exploitation testing• Best practice at APG• Service Level Management and Testing• Using State Transition Test to test Availability
  7. 7. All Pension Group• Pension provider for international pension funds• Approximately 4000 staff provide pensions for around 4 million participants• Internal IT department (CIS) exists of: • Application Services (AS) – Software development • Exploitation Services (ES) – Service management • And others, like QA and Financial control• Mission of CIS: Allow business to excel by delivering ICT services
  8. 8. Test optimization• One of the optimization targets: Involve ES in an earlier stage in project and incorporate a formal test approach• Implemented elements: • Involvement at reviews of specifications • Introduction of Test specification techniques at Exploitation tests• Result: • Improved the quality of advice given to stakeholders • Better understanding of the software and daily tasks • Increased confidence
  9. 9. Agenda• Understanding Exploitation testing• Best practice at APG• Service Level Management and Testing• Using State Transition Test to test Availability
  10. 10. Short view on ITIL v3 Service Validation and Testing
  11. 11. ITIL version 2 Service Support Incident Operational Management Release Problem Management Software testing Management Configuration ChangeSupport Management Management Security desk management Service Delivery Service level Strategic Management Financial Availability Management Management Continuity Capacity Management Management
  12. 12. Exploitation test within the V-model Exploitation User Business Acceptance Acceptancerequirements Business Test Test SLA IT Service Exploitation Test Management System UC/ OLA System System specifications Test Integration System System System specifications Integration Test Technical Technical design Component Test Development Development UC = Underpinning contract OLA = Operational Level Agreement
  13. 13. Stakeholders
  14. 14. SLA items versus Software testingSLA Items Quality attributes Test approachAvailability Maturity / State Transition Test Fault toleranceDown/maintenance time Maintainability / Review / Recoverability During availability testSecurity Security Review / ‘hack test’Capability Efficiency Performance and stress testService related Items Quality attributes Test approachVerifiability Understandability / During availability Analyzability TestServiceability Operability Scenario / Real life test (IT Service perspective)
  15. 15. Agenda• Understanding Exploitation testing• Best practice at APG• Service Level Management and Testing• Using State Transition Test to test Availability
  16. 16. Use of State Transition TestingTest specification steps:1. Specify the system components influencing the availability of the system2. Specify the failures that may occur3. Specify the measures taken to prevent these failures4. Conduct State Transition Diagram a. Define the states in relation to these measures b. Visualize the status and transitions between5. Specify the test cases
  17. 17. Example State Transition Testing (1) 1. Critical system component: application and database servers 2. Business risk: systems unavailable, due to AC power break 3. Preventive measure: uninterruptible power supply (UPS) 4. State Transition Diagram AC Power break Supply power via UPS Normal ON power End of AC power break 15% of batteryCharged UPS Supply power via AC energy left Start up Shut down Systems Systems Charging OFF End of AC power break Charge UPS
  18. 18. Example State Transition Testing (2) 5. Specify test cases TC1 TC2 TC3 TC4 TC5Start State Normal power ON OFF CHARGING ONTrigger AC Power break 15% energy left End of AC power break 20% charged UPS End of AC power breakAction Supply power via UPS Shut down systems Charge of UPS Start up systems Supply power via ACEnd State ON OFF CHARGING Normal power Stand ByExpectation Servers are operating Administrator is notified UPS is charging Servers are started up Servers are operating Routers are operating Related systems are Systems are down Temp. data is reprocessed Routers are operating UPS is active notified Routers are down Routers are started up UPS is in standby Routers are shutdown Related systems are modus Active processes are notified stopped Temporary data is saved Servers are shut down [Based on a 0-switch coverage]
  19. 19. Questions

×