T12 Exploitation Testing Presentation 1.2

1,019 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,019
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Turned to power down during a test?
  • T12 Exploitation Testing Presentation 1.2

    1. 1. T12: Exploitation testing For testing Service Level Agreements 1 december 2009 based on a best practice at
    2. 2. Agenda <ul><li>Understanding Exploitation testing </li></ul><ul><li>Best practice at APG </li></ul><ul><li>Service Level Management and Testing </li></ul><ul><li>Using State Transition Test to test Availability </li></ul>
    3. 3. Understanding Exploitation testing <ul><li>Based on Dutch term ‘Exploitatie test’ </li></ul><ul><li>Definition: Check whether the agreed or expected Service level related to IT infrastructures can be achieved. </li></ul><ul><li>Those agreements and/or expectations are formalized in Service Level Agreements (SLA) </li></ul><ul><li>Represents a group of quality attributes </li></ul>
    4. 4. Service Level Agreement (SLA) <ul><li>A negotiated agreement between two parties </li></ul><ul><ul><li>customer and service provider </li></ul></ul><ul><li>Containing measurable KPI’s, such as: </li></ul><ul><ul><li>99.9% uptime per month => max. downtime: 43.2 minutes </li></ul></ul><ul><ul><li>95% of transactions completed within 1 second </li></ul></ul><ul><ul><li>Mean Time To Recovery of 4 hours </li></ul></ul><ul><ul><li>98% of batch jobs executed within batch window </li></ul></ul><ul><ul><li>Maximum response time of 30 minutes </li></ul></ul>
    5. 5. Trends in IT service providing <ul><li>Formalization of SLA’s, due to: </li></ul><ul><ul><li>Increased Integration of systems </li></ul></ul><ul><ul><li>Globalization (higher availability demands) </li></ul></ul><ul><ul><li>Third party involvement </li></ul></ul><ul><ul><li>Growing complexity of infrastructures and architectures </li></ul></ul><ul><ul><li>Etc… </li></ul></ul>
    6. 6. Agenda <ul><li>Understanding Exploitation testing </li></ul><ul><li>Best practice at APG </li></ul><ul><li>Service Level Management and Testing </li></ul><ul><li>Using State Transition Test to test Availability </li></ul>
    7. 7. All Pension Group <ul><li>Pension provider for international pension funds </li></ul><ul><li>Approximately 4000 staff provide pensions for around 4 million participants </li></ul><ul><li>Internal IT department (CIS) exists of: </li></ul><ul><ul><li>Application Services (AS) – Software development </li></ul></ul><ul><ul><li>Exploitation Services (ES) – Service management </li></ul></ul><ul><ul><li>And others, like QA and Financial control </li></ul></ul><ul><li>Mission of CIS: Allow business to excel by delivering ICT services </li></ul>
    8. 8. Test optimization <ul><li>One of the optimization targets: Involve ES in an earlier stage in project and incorporate a formal test approach </li></ul><ul><li>Implemented elements: </li></ul><ul><ul><li>Involvement at reviews of specifications </li></ul></ul><ul><ul><li>Introduction of Test specification techniques at Exploitation tests </li></ul></ul><ul><li>Result: </li></ul><ul><ul><li>Improved the quality of advice given to stakeholders </li></ul></ul><ul><ul><li>Better understanding of the software and daily tasks </li></ul></ul><ul><ul><li>Increased confidence </li></ul></ul>
    9. 9. Agenda <ul><li>Understanding Exploitation testing </li></ul><ul><li>Best practice at APG </li></ul><ul><li>Service Level Management and Testing </li></ul><ul><li>Using State Transition Test to test Availability </li></ul>
    10. 10. Short view on ITIL v3 Service Validation and Testing
    11. 11. ITIL version 2 Service Support Operational Service Delivery Strategic Software testing Incident Management Problem Management Configuration Management Change Management Release Management Service level Management Availability Management Continuity Management Capacity Management Financial Management Security management Support desk
    12. 12. System Integration Component Development Business System Integration IT Service Management UC = Underpinning contract OLA = Operational Level Agreement Exploitation test within the V-model System specifications System Test Business requirements System specifications Technical design Development Technical Test System Test Exploitation Test Exploitation Acceptance Test SLA UC/ OLA User Acceptance Test
    13. 13. Stakeholders
    14. 14. SLA items versus Software testing SLA Items Quality attributes Test approach Availability Maturity / Fault tolerance State Transition Test Down/maintenance time Maintainability / Recoverability Review / During availability test Security Security Review / ‘hack test’ Capability Efficiency Performance and stress test Service related Items Quality attributes Test approach Verifiability Understandability / Analyzability During availability Test Serviceability Operability (IT Service perspective) Scenario / Real life test
    15. 15. Agenda <ul><li>Understanding Exploitation testing </li></ul><ul><li>Best practice at APG </li></ul><ul><li>Service Level Management and Testing </li></ul><ul><li>Using State Transition Test to test Availability </li></ul>
    16. 16. Use of State Transition Testing <ul><li>Test specification steps: </li></ul><ul><li>Specify the system components influencing the availability of the system </li></ul><ul><li>Specify the failures that may occur </li></ul><ul><li>Specify the measures taken to prevent these failures </li></ul><ul><li>Conduct State Transition Diagram </li></ul><ul><ul><li>Define the states in relation to these measures </li></ul></ul><ul><ul><li>Visualize the status and transitions between </li></ul></ul><ul><li>Specify the test cases </li></ul>
    17. 17. Example State Transition Testing (1) <ul><li>Critical system component: application and database servers </li></ul><ul><li>Business risk: systems unavailable, due to AC power break </li></ul><ul><li>Preventive measure: uninterruptible power supply (UPS) </li></ul><ul><li>State Transition Diagram </li></ul>AC Power break 15% of battery energy left Shut down Systems Supply power via UPS End of AC power break Charge UPS End of AC power break Supply power via AC Charged UPS Start up Systems Normal power ON OFF Charging
    18. 18. Example State Transition Testing (2) <ul><li>Specify test cases </li></ul>[Based on a 0-switch coverage] TC1 TC2 TC3 TC4 TC5 Start State Normal power ON OFF CHARGING ON Trigger AC Power break 15% energy left End of AC power break 20% charged UPS End of AC power break Action Supply power via UPS Shut down systems Charge of UPS Start up systems Supply power via AC End State ON OFF CHARGING Normal power Stand By Expectation Servers are operating Routers are operating UPS is active Administrator is notified Related systems are notified Routers are shutdown Active processes are stopped Temporary data is saved Servers are shut down UPS is charging Systems are down Routers are down Servers are started up Temp. data is reprocessed Routers are started up Related systems are notified Servers are operating Routers are operating UPS is in standby modus
    19. 19. Questions

    ×