Governance, Management and Testing in an Outsourced Model Ian Londesbrough Principal Consultant 8 th October 08

Agenda Understand the context of Outsourcing Focus in on the Quality – top to tail of the contract Implementing the Testing Approach Roles and Responsibilities Who will Test Who will Accept Measurement and Metrics Governance Conclusion Questions

Understand the context of Outsourcing

Focus in on the Quality – top to tail of the contract

Implementing the Testing Approach

Roles and Responsibilities

Who will Test

Who will Accept

Measurement and Metrics

Governance

Conclusion

Questions

Outsourcing Challenges Globalisation drives constant Change Businesses are engaged in complex programmes to transform: Business Operating Models Business Processes Business Systems Phase 2: The Evaluation & Selection Phase 1: The Decision to Outsource Phase 3: Contract for Success Phase 4: Transition & Go-Live Phase 5: Operation & ROI Phase 2: The Evaluation & Selection Phase 1: The Decision to Outsource Phase 3: Contract for Success Phase 4: Transition & Go-Live Phase 5: Operation & ROI The Path to Outsourcing Phase 2: The Evaluation & Selection Phase 1: The Decision to Outsource Phase 3: Contract for Success Phase 4: Transition & Go-Live Phase 5: Operation & ROI

Globalisation drives constant Change

Businesses are engaged in complex programmes to transform:

Business Operating Models

Business Processes

Business Systems

The Path to Outsourcing

Key Points to Outsourcing Understand what Outsourcing is attempting to achieve: Reduce cost Resource flexibility Shorter time to market Faster business growth Assess existing processes Engage the business – tell them what’s happening Emphasise sound Quality Governance and Testing Construct the RFP to differentiate competitors Select using “testable” & “measurable” criteria Consider using sourcing professionals

Understand what Outsourcing is attempting to achieve:

Reduce cost

Resource flexibility

Shorter time to market

Faster business growth

Assess existing processes

Engage the business – tell them what’s happening

Emphasise sound Quality Governance and Testing

Construct the RFP to differentiate competitors

Select using “testable” & “measurable” criteria

Consider using sourcing professionals

Contracting for Quality Expand the scope of the outsourcing into measurable and testable clauses Define how each Quality Gate will be measured and signed off, and by whom Define acceptable levels of variance, if any Build in a review and improvement process for each delivery iteration Use the Quality Gates to ensure the supplier delivers what is contracted Define any penalties Address any cultural barriers To fail to prepare is to prepare to fail

Expand the scope of the outsourcing into measurable and testable clauses

Define how each Quality Gate will be measured and signed off, and by whom

Define acceptable levels of variance, if any

Build in a review and improvement process for each delivery iteration

Use the Quality Gates to ensure the supplier delivers what is contracted

Define any penalties

Address any cultural barriers

Implementing the Testing Approach Agreeing roles and responsibilities for all partners Establishing who will test the solutions provided at each stage of the lifecycle Establishing who will integrate and accept the solutions into the business Understanding what metrics and quality measures need to be in place to ensure the partners add value to your business Defining contracts and commercial models that put the onus on service providers to deliver against these Managing and governing the deliverables and services against these contracts Begin with the end in mind

Agreeing roles and responsibilities for all partners

Establishing who will test the solutions provided at each stage of the lifecycle

Establishing who will integrate and accept the solutions into the business

Understanding what metrics and quality measures need to be in place to ensure the partners add value to your business

Defining contracts and commercial models that put the onus on service providers to deliver against these

Managing and governing the deliverables and services against these contracts

Lifecycle, Testing and Quality Gates Review (Internal Static Test) The Idea Review (Static Test) Review (Static Test) Dynamic Test Review (Internal Static Test) Requirements Definition Specification & Design Build Implementation Benefits Realisation Outsourced Responsibility (Simple model) 1 2 3 4 Dynamic Test 5 6

Agree Roles and Responsibilities Define the scope of the outsourcing Decide quality & testing responsibilities before contract negotiation Business areas have roles and responsibilities for testing Appoint a strong Test Programme Manager Include Test Roles and Responsibilities in the contract Confirm in-house test activities at initial handover and acceptance Need clear responsibilities if Penalties and Warranties are challenged “ We thought YOU were doing it” is not an option

Define the scope of the outsourcing

Decide quality & testing responsibilities before contract negotiation

Business areas have roles and responsibilities for testing

Appoint a strong Test Programme Manager

Include Test Roles and Responsibilities in the contract

Confirm in-house test activities at initial handover and acceptance

Need clear responsibilities if Penalties and Warranties are challenged

Who will Test? The business risk stays with the organisation Testing at each stage of the lifecycle must match Acceptance Criteria Define responsibility for Testing at Contract stage Involve Business and Supplier in Static Testing (e.g for Requirements and Design) Is Supplier Testing “marking own homework”? Independent testing clearly demonstrates criteria are met (or not) Testing validates that the right thing has been built and verifies that it has been built right

The business risk stays with the organisation

Testing at each stage of the lifecycle must match Acceptance Criteria

Define responsibility for Testing at Contract stage

Involve Business and Supplier in Static Testing (e.g for Requirements and Design)

Is Supplier Testing “marking own homework”?

Independent testing clearly demonstrates criteria are met (or not)

Who will Accept? Contract specifies unambiguous Acceptance Criteria The solution needs integrating into existing systems Ensure end users are available for Acceptance stage Supplier support is essential In-house Operations, both IT and Business, need to be included In-house organisation may not have ability to Test for Acceptance Consider experienced third-party Acceptance Testing Acceptance requires the same visibility and joint sign-off as other Quality Gates Accept it at the end only if it matches what you defined at the beginning

Contract specifies unambiguous Acceptance Criteria

The solution needs integrating into existing systems

Ensure end users are available for Acceptance stage

Supplier support is essential

In-house Operations, both IT and Business, need to be included

In-house organisation may not have ability to Test for Acceptance

Consider experienced third-party Acceptance Testing

Acceptance requires the same visibility and joint sign-off as other Quality Gates

Measure success - or, how was it for you? Define metrics and data capture in line with contract Appropriate Benchmarks are needed to measure compliance What you measure will depend on the nature of the outsourced service: The metrics identified in the contract Agreed progress metrics Agreed quality metrics (e.g. defects found) Measure throughout the life-cycle Include in regular reports – not just Quality Gates Use for review and improvement process “ The only man who behaves sensibly is my tailor; he takes my measurements anew every time he sees me, while all the rest go on with their old measurements and expect me to fit them” George Bernard Shaw

Define metrics and data capture in line with contract

Appropriate Benchmarks are needed to measure compliance

What you measure will depend on the nature of the outsourced service:

The metrics identified in the contract

Agreed progress metrics

Agreed quality metrics (e.g. defects found)

Measure throughout the life-cycle

Include in regular reports – not just Quality Gates

Use for review and improvement process

Governance against contract Governance is an in-house responsibility Requires an owner and necessary commitment Monitor the contracted acceptance criteria at each stage Ensure the quality of the deliverables is being measured through Testing Time Quality Gate reviews to ensure all interested parties attend Assess whether the outsourcing objectives are being met Review and improve for each iteration of the delivery life-cycle To fail to prepare is to prepare to fail. To prepare well and then fail to govern invites failure back in!

Governance is an in-house responsibility

Requires an owner and necessary commitment

Monitor the contracted acceptance criteria at each stage

Ensure the quality of the deliverables is being measured through Testing

Time Quality Gate reviews to ensure all interested parties attend

Assess whether the outsourcing objectives are being met

Review and improve for each iteration of the delivery life-cycle

Conclusions Understand the reasons for outsourcing and test against these Outsourcing needs strong management as well as in-house delivery It is not a silver bullet - you need to get your own house in order Additional effort prior to contract signature increases chance of success Focus on quality at all stages of the lifecycle – testing isn’t a bolt-on activity Be strong and rigorous with quality gates and measurement Don’t accept the unacceptable

Understand the reasons for outsourcing and test against these

Outsourcing needs strong management as well as in-house delivery

It is not a silver bullet - you need to get your own house in order

Additional effort prior to contract signature increases chance of success

Focus on quality at all stages of the lifecycle – testing isn’t a bolt-on activity

Be strong and rigorous with quality gates and measurement

Don’t accept the unacceptable

Questions? Please e-mail: [email_address]