18 th June 2008 Performance Testing Helping IT Help The Business

Performance Failures = Lost Revenue Yahoo News: Fri Jun 6, 2008 4:50 PM ET Online retailer Amazon Inc said its U.S. website was down for about 2 hours on Friday and it was unclear when service would be restored in full… Amazon shares fell 4.59 percent to close at $80.63 MSNBC News: Monday March 24 , 2008 NetFlix suffers 11 hour outage of its Retail as well as the shipping and distribution Websites… Affected customers compensated $5 each due to downtime Internet Business News: July 27, 2006 Australia's Jetstar airline Website crashed yesterday (26 July) as travelers went online to take advantage of promotional fares for the carrier's new international services.

Yahoo News: Fri Jun 6, 2008 4:50 PM ET

Online retailer Amazon Inc said its U.S. website was down for about 2 hours on Friday and it was unclear when service would be restored in full…

Amazon shares fell 4.59 percent to close at $80.63

MSNBC News: Monday March 24 , 2008

NetFlix suffers 11 hour outage of its Retail as well as the shipping and distribution Websites…

Affected customers compensated $5 each due to downtime

Internet Business News: July 27, 2006

Australia's Jetstar airline Website crashed yesterday (26 July) as travelers went online to take advantage of promotional fares for the carrier's new international services.

Performance Failures And Remember those high profile sites that failed to handle customer load during peak events: Victoria Secret Fashion Show… The FIFA World Cup Soccer website… CNET News.com: Jan 4 2001 After a day of lengthy 11 hour outages, eBay said Thursday that it has restored its auction system and intends to upgrade its hardware in the coming weeks to provide improved service. Outages of eBay's Web site attract attention because the vendor is so popular, but the problem occurs industry-wide--and not just with consumer sites. More hardware won't solve Web site outages -- Donna Scott, Gartner Analyst

And Remember those high profile sites that failed to handle customer load during peak events:

Victoria Secret Fashion Show…

The FIFA World Cup Soccer website…

CNET News.com: Jan 4 2001

After a day of lengthy 11 hour outages, eBay said Thursday that it has restored its auction system and intends to upgrade its hardware in the coming weeks to provide improved service.

Outages of eBay's Web site attract attention because the vendor is so popular, but the problem occurs industry-wide--and not just with consumer sites.

More hardware won't solve Web site outages -- Donna Scott, Gartner Analyst

Reluctance for Requirements The reluctance to defining performance requirements usually comes from: A lack of understanding for the purpose of performance requirements Time constraints due to tight and unrealistic deadlines – “market driven” and not “realism driven” Limited budgets - We have so many other things to worry about – it will be fine Ownership of systems is not understood – Key Stakeholders take a back seat when then they should be driving!

The reluctance to defining performance requirements usually comes from:

A lack of understanding for the purpose of performance requirements

Time constraints due to tight and unrealistic deadlines – “market driven” and not “realism driven”

Limited budgets - We have so many other things to worry about – it will be fine

Ownership of systems is not understood – Key Stakeholders take a back seat when then they should be driving!

Performance Goals and their importance “ Projects without clear goals will not achieve their goals clearly ” Tom Gilb -- The Tom Gilb Principle Recent studies have attributed many application’s poor performance to a lack of clear performance goals when the software was designed and developed. Further analysis has shown that much of this is caused by non-existent or incomplete performance specifications

“ Projects without clear goals will not achieve their goals clearly ”

Tom Gilb -- The Tom Gilb Principle

Recent studies have attributed many application’s poor performance to a lack of clear performance goals when the software was designed and developed. Further analysis has shown that much of this is caused by non-existent or incomplete performance specifications

Achieving Performance Goals Today’s discussion is intended to address 3 main points: Describe considerations for a systems performance Break down performance goals into tasks Emphasize the importance of making performance goals part of the IT software requirements definition

Today’s discussion is intended to address 3 main points:

Describe considerations for a systems performance

Break down performance goals into tasks

Emphasize the importance of making performance goals part of the IT software requirements definition

Planning Performance Goals When Planning for Performance Requirements: Consumer Requirements: It is critical to understand who or what will be using the Application Data: It is important to understand the types and amounts of data that are anticipated or required to meet business needs Traffic: It is important to understand the rates at which the data will need to be transacted in order to meet business goals Business: It is important to understand from a business perspective how much business needs conducting to make it a success – this will ultimately drive change

When Planning for Performance Requirements:

Consumer Requirements: It is critical to understand who or what will be using the Application

Data: It is important to understand the types and amounts of data that are anticipated or required to meet business needs

Traffic: It is important to understand the rates at which the data will need to be transacted in order to meet business goals

Business: It is important to understand from a business perspective how much business needs conducting to make it a success – this will ultimately drive change

Performance Requirements – Roles Planning Requirements Phases and Responsibilities: The business analysts and owners should be defining the performance requirements The technologists and developers will be designing and developing the application to meet the requirements The testers will be evaluating that the application does effectively meet the requirements The business and its stakeholders MUST sign the application off as part of the implementation process

Planning Requirements Phases and Responsibilities:

The business analysts and owners should be defining the performance requirements

The technologists and developers will be designing and developing the application to meet the requirements

The testers will be evaluating that the application does effectively meet the requirements

The business and its stakeholders MUST sign the application off as part of the implementation process

Performance Requirements Ingredients for Performance Requirements: Define requirements in order of priority Keep the performance requirements within the context of the organization's business Align the performance goals with specific business goals Create performance goals that can be measured Create performance goals that are inline with industry standards Determine goal achievement criteria Decide which goals will have pass fail grades and which will have varying degrees of tolerance Keep in mind the business that needs to be conducted within the application to meet the consumer needs or meet the financial targets

Ingredients for Performance Requirements:

Define requirements in order of priority

Keep the performance requirements within the context of the organization's business

Align the performance goals with specific business goals

Create performance goals that can be measured

Create performance goals that are inline with industry standards

Determine goal achievement criteria

Decide which goals will have pass fail grades and which will have varying degrees of tolerance

Keep in mind the business that needs to be conducted within the application to meet the consumer needs or meet the financial targets

Inter-Team Communication Objectives may have been set and requirements defined but sometimes the business needs change The key to successfully implementing performance testing is continual communication among team members – manage change accordingly An effective mechanism of communication and collaboration will allow the team to make good performance related decisions, whilst changes are easy to make The methods you use to communicate plans, priorities and changes are not that important as long as you are able to adapt those changes without requiring significant re-work

Objectives may have been set and requirements defined but sometimes the business needs change

The key to successfully implementing performance testing is continual communication among team members – manage change accordingly

An effective mechanism of communication and collaboration will allow the team to make good performance related decisions, whilst changes are easy to make

The methods you use to communicate plans, priorities and changes are not that important as long as you are able to adapt those changes without requiring significant re-work

Anatomy of a Performance Test Requirements Discovery Test Planning Automation Testing Measurements Analysis Tuning and Re-Testing

Requirements

Discovery

Test Planning

Automation

Testing

Measurements

Analysis

Tuning and Re-Testing

Conclusion Costs associated with lack of testing and understand application performance are evident Performance Requirements need to be oriented around business goals – its usually the business who wants and owns the system The roles of performance testing Business Development Testing “ If all these are successful, you can be sure that IT really can help the Business"

Costs associated with lack of testing and understand application performance are evident

Performance Requirements need to be oriented around business goals – its usually the business who wants and owns the system

The roles of performance testing

Business

Development

Testing

“ If all these are successful, you can be sure that IT really can help the Business"