T3 Consortium's Performance Center of Excellence


Published on

Learn more about building and mangaging a IT Test Lab that centralizes process and tools to create a more robust CoE (Center of Excellence)

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • T3 Consortium's Performance Center of Excellence

    1. 1. Professional Services Performance Testing Center of Excellence Application Performance Management Solution Click to Advance Presentation Mute sound if necessary (sound icon) Information Technology Services
    2. 2. Today’s Challenges Business lines rarely share information on tool research, usage, testing practices, and cost containment. Software Development Life Cycle models frequently compromise testing initiatives to meet deliverable deadlines. Performance Testing Practices vary in most projects. Knowledge Transfer – performance testing information leaves with consultants. Performance Tools are expensive, and very complex to integrate. No global approach to Performance Testing tools, techniques, or costs. Expensive consultants are frequently hired for Performance Testing Many projects cannot afford to make long term commitments to Performance testing. Decentralized approach to building and testing software.
    3. 3. Industry Trends Application owners often ask the following: Does my application scale to meet business goals? 75% of ERP systems fail to do so (Gartner) Does my application scale to meet business goals? 70% of performance problems can be resolved by configuration changes (survey of 3,000 HP/Mercury engagements) Can I resolve application bottlenecks? 40% of app bottlenecks are first noticed by a customer or executive (Mercury Survey) $2B + spent in app server overcapacity from 2001 to 2003 (Gartner) Have I overspent on infrastructure?
    4. 4. One Platform Solution T3 works to empower Business, Quality Assurance, and Development teams to deliver better software by centralizing all aspects of testing and performance management. “ Bringing the pieces together” Become Performance center of excellence Centralize performance testing and monitoring tools Global Performance testing execution capability Focus on quick “cost savings” wins! Reduce dependency on expensive consultancy QUICK START- Bring automation in quickly that addresses an immediate need. Reduce infrastructure support for testing Take a ‘value’ approach and network it Forge alliances & partnerships Leverage vendor Purchasing Power
    5. 5. Performance Approach Phase-driven approach to Performance Optimization Phase 1: Plan Fully Plan the project Organize the Team Determine the business goals Define the business process Inventory HW, SW and network Identify key participants Mobilize test team Create test plan Agree execution schedule Phase 2: Baseline Quantify the System Performance Create test cases Deploy monitoring agents Run initial test plan Document the baseline profile Compare baseline with target Phase 3: Optimize Iteratively Isolate and Reduce Bottlenecks Emulate production load Monitor system performance Identify problem areas Analyze root cause Determine resolution Apply modification Assess the Performance Improvements and Report Phase 4: Report <ul><li>Document </li></ul><ul><ul><li>improved throughput </li></ul></ul><ul><ul><li>Increased capacity </li></ul></ul><ul><ul><li>reduced error rate </li></ul></ul><ul><ul><li>greater stability </li></ul></ul><ul><ul><li>better user response time </li></ul></ul><ul><li>Compare with baseline and target </li></ul><ul><li>Produce findings/recommendations </li></ul><ul><li>Retain data for future comparison </li></ul>
    6. 6. Why HP/Mercury <ul><li>Proven Leadership </li></ul><ul><li>Market Visionaries </li></ul><ul><li>Strategic Alliance </li></ul>Worldwide Performance Testing Market Share Source Newport Group, Inc © 2004 Empirix 4% Compuware 4% Other 9% Segue 2% Rational 9% RadView 1% HP/Mercury 71%
    7. 7. Automated Performance Testing Web Server App. Server Database Internet/ WAN Controller User Simulation PERFORMANCE MONITORS <ul><li>Replaces real users with thousands of virtual users </li></ul><ul><li>Generates consistent, measurable, and repeatable load, managed from a single point of control </li></ul><ul><li>Efficiently isolates performance bottlenecks across all tiers/layers with automated reporting and analyses </li></ul>
    8. 8. Types of Performance Testing <ul><li>Stress Testing </li></ul><ul><li>Capacity Planning </li></ul><ul><li>Throughput Improvement </li></ul><ul><li>Server Consolidation </li></ul><ul><li>Baseline Assessment </li></ul><ul><li>New Version Impact Analysis </li></ul><ul><li>Ensure Business Performance </li></ul>
    9. 9. Application Monitoring <ul><li>98% of sites experience critical performance problems </li></ul><ul><ul><li>Typical applications encounter problems at 15% of design capacity </li></ul></ul><ul><li>Problems occur both inside & outside firewall </li></ul><ul><ul><li>25% - network and bandwidth related </li></ul></ul><ul><ul><li>23% - application server related </li></ul></ul><ul><ul><li>20% - load balancers, web server or proxy server issues </li></ul></ul>Source: Mercury Interactive hosted services Application Management monitors the entire infrastructure from the users perspective! Application Server Web Servers Database Server Router Router Load Balancer Firewall ISP ISP ISP
    10. 10. A Day In the Life… Daily Challenges 07:17AM Ticket opened 07:05AM Problem detected - data center serving e-Bank customers is down 07:31AM Bridge-line opened - 8 people 11:48AM 4 more people paged , DBA joins line 04:05PM Root cause identified : mis-configured connection pooling caused DB crash at peak traffic. At least 800 customers affected, business impact unknown 04:20PM IT representative sent to client impact assessment meeting 08:03AM Check UNIX issue, escalate to hardware vendor 08:06AM Business urgency discussed , not well understood 10:51AM Hardware is ok, Informix DB appears down 10:20AM Security clearance delays hardware vendor from entering data center
    11. 11. Diagnose Root Cause Define SLAs Business Console Customer Impact SLM Event Viewer Tier 2 Top View BUSINESS AVAILABILITY CENTER Tier 1 Tier 3 RESOLUTION CENTER
    12. 12. Application Management Dashboard Define SLAs Business Console Customer Impact SLM Event Viewer Tier 1 Tier 2 Tier 3 Top View BUSINESS AVAILABILITY CENTER RESOLUTION CENTER
    13. 13. Appendix <ul><li>Supplemental Information </li></ul>
    14. 14. Introducing LoadRunner Our Center of Excellence approach uses Mercury Interactive LoadRunner product to deliver an integrated solution for business technology optimization. Load Runner System Monitor Automated Scripts Data Repository <ul><li>It provides an end-to-end tool set for performance tuning </li></ul><ul><ul><li>a user simulation module to create business transaction load </li></ul></ul><ul><ul><li>a system monitoring module to display infrastructure behavior and application errors </li></ul></ul><ul><ul><li>a set of automated scripts that identifies modifications </li></ul></ul><ul><ul><li>a data repository for future comparisons </li></ul></ul>Network bandwidth Routers Switches Hosting equipment Load balancers Application servers Databases Middleware Back office Firewalls IDS systems DDoS systems Gateways Web servers Cache servers LAN / WAN Client Systems Legacy system .NET services Sun ONE services Security Infrastructure Application
    15. 15. Powering the CoE Application Delivery Foundation Application Monitoring Business Availability Center • Resolution Center Application Delivery Console Virtual Users Protocols Monitors PERFORMANCE CoE Global Management Multi-project Management Resource Management Diagnostics Tuning LoadRunner Capacity Planning
    16. 16. LoadRunner Coverage Recommendation Resolution Problem Identification Monitoring User Simulation Infrastructure (e.g. Linux, Solaris, Windows) Vendor Product (e.g. Oracle, Websphere) Performance Tuning Steps Tunable Components LoadRunner Capabilities in Optimization Exercise In-house Apps <ul><li>method level for J2EE apps </li></ul><ul><li>program level for other types </li></ul>* J2EE source code line-level identification available using OptiBench add-on.
    17. 17. LoadRunner Overview Web Server App. Server Database Internet/ Intranet USER SIMULATION PROTOCOLS Clients PERFORMANCE MONITORS <ul><li>SAP </li></ul><ul><li>Oracle </li></ul><ul><li>Siebel </li></ul><ul><li>PeopleSoft </li></ul>ERP/CRM <ul><li>HTTP(S) </li></ul><ul><li>XML </li></ul><ul><li>Citrix ICA </li></ul><ul><li>SOAP </li></ul><ul><li>WAP </li></ul>Web <ul><li>EJBs </li></ul><ul><li>CORBA </li></ul><ul><li>COM </li></ul><ul><li>RMI </li></ul><ul><li>MQSeries </li></ul><ul><li>3270 </li></ul><ul><li>5250 </li></ul><ul><li>VT100 </li></ul><ul><li>Oracle </li></ul><ul><li>MS SQLServer </li></ul><ul><li>DB2 </li></ul><ul><li>ODBC </li></ul>Middleware Legacy Databases Operating Systems Network Web Servers App Servers Java Databases <ul><li>Windows </li></ul><ul><li>Unix </li></ul><ul><li>Linux </li></ul><ul><li>SNMP </li></ul><ul><li>WAN Emulation </li></ul><ul><li>MS IIS </li></ul><ul><li>iPlanet </li></ul><ul><li>Apache </li></ul><ul><li>EJB </li></ul><ul><li>JDBC </li></ul><ul><li>JSP </li></ul><ul><li>Sitraka JMonitor </li></ul><ul><li>Oracle </li></ul><ul><li>MSSQL Server </li></ul><ul><li>DB2 </li></ul><ul><li>BEA WebLogic </li></ul><ul><li>IBM WebSphere </li></ul><ul><li>ATG Dynamo </li></ul><ul><li>iPlanet App Server </li></ul>
    18. 18. LoadRunner Deployment Application Tier Applications Infrastructure Tier Database Tier Distributed System App & System Monitoring Measure Performance -> Identify Constraints -> Apply Modification ~~~ ~~~ Stress Simulation Telecomm Fabric Automated & Manual Tuning Apply Tuning Process Gather Metrics Step 1 <ul><li>Define business process </li></ul><ul><li>Capture user behavior </li></ul><ul><li>Create simulation profile </li></ul><ul><li>Apply controlled load </li></ul>Step 2 Step 4 Sample Recommendations WebLogic - misconfigured Java VM heap size Database - missing indexes, full table scan ... Step 3 Environment under User Load Typical Statistics Application Servers Fire wall Web Servers Load Balancer Fire wall Database Servers Storage Infrastructure Consumption CPU 28 CPUs Memory 2,560MB Disk I/O 70% Network 80% System Capacity User count 100 Tranx rate 100/min User Experience Response time 10 sec Error rate 10 %
    19. 19. Metrics Databases <ul><li>Oracle </li></ul><ul><ul><li>Total file opens </li></ul></ul><ul><ul><li>Enqueue deadlocks </li></ul></ul><ul><ul><li>Enqueue waits </li></ul></ul><ul><ul><li>Opened cursors current </li></ul></ul><ul><ul><li>SQL*Net roundtrips to/from client </li></ul></ul><ul><ul><li>Plus 164 other counters </li></ul></ul><ul><li>Sybase </li></ul><ul><ul><li>Disk Reads </li></ul></ul><ul><ul><li>Memory </li></ul></ul><ul><ul><li>Disk Writes </li></ul></ul><ul><ul><li>Disk Waits </li></ul></ul><ul><ul><li>Locks count </li></ul></ul><ul><ul><li>% Hits </li></ul></ul><ul><ul><li>% Processor Time (process) % </li></ul></ul><ul><ul><li>Plus 49 other counters </li></ul></ul><ul><li>Microsoft SQL Server </li></ul><ul><ul><li>SQL Re-Compilations/Sec </li></ul></ul><ul><ul><li>I/O - Outstanding Reads </li></ul></ul><ul><ul><li>Lock Wait Time (ms) </li></ul></ul><ul><ul><li>Total Latch Wait Time (ms) </li></ul></ul><ul><ul><li>SQL Re-Compilations/Sec </li></ul></ul><ul><ul><li>Plus 38 other counters </li></ul></ul><ul><li>DB2 </li></ul><ul><ul><li>Local_con_exec </li></ul></ul><ul><ul><li>Total_sorts </li></ul></ul><ul><ul><li>Plus 174 other counters </li></ul></ul>App. Servers <ul><li>IBM WebSphere </li></ul><ul><ul><li>ThreadCreates </li></ul></ul><ul><ul><li>ActiveThreads </li></ul></ul><ul><ul><li>ConnectionPoolSize </li></ul></ul><ul><ul><li>SessionsActive </li></ul></ul><ul><ul><li>Plus 82 other counters </li></ul></ul><ul><li>BEA WebLogic </li></ul><ul><ul><li>HeapSizeCurrent </li></ul></ul><ul><ul><li>ActiveConnectionsCurrentCount </li></ul></ul><ul><ul><li>WaitingForConnectionCurrentCount </li></ul></ul><ul><ul><li>Plus 118 other counters </li></ul></ul><ul><li>Iplanet Application Server </li></ul><ul><ul><li>nasKesEngAutoStart </li></ul></ul><ul><ul><li>nasEngSYBPreparedQueryTotal </li></ul></ul><ul><ul><li>nasEngThreadWait </li></ul></ul><ul><ul><li>Plus 118 other counters </li></ul></ul><ul><li>Microsoft Active Server Pages </li></ul><ul><ul><li>Requests Executing </li></ul></ul><ul><ul><li>Requests Queued </li></ul></ul><ul><ul><li>Request Bytes Out Total </li></ul></ul><ul><ul><li>Requests/Sec </li></ul></ul><ul><ul><li>Transactions Aborted </li></ul></ul><ul><ul><li>Transactions Pending </li></ul></ul><ul><li>Oracle 9iAS HTTP Server </li></ul><ul><li>Allaire ColdFusion </li></ul><ul><li>SilverStream </li></ul><ul><li>Ariba </li></ul><ul><li>ATG Dynamo </li></ul><ul><li>Microsoft COM+ Monitor </li></ul><ul><li>BroadVision </li></ul>Web Servers <ul><li>Apache </li></ul><ul><ul><li>#Busy Servers </li></ul></ul><ul><ul><li>#Idle Servers </li></ul></ul><ul><ul><li>CPU Usage </li></ul></ul><ul><ul><li>Hits/Sec </li></ul></ul><ul><ul><li>KBytes Sent/Sec </li></ul></ul><ul><li>MS IIS </li></ul><ul><ul><li>Connection Attempts/Sec </li></ul></ul><ul><ul><li>Files Received/Sec </li></ul></ul><ul><ul><li>Logon Attempts/Sec </li></ul></ul><ul><ul><li>Total Files Transferred </li></ul></ul><ul><ul><li>Plus 50 other counters </li></ul></ul><ul><li>Iplanet (Netscape) </li></ul><ul><ul><li>Bad requests/Sec </li></ul></ul><ul><ul><li>Bytes Sent/Sec </li></ul></ul><ul><ul><li>Hits/Sec </li></ul></ul><ul><ul><li>Plus 11 other counters </li></ul></ul>Other Software and Hardware Streaming Media Monitors Checkpoint Firewall Server ERP Performance Monitors Middleware Monitors Java Performance Monitors Citrix MetaFrame Monitors Network Delay Customized Counters Capability Customized Counters Capability
    20. 20. The Tuning Process LoadRunner’s tuning agent can be deployed to automatically recommend optimum settings for a wide range of industry-standard applications. IT specialists will manually optimize additional hardware and software based on LoadRunner performance metrics. <ul><li>Apache Web Server 1.x/2.x </li></ul><ul><li>BEA Weblogic 6.x/7.x </li></ul><ul><li>IBM HTTP Server </li></ul><ul><li>IBM Websphere Advanced 4.x </li></ul><ul><li>IBM Websphere Single Server 4.x </li></ul><ul><li>iPlanet Enterprise Server 6 & higher </li></ul><ul><li>Microsoft IIS 4/5 </li></ul><ul><li>Microsoft Active Server Pages 2/3 </li></ul><ul><li>Oracle Database </li></ul><ul><li>Oracle 9iAS </li></ul><ul><li>PeopleSoft 8.x </li></ul><ul><li>SAP Enterprise Portals 5 </li></ul><ul><li>Siebel 7.x </li></ul><ul><li>SQL Server 7.5/2000 </li></ul><ul><li>Windows: NT, 2000 and XP </li></ul><ul><li>UNIX: Solaris, HP, AIX and Linux </li></ul>
    21. 21. Benefits of Tuning <ul><li>Optimizes application and infrastructure performance </li></ul><ul><li>Isolates and resolves performance bottlenecks </li></ul><ul><li>Establishes optimized configuration settings for production </li></ul>
    22. 22. Benefits of LoadRunner Diagnostics <ul><li>Pinpoints application bottlenecks, e.g., J2EE to method/ SQL level </li></ul><ul><li>Reduces time to resolution for application issues </li></ul><ul><li>Integrated with Mercury LoadRunner— combines end user response time with diagnostics </li></ul>
    23. 23. Visit Us Online For additional information contact: [email_address] Click Here or copy/paste www.t3consortium.com Global Virtual Testing Capabilities