An Introduction to Software Performance Engineering

Enterprise SoftwarePerformance EngineeringBuilding systems that scream so your users don’t:A key component of risk managementPresented By: Walter KuketzHosted By: Frank Days © 2012 Collaborative Consulting

Housekeeping Presentation will last 45 Submit questions via the Slides will be made minutes chat window available 2 2012 Collaborative Consulting

Purpose of today’s webinar Review the current challenges of being a performance engineer Overview of the five knowledge areas of Software performance engineering body of knowldge Where to inject PE tasks and activities into the SDLC to build more performant applications Overview of application performance management (end-to-end user experience) Roles and career path overview Case study – 10X event 3 2012 Collaborative Consulting

Performance engineering challenges today Highly • Complex and tangled system of systems distributed • Distributed Infrastructure apps • Distributed production operations • Performance engineering and testing Distributed • Architecture and development teams • Business Evolving & • Mobile devices changing • Third party services application • SaaS products architectures • Cloud Changing • Driven by mobile anywhere usage • Flash traffic events patterns 4 2012 Collaborative Consulting

Performance engineering challenges User Enterprise Budget Team skills experience • Release • User populations • Constant cost • Performance schedules, mont are exploding and budget organization; hly, quarterly pressure skillsets, multi • Web site disciplined, • Project based performance • Maintaining a communication, directly impacts meaningful split team revenue performance lab (production-like) • Training and career paths 5 2012 Collaborative Consulting

No clear standards in the market for PE Test automation technician Capacity planner ? Performance tester ? ? Performance Engineer Performance analyst ? Performance Architect ? Benchmarks ? Troubleshooting Performance Data engineer ? 6 2012 Collaborative Consulting

Goal of the SPE Body of Knowledge Define the profession of performance Define profession engineering Establish Body of Establish a body of knowledge for Knowldge performance engineering Establish Establish a common language for roles, common responsibilities, and practice areas using language the form of Knowledge Areas Define Define the competencies shared across competencies the knowledge areas Capture common Capture common techniques for sharing techniques across the industry 7 2012 Collaborative Consulting

The SPE Body of Knowledge Enterprise Software Performance Engineering Planning, coordination, information sharing, and control Knowledge areas Performance validation and testing Problem Application SDLC & (PVT) detection performance architecture and management (SA) Capacity planning (APM) resolution (PDR) (CP) Underlying competencies and roles for delivering Techniques supporting the knowledge areas 8 2012 Collaborative Consulting

The Enterprise PE team: There must be a practice leader A shared service and clearing house for all performance, scalability and capacity planning information. Transforms performance information to business knowledge and advantage. Communicates across the teams to share information For instance, the production workload and the Operations PT performance test workload – are they in sync? Architecture and Make sure the architecture team knows how the production new release, or newest build behave in production. The business risks are clearly identified and Business direction communicated. Can provide performance guidelines to the Development teams developers, must influence the unit testing process. 9 2012 Collaborative Consulting

Information sharing across the knowledge areas Enterprise PE Capacity planning and planning Triage monitoring SPE impacts Architects, Results, Developers, testing, and Forecast Workload operations Workload scalability NFR’s Performance User Exp Volumetrics, validation, Workload Utilization workload testing SDLC, Results Architecture Observations APM , NFR Growth plans Workload NFR’s 10 2012 Collaborative Consulting

PE: Knowledge areas SA PVT CP APM PDR Software development Performance testing and Capacity planning Application performance Problem detection andlifecycle and architecture validation management resolution 11 2012 Collaborative Consulting

PE: Software development lifecycle and architecture SA PVT CP APM PDR Software development Performance testing and Capacity planning Application performance Problem detection andlifecycle and architecture validation management resolution 12 2012 Collaborative Consulting

PE: SA (software development lifecycle) Waterfall methodologies Functional Performance Define Design Develop Deploy test test Need to plan for Build code to performance performance testing requirements Define Deploy Agile Design methods Test Build Can a system be designed to support 10 TPS and 1 second response time work @ 100 TPS and 1 second? 13 2012 Collaborative Consulting

SDLC: Define and design Define Design 1. Project risk scorecard 1. Technology Architecture validated - Key business transactions to the performance Usage identified Scenarios. How will it achieve the - Workflow constraints key performance requirements? Sub-second - SLA’s and penalties response time. Define the scalability approach. Flash marketing 2. Workload model and usage event. 2. Scalability to support flash events scenarios 3. Environments: include any 3. Business volumes with peak required changes to develop and factors test the performance usage 4. Business performance scenarios. requirements Communication to your downstream processes and people Prepare to deal with flash events Design to avoid flash events (they are expensive. E.g. Work with business why does the outbound email have to send email to all xM customers at the same time 14 2012 Collaborative Consulting

SDLC: Develop and functional testing Functions are tested. Performance needs to be analyzed. Develop Functional test • The functions are mapped to software • Verify business functions and key components. The performance req’s are performance functions. assigned to the components. • Verify the key performance workflows • Unit testing for early validation of the and functions. performance req’s for the software components. Response time • Define and execute micro-performance measurements. testing in the QA process. • Integration testing include micro- • Produce report on for each key performance testing of key functions. performance Usage Scenario with Code review of the key performance response time and throughput. functions. • Enhance unit and integration test for early validation of key performance functions Greatest impact on performance and scalability, but you must know the goals. Communication to your downstream processes and people 15 2012 Collaborative Consulting

Performance budgets Do your developers know how much time they have for their piece of the transaction? Code runs everywhere. SERVER WEB APP SVR PROXY MQ/ESB LDAP CORBA DCOM Respond to the user in 2 seconds. Web Services Datacenter 16 2012 Collaborative Consulting

PE: Performance testing and validation SA PVT CP APM PDR Software development Performance testing and Capacity planning Application performance Problem detection andlifecycle and architecture validation management resolution 17 2012 Collaborative Consulting

PE: PTV (performance testing and validation) What is changing? Where does it need to be validated? What (all) is being measured? The project: Why are we testing and what are the success criteria? Prerequisites: Business goals and non-functional requirements are defined Workload and scenarios defined Environment defined (where are we testing) Application in a stable state Configuration management and release management ready Core performance team identified Supporting team committed 18 2012 Collaborative Consulting

Organization: Performance team value You still need to explain to the rest of the organization what you do Business and Early involvement application area with the applications Test execution Environment knowledge team• Deep understanding of • Performance engineers • Well-understood • Testing environment is the application being involved in design workload model and a close approximation tested sessions characterization of production, including database size• Deep understanding of • Understand the nature • All components in the the technical of the data required for test are monitored and • Differences between architecture of the testing reported on test and production are application clearly known • Able to understand • A defined triage process• Understanding of the technical gaps to meet for troubleshooting • Release and business area requirements not configuration • A process to manage supported by the supported by the tool management in place the inevitable applications unplanned tests and and procedures enforced test archive exists Performance team value 19 2012 Collaborative Consulting

Performance team reference model Primary test, design, and execution team Performance engineering support skills Planning and Application project Performance architect management engineer Technology SME Test Results automation analyst Data technician engineering Must determine the core team members vs. on-demand team 20 2012 Collaborative Consulting

Enterprise performance team PT problem Strategy and PT technology resolution planning PMO board (architecture)** Infrastructure** Performance Performance test engineering and execution team optimization Application Vendor Application support management deployment Vendor IBM Oracle EMC Informatica Vendor Vendor Vendors 21 2012 Collaborative Consulting

Model test case execution In the end, the test has to be run. Prepare test Prepare test Confirm Select test harness online Initiate test 0 case or batch monitoring components environment readiness case 1 messages No Problem analysis and Evaluate triage team impact of Select test Normal test 1 case completion? change Yes Yes Gather test Analysis for Create test results specific report; publish Txn RT, CPU, DB change? to repository 2 No Are Categorize Can you Implement changes change – enter retest with Rerun the test 2 required for Yes change to CM configuration change the Yes case retest? tracking tool change? No No Move to next Move to next test case 0 test scenario 22 2012 Collaborative Consulting

PE: Capacity planning SA PVT CP APM PDR Software development Performance testing and Capacity planning Application performance Problem detection andlifecycle and architecture validation management resolution 23 2012 Collaborative Consulting

PE: CP (capacity planning) The right system at the right price! Plan for • Forecast workloads future • Plan for usage Analyze • Track utilization of existing systems current capacity • Track workload Determine • Define system to capacity support workloads requirements • Test • Define service levels Workloads drive system transactions, which drive resource utilization 24 2012 Collaborative Consulting

Capacity: Use of resources The overall system has a capacity; each component has a capacity. SERVER WEB APP SVR PROXY MQ/ESB LDAP CORBA DCOM Web Services Datacenter 25 2012 Collaborative Consulting

Workload model validation Actual workload Synthetic workload Model calibration System System Measured response times, Measured response times, throughputs, utilizations, etc. throughputs, utilizations, etc. Acceptable No Yes 26 2012 Collaborative Consulting

PE: Application performance management SA PVT CP APM PDR Software development Performance testing and Capacity planning Application performance Problem detection andlifecycle and architecture validation management resolution 27 2012 Collaborative Consulting

Five elements of APM (Gartner) Monitoring the performance of complex distributed applications End user experience measurement – browser/mobile 1 device 2 Create a model of the run-time environment (discovery) Profile the performance and behavior of user-defined 3 transactions Performance metrics from each of the 4 applications/systems technical components (Webserver, App server, Database, etc.) 5 Application performance management database 28 2012 Collaborative Consulting

End user experience monitoring 29 2012 Collaborative Consulting

Track key user experience metrics Mobile Total end-user response time Browser rendering time Network latency Mobile Real User Monitoring Rendering 30 2012 Collaborative Consulting

And see every hop of the transaction Mobile Proxy server Web server App server Data warehouse gateway Mainframe Database Dynamic Transaction Path Detection Rendering 31 2012 Collaborative Consulting

PE: Problem detection and resolution SA PVT CP APM PDR Software development Performance testing and Capacity planning Application performance Problem detection andlifecycle and architecture validation management resolution 32 2012 Collaborative Consulting

Performance issues and root-cause analysis This leverages several knowledge areas A solution must be found quickly (under pressure) Performance issues are found in: • Preproduction environment • Production and operations environments Key business transactions are not meeting their service level objectives: • End user experience is degrading (acute or chronic) • Overnight processing is taking longer • Stability and performance issues 33 2012 Collaborative Consulting

When problems occur Flash web site events (all at once) New code released causes problems Infrastructure upgrade Consolidation of applications and servers New workload model (batch and online) 34 2012 Collaborative Consulting

Underlying competencies Analytical Project Communication thinking management skills problem solver Business Monitoring and Statistics Knowledge tuning Workload Behavioral Application and model characteristics tool knowledge Underlying development competencies Test data Test execution Forecasting management process 35 2012 Collaborative Consulting

Technical competencies Code Deep dive Resource Database profiling tools utilization monitoring tools (HP Diagnostics, tools CA Wily, etc, system (Static analysis, Jprobe, etc) diagnostics) level End to end Comprehensive Statistics and performance results collection Distributions tools and reporting Technical competencies Test design Queuing and and Test data Little’s law execution modeling creation tools tools 36 2012 Collaborative Consulting

PE Techniques Brainstorming Data modeling Creating a workload model Performance testing Triage approach Static code analysis Logging and System metrics collection instrumentation 37 2012 Collaborative Consulting

PE roles: The team – many possible roles Managing and developing the core team members 1 Technical Project manager 2 Performance Architect 3 Sr. Performance Engineer 4 Performance Engineer 5 Sr. Test automation technician 6 Test automation technician 7 Performance data engineer 8 Sr. Capacity planner 9 Capacity planner 10 Modeler 38 2012 Collaborative Consulting

Career path: Performance engineer foundation Performance Engineer Grow your Performance testing leadership and Lead communication technical Workload role models Technical Production architecture triage Core Additional technical tech competency Communication competency Grow your breadth of capabilities 39 2012 Collaborative Consulting

Case study: Preparing for a flash marketing event Planning a large event to introduce a new product • Send out email to all registered customers (500,000 currently), there is a link in the email. • Run multiple TV ad’s encouraging people to register and then come to the event, results in 100,000 new registrations • No new functionality required to support the event • Potential 600,000 users of the web site The business is forecasting at least a 10X increase during the event, in terms of product sales • How does this translate into system workload and utilization? • Do we need more servers? • The current system was not really designed for such a flash event 40 2012 Collaborative Consulting

PE: Process to prepare for the event (10X) CP CP PVT SA Forecast the workload Tune application based on results APM PDR Measure the current Plan and execute workload and PT for forecast resource utilization workload 41 2012 Collaborative Consulting

CP: Workload model CP The CP team uses this information to forecast the new workload 10X. Step Transaction – by product Percentage 1 Login 100 2 Search product 100 3 View product detail 100 4 Add to shopping cart 50 5 Confirm purchase method 50 6 Shipping information 50 7 Send confirmation email 50 42 2012 Collaborative Consulting

CP: Analyze current capacity CP Review current Production workload, and the under lying system utilization. • Average workload • Current peak workload Response time Service level 100% System utilization 43 2012 Collaborative Consulting

PVT: planning, executing, and analysis PVT • Use the workload model from CP • Enable APM during testing • Work with the Application Architect Methodology is iterative and tactical Define business Review infrastructure activity profiles & Design & build tests & architecture service levels • Identify risk areas • Types & numbers of • Test data generation • Review configuration users • Create test scripts settings, topology, & sizing • Business activities & • User & transaction profiles • Define points of frequencies • Infrastructure configuration measurement Iterate testing & tuning 44 2012 Collaborative Consulting

Questions? Thank you! Get your free copy SharePath RUM today! More information: www.collaborative.com info@correlsense.com www.correlsense.com/demo www.real-user-monitoring.com 45 2012 Collaborative Consulting

http://www.correlsense.com	156
http://stage.correlsense.com	155
http://www.linkedin.com	59
http://applicationperformanceengineeringhub.com	48

An Introduction to Software Performance Engineering

by Correlsense on Jan 27, 2012

Accessibility

Categories

Tags

Upload Details

Usage Rights

4 Embeds 418

Statistics

An Introduction to Software Performance Engineering — Presentation Transcript