Realizing Continuous Performance Management

916 views
875 views

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
916
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
19
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Realizing Continuous Performance Management

  1. 1. Your logo here Realizing Continuous Performance Management Steven Haines Quest Software Java Domain Expert February 14, 2008
  2. 2. Agenda  State of the Market  Cost of Mistakes  Test-Driven Development  Continuous Integration  Continuous Performance Management  CPM in Practice  Business Value  Next Steps…  Q&A
  3. 3. Industry Experts Say… Forrester Research reports nearly 85 percent of companies with revenue of more than $1 billion reported incidents of significant application performance problems. Survey respondents identified the architecture and deployment as the primary causes of these problems. -Forrester Research Medium-sized businesses* are losing an average of 1% of their annual revenue, or $867K, to downtime. Application outages and degradations are the biggest sources of downtime, costing these companies $213K annually. -Infonetics Research *101 to 1000 employees
  4. 4. Impact of Poorly Performing Applications  Business-to-consumer − Direct loss of customer revenue − Loss of confidence  Business-to-business − Loss of reputation − Potential loss of business relationships  Intranet − Loss of productivity − Longer time-to-market = loss of competitive advantage
  5. 5. Cost of Mistakes Performance problems found later in production cost almost 10x more to fix… Test Early, Test Often
  6. 6. Why is the Current State of Testing Failing?  Test suites are written to out-of-date artifacts  Test suites are not comprehensive enough  Test suites are not automated − Not performed regularly − No regression testing
  7. 7. Test-Driven Development  Test cases must be written prior to writing code  Test cases written by developers  Each code component must include a test suite
  8. 8. Test-Driven Development Test-Driven Development Continuous Integration Code Test Code Test Test Suite Source code Code Test Repository Continuous Performance Management Picture to show profiling of a test: JProbe or PAS type of screen Profile Analyze React
  9. 9. Test-Driven Development Lifecycle Add a Test Test Suite
  10. 10. Test-Driven Development Lifecycle Add a Test Prove Failure Test Suite
  11. 11. Test-Driven Development Lifecycle Add a Test Prove Failure Test Suite Implement New Functionality
  12. 12. Test-Driven Development Lifecycle Add a Test Prove Failure Test Suite Implement New Functionality Prove Success
  13. 13. Test-Driven Development Lifecycle Business Value:  Shorter Development Add a Test Prove Cycles Failure  Limited Debugging  Faster Triaging  Clearly Solved Business Test Suite Problems Re-factor Implement New the Code  Confidence in Functionality Deployment Prove Success
  14. 14. Continuous Integration  The Challenge − With complicated applications, integration can take longer than development  The Solution − Integrate after each code check-in  The Tool − Continuous Integration Server
  15. 15. Test-Driven Development + Continuous Integration Test-Driven Development Continuous Integration Code Test Build Code Test Test Suite CI Test Server Source code Code Test Repository Publish Continuous Performance Management Picture to show profiling of a test: JProbe or PAS type of screen Profile Analyze React
  16. 16. Continuous Integration Lifecycle Monitors SourceCode Repository Continuous Integration Server
  17. 17. Continuous Integration Lifecycle Monitors SourceCode Detects New Repository Check-Ins Continuous Integration Server
  18. 18. Continuous Integration Lifecycle Monitors SourceCode Detects New Repository Check-Ins Continuous Integration Server Compiles Entire Application
  19. 19. Continuous Integration Lifecycle Monitors SourceCode Detects New Repository Check-Ins Continuous Integration Server Compiles Entire Application Runs Exhaustive Test Suite • Commit Build • Secondary Builds
  20. 20. Continuous Integration Lifecycle Business Value: Monitors  Shorter integration SourceCode Detects New Repository Check-Ins times  Integration issues Continuous resolved as they Integration occur Server Publishes Compiles Entire Build Results Application  Higher quality end product Runs Exhaustive Test Suite • Commit Build • Secondary Builds
  21. 21. What is Continuous Performance Management?  Challenge in Performance Testing − Time consuming and error prone because manual − Problems discovered at the end of development effort often affect assumptions made at the beginning of a development effort  Requires re-factoring the architecture  Solution − Performance Management in a Continuous Integration Environment  Proactive approach  Execute performance tests on every developer commit − On every code check-in
  22. 22. Continuous Performance Management (CPM) • Create additional Secondary Builds that Secondary implement performance Build 1 tests 1. Performance Unit Tests 2. Performance CPM Integration Tests • Business case Secondary Secondary • Use case scenario Build 2 Build 3 3. Performance Stress/Load Tests
  23. 23. Test-Driven Development + Continuous Integration + CPM Test-Driven Development Continuous Integration Code Test Build Code Test Test Suite CI Test Server Source code Code Test Repository Publish Continuous Performance Management Profile Analyze Repair
  24. 24. Performance Unit Tests • Memory – Lingering Object References (memory leaks) – Object cycling • Performance – Identify slow running code • Coverage – Quantify the percentage of classes, methods, lines of code and conditions exercised by unit tests
  25. 25. Performance Integration Tests  Performance analysis of a single user − Identify request response times that exceed service-level agreements − Triage to offending methods and SQL statements − Follow requests across tiers (and across JVMs)  Identify application problems prior to load − Find problems in isolation − Easier to pinpoint root cause
  26. 26. Performance Stress/Load Tests  Performance analysis under load (multiple users) − Identify request response times that exceed service-level agreements − Triage to offending methods and SQL statements − Follow requests across tiers (and across JVMs) − Validate environmental configuration − Identify resource contentions  Identify scalability problems prior to deployment − Identify environmental configuration issues − Resolve scalability and reliability problems
  27. 27. Continuous Performance Management Toolset  Continuous Integration Server – CruiseControl  Build Environment – Apache Ant  Test Bed – JUnit  Load Generator – Apache JMeter  Performance Testing: − Unit tests: Quest JProbe − Integration/Load tests: Quest PerformaSure
  28. 28. CPM – Too Arduous a Task?  It is a common misconception that CPM is too risky and time consuming of an undertaking  We’ve gone through painstaking efforts to make CPM as easy as possible to implement  Only requires a handful of build script additions
  29. 29. CPM In Practice – Performance Unit Tests  Additions to Ant build script to profile all your JUnit tests  Easy to do – can paste on a single slide <!-- Set location of the JProbe Enterprise Tools --> <property name="jprobe.enterprisetools.home“ location="c:jprobe-ent" /> <!-- Define the Pre-processor CLASSPATH --> <path id="classpath.preprocessor"> <fileset dir="${jprobe.enterprisetools.home}" includes="lib/*.jar"/> </path> <!-- Execute the preprocessor --> <java classname="com.javasrc.anttools.JProbePreprocessor"> <sysproperty key="src.dirs" value="${src};${src.test}" /> <sysproperty key="jprobe.home" value="C:Program FilesJProbe 7.0" /> <sysproperty key="jprobe.build.dest" value="${jprobe.enterprisetools.home}jprobe.xml" /> <classpath refid="classpath.preprocessor"/> </java> <!-- Import the Generated Script --> <import file="${jprobe.enterprisetools.home}jprobe.xml" />
  30. 30. CPM In Practice – Unit Test Report
  31. 31. CPM In Practice – Unit Test Report
  32. 32. Performance Unit Testing – Quest JProbe  Three analysis enablers: − Memory − Performance − Coverage
  33. 33. CPM In Practice – Integration / Load Tests  Additions to Ant build script to launch JMeter  Easy to do – can paste on a single slide <!-- Import the JMeter task --> <taskdef name="jmeter“ classname="org.programmerplanet.ant.taskdefs.jmeter.JMeterTask"/> <!-- Execute the test script --> <jmeter jmeterhome="C:libjakarta-jmeter-2.2" testplan="${basedir}/mytests/MyTest.jmx" resultlog="${basedir}/mytests/MyTestResults.jtl"> <property name="jmeter.save.saveservice.output_format" value="xml" /> </jmeter> <!-- Generate an HTML Report --> <xslt in="${basedir}/mytests/MyTestResults.jtl" out="${basedir}/mytests/MyTestResults.html" style="${basedir}/mytests/jmeter-results-report.xsl" />
  34. 34. CPM In Practice – Integration / Load Tests  Additions to Ant build script to record a performance session <property name="pas.home" location="j:PerformaSure5.0" /> <target name="pas" description= "Start a PerformaSure recording" > <!-- Execute nexusctl --> <exec executable="cmd"> <arg value="/c" /> <arg value="${pas.home}/scripts/nexusctl.cmd" /> <arg value="start-recording" /> <arg value="-user" /> <arg value="user" /> <arg value="-l" /> <arg value="30m" /> <arg value="-t" /> <arg value="10s" /> <arg value="-f" /> </exec> </target>
  35. 35. Performance Integration/Load Testing – Quest PerformaSure Quest Management Server Tag-and-Follow™
  36. 36. Performance Integration/Load Testing – Quest PerformaSure
  37. 37. Business Value  Benefits − Reduce R&D and QA time in performance testing − Better performing applications  Results − Faster time-to-market = more time to add features that give you a competitive advantage over your competition − Higher quality applications = higher customer satisfaction
  38. 38. Next Steps…  Download one of my whitepapers from www.quest.com under Application Management, solutions for Java: − Extending the Continuous Integration Environment with Performance Testing (Part 1) − Achieving Continuous Performance Management (Part II) − Best Practices in Java Environment Performance Testing − A Formal Performance Tuning Methodology: Wait-Based Tuning  Sign up for my upcoming “Java Performance Winner’s Circle” webcast series on performance testing, tuning, and management at www.quest.com under Application Management, solutions for Java (starts in March 2008)  Watch my Continuous Integration OnSoftware Video Podcast on www.informit.com (or iTunes)  Read my forthcoming digital shortcut book, Agile Java Development with Test-Driven Development and Continuous Integration, on www.informit.com  Read my latest book, Pro Java EE 5 Performance Management and Optimization, at your favorite book store
  39. 39. Next Steps cont’d…  Quest Software Product Information: JProbe – www.quest.com/jprobe PerformaSure – www.quest.com/performasure  Other Resources: JProbe Live Product Demo – www.quest.com/events PerformaSure Product Demand – www.quest.com/performasure  Contact Quest Software Sales: www.quest.com/company/contact-us.aspx
  40. 40. Agenda  State of the Market  Test-Driven Development  Continuous Integration  Continuous Performance Management  CPM in Practice  Business Value  Next Steps…  Thank You -- Q&A

×