Evaluating SaaS APM - A quick whitepaper from New Relic
Evaluating On-Demand APMA Cost-Benefit AnalysisFebruary, 2011Executive SummaryAs web applications have grown increasingly vital to building a healthy business andachieving competitive advantages, application performance management (APM) hasbecome a well-established discipline. Yet several factors are driving the need for a newapproach to APM. Major innovations in the IT landscape, mounting pressure for morerapid, iterative development processes, and significant economic factors have allcontributed to making traditional methods of APM increasingly obsolete. Next-generation APM solutions are needed to ensure the health and availability of webapplications now and in the future. The on-demand, SaaS delivery model meets therequirements of today’s businesses for reliable, high-performance applicationsdeployed in modern environments, while also providing significant economicadvantages.
New Challenges Require New SolutionsThe traditional means of monitoring online application performance are no longer viableas a result of technological advancements and market trends that have set a newstandard for infrastructures and development environments.Cloud computing is leading the push for new management tools. Cloud-basedarchitectures allow businesses to build and deploy applications faster and easier thanever before, but on-premise APM solutions were not designed to handle such a constantstream of dynamically occurring application instances. Additionally, the ongoingtransition toward Agile development practices has also made traditional APM softwareuntenable. Built for long development cycles and phased feature enhancements, it lacksthe implementation capabilities to get up and running on new applications quickly orrespond to frequent iteration.The current polyglot nature of many application environments presents new challengesas well. It’s now common for organizations to employ multiple web applications writtenin several different languages (Ruby, PHP, Java, .NET) — something earlier-generationAPM solutions can’t support — certainly not in the same console, using the same coreproduct and technology.Important economic factors are also driving the need to cut costs and optimizeresources with scaled-out infrastructures comprising hundreds or thousands ofcommodity servers, and management tools that involve far less maintenance andsupport. So when applications can be deployed rapidly on premise, in the cloud, or inhybrid environments, how do you choose an effective solution that bridges the gapbetween established, traditional APM and these new business requirements?SaaS Brings Business Up to SpeedWhile the first generation of APM solutions was adequate for simpler architectures andlargely homogenous environments, the demands of today’s more sophisticated modelshave sparked the evolution of advanced APM solutions. Specifically, SaaS-basedtechnologies that install in minutes with no configuration, support compositeapplications with robust functionality, and are equally effective in dedicated datacenters,
public and/or private clouds, and hybrid environments. These on-demand solutionsprovide all the benefits of traditional on-premise tools without many of the drawbacks.Fig. 1 – Comparing capabilities of traditional on-premise vs. advanced on-demand APM solutions. On-Premise APM On-Demand APM End-to-end visibility of transactions ! ! Proactive 24x7 application monitoring ! ! Ability to pinpoint and diagnose ! ! performance issues Optimize resource allocation and ! ! accelerate transactions Improve service delivery and user ! ! experience Monitor application availability to avoid ! ! downtime Supports applications in hybrid ! environments, including public clouds Immediate implementation with no ! configuration required Simple service-based delivery model ! Single interface for managing composite ! applications Instant, automatic management of ! application updatesFigure 1 illustrates the disadvantages of traditional APM software when applied tocurrent environments. The last item is especially troubling. With pressure to speed therelease of new features, particularly for burgeoning social media and SaaS-basedapplications, IT organizations must meet this faster pace with monthly, weekly, andsometimes even daily updates. New frameworks, new libraries, and new features areconstantly being utilized or added. On-premise solutions simply can’t keep up. Still,functional limitations are only part of the whole story.
Without a doubt, one of the biggest flaws in the traditional APM model is its prohibitivecost. It prevents many companies from implementing an APM solution in the first place,or restricts their use to only a small subset of their applications because very few areable to justify the high cost. On-demand APM, on the other hand, eliminates thisobstacle.Calculating Total Cost of OwnershipThus, a principle benefit of SaaS-based on-demand APM is enterprise-level functionalitywithout enterprise pricing. A point-by-point cost comparison of the two modelsillustrates just how great the budgetary implications can be.Cost of AcquisitionTraditional on-premise APM software is sold by direct sales reps who receive largesalaries, lucrative bonuses and perks (think Hawaii). These costs are, of course, passedon to customers. After manually counting users, agents, and applications for licensecompliance, the software must then be configured for the environment – which requiresseveral weeks of professional services at up to $400/hour. An on-demand solutionallows unlimited users, knows automatically how many applications to monitor, and itsWeb-based delivery makes acquisition quick and easy without ever having to talk to asales person.Maintenance and Support CostsBecause on-demand APM is delivered using the SaaS model, all back-end componentsare hosted on the provider’s side. Upgrades are included in the base cost, and noinfrastructure or employee resources are needed to maintain the solution. That’s not thecase with traditional APM software, which typically requires an 18-20% annual fee forupgrading to the newest version – not to mention additional hardware and manpower.Extra servers are needed to run the business logic and collect the metrics, and one ormore full-time employees may have to be hired to administer the solution.Time to Value = Opportunity CostOn-premise APM software can take months to implement for each application, plusmore time to reconfigure for updates. In Agile operations, frequent releases can getexpensive and produce an opportunity cost. Each performance issue or failed
transaction during this lengthy process wastes resources and sacrifices revenue. Then,every 12-18 months, upgrades to the APM software itself require taking down andrestarting each application. In a huge datacenter with 100 applications or more, that’s anightmare. On-demand APM installs in minutes with no configuration, even forsubsequent updates. New features arrive automatically every week, while applicationsstay running. The result: immediate time to value, zero opportunity cost.Education CostsThe complexity of on-premise APM extends beyond its implementation. Poor usability inthe solutions’ designs can lead employees to avoid them altogether. So after all theexpense of acquiring the software, it oftentimes remains largely unused. That is, unlessthe company pays even more money for the vendor’s education services. Training isunnecessary with an on-demand model, where ease-of-use and an intuitive interfaceare key benefits common to SaaS offerings.Infrastructure CostsDesigned for virtualized and distributed environments, hosted by the provider anddelivered over the Web, SaaS-deployed APM solutions incur no extra costs related toscaling up the IT infrastructure. Conversely, traditional software options are hosted onthe enterprise network – setup generally requires the purchase of additional servers andother equipment.Fig. 2 – Total cost of ownership breakdown of traditional on-premise vs. advanced on-demand APM solutions. On-Premise APM On-Demand APM Annual cost (base) $7,500/CPU license fee + $175/month 10% ($750)/CPU initial setup Maintenance and support costs 18% ($1,350)/year $0 Admin/Feature Upgrades $150,000/year for full-time $0 admin Infrastructure costs $10,000 (8 core CPU box) $0
Fig. 3 – Total cost of ownership breakdown of traditional on-premise vs. advanced on-demand APM solutionscalculated for 20 hosts. On-Premise APM On-Demand APM Annual cost (base) $165,000 $42,000 Maintenance and support costs $29,700 $0 Admin/Feature Upgrades $150,000 $0 Infrastructure costs $10,000 $0 Total $354,700 $42,000Case Study: The ROI of On-Demand APMIn a head-to-head comparison, it’s clear which APM model provides the most cost-effective solution. The expense of SaaS-deployed solutions is a tiny fraction of the TCOfor on-premise alternatives. But how do the benefits of on-demand APM play out in thereal world? Here’s just one example from a THINKstrategies report on its Best of SaaSShowplace Awards, which highlights the method’s measurable benefits. A retail client ofthe industry’s leading on-demand APM provider achieved the following results:Within one month • Average site response time reduced by 66% • 300% increase in application speed • Visitor traffic went up 53%Within ten months • 500% boost in application speed • Conversion rate jumped 18%Results like these are not unusual. And when you evaluate the scope of cost savingspreviously discussed alongside the kind of benefits that truly move a business forward,a significant ROI can be expected, especially over the long run. For the company in theexample above, the ROI of on-demand APM in the first six months alone was enough topay for the solution for more than 18 years (with the original infrastructure).ConclusionWith the IT landscape evolving and organizations increasing their focus on optimizingthe user experience, employing effective APM tools is paramount to reliable world-class
business applications. The trends toward platform diversification, cloud-basedarchitectures, and Agile development are on the rise. The tools for managing business-critical applications in such environments must keep pace. Yet the growing need to cutcosts and accelerate the speed of development requires solutions that are alsoeconomical and deliver rapid returns — without sacrificing important capabilities. Onlyon-demand APM solutions delivered via a SaaS model can offer this advantage ofenterprise-level functionality without enterprise software prices. Thus, their value inparticular will continue climbing in league with these technology and market forces asSaaS-deployed APM becomes the standard for businesses of all sizes.