Enterprise VoIP


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Enterprise VoIP

  1. 1. WHITEPAPER Enterprise VoIP Moving from Despair to Repair to Prepare Using Processes, Tools, and Measurement to Realize the Benefits of IP Telephony Applications
  2. 2. Executive Overview VoIP is here to stay, and nearly all enterprises are considering when and how to take advantage of this new technology. There are a variety of business motivations driving VoIP adoption in the enterprise, and there are real costs savings, productivity increases, and image benefits to be gained. These benefits are typically enabled by VoIP infrastructure on a converged network, but achieved through IP Telephony applications such as messaging, conferencing, contact center, and mobility. Realizing these benefits can be a challenge, and organizations may experience frustration, stress and even despair as they work to deploy VoIP in their environment. Simply put, VoIP is much more than “just another application on the network,” and most organizations have never managed an application with high availability and performance requirements like those of VoIP. With the right tools and training, however, organizations may become more skilled and responsive at troubleshooting and resolving issues — repair. The best approach is to consider VoIP deployment as a lifecycle and prepare. Careful planning throughout the entire lifecycle can help enterprises meet the challenges of VoIP deployment before they turn into quality of service issues. The best way to prepare for and combat the quality issues inherent in a converged network is to test the network thoroughly before rollout, and the best way to prepare for VoIP application rollout and troubleshooting is to baseline VoIP applications and then test across a range of conditions to anticipate and resolve problems before they arise in production. When good tools and processes are used, VoIP deployments can go smoothly and provide great business benefit. This paper is part one of a three-part series on enterprise VoIP, which was developed to provide information about the factors affecting both the infrastructure and applications during VoIP deployments. It lays out a lifecycle approach for VoIP deployment and describes the multiple levels of measurement needed to manage VoIP applications. Tools and techniques are offered for baselining, proactive problem identification, and troubleshooting. Testing is not an option, it is essential, so specific examples of how testing addresses common VoIP issues are described to help enterprises confront these issues before they impact quality and erode the business benefits of VoIP. The State of Enterprise VoIP Motivations and Challenges VoIP is here. The proliferation of converged networks has opened the door to a new world of enhanced applications and services, including consumer and carrier services, and a strong surge in enterprise VoIP adoption. VoIP Adoption in Enterprises When and How — Not If For enterprises, VoIP adoption has become a matter of when and how — not if. With all major vendors focused on VoIP solutions, using VoIP is becoming the clear choice for new “greenfield” installations. The year 2005 will be a “tipping point” when the number of new VoIP phones shipped to enterprises exceeds the number of TDM phones. A migration to VoIP is in the future of nearly all enterprises: a recent Empirix survey reported that 52% are already deploying VoIP at some level, 46% have plans for deployment, and only 2% have no plans to deploy VoIP at all. See, Figure 1 below. Figure 1: VoIP deployment plans among enterprises. 2
  3. 3. WHITEPAPER Business Benefits It’s the Applications The benefits that are used to justify VoIP deployments have evolved along with the technology. Three years ago, for example, projects were typically justified on the basis of reduced operating costs, either for toll bypass or the lower operating costs of a single network. Then two years ago, the single biggest justification was virtualization, the ability to work anywhere and tie geographically dispersed and mobile employees into a single system. Both of these justifications remain strong, but increasingly, enterprises have found that the strongest recurring value is in the applications themselves. These include messaging (voicemail and unified messaging), contact centers, auto-attendants, CTI, and conferencing — not new applications but traditional ones that VoIP makes more manageable, more cost-effective, and more uniformly available. The technology also supports new applications: presence-based contact centers, video streaming, dual-mode mobile phones, etc. But business benefits from the traditional applications alone provide ample justification for the migration to VoIP. They provide reduced operating costs, simpler network administration, improved team collaboration and productivity, and better communications with customers, partners and suppliers. So, if VoIP delivers such value, then what’s the catch? Like any major new technology, VoIP brings with it a unique set of challenges as well as opportunities. Challenges VoIP is not just another Application on the Network The concept of VoIP is so simple that organizations often underestimate just how complex the reality of VoIP is. VoIP is still an emerging technology, so there are new standards, moving feature sets, and competing approaches, which make interoperability a major challenge. In addition, voice is a difficult medium for IP: it is very sensitive to the inherent characteristics of IP networks, including latency, jitter, and packet loss. Voice quality, availability, and security are critical, but are difficult to measure and manage. (Quality also depends heavily on the characteristics of the phones, equipment, and incoming signal, since echo, distortion, and noise matter, as do packet-level impairments). Yet expectations are high based on the service levels people have become accustomed to from phone systems: five 9’s availability, toll quality, clean interoperability and applications, and dial tone even when the lights go out. Most companies have never managed an application with such high availability and performance requirements. At the same time, VoIP deployment is nearly always a migration, which means that legacy systems must be managed simultaneously and slowly upgraded while interoperating with new packet-based equipment. And all of this occurs in newly converged IT teams, where neither the voice people nor the data people have worked with anything quite like it before. Experiences of Despair Pitfalls of IP Telephony Since enterprise VoIP deployments first began in earnest around 2001, there have been a number of high profile disasters. Dow Chemical, Carnegie Mellon, Merrill Lynch, and others began wide-scale deployments only to back out or change direction due to quality and acceptance issues. In cases like these, the cost of failed deployment can be millions of dollars — and this doesn’t include the inevitable disruption to business operations. To call the feelings of those in this situation despair may be putting it mildly. Leaking Away the Promised Business Benefits Less visible but more common are projects that run into trouble but persevere in quiet desperation, losing the business benefit along the way. Project managers and vendors usually don’t want to talk about these cases, but they are far too frequent. Here are just three examples: A major consumer product company.A major consumer product company deployed 1,000 endpoints over one and a half years. “A learning experience,” according to the telecom director. “At first it was no different from any other PBX deployment. Then we learned you had to be careful about the network. Pushing new rules to the firewall caused the phones to reboot. We learned we needed QoS from end to end and we had to upgrade our entire network to support QoS and PoE. To top it off, 50 call center agents moved in. Traffic went up; we started having phantom problems that were hard to troubleshoot. Then we had lots of patches, with each patch causing an outage. Our call center is online 24x7; there’s no time to do offline management or apply patches.” After more than a year, the company realized it needed troubleshooting and testing tools. “The system is stable now, but the damage has been done; end users still blame everything on the phones. Now we’re in a PR situation internally,” according to this manager. There are still many miles to go to reach full deployment, and the promised cost savings have evaporated, at least for the next few years. A major retailer. A major retailer deployed VoIP at one of its main outlets as the first part of a substantial multi-site rollout. During the planning phase, the project team underestimated the amount of traffic that would traverse the network. Due to congestion, they had to redesign and upgrade the network mid-stream, incurring substantial unanticipated costs and causing a flood of complaints from both voice and data users over a four-month period. 3
  4. 4. A major mortgage provider. A major mortgage provider has deployed VoIP in ten out of 19 offices, with 3,400 endpoints to date. The Director of Converged Infrastructure has watched a one-year project turn into a three years (and counting) endeavor. “I’m holding off on our call centers, trading desks, and executives. The way this project has gone, I’m afraid to put a VoIP phone on my CEO’s desk!” Although deployment was planned thoroughly in advance, the stability and feature interactions of the IP Telephony server left much to be desired. “We had frequent upgrades and patches, and each one caused disruption to our users because it changed how key features worked. Interoperability was a huge challenge, and key applications like voicemail and call center basically didn’t work. Our voice quality is still troublesome, but we don’t really know how much of a problem it is. And the payback for this project is long gone.” Common Pitfalls and Issues For those who have successfully deployed VoIP and IP Telephony applications, the process is generally a learning experience. The combination of new technology and new organizational structures, as well as operating at several layers simultaneously, makes each deployment project a challenge. Though each project will have specific issues, there are a number of common pitfalls and successful practices to avoid each one: Lack of Organizational Readiness. VoIP is new for data and voice teams in an IT organization; it requires skill and commitment from both groups and can provide a growth opportunity for everyone. To be successful, however, good cooperation and clear lines of responsibility must be in place. One enterprise CIO said, “I walked into the kickoff meeting and announced that we were doing a VoIP rollout and that it would provide significant business benefits. Within ten minutes, it was as if a trench had opened in the middle of the floor, and grenades were being passed back and forth!” One successful large-scale deployment avoided this pitfall by describing voice infrastructure and voice applications and setting up responsibilities for each. Underestimating VoIP as just Another Application on the Network. Although awareness of the special place of voice and IP telephony is now widespread in IT departments, it is still all too common to launch into a deployment under-prepared. Unlike most existing IP applications, voice and video applications are extremely sensitive to delay, jitter, and packet loss. QoS must be implemented in most networks to achieve acceptable call quality. And the expectations for voice availability across the enterprise are far higher than for most other applications. A useful technique is to provide cross-training, case studies, and the necessary tools for the staff involved with a deployment. Expecting Telephony-Grade Support. Many IP telephony vendors have a different business model and philosophy than those that IT departments are used to for voice. (Tried to fix it, but that sentence is still a little clumsy and I had to read it several times to understand what you were saying because the reference to telephony vendors sort of comes from left field — by the way, though in general your writing is excellent!) Telephony services are run on standard servers, making the lines of responsibilities less clear than with refrigerator-scale legacy PBXs. Compounding the issue is the fact that many enterprises use multiple vendors for different parts of their infrastructure. Preparing ahead of time for troubleshooting and system management can help to avoid the finger-pointing and delays in problem resolution that often occur. Depending on a Single-Shot Network Assessment. Though extremely useful, a network audit cannot discover the dynamic behavior of a complex network. As a result, multiphase deployments often have trouble in later phases because conditions have changed since the initial assessment. Testing a network with actual VoIP traffic on a regular basis can avoid ongoing headaches — many of which are caused by mis-configurations that are a side effect of some other project. Lack of a Lifecycle View. One classic pitfall is to neglect troubleshooting until after deployment; another is to neglect the proper testing to extrapolate from a pilot to a full-scale deployment. Viewing deployment as a set of concrete phases helps to ensure clean deployments and management. Focusing Solely on Infrastructure and Ignoring Applications. In a recent survey, the majority of VoIP deployments were motivated by applications such as messaging, contact centers, and telework rather than by a less expensive or more manageable infrastructure. There are three levels at play in every VoIP deployment: packet-level, call-level, and application-level. The most successful deployments pay attention to all three layers throughout the lifecycle. Although the picture painted above — with complex pitfalls and challenges, and interdependent planning and testing required — may seem daunting, it is possible to have a clean deployment and realize significant business benefits. Moreover, every business must eventually go through this migration. While no complete process is available today, the key is to seek the best practices available and build these into the plan and the budget. 4
  5. 5. WHITEPAPER Moving to Repair For those enterprises that have reached the zone of despair in their deployment process, the way out is clear: fix current problems and plan the next phases with more foresight and wisdom. In most cases, this means equipping the team with better knowledge and tools. Multiple Layers of Measurement Measurement is crucial for improvement; it’s an old adage that you can’t manage what you can’t measure. At any stage in a deployment it is important to capture baselines and have the capability to measure at multiple levels. Table 1 shows a view of the layers involved in a VoIP deployment, some important measurements, and some useful tools. Table 1: VoIP Management and Tools LAYER MEASUREMENTS TOOLS • Packet Layer • Packet loss • Packet Sniffers and Testers • Jitter • Element-Level Network Management • Delay • Switch/router Consoles/statistics • Call Layer • Call Completion Rates and Latency • Call and Protocol Analyzers • Voice Quality • Infrastructure Testers • Equipment Status • Equipment/Server Consoles • Application Layer (format) • Application Completion Rates and Latency • Application Testers • Cross-Platform Status • Transaction-Level Management • Provider SLAs • SLA Management By measuring the operations of currently deployed IP telephony applications regularly, IT staff can gain the information needed for troubleshooting, problem resolution, growth, and further deployments. A positive side effect of a measurement program is that staff become exposed and “trained” in how the infrastructure and applications work. Troubleshooting Tools and Techniques Resolving VoIP problems, especially intermittent problems, is very difficult. A skilled troubleshooter needs a variety of tools at his or her disposal. One indispensable tool is an analyzer capable of tracing and troubleshooting VoIP at the call level. In part two of this series, we describe specific scenarios and techniques for tracing and troubleshooting using a specific tool — the Hammer Call Analyzer — as an example. Testing Tools as a Means to Repair Testing tools are another invaluable addition to the VoIP deployer’s arsenal. While we will cover useful testing techniques below, it is important to remember that while ideally testing should occur early in the process, it is never too late to test. Many deployment projects have come back to baselining or network testing mid- project, after starting without it and running into problems. The baseline scenarios that can be captured through automated testing of VoIP infrastructure and applications are invaluable. They create a captured record of normal operation for a given environment, help find remaining issues before users do, and are one of the best ways to train IT staff in VoIP operations and troubleshooting. 5
  6. 6. Prepare and Avoid Despair Being Ready for Deployment Of course, avoiding issues during VoIP deployment is the goal, and the vast majority of issues are avoidable with proper planning, training, processes, and tools. But, no amount of preparation will provide a rollout that is 100% trouble-free, so being ready to repair is important as well. By viewing deployment as a lifecycle with distinct phases and by being ready for each phase ahead of time, deployments can be accomplished smoothly, achieving benefits more fully while saving significant amounts of time and money. The VoIP Deployment Lifecycle Migration in Phases Nearly every VoIP deployment is a migration from the existing TDM infrastructure and can be done in sub-projects. In turn, each sub-project goes through distinct phases, as shown in Figure 2 below. Ideally, project staff should plan, design, implement, operate, and optimize each sub-project, and use the lessons learned for the next sub-project. Even when the situation does not allow a careful, measured rollout, it is very useful to think of a project in terms of these phases. By keeping the phases in mind AND remembering to consider the applications, not just the infrastructure, successful deployment can be better assured. Planning for Successful Deployment Enterprise VoIP deployments, like any other complex IT project, involve careful planning and project management. Figure 3 below shows a typical timeline for a single-site IPT deployment. Multi-site deployments often have similar timelines but rely on multiple cutover dates because sites are completed independently. 6
  7. 7. WHITEPAPER In any scenario, planning for quality assurance measures throughout the project is extremely beneficial. This includes baselining the current network during needs assessment, testing the network infrastructure multiple times as the network is prepared for VoIP, testing vendor solutions during the procurement process, testing applications and infrastructure during installation and through cutover, and ongoing testing, troubleshooting, and management after the cutover. It is very important to prepare IT staff ahead of time by making the organizational responsibilities clear. Voice applications are poorly understood by most data network specialists; the impact of infrastructure changes on these applications is typically unpredictable. One large financial institution created separate teams for VoIP Infrastructure and VoIP Applications and reinforced the fact that VoIP deployment was a growth and learning opportunity for everyone involved. This covered 90% of those involved and greatly improved teamwork among groups that had been uncomfortable working together previously. Design the Infrastructure Getting Networks Ready for VoIP Voice over IP is very sensitive to IP network problems; so it is highly desirable to create a clean network environment prior to deployment. This can be achieved by well understood network design practices. However, it is important to remember that network problems, which may not cause issues for data applications, may be very problematic for VoIP. Implement Network Infrastructure Testing Pre-deployment testing is the systematic testing of the ability of the network to support Voice-over-IP traffic. Because this generally involves sampling network performance, it is not a substitute for planning. It can, however, help provide a level of confidence and identify potential problems. VoIP-specific testing is a critical, often overlooked, practice to check a variety of key factors, including: Ability and expertise to configure the network properly Interoperability issues Capacity Management tools Part three of this series discusses testing techniques in more depth, and describes specific scenarios and techniques for benchmarking, assessment, and testing using a specific tool — the Hammer VoIP Test Solution for Enterprises — as an example. Implement: System and Application Testing Voice applications — such as conferencing, messaging and voice mail, call routing and CTI, contact centers, voice self service/IVR, voice transaction systems, trading turrets, teleworking, click-to-talk, etc. — are what typically provides value from IP Telephony. These applications benefit from another layer of testing, over and above the infrastructure testing described above. Applications tend to have a large number of potential paths and many configuration parameters. IP telephony applications are no exception, and they have the additional twist that they interact with a new infrastructure that can change the effectiveness of the application — sometimes dramatically, sometimes subtly. Once the VoIP infrastructure is tested and good baseline measurements are captured, application-level testing can be done very effectively. Typical areas to test are: Signaling latency (speed of dial tone, speed of call transfer, etc.), Reliability of application information delivery (screen pops, information elements use for routing, etc.), Application performance (IVR responsiveness, Application performance under load, etc.) Impact of VoIP on applications (speech recognition accuracy with packet loss, conference bridge loudest-speaker detection, etc.) All-paths testing (correct configuration of all forwarding, hunting, routing, voicemail and messaging configuration, etc.) 7
  8. 8. WHITEPAPER Manage and Optimize: Prepare for Ongoing Improvements Day 2 of VoIP deployment does not bring an end to problems. The experience of many enterprises is that the issues have just begun, as the new system has now been exposed to the user base. In addition to continuing with the next phase of deployment, ongoing management of the network and customer service for the users must be completed for this phase of the deployment. Of course, proper preparation and testing ahead of time can help organizations avoid the majority of problems, and troubleshooting tools and techniques can speed resolution and minimize the impact of the problems that do occur. Ongoing management, measurement, and optimization are important as well, however. Enterprises that plan for this phase from the start have all their bases covered and will have the best experiences with VoIP deployments. There are many moving parts to manage in enterprise VoIP deployments. Most VoIP deployments are multi-site and multi-vendor. Interoperability remains somewhat problematic even after cutover, and subtle interactions between infrastructure and applications often surface with changes in traffic or configuration. Many VoIP-related problems are transient in nature and can occur at many places in the infrastructure. Network architects and managers should address call quality and performance management problems as they plan and deploy their IP networks, but they should be aware that these problems also occur during normal day-to-day network operation. Optimization means more than monitoring or management — it means periodic benchmarking, measurement, and analysis. Many of the same techniques and tools used prior to deployment can apply here: voice quality measurement, VoIP infrastructure and application testing, and needs assessment. In the exhilaration and/or exhaustion following cutover, teams tend to overlook re-benchmarking or optimization. While just keeping it running can be a challenge in itself, there is often a substantial business benefit to be gained from periodic optimization. By preparing ahead of time, ongoing management can be less consuming and the benefits of optimization can be realized much more easily. Conclusion Moving from despair to repair to prepare is not a straight-line path. All enterprises are unique, and every VoIP deployment project has many phases, each with its own degree of despair, repair and prepare. But the same principles, tools, and processes can be used to reduce headaches and improve business benefits regardless of where an organization is in this continuum. Prepare. The ideal situation is to prepare ahead of time. This includes organization, training, tools, and planning. Benchmarking very early, with network testing and current application measurement, is very powerful in guiding plans, and lays the groundwork for optimization after cutover. It can also be a great training experience. Planning ahead for all phases of deployment may seem overwhelming, but it pays off. Testing is an essential process at all stages of deployment; it is not an option. Repair. Organizations will inevitably find themselves needing to repair. (Those who are well prepared have much less repair work and will be more effective when they do have it because the tools, training, and processes are in place.) Troubleshooting is difficult with VoIP, and VoIP-specific tools are indispensable. Testing plays a major role here as well, both for isolating and recreating problems, determining how critical they are through regular benchmarking, and ultimately resolving them. Despair. Too many organizations find themselves in despair during their VoIP deployment projects. In some cases, this despair is dramatic, but quiet desperation is even more widespread. For organizations in despair, there is hope. First, get good at repair, and stop the bleeding that happens daily in these situations. Second, go back and prepare. This will make ongoing operations manageable and make the next deployment phase a positive experience. The business benefits of VoIP are real, and the adoption of this technology is rapid and widespread. Realizing these benefits can be a challenge, however, and organizations may find themselves in despair too frequently and too easily. By being aware of common pitfalls, viewing VoIP deployment as a lifecycle, and using the best tools and practices, deployments can go smoothly and deliver significant business benefits. ©2005 Empirix. All rights reserved. All descriptions, specifications and prices are intended for general information only and are subject to change without notice. Some mentioned features are optional. All names, products, services, trademarks or registered trademarks are used for identification purposes only and are the property of their WP-1-EVOIP04.05 respective organizations. Corporate HQ | 20 Crosby Drive | Bedford, MA 01730 | USA | tel: +1 781.266.3200 | fax: +1 781.266.3201 | www.empirix.com European HQ | Building A, Trinity Court, Wokingham Road | Bracknell, Berkshire RG42 1PL | United Kingdom | tel: +44 (0)1344 66 8080 | fax: +44 (0)1344 66 8172