From Pilot to Production: Managing Your VoIP Service ...
From Pilot to
Your VoIP Service
Unifying Communications for
Business Advantage............... 3 White Paper
Why the Pilot Was Vital:
Lessons Learned .................... 4 July 2007
Evaluating the Pilot for
Success .................................... 4
Moving Forward: Putting
VoIP into Production ............ 6
This paper describes the VoIP network management life cycle,
Best Practices for Rolling Out discusses what you need to know to put VoIP into production,
VOIP to a Multi-Branch and presents best practices for rolling out VoIP.
Company ................................ 8
Do-It-Yourself vs. Using a
Solution Suite ......................... 9
Summary .............................. 11
For many enterprises, there comes a point when the move toward the next generation of convergence
and integration stalls. You made the decision to do a VoIP pilot program. You invested the effort in
doing all the right prep work. Now you’re ready to move from the pilot phase to production—or are
In far too many cases, businesses never quite make the leap out of the pilot phase into production
services with IP Telephony (IPT). There are a number of legitimate reasons for a project to stall.
Some enterprises fear that quality of voice communications might not be what they had hoped for,
leading them to re-evaluate if, when, and how they will deploy VoIP into their business environment.
A shift into analysis paralysis, or deferring action while “we think about it further,” can delay an
implementation for months. But why let the VoIP implementation project stall after the pilot or
concept testing is completed?
For many organizations, two facets of the VoIP implementation get downplayed or overlooked
entirely: management and monitoring. Voice communication is a mission-critical service in the
enterprise. It’s as crucial as the data services. You know from the network readiness assessment and
pilot testing that your IP networking infrastructure can support VoIP services, but are you ready to
embrace the necessary management to support IP-based voice services?
Several key metrics impact the smoothness of the transition from traditional telephone services to
• Call quality—Call quality embraces far more than voice quality. It’s easy to get caught up in
technical concerns over bandwidth, delay, and jitter, but call quality goes beyond that. If you
think of Quality of Service (QoS) as the technical side of VoIP services, consider the Quality of
Experience (QoE), or the user experience, as the human side. On the QoE side of the equation,
you monitor factors such as how quickly dial tone is delivered, call setup and teardown times,
ease of use, and of course, the quality of the voice conversation.
• Cutover downtime impact—The cutover itself will have some impact. Methodical planning and
use of a phased approach can minimize the disruption to daily business operations.
• Internal communications—How you share information about the transition will have a huge
impact on how smoothly it goes. If users know what is coming and understand the phases of the
rollout, they are generally more receptive and supportive than if they get an email that simply
says “On Friday we’ll be converting to a new phone system.” Training and open discussion of the
change help eliminate general complaints about the phone system being down. And the system
doesn’t need to leave users out of service. Good planning can ensure a smooth, painless
• Security when connecting to external resources—Because the proof-of-concept or pilot testing is
often done inside the enterprise, connections to outside networks, such as the Public Switched
Telephone Network (PSTN) or Session Initiation Protocol (SIP) peering with an extranet business
partner, may be overlooked in the early phases of planning. If they weren’t considered in the pilot
phase, they have to be address before moving into production.
• Service monitoring—For many businesses, the readiness assessment and pilot testing overlook
the need to monitor. Voice services are critical to the life of an enterprise. Including monitoring
plans is a vital component of the VoIP migration project plan from pilot to cutover into live
This paper reviews pain points and best practices for managing the transition of VoIP deployments
from the pilot or proof-of-concept phase into production and culminates in how NetIQ’s VoIP
solution supports the complete VoIP service life cycle.
Figure 1: The basic service management life cycle.
This paper takes a general view of the network management life cycle as Figure 1 shows. The life
cycle encompasses four basic phases:
• Assessment—In this phase, you identify business requirements and conduct the readiness
assessment to establish baseline capabilities of the network and determine whether the network
can support the integration of VoIP, and perhaps video, with the IP data infrastructure.
• Pre-Deployment—At this stage, you create service maps of all network services and develop
service assurance levels and service level agreements (SLAs). It is here that the monitoring
policies and management methods are defined as well.
• Ongoing Monitoring—In daily operations, you monitor service, prioritize changes, and respond
to any service impairments. For ease of operations, automate as much of this as possible to speed
problem diagnosis and resolution.
• Reporting and Planning—Ongoing, continual monitoring and measurement in operations
produces reports on service levels for analysis, as well as information about outages and
problems. This business intelligence provides input for service improvement plans to sustain the
mission-critical service. These plans then feed directly into the assessment phase, allowing you to
understand any impact they will have when implemented in production.
By taking a methodical approach to every phase of the life cycle, you help ensure comprehensive
management and monitoring of the VoIP service network as well as ease the deployment and multi-
branch rollout of any growth or new VoIP environments.
2 White Paper
Unifying Communications for Business Advantage
Convergence is a hot buzzword that has been cycling around voice and data networking technologies
for 10 years or more. Unified communications is just another slant on convergence or integration. It’s
important for a variety of reasons.
One of the foremost drivers of integrating voice and data is cost consolidation. The expense of
maintaining dual networks, carrying voice traffic on one and data on another, is simply unnecessary.
These two networks operate on the same cable plant or wiring scheme used through an office
location. Internally, within the enterprise, it should only require half the physical wiring.
The cost consolidation of outward networking is another factor. A connection from the PBX to the
PSTN costs a significant amount of money. So does a connection from the data network to the
Internet. It’s reasonable to expect that consolidating those two, onto a single circuit to a unified
communications provider, should drive down cost. And it should drive down billing complexity as
Those are two aspects of infrastructure convergence, but there are others. For large enterprise
businesses, the issue of managing separate voice and data networks also encompasses soft costs. For
many organizations, two separate technical groups provide service management, monitoring, and
support. Unifying onto a single IP-based architecture also allows for the consolidation of staff in
terms of cost and technical resources.
A Cost Consolidation Warning
• An overly aggressive manager might approach the integration of voice and data services with
the wild-eyed idea that there is a linear reduction in staff when voice and data converge.
Practical experience has shown that this often isn’t the case. Because the IT staff supporting the
data network will be learning VoIP as something new, and the telephony support staff will be
learning IP-networking technologies, it’s wise to not get too aggressive in considering staff
• Most organizations experience a gain in resources, but from a different angle. As the converged
service delivery team evolves, the skill sets broaden and overlap. This can allow for some
thoughtful reduction, but more often provides knowledgeable resources that can begin to
engage in new projects.
Unified communications has broadened the concept of convergence well beyond simply integrating
voice and data onto a single infrastructure. Today, companies are exploring Software Oriented
Architectures (SOAs). From an SOA view, data and voice evolve to become services on the network
that can be called from any network resource.
In the past, telephones and PCs called voice and data services. SOA moves beyond these resources to
the applications running within the network. The integration of services moving voice to VoIP on the
IP network sets the stage for the next evolutionary cycle of integration and convergence.
SOA presents a vision of voice services coupled with enterprise applications. The Customer
Relationship Management (CRM) system might be tightly coupled with outbound voice calling
services. The Enterprise Resource Planning (ERP) system might integrate a supply chain management
module with data integration to extranet vendor partners. Even the human resource systems might
provide simpler mechanisms for requesting and deploying new phone numbers, LAN connections,
and workstations for newly hired employees.
The SOA approach is young and evolving, but it’s being embraced by the entire vendor community.
Many application vendors are now using their ability to integrate software with voice and data
services as a differentiator. If you accept that convergence as a concept has evolved to become what
is now called Unified Communications, the next cycle of evolution is likely to be the shift to SOA.
Why the Pilot Was Vital: Lessons Learned
The proof-of-concept or pilot testing provides valuable insight into the network’s capabilities. The
process of readiness assessment and pilot testing provides crucial business intelligence about the state
of your network and its ability to deliver VoIP services.
Through these efforts, you come to better understand your business call flows and volumes. The
patterns for inbound and outbound calls may be very different, but you’ve identified what they are.
You have critical knowledge of your requirements and capabilities.
Remember that the telephone network is a mission-critical facet of business operations. The phone
network is built to support 99.999 percent uptime, which equates to roughly five minutes of
downtime per year. Corporate networks rarely provide this level of reliability today. Thus, some
redesign effort may be required to provide suitable reliability in the data network.
Many VoIP projects begin with the implicit understanding that network upgrades will be required.
Network elements such as routers may already be running at high CPU utilization or unable to
support VoIP services. WAN links may be overburdened supporting existing data applications.
Network capacity in terms of bandwidth and processing power are vital factors in deploying any new
The pitfall of upgrading before implementing VoIP is the need for multiple upgrades. A smart
approach is to assess and understand calling requirements in conjunction with existing data services.
Evaluate planned new data services as well as voice requirements. Rather than upgrading
prematurely, focus early on data gathering and information analysis. This methodical approach yields
a better network design with the capacity to support a successful pilot deployment and expand to
provide for all the needs of the business.
Evaluating the Pilot for Success
In the planning and assessment phase, you gathered requirements for the new VoIP service. You also
have information gathered and took steps that set the stage for the design and pilot testing.
Was the Pilot a Success? How Do You Determine the Next
Once the pilot is complete, it is common to ask the wrong question: Was it successful? The more
relevant question to ask is whether you learned what you need to know. A pilot should be viewed
much like an exercise. It’s more than just evaluation of a particular vendor’s technology and testing of
design concepts. The pilot is also an opportunity for gap analysis in testing earlier assumptions you
had made about the network.
You have identified the requirements. The pilot provided real user interaction, so you can tweak
expectations of the production environment.
4 White Paper
Now is the time to review the findings and analyze any network shortfalls. Are upgrades needed to
network hardware? Do you need to implement QoS? Should you do so by adding virtual local area
network (VLAN) technology or Multiprotocol Label Switching (MPLS)? Do you meet the call
quality and service delivery needs?
The Success Factors for Progress
This is not the time to let the VoIP migration project stall. If that happens, when you readdress it a
year from now, or more, you will have to repeat all the work you just finished. This is a critical time
to map your strategy for moving forward and not losing momentum.
Business Decisions and Factors
From a business perspective, it’s vital that you document and validate the business drivers behind
your migration to converged technologies. Cost recovery and reduction are always factors in business.
When integrating data and VoIP services, there may be staff reorganization and management factors
to consider as well.
There is a danger in overlooking improved efficiencies in what is perhaps the most important business
factor: day-to-day work flow. Business process changes might be tweaked and honed for greater
efficiency and improved productivity. For many organizations, VoIP implementation leads to a
business process re-engineering exercise to tighten up work tasks and become more competitive. This
is a good time to develop the roadmap to further integration with business applications you’re
planning to deploy in the coming year.
Technical Decisions and Factors
It’s time to plan the three-to-five-year roadmap for the future of the technology infrastructure while
forging ahead with the planned VoIP implementation. As you move from pilot to production, you will
deploy new methods for managing and monitoring the network. For some organizations, this
transition from pilot into production stalls due to a needless “deer in the headlights” moment of panic.
If you overreact, the project might stall, wasting all the effort you’ve put in. Instead, you must
recognize that voice services are not new to the VoIP implementation, but have been a mission-
critical business tool for years.
Convergence doesn’t put voice at risk by moving it to the IP network. Rather, integrating voice and
data embraces a new reality of the information age; one often left unspoken. The IP network is also a
mission-critical business tool. Gone are the days when the corporate LAN can deliver unreliable
service or leave the business without email for a day. Convergence and integration signal that you
have accepted technology as a vital business tool—vital for business, for competition, and even for
business survival. Your strategies for managing and monitoring the network are shifting to become
your technology dashboard for monitoring the health of business workflows.
Moving Forward: Putting VoIP into Production
When you move forward into production with VoIP services, you want to adhere, as much as
practical, to the earlier baselines established during the assessment, design, and pilot testing phases.
You’re moving beyond planning and probing into the most exciting part of the deployment. You will
get to reap the benefits of the hard work that has gotten you this far.
What Do You Need to Monitor and Manage?
Over the years, telephone callers have come to expect a preconceived level of performance in a
number of areas. Through planning and testing, you’ve identified which QoS and QoE measures
count. Now it’s simply a matter of ensuring the network is ready to deliver, then monitoring to make
sure conditions are stable and that you’re getting what you need from your resources.
The Elements of the Network to Monitor and Manage and Why
You need comprehensive monitoring of the VoIP service and the network elements that deliver that
service. Whenever a new device registers with the network—whether it’s a telephone set, a gateway,
or some other element—there will be an audit entry to review. Registration problems can lead to
service availability problems. You’ll want to be alerted when the number of registration attempts or
failures exceeds predefined thresholds. An unexpected change in the number of registered telephones
could indicate a problem with the network. Monitoring gateway registration can help identify new or
Call monitoring isn’t eavesdropping on individual calls. You need to monitor calls for traffic
monitoring. This means monitoring incoming and outgoing call volumes to identify failures. If the
VoIP system supports fax calling, attempted fax calls also need to be monitored. For most
organizations, call monitoring focuses on the following four areas:
• Calls in progress—Every time a VoIP phone goes “off hook,” a call is in progress until it goes
back “on hook.” Calls in progress that are successful are active calls.
• Active calls—Active calls have successfully connected a voice path. In short, two people can
talk. Remember back in the assessment phase, you designed the system to determine how many
active calls it needed to support. Monitoring the active calls ensures you aren’t outgrowing your
design. As the business grows, this will be one of the first indicators that you need to start
increasing calling capacity.
• Attempted calls—Ideally every call in progress will complete as an active call, but in reality that
just doesn’t happen. Monitoring calls that were attempted provides information about the peak
periods. It helps you identify the busy hour call attempt (BHCA) value for network sizing and
• Completed calls—A completed call is any phone call that is successful and didn’t end
abnormally. Monitoring completed calls also helps you understand the peak periods and the
Because VoIP services usually interconnect to the PSTN through gateways, you need to monitor
gateways and the PSTN side of the VoIP service network, which is often made up of T-1 circuits.
Monitoring these links to the PSTN provides important information about calling patterns and busy
hour peak call volumes. Baseline data can help identify underutilized circuits so you don’t pay for
capacity you aren’t using.
6 White Paper
IP phone functionality needs continual monitoring to ensure service levels are meeting your needs.
You should monitor IP phones for their registration status, the validity of their dial tones, jitter,
latency, and lost packet count. These QoS parameters directly affect service delivery.
Monitoring Bandwidth and QoS
As new applications are added to the network over time, there is always a risk of oversubscribing
network links. Oversubscription leads to congestion; congestion may degrade call quality. Packet loss
and increased delay are common side effects of congestion. Both can render VoIP services unusable,
so you need to monitor basic packet delivery across the network.
You may have determined in earlier readiness, design, and testing that VoIP traffic needed some kind
of prioritization over other kinds of network traffic. The main objective of any QoS mechanism is to
ensure that each type of traffic—data, voice, and video—receives the preferential treatment it
deserves. If you have implemented QoS to reduce or eliminate delay, for example, now you’ll need to
monitor it to make sure your design is working effectively.
The basic metrics you will monitor to keep your service in top condition aren’t daunting. And they’re
easily managed by network tools. You are simply ensuring that you are bringing the right tools into
the mix to support the work you’ve put into developing VoIP services:
• Delay, or latency, is simply an estimate of the network delivery time expressed in milliseconds.
The end-to-end delay, or latency, as measured between endpoints is a key factor in determining
VoIP call quality, and is considered a routine part of monitoring.
• Jitter is nothing more than the variation in the arrival rate of packets. Jitter gives you a view of
the consistency or predictability of your network. You know jitter causes problems with VoIP
service. Networks can compensate for jitter by implementing jitter buffers to normalize the timing
of the traffic flow. Once you’ve performed a pilot and know what you need, jitter monitoring is
just another technical tool for providing a continuous health check on the state of the network.
• Packet loss indicates a packet lost during transmission. In VoIP, packet loss could mean the loss
of an entire syllable or word during the course of a conversation. This loss can severely impact
voice calls, and is tracked by monitoring the number of packets that were expected against the
number actually received.
• Mean Opinion Score (MOS) is a subjective measure used to score the quality of how the audio
sounds in telephone calls. There are a number of technical solutions for monitoring MOS.
Voice quality measurement can be either non-intrusive or intrusive. Non-intrusive tests are usually
based on monitoring actual voice conversations that take place during daily business. Intrusive testing
requires placing test calls across the network.
Monitoring and Managing the VoIP Service Delivery
QoS is a measurement of how you treat the packets traversing the network. It includes the monitoring
of delay, jitter, packet loss, and network availability. Although a measure of instantaneous QoS
metrics is important, it only provides a rough approximation of the user’s complete QoE.
In VoIP systems, QoE is defined as a telephony system user’s perception of the quality of the
communication being experienced. QoE takes into account the cumulative effect of network
characteristics on the human speech being transmitted.
QoE vs. QoS
QoS may be an important element of technical issues such as SLA compliance, but meeting the user’s
QoE expectations is the key to a successful VoIP implementation. QoE is more than just the audio
quality of the voice conversation. Monitoring solutions also provide tools to monitor how quickly
users get dial tone when picking up the receiver, how quickly calls ring through to the other end, how
efficiently connections are torn down when the caller hangs up, echo or noise on the line, and a
number of other factors.
Many aspects of QoS might be measured in incremental links across the network but total QoS and
real-world QoE are end-to-end measures of the complete user experience. Solid network monitoring
and management tools provide for end-to-end testing and alerting capabilities when expected
parameters aren’t met.
Best Practices for Rolling Out VOIP to a Multi-Branch
When deploying VoIP across the enterprise, one of the most successful approaches is to keep
simplicity in mind and try not to do everything at once. Perhaps the riskiest thing an implementation
project manager can undertake is a “flash cut.” The idea that on Monday everyone will come in and
have new phones and be operational on the new system sounds so appealing that it can be very
tempting. In a single site implementation, this can be quite successful, and may indeed be the best
In a multi-branch company, it makes far more sense, and yields a much higher success rate, to take a
phased approach. Cut over one or more sites each week on an implementation rollout plan. The larger
the enterprise, the greater the likelihood you will learn lessons unique to your company along the
way. In other words, although cutovers might have challenges at the beginning, by the time the
migration wave washes across the entire enterprise, the problems will have all been identified; a
methodical cutover means smooth sailing. For example, in a nationwide enterprise, use the idea of
following the sun for cutover work. Don’t start work at 1:00 PM on the West Coast if people on the
East Coast are involved in the cutover work. Plan the cutover to coincide with working hours, or
coordinate time-of-day planning throughout the implementation project.
One danger with pilot testing can be enticing: You cannot afford to plan a pilot, then decide partway
in that it is good enough and just start rolling out production services. Do not give in to the
temptation. It’s important to complete a pilot and assess any lessons learned. By letting a pilot
proceed to production without undergoing a thorough test, you risk learning your lessons during real-
life business operations. It is best to avoid identifying problems with the VoIP service while the CEO
is on a crucial business call with the board of directors. Make sure your pilot is truly a pilot.
You will come away from a completed pilot having learned several lessons. Some of these lessons
will validate things you already know; many others will provide insight into your network’s
architecture and capabilities. A good pilot provides data for reassessing your network’s design. When
moving forward from a pilot, keep in mind these best practices, which are simple and general in
• Build for today
• Upgrade the network in stages as needed
• Plan for tomorrow
8 White Paper
Do-It-Yourself vs. Using a Solution Suite
When it comes to management and monitoring, you might choose from two fundamentally different
approaches. You might choose to do it all yourself. By using commercial software packages,
shareware, and open source tools, an organization can build an effective management and monitoring
suite on its own.
If you’re considering that approach, the first thing you need to ask yourself is why? For most
organizations that seriously look at the do-it-yourself approach, the baseline driver is often cost.
There is a preconceived notion that commercial management and monitoring packages are just too
The danger of doing it yourself is far greater than this simple cost-based mindset demonstrates:
• Who will integrate management tools from different sources into a common corporate dashboard?
• Do you have the resources in house?
• Do you want or need the resources for this sort of work in house?
• Would you rather focus on the core mission of your business?
• Will your staff think of everything you need to manage and monitor, or would you rather partner
with a vendor to incorporate an integrated suite of tools and applications designed to let you
manage the resources of your network without becoming your own vendor?
Consider this: Should you choose to assemble your own custom toolset for management and
monitoring, will you have the in-house resources to help when a service disruption occurs or a
security breach endangers your network? You may not have a trusted vendor or partner who can step
in to help identify and analyze a problem so that you can quickly take the right steps for remediation.
The danger of doing it yourself is that you might have to do more than you want to.
Doing it yourself is a business decision that is not without risk. And although there is risk in any
business decision, working with a reputable solution vendor can bring tremendous strength of
knowledge to the effort.
Consider the value-added NetIQ VoIP solution. It supports the complete life-cycle management (see
Figure 2) and provides support for both Cisco and Nortel applications, thereby easing the
management of and daily operations for several of the major IP telephony solutions deployed in
enterprise VoIP networks.
Report on and review VoIP service
levels Assess network readiness for VoIP
AppManager Analysis Center Vivinet Assessor
Call Data Analysis
Assure VoIP services Define VoIP SLAs and
AppManager for VoIP set monitoring policies
Vivinet Diagnostics AppManager for VoIP
VoIP Security Solution AppManager Control Center
Figure 2: NetIQ support for VoIP management life cycle.
Emerging telecommunication technologies such as VoIP can provide a competitive edge in business.
Managing these technologies to sustain the necessary performance, availability, and reliability can be
a complex role. The NetIQ VoIP solution can help play a vital role in ensuring:
• VoIP servers and voicemail systems are up and working
• IP data networks continue operating properly to sustain business operations
• Call quality meets user needs both in QoS and QoE
• VoIP services are secure, assuring system integrity and availability
In doing so, the VoIP solution helps organizations avoid problems and enhance efficiency in IT
operations. Automating oversight of service delivery maximizes use of staff time and helps make the
most effective use of server and network resources.
NetIQ Vivinet Assessor helps you identify and document how well your network can support VoIP
before you deploy anything new. It helps identify the call quality you will be able to expect from your
network after implementation. It provides predictive modeling capabilities to ensure you can match
your live production network, helps gather the appropriate metrics to ensure accuracy, and simulates
background traffic to closely match the real-world operating environment. Vivinet Assessor also
helps build a network inventory through a set of automated tools. Assessing your network on a
regular basis will continue to ensure that your network will be able to support changes and updates
that occur not only as a result of VoIP updates such as new phones, but changes to other IT
operations, such as newly deployed servers or applications.
You need continuous management and monitoring throughout the life cycle in order to guarantee the
QoS and experience that business communications requires. NetIQ AppManager for VoIP delivers a
solution for managing, securing, diagnosing, and analyzing the performance and availability of VoIP
By simplifying management of VoIP services and incorporating powerful reporting capabilities, you
can maximize the availability and performance of the network. You can use these tools to extend
VoIP expertise to the corporate dashboard environment, always monitoring the health of the network
and services running within the enterprise.
10 White Paper
NetIQ Vivinet Diagnostics helps quickly pinpoint any VoIP call quality issues and narrows
remediation efforts to the root cause quickly and effectively. It brings advanced capability in
automated monitoring, reducing the technical skill level required to maintain a healthy VoIP service.
Vivinet Diagnostics is easy to use in daily operations, and can give your staff the information it needs
to troubleshoot complex VoIP problems in Cisco and Nortel environments. Data pulled by Vivinet
Diagnostics can be communicated to NetIQ AppManager for enhanced reporting via Analysis Center
and predictive performance management with AppManager Performance Profiler as well.
You completed much of the difficult project implementation work involved when you performed
network readiness assessment, network design, and pilot testing. That substantial work effort was
time well spent. Now it’s important to maintain momentum and keep moving forward with service
By implementing a methodical plan and setting proper expectations, you ensured a smooth and
seamless cutover from vision to production operations, but the work doesn’t stop there. At cutover
day plus one, management and monitoring of VoIP services become part of the daily routine. Just as
corporate cultures evolve, operational strategies evolve with them. Technologies change to support
the needs of your business, and the way you manage them has to keep pace.
Sustaining a competitive business network isn’t a product that you drop in place. Just as you have
day-to-day business operations, you have routine management and monitoring. You’re living the life
cycle, and the operations that begin when your VoIP deployment is placed into production are the
first steps in gathering information and analysis for the next evolution of enterprise network services.