Performance Test VoLTE diversifEye Large Scale Tunnel Requests Use real phone Credentials Stateful Application Requests Millions of GTP tunnels with real application requests MME Emulated server S11 Side responses eNodeB S1u S5 SGi SGW PDN GW Fully load S1 - S1u interfacesDelivering high quality Voice over LTE networks (VoLTE)IntroductionOne of the major challenges associated with Voice over LTE (VoLTE) for carriers is defining the Quality of Service (QoS)features for the LTE network.VoLTE is positioned as a high quality voice service enabling call quality equivalent to or better than a 2.5G/3G cellular call,except the network is now all IP. The challenge for the carrier is to ensure each element of the network from the User Equip-ment (UE), Radio Access Network (RAN), Evolved Packet Core (EPC) and IP Multimedia Subsystem (IMS) core are optimizedcorrectly to deliver voice with an uncompromising level of quality. In addition it will most likely be the first time all compo-nents are tested with a mission critical application.The ChallengeVoice over LTE will play a key part in determining a subscriber’sQuality of Experience (QoE) on a 4G network. However, a QoS testplan cannot be solely focused on voice and must include otherservices, which include video and data. The VoLTE application willcompete with other applications for bearer paths and bandwidth.A common reason for 4G carriers to test performance at an appli-cation layer level is to ensure the QoS prioritization of the voiceelement is accurate, but to also ensure that there is no negativeperformance impact on other revenue generating sources whichinclude IMS enablers of presence and location.With the proliferation of smartphones and tablet computingdevices, more applications are appearing which are enabled by Figure 1: VoLTE call quality is dependant on the LTEthe Session Initiation Protocol (SIP). The increasing capacity network handling thousands of concurrent SIP sessionsrequirement on the management layer, plus the addition of appli- per second. Dedicated QoS for Voice in LTE will impact Data applications running on the same limited band-cations means QoS optimization for VoLTE is a two dimensional width resources.challenge in which both layers need attention.
VOLTEThe successful delivery of any application on a 4G network is dependent on the smooth transition of packets betweendistinct elements on the 4G bearer path. With the focus on VoLTE, a number of test requirements immediately appear whenconsidering QoS.At a basic level, performance testing will include call establishment, call billing and call termination procedures. However, tounderstand the impact that variances in LTE network QoS settings have, carriers need to look to the actual subscriber orQuality of Experience (QoE) performance measurement. An example subscriber QoE measurement which is impacted byQoS settings is the individual call quality rating, which must be measured in real-time. This is achieved using subjective callanalysis, giving an insight into the actual audio quality over the duration of the call.Defining a test strategy for VoLTE4G network performance testing requires multiple approaches. This application note focuses in on the IP aspect. At an IPpacket level, there are many performance tests required with varying degrees of complexity. These complex test challengesare overcome by using per flow emulation with stateful application flows.The advantages of per flow stateful application traffic emulationand performance testing are highlighted when it comes to testingVoLTE QoS performance on mobility or in call handover scenarios.To identify some basic LTE QoS performance test requirements anexample approach is to step through each element on the 4Gnetwork bearer path:Radio Access NetworkenodeBs manage a number of complex processes which includebandwidth management of the subscriber applications, manage-ment of the User Equipment (UE) measurement reports, communi-cation with the Mobility Management Entity (MME) and ServingGateway (SGW).A critical factor in any cellular network is call continuity when roam-ing. Call continuity success is based on the ability of the enodeBs toimplement call handovers. A basic LTE QoS test ensures connectiv-ity between enodeBs, alongside packet latency measurementsbetween enodeBs. Tight control over QoS settings covering thedelivery of packets passing between enodeBs is critical, furthertesting will include scaling to include multiple device handovers orincreased traffic on the enodeB interconnect path (X2). Figure 2: 4G/LTE radio access networks areLTE subscribers will have multiple bearer paths assigned to them, so smarter, with more functionality based on theQoE performance tests should include as many different applica- reach of IP to the tower. QoS for VoLTE is more thantions as possible. Per flow performance testing enables carriers analyzing power of the radio network, analysismeasure actual voice quality and to also show how the focused QoS must be performed at a packet level. Testing is notsettings on the dedicated voice bearer, enforced by the enodeB, limited to the call itself but must also includeimpacts bandwidth management on the other application types. security attack mitigation type tests, assessingThe aim is to find a reliable QoS bearer path setting for VoLTE, whilst known vulnerabilities associated with IP.maintaining connectivity for all other applications running. Performance TESTING
UE Performance test eNodeB, SGW, PDN GWUE eNodeB & mobile DPI devices :Test per application flow QoE for voice, video and data. UE emulator Serving diversifEye : Emulate multiple Gateway applications per emulated UE Packet Data Network Gateway + DPI devices Stateful Applications (Video, Voice, Web, Email, etc) INTERNET UE IPv4/6 UE IPv4/6 GTP TEID GTP TEID diversifEye : Emulate application diversifEye server responses or connect to external 3rd party servers Figure 3 : diversifEye provides large scale emulation of stateful application flows, by accurately representing UEs (IPv4/IPv6) and parameters such as IMSI and user authentication details. diversifEye per flow architecture and large scale test capabilities are not only used to test the scalability of the LTE EPC, but provide vital details on a per subscriber Quality of Experience basis. Evolved Packet Core (EPC) A function of the LTE EPC is to scale to meet the increasing demands made by subscribers and their applications. In most instances each subscriber device will have multiple bearer paths assigned, dependant on the application, each bearer path will be assigned a QoS priority level. Initial VoLTE application testing will assess that the application QoS quality level is correct and the network core is prioritiz- ing VoLTE flows. Following on from that, it’s possible to examine performance on other applications. Examination of QoS is not limited to a single user entity. The level of scalability that is achievable in the EPC is dependent on the number of sessions the MME will establish, closely linked with the volume of traffic flows the SGW can cope with. QoS performance testing has a knock on effect in terms of utilization on the control/signalling plane in LTE. See Shenick’s companion note for more details http://www.shenick.com/media_lib/files/Shenick_Optimizing_4G_networks.pdf. Essen- tially, QoS is put in place to handle the millions of session requests accessing unique applications connecting to global web services. Large scale activity testing emphasizes the need for emulation testing in the EPC. Scale is not the only limiting factor to VoLTE; performance testing needs to examine the Session Initiation Protocol (SIP), in all instances there will be a large volume of concurrent SIP sessions. Detailed testing should outline the overhead associ- ated with maintaining each of those sessions. VoLTE Performance TESTING
As in the RAN, roaming UEs place additional resource requirements on the EPC. Once the enodeBs and the UE have finalizedhandover, it is up to the target enodeB to communicate this message with the MME. It’s critical that the latency and the avail-able memory buffers on interlinking paths (S1-MME) are set correctly. With packet latency performance, it’s often the casethat buffer sizes are set too aggressively which has a negative impact on QoS performance especially during a period of lowtraffic requests. It only requires tens of milliseconds of latency to perceive a VoLTE call as poor quality.Testing the management paths with varied traffic load scenarios will result in an optimized QoS setting, preserving VoLTEcall continuity. UE SIP Metrics DescriptionThe EPC MME manages both the interconnected enodeBs andInaRTP Bits/sec number of gateways in which the bearer traffic UE varying The number of RTP bits/second received in by this UE.passes through. Returning to the roaming subscriber’s call; call continuity UE Out RTP Bits/sec is further dependent on the speed in which the The number of RTP bits/second sent out by this UE. UE In RTP Packets/sec UE to the user plane The number of RTP packets/second receivedby by this UE.MME updates the SGW, with a message indicating the change Out RTP Packets/sec path for the UE. Latency measurements The number of RTP packets/second sent out in this UE.again are an important parameter to maintaining the VoLTE callRTP Out of Sequence UE quality. The number of packets out of sequence sent out by this UE. Packets UE RTP Dropped Packets The number of packets dropped by this UE.Another important QoS feature in the LTE EPC concerns the speed in which the address redirection request in by this UE. out, it UE Duplicate RTP Packets The number of duplicate packets received is carriedmust be managed by the SGW so as to minimize the volume UE Out Calls Attempted of traffic heading backnumber of calls out attempted by this UE. sample QoS The to the source enodeB. Aperformance optimization test will cover bandwidth utilization, Out Calls Established UE anCalls Rejected test isThe number ofan excessivethis UE. UE Out example to force calls established by number of redirec- The number of calls rejected by this UE.tion updates for the MME to handle. UE In Calls Attempted The number of incoming calls that this UE attempted to receive. UE In Calls Established The number of incoming calls established by this UE.As each request increases the CPU cycles required in the MME, this has an effect of The number ofthe MME’s abilitythis UE. UE In Calls Rejected slowing incoming calls rejected by to manage theSGWs. To determine what impact these loads have on performance, analysis is done covering the number of packets still UE Calls Errored The number of calls with errors logged by this UE.arriving to the UE address after the UE handover request has beenOut Messages by theThe number of SIP messages sent out byby this UE. UE SIP initiated UE SIP Messages Resent source enodeB. The number of SIP messages resent out this UE. UE SIP In Messages The number of SIP messages received by this UE. UE In RTCP Packets The number of RTCP packets received in by this UE. UE Out RTCP Packets The number of RTCP packets sent out by this UEUE SIP Metrics Description UE Registrations Attempted The number of registrations attempted by this UE.UE In RTP Bits/sec The number of RTP bits/second received in by this UE. UE Registrations Successful The number of successful registrations by this UE.UE Out RTP Bits/sec The number of RTP bits/second sent out by this UE.UE In RTP Packets/sec The number of RTP packets/second received in by this UE. UE Registrations Rejected The number of registrations rejected by this UE.UE Out RTP Packets/sec The number of RTP packets/second sent out by this UE. UE Registrations Errored The number of registrations with errors logged by this UE.UE RTP Out of Sequence UE Calls Received Ringing The number of ringing calls received in by this UE. The number of packets out of sequence sent out by this UE.Packets UE Mean Time to Ringing (ms) The average time for incoming calls to this UE to ring.UE RTP Dropped Packets The number of packets dropped by this UE. UE Min Time to Ringing (ms) The minimum time for incoming calls to this UE to ring.UE Duplicate RTP Packets The number of duplicate packets received in by this UE. UE Max Time to Ringing (ms) The maximum time for incoming calls to this UE to ring.UE Out Calls Attempted The number of calls out attempted by this UE. The number of messages with RTP packets received in byUE Out Calls Established The number of calls established by this UE. UE Calls Received RTP Packet this UE.UE Out Calls Rejected The number of calls rejected by this UE. UE Mean Time to RTP Packet The number of incoming calls that this UE attempted to The Mean time for this UE to receive the first RTP packet.UE In Calls Attempted (ms) receive. UE Min Time to RTP PacketUE In Calls Established The number of incoming calls established by this UE. The Minimum time for this UE to receive the first RTP packet. (ms)UE In Calls Rejected The number of incoming calls rejected by this UE. UE Max Time to RTP Packet The Maximum time for this UE to receive the first RTP packetUE Calls Errored The number of calls with errors logged by this UE. (ms)UE SIP Out Messages The number of SIP messages sent out by this UE. UE RTP Jitter (RFC 3350) ms The Jitter per ms.UE SIP Messages Resent The number of SIP messages resent out by this UE. UE RTP Max Jitter (RFC 3350) The maximum Jitter per ms.UE SIP In Messages The number of SIP messages received by this UE.UE In RTCP Packets The number of RTCP packets received in by this UE.UE Out RTCP Packets The number of RTCP packets sent out by this UEUE Registrations Attempted The number of registrations attempted by this UE.UE Registrations Successful The number of successful registrations by this UE.UE Registrations Rejected The number of registrations rejected by this UE.UE Registrations Errored The number of registrations with errors logged by this UE.UE Calls Received Ringing The number of ringing calls received in by this UE. Table 1 : Dependanttime forthe SIPcalls to this UE to ring. (typically 9 to 16 SIP messages per session) the signalling plane willUE Mean Time to Ringing (ms) The average on incoming configurationUE Min Time to require tight management. To identify how ring. Ringing (ms) The minimum time for incoming calls to this UE to configuration optimization impacts quality a unique set of metrics isUE Max Time to Ringing (ms)detailing each step in the VoLTE call process. Highlighted above are a sample set of metrics. required The maximum time for incoming calls to this UE to ring. The number of messages with RTP packets received in byUE Calls Received RTP Packet this UE.UE Mean Time to RTP Packet The Mean time for this UE to receive the first RTP packet.(ms)UE Min Time to RTP Packet The Minimum time for this UE to receive the first RTP packet.(ms)UE Max Time to RTP Packet(ms)UE RTP Jitter (RFC 3350) ms The Maximum time for this UE to receive the first RTP packet The Jitter per ms. VoLTE Performance TESTING
IMS CoreThe IMS core contains a number of elements, for the purpose of VoLTE QoS performance testing, it’s crucial to examine thecore in terms of the SIP functionality and handling.Focusing on the Call Session Control Function (CSCF), the dual purpose of this element is to handle SIP messaging andmanage the IP address assignment requests from the UE (DHCP). The CSCF may also use encryption (IPSec or TLS) for secu-rity purposes on its ports.QoS performance testing aimed at VoLTE in the IMS core should cover all the protocols in use. As a minimum, VoLTE QoEtests in the IMS core must cover: - UE IP address assignment timing performance - SIP registration performance - Actual voice quality analysis in real-time - Call release and teardown performanceAn example performance test is to see how quickly an IP address is assigned to the UE or how quickly that device can sendthe first SIP “Invite” message. This scaled to several thousand users will give an accurate representation on how well the IMScore is primed.The IMS Core will pass millions of SIP messages a second. This puts the rates of concurrent SIP sessions in the order of tensof thousands. Clearly it’s not feasible to try to actively test these numbers with a group of friendly users. To create largevolumes of concurrent SIP sessions requires high performance, scalable emulation tools that can analyze each and everysession in real-time. Figure 4 : Optimization is a key requirement for LTE as each SIP session enabled on the carrier costs revenue. IMS enablers such as Presence and Location are one way in which carriers can counteract revenue loss to OTT content providers and benefit from an all IP network. For these services to a success carriers must performance test in the application layer. VoLTE Performance TESTING
VoLTE Performance TESTINGdiversifEyediversifEye™ is a test and performance measurement solution for next generation networks such as 4G-LTE and Cloud Com-puting. diversifEye provides emulation, test and performance measurements of individual application flows such as Video,Voice and Data applications running in the LTE network. diversifEye’s per flow architecture with real time analysis providesthe granularity necessary to determine any Quality of Experience performance issues on a per application basis relating topoorly configured QoS settings on the 4G network.As an emulation tool diversifEye saves considerable time when testing 4G network performance with its pre-configuredapplications and emulated UE end-points. Testing includes performance analysis of SIP, SIP over IPSec, SIP over GTP, etc.Furthermore diversifEye provides IMS type applications such as SIP enabled RTSP or SIP enabled HTTP flows.diversifEye provides large scale emulation of stateful application flows, by accurately representing UEs and parameters suchas IMSI and user authentication details. diversifEye’s emulated traffic flows are used to load various interfaces on the 4G path.diversifEye is not limited to solely loading the LTE network paths but also enables the delivery of real user activity on a perUE basis. diversifEye’s per flow architecture means multiple application flow requests per registered UE, representing a realsubscriber’s smartphone activity. diversifEye enables testing utilization performance of the LTE EPC.diversifEye’s per flow architecture provides performance measurements on each and every emulated application flow asso-ciated with each and every emulated UE, enabling users to quantify how LTE QoS feature changes impact a subscriber’sVoLTE service. Figure 5 : Dedicated VoLTE performance windows provide real-time views on QoE. Capture live performance issues when changing core LTE QoS settings.SummaryOptimizing 4G networks to save costs has been greatly simplified by Shenick’s per-flow emulation and test approach. Withper-flow emulation and real time analysis it’s possible to limit the impact to customer QoS. Using Shenick’s solutions, serviceproviders and network equipment manufacturers are given the necessary granular view of end-user quality in real time. Anysubtle changes in quality are picked up by Shenick’s thresholding facility, enabling service providers to quickly judge, in realtime, if a decision to change or optimize a device setting is the correct one. This win-win situation ensures service providerscontinue to deliver services at the right price!About Shenick Network SystemsShenick is an award winning provider of per-flow IP communications test systems, enabling converged IP network service providers andcommunications equipment vendors to test the complete service delivery life cycle from lab to live deployment. Established in 2000,Shenick has deployed its diversifEye™ integrated network, application and security attack emulation and performance monitoringsystems to converged IP-oriented network service providers, communications equipment manufacturers, large enterprise and govern-ments globally. diversifEye is a registered trademark of Shenick Network Systems. For more information contact: email@example.com US : 533 Airport Boulevard, Burlingame, CA 94010, USA Tel: +1-650-288-0511 EU : Brook House, Corrig Avenue, Dun Laoghaire, Co Dublin, Ireland Tel: +353-1-236-7002