Technology Update
                                                      By Tara Van Unen                   CompactPCI & Ad...
A household receiving triple play services
                              Technology Update
from a single provider may be s...
Technology Update

device/system under test. By scaling the         UDP/IPv4, and multiple VLAN tags)           highlights...
Technology Update

n Maximum acceptable DF: 9-50 ms
n Maximum acceptable MLR
  (all codecs): 0.004 for SDTV and
  VOD and ...
Upcoming SlideShare
Loading in …5

Validating IPTV service quality under realistic triple play ...


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Validating IPTV service quality under realistic triple play ...

  1. 1. Technology Update By Tara Van Unen CompactPCI & AdvancedTCA Systems Validating IPTV service quality under realistic triple play network conditions In this column Tara discusses the emerg- ing test methodologies and metrics for verifying IPTV Quality of Experience (QoE) across the IPTV delivery network, as illustrated in Figure 1. What happens during the converging and scaling of time/loss-sensitive video traffic with existing voice and data services con- tending for the same network resources? Viewers have come to expect a predict- able level of service quality with their broadcast and satellite TV services, and they will not be tolerant to service inter- ruptions, picture degradation, or long Figure 1 waiting periods to change channels with their new IPTV service. the correct channel. Acceptable channel Network characteristics and zapping delay is generally considered to be media quality IPTV quality of experience around 1 second total, end-to-end. A chan- It’s estimated that the number of IPTV With so much at stake, it is not surprising nel zapping time of 100-200 ms is consid- subscribers worldwide will reach 53.7 mil- that IPTV QoE has become one of the most ered by viewers to be instantaneous. lion in 2009[1]. IPTV subscriber growth in popular buzz words within industry publi- North America alone will increase 12,985 cations, tradeshows, and forums. IPTV QoE Multicast protocols enable channel zap- percent between 2004 and 2009. France, refers to how well the video service satis- ping within the network infrastructure. the leading European country in rolling fies users’ expectations. The IPTV quality Internet Group Management Protocol out IPTV services, had 281,000 subscrib- subscribers experience must be equal to or (IGMP) or Multicast Listener Discov- ers registered to the three main IPTV ser- better than today’s cable and satellite TV ery (MLD) leave/join delay has a direct vices (Maligne, Free, and Neuf) at the end services, or else service providers run the impact on channel zapping delay. To keep of 2005[2]. Digital video distributed via risk of significant subscriber churn. overall channel zapping delay within IP multicast (IGMP in most networks) 1 second, the target multicast leave/join doesn’t ensure consistent video quality QoE is influenced by commercial factors, delay of each network component needs among all the users watching the same such as the price, content, and features of to be about 10 ms to 200 ms. channel. It is therefore difficult to ensure the service, as well as technical factors that each and every subscriber is receiv- including channel change response times, Many factors can compromise the per- ing the video properly. With IPTV deliv- and the media quality itself. Measuring ceived media quality, so that measuring it ery network bandwidth and processing IPTV QoE refers to testing the techni- is a formidable challenge. The scale and resources finite, it follows that the more cal aspects that influence the subscriber’s behavior of IPTV subscribers and conver- subscribers requesting the IPTV service, service experience. Acquisition costs are gence of other triple play traffic contend- the higher the threat of compromised QoE. estimated are at $1,000 per subscriber, ing for finite network resources have a It is critical that network equipment be and it is important to validate IPTV QoE significant impact on the timely and accu- tested under an increasing scale of both on a per-subscriber basis. rate forwarding of IPTV packets. Result- subscribers and IPTV channels to iden- ing network impairments (packet loss and tify the point at which per-subscriber QoE There are two fundamental areas of IPTV sequence errors, latency, and jitter) can reaches an unacceptable level. If the access QoE testing: have various detrimental effects on visible network can only accommodate, for exam- video quality such as blocking, blurring, ple, 1,000 simultaneous subscribers for the n Channel zapping measurements edge distortion, judder (choppy picture), World Cup in one community, user 1,001 n Media (audio and video) quality and visual noise. Therefore, a complex could potentially degrade the experience metrics network environment that accurately for all of the 1,001 viewers. reflects the characteristics of triple play Channel zapping measurements evaluate networks must be represented in the lab in Dynamic subscriber behaviors how quickly subscribers can change order to sufficiently stress network equip- In a realistic triple play user environment, channels, and verify viewers are receiving ment and evaluate IPTV media quality. subscribers behave in a dynamic fashion. Reprinted from CompactPCI and AdvancedTCA Systems / September 2006 Copyright 2006
  2. 2. A household receiving triple play services Technology Update from a single provider may be simulta- neously initiating channel-change and new Internet-connection requests while having multiple VoIP telephone conversa- tions. When scaled across the subscriber base, this dynamic behavior can be very Figure 2 demanding on the control plane of IPTV network elements and potentially jeopar- boxes and video sources, along with VoIP using a single test system to simulate and dize IPTV viewers’ QoE. phones and personal computers to repre- make IPTV QoE measurements on a net- sent voice and Internet service traffic, is work environment that accurately reflects Take, for example, a large number of not a very practical approach to testing the scale and characteristics of real-world simultaneous channel change requests IPTV QoE. Obvious concerns include the triple play networks. Figure 3 illustrates (channel zapping) during the commercial high capital outlay required to source the how a large test bed of equipment can be break of the Academy Awards. The rapid equipment, space requirements for hous- replaced with a single test tool that can: transition from a steady state of long- ing the test bed, and the labor-intensive term viewing to a huge series of changes configuration required to control the tests. n Simulate IPTV subscribers and can significantly stress a Broadband Net- This methodology also has serious flaws channels with scalability work Gateway (BNG) or edge router as it since it doesn’t scale well to reflect real- n Emulate dynamic subscriber struggles to process thousands of IGMP world subscriber numbers, which can behaviors group join/leave requests, update mul- reach from the thousands into the hun- n Generate a combination of triple play ticast forwarding tables, and replicate dreds of thousands. Set top boxes over- traffic multicast traffic over the correct outgo- heating and requiring frequent rebooting n Provide relevant test metrics for ing interfaces. This stress can result in compromise the reliability of the tests. thousands of individual subscribers packet forwarding delay and loss and Within this setting, IPTV QoE is deter- affect IPTV QoE. It is therefore critical mined by individuals who watch and surf By emulating and scaling multiple to model dynamic subscriber behaviors in television programming for hours on end protocols simultaneously over the same the test environment in order to measure and rate the overall quality of their expe- test port, such as Point to Point Protocol the impact on IPTV QoE metrics. Includ- rience within a defined numeric scale. over Ethernet (PPPoE), Dynamic Host ing a combination of triple play traffic Configuration Protocol (DHCP) and within the test environment to identify While this type of subjective testing can IGMP, a realistic (and cost effective) test how the presence (or interference) of identify when service quality expecta- environment that reflects IPTV subscriber other service traffic influences quality tions aren’t being met, it provides little growth expectations can be simulated. of service and timely forwarding of high insight into what’s actually causing or Test systems must be able to emulate and priority video traffic is a must. contributing to the service degradation. make measurements on tens of thousands Test measurements are difficult (if not of subscribers, watching hundreds of Traditional approaches to impossible) to correlate with specific different channels, located behind a measuring IPTV quality of problems at the network layer, thus the single device. experience measurements have little value when it The traditional, yet problematic, approach comes to troubleshooting and isolating This simulated topology is much easier to achieving channel zapping and media network configuration problems. to manipulate and control than using real quality measurements has been to build set top boxes, while providing excellent an extensive test bed of equipment in the Next generation methodology for granularity. A unique channel zapping lab, as shown in Figure 2. measuring IPTV QoE profile can be applied to each individual Next generation IPTV QoE test method- subscriber, or group of subscribers, to This type of test environment, which ologies overcome the issues and limita- identify how different channel changing incorporates hundreds of real set top tions of traditional test approaches by behaviors impact the performance of the Figure 3 Reprinted from CompactPCI and AdvancedTCA Systems / September 2006 Copyright 2006
  3. 3. Technology Update device/system under test. By scaling the UDP/IPv4, and multiple VLAN tags) highlights packet loss events as contribu- channel zapping activity, it is possible to n Video unicast traffic tors to poor video quality. This provides saturate the device under test with thou- (representing Video on Demand) much greater insight into the network sands of individual requests for the same n VoIP (voice) traffic conditions that contribute to video qual- channel, as well as thousands of multi- n Internet traffic ity than, say, a simple video quality score cast group join/leave requests from sub- on an arbitrary scale[3]. scribers flipping through channels in The oversubscription and convergence sequence. Developers can use a simulated of this traffic over the same port will MDI values are also more relevant for topology (Figure 4) to quickly achieve per- force the device/system under test to assessing network equipment performance subscriber channel zapping measurements, forward traffic according to service than video quality metrics. Video quality as well as the average/minimum/maximum prioritizations. metrics decode the video and consider for different subscriber groups. Key chan- compression and codec properties. Yet nel zapping test scenarios include: The realistic simulation of triple play network devices can only switch, delay, subscribers and services creates an ideal or drop packets. Isolating network equip- n Internet group management protocol environment for verifying overall IPTV ment’s effect on IPTV QoE demands met- or multicast listener discovery leave media quality. The media quality test rics based on packet-level measurements. and join delay for subscribers metrics must be scalable, repeatable, and Furthermore, as MDI does not rely on the n Sustained channel zapping provide insight into the reasons behind processor-intensive activity of decoding, performance relevant performance problems. measurements can be scaled to tens of n Channel zapping performance under thousands of subscribers at once to achieve peak load The Media Delivery Index (MDI) is gain- critical media quality metrics including: ing widespread industry acceptance for Defining their interaction with other triple testing IPTV QoE over network elements n The effect of protocol stress and play voice and data services can further in a video delivery infrastructure. MDI is network instability on media quality scope the behavior of each subscriber. an industry standard defined in RFC 4445 n Voice and data services impact on Applying a dynamic and comprehensive and endorsed by the IP Video Quality media quality profile to each subscriber allows the mod- Alliance. MDI’s two components, the n The ramifications of channel zapping eling of ever-changing network condi- Delay Factor (DF) and the Media Loss on media quality tions, such as DHCP session-flapping and Rate (MLR) are based on concepts that n The number of IPTV subscribers new Internet connection requests, which translate directly into networking terms: and channels the device can support can put significant stress on network jitter and loss. MDI correlates network within an acceptable media quality equipment, and impact IPTV media qual- impairments with video quality, which is ity. These profiles can be used to define vital for isolating problems and determin- QoE standards for IPTV are still under and generate different combinations of ing their root cause. A high delay factor debate, but the DSL Forum’s WT-126 triple play traffic types, with correspond- directly indicates that increased latency, recommends a maximum loss of up to ing priorities, including: which can degrade video quality, has been 5 consecutive IP packets per 30 minutes introduced by the device/system under for SDTV and VOD, and 4 hours for n Video multicast traffic (representing test. It also warns of possible impending HDTV. If translated into MLR terms, this broadcast IPTV) with different video packet loss, as device buffers approach assumes the loss is a single IP packet in the payloads such as MPEG-2 SD and overflow or underflow levels. This points specified timeframe to account for the fact MPEG-4 AVC/H.264 SD/HD traffic, to congestion in the network or inade- that the packet loss must be consecutive. and Windows Media 9/VC-1 and quate buffer resources as potential reasons Agilent Technologies’ study recommends encapsulations (for example, MPEG-2 for the poor performance. Similarly, the the following cumulative MDI measure- TS/RTP/UDP/IPv4, MPEG-2 TS/ MDI’s media loss rate component clearly ments throughout the delivery network: Figure 4 Reprinted from CompactPCI and AdvancedTCA Systems / September 2006 Copyright 2006
  4. 4. Technology Update n Maximum acceptable DF: 9-50 ms n Maximum acceptable MLR (all codecs): 0.004 for SDTV and VOD and 0.0005 for HDTV[4] Figure 5 illustrates how packet loss is manifested in the viewed video quality. Conclusion IPTV QoE is the critical factor under- lying the success of IPTV deployments and service provider triple play service bundles. Several widely publicized IPTV trials have already failed to turn into actual deployments, and others have been Figure 5 delayed for more than a year. The losses of face and impaired financials have been Tara Van Unen is a product manager very costly. Ensuring customers’ viewing for Agilent’s Data Networks Operations, If you found this article valuable, experience meets expectations from day focused on IPTV and triple play test click here for your complimentary one mitigates this risk. methodologies. She has been working subscription to CompactPCI and on the development of Agilents N2X AdvancedTCA Systems magazine. It is critical to select test tools and method- Multiservices Test Solution since 1999. You can also subscribe to the ologies with the ability to test new video Tara is an expert in IPTV, multicast, and CompactPCI and AdvancedTCA services in the larger context of triple MPLS technologies and is a member of E-Letter, our free e-mail play networks to verify IPTV QoE under the IPTV Interoperability Forum within newsletter, by clicking here. increasing scale, and in the presence of ATIS. She holds a MBA degree from other voice and data services. Diligent Simon Fraser University. and thorough pre-deployment testing, under realistic and dynamic network To learn more, contact Tara at: conditions, will have a direct influence on subscriber satisfaction. Agilent Technologies 395 Page Mill Road References Palo Alto, CA 94306 [1] Infonetics Research, Inc., IPTV Equipment Tel: 877-424-4536 and Services Market Outlook, 2005, p. 28 E-mail: [2] Franz Kozamernik, “IPTV – a different Website: television,” July 2006; available from tcm_6-46276.php, accessed August 3, 2006 [3] Andre Dufour, Agilent White Paper: Scalable IPTV Quality Testing, 2006, p. 3 [4] Andre Dufour, Agilent White Paper: Understanding the Media Delivery Index, 2006 Reprinted from CompactPCI and AdvancedTCA Systems / September 2006 Copyright 2006