Your SlideShare is downloading. ×

Thanks for flagging this SlideShare!

Oops! An error has occurred.


Saving this for later?

Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime - even offline.

Text the download link to your phone

Standard text messaging rates apply

Telepresence Testing Approach by Shenick


Published on

Presented at IMTC Telepresence Workshop …

Presented at IMTC Telepresence Workshop
June 15, 2010
Jesi, Italy

describes issues and solutions associated with Telepresence system testing

Published in: Technology

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • Shenick is an award winning provider of IP test solutions for Next Generation Networks and Applications. Shenick’s unique selling point is the ‘Per flow’ architecture enabling users emulate unique end-points running various applications like Telepresence, with as close to real activity. This provides users with the unique ability to build test scenarios with varying load conditions and traffic types.
  • Telepresence success is based on an almost flawless presentation of HD video and Audio. Clearly the video and audio quality are clearly the most relevant metrics. But in relation to understanding the full picture of quality of experience more test parameters need to be included in the test strategy. Evidently Telepresence is not the only service ro run on a network, therefore its important to test with varying applications and load conditions.
  • Its clear that different people will require different test processes and procedures along the production path. It’s important to select a test partner/solution that will work efficiently and effectively at each stage, including the full implemented live solution.
  • Closed testing, enabling users pre-empt the likely outcome of the test, these type tests are useful in examining performance of the basic system in isolation.
  • At a network wide level performance can still be pre-empted, this enables fine tuning of components and various QoS settings, these type tests should still be seen as closed.
  • A key component of Telepresence is the management of video and audio flows from unique end-points. Its clear to test Telepresence requires a per flow approach in which each end-point has its own unique parameters. As part of the overall Telepresence experience its important to include tests such as DHCP server timing performance and test end-point management databases access times, all of which contribute to a successful call attempt .
  • Adding dynamic user behaviour is a step towards an open test environment. This means tests conditions can be altered during live runs, making it a little harder to predict the actual outcome e.g. what happens when all end-points mute calls and un-mute at the same time, with high decibel noise?
  • Order is critical
  • By adding additional user traffic flows to the test such as web, email, P2P the test strategy becomes Open. Open means its less likely the outcome of the test can be predicted.
  • `Shenick has two distinct product lines diversifEye and servicEye. diversifEye is a lab based tool.
  • servicEye Spider enables users test from remote end-points on the network against real services.
  • If you require more details, or alternatively would like to organise a demo please do not hesitate in getting in touch. Thank you.
  • Transcript

    • 1. Mark Lambe Product Marketing Manager Shenick Network Systems Use of Traffic Emulation in Interoperability Testing
    • 2. Overview
      • Telepresence is the delivery of high performance voice and video which plays a critical role in service differentiation.
      • A clear understanding of quality and performance limitations under a variety of operational conditions, including interoperability, is imperative to success.
      • Today’s discussion covers the benefits of adopting real-time, per end point user emulation and analysis to deliver a highly credible and pragmatic quality test program.
    • 3. Define Interoperability
      • Interoperability means different things to different people at each stage
        • The Telepresence deployment will function correctly i.e. each component in the system can communicate and operate with each other.
        • The Telepresence deployment operates correctly on the proposed infrastructure with minimum disruption.
        • The Telepresence deployment will operate alongside other services (video, voice, data) without impacting QoE/QoS.
        • The use of different Vendor end-point systems/components.
      • Telepresence has unified the way we communicate
        • Therefore a unified test solution covering all aspects of Interoperability is required.
      Vendor Interoperability
    • 4. System Functionality
      • Emulate the most realistic components, therefore giving the most accurate results
        • A Complete test process - examine performance from end-point boot-up to call ended
      • Performance test configuration/management server responses
        • Ability of end-points to connect to management servers
        • Time to download end-point firmware settings, upgrades, etc
        • Time to register through to time to join calls
      • Participate in real/live Telepresence calls
        • Measure actual video/audio quality
      • Security Testing
        • Attempt registration with un-assigned MAC addresses
    • 5. Operational Functionality
      • Preliminary testing
        • Network Performance
          • Connectability – End-points can communicate with management servers
            • What impact will firewalls have on Telepresence traffic flows coming from unsecure to secure networks
          • Test Telepresence management servers under varying network and load conditions
            • If a DDoS attack occurs at the firewall, what impact has this on the Telepresence conference
        • Hardware Performance
          • Determine latency measurements
          • CPU Processing performance limitations
        • Quality
          • Test end-to-end video/audio quality using perceptive analysis tools
          • Test bandwidth performance use various video grades (low, medium, high resolutions)
      • Classification Testing
        • QoS Testing
          • Are Telepresence flows identified and handled correctly
          • False negative tests – test with flows of similar nature ( e.g. P2P)
          • False positive tests – a Telepresence flow is identified and handled incorrectly
    • 6. Meeting Functionality
      • Flow Control Testing
        • Video Switching – test video switching policies
          • who has the floor testing
          • Who is shouting the loudest to win control of the screen
          • Sporadic Interruption through dynamic behaviour or un-muting of a participant
        • Audio – test the capability to deliver every end-points audio flows
          • a heated discussion
      • Security feature Testing
        • Assess muted room appearance functionality
        • Attempt to join conference with unknown/un-invited
        • numbers
    • 7. Emulated Telepresence Activity
      • Per Flow Emulation & Performance Analysis
        • Emulate real stateful end-points, with real user activity
          • Register with Telepresence Management Servers
          • Participate in real calls
        • Accurate representations of real world devices
          • Per end-point unique MAC address assignment
          • Static and Dynamic IP address allocation
          • IPv4 and IPv6 capabilities
      diversifEye’s flexibility enables users configure device properties easily, matching those in the Telepresence device management database
    • 8. Real User Activity
      • Participation with as close to real activity is key in performance testing Telepresence systems and deployments, dynamic behavior adds a level of un-predictability to the likely results outcome
        • Unique video and audio per emulated Telepresence
        • end-point
        • Ability to vary codecs for video and audio
        • Auxiliary inputs e.g. Videos, Presentations
        • Dynamic behaviour to emulated end-points
          • Loudness (peak level, duration, etc)
          • Mute / Un-mute calls in live tests
      Shenick’s diversifEye enables users set various levels of loudness per emulated Telepresence end-points Mute / Un-mute calls during live tests
    • 9. Sample Performance Analysis
      • Measure performance on each and every individual emulated Telepresence end-point
        • In multi-point calls determine how varying network architectures/settings impact call performance
      • Sample Performance Metrics per unique Telepresence end-point
        • Video Quality – Use perceptive analysis tools to deliver MOS scores
        • Audio Quality – Use perceptive analysis tools to deliver MOS scores
        • RTP Bandwidth – bits or packets per second
        • SSRC rate – Sent / received
        • RTP packet metrics – Dropped/Duplicate
        • RTP Video frame count / Video frame jitter
        • Out Calls Attempted / Established / Rejected
      Shenick’s diversifEye delivers real-time results analysis, on a per emulated Telepresence end-point basis. Examine during live tests the impact of different traffic volumes and application types
    • 10. Open Test Strategy
      • Telepresence will run in parallel to other services such as voice, video and data.
        • It’s essential to determine the impact Telepresence deployments have on existing services
        • It’s also important to understand how Telepresence is impacted by adding other traffic sources and varying these flows loads.
        • Test with both legitimate and illegal traffic flows, spam, virus, worms, DDOS, etc
      • In the following open test scenario
        • What’s a relevant / realistic MOS score for Telepresence in live networks?
        • Who would you perceive as the happiest customer?
      4.5 4.6 3 VoIP MoS 2,000 5,000 950 Email send time (ms) 10 1 25 HTTP Downloads (seconds)  P2P traffic flows (present) 4.2 / 4.4 Enterprise C 3.9 / 4.0 Enterprise B 4.5 / 4.5 Enterprise A Telepresence MOS (video / audio)
    • 11. diversifEye
      • diversifEye emulates stateful clients with real world functionality and features for Telepresence
      • diversifEye provides stateful traffic emulation of many common applications and traffic flow types:
      • IGMP / MLD, VoD (RTSP), VoIP, Dual Hosted VoIP, RTP, PPPoE, HTTP, P2P, VLAN, DHCP, P2P TCP Playback, SMTP, POP3, FTP, TWAMP, SSL/TLS
      • Mix legal and illegal flows: include DDoS type attacks, emails with vrius, spam attacks, etc
      • diversifEye may support a mix of client and/or server applications using IPv4 and/or IPv6 on a single test interface port.
      • Support for IPv6 transition protocols: Dual-Stack Lite, 6rd / 6to4
      • diversifEye is used in ‘Per flow’ application emulation and performance analysis of multiple environments including : xDSL, xPON, IMS, CABLE, WiMAX & LTE
      • Capture and replay TCP, UDP (files >1Gb)
      • Fully compliant TCP stack
    • 12. servicEye Spider
      • Remote Network Test with centralized management and control
      • Tap into the power of a sophisticated lab tester from multiple remote network sites
    • 13. Thank You! Questions & Answers email :