The document discusses different network environments that can be used for testing interoperability and robustness of devices, including: pure networks, real networks, and emulated networks. Pure networks are the simplest but only test for interoperability, not robustness. Real networks best test robustness but are difficult to control and reproduce issues. Network emulators aim to provide controllable and reproducible impaired network conditions to identify defects, but add complexity to testing. Key considerations discussed include the types of client and server devices used for testing (QE-UAs, QE-Servers) as well as how to determine appropriate network impairments and parameter values when using emulation.

1. Interoperability WG –
Network Environment

2. 2
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Pure Network
RFC 3261
SIP Server
QE - Servers
(TBD)
QE - Server
SIP UA
DUT
Functional
Operation
QE - UA
QE - UAs
(TBD)
Functional
Operation
Standard UA
SIP MessagesSIP Messages
ViTA IOT
Test Spec

3. 3
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Real Network
QE - UAs
(TBD)
SIP UA
DUT
Functional
Operation
Real Network
(TBD)
QE - Servers
(TBD)
ViTA IOT
Test Spec

4. 4
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Real Network Requirements
 Network Provider
 HiNet
 SeedNet
 …
 Network Status Report
 Support Network Impairments Monitoring
 Support RMON Report
 …
 QE-UAs
 QE-Servers
 ViTA IOT Test Spec.

5. 5
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Network Emulator
Network
Emulator
QE - Servers
(TBD)
QE - Server QE - UA
QE - UAs
(TBD)
Functional
Operation
Standard UA
SIP UA
DUT
Functional
Operation
ViTA IOT
Test Spec

6. 6
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Network Emulator Requirements
 Network Emulator
 Define Network Impairments
 Reference next pages
 QE-UAs
 QE-Servers
 ViTA IOT Test Spec.

7. 7
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Network Emulator - Impairments
 Latency
 Packet Loss
 Packet Effect
 Link Fault
 Congestion
 Bandwidth

8. 8
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Latency
 Fixed Latency
 A fixed latency value is
applied to all packets
 Configurable parameter:
Latency
 Uniform Distributed
Latency
 A changing latency over time
 Can emulate jitter condition
 Configurable parameter:
Minimum Latency, Maximum
latency
delay
count
delay
count

9. 9
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Latency
 Normal Distributed Latency
 Latency fluctuate randomly
and non-uniformly around
average value
 Can emulate jitter condition
 Configurable parameter:
Average, Standard Deviation
 Linear Latency
 Latency increase as time
increase
 Can emulate jitter condition
 Configurable parameter:
Minimum Latency, Maximum
Latency, Time Duration
delay
count
delay
Time

10. 10
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Packet Loss
 Periodic Loss
 Discards one packet every 2 to 65535 packets
 Count packets separately on both direction
 Random Loss
 Packet losses base on a probability from 0.01% to 100%
 Bust Loss
 Packet losses base on a probability from 0.01% to 90%
 Loss a random number of packets between 1 and 65535

11. 12
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Packet Effect
 Out of Order
 Base on a probability between 1% and 50%, a packet would be
moved from original location with an offset from 1 to 64 packets
 Duplicate Packets
 Base on a probability between 1% and 99%, a packet would be
duplicated for 1 to 20 copies.
 Fragmentation
 MTU can be set from 64 bytes to 1460 bytes
 Handle “Do Not Fragment” bit
 TTL Effects
 A packet’s TTL value could be decreased from 1 to 255
 An ICMP message may be returned to the originator of the packet

12. 13
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Link Fault
 Bit Error
 Toggle one bit every 102 to 1012 bits
 Toggle from 1 to 500 bit if Bit Error occurs
 Disconnect
 Disconnect network from 10 ms to 1 hour
 All packets are lost during “disconnection”

13. 15
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Congestion
 Congestion occurs every 1 second to 300 seconds
 A congestion lasts from 10 ms to 120000 ms
 Fixed latency from 0 ms to 8000 ms
 Packet loss probability from 0% to 90%
 Latency and Packet loss setting in Congestion
overwrite original setting when Congestion events
occur

14. 16
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Gateway-Bandwidth Control
 Bandwidth from 2.4 Kbps to 622 Mbps
 Packet Overhead Length from 0 to 1000 bytes
 Bandwidth Utilization from 0 % to 100 %
 Queue Size from 2 KB to 65 MB
 Queue Size Policy: Packet Mode or Byte Mode
 Queue Management: Drop Tail or RED (Random
Early Detection)

15. Summary

16. 18
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Pure Network
 Adv.
 環境單純, 可明確identify DUT defects
 可reproduce DUT defects
 Test Report 較為可靠客觀
 Disadv.
 只證明Interop 沒問題，Vender自己需進行 Real Network Test
 Open Issues
 QE-UAs, QE-Servers

17. 19
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Real Network
 Adv.
 可測得DUT 之Robustness
 最能滿足Vender需求
 Disadv.
 往往無法Reproduce DUT defects, 因為造成defects之Network
Impairments有太多種組合
 Test Report 較不客觀, 最多只能指出當時環境狀況
 增加了環境不確定度，加重測試事務與時程
 2個run * 3個QE System * 40 Test Cases + Unpredicted Network Status
 降低了通過率
 Open Issues
 QE-UAs, QE-Servers
 Network Provider
 Network Status Report

18. 20
The information contained herein is the exclusive property of ITRI and shall not be distributed, reproduced, or disclosed in whole or in part without prior written permission of ITRI.
Network Emulator
 Adv.
 可調整Impairments，使得Emulator Network 接近真實
 Impairments可組合，可給設定參數值
 可 Reproduce Network Status, 並Identify Defects
 Disadv.
 增加了測試複雜度，加長測試時程
 2個run * 2個phase(Network Env.) * 3個QE System*40 Test Cases
 降低了通過率
 Open Issues
 QE-UAs, QE-Servers
 Survey Impairments其組合與參數之合理值範圍