This is my POC report to our customer - AWB and I use my professional skill to present them how to make their testing be automatically with whole stuff!

1. Standards of Excellence
AWB WiMAX Indoor
CPE Performance
Test
- Requirement &
Architecture
May 11, 2011 Richard Hsu
QualiSystems Proprietary & Confidential

2. AWB WiMAX / WiFi Indoor CPE Performance Test
- Architecture
Server IP : 1.1.4.101 Installed Products on Server:
• Remote Runner (for
Remote DUT script execute)
Internet
WiMAX Server:
1.1.4.102 WiMAX
Throughput Test Procedure:
AWB Indoor CPE
1. Set Rate on WiMax Server.
2. Set WiFi Mode.
3. Iperf from Server to Client 1.
WAN : 172.30.104.131 4. Iperf from Server to Client 2.
5. Iperf from Client 1 to Server.
LAN : 192.168.1.1 6. Iperf from Client 2 to Server.
Ethernet 2 WIFI Client 2 :
Ethernet 1 Ethernet : 192.168.1.141
Installed Products on Client2: WIFI : 192.168.1.142
• Studio (for Create) Ethernet
• Runner (for Execute) Client 1 : Installed Products on
• License Server (for Control Ethernet 1: 192.168.1.140 Client1:
Floating license) Ethernet 2: 1.1.4.100 • Remote Runner (for
Remote DUT script execute)
QualiSystems Proprietary & Confidential

3. HOST
Installed Products on Host PC:
• Studio (for Create)
Ethernet IP: 192.168.0.101
• Runner (for Execute)
WiFi IP: 192.168.1.100
• License Server (for Control
Floating license)
AP LAN IP: 192.168.1.1
Installed Products on DUT PC:
Ethernet IP: 192.168.0.121
• Remote Runner (for Remote
WiFi IP: 192.168.1.120
DUT script execute)
DUT 1
QualiSystems Proprietary & Confidential

4. Testing Requirements for testing automation in the future
1. AWB uses Iperf to send packets for testing performance of
CPE.
2. AWB wants to let every scenario be testing component and
engineers can choose which test to run alone or run at the
same time.
3. AWB also wants to schedule their tests and can set how
much time to run for any tests on every test station.
4. AWB’s engineers can run test cases easily and increase
testing coverage. Their engineers can focus on problem-
solved without spending much time to wait for testing done.
QualiSystems Proprietary & Confidential

5. Testing Challenges & TestShell Solution
AWB Testing Challenge TestShell Solution
• Current test process just can run • All test cases are reusable as
manually. components that is easy to create new
• Test case must be re-builded once one.
there are new requirements for any • The test cases are builded from
change or update. different team member's abilities and
efforts.
• Bottleneck of manual testing • TS provide parallel testing
(1 operators → 1~3 DUTs) and (Runner →10~ 20 DUTs)
cannot hire more test operators.
• Testing coverage is limited by • Scheduling execution for tests and
manpower. regressions (running 24/7)
• Resource Management
QualiSystems Proprietary & Confidential

6. Testing Challenges and TestShell Solution
Pegatron Testing Challenge TestShell Solution
• Manage and maintain test • TestShell-Foundation is a centralize
case/result/tool from different teams. DB for all test cases, test results and
managing all DUT’s, Tests, Regressions
and Documentation (specification,
requirements, test plan).
• User Authorization
• Generate reports easily • Generate/export reports automatically.
• Real time dashboard for viewing trends
and statistics of test results
QualiSystems Proprietary & Confidential

7. 1st phase POV – Testing Architecture / Requirements
Test Scenario:
1. Client 1 to Client 2.
AWB Indoor CPE
Reruirements:
LAN 1. We don’t need to connect to WAN
WiFi due to the fact that we cannot
Ethernet connect to WiMAX.
2. When we do POV, we needs
including CPE setting by website
Client 1 Client 2 configuration GUI recording.
3. We can test couple WiFi channels
to approach testing automation.
4. Send packets by Iperf tool.
QualiSystems Proprietary & Confidential

8. TestShell Test Structure
Login to Router
Load Configuration
Wait for reboot
Launch
iPerf Login to Router after reboot
Server Case
WiFi to LAN LAN to WiFi
QualiSystems Proprietary & Confidential

9. Test execution
QualiSystems Proprietary & Confidential

10. Test execution – Production Runner View
QualiSystems Proprietary & Confidential

11. Test Parameters
1. Configuration File to Load
2. Direction
1. LAN to WiFi
2. WiFi to LAN
Same test is running with different configuration
QualiSystems Proprietary & Confidential