Testing station analogs, binaries, and controls of a RTU at a single substation can be time consuming and error prone. Multiply these challenges by 600 substations and suddenly these challenges can seem insurmountable. This is where the power of software automation can deliver efficiencies and eliminate human errors. This paper summarizes the approach used and lessons learned when automating the point-to-point testing of the legacy D20 RTU. Automation included creation of test plans by sanitizing data from multiple sources, live testing automation with a text-based user interface for the tester, and the automatic recording of the test results.

1. Automating the Point-to-Point Testing of
Hundreds of Substations
Nathan Wallace, PhD, PE; Luke Hebert
8/24/2022

2. Agenda
2
Why
How
Results
Test
Plans
Test
Execution
Test
Analysis
1
2
3
?
?
?
What

3. Overall Why
3
SCADA: RTSCADA to ADMS
“Testing BKR1011 MW
You should see 11”
“Going to BKR1011 MW
… seeing .11”
Control
Site Specific
Test Plan
1 Sub
600+ Subs

4. (1) Test Plans - Why
4
600+ Subs
ADMS
RTSCADA
Migration File
Challenges
• What is the corresponding signal name in the RTU?
• Requires manually aligning indexes across all 3 places (+1 for RTU internal db)
• Have to manually select desired % of controls, analogs, and binaries
Site Specific Test Plan
Given
Need
Manual Creation: 2.5 hours
RTU Config

5. (1) Test Plans - How
5
600+ Subs
Software Creation: 15s + 10min QC
Migration File RTU Config
DPA Export
Formatted Site Specific Test Plan

6. (2) Test Execution - Why
6
600+ Subs
Challenges
• Multiple menus to navigate
• Navigation uses Ctl+X keys
• Repetitive manual work easy to lose focus
• Have to flip on tiny laptop in field b/t Test Plan and Terminal App
• Easy to leave station with points still forced. (True Story – 10pm work outing)
• Testing Controls Options: (1) Alarm light on Card or (2) Pulling SER from Relay
• What actual Card or Relay is control command associated with?
Manual: 35s/point
Test Plan
…

7. (2) Test Execution - How
7
600+ Subs
Formatted Site Specific Test Plan
PySerial
…

8. (2) Test Execution – How (Cont)
8
600+ Subs

9. (2) Test Execution – How (Cont)
9
600+ Subs
Binary Point

10. (2) Test Execution – How (Cont)
10
600+ Subs
Analog Point

11. (2) Test Execution – How (Cont)
11
600+ Subs
Controls (Super Magic)
Live Serial Traffic -> PCAP -> tshark -> Live Parse

12. (3) Test Analysis - Why
12
600+ Subs
Completed
Test Plan
Station A
Challenges
• Manually Determining Points %Pass, %Fail, %Untested Overall and by Type
• Manually Trending Across Stations searching for common failures
Manual: IT Depends… 
Completed
Test Plan
Station B
…
Completed
Test Plan
Station C

13. (3) Test Analysis - How
13
600+ Subs

14. Results
14 Future: Closed Loop (Stay Tunned)
600+ Subs
Without Automation With Automation
Test Plan 2.5 Hours 15 sec + 10min QC
Testing 35 sec / point 5 sec / point
Average Station Points 300
Test Plan 2.5 hours 0.17 hours
Testing 3 hours 0.42 hours
Total Time / Station 5.42 hours 0.58 hours
Total Stations 600
Total Time / Project 3250 hours 350 hours
406.25 days 43.75 days
Time Saved 2900 hours
362.5 days

15. 5
Nathan Wallace, PhD, PE
Thank You
Questions?