This document describes a flexible software test framework called Flex Test that was developed to overcome common problems with software test systems. Flex Test allows tests to be easily configured and run across different instruments. It stores all test parameters, data, and results in an organized format. The interface provides tabs to switch between different pre-programmed tests or to queue multiple tests. Configuration files separate test setup from execution. Flex Test notifies users when tests complete and generates detailed reports. Its instrument drivers abstract commands to support changing instruments and it can simulate connections during development. The developer seeks feedback on features and tests of interest.

1. Flex Test: When You Care Enough to Test
The VERY BEST
Matthew J. Bradley
Bradley Technologies
December 28, 2016
1. Introduction
After developing tests (particularly in the RF industry) for over 20 years, I have come to some
conclusions that are probably not too surprising:
1) Software is usually not re-useable from project to project. If it is re-useable,it usually is not easy.
2) Software often changes dramatically throughout the product cycle. This is because different
equipment and often different objectives are involved in each step.
3) Data is usually not going to be in the format that you want.
4) Changing instruments requires nothing short of an act of Congress. I have seen people struggle
with 40 year old spectrum analyzers because no one wants to touch the software that runs the test
system.
5) Everyone says they want traceability. Almost no one will wait for it before going into
production.
Finally, everyone says they want to solve these problems. But no one actually wants to pay for the
solution or take the time to solve it and so it remains an issue.
As a result, I decided to develop the test framework that would overcome these problems. I have
implemented it with a few customers and they have been extremely pleased. I am expanding the types of
tests that it performs as well as the instruments supported. I hope you will consider it to meet your
engineering and production needs.
Please note that development was performed primarily in LabVIEW.
2. User Interface
Figure 1 shows the engineering interface, ready to perform a harmonics test. Instrument parameters,test
conditions, and limits are all stored in a configuration file. As a result, the exact same test can be called
from another test executive (such as TestStand). Furthermore, by using multiple configuration files,
different conditions or limits can be tested with the same software.

2. Figure 1: Flex Test showing the engineering interface forthe Harmonics Test. Note that although the plot legend
indicates that the one half harmonic(and limit) will be tested, the actual harmonics testedwill be determinedby the
configuration file.
Tabs for additional tests (such as RF calibration, Gain Compression, S Parameters,and more) are also
shown. This enables the operator to switch between tests as needed for hardware validation and
verification, repair, or other tasks as needed.

3. It may be desirable, however, to have the ability to queue up severaltests and run them consecutively.
This can be done through the Multi Test tab, as shown in figure 2.
Figure 2: Not only does this allowthe operator to setupa list of tests to run, note that it also gives you the option to set a
warm up time before executing each test.
In addition, some tests simply take a long time to execute. Generally, this requires operators to frequently
check on systems to determine if the tests have completed, failed, or stopped executing. Failure to check
on these systems can lead to lower test system utilization. But, with Flex Test,this does not have to be a
problem. Text messages can be sent to groups of users to notify them when tests pass,fail, or abort. In
my experience, operators appreciate this feature most of all.
Naturally, the tests themselves re-use a great deal of code. Furthermore, they are highly structured. This
decreases development time dramatically. Typically, a new test can be built and ready to go in two days.
This is even true for custom tests that are specific to a single product. Existing tests (such as the
harmonics test shown below) require even less time to verify.

4. 3. TestConfiguration
Configuring the tests is straightforward. For example, configuration screens for the harmonics test are
shown in figures 3 and 4. Other tests have similar appearances. This is separated from the test interface
to give our clients more control over who can change configuration files.
Figure 3: Although the format of configuration files can be customizedas needed, we have triedto make them as general
as possible. For example, multiple frequency bands can be specifiedas shown and the Harmonics measurement cluster
(in the lower left) can have piecewise linearlimits.

5. Figure 4: This is the same as figure 3, but in this case we are showing that we can test for more than one harmonic at a
time. In figure 3, we showed the half harmonicwhereas in this case we are showing the first harmonic. There is no limit
to the number that can be tested.
4. Output Data Files
When I have shown this in the past, the next question that is normally asked concerns the output. There
are actually a number of files that are created,including:
 Copies of all RF calibrations
 The configuration file
 A record of all instruments (including model numbers and serial numbers)
 The raw data (in Excel compatible format)
 A summary file (for some tests,also in Excel compatible format)
 A PDF report (optional). This is typically distributed to customers.
All of these files are placed in the same folder which is specific to the UUT model number, serial number,
and the test time. The configuration file and calibration file are stored so that tests can be analyzed or
repeated even if the configuration changes or the system is re-calibrated.

6. A page from a sample report is shown in figure 5. Reports are frequently customized to individual client
and customer needs. We take special care to ensure that the reports are precisely what you want and
need.
Figure 5: One sample page of a test report. Reports are created from the data files, typically using VBA, and are
publishedto PDF format. Frequently, they include a mix of both plots and text data. We have found that our clients’
customers have been very pleasedwith both the level of detail and the clarity of presentation.

7. 5. Instrument Drivers
I have also taken special care in the architecture of the instrument drivers that are used by Flex Test. As
stated earlier, the inability to easily change instruments to meet new requirements can present major
obstacles to the success of software. Quite frankly, I have found that many of the available drivers are not
adequate. Although there are some exceptions, many do not include the functionality that I need or have
significant errors. For example, I know of one oscilloscope driver that sets both the time per division as
well as the time for the entire sweep within the same vi. Which takes precedence when they conflict?
More importantly, why would you have one vi for both?
As a result, I have abstracted the instrument architecture. As an example, let us consider a spectrum
analyzer. From the perspective of the harmonics test,we do not set the center frequency for a specific
model. Rather,we simply set it for a generic spectrum analyzer. The code determines which spectrum
analyzer is being used and sends the appropriate commands. Without dwelling on too many details, rest
assured that it is fast and can support multiple instruments of the same type in the same test (such as
power supplies).
There is always (or at least should be!) concern, however, when replacing instruments in terms of
capabilities. I once came across a test system where they had replaced a spectrum analyzer with a
different model but did not realize that the new model could not operate at the same resolution bandwidth
settings as the old. This caused enormous problems for them. As a result, our instrument drivers have a
“Verify” mode. When turned on, this will check each setting as it is made and confirm that the
instrument is configured properly. It will also check to see if there are any error conditions present on the
instrument. This can be turned on or off quite easily. I usually leave it on, but have turned it off if it
impacts test time.
Finally, software developers often find that the instruments they need are unavailable. RF
instrumentation can be quite expensive and the equipment may be needed elsewhere. Therefore,our
instrument drivers have a “Live” mode option. When turned on, commands and queries will be sent to
the instruments. When turned off, they are not – but work can continue for the rest of the test. This can
be set for individual instruments or for all instruments in a system.
6. Conclusion
Not surprisingly, there are more details than I could discuss in this description. Some would be technical
in nature whereas others would be financial.
I would appreciate hearing from you. In particular, I am interested in:
 Which instruments do you need supported?
 What tests would you like to see?
 What other features would you want to include?
Please feelfree to send them to me at mbradley@brad-tech.com or call me at (707) 837-2893.