testing chapter

1. 10.0 DEBUGGING TECHNIQUES
 Introduction
 Rule of Thumb: Write good, bug-free code from start if you could
 Testing/Debugging embedded software is more difficult than application
software
 Post-shipment application problems are more tolerable than embedded (real-
time or life-critical) software

2. 10.0 DEBUGGING TECHNIQUES
 10.1 Testing on Host Machine
 Some reasons why you can’t test (much, if any) on target machine:
 Test early (target may not ready or completely stable)
 Exercise all code, including exceptions (real situations may be difficult to
exercise)
 Develop reusable, repeatable test (difficult to do in target environment, and
likelihood of hitting the same bug is low)
 Store test results (target may not even have disk drive to store results)

3. 10.0 DEBUGGING TECHNIQUES
 10.1 Testing on Host Machine – 1
 Basic Techniques
 Fig 10.1 –
 Target system on the left: (hardware-indep code, hardware-dep code, hw)
 Test system (on host) on the right: (hardware-indep code – same, scaffold – rest)
 Scaffold provides (in software) all functionalities and calls to hardware as in the
hardware-dep and hardware components of the target system – more like a simulator
for them!

5. 10.0 DEBUGGING TECHNIQUES
 10.1 Testing on Host Machine – 2
 Basic Techniques
 Fig 10.2 –

Radio.c -- hardware independent code

Radiohw.c – hardware dependent code (only interface to hw: inp() and outp()
supporting vTurnOnTransmitter() and vTurnOffTransmitter() functions

Inp() and outp() must have real hardware code to read/write byte data correctly -
makes testing harder!!
 Fig 10.3 –

Replace radiohw.c with scaffold, eliminating the need for inp() and outp() – both
are simulated in software – a program stub!!

6. Figure 10.2 A Poor Plan for Testing
/* File: radio.c */
void vRadioTask (void)
{
.
.
.
!! Complicated code to determine if turning on the radio now
!! is within FCC regulations.
.
.
.
!! More complicated code to decide if turning on the radio now
!! makes sense in our protocol.
If (!! Time to send something on the radio)
{
vTurnOnTransmitter (FREQ_NORMAL);
!! Send data out
vTurnOffRadio ();
}
}
-----------------------------------------------
(continued)

7. Figure 10.2 (continued)
/* File: radiohw.c */
void vTurnOnTransmitter (int iFrequencyValue)
{
BYTE byPower; /* Byte read from device controlling power. */
int i;
/* Turn on main power for the radio. */
disable_interrupts ();
byPower = inp (PWR_CONTROL_ADDR);
byPower |= PWR_CONTROL_RADIO_MAIN;
outp (PWR_CONTROL_ADDR, byPower);
enable_interrupts ();
/* Shift the frequency value out to hardware. */
for (i = 0; i < 16; ++i)
{
/* Send out the lowest bit of iFrequencyValue */
if (iFrequencyValue & 0x0001)
{
/* The data is a binary 1 */
/* Put a '1' on the data line; pulse the clock line. */
outp (FRQ_CONROL, DATA_1 & CLOCK_LOW)
outp (FRQ_CONROL, DATA_1 & CLOCK_HIGH);
}
(continued)

8. Figure 10.2 (continued)
else
{
/* The data is a binary 0 */
/* put a '0' on the data line; pulse the clock line. */
outp (FRQ_CONROL, DATA_0 & CLOCK_LOW)
outp (FRQ_CONROL, DATA_0 & CLOCK_HIGH);
}
/* Shift iFrequencyValue to get the next lowest bit. */
iFrequencyValue >>= 1;
}
/* Turn on the receiver. */
byPower = inp (PWR_CONTROL_ADDR);
byPower |= PWR_CONTROL_RADIO_RECEIVER;
outp (PWR_CONTROL_ADDR, byPower);
enable_interrupts ();
}
void vTurnOffRadio (void)
{
BYTE byPower; /* Byte read from device controlling power. */
/* Turn off main power for the radio. */
disable_interrupts ();
byPower = inp (PWR_CONTROL_ADDR);
byPower &= ~PWR_CONTROL_RADIO_MAIN;
outp (PWR_CONTROL_ADDR, byPower);
enable_interrupts ();
}
------------------------------------------- (continued)

9. Figure 10.2 (continued)
/* File: test.c */
void outp (int Address, BYTE byData)
{
#ifdef LET_USER_SIMULATE_HARDWARE
PRINTF ("program wrote %02x to %04x.", byData, iAddress);
#endif
#ifdef SIMULATE_HARDWARE
!! Remember that software wrote byData to iAddress
!! Update state of simulated hardware.
#endif
}
BYTE inp (int iAddress)
{
int iData;
#ifdef LET_USER_SIMULATE_HARDWARE
PRINTF ("program wrote %02x to %04x. Enter value.",
iAddress);
scanf ("%x", &iData);
#endif
#ifdef SIMULATE_HARDWARE
!! Figure out what the real hardware would return
#endif
return ((BYTE) iData);
}

10. Figure 10.3 Better Plan for Testing
/* File: radio.c */
void vRadioTask (void)
{
.
.
.
!! Complicated code to determine if turning on the radio now
!! is within FCC regulations.
.
.
.
!! More complicated code to decide if turning on the radio now
!! makes sense in our protocol.
If (!! Time to send something on the radio)
{
vTurnOnTransmitter (FREQ_NORMAL);
!! Send data out
vTurnOffRadio ();
}
}
-----------------------------------------------
(continued)

11. Figure 10.3 (continued)
/* File: test.c */
static BOOL fRadionOn;
static int iRadioFrequencyValue;
void vTurnOnTransmitter (int iFrequencyValue)
{
/* Record the state of the radio. */
fRadionOn = TRUE;
iRadioFrequencyValue = iFrequencyValue;
/* Tell the user */
printf ("Radio turned on with frequency %04x", iFrequencyValue);
}
void vTurnOffRadio (void)
{
/* Record the state of the radio. */
fRadioOn = FALSE;
/* Tell the user */
printf ("Radio now off");
}

12. 10.0 DEBUGGING TECHNIQUES
 10.1 Testing on Host Machine – 3
 Calling Interrupt Routines –
 Embedded systems are interrupt-driven, so to test based on interrupts
 1) Divide interrupt routines into two components

A) a component that deals with the hardware

B) a component of the routine which deals with the rest of the system
 2) To test, structure the routine such that the hardware-dependent component (A) calls
the hardware-independent part (B).
 3) Write component B in C-language, so that the test scaffold can call it
 E.g., Fig 10.4 –
 Hw component (A) is vHandleRxHardware(), which reads characters from the hw
 Sw component (B) is vHandleByte, called by A to buffer characters, among others
 The test scaffold, vTestMain(), then calls vHandleByte(), to test if the system works
[where vTestMain() pretends to be the hardware sending the chars to vHandleByte()]

13. Figure 10.4 Dividing Interrupt Routines into Two Parts
/* File: serial.c */
#define CR 0x0d
#define SIZEOF_CMD_BUFFER 200
BYTE a_byCommandBuffer[SIZEOF_CMD_BUFFER];
/* Queue to send message to command-handling task. */
extern unsigned long qidCommands;
void interrupt vHandleRxHardware (void)
{
BYTE byChar; /* The character we received */
int iHwError; /* Hardware error, if any */
iHwError = !! Get status from hardware;
if (iHwError == CHARACTER_RXD_OK)
{
/* We received a character; deal with it. */
byChar = !! Read byte from hardware;
vHandleRxByte (byChar);
}
else
!! Deal with hardware error
!! Reset the hardware as necessary.
!! Reset interrupt controller as necessary.
} (continued)

14. Figure 10.4 (continued)
void vHandleRxByte (BYTE byReceived)
{
static BYTE *p_byCommandBufferTail = a_ byCommandBuffer;
extern BYTE *p_byCommandBufferHead;
unsigned long a_ulMessage[4]; /* Message buffer. */
/* Advance the tail pointer and wrap if necessary */
++ p_byCommandBufferTail;
if (p_byCommandBufferTail == &a_ byCommandBuffer
[SIZEOF_CMD_BUFFER])
p_byCommandBufferTail = a_ byCommandBuffer;
/* If the buffer is not full . . . . */
if (p_byCommandBufferTail != p_byCommandBufferHead)
{
/* Store the character in the buffer. */
*p_byCommandBufferTail = byReceived;
/* If we got a carriage return, wake up the command-handling task. */
if (*p_byCommandBufferTail == CR)
{
/* Build the message . . . */
a_ulMessage[0] = MSG_COMMAND_ARRIVED;
a_ulMessage[1] = 0L;
a_ulMessage[2] = 0L;
a_ulMessage[3] = 0L;
(continued)

15. Figure 10.4 (continued)
/* . . . and send it. */
q_send (qidCommands, a_ulMessage);
}
}
else
{
/* Discard the character; move the pointer back. */
if (p_byCommandBufferTail == a_ byCommandBuffer)
p_byCommandBufferTail ==
&a_ byCommandBuffer[SIZEOF_CMD_BUFFER];
-- p_byCommandBufferTail;
}
}
--------------------------------------------
/* File: test.c */
void vTestMain (void)
{
BYTE a_byTestCommand[] = "THUMBS UPx0dSIMON SAYS THUMBS UPx0d";
BYTE *p_by;
.
.
/* Send each of the characters in a_byTestCommand */
p_by = a_byTestCommand;
while (*p_by)
{
/* Send a single character as though received by the interrupt */
vHandleRxByte (*p_by);
/* Go to the next character */
++p_by;
}
.
.
}

17. 10.0 DEBUGGING TECHNIQUES
 10.1 Testing on Host Machine – 4
 Calling the Timer Interrupt Routine
 Design the test scaffold routine to directly call the timer interrupt routine, rather than
other part of the host environment, to avoid interruptions in the scaffold’s timing of
events
 This way, the scaffold has control over sequences of events in the test which must
occur within intervals of timer interrupts
 Script Files and Output Files
 To let the scaffold test the system in some sequence or repeated times, write a script
file (of commands and parameters) to control the test
 Parse the script file, test system based on commands/parameters, and direct output –
intermixture of the input-script and output lines – into an output file
 The commands in the script cause the scaffold to call routines in the B (sw-indp)
component -- See Fig 10.5 and Fig 10.6 – for the cordless bar-code scanner

20. 10.0 DEBUGGING TECHNIQUES
 10.1 Testing on Host Machine – 5
 More Advanced Techniques
 Making the scaffold automatically control sequence of events – e.g., calling the printer
interrupt many times but in a controlled order to avoid swamping
 Making the scaffold automatically queue up requests-to-send output lines, by
automatically controlling the button interrupt routine, which will cause successive
pressing of a button to let the next output line be received from the hardware (the
printer interrupt routine). In this way, the hardware-independent software is controlled
by the scaffold, where the button interrupts serve as a switch
 The scaffold may contain multiple instances of the software-independent code, and
the scaffold serves as a controller of the communication between the instances –
where each instance is called by the scaffold when the hardware interrupt occurs
(e.g., the scanner or the cash register). In this way, the scaffold simulates the
hardware (scanner or register) and provides communication services to the software-
independent code instances it calls. – See Fig 10.7

22. 10.0 DEBUGGING TECHNIQUES
 10.1 Testing on Host Machine – 6
 Objections, Limitations, and Shortcomings
 1) Hard to test parts which are truly hardware dependent, until the target system
is operational. Yet, good to test most sw-independent parts on host (see Fig
10.8)
 2) Time and effort in writing scaffold – even if huge, it is worthwhile
 3) Having the scaffold run on the host and its RTOS – scaffold can run as low
priority task within the RTOS and have nicely integrated testing environment
 4) The hard to justify limitations – can’t tell in scaffold until the actual test
 Writing to the wrong hardware address – software/hardware interactions
 Realistic interrupt latency due to differences in processor speeds (host v. target)
 Real interrupts that cause shared-data problems, where real enable/disable is the key
 Differences in network addressing, size of data types, data packing schemes –
portability issues

24. 10.0 DEBUGGING TECHNIQUES
 10.2 Instruction Set Simulators
 Using software to simulate:
 The target microprocessor instruction set
 The target memory (types - RAM)
 The target microprocessor architecture (interconnections and components)
 Simulator – must understand the linker/locator Map format, parse and interpret it
 Simulator – takes the Map as input, reads the instructions from simulated ROM,
reads/writes from/to simulated registers
 Provide a user interface to simulator for I/O, debugging (using, e.g., a macro
language)

25. 10.0 DEBUGGING TECHNIQUES
 10.2 Instruction Set Simulators – 1
 Capabilities of Simulators:
 Collect statistics on # instructions executed, bus cycles for estimating actual times
 Easier to test assembly code (for startup software and interrupt routines) in simulator
 Easier to test for portability since simulator takes same Map as the target
 Other parts, e.g., timers and built-in peripherals, can be tested in the corresponding
simulated versions in the simulated microprocessor architecture
 What simulators can’t help:
 Simulating and testing ASICs, sensors, actuators, specialized radios (perhaps, in
future systems!!)
 Lacking I/O interfaces in simulator to support testing techniques discussed (unless
additional provision is made for I/O to support the scaffold; and scripts to format and
reformat files between the simulator, simulated memory, and the scaffold)

26. 10.0 DEBUGGING TECHNIQUES
 10.3 The assert Macro
 The assert is used (with a boolean-expression parameter) to check assumptions
 If the expression is TRUE nothing happens, if FALSE, a message is printed and
the program crashes
 Assert works well in finding bugs early, when testing in the host environment
 On failure, assert causes a return to the host operating systems (can’t do on
target, and can’t print such message on target – may not have the display unit)
 Assert macro that runs on the target are useful for spotting problems:
 1) disabling interrupts and spin in infinite loop – effectively stopping the system
 2) turn on some pattern of LEDs or blinking device
 3) write special code memory for logic analyzer to read
 4) write location of the instruction that cause problem to specific memory for logic
analyzer to read (the Map can help isolate which source code is the culprit!)
 5) execute an illegal op or other to stop the system – e.g., using in-circuit emulators

28. 10.0 DEBUGGING TECHNIQUES
 10.4 Using Laboratory Tools – Hardware-focused
 Lab tools help reveal hard-to-find, very infrequently occurring bugs
 Types useful to software engineers:
 Voltmeters (measure voltage diff); Ohmmeters (measure resistance/connectedness)
 Oscilloscopes (scopes) test events that repeat periodically – monitoring one or two
signals (graph of time v. voltage), triggering mechanism to indicate start of monitoring,
adjust vertical to know ground-signal, used as voltmeter (flat graph at some vertical
relative to ground signal), test if a device/part is working – is graph flat? Is the digital
signal coming through – expecting a quick rising/falling edge (from 0 – VCC or VCC –
0) – if not, scope will show slow rising/falling – indicating loading, bus fight, or other
hardware problem
 (See Fig 10.10, Fig 10.11, Fig 10.12, Fig 10.13, Fig 10.14)

34. 10.0 DEBUGGING TECHNIQUES
 10.4 Using Laboratory Tools – Hardware-focused - 1
 Logic Analyzer
 Like storage scopes that (first) capture many signals and displays them
simultaneously
 It knows only of VCC and ground voltage levels (displays are like timing diagrams) –
Real scopes display exact voltage (like analog)
 Can be used to trigger on-symptom and track back in stored signal to isolate problem
 Many signals can be triggered at their low and/or high points and for how long in that
state
 Used in Timing or State Mode

35. 10.0 DEBUGGING TECHNIQUES
 10.4 Using Laboratory Tools – Hardware-focused – 2
 Logic Analyzers in Timing Mode
 Find out if an event occurred – did cordless scanner turn on the radio?
 Measure how long it took software to respond to an interrupt (e.g., between a button
interrupt signal and activation signal of a responding device – to turn off an bell)
 Is the software putting out the right pattern of signals to control a hardware device –
looking back in the captured signal for elapsed time
 (See Fig 10.15 on response time)
 (See Fig 10.16 on elapsed time)
 (See Fig 10.17 – a typical Logic Analyzer with on-screen button, mouse, keyboard,
network adaptor, disk storage for storing configurations/settings, ribbon cables)

39. 10.0 DEBUGGING TECHNIQUES
 10.4 Using Laboratory Tools – Hardware-focused – 3
 Logic Analyzers in State Mode
 Captures signals only on clock-event occurring from the attached hardware
 Typical use: instructions read/fetched by microprocessor, data read from or written to
ROM, RAM, or I/O devices
 To do so, connect LA to address and data signals and RE/ signal on the ROM (or
RAM)
 If triggering is on rising edge of RE/ pin, address and data signals will be captured
 Output of LA, called trace, is stored for later analysis – see Fig 10.18
 LA can be triggered on unusual event occurrences, then capture signals therefrom –
especially for debugging purposes (e.g., writing data to wrong address, tracking a
rarely occurring bug, filtering signals for select devices or events)
 LA can’t capture all signals, e.g., on fetch from caches, registers, un-accessed
memory

41. 10.0 DEBUGGING TECHNIQUES
 10.4 Using Laboratory Tools – Hardware-focused – 4
 In-Circuit Emulators (ICE)
 Replaces target microprocessor in target circuitry (with some engineering)
 Has all the capabilities of a software debugger
 Maintains trace, similar to that of an LA’s
 Has overlay memory to emulate ROM and RAM for a specified range of address within
the ICE (rather than the system’s main ROM or RAM) – facilitates debugging
 ICE v. LA

LA’s have better trace and filtering mechanism, and easier to detail and find problems

LA’s run in timing mode

LA’s work with any microprocessor – ICE is microprocessor-specific

LA’s support many but select signals to attach, ICE requires connecting ALL signals

ICE is more invasive

42. 10.0 DEBUGGING TECHNIQUES
 10.4 Using Laboratory Tools – Hardware-focused – 5
 Hardware Peculiarities that Make Debugging Difficult
 Inter-pin distances/intervals for attaching probes – getting smaller
 Providing sockets for debugging hardware – simply increases product size
 ASIC’s encase signals that are hard to probe and track using LA’s or ICE’s
 Use of RISC architectural designs makes it difficult to track when read/write happen in
on-board (microprocessor) caches – different from the external RAM or ROM
 Increasingly necessary to know the lab tool chain as it influences the design of product

43. 10. DEBUGGING TECHNIQUES
 10.4 Using Laboratory Tools – Hardware-focused – 6
 Software-Only Monitors
 Monitors allow running an embedded system in the target environment, while
providing debugging interfaces on both the host and target environments
 A small portion of the Monitor resides in the target ROM (debugging kernel or
monitor):

The codes receives programs from serial port, network, copies into target’s RAM, and run it
with full debugging capabilities to test/debug the programs
 Another portion of monitor resides on host – provides debugging capability and
communicates with the debugging kernel over serial port or network, without hardware
modifications
 Compiled, linked (may be located into Map) code is downloaded from the host (by the
portion on the host) to the target RAM or flash (received by the kernel)
 Other designs: ROM Emulator interface and JPAG comm. port on the
target processor
 (See Fig 10.19)