Data Science - Part VIII - Artifical Neural Network
Neural Network Project Identifies Sine & Square Waves
1. Laboratory #6:
Neural Network Project
Kyle Villano, Tim Kindervatter and Kelvin Resto
ELC 343-01
Dr. Jesson
The College of New Jersey
December 10th, 2014
2. 2
Table of Contents
Introduction ............................................................................................................................... 3
Problem Description................................................................................................................... 3
Materials and Procedure ............................................................................................................. 5
Results .........................................................................................................................................5
Discussion .................................................................................................................................... 8
Conclusion ................................................................................................................................... 9
Appendix…....................................................................................................................................11
3. 3
Introduction
In this lab we used PSoC 5LP board, along with its creator software in order to create a
neural network program. This neural network needed to be able to identify types of signals that
are input to the board based on their Fourier transforms, specifically a sine and a square wave.
The network would also use a specific weighting system that aids in teaching the neural network
to learn how to identify these aforementioned input signals. In order to do this, we wrote C code
that would simulate the learning of this neural network, and then implemented it on the PSoC
5LP board. Once the code was written, we used the oscilloscope and waveform generator to
produce inputs to the system and observe the corresponding outputs, which would allow us to
finally confirm that it indeed functioned in the desired manner. After completing the design of
the code, we finished our work by observing that the neural network did indeed correctly identify
the two types of input waves. However, our learning algorithm did seem to have some trouble,
which we believe can be attributed to the incorrect assigning of weight values to certain nodes.
Problem Description
The given task in this lab was stated quite simply, that we were to create a neural network
program that was able to learn and adapt to identify two types of given input signals. The
network would need to be based mainly upon assigning weight values to various nodes, which
would then allow the program to be able to learn from its mistakes and correctly identify square
and sine input waves after many instances. The lab did give some guiding information, telling us
that the network needed 64 input nodes that were obtained by performing a 128-point FFT on the
input signal, which would then be connected to 24 points in the hidden layer that allowed for the
two output points to be formulated and produced by the board. To teach this board to perform
such a task, we needed to create both a training mode (to allow the board to learn what output
4. 4
bits result in which answers) and a identification mode (to verify that it worked in the correct
manner). Appendix A attached to the lab sheet provided us with much relevant information
regarding how to attack this problem, such as a diagram of the training procedure and methods
that we could use to assign weight values to each node in our program. The aforementioned node
diagram describing the training procedure can be seen below in Figure 1.
Using this information, along with background knowledge and examples of neural networks
found online, we became well-equipped to try to solve this complex problem.
Figure 1: Illustration of Training Procedure
5. 5
Materials and Procedure
Materials Used:
• PSoC Creator Software
• CY8CKIT-001 with PSoC 5 LP Board
• PSoC Programmer
• Oscilloscope and corresponding probes
• Waveform generator
Procedure:
• Perform background research on neural networks and view working examples
• Create training mode for network based on back-error propagation techniques
• Run simulation protocol to allow network to learn and determine weights for nodes
• Create identification mode for network to verify accurate functionality
• Run network and observe output results to verify success of program
Results
We first started off this lab by researching neural networks and working examples. In the
realm of computer science, artificial neural networks are essentially forms of computer
architecture inspired by biological neural networks, such as the brain, and are used to estimate
or approximate functions that depend on a large number of inputs that are generally unknown.
An artificial neural network is made up of systems of interconnected neurons that can compute
values from inputs, and are basically capable of machine learning as well as pattern recognition
due to their adaptive nature.
The next part of the lab involved creating a trading mode for the neural network using
back-error propagation techniques. By giving the network a set of input, it is trained to give a
6. 6
desired response. If the network gives the wrong answer, then the network is corrected by
adjusting its parameters so that the error is reduced. During this correction process, one starts
with the output nodes and propagation is backward to the input nodes and is repeated until all the
weight values converge. For this lab the weight values were calculated by using a C++ program
since a PSoC board does not have the capacity to calculate the weights. Even though the weights
were calculated on a computer the program took extremely long to run (approximately 4 hours!).
The final part of the Lab involved taking a 128-point FFT of a signal and feeding the
generated points into the neural network. The PSoC board, programed with the neural network
(Figure 2), had to be able to recognize two distinct input signals scaled between 0 and 1. The
first signal was a sinusoidal input with a value of +1 for node 0 and -1 for node 1 in the output
layer. The other signal was a square wave input with a value of -1 for node 0 and +1 for node 1
in the output layer. When a sine signal was generated from a waveform generator at 1 volt peak
to peak at 100 Hz the PSoC board recognized the input signal through its FFT inputs which were
fed through the neural network and displayed ‘Sine’ on the LCD screen (Figure 3). Also when a
square wave signal was fed to the PSoC board the LCD screen displayed ‘Square’ (Figure 4).
Thus our neural network was a success and the board was able to recognize both input signals.
7. 7
Figure 2: Neural network block diagram with ADC converter and LCD
Figure 3: Neural network recognizing sine wave
8. 8
Figure 4: Neural network recognizing square wave
Discussion
In this project, we were instructed to implement a neural network that is able to recognize
a sine wave and a square wave, both of unity amplitude. This is accomplished by taking a 128-
point FFT of the signal and feeding the generated points into the neural network. This neural
network uses a three-layer structure, which means that there is a set of nodes that accepts the
FFT points as input, a middle hidden layer with a transfer function that alters the output of the
first layer, and finally an output layer that forms the waveform that is desired.
The network is told from the outset to expect a certain output from the final layer of
nodes given a certain input into the input layer. In our case, we want the output to be the
waveform we wish to generate, and we wish the input to be the FFT of that waveform. The
learning algorithm uses values called weights to calculate the error in the output nodes (in other
words, how close it got to the expected output), and then uses that error value to adjust the
9. 9
weights in an attempt to get closer to the desired output. Each node in the middle and output
layers has certain weights associated with the node that it is receiving an input from. We
assigned arbitrary weights at first to give the algorithm a starting place. Subsequently, using
formulas given to us, we implemented an algorithm in C++ that calculates the error in the output,
and then uses that error value to adjust the values of the weights. As a result, every time the
learning algorithm is run, it gets closer and closer to the correct output value until all of the error
values converge to 0, which means we have the exact output we desire.
Our neural net was able to give a reasonable output for the square wave, but was unable
to reproduce a sine wave perfectly. In order to fix this, we would need to run the learning
algorithm at least one more time to recalculate the weights. Unfortunately, this process takes
many hours and we did not have the time to do this. If we had the time, the algorithm would be
able to adjust the weights to give an adequate output.
Conclusion
In this lab, we learned how to work with and write very high-level C code, and how one
can simulate and verify these results using the PSoC 5LP board. We also learned how to design a
functioning neural network, and how to effectively code and implement such a program in
regards to solving microcomputer projects. We also were able to ascertain information regarding
various errors that could occur during the coding process, and how to identify and correct them
with the software, hardware, and various debugging methods.
We were able to incorporate knowledge and code used in many examples that we
researched online in completing our project. These newly learned skills (along with knowledge
retained from previous projects) then allowed us to code and analyze the two routines that we
needed to successfully implement in order to create this neural network. We were also able to
understand the purpose of the many functions and variables utilized in our network programs,
10. 10
but perhaps more importantly, we were able to identify the ways in which we needed to alter
these components to fit the necessary tasks for our overall project.
In performing the project, we were initially overwhelmed with the scope of what we were
tasked with doing. However, through much background research and the studying of examples,
we learned the various concepts and ideas that a neural network contained, and how to
implement these concepts into our C code. We did come across a good amount of trouble when
developing the learning algorithm, as the weights assigned to various nodes did not seem to be
correctly allowing our network to learn to identify signals, among other things. We were also
surprised by the amount of time that the learning routine needed to run in order to be able to
correctly identify these input signals, as our program ran routines for several hours before it
could be used to identify a square and sine wave.
Overall, the neural network is a program with wide-reaching applications in the field of
microcomputer systems. For example, neural networks can be used in items ranging from home
appliances to entire power grids, as the learning algorithms will self-adjust to allocate power
more efficiently based upon observations of usage and intensity. Therefore, it is quite apparent
that the knowledge and experiences gained from successfully completing this project will also
prove to be useful in the future. This lab is also important in that it helps us understand how
neural networks are implemented in various programs, as well as the many applications that this
type of network can be used for in order to further our knowledge in this field and help us solve
future electrical and computer engineering projects.
11. 11
Appendix: Code for Neural Network
#include <device.h>
#include <math.h>
// Creates the complex structure that will be used throughout the code. There
// is one real part and one imaginary part.
struct cmpx{double real; double imag;};
// Initializes COMPLEX to complex structure type
typedef struct cmpx COMPLEX;
// This function performs the FFT of an input sample array
void FFT(COMPLEX *Y, COMPLEX *w ) // input sample array
{
COMPLEX temp1, temp2; //Temporary storage variables
int i, j, k; //For loop counter variables
int upper_leg, lower_leg, leg_diff; //Index of upper/lower butterfly
leg and difference between upper/lower leg
int index, step; //Index and step between twiddle factors
leg_diff = 64; // starting difference between upper and lower butterfly
// leg
step = 1; // Step between values in twiddle factors array
// for 128-point FFT
// Fills up array
for(i=0; i<7; i++)
{
index = 0;
for(j=0; j<leg_diff; j++)
{
for(upper_leg=j; upper_leg<128; upper_leg+=2*leg_diff)
{
lower_leg = upper_leg + leg_diff;
temp1.real = Y[upper_leg].real + Y[lower_leg].real;
temp1.imag = Y[upper_leg].imag + Y[lower_leg].imag;
temp2.real = Y[upper_leg].real - Y[lower_leg].real;
temp2.imag = Y[upper_leg].imag - Y[lower_leg].imag;
Y[lower_leg].real = temp2.real * w[index].real -
temp2.imag * w[index].imag;
Y[lower_leg].imag = temp2.real * w[index].imag +
temp2.imag * w[index].real;
Y[upper_leg].real = temp1.real;
Y[upper_leg].imag = temp1.imag;
}
index += step;
}
leg_diff = leg_diff/2;
step *= 2;
}
// bit reversal for re-sequencing data
j = 0;
for(i=1; i<127; i++)
16. 16
3.03859, -9.87563, -5.60347, 0.449905, -0.887437, 0.593624, 0.886812,
1.00681, -0.203718, -0.308984}};
/*_________End Weights ji______________________*/
/*___________Begin Weights kj__________________*/
double wkj[2][24]={{0.0644177, -1.66273, -3.62619, -10.4896, 0.16428, -
12.4576, 8.8171, -12.322, -0.289505, -0.392423, -0.490231, 14.4188, -
0.747429, 0.102296, 0.187079, -2.74119, -3.31169, -0.201028, -18.5548,
0.162079, -0.578729, -8.16069, -11.4766, -0.315152},
{0.558525, 2.07238, 3.5664, 11.1896, 1.20781, 12.7963, -7.48796, 12.2755,
0.781586, 1.76665, 0.98428, -12.8118, 1.69934, 1.06168, 0.817385, 3.44866,
3.45631, 0.487867, 18.3783, 0.579047, 0.610616, 9.05575, 11.7729, 1.15769}};
/*_____________End Weights kj___________*/
ADC_Start();// Begin analog to digital converter
ADC_StartConvert();// Begin analog to digital conversion
LCD_Char_Start(); //Start LCD display
LCD_Char_ClearDisplay();// Clear LCD display
LCD_Char_Position(0,0); // Set LCD character position to point 0,0
double magnitude[64];
COMPLEX wave1[128];
double xi[64];//values in nodes in layer 1 (switches for signal 1/2)
double xj[24];//values in nodes in layer 2
double xk[2];//values in nodes in layer 3
double sj[64];//intermediate sj
double sk[24];//intermediate sk
int i;//nodes in layer 1
int j;//nodes in layer 2
int k;//nodes in layer 3
int a;//variable for magnitude
int b = 0;//switch reading variable
int c = 0;//switch reading variable
int d = 0;//switch reading variable
double sumj = 0;//actual values of sj
double sumk = 0;//actual values of sk
// GENERATES TWIDDLE CONSTANTS
// 64 complex points
double arg;
COMPLEX z[64];
arg = 2 * 3.141592654 / 128;
for(i=0; i<64; i++)
{
z[i].real = cos(i * arg);
z[i].imag = -sin(i * arg);
}
//end initialization
17. 17
for(;;)
{
/*______LCD Message_____________*/
LCD_Char_PrintString("Press to Start");// Displayed on LCD on start up
while(d == 0)
{
c = Pin_1_Read();
b = Pin_2_Read();
if (c != b)
{
d = 1;
}
}
d=0;
while(d == 0)
{
c = Pin_1_Read();
b = Pin_2_Read();
if (c == b)
{
d = 1;
}
}
LCD_Char_ClearDisplay();
d = 0;
wave1[0].real = 1.0*ADC_GetResult16();
CyDelay(1);
/*______________Wave 1_____________*/
for(i = 0; i < 128; i++)
{
wave1[i].real = 1.0*ADC_GetResult16();
if(wave1[i].real < -100)
{
wave1[i].real =1.0- (-wave1[i].real)/100.0;
}
else if(wave1[i].real > -100)
{
wave1[i].real =1.0- (wave1[i].real)/100.0;
}
CyDelayUs(15);
wave1[i].imag = 0;
}
/*_________End Wave 1_______________*/
CyDelay(10);
/*_________FFT Input Signal**********/
FFT(wave1, z);
for(a = 64; a <128; a++)//calculate the magnitude with 1/2 the 128 points
{
magnitude[(-64 + a)] =
sqrt(wave1[a].real*wave1[a].real+wave1[a].imag*wave1[a].imag);
}
for(i = 0; i <64; i++)
{
xi[i] = magnitude[i];
18. 18
}
/*___________________End FFT_____________*/
CyDelay(10);
/*________________Calculate Outputs____________*/
for (j = 0; j <24; j++)
{
sumj = 0;
for (i = 0; i < 64; i ++)
{
sumj = sumj + wji[j][i]*xi[i];//finding the sum of wji*xi
}
sj[j] = sumj/64.0;//putting the sum sj into its array dividing by
64
xj[j] = tanh(sumj/64.0);//xj = the tanh of sj
}
for (k = 0; k < 2; k++)
{
sumk = 0;
for (j = 0; j < 24; j++)
{
sumk = sumk + wkj[k][j]*xj[j];//find the sum of wkj*xj
}
sk[k] = sumk/24.0;//putting the sum sk into its array dividing by 24
xk[k] = tanh(sk[k]/24.0);//xk = the tanh of sk
}
for(k = 0; k <2; k++)
{
if (xk[k] < 0)
{
xk[k] = -1;//rounding function down
}
else if(xk[k] > 0)
{
xk[k] = 1;//rounding function up
}
}
/*__________End Calculations*___________*/
CyDelay(10);
/*__________Output waveform__________*/
if((xk[0] == 1)&&(xk[1] == -1))
{
LCD_Char_ClearDisplay();
LCD_Char_PrintString("Sine");// If sine wave is inputted via wave form
//generator than LCD will display Sine
}
else if ((xk[0] == -1)&&(xk[1] == 1))
{
LCD_Char_ClearDisplay();
LCD_Char_PrintString("Square");// If square wave is inputted via wave form
//generator than LCD will display Square
}
19. 19
}
/*_______________End Output______________*/
/*___________LCD Message____________*/
while(d == 0){
c = Pin_1_Read();// Read pin 1, store value in c
b = Pin_2_Read();// Read pin 2, store value in b
if (c != b){
d = 1;
}
}
d=0;
while(d == 0){
c = Pin_1_Read();// Read pin 1, store value in c
b = Pin_2_Read();// Read pin 2, store value in b
if (c == b){
d = 1;
}
}
LCD_Char_ClearDisplay();// Clear LCD screen
d = 0;
/***********End LCD Message**********/
}
return;
}
