calculating the weights for an adaline using the delta rule - codeproject

Articles » General Programming » Algorithms & Recipes » Neural Networks
Calculating the weights for an ADALINE using the Delta
Rule
By Ben Aldhouse, 25 Jun 2007
Download source code - 20.5 KB
Download compiled application - 9.23 KB
Introduction
This is actually one of the simplest C++ programs I have ever completed. There really isn't much to it. It won't be of
interest to either accomplished C++ programmers, or to experts in Neural Networks. I hope however, it may be of
some interest to people who are roughly at the same stage as me in learning at Neural Networks, i.e., pretty much at
the start. It is also a fairly easy Visual C++ program to write. In other articles I have written, I have described how to
build the application up with a fair amount of detail, starting with describing how to deal with the Application Wizard.
In this article, I will assume the reader knows how to create an application shell using the Application Wizard, and how
to place controls on a form and attach variables to those controls, and how to make an application handler function
for a button control.
For those who aren't interested in the coding but would like to play with the calculator, the compiled application is
available by clicking on the link above. It is easy to use. You enter your starting weights, training set, desired output,
and value for eta, and press 'Go'. The program should stop running pretty quickly. The compiled program will run for

4.14 (3 votes)
one hundred passes, which, as far as I can tell, is fully adequate. If you scroll down the output box until the numbers in
column 'y' (the output) start to correspond with the numbers in column 'd' (the desired output), then the values
shown in the output box under 'w0', 'w1', and 'w2' on that line are the correct weights for the network. You can test
this by replacing the 'Starting weights' with these values and pressing 'Go' again. This time, every value in 'y' should
equal the corresponding value in 'd' right from the start of the output.
Background
I have been looking at Phil Picton's book 'Neural Networks', having had the book for some time and not getting
through it as fast as I would like. It has only been in the last few days, however, that I had the bright idea of working
through the iterative process used for finding the weights on an ADALINE. I wrote out the numbers on a piece of

paper, and did the sums in my head between serving customers at the supermarket where I work. I felt pleased
because I felt that I had got the process more or less right. However, the process wasn't fast enough for me to get
satisfactory results. People in shops can be very demanding, you know. Besides, I suck at mental arithmetic. So, on a
Saturday night, I set to and wrote this rather undemanding little program.
I am not going to try and describe the ADALINE here. I am only a learner. I will provide the appropriate Google search
for the reader here - ADALINE Neural Network description. I am sure there is something useful out there. If not, then
you could do worse than to buy Phil Picton's book.
Using the code
OK, very briefly, I set up an SDI using inheritance from CFormView. On the dialog, I added a number of text boxes for
the input, and one for the output, plus one button to set everything going. All the code of any interest is in the event
handler for the button CAdalineView::OnButtonGo() and one other member function I created called
CAdalineView::adaline( ).
void CAdalineView::OnButtonGo()
{
// TODO: Add your control notification handler code here
float d1, d2, d3, d4, eta;
float y1, y2, y3, y4;
float w10, w11, w12;
float w20, w21, w22;
float w30, w31, w32;
float w40, w41, w42;
float x10, x11, x12;
float x20, x21, x22;
float x30, x31, x32;
float x40, x41, x42;

float net1, net2, net3, net4;
float dw10, dw11, dw12;
float dw20, dw21, dw22;
float dw30, dw31, dw32;

float dw40, dw41, dw42;
eta = atof(m_strEta);
d1 = atof(m_strD1);
d2 = atof(m_strD2);
d3 = atof(m_strD3);
d4 = atof(m_strD4);
CString str="";
m_strOutput="";
////Starting data
w10 = atof(m_strW0);
w11 = atof(m_strW1);
w12 = atof(m_strW2);
int passes = 0;
int matches = 0;
x10 = atof(m_strX10);
x11 = atof(m_strX11);
x12 = atof(m_strX12);
x20 = atof(m_strX20);
x21 = atof(m_strX21);
x22 = atof(m_strX22);
x30 = atof(m_strX30);
x31 = atof(m_strX31);
x32 = atof(m_strX32);
x40 = atof(m_strX40);
x41 = atof(m_strX41);
x42 = atof(m_strX42);
while (passes <= 100){
///// input pattern 1
adaline(x10, x11, x12, w10, w11, w12, d1, net1, y1,
dw10, dw11, dw12, eta);

//// change weights from pass1 to pass 2
w20 = w10+dw10;
w21 = w11+dw11;
w22 = w12+dw12;
///// input pattern 2
adaline(x20, x21, x22, w20, w21, w22, d2, net2, y2,
dw20, dw21, dw22, eta);
//// change weights from pass2 to pass 3
w30 = w20+dw20;
w31 = w21+dw21;
w32 = w22+dw22;
///// input pattern 3
adaline(x30, x31, x32, w30, w31, w32, d3, net3, y3,
dw30, dw31, dw32, eta);
//// change weights from pass3 to pass 4
w40 = w30+dw30;
w41 = w31+dw31;
w42 = w32+dw32;
///// input pattern 4
adaline(x40, x41, x42, w40, w41, w42, d4, net4, y4,
dw40, dw41, dw42, eta);
UpdateData(FALSE);
//// change weights from pass4 to pass 1
w10 = w40+dw40;
w11 = w41+dw41;
w12 = w42+dw42;
passes++;
}
}
So, this function takes the data from the edit boxes and handles the iteration process, passing the data for each of the

four input patterns to the adaline function in turn. The weights are adjusted here, also. However, the values by which
the weights are adjusted are calculated in the adaline function.
void CAdalineView::adaline(float x0, float x1, float x2,
float w0, float w1, float w2,
float d, float net, float y,
float &dw0, float &dw1, float &dw2, float eta)
{
CString str = "";
//Calculate 'net'
net = x0*w0 + x1*w1 + x2*w2;
//Calculate output value by passing 'net' through
//a hard-limiter
if (net >= 0)
y = 1;
else
y = -1;

//Here the 'Delta Rule' is applied
dw0 = eta*x0*(d - net);
dw1 = eta*x1*(d - net);
dw2 = eta*x2*(d - net);
//And the output is shuffled off to the appropriate text box
//inputs
str.Format("%.2f\t%.2f\t%.2f\t",x0,x1,x2);
m_strOutput += str;
//weights
str.Format("%.2f\t%.2f\t%.2f\t",w0,w1,w2);
m_strOutput += str;
//d, net, y
str.Format("%.2f\t%.2f\t%.2f\t",d,net,y);

m_strOutput += str;
//weight adjustments
str.Format("%.2f\t%.2f\t%.2f\t\r\n",dw0,dw1,dw2);
m_strOutput += str;
}
Note that dw1, dw2, and dw3 are passed 'by reference' (indicated by the ampersand prefix). It is both easier and more
efficient to do this, allowing the variables to be assigned new values in the called function and have those new values
instantly available in the calling function (because both functions are looking at the same memory locations). This
precludes the various overheads incurred by passing the variables 'by value'. The other parameters are left as 'by value'
passes since they are not required to provide information back to the calling function. If you are looking for more
efficiency and want to keep with the principle of minimum privilege (i.e., the function being called can't interfere with
the variable in the calling function) that 'by value' passes provide, then you could have all of the variables passed 'by
reference', and declare all of them, other than the ones you specifically want to manipulate, in the calling function (i.e.,
dw0, dw1, as dw2) as 'const'.
The following snippet is where the Delta Rule is applied:
//Here the 'Delta Rule' is applied
dw0 = eta*x0*(d - net);
dw1 = eta*x1*(d - net);
dw2 = eta*x2*(d - net);
and here the output 'y' is calculated by passing the value in 'net' through a 'hard-limiter':
//Calculate output value by passing 'net' through
//a hard-limiter
if (net >= 0)
y = 1;
else
y = -1;
which, somehow, turns out to be a very friendly looking couple of lines of code to execute such a severe sounding
function.
Points of interest
If I hadn't written this program, and carried on trying to do the calculations on a piece of paper, I would have been

doing it till kingdom come, since I had been subtracting the weight changes (the dw values) from the weights (the w
values). My program showed very efficiently that this led nowhere fast. After a moment or two of thinking about it, I
changed the code (in the OnButtonGo function) to add the weight changes, and thankfully, it worked.
w20 = w10+dw10;
w21 = w11+dw11;
w22 = w12+dw12;
License
This article has no explicit license attached to it but may contain usage terms in the article text or the download files
themselves. If in doubt please contact the author via the discussion board below.
A list of licenses authors might use can be found here
About the Author
Ben Aldhouse
United Kingdom
Member
random and twisted
Permalink | Advertise | Privacy | Mobile
Web03 | 2.6.121031.1 | Last Updated 25 Jun 2007
Article Copyright 2007 by Ben Aldhouse
Everything else Copyright © CodeProject, 1999-2012
Terms of Use
Comments and Discussions
0 messages have been posted for this article Visit />the-weights-for-an-ADALINE-using-the-D to post and view comments on this article, or click here to get a print
view with messages.