Download Lab 4 - De Montfort University

Bob John and Ralph Birkenhead – De Montfort University, Leicester, UK [email protected]
Lab 4
Learning outcomes
You will be able to construct more complicated networks and test the effect of
the use of different transfer functions. You will be able to describe the network
object used by Matlab more fully and you will be able to access more
information about a network. There will be an assessment this week based on
setting up and training a network and then discussing if it generalizes.
We will now construct a network with two neurons in the first layer and then another
layer with one neuron in it. We will then try to model the data from
perceptronData3.txt with this network. We will see that it doesn't train all that easily
and then modify it to get a better fit.
So get the data from perceptronData3.txt into a matrix in the workspace and set up the
IN matrix and the TARGET matrix just as you did in the last lab. Recall that IN is a
matrix with two rows –the first row is x-coordinates and the second is y-coordinates.
The TARGET matrix is actually a vector of target values that the function we are
modelling takes at the corresponding x-y points. In the case of our data the target
values are either 0 or 1. Again you will need to find the max values and the min
values of the x and y co-ordinates, in order to pass the parameters to the network
creator.
In what follows I have assumed that the matrix dmin has the min values and dmax has
the max values – just as in the last lab.
net = newff([dmin(1) dmax(1);dmin(2) dmax(2)],[2
1],{'logsig' 'purelin'});
What has this command done? Net is a network object, returned by the function
newff. The first parameter to newff is a 2x2 matrix. The input to our function is a
vector x-y value. The net wants to know how much scaling it needs to do – so we give
it a min-x value and max-x value as the top row of the 2x2 and a min-y max-y as the
bottom row. Because there are two rows in this matrix the net will be told that it is
modelling a function of two variables. The next entry is a vector telling the newff how
many neurons are in each layer of the net. We do not tell newff how many layers – it
can tell that we want two layers because we say we want 2 neurons in the first hidden
layer and 1 neuron in the output layer. Then we give as a parameter an array of
transfer function names. For good reasons we will have the transfer in the first hidden
layer as logsig and we use linear in the second hidden layer.
We have created the network – let's initialise it.
net=init(net);
This has set up initial weights and biases – exactly how depends on some parameters
that were set up by default by newff when it created net. If we don't like the defaults
then we can set them ourselves. At present let's use defaults where possible for
convenience.
Bob John and Ralph Birkenhead – De Montfort University, Leicester, UK [email protected]
Now we will train the network – using train.
net=train(net,IN,TARGET);
We have trained the net on the IN inputs against the target TARGET. But has it
worked – use sim to find the errors:
err=TARGET-sim(net,IN)
As you remember from last week sim tells me what the network computes, by taking
the difference between TARGET and the simulated value. I could plot the errs vector
to get a visual idea of how good the net is.
bar(err)
Neural nets depend on the initialisation values – they do not always build to the same
outcome. Repeat the sequence of commands starting from
net=init(net);
up to
bar(err)
Did you get the same output? This is due to the randomising in the init function.
Try a few more and see if you get to make a net which models the data well.
If none of these networks are good enough we will suggest a different tack. Let us
create a network with 3 neurons in the first hidden layer – leave everything else the
same (use logsig and purelin as transfers as before). You have very little code to
change to get the new network. See if the new network is any better. Again you might
want to try several runs. If you think that your new network is still not up to scratch
maybe add another neuron. Why does adding a neuron help?
To really understand what the toolbox is doing we should briefly explore the network
object in Matlab. Use the net with two neurons in the first hidden layer and one output
neuron. Type the following commands and look at the output:
net
net.LW
net.LW{2,1}
net.b{1,1}
Bob John and Ralph Birkenhead – De Montfort University, Leicester, UK [email protected]
net.b{2,1}
The output gives information about the weights and biases used by the network, as
well as information used by the train and init functions. By referencing these
attributes directly modifications to the network can be made.
The next part of the laboratory session will be assessed by your lab tutor. You must
produce a working network to pass the assessment.
We will create a network which tries to match the output in column3 of the data.txt
file that you saved in lab 2 (so we are trying to model a function of one variable). First
of all work out the range of the x values (look at the data); then set up a feed-forward
network with one input neuron, 3 hidden neurons and one output neuron, by using the
newff function (see help on newff to see what you need to input as parameters.) Make
what you consider to be a sensible choice for the transfer functions.
Initialise and train the network for a maximum of 100 epochs but stop if the error
drops below 0.01. Do a couple of runs and find the best network you can. (It will help
if you call your networks net1, net2 etc. because then you won't lose your best!) You
might want to plot the network simulation of the function and the actual function to
do a visual comparison. When you have a working network tell your tutor how you
could check if your network generalises well.