Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Biological neuron model wikipedia , lookup
Central pattern generator wikipedia , lookup
Synaptic gating wikipedia , lookup
Nervous system network models wikipedia , lookup
Catastrophic interference wikipedia , lookup
Convolutional neural network wikipedia , lookup
Bob John and Ralph Birkenhead – De Montfort University, Leicester, UK [email protected] Lab 4 Learning outcomes You will be able to construct more complicated networks and test the effect of the use of different transfer functions. You will be able to describe the network object used by Matlab more fully and you will be able to access more information about a network. There will be an assessment this week based on setting up and training a network and then discussing if it generalizes. We will now construct a network with two neurons in the first layer and then another layer with one neuron in it. We will then try to model the data from perceptronData3.txt with this network. We will see that it doesn't train all that easily and then modify it to get a better fit. So get the data from perceptronData3.txt into a matrix in the workspace and set up the IN matrix and the TARGET matrix just as you did in the last lab. Recall that IN is a matrix with two rows –the first row is x-coordinates and the second is y-coordinates. The TARGET matrix is actually a vector of target values that the function we are modelling takes at the corresponding x-y points. In the case of our data the target values are either 0 or 1. Again you will need to find the max values and the min values of the x and y co-ordinates, in order to pass the parameters to the network creator. In what follows I have assumed that the matrix dmin has the min values and dmax has the max values – just as in the last lab. net = newff([dmin(1) dmax(1);dmin(2) dmax(2)],[2 1],{'logsig' 'purelin'}); What has this command done? Net is a network object, returned by the function newff. The first parameter to newff is a 2x2 matrix. The input to our function is a vector x-y value. The net wants to know how much scaling it needs to do – so we give it a min-x value and max-x value as the top row of the 2x2 and a min-y max-y as the bottom row. Because there are two rows in this matrix the net will be told that it is modelling a function of two variables. The next entry is a vector telling the newff how many neurons are in each layer of the net. We do not tell newff how many layers – it can tell that we want two layers because we say we want 2 neurons in the first hidden layer and 1 neuron in the output layer. Then we give as a parameter an array of transfer function names. For good reasons we will have the transfer in the first hidden layer as logsig and we use linear in the second hidden layer. We have created the network – let's initialise it. net=init(net); This has set up initial weights and biases – exactly how depends on some parameters that were set up by default by newff when it created net. If we don't like the defaults then we can set them ourselves. At present let's use defaults where possible for convenience. Bob John and Ralph Birkenhead – De Montfort University, Leicester, UK [email protected] Now we will train the network – using train. net=train(net,IN,TARGET); We have trained the net on the IN inputs against the target TARGET. But has it worked – use sim to find the errors: err=TARGET-sim(net,IN) As you remember from last week sim tells me what the network computes, by taking the difference between TARGET and the simulated value. I could plot the errs vector to get a visual idea of how good the net is. bar(err) Neural nets depend on the initialisation values – they do not always build to the same outcome. Repeat the sequence of commands starting from net=init(net); up to bar(err) Did you get the same output? This is due to the randomising in the init function. Try a few more and see if you get to make a net which models the data well. If none of these networks are good enough we will suggest a different tack. Let us create a network with 3 neurons in the first hidden layer – leave everything else the same (use logsig and purelin as transfers as before). You have very little code to change to get the new network. See if the new network is any better. Again you might want to try several runs. If you think that your new network is still not up to scratch maybe add another neuron. Why does adding a neuron help? To really understand what the toolbox is doing we should briefly explore the network object in Matlab. Use the net with two neurons in the first hidden layer and one output neuron. Type the following commands and look at the output: net net.LW net.LW{2,1} net.b{1,1} Bob John and Ralph Birkenhead – De Montfort University, Leicester, UK [email protected] net.b{2,1} The output gives information about the weights and biases used by the network, as well as information used by the train and init functions. By referencing these attributes directly modifications to the network can be made. The next part of the laboratory session will be assessed by your lab tutor. You must produce a working network to pass the assessment. We will create a network which tries to match the output in column3 of the data.txt file that you saved in lab 2 (so we are trying to model a function of one variable). First of all work out the range of the x values (look at the data); then set up a feed-forward network with one input neuron, 3 hidden neurons and one output neuron, by using the newff function (see help on newff to see what you need to input as parameters.) Make what you consider to be a sensible choice for the transfer functions. Initialise and train the network for a maximum of 100 epochs but stop if the error drops below 0.01. Do a couple of runs and find the best network you can. (It will help if you call your networks net1, net2 etc. because then you won't lose your best!) You might want to plot the network simulation of the function and the actual function to do a visual comparison. When you have a working network tell your tutor how you could check if your network generalises well.